DevOps Pipeline Setup: Hands-On GitHub Actions Lab

Lab Overview: Production DevOps Pipeline Setup Architecture

In this hands-on engineering lab, you will configure a complete, automated continuous delivery system. The goal of this devops pipeline setup is to demonstrate how modern engineering organizations move code from a developer's local commit to an Amazon EKS or Kubernetes cluster in under four minutes without exposing static cloud credentials.

To understand how this architecture compares against legacy centralized build farms, read our detailed comparison of GitHub Actions vs Jenkins. For deep architectural guidance, consult our companion CI/CD pipeline setup guide.

Step-by-Step DevOps Pipeline Setup Implementation

Follow these five progressive steps to build, validate, containerize, and deploy a production-grade microservice pipeline.

Step 1: Clone the Forkable Lab Repository Structure

Set up your local directory workspace adhering to clean separation between application source code, Docker build files, CI workflow definitions, and Kubernetes deployment manifests.

# Initialize the production lab directory structure
mkdir -p devops-pipeline-lab/{.github/workflows,cmd/api,deploy/k8s}
cd devops-pipeline-lab

# Verify directory layout
tree -a
# Output:
# .
# ├── .github
# │   └── workflows
# ├── cmd
# │   └── api
# └── deploy
#     └── k8s

Step 2: Create the High-Performance Multi-Stage Dockerfile

Create a container build specification optimized for Docker BuildKit. This configuration uses a builder stage for dependency caching and compiles a minimal, distroless production container image.

# File: Dockerfile
# syntax=docker/dockerfile:1.7

# Stage 1: Dependency Caching & Compilation
FROM golang:1.24-alpine AS builder
WORKDIR /workspace
RUN apk add --no-cache git ca-certificates

# Cache dependencies independently from source code
COPY go.mod go.sum ./
RUN --mount=type=cache,target=/go/pkg/mod \
    go mod download

COPY . .
RUN --mount=type=cache,target=/go/pkg/mod \
    --mount=type=cache,target=/root/.cache/go-build \
    CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags="-s -w" -o /bin/service ./cmd/api

# Stage 2: Distroless Non-Root Runtime
FROM gcr.io/distroless/static-debian12:nonroot
WORKDIR /
COPY --from=builder /bin/service /service
USER nonroot:nonroot
EXPOSE 8080
ENTRYPOINT ["/service"]

Step 3: Configure the Automated GitHub Actions Workflow

Create the primary continuous delivery engine in .github/workflows/pipeline.yml. This workflow activates parallel matrix testing, authenticates to cloud registries using OpenID Connect (OIDC), compiles with remote BuildKit caching, scans with Trivy, and triggers GitOps synchronization.

# File: .github/workflows/pipeline.yml
name: Production DevOps Pipeline
on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

permissions:
  contents: write
  id-token: write

jobs:
  validate:
    name: Lint & Secret Detection
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Code
        uses: actions/checkout@v4

      - name: Scan for Hardcoded Secrets
        uses: trufflesecurity/trufflehog@v3.63.0
        with:
          path: ./
          base: ${{ github.event.repository.default_branch }}
          head: HEAD

  test:
    name: Unit & Race Detection
    needs: validate
    runs-on: ubuntu-latest
    strategy:
      matrix:
        go: ['1.23', '1.24']
    steps:
      - name: Checkout Repository
        uses: actions/checkout@v4

      - name: Setup Go Runtime
        uses: actions/setup-go@v5
        with:
          go-version: ${{ matrix.go }}
          cache: true

      - name: Run Tests with Race Detector
        run: go test -v -race -coverprofile=coverage.out ./...

  build-and-scan:
    name: BuildKit & Vulnerability Gate
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Repository
        uses: actions/checkout@v4

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Build Local Container for Vulnerability Audit
        uses: docker/build-push-action@v5
        with:
          context: .
          load: true
          tags: local-app:${{ github.sha }}
          cache-from: type=gha
          cache-to: type=gha,mode=max

      - name: Security Scan Container with Trivy
        uses: aquasecurity/trivy-action@master
        with:
          image-ref: 'local-app:${{ github.sha }}'
          format: 'table'
          exit-code: '1'
          ignore-unfixed: true
          severity: 'CRITICAL,HIGH'

Step 4: Establish the Declarative Kubernetes Deployment Manifest

Define the production workload specifications in deploy/k8s/deployment.yaml. Ensure explicit CPU/memory resource requests and limits, readiness/liveness health probes, and hardened security contexts are configured.

# File: deploy/k8s/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-microservice
  namespace: production
  labels:
    app.kubernetes.io/name: api-microservice
spec:
  replicas: 3
  selector:
    matchLabels:
      app: api-microservice
  template:
    metadata:
      labels:
        app: api-microservice
    spec:
      securityContext:
        runAsNonRoot: true
        runAsUser: 65532
        fsGroup: 65532
      containers:
        - name: api
          image: ghcr.io/naveedahmed/api-microservice:latest
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 8080
          resources:
            requests:
              cpu: 100m
              memory: 128Mi
            limits:
              cpu: 500m
              memory: 256Mi
          readinessProbe:
            httpGet:
              path: /healthz
              port: 8080
            initialDelaySeconds: 3
            periodSeconds: 5
          livenessProbe:
            httpGet:
              path: /healthz
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 10

To validate your manifest syntax and verify security contexts before pushing to Git, use the Kubernetes YAML Generator & Validator.

Step 5: Configure the ArgoCD Declarative GitOps Application

Connect your repository to ArgoCD by applying the deploy/k8s/argocd-app.yaml manifest. ArgoCD monitors the Git repository and reconciles state automatically without exposing cluster credentials to external build runners.

# File: deploy/k8s/argocd-app.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: api-microservice-prod
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: default
  source:
    repoURL: https://github.com/naveedahmed/devops-pipeline-lab.git
    targetRevision: main
    path: deploy/k8s
  destination:
    server: https://kubernetes.default.svc
    namespace: production
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

# Apply the GitOps Application into your Kubernetes cluster
kubectl apply -f deploy/k8s/argocd-app.yaml

# Verify application synchronization status
kubectl get application api-microservice-prod -n argocd -o wide
# Output:
# NAME                    SYNC STATUS   HEALTH STATUS
# api-microservice-prod   Synced        Healthy

Verification & Common Troubleshooting Scenarios

Once your devops pipeline setup is operational, test failure conditions to verify that quality gates trigger properly:

How to verify test failure gates in GitHub Actions?

Trigger an intentional validation failure by committing an unformatted file or injecting a dummy AWS secret. Verify that TruffleHog intercepts the commit and prevents the pull request from merging.

Why does BuildKit multi-stage caching cut build times by 70%?

BuildKit analyzes layer dependencies in a Directed Acyclic Graph (DAG) and reuses untouched stages from remote registry caches. Only modified layers and downstream dependents are re-compiled.

For troubleshooting pods failing to schedule after deployment, reference our triage guide for pods stuck in Pending and bookmark the Kubectl Production Cheat Sheet.