CI/CD Pipeline Design: From Push to Production in Minutes

A slow CI/CD pipeline kills productivity. Developers wait 30 minutes for feedback. A good pipeline gives feedback in 3 minutes. This guide covers pipeline architecture.

Pipeline Architecture

# .github/workflows/ci-cd.yml
name: CI/CD

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  # Stage 1: Quick feedback (3 minutes)
  lint-and-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
        with:
          node-version: '18'
          cache: 'npm'

      - run: npm ci
      - run: npm run lint
      - run: npm run test
      - run: npm run type-check

  # Stage 2: Build (2 minutes)
  build:
    needs: lint-and-test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
        with:
          node-version: '18'
          cache: 'npm'

      - run: npm ci
      - run: npm run build
      
      - uses: actions/upload-artifact@v3
        with:
          name: build-artifact
          path: dist/

  # Stage 3: Security scan (parallel, 1 minute)
  security-scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - run: npm audit --audit-level=moderate
      
      - name: Run SAST scan
        uses: securelabs/sast-scan@v1

  # Stage 4: Deploy to staging (5 minutes)
  deploy-staging:
    needs: [build, security-scan]
    runs-on: ubuntu-latest
    if: github.event_name == 'push'
    steps:
      - uses: actions/checkout@v3
      - uses: actions/download-artifact@v3
        with:
          name: build-artifact
          path: dist/

      - name: Deploy to Vercel Staging
        run: |
          npx vercel deploy --prod \
            --token ${{ secrets.VERCEL_TOKEN }} \
            --scope ${{ secrets.VERCEL_ORG_ID }}

      - name: Smoke tests
        run: npm run test:e2e

  # Stage 5: Manual approval + production deploy
  deploy-production:
    needs: deploy-staging
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/main'
    environment:
      name: production
      url: https://myapp.com
    steps:
      - uses: actions/checkout@v3
      - uses: actions/download-artifact@v3
        with:
          name: build-artifact

      - name: Deploy to production
        run: |
          npx vercel deploy --prod \
            --token ${{ secrets.VERCEL_TOKEN }}

      - name: Health check
        run: |
          curl -f https://myapp.com/health || exit 1

      - name: Notify Slack
        if: success()
        run: |
          curl -X POST ${{ secrets.SLACK_WEBHOOK }} \
            -d '{"text":"Deployed to production"}'

Total time: ~12 minutes (lint: 3, build: 2, security: 1, staging: 5, production: 1)

Stage 1: Lint & Test (Fast Feedback)

Run this first. Catches obvious bugs in seconds.

lint-and-test:
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v3
    - uses: actions/setup-node@v3
      with:
        cache: npm  # 💡 Cache npm dependencies
    
    - run: npm ci --prefer-offline --no-audit
    - run: npm run lint
    - run: npm run test -- --coverage
    - run: npm run type-check

  # Fail fast on type errors
  continue-on-error: false

Optimization: Use caching to skip dependency downloads.

setup-node:
  with:
    cache: npm  # Saves 30+ seconds per run

Without caching: 45s With caching: 5s

Caching Strategy

- name: Cache dependencies
  uses: actions/cache@v3
  with:
    path: ~/.npm
    key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
    restore-keys: |
      ${{ runner.os }}-node-

- name: Cache build
  uses: actions/cache@v3
  with:
    path: .next/cache
    key: ${{ runner.os }}-nextjs-${{ hashFiles('**/package-lock.json') }}
    restore-keys: |
      ${{ runner.os }}-nextjs-

Cache key includes hash of package-lock.json. Changes to dependencies invalidate cache automatically.

Stage 2: Build (Production Artifact)

build:
  needs: lint-and-test
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v3
    - uses: actions/setup-node@v3
    
    - run: npm ci
    - run: npm run build
    
    # Upload build as artifact
    - uses: actions/upload-artifact@v3
      with:
        name: build-output
        path: dist/
        retention-days: 7  # Keep 7 days then delete

Artifacts are compressed and cached. Saves recompiling in stage 3.

Stage 3: Parallel Security Scans

Run these in parallel—they don't depend on each other.

security-scan:
  runs-on: ubuntu-latest
  steps:
    # SAST: Static code analysis
    - uses: actions/checkout@v3
    - uses: securelabs/sast-scan@v1

    # Dependency audit
    - run: npm audit --audit-level=moderate

    # Container scanning (if building Docker images)
    - uses: aquasecurity/trivy-action@master
      with:
        image-ref: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.VERSION }}

    # License compliance
    - run: npm ls --all | grep UNLICENSED && exit 1 || true

All run in parallel in same job. One failure stops pipeline.

Stage 4: Deploy to Staging

deploy-staging:
  needs: [build, security-scan]
  runs-on: ubuntu-latest
  if: github.event_name == 'push'  # Only on push, not PR
  
  steps:
    - uses: actions/download-artifact@v3
      with:
        name: build-output

    - name: Deploy to staging
      run: |
        npm run deploy:staging

    # Smoke tests on staging
    - name: Smoke tests
      run: |
        curl -f https://staging.myapp.com/health || exit 1
        npm run test:e2e -- --baseUrl=https://staging.myapp.com

Tests against real staging environment. Catches integration bugs.

Stage 5: Manual Approval + Production

deploy-production:
  needs: deploy-staging
  runs-on: ubuntu-latest
  if: github.ref == 'refs/heads/main'
  
  # 🔐 Require manual approval
  environment:
    name: production
    url: https://myapp.com
    
  steps:
    - uses: actions/download-artifact@v3
    
    - name: Deploy
      run: npm run deploy:prod

    # Verify deployment
    - name: Health check
      run: |
        for i in {1..5}; do
          if curl -f https://myapp.com/health; then
            echo "Health check passed"
            exit 0
          fi
          echo "Attempt $i failed, retrying..."
          sleep 10
        done
        exit 1

    # Notify
    - name: Slack notification
      if: success()
      run: |
        curl -X POST ${{ secrets.SLACK_WEBHOOK }} \
          -H 'Content-Type: application/json' \
          -d '{
            "text": "✅ Deployed to production",
            "blocks": [{
              "type": "section",
              "text": {
                "type": "mrkdwn",
                "text": "*Production Deployment Successful*\nCommit: ${{ github.sha }}"
              }
            }]
          }'

GitHub UI prompts for approval before production:

⏳ Waiting for approval
Reviewers can approve this deployment

Conditional Execution

if: github.event_name == 'push'          # Only on push
if: github.ref == 'refs/heads/main'      # Only main branch
if: github.actor != 'dependabot'         # Skip bots
if: success()                             # Only if previous succeeded
if: failure()                             # Only if previous failed
if: always()                              # Always run

Matrix Testing (Multiple Versions)

Test against multiple Node/OS versions:

test:
  runs-on: ${{ matrix.os }}
  strategy:
    matrix:
      node: [16, 18, 20]
      os: [ubuntu-latest, macos-latest, windows-latest]
  
  steps:
    - uses: actions/setup-node@v3
      with:
        node-version: ${{ matrix.node }}
    - run: npm test

Runs 9 jobs in parallel (3 nodes × 3 OS). Finds version incompatibilities early.

Failure Handling

# Notify on failure
- name: Notify failure
  if: failure()
  run: |
    curl -X POST ${{ secrets.SLACK_WEBHOOK }} \
      -d '{"text":"❌ Pipeline failed: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"}'

# Retry failing jobs
- uses: nick-invision/retry@v2
  with:
    timeout_minutes: 10
    max_attempts: 3
    command: npm run deploy

# Continue despite failures
- run: npm run test || true  # Non-blocking test
  continue-on-error: true

Performance Benchmarks

Stage	Time	Optimization
Lint & test	3m	Cache dependencies
Build	2m	Cache build artifacts
Security	1m	Run in parallel
Staging	5m	Keep alive between runs
Production	1m	Minimize deployment script
Total	12m	All running with minimal waste

Checklist

Conclusion

A well-designed pipeline:

Gives feedback in <5 minutes
Prevents bad code from reaching production
Enables frequent, safe deploys
Reduces manual work

Start simple. Add parallelization and caching. Measure times at each stage. Optimize the slowest bottleneck.

Your team's velocity depends on it.

SUBID