Reminders

• HW3 due tonight!
• HW4 pushed back a week, will release March 25th
• Final Project guidelines released, proposal due April 8th
• Joy Liu guest lecture April 15th

Agenda

1. Motivation
2. Continuous Integration (CI)
3. Continuous Deployment/Delivery (CD)
4. CI/CD Pipelines
5. Best Practices

Manual Development Workflow

• Developer writes code locally
• Manually runs tests (sometimes)
• Pushes to shared repository after review
• Someone manually pulls latest changes and builds an image
• Someone manually deploys image to production
• Deployment may involve running migrations

What could go wrong?

Manual Deployment Issues

• Integration failures: Code works alone but breaks when combined

  • Different developers working on conflicting features
  • Dependencies mismatch between environments

• Deployment delays: Weeks between code completion and production

  • Can lead to large, risky releases

• Human error: Manual steps are error-prone

• Costly: Can take an engineer's whole day to do this!

The Cost of Manual Processes

Key insight: Manual processes don't scale as teams grow

What is Continuous Integration?

Automatically build and test code every time changes are pushed

Core ideology: Integrate early, integrate often

Goal: Catch integration problems early, when they're cheap to fix

Benefits of CI

• Fast feedback: Know within minutes if code breaks
• Reduced integration risk: Small, frequent integrations easier than big merges
• Automated testing: Tests run consistently every time
• Documentation: Build history shows what works and what doesn't
• Confidence: Green builds give confidence to deploy

Key metric: Time from commit to feedback

What Tests Should CI Run?

Example: Simple CI Configuration

# .github/workflows/ci.yml (GitHub Actions)
name: CI
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install uv
        uses: astral-sh/setup-uv@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install dependencies
        run: uv sync --all-extras --dev

      - name: Run tests
        run: uv run pytest

Every push triggers: checkout → setup → install → test

What is Continuous Deployment?

Automatically deploy code that passes CI checks

Two flavors:

• Continuous Delivery: Code is always ready to deploy (manual trigger)
• Continuous Deployment: Code is automatically deployed to production

Goal: Minimize time from code written to code running in production

Continuous Delivery vs. Deployment

Both require: Robust automated testing and monitoring

CD Prerequisites

You need:

• Comprehensive automated tests
• Good monitoring and alerting
• Fast rollback capability
• Feature flags for incomplete work
• Team confidence in the process

Without these: CD becomes "continuous disaster"

Deployment Strategies

Benefits of CD

• Smaller deployments: Less risky than "big-bang" releases
• Faster feedback: See user reactions quickly
• Less deployment stress: Deploying becomes routine
• Better quality: Frequent deploys encourage better practices

Cultural shift: Deployment becomes a non-event, confidence is built with frequent deployments

IaC in the CD Pipeline

We can also use CD for infrastructure changes!

1. Developer changes infrastructure code (Terraform, CloudFormation)
2. CI runs validation and tests (terraform plan)
3. CD applies changes automatically (terraform apply)
4. Infrastructure changes are tracked in git history

Infrastructure changes go through same review process as application code

What is a Pipeline?

A series of automated steps that take code from commit to production. This combines CI with CD. A pipeline creates the flow:

1. Source: Code is pushed to repository
2. Build: Compile, package, create artifacts/images
3. Test: Run automated tests
4. Deploy: Release to environment(s)
5. Monitor: Track health and metrics

Each stage can fail and stop the pipeline

Pipeline as Code

pipeline:
  - stage: build
    jobs:
      - compile
      - package
  - stage: test
    jobs:
      - unit-tests
      - integration-tests
      - security-scan
  - stage: deploy-production
    jobs:
      - deploy-app
      - smoke-tests
      - monitor

Secrets and Configuration

Pipelines need credentials to deploy (AWS role), API keys, etc.

Solutions:

• Store secrets in CI platform (GitHub Secrets, etc.)
  • Never commit secrets to repository
• Use environment-specific configurations
• Rotate credentials regularly

Pipeline Monitoring

What to track:

• Build success/failure rates
• Pipeline duration (time from commit to deploy)
• Test flakiness (tests that fail randomly)
• Deployment frequency
• Mean time to recovery (MTTR)

Red flags:

• Pipelines taking longer over time
• Increasing failure rates
• Developers skipping CI checks

CI/CD Tool Landscape

Keep Builds Fast

Why: Fast feedback loops improve productivity

How:

• Parallelize tests
• Cache dependencies and build artifacts
• Use powerful CI runners for critical pipelines
• Split slow tests into separate jobs

Trade-off: Speed vs. test coverage

Fail Fast, Fail Clearly

Fail fast: Run quick checks first (linting, compilation) before slow tests

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - run: npm run lint

  test:
    needs: lint  # Only run if lint passes
    runs-on: ubuntu-latest
    steps:
      - run: npm test

Fail clearly: Provide actionable error messages

• Show which test failed
• Include relevant logs
• Link to documentation when possible

Keep Main Branch Green

Rule: Main/master branch should always build successfully. Fixing broken main should always be top priority.

Enforce with:

• Required status checks on PRs (can't merge if not green)
• Automated rollback if main breaks (git revert)

Monitor Deployments

Deployment isn't done when code is pushed

Monitor:

• Application errors and exceptions
• Performance metrics (latency, throughput)
• Resource usage (CPU, memory)
• Business metrics (signups, conversions)

Practice: Watch metrics for 15-30 minutes after deployment

Rollback Strategies

When things go wrong, you need a panic button

Security in CI/CD

Vulnerabilities:

• Leaked secrets in logs or code
• Compromised pipeline access
• Malicious dependencies
• Unsafe Docker images

Protections:

• Secret scanning in repositories
• Dependency vulnerability scanning
• Image scanning for containers
• Principle of least privilege for pipeline permissions

CI/CD Maturity Model

Level 1: No automation, manual everything
Level 2: Automated builds, manual tests and deploys
Level 3: Automated tests (CI)
Level 4: Automated deployments to staging (Continuous Delivery)
Level 5: Automated deployments to production (Continuous Deployment)