9/11 Lab Errata

• When testing the authenticated /notes endpoint, you actually probably got the error {"detail":"Service unavailable"}
• This is because in order for separate docker containers to talk to each other, they need to be on the same network
  • You could have set the IP address assigned to the auth container by docker as AUTH_SERVICE_URL on the notes service, but this is unwieldy and manual
• We will solve this problem today!

Agenda

1. Docker Compose Introduction
2. YAML Overview
3. Docker Compose Specification
4. Lab

Recap

• So far Jarvis is a mono-repo where each service is a separate uv project
  • This setup is common for large companies where each service is owned by a team
• We created configurations for our services using environment variables or a .env file
  • This paradigm of loading a file then environment variables is called dotenv
• We created Dockerfiles for each service to containerize them
• Goal for today: finish creating a robust local development environment that mirrors the components we need in production

Recap

We have a problem, running Jarvis locally is a pain!

• We need to open a terminal for each service and run its container
  • If we add a service (chat), this is only going to get worse
• We need to make sure we use the right commands for running each service, i.e. picking the right port
• If we're actively developing on a service, we need to make code changes, build the Docker image, then run it again

Enter: Docker Compose

Docker Compose is a tool that allows us to define and run multi-container applications.

• An application is defined as a collection of containers using a YAML file compose.yml
• Run all containers using a single command docker-compose up
• Provides configurations for almost anything you need within a container
• Mainly used locally, typically not recommended for production

https://builder.aws.com/content/2qi9qQstGnCWguDzgLg1NgP8lBF/file-structure-of-docker-composeyml-file

YAML: Overview

YAML stands for "YAML Ain't Markup Language"

• Meant to be a human readable serialization format
• Used very commonly for various DevOps tools
  • Docker, Kubernetes, Terraform, CI/CD, etc
• File extension is either .yaml or .yml

# Key-value pairs (scalars)
name: "John Doe"
age: 30
is_active: true
salary: 75000.50
department: null
# Strings (quotes optional for simple strings)
simple_string: Hello World
quoted_string: "Special chars: @#$%"
multiline_string: |
  This is a multiline string
  that preserves line breaks
  and formatting.
folded_string: >
  This is a folded string that
  will be converted to a single
  line with spaces replacing newlines.

Docker Compose Specification

• Docker Compose introduces a few new concepts: services, networks, and volumes
• Each of these are defined as top level objects in the compose.yaml file
• Services are effectively a container running a program, e.g. a FastAPI application, Next.js frontend, Postgres database, etc
• Networks allow services to communicate with each other
• Volumes store data outside containers so it survives container restarts and updates

Services: Defining Your Containers

Services are the containers that make up your application. Each service runs one part of your system.

services:
  web:
    image: nginx:alpine
    ports:
      - "80:80"
  database:
    image: postgres:15
    environment:
      POSTGRES_DB: myapp

More Service Syntax

services:
  api:
    build: ./backend          # Build from a Dockerfile
    ports:
      - "3000:3000"
    env_file:                 # Inject env variables from local file
      - ./api/.env
    depends_on:               # Start only after database service is ready
      - database
    restart: unless-stopped   # Restart policy
  database:
    image: postgres:15
    volumes:
      - db_data:/var/lib/postgresql/data    # Use volume for persistence

Networks: Connecting Services

• Default network: Unless specified, all services are included in the default network. This allows them to connect to each other using service names as hostnames, e.g. http://api connects to the api service.
• If you have a need to split network connections from each service, you can explicitly add services to networks

Networks Syntax

services:
  api:
    ...
    networks:
      - web-tier     # include api service in web-tier and db-tier networks
      - db-tier
networks:
  web-tier:
    driver: bridge    # The network driver to use, typically bridge
  db-tier:
    driver: bridge
    internal: true    # Prevent external access, false by default

Volumes: Persisting Data

• We can create volumes to store persistent data
  • Think of these as folders
• We mount these volumes in our containers to use them
• We can also mount local files to our Docker containers
  • Useful for local development, e.g. mount the app/ directory so that FastAPI can restart automatically within the container when it sees files have changed!

Volumes Syntax

services:
  database:
    image: postgres:15
    volumes:
      # Mount db_data volume at /var/lib/postgresql/data
      - db_data:/var/lib/postgresql/data
  api:
    volumes:
      # Mount local folder ./api/app as /app on container
      - ./api/app:/app
volumes:
  db_data:        # Managed by Docker
  static_files:   # Shared between services

Docker Compose CLI

Common commands:

• docker compose up - start services defined in compose.yaml
  • adding the -d flag runs in detached mode, i.e. the background
• docker compose down - stop and remove services
• docker compose ps - get status of containers
• docker compose logs - get logs of all containers
  • Only really needed if you started in detached mode

Docker Compose Specification

There's a lot more you can do with Docker Compose, we've only just gone over the basics we need. If you're curious, check out the specification on the docker documentation.

https://docs.docker.com/reference/compose-file/