Docker Compose for AI Development: Local Environment Setup Guide
While individual Docker containers are useful, AI applications typically involve multiple services working together. Docker Compose orchestrates these services locally, giving you a production-like environment on your development machine.
Through setting up development environments for various AI projects, I’ve learned that a well-configured Docker Compose setup accelerates development significantly, you can test integrations locally that would otherwise require cloud deployment.
Why Docker Compose for AI
AI development benefits specifically from Docker Compose:
Multi-service complexity is standard. AI applications typically need: application server, vector database, cache, and often more.
Environment consistency between developers. The same compose file works on every machine.
Quick environment reset for debugging. Tear down and recreate the entire stack in seconds.
Production parity using the same containers you’ll deploy.
Basic Structure
Understanding Docker Compose structure is essential.
Compose File
The docker-compose.yml defines your environment:
Services are the containers in your stack. Each service has an image or build context.
Networks connect services together. Default bridge network usually suffices.
Volumes persist data between restarts. Essential for databases and model files.
Environment variables configure each service. Secrets, ports, and feature flags.
Service Definition
Each service needs key configuration:
Image or build specifies what to run. Use official images or build from Dockerfile.
Ports expose services to your host. Map container ports to localhost.
Volumes mount host directories. Code, data, and configuration.
Depends_on controls startup order. Databases before apps.
Environment sets configuration variables.
Common AI Service Stack
Most AI applications share similar service requirements.
Application Server
Your main AI application:
Build from local Dockerfile for development. See code changes immediately.
Volume mount source code for live reloading. Changes don’t require rebuild.
Expose API port to localhost. Typically 8000 or 8080.
Environment variables for configuration. API keys, database URLs.
Vector Database
Store and search embeddings:
Chroma, Qdrant, or Weaviate for local vector search.
Persistent volume for embedding data. Survives container restarts.
Port mapping for direct database access. Useful for debugging.
PostgreSQL with pgvector
When using PostgreSQL for vectors:
Official postgres image with pgvector extension.
Initialization scripts run on first start. Create extensions, tables, indexes.
Volume for data directory preserves database between sessions.
Redis
Caching and queuing:
Official Redis image works immediately.
Optional persistence if needed between restarts.
Port 6379 for local access.
Object Storage
Mock S3 for file storage:
MinIO provides S3-compatible storage locally.
Bucket initialization via environment variables or scripts.
Console access for debugging uploads.
GPU Configuration
AI development often requires GPU access.
NVIDIA Container Runtime
Enable GPU access in Compose:
deploy.resources.reservations.devices specifies GPU requirements.
driver: nvidia for NVIDIA GPUs.
count or device_ids control which GPUs.
capabilities: [gpu] enables GPU features.
GPU Resource Management
Multiple services sharing GPUs:
Allocate specific GPUs to specific services. Prevent contention.
Memory limits prevent one service monopolizing.
Development vs production GPU allocation.
CPU-Only Development
When GPUs aren’t available:
Conditional GPU configuration based on environment.
Override files for different hardware.
Mock services for GPU-dependent features.
Development Workflow
Integrate Compose into daily development.
Starting Services
Launch your environment:
docker compose up starts all services. Add -d for detached mode.
docker compose up service-name starts specific services.
docker compose logs -f follows log output.
Hot Reloading
See changes without rebuilding:
Volume mount source code into running containers.
Use development servers with auto-reload. Uvicorn with —reload.
Install dependencies at startup for flexibility.
Rebuilding
When dependencies change:
docker compose build rebuilds images.
docker compose up —build rebuilds then starts.
—no-cache forces complete rebuild.
Teardown
Clean up environments:
docker compose down stops and removes containers.
-v flag removes volumes too. Nuclear option for fresh start.
—rmi all removes images. Complete cleanup.
Environment Configuration
Manage configuration properly.
Environment Files
Separate configuration from compose file:
.env file loaded automatically. Simple key=value format.
env_file directive for service-specific files.
Multiple environments via different env files.
Secrets Management
Handle sensitive configuration:
Never commit secrets to version control.
.env.example documents required variables.
Docker secrets for production-like handling.
Override Files
Environment-specific overrides:
docker-compose.override.yml loaded automatically. Development settings.
docker-compose.prod.yml for production configuration.
-f flag specifies which files to use.
Network Configuration
Services communicate over Docker networks.
Default Network
Compose creates a default network:
Service names as hostnames. postgres not localhost.
Automatic DNS resolution between services.
Internal traffic doesn’t need port exposure.
Custom Networks
For complex topologies:
Multiple networks for service isolation.
External networks for cross-project communication.
Network aliases for service discovery.
Volume Strategies
Persist data effectively.
Named Volumes
Managed by Docker:
postgres_data: mounts to /var/lib/postgresql/data.
Persist between restarts automatically.
docker volume commands for management.
Bind Mounts
Host directory mapping:
./src:/app/src for code mounting.
Development workflow with live editing.
Performance considerations on macOS/Windows.
tmpfs Mounts
Memory-backed storage:
Fast temporary storage for caches.
Cleared on restart by design.
Useful for test artifacts.
Health Checks
Ensure services are ready.
Container Health Checks
Define in compose file:
test command to check health.
interval between checks.
timeout for check execution.
retries before marking unhealthy.
Dependency Health
Wait for healthy dependencies:
depends_on with condition waits for health.
condition: service_healthy requires passing health checks.
Better than simple depends_on.
Testing Configuration
Run tests in Compose.
Test Services
Dedicated test containers:
Test service runs your test suite.
Depends on application and dependencies.
Volume mount for code access.
CI/CD Integration
Use Compose in pipelines:
docker compose up -d starts services.
docker compose run tests executes tests.
docker compose down cleans up.
Production Considerations
Bridge development to production.
Compose to Kubernetes
Migration path:
Similar concepts translate to Kubernetes.
kompose converts compose files.
Abstractions differ but principles transfer.
Environment Parity
Match production:
Same images used in dev and prod.
Similar resource constraints if possible.
Feature parity for realistic testing.
What AI Engineers Need to Know
Docker Compose mastery for AI means understanding:
- Service orchestration for multi-container AI apps
- GPU configuration for local inference
- Volume strategies for data persistence
- Development workflow integration
- Environment management across stages
- Health checks for reliable startup
- Production transition path
The engineers who master these patterns develop faster with environments that match production while maintaining the flexibility needed for rapid iteration.
For more on AI development infrastructure, check out my guides on Docker for AI engineers and deploying AI with Docker and FastAPI. Local development environment mastery is foundational for production AI work.
Ready to set up proper AI development environments? Watch the implementation on YouTube where I configure real Docker Compose setups. And if you want to learn alongside other AI engineers, join our community where we share development environment patterns daily.
