Enhancing Your Python TDD Workflow with Docker

Table of Contents

1. What is Test-Driven Development (TDD)?
2. Why Docker Enhances Python TDD
3. Prerequisites
4. Setting Up Your Docker Environment for Python TDD
5. Writing Your First Python Test with TDD
6. Integrating Docker into the TDD Workflow
7. Advanced Tips for Dockerized TDD
8. Troubleshooting Common Issues
9. Conclusion
10. References

What is Test-Driven Development (TDD)?

TDD follows a simple, iterative cycle: Red → Green → Refactor:

• Red: Write a test that fails (because the feature isn’t implemented yet).
• Green: Write the minimal code required to make the test pass.
• Refactor: Improve the code’s readability, performance, or structure without changing its behavior (tests should still pass).

This cycle ensures that code is testable by design and reduces the risk of regressions. However, without a consistent environment, tests might pass locally but fail in production (or vice versa), undermining TDD’s benefits. Docker solves this by encapsulating your application and its dependencies in a container.

Why Docker Enhances Python TDD

Docker complements TDD in several key ways:

• Environment Consistency: Docker containers ensure everyone on your team (and your CI/CD pipeline) uses the exact same Python version, dependencies, and system libraries. No more “but it worked on my laptop!”
• Isolation: Tests run in a sandboxed environment, so they won’t interfere with your system Python or other projects.
• Reproducibility: Anyone can recreate your development environment with a single docker-compose up command.
• Scalability: Easily add services (e.g., databases, APIs) to your tests using Docker Compose for integration testing.
• CI/CD Readiness: Dockerized tests integrate seamlessly with CI tools like GitHub Actions, GitLab CI, or Jenkins.

Prerequisites

Before we start, ensure you have the following installed:

• Docker (v20.10+ recommended)
• Docker Compose (included with Docker Desktop on Windows/macOS)
• Basic familiarity with Python and TDD concepts (we’ll use pytest for testing)

Setting Up Your Docker Environment for Python TDD

Let’s build a Docker environment for a simple Python project (a “calculator” app) to demonstrate TDD. We’ll use:

• A Dockerfile to define the application image.
• docker-compose.yml to simplify running tests and managing services.

Step 1: Project Structure

First, create a project directory with this structure:

python-tdd-docker/  
├── calculator/          # Source code  
│   ├── __init__.py  
│   └── calculator.py    # Core logic  
├── tests/               # Test files  
│   ├── __init__.py  
│   └── test_calculator.py  # TDD tests  
├── requirements.txt     # Dependencies (pytest, etc.)  
├── Dockerfile           # Defines the Docker image  
└── docker-compose.yml   # Orchestrates the environment  

Step 2: Define Dependencies

Add pytest to requirements.txt (we’ll use it for testing):

# requirements.txt  
pytest==7.4.0  

Step 3: Write the Dockerfile

The Dockerfile defines how to build the Docker image for your app. We’ll use Python’s official slim image for a balance of size and functionality:

# Dockerfile  
# Use an official Python runtime as the base image  
FROM python:3.11-slim  

# Set environment variables to prevent Python from writing .pyc files and buffering output  
ENV PYTHONDONTWRITEBYTECODE=1  
ENV PYTHONUNBUFFERED=1  

# Set the working directory in the container  
WORKDIR /app  

# Copy the requirements file first to leverage Docker's caching  
# This way, dependencies are only reinstalled if requirements.txt changes  
COPY requirements.txt .  

# Install dependencies  
RUN pip install --no-cache-dir -r requirements.txt  

# Copy the entire project into the container (after dependencies for caching)  
COPY . .  

Key Notes:

• PYTHONDONTWRITEBYTECODE avoids cluttering the container with .pyc files.
• PYTHONUNBUFFERED ensures print statements and logs appear immediately.
• Copying requirements.txt before the rest of the code caches dependencies, speeding up rebuilds when only code (not dependencies) changes.

Step 4: Write docker-compose.yml

docker-compose.yml simplifies running commands in the container (e.g., running tests). We’ll add a service for the app and configure volume mounts for live code reloading:

# docker-compose.yml  
version: '3.8'  

services:  
  app:  
    build: .  # Build the image from the Dockerfile  
    volumes:  
      - .:/app  # Mount local code into the container (live reloading)  
      - /app/__pycache__  # Ignore __pycache__ to avoid conflicts  
    command: pytest  # Default command: run tests  

Why Volumes? The .:/app mount syncs your local code with the container. This means you can edit files locally, and changes are immediately reflected in the container—no need to rebuild the image to test updates!

Writing Your First Python Test with TDD

Now, let’s implement the TDD cycle using our Docker environment. We’ll build a Calculator class with an add method.

Phase 1: Red (Write a Failing Test)

First, write a test for the add method in tests/test_calculator.py:

# tests/test_calculator.py  
import pytest  
from calculator.calculator import Calculator  

def test_addition():  
    calculator = Calculator()  
    result = calculator.add(2, 3)  
    assert result == 5, "2 + 3 should equal 5"  

At this point, Calculator doesn’t exist, so the test will fail. Let’s confirm by running the test in Docker.

Run the Test in Docker

Build the image and run the test with Docker Compose:

# Build the Docker image (only needed once or after Dockerfile changes)  
docker-compose build  

# Run the test command defined in docker-compose.yml  
docker-compose run --rm app  

Expected Output (Red Phase):

ImportError: cannot import name 'Calculator' from 'calculator.calculator'  

The test fails—good! We’re in the Red phase.

Phase 2: Green (Make the Test Pass)

Now, implement the minimal code to pass the test. Create calculator/calculator.py:

# calculator/calculator.py  
class Calculator:  
    def add(self, a: int, b: int) -> int:  
        return a + b  # Simple implementation to pass the test  

Re-Run the Test

Since we mounted the code as a volume, we don’t need to rebuild the image. Just re-run the test:

docker-compose run --rm app  

Expected Output (Green Phase):

============================= test session starts ==============================  
collected 1 item  

tests/test_calculator.py .                                              [100%]  

============================== 1 passed in 0.01s ===============================  

The test passes—we’re in the Green phase!

Phase 3: Refactor (Improve Code)

Our code works, but maybe we can make it cleaner (e.g., add type hints, docstrings). Refactor calculator.py:

# calculator/calculator.py  
class Calculator:  
    """A simple calculator for basic arithmetic operations."""  

    def add(self, a: int, b: int) -> int:  
        """Add two integers and return the result.  

        Args:  
            a: The first integer to add.  
            b: The second integer to add.  

        Returns:  
            The sum of `a` and `b`.  
        """  
        return a + b  

Re-run the test to ensure refactoring didn’t break anything:

docker-compose run --rm app  

The test still passes—success!

Integrating Docker into the TDD Workflow

Now that we have a basic setup, let’s formalize the TDD workflow with Docker:

1. Write a Failing Test (Red)

Add a new test to test_calculator.py (e.g., for subtraction):

# tests/test_calculator.py  
def test_subtraction():  
    calculator = Calculator()  
    result = calculator.subtract(5, 3)  
    assert result == 2, "5 - 3 should equal 2"  

Run the test in Docker—expect a failure (no subtract method):

docker-compose run --rm app  

2. Implement the Feature (Green)

Add the subtract method to calculator.py:

# calculator/calculator.py  
def subtract(self, a: int, b: int) -> int:  
    return a - b  

3. Refactor (Optional)

No refactoring needed here, but if you wanted to add error handling (e.g., for non-integer inputs), you could do so now and re-run tests.

Key Workflow Tip: Live Test Reloading

To avoid typing docker-compose run --rm app repeatedly, use pytest-watch to auto-run tests when files change. Install it by adding pytest-watch==4.2.0 to requirements.txt, then update the command in docker-compose.yml:

# docker-compose.yml (updated)  
services:  
  app:  
    build: .  
    volumes:  
      - .:/app  
      - /app/__pycache__  
    command: ptw --poll  # Auto-run tests on file changes (--poll for Docker compatibility)  

Now run:

docker-compose up  

Any changes to test_calculator.py or calculator.py will trigger an automatic test run—perfect for TDD!

Advanced Tips for Dockerized TDD

1. Multi-Stage Builds for Smaller Images

Use multi-stage builds to separate the “build/test” environment from the “production” environment, reducing image size. Update your Dockerfile:

# Stage 1: Build and test  
FROM python:3.11-slim AS builder  
WORKDIR /app  
COPY requirements.txt .  
RUN pip wheel --no-cache-dir --no-deps --wheel-dir /app/wheels -r requirements.txt  
COPY . .  
RUN pytest  # Run tests in the builder stage  

# Stage 2: Production image (only runtime dependencies)  
FROM python:3.11-slim  
WORKDIR /app  
COPY --from=builder /app/wheels /wheels  
COPY --from=builder /app/requirements.txt .  
RUN pip install --no-cache /wheels/*  
COPY --from=builder /app/calculator /app/calculator  # Only copy source code  
CMD ["python", "-m", "calculator.calculator"]  # Example runtime command  

Now build with docker build -t calculator-app .—the final image will be smaller!

2. Cache Dependencies Aggressively

Docker caches layers, so order your COPY commands to avoid re-installing dependencies unnecessarily. Always copy requirements.txt before copying code:

# Good: Dependencies are cached unless requirements.txt changes  
COPY requirements.txt .  
RUN pip install -r requirements.txt  
COPY . .  # Code changes won’t trigger dependency re-installs  

3. Run Tests in CI with GitHub Actions

Add a .github/workflows/ci.yml file to run tests on every push:

name: Run Tests  
on: [push, pull_request]  

jobs:  
  test:  
    runs-on: ubuntu-latest  
    steps:  
      - name: Checkout code  
        uses: actions/checkout@v4  

      - name: Build and test  
        run: |  
          docker-compose build  
          docker-compose run --rm app  

Now every commit will trigger tests in Docker—no more environment surprises!

4. Test Against Multiple Python Versions

Use Docker’s --build-arg to test against multiple Python versions. Update the Dockerfile to accept a PYTHON_VERSION argument:

ARG PYTHON_VERSION=3.11-slim  
FROM python:${PYTHON_VERSION}  
# ... rest of the Dockerfile ...  

Then test with:

docker build --build-arg PYTHON_VERSION=3.10-slim -t calculator-3.10 .  
docker run --rm calculator-3.10 pytest  

5. Integration Testing with Docker Compose

For tests requiring external services (e.g., a PostgreSQL database), add the service to docker-compose.yml:

# docker-compose.yml (with PostgreSQL)  
services:  
  app:  
    build: .  
    volumes:  
      - .:/app  
    depends_on:  
      - db  
    environment:  
      - DATABASE_URL=postgresql://user:pass@db:5432/mydb  
    command: pytest  

  db:  
    image: postgres:15-alpine  
    environment:  
      - POSTGRES_USER=user  
      - POSTGRES_PASSWORD=pass  
      - POSTGRES_DB=mydb  
    ports:  
      - "5432:5432"  

Now your tests can connect to the database running in the db service!

Troubleshooting Common Issues

Issue: Tests fail due to permission errors in the container.

Fix: Ensure the container’s user has access to the mounted volume. Add a non-root user to the Dockerfile:

RUN adduser --disabled-password --gecos '' appuser  
USER appuser  

Issue: Changes to code aren’t reflected in the container.

Fix: Verify the volume mount in docker-compose.yml (- .:/app). If using WSL2 on Windows, ensure your project is in the WSL filesystem (not /mnt/c).

Issue: Slow test runs in Docker.

Fix:

• Use pytest-xdist to run tests in parallel (add pytest-xdist==3.3.1 to requirements.txt, then run pytest -n auto).
• Cache dependencies aggressively in the Dockerfile.

Conclusion

By integrating Docker into your Python TDD workflow, you eliminate environment inconsistencies, streamline testing, and build a foundation for scalable, CI/CD-ready applications. The combination of TDD’s disciplined testing and Docker’s reproducibility ensures your code is robust, maintainable, and ready for production.

Start small (as we did with the calculator app), then expand to multi-service applications and CI/CD pipelines. Your future self (and team) will thank you!

References

• Docker Documentation
• pytest Documentation
• Test-Driven Development with Python (Book)
• GitHub Actions Documentation
• Python Official Docker Images