Python Testing: Common Pitfalls and How to Avoid Them

Table of Contents

1. Not Testing Edge Cases
2. Testing Implementation Details
3. Flaky Tests
4. Over-Mocking
5. Insufficient Test Coverage (or Obsessing Over It)
6. Ignoring Test Readability
7. Not Testing Error Conditions
8. Skipping Test Automation
9. Conclusion
10. References

1. Not Testing Edge Cases

The Pitfall

Many tests focus only on “happy path” scenarios (e.g., valid inputs, typical use cases) but ignore edge cases—extreme or unexpected inputs that could break your code. Examples include:

• Empty collections (e.g., [], {}).
• Boundary values (e.g., 0, None, maximum integers).
• Malformed inputs (e.g., strings where numbers are expected).

Why it’s a problem: Edge cases are where bugs often hide. A function that works for [1, 2, 3] might crash with [] or return incorrect results for [None, 5].

How to Avoid It

• Use Parameterized Testing: Tools like pytest.mark.parametrize let you test multiple inputs (including edge cases) in a single test function.
• Think Like a Tester: Ask: What if the input is empty? Zero? Negative? Null?
• Leverage Property-Based Testing: Libraries like hypothesis generate thousands of inputs (including edge cases) to validate “properties” of your code (e.g., “summing a list twice returns the same result”).

Example

Suppose you’re testing a function sum_numbers(numbers) that sums a list of integers:

# Bad: Only tests the happy path  
def test_sum_numbers_happy_path():  
    assert sum_numbers([1, 2, 3]) == 6  


# Good: Tests edge cases with parametrization  
import pytest  

@pytest.mark.parametrize("numbers, expected", [  
    ([1, 2, 3], 6),          # Happy path  
    ([], 0),                 # Empty list  
    ([0], 0),                # Single zero  
    ([None, 5], 5),          # Mixed None and int (if allowed)  
    ([10**18, 10**18], 2*10**18),  # Large numbers  
])  
def test_sum_numbers_edge_cases(numbers, expected):  
    assert sum_numbers(numbers) == expected  

2. Testing Implementation Details

The Pitfall

Tests should validate behavior, not how the code works internally. Testing implementation details (e.g., private methods, variable names, or helper functions) leads to brittle tests that break when you refactor—even if the external behavior is unchanged.

Example: Suppose you have a function calculate_total(items) that uses a private helper _apply_discount(price). Testing that _apply_discount was called with a specific argument (via mocks) instead of checking the final total is a classic implementation test.

How to Avoid It

• Test Inputs and Outputs: Focus on “given X input, does the code return Y output?” or “does it raise Z error?”
• Avoid Mocking Internal Functions: Only mock external dependencies (e.g., APIs, databases), not internal helpers.
• Use Black-Box Testing: Treat the code as a “black box”—you don’t care how it works, only that it works.

Example

# Bad: Tests implementation (mocks internal helper)  
def test_calculate_total_implementation():  
    with patch("my_module._apply_discount") as mock_discount:  
        calculate_total([{"price": 100}])  
        mock_discount.assert_called_once_with(100)  # Breaks if _apply_discount is renamed  


# Good: Tests behavior (output)  
def test_calculate_total_behavior():  
    items = [{"price": 100}, {"price": 50}]  
    assert calculate_total(items) == 135  # Assumes 10% discount on $150 total  

3. Flaky Tests

The Pitfall

Flaky tests are tests that pass sometimes and fail other times, often for no obvious reason. Common causes:

• Shared State: Tests modifying global variables or a shared database.
• External Dependencies: Tests relying on live APIs, networks, or time-sensitive data (e.g., datetime.now()).
• Timing Issues: Asynchronous code (e.g., asyncio) or race conditions.

How to Avoid It

• Isolate Tests: Use pytest fixtures with scope="function" to reset state between tests (e.g., a fresh database connection per test).
• Mock External Dependencies: Replace live APIs/databases with mocks (e.g., unittest.mock) to control inputs.
• Avoid Time-Dependent Code: Use freezegun to mock datetime or time functions.
• Retry Flaky Tests (Temporarily): Tools like pytest-rerunfailures can rerun failed tests to identify flakiness, but fix the root cause long-term.

Example

A test that checks if a “daily report” is generated might fail if run at 11:59 PM vs. 12:01 AM. Fix it with freezegun:

from freezegun import freeze_time  

def test_daily_report_generation():  
    with freeze_time("2023-10-01 09:00:00"):  # Mock time to a fixed value  
        report = generate_daily_report()  
        assert report.date == "2023-10-01"  

4. Over-Mocking

The Pitfall

Mocks are powerful, but overusing them (e.g., mocking every function call) leads to tests that:

• Are hard to read (too many mocks).
• Don’t validate real behavior (they test “mocks interact correctly” instead of “code works”).

How to Avoid It

• Mock Only What You Own: Mock external systems (e.g., requests.get), but use real code for internal logic.
• Prefer Fakes Over Mocks: Use lightweight “fake” implementations (e.g., an in-memory SQLite database instead of a mock DB driver).
• Keep Mocks Simple: Avoid over-specifying mocks (e.g., don’t check mock.assert_called_with(exact_arg) unless critical).

Example

# Bad: Over-mocking internal logic  
def test_user_service_over_mocked():  
    with patch("user_service.get_db"), \  
         patch("user_service.validate_email"), \  
         patch("user_service.hash_password"):  
        UserService().create_user("[email protected]", "pass")  
        # Too many mocks; tests nothing useful  


# Good: Mock only external DB, use real validation/hashing  
def test_user_service_fake_db():  
    db = InMemoryDB()  # Fake DB, not a mock  
    service = UserService(db=db)  
    user = service.create_user("[email protected]", "pass")  
    assert db.get_user(user.id) is not None  # Tests real behavior  

5. Insufficient Test Coverage (or Obsessing Over It)

The Pitfall

• Too Little Coverage: Tests that miss critical code paths (e.g., error handlers, conditional branches).
• Blindly Chasing 100% Coverage: Obsessing over coverage metrics leads to “coverage theater”—tests that hit lines but don’t validate behavior.

How to Avoid It

• Use Coverage Tools: pytest-cov identifies untested code, but focus on critical paths (e.g., payment processing) over trivial ones (e.g., simple getters).
• Test for Behavior, Not Lines: A test with 80% coverage that validates edge cases is better than 100% coverage with useless tests.

Example

# Run pytest with coverage report  
pytest --cov=my_module tests/  

This outputs which lines are untested, helping you target gaps (e.g., an untested except ValueError block).

6. Ignoring Test Readability

The Pitfall

Tests are code too! Unreadable tests (e.g., vague names, messy setup, overly complex logic) are hard to debug when they fail.

How to Avoid It

• Clear Naming: Use descriptive names like test_checkout_returns_error_when_cart_empty instead of test_checkout_1.
• Simplify Setup: Use pytest fixtures to reuse setup code (e.g., @pytest.fixture def user(): return User(...)).
• Keep Tests Short: A test should fit on one screen and test one behavior.

Example

# Bad: Unreadable test  
def test_login():  
    u = User(email="[email protected]", pwd="123")  
    db.add(u)  
    db.commit()  
    t = client.post("/login", data={"e": "[email protected]", "p": "123"})  
    assert t.status_code == 200  


# Good: Readable test with fixtures  
@pytest.fixture  
def db_setup():  
    db = InMemoryDB()  
    db.add(User(email="[email protected]", password="hashed_pass"))  
    return db  

def test_login_success(db_setup, client):  
    response = client.post(  
        "/login",  
        data={"email": "[email protected]", "password": "hashed_pass"}  
    )  
    assert response.status_code == 200  
    assert "token" in response.json()  

7. Not Testing Error Conditions

The Pitfall

Many tests only validate “success” scenarios but ignore errors (e.g., invalid inputs, missing data). This leaves your code vulnerable to crashes when things go wrong.

How to Avoid It

• Test Expected Exceptions: Use pytest.raises to verify that invalid inputs raise the right errors.
• Validate Error Messages: Ensure errors are descriptive (e.g., “Email is required” instead of “Invalid input”).

Example

def test_divide_by_zero():  
    with pytest.raises(ZeroDivisionError) as exc_info:  
        divide(5, 0)  
    assert "division by zero" in str(exc_info.value)  # Validate error message  


def test_create_user_invalid_email():  
    with pytest.raises(ValueError) as exc_info:  
        UserService().create_user(email="not-an-email", password="pass")  
    assert "Invalid email format" in str(exc_info.value)  

8. Skipping Test Automation

The Pitfall

Manual testing (e.g., running pytest locally before commits) is error-prone and slow. Without automation, regressions slip into production.

How to Avoid It

• Integrate with CI/CD: Use GitHub Actions, GitLab CI, or Jenkins to run tests on every commit.
• Block Merges on Failures: Configure CI to prevent merging PRs if tests fail.

Example GitHub Actions Workflow

# .github/workflows/tests.yml  
name: Tests  
on: [push, pull_request]  

jobs:  
  test:  
    runs-on: ubuntu-latest  
    steps:  
      - uses: actions/checkout@v4  
      - uses: actions/setup-python@v5  
        with: { python-version: "3.11" }  
      - run: pip install -r requirements.txt  
      - run: pytest tests/ --cov=my_module  

Conclusion

Testing in Python is as much about avoiding pitfalls as it is about writing tests. By focusing on edge cases, behavior over implementation, readability, and automation, you’ll build a test suite that catches bugs, supports refactoring, and scales with your codebase. Remember: good tests are a safety net, not a burden.

References

• pytest Documentation
• hypothesis: Property-Based Testing
• pytest-cov
• Freezegun
• Testing on the Toilet: Avoid Over-Mocking
• Python Testing with pytest (Book by Brian Okken)