Debugging and Testing with Python's Standard Library

Table of Contents

1. Debugging Tools
   • 1.1 pdb: The Python Debugger
   • 1.2 logging: Beyond Print Statements
   • 1.3 traceback: Detailed Error Tracking
2. Testing Tools
   • 2.1 unittest: The Built-in Testing Framework
   • 2.2 doctest: Testing via Docstrings
   • 2.3 unittest.mock: Mocking and Patching
3. Best Practices
4. Conclusion
5. References

Debugging Tools

Debugging is the process of identifying, isolating, and resolving errors in code. Python’s standard library offers several tools to simplify this process.

1.1 pdb: The Python Debugger

The pdb module is Python’s built-in interactive debugger, providing a command-line interface to inspect code execution. It allows you to pause execution, inspect variables, step through code, and set breakpoints.

Key Features:

• Breakpoints: Pause execution at specific lines.
• Stepping: Execute code line-by-line (step into, over, or out of functions).
• Inspection: Examine variables, call stacks, and code context.

Getting Started:

To use pdb, you can either:

• Run a script directly with pdb:

  python -m pdb my_script.py  

• Insert a breakpoint within the script (Python 3.7+):

  breakpoint()  # Equivalent to pdb.set_trace() in older versions  

Common pdb Commands:

Example: Debugging a Buggy Function

Suppose we have a function sum_list that’s supposed to sum elements of a list but returns incorrect results:

def sum_list(numbers):  
    total = 1  # Bug: Should initialize to 0  
    for num in numbers:  
        total += num  
    return total  

# Let's debug this!  
breakpoint()  # Execution pauses here  
result = sum_list([1, 2, 3])  
print(f"Sum: {result}")  # Expected: 6, Actual: 7  

Running the script triggers pdb. Use n to step through, p total to check the variable, and you’ll spot the bug (initializing total to 1 instead of 0).

1.2 logging: Beyond Print Statements

While print() is useful for quick checks, the logging module offers a flexible, configurable way to track code execution. It supports different severity levels, output destinations (files, consoles), and formatting—making it far more powerful than ad-hoc print statements.

Key Features:

• Severity Levels: DEBUG, INFO, WARNING, ERROR, CRITICAL (control verbosity).
• Configuration: Define output format, log files, and handlers.
• Flexibility: Enable/disable logs without modifying code.

Basic Usage:

import logging  

# Configure logging (run once at startup)  
logging.basicConfig(  
    level=logging.DEBUG,  # Capture all levels from DEBUG upwards  
    format="%(asctime)s - %(levelname)s - %(message)s",  # Include timestamp  
    filename="app.log"  # Log to a file (omit to log to console)  
)  

def divide(a, b):  
    logging.debug(f"Dividing {a} by {b}")  # Detailed debug info  
    try:  
        result = a / b  
        logging.info(f"Success: {a}/{b} = {result}")  # General info  
        return result  
    except ZeroDivisionError:  
        logging.error("Cannot divide by zero!")  # Error condition  
        return None  

divide(10, 2)  
divide(5, 0)  

Output (app.log):

2024-05-20 14:30:00,123 - DEBUG - Dividing 10 by 2  
2024-05-20 14:30:00,124 - INFO - Success: 10/2 = 5.0  
2024-05-20 14:30:00,124 - DEBUG - Dividing 5 by 0  
2024-05-20 14:30:00,124 - ERROR - Cannot divide by zero!  

Why logging Over print()?

• Levels: Filter logs by severity (e.g., disable DEBUG logs in production).
• Persistence: Log to files for post-mortem analysis.
• Formatting: Include timestamps, module names, or process IDs for context.

1.3 traceback: Detailed Error Tracking

When exceptions occur, Python prints a traceback—a stack trace showing where the error originated. The traceback module lets you programmatically access and customize these tracebacks, making it easier to log or display detailed error information.

Common Functions:

• traceback.print_exc(): Print the traceback for the last exception.
• traceback.format_exc(): Return the traceback as a string (for logging).

Example: Logging Tracebacks

import traceback  

def risky_operation():  
    return 1 / 0  # Raises ZeroDivisionError  

try:  
    risky_operation()  
except Exception:  
    # Log the full traceback to a file  
    with open("error.log", "w") as f:  
        traceback.print_exc(file=f)  
    print("An error occurred. Check error.log for details.")  

Output (error.log):

Traceback (most recent call last):  
  File "script.py", line 8, in <module>  
    risky_operation()  
  File "script.py", line 4, in risky_operation  
    return 1 / 0  
ZeroDivisionError: division by zero  

Testing Tools

Testing ensures code works as intended. Python’s standard library includes unittest (for structured unit tests), doctest (for testing via docstrings), and unittest.mock (for mocking external dependencies).

2.1 unittest: The Built-in Testing Framework

Inspired by Java’s JUnit, unittest is a full-featured testing framework for writing and running unit tests. It supports test cases, assertions, setup/teardown logic, and test discovery.

Core Concepts:

• Test Case: A class inheriting from unittest.TestCase, containing test methods (named test_*).
• Assertions: Methods like assertEqual(a, b) or assertTrue(condition) to verify outcomes.
• Setup/Teardown: setUp() (runs before each test) and tearDown() (runs after each test) for shared logic.

Example: Testing a Function

Let’s test a simple add function:

# math_functions.py  
def add(a, b):  
    return a + b  

Create a test file test_math_functions.py:

import unittest  
from math_functions import add  

class TestAdd(unittest.TestCase):  
    def test_add_positive_numbers(self):  
        self.assertEqual(add(2, 3), 5)  # 2 + 3 should be 5  

    def test_add_negative_numbers(self):  
        self.assertEqual(add(-1, -1), -2)  

    def test_add_zero(self):  
        self.assertEqual(add(0, 5), 5)  

if __name__ == "__main__":  
    unittest.main()  # Run tests when the script is executed  

Running Tests:

python test_math_functions.py  

Output:

...  
----------------------------------------------------------------------  
Ran 3 tests in 0.001s  

OK  

Advanced Features:

• Test Discovery: Run all tests in a directory with python -m unittest discover.
• Skipping Tests: Use @unittest.skip("Reason") to skip tests conditionally.
• Expected Failures: @unittest.expectedFailure for known bugs.

2.2 doctest: Testing via Docstrings

doctest extracts and runs Python code examples from docstrings, ensuring documentation stays in sync with code behavior. It’s ideal for simple tests and illustrative examples.

How It Works:

doctest searches for lines starting with >>> (simulating the Python REPL) and checks if the output matches the expected result.

Example: Testing with Docstrings

# string_utils.py  
def reverse_string(s):  
    """Reverse a string.  

    Examples:  
    >>> reverse_string("hello")  
    'olleh'  

    >>> reverse_string("")  
    ''  

    >>> reverse_string("a")  
    'a'  
    """  
    return s[::-1]  

Run doctest from the command line:

python -m doctest -v string_utils.py  # -v for verbose output  

Output:

Trying:  
    reverse_string("hello")  
Expecting:  
    'olleh'  
ok  
Trying:  
    reverse_string("")  
Expecting:  
    ''  
ok  
Trying:  
    reverse_string("a")  
Expecting:  
    'a'  
ok  
1 items had no tests:  
    string_utils  
1 items passed all tests:  
   3 tests in string_utils.reverse_string  
3 tests in 1 items.  
3 passed and 0 failed.  
Test passed.  

Pros/Cons:

• Pros: Simple, integrates with documentation, no extra test files.
• Cons: Not ideal for complex tests; sensitive to whitespace/formatting.

2.3 unittest.mock: Mocking External Dependencies

unittest.mock (built into Python 3.3+) lets you replace real objects with “mocks” to isolate code during testing. It’s critical for testing code that interacts with databases, APIs, or other external systems.

Key Components:

• Mock: A flexible mock object to simulate dependencies.
• patch: A decorator/context manager to temporarily replace objects (e.g., requests.get).

Example: Mocking an API Call

Suppose we have a function that fetches data from a weather API:

# weather.py  
import requests  

def get_weather(city):  
    url = f"https://api.weather.com/{city}"  
    response = requests.get(url)  
    return response.json()["temperature"]  

To test get_weather without hitting the real API, mock requests.get:

# test_weather.py  
import unittest  
from unittest.mock import patch, Mock  
from weather import get_weather  

class TestGetWeather(unittest.TestCase):  
    @patch("weather.requests.get")  # Mock requests.get in the weather module  
    def test_get_weather_success(self, mock_get):  
        # Configure the mock to return a JSON response  
        mock_response = Mock()  
        mock_response.json.return_value = {"temperature": 22}  
        mock_get.return_value = mock_response  

        # Call the function under test  
        temp = get_weather("London")  

        # Assertions  
        self.assertEqual(temp, 22)  
        mock_get.assert_called_once_with("https://api.weather.com/London")  # Verify API URL  

if __name__ == "__main__":  
    unittest.main()  

Explanation:

• @patch("weather.requests.get") replaces requests.get with a mock during the test.
• mock_get is the mock object passed to the test method.
• We configure mock_response.json() to return {"temperature": 22}, simulating a successful API call.

Best Practices

1. Combine Debugging and Testing: Write tests first (TDD) to catch bugs early, then use pdb to debug failures.
2. Use logging Over print(): Avoid cluttering code with print statements; use logging levels to control verbosity.
3. Isolate Tests: Use unittest.mock to ensure tests don’t depend on external systems (databases, APIs).
4. Test Edge Cases: Include tests for invalid inputs, empty values, and boundary conditions.
5. Document Tests: Use docstrings in test cases to explain what’s being tested.

Conclusion

Python’s standard library provides a rich toolkit for debugging and testing, from the interactive pdb debugger to the structured unittest framework. These tools are lightweight, reliable, and require no external dependencies, making them essential for every Python developer. By mastering pdb, logging, unittest, and doctest, you’ll write more robust, maintainable code and catch bugs before they reach production.

References

• Python pdb Documentation
• Python logging Documentation
• Python traceback Documentation
• Python unittest Documentation
• Python doctest Documentation
• Python unittest.mock Documentation