Python’s Interpreter Pattern: Building Custom Interpreters

What is the Interpreter Pattern?
Core Components of the Interpreter Pattern
When to Use the Interpreter Pattern
Python Implementation: Building a Simple Arithmetic Interpreter
Advantages and Disadvantages
Real-World Applications
Conclusion
References

What is the Interpreter Pattern?

The Interpreter Pattern is defined by the “Gang of Four” (GoF) as:
“Given a language, define a representation for its grammar along with an interpreter that uses the representation to interpret sentences in the language.”

In simpler terms, it lets you model a language’s grammar as a set of classes, where each grammar rule is represented by an “interpreter” object. These interpreters collaborate to evaluate or execute expressions written in the language.

Key Idea:

At its core, the Interpreter Pattern transforms abstract syntax (grammar rules) into concrete behavior (evaluation logic). It uses a composite structure to represent expressions, where simple “terminal” expressions (e.g., numbers, variables) are combined into complex “non-terminal” expressions (e.g., addition, subtraction) using recursion.

Core Components of the Interpreter Pattern

To implement the Interpreter Pattern, you’ll need the following components:

1. Abstract Expression

An abstract base class (or interface) that declares an interpret method. This method is responsible for evaluating or executing the expression. All concrete expressions (terminal and non-terminal) implement this interface.

2. Terminal Expression

Represents the “leaves” of the grammar—simple, indivisible elements (e.g., numbers, variable names). They implement the interpret method to return a value directly (e.g., returning the value of a number or looking up a variable).

3. Non-Terminal Expression

Represents complex expressions formed by combining terminal or other non-terminal expressions (e.g., a + b, x > 5). They implement interpret by recursively interpreting their child expressions and combining the results.

4. Context

A data structure that holds global information (e.g., variable values, configuration) needed by the interpreter. It’s passed to the interpret method to resolve dependencies like variables.

5. Client

Constructs the Abstract Syntax Tree (AST)—a tree representation of the input expression—using the terminal and non-terminal expressions. The Client then invokes interpret on the root of the AST to evaluate the expression.

When to Use the Interpreter Pattern

The Interpreter Pattern shines in specific scenarios:

Small, Simple Grammars: It’s ideal for languages with a limited set of rules (e.g., 2–5 operations).
Frequent Evaluation: When expressions in the language need to be evaluated repeatedly (e.g., real-time rule engines).
Easy Extensibility: When you need to add new grammar rules without rewriting existing code.

When to Avoid It:

For complex grammars (e.g., SQL, Python itself), use parser generators like ANTLR or Lark instead—they handle ambiguity, optimization, and scalability better.
For performance-critical applications with large inputs, as the pattern can become inefficient due to recursion and object overhead.

Python Implementation: Building a Simple Arithmetic Interpreter

Let’s build a practical example: a custom interpreter for a simple arithmetic language that supports:

Numbers (e.g., 42)
Variables (e.g., x, y)
Operations: addition (+), subtraction (-), and later multiplication (*).

We’ll use Python’s abc module for abstract base classes and demonstrate all core components.

Step 1: Define the Abstract Expression

First, create an abstract base class (ABC) for all expressions. It declares the interpret method, which takes a context (dictionary of variables) and returns a value.

from abc import ABC, abstractmethod

class AbstractExpression(ABC):
    @abstractmethod
    def interpret(self, context: dict) -> int:
        """Evaluate the expression using the provided context."""
        pass

Step 2: Implement Terminal Expressions

Terminal expressions represent the simplest elements of our language: numbers and variables.

Number Terminal

Evaluates to a fixed integer value (ignores the context).

class Number(AbstractExpression):
    def __init__(self, value: int):
        self.value = value

    def interpret(self, context: dict) -> int:
        return self.value  # Numbers don't depend on context

Variable Terminal

Looks up a variable’s value from the context.

class Variable(AbstractExpression):
    def __init__(self, name: str):
        self.name = name  # Name of the variable (e.g., "x")

    def interpret(self, context: dict) -> int:
        if self.name not in context:
            raise ValueError(f"Variable '{self.name}' not defined in context.")
        return context[self.name]

Step 3: Implement Non-Terminal Expressions

Non-terminal expressions combine other expressions using operations. Let’s start with addition and subtraction.

Addition

Adds the results of two child expressions.

class Add(AbstractExpression):
    def __init__(self, left: AbstractExpression, right: AbstractExpression):
        self.left = left  # Left operand (e.g., Number(3) or Variable("x"))
        self.right = right  # Right operand

    def interpret(self, context: dict) -> int:
        # Recursively interpret left and right, then add
        return self.left.interpret(context) + self.right.interpret(context)

Subtraction

Subtracts the right operand from the left.

class Subtract(AbstractExpression):
    def __init__(self, left: AbstractExpression, right: AbstractExpression):
        self.left = left
        self.right = right

    def interpret(self, context: dict) -> int:
        return self.left.interpret(context) - self.right.interpret(context)

Step 4: Tokenization and Parsing

To evaluate a string input (e.g., "x + 5 - y"), we need to:

Tokenize: Split the input into meaningful tokens (e.g., ["x", "+", "5", "-", "y"]).
Parse: Convert tokens into an AST using our terminal/non-terminal expressions.

Tokenizer

A simple function to split the input string into tokens (whitespace-separated):

def tokenize(expression: str) -> list[str]:
    """Split an input string into tokens (variables, numbers, operators)."""
    return expression.strip().split()

Parser

A recursive descent parser to build the AST. For simplicity, we’ll handle left-associative operations (e.g., a + b - c is (a + b) - c).

def parse(tokens: list[str]) -> AbstractExpression:
    """Parse tokens into an Abstract Syntax Tree (AST)."""
    if not tokens:
        raise ValueError("Empty expression.")

    # Helper to check if a token is a number
    def is_number(token: str) -> bool:
        return token.lstrip('-').isdigit()  # Handle negative numbers

    # Build the root of the AST (leftmost token)
    left_token = tokens[0]
    if is_number(left_token):
        left = Number(int(left_token))
    else:
        left = Variable(left_token)  # Assume variables are non-numeric tokens

    # Process remaining tokens (operator + operand pairs)
    i = 1
    while i < len(tokens):
        operator = tokens[i]
        right_token = tokens[i + 1]

        # Create right operand (Number or Variable)
        if is_number(right_token):
            right = Number(int(right_token))
        else:
            right = Variable(right_token)

        # Combine left and right with the operator
        if operator == '+':
            left = Add(left, right)
        elif operator == '-':
            left = Subtract(left, right)
        else:
            raise ValueError(f"Unknown operator: '{operator}'")

        i += 2  # Move to next operator-operand pair

    return left  # Root of the AST

Step 5: Interpret the Expression

Now, let’s tie it all together. The Client will:

Accept an input string (e.g., "x + 3 - y").
Tokenize and parse it into an AST.
Evaluate the AST using a context (variable values).

Client Code

if __name__ == "__main__":
    # Example 1: Evaluate "3 + 4 - 2" (no variables)
    expression1 = "3 + 4 - 2"
    tokens1 = tokenize(expression1)
    ast1 = parse(tokens1)
    result1 = ast1.interpret(context={})  # No variables needed
    print(f"{expression1} = {result1}")  # Output: 3 + 4 - 2 = 5

    # Example 2: Evaluate "x + 5 - y" with variables
    expression2 = "x + 5 - y"
    tokens2 = tokenize(expression2)
    ast2 = parse(tokens2)
    context = {"x": 10, "y": 3}  # Define variables
    result2 = ast2.interpret(context)
    print(f"{expression2} (x=10, y=3) = {result2}")  # Output: x + 5 - y (x=10, y=3) = 12

Extending the Language: Adding Multiplication

The Interpreter Pattern makes it easy to extend the grammar. Let’s add multiplication (*) by:

Adding a new non-terminal expression class:

class Multiply(AbstractExpression):
    def __init__(self, left: AbstractExpression, right: AbstractExpression):
        self.left = left
        self.right = right

    def interpret(self, context: dict) -> int:
        return self.left.interpret(context) * self.right.interpret(context)

Updating the parser to handle *:

# In the parse function's operator handling:
elif operator == '*':
    left = Multiply(left, right)

Now we can evaluate expressions like "x * 2 + 3":

expression3 = "x * 2 + 3"
tokens3 = tokenize(expression3)
ast3 = parse(tokens3)
result3 = ast3.interpret({"x": 4})  # (4 * 2) + 3 = 11
print(f"{expression3} (x=4) = {result3}")  # Output: x * 2 + 3 (x=4) = 11

Advantages and Disadvantages

Advantages

Easy to Extend: Adding new grammar rules (e.g., *, /) requires only new NonTerminalExpression classes and minor parser updates.
Simple for Small Languages: No need for complex parser generators—hand-coding is feasible for small rule sets.
Clear Separation of Concerns: Each expression type (e.g., Add, Variable) is a separate class, making code modular.

Disadvantages

Complex Grammars Become Unwieldy: With 10+ operations, the number of classes and parser logic grows exponentially.
Inefficient for Large Inputs: Recursion and object overhead can slow down evaluation of large/complex expressions.
Hard to Debug: ASTs can be deep and complex, making debugging errors in expressions challenging.

Real-World Applications

The Interpreter Pattern is used in many tools and frameworks, often for embedded or domain-specific languages:

Django Template Language: Interprets template tags (e.g., {% if user.is_authenticated %}) by parsing and evaluating expressions.
Configuration Parsers: Tools like ansible or docker-compose use simplified interpreters to parse YAML/JSON configs with custom logic (e.g., variable interpolation).
Testing Frameworks: pytest uses an interpreter to evaluate custom assertion expressions (e.g., assert x > 5).
Rule Engines: Business rule engines (e.g., for insurance eligibility) interpret custom rule expressions (e.g., age > 18 AND income > 50000).

Conclusion

The Interpreter Pattern is a powerful tool for building custom interpreters for small, focused languages. By modeling grammar rules as classes and combining them into an AST, you can easily evaluate expressions tailored to your domain.

Use it when you need a lightweight, extensible solution for simple languages. For complex grammars, opt for parser generators like ANTLR or Lark. With Python’s flexibility, implementing the Interpreter Pattern is straightforward—empowering you to build everything from tiny calculators to embedded rule engines.

References

Gamma, E., Helm, R., Johnson, R., & Vlissides, J. (1994). Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley.
Python Software Foundation. (n.d.). Abstract Base Classes (abc). https://docs.python.org/3/library/abc.html
Fowler, M. (2010). Domain-Specific Languages. Addison-Wesley.
Lark Parser. (n.d.). Lark: A Modern Parser Generator for Python. https://lark-parser.readthedocs.io/

Table of Contents