← Back to Tool bridge

PYTHON TOOL BRIDGE

Documentation for PYTHON_TOOL_BRIDGE from the Dspex repository.

Of course. Building a robust remote procedure call (RPC) bridge between two different language runtimes requires a clear plan. Here is a detailed technical specification for creating a tool bridge that allows Python’s dspy.ReAct module, running via Snakepit, to securely and efficiently call back into Elixir functions.

Technical Specification: DSPex Elixir-Python Tool Bridge

1. Overview

1.1. Purpose The primary goal of this specification is to define an architecture for a bidirectional communication bridge that enables the dspy.ReAct module in Python to invoke and receive results from tool functions written in Elixir. This allows developers to leverage the BEAM’s concurrency and fault tolerance for tool implementations while using DSPy’s advanced reasoning capabilities.

1.2. Problem Statement The dspy.ReAct module requires its tools to be Python callable objects (e.g., functions or class instances with a __call__ method). Elixir functions are not directly callable from the Python runtime. A simple placeholder, as identified in the debugging session, results in a ValidationError.

1.3. Solution Architecture The proposed solution is a synchronous, in-band RPC mechanism built on top of the existing Snakepit port communication protocol. The bridge will:

  1. Register Elixir functions in a secure, session-aware registry on the Elixir side.
  2. Proxy these registered functions as callable Python objects (RPCProxyTool) on the Python side.
  3. When a Python proxy tool is called, it will send a special RPC request back to the Elixir worker process that initiated the dspy.ReAct call.
  4. The Elixir worker will dispatch the call to the correct Elixir function, execute it, and send the result back to the Python process.
  5. The Python process will receive the result and return it, unblocking the dspy.ReAct module to continue its reasoning loop.

This architecture avoids the complexity of out-of-band communication (e.g., separate HTTP servers or message queues) by reusing the existing, performant Port connection.

2. Core Components

2.1. Elixir-Side Components (dspex)

  1. DSPex.ToolRegistry (New Module)

    • Type: GenServer.
    • Responsibility: Securely stores mappings from a unique tool_id (string) to an Elixir Module-Function-Arity (MFA).
    • State: A Map holding {tool_id => {module, function, arity}}.
    • API:
      • start_link/1: Starts the GenServer.
      • register(fun :: function()) :: {:ok, tool_id :: String.t()}: Accepts an Elixir function, captures its MFA, generates a unique and secure ID, stores the mapping, and returns the ID.
      • lookup(tool_id :: String.t()) :: {:ok, mfa} | {:error, :not_found}: Retrieves the MFA for a given ID.

  2. Snakepit.Pool.Worker (Modification)

    • Responsibility: This existing GenServer needs to be enhanced to handle the new in-band RPC calls from the Python bridge.
    • Logic: Its handle_info({port, {:data, data}}, state) function will be modified to recognize a new message type for RPC calls, dispatch the execution, and send the result back through the port.

2.2. Python-Side Components (enhanced_bridge.py)

  1. RPCProxyTool (New Class)

    • Type: A standard Python class.
    • Responsibility: Acts as the callable proxy that dspy.Tool will wrap. It will be instantiated with a tool_id and a reference to the ProtocolHandler.
    • Methods:
      • __init__(self, tool_id, protocol_handler): Stores the tool_id and the handler for communication.
      • __call__(self, *args, **kwargs): The core RPC logic. This method will:
        1. Construct an rpc_tool_call request packet.
        2. Send it to Elixir using protocol_handler.write_message().
        3. Block and wait for the corresponding rpc_tool_response using protocol_handler.read_message().
        4. Return the result from the response.

  2. EnhancedCommandHandler (Modification)

    • Responsibility: The _execute_dynamic_call method will be updated to detect when it’s creating a dspy.ReAct module. It will transform the placeholder tool definitions from Elixir into instances of RPCProxyTool.

3. Communication Protocol Extension

The existing Snakepit length-prefixed protocol will be extended with two new message types for the RPC mechanism.

3.1. Python -> Elixir: RPC Call Request

When the RPCProxyTool is called, it sends this message to the Elixir worker’s port.

  • Format: JSON or MessagePack
  • Schema:
    {
  "type": "rpc_call",
  "rpc_id": "<unique_id_for_this_call>",
  "tool_id": "<id_from_tool_registry>",
  "args": [...], // Positional arguments
  "kwargs": {...} // Keyword arguments
}

3.2. Elixir -> Python: RPC Call Response

The Elixir worker sends this message back through the port after executing the tool function.

  • Format: JSON or MessagePack
  • Schema (Success):
    {
  "type": "rpc_response",
  "rpc_id": "<same_id_as_request>",
  "status": "ok",
  "result": <return_value_from_elixir_function>
}
  • Schema (Error):
    {
  "type": "rpc_response",
  "rpc_id": "<same_id_as_request>",
  "status": "error",
  "error": {
    "type": "<elixir_exception_type>", // e.g., "RuntimeError"
    "message": "<exception_message>",
    "stacktrace": "<optional_stacktrace_string>"
  }
}

4. End-to-End Workflow

  1. Setup (DSPex.Modules.ReAct.create/3):

    • An Elixir developer defines a list of tools: tools = [%{name: "search", func: &MyApp.search/1, ...}].
    • DSPex.Modules.ReAct.create/3 is called. It iterates through the tools.
    • For each tool, it calls DSPex.ToolRegistry.register(tool.func), which returns a unique tool_id.
    • It constructs the kwargs for the dspy.ReAct Python call, replacing the Elixir function with the tool_id: python_tools = [%{name: "search", tool_id: "tool_abc123", ...}].
    • The Snakepit.Python.call("dspy.ReAct", %{tools: python_tools, ...}) command is sent.

  2. Python-Side Instantiation (EnhancedCommandHandler):

    • The _execute_dynamic_call method receives the request to instantiate dspy.ReAct.
    • It sees the tool_id in the tool definitions. For each one, it creates an RPCProxyTool(tool_id, self.protocol_handler).
    • It then creates the final dspy.Tool object, passing the RPCProxyTool instance as the func argument: dspy.Tool(func=rpc_proxy_instance, ...).
    • The dspy.ReAct module is instantiated with these valid, callable Python tool objects and stored.

  3. Runtime Execution (dspy.ReAct calls a tool):

    • The dspy.ReAct module decides to use the “search” tool and calls it: search_tool("what is elixir").
    • This invokes the __call__ method on the corresponding RPCProxyTool instance.

  4. RPC Call (Python -> Elixir):

    • RPCProxyTool.__call__ generates a unique rpc_id and sends an rpc_call message through its protocol_handler to the Elixir port.
    • The Python bridge then blocks, waiting for a response message with the matching rpc_id.

  5. RPC Dispatch (Elixir):

    • The Snakepit.Pool.Worker’s handle_info receives the port data.
    • It decodes the message and sees %{type: "rpc_call"}.
    • It calls DSPex.ToolRegistry.lookup(tool_id) to get the MFA.
    • It safely executes the function using apply(module, function, args). It wraps this call in a try/catch block to handle Elixir exceptions.

  6. RPC Response (Elixir -> Python):

    • If the Elixir function succeeds, the worker constructs and sends an rpc_response message with status: "ok" and the result.
    • If the function fails, it sends an rpc_response with status: "error" and the exception details.
    • The worker then goes back to waiting for more port messages (either another RPC call or the final response from dspy.ReAct).

  7. Runtime Resumption (Python):

    • The RPCProxyTool.__call__ method receives the rpc_response, extracts the result (or raises a Python exception if an error was returned), and returns it.
    • The dspy.ReAct module receives the result from the tool call and continues its execution.

  8. Final Response:

    • Eventually, dspy.ReAct finishes its work and returns a final Prediction object.
    • The EnhancedCommandHandler serializes this final result and sends it back to the Snakepit.Pool.Worker as the response to the original execute call, completing the entire workflow.

5. Security Considerations

  • Function Whitelisting: The ToolRegistry acts as a natural whitelist. Only functions explicitly registered during setup can be called from Python. Python cannot invoke arbitrary Elixir code.
  • Data Serialization: All arguments and return values are passed as standard data types (strings, numbers, lists, maps), not executable code, preventing code injection vulnerabilities across the boundary.
  • Resource Limits: The execution happens within the context of the Elixir worker process. Standard BEAM process limits and timeouts can be applied to prevent runaway tool functions from destabilizing the system.

6. Future Enhancements

  • Asynchronous Tools: The initial design is synchronous. A future version could support async tools by having the Elixir worker immediately acknowledge the RPC call and send the result later in a separate message, requiring a more complex state machine on the Python side.
  • Streaming Tools: For tools that produce continuous output (e.g., tailing a log file), the protocol could be extended to support multiple rpc_stream_chunk messages followed by a final rpc_stream_end message.
  • Argument/Return Type Marshalling: Add support for more complex Elixir types (structs, tuples) by defining a clear serialization/deserialization scheme.