← Back to Docs

DIAGS PYTHON

Documentation for DIAGS_PYTHON from the Ds ex repository.

Of course. Given that you’re porting DSPy to Elixir (DSPEx), I will create a series of architectural diagrams and explanations that focus on the core concepts, data flows, and design patterns of the original Python DSPy. This will help you understand not just what the code does, but why it’s structured the way it is, which is crucial for a successful port.

I will structure this analysis from a high-level overview down to the specifics of key components like optimization and execution.

1. High-Level Conceptual Architecture

This diagram shows the main layers of the DSPy framework. It separates the user-facing programming model from the optimization (compilation) and execution (runtime) layers.

graph TD subgraph UserApp["User Application"] A["User-Defined Program: e.g., RAG(dspy.Module)"] end subgraph DSPyFramework["DSPy Framework"] subgraph ProgrammingModel["Programming Model"] B("Primitives: dspy.Module, dspy.Predict, dspy.Signature") end subgraph Optimization["Teleprompter (Compiler)"] C("Teleprompter: BootstrapFewShot, MIPRO, etc.") D{{"Trainset (dspy.Example)"}} E("Metric Function") F["Optimized Program (with demos/instructions)"] end subgraph Execution["Runtime"] G("Adapter: ChatAdapter, JSONAdapter") H("LM Client: dspy.LM") I("RM Client: dspy.Retrieve") J{{"Cache"}} end end subgraph ExternalServices["External Services"] K("LLM API: OpenAI, Gemini") L("Vector DB / Search: Weaviate, Colbert") end A -- "Uses" --> B C -- "Takes as input" --> A C -- "Uses" --> D C -- "Uses" --> E C -- "Outputs" --> F F -- "Is a" --> A A -- "Calls at runtime" --> G A -- "Calls at runtime" --> I G -- "Calls" --> H H -- "Manages" --> J H -- "Calls" --> K I -- "Calls" --> L

Architectural Insights & Elixir Porting Notes:

  • Separation of Concerns: DSPy elegantly separates the what (the program logic defined in dspy.Module) from the how (the specific prompt format, which is handled by the Adapter and Teleprompter).
  • Compilation vs. Runtime: The “Teleprompter” is a compiler. It takes a program (student), data (trainset), and an objective (metric) and outputs a new, optimized program. This compilation is often a slow, data-intensive process.
    • For DSPEx: This compilation step is a perfect candidate for massive concurrency using Task.async_stream or distributed computing across a BEAM cluster, as your dspex/evaluate.ex file suggests you’re already doing.
  • Runtime Execution: A forward() call on a program is the runtime execution path. This is where the Adapter and LM clients come into play to make a single prediction.
  • State: The “optimized” state (few-shot examples, improved instructions) is stored within the dspy.Predict modules of the returned program. In Python, this is done by modifying the object’s attributes. In Elixir, your DSPEx.OptimizedProgram struct correctly captures this by wrapping the original program and adding demos, which is the idiomatic functional approach.

2. Core Primitives and Program Composition

This diagram details how users build programs using DSPy’s core building blocks.

graph TD subgraph Core Primitives P_Module[primitives/module.py
dspy.Module
Base class for composition] P_Predict[predict/predict.py
dspy.Predict
A single LLM call] P_Signature[signatures/signature.py
dspy.Signature
I/O contract for Predict] P_Example[primitives/example.py
dspy.Example
Data record] P_Retrieve[retrieve/retrieve.py
dspy.Retrieve
A single retrieval call] end subgraph Example User Program: RAG RAG["RAG(dspy.Module)"] RAG_Retrieve["self.retrieve = dspy.Retrieve(k=3)"] RAG_CoT["self.generate_answer = dspy.ChainOfThought(...)"] end subgraph Built-in Modules CoT[predict/chain_of_thought.py
dspy.ChainOfThought] ReAct[predict/react.py
dspy.ReAct] end RAG -- "Inherits from" --> P_Module RAG -- "Contains" --> RAG_Retrieve RAG -- "Contains" --> RAG_CoT RAG_Retrieve -- "Is an instance of" --> P_Retrieve RAG_CoT -- "Is an instance of" --> CoT CoT -- "Is a" --> P_Module CoT -- "Internally uses" --> P_Predict ReAct -- "Is a" --> P_Module ReAct -- "Internally uses" --> P_Predict P_Predict -- "Is configured with a" --> P_Signature

Architectural Insights & Elixir Porting Notes:

  • Composition over Inheritance: Users build complex programs by composing smaller, pre-defined modules (ChainOfThought, Retrieve) inside their own dspy.Module. This is a powerful pattern.
  • dspy.Predict is the Atomic Unit: The most fundamental action is dspy.Predict, which represents a single, specific query to an LLM. More complex modules like ChainOfThought and ReAct are themselves dspy.Modules that orchestrate one or more dspy.Predict calls.
  • dspy.Signature is the “Type Spec”: A signature is not just a docstring; it’s a structured object that defines the input and output fields for a Predict module. It’s used by the Adapter to format prompts and parse outputs correctly. Your macro-based DSPEx.Signature is an excellent, compile-time-safe Elixir equivalent.
  • State Management: An optimized program is one where the demos attribute of its dspy.Predict instances has been populated by a Teleprompter. Your DSPEx.OptimizedProgram struct handles this immutably.

3. The Execution Flow: A forward() Call

This diagram traces a single call to a dspy.Predict module at runtime. This is the most critical flow to understand for the core runtime.

sequenceDiagram participant User participant Program as MyRAG(dspy.Module) participant Predict as dspy.Predict participant Settings as dspy.settings participant Adapter as dspy.ChatAdapter participant LM as dspy.LM participant Cache as dspy.Cache participant LLM_API as External LLM API User->>+Program: program(question="...") Program->>+Predict: self.generate_answer(context=..., question=...) Predict->>+Settings: Get current Adapter and LM Settings-->>-Predict: adapter, lm Predict->>+Adapter: format(signature, demos, inputs) Adapter-->>-Predict: formatted_messages Predict->>+LM: __call__(messages=formatted_messages) LM->>+Cache: get(request_key) alt Cache Hit Cache-->>LM: Cached Response else Cache Miss Cache-->>LM: Not Found LM->>+LLM_API: HTTP Request (messages) LLM_API-->>-LM: Raw JSON Response LM->>Cache: set(request_key, response) end Cache-->>-LM: Response LM-->>-Predict: Raw JSON Response Predict->>+Adapter: parse(signature, response) Adapter-->>-Predict: Parsed structured output (dict) Predict-->>-Program: dspy.Prediction object Program-->>-User: Final dspy.Prediction object

Architectural Insights & Elixir Porting Notes:

  • Global Settings (dspy.settings): This is a key architectural choice in Python. It’s a thread-local global object that holds the configured LM, RM, and Adapter. This makes it easy to swap out backends without changing the program code.
    • For DSPEx: This pattern is less common in Elixir. Your DSPEx.Services.ConfigManager GenServer is the correct idiomatic replacement. Instead of a global, you fetch configuration from a known, stateful process.
  • The Adapter’s Role: The Adapter is the crucial middleman. It takes the high-level, structured Signature and Examples and translates them into the specific string format required by the LLM (e.g., a JSON object for some models, or a specific chat message format for others). It also does the reverse (parse). This decouples the program’s logic from the LLM’s prompt engineering quirks.
  • Caching: Caching is built into the dspy.LM client layer. It hashes the request parameters (prompt, model, temperature, etc.) to create a cache key. This is a critical feature for reducing costs and speeding up development, especially during teleprompting.
  • Client Abstraction: The dspy.LM class wraps litellm, which provides a unified interface to hundreds of LLMs. This is a powerful abstraction. Your DSPEx.ClientManager and DSPEx.Client appear to be building a similar, robust abstraction using Elixir’s strengths (GenServers, supervision).

4. The Optimization Flow: The “Compilation” Loop

This diagram shows how a Teleprompter like BootstrapFewShot optimizes a program.

graph TD subgraph Inputs A["Student Program (unoptimized)"] B["Trainset (labeled dspy.Examples)"] C["Teacher Program (optional, often a more powerful LLM)"] D["Metric Function (e.g., exact_match)"] end subgraph Teleprompter["BootstrapFewShot.compile()"] E{Start Compilation} --> F{For each example in Trainset} F --> G["1 Generate a trace using the Teacher"] G --> H{2 Evaluate the trace with the Metric} H -- "Score >= Threshold" --> I["3 Collect this trace as a candidate demo"] H -- "Score < Threshold" --> F I --> F F -- "End of Trainset" --> J["4 Gather all candidate demos"] J --> K["5 Select the best k demos for each predictor"] K --> L["6 Create a new Optimized Program"] L --> M["7 Inject selected demos into the new program's Predictors"] end subgraph Output N["Optimized Program (with few-shot demos)"] end A --> E B --> E C --> E D --> E M --> N

Architectural Insights & Elixir Porting Notes:

  • Data-Driven Optimization: This is the core idea of DSPy. Instead of manually writing prompts, you provide data and a metric, and the framework learns the best few-shot demonstrations to include in the prompt.
  • Teacher/Student Model: The teacher (a more powerful/expensive model) is used to generate high-quality outputs (the “reasoning” in Chain of Thought, for example). These outputs become part of the demonstrations for the student (a cheaper/faster model), effectively teaching it to perform the task well.
  • Parallelism is Key: Step F (For each example...) is embarrassingly parallel. This is where the performance of the teleprompter is determined. The Python implementation uses dspy.utils.parallelizer.ParallelExecutor, which is a wrapper around ThreadPoolExecutor.
    • For DSPEx: Your use of Task.async_stream in dspex/teleprompter/bootstrap_fewshot.ex is the perfect, more performant BEAM equivalent. It will handle I/O-bound LLM calls much more efficiently than Python threads.
  • The Output is a Program: The result of compilation isn’t just a set of prompts; it’s a new, complete, and executable dspy.Module instance that has been “programmed” with the optimal demonstrations.

Summary for Your DSPEx Port

Your current architecture for DSPEx seems to be on the right track by translating Python’s imperative/object-oriented patterns into Elixir’s functional and concurrent ones.

  1. State Management: You’ve correctly identified that Python’s global dspy.settings should be replaced by a managed process like your ConfigManager. This is more robust and idiomatic in Elixir.
  2. Concurrency: You’ve correctly identified that the evaluation and optimization loops are prime candidates for Task.async_stream, which will be a significant performance advantage over the Python implementation.
  3. Core Primitives: Your DSPEx.Signature, DSPEx.Program, and DSPEx.Example modules are the right foundation. The macro-based DSPEx.Signature is particularly clever, providing compile-time safety that Python lacks.
  4. Client Layer: Building your DSPEx.ClientManager on GenServers is the right long-term approach for resilience, allowing you to easily add supervision, circuit breakers (Fuse), and stateful features like rate limiting.

By understanding these core architectural patterns from the original DSPy, you can ensure your DSPEx port is not only functionally equivalent but also idiomatically Elixir and architecturally superior in terms of concurrency and fault tolerance.