Of course. Based on the provided documentation and our existing architectural analysis, here is a detailed technical plan for porting DSPy to Elixir (DSPEx).
This plan extracts the necessary technical information from the provided documentation and organizes it into a phased implementation strategy. It highlights key decisions, dependencies, and challenges for each component.
Project Plan: Porting DSPy to DSPEx
This document outlines the technical requirements and implementation strategy for creating DSPEx, an Elixir-native port of the DSPy framework.
Guiding Principles
- Embrace OTP: Do not just transliterate Python. Re-architect components to leverage OTP primitives (
GenServer,Supervisor,Task,Stream) for concurrency, state management, and fault tolerance. - Declarative API: Maintain DSPy’s user-friendly, declarative nature through Elixir macros and behaviours.
- Phased Implementation: Focus on a stable core before building advanced optimizers and features.
- Isolate Python Dependencies: Acknowledge that some features (e.g., local model serving, finetuning) are heavily Python-based. Isolate these dependencies via well-defined interop boundaries (
Port, NIFs).
Phase 1: The Foundational Core
Objective: Implement the minimum viable components to define, run, and evaluate a basic DSPy program. This aligns with the “Polishing the core functionality” objective from the DSPy roadmap.
| Component / Feature | Python (DSPy) Implementation | Elixir (DSPEx) Technical Plan & Info Needed | Docs Reference |
|---|---|---|---|
| 1. Primitives | dspy.Signature, dspy.InputField, dspy.OutputField, dspy.Example, dspy.Prediction, dspy.Module, dspy.Program | DSPEx.Signature: Create a defsignature macro to parse string signatures ("in -> out") and class-based definitions into a compile-time struct. Info Needed: How to handle advanced types like Literal, list[dict[str, str]], and custom types (dspy.Image, dspy.History). This will require a robust type-parsing and validation system within the macro.DSPEx.Example & Prediction: Implement as simple Elixir structs. The .with_inputs() method will return a new, immutable struct. Prediction will need a completions field, which itself is a list of structs.DSPEx.Program: Define as a behaviour with forward/2 and acall/2 callbacks. | learn/programming/signatures.md, learn/evaluation/data.md, api/modules/Module.md |
2. LM Client (dspy.LM) | Thin wrapper around litellm. Handles configuration, caching, and retries. | DSPEx.Client.LM: Implement as a named, supervised GenServer. Info Needed:1. Configuration: Holds provider, model name, API key, etc. Fetches config from the OTP Application environment. 2. Request Handling: The public API ( request/2) will cast to the GenServer. The GenServer will spawn a supervised Task for the actual HTTP call to avoid blocking.3. Retries: The supervising process (the one calling request/2) will trap exits from the Task and implement retry logic (e.g., using a library like backoff). This replaces Python’s tenacity.4. Streaming & Async: The streamify feature requires the client to support streaming responses. The Task can handle this by streaming the HTTP response body back to the caller via messages. | roadmap.md (On LMs), api/models/LM.md, api/utils/streamify.md |
3. RM Client (dspy.Retrieve) | Wrappers for various vector DBs (ColBERT, Weaviate, etc.). | DSPEx.Client.RM: Define a retrieve/2 behaviour. Info Needed: A strategy for each backend.1. HTTP-based RMs (e.g., ColBERTv2): Implement similarly to the LM client.2. Library-based RMs (e.g., Weaviate, Pinecone): These require Elixir client libraries. If they don’t exist, we must decide between writing new ones or using Python interop (via Ports). The interop seems more practical initially. Each RM module will manage its own Port to a dedicated Python script. | api/tools/ColBERTv2.md, various retrieve/*.py files |
| 4. Basic Modules | dspy.Predict, dspy.ChainOfThought | DSPEx.Predict: A simple module implementing the Program behaviour. Its forward/2 function will call the specified LM client.DSPEx.ChainOfThought: A module that programmatically modifies the signature to add a reasoning field before calling DSPEx.Predict. This demonstrates module composition. | learn/programming/modules.md, api/modules/Predict.md, api/modules/ChainOfThought.md |
5. Evaluation (dspy.Evaluate) | Uses a thread pool to run a program over a dev set. | DSPEx.Evaluate.run/3: This function will be the primary evaluation entry point. Technical Plan:1. Use Task.async_stream/3 to process the entire dev set concurrently. Each example will run in its own lightweight process. This is a direct, more efficient replacement for Python’s ThreadPoolExecutor.2. The metric will be an Elixir function reference ( &MyMetric.calculate/2).3. Needs to handle and report failures gracefully from the stream of results. | learn/evaluation/overview.md, api/evaluation/Evaluate.md |
Phase 2: Core Optimization Logic (Teleprompters)
Objective: Implement the core optimizers that represent DSPy’s main value proposition. This is a complex phase requiring careful state management.
| Component / Feature | Python (DSPy) Implementation | Elixir (DSPEx) Technical Plan & Info Needed | Docs Reference |
|---|---|---|---|
| 1. Optimizer Base | dspy.teleprompt.Teleprompter | DSPEx.Teleprompter: A behaviour defining a compile/3 callback. | learn/optimization/optimizers.md |
2. Few-Shot Learning (BootstrapFewShot) | Bootstraps examples by running a teacher program and filtering traces based on a metric. | DSPEx.Teleprompter.BootstrapFewShot: Implement as a supervised GenServer. Technical Plan:1. The compile call will cast to the GenServer, which starts the process asynchronously.2. The GenServer’s state will hold the trainset and the growing list of bootstrapped demos.3. In its main loop, it will use Task.async_stream to run the teacher program over the trainset.4. It will process the stream of results, apply the metric, and add successful traces to its state. 5. Finally, it will construct and return the optimized program. | api/optimizers/BootstrapFewShot.md, faqs.md |
3. Instruction Optimization (MIPROv2) | A complex optimizer involving bootstrapping, instruction proposal, and Bayesian Optimization. | DSPEx.Teleprompter.MIPROv2: Implement as a supervised GenServer. Technical Plan & Info Needed:1. State Management: The GenServer state is more complex, tracking candidate instructions, few-shot demo sets, and trial scores.2. Sub-programs: The “grounded proposal stage” is itself a DSPy program. The GenServer will invoke this sub-program to generate new instructions.3. Bayesian Optimization: This is a major external dependency. We need to find or build an Elixir library for Bayesian Optimization (e.g., a port of scikit-optimize or similar). If none exists, this is a significant roadblock that might require Python interop.4. Minibatching: The optimizer evaluates on minibatches. The GenServer will manage sampling from the valset for each trial. | api/optimizers/MIPROv2.md, roadmap.md |
4. Finetuning (BootstrapFinetune) | Generates a dataset from traces and launches a finetuning job. | DSPEx.Teleprompter.BootstrapFinetune: This is the most significant Python interop challenge. Technical Plan:1. The GenServer will perform the data generation step, creating a JSONL file with training examples.2. It will then use an Elixir Port to execute a Python script. This script will handle the actual finetuning using libraries like transformers and trl.3. The GenServer will monitor the Port for completion status and the path to the finetuned model artifacts. Info Needed: The exact format of the training data generated by the Python version’s adapters needs to be replicated. | roadmap.md (On Finetuning), api/optimizers/BootstrapFinetune.md |
Phase 3: Advanced Modules & Features
Objective: Implement more complex modules and features that improve control, interactivity, and power.
| Component / Feature | Python (DSPy) Implementation | Elixir (DSPEx) Technical Plan & Info Needed | Docs Reference |
|---|---|---|---|
1. Agentic Modules (dspy.ReAct) | An iterative module that uses tools to answer questions. It maintains a stateful “trajectory.” | DSPEx.ReAct: Implement as a GenServer per forward call. Technical Plan:1. The forward call spawns a temporary ReAct GenServer.2. The GenServer’s state is the trajectory.3. In its loop, it spawns Tasks for LM calls (to generate thought/action) and tool calls.4. Tools will be passed as Elixir module/function references ( &MyTool.run/1).5. This naturally isolates the state of each agentic run. | api/modules/ReAct.md |
2. Code Execution (dspy.ProgramOfThought, dspy.CodeAct) | Uses a PythonInterpreter to execute generated code in a sandbox. | DSPEx.CodeAct: This requires a secure code execution sandbox. Technical Plan & Info Needed:1. The dspy.PythonInterpreter uses Deno. We can replicate this by creating a Port to a long-lived Deno process from a GenServer. The GenServer would manage the sandbox lifecycle.2. The logic for passing tools (as functions) into the sandbox needs to be carefully designed. The Python version injects code; the Elixir version would need to serialize the tool’s definition and inject it into the Deno/Python environment. This is a complex interop task. | api/modules/CodeAct.md |
| 3. Assertions | dspy.Assert and dspy.Suggest use backtracking and dynamic signature modification to enforce constraints. | DSPEx.Assert & DSPEx.Suggest: This can be implemented more idiomatically in OTP. Technical Plan:1. An assertion function will not raise an exception. Instead, upon failure, it will send a message (e.g., {:assertion_failed, feedback}) to its own process PID.2. The ProgramProcess (from Diagram 2) will have a handle_info clause for this message.3. Upon receiving the failure message, it can decide to re-run the last Task with a modified prompt, effectively implementing the backtrack. The number of retries is managed in the ProgramProcess state. This avoids exceptions for control flow. | learn/programming/7-assertions.md |
| 4. Streaming | dspy.streamify, StreamListener, StatusMessage | DSPEx.Streaming: Elixir is built for this. Technical Plan:1. Implement using Elixir’s native Stream module or, for more power (back-pressure), GenStage.2. A streamify function will wrap a Program, returning a Stream.3. The LM client must support yielding chunks from the HTTP response.4. StreamListener and StatusMessage would be GenStage consumers that transform the raw token stream into structured events. | api/utils/streamify.md, api/utils/StreamListener.md |
Phase 4: Productionization & Developer Experience
Objective: Add features that make DSPEx robust for production use, including saving, loading, caching, and observability.
| Component / Feature | Python (DSPy) Implementation | Elixir (DSPEx) Technical Plan & Info Needed | Docs Reference |
|---|---|---|---|
| 1. Caching | In-memory cachetools and on-disk diskcache. | DSPEx.Cache: Implement as a supervised GenServer + ETS table. Technical Plan:1. In-memory: Use an ETS table for O(1) concurrent reads. The GenServer will handle writes.2. On-disk: The GenServer will manage file I/O for persistence, replacing diskcache. The pyproject.toml dependency diskcache>=5.6.0 confirms its importance.3. The configure_cache function will start/configure this supervised service. | faqs.md, api/utils/configure_cache.md |
| 2. Saving & Loading | program.save() and dspy.load() using JSON and cloudpickle. | DSPEx.Program.save/2: Info Needed:1. For simple state (instructions, demos), serialize to JSON. 2. For complex, compiled programs, Elixir’s :erlang.term_to_binary/1 is the native equivalent of cloudpickle, allowing the entire state of a GenServer or the compiled program struct to be saved and loaded reliably. We will prefer this over JSON for full program state. | faqs.md, api/utils/load.md |
| 3. Observability & Tracking | Integrations with MLflow, logging, inspect_history. | DSPEx.Observability: Technical Plan:1. Logging: Use Elixir’s standard Logger library.2. History: The dspy.inspect_history functionality can be replicated by having LM clients log all requests to a central, in-memory store (e.g., an ETS table or Agent) when enabled.3. Tracing: Replicate the with_callbacks decorator using :telemetry handlers. This is the standard Elixir way to create hooks. An external library like opentelemetry_telemetry can then automatically convert these events into OpenTelemetry traces for tools like MLflow. | roadmap.md, production/index.md |