20250713 p2of4

Documentation for 20250713_p2of4 from the Dspex repository.

Excellent. Let’s proceed with the detailed plan for Phase 2.

This phase marks the strategic shift from relying entirely on the Python bridge to building a high-performance native core in Elixir. The key is to do this incrementally, replacing the components that will benefit most from OTP’s concurrency and fault tolerance, while leaving the more complex, dependency-heavy parts in Python for now.

4. Detailed Roadmap: Phase 2 - The Native Execution Core

Objective: To achieve a significant performance and reliability leap by re-implementing dspy’s core prediction and reasoning logic in native Elixir. The outcome will be a hybrid system where common workflows run natively, and the Python bridge becomes a tool for specialized tasks.

Month 3: Native Modules & Hybrid Execution

Week	Epic	Key Tasks & Implementation Details	Success Criteria
Week 9	(2.1) The Native Module System	Based on `stage2_03_module_system.md`: • `DSPex.Module.Native`: Implement the core `GenServer`-based module `behaviour`. This includes state management (`init`, `handle_call`), parameter tracking, and lifecycle hooks. • `DSPex.Module.Registry`: A new registry (or an extension of the existing adapter registry) to manage and discover running native module processes. • `DSPex.Module.State`: A struct defining the comprehensive state for a module, including parameters, history, and performance metrics. • `DSPex.Module.Parameter`: Implement the struct for managing learnable parameters with history tracking.	✅ A native module can be started, supervised, and have its state inspected via GenServer calls. ✅ The module registry can track and look up active native module PIDs.
Week 10	(2.2) Native `Predict` Module	Based on `stage2_05_prediction_pipeline.md`: • Create `DSPex.Native.Predict` implementing the `DSPex.Module.Native` behaviour. • Implement the `forward/2` pass which: 1. Builds a prompt from the signature and inputs. 2. Calls a native Elixir LM Client (see Epic 2.3). 3. Parses the LLM response to populate output fields. • Implement basic prompt templating logic.	✅ `DSPex.Native.Predict.forward(pid, %{...})` successfully calls an LLM API and returns a structured result without touching Python.
Week 11	(2.3) Native Provider Framework	Based on `stage2_01_signature_compilation.md` (Schema) & `stage2_03_module_system.md` (Client): • `DSPex.Providers.Client`: Implement a robust HTTP client using `Finch` for connection pooling. • `DSPex.Providers.OpenAI`: Create a native client for the OpenAI API that uses the `JsonSchema` from Stage 1 to enable function-calling. • `DSPex.Providers.Anthropic`: Create a native client for the Anthropic API. • Implement rate limiting and exponential backoff retry logic.	✅ The native `Predict` module can successfully use the `OpenAI` client to get a response. ✅ The framework can handle API errors and rate limits gracefully.
Week 12	(2.4) Hybrid Execution Engine	This is the critical integration step. • Enhance `DSPex.Program` Resource: Add an `execution_backend` field to each module definition within the program (`native` or `python`). • Enhance `ExecutionEngine`: The orchestrator must now be “backend-aware." - When a step uses a `:native` module, it calls the GenServer PID directly. - When a step uses a `:python` module, it calls the `PythonPort` adapter as before. • Update the `DataLayer` to handle this routing logic.	✅ A single `DSPex.Program` can be defined with a mix of native and Python-bridged modules and execute successfully, with the engine routing calls to the correct backend.

Month 4: Native Reasoning & Assertions

Week	Epic	Key Tasks & Implementation Details	Success Criteria
Week 13	(2.5) Native `ChainOfThought` Module	Based on `stage2_06_chain_of_thought.md`: • Implement `DSPex.Native.ChainOfThought` module. • `SignatureEnhancer`: Create logic to dynamically inject a `:reasoning` field into a signature for the CoT prompt. • Implement the `forward/2` pass to make one LLM call for the combined reasoning and answer, then parse the result. • `StepValidator`: A simple version to start, parsing the reasoning string into a list of steps.	✅ The native `ChainOfThought` module produces a structured output containing both the final answer and the reasoning chain.
Week 14	(2.6) Native `ReAct` Module & Tool System	Based on `stage2_07_react_patterns.md`: • `DSPex.React.Engine`: Implement the core “Thought -> Action -> Observation” loop in an Elixir process. • `ToolRegistry`: A GenServer to register and look up native Elixir functions as tools. • `ActionParser`: Parse the `Action[Tool](args)` string format generated by the LLM. • Implement `DSPex.Native.ReAct`: The module that orchestrates the ReAct loop, calling tools from the `ToolRegistry`.	✅ A `ReAct` program can successfully call a registered native Elixir tool (e.g., a simple `:calculator` tool) to solve a problem.
Week 15	(2.7) Native Assertion Framework	This is a major DSPy feature. Based on the `ASSERTIONS` section of the gap analysis: • Implement `dspy.Assert`: Create a mechanism where a validation failure within a `forward` pass triggers a retry. • The `ExecutionEngine` must catch a specific exception (e.g., `AssertionError`). • On catching the error, it should re-run the same LLM call but with an augmented prompt that includes the failure reason and a request to correct it. • Implement `dspy.Suggest`: A similar mechanism for “soft” constraints that adds feedback to the prompt but doesn’t cause a hard retry on failure.	✅ A program using `DSPex.Assert.ensure(condition, message)` will automatically retry with feedback upon failure, ultimately producing a valid output.
Week 16	(2.8) Native `BootstrapFewShot` Optimizer	Based on the `Optimizers` section of the gap analysis: • Implement a native version of this teleprompter. • Candidate Generation: The optimizer runs the student program on the training set using the native execution engine. • Filtering & Selection: It collects successful traces (where the prediction matched the ground truth) and filters them to use as few-shot examples. • Program Update: It updates the student program’s `Ash` resource with the newly compiled set of few-shot demonstrations.	✅ Can run `DSPex.Optimizers.BootstrapFewShot.compile(program, trainset)` and see a measurable improvement in the program’s score on a dev set.

Phase 2 Deliverables & Outcome

By the end of Phase 2, DSPex will be a true hybrid system with a powerful native core.

Key Deliverables:
1. A performant, fault-tolerant native execution engine for Predict, ChainOfThought, and ReAct.
2. A native tool-use framework.
3. A native assertion/self-correction framework.
4. The first native teleprompter (BootstrapFewShot).
5. A hybrid execution model that seamlessly combines native Elixir and Python-bridged modules.
Resulting State of the System:
- Performance: Common prediction and reasoning tasks will be significantly faster and more concurrent, free from the Python bridge bottleneck.
- Reliability: The most frequently used components will now be managed by OTP supervisors, making the application far more resilient.
- Capability: The system can now perform complex reasoning, tool use, and self-correction natively.
- Strategic Position: DSPex is no longer just a “wrapper.” It has a provably superior native core for orchestration while retaining access to the Python ecosystem for specialized, hard-to-port components like MIPRO. The “Frankenstein” risk is successfully mitigated in favor of a “cyborg” model.

This phase firmly establishes the technical superiority of the Elixir-based approach for the core orchestration and execution logic of an LLM programming model.

Shall we proceed to detail Phase 3?