Excellent. This is the core innovation of your Elixir port and the key to solving the problem posed by the DSPy community. Visualizing how the Variable abstraction achieves this decoupling is crucial.
Here are a series of detailed diagrams that illustrate the architecture of the Variable system, focusing on how it decouples parameter definition from the optimization process, as per the design specification you provided.
Introduction: The Decoupling Principle
The fundamental shift introduced by the Variable system is moving from a world where an optimizer needs to know what it’s optimizing (e.g., a “prompt” or a “model parameter”) to a world where it only needs to know that it’s optimizing a “variable” with certain properties (e.g., a :discrete choice or a :continuous range).
This abstraction layer is what enables any optimizer to tune any parameter. The following diagrams break down how this is achieved architecturally.
Diagram 1: The Core Variable Abstraction
This diagram shows how different concrete, high-level concepts (like choosing an adapter or setting a temperature) are all unified under the single ElixirML.Variable struct. This is the first step of decoupling.
(JSON vs. Markdown)"] Param2["Reasoning Module
(Predict vs. CoT vs. PoT)"] Param3["LLM Temperature"] end subgraph "Unified Variable Abstraction" AbstractVar["ElixirML.Variable Struct
(The single, unified representation)"] end subgraph "Variable Implementation Examples" direction LR Struct1["%Variable{
id: :adapter,
type: :discrete,
choices: [JSON, MD]
}"] Struct2["%Variable{
id: :reasoning_module,
type: :discrete,
choices: [Predict, CoT, PoT]
}"] Struct3["%Variable{
id: :temperature,
type: :continuous,
range: {0.0, 2.0}
}"] end %% Connect Concrete to Abstract Param1 -- "Is represented as" --> AbstractVar Param2 -- "Is represented as" --> AbstractVar Param3 -- "Is represented as" --> AbstractVar %% Connect Abstract to Implementation AbstractVar -- "Is instantiated as" --> Struct1 AbstractVar -- " " --> Struct2 AbstractVar -- " " --> Struct3 %% Apply Styles class Param1,Param2,Param3 Concrete class AbstractVar Abstract class Struct1,Struct2,Struct3 StructDef
Explanation: Instead of having different types for different parameters, the system maps everything to a Variable struct. The optimizer doesn’t see “an adapter setting”; it sees a :discrete variable named :adapter with a list of choices. This abstraction is what allows any optimizer to work with any parameter.
Diagram 2: Defining the Search Space
A DSPEx.Program uses the Variable DSL to declare which of its parameters are tunable. These declarations are collected into a Variable.Space, which defines the complete, unified search space for that program.
use ElixirML.Resource
variable :adapter, :discrete,
choices: [:json, :markdown]
variable :temperature, :continuous,
range: {0.1, 1.2}
variable :reasoning, :module,
modules: [Predict, CoT]
end"] end subgraph "Compilation / Instantiation Process" Extractor["Variable Extractor
(Compile-time or Runtime)"] end subgraph "Resulting Search Space" Space["Variable.Space Struct"] Var1["%Variable{id: :adapter, type: :discrete, ...}"] Var2["%Variable{id: :temperature, type: :continuous, ...}"] Var3["%Variable{id: :reasoning, type: :module, ...}"] end %% Flow Program -- "1 Declares optimizable parameters" --> Extractor Extractor -- "2 Populates" --> Space Space -- "Contains" --> Var1 Space -- "Contains" --> Var2 Space -- "Contains" --> Var3 %% Apply Styles class Program ProgramDef class Extractor Process class Space,Var1,Var2,Var3 Artifact
Explanation: The developer declares variables directly within their program module. The framework then extracts these declarations to create a Variable.Space struct. This Variable.Space is a self-contained, serializable description of the entire optimizable surface of the program.
Diagram 3: The Decoupling Point - Optimizer Interaction
This is the most critical diagram. It shows that the optimizer only interacts with the Variable.Space, not with the program’s internal details. This is the “decoupling” Omar Khattab described.
(e.g., SIMBA)"] subgraph "The Decoupling Interface" VariableSpace["Variable.Space"] end %% The Abstraction Boundary is now represented by this subgraph. %% It contains the internal implementation details that the optimizer cannot see directly. subgraph "Implementation Details (Hidden from Optimizer)" direction LR ProgramAdapter["Adapter Selection Logic"] ProgramModule["Reasoning Module Logic"] ProgramParam["LLM Parameter Settings"] end %% Optimizer interacts ONLY with the VariableSpace Optimizer -- "1 Asks: 'What are the variables?'" --> VariableSpace VariableSpace -- "2 Responds: \`%{adapter: %Variable{...}}\`" --> Optimizer Optimizer -- "3 Suggests new configuration:
\`%{adapter: :json, temperature: 0.9}\`" --> VariableSpace %% The VariableSpace is the only bridge across the boundary VariableSpace -- "4 Is used to configure" --> ProgramAdapter VariableSpace -- " " --> ProgramModule VariableSpace -- " " --> ProgramParam %% The optimizer has no direct knowledge of these internals (visualized as red dashed lines) Optimizer -.-> ProgramAdapter Optimizer -.-> ProgramModule Optimizer -.-> ProgramParam %% Apply Styles to nodes class Optimizer Optimizer class VariableSpace Interface class ProgramAdapter,ProgramModule,ProgramParam Program %% Apply Styles to specific links to create the red dashed "forbidden" connections %% The numbers correspond to the link order in the diagram (0-indexed) linkStyle 6 stroke:red,stroke-dasharray:5 5,stroke-width:2px linkStyle 7 stroke:red,stroke-dasharray:5 5,stroke-width:2px linkStyle 8 stroke:red,stroke-dasharray:5 5,stroke-width:2px
Explanation: The optimizer (e.g., SIMBA) is completely blind to what :adapter or :temperature actually mean. It only queries the VariableSpace to understand the search landscape (e.g., “there’s a discrete variable named ‘adapter’ with two choices”). It then proposes a new configuration (a simple map like %{adapter: :json, ...}), which is then applied to the program by a separate mechanism. The optimizer’s logic is completely generic and reusable for any set of variables.
Diagram 4: End-to-End Adaptive Optimization Workflow
This final diagram shows the full loop, putting all the pieces together to answer Maxime’s question: “how are they all evaluated and selected automatically?”
(e.g., SIMBA / BEACON)"] subgraph "1 Propose Configuration" Optimizer -- "Samples from" --> VariableSpace["Variable Space"] VariableSpace -- "Generates" --> Configuration["Candidate Configuration
\`e.g., %{adapter: :json, ...}\`"] end subgraph "2 Apply & Evaluate" ProgramTemplate["Base Program Template"] ConfigApplicator["Configuration Applicator"] ConfiguredProgram["Configured Program Instance"] EvaluationSystem["Evaluation System"] Devset["Development Dataset"] FinalScore["Multi-Objective Score
(Accuracy, Cost, Latency)"] Configuration --> ConfigApplicator ProgramTemplate --> ConfigApplicator ConfigApplicator --> ConfiguredProgram ConfiguredProgram -- "Runs on" --> Devset Devset --> EvaluationSystem ConfiguredProgram -- "Is evaluated by" --> EvaluationSystem EvaluationSystem --> FinalScore end subgraph "3 Update & Repeat" FinalScore -- "Feedback to" --> Optimizer end end Optimizer -- "Loop until budget exhausted" --> Optimizer Optimizer -- "Outputs" --> BestConfiguration["Best Found Configuration"] %% Apply Styles class Optimizer Optimizer class ProgramTemplate,Devset,VariableSpace,Configuration,FinalScore,BestConfiguration Data class EvaluationSystem,ConfigApplicator Evaluation
Explanation:
- The Optimizer starts and consults the Variable Space.
- It proposes a candidate configuration (e.g.,
%{adapter: JSONAdapter, reasoning: ChainOfThought, temperature: 0.9}). - A Configuration Applicator takes this map and the base program template to create a concrete, runnable instance of the program.
- The Evaluation System runs this specific instance against the devset.
- It calculates a multi-objective score (considering accuracy, cost, etc.).
- This score is fed back to the optimizer, which updates its internal models and proposes the next, hopefully better, configuration.
- This loop repeats until an optimization budget (e.g., number of trials) is met, finally yielding the best configuration found.
This system directly solves the challenge by creating an abstract layer (Variable) that allows any optimizer to explore a search space of program configurations without needing to know the specific implementation details of those configurations.