Of course. Here is a detailed set of diagrams showcasing how our evolved DSPex Foundation layer, its dependencies, and the underlying foundation hex package integrate with the broader DSPEx architecture.
This provides a holistic view, from the highest-level application logic down to the infrastructure that powers it.
Overview of the Diagrams
- Diagram 1: The Layered Architecture Stack: A static, high-level view showing the layers of abstraction and dependency, from the user’s application down to Elixir/OTP. This illustrates what depends on what.
- Diagram 2: Dynamic Execution Flow of a
PredictCall: A sequence diagram tracing a single, successfulforwardcall through the entire stack. This illustrates who talks to whom, and when. - Diagram 3: Advanced Component Deep Dive - The Semantic Cache: A component interaction diagram showing how the
DSPex.Cacheuses its various dependencies (Nebulex,Cachex,Nx,Bumblebee,Rag) to fulfill a request. This illustrates how a complex component works internally.
Diagram 1: The DSPEx Layered Architecture Stack
Purpose: This diagram provides a static, “30,000-foot view” of the entire system architecture. It clearly delineates the responsibilities of each layer and shows the strict, downward-only dependency flow, which is a hallmark of a well-designed, maintainable system.
graph LR
subgraph " "
direction TB
subgraph "Layer 4: User Application / High Level Programs"
A1["User's Application Code"]
A2["\`DSPEx.Teleprompter\` (Optimizers)"]
end
subgraph "Layer 3: DSPEx Core Modules (The 'DSPy' Logics)"
B1["\`DSPEx.Predict\`"]
B2["\`DSPEx.ChainOfThought\`"]
B3["\`DSPEx.ReAct\`"]
B4["\`DSPEx.Evaluate\`"]
end
subgraph "Layer 2 DSPex Foundation (AI-Specific Infrastructure)"
C1["\`DSPEx.Workflow\` Engine"]
C2["\`DSPEx.Client\` (Provider Manager)"]
C3["\`DSPEx.PromptTemplate\` Engine"]
C4["\`DSPEx.Cache\` (Semantic/Tiered)"]
C5["\`DSPEx.Analytics\` (Cost/Perf)"]
end
subgraph "Layer 1: Elixir Dependency Ecosystem"
D1["\`Req\` / \`Jason\` (HTTP/JSON)"]
D2["\`Nebulex\` / \`Cachex\` (Caching)"]
D3["\`Nx\` / \`Axon\` / \`Bumblebee\` (ML)"]
D4["\`Rag\` / \`Explorer\` (Vector/Data)"]
end
subgraph "Layer 0: Generic \`foundation\` Hex Package"
E1["\`Foundation.Infrastructure\` (Fuse, Rate Limiter, Pool)"]
E2["\`Foundation.Observability\` (Events, Telemetry)"]
E3["\`Foundation.Services\` (Config, Registry)"]
end
subgraph "Base: Elixir OTP"
F1["Elixir / OTP (GenServer, Task, Supervisor)"]
end
end
A1 --> B1
A2 --> B4
B1 --> C1
B2 --> C1
B3 --> C2
B4 --> B1
C1 --> E2
C2 --> E1
C3 --> F1
C4 --> D2
C4 --> D3
C4 --> D4
C5 --> E2
C2 --> D1
D1 --> F1
D2 --> F1
D3 --> F1
D4 --> F1
E1 --> F1
E2 --> F1
E3 --> F1
linkStyle 10 stroke-width:2px,stroke:orange,fill:none;
linkStyle 11 stroke-width:2px,stroke:orange,fill:none;
linkStyle 12 stroke-width:2px,stroke:orange,fill:none;
linkStyle 13 stroke-width:2px,stroke:orange,fill:none;
style A1 fill:#e3f2fd,stroke:#1e88e5,color:#000
style A2 fill:#e3f2fd,stroke:#1e88e5,color:#000
style B1 fill:#e8f5e9,stroke:#388e3c,color:#000
style B2 fill:#e8f5e9,stroke:#388e3c,color:#000
style B3 fill:#e8f5e9,stroke:#388e3c,color:#000
style B4 fill:#e8f5e9,stroke:#388e3c,color:#000
style C1 fill:#fff3e0,stroke:#f57c00,color:#000
style C2 fill:#fff3e0,stroke:#f57c00,color:#000
style C3 fill:#fff3e0,stroke:#f57c00,color:#000
style C4 fill:#fff3e0,stroke:#f57c00,color:#000
style C5 fill:#fff3e0,stroke:#f57c00,color:#000
style D1 fill:#ede7f6,stroke:#5e35b1,color:#000
style D2 fill:#ede7f6,stroke:#5e35b1,color:#000
style D3 fill:#ede7f6,stroke:#5e35b1,color:#000
style D4 fill:#ede7f6,stroke:#5e35b1,color:#000
style E1 fill:#fce4ec,stroke:#c2185b,color:#000
style E2 fill:#fce4ec,stroke:#c2185b,color:#000
style E3 fill:#fce4ec,stroke:#c2185b,color:#000
style F1 fill:#fafafa,stroke:#37474f,color:#000
Diagram 2: Dynamic Execution Flow of a Predict Call
Purpose: This sequence diagram shows the real-time flow of a single, successful call through the stack. It highlights how DSPex Foundation components orchestrate work and delegate to the foundation hex package for resilience and resource management.
sequenceDiagram
actor User
participant Predict as DSPEx.Predict
participant Workflow as DSPEx.Workflow
participant Template as DSPEx.PromptTemplate
participant Client as DSPEx.Client
participant Foundation as Foundation.Infrastructure
participant Worker as "HttpWorker (via Req)"
participant LM_API as "External LM API"
User->>+Predict: forward(program, inputs)
Predict->>+Workflow: run(pipeline, initial_data)
Note over Workflow: Creates a parent ErrorContext for the entire trace.
Workflow->>+Template: render(template, vars)
Template-->>-Workflow: returns rendered_prompt
Workflow->>+Client: request(client_id, messages)
Note over Client: Calculates token costs for rate limiting.
Client->>+Foundation: execute_protected(..., api_call_fun)
Note over Foundation: Orchestrates:
1. Rate Limiter check (RPM/TPM)
2. Circuit Breaker check
3. Pool Checkout Foundation->>+Worker: post(url, body) Note over Worker: Uses Req to make the HTTP POST request. Worker->>+LM_API: POST /v1/completions LM_API-->>-Worker: 200 OK (response_body) Worker-->>-Foundation: {:ok, parsed_response} Foundation-->>-Client: {:ok, parsed_response} Note over Foundation: Checks worker back into pool. Client-->>-Workflow: {:ok, lm_response} Workflow->>Workflow: parse_response_step(...) Workflow-->>-Predict: {:ok, %{answer: "..."}} Predict-->>-User: {:ok, %DSPEx.Prediction{...}}
1. Rate Limiter check (RPM/TPM)
2. Circuit Breaker check
3. Pool Checkout Foundation->>+Worker: post(url, body) Note over Worker: Uses Req to make the HTTP POST request. Worker->>+LM_API: POST /v1/completions LM_API-->>-Worker: 200 OK (response_body) Worker-->>-Foundation: {:ok, parsed_response} Foundation-->>-Client: {:ok, parsed_response} Note over Foundation: Checks worker back into pool. Client-->>-Workflow: {:ok, lm_response} Workflow->>Workflow: parse_response_step(...) Workflow-->>-Predict: {:ok, %{answer: "..."}} Predict-->>-User: {:ok, %DSPEx.Prediction{...}}
Diagram 3: Advanced Component Deep Dive - The Semantic Cache
Purpose: This component interaction diagram illustrates the internal logic of the DSPEx.Cache module when a get_or_compute call is made. It shows the cascade of cache lookups and the integration of the ML embedding pipeline.
graph TD
subgraph " "
direction LR
A["Calling Program"] --> B{"DSPEx.Cache.get_or_compute"};
end
subgraph L1_Cache ["L1: In Memory (Cachex/ETS)"]
C{"Check for Exact Key"}
end
subgraph L2_Cache ["L2: On Disk (Persistent)"]
D{"Check for Exact Key"}
end
subgraph L3_Cache ["L3: Semantic (Vector Store)"]
E["Embedding Pipeline
(Bumblebee/Axon/Nx)"] F["Vector DB Client
(Rag)"] end subgraph Compute ["Compute Path"] G["Execute \`compute_fun\`
(Calls DSPEx.Client)"] end subgraph Write_Path ["Cache Write Path"] H["Write to L1"] I["Write to L2"] J["Write to L3"] end B -- "1. Key & Prompt Text" --> C; C -- "Miss" --> D; D -- "Miss" --> E; E -- "Generates Embedding" --> F; F -- "Semantic Search" --> B; C -- "Hit" --> Z([Return Value]); D -- "Hit" --> Z; F -- "Hit" --> Z; B -- "Total Miss" --> G; G -- "Result" --> H; H --> I; I --> J; J --> Z; classDef l1 fill:#e0f7fa,stroke:#00796b,stroke-width:2px,color:#000 classDef l2 fill:#e8eaf6,stroke:#303f9f,stroke-width:2px,color:#000 classDef l3 fill:#f3e5f5,stroke:#7b1fa2,stroke-width:2px,color:#000 classDef compute fill:#fbe9e7,stroke:#d84315,stroke-width:2px,color:#000 classDef write fill:#e8f5e9,stroke:#388e3c,stroke-width:2px,color:#000 class C L1_Cache; class D L2_Cache; class E,F L3_Cache; class G compute; class H,I,J write;
(Bumblebee/Axon/Nx)"] F["Vector DB Client
(Rag)"] end subgraph Compute ["Compute Path"] G["Execute \`compute_fun\`
(Calls DSPEx.Client)"] end subgraph Write_Path ["Cache Write Path"] H["Write to L1"] I["Write to L2"] J["Write to L3"] end B -- "1. Key & Prompt Text" --> C; C -- "Miss" --> D; D -- "Miss" --> E; E -- "Generates Embedding" --> F; F -- "Semantic Search" --> B; C -- "Hit" --> Z([Return Value]); D -- "Hit" --> Z; F -- "Hit" --> Z; B -- "Total Miss" --> G; G -- "Result" --> H; H --> I; I --> J; J --> Z; classDef l1 fill:#e0f7fa,stroke:#00796b,stroke-width:2px,color:#000 classDef l2 fill:#e8eaf6,stroke:#303f9f,stroke-width:2px,color:#000 classDef l3 fill:#f3e5f5,stroke:#7b1fa2,stroke-width:2px,color:#000 classDef compute fill:#fbe9e7,stroke:#d84315,stroke-width:2px,color:#000 classDef write fill:#e8f5e9,stroke:#388e3c,stroke-width:2px,color:#000 class C L1_Cache; class D L2_Cache; class E,F L3_Cache; class G compute; class H,I,J write;
These three diagrams provide a comprehensive, multi-faceted view of the proposed DSPEx architecture. They show how a clear, layered design allows for the integration of powerful, specialized libraries at each level, all while building on the solid, generic infrastructure provided by the foundation package.