Looking at the DSPEx implementation plan and the Ash Framework/AshAI documentation, here’s how they would fit into the DSPEx port:
Ash Framework Integration Points
Stage 1-2: Foundation & Client
No immediate integration needed - The core signature parsing, client, and HTTP layers should remain independent of Ash for maximum flexibility.
Stage 3: Adapters & Message Formatting
Optional Integration Point:
- Could create an
DSPEx.Adapter.Ash
that formats prompts using Ash resource schemas - This would allow DSPEx signatures to be automatically derived from Ash resource attributes
# Example: Auto-generate DSPEx signatures from Ash resources
defmodule DSPEx.Adapter.Ash do
def signature_from_resource(resource_module, action_name) do
# Extract input/output fields from Ash action definition
# Generate DSPEx signature automatically
end
end
Stage 4: Prediction Engine
Key Integration Point:
- DSPEx.Example could be backed by Ash resources for persistence
- Training/dev sets could be stored as Ash resources with proper querying
- Demo management could use Ash’s data layer abstraction
# Enhanced example storage
defmodule MyApp.TrainingData do
use Ash.Resource, extensions: [AshAi]
attributes do
attribute :question, :string
attribute :answer, :string
attribute :quality_score, :float
end
actions do
read :high_quality_examples do
filter expr(quality_score > 0.8)
end
end
end
Stage 5-6: Evaluation & Advanced Features
Major Integration Opportunity:
- Tool Integration: DSPEx programs could expose Ash actions as LLM tools via AshAI’s tool system:
defmodule MyApp.Domain do
use Ash.Domain, extensions: [AshAi]
tools do
tool :search_knowledge, MyApp.Knowledge, :search
tool :create_ticket, MyApp.Support.Ticket, :create
end
end
# DSPEx program that can use these tools
defmodule SmartAssistant do
use DSPEx.Signature, "user_request -> response, actions_taken"
def forward(program, inputs) do
# DSPEx orchestrates the LLM reasoning
# AshAI provides the actual tool execution
DSPEx.Predict.forward(program, inputs, tools: MyApp.Domain.tools())
end
end
- Vectorization for RAG: AshAI’s vectorization would be perfect for DSPEx retrieval:
defmodule MyApp.KnowledgeBase do
use Ash.Resource, extensions: [AshAi]
vectorize do
full_text do
text(fn record -> "#{record.title}: #{record.content}" end)
end
strategy :ash_oban
embedding_model MyApp.OpenAiEmbeddingModel
end
actions do
read :vector_search do
argument :query_vector, {:array, :float}
filter expr(vector_cosine_distance(full_text_vector, ^query_vector) < 0.7)
sort vector_cosine_distance(full_text_vector, ^query_vector)
end
end
end
# DSPEx retriever using Ash vectorization
defmodule DSPEx.Retriever.Ash do
def retrieve(query, opts) do
embedding = generate_embedding(query)
MyApp.KnowledgeBase
|> Ash.Query.for_read(:vector_search, %{query_vector: embedding})
|> MyApp.Api.read!()
end
end
AshAI Integration Strategy
Complementary, Not Competitive
DSPEx and AshAI solve different problems and would work excellently together:
- DSPEx: Optimizes prompts and reasoning chains
- AshAI: Provides tool execution and data integration
Integration Architecture:
# DSPEx handles the reasoning optimization
defmodule CustomerSupportAgent do
use DSPEx.Signature, "customer_issue, context -> response, recommended_actions"
# AshAI provides the tools this agent can use
@tools [
:search_knowledge_base,
:create_ticket,
:escalate_to_human,
:update_customer_record
]
end
# AshAI defines what those tools actually do
defmodule MyApp.Support do
use Ash.Domain, extensions: [AshAi]
tools do
tool :search_knowledge_base, MyApp.Knowledge, :search
tool :create_ticket, MyApp.Tickets, :create
tool :escalate_to_human, MyApp.Escalation, :create
tool :update_customer_record, MyApp.Customers, :update
end
end
# Combined system
defmodule OptimizedSupportBot do
def handle_customer_issue(issue_text) do
# 1. DSPEx optimizes the reasoning
agent = %CustomerSupportAgent{client: :openai}
|> DSPEx.Teleprompter.BootstrapFewShot.compile(training_data, metric_fn)
# 2. Execute with AshAI tools available
DSPEx.Program.forward(agent, %{
customer_issue: issue_text,
context: get_customer_context()
}, tools: MyApp.Support.tools())
end
end
Why No Phoenix Needed
You’re absolutely right - Phoenix isn’t needed for the core DSPEx port because:
- DSPEx is a library: Like DSPy, it’s meant to be embedded in applications
- Ash handles persistence: Data layer is covered by Ash resources
- HTTP client is built-in: DSPEx.Client handles LLM API communication
- Tool execution via AshAI: Web interfaces can be added later if needed
The only Phoenix use case would be if you wanted to build a web UI for DSPEx optimization runs, but that would be a separate application layer.
Recommended Integration Timeline
- Stages 1-4: Build DSPEx independently
- Stage 5: Add Ash integration for example/demo persistence
- Stage 6: Add AshAI tool integration for advanced reasoning
- Post-MVP: Add optional Phoenix UI for monitoring optimization runs
This approach gives you maximum flexibility while leveraging Ash’s strengths where they add the most value.