← Back to Dev

NEXT STEPS

Documentation for NEXT_STEPS from the Ds ex repository.

DSPEx Next Steps - Comprehensive Development Plan

Executive Summary

DSPEx is well-positioned to become the definitive Elixir port of DSPy, leveraging BEAM’s unique strengths for distributed AI systems. After thorough analysis, the project has solid foundations but requires strategic realignment to achieve its ambitious vision.

Key Insight: The convergence of Foundation 2.0’s distributed infrastructure vision with DSPEx’s AI programming needs creates a unique opportunity to build something significantly better than the original Python DSPy.

Current State Analysis

✅ Strengths

  • Excellent Foundation: DSPEx.Signature and DSPEx.Example are production-ready with sophisticated macro-based parsing
  • Strong Engineering Practices: 100% test coverage on implemented features, zero Dialyzer warnings
  • Clear Architecture Vision: Well-documented layered dependency graph
  • Foundation Integration Path: Clear technical blueprints for leveraging Foundation library

⚠️ Critical Gaps

  • Documentation vs Reality Mismatch: Claims of “Phase 1 Complete” are premature
  • Missing Core Components: DSPEx.Evaluate and DSPEx.Teleprompter are completely absent
  • Basic Client Implementation: Current client is simple HTTP wrapper, not the envisioned GenServer-based resilient architecture
  • No Optimization Engine: The “self-improving” aspect of DSPy is not implemented

🎯 Strategic Opportunity

The Foundation 2.0 vision of distributed BEAM applications (1000+ node clusters, Partisan networking) perfectly aligns with DSPEx’s need for distributed AI workloads. This creates potential for DSPEx to be the first AI framework built for massive distributed inference and optimization.

Phase 1A: Foundation Consolidation (Weeks 1-2)

Objective: Complete the actual Phase 1 with Foundation integration

Critical Tasks:

  1. Implement True GenServer-based Client

    • Replace basic HTTP wrapper with supervised GenServer architecture
    • Integrate Foundation’s ConnectionManager, CircuitBreaker, and RateLimiter
    • Add multi-provider support (OpenAI, Anthropic, Gemini)
    • Location: lib/dspex/client.ex

  2. Complete DSPEx.Program Behavior

    • Define the missing behavior module referenced throughout codebase
    • Update DSPEx.Predict to properly implement the behavior
    • Add composition patterns for complex programs
    • Location: lib/dspex/program.ex

  3. Enhance Adapter Layer

    • Abstract provider-specific logic
    • Add signature manipulation helpers (for ChainOfThought)
    • Implement robust parsing with better error handling
    • Location: lib/dspex/adapter/

  4. Documentation Reality Check

    • Update CLAUDE.md to reflect actual implementation status
    • Remove “Phase 1 Complete” claims until actually complete
    • Separate architectural vision from current capabilities

Success Criteria:

  • GenServer-based client with circuit breakers working
  • Multi-provider support functional
  • All 126 tests passing with >90% coverage
  • Zero Dialyzer warnings maintained
  • Documentation matches implementation reality

Phase 1B: Basic Evaluation Engine (Weeks 3-4)

Objective: Implement minimal viable evaluation to enable optimization

Critical Tasks:

  1. DSPEx.Evaluate Core

    • Implement concurrent evaluation using Task.async_stream
    • Add basic metrics collection (accuracy, latency)
    • Progress tracking and error isolation
    • Location: lib/dspex/evaluate.ex

  2. DSPEx.Example Enhancements

    • Add validation functions (has_field?, equal?)
    • Implement utility functions (copy/2, without/2)
    • JSON serialization support
    • Location: lib/dspex/example.ex

  3. Test Infrastructure

    • Convert placeholder tests in test_phase2/ to real tests
    • Add property-based testing for evaluation edge cases
    • Mock framework for LLM responses during testing

Success Criteria:

  • Can evaluate programs against datasets
  • Concurrent evaluation working with fault isolation
  • Basic metrics collection functional
  • Performance benchmarking available

Phase 2: Optimization Engine (Weeks 5-8)

Objective: Implement the “self-improving” core of DSPy

Major Components:

  1. DSPEx.Teleprompter

    • BootstrapFewShot optimizer implementation
    • Teacher/student training pipeline
    • Performance-based demo filtering
    • Location: lib/dspex/teleprompter/

  2. Advanced Programs

    • DSPEx.ChainOfThought with reasoning injection
    • DSPEx.ReAct for agentic workflows
    • Program composition patterns
    • Location: lib/dspex/programs/

  3. Distributed Optimization

    • Leverage Foundation’s distributed capabilities
    • Parallel optimization across multiple nodes
    • Shared state management for optimization results

Success Criteria:

  • BootstrapFewShot optimizer working
  • Can automatically improve programs with more data
  • Distributed optimization functional
  • Performance improvements demonstrable

Phase 3: Foundation 2.0 Integration (Weeks 9-12)

Objective: Leverage Foundation 2.0’s distributed capabilities

Revolutionary Features:

  1. Massive Scale AI

    • 1000+ node distributed inference
    • Partisan networking for AI workloads
    • Dynamic topology adaptation

  2. BEAM-Native AI Patterns

    • Process-per-agent architectures
    • Fault-tolerant AI pipelines
    • Zero-copy message optimization for large models

  3. Self-Managing AI Infrastructure

    • Automatic load balancing across nodes
    • Predictive scaling based on workload
    • Self-healing distributed AI systems

Success Criteria:

  • 100+ node distributed inference working
  • Automatic failover and recovery
  • Performance superior to centralized approaches
  • Zero-risk migration from Phase 2

Architecture Decisions

Foundation Library Integration

Decision: Use Foundation 0.1.4 as core infrastructure dependency

  • Rationale: Provides production-ready circuit breakers, connection pooling, and observability
  • Risk: Early-stage library, but we’re involved in its development
  • Mitigation: Contribute to Foundation development as needed for DSPEx requirements

Hybrid Approach: Simple + Sophisticated

Decision: Support both simple struct-based programs and GenServer-based stateful programs

  • Simple: DSPEx.Predict remains struct-based for composability
  • Sophisticated: DSPEx.Teleprompter uses GenServer for optimization state
  • Rationale: Matches DSPy’s flexibility while leveraging OTP strengths

Distributed-First Design

Decision: Design for distributed execution from the start

  • Rationale: BEAM’s unique advantage over Python
  • Implementation: Use Foundation’s distributed primitives
  • Benefit: Scales to workloads impossible in Python DSPy

Risk Assessment

High Risk

  • Foundation Library Dependency: Early-stage library could have breaking changes
  • Complexity Scope: Ambitious distributed AI vision may be overreach
  • Performance Unknowns: Distributed overhead vs. parallelization benefits

Medium Risk

  • Python DSPy Compatibility: Some patterns may not translate cleanly to Elixir
  • LLM Provider Changes: APIs evolve rapidly, requiring adapter updates
  • Community Adoption: Elixir AI community is smaller than Python

Low Risk

  • Technical Implementation: Strong OTP patterns reduce implementation risk
  • Team Expertise: Demonstrated competency in implemented components
  • Test Coverage: Comprehensive testing reduces regression risk

Success Metrics

Technical Excellence

  • Performance: 10x faster evaluation than Python DSPy (distributed)
  • Reliability: 99.9% uptime with automatic failover
  • Scalability: 1000+ concurrent optimizations
  • Quality: Zero Dialyzer warnings, >95% test coverage

Community Impact

  • Adoption: 100+ stars on GitHub within 6 months
  • Integration: Used in production by 5+ companies
  • Ecosystem: 3+ derivative projects built on DSPEx
  • Documentation: Comprehensive guides and examples

Strategic Value

  • Innovation: First distributed AI programming framework
  • Differentiation: Capabilities impossible in Python DSPy
  • Foundation: Basis for next-generation AI infrastructure
  • Legacy: Reference implementation for distributed AI on BEAM

Immediate Next Actions

This Week

  1. Implement GenServer-based Client - Replace basic HTTP wrapper
  2. Define DSPEx.Program Behavior - Fix missing behavior module
  3. Update Documentation - Align with actual implementation status
  4. Plan Foundation Integration - Coordinate with Foundation 2.0 development

Next Week

  1. Complete Client Architecture - Circuit breakers, connection pooling
  2. Multi-provider Support - OpenAI, Anthropic, Gemini adapters
  3. Begin Evaluation Engine - Concurrent evaluation framework
  4. Test Infrastructure - Replace placeholder tests

Month 1 Goal

  • Phase 1A complete with all core components actually implemented
  • Working end-to-end pipeline from signature to evaluation
  • Foundation integration providing resilience and observability
  • Documentation accurately reflecting capabilities

Conclusion

DSPEx has the potential to be the definitive AI programming framework for distributed systems. The convergence of Foundation 2.0’s distributed infrastructure vision with DSPEx’s AI programming needs creates a unique opportunity.

The key is disciplined execution: complete Phase 1 foundations properly, then leverage those foundations for the revolutionary distributed AI capabilities that only BEAM can provide.

The vision is clear. The technology is proven. The opportunity is now.

This plan synthesizes analysis from CLAUDE_PHASE1_REVIEW.md, Foundation 2.0 vision documents, and early technical blueprints. It provides a practical roadmap for building the distributed AI framework that shows the world why BEAM is the superior platform for AI applications.