← Back to 20250617

211 ELIXACT

Documentation for 211_ELIXACT from the Ds ex repository.

211_ELIXACT.md: DSPEx Elixact Integration Requirements & Implementation Plan

Executive Summary

Based on comprehensive analysis of the current DSPEx codebase and the Pydantic usage patterns in DSPy (documented in 210_PYDANTIC_INTEG.md), this document outlines specific technical requirements and implementation strategies for integrating Elixact into DSPEx.

Key Finding: DSPEx already has a solid foundation for Elixact integration, with existing bridge modules, enhanced parsers, and comprehensive test coverage. The integration path is low-risk and builds on working code.

Current DSPEx Architecture Analysis

Existing Foundation (✅ Already Implemented)

1. Elixact Bridge Module (/lib/dspex/elixact_bridge.ex)

  • Status: 820 lines, fully implemented and tested
  • Functionality:
    • Signature to Elixact schema conversion
    • Type mapping (string, integer, float, boolean, list, map)
    • Basic validation integration
    • Error handling with structured feedback

2. Enhanced Signature Parser (/lib/dspex/signature/enhanced_parser.ex)

  • Status: 837 lines, production-ready
  • Functionality:
    • Type annotations support: question:string -> answer:string, confidence:float
    • Constraint parsing: count:integer(min=1,max=100)
    • Nested type definitions
    • Rich field metadata extraction

3. Configuration Schema System (/lib/dspex/config/schema.ex)

  • Status: 480 lines, comprehensive validation
  • Functionality:
    • Type-safe configuration validation
    • Nested configuration schemas
    • Default value handling
    • Error reporting with field paths

4. Test Infrastructure

  • Status: 13 integration tests passing, full unit test coverage
  • Coverage:
    • Bridge functionality validation
    • Parser integration testing
    • Configuration schema validation
    • Error handling scenarios

Current Signature System Architecture

DSPEx.Signature (behavior)
├── DSPEx.Signature.Parser (basic: "question -> answer")
├── DSPEx.Signature.EnhancedParser (typed: "question:string -> answer:string")
└── DSPEx.ElixactBridge (convert to Elixact schemas)

Current Validation Points

  1. Input Validation (/lib/dspex/predict.ex:45-67)

    • Basic field presence checking
    • Type coercion for simple types
    • Manual validation logic

  2. Output Validation (/lib/dspex/predict.ex:112-134)

    • Response parsing validation
    • Field extraction and verification
    • Basic type checking

  3. Adapter Validation (/lib/dspex/adapters/instructor_lite_gemini.ex:89-156)

    • Manual JSON schema construction
    • Response format validation
    • Error handling for malformed responses

  4. Configuration Validation (/lib/dspex/config/schema.ex)

    • Comprehensive config validation using existing schema system
    • Nested configuration support
    • Type-safe parameter handling

Integration Opportunities & Requirements

Priority 1: Enhanced JSON Schema Generation

Current State

File: /lib/dspex/adapters/instructor_lite_gemini.ex
Lines: 89-156 (67 lines of manual schema construction)

# Current manual approach
defp build_json_schema(signature) do
  output_fields = signature.output_fields()
  properties = output_fields
    |> Enum.map(fn field -> {to_string(field), %{"type" => "string"}} end)
    |> Enum.into(%{})
  
  %{
    "type" => "object",
    "properties" => properties,
    "required" => output_fields |> Enum.map(&to_string/1),
    "additionalProperties" => false
  }
end

Enhancement Required

Integration Point: Use Elixact for automatic schema generation with constraints

# Enhanced approach with Elixact
defp build_json_schema(signature) do
  case DSPEx.ElixactBridge.signature_to_schema(signature) do
    {:ok, elixact_schema} ->
      Elixact.JsonSchema.from_schema(elixact_schema)
    {:error, _} ->
      # Fallback to manual schema
      build_json_schema_legacy(signature)
  end
end

Benefits:

  • Support for complex types (nested objects, arrays, unions)
  • Automatic constraint validation (min/max, patterns, formats)
  • Rich field descriptions and examples
  • Comprehensive type safety

Priority 2: Validation Pipeline Integration

Current State

File: /lib/dspex/predict.ex
Lines: 45-67 (input validation), 112-134 (output validation)

# Current basic validation
defp validate_inputs(signature, inputs) do
  required_fields = signature.input_fields()
  provided_fields = Map.keys(inputs)
  
  missing_fields = required_fields -- provided_fields
  if missing_fields == [] do
    :ok
  else
    {:error, {:missing_fields, missing_fields}}
  end
end

Enhancement Required

Integration Point: Add comprehensive Elixact validation

# Enhanced validation with Elixact
defp validate_inputs(signature, inputs) do
  case DSPEx.ElixactBridge.signature_to_schema(signature) do
    {:ok, elixact_schema} ->
      input_fields = extract_input_fields(elixact_schema)
      input_data = Map.take(inputs, input_fields)
      
      case Elixact.validate(input_data, elixact_schema) do
        {:ok, validated} -> {:ok, validated}
        {:error, errors} -> {:error, {:validation_failed, format_errors(errors)}}
      end
    {:error, reason} ->
      # Fallback to basic validation
      validate_inputs_basic(signature, inputs)
  end
end

Benefits:

  • Rich type validation (strings, numbers, booleans, complex types)
  • Constraint enforcement (length, range, format)
  • Structured error reporting with field paths
  • Type coercion and normalization

Priority 3: Enhanced Type System Support

Current State

File: /lib/dspex/signature/enhanced_parser.ex
Lines: 200-350 (type annotation parsing)

# Current type support (basic)
defp parse_type_annotation("string"), do: {:ok, :string}
defp parse_type_annotation("integer"), do: {:ok, :integer}
defp parse_type_annotation("float"), do: {:ok, :float}
defp parse_type_annotation("boolean"), do: {:ok, :boolean}

Enhancement Required

Integration Point: Add complex type support via Elixact

# Enhanced type support
defp parse_type_annotation("string(min=1,max=100)") do
  {:ok, {:string, [min_length: 1, max_length: 100]}}
end

defp parse_type_annotation("array(string)") do
  {:ok, {:array, :string}}
end

defp parse_type_annotation("object{name:string,age:integer}") do
  {:ok, {:object, [name: :string, age: :integer]}}
end

Benefits:

  • Complex nested object support
  • Array and union type support
  • Rich constraint specifications
  • Compile-time type checking

Specific Technical Requirements

1. Elixact Feature Requirements

Based on the Pydantic patterns analysis, Elixact needs to support:

Core Requirements (Must Have)

  1. Runtime Schema Generation

    • Dynamic schema creation from parsed signatures
    • Configuration-based schema modification
    • Schema composition and merging

  2. Advanced Validation

    • Field-level constraint validation
    • Cross-field validation rules
    • Custom validator functions
    • Conditional validation logic

  3. JSON Schema Generation

    • Complete JSON Schema Draft 7 support
    • OpenAI/Anthropic/Gemini compatibility
    • Reference resolution for nested schemas
    • Schema manipulation utilities

  4. Error Handling

    • Structured error reporting with field paths
    • Error code categorization
    • Detailed validation failure information
    • Integration with DSPEx error handling

Advanced Requirements (Should Have)

  1. Type Coercion

    • String to number conversion
    • Flexible type casting
    • Format-based parsing (dates, URLs)
    • Safe type coercion with validation

  2. Configuration Support

    • Schema configuration options
    • Validation mode selection (strict/lax)
    • Custom type definitions
    • Extension points for custom logic

2. DSPEx Integration Points

A. Signature System Enhancement

Target: /lib/dspex/signature.ex

defmodule DSPEx.Signature do
  # Add Elixact-aware signature behavior
  @callback to_elixact_schema() :: {:ok, Elixact.Schema.t()} | {:error, term()}
  @callback validate_with_elixact(data :: map()) :: {:ok, map()} | {:error, term()}
  
  # Enhanced signature creation
  def create_with_elixact(signature_string, opts \\ []) do
    with {:ok, parsed} <- EnhancedParser.parse(signature_string),
         {:ok, schema} <- ElixactBridge.signature_to_schema(parsed) do
      create_signature_module(schema, opts)
    end
  end
end

B. Predict Module Enhancement

Target: /lib/dspex/predict.ex

defmodule DSPEx.Predict do
  # Enhanced validation in forward/3
  defp validate_inputs_with_elixact(program, inputs) do
    case program.signature.to_elixact_schema() do
      {:ok, schema} ->
        input_schema = extract_input_schema(schema)
        Elixact.validate(inputs, input_schema)
      {:error, _} ->
        # Fallback to current validation
        validate_inputs_basic(program.signature, inputs)
    end
  end
end

C. Adapter Enhancement

Target: /lib/dspex/adapters/instructor_lite_gemini.ex

defmodule DSPEx.Adapters.InstructorLiteGemini do
  # Enhanced schema generation
  defp build_json_schema_with_elixact(signature) do
    case signature.to_elixact_schema() do
      {:ok, schema} ->
        output_schema = extract_output_schema(schema)
        Elixact.JsonSchema.generate(output_schema)
      {:error, _} ->
        build_json_schema_basic(signature)
    end
  end
end

3. Configuration Integration

Enhanced Configuration Schema

Target: /lib/dspex/config/schema.ex

defmodule DSPEx.Config.Schema do
  # Use Elixact for configuration validation
  use Elixact.Schema
  
  schema "DSPEx Configuration" do
    field :signatures, {:array, SignatureConfig}
    field :validation, ValidationConfig
    field :adapters, {:map, AdapterConfig}
  end
  
  defmodule SignatureConfig do
    use Elixact.Schema
    
    schema "Signature Configuration" do
      field :type_checking, :boolean, default: true
      field :constraint_validation, :boolean, default: true
      field :error_detail_level, :atom, enum: [:basic, :detailed, :verbose]
    end
  end
end

Implementation Strategy

Phase 1: Core Integration (2-3 weeks)

  1. Enhance Elixact Bridge (Week 1)

    • Add missing Pydantic pattern support
    • Implement TypeAdapter equivalent
    • Add schema reference resolution

  2. Update Validation Pipeline (Week 1-2)

    • Integrate Elixact validation in Predict module
    • Enhance error handling and reporting
    • Add validation configuration options

  3. Adapter Enhancement (Week 2-3)

    • Replace manual JSON schema generation
    • Add constraint-aware schema building
    • Test with OpenAI/Anthropic/Gemini APIs

Phase 2: Advanced Features (2-3 weeks)

  1. Enhanced Type System (Week 3-4)

    • Add complex type parsing support
    • Implement nested object schemas
    • Add union and array type support

  2. Configuration Enhancement (Week 4-5)

    • Type-safe configuration schemas
    • Validation mode configuration
    • Custom type definitions

  3. Testing and Optimization (Week 5-6)

    • Comprehensive test coverage
    • Performance optimization
    • Documentation and examples

Phase 3: Advanced Patterns (2-3 weeks)

  1. Advanced Pydantic Patterns (Week 6-7)

    • Implement remaining patterns from 210_PYDANTIC_INTEG.md
    • Add wrapper model support
    • Implement custom serialization

  2. Migration Tools (Week 7-8)

    • Signature migration utilities
    • Automated conversion tools
    • Documentation and guides

Risk Assessment & Mitigation

Low Risk Areas ✅

  • Foundation Integration: Existing bridge module provides solid base
  • Test Coverage: Comprehensive test infrastructure already in place
  • Backward Compatibility: Current signature system continues to work
  • Performance: Minimal overhead with existing validation patterns

Medium Risk Areas ⚠️

  • Elixact Feature Gaps: Some Pydantic patterns may need Elixact enhancement
  • Complex Type Support: Advanced types require careful implementation
  • API Compatibility: JSON schema generation must work with LLM providers

High Risk Areas 🔴

  • Migration Complexity: Large signatures may be complex to migrate
  • Performance Impact: Validation overhead needs monitoring
  • External Dependencies: Elixact evolution and compatibility

Mitigation Strategies

  1. Incremental Integration: Build on existing working foundation
  2. Fallback Support: Maintain current validation as fallback
  3. Comprehensive Testing: Extensive test coverage for all integration points
  4. Performance Monitoring: Benchmark validation overhead
  5. Migration Tools: Automated tools for signature conversion

Success Metrics

Technical Metrics

  • ✅ Zero Breaking Changes: All existing signatures continue to work
  • ✅ Enhanced Validation: Rich type checking and constraint validation
  • ✅ JSON Schema Quality: Full constraint support in generated schemas
  • ✅ Error Reporting: Structured errors with field paths and details
  • ✅ Performance: <10% validation overhead

User Experience Metrics

  • ✅ Migration Ease: Simple signature migration process
  • ✅ Documentation Quality: Clear guides and examples
  • ✅ Error Messages: Actionable and informative error reporting
  • ✅ Type Safety: Compile-time and runtime type checking
  • ✅ IDE Support: Rich development experience

Conclusion

DSPEx is exceptionally well-positioned for Elixact integration, with a solid foundation already in place. The existing bridge module, enhanced parser, and comprehensive test coverage provide a low-risk path to adding substantial type safety and validation capabilities.

The integration strategy builds incrementally on working code, maintaining backward compatibility while adding powerful new features. The specific technical requirements identified here provide a clear roadmap for implementation, with well-defined integration points and success metrics.

By following this plan, DSPEx will achieve comprehensive Pydantic feature parity while leveraging Elixir’s strengths for superior performance, reliability, and developer experience.