● Technical Design Document: Complete Dynamic Signature System for DSPex

Executive Summary

This document outlines the comprehensive design for implementing a full dynamic signature system that enables DSPex to support arbitrary input/output field definitions beyond the current hardcoded “question → answer” pattern. The implementation will bridge Elixir’s innovative signature format with Python DSPy’s signature system.

Current State Analysis

What We Have Working ✅

• SessionPoolV2: Robust concurrent execution with pool management
• Basic DSPy Integration: Hardcoded QuestionAnswer signatures work perfectly
• Real ML Inference: Sentiment analysis, translation, summarization via Q&A format
• Adapter Infrastructure: Complete bridge between Elixir and Python

Current Limitation ❌

• Hardcoded Signatures: All operations default to question → answer regardless of defined signature
• Signature Conversion Broken: convert_signature() functions exist but don’t properly translate to DSPy

Design Goals

1. Full Signature Flexibility: Support arbitrary field names (text→sentiment, english→french, etc.)
2. Type Safety: Validate signatures at creation and execution time
3. Backward Compatibility: Maintain existing Q&A functionality
4. Performance: Minimal overhead for signature conversion
5. Developer Experience: Intuitive signature definition API

Architecture Overview

┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ Elixir DSPex │ │ Signature │ │ Python DSPy │ │ Signature │───▶│ Converter │───▶│ Dynamic │ │ Definition │ │ Pipeline │ │ Signature │ └─────────────────┘ └─────────────────┘ └─────────────────┘

Component Breakdown

1. Enhanced Signature Definition System

1.1 Elixir Signature Format (Current)

signature = %{ name: “SentimentAnalysis”, inputs: [%{name: “text”, type: “string”}], outputs: [%{name: “sentiment”, type: “string”}] }

1.2 Enhanced Signature Format (Proposed)

signature = %{ name: “SentimentAnalysis”, description: “Analyze the sentiment of input text”, inputs: [ %{name: “text”, type: “string”, description: “Text to analyze”, required: true} ], outputs: [ %{name: “sentiment”, type: “string”, description: “Sentiment classification”, enum: [“positive”, “negative”, “neutral”]} ], examples: [ %{inputs: %{text: “I love this!”}, outputs: %{sentiment: “positive”}} ] }

2. Python Bridge Signature Converter

2.1 Current Issues in convert_signature()

• Converts to dictionary format but DSPy doesn’t use the field names
• No dynamic DSPy signature class creation
• Falls back to hardcoded patterns

2.2 Proposed Python Bridge Enhancement

File: /priv/python/dspy_bridge.py

class DynamicSignatureGenerator: “““Generates DSPy signatures dynamically from Elixir definitions”””

  def create_signature(self, signature_def):
      """
      Convert Elixir signature to DSPy signature class

      Args:
          signature_def: Dict from Elixir with inputs/outputs

      Returns:
          DSPy signature class instance
      """
      inputs_str = self._build_inputs_string(signature_def['inputs'])
      outputs_str = self._build_outputs_string(signature_def['outputs'])

      # Create the signature string for DSPy
      signature_string = f"{inputs_str} -> {outputs_str}"

      # Create dynamic class
      class_name = signature_def.get('name', 'DynamicSignature')
      signature_class = type(class_name, (dspy.Signature,), {
          '__signature__': signature_string,
          '__doc__': signature_def.get('description', ''),
      })

      return signature_class

  def _build_inputs_string(self, inputs):
      """Convert input field definitions to DSPy format"""
      fields = []
      for field in inputs:
          field_name = field['name']
          field_type = field.get('type', 'str')
          description = field.get('description', '')

          if description:
              fields.append(f"{field_name}: {field_type} = dspy.InputField(desc='{description}')")
          else:
              fields.append(f"{field_name}: {field_type}")

      return ", ".join(fields)

  def _build_outputs_string(self, outputs):
      """Convert output field definitions to DSPy format"""
      fields = []
      for field in outputs:
          field_name = field['name']
          field_type = field.get('type', 'str')
          description = field.get('description', '')

          if description:
              fields.append(f"{field_name}: {field_type} = dspy.OutputField(desc='{description}')")
          else:
              fields.append(f"{field_name}: {field_type}")

      return ", ".join(fields)

2.3 Enhanced Bridge Command Handler

def handle_create_program(self, args): “““Create a program with dynamic signature””” signature_def = args.get(‘signature’, {}) program_id = args.get(‘id’, f"program_{uuid.uuid4().hex[:8]}")

  try:
      # Generate dynamic signature
      signature_generator = DynamicSignatureGenerator()
      signature_class = signature_generator.create_signature(signature_def)

      # Create DSPy predictor with dynamic signature
      predictor = dspy.Predict(signature_class)

      # Store program
      self.programs[program_id] = {
          'predictor': predictor,
          'signature': signature_class,
          'signature_def': signature_def,
          'created_at': time.time()
      }

      return {
          'program_id': program_id,
          'signature': signature_def,
          'status': 'created'
      }

  except Exception as e:
      raise ValueError(f"Failed to create program: {str(e)}")

def handle_execute_program(self, args): “““Execute program with dynamic input/output handling””” program_id = args.get(‘program_id’) inputs = args.get(‘inputs’, {})

  if program_id not in self.programs:
      raise ValueError(f"Program not found: {program_id}")

  program = self.programs[program_id]
  predictor = program['predictor']
  signature_def = program['signature_def']

  try:
      # Validate inputs match signature
      self._validate_inputs(inputs, signature_def['inputs'])

      # Execute with dynamic inputs
      result = predictor(**inputs)

      # Extract outputs according to signature
      outputs = self._extract_outputs(result, signature_def['outputs'])

      return {
          'program_id': program_id,
          'inputs': inputs,
          'outputs': outputs,
          'execution_time': time.time()
      }

  except Exception as e:
      raise ValueError(f"Execution failed: {str(e)}")

def _validate_inputs(self, inputs, input_schema): “““Validate inputs match signature requirements””” required_fields = [field[’name’] for field in input_schema if field.get(‘required’, True)]

  for field in required_fields:
      if field not in inputs:
          raise ValueError(f"Missing required input field: {field}")

  # Additional type validation could go here

def _extract_outputs(self, result, output_schema): “““Extract outputs from DSPy result according to signature””” outputs = {}

  for field in output_schema:
      field_name = field['name']
      if hasattr(result, field_name):
          outputs[field_name] = getattr(result, field_name)
      else:
          # Fallback for compatibility
          outputs[field_name] = str(result)

  return outputs

3. Elixir Adapter Enhancement

3.1 Enhanced convert_signature() in PythonPort

defp convert_signature(signature) when is_map(signature) do # Enhanced signature conversion with full field mapping %{ “name” => Map.get(signature, :name, Map.get(signature, “name”, “DynamicSignature”)), “description” => Map.get(signature, :description, Map.get(signature, “description”, “”)), “inputs” => convert_fields(Map.get(signature, :inputs, Map.get(signature, “inputs”, []))), “outputs” => convert_fields(Map.get(signature, :outputs, Map.get(signature, “outputs”, []))) } end

defp convert_fields(fields) when is_list(fields) do Enum.map(fields, fn field -> %{ “name” => Map.get(field, :name, Map.get(field, “name”)), “type” => Map.get(field, :type, Map.get(field, “type”, “string”)), “description” => Map.get(field, :description, Map.get(field, “description”, “”)), “required” => Map.get(field, :required, Map.get(field, “required”, true)) } end) end

3.2 Input Validation Layer

defmodule DSPex.Signature.Validator do @moduledoc """ Validates inputs against signature definitions """

def validate_inputs(inputs, signature) do
  required_fields = get_required_fields(signature)

  with :ok <- check_required_fields(inputs, required_fields),
       :ok <- validate_field_types(inputs, signature) do
    :ok
  else
    {:error, reason} -> {:error, "Input validation failed: #{reason}"}
  end
end

defp get_required_fields(signature) do
  signature
  |> Map.get(:inputs, [])
  |> Enum.filter(fn field -> Map.get(field, :required, true) end)
  |> Enum.map(fn field -> Map.get(field, :name) end)
end

defp check_required_fields(inputs, required_fields) do
  missing_fields = required_fields -- Map.keys(inputs)

  if Enum.empty?(missing_fields) do
    :ok
  else
    {:error, "Missing required fields: #{Enum.join(missing_fields, ", ")}"}
  end
end

defp validate_field_types(inputs, signature) do
  # Type validation logic here
  :ok
end

end

Implementation Plan

Phase 1: Python Bridge Enhancement (Week 1)

1. Implement DynamicSignatureGenerator in dspy_bridge.py
2. Update create_program handler to use dynamic signatures
3. Update execute_program handler for dynamic I/O
4. Add validation and error handling

Phase 2: Elixir Adapter Update (Week 2)

1. Fix convert_signature() functions in PythonPort
2. Add input validation layer
3. Update error handling for signature mismatches
4. Add signature caching for performance

Phase 3: Enhanced API Layer (Week 3)

1. Update DSPex.create_program() with signature validation
2. Add signature builder helpers for common patterns
3. Update documentation and examples
4. Add migration guide from Q&A format

Phase 4: Testing & Examples (Week 4)

1. Comprehensive test suite for all signature types
2. Update existing examples to use new signatures
3. Create advanced examples showcasing signature flexibility
4. Performance benchmarking vs current Q&A system

Signature Examples

1. Sentiment Analysis

signature = %{ name: “SentimentAnalysis”, inputs: [%{name: “text”, type: “string”}], outputs: [%{name: “sentiment”, type: “string”}, %{name: “confidence”, type: “float”}] }

inputs = %{text: “I love this product!”}

Returns: %{sentiment: “positive”, confidence: 0.95}

2. Translation

signature = %{ name: “Translation”, inputs: [ %{name: “text”, type: “string”}, %{name: “target_language”, type: “string”} ], outputs: [%{name: “translated_text”, type: “string”}] }

inputs = %{text: “Hello world”, target_language: “French”}

Returns: %{translated_text: “Bonjour le monde”}

3. Multi-Field Analysis

signature = %{ name: “TextAnalysis”, inputs: [%{name: “text”, type: “string”}], outputs: [ %{name: “sentiment”, type: “string”}, %{name: “language”, type: “string”}, %{name: “summary”, type: “string”}, %{name: “keywords”, type: “list”} ] }

Migration Strategy

Backward Compatibility

• Maintain Q&A fallback: If no signature specified, use QuestionAnswer
• Gradual migration: Allow both old and new format during transition
• Deprecation warnings: Log when using legacy format

Migration Steps

1. Phase 1: New signature system available alongside Q&A
2. Phase 2: Examples updated to showcase new signatures
3. Phase 3: Q&A format marked as legacy (still supported)
4. Phase 4: Full migration complete

Risk Assessment

Technical Risks

• DSPy Compatibility: Ensure dynamic signatures work with all DSPy features
• Performance Impact: Signature conversion overhead
• Error Handling: Complex error propagation from Python to Elixir

Mitigation Strategies

• Extensive Testing: Comprehensive test suite across signature types
• Performance Monitoring: Benchmark dynamic vs static signatures
• Gradual Rollout: Phase-by-phase implementation with fallbacks

Success Metrics

1. Functional: All example signature types work correctly
2. Performance: <10% overhead vs current Q&A system
3. Developer Experience: Intuitive signature definition API
4. Compatibility: 100% backward compatibility with existing code
5. Adoption: Clear migration path from Q&A to dynamic signatures

Conclusion

This design provides a comprehensive path to full signature flexibility while maintaining the robust concurrent execution capabilities we’ve built. The implementation leverages existing infrastructure while adding the dynamic signature generation layer that bridges Elixir’s innovative signature format with DSPy’s powerful capabilities.

Next Step: Begin Phase 1 implementation with the Python bridge enhancement to unlock true signature flexibility in DSPex.