← Back to Analysis

03 workflow optimization

Documentation for 03_workflow_optimization from the Pipeline ex repository.

Workflow Optimization Strategies

Core Problem: Making AI Automation Reliable

Your key insight is crucial: “It’s not about pipelines, it’s about evals”. The current system generates YAML and prays it works. We need systematic approaches to make AI assistance reliable and predictable.

Immediate Optimization Strategies

1. Prompt Engineering for Reliability

Current Problem:

  • Prompts are ad-hoc and inconsistent
  • No systematic approach to prompt construction
  • Context setup is inconsistent

Solution: Structured Prompt Templates

# Template: elixir_analysis_prompt.yaml
prompt_template:
  name: elixir_code_analysis
  context_setup:
    - "You are an expert Elixir/OTP developer"
    - "Focus on functional programming principles"
    - "Consider supervision tree patterns"
    - "Avoid imperative patterns and sleep calls"
  
  task_definition:
    - "Analyze the provided Elixir code"
    - "Identify potential issues and improvements"
    - "Suggest OTP-compliant alternatives"
  
  output_format:
    - "Provide analysis in structured format"
    - "Include severity levels for issues"
    - "Suggest specific code improvements"
  
  validation_criteria:
    - "Check for OTP compliance"
    - "Verify functional programming principles"
    - "Ensure no anti-patterns"

2. Multi-Step Validation Pipeline

The TLC Problem:

You need “TLC agents” - automated reviewers that catch Claude’s mistakes.

Solution: Validation Chain

# validation_pipeline.yaml
workflow:
  name: reliable_code_analysis
  steps:
    - name: initial_analysis
      type: claude_smart
      prompt: "Analyze this Elixir code for issues"
      
    - name: validate_analysis
      type: claude_robust
      prompt: |
        Review the previous analysis for accuracy:
        - Are identified issues actually problems?
        - Are suggestions OTP-compliant?
        - Are there missed issues?
        
    - name: cross_validate
      type: gemini
      prompt: "Independently analyze the same code and compare findings"
      
    - name: synthesize_final
      type: claude_smart
      prompt: "Combine all analyses into final, validated report"

3. Context-Fresh Prompt Strategy

Your Insight:

“Clean prompts that are small and testable with all new required reading for context”

Optimization Pattern:

# context_fresh_template.yaml
workflow:
  name: context_fresh_analysis
  steps:
    - name: setup_context
      type: file_ops
      operation: read_context_files
      files:
        - "relevant_modules.ex"
        - "test_examples.exs"
        - "otp_patterns.md"
        
    - name: focused_analysis
      type: claude_smart
      prompt: |
        Context: {{context_files}}
        
        Task: Analyze only this specific function
        Code: {{target_code}}
        
        Constraints:
        - Focus only on this function
        - Use provided context only
        - No external assumptions
        
    - name: validate_against_context
      type: claude_robust
      prompt: "Verify analysis uses only provided context"

Sequential Pipeline Optimization

Your Requirement:

“Sequential pipelines that run nice and slow and steady, each pipeline finishes to 100% completion”

Optimization Strategy: Pipeline Stages

Stage 1: Comprehensive Analysis

analysis_stage:
  steps:
    - name: understand_requirement
      type: claude_smart
      prompt: "Thoroughly understand what needs to be done"
      validation: "Requirement clarity check"
      
    - name: identify_dependencies
      type: claude_extract
      prompt: "Identify all dependencies and constraints"
      validation: "Dependency completeness check"
      
    - name: plan_approach
      type: claude_smart
      prompt: "Create detailed implementation plan"
      validation: "Plan feasibility check"

Stage 2: Implementation with Validation

implementation_stage:
  steps:
    - name: implement_solution
      type: claude_robust
      prompt: "Implement following the validated plan"
      validation: "Code syntax and logic check"
      
    - name: test_implementation
      type: claude_smart
      prompt: "Generate comprehensive tests"
      validation: "Test coverage verification"
      
    - name: verify_requirements
      type: claude_robust
      prompt: "Verify all requirements are met"
      validation: "Requirements compliance check"

Stage 3: Quality Assurance

qa_stage:
  steps:
    - name: code_review
      type: claude_smart
      prompt: "Perform thorough code review"
      validation: "Review quality check"
      
    - name: performance_analysis
      type: claude_extract
      prompt: "Analyze performance implications"
      validation: "Performance metrics validation"
      
    - name: final_validation
      type: claude_robust
      prompt: "Final validation of complete solution"
      validation: "Completion criteria check"

Critical Thinking Integration

Problem: Lack of Critical Thinking

Current system doesn’t apply critical thinking to improve the process.

Solution: Reflective Pipelines

# reflective_pipeline.yaml
workflow:
  name: critical_thinking_pipeline
  steps:
    - name: initial_solution
      type: claude_smart
      prompt: "Provide initial solution"
      
    - name: critique_solution
      type: claude_robust
      prompt: |
        Critically analyze the previous solution:
        - What could go wrong?
        - What edge cases are missed?
        - Are there better approaches?
        
    - name: improve_solution
      type: claude_smart
      prompt: "Improve solution based on critique"
      
    - name: validate_improvement
      type: claude_robust
      prompt: "Verify improvements address critiques"

Addressing “Claude Doing Dumb Shit”

Common Claude Mistakes in Elixir:

  1. Using sleep calls instead of proper OTP patterns
  2. Imperative patterns instead of functional approaches
  3. Poor error handling strategies
  4. Ignoring supervision tree patterns

Optimization: Error-Aware Prompts

# elixir_optimized_prompt.yaml
prompt_template:
  name: elixir_safe_coding
  anti_patterns_to_avoid:
    - "Never use Process.sleep for timing"
    - "Avoid imperative loops, use Enum functions"
    - "Don't ignore supervision tree patterns"
    - "Always handle errors with proper patterns"
  
  required_patterns:
    - "Use GenServer for stateful processes"
    - "Implement proper supervision strategies"
    - "Use pattern matching for control flow"
    - "Follow OTP principles consistently"
  
  validation_checks:
    - "Check for OTP compliance"
    - "Verify error handling patterns"
    - "Ensure functional programming approach"
    - "Validate supervision tree design"

Practical Implementation Steps

Phase 1: Immediate Improvements (This Week)

  1. Create Prompt Templates: Build 5-10 proven prompt templates
  2. Add Validation Steps: Every pipeline needs validation
  3. Test Small Workflows: Start with documentation and analysis

Phase 2: Quality Assurance (Next 2 Weeks)

  1. Multi-Step Validation: Implement validation chains
  2. Error Recovery: Add retry and fallback mechanisms
  3. Context Management: Implement context-fresh patterns

Phase 3: Workflow Integration (Next Month)

  1. Sequential Pipelines: Build reliable multi-stage workflows
  2. Critical Thinking: Add reflective analysis steps
  3. Performance Monitoring: Track success rates and improve

Key Success Metrics

Quality Metrics:

  • Validation Success Rate: % of validations that pass
  • Error Recovery Rate: % of errors successfully recovered
  • Context Compliance: % of responses using only provided context

Productivity Metrics:

  • Manual Review Time: Reduction in human review time
  • Task Completion Rate: % of tasks completed without human intervention
  • Prompt Reusability: Number of times prompts are successfully reused

Bottom Line Optimization Strategy

Focus on reliability over features:

  1. Small, Testable Prompts: Build library of proven prompt patterns
  2. Multi-Step Validation: Never trust single AI responses
  3. Context Control: Strict context management for predictable results
  4. Error Awareness: Design prompts to avoid known failure modes
  5. Human Checkpoints: Strategic human validation points

This approach addresses your core insight: the problem isn’t automation, it’s reliability. By building systematic validation and error recovery into every pipeline, you can achieve the “TLC” you need while maintaining the benefits of AI assistance.