← Back to Prompts

V2 POOL PROMPTS PHASE4 TEST INFRASTRUCTURE REVISED2

Documentation for V2_POOL_PROMPTS_PHASE4_TEST_INFRASTRUCTURE_REVISED2 from the Dspex repository.

V2 Pool Implementation Prompts - Phase 4: Test Infrastructure Enhancement (REVISED2)

Overview

Phase 4 focuses on enhancing existing test infrastructure rather than reinventing it. The codebase already has excellent test foundations (DSPex.UnifiedTestFoundation, SupervisionTestHelpers) that follow unified testing guide patterns. Phase 4 adds pool-specific testing capabilities, performance benchmarking, and chaos engineering on top of this solid foundation.

Key Strategy: Extend existing infrastructure rather than replace it.

Session 4.1: Pool Test Extensions

Prompt 4.1.1 - Extend UnifiedTestFoundation for Pool Testing

We're implementing Phase 4 of the V2 Pool design, specifically enhancing test infrastructure.

First, examine existing test infrastructure:
1. Read test/support/unified_test_foundation.ex 
2. Read test/support/supervision_test_helpers.ex
3. Check docs/code-standards/UNIFIED_TESTING_GUIDE.md

Our goal is to ADD pool-specific testing mode to existing foundation, not replace it.

Enhance test/support/unified_test_foundation.ex:
1. Add :pool_testing isolation mode alongside existing modes
2. Implement setup_pool_testing/1 function 
3. Add pool-specific configuration and warmup
4. Integrate with existing supervision testing patterns
5. Add pool performance monitoring capabilities

Show me:
1. The new :pool_testing mode implementation
2. How it leverages existing supervision testing
3. Pool-specific enhancements added

Prompt 4.1.2 - Create Pool Test Helpers

Let's create pool-specific test helpers that work with existing infrastructure.

First, read existing helpers:
1. test/support/pool_v2_test_helpers.ex (current pool helpers)
2. test/support/supervision_test_helpers.ex (supervision patterns)
3. Check what pool operations need testing

Create test/support/enhanced_pool_test_helpers.ex:
1. Import and extend existing pool helpers
2. Add pool warming and pre-configuration functions
3. Add concurrent pool operation testing
4. Add session affinity testing helpers
5. Add pool scaling and load testing utilities

Key functions to implement:
- setup_isolated_pool/2 (leveraging existing isolation)
- warm_pool_workers/3 (parallel worker creation)
- test_concurrent_operations/3 (load testing)
- verify_session_affinity/2 (session tracking)
- monitor_pool_performance/2 (metrics collection)

Show me the enhanced helper implementations.

Prompt 4.1.3 - Pool Performance Test Framework

Let's create a performance testing framework for pools.

First, understand performance requirements:
1. Current pool performance characteristics
2. Performance optimization notes in CLAUDE.md
3. Existing benchmarking needs

Create test/support/pool_performance_framework.ex:
1. Benchmark configuration and setup
2. Performance test execution engine
3. Metrics collection and analysis
4. Performance regression detection
5. Load testing coordination

Key components:
- PerformanceBenchmark struct for test configuration
- benchmark_pool_operations/2 with warmup and measurement
- collect_performance_metrics/2 for comprehensive monitoring
- analyze_performance_results/1 for automated analysis
- performance_regression_detector/2 for CI integration

Show me the performance framework implementation.

Session 4.2: Deterministic Test Helpers

Prompt 4.2.1 - Enhance Existing Wait Functions

Let's enhance existing deterministic helpers rather than create new ones.

First, review existing patterns:
1. test/support/supervision_test_helpers.ex wait_for functions
2. Current event-driven coordination patterns
3. Pool-specific timing requirements

Enhance test/support/supervision_test_helpers.ex:
1. Add pool-specific wait conditions to existing wait_for/2
2. Add wait_for_pool_ready/3 function
3. Add wait_for_workers_initialized/3 function  
4. Add synchronize_pool_operations/2 function
5. Enhance existing failure injection for pools

Show me:
1. Enhanced wait_for functions with pool conditions
2. New pool-specific waiting helpers
3. How they integrate with existing patterns

Prompt 4.2.2 - Pool Chaos Testing Helpers

Let's add chaos testing specifically for pools.

First, understand pool failure modes:
1. Worker process crashes
2. Port communication failures
3. Session affinity disruption
4. Pool scaling issues

Create test/support/pool_chaos_helpers.ex:
1. Import existing supervision test helpers
2. Add pool-specific failure injection
3. Add cascading failure simulation
4. Add resource exhaustion simulation
5. Add recovery verification

Key functions:
- inject_worker_failure/3 (kill specific workers)
- simulate_port_corruption/2 (break communication)
- create_memory_pressure/2 (resource exhaustion)
- verify_pool_recovery/3 (check recovery completeness)
- chaos_test_orchestrator/3 (coordinate multiple failures)

Show me the chaos testing implementation.

Session 4.3: Enhanced Integration Tests

Prompt 4.3.1 - Update Existing Pool Tests

Let's enhance existing pool tests with new infrastructure.

First, identify existing tests to enhance:
1. List all files matching test/*pool* 
2. Check current test patterns and isolation
3. Identify tests that need performance monitoring

Update existing pool tests to use enhanced infrastructure:
1. Convert to use :pool_testing isolation mode
2. Add performance monitoring to key tests
3. Add chaos testing scenarios
4. Ensure proper resource monitoring
5. Add session affinity verification

Show me:
1. Updated test file structure
2. How existing tests are enhanced (not replaced)
3. New test scenarios added

Prompt 4.3.2 - Multi-Layer Pool Testing

Let's implement comprehensive multi-layer testing.

First, understand layer requirements:
1. Review existing layer configurations (layer_1, layer_2, layer_3)
2. Pool behavior differences across layers
3. Integration testing needs

Create test/dspex/python_bridge/pool_multi_layer_test.exs:
1. Use enhanced pool testing infrastructure
2. Test layer transitions and fallbacks
3. Test pool behavior consistency across layers
4. Test performance characteristics per layer
5. Test error handling across layers

Include:
- Layer-specific pool configurations
- Cross-layer session affinity testing
- Performance comparison across layers
- Error propagation testing
- Resource usage monitoring per layer

Show me the multi-layer test implementation.

Session 4.4: Performance Testing Suite

Prompt 4.4.1 - Pool Performance Benchmarks

Let's create comprehensive pool performance tests.

First, establish performance baselines:
1. Review performance optimization notes in CLAUDE.md
2. Current pool operation timing expectations
3. Performance regression detection needs

Create test/dspex/python_bridge/pool_performance_test.exs:
1. Use pool performance framework from 4.1.3
2. Benchmark single operation latency
3. Benchmark concurrent throughput
4. Test pool scaling performance
5. Test session affinity performance impact

Key benchmarks:
- Single operation latency (p50, p90, p95, p99)
- Concurrent operation throughput
- Worker initialization time
- Session affinity lookup performance
- Pool scaling responsiveness

Show me the performance test implementation.

Prompt 4.4.2 - Load Testing Framework

Let's implement realistic load testing.

First, understand load testing requirements:
1. Realistic usage patterns
2. Scaling behavior under load
3. Error handling under stress

Enhance pool_performance_test.exs:
1. Add gradual load ramping
2. Add sustained load testing
3. Add spike testing scenarios
4. Add mixed operation type testing
5. Add load testing with chaos injection

Load scenarios:
- Ramp to 1000 ops/sec over 2 minutes
- Sustain 500 ops/sec for 5 minutes
- Spike from 100 to 2000 ops/sec instantly
- Mixed predict/batch operations
- Load testing with worker failures

Show me load testing implementation and sample results.

Session 4.5: Chaos Engineering

Prompt 4.5.1 - Pool Chaos Test Suite

Let's implement comprehensive chaos testing for pools.

First, identify chaos scenarios:
1. Pool-specific failure modes
2. Cascading failure patterns
3. Recovery verification requirements

Create test/dspex/python_bridge/pool_chaos_test.exs:
1. Use pool chaos helpers from 4.2.2
2. Test individual worker failures
3. Test cascading pool failures  
4. Test session affinity disruption
5. Test pool recovery under load

Chaos scenarios:
- Kill 60% of workers during operation
- Corrupt session affinity data
- Exhaust pool resources gradually
- Simulate network partitions
- Combine multiple failure types

Show me chaos test implementation.

Prompt 4.5.2 - Chaos with Load Testing

Let's combine chaos engineering with load testing.

First, understand realistic failure patterns:
1. Failures during peak load
2. Recovery behavior under stress
3. Performance degradation patterns

Enhance pool_chaos_test.exs:
1. Add chaos during load testing
2. Test partial pool failures
3. Test recovery performance metrics
4. Test graceful degradation
5. Test error rate monitoring

Combined scenarios:
- Worker failures during 500 ops/sec load
- Session data corruption during session-heavy load
- Pool scaling failures during ramp-up
- Network issues during sustained operations
- Multiple simultaneous issues

Show me implementation and results analysis.

Session 4.6: CI/CD Integration

Prompt 4.6.1 - Test Configuration for CI

Let's configure enhanced tests for CI/CD.

First, review CI requirements:
1. Current config/test.exs configuration
2. Performance thresholds for CI
3. Test execution time limits

Update config/test.exs:
1. Add pool testing configuration
2. Set performance thresholds for CI
3. Configure test isolation settings
4. Set appropriate timeouts
5. Configure monitoring levels

Configuration sections:
- Pool testing specific config
- Performance test thresholds  
- Chaos test safety limits
- Layer-specific configurations
- CI-optimized settings

Show me updated configuration.

Prompt 4.6.2 - Test Execution Scripts

Let's create enhanced test execution scripts.

First, understand test organization:
1. Current test categorization
2. Performance vs functional test separation
3. CI pipeline requirements

Update scripts/ directory:
1. Enhance existing scripts for pool tests
2. Add performance test execution
3. Add chaos test execution (optional)
4. Add comprehensive test reporting
5. Add performance regression detection

Scripts to enhance/create:
- Enhanced run_integration_tests.sh with pool tests
- New run_performance_tests.sh
- New run_chaos_tests.sh (marked optional)
- Enhanced run_all_tests.sh
- New performance_regression_check.sh

Show me script implementations.

Session 4.7: Test Documentation

Prompt 4.7.1 - Update Testing Guide

Let's update existing testing documentation.

First, review existing docs:
1. docs/code-standards/UNIFIED_TESTING_GUIDE.md
2. Current testing patterns
3. New pool testing additions

Update docs/code-standards/UNIFIED_TESTING_GUIDE.md:
1. Add pool testing mode documentation
2. Add performance testing patterns
3. Add chaos testing guidelines
4. Add pool-specific best practices
5. Add troubleshooting for pool tests

New sections to add:
- Pool Testing Mode usage
- Performance testing patterns
- Chaos testing safety guidelines
- Pool test troubleshooting
- Integration with existing patterns

Show me documentation updates.

Prompt 4.7.2 - Pool Testing Cookbook

Let's create a pool testing cookbook.

Create docs/POOL_TESTING_COOKBOOK.md:
1. Quick start for pool testing
2. Common pool testing patterns
3. Performance testing recipes
4. Chaos testing examples
5. Troubleshooting guide

Include:
- Copy-paste test templates
- Performance benchmark examples
- Chaos scenario recipes
- Common gotchas and solutions
- Best practices checklist

Show me the cookbook content.

Session 4.8: Validation and Integration

Prompt 4.8.1 - Run Enhanced Test Suite

Let's validate the enhanced test infrastructure.

Execute comprehensive validation:
1. Run enhanced pool tests
2. Run performance benchmarks
3. Run chaos tests (if safe)
4. Generate performance baselines
5. Verify no test contamination

Validation checklist:
- All enhanced tests pass?
- Performance within expected ranges?
- No resource leaks detected?
- Chaos tests show proper recovery?
- Integration with existing tests works?

Show me validation results and any issues found.

Prompt 4.8.2 - Performance Baseline Documentation

Let's document performance baselines and test capabilities.

Create comprehensive Phase 4 report:
1. Enhanced infrastructure overview
2. Performance baseline measurements
3. Test coverage improvements
4. Chaos testing capabilities
5. CI/CD integration status

Update CLAUDE.md:
1. Add Phase 4 completion status
2. Document performance baselines
3. Add test execution guidance
4. Document new testing capabilities
5. Add troubleshooting notes

Show me the complete Phase 4 report and CLAUDE.md updates.

Session 4.9: Continuous Improvement

Prompt 4.9.1 - Test Health Monitoring

Let's implement test health monitoring.

Create scripts/test_health_monitor.sh:
1. Monitor test execution times
2. Detect flaky tests
3. Track performance trends
4. Monitor resource usage patterns
5. Generate health reports

Monitoring capabilities:
- Test execution time tracking
- Failure rate monitoring
- Performance regression detection
- Resource leak detection
- Test dependency analysis

Show me monitoring implementation.

Prompt 4.9.2 - Automated Test Maintenance

Let's create automated test maintenance tools.

Create scripts/test_maintenance.sh:
1. Automated test cleanup
2. Performance threshold updates
3. Test configuration validation
4. Unused test helper detection
5. Test documentation updates

Maintenance tasks:
- Remove obsolete test helpers
- Update performance thresholds based on trends
- Validate test configuration consistency
- Clean up test artifacts
- Update test documentation

Show me maintenance automation.

Key Differences from Original Phase 4

✅ Leverage Existing Infrastructure

  • Build on DSPex.UnifiedTestFoundation instead of creating new isolation
  • Enhance SupervisionTestHelpers instead of replacing
  • Extend existing pool helpers rather than rewrite

🎯 Focus on Pool-Specific Value

  • Pool performance testing (legitimately new)
  • Pool chaos engineering (new scenarios)
  • Session affinity testing (pool-specific)
  • Multi-layer pool testing (integration focus)

🔧 Enhancement Strategy

  • Add :pool_testing mode to existing foundation
  • Create specialized helpers that import existing ones
  • Focus on performance and chaos testing (the missing pieces)
  • Maintain backward compatibility

📈 Measurable Outcomes

  • Performance baselines established
  • Chaos testing validates resilience
  • Enhanced test coverage for pools
  • CI/CD integration with performance monitoring
  • Comprehensive test documentation

This revised approach respects the excellent existing test infrastructure while adding the genuinely needed pool-specific testing capabilities.