Skip to content

Testing Documentation

This section provides comprehensive testing documentation for RAG Modulo, ensuring the development workflow and Makefile targets work reliably across all environments.

🎯 Testing Strategy

Our testing approach follows a multi-layer strategy to ensure comprehensive validation:

Priority Order:

  1. Fresh Environment Simulation (Most Important) - Validates real developer experience
  2. Automated Integration Tests - Ensures reliability in CI/CD
  3. Manual Validation - Catches edge cases
  4. Documentation Testing - Ensures usability

📚 Testing Documentation

Testing Strategy

Comprehensive overview of the testing approach for RAG Modulo, including Docker-in-Docker alternatives and test infrastructure.

What it covers: - Overall testing strategy and approaches - Docker-in-Docker alternatives for macOS - Test infrastructure and tools - Platform-specific considerations - Future testing roadmap

Makefile Testing Guide

Detailed guide for testing Makefile targets with focus on Docker-in-Docker alternatives.

What it covers: - Direct testing vs container testing approaches - macOS-specific Docker considerations - Smart test runner usage - Common issues and troubleshooting - Best practices for test development

Comprehensive Testing Guide

Complete testing strategy with detailed instructions for running all test types.

What it covers: - Fresh environment simulation testing - Automated integration tests - Manual validation procedures - Documentation accuracy testing - Troubleshooting common issues - CI/CD integration

Manual Validation Checklist

Detailed checklist for manual testing of all Makefile targets and edge cases.

What it covers: - Core development targets testing - Error handling validation - Performance benchmarks - Integration workflow testing - Documentation accuracy checks

🚀 Quick Start Testing

# Check Docker requirements first
make check-docker

# Use smart test runner (auto-detects best approach)
./tests/run_makefile_tests.sh

# OR: Run direct tests (macOS/local development)
python -m pytest tests/test_makefile_targets_direct.py -v

For macOS Development Workflow

# Quick validation (5 minutes)
make dev-init
make dev-build
make dev-up
make dev-validate
make dev-down
make clean-all

For Contributors

# Complete test suite (30 minutes)
./scripts/test-fresh-environment.sh  # if available
python -m pytest tests/test_makefile_targets_direct.py -v
./scripts/test-documentation.sh     # if available

For Release Validation

Follow the Manual Validation Checklist for comprehensive testing before releases.

🧪 Test Types

Fresh Environment Simulation

  • Purpose: Simulates completely fresh developer machine
  • Script: scripts/test-fresh-environment.sh
  • Duration: 10-15 minutes
  • Why Important: Validates real developer onboarding experience

Automated Integration Tests

  • Purpose: Automated testing for CI/CD pipelines
  • Script: tests/test_makefile_targets.py
  • Duration: 5-10 minutes
  • Why Important: Ensures reliability and regression testing

Manual Validation

  • Purpose: Comprehensive manual testing for edge cases
  • Guide: Manual Validation Checklist
  • Duration: 30-60 minutes
  • Why Important: Catches issues automated tests might miss

Documentation Testing

  • Purpose: Ensures all documentation is accurate
  • Script: scripts/test-documentation.sh
  • Duration: 2-5 minutes
  • Why Important: Prevents documentation drift

🎯 Success Criteria

All Tests Must Pass

  • ✅ Fresh environment simulation passes
  • ✅ Automated integration tests pass
  • ✅ Documentation tests pass
  • ✅ Manual validation checklist completed

Performance Benchmarks

  • Build time: < 5 minutes for fresh build
  • Startup time: < 2 minutes for service startup
  • Validation time: < 30 seconds for environment validation

User Experience

  • New developers: Can get started in < 10 minutes
  • Documentation: All examples work as documented
  • Error messages: Clear and actionable
  • Help system: Comprehensive and accurate

🔧 Test Scripts

Available Scripts

  • scripts/test-fresh-environment.sh - Fresh environment simulation
  • scripts/test-documentation.sh - Documentation validation
  • tests/test_makefile_targets.py - Automated integration tests

Running Tests

# Make scripts executable
chmod +x scripts/*.sh

# Run individual tests
./scripts/test-fresh-environment.sh
./scripts/test-documentation.sh
python -m pytest tests/test_makefile_targets.py -v

📊 Continuous Testing

CI/CD Integration

Tests are designed to run in CI/CD pipelines: - Fresh environment simulation for validation - Automated tests for regression testing - Documentation tests for accuracy

Pre-commit Hooks

Consider adding test validation to pre-commit hooks for immediate feedback.

Regular Validation

  • Weekly: Run automated tests
  • Before releases: Run all tests
  • After changes: Run relevant tests
  • New team members: Follow manual checklist

🆘 Getting Help

Test Failures

See the Comprehensive Testing Guide troubleshooting section for common issues and solutions.

Questions

Comprehensive testing ensures RAG Modulo's development workflow is robust, reliable, and user-friendly.