Miguel
/
AutoBackups
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
AutoBackups/PLAYWRIGHT_README.md

7.4 KiB
Raw Blame History

AutoBackups - Playwright E2E Testing Integration

Overview

This project now includes comprehensive end-to-end (E2E) testing using Playwright, integrated with the Model Context Protocol (MCP) for automated testing of the AutoBackups Flask application.

Features

  • 🎭 Cross-browser testing (Chromium, Firefox, WebKit)
  • 🔧 Automated setup with the Flask development server
  • 📊 Comprehensive reporting with HTML reports and screenshots
  • 🐛 Debug mode for test development
  • 🚀 CI/CD ready with GitHub Actions integration
  • 📱 Responsive testing across different viewport sizes
  • 🔌 API testing for backend endpoints

Test Suites

1. Dashboard Tests (dashboard.spec.js)

  • Main dashboard loading and rendering
  • Navigation elements verification
  • Responsive design testing across different viewports
  • Visual regression testing with screenshots

2. Configuration Tests (configuration.spec.js)

  • Configuration page accessibility
  • Form elements detection and interaction
  • Data submission workflow
  • Input validation testing

3. API Tests (api.spec.js)

  • Health check endpoints
  • Backup API endpoints testing
  • CORS headers validation
  • POST request handling

4. Backup Workflow Tests (backup-workflow.spec.js)

  • Backup status display
  • Manual backup trigger functionality
  • Project list visualization
  • Backup scheduling interface
  • Progress and logging display

Quick Start

Prerequisites

  1. Conda environment with AutoBackups dependencies:

    conda env create -f environment.yml
conda activate autobackups

  2. Node.js dependencies:

    npm install

  3. Playwright browsers:

    npx playwright install

Running Tests

Windows (PowerShell)

# Run all tests
.\run-tests.bat

# Run tests in headed mode (visible browser)
.\run-tests.bat headed

# Run specific test file
.\run-tests.bat specific tests/e2e/dashboard.spec.js

# Debug mode
.\run-tests.bat debug

# Show test report
.\run-tests.bat report

Linux/Mac (Bash)

# Make script executable
chmod +x run-tests.sh

# Run all tests
./run-tests.sh

# Run tests in headed mode
./run-tests.sh headed

# Run specific test file
./run-tests.sh specific tests/e2e/dashboard.spec.js

# Debug mode
./run-tests.sh debug

# Show test report
./run-tests.sh report

Direct NPX Commands

# Run all tests
npx playwright test

# Run tests with visible browser
npx playwright test --headed

# Run specific test file
npx playwright test tests/e2e/dashboard.spec.js

# Run in debug mode
npx playwright test --debug

# Generate and show HTML report
npx playwright show-report

Test Configuration

Playwright Config (playwright.config.js)

Key configurations:

  • Base URL: http://localhost:5000 (Flask dev server)
  • Timeout: 30 seconds per test
  • Retries: 2 retries on CI, 0 locally
  • Browsers: Chromium, Firefox, WebKit
  • Web Server: Automatically starts Flask app before tests

Test Data and Helpers

The tests/e2e/helpers/test-helpers.js file provides:

  • Common test utilities
  • Form filling helpers
  • Screenshot utilities
  • Console logging setup
  • Test data generators

Project Structure

├── tests/
│   └── e2e/
│       ├── dashboard.spec.js           # Dashboard functionality tests
│       ├── configuration.spec.js      # Configuration page tests
│       ├── api.spec.js                 # API endpoint tests
│       ├── backup-workflow.spec.js     # Backup workflow tests
│       ├── setup.js                    # Global test setup
│       └── helpers/
│           └── test-helpers.js         # Test utilities
├── playwright.config.js               # Playwright configuration
├── package.json                       # Node.js dependencies
├── run-tests.sh                       # Linux/Mac test runner
├── run-tests.bat                      # Windows test runner
├── playwright-report/                 # HTML test reports
└── test-results/                      # Test artifacts and screenshots

Environment Variables

You can customize test behavior with environment variables:

# Custom base URL
export PLAYWRIGHT_BASE_URL=http://localhost:8080

# CI mode
export CI=true

# Debug mode
export DEBUG=pw:api

Continuous Integration

GitHub Actions

The project includes a GitHub Actions workflow (.github/workflows/e2e-tests.yml) that:

  1. Sets up Python 3.12 with Conda
  2. Installs dependencies
  3. Runs Playwright tests
  4. Uploads test artifacts
  5. Generates test reports

Running in CI

Tests automatically run on:

  • Push to main or develop branches
  • Pull requests to main or develop branches

Test Reports

HTML Report

  • Interactive report with test results
  • Screenshots and videos of failures
  • Trace viewer for debugging
  • Access via: npx playwright show-report

JSON Report

  • Machine-readable test results
  • Located at: test-results/playwright-results.json
  • Useful for CI/CD integrations

Best Practices

Writing Tests

  1. Use Page Object Model for complex interactions
  2. Wait for elements properly with page.waitForSelector()
  3. Take screenshots for visual verification
  4. Use data-testid attributes for reliable element selection
  5. Test across browsers by running full suite

Debugging Tests

  1. Headed mode: See browser interactions in real-time
  2. Debug mode: Step through tests with Playwright Inspector
  3. Console logs: Browser console messages are captured
  4. Screenshots: Automatic screenshots on failures
  5. Video recording: Videos of failed test runs

Performance

  1. Parallel execution: Tests run in parallel by default
  2. Selective testing: Run specific test files during development
  3. CI optimization: Different configurations for CI vs local development

Troubleshooting

Common Issues

  1. Port conflicts: Ensure Flask app port (5000) is available
  2. Conda environment: Make sure autobackups environment is activated
  3. Browser installation: Run npx playwright install if browsers are missing
  4. Timeouts: Increase timeout in playwright.config.js if needed

Debug Commands

# Check Playwright installation
npx playwright --version

# Verify browser installation
npx playwright install --dry-run

# Test configuration
npx playwright test --list

# Debug specific test
npx playwright test tests/e2e/dashboard.spec.js --debug

Integration with MCP

This Playwright setup is designed to work seamlessly with Model Context Protocol (MCP):

  1. Automated test generation based on application features
  2. Dynamic test adaptation as the application evolves
  3. Intelligent test selection based on code changes
  4. Report analysis and recommendation generation

Next Steps

  1. Expand test coverage for specific AutoBackups features
  2. Add visual regression testing with baseline screenshots
  3. Integrate performance testing with Playwright's performance APIs
  4. Set up test data management for more complex scenarios
  5. Add accessibility testing with axe-playwright integration

Resources