Junchuan Zhang | Jacky (He/His)

Junchuan Zhang | Jacky (He/His)

  • Graduates Software Engineer at Jaguar Land Rover, Gaydon, UK
  • MSc in Rehabilitation Engineering and Assistive Technologies from University College London
  • BSc in Computer Science and Electronic Engineering from University of Liverpool

5.Python Rest from OAS Simulator (PyROS)

Published:

PyROS (Python Rest from OAS Simulator) - Project Documentation

1. Project Overview & Core Features

  • Objective: Implement a comprehensive REST API server simulator that dynamically generates mock HTTP endpoints from OpenAPI Specification (OAS) files, enabling efficient API testing and development without requiring actual backend services.

  • Core Functionality: PyROS consumes OAS/YAML files to automatically create mock endpoints, supports custom response configuration, provides a graphical user interface for endpoint management, and offers comprehensive API control through REST endpoints.

2. Technical Architecture

  • Backend Framework: FastAPI with Uvicorn ASGI server
  • GUI Framework: Tkinter for desktop application interface
  • API Specification: OpenAPI 3.0.0 (OAS) YAML file parsing
  • Containerization: Docker with Ubuntu 22.04 base image
  • Additional Libraries: PyYAML for YAML parsing, Requests for HTTP communication, Threading for concurrent operations

3. Core Data Models & Components

  • APIInterpreter: Parses OAS YAML files and extracts endpoint definitions, methods, and URL parameters with enum support

  • EndpointFactory: Generates concrete endpoint instances from API specifications, handling URL variable substitution and enum expansion

  • Endpoint: Individual endpoint representation with URL, supported HTTP methods, and custom response management

  • RestResponse: Response configuration model with HTTP status codes and JSON body definitions

  • DynamicEndpointAPI: Core server class managing dynamic endpoint registration, request routing, and custom response handling

  • PyROSGUI: Desktop application providing visual interface for endpoint management, custom response configuration, and server control

4. Authentication & Security

  • No Authentication Required: PyROS is designed as a development/testing tool and operates without authentication
  • Local Development Focus: Server runs on localhost (127.0.0.1:6767) by default
  • Docker Isolation: Containerized deployment provides environment isolation
  • X11 Forwarding: GUI support through X11 forwarding for remote desktop access

5. Flexibility & Maintainability

  • Dynamic Endpoint Generation: Automatic endpoint creation from OAS specifications without code changes
  • Custom Response Configuration: Runtime modification of endpoint responses without server restart
  • Template Support: URL template matching with variable substitution and enum expansion
  • Modular Architecture: Clear separation of concerns with dedicated classes for each responsibility
  • Extensible Design: Easy addition of new features through the existing plugin-like architecture

6. Key Pages & Features

Server Management

  • Dynamic Endpoint Registration: Add endpoints with URL patterns and HTTP methods
  • Custom Response Configuration: Set specific HTTP codes and JSON responses for endpoints
  • Endpoint Listing: View all registered endpoints and their configurations
  • Server Control: Start, stop, and manage server operations

GUI Interface

  • Connection Management: Connect to PyROS server with configurable URL
  • YAML Configuration: Load and process OAS specification files
  • Endpoint Management: Visual interface for viewing and managing registered endpoints
  • Custom Response Editor: Interactive dialog for configuring custom responses
  • Real-time Updates: Live synchronization with server state

API Control Interface

  • REST API: Complete HTTP API for programmatic control
  • Postman Integration: Pre-configured Postman collection for API testing
  • Endpoint CRUD Operations: Create, read, update, delete endpoints
  • Response Management: Configure and manage custom responses
  • Server Simulation: Simulate Postman-like behavior for OAS files

7. Advanced Features

  • OAS File Processing: Automatic parsing and endpoint generation from OpenAPI specifications
  • URL Template Matching: Support for path parameters with enum validation
  • Enum Expansion: Automatic generation of concrete URLs from enum definitions
  • Method Support: Full HTTP method support (GET, POST, PUT, DELETE)
  • Custom Response Override: Override default responses with custom HTTP codes and bodies
  • Real-time GUI Updates: Live synchronization between GUI and server state
  • Docker Integration: Containerized deployment with X11 forwarding support
  • Threading Support: Concurrent operations for non-blocking server management

8. Page Structure & Navigation

Server Startup Flow

  • Docker container initialization with Ubuntu 22.04 base
  • Python environment setup with required dependencies
  • FastAPI server startup on port 6767
  • Optional YAML file processing for automatic endpoint generation

GUI Application Flow

  • Connection Dialog: Server URL configuration and connection testing
  • Main Interface: Three-panel layout with YAML config, endpoints, and custom responses
  • Endpoint Management: Listbox with endpoint details and right-click context menus
  • Custom Response Management: Listbox with response configurations and editing dialogs
  • YAML Processing: File browser and automatic endpoint generation from OAS files

API Endpoint Structure

  • Management Endpoints:
    • /add_dynamic_endpoint - Register new endpoints
    • /list_endpoints - Get all registered endpoints
    • /update_endpoint - Modify existing endpoints
    • /delete_endpoint - Remove endpoints
    • /add_custom_response - Configure custom responses
    • /list_custom_responses - Get all custom responses
    • /delete_custom_response - Remove custom responses
    • /simulate_postman - Process OAS files
    • /shutdown - Server shutdown
  • Dynamic Endpoints: All registered endpoints from OAS files or manual creation

GUI Panel Organization

  • YAML Configuration Panel: File path input, browse button, and load functionality
  • API Endpoints Panel: Scrollable listbox with endpoint details and management options
  • Custom Responses Panel: Scrollable listbox with response configurations and editing capabilities

Context Menu Operations

  • Endpoint Right-Click: Add custom response, edit endpoint, delete endpoint
  • Custom Response Right-Click: Edit response, delete response
  • Double-Click Actions: Direct editing of selected items

9. Frontend Development Highlights

  1. Tkinter GUI: Native desktop application with cross-platform compatibility
  2. Three-Panel Layout: Organized interface with YAML config, endpoints, and custom responses
  3. Real-time Synchronization: Live updates between GUI and server state
  4. Context Menus: Right-click operations for efficient endpoint and response management
  5. Dialog System: Modal dialogs for configuration and editing operations
  6. Connection Management: Robust server connection with error handling
  7. File Integration: Native file browser for OAS YAML file selection
  8. Scrollable Interfaces: Large dataset handling with scrollbars
  9. Error Handling: Comprehensive error messages and validation
  10. X11 Integration: Remote desktop support through X11 forwarding

10. Deployment & Environment

  • Docker Containerization: Ubuntu 22.04 base with Python 3 environment
  • Port Configuration: Default server port 6767 with Docker port mapping
  • Volume Mounting: OAS files mounted at /data for easy access
  • X11 Forwarding: GUI support through X11 socket mounting
  • Build Process: Multi-stage Docker build with dependency installation
  • Environment Variables: Configurable display and host settings

Deployment Commands

# Build the container
docker compose build pyros

# Start the server
docker compose up pyros

# Start the GUI (in separate terminal)
docker exec -it pyros bash & ./scripts/start_pyros_gui.sh

11. API Integration & Testing

Postman Collection

  • Pre-configured Requests: Complete set of API calls for PyROS management
  • Environment Variables: Configurable server URLs and endpoints
  • Request Templates: Standardized request formats for all operations
  • Response Examples: Sample responses for testing and validation

OAS File Processing

  • Automatic Endpoint Generation: Parse OAS files and create mock endpoints
  • Enum Support: Handle path parameters with enum constraints
  • Method Extraction: Extract supported HTTP methods from specifications
  • URL Template Processing: Convert OAS path templates to concrete endpoints

Testing Capabilities

  • Mock Server Testing: Test client applications against mock endpoints
  • Response Validation: Verify custom response configurations
  • Endpoint Verification: Confirm endpoint registration and behavior
  • Integration Testing: End-to-end testing with real API specifications

12. Future Enhancements

  • Authentication System: Optional authentication for production deployments
  • Response Templates: Pre-defined response templates for common scenarios
  • Request Logging: Comprehensive request/response logging and analytics
  • Performance Monitoring: Server performance metrics and monitoring
  • Plugin System: Extensible architecture for custom endpoint behaviors
  • Web-based GUI: Modern web interface as alternative to Tkinter
  • Multi-server Support: Management of multiple PyROS instances
  • API Versioning: Support for multiple API versions and specifications
  • Advanced Matching: Complex URL matching patterns and rules
  • Export/Import: Configuration backup and restore functionality
  • Real-time Collaboration: Multi-user support for team development
  • Integration APIs: Webhooks and external system integration
  • Advanced Validation: Request/response schema validation
  • Load Testing: Built-in load testing capabilities
  • Documentation Generation: Automatic API documentation from OAS files

13. Development Workflow

Local Development

  1. Setup: Install Python dependencies and configure environment
  2. Development: Modify PyROS code with hot-reload support
  3. Testing: Use Postman collection for API testing
  4. GUI Testing: Test Tkinter interface with various scenarios
  5. OAS Integration: Test with real OpenAPI specification files

Docker Development

  1. Build: Create Docker image with current codebase
  2. Run: Start containerized environment
  3. GUI Access: Connect to GUI through X11 forwarding
  4. File Mounting: Mount OAS files for testing
  5. Debugging: Access container logs and debugging tools

Production Deployment

  1. Image Building: Create production-ready Docker image
  2. Environment Configuration: Set production environment variables
  3. Service Deployment: Deploy to container orchestration platform
  4. Monitoring: Implement logging and monitoring solutions
  5. Scaling: Configure horizontal scaling for high availability

14. Troubleshooting & Support

Common Issues

  • X11 Connection: Ensure X11 forwarding is properly configured
  • Port Conflicts: Verify port 6767 is available and not blocked
  • File Permissions: Check Docker volume mounting permissions
  • OAS Parsing: Validate YAML syntax and OpenAPI specification compliance
  • GUI Connectivity: Verify server URL and network connectivity

Debugging Tools

  • Server Logs: Container logs for server-side debugging
  • GUI Console: Tkinter application console output
  • API Testing: Postman collection for endpoint verification
  • Network Tools: TCP dump and network monitoring utilities

Performance Optimization

  • Endpoint Caching: Optimize endpoint lookup and matching
  • Memory Management: Efficient handling of large OAS files
  • Concurrent Requests: Threading optimization for multiple clients
  • Response Time: Minimize latency for custom response processing