Runtime Implementation

Deep dive into Gati's runtime architecture, queue fabric, and worker pool design.

Overview

Gati's runtime achieves 172K RPS through a queue-based architecture with zero-copy request handling and optimized worker pools.

Queue Fabric Architecture

Request Pipeline

HTTP Request → Parse Queue → Route Queue → Handler Queue → Response Queue → HTTP Response
     ↓            ↓             ↓             ↓              ↓
  <6μs         <0.5ms        <0.5ms        Variable       <1ms

Queue Types

1. Parse Queue - Request parsing and validation

• Zero-copy buffer handling
• Streaming JSON parser
• Header extraction

2. Route Queue - Route matching and parameter extraction

• Trie-based routing (2.6M matches/sec)
• Path parameter extraction
• Method validation

3. Handler Queue - Business logic execution

• Worker pool management
• Context injection
• Error handling

4. Response Queue - Response serialization

• JSON serialization
• Header setting
• Compression

Worker Pool

Pool Configuration

const pool = createWorkerPool({
  minWorkers: 4,
  maxWorkers: 16,
  idleTimeout: 30000,
  taskTimeout: 5000
});

Worker Lifecycle

Idle → Assigned → Executing → Completed → Idle
  ↓                                         ↑
Timeout → Terminated → Spawn New ──────────┘

Load Balancing

• Round-robin task assignment
• Worker health monitoring
• Automatic scaling based on queue depth

Performance Optimizations

1. Zero-Copy Buffers

// Avoid buffer copies
const body = req.rawBody; // Direct buffer reference

2. Object Pooling

// Reuse context objects
const lctx = contextPool.acquire();
// ... use context
contextPool.release(lctx);

3. Fast Path Routing

// Static routes bypass trie
if (staticRoutes.has(path)) {
  return staticRoutes.get(path);
}

Benchmarks

• Throughput: 172K RPS
• Latency: p95 <10ms, p99 <25ms
• Pipeline: <6μs overhead
• Route Matching: 2.6M ops/sec

Related

• Performance Guide
• Benchmarking Guide
• Architecture Overview