System Canvas

The Foundation: Little's Law

The entire simulation is anchored by Little's Law — the fundamental theorem of queuing theory. It guarantees that for any stable system, the average concurrency (L) equals throughput (λ) times latency (W).

Nested Private Services

In a Project Workspace you can drop one private service inside another as a Microservice node. The engine does notreplay a frozen benchmark snapshot for that node. Instead, before sampling begins, it simulates the nested service's entire internal architecture against the exact RPS it receives in the current run, then feeds the resulting latency, error rate, and saturation back into the parent simulation.

Each Microservice node also exposes a Request Absorption Rate. When a service references another service internally, this rate is the probability that an inbound request cascades to the downstream service — modelling caching, early returns, or validation that "absorbs" traffic at the edge.

Network Physics

Every hop between nodes pays a network cost before any component logic runs. The cost depends on whether the nodes share a VPC and region.

CPU performance degrades exponentially as effective RPS per core approaches a saturation threshold (~800 RPS/vCPU). Low RAM adds an additional multiplier from page-fault pressure.

Uses the same exponential CPU model as Server/VM. Containers typically run with fractional CPU shares and low RAM, so the RAM penalty is almost always active at default values (0.5 GB → 3× multiplier).

Every invocation incurs a cold-start penalty. Higher memory allocation gives proportionally more CPU, so 1 GB functions start roughly 2.8× faster than 128 MB functions.

Long-lived connections have very low per-message overhead. The main constraint is the total concurrent connection ceiling — once saturated, new handshakes are rejected. Heartbeat adds 0.5 ms overhead only when heartbeatInterval > 0 (i.e. actively configured). Setting it to 0 disables ping/pong entirely.

Utilisation uses Little's Law directly: concurrent connections ≈ throughput × avg session (30 s). At 1 000 new connections/s with a 30 s session lifetime → 30 000 concurrent, on a 50 000-conn server → 60% load.

Models a background worker pool — Node.js worker_threads, Python Celery, Go goroutine pool, Java ExecutorService. Each worker processes one job at a time with a deterministic service time. The engine uses the M/D/1 Pollaczek–Khinchine (P-K) formula: Poisson arrivals, deterministic (fixed) service time — the most accurate closed-form model for a fixed thread pool.

Models a cluster of worker nodes managed by a control plane. Throughput capacity scales linearly with node count, CPU per node, and overcommit ratio. It simulates scheduling overhead and optional autoscaling latency.

GraphQL introduces a parsing and resolution layer. Performance is sensitive to query depth and the number of underlying resolvers invoked per request.

A Microservice node represents another private service composed inside this one. It is a black box at this level — but not a static one. The engine recursively simulates the referenced service's full internal canvas at the effective RPS this node receives, so its latency, error rate, and saturation always reflect the real load. See Nested Private Services for the full model.

The gateway adds a fixed 2 ms routing overhead and enforces rate limits. Requests above the rate limit are immediately dropped and counted as errors.

The LB routes each request to exactly one downstream using a weighted random selection. The routing weight depends on the algorithm configured.

Deep packet inspection adds latency proportional to the number of active rules. Shallow mode skips payload analysis.

Example: 100 active rules → log₁₀(100) × 2 = 4 ms. Deep mode total: 8 + 4 = 12 ms per hop. 5 000 rules → log₁₀(5000) × 2 ≈ 7.4 ms penalty.

DNS resolution latency is modelled with a probabilistic cache-hit using a hyperbolic saturation curve. TTL=30 s gives a 50% hit rate; TTL=300 s gives ~91%; TTL=3 600 s gives ~99.2%. This reflects real resolver behaviour better than a linear cap — most of the caching benefit is captured in the first few minutes of TTL.

On every request the CDN rolls a cache-hit against the configured ratio. Hits are served from the nearest edge PoP (fast); misses add a 75 ms origin-pull penalty on top of the 5 ms edge base, giving ~80 ms total on miss.

Models 3rd-party dependencies (Twilio, Stripe, Auth0). These services have their own latency distributions, error rates, and throughput limits outside your direct control.

SQL performance is bound by IOPS (disk I/O throughput). Partitioning doubles effective IOPS. RAM above 8 GB unlocks a 1.5× buffer-pool bonus (hot pages served from memory). Sharding multiplies total capacity linearly across shard nodes. The utilisation metric uses the same effective IOPS formula as latency — partitioning and RAM bonuses are reflected in both so the load gauge is always consistent.

NoSQL stores are optimized for high write throughput and horizontal scale. Sharding multiplies base throughput capacity by distributing data across nodes.

Search latency decreases with more cluster nodes (parallel shard execution). Low RAM per node causes JVM heap pressure and frequent GC pauses (+10 ms penalty). Utilisation is modelled against a capacity of 1 000 QPS per node — a conservative real-world estimate for mixed search/index workloads on moderate hardware.

Provider-agnostic object store model. Latency is composed of six additive steps: storage class retrieval tier, auth overhead (standard SigV4 vs. presigned), optional KMS encryption round-trip, Transfer Acceleration edge routing, per-prefix request-rate throttling, and bandwidth saturation. All steps respect the configured workload type (read vs. write).

Redis is single-threaded and memory-bound. Its compute cost is negligible (0.5 ms) on top of the network round-trip. Enable sharding (Redis Cluster) to scale throughput linearly across shard nodes — each shard independently handles its key-space slice at the same 50 000 ops/s per GB rate.

Memcached is a distributed, multi-threaded, volatile cache. Unlike Redis it has no persistence or complex data structures — just fast key/value storage. Its key advantage is multi-threading: throughput scales with both memory allocation and thread count. Sharding adds further horizontal capacity by distributing keys across independent Memcached nodes (client-side consistent hashing).

Optimized for append-only streaming writes and window-based reads. It models heavy replication and background downsampling (merging old data points).

Relies on pointer-following for deep relationship traversals. Latency grows non-linearly with traversal depth (hops) and average node degree (connections per node).

Models simple O(1) lookups (DynamoDB, etcd, Consul). Highly parallel, deterministic performance intended for transient configuration or distributed locking.

Optimized for large object storage (images, MP4, backups). Unlike the general Object Storage node, it specifically models bandwidth-bound transfer latency with CDN bypass, multipart upload, replication penalties, and storage-class-aware first-byte cost.

OLAP store for multi-terabyte queries. Columnar storage and massive query parallelism reduce scan times for "Very Complex" analytics jobs.

Kafka latency is shaped by partition count (parallelism) and broker count (replication overhead). More partitions reduce per-partition queue depth via log₂ bonus; more brokers add a small ISR sync penalty. Latency is floored at 1 ms — even a massively over-provisioned cluster has serialization overhead.

FIFO queues guarantee exactly-once ordered delivery at the cost of extra coordination overhead. Standard queues trade ordering guarantees for lower latency.

Fan-out adds a per-subscriber delivery scheduling cost to publish latency. 0.5 ms per subscriber reflects the broker's cost of dispatching to each downstream delivery agent.

SMTP is inherently high-latency (TCP handshake + EHLO + DATA phase + server ACK: 100–250 ms baseline). The connection pool queuing follows M/M/c queueing theory — latency diverges sharply as utilisation approaches and exceeds 1.0. Above 1.5× saturation, connections are dropped as errors.

An AMQP broker implementing smart-routing via exchanges. Performance depends on persistence settings, acknowledgment modes, and the total consumer pool size across all queues.

Models a background task scheduler. Unlike the Service Worker (which focuses on thread pooling), the Scheduler focuses on long-running jobs and concurrent execution limits. It follows a variation of Little's Law for queueing wait times.

LLM latency is dominated by token generation speed. The engine estimates output tokens as 10% of the context window size and converts generation time to milliseconds.

ANN search latency grows with vector dimensionality. The index type determines the search algorithm: HNSW (graph-based) is fastest; Flat (brute-force) is slowest.

Embedding generation is a fixed-cost operation whose per-item cost is amortized by batching. Larger batches reduce effective per-request latency.

Agentic workflows call the LLM multiple times per user request. The orchestrator models only its own routing overhead — each downstream LLM call accrues its own latency separately.

Document parsing latency grows with chunk size (more data per unit) and shrinks with parallelism.