Felix

Early Development

Felix is in early active development and has not been run in production by anyone. The design and implementation are still moving.

Where these pages describe something that is not built, they say so. The status table is the one place that is kept current per capability, and it is the page to trust when another disagrees with it.

Felix is a low-latency distributed data backend that serves streams, caches and work queues from one replicated log, over a single QUIC-based transport layer.

What makes it different

New here? Why Felix? is the plain-language answer — why one system instead of a log, a cache and a queue, and who should not use it. The cards below are the same argument for someone who already knows the territory.

Low tail latency

Designed for predictable p99/p999 under load, not just aggregate throughput. QUIC gives multiplexed streams without head-of-line blocking, and explicit backpressure stops one slow path cascading into the rest.

Unified primitives

One core log abstraction behind several semantics — streams (per-subscription fanout cursors), queues (shared consumer-group cursors with acks), and cache (key → value with TTL). One system instead of three.

QUIC from the ground up

Encrypted by default with TLS 1.3, multiplexed streams per connection, built-in flow control and congestion awareness, and resistance to head-of-line blocking.

Tunable, explicitly

Connection pooling, stream multiplexing, time- and count-bounded batching, flow-control window sizing, and optional per-stage telemetry — all under your control rather than hidden behind defaults.

See it behave

The demos are not simulations — each one runs a real broker and real clients, and each asserts the property it demonstrates as a test.

Slow-consumer isolationOne consumer stalls. Under drop_new the healthy consumers never notice; under block it takes down the publisher. The same workload, twice.

Local state divergenceThe counterpart: what an at-most-once configuration costs a consumer that holds local state, and what lossless delivery costs in throughput.

How It Works

Felix uses a framed protocol (felix-wire) over QUIC streams to unify event streaming and request/response caching, with explicit control over multiplexing, batching, and flow control.

Streams, caches and queues are three readings of the same bytes, not three subsystems. They share one durability path, one recovery path, one placement rule and one replication path.

Pub/Sub data flow

1. The client opens a bidirectional control stream to publish/subscribe and receive acknowledgements.
2. The broker validates scope, enqueues publish jobs, and fans out to subscribers.
3. Each subscription gets a dedicated unidirectional event stream for delivery.
4. Events travel as binary EventBatch frames with configurable batching.

Cache data flow

1. The client maintains a cache connection pool with long-lived stream workers.
2. Cache requests carry a request_id and are multiplexed over those streams.
3. The broker processes request frames in a read loop and replies on the same stream.
4. This avoids per-request stream setup cost and improves tail latency under concurrency.

Queue data flow

1. A consumer polls a group rather than being pushed to, because only it knows when it has capacity.
2. The broker claims records against a visibility timeout and hands them over with their attempt counts.
3. An acknowledgement settles one record; the group’s durable cursor advances over a contiguous run of them.
4. Anything unanswered is redelivered, and dead-lettered once it has been attempted too many times.

Composed data flow

Because every semantic is a reading of one log, they compose — each flow below is the primitive a whole class of application is usually hand-built from:

1. Watch a key or prefix (keyed watch): config push and cache invalidation — every instance learns of a change the moment it lands, with an offset to resume from instead of a poll loop.
2. Current state first, then changes (retained delivery): presence and state sync — join a room, hold the roster immediately, stay current; the confirmation says the exact moment your state is complete.
3. Deltas folded into a durable sum (counters): rate limits, usage metering, live tallies — increment and learn where you stand in one round trip, with a sum that survives restart, compaction, and failover.

Key Features

Feature	What it gives you
QUIC transport	Modern, multiplexed, encrypted transport layer
High fanout	Efficient delivery to many subscribers, with isolation between them
Connection pooling	Reusable connections and streams, so hot paths skip setup cost
Batch processing	Time- and count-bounded batching for throughput
TTL support	Time-to-live on cache entries with lazy expiration
Work queues	Consumer groups with acknowledgements, redelivery and dead letters
Backpressure	Explicit flow control at six checkpoints to prevent cascade failures
Metrics	Prometheus-compatible metrics endpoint
Observability	Structured logging with optional per-stage telemetry

Use Cases

Felix is a good fit today for:

• Real-time streaming with high fanout and tunable latency/throughput trade-offs
• Event pipelines with batch publishing and batch delivery
• Low-latency caching over QUIC with predictable tail latency under load
• Work distribution across consumers, where each job goes to one of them and is retried if nobody says it was handled
• Microservice communication with unified stream, cache and queue semantics

See What Felix Is For for the honest boundaries — including what Felix is deliberately not.

Current Status

Working today

• Multi-broker clusters with rendezvous-hashed shard placement
• Durable log-structured storage, with retention and torn-tail repair
• Replication with leader leases, quorum acknowledgement and failover
• A log-backed cache: routed to one owner, replicated, put/get/delete
• Consumer groups: poll, acknowledge, redeliver, dead-letter, redrive
• A control plane over REST, holding metadata in Raft — no external database — or in Postgres, and capability negotiation on the wire
• Tenant-scoped tokens, RBAC, and OIDC token exchange
• Mutually authenticated broker-to-broker QUIC, when certificates are configured

Not built yet

• Rebalancing: a shard whose leader is alive is never moved, however uneven that leaves things
• End-to-end payload encryption, encryption at rest, and audit logging
• Tiered storage and cold-tier reads
• Cross-region bridges with explicit data movement
• Clients in Go or C# (Rust, Python and TypeScript exist today)

Quick Start

1. Clone the repository.

   git clone https://github.com/gabloe/felix.git
   cd felix

2. Build the workspace.

   cargo build --workspace --release

3. Run a demo — each one is self-contained and starts its own broker.

   task demo:slow-consumer
   task demo:state-divergence

4. Or start a real cluster — control plane, credentials and three brokers — and publish through one broker to a shard another one owns.

   cargo run --release -p felix-cluster -- up --nodes 3
   cargo run --release -p felix-cluster -- subscribe orders   # second window
   cargo run --release -p felix-cluster -- publish orders hi  # third

See the Demos Overview or the Quickstart Guide for the full set.

Community & Support

Source on GitHubThe workspace, the demos, and the CI that keeps them honest.

IssuesReport a bug or request a feature.

ContributingHow to build, test, and land a change.

How Felix WorksThe end-to-end tour, from a publish call to a subscriber's socket.