Configuration Reference
Complete reference for all Felix broker configuration options.
Configuration Methods
Section titled “Configuration Methods”Felix supports three configuration methods, applied in order (later sources override earlier):
- Built-in defaults: Sensible defaults for development
- Environment variables:
FELIX_*variables for quick overrides - YAML config file: Structured configuration for production
Precedence Example
Section titled “Precedence Example”# Default: quic_bind = 0.0.0.0:5000# Environment: FELIX_QUIC_BIND=127.0.0.1:5001# YAML: quic_bind: "0.0.0.0:6000"# Result: 0.0.0.0:6000 (YAML wins)Configuration Structure
Section titled “Configuration Structure”YAML Config File
Section titled “YAML Config File”Location priority:
$FELIX_BROKER_CONFIG(explicit path)/usr/local/felix/config.yml(default, optional)
Example:
quic_bind: "0.0.0.0:5000"metrics_bind: "0.0.0.0:8080"controlplane_url: "http://controlplane:8443"controlplane_sync_interval_ms: 2000ack_on_commit: falsemax_frame_bytes: 16777216publish_queue_wait_timeout_ms: 2000ack_wait_timeout_ms: 2000disable_timings: falsecontrol_stream_drain_timeout_ms: 50cache_conn_recv_window: 268435456cache_stream_recv_window: 67108864cache_send_window: 268435456event_batch_max_events: 64event_batch_max_bytes: 65536event_batch_max_delay_us: 250fanout_batch_size: 64pub_workers_per_conn: 4pub_queue_depth: 64pub_inflight_bytes: 67108864pub_conn_inflight_bytes: 16777216subscriber_queue_capacity: 512max_subscriptions_per_conn: 4096subscriber_writer_lanes: 4subscriber_lane_queue_depth: 64max_subscriber_writer_lanes: 8subscriber_lane_shard: autoNetwork Configuration
Section titled “Network Configuration”quic_bind
Section titled “quic_bind”Description: QUIC listener bind address and port.
Type: SocketAddr (IP:Port)
Default: 0.0.0.0:5000
Environment: FELIX_QUIC_BIND
Example:
quic_bind: "0.0.0.0:5000"Notes:
- UDP port for QUIC transport
- Use
0.0.0.0to listen on all interfaces - Use
127.0.0.1for localhost only
metrics_bind
Section titled “metrics_bind”Description: HTTP metrics and health endpoint bind address.
Type: SocketAddr (IP:Port)
Default: 0.0.0.0:8080
Environment: FELIX_BROKER_METRICS_BIND
Example:
metrics_bind: "0.0.0.0:8080"Endpoints:
/healthz: Health check/metrics: Prometheus metrics (if enabled)
Control Plane Configuration
Section titled “Control Plane Configuration”controlplane_url
Section titled “controlplane_url”Description: Optional control plane base URL for metadata sync.
Type: String (URL)
Default: None
Environment: FELIX_CONTROLPLANE_URL
Example:
controlplane_url: "http://felix-controlplane:8443"Notes:
- Optional for single-node deployments
- Required for multi-node clusters
- Should include scheme (
http://orhttps://)
controlplane_sync_interval_ms
Section titled “controlplane_sync_interval_ms”Description: Interval for polling control plane changes.
Type: u64 (milliseconds)
Default: 2000
Environment: FELIX_CONTROLPLANE_SYNC_INTERVAL_MS
Example:
controlplane_sync_interval_ms: 2000Recommendations:
- Fast changes:
500-1000ms - Normal operation:
2000-5000ms - Stable clusters:
5000-10000ms
Publishing Configuration
Section titled “Publishing Configuration”ack_on_commit
Section titled “ack_on_commit”Description: Send acknowledgements after message commit.
Type: bool
Default: false
Environment: FELIX_ACK_ON_COMMIT (1, true, yes = enabled)
Example:
ack_on_commit: trueTrade-offs:
false: Lower latency, fire-and-forget semanticstrue: Higher latency, explicit acknowledgement
max_frame_bytes
Section titled “max_frame_bytes”Description: Maximum frame size accepted on QUIC streams.
Type: usize (bytes)
Default: 16777216 (16 MiB)
Environment: FELIX_MAX_FRAME_BYTES
Example:
max_frame_bytes: 16777216Notes:
- Limits individual message size
- Affects memory usage per stream
- Must match client expectations
publish_queue_wait_timeout_ms
Section titled “publish_queue_wait_timeout_ms”Description: Maximum time to wait when backpressuring publish enqueue.
Type: u64 (milliseconds)
Default: 2000
Environment: FELIX_PUBLISH_QUEUE_WAIT_MS
Example:
publish_queue_wait_timeout_ms: 2000Behavior:
- Publish blocks if queue is full
- Returns error after timeout
- Prevents unbounded memory growth
ack_wait_timeout_ms
Section titled “ack_wait_timeout_ms”Description: Maximum time to wait for ack-on-commit completion.
Type: u64 (milliseconds)
Default: 2000
Environment: FELIX_ACK_WAIT_TIMEOUT_MS
Example:
ack_wait_timeout_ms: 2000Notes:
- Only applies when
ack_on_commit: true - Publisher receives error if exceeded
Event Delivery Configuration
Section titled “Event Delivery Configuration”event_batch_max_events
Section titled “event_batch_max_events”Description: Maximum events per batched subscription frame.
Type: usize (count)
Default: 64
Environment: FELIX_EVENT_BATCH_MAX_EVENTS
Example:
event_batch_max_events: 64Tuning:
- Low latency:
1-16 - Balanced:
32-64 - High throughput:
128-256
event_batch_max_bytes
Section titled “event_batch_max_bytes”Description: Maximum bytes per batched subscription frame.
Type: usize (bytes)
Default: 65536 (64 KiB)
Environment: FELIX_EVENT_BATCH_MAX_BYTES
Example:
event_batch_max_bytes: 65536Notes:
- Whichever limit hits first triggers batch send
- Consider payload size when tuning
event_batch_max_delay_us
Section titled “event_batch_max_delay_us”Description: Maximum delay before flushing a subscription batch.
Type: u64 (microseconds)
Default: 250
Environment: FELIX_EVENT_BATCH_MAX_DELAY_US
Example:
event_batch_max_delay_us: 250Tuning:
- Ultra-low latency:
50-100us - Balanced:
250-500us - High throughput:
1000-5000us
fanout_batch_size
Section titled “fanout_batch_size”Description: Publish-side fanout batching hint and subscribe delivery batch cap.
Type: usize (count)
Default: 64
Environment: FELIX_FANOUT_BATCH
Example:
fanout_batch_size: 64Recommendations:
- Low fanout (1-10):
16-32 - Medium fanout (10-100):
64-128 - High fanout (100+):
128-256
Event Frame Encoding
Section titled “Event Frame Encoding”Subscription event delivery uses binary EventBatch frames.
Outbound Writer Lanes
Section titled “Outbound Writer Lanes”Broker outbound subscribe delivery uses lane-sharded writer tasks to reduce contention under high fanout / large payload workloads.
subscriber_queue_capacity
Section titled “subscriber_queue_capacity”Description: Per-subscriber queue capacity in broker core (drop-on-full boundary).
Type: usize (count)
Default: 512
Environment: FELIX_SUBSCRIBER_QUEUE_CAPACITY (alias: FELIX_SUB_QUEUE_CAPACITY)
subscriber_queue_capacity: 512max_subscriptions_per_conn
Section titled “max_subscriptions_per_conn”Description: Max concurrent subscriptions a single QUIC connection may hold. subscriber_queue_capacity bounds the size of one subscription’s buffer; this bounds how many subscriptions one connection can open in total, which is otherwise unbounded — a connection issuing unlimited Subscribe requests could grow broker memory without limit.
Type: usize (count)
Default: 4096
Environment: FELIX_MAX_SUBSCRIPTIONS_PER_CONN
max_subscriptions_per_conn: 4096subscriber_queue_policy
Section titled “subscriber_queue_policy”Description: Backpressure policy when a subscriber’s broker-core queue (subscriber_queue_capacity) is full — this is the fanout enqueue path, upstream of the writer lanes below.
Type: enum (block, drop_new, drop_old)
Default: drop_new
Environment: FELIX_SUB_QUEUE_POLICY
subscriber_queue_policy: drop_newTuning:
drop_new(default): sheds the newest event when the queue is full — bounds latency, overload becomes visible as drops (felix_subscribe_dropped_total).block: publish waits for queue space — strongest delivery guarantee, but a single slow subscriber can add latency to publishers. Used by the benchmark harness’s lossless throughput mode alongsidepub_ingress_wait.drop_old: currently emulated withdrop_newsemantics; tracked separately in metrics.
subscriber_writer_lanes
Section titled “subscriber_writer_lanes”Description: Requested number of outbound writer lanes.
Type: usize (count)
Default: 4
Environment: FELIX_SUB_WRITER_LANES
subscriber_writer_lanes: 4subscriber_lane_queue_depth
Section titled “subscriber_lane_queue_depth”Description: Bounded command queue depth per writer lane.
Type: usize (count)
Default: 64
Environment: FELIX_SUB_LANE_QUEUE_DEPTH (alias: FELIX_SUB_QUEUE_BOUND)
subscriber_lane_queue_depth: 64subscriber_lane_queue_policy
Section titled “subscriber_lane_queue_policy”Description: Backpressure policy for the writer-lane command queue (downstream of subscriber_queue_policy; gates the actual QUIC write).
Type: enum (block, drop_new, drop_old)
Default: drop_new
Environment: FELIX_SUB_QUEUE_MODE (alias: FELIX_SUB_LANE_QUEUE_POLICY)
subscriber_lane_queue_policy: drop_newSame semantics as subscriber_queue_policy, applied one stage later in the pipeline. Control commands (subscriber register/unregister) always use blocking send regardless of this setting.
max_subscriber_writer_lanes
Section titled “max_subscriber_writer_lanes”Description: Safety clamp for subscriber_writer_lanes to avoid oversubscription regressions.
Type: usize (count)
Default: 8
Environment: FELIX_MAX_SUB_WRITER_LANES
max_subscriber_writer_lanes: 8subscriber_lane_shard
Section titled “subscriber_lane_shard”Description: Lane assignment policy for subscriber outbound writes.
Type: enum (auto, subscriber_id_hash, connection_id_hash, round_robin_pin)
Default: auto
Environment: FELIX_SUB_LANE_SHARD
subscriber_lane_shard: autoPolicy guidance:
auto: Prefer connection-aware routing when connection id is available; fallback to subscriber id.subscriber_id_hash: Good general distribution independent of connection topology.connection_id_hash: Useful when many subscribers share connections and connection-local contention dominates.round_robin_pin: Pins lane at subscribe-time; preserves ordering, but can underperform in skewed workloads.
subscriber_single_writer_per_conn
Section titled “subscriber_single_writer_per_conn”Description: If true, route all subscribers on the same QUIC connection to one writer lane (serializes writes per connection instead of spreading them across lanes).
Type: bool
Default: false
Environment: FELIX_SUB_SINGLE_WRITER_PER_CONN (1, true, yes = enabled)
subscriber_single_writer_per_conn: falseTuning: The latency-focused benchmark profile (batch = 1) enables this for stable per-message ordering; the throughput profile leaves it off to use parallel lanes.
subscriber_flush_max_items
Section titled “subscriber_flush_max_items”Description: Maximum queued lane commands drained per flush before a write is issued.
Type: usize (count)
Default: 16
Environment: FELIX_SUB_FLUSH_MAX_ITEMS
subscriber_flush_max_items: 16subscriber_flush_max_delay_us
Section titled “subscriber_flush_max_delay_us”Description: Maximum time spent waiting to fill a lane flush buffer before writing what’s accumulated.
Type: u64 (microseconds)
Default: 50
Environment: FELIX_SUB_FLUSH_MAX_DELAY_US
subscriber_flush_max_delay_us: 50subscriber_max_bytes_per_write
Section titled “subscriber_max_bytes_per_write”Description: Upper bound on coalesced bytes per QUIC write call to a subscriber stream.
Type: usize (bytes)
Default: 65536 (64 KiB)
Environment: FELIX_SUB_MAX_BYTES_PER_WRITE
subscriber_max_bytes_per_write: 65536Delivery Stream Topology
Section titled “Delivery Stream Topology”sub_streams_per_conn
Section titled “sub_streams_per_conn”Description: Number of delivery streams to use per connection in hashed-pool mode (sub_stream_mode: hashed_pool).
Type: usize (count)
Default: 4
Environment: FELIX_SUB_STREAMS_PER_CONN
sub_streams_per_conn: 4sub_stream_mode
Section titled “sub_stream_mode”Description: Strategy for mapping subscribers to event streams.
Type: enum (per_subscriber, hashed_pool)
Default: per_subscriber
Environment: FELIX_SUB_STREAM_MODE
sub_stream_mode: per_subscriberNotes: hashed_pool is not yet enabled — the broker currently falls back to per_subscriber and logs a debug warning if hashed_pool is requested.
Cache Configuration
Section titled “Cache Configuration”cache_conn_recv_window
Section titled “cache_conn_recv_window”Description: Cache connection flow-control receive window.
Type: u64 (bytes)
Default: 268435456 (256 MiB)
Environment: FELIX_CACHE_CONN_RECV_WINDOW
Example:
cache_conn_recv_window: 268435456Notes:
- Per-connection receive credit
- Multiplied by connection pool size
- Affects burst tolerance
cache_stream_recv_window
Section titled “cache_stream_recv_window”Description: Cache stream flow-control receive window.
Type: u64 (bytes)
Default: 67108864 (64 MiB)
Environment: FELIX_CACHE_STREAM_RECV_WINDOW
Example:
cache_stream_recv_window: 67108864Notes:
- Per-stream receive credit
- Multiplied by streams per connection
- Total credit =
stream_window × streams_per_conn × conn_pool
cache_send_window
Section titled “cache_send_window”Description: Cache connection send window.
Type: u64 (bytes)
Default: 268435456 (256 MiB)
Environment: FELIX_CACHE_SEND_WINDOW
Example:
cache_send_window: 268435456Notes:
- Per-connection send credit
- Affects concurrent request throughput
Worker and Queue Configuration
Section titled “Worker and Queue Configuration”pub_workers_per_conn
Section titled “pub_workers_per_conn”Description: Publish worker count per QUIC connection.
Type: usize (count)
Default: 4
Environment: FELIX_BROKER_PUB_WORKERS_PER_CONN
Example:
pub_workers_per_conn: 4Recommendations:
- Low concurrency:
2-4 - High concurrency:
8-16 - Match to expected concurrent publishers per connection
pub_queue_depth
Section titled “pub_queue_depth”Description: Per-worker publish queue depth.
Type: usize (count)
Default: 64
Environment: FELIX_BROKER_PUB_QUEUE_DEPTH
Example:
pub_queue_depth: 64Tuning:
- Larger values allow more buffering under burst but increase saturation latency
- Affects memory usage per worker
- Consider with
publish_queue_wait_timeout_ms
pub_inflight_bytes
Section titled “pub_inflight_bytes”Description: Shared in-flight publish byte budget across all publish workers (process-wide, not per-connection).
Type: usize (bytes)
Default: 67108864 (64 MiB)
Environment: FELIX_BROKER_PUBLISH_INFLIGHT_BYTES
Example:
pub_inflight_bytes: 67108864Tuning:
pub_queue_depthbounds the number of queued jobs, but a job’s payload can be as large asmax_frame_bytes;pub_inflight_bytesbounds actual queued-or-processing bytes regardless of item count.- The budget is acquired before a job is handed to a worker queue and released only once the job finishes processing, so it reflects real resident memory, not just admission-time bytes.
- Should be set well above
max_frame_bytes— a job larger than the remaining budget waits (and can time out underEnqueuePolicy::Wait) rather than being admitted. - Lower this to shrink worst-case ingress memory under large-payload workloads; raise it to allow more large batches in flight concurrently.
pub_conn_inflight_bytes
Section titled “pub_conn_inflight_bytes”Description: Per-connection share of pub_inflight_bytes. pub_inflight_bytes is intentionally process-wide (see its description above), which on its own means nothing stops one connection from occupying the entire shared budget. pub_conn_inflight_bytes closes that gap: it’s a second, independent byte-budget gate sized per connection, checked before the shared budget on every publish admission.
Type: usize (bytes)
Default: 16777216 (16 MiB)
Environment: FELIX_BROKER_PUBLISH_CONN_INFLIGHT_BYTES
Example:
pub_conn_inflight_bytes: 16777216Tuning:
- Must be smaller than
pub_inflight_bytesto have any effect; setting it equal to or abovepub_inflight_bytesmeans a single connection can once again claim the whole shared budget. - Roughly
pub_inflight_bytes / Nfor the expected number of concurrently active connections gives each a fair share while still allowing the shared budget to absorb bursts from fewer connections.
pub_ingress_wait
Section titled “pub_ingress_wait”Description: When true, un-acked (fire-and-forget) publishes wait — bounded by publish_queue_wait_timeout_ms — for ingress capacity instead of being shed when the publish queue or byte budget is full.
Type: bool
Default: false
Environment: FELIX_PUB_INGRESS_WAIT
Example:
pub_ingress_wait: trueTuning:
- Off (default): overload sheds fire-and-forget publishes visibly (
felix_broker_ingress_dropped_total) and keeps latency bounded. - On: backpressure propagates through QUIC flow control to the publisher — nothing is shed, producers slow down. Use for lossless pipelines and sustainable-throughput benchmarking.
core_shards
Section titled “core_shards”Description: Number of core-pinned shard executors owning stream work (thread-per-core). Each stream’s handle id deterministically selects an owning shard; that shard runs the stream’s publish worker and its subscriptions’ lane feeders on one dedicated single-threaded runtime (pinned to a CPU core on Linux). Publish append, fanout enqueue, and subscriber dequeue all stay core-local.
Type: usize (count; 0 = disabled)
Default: 0
Environment: FELIX_CORE_SHARDS
Example:
core_shards: 4Tuning:
- When enabled, the publish worker count becomes the shard count (one worker per shard), superseding
pub_workers_per_conn. - Benefits scale with stream count: workloads spread across many streams gain parallel, contention-free per-core pipelines (measured +34% delivered throughput at 4 streams × 4 shards, unpinned). Single-stream workloads serialize on one shard by design — neutral to mildly positive.
- Core pinning requires Linux (
sched_setaffinity); elsewhere shards still get dedicated threads, preserving the single-writer ownership model without hard affinity. - Reasonable starting point: number of physical cores minus 2 (leaving headroom for QUIC I/O on the main runtime).
Performance Configuration
Section titled “Performance Configuration”disable_timings
Section titled “disable_timings”Description: Disable per-stage timing collection for lower overhead.
Type: bool
Default: false
Environment: FELIX_DISABLE_TIMINGS (1, true, yes = disabled)
Example:
disable_timings: trueTrade-offs:
false: Detailed latency metrics, slight overheadtrue: Maximum performance, no per-stage timings
Recommendations:
- Development:
false(debug performance) - Production low-load:
false(observability) - Production high-load:
true(reduce overhead)
control_stream_drain_timeout_ms
Section titled “control_stream_drain_timeout_ms”Description: Maximum time to wait for control-stream writer to drain.
Type: u64 (milliseconds)
Default: 50
Environment: FELIX_CONTROL_STREAM_DRAIN_TIMEOUT_MS
Example:
control_stream_drain_timeout_ms: 50Notes:
- Affects graceful connection shutdown
- Balance between responsiveness and reliability
Client-Side Configuration
Section titled “Client-Side Configuration”While this reference covers broker configuration, clients also have tunable parameters:
Event Connection Pool
Section titled “Event Connection Pool”Environment: FELIX_EVENT_CONN_POOL
Default: 8
Description: Number of QUIC connections in the event pool.
Cache Connection Pool
Section titled “Cache Connection Pool”Environment: FELIX_CACHE_CONN_POOL
Default: 8
Description: Number of QUIC connections for cache operations.
Cache Streams Per Connection
Section titled “Cache Streams Per Connection”Environment: FELIX_CACHE_STREAMS_PER_CONN
Default: 4
Description: Concurrent cache streams per connection.
Publish Chunk Bytes
Section titled “Publish Chunk Bytes”Environment: FELIX_PUBLISH_CHUNK_BYTES
Default: 16384 (16 KiB)
Description: Chunk size for publishing large messages.
Configuration Validation
Section titled “Configuration Validation”Felix validates configuration at startup:
# Test configurationcargo run --release -p broker -- --dry-run
# Explicit config fileFELIX_BROKER_CONFIG=/path/to/config.yml cargo run --release -p brokerCommon validation errors:
- Invalid socket address format
- Negative or zero values where positive required
- Conflicting settings
Performance Profiles
Section titled “Performance Profiles”Low Latency (p50-optimized)
Section titled “Low Latency (p50-optimized)”event_batch_max_events: 1event_batch_max_delay_us: 50fanout_batch_size: 16subscriber_writer_lanes: 2subscriber_lane_shard: autosubscriber_queue_capacity: 64disable_timings: trueBalanced (recommended)
Section titled “Balanced (recommended)”event_batch_max_events: 64event_batch_max_bytes: 65536event_batch_max_delay_us: 250fanout_batch_size: 64subscriber_writer_lanes: 4subscriber_lane_shard: autosubscriber_queue_capacity: 512disable_timings: falsecache_conn_recv_window: 268435456High Throughput (batch-optimized)
Section titled “High Throughput (batch-optimized)”event_batch_max_events: 256event_batch_max_bytes: 1048576event_batch_max_delay_us: 1000fanout_batch_size: 128subscriber_writer_lanes: 8max_subscriber_writer_lanes: 8subscriber_lane_shard: autosubscriber_queue_capacity: 4096pub_ingress_wait: truecore_shards: 4disable_timings: trueHigh Memory (burst-tolerant)
Section titled “High Memory (burst-tolerant)”cache_conn_recv_window: 536870912cache_stream_recv_window: 134217728cache_send_window: 536870912subscriber_queue_capacity: 2048subscriber_lane_queue_depth: 16384pub_queue_depth: 2048Configuration Examples
Section titled “Configuration Examples”Development
Section titled “Development”quic_bind: "127.0.0.1:5000"metrics_bind: "127.0.0.1:8080"disable_timings: falseevent_batch_max_events: 32Production Single-Node
Section titled “Production Single-Node”quic_bind: "0.0.0.0:5000"metrics_bind: "0.0.0.0:8080"ack_on_commit: truedisable_timings: trueevent_batch_max_events: 64event_batch_max_delay_us: 250subscriber_writer_lanes: 4subscriber_lane_shard: autocache_conn_recv_window: 268435456Production Cluster
Section titled “Production Cluster”quic_bind: "0.0.0.0:5000"metrics_bind: "0.0.0.0:8080"controlplane_url: "http://felix-controlplane:8443"controlplane_sync_interval_ms: 2000ack_on_commit: truedisable_timings: trueevent_batch_max_events: 128event_batch_max_bytes: 262144fanout_batch_size: 128subscriber_writer_lanes: 4subscriber_lane_shard: autoNext Steps
Section titled “Next Steps”- Environment variables reference: Environment Variables
- Troubleshooting issues: Troubleshooting Guide
- Performance tuning: Performance Guide