Maschinenraum
A multiplayer social room drawn as a neon isometric world. People open a link, pick a nickname, walk around, talk, sit at tables, open doors, and rebuild the room together. A Rust server keeps each room in a single task. A Rust client compiled to WebAssembly draws everything as vector geometry on the GPU.
Goals
- Zero-install: runs in any browser with WebGPU or WebGL2.
- Up to 50 people per room, around 500 per process, with latency low enough that it feels local.
- Live editing by anyone let in (or only by holders of the builder key), with every change recorded.
- A predictable hot path: no locks, no steady-state allocation, bounded queues everywhere.
Out of scope for v1
- Per-room permissions: access and building are process-wide keys (
MR_ACCESS_KEY,MR_BUILDER_KEY). Voice and claimed nicks came later. (Commands are rate limited per connection: a token bucket of 10/s, burst 20, that also charges for search work.) - Horizontal scaling across processes (rooms are in-process actors).
- A rollback command (the event log makes one possible; it isn't built).
Decision record
Choices made before any code was written, and the reasoning behind them. Select a row for the alternatives.
Crates and runtime
Six crates, and xtask for the checks. protocol is the shared contract (messages, floors, the world tick, walking) and holds every game's rules (the half pipe, transfer, football, light cycles, breakout, Spindizzy, Bomberman, Commando, Kobold, the lexicon's fold, the lift and paternoster), so the client and server cannot disagree within one build; race (ship physics, AI, weapons) and props (deterministic ball and glass physics) are simulation code both sides run. Commando is lockstep: the runner's browser, every onlooker's and the room step the same deterministic game from one input byte a tick. Kobold is integer only, so a run is fixed by its job, its case and the programs; the room scores a submission's 100 cases off its own task. A Kobold duel is two games with the homes swapped, side 1 reading links through the arena's mirror; the room plays it on its own task in well under a millisecond. A relay shift is its line, its seed and its swaps; browsers play it half a second behind the room and replay from the start when a swap comes late, and the room replays it whole at the end to score it. A light show is its program, its seed and its beat period, the period taken from the disco's tempo; only browsers step it, and the room checks a set piece on its own task. The server is split into a library and a binary so tests and benchmarks can drive a room without sockets.
flowchart LR protocol["protocol
serde + postcard"] --> server["server
axum · tokio · duckdb"] protocol --> client["client
wasm · wgpu · lyon"] protocol --> loadgen["loadgen
bot swarm"] race["race
ships · AI · weapons"] --> server race --> client props["props
deterministic physics"] --> server props --> client server --> tests["tests/alloc.rs
benches/room.rs"] client --> tbench["benches/tessellate.rs
(native, scene only)"]
| Component | Owns | Concurrency |
|---|---|---|
| Room directory | name → room inbox sender, the world map | One parking_lot mutex, taken on joins, hand-offs, neighbour observation and retirement, never per message |
| Room task | Players, residents, objects, the walk grid, A* scratch, props, the bartender, the games under way (and Commando's copy of each run), the DJ deck, the lexicon's tree, broadcast arena | One tokio task per room; single owner, no locks; rooms message each other only with try_send |
| Race session | One race at a gate | One task per gate, 60 Hz; inputs bypass the room |
| Kobold scoring | One submission's 100 cases | spawn_blocking; the verdict comes back to the room's inbox as RoomMsg::KoboldScored |
| Load sampler | The process's CPU and memory, two minutes | One OS thread, a watch rooms read without a lock |
| Census | Each room's headcount | A watch in the directory, written on join and leave |
| Tempo | The disco's tempo | A watch in the directory (Directory::tempo), written by the deck, read when a Kobold light show starts |
| Reader task | Nothing; forwards raw frames | One per connection |
| Writer task | The socket sink | One per connection; feeds every queued frame, flushes once per batch |
| Event log thread | The DuckDB connection | One OS thread; batched appends. Loads and the hourly archive run on a second thread with a cloned connection |
From a click to every screen
Pick an action to watch it travel through the server. Each hop is a bounded channel. Watch where frames are decoded, where they are encoded, and what gets logged.
The lobby, live
This is the real rooms/lobby.ron, using the same projection (64 × 32 tiles), the same blocking rules and a breadth-first search (on a flat floor it finds paths as short as the server's A*). Click a tile to walk there, or click a door to toggle it. Hover to read the tile coordinate and the depth value the GPU uses to order it.
The server's A* runs over a per-tile walk grid that also encodes stairs (ramps climb along their rotation and link to the landing a floor up) and checks each step's height: parts of a floor can be raised or sunken, and a step taller than 0.5 is not walked. A search has a 200,000-tile budget. The server keeps the grid up to date as objects change, so a search never rebuilds it. A search marks tiles with a generation number instead of clearing its arrays, and it reuses one queue and one path buffer. Seats and plants are walkable, doors only when open. The start tile is never checked, so a player an edit has boxed in can always walk off.
Postcard, byte by byte
One binary WebSocket frame per message. Postcard writes the enum variant index, then the fields in order: integers as varints (signed ones zig-zag encoded), strings as a length followed by UTF-8. There are no field names and no version byte. These are real encodings produced by the protocol crate.
Decoding is zero-copy: C2S<'a> borrows &str straight from the received frame. S2C<'a> uses Cow, so the server encodes from its own state without cloning. Because variant order is the format, new variants are only ever appended.
Base file plus an append-only log
A room's layout is always rooms/<name>.ron with its logged edits replayed on top, from the room's latest snapshot (the compacted edits; a new one after 256 more). Chat, join, leave and served are history, archived to Parquet after a day. Game results, Kobold programs and champions, and letters are rows read at load; Kobold duels replay into the ratings, and the lexicon's lines into its tree. Runtime state a crash should not lose (open doors, the DJ deck, matches under way) is written to room_state; seats reset.
CREATE TABLE events ( room VARCHAR NOT NULL, seq BIGINT NOT NULL, -- per room, contiguous from 1 ts_us BIGINT NOT NULL, -- UTC µs since epoch (batch flush time) nick VARCHAR NOT NULL, kind VARCHAR NOT NULL, -- place | remove | rotate | screen | portal | drop | take | chat | ... payload VARCHAR NOT NULL, -- JSON of eventlog::Event; append-only contract PRIMARY KEY (room, seq) );
Replay
| Object | Kind | Tile | rot | State after replay |
|---|
Two id ranges
Base-file ids stay below 220. Objects placed at runtime are numbered from 220 upward. Logged remove and rotate events name objects by id, so adding objects to a base file can never shift what an old event points at.
Write path
- The room validates an edit, applies it, broadcasts it, then sends
LogMsg::Appendinto a channel bounded at 4096. If the channel is full, the room waits; edits are never dropped. - The log thread takes a row, waits 200 ms, drains up to 1024 more rows, and writes them in one
Appenderflush. LoadandFlushrequests are processed after the rows queued before them. A room that is loading, or a shutdown in progress, sees every event accepted earlier.- SIGTERM flushes before exit. A hard kill can lose up to about 200 ms of events. A DuckDB error exits the process, so memory and disk never disagree.
Beams, a light volume, six looks
The client uses no image assets. Every shape is built in code: fills as triangles, edges as analytic glowing beams (tracy's line renderer), avatars as rigged instanced templates. It draws with wgpu (WebGPU, falling back to WebGL2) into an HDR target with a depth buffer: fills write depth with a slope-scaled bias, beams test it and never write it. Text such as names and chat is DOM over the canvas; signs, boards and billboards are stroke-font lines in the scene.
| Data | Contents | Rebuilt | Per-frame CPU |
|---|---|---|---|
| Static scene | Floors (raised parts and risers), walls and their style's dressing, every object, the walls along the seams next door, the city (in photo mode, every room of the world if asked) | When an object or a neighbour changes | none |
| Light volume | A 3D grid of light from lamps, glowing things and neon, over the room you are in | When light changes; an edit rebakes only its region | none |
| Templates | Avatars, props, markers | Once at startup | none |
| Instances | Position, pose, colours, 80 bytes each | Every frame | one buffer write |
| Dynamic lines | Pings, the model's wall, live billboards, games, the lexicon's tree | When they change | rare; every frame while a game moves |
| Dynamic fills | A Commando battlefield, painted by the room's painter as it scrolls; its ground stamps the floor stencil so shadows fall on it. Or a Kobold machine floor: the job's hosts, files and kobolds. A room has one or the other | Every frame while a run or the table's demo shows; every frame in a room with a Kobold floor | one buffer write |
Light. The volume holds, per cell, the light's colour and its direction per channel; a surface reads four samples and shades by its own normal, so any number of lights costs the same per pixel. Walls block light by the rule that blocks sight. On WebGPU a compute pass traces bounce light, neon and ambient occlusion over the volume's voxels. Only the room you are in is drawn and lit; next door shows just its walls, doors and windows along the seam.
Looks. Six shader looks (Diorama, Lounge, Atmosphere, Classic vector, Graphic, Comic) share one scene. Every look in the menu (52) is a profile on one of them that sets every display setting, the vertex, edge and face themes and the line weights, and grades the frame; a Quality setting (Low, Medium, High, Auto, which steps down while frames stay slow) caps what a look may switch on. Photo mode draws lighter while its camera moves and at its best, past Quality, once still.
flowchart LR G["GI compute
WebGPU only"] --> S S["scene pass
HDR + depth, MSAA"] --> B["bloom chain
down and up mips"] S --> C["composite
AgX or ACES · grade · streak · text mode · film"] B --> C --> O["canvas"]
The HDR target is Rgba16Float when the adapter can render to, filter and blend it, otherwise Rgba8Unorm on the WebGL2 path. Struct layouts shared with WGSL are mirrored by hand: CameraUniform 304 bytes, Line 64, Instance 80.
Device, targets, bindings, one frame
One Gfx (client/src/gfx.rs) owns the wgpu device, every target and pipeline, and records each frame into a single command encoder. Three WGSL modules: scene.wgsl (everything in the world), post.wgsl (bloom and the composite) and gi.wgsl (the light tracer, compute). The app hands it a Frame each animation frame: the view, the look and settings, the instances and the dynamic lines.
Backends
new_instance_with_webgpu_detection picks WebGPU where the browser has it and WebGL2 otherwise; the status line shows which. The same code runs on both; what differs is what each can do.
| WebGPU | WebGL2 | |
|---|---|---|
| Traced light (compute) | Off, On or Ultra | none: the baked volume only |
| HDR targets | Rgba16Float | Rgba16Float if it can render to, filter and blend it, else Rgba8Unorm |
| Rich extras | dust motes, the nebula (Camera.look.y) | left out |
| GPU timings (F3) | timestamp queries per segment | none |
| Quality on Auto | High | Low |
Pipelines only some themes need (ink, hidden lines, vertices) are built the first time a theme asks: WebGL2's shader compiler is slow, and nobody should pay for them at join. client/tests/shaders.rs validates every entry point with naga and translates it to WebGL2 GLSL with each override the client sets (TRAILS, AGX).
Targets
| Target | Format | Lives | Purpose |
|---|---|---|---|
| Scene | HDR, 4× MSAA resolved to single-sample | per frame | The world: fills, beams, rain, reflections |
| Depth + stencil | Depth24PlusStencil8, multisampled | per frame | Fills write depth with a slope bias, beams only test it; the stencil marks your floor for shadows and reflections |
| Phosphor | HDR, persistent | across frames | Avatar trails: decayed each frame, then deposited into |
| Bloom chain | HDR, 6 mips | per frame | Soft-knee threshold, 6 down, 5 up |
| Light volume | 4 × 3D Rgba16Float | until the layout changes | Colour, and direction per channel, over the room and its neighbours' band |
| Voxels, traced light | 3D Rgba8, 3D Rgba16Float ×3 | voxels on change; traced light ping-pongs | The tracer's scene and its two result textures |
| Tile map | 2D array Rgba8, a layer per floor | on change and on your steps | Fog of war and flat lamp light; the race's light strip |
| Canvas | the surface's, sRGB or encoded by hand | per frame | The composite's output; 1.5× when supersampled, 2× for a capture |
Bindings
scene.wgsl, group 0
| 0 | Camera uniform, 304 bytes (world last: elevation and the line weights) |
| 2 | Lights: up to 32 (avatar glows), two vec4 each |
| 3–4 | Tile map and its linear sampler |
| 5–8 | The light volume: colour, then direction for r, g, b |
| 9 | Traced light (the current of two) |
| 10 | Neon voxels, for Ultra's floor reflections |
Template lines are vertex data (locations 10–13), never a uniform array: Direct3D took minutes per pipeline over one indexed by vertex_index.
post.wgsl and gi.wgsl
| 0–2 | Post: the textures of the pass (scene, bloom or phosphor) |
| 3–4 | Post: sampler, Params (texel size, threshold, bloom strength, sRGB flag) |
| 5 | Post: Photo, two vec4: photo mode, drinks, a look's film |
| 6 | Post: Grade, ten vec4: the look's grade, text mode, streak |
| 0–6 | GI: Params (four vec4), avatars, voxels, neon, direct light, previous result, next (storage) |
One frame
Pick a backend and settings to see which passes run. Skipped passes cost nothing: they are not drawn at a lower setting, they are not drawn.
What crosses to the GPU, and when
| When | Upload | Size |
|---|---|---|
| Every frame | Camera uniform | 304 B |
| Every frame | Instances: people, props, markers | 80 B each |
| Every frame | Avatar glows, the nearest to the camera | ≤ 1 KB |
| Every frame, while it shows | Photo: photo mode, a drink's effect, a look's moving grain | 32 B |
| When they change | Dynamic lines: pings, the model's wall, live billboards, games, the lexicon's tree | 64 B a line |
| Every frame, while it shows | Dynamic fills: a Commando battlefield or a Kobold floor | ≈ 0.2–0.5 MB |
| On your step | Fog of war, the rectangle around you | small |
| On an edit | The static mesh; the light volume's box around the edit; voxels | region only |
| On joining, or a neighbour's change | The whole light volume, room and neighbours' band | ≈ 1–5 MB |
The F3 overlay splits the GPU frame into the same segments as the walk above: fills, avatars, shadows, glass, beams, mirror, avatar beams, phosphor, bloom, composite and gi.
Measured, not assumed
All numbers come from one desktop machine. Load tests ran the bot swarm on the same host, so the bots and the server competed for CPU.
Chat round-trip under load
Log scale. The bottom row is about 4× the design target.
Static rebuild per edit
Native timing. The dashed line is one 60 Hz frame. Large rooms drop a frame on every edit: known hot spot #1.
One move, decode to fan-out
Worst-case search
A serpentine maze where the path visits most tiles. About 217 M tiles/s at every size.
Capacity estimate
from benchesFrames out grows with the square of room size. The room-task figure uses the measured fan-out costs (4.1 / 5.8 / 8.2 µs at 8 / 32 / 64 players), interpolated. Socket writes are the larger real cost and aren't included. Measured at 500 users: 49 k frames/s delivered, p99 5.8 ms.
How it is kept honest
| Tool | What it proves |
|---|---|
tests/alloc.rs | A counting global allocator warms up 50 players, confirms the counter works, then asserts zero allocations across 10,000 moves with fan-out. |
benches/room.rs | Pathfinding on a worst-case maze, the full move path (decode to try_send), and the codec. |
benches/tessellate.rs | The cost of rebuilding the static mesh after an edit, plus template construction. |
loadgen | Real sockets with bots measuring their own chat round-trip through the broadcast path. Reports p50, p90, p99, p99.9 and max. |
--profile profiling | A release build with symbols, for Superluminal, perf or samply. |
What happens when things fill up or fail
| Condition | Behaviour | Why |
|---|---|---|
| Player outbox full (256) | kick after the current message | One stalled client must not stall 63 others |
| Room inbox full (1024) | wait The reader stops reading and TCP pushes back | Backpressure reaches the sender that caused it |
| Event log channel full (4096) | wait The room pauses | Edits are never dropped |
| DuckDB write error | exit The process terminates | Serving on would let memory and disk diverge |
| Corrupt payload on load | panic in that room only | Never run a room with silently lost edits |
| Malformed frame after join | kick | The client is broken or hostile |
| Command bucket empty | drop the command | One client cannot monopolise a room task (10/s, burst 20, searches charged by size) |
| Neighbour room's inbox full | skip that observation or hand-off attempt | A blocking send between two full inboxes would deadlock |
| SIGTERM / Ctrl-C | flush then exit | Everything accepted before the signal is durable |
| Client from an older build | bad frame Reload needed | No version handshake; the server always serves a matching client |
Known hot spots
- A* in the worst case. 275 µs for a corner-to-corner serpentine maze at 128², bounded by the search budget and charged to the client's command bucket. (The static rebuild that was hot spot one is down to 2.1 ms at 128².)
- Latency checkpoints run in every room, O(n²) relays, whether anyone looks or not; past about 120 people in one room they would starve moves and chat. Gate them on the panel or a
#latencyboard. - Text mode and the anamorphic streak are per-pixel heavy (the composite five times per pixel; 20 bloom taps). Both want a low-resolution pre-pass.
- Chat logging shares the room's backpressure. A DuckDB stall long enough to fill 4096 rows pauses the room.
Review checklist
Rules that break silently. Tick them off when reviewing a change; ticks are stored only in this browser.