The system beneath every moving world.
Trace one live session from a typed command to a generated frame. Each layer stays independently observable while sharing one session boundary.
Live system cross-section
One session,
five observable layers.
Select a layer to inspect its contract. The command path moves toward compute; generated frames return through the same live session.
Endpoint
/v1/sessions
Transport
WebRTC + WS
Usage
Ready time
Session state
Generation is treated as a living session. The runtime holds model state between commands and gives the client one canonical lifecycle to reconnect to.
- Lifecycle
- disconnected → connecting → queued → ready
- Reconnect
- Rejoin by session ID after a recoverable disconnect
- Timeline
- Commands apply at model-specific chunk boundaries
- Observability
- State messages, chunk events, and command errors
- Metering
- Usage begins only after the session reaches ready
Platform / next step
One protocol.
Video and interactive worlds.
Explore modelsPlatform reference
Typed contract
The SDK and runtime agree on lifecycle, capabilities, tracks, commands, messages, and recoverability. A model changes capability—not the shape of the product integration.
- SDKs
- JavaScript, React, and Python
- Files
- uploadFile() for image and clip inputs
- Messages
- State, generation, chunk, and command events
- Errors
- Command failures reported through protocol messages
- Billing
- Connecting and waiting are free; ready is metered
Streaming transport
A bidirectional connection designed around frames instead of requests. Control signals travel upstream while generated video returns on a named realtime track.
- Transport
- LiveKit WebRTC or direct-worker WebSocket
- Session API
- sdk.realtime.v2 · /v1/sessions
- Media out
- recvonly main_video track
- Media in
- Model-dependent camera or uploaded source tracks
- Handshake
- Commands, tracks, and limits advertised on connect
Session state
Generation is treated as a living session. The runtime holds model state between commands and gives the client one canonical lifecycle to reconnect to.
- Lifecycle
- disconnected → connecting → queued → ready
- Reconnect
- Rejoin by session ID after a recoverable disconnect
- Timeline
- Commands apply at model-specific chunk boundaries
- Observability
- State messages, chunk events, and command errors
- Metering
- Usage begins only after the session reaches ready
Model adaptation
Every research release arrives with a different control surface and runtime profile. Windflow translates those differences without hiding model-specific capability.
- Optimization
- Compiled paths, quantized profiles, and custom kernels
- Commands
- Only supported controls are exposed per model
- Tracks
- Generated and source media are explicitly named
- Limits
- Resolution, chunk size, latency, and scene budgets stay visible
- Status
- Production lifecycle published independently from live capacity
GPU orchestration
Long-lived model workers run behind a provider-neutral HTTP boundary. Capacity can move between supported GPU hosts without changing the session API.
- Worker
- Long-lived authenticated service with health reporting
- Weights
- Persistent model data mounted beside the worker
- Media
- Workers publish generated tracks through LiveKit
- Profiles
- GPU class selected per model and optimization profile
- Recovery
- Worker health and session state remain independently visible