# Windflow realtime inference platform

Source: https://windflow.online/platform · Windflow Labs

## Typed contract

The SDK and runtime agree on lifecycle, capabilities, tracks, commands, messages, and recoverability. A model changes capability—not the shape of the product integration.

- SDKs: JavaScript, React, and Python
- Files: uploadFile() for image and clip inputs
- Messages: State, generation, chunk, and command events
- Errors: Command failures reported through protocol messages
- Billing: Connecting and waiting are free; ready is metered

## Streaming transport

A bidirectional connection designed around frames instead of requests. Control signals travel upstream while generated video returns on a named realtime track.

- Transport: LiveKit WebRTC or direct-worker WebSocket
- Session API: sdk.realtime.v2 · /v1/sessions
- Media out: recvonly main_video track
- Media in: Model-dependent camera or uploaded source tracks
- Handshake: Commands, tracks, and limits advertised on connect

## Session state

Generation is treated as a living session. The runtime holds model state between commands and gives the client one canonical lifecycle to reconnect to.

- Lifecycle: disconnected → connecting → queued → ready
- Reconnect: Rejoin by session ID after a recoverable disconnect
- Timeline: Commands apply at model-specific chunk boundaries
- Observability: State messages, chunk events, and command errors
- Metering: Usage begins only after the session reaches ready

## Model adaptation

Every research release arrives with a different control surface and runtime profile. Windflow translates those differences without hiding model-specific capability.

- Optimization: Compiled paths, quantized profiles, and custom kernels
- Commands: Only supported controls are exposed per model
- Tracks: Generated and source media are explicitly named
- Limits: Resolution, chunk size, latency, and scene budgets stay visible
- Status: Production lifecycle published independently from live capacity

## GPU orchestration

Long-lived model workers run behind a provider-neutral HTTP boundary. Capacity can move between supported GPU hosts without changing the session API.

- Worker: Long-lived authenticated service with health reporting
- Weights: Persistent model data mounted beside the worker
- Media: Workers publish generated tracks through LiveKit
- Profiles: GPU class selected per model and optimization profile
- Recovery: Worker health and session state remain independently visible