# How to turn an open-source video model into a realtime API

> Prepare a video model for a hosted realtime API: licensed weights, repeatable inference, streaming transport, session controls and measured performance.

Source: https://windflow.online/guides/turn-an-open-source-video-model-into-a-realtime-api · Updated 2026-10-11 · Windflow Labs

A video model becomes a realtime API when it can produce a second of video in under a second, in chunks, with a way to take new controls between chunks, and when something around it holds the session, streams the frames and meters the time. The first part is the model's job. The second part is infrastructure, and it is most of the work.

This guide covers both halves: how to tell whether a model can run in realtime, and what a platform adds to make it a product. If you want a model hosted rather than building the platform yourself, the last section says what to prepare.

## Part 1: can the model run in realtime?

Run through five checks before investing in serving.

1. **Generation is faster than playback, with margin.** At the resolution you will deliver, the model must produce a second of video in clearly less than a second on the GPU you plan to use. Margin matters, because a stream that only just keeps up will stall the first time the GPU is busy.
2. **It generates in chunks, not whole clips.** Autoregressive or chunked models emit a few frames at a time, each conditioned on the last. A model that denoises the entire clip at once has to finish before the first frame exists.
3. **It takes controls between chunks.** A realtime model needs a place in its loop to accept a new prompt, image, key press or camera frame. If the only input is a prompt fixed at the start, it can stream but not be steered.
4. **It runs in a few denoising steps.** Standard diffusion uses dozens of steps per chunk. Realtime models are usually distilled to a handful, using techniques such as Self-Forcing or DMD2. If you have a high-quality many-step model, distillation is the first project.
5. **It stays coherent for a long run.** Check drift: run it for minutes, not seconds, and see whether identity and geometry hold. Models that fail here need anchoring, memory or periodic re-conditioning.

If a model fails check 1 or 4 at your target quality, serving will not fix it; the work is in the model.

## Part 2: what a platform adds around the model

A model that passes the checks is still not an API. The pieces around it are:

| Layer | What it does |
|---|---|
| Runtime image | Pins source and dependencies, restores weights, makes startup repeatable, exposes health |
| Session contract | Maps readiness, lifecycle, reconnects, named output tracks, messages, limits and failures |
| Controls and media | Wires only the prompts, images, video, movement, camera or actions the model really supports; encodes and delivers frames |
| Placement | Finds a GPU for each session, brings capacity up on demand, and replaces a worker that fails |
| Metering and recording | Meters ready time, applies reservations and limits, and manages temporary recordings |
| Observability | Measures first-frame time, delivered frame rate, stalls and control latency from the client |

Each layer has failure modes that only show up under real traffic: a worker that is healthy but slow, a reconnect that arrives after the session was reaped, a control that lands one chunk late. That is why teams that wrap a model themselves often spend longer on the layers than on the model.

## What to prepare if you want Windflow to host it

Windflow hosts and optimises realtime models on NVIDIA and AMD GPUs and publishes them behind the same session API as its [model catalog](https://windflow.online/models). It asks for four things:

1. **A complete checkpoint at an exact revision,** with every required asset and a stable way to fetch it.
2. **Permission to store, package and serve it.** A public licence or written authorisation must cover hosted inference.
3. **One clean launch path** from input to output on a Linux GPU environment, container or command, with a fixed reference input.
4. **A baseline we can reproduce.** Name the hardware, resolution, precision, steps, batch, warm-up, sample duration and whether the frame rate is native or interpolated.

There are three ways to work together: licensed weights hosted by Windflow, a vendor API integrated behind the same session contract, or a joint project with a scoped goal. Integration is non-exclusive, performance numbers are reviewed with the model's authors before anything is published, and a model can be delisted on request. Details are on the [bring a model](https://windflow.online/bring-a-model) page.

## What does not work

- **An announcement without an inference release.** A paper, teaser or placeholder repository is not something that can be served.
- **Weights that cannot be fetched** and no endpoint to evaluate instead.
- **Unclear hosting rights.** A licence that does not cover served inference needs a written exception.
- **A baseline that does not reproduce,** because it depends on private services, manual steps or undisclosed interpolation.

## FAQ

### Can I bring my own open-source video model and get a realtime API for it?

Yes, if it can generate in chunks faster than playback and accept controls between chunks. Windflow hosts qualifying models behind its session API; you provide the checkpoint, hosting rights, a runnable path and a reproducible baseline. See [bring a model](https://windflow.online/bring-a-model).

### What frame rate counts as realtime?

The model must generate faster than it plays back at the resolution you deliver. For 24 frames-per-second playback that means producing more than 24 frames per second sustained, with margin, not a burst.

### Does Windflow charge model teams?

For selected open models, Windflow does not charge the model team for integration and covers public catalog serving. Dedicated capacity, vendor API integration and joint projects are scoped and priced in writing before GPU work starts.

### Can I host the model elsewhere too?

Yes. Integration is non-exclusive, and your team keeps its rights.