# How to use the LongLive 2 API

> Use the LongLive-2.0 API to stream a multi-shot video: stage the opening shot, start, then steer with soft shots, hard cuts and shots scheduled ahead of time.

Source: https://windflow.online/guides/longlive-2-api · Updated 2026-11-09 · Windflow Labs

The **LongLive 2 API** on Windflow runs [LongLive-2.0](https://windflow.online/models/longlive-2) as a realtime session: you describe a shot in text, the model streams it at 1280 × 704, and you direct what happens next while it plays. A soft shot steers the running scene without breaking it; a hard cut starts a new scene. Shots can also be scheduled ahead on a shared chunk counter, so a whole sequence can be planned before the first frame.

LongLive-2.0 directs with text only. There is no reference image in this release.

## Start from something runnable

The [public browser examples](https://github.com/windflow-labs/realtime-examples) provide a local Node server and a plain JavaScript client for LingBot, SANA-Streaming, Helios and LongLive. They keep the account key on the server, request a single-session token capped at 180 seconds, and include an explicit release action. The repository states its verification limits; approved beta access and billable model capacity are required for a live run.

You need an approved Windflow account and an API key that begins with `wf_live_` ([request access](https://windflow.online/auth?mode=request)). Requests go to `https://api.windflow.online`.

**1. Check the key (server).** The response lists this model's commands, messages and limitations.

```bash
curl https://api.windflow.online/v1/models/longlive-v2 \
  -H "Authorization: Bearer $WINDFLOW_API_KEY"
```

The model id on the API is `longlive-v2`; the model page lives at `/models/longlive-2`. Use `longlive-v2` everywhere a request names the model.

**2. Mint a browser token (server).** The account key never reaches the browser.

```ts
// For example app/api/windflow-token/route.ts. Authenticate your own user first.
export async function POST() {
  const res = await fetch("https://api.windflow.online/v1/tokens", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.WINDFLOW_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ model: "longlive-v2", max_sessions: 1 }),
  });
  return Response.json(await res.json(), { status: res.status });
}
```

The token is bound to this model (another model gets `403 model_mismatch`) and can create one session. Read `expires_at` from the response instead of assuming a lifetime.

**3. Open the session and draw frames (browser).** Save this as `windflow-session.js`. It is the same file the [Helios guide](https://windflow.online/guides/helios-api) walks through line by line.

```js
// windflow-session.js: plain fetch and WebSocket, no dependencies
const API = "https://api.windflow.online";

export async function call(path, jwt, body) {
  const res = await fetch(API + path, {
    method: body ? "POST" : "GET",
    headers: { Authorization: `Bearer ${jwt}`, "Content-Type": "application/json" },
    body: body && JSON.stringify(body),
  });
  const data = await res.json();
  if (!res.ok) throw Object.assign(new Error(data.error.message), data.error, { status: res.status });
  return data;
}

export function openSession(model, canvas, onMessage = () => {}) {
  const ctx = canvas.getContext("2d");
  let session, ws, closed = false, released = false;
  let painting = Promise.resolve();

  const paint = async (bytes) => {
    const frame = await createImageBitmap(new Blob([bytes], { type: "image/jpeg" }));
    if (canvas.width !== frame.width) Object.assign(canvas, { width: frame.width, height: frame.height });
    ctx.drawImage(frame, 0, 0);
    frame.close();
  };

  const release = () => {
    closed = true;
    ws?.close();
    if (!session || released) return;
    released = true;
    fetch(`${API}/v1/sessions/${session.session_id}`, {
      method: "POST",
      keepalive: true, // still sent while the page unloads
      headers: { Authorization: `Bearer ${session.jwt}`, "Content-Type": "application/json" },
      body: JSON.stringify({ action: "detach", connection_id: session.connection_id, release: true }),
    });
  };

  const ready = (async () => {
    const { jwt } = await (await fetch("/api/windflow-token", { method: "POST" })).json();
    session = await call("/v1/sessions", jwt, { model, transport: "ws-h264" });
    while (session.status === "queued" && !closed) {
      await new Promise((resolve) => setTimeout(resolve, 2000));
      const poll = await call(`/v1/sessions/${session.session_id}`, session.jwt);
      onMessage({ type: "queued", position: poll.queue_position });
      if (poll.status !== "queued") {
        session = await call("/v1/sessions", session.jwt, { model, session_id: session.session_id, transport: "ws-h264" });
      }
    }
    const offer = session.transport.offers.find((o) => o.kind === "ws-h264");
    if (closed || !offer) throw new Error(closed ? "released" : `no stream offer (status ${session.status})`);
    const url = new URL(offer.relay_path, API);
    url.protocol = "wss:";
    ws = new WebSocket(url);
    ws.binaryType = "arraybuffer";
    return new Promise((resolve, reject) => {
      ws.onopen = () => {
        ws.send(JSON.stringify({ type: "authenticate", token: offer.launch_token }));
        ws.send(JSON.stringify({ type: "client_hello", codecs: ["jpeg"] }));
      };
      ws.onmessage = ({ data }) => {
        if (typeof data !== "string") {
          painting = painting.then(() => paint(data)).catch(console.warn);
          return;
        }
        const msg = JSON.parse(data);
        if (msg.type === "worker_ready") resolve(msg);
        if (msg.type === "failed") reject(new Error(msg.message ?? msg.error));
        onMessage(msg);
      };
      ws.onclose = (event) => {
        onMessage({ type: "socket_closed", code: event.code });
        reject(new Error(`socket closed (${event.code})`));
      };
    });
  })();
  ready.catch(release); // a failed start must not leave a session running

  return {
    ready,
    release,
    get session() { return session; },
    get socket() { return ws; },
    command: (type, fields = {}) =>
      call(`/v1/sessions/${session.session_id}/commands`, session.jwt, { command: { type, ...fields } }),
    upload: async (file) => {
      const intent = await call("/v1/uploads", session.jwt, {
        filename: file.name, mime: file.type, size: file.size, session_id: session.session_id,
      });
      const put = await fetch(intent.upload_url, { method: "PUT", headers: intent.headers, body: file });
      if (!put.ok) throw new Error(`upload failed (${put.status})`);
      return call(`/v1/uploads/${intent.file_id}/complete`, session.jwt, {});
    },
  };
}
```

`queued` and `socket_closed` are messages this file generates; every other message comes from the session.

**4. Stage the opening shot and start.**

```js
import { openSession } from "./windflow-session.js";

const film = openSession("longlive-v2", document.querySelector("canvas"), (msg) => console.log(msg.type, msg));
await film.ready; // worker_ready: commands are accepted from here
await film.command("set_shot", { prompt: "A chef chops vegetables in a quiet kitchen at dawn, medium shot" });
await film.command("start");
```

## Wait until the session is ready

Capacity is brought up on demand, so `POST /v1/sessions` can answer `status: "queued"` with a `queue_position` and no transport offers. Keep polling `GET /v1/sessions/{id}`: a queue ticket nobody polls expires with `error: "scheduler_queue_expired"`. When the status leaves `queued`, `POST /v1/sessions` with the same `session_id` returns the WebSocket offer without creating a second session.

On the socket, `authenticate` with the offer's `launch_token` must be the first message; `client_hello` with `codecs: ["jpeg"]` asks for one JPEG per binary message. Commands are accepted after `worker_ready`. Before it they fail with `409 queued` or `409 no_active_session`.

## Direct the film with shots and cuts

LongLive-2.0 generates 29-frame chunks and keeps a rolling history of the current scene. Every shot command lands on a chunk boundary.

| Command | What it does | When to use it |
|---|---|---|
| `set_shot` | Sets the opening shot before `start`, or queues a **soft** transition during the run. The new shot steers inside the rolling context, so the scene stays continuous | The same scene, a new action or camera move |
| `scene_cut` (alias `cut`) | Queues a **hard** cut: the rolling context is erased and re-primed, and the new scene gets a fresh budget | A new place, time or cast |
| `schedule_shot` | A soft shot at a future `at_session_chunk` | A planned beat |
| `schedule_scene_cut` | A hard cut at a future `at_session_chunk` | A planned scene change |
| `start`, `pause`, `resume`, `reset`, `set_seed` | Run control; `reset` aborts and clears the active shot state | |

A soft shot switches live, well under a second. A hard cut takes about two seconds while the engine sets up the new scene. Reserve cuts for real changes of scene.

```js
await film.command("set_shot", { prompt: "The chef slides the vegetables into a hot pan, close-up, steam rising" });
await film.command("scene_cut", { prompt: "The plated dish is served at a candlelit table, slow push-in" });
```

### Schedule a sequence ahead of time

`schedule_shot` and `schedule_scene_cut` take `at_session_chunk`, a global counter that keeps counting across cuts. The per-scene `chunk_index` resets at every scene, while `session_chunk` keeps counting for the whole session. Schedule against `session_chunk`.

```js
const beats = [
  { at_session_chunk: 6, prompt: "A slow push-in on the finished dish" },
  { at_session_chunk: 12, prompt: "The same restaurant from the street at night", cut: true },
];
for (const { cut, ...beat } of beats) {
  await film.command(cut ? "schedule_scene_cut" : "schedule_shot", beat);
}
```

The session acknowledges each command: `shot_set`, `scene_cut`, `shot_scheduled` and `scene_cut_scheduled` confirm acceptance, and `shot_started` arrives when a shot actually begins on a chunk boundary. Update your interface on `shot_started`, not on the acknowledgement.

### Plan around the scene budget

Each scene runs up to 48 chunks. When a scene reaches that ceiling the session sends `scene_budget_reached` and the run emits `generation_complete`. The session itself stays open: send `set_shot` or `scene_cut` to continue with the next scene instead of reconnecting. A `scene_cut` always opens a fresh 48-chunk budget.

For how to word each shot, see the [LongLive 2 prompting guide](https://windflow.online/guides/longlive-2-prompting-guide).

## Track arrived is not first frame visible

| Milestone | Signal | What the viewer should see |
|---|---|---|
| Ready | `worker_ready` | Preparing, controls enabled |
| Generating | `generation_started`, after `start` | Generating, no picture yet |
| Shot began | `shot_started` with `shot_index` 0 | Still the loader |
| Stream arrived | `video_config`, then the first binary message | Still the loader |
| First frame visible | The first `drawImage` returns | The picture |

Start your first-frame timer at `generation_started`. The session's `transport.fallback_policy.first_frame_timeout_ms` (10 seconds by default) is how long the platform's own client waits for media before it treats the stream as failed.

A canvas paints without `play()`, so autoplay rules do not apply. If you move the picture into a `<video>` element, through `canvas.captureStream()` or the session's `livekit` offer, set `muted` and `playsInline` and handle the promise from `play()`.

## Release the session on unmount

Call `film.release()` when the component unmounts and on `pagehide`. It sends `{ action: "detach", connection_id, release: true }` with `keepalive`, which ends the session at the worker and hands the capacity back; the response reports `session_released: true`. Paused time is billed, so release is what stops the meter. If a tab dies without releasing, the worker keeps the session for a reconnect grace (30 seconds by default). For drops, see [recovering a dropped stream](https://windflow.online/guides/recover-a-dropped-realtime-video-stream).

## Troubleshooting by the stage where startup stopped

| Stage | Symptom | Cause and fix |
|---|---|---|
| Token | `401 invalid_api_key` | Send the `wf_live_` key from your server only |
| Create | `403 model_mismatch` or `token_sessions_exhausted` | Token minted for another model, or its `max_sessions` used up |
| Create | `402 insufficient_balance` or `429 rate_limited` | Balance below the shortest useful session, or too many new sessions; honour `Retry-After` |
| Queue | `failed` with `scheduler_queue_expired` | Polling stopped while queued |
| Socket | Closes with `4401` or `4408` | `authenticate` not sent first; or the session already hit its time limit |
| Ready | `start` rejected: "prompt is required" | Send the opening `set_shot` before `start` |
| Running | A scheduled shot fires at the wrong moment | It was scheduled against `chunk_index`, which resets per scene; use `session_chunk` |
| Running | A short pause at a scene change | A hard cut is setting up the new scene; use `set_shot` when the scene continues |
| Running | A scene ends after 48 chunks | `scene_budget_reached`: send the next `set_shot` or `scene_cut` |
| Running | A soft shot changes more than intended | Too much changed at once; change one thing per shot |

Rejected commands return `422` with a `command_…` code over HTTP and a `command_error` message on the socket.

## FAQ

### Can LongLive 2 start from an image?

Not in this release. LongLive-2.0 on Windflow takes text shot prompts only. For image-to-video, use [Helios](https://windflow.online/guides/helios-api) or [LTX-2.5](https://windflow.online/guides/ltx-2-5-api).

### What is the difference between set_shot and scene_cut?

`set_shot` is a soft transition that keeps the rolling scene context, so subjects and setting carry over. `scene_cut` erases that context, re-primes the model with the new prompt and opens a fresh 48-chunk scene budget, taking about two seconds to set up.

### How long can one session run?

Each scene runs up to 48 chunks. The session continues across scenes for as long as you keep sending shots or cuts, within the session time limit the session response states as `session_max_seconds`.