# How to recover a realtime video stream after a dropped connection

> Tell a dropped connection from an ended session, retry inside a bounded grace, reattach to the same session and restore playback, with a test checklist.

Source: https://windflow.online/guides/recover-a-dropped-realtime-video-stream · Updated 2026-10-20 · Windflow Labs

When a realtime video stream stops, decide first whether the **connection dropped** or the **session ended**. Retry a dropped connection on the same session inside a bounded grace, without calling anything that ends the session, then restore playback from the worker's state snapshot. An ended session is over: say why, and let the user choose whether to start a new one.

The examples use Windflow's WebSocket transport (`ws-h264`) and the `call` helper from `windflow-session.js` in the [Helios](https://windflow.online/guides/helios-api), [JoyAI-Video-Edit](https://windflow.online/guides/joyai-video-edit-api), [LingBot](https://windflow.online/guides/lingbot-api) and [ABot-World](https://windflow.online/guides/abot-world-api) guides. The same rules apply to WebRTC.

## Dropped connection or ended session?

Read the last messages and the close code before you reconnect.

| What you saw | What it means | What to do |
|---|---|---|
| `session_ended`, then the socket closed | The worker ended the session; `reason` is `time_limit`, `pause_limit`, `source_complete` or `faulted` | Show the reason and offer a new session |
| `generation_ended` with no new `generation_started`, then the close | The run finished and the worker retired the session | Same |
| Close code `4408` | The session reached its time limit | Same |
| Close code `4409` | Another lease now holds the worker | Stop retrying; the session cannot continue there |
| Close code `4401` | The worker refused the launch credentials | Fetch fresh offers once, then stop |
| Any other close, or the browser went offline | A dropped connection | Reconnect to the same session |

When in doubt, ask the control plane. `GET /v1/sessions/{id}` returns `status`, where `ended`, `failed` and `canceled` are final, and `error` says why.

```js
// `lastEnd` is the last session_ended or generation_ended message,
// cleared again when a generation_started arrives.
function classifyClose(event, lastEnd) {
  if (lastEnd || event.code === 4408) return "ended";
  if (event.code === 4409) return "taken";
  if (event.code === 4401) return "refresh";
  return "retry";
}
```

## Retry inside a bounded grace

Two clocks bound the retries.

- **The client grace** is `transport.fallback_policy.reconnect_grace_ms` in the session response, 9 seconds by default. Inside it, reopen the same offer: its launch token is still valid and the worker still holds the session. Back off 100, 200, 400 and 800 ms, then every 1.5 seconds. Also stop after six sockets in a row that close without a single frame, because a worker that accepts and then drops would otherwise reset your timer forever.
- **The worker grace** is how long the worker keeps a detached session before releasing it, 30 seconds by default. Past the client grace, ask for fresh offers on the same session and connection: `POST /v1/sessions` with `session_id` and `connection_id`, using the session `jwt`. Reattaching is not a new session, so it is not rate-limited and does not use up a token's `max_sessions`.

Time the socket spends detached inside the worker grace is not billed.

```js
import { call } from "./windflow-session.js";

const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
const WORKER_GRACE_MS = 30_000; // the worker's default; the session response does not carry it
let inFlight = null;

// Wire ws.onclose (when classifyClose says "retry" or "refresh"), the window
// "online" event and your retry button to this. A second call joins the first.
export function recover(stream) {
  // Sockets in a row that closed before any frame arrived; reset it when a frame paints.
  if (stream.framelessCloses >= 6) return Promise.resolve("lost");
  inFlight ??= reconnect(stream).finally(() => (inFlight = null));
  return inFlight;
}

async function reconnect(stream) {
  const startedAt = performance.now();
  const graceMs = stream.session.transport.fallback_policy?.reconnect_grace_ms ?? 9_000;

  // 1. The same offer and launch token, with a short backoff.
  for (let attempt = 0; performance.now() - startedAt < graceMs; attempt++) {
    await sleep(Math.min(1_500, 100 * 2 ** Math.min(attempt, 4)));
    const outcome = await stream.reopen(); // "ready", or classifyClose() of a socket that closed first
    if (outcome === "ready") return "resumed";
    if (outcome === "ended" || outcome === "taken") return outcome;
    if (outcome === "refresh") break;
  }

  // 2. Fresh offers for the same session and connection, until the worker's grace is spent.
  while (performance.now() - startedAt < WORKER_GRACE_MS) {
    try {
      const fresh = await call("/v1/sessions", stream.session.jwt, {
        model: stream.session.model,
        session_id: stream.session.session_id,
        connection_id: stream.session.connection_id,
        transport: "ws-h264",
      });
      if (["ended", "failed", "canceled"].includes(fresh.status)) return "ended";
      stream.session = fresh; // a new jwt and launch token, the same session_id
      if (fresh.status === "queued") return "queued"; // wait for placement as at startup
      const outcome = await stream.reopen();
      if (outcome === "ready") return "resumed";
      if (outcome !== "retry") return outcome === "refresh" ? "lost" : outcome; // refused twice, ended or taken
    } catch (error) {
      if (error.code === "token_expired") {
        stream.session.jwt = await sessionToken(stream.session.session_id);
        continue;
      }
    }
    await sleep(1_500); // most likely still offline
  }
  return "lost";
}
```

Here `stream` is your own object: `stream.session` holds the latest `POST /v1/sessions` response, `stream.socket` the open socket and `stream.command()` sends commands as in the model guides. `stream.reopen()` repeats the socket setup from startup on the current `ws-h264` offer, `authenticate` with the `launch_token` and then `client_hello`, and resolves `"ready"` on `worker_ready` or with the classification of a socket that closed first. `sessionToken()` asks your server for a token bound to this session, described below.

`resumed` continues with the restore steps below and `queued` waits for placement as at startup. `ended`, `taken` and `lost` mean this session is not coming back: say so, and offer a new session.

## Do not end the session while you retry

- Do not send `{ action: "detach", release: true }` or `DELETE /v1/sessions/{id}?connection_id=…&release=true` before or during a retry. Release ends the session for good.
- Keep the player mounted while it recovers. A component that unmounts on a "disconnected" state and releases in its cleanup ends the very session you are trying to reach.
- Do not send `stop` or `reset` to clean up: `stop` ends the run, and `reset` throws away the session's staged state.
- Do not open a new session as the retry. It is a second billed session, new sessions are rate-limited (`429 rate_limited` with `Retry-After`), and a token minted with `max_sessions` may have none left.
- Keep `session_id`, `connection_id` and the latest session `jwt`. If that JWT has expired (`401 token_expired`), have your server call `POST /v1/tokens` with `{ "session_id": "sess_…" }`. That token is bound to this one session and cannot create another.

## Restore playback after reconnecting

On a reattach inside the worker grace, the worker sends `worker_ready` and then a `state` message whose `data` is a snapshot of the session, and generation carries on with the model state it had. Rebuild the client around it:

1. Put the loading state back and keep it until the next frame is painted, exactly as at startup.
2. Send the current controls, not their history. A key released during the drop never reached the worker, so send one `set_input_state` with the keys held now, often none. The snapshot's `input_seq` is the last one the worker holds; number yours above it.
3. Read the snapshot. If `running` is `false` although the stream was running, nothing is generating, usually because the session came back on a fresh worker: re-send the latest of each setup command, such as `set_mode`, `set_video` or `set_conditioning`, then `start`, and `pause` if the user had paused. Do not replay one-shot commands such as `trigger_event`, `schedule_prompt` or `scene_cut`, and do not replay held keys: a fresh worker starts with neutral movement.
4. If the picture is in a `<video>` element, call `play()` again and handle `NotAllowedError` with a play button.

```js
const setup = new Map(); // command type -> latest fields, e.g. "set_conditioning" -> { prompt, image }
let streaming = false;   // true once frames were showing, false after the user stops
let userPaused = false;

// Call with `data` from the first `state` message after worker_ready on a reopened socket.
async function restore(snapshot) {
  // LingBot and ABot-World: the keys held now, usually none.
  inputSeq = Math.max(inputSeq, snapshot.input_seq ?? 0);
  stream.socket.send(JSON.stringify({ type: "set_input_state", input_seq: ++inputSeq, pressed_keys: [...held] }));
  if (snapshot.running || !streaming) return; // same worker, same state: nothing to re-send
  for (const [type, fields] of setup) await stream.command(type, fields);
  await stream.command("start");
  if (userPaused) await stream.command("pause");
}
```

Give each command a `command_id` and reuse it when you retry that command. The worker remembers recent ids and drops the duplicate; a rejection for `ENGINE_ERROR` or `INTERNAL` is not remembered, so the same id gets a real second attempt.

## If you use WebRTC

On the session's `livekit` offer, the WebRTC client reports that it is reconnecting before it gives up; treat its final disconnect with the same rules. To move a live session from one transport to the other instead, call `POST /v1/sessions/{id}/transport` with `to`, the `from_rev` you last saw in `transport.rev`, and an optional `reason`. A stale `from_rev` answers `409`, and repeating a switch that already happened is safe.

## Test checklist

Run each test on the kind of network your users have, with the browser's network panel open.

1. **Short drop.** Turn the network off for about 3 seconds. Expect the same `session_id`, no new `POST /v1/sessions` call, `worker_ready` and a `state` message on the new socket, and frames that resume.
2. **Long drop.** Turn it off for about 15 seconds, then for about 45. At 15 seconds, expect a `POST /v1/sessions` refresh with the same `session_id` and `connection_id` once the network is back, and a new launch token. At 45 seconds, past the worker's default grace, the session has normally ended: expect a final status and an interface that says so without opening a new session on its own.
3. **Expired token.** Make the refresh fail with `401 token_expired`, by waiting past the session JWT's `expires_at` or by stubbing the response. Expect your server to mint a token with `session_id` and the same session to resume.
4. **Double-clicked retry.** While disconnected, click retry twice quickly. Expect one socket, one refresh, one connection when you `POST /v1/sessions/{id}` with `{ "action": "list" }`, and a retried command applied once.

## FAQ

### Why does the stream reconnect but stay black?

The socket is back but no frame has been painted. Check the `state` snapshot: a paused session sends no new frames, and a session on a fresh worker needs its setup commands and `start` again.

### Is the time spent reconnecting billed?

Time the socket spends detached inside the worker's reconnect grace is not billed. Once the client reattaches, the meter runs as before, and paused time counts.

### Should the app reconnect without asking the user?

Inside the graces, yes: it is the same session. Once the session has ended, ask before starting a new one, because that opens a new billed session.