Observability: live view, screencast, and traces

Every computer is fully observable while it runs: a live view in the dashboard, a raw video stream, an event stream of every action, and per-step task traces. This page covers each surface and when to use it.

Live view in the dashboard

Every computer has a live view at:

https://lightcone.ai/c/{computer_id}

Open it in a browser to watch the session in real time, no code required. After the session ends, the same URL serves the replay.

When you run a Task, the started event in the stream carries the computer_id, so you can link to the live view as soon as the run begins:

stream = client.agent.tasks.start_stream(instruction="...", kind="browser")

for event in stream:
    if event.get("type") == "started":
        print(f"Watch live: https://lightcone.ai/c/{event['computer_id']}")

const stream = await client.agent.tasks.startStream({ instruction: "...", kind: "browser" });

for await (const event of stream) {
  if (event.type === "started") {
    console.log(`Watch live: https://lightcone.ai/c/${event.computer_id}`);
  }
}

Screencast stream

GET /computers/{id}/screencast is a Server-Sent Events stream of the session’s video. The event shapes depend on the session kind:

Session kind	SSE events	Payload
Browser	Unnamed (default) events	JSON with a base64-encoded JPEG frame in image_data, plus metadata (timestamp, viewport dimensions)
Desktop	event: h264	JSON with nalu_data, base64-encoded raw H.264 NAL units, suitable for a WebCodecs VideoDecoder
Desktop	event: cursor_update	Cursor sprite bitmap and hotspot; emitted only when the cursor shape changes
Desktop	event: cursor_position	Cursor x/y; emitted once per video frame while the cursor moves

A : heartbeat SSE comment is sent every 30 seconds to keep intermediaries from closing the connection. Action responses are deliberately excluded; those live on /events.

import json

with client.computers.with_streaming_response.retrieve_screencast(computer.id) as response:
    for line in response.iter_lines():
        if line.startswith("data: "):
            frame = json.loads(line[len("data: "):])
            # browser: frame["image_data"] is a base64 JPEG

const response = await client.computers.retrieveScreencast(computer.id!);
// Parse the SSE body: unnamed events carry JPEG frames (browser),
// named "h264" events carry NAL units (desktop).

Desktop H.264 frames are raw NAL units, not a container format. In a web app, feed them to a WebCodecs VideoDecoder; on a server, pipe them to ffmpeg (below). Cursor events are separate so you can composite the cursor sprite over the decoded video yourself.

Recording a session to MP4

Pipe the screencast into ffmpeg to produce a video file. For a desktop session, decode the base64 NAL units and write them to ffmpeg’s stdin:

import base64
import json
import subprocess

from tzafon import Lightcone

client = Lightcone()
computer = client.computers.create(kind="desktop")

ffmpeg = subprocess.Popen(
    ["ffmpeg", "-y", "-i", "pipe:", "-c:v", "copy", "recording.mp4"],
    stdin=subprocess.PIPE,
)

try:
    with client.computers.with_streaming_response.retrieve_screencast(computer.id) as response:
        for line in response.iter_lines():
            if not line.startswith("data: "):
                continue  # skips "event: ..." lines and ": heartbeat" comments
            payload = json.loads(line[len("data: "):])
            if "nalu_data" in payload:
                ffmpeg.stdin.write(base64.b64decode(payload["nalu_data"]))
finally:
    ffmpeg.stdin.close()
    ffmpeg.wait()
    client.computers.delete(computer.id)

For a browser session the frames are JPEGs, so use image2pipe instead and write the decoded image_data bytes:

ffmpeg = subprocess.Popen(
    ["ffmpeg", "-y", "-f", "image2pipe", "-framerate", "10", "-i", "pipe:", "recording.mp4"],
    stdin=subprocess.PIPE,
)
# ... ffmpeg.stdin.write(base64.b64decode(payload["image_data"]))

Browser frames arrive when the page changes, not at a fixed rate, so pick a -framerate close to your observed frame rate; a fixed value like 10 is fine for most automation recordings.

Event stream and WebSocket

GET /computers/{id}/events is an SSE stream of every action on the session (video excluded), which makes it a step-by-step trace of programmatic control. Each data: line is a JSON object in one of three categories:

Category	What it carries
Action request	An action has been queued for execution
Action response	A completed action, with a request_id for correlation; shell commands include exit_code, stdout, stderr
Command output chunk	Streaming output from a long-running /exec command: request_id, base64 data, is_stderr

GET /computers/{id}/ws is the WebSocket variant of the same stream, with one addition: it’s bidirectional, so you can send actions over the same connection instead of making HTTP calls. Use it when you want a single connection for both control and observation.

Task traces

For Tasks, the SSE stream is itself the trace. Each step of the run emits events you can log or replay:

Event	What it carries
screenshot	URL of the screenshot the agent saw at this step
executed	The action the agent took
thinking / status	The agent’s reasoning and progress updates
plan	The agent’s current plan

stream = client.agent.tasks.start_stream(instruction="...", kind="browser")

for event in stream:
    print(event)  # persist these; they are the full run trace

const stream = await client.agent.tasks.startStream({ instruction: "...", kind: "browser" });

for await (const event of stream) {
  console.log(event); // persist these; they are the full run trace
}

Screenshot URLs in task events are presigned and expire after roughly 7 days. If you need traces beyond that, download the images when you receive the events.

Fused act-and-observe

For a tight see-act loop you don’t need a separate screenshot call after each action. Add ?screenshot_after=true to any action endpoint and the response includes a post-action screenshot taken after the UI settles:

result = client.computers.click(
    computer.id,
    x=100,
    y=200,
    extra_query={"screenshot_after": "true", "settle_ms": "500"},
)

const result = await client.computers.click(computer.id!, { x: 100, y: 200 }, {
  query: { screenshot_after: "true", settle_ms: "500" },
});

Query param	Effect
screenshot_after=true	Return a post-action screenshot in the same response
settle_ms=N	Wait up to N ms for the UI to settle before capturing
base64=true	Return the fused frame as base64 in the response body instead of a URL

This halves the round trips in a screenshot-act-screenshot loop and guarantees the frame reflects the action you just took.

See also

• Computers: computer lifecycle and actions
• Tasks: autonomous runs and their event streams
• Limits and behaviors: platform behaviors and constraints
• Troubleshooting: common issues and solutions