Kernel

Kernel is a Browsers-as-a-Service platform for AI agents. It provides cloud Chrome browsers with stealth mode, residential proxies, and session replay. Connect Northstar to Kernel’s browsers to build computer-use agents that deploy with a single command.

Kernel provides the browser. Lightcone provides the model. You use Kernel’s Computer Controls API to take screenshots and execute actions, and Lightcone’s Responses API to decide what to do next.

Quickstart

1. Install

pip install kernel tzafon

npm install @onkernel/sdk @tzafon/lightcone

2. Set your API key

export TZAFON_API_KEY=sk_your_api_key_here

3. Create from template

kernel create
# Select "tzafon-computer-use" from the template list

This scaffolds a complete CUA agent with:

• A Kernel app entry point
• A CUA sampling loop using Northstar
• Action mapping from Northstar’s output to Kernel’s Computer Controls

4. Deploy

kernel deploy app.py --env-file .env

5. Invoke

kernel invoke my-app cua-task --payload '{"query": "Go to wikipedia.org and search for Ada Lovelace"}'

How it works

The template implements a computer-use loop: take a screenshot → send it to Northstar → execute the action → repeat.

┌──────────────┐                              ┌──────────────┐
│              │  screenshot (base64)          │              │
│    Kernel    │ ──────────────────────────── >│   Northstar  │
│    Browser   │                               │  (Lightcone) │
│              │< ──────────────────────────── │              │
└──────────────┘  action (click, type, drag…)  └──────────────┘
       │
       │  execute via Computer Controls API
       v
   clicks, types, scrolls on a real Chrome browser

Each iteration:

1. Screenshot — Kernel’s capture_screenshot() returns a PNG
2. Decide — Lightcone’s responses.create() takes the screenshot and returns the next action
3. Execute — the template maps Northstar’s action to Kernel’s Computer Controls (click_mouse, type_text, press_key, scroll, etc.)
4. Repeat — send the new screenshot back with previous_response_id

The CUA loop

Here’s the core loop. The template handles this for you, but if you’re building from scratch:

import kernel
from tzafon import Lightcone
import base64

tzafon = Lightcone()

# Create a Kernel browser session
session = kernel.browsers.create(
    stealth_mode=True,
    viewport={"width": 1280, "height": 800},
)

# Take initial screenshot
png_bytes = kernel.browsers.computer.capture_screenshot(session.id)
screenshot_b64 = base64.b64encode(png_bytes).decode()

# First request to Northstar
response = tzafon.responses.create(
    model="tzafon.northstar-cua-fast",
    input=[{
        "role": "user",
        "content": [
            {"type": "input_text", "text": "Go to wikipedia.org and search for Ada Lovelace"},
            {"type": "input_image", "image_url": f"data:image/png;base64,{screenshot_b64}", "detail": "auto"},
        ],
    }],
    tools=[{
        "type": "computer_use",
        "display_width": 1280,
        "display_height": 800,
        "environment": "browser",
    }],
)

# Loop until done
for step in range(50):
    computer_call = next(
        (o for o in (response.output or []) if o.type == "computer_call"), None
    )
    if not computer_call:
        break

    action = computer_call.action
    if action.type in ("terminate", "done", "answer"):
        print(f"Done: {getattr(action, 'result', getattr(action, 'text', ''))}")
        break

    # Execute the action on Kernel's browser
    if action.type == "click":
        kernel.browsers.computer.click_mouse(session.id, action.x, action.y)
    elif action.type == "double_click":
        kernel.browsers.computer.click_mouse(session.id, action.x, action.y, num_clicks=2)
    elif action.type == "type":
        kernel.browsers.computer.type_text(session.id, action.text)
    elif action.type in ("key", "keypress"):
        # Map Northstar's key format to Kernel's format
        # Northstar: ["Control", "c"] → Kernel: ["Ctrl+c"]
        kernel.browsers.computer.press_key(session.id, action.keys)
    elif action.type == "scroll":
        kernel.browsers.computer.scroll(
            session.id, action.x or 640, action.y or 400,
            delta_x=0, delta_y=action.scroll_y or 0,
        )
    elif action.type == "drag":
        kernel.browsers.computer.drag_mouse(
            session.id, path=[[action.x, action.y], [action.end_x, action.end_y]],
        )
    elif action.type == "navigate":
        kernel.browsers.playwright.execute(
            session.id, code=f'page.goto("{action.url}")',
        )
    elif action.type == "wait":
        import time; time.sleep(2)

    # Screenshot and continue
    import time; time.sleep(1)
    png_bytes = kernel.browsers.computer.capture_screenshot(session.id)
    screenshot_b64 = base64.b64encode(png_bytes).decode()

    response = tzafon.responses.create(
        model="tzafon.northstar-cua-fast",
        previous_response_id=response.id,
        input=[{
            "type": "computer_call_output",
            "call_id": computer_call.call_id,
            "output": {"type": "input_image", "image_url": f"data:image/png;base64,{screenshot_b64}", "detail": "auto"},
        }],
        tools=[{
            "type": "computer_use",
            "display_width": 1280,
            "display_height": 800,
            "environment": "browser",
        }],
    )

kernel.browsers.delete(session.id)

import Kernel from "@onkernel/sdk";
import Lightcone from "@tzafon/lightcone";

const kernel = new Kernel();
const tzafon = new Lightcone();

// Create a Kernel browser session
const session = await kernel.browsers.create({
  stealthMode: true,
  viewport: { width: 1280, height: 800 },
});

// Take initial screenshot
const pngBuffer = await kernel.browsers.computer.captureScreenshot(session.id);
const screenshotB64 = pngBuffer.toString("base64");

// First request to Northstar
let response = await tzafon.responses.create({
  model: "tzafon.northstar-cua-fast",
  input: [{
    role: "user",
    content: [
      { type: "input_text", text: "Go to wikipedia.org and search for Ada Lovelace" },
      { type: "input_image", image_url: `data:image/png;base64,${screenshotB64}`, detail: "auto" },
    ],
  }],
  tools: [{
    type: "computer_use",
    display_width: 1280,
    display_height: 800,
    environment: "browser",
  }],
});

// Loop until done
for (let step = 0; step < 50; step++) {
  const computerCall = response.output?.find((o) => o.type === "computer_call");
  if (!computerCall) break;

  const action = computerCall.action!;
  if (["terminate", "done", "answer"].includes(action.type!)) break;

  // Execute the action on Kernel's browser
  switch (action.type) {
    case "click":
      await kernel.browsers.computer.clickMouse(session.id, action.x!, action.y!);
      break;
    case "double_click":
      await kernel.browsers.computer.clickMouse(session.id, action.x!, action.y!, { numClicks: 2 });
      break;
    case "type":
      await kernel.browsers.computer.typeText(session.id, action.text!);
      break;
    case "key":
    case "keypress":
      await kernel.browsers.computer.pressKey(session.id, action.keys!);
      break;
    case "scroll":
      await kernel.browsers.computer.scroll(session.id, action.x ?? 640, action.y ?? 400, {
        deltaX: 0, deltaY: action.scroll_y ?? 0,
      });
      break;
    case "drag":
      await kernel.browsers.computer.dragMouse(session.id, {
        path: [[action.x!, action.y!], [action.end_x!, action.end_y!]],
      });
      break;
    case "navigate":
      await kernel.browsers.playwright.execute(session.id, {
        code: `page.goto("${action.url}")`,
      });
      break;
  }

  // Screenshot and continue
  await new Promise((r) => setTimeout(r, 1000));
  const newPng = await kernel.browsers.computer.captureScreenshot(session.id);
  const newB64 = newPng.toString("base64");

  response = await tzafon.responses.create({
    model: "tzafon.northstar-cua-fast",
    previous_response_id: response.id!,
    input: [{
      type: "computer_call_output",
      call_id: computerCall.call_id!,
      output: { type: "input_image", image_url: `data:image/png;base64,${newB64}`, detail: "auto" },
    }],
    tools: [{
      type: "computer_use",
      display_width: 1280,
      display_height: 800,
      environment: "browser",
    }],
  });
}

await kernel.browsers.delete(session.id);

Action mapping

Northstar’s action format maps to Kernel’s Computer Controls:

Northstar action	Kernel method
click (x, y)	click_mouse(session_id, x, y)
double_click (x, y)	click_mouse(session_id, x, y, num_clicks=2)
right_click / click with button: "right"	click_mouse(session_id, x, y, button="right")
type (text)	type_text(session_id, text)
key / keypress (keys)	press_key(session_id, keys)
scroll (scroll_y, x, y)	scroll(session_id, x, y, delta_y=scroll_y)
drag (x, y, end_x, end_y)	drag_mouse(session_id, path=[[x,y],[end_x,end_y]])
navigate (url)	playwright.execute(session_id, code='page.goto("url")')

Northstar returns right-clicks as click with button: "right", not as a separate right_click action type. Check action.button to distinguish left from right clicks.

Why Kernel + Northstar?

• One-click deploy — kernel deploy puts your agent in production
• Stealth browsers — Kernel’s anti-bot fingerprinting lets Northstar operate on sites that block automation
• Session replay — record every run as an MP4 video for debugging
• Live view — watch Northstar operate in real time via Kernel’s live view URLs
• Residential proxies — rotate IPs to avoid rate limiting
• Scale — spin up hundreds of browser sessions in parallel

Resources

• Kernel documentation
• Kernel CLI
• Tzafon template PR

See also

• Computer-use loop — how the CUA loop works under the hood
• Coordinates — how Northstar’s coordinate system and scaling works
• Responses API — reference for the model API used in the loop
• Playwright — alternative: connect Playwright to Lightcone’s own cloud browsers