Vercel AI SDK

Use Lightcone with the Vercel AI SDK to give your AI agents the ability to browse and interact with web pages.

Install

npm install ai @ai-sdk/openai @tzafon/lightcone

Example

import { generateText, tool } from "ai";
import { openai } from "@ai-sdk/openai";
import Lightcone from "@tzafon/lightcone";

const lightcone = new Lightcone();

const result = await generateText({
  model: openai("gpt-4o"),
  tools: {
    browse: tool({
      description: "Visit a URL and take a screenshot of the page",
      parameters: {
        url: { type: "string", description: "The URL to visit" },
      },
      execute: async ({ url }) => {
        const computer = await lightcone.computers.create({ kind: "browser" });
        const id = computer.id!;

        await lightcone.computers.navigate(id, { url });
        const result = await lightcone.computers.screenshot(id);
        await lightcone.computers.delete(id);

        return { screenshot: result.result?.screenshot_url };
      },
    }),
  },
  prompt: "Visit https://news.ycombinator.com and describe what you see",
});

console.log(result.text);

How it works

The Vercel AI SDK supports tool calling. Lightcone’s browser actions are wrapped as tools that the AI model can invoke during text generation. When the model decides it needs to visit a web page, it calls the browse tool, which:

1. Creates a Lightcone browser session
2. Navigates to the requested URL
3. Takes a screenshot
4. Returns the result to the model
5. Terminates the session

You can extend this pattern with additional tools for clicking, typing, form filling, and data extraction.

Resources

• Vercel AI SDK documentation
• Vercel AI SDK tool calling

See also

• Mastra — another TypeScript agent framework with Lightcone
• LangChain — Python and TypeScript agent tools
• Web scraping — common use case for browser tools