Skip to content
Lightcone Lightcone
Dashboard
API Reference
Chat

Create Chat Completion

client.chat.createCompletion(ChatCreateCompletionParams { messages, add_generation_prompt, add_special_tokens, 52 more } body, RequestOptionsoptions?): ChatCreateCompletionResponse
POST/chat/completions

Create Chat Completion

ParametersExpand Collapse
body: ChatCreateCompletionParams { messages, add_generation_prompt, add_special_tokens, 52 more }
messages: Array<ChatCompletionDeveloperMessageParam { content, role, name } | ChatCompletionSystemMessageParam { content, role, name } | ChatCompletionUserMessageParam { content, role, name } | 5 more>
One of the following:
ChatCompletionDeveloperMessageParam { content, role, name }

Developer-provided instructions that the model should follow, regardless of messages sent by the user. With o1 models and newer, developer messages replace the previous system messages.

content: string | Array<UnionMember1>
One of the following:
string
Array<UnionMember1>
text: string
type: "text"
role: "developer"
name?: string
ChatCompletionSystemMessageParam { content, role, name }

Developer-provided instructions that the model should follow, regardless of messages sent by the user. With o1 models and newer, use developer messages for this purpose instead.

content: string | Array<UnionMember1>
One of the following:
string
Array<UnionMember1>
text: string
type: "text"
role: "system"
name?: string
ChatCompletionUserMessageParam { content, role, name }

Messages sent by an end user, containing prompts or additional context information.

content: string | Array<ChatCompletionContentPartTextParam { text, type } | ChatCompletionContentPartImageParam { image_url, type } | ChatCompletionContentPartInputAudioParam { input_audio, type } | File { file, type } >
One of the following:
string
Array<ChatCompletionContentPartTextParam { text, type } | ChatCompletionContentPartImageParam { image_url, type } | ChatCompletionContentPartInputAudioParam { input_audio, type } | File { file, type } >
ChatCompletionContentPartTextParam { text, type }

Learn about text inputs.

text: string
type: "text"
ChatCompletionContentPartImageParam { image_url, type }

Learn about image inputs.

image_url: ImageURL { url, detail }
url: string
detail?: "auto" | "low" | "high"
One of the following:
"auto"
"low"
"high"
type: "image_url"
ChatCompletionContentPartInputAudioParam { input_audio, type }

Learn about audio inputs.

input_audio: InputAudio { data, format }
data: string
format: "wav" | "mp3"
One of the following:
"wav"
"mp3"
type: "input_audio"
File { file, type }

Learn about file inputs for text generation.

file: File { file_data, file_id, filename }
file_data?: string
file_id?: string
filename?: string
type: "file"
role: "user"
name?: string
ChatCompletionAssistantMessageParam { role, audio, content, 4 more }

Messages sent by the model in response to user messages.

role: "assistant"
audio?: Audio | null

Data about a previous audio response from the model. Learn more.

id: string
content?: string | Array<ChatCompletionContentPartTextParam { text, type } | ChatCompletionContentPartRefusalParam { refusal, type } > | null
One of the following:
string
Array<ChatCompletionContentPartTextParam { text, type } | ChatCompletionContentPartRefusalParam { refusal, type } >
ChatCompletionContentPartTextParam { text, type }

Learn about text inputs.

text: string
type: "text"
ChatCompletionContentPartRefusalParam { refusal, type }
refusal: string
type: "refusal"
function_call?: FunctionCall | null

Deprecated and replaced by tool_calls.

The name and arguments of a function that should be called, as generated by the model.

arguments: string
name: string
name?: string
refusal?: string | null
tool_calls?: Array<ChatCompletionMessageFunctionToolCallParam { id, function, type } | ChatCompletionMessageCustomToolCallParam { id, custom, type } >
One of the following:
ChatCompletionMessageFunctionToolCallParam { id, function, type }

A call to a function tool created by the model.

id: string
function: Function { arguments, name }

The function that the model called.

arguments: string
name: string
type: "function"
ChatCompletionMessageCustomToolCallParam { id, custom, type }

A call to a custom tool created by the model.

id: string
custom: Custom { input, name }

The custom tool that the model called.

input: string
name: string
type: "custom"
ChatCompletionToolMessageParam { content, role, tool_call_id }
content: string | Array<UnionMember1>
One of the following:
string
Array<UnionMember1>
text: string
type: "text"
role: "tool"
tool_call_id: string
ChatCompletionFunctionMessageParam { content, name, role }
content: string | null
name: string
role: "function"
CustomChatCompletionMessageParam { role, content, name, 4 more }

Enables custom roles in the Chat Completion API.

role: string
content?: string | Array<ChatCompletionContentPartTextParam { text, type } | ChatCompletionContentPartImageParam { image_url, type } | ChatCompletionContentPartInputAudioParam { input_audio, type } | 11 more>
One of the following:
string
Array<ChatCompletionContentPartTextParam { text, type } | ChatCompletionContentPartImageParam { image_url, type } | ChatCompletionContentPartInputAudioParam { input_audio, type } | 11 more>
ChatCompletionContentPartTextParam { text, type }

Learn about text inputs.

text: string
type: "text"
ChatCompletionContentPartImageParam { image_url, type }

Learn about image inputs.

image_url: ImageURL { url, detail }
url: string
detail?: "auto" | "low" | "high"
One of the following:
"auto"
"low"
"high"
type: "image_url"
ChatCompletionContentPartInputAudioParam { input_audio, type }

Learn about audio inputs.

input_audio: InputAudio { data, format }
data: string
format: "wav" | "mp3"
One of the following:
"wav"
"mp3"
type: "input_audio"
File { file, type }

Learn about file inputs for text generation.

file: File { file_data, file_id, filename }
file_data?: string
file_id?: string
filename?: string
type: "file"
ChatCompletionContentPartAudioParam { audio_url, type }
audio_url: AudioURL { url }
url: string
type: "audio_url"
ChatCompletionContentPartVideoParam { type, video_url }
type: "video_url"
video_url: VideoURL { url }
url: string
ChatCompletionContentPartRefusalParam { refusal, type }
refusal: string
type: "refusal"
CustomChatCompletionContentSimpleImageParam { image_url, uuid }

A simpler version of the param that only accepts a plain image_url. This is supported by OpenAI API, although it is not documented.

Example: { "image_url": "https://example.com/image.jpg" }

image_url?: string | null
uuid?: string | null
ChatCompletionContentPartImageEmbedsParam { type, image_embeds, uuid }
type: "image_embeds"
image_embeds?: string | Record<string, string> | null
One of the following:
string
Record<string, string>
uuid?: string | null
ChatCompletionContentPartAudioEmbedsParam { type, audio_embeds, uuid }
type: "audio_embeds"
audio_embeds?: string | Record<string, string> | null
One of the following:
string
Record<string, string>
uuid?: string | null
CustomChatCompletionContentSimpleAudioParam { audio_url }

A simpler version of the param that only accepts a plain audio_url.

Example: { "audio_url": "https://example.com/audio.mp3" }

audio_url?: string | null
CustomChatCompletionContentSimpleVideoParam { uuid, video_url }

A simpler version of the param that only accepts a plain audio_url.

Example: { "video_url": "https://example.com/video.mp4" }

uuid?: string | null
video_url?: string | null
string
CustomThinkCompletionContentParam { thinking, type, closed }

A Think Completion Content Param that accepts a plain text and a boolean.

Example: { "thinking": "I am thinking about the answer", "closed": True, "type": "thinking" }

thinking: string
type: "thinking"
closed?: boolean
name?: string
reasoning?: string | null
tool_call_id?: string | null
tool_calls?: Array<ToolCall> | null
id: string
function: Function { arguments, name }

The function that the model called.

arguments: string
name: string
type: "function"
tools?: Array<Tool> | null
function: Function { name, description, parameters, strict }
name: string
description?: string
parameters?: Record<string, unknown>
strict?: boolean | null
type: "function"
Message { author, channel, content, 2 more }
author: Author { role, name }
role: "user" | "assistant" | "system" | 2 more

The role of a message author (mirrors chat::Role).

One of the following:
"user"
"assistant"
"system"
"developer"
"tool"
name?: string | null
channel?: string | null
content?: Array<unknown>
content_type?: string | null
recipient?: string | null
add_generation_prompt?: boolean

If true, the generation prompt will be added to the chat template. This is a parameter used by chat template in tokenizer config of the model.

add_special_tokens?: boolean

If true, special tokens (e.g. BOS) will be added to the prompt on top of what is added by the chat template. For most models, the chat template takes care of adding the special tokens so this should be set to false (as is the default).

allowed_token_ids?: Array<number> | null
bad_words?: Array<string>
cache_salt?: string | null

If specified, the prefix cache will be salted with the provided string to prevent an attacker to guess prompts in multi-user environments. The salt should be random, protected from access by 3rd parties, and long enough to be unpredictable (e.g., 43 characters base64-encoded, corresponding to 256 bit).

chat_template?: string | null

A Jinja template to use for this conversion. As of transformers v4.44, default chat template is no longer allowed, so you must provide a chat template if the tokenizer does not define one.

chat_template_kwargs?: Record<string, unknown> | null

Additional keyword args to pass to the template renderer. Will be accessible by the chat template.

continue_final_message?: boolean

If this is set, the chat will be formatted so that the final message in the chat is open-ended, without any EOS tokens. The model will continue this message rather than starting a new one. This allows you to "prefill" part of the model's response for it. Cannot be used at the same time as add_generation_prompt.

documents?: Array<Record<string, string>> | null

A list of dicts representing documents that will be accessible to the model if it is performing RAG (retrieval-augmented generation). If the template does not support RAG, this argument will have no effect. We recommend that each document should be a dict containing "title" and "text" keys.

echo?: boolean

If true, the new message will be prepended with the last message if they belong to the same role.

frequency_penalty?: number | null
ignore_eos?: boolean
include_reasoning?: boolean
include_stop_str_in_output?: boolean
kv_transfer_params?: Record<string, unknown> | null

KVTransfer parameters used for disaggregated serving.

length_penalty?: number
logit_bias?: Record<string, number> | null
logits_processors?: Array<string | LogitsProcessorConstructor { qualname, args, kwargs } > | null

A list of either qualified names of logits processors, or constructor objects, to apply when sampling. A constructor is a JSON object with a required 'qualname' field specifying the qualified name of the processor class/factory, and optional 'args' and 'kwargs' fields containing positional and keyword arguments. For example: {'qualname': 'my_module.MyLogitsProcessor', 'args': [1, 2], 'kwargs': {'param': 'value'}}.

One of the following:
string
LogitsProcessorConstructor { qualname, args, kwargs }
qualname: string
args?: Array<unknown> | null
kwargs?: Record<string, unknown> | null
logprobs?: boolean | null
max_completion_tokens?: number | null
Deprecatedmax_tokens?: number | null
min_p?: number | null
min_tokens?: number
mm_processor_kwargs?: Record<string, unknown> | null

Additional kwargs to pass to the HF processor.

model?: string | null
n?: number | null
parallel_tool_calls?: boolean | null
presence_penalty?: number | null
priority?: number

The priority of the request (lower means earlier handling; default: 0). Any priority other than 0 will raise an error if the served model does not use priority scheduling.

prompt_logprobs?: number | null
reasoning_effort?: "low" | "medium" | "high" | null
One of the following:
"low"
"medium"
"high"
repetition_penalty?: number | null
request_id?: string

The request_id related to this request. If the caller does not set it, a random_uuid will be generated. This id is used through out the inference process and return in response.

response_format?: ResponseFormat { type, json_schema } | StructuralTagResponseFormat { format, type } | LegacyStructuralTagResponseFormat { structures, triggers, type } | null
One of the following:
ResponseFormat { type, json_schema }
type: "text" | "json_object" | "json_schema"
One of the following:
"text"
"json_object"
"json_schema"
json_schema?: JsonSchema | null
name: string
description?: string | null
schema?: Record<string, unknown> | null
strict?: boolean | null
StructuralTagResponseFormat { format, type }
format: unknown
type: "structural_tag"
LegacyStructuralTagResponseFormat { structures, triggers, type }
structures: Array<Structure>
begin: string
end: string
schema?: Record<string, unknown> | null
triggers: Array<string>
type: "structural_tag"
return_token_ids?: boolean | null

If specified, the result will include token IDs alongside the generated text. In streaming mode, prompt_token_ids is included only in the first chunk, and token_ids contains the delta tokens for each chunk. This is useful for debugging or when you need to map generated text back to input tokens.

return_tokens_as_token_ids?: boolean | null

If specified with 'logprobs', tokens are represented as strings of the form 'token_id:{token_id}' so that tokens that are not JSON-encodable can be identified.

seed?: number | null
maximum9223372036854776000
minimum-9223372036854776000
skip_special_tokens?: boolean
spaces_between_special_tokens?: boolean
stop?: string | Array<string> | null
One of the following:
string
Array<string>
stop_token_ids?: Array<number> | null
stream?: boolean | null
stream_options?: StreamOptions | null
continuous_usage_stats?: boolean | null
include_usage?: boolean | null
structured_outputs?: StructuredOutputs | null

Additional kwargs for structured outputs

_backend?: string | null
_backend_was_auto?: boolean
choice?: Array<string> | null
disable_additional_properties?: boolean
disable_any_whitespace?: boolean
disable_fallback?: boolean
grammar?: string | null
json?: string | Record<string, unknown> | null
One of the following:
string
Record<string, unknown>
json_object?: boolean | null
regex?: string | null
structural_tag?: string | null
whitespace_pattern?: string | null
temperature?: number | null
tool_choice?: "none" | "auto" | "required" | ChatCompletionNamedToolChoiceParam { function, type } | null
One of the following:
"none" | "auto" | "required"
"none"
"auto"
"required"
ChatCompletionNamedToolChoiceParam { function, type }
function: Function { name }
name: string
type?: "function"
tools?: Array<Tool> | null
function: Function { name, description, parameters }
name: string
description?: string | null
parameters?: Record<string, unknown> | null
type?: "function"
top_k?: number | null
top_logprobs?: number | null
top_p?: number | null
truncate_prompt_tokens?: number | null
maximum9223372036854776000
minimum-1
user?: string | null
vllm_xargs?: Record<string, string | number | Array<string | number>> | null

Additional request parameters with (list of) string or numeric values, used by custom extensions.

One of the following:
string
number
Array<string | number>
string
number
ReturnsExpand Collapse
ChatCreateCompletionResponse = unknown

Create Chat Completion

import Lightcone from '@tzafon/lightcone';

const client = new Lightcone({
  apiKey: process.env['TZAFON_API_KEY'], // This is the default and can be omitted
});

const response = await client.chat.createCompletion({
  messages: [{ content: 'string', role: 'developer' }],
});

console.log(response);
{}
Returns Examples
{}