@bastani/atomic-ai is the LLM library Atomic itself runs on. It ships in every Atomic release at Atomic’s version, and you can install it on its own:
Supported Providers
- OpenAI (API key or Sign in with ChatGPT)
- Ant Ling
- Azure OpenAI (Responses)
- OpenAI Codex (ChatGPT subscription, requires OAuth, see below)
- TypeSafe (System One classifier API)
- DeepSeek
- NVIDIA NIM
- Anthropic
- Vertex AI (Gemini via Vertex AI)
- Mistral
- Groq
- Cerebras
- Cloudflare AI Gateway
- Cloudflare Workers AI
- xAI
- OpenRouter
- Vercel AI Gateway
- ZAI Coding Plan (Global) (with separate China provider)
- MiniMax (with separate China provider)
- Together AI
- Baseten
- Hugging Face
- Moonshot AI (with separate China provider)
- GitHub Copilot (requires OAuth, see below)
- Amazon Bedrock
- OpenCode Zen
- OpenCode Go
- Fireworks (uses OpenAI- and Anthropic-compatible APIs)
- Kimi For Coding (Moonshot AI subscription endpoint, uses OpenAI Chat Completions)
- Meta (Model API, uses OpenAI Responses-compatible API)
- Qwen Token Plan (separate Team and Individual providers, each with a China provider)
- Xiaomi MiMo (defaults to API billing endpoint, with separate Token Plan providers for
cn/ams/sgpregions) - Any OpenAI-compatible API: Ollama, vLLM, LM Studio, etc.
Installation
@bastani/atomic-ai: Type, Static, and TSchema.
Quick Start
You build aModels collection of providers and stream through it. The quickest start registers every built-in provider; apps that care about bundle size register individual providers instead (see Provider Factories and Bundling and Tree Shaking).
usage.cost.totalFromProvider is true when usage.cost.total is the billed cost reported by the provider and false when it is estimated from the model’s catalog rates. A missing value, as in sessions recorded before this field existed, also means false.
Snippets in the rest of this page assume a models collection set up like this (with the relevant providers registered).
usage.cost.total is the best available total cost. For OpenRouter chat responses, it uses the cost reported by OpenRouter, including both the OpenRouter charge and upstream inference cost for BYOK requests. When provider accounting is unavailable or invalid, it falls back to the model’s catalog rates. The input, output, cacheRead, and cacheWrite cost components remain catalog estimates and may not sum to a provider-reported total.
Providers and Models
A provider is the runtime unit: it owns its model catalog, its auth (API key resolution, OAuth flows), and its stream behavior. AModels collection holds providers and routes every request to the provider that owns the model.
Providers internally share API implementations (the wire protocols): Anthropic models use anthropic-messages, OpenAI uses openai-responses, while xAI, Groq, Cerebras, OpenRouter, and most others share openai-completions. Mixed-API providers (GitHub Copilot, OpenCode Zen) dispatch per model.
Provider Factories
For apps that only need specific providers, there is one factory per built-in provider, each a subpath import that pulls only that provider’s catalog:@anthropic-ai/sdk, openai, @google/genai, etc.) stay in lazy chunks loaded on the first request to a model of that API.
For a collection without TypeBox, built-in catalogs or provider SDKs, import createModels and createProvider from @bastani/atomic-ai/models. Import the provider factories you need separately.
All Built-in Providers
For apps that want everything (as in Quick Start):builtinModels() accepts the same options as createModels() (credentials, authContext); builtinProviders() returns the provider array if you want to register them on your own collection.
Querying Models
Reads are synchronous and return the last-known lists:getModels()/getModel()/getAvailable() return chat models (Model<Api>) usable with stream(). The *OfType reads return one model type, and getAllModels()/getAllAvailable() return every type as AnyModel:
type decides which operation accepts it: chat models stream, type: "image" models generate images, and type: "classifier" models classify structured state. type is optional on chat models, so a model without type is a chat model. Do not compare type directly; narrow mixed lists with isModelType() or read the effective type with getModelType():
getModels() returns chat models and the optional getAllModels() returns every type; providers with only chat models can omit it.
Dynamically listed chat models are typed Model<Api>. Narrow with the hasApi() guard when you need API-specific option typing:
Static Catalog Reads
For tooling that wants the generated built-in catalog with full literal typing (provider and model IDs auto-complete), independent of any collection:Dynamic Providers
Providers may have dynamic model lists (a llama.cpp server, a live OpenRouter listing). Reads stay sync; fetching is an explicit async verb:refresh(). See createProvider() for building a dynamic provider.
Auth
Every provider owns its auth: how API keys resolve (stored credentials, environment variables, ambient sources like AWS profiles or gcloud ADC) and, where supported, OAuth login/refresh flows.How Auth Resolves
When you callmodels.stream(), the collection resolves auth through the owning provider and merges it into the request. Explicit per-request values always win:
model.headers:
apiKey, headers, or baseUrl. getAuth() resolves undefined for unconfigured providers and rejects with ModelsError when something is actually broken ("oauth": token refresh failed, credential preserved for re-login; "auth": key resolution or credential store failure). Request paths surface the same failures as stream errors.
A provider can reuse other providers’ stored API keys instead of offering its own login. List their IDs in ApiKeyAuth.borrowCredentialsFrom, or build the auth with sharedApiKeyAuth(name, envVars, providerIds). The provider then has no stored credential of its own: the first listed provider with a stored API key supplies it, in the order given. Stored OAuth credentials are skipped, and the environment variables apply when none qualifies. openai-decisions and openai-images use this with ['openai-api', 'openai'].
getAuth(), checkAuth(), getAvailable(), login, and logout accept optional caller cancellation through their existing options or interaction objects and remain unbounded when no signal is supplied. Provider login, ApiKeyAuth.check, ApiKeyAuth.resolve, and OAuthAuth.refresh implementations always receive a concrete signal and must honor it for blocking work.
Transforming Request Headers
Models.stream(), complete(), streamSimple(), and completeSimple() accept a Models-only transformHeaders option. It runs once after provider auth, model.headers, and explicit options.headers have been merged, but before provider dispatch:
null for a header suppresses lower-level defaults that support deletion.
transformHeaders belongs to Models, not Provider. A Models implementation must consume it and remove it before calling Provider.stream*(). Provider implementations continue receiving ordinary ApiStreamOptions or SimpleStreamOptions and never handle the transform themselves. Use this option instead of calling getAuth(model) before stream*(), which would resolve request auth twice.
Client Identity
identity controls how requests identify the calling app to providers. Set it on createModels() as a default, or per request; a request’s fields override the Models default field by field:
-
userAgent:"detailed"(default) sendsatomic/<version> (<platform>; <runtime>; <arch>),"short"sendsatomic, and{ custom: "..." }sends the given value. Set{ custom: "my-app/1.0" }to name your own app. On Bedrock the value is appended to the AWS SDK user agent. -
version: the version in the detailedUser-Agent. Default: the@bastani/atomic-aiversion. -
attribution: provider attribution headers. Absent (default) ornullsends none;nullalso turns off attribution from a merged identity.{ name, url, categories? }maps to each provider’s headers:- OpenRouter:
HTTP-Referer: <url>(the app’s identity in OpenRouter’s rankings, grouped by origin),X-OpenRouter-Title: <name>, andX-OpenRouter-Categories(comma-separated). - NVIDIA NIM:
X-BILLING-INVOKE-ORIGIN: <name>with its first letter upper-cased. - OpenCode:
x-opencode-client: <name>.
ATOMIC_ATTRIBUTIONis Atomic’s own attribution,{ name: "Atomic", url: "https://github.com/bastani-inc/atomic", categories: ["cli-agent"] };PI_ATTRIBUTIONis a deprecated alias of it. Pass your own app’s attribution instead. - OpenRouter:
null request header removes one. Providers that send a fixed client identity, such as GitHub Copilot and Anthropic OAuth, keep it. withClientIdentity(models, identity) returns a wrapper of a Models (or MutableModels) whose requests use identity over the one the Models was created with, for libraries that receive a Models from their host. identity is plain JSON, so it can be persisted with other request options.
Credential Store
Stored credentials (API keys entered interactively, OAuth tokens) live in aCredentialStore — one type-tagged credential per provider. @bastani/atomic-ai ships an in-memory default; apps inject persistent storage:
read(providerId), list() for non-secret { providerId, type } metadata, modify(providerId, fn) (the only write path — a serialized read-modify-write), and delete(providerId). Each operation accepts optional cancellation options. Enumeration must not resolve secrets or execute configured key commands. OAuth token refresh runs inside modify, so concurrent requests and processes cannot double-refresh a rotated token. A stored credential owns its provider: environment variables are only consulted when nothing is stored, and a failed refresh never silently falls back to an env key.
API-key credentials use the same discriminator as Atomic’s auth.json and can carry provider-scoped env/config values:
Environment Variables
Built-in providers resolve these env vars (Node.js; in browsers passapiKey explicitly):
Qwen Token Plan has a Team and an Individual (Personal) provider per region; each exposes only the
models documented for its subscription. Set the key variable of your plan.
Stored credentials remain provider-scoped, so save the key under the provider ID you register.
Amazon Bedrock resolves ambient AWS credentials (
AWS_PROFILE, access key pairs, AWS_BEARER_TOKEN_BEDROCK, ECS task roles, web identity tokens); its provider-owned login flow supports bearer tokens, AWS profiles, and the existing credential chain. Vertex AI resolves either an explicit key or gcloud Application Default Credentials plus project/location, with a provider-owned login flow for API keys, ADC, and service-account files.
Tools
Tools enable LLMs to interact with external systems. This library uses TypeBox schemas for type-safe tool definitions with automatic validation using TypeBox’s built-in validator and value conversion utilities. TypeBox schemas can be serialized and deserialized as plain JSON, making them ideal for distributed systems.Defining Tools
Constrained Sampling for Tools
Tools can opt in to provider-side constrained sampling. For JSON-schema tools,strict: 'prefer' uses provider-side strict schema enforcement when supported and otherwise falls back to normal tool calling. strict: 'require' fails the request when the active provider/model cannot honor it. Set constrainedSampling: false to explicitly opt out; it behaves the same as omitting the field.
VALIDATED function-calling mode (or ANY when explicitly requested); earlier Gemini versions fall back for strict: 'prefer' and reject strict: 'require' because they do not enforce required parameters. Bedrock strict-tool capability is generated from model structured-output metadata; custom Bedrock models can override compat.supportsStrictMode. OpenAI Responses and Chat Completions can also emit grammar-constrained custom tools with OpenAI Lark or regex grammar variants. If multiple OpenAI variants are supplied, Lark is preferred over regex. Grammar constraints are enforced when the active model supports grammar tools; otherwise the tool falls back to normal function/JSON-schema handling. Grammar tool capability is model metadata: the generated catalog sets compat.supportsOpenAIGrammarTools for GPT-5+ models on endpoints that pass OpenAI custom tools through (OpenAI, OpenAI Codex, Azure OpenAI Responses, GitHub Copilot, opencode, and Cloudflare AI Gateway). OpenAI rejects type: "custom" tools for pre-GPT-5 models, and gateways that normalize tool schemas (e.g. OpenRouter) mangle them, so the flag stays off elsewhere. Custom model definitions can opt in via compat. Grammar-capable models reject grammar configurations without a non-empty supported variant. Native grammar tools must have an object parameter schema with exactly one required string property:
Handling Tool Calls
Tool results use content blocks and can include both text and images:Fireworks deferred tools
Fireworks models using the Anthropic Messages API support native deferred tool loading. Keep your discovery tool incontext.tools. After the discovery tool result, append a chronological system message with toolsAdded containing the newly discovered tool definitions (toolsRemoved when tools drop out). The adapter sends deferred schemas and tool_reference blocks while preserving ordinary result text and unsigned thinking during replay.
Name the discovery tool ToolSearch or tool_search for Fireworks prompt-prefix deferral. Other discovery names serialize correctly but do not get that prefix optimization. Set model.compat.supportsToolReferences to false to send ordinary tool schemas instead. This feature does not apply to Fireworks Chat Completions models.
Streaming Tool Calls with Partial JSON
During streaming, tool call arguments are progressively parsed as they arrive. This enables real-time UI updates before the complete arguments are available:- During
toolcall_deltaevents,argumentscontains the best-effort parse of partial JSON - Fields may be missing or incomplete - always check for existence before use
- String values may be truncated mid-word
- Arrays may be incomplete
- Nested objects may be partially populated
- At minimum,
argumentswill be an empty object{}, neverundefined - The Google provider does not support function call streaming. Instead, you will receive a single
toolcall_deltaevent with the full arguments.
Validating Tool Arguments
When implementing your own tool execution loop, usevalidateToolCall to validate arguments before passing them to your tools:
Complete Event Reference
Successful generation followsstart → updates* → done. A failure after generation starts follows start → updates* → error. Request setup may fail before generation starts, in which case the stream contains only error; done and update events are invalid before start. Direct API streamSimple() calls throw synchronously when request auth is missing.
Tool-call arguments at toolcall_start are provider-specific; toolcall_delta carries subsequent JSON updates.
All streaming events emitted during assistant message generation:
Streaming events for different content blocks are not guaranteed to be contiguous. Providers may emit deltas for text, thinking, and tool calls in the same upstream chunk, and the library may surface corresponding events interleaved, for example
text_start, text_delta, toolcall_start, text_delta, toolcall_delta. Consumers must use contentIndex to associate each delta/end event with its block and must not assume that a block’s *_start/*_delta/*_end sequence is uninterrupted by events for other blocks.
Image Input
Models with vision capabilities can process images. You can check if a model supports images via theinput property. If you pass images to a non-vision model, they are silently ignored.
Image Generation
Image models live in the sameModels collection and on the same Provider as chat models, so one credential per provider covers both. They are typed ImageModel with type: "image" and are used through generateImages(), a one-shot API that waits for the provider response and returns the final AssistantImages result. Do not use the chat/stream APIs for them; stream() rejects image models.
Basic Image Generation
generateImages() accepts only ImageModel values. If an upstream model supports both chat and image generation, the catalog contains separate entries with the same provider and ID: getModel() returns its chat operation and getModelOfType('image', ...) returns its image operation. Failures never reject; they return an AssistantImages with stopReason: "error", including unknown providers, unconfigured auth, and providers without an image implementation.
A provider declares image support with the images option of createProvider(): a map from model.api to an implementation with generateImages(). Image models go into the same models list as chat models. api becomes optional when images is present, so an image-only provider is just a provider without chat models:
getImageModel() / getImageModels() / getImageProviders() / generateImages()) remains available on the compat entrypoint. Its generateImages() dispatches on model.api and takes apiKey explicitly:
registerImagesApiProvider({ api, generateImages }) from the compat entrypoint is the extension point for image backends that are not built in. It registers an implementation for an image API ID, and the global generateImages() then dispatches models with that api to it:
OpenAI Images API
Theopenai-images provider serves OpenAI’s GPT Image models (gpt-image-2.6-sunburst, gpt-image-2.6-flare, gpt-image-2.5-sunburst, gpt-image-2.5-flare, gpt-image-2, gpt-image-1.5, gpt-image-1, gpt-image-1-mini) through the Images API. It borrows the API key stored for openai-api, then the one stored for openai, then OPENAI_API_KEY; a ChatGPT OAuth credential on openai is skipped, so with only that credential the models are unavailable and generateImages() returns Provider is not configured: openai-images. Text blocks become the prompt. Without image blocks the request goes to /images/generations; with them it goes to /images/edits, uploading each image block. OpenAIImagesOptions adds the request fields below, each sent only when set:
ImageContent block whose mimeType follows the response’s output_format: image/png, image/jpeg, image/webp, image/gif or image/svg+xml. An SVG block holds UTF-8 XML in data as base64; save a GIF with .gif and an SVG with .svg.
GIF and SVG output are generation-only: they cannot be combined with image blocks, SVG needs
n of 1, and neither supports streaming, partial images or outputCompression. outputFormat: 'gif' costs more tokens as frameCount grows; GIF size is the size of each frame, and the frames must fit the model’s size limits, so a small size with few frames can be refused (512x512 with 3 frames worked).
Reference images, markup and mask. With image blocks the first image without a role is the base image to edit and the others are references. Set role: 'markup' on a transparent PNG with annotations (circles, sketches) to send it as the markup field, which guides the edit of the first image and must have its dimensions; set role: 'mask' on an image with an alpha channel to send it as the mask field for the first image. A markup or mask image is not counted as a reference, a request takes at most one of each, and they cannot be combined. Markup-guided edits work on GPT Image 2.5 and 2.6.
ImageGenerationConfigError naming the field, the reason and what to change: a value out of range, frameCount or frameDurationMs without GIF output, GIF or SVG output on an edit, SVG with n above 1, outputCompression with anything but jpeg or webp, svg without SVG output, markup together with a mask, a role image without a base image, two images with the same role, and complexity, GIF, SVG or markup on a model that does not declare it. Models declare the optional features in ImageModel.capabilities; a feature that is not declared is unsupported, so a custom openai-images model needs capabilities to use them. The checks run before an onPayload hook sees the request and again on the request that is sent, whether the hook returns a replacement or changes it in place. reason is one of:
Errors.
generateImages() still resolves with stopReason: 'error' and errorMessage, and now also error, a JSON-safe ImageGenerationErrorData. Rebuild the typed error with imageGenerationErrorFromData(result.error) and branch on its class or guard, code or retryable, never on the message. Cancellation resolves with stopReason: 'aborted' and no error. The typed error holds only the HTTP status and the provider’s type, code and request ID, in detail and in its message; it never holds the provider’s message, which can quote the prompt. errorMessage keeps the provider’s text.
All extend
ImageGenerationError, and isImageGenerationError() matches any of them. Guards check the value itself: an instance, or an object from another process whose name is the class name and whose code is that class’s code. To search a cause chain (at most eight links), use findImageGenerationError(error, guard), for example findImageGenerationError(error, isImageGenerationQuotaError); isRetryableImageGenerationError() answers whether the nearest one in the chain is retryable. result.error keeps up to eight cause-chain links: family causes retain their data, while foreign causes retain a safe name and known errno code, if any, with their message replaced by [foreign error message omitted]. Rehydration preserves these links and codes. With maxRetries, rate-limit and server errors are retried; quota and user errors never are. Usage from an SVG request covers the image model only; OpenAI reports the tokens of the SVG-authoring model separately (svg_usage), and they are not added to usage or its cost.
Notes and Limitations
- Image models and chat models share
ModelsandProvider; list them withgetModelsOfType('image')and run them withgenerateImages(), never the chat/stream APIs. - Image-generation models do not participate in tool calling.
- Outputs are returned in
AssistantImages.outputand can include both base64-encodedImageContentblocks andTextContentblocks. - Some models return only images, others return images plus text. Check
model.output. - Some models accept image input, others are text-to-image only. Check
model.input. - Like the streaming APIs, image generation supports options such as
apiKey,signal,headers,onPayload, andonResponse, and results may includestopReason,responseId, andusage. - If you want a model to analyze images in a conversation or call tools, use the regular chat APIs with a model that supports image input.
- At the moment, image generation is available through only one provider, OpenRouter.
Classification
Classifier models consume structured JSON state and answer one or more typed questions. They do not use chat or image-generation APIs. Built-in providers include TypeSafe’s Jev and Microsoft-Decision-1:bool questions and { type: "bool", probability } answers. The TypeSafe adapter translates those to and from its noul wire representation. Like image generation, classify() resolves to a result with stopReason: "error" instead of rejecting for provider, authentication, or response errors.
System One results include usage when the service reports input or output token counts. Costs use the model’s catalog pricing. A billed response can retain usage even when its answers are malformed; absent usage means unknown, not zero.
ClassifierOptions.temperature divides the answer logits by the given value before they are normalized; values above 1 soften the distribution. APIs that cannot apply it, such as System One, ignore it.
System One images
ClassifierContext.images holds image blocks ({ type: "image", data, mimeType }) judged together with the state. Only models whose input includes "image" accept them; classify() returns an error result for other models. System One services take images in three forms. The model’s compat.imageInput chooses the form ("assets", "images", or "state-image-urls"), and compat.sendImageOptInHeader controls TypeSafe’s opt-in header:
- TypeSafe (
typesafe, and Jev on Workers AI) takes PNG and JPEG images asassets, each under a fresh UUID reference that is also listed in order understate.images, so a state with its ownimagesfield cannot send images. Images are experimental on TypeSafe’s own API and need theTypeSafe-Danger-Zone-Images: trueheader, which the API sends with requests that carry images. - Clef on Workers AI takes up to four PNG, JPEG, or WebP images as an
imageslist of data URLs, next to the unchanged state. - OpenRouter takes images as
{ type: "image_url", image_url: { url } }parts before the state:[part, ..., state]. It answers requests withassetsbut drops the images, without an error.
compat explicitly, so they keep their form when baseUrl points at a proxy. For models without it, the APIs detect the form: OpenRouter by provider openrouter or a baseUrl containing openrouter.ai, Clef by a model ID starting with @cf/cloudflare/clef, and TypeSafe’s form with the opt-in header otherwise. A custom provider that reaches OpenRouter through a proxy matches neither check and must set compat.imageInput: "state-image-urls" and compat.sendImageOptInHeader: false; otherwise OpenRouter drops its images.
Vercel AI Gateway and OpenCode list their Jev models with text input only.
Chat models on llama.cpp
Thellama-cpp-classify API turns a chat model served by llama.cpp’s llama-server into a classifier. Each question becomes one chat prompt: the state, every question of the request, the state again, and the question with its answers under single-token labels (letters for a choice, Yes/No for a bool, digits for a score). The prompt up to the final question is shared by all questions of a request, so the server’s prompt cache evaluates the state once per request. The server returns the log-probabilities of the next token, and the answer is the softmax over the label tokens. Choices support up to 62 options and scores up to 10 levels. The model’s baseUrl is the server URL; a trailing /v1 is ignored. In router mode, the model ID selects the model.
temperature above 1 to soften them.
Custom providers register classifier models and implementations by API ID:
Thinking/Reasoning
Many models support thinking/reasoning capabilities where they can show their internal thought process. You can check if a model supports reasoning via thereasoning property. If you pass reasoning options to a non-reasoning model, they are silently ignored.
Unified Interface (streamSimple/completeSimple)
xhigh and max are model-specific, opt-in levels. Use getSupportedThinkingLevels(model) to determine whether a concrete model exposes either level; models such as GPT-5.6 can expose both.
Provider-Specific Options (stream/complete)
models.stream()/complete() accept the owning API’s full option set. Use hasApi() to narrow a dynamically looked-up model to its API for full option typing:
Streaming Thinking Content
When streaming, thinking content is delivered through specific events:Stop Reasons
EveryAssistantMessage includes a stopReason field that indicates how the generation ended:
"pending"- Only present in partial messages when we do not know what the stop reason will be"stop"- This is the final message the model will produce this turn"length"- Output hit the maximum token limit"toolUse"- Model is calling tools and expects tool results"error"- An error occurred during generation"aborted"- Request was cancelled via abort signal
AssistantMessage may also include responseId, a provider-specific upstream response or message identifier when the underlying API exposes one. Do not assume it is always present across providers.
Error Handling
Request failures after a stream is returned never throw: when a request ends with an error (including aborts and tool call validation errors), the streaming API emits an error event and the final message carries the details. Direct APIstreamSimple() calls throw synchronously when their required auth is absent:
stopReason: "error". Direct API streamSimple() calls instead throw synchronously when their required auth is absent.
Aborting Requests
The abort signal allows you to cancel in-progress requests. Aborted requests havestopReason === 'aborted':
Continuing After Abort
Aborted messages can be added to the conversation context and continued in subsequent requests:Debugging Provider Payloads
Use theonPayload callback to inspect the request payload sent to the provider. This is useful for debugging request formatting issues or provider validation errors.
stream, complete, streamSimple, and completeSimple.
Observing Provider Stream Events
UseonProviderStreamEvent to inspect provider-specific fields that the library does not include in AssistantMessage. The callback receives the parsed event available to the adapter before the library normalizes it. Treat the event as read-only because mutations can affect normalization. This is not guaranteed to be the original HTTP bytes or SSE frame.
Custom Providers
createProvider()
createProvider() builds a provider from parts: identity, auth, a model list, and an API implementation (api for chat models, images for image generation, classifiers for classification; at least one is required, see Image Generation). Use it for local inference servers, proxies, or any OpenAI/Anthropic-compatible endpoint:
envApiKeyAuth(displayName, envVars) gives the standard behavior (stored credential wins, then the first set env var):
model.api; each model dispatches to its API’s implementation:
ProviderStreams you pass as api so every request goes through the transformation before dispatch. The Cloudflare providers do this to materialize account/gateway endpoint placeholders from the resolved provider env:
fetchModels, which can return models of every type. Models.refresh() refreshes every configured dynamic provider, passing its effective API-key or refreshed OAuth credential. A ModelsStore persists dynamic catalogs; both stores default to in-memory implementations. Its read, write, and delete operations accept optional cancellation, and Models binds those waits to the provider refresh signal.
Models.refresh() is unbounded when its optional signal is omitted. Providers always receive a concrete RefreshModelsContext.signal and must honor it for network requests and other blocking work. When a caller supplies a signal, Models.refresh() returns promptly with aborted: true after cancellation even if a custom provider fails to cooperate; the provider must still honor the signal to stop its underlying work.
Use models.refresh({ providers: ['openrouter'] }) to restrict work to selected providers, models.refresh({ allowNetwork: false }) to restore persisted catalogs without network access, or models.refresh({ force: true }) to bypass provider freshness checks. Model reads stay synchronous and return the last restored or refreshed list.
Handwritten providers whose catalog source needs no provider credential can set Provider.refreshModelsAuth to "configured". The network refresh still skips unconfigured providers, but does not refresh OAuth and receives context.credential === undefined. The default, "credential", resolves the effective credential and refreshes expired OAuth before fetching models. Keep that default for account-specific catalogs and hooks that read context.credential.
createProvider() handles dynamic publication and persistence automatically. Handwritten Provider.refreshModels() implementations receive the read-only context.stored snapshot and publish through context.publish({ persist?, update? }). Omit persist to leave storage unchanged, pass a ModelsStoreEntry to write it, or pass persist: null to delete it. ModelsStoreEntry.models contains models of every type. Publication is generation-checked; put synchronous in-memory catalog changes in update rather than mutating state before publication.
Custom models can carry headers (e.g. proxies behind bot detection) and compat flags. Models.getAuth(model) includes those model headers, and stream methods merge them before explicit request headers and transformHeaders. See OpenAI Compatibility Settings.
Some OpenAI-compatible servers do not understand the developer role used for reasoning-capable models. For those providers, set compat.supportsDeveloperRole to false so the system prompt is sent as a system message instead. If the server also does not support reasoning_effort, set compat.supportsReasoningEffort to false too. This commonly applies to Ollama, vLLM, SGLang, and similar OpenAI-compatible servers.
Use model-level thinkingLevelMap to describe model-specific thinking controls. Keys are the library’s thinking levels (off, minimal, low, medium, high, xhigh, max). Missing standard levels through high use provider defaults; xhigh and max are opt-in and require a non-null map entry. String values are sent to the provider, null marks a level unsupported, and maps may skip levels.
Calling API Implementations Directly
The API implementations are importable on their own. Each module exports exactlystream and streamSimple with that API’s full option typing. Direct calls bypass provider auth and context normalization — pass apiKey explicitly and wrap the context in normalizeContext():
./api/<api-id>:
Importing an implementation module loads its SDK. The
./api/<id>.lazy wrappers (used by the provider factories) defer that load to the first request when the runtime or bundler supports dynamic import chunking. Legacy raw API subpaths from older releases (./anthropic, ./google, ./mistral, ./openai-completions, …) were removed; use @bastani/atomic-ai/api/<api-id>.
Anthropic message threads (beta): with sessionId set and cacheRetention not "none", Anthropic requests continue from the previous reply on the server (message-threads-2026-08-12 beta) and send only the new messages. When anything earlier in the request changed, or the server can’t continue, the full request is sent. Token counts and prompt caching are unchanged. On by default only for a baseUrl on api.anthropic.com; set compat.supportsMessageThreads to change that, for example for gateways that forward the thread field. Requests made with a caller-supplied client never use threads. Set disableContinuation: true on requests that repeat an earlier one, such as cache warming. Call cleanupSessionResources(sessionId) when a session ends.
OpenAI Compatibility Settings
Theopenai-completions API is implemented by many providers with minor differences. By default, the library auto-detects compatibility settings based on baseUrl for a small set of known OpenAI-compatible providers (Cerebras, xAI, Chutes, DeepSeek, NVIDIA NIM, Together AI, zAi, OpenCode, Cloudflare Workers AI, etc.). For custom proxies or unknown endpoints, you can override these settings via the compat field. For openai-responses models, the compat field supports Responses-specific flags.
x-session-id from sessionId when prompt caching is enabled. Chat Completions and Anthropic Messages both auto-detect OpenRouter endpoints unless sendSessionAffinityHeaders is explicitly false. On Anthropic-compatible models, sessionAffinityFormat: "openrouter" selects x-session-id; when unset on non-OpenRouter endpoints, the existing x-session-affinity format is used. Explicit request headers take precedence over generated headers.
If compat is not set, the library falls back to URL-based detection. If compat is partially set, unspecified fields use the detected defaults. This is useful for:
- LiteLLM proxies: May not support
storefield - Custom inference servers: May use non-standard field names
- Self-hosted endpoints: May have different feature support
- Codex-protocol gateways: May reject the
max_output_tokensparameter
Faux Provider for Tests
fauxProvider() builds an in-memory provider with scripted responses for tests and demos:
- Responses are consumed from a queue in request start order.
- If the queue is empty, the faux provider returns an assistant error message with
errorMessage: "No more faux responses queued". - Use
faux.setResponses([...])to replace the remaining queue andfaux.appendResponses([...])to add more responses. faux.modelsexposes all faux models.faux.getModel()returns the first one, andfaux.getModel(id)returns a specific one.- Use
fauxAssistantMessage(...)for scripted assistant replies. UsefauxText(...),fauxThinking(...), andfauxToolCall(...)to build content blocks without filling in low-level fields manually. - Usage is estimated at roughly 1 token per 4 characters. When
sessionIdis present andcacheRetentionis not"none", prompt cache reads and writes are simulated automatically. - Tool call arguments stream incrementally via
toolcall_deltachunks. - By default, each streamed chunk is emitted on its own microtask. Set
tokensPerSecondto pace chunk delivery in real time. - The intended use is one deterministic scripted flow per handle. If you need independent concurrent flows, create separate faux providers with distinct
providerids.
Cross-Provider Handoffs
The library supports seamless handoffs between different LLM providers within the same conversation. This allows you to switch models mid-conversation while preserving context, including text, tool calls, and tool results. When messages from one provider are sent to a different provider, the library automatically transforms them for compatibility:- User and tool result messages are passed through unchanged
- Assistant messages from the same provider, API, and model are preserved as-is, including thinking blocks
- Assistant messages from a different model have their thinking blocks dropped. Signatures and encrypted reasoning only work with the model that produced them, and readable thinking sent as plain text makes the new model treat it as part of the reply. Anthropic models whose API decides which thinking blocks a model may read receive signed blocks from other models on the same provider unchanged
- Tool calls and regular text are preserved unchanged
System Messages
Public entry points (Models.stream(), streamSimple(), complete(), and completeSimple()) accept Context. They normalize systemPrompt and tools into a leading system message. Provider implementations, ProviderStreams, and direct API modules receive a TranscriptContext containing only messages.
null removes a section), and add or remove tools. Section text is opaque and rendered verbatim, joined by blank lines. getCurrentSystemPrompt(context.messages) and getCurrentTools(context.messages) replay these changes for custom providers; do not read context.systemPrompt or context.tools inside provider implementations.
Models with supportsMidConvoSystemMessages receive updates in place. Other models receive the replayed prompt and tools as a leading checkpoint. Anthropic’s supportsMidConvoToolChanges, OpenAI Responses’ supportsAdditionalTools/supportsToolSearch, and OpenAI Completions’ supportsMidConvoToolAdditions enable native tool transitions where representable. Unsupported transitions fall back to the current top-level tool list and may invalidate the cached prefix.
With Anthropic’s supportsMidConvoToolChanges, the inline-tools-2026-09-15 beta keeps the initial top-level tools and a deferred placeholder fixed. Later tool_addition blocks carry complete definitions, including replacements under an existing name; tool_removal blocks reference names. This preserves the cached prefix across tool changes. At least one initial tool is required; otherwise the current tool list is sent at the top level.
With OpenAI Responses’ supportsAdditionalTools, every tool is declared in additional_tools items, like Codex: the initial tools in one at the head of the input, and later additions and redefinitions at their message, with no top-level tool list. A later item replaces same-name definitions from earlier items but cannot remove a tool, so removals are announced as text, later updates restate the tools that are still unavailable, and calls to removed tools fail at execution. With only supportsToolSearch, or OpenAI Completions’ supportsMidConvoToolAdditions, new tools are anchored at their message; a removal or a redeclared name moves the tool state up to that point into the top-level list once.
Context Serialization
TheContext object can be easily serialized and deserialized using standard JSON methods, making it simple to persist conversations, implement chat history, or transfer contexts between services:
JSON.stringify away.
Note: If the context contains images (encoded as base64 as shown in the Image Input section), those will also be serialized.
Browser Usage
The library supports browser environments. The core entrypoint and provider factories are side-effect free and bundle cleanly. Environment variables are not available in browsers, so pass API keys explicitly — or inject aCredentialStore (e.g. localStorage-backed) and let provider auth resolve from stored credentials:
Security Warning: Exposing API keys in frontend code is dangerous. Anyone can extract and abuse your keys. Only use this approach for internal tools or demos. For production applications, use a backend proxy that keeps your API keys secure.Browser compatibility notes:
- Amazon Bedrock (
bedrock-converse-stream) is not supported in browser environments. It can still appear in model lists; calls fail at runtime. - OAuth login flows are Node-only. They are lazy-loaded behind bundler-opaque imports, so registering an OAuth-capable provider does not pull Node-only code into a browser bundle — only actually logging in would.
- Use a server-side proxy or backend service if you need Bedrock or OAuth-based auth from a web app.
Bundling and Tree Shaking
For small bundles, import only the providers you need:@bastani/atomic-aiis the core entrypoint and does not import built-in catalogs, provider factories, or SDK implementations.@bastani/atomic-ai/providers/<provider>imports that provider’s catalog and lazy API wrapper only.@bastani/atomic-ai/providers/allimports every built-in provider factory and all catalogs. Use it only when you want the full built-in set.- With code splitting, provider SDKs stay in lazy chunks and load on first request.
- Without code splitting, bundlers fold reachable lazy API implementations into the single bundle. A single-provider bundle then includes that provider’s SDK;
providers/allincludes all statically visible SDKs. Bedrock is the exception: its AWS SDK implementation is loaded through a bundler-opaque Node-only import. - Importing
@bastani/atomic-ai/api/<api-id>directly loads that API implementation and its SDK immediately.
@bastani/atomic-ai/compat in new bundled apps; it preserves the old global API and imports the full built-in catalog surface.
For single-file Node ESM bundles, some SDK dependencies may still use dynamic CommonJS require() internally. If you see errors such as Dynamic require of "child_process" is not supported, add a Node require shim to the bundle. With esbuild:
splitting: true, bun build --splitting) can keep Bedrock and the OAuth login flows inside the bundle but lazy: register loaders with literal dynamic imports, which the bundler turns into chunks that load on first use:
Provider-Scoped Environment Overrides
Passenv in stream options to scope provider configuration to a request. Values in env are used before process environment variables for provider auth and configuration such as Cloudflare account IDs, Azure OpenAI settings, Vertex project/location, Bedrock settings, PI_CACHE_RETENTION, and HTTP_PROXY/HTTPS_PROXY.
OAuth Providers
Several providers support OAuth authentication instead of static API keys:- Anthropic (Claude Pro/Max subscription)
- OpenAI (Sign in with ChatGPT, direct Responses API access)
- OpenAI Codex (ChatGPT subscription, Codex Responses access)
- GitHub Copilot (Copilot subscription)
- OpenRouter (OAuth PKCE that mints a user-controlled API key)
OAuthAuth on provider.auth.oauth with three operations: login(interaction) uses the provider-neutral AuthInteraction.prompt()/notify() protocol and returns a credential, refresh(credential, signal) refreshes expiring credentials when applicable, and toAuth(credential) derives request auth (GitHub Copilot’s per-account base URL comes from here). Provider login interactions and refresh calls always carry a concrete abort signal. Refresh is automatic: models.getAuth(providerId) and request paths refresh expired tokens under a credential-store lock, so concurrent requests and processes cannot double-refresh. OpenRouter’s OAuth flow instead returns a permanent API key, so its refresh operation is a no-op.
LoginOptions argument with getDeviceId: () => installationUuid. Persist this UUID and return the same value on later logins. The login registers a user-owned client and stores its issued client ID and granted scopes with the credential. Direct ChatGPT access omits unsupported temperature, output-token limits and cache-retention fields. If its shared subscription limit is exhausted, check ChatGPT usage.
Vertex AI
Vertex AI models support either a Google Cloud API key or Application Default Credentials (ADC). Its provider-owned API-key login flow can configure either method:- API key: Set
GOOGLE_CLOUD_API_KEYor passapiKeyin the call options. - Local development (ADC): Run
gcloud auth application-default login - CI/Production (ADC): Set
GOOGLE_APPLICATION_CREDENTIALSto point to a service account JSON key file
GOOGLE_CLOUD_PROJECT (or GCLOUD_PROJECT) and GOOGLE_CLOUD_LOCATION. You can also pass project/location in the call options. When using GOOGLE_CLOUD_API_KEY, project and location are not required.
CLI Login
The quickest way to authenticate:auth.json in the current directory.
Programmatic OAuth
Built-in login and refresh flows are private provider implementations. Use provider-ownedOAuthAuth, which composes with CredentialStore and gets locked auto-refresh through Models. The @bastani/atomic-ai/oauth entry point retains only type declarations required by coding-agent extension OAuth compatibility.
Provider notes:
OpenAI Codex: Requires a ChatGPT subscription with model access. Includes GPT-6.2 Sol and Codex models with extended context windows and reasoning capabilities. The library handles session-based prompt caching when sessionId is provided unless cacheRetention is "none". Set transport to "sse", "websocket", or "auto" for Codex Responses transport selection. WebSocket connections with a sessionId and caching enabled are reused per session and expire after 5 minutes of inactivity.
Call cleanupSessionResources(sessionId) when finished with a Codex session so its pooled WebSocket connection does not keep the process alive. Import it from @bastani/atomic-ai.
Azure OpenAI (Responses): Uses the Responses API only. Set AZURE_OPENAI_API_KEY and either AZURE_OPENAI_BASE_URL or AZURE_OPENAI_RESOURCE_NAME. AZURE_OPENAI_BASE_URL supports both https://<resource>.openai.azure.com and https://<resource>.cognitiveservices.azure.com; root endpoints are normalized to .../openai/v1 automatically. Use AZURE_OPENAI_API_VERSION (defaults to v1) to override the API version if needed. Deployment names are treated as model IDs by default, override with azureDeploymentName or AZURE_OPENAI_DEPLOYMENT_NAME_MAP using comma-separated model-id=deployment pairs (for example gpt-4o-mini=my-deployment,gpt-4o=prod). Legacy deployment-based URLs are intentionally unsupported.
GitHub Copilot: If you get “The requested model is not supported” error, enable the model manually in VS Code: open Copilot Chat, click the model selector, select the model (warning icon), and click “Enable”.
Migrating from the Old Global API
Older versions exposed a global API:stream()/complete() dispatching on model.api via a global registry, sync getModel()/getModels()/getProviders() catalog reads, registerApiProvider(), getEnvApiKey(), and per-API lazy stream functions. That surface lives unchanged on the compat entrypoint:
createModels() + provider factories:
The separate
ImagesModels/ImagesProvider collection that existed briefly (createImagesModels(), createImagesProvider(), openrouterImagesProvider(), builtinImagesModels()) is gone: image models now live on the regular provider. Replace builtinImagesModels() with builtinModels(), imagesModels.getModel() with models.getModelOfType('image', ...), and createImagesProvider({ models, api }) with createProvider({ models, images }). The old plural image type names are removed; use ImageModel and ImageApi, and add type: "image" to image model literals.