One OpenAI-compatible API in front of models served by several providers. The platform picks an eligible provider for each request, applies your keys’ limits and data constraints, and meters every request to the token against prepaid credit.
Free to sign up. To use the API, contact FreshToken to sign an offline contract; an administrator then adds your credit, and you pay per token used. There is no self-service top-up.
# Python 3, standard library only: save as first_request.py, run: python3 first_request.py
import json
import os
import urllib.request
import uuid
request = urllib.request.Request(
"https://api.freshtoken.ai/v1/chat/completions",
data=json.dumps({
"model": "zai-org/glm-4-flash-250414",
"messages": [{"role": "user", "content": "Hello"}],
}).encode(),
headers={
"Authorization": "Bearer " + os.environ["FRESHTOKEN_API_KEY"],
"Content-Type": "application/json",
# A new key for each new request; the same key only to retry this one."Idempotency-Key": str(uuid.uuid4()),
},
)
with urllib.request.urlopen(request) as response:
reply = json.load(response)
print(reply["choices"][0]["message"]["content"])
// Node.js 18 or later, no packages: save as first-request.mjs, run: node first-request.mjs
import { randomUUID } from "node:crypto";
const response = await fetch("https://api.freshtoken.ai/v1/chat/completions", {
method: "POST",
headers: {
Authorization: `Bearer ${process.env.FRESHTOKEN_API_KEY}`,
"Content-Type": "application/json",
// A new key for each new request; the same key only to retry this one."Idempotency-Key": randomUUID(),
},
body: JSON.stringify({
model: "zai-org/glm-4-flash-250414",
messages: [{ role: "user", content: "Hello" }],
}),
});
const reply = await response.json();
if (!response.ok) throw new Error(reply.error?.message ?? response.statusText);
console.log(reply.choices[0].message.content);
What the gateway does between your code and the model, on every request.
Your applicationYour code, your SDK
An OpenAI-compatible client, pointed at one base URL with one sk- key. An Idempotency-Key on each request makes a retry safe.
Token GearOn every request
Checks the key
Applies limits and data constraints
Picks an eligible deployment
Meters tokens and the charge
ProvidersDeployments of the model
Provider A: eligible
Provider B: eligible
Provider C: not eligible
If Provider A fails before any output is sent, the request goes to Provider B.
An illustration of one request, not live traffic. The providers are examples; routing is the platform’s, and customers do not choose them. Routing and failover are invite-only for now.
What Token Gear does
Token Gear is offered by invitation, under an offline contract, while we finish testing it with real providers. Nothing here is generally available yet, and each capability says where it stands. How keys and data are handled →
Invite-only
Unified model API
One OpenAI-compatible endpoint for /v1/chat/completions, /v1/completions, /v1/embeddings and a subset of /v1/responses, streamed over SSE. Keep your SDK and change the base URL; an Idempotency-Key on each request keeps a retry from being charged twice.
For each request the platform picks an eligible deployment of the model — by version, capability, data constraints and cost — and fails over before any output is sent. Customers do not choose or pin providers; each request carries the context it needs.
The most a request can cost is reserved before it goes upstream and settled to the token when it ends; a retry with the same Idempotency-Key is not charged twice. The console shows usage and charges as they happen.
FreshToken’s AI gateway and model-service platform. It puts models served by several providers behind one OpenAI-compatible API, and handles identity, routing, limits, data constraints and metering along the way.
How do I get access?
Sign up, then contact FreshToken to sign an offline contract. An administrator adds the credit you bought to your account, and you create API keys in the console. There is no self-service top-up or card payment.
How is it different from calling model providers directly?
You hold one key and one balance instead of an account per provider. Token Gear picks an eligible provider for each request, fails over before any output is sent, applies your limits and data constraints, and meters every request to the token.
What is an NCP, and what is a Token Engine?
An NCP is an upstream provider that runs model inference. Each NCP runs a Token Engine: execution state and an inference scheduler (P/D disaggregation, KV cache, batching, DP/EP) on top of its LLM hosting and GPU infrastructure.
Does Token Gear keep my conversation?
Token Gear holds no conversation between requests. Each request carries the context the model needs, as with the OpenAI API, and the platform routes each one on its own, so no conversation is tied to one provider; a reference to something a provider stored earlier is refused rather than guessed. What is kept of a request is under “What happens to my data?”.
How does billing work?
Prepaid credit, bought under an offline contract and metered per token. The most a request can cost is reserved before it goes upstream, and the actual tokens are settled when it ends. Organizations share a balance, and the console shows usage and charges as they happen.
What happens to my data?
Your content is passed to the engine or provider that serves the request and back to you. We keep what is needed to run and bill an account: keys, usage records and balances. We do not store your prompts or responses, with three exceptions: a response is kept for 24 hours (up to 1 MiB) so that a retry with the same Idempotency-Key gets it again; a request flagged for abuse is kept for a short period; and content is kept if your account turns content logging on. See the privacy policy and security and deployment.
Is it compatible with the OpenAI SDKs?
For the endpoints and fields it supports: chat completions, completions, embeddings and a subset of the Responses API, SSE streaming included. Point the SDK’s base URL at Token Gear, and use an sk- key. An Idempotency-Key header on each request is recommended, so that a retry is not charged twice. The compatibility matrix says which tools work.
How does an NCP join?
Write to support@freshtoken.ai. We review the models you host, their prices and evidence, connect your endpoint, and admit each deployment before it serves requests.
Talk to us about access
Sign up, then contact FreshToken to sign an offline contract; an administrator adds your credit.