Understanding Tokens: How AI Models Count and Charge for Your Work

If you run agents for any length of time, the bill shows up. And unlike a flat SaaS subscription, an AI bill is the sum of thousands of tiny per-call charges, each measured in a unit most people never had to think about before: the token.

This post explains what a token actually is, how the major providers turn tokens into dollars, and — most importantly — the handful of habits that separate a cheap agent workflow from an expensive one.

What Is a Token?

A token is a chunk of text the model processes. It is not a word, not a character, and not always a syllable. It is whatever unit the provider’s tokenizer decided was frequent enough in its training data to earn its own ID.

As a rough rule for English prose:

• 1 token ≈ 4 characters
• 1 token ≈ 0.75 words
• 100 tokens ≈ 75 words, or about a short paragraph

The tokenizer splits text using an algorithm like Byte Pair Encoding (BPE) or SentencePiece. Common words like the are a single token. Rarer words get split: tokenization becomes token + ization. Code, punctuation-heavy text, emoji, and non-English languages tend to fragment into more tokens per visible character — sometimes dramatically so.

The same sentence tokenizes differently on every provider. Claude’s tokenizer is roughly 10–20% more efficient than GPT’s for English prose — less so for code.

What Gets Counted

Every request is billed on two separate streams, and understanding the split is the foundation of every cost decision that follows.

That asymmetry is why output tokens are typically three to five times more expensive than input tokens. It is also why a chatty model is an expensive model, regardless of how cheap its input pricing looks.

How Cost Is Calculated

Providers quote prices per 1 million tokens. The formula is boring but exact:

cost = (input_tokens  / 1,000,000) × input_price
     + (output_tokens / 1,000,000) × output_price

Rough shape of the market in 2026 (USD per 1M tokens):

The gap between tiers is roughly 10×–50×. Picking the right tier for the task is the single largest cost lever you have.

Why the Same Prompt Costs Different Amounts

Three things surprise people:

1. Tokenizers differ. The same sentence tokenizes differently in each provider’s vocabulary.
2. Conversations grow quadratically. Each turn re-sends the entire history as input.
3. Reasoning tokens are invisible but billed. o1, o3, and Claude “thinking” models emit hidden chain-of-thought that counts as output. A one-sentence question can produce 10,000+ reasoning tokens you never see but definitely pay for.

Turn 20 pays for turns 1 through 19 again.

Cost-Reduction Tactics That Actually Work

The two clusters above represent the same conversation at two different points. On the left, a fresh, focused exchange — a handful of tokens, easy to reason about, cheap to run. On the right, the same conversation fifty turns later, dragging every prior message forward on every request.

Here are the levers that move the needle, roughly in order of impact.

1. Keep sessions short and scoped

The single most common way to burn money is a long-running chat session where the full history is re-sent on every turn. By turn 50, a modest 2K-token exchange has ballooned into 100K+ tokens of context being shipped every request.

Rule of thumb: if the task changes, start a new session. Weave’s agent model encourages this — each specialist runs in a fresh context and returns a summary, so the orchestrator’s context stays lean.

2. Use smaller context windows deliberately

Just because a model supports 1M tokens of context does not mean you should fill it. Many providers tier pricing above certain thresholds (e.g. Gemini charges more past 128K), and even where they don’t, more input tokens means more money and often worse quality due to attention dilution.

Trim ruthlessly: drop old tool results, summarize earlier turns, and only attach the specific files the model needs right now.

3. Match the model to the task

A code review does not need a frontier reasoning model. A syntax fix does not need Opus. Route mechanical work to small/fast models and reserve frontier tiers for genuinely hard planning or novel problem-solving.

Weave’s multi-agent design makes this natural: cheap agents (Thread for exploration, Shuttle for focused edits) do the volume work, and expensive agents (Pattern for planning, Warp for security audits) are invoked only when justified.

4. Turn on prompt caching

Anthropic, OpenAI, and Google all offer prompt caching: mark a stable prefix (your system prompt, tool definitions, reference docs) as cacheable, and subsequent calls read it at ~10% of the normal input price. Cache writes cost slightly more (~1.25×), so the break-even is usually 2–3 reuses.

For agent workloads with long, stable system prompts, prompt caching can cut input costs by 80% or more. It is the closest thing to a free lunch in this space.

5. Use the Batch API for anything asynchronous

If your job can wait up to 24 hours (evals, bulk analysis, offline processing), the batch APIs from OpenAI and Anthropic offer a flat 50% discount. No code changes beyond the endpoint.

6. Watch output tokens like a hawk

Because output is 3–5× input pricing, a chatty model is an expensive model. Set max_tokens to something realistic for the task. Ask for structured output (JSON schema) instead of prose when you can — it’s shorter and easier to parse. Tell the model to be concise; it will listen.

7. Prune tool definitions

Tool schemas are re-sent as input on every turn. An agent with 30 registered tools can easily be shipping 5K+ tokens of schema per request before any actual work happens. Only expose the tools the current step needs.

How to Measure Before You Pay

Every provider gives you a way to count tokens without spending money on generation:

• OpenAI: the tiktoken library, or the usage field returned on every API response.
• Anthropic: the /v1/messages/count_tokens endpoint, plus a usage object in responses that breaks out cache_creation_input_tokens and cache_read_input_tokens separately.
• Google: the countTokens method on each model.
• Back-of-envelope: characters / 4 is close enough for English prose budgeting; multiply by 1.3 for code or non-English text.

Log the usage field from every response into your telemetry. You cannot manage what you don’t measure, and the raw numbers are already in the API response — you just have to record them.

The Short Version

Tokens are the unit. Input and output are billed separately, output costs several times more, and reasoning models bill invisible thinking as output. The three habits that keep costs sane:

1. Short, scoped sessions — don’t drag a long history through every turn.
2. Right-sized models — reserve frontier tiers for genuinely hard problems.
3. Prompt caching on stable prefixes — the largest easy win for agent workloads.

Weave’s design leans into all three: specialized agents in fresh contexts, cheap explorers for volume work, and a lean orchestrator that delegates rather than accumulates. The architecture is not just about quality — it is also about not lighting money on fire.