TL;DR

  • Tokens are the unit of cost. A rough estimate is 1 token ~ 4 English characters or about 0.75 words.
  • Track inputs and outputs separately. Output tokens often carry a different rate and are easier to control with concise response contracts.
  • Log cache behavior. Reused system prompts, retrieved context, and stable instructions should be measured separately from fresh input.
  • Budget by workload. A ledger is more useful when it records what the system is doing, who owns it, how often it runs, and when assumptions expire.
  • Review periodically. Rates, route capabilities, and product behavior change, so the ledger should hold assumptions instead of hard-coded provider claims.

Why Token Economics Matter

LLMs charge by the token, not the word or the request. If you're building at scale, token efficiency becomes a core part of your product architecture. Small differences in tokenization, route tier, and output budget can change the cost profile of an entire platform.

This guide starts with the fundamentals: tokens, embeddings, and tokenizers. It then turns those ideas into a lightweight ledger pattern for planning real workloads.

What Are Tokens

LLMs do not read characters or words directly. They read tokens, which are subword units produced by a tokenizer. You pay per token for input and output, so tokenization is the first layer of AI cost economics.

English prose is usually compact, but code, JSON, logs, and structured prompts often expand into more tokens because punctuation, separators, and symbols get split differently. That is why a short-looking prompt can still be expensive.

Token density by content type (approx):
English prose: 25 tokens per 100 chars
Source code: 30-40 tokens per 100 chars
JSON/YAML: 35-45 tokens per 100 chars
Chinese/Korean: 50-80 tokens per 100 chars

The practical implication: if you're processing logs, code snippets, or API responses, your token cost is often higher than plain English text. Token awareness is a design discipline, not an afterthought.

What Are Embeddings

Once tokenized, each token is converted into an embedding - a numerical vector that captures meaning in context. Embeddings help the system represent relationships between concepts, such as similarity, intent, and dependency.

The context window is the amount of text the runtime can reason over at one time. Bigger context windows reduce truncation risk, but they also increase the ceiling for a single request if you fill them.

The cost ceiling for any request is still simple: Input Cost = Input Tokens x Input Price per Token. Output cost is separate and usually higher, because generated tokens are produced one by one.

For production systems, use provider token-count APIs rather than rough character estimates. Content type, tool calls, files, and schemas can all shift the real count.

How Tokenizers Work & Why They Matter

Tokenizers split text into system-readable chunks. Each tokenizer can process the same content differently, especially for code, structured data, punctuation-heavy prompts, and non-English text. These systems are optimized for efficient text representation, but they do not always split the same prompt the same way.

That means the same English text can land in slightly different token counts across providers. The difference is usually small; the larger factor is the route tier and the output you ask it to generate.

Key insight: track costs, not just token counts. Tokens x price per token is the real equation. Ledger estimates can use a rough English heuristic, but actual token counts vary by tokenizer, content type, tools, files, and runtime behavior.

Most AI systems eventually use tiers: low-cost routes for simple repeatable work, balanced routes for day-to-day production tasks, and premium routes for complex reasoning. The ledger should capture the route class, not a stale vendor-specific name.

Low-cost route: classification, extraction, tagging, short summaries, and other high-volume tasks where failure modes are easy to validate.
Balanced route: product workflows that need stronger instruction following, moderate reasoning, and reliable structured output.
Premium route: complex synthesis, deep code reasoning, multi-step planning, and tasks where quality risk outweighs token savings.

The key economic insight: route by task, not by prestige. Track your real workload rather than assuming a route label tells the full cost story.

The Token Cost Formula

Cost is the sum of input and output token costs. Output tokens are typically more expensive than input tokens because generated text is produced token-by-token.

Total Cost = (Input Tokens x Input Price) + (Output Tokens x Output Price)

Use measured token counts wherever possible, then log each workload with its route tier, input tokens, output tokens, request volume, and monthly run frequency.

Monthly Cost = ((Input Tokens x Input Rate) + (Cached Input Tokens x Cached Rate) + (Output Tokens x Output Rate)) x Requests per Month

The ratio of output price to input price varies by provider and route tier. The practical implication: output budget is precious. If you don't need 500 tokens of output, ask for 200.

At Scale, Route Choice Multiplies

Single-request costs are small. At scale, they compound. A workload that processes thousands of documents, tickets, builds, or support messages each month needs an owner and a review rhythm.

Example ledger row

Workload: nightly incident summarization
Route: balanced production route
Formula: ((avg input x input rate) + (avg output x output rate)) x monthly runs
Review: refresh token counts and rates each billing cycle

The ledger turns usage into an engineering decision instead of a surprise invoice. A team can save meaningful budget simply by routing simple workloads to lower-cost paths and revisiting assumptions regularly.

Token Ledger Template

Keep one lightweight ledger per product area. This gives engineering, product, and finance the same view of expected usage before the bill arrives.

Recommended columns: workload, owner, route tier, input tokens, cached input tokens, output tokens, requests per day, days per month, monthly estimate, assumptions, and review date.

Cost Optimization Levers

  • Route by task: Not every request needs the most expensive route. Start with the lowest reliable tier, then escalate only when confidence, complexity, or risk requires it.
  • Prompt Caching: If the same system prompt or context is reused across requests, enable prompt caching where supported. Cached tokens cost far less than fresh input tokens and should be on by default.
  • Output Budget: You pay for every output token. Don't ask for 500 tokens if 100 will do. Constrain `max_tokens` to what you actually need.
  • Batch APIs: All major providers now offer batch-style or async workflows for non-urgent tasks. For lower-priority workloads, batching is often the most economical route.

Code Examples: Token Counting & Cost Calculation

Basic token counter (JavaScript):

function estimateTokens(text) {
  // Rough heuristic: 1 token ~ 0.75 words in English text
  const words = text.trim().split(/\s+/).length;
  return Math.round(words / 0.75);
}

const prompt = "Explain machine learning in 100 words";
const tokens = estimateTokens(prompt);
console.log(`~${tokens} tokens`); // Output: ~47 tokens

Cost calculator (Python):

ROUTES = {
  "low_cost": {"input": 0.25, "cached": 0.05, "output": 1.00},
  "balanced": {"input": 1.00, "cached": 0.20, "output": 5.00},
  "premium": {"input": 5.00, "cached": 1.00, "output": 20.00},
}

def calculate_cost(route, input_tokens, cached_tokens, output_tokens):
  rates = ROUTES[route]
  cost = (input_tokens * rates["input"] +
        cached_tokens * rates["cached"] +
        output_tokens * rates["output"]) / 1_000_000
  return round(cost, 6)

cost = calculate_cost("balanced", 2000, 500, 500)
print(f"Cost: ${cost:.4f}")