Token costs
No other news API tells you this, and it is the single most useful number when the caller is a language model. Measured on real responses; figures are for a typical GPT/Claude tokenizer at roughly four characters per token.
Per article
| Mode | Bytes | Tokens | What you get |
|---|---|---|---|
Metadata only, full_text=false | ~600 | ~150 | Title, description, publisher, country, language, times, URL |
With body, full_text=true | ~3,600 | ~900 | All of the above plus 2,000–4,000 characters of text |
With highlight=true | ~1,000 | ~250 | Metadata plus two 200-character matched fragments |
Per response
| Call | Tokens | Verdict |
|---|---|---|
/v1/stats, any filter | ~200 | Cheapest useful call in the API |
/v1/search?size=5 | ~800 | Right size for answering a question |
/v1/search?size=20 | ~3,000 | Default. Fine. |
/v1/search?size=100 | ~15,000 | Only when code, not a model, reads the result |
/v1/search?size=20&full_text=true | ~18,000 | expensive Select first, then fetch |
/v1/search?size=100&full_text=true | ~90,000 | Will overflow most context windows |
/v1/article, one article | ~900 | The right way to get a body |
The rule
Never let a model set full_text=true together with a large
size. Scanning is cheap, reading is expensive. Retrieve metadata for
many, fetch bodies for few. A well-built pipeline spends 80% of its tokens on the eight
articles it chose and 20% on the sixty it rejected.
Cost of the same question, three ways
| Approach | Calls | Tokens |
|---|---|---|
Naive: size=50&full_text=true, feed it all in |
1 | ~45,000 |
Better: size=50, model picks five, fetch those |
6 | ~12,000 |
Best: stats, then size=60, rank in code, fetch eight |
10 | ~9,000 |
Five times cheaper, and the answer is better, because the eight articles were chosen rather than the first fifty being dumped in.
Trimming responses in your dispatcher
# keep only what the model needs to decide
KEEP = ("title", "description", "url", "published_at", "host", "country", "lang")
def slim(results):
return [{k: a.get(k) for k in KEEP if a.get(k)} for a in results]Dropping id, crawled_at, image,
categories, country_source and tld cuts roughly 35%
off a metadata response. Keep them when your own code needs them; strip them before the
text reaches a model.
Latency, while we are counting
| Call | Typical | p99 |
|---|---|---|
/v1/search, filtered | 40 ms | 107 ms |
/v1/search, full-text query | 60 ms | 150 ms |
/v1/stats | 90 ms | 220 ms |
/v1/article | 25 ms | 60 ms |
Measured at 1,247 requests per second across the cluster with zero errors.
