Pagination

Offset paging works to 9,900 results. Beyond that, slice by time — which is faster anyway.

Offset paging

bash

curl "https://freenewsapi.ai/v1/search?q=budget&size=100&offset=0"\ncurl "https://freenewsapi.ai/v1/search?q=budget&size=100&offset=100"\ncurl "https://freenewsapi.ai/v1/search?q=budget&size=100&offset=200"

Maximum size is 100, maximum offset is 9,900. Together they cap a single query at 10,000 retrievable results.

The ceiling is not a policy, it is physics. Deep offsets force the engine to sort every matching document on every shard to throw almost all of them away. At offset 9,900 you are already paying for 10,000 sorted documents to receive 100.

Walking a whole day

A day holds roughly 110,000 articles — eleven times the offset ceiling. Slice by time and reset the offset for each slice. Because to is exclusive, adjacent windows never overlap and never drop an article:

python

import requests

BASE = "https://freenewsapi.ai/v1/search"

def walk_day(day, **filters):
    """Yield every article published on `day`, hour by hour."""
    for h in range(24):
        frm = f"{day}T{h:02d}:00:00Z"
        to  = f"{day}T{h:02d}:59:59Z"
        offset = 0
        while True:
            r = requests.get(BASE, params={
                "from": frm, "to": to, "sort": "date_asc",
                "size": 100, "offset": offset, **filters,
            }).json()
            hits = r["results"]
            if not hits:
                break
            yield from hits
            offset += len(hits)
            if offset >= 9900:      # hour too dense \u2014 split further
                break

for a in walk_day("2026-08-17", country="UA"):
    print(a["published_at"], a["title"])

If a single hour still exceeds 9,900 — rare, and only without filters — subdivide into ten-minute windows, or add a lang or country filter and iterate over those values instead.

Polling for new articles

For a continuously running agent, do not paginate at all. Sort by crawled and ask only for what arrived since the last poll:

bash

curl "https://freenewsapi.ai/v1/search?q=merger&sort=crawled&from=now-15m&size=100"

Sort by crawled rather than date here. Publishers backdate publication times; crawl time is assigned by the pipeline and only moves forward, so nothing can slip in behind your cursor.

New batches land once an hour — see snapshots for the actual cadence. Polling more often than every fifteen minutes buys you nothing.

Cursor paging

A search_after cursor that removes the ceiling entirely is on the roadmap. Until then, time slicing is the supported way to do a full sweep, and it parallelises better: 24 hourly windows can be fetched concurrently, whereas offset paging is inherently sequential.