Ingest snapshots
Every batch we process, published with its numbers. No other news API shows you this, because most would rather you did not know how their sausage is made. Ours comes from a public source, so there is nothing to hide.
How to read this. Our upstream sources release a new archive about once an hour. We fetch each one, extract articles, drop duplicates and content farms, and index what remains. Each row below is one archive. If ingest ever stalls, this page shows it immediately — that is the point of publishing it.
| Crawled at | Pages | Articles kept | Keep rate | Country known | Duplicates | Dropped |
|---|---|---|---|---|---|---|
2026-09-07 08:57Z | 21,169 | 13,820 | 65% | 98.5% | 1,516 | 5,832 |
2026-09-07 07:06Z | 20,476 | 13,477 | 66% | 99.6% | 1,185 | 5,809 |
2026-09-07 04:46Z | 20,053 | 11,966 | 60% | 99.3% | 619 | 7,465 |
2026-09-07 01:14Z | 20,361 | 11,820 | 58% | 99.0% | 403 | 8,134 |
2026-09-06 21:59Z | 20,043 | 10,377 | 52% | 98.7% | 1,226 | 8,440 |
2026-09-06 19:36Z | 17,636 | 11,148 | 63% | 98.3% | 770 | 5,718 |
2026-09-06 17:40Z | 20,680 | 11,036 | 53% | 98.8% | 2,961 | 6,683 |
2026-09-06 15:26Z | 19,737 | 13,190 | 67% | 98.8% | 675 | 5,868 |
2026-09-06 13:14Z | 19,890 | 13,229 | 67% | 98.8% | 613 | 6,047 |
2026-09-06 11:00Z | 20,327 | 13,665 | 67% | 99.1% | 944 | 5,715 |
2026-09-06 08:53Z | 20,496 | 13,380 | 65% | 99.2% | 858 | 6,254 |
2026-09-06 06:11Z | 20,644 | 13,079 | 63% | 99.5% | 1,069 | 6,492 |
2026-09-06 03:03Z | 19,814 | 11,821 | 60% | 99.5% | 390 | 7,601 |
2026-09-05 23:34Z | 21,592 | 11,452 | 53% | 99.6% | 309 | 9,829 |
2026-09-05 20:53Z | 18,495 | 10,132 | 55% | 97.8% | 995 | 7,368 |
2026-09-05 18:43Z | 19,171 | 11,580 | 60% | 98.3% | 457 | 7,134 |
2026-09-05 16:32Z | 20,527 | 13,398 | 65% | 98.9% | 630 | 6,496 |
2026-09-05 14:25Z | 20,282 | 13,492 | 67% | 98.8% | 610 | 6,180 |
2026-09-05 12:29Z | 20,023 | 12,243 | 61% | 99.0% | 1,217 | 6,560 |
2026-09-05 10:27Z | 20,110 | 12,570 | 63% | 98.3% | 1,075 | 6,461 |
2026-09-05 08:21Z | 19,825 | 12,077 | 61% | 98.9% | 846 | 6,897 |
2026-09-05 06:07Z | 20,734 | 11,966 | 58% | 98.9% | 643 | 8,123 |
2026-09-05 03:12Z | 20,272 | 11,308 | 56% | 99.8% | 360 | 8,602 |
2026-09-04 23:33Z | 23,412 | 11,958 | 51% | 99.1% | 732 | 10,720 |
2026-09-04 21:15Z | 20,540 | 11,064 | 54% | 98.2% | 1,067 | 8,409 |
2026-09-04 19:25Z | 19,481 | 12,379 | 64% | 97.8% | 1,038 | 6,064 |
2026-09-04 17:44Z | 19,659 | 12,332 | 63% | 98.7% | 1,186 | 6,141 |
2026-09-04 16:09Z | 20,100 | 13,465 | 67% | 98.7% | 703 | 5,930 |
2026-09-04 14:35Z | 19,900 | 13,399 | 67% | 99.0% | 695 | 5,806 |
2026-09-04 13:04Z | 21,564 | 14,862 | 69% | 98.8% | 870 | 5,831 |
2026-09-04 11:38Z | 20,699 | 14,100 | 68% | 99.1% | 1,253 | 5,345 |
2026-09-04 09:55Z | 21,327 | 14,489 | 68% | 98.8% | 1,114 | 5,722 |
2026-09-04 08:25Z | 21,074 | 13,640 | 65% | 99.1% | 1,655 | 5,777 |
2026-09-04 06:25Z | 22,114 | 14,084 | 64% | 99.5% | 1,621 | 6,408 |
2026-08-27 05:53Z | 1,946 | 915 | 47% | 99.3% | 46 | 985 |
2026-09-04 04:13Z | 19,230 | 11,389 | 59% | 99.7% | 482 | 7,354 |
2026-09-04 01:21Z | 20,811 | 11,287 | 54% | 99.8% | 324 | 9,199 |
2026-09-03 22:44Z | 21,218 | 11,539 | 54% | 99.4% | 519 | 9,160 |
2026-09-03 20:55Z | 19,610 | 11,593 | 59% | 99.4% | 1,183 | 6,834 |
2026-09-03 19:05Z | 19,970 | 12,404 | 62% | 99.6% | 1,207 | 6,359 |
2026-09-03 17:30Z | 20,607 | 13,387 | 65% | 99.0% | 1,726 | 5,494 |
2026-09-03 16:02Z | 19,913 | 13,364 | 67% | 98.5% | 771 | 5,776 |
2026-09-03 14:38Z | 20,033 | 14,143 | 71% | 99.1% | 592 | 5,298 |
2026-09-03 13:18Z | 19,835 | 13,428 | 68% | 98.3% | 855 | 5,548 |
2026-09-03 11:50Z | 20,845 | 14,291 | 69% | 98.9% | 871 | 5,681 |
2026-09-03 10:17Z | 20,299 | 14,849 | 73% | 98.6% | 1,144 | 4,303 |
2026-09-03 08:47Z | 21,803 | 14,931 | 68% | 98.0% | 1,960 | 4,911 |
2026-09-03 06:56Z | 21,431 | 13,686 | 64% | 99.5% | 916 | 6,827 |
2026-09-03 04:48Z | 20,524 | 12,747 | 62% | 99.3% | 875 | 6,900 |
2026-09-03 02:00Z | 20,972 | 12,146 | 58% | 99.9% | 340 | 8,484 |
2026-09-02 22:58Z | 22,250 | 12,037 | 54% | 99.6% | 484 | 9,727 |
2026-09-02 20:54Z | 19,088 | 12,913 | 68% | 99.6% | 1,290 | 4,885 |
2026-09-02 19:12Z | 18,797 | 12,849 | 68% | 99.4% | 1,339 | 4,609 |
2026-09-02 17:30Z | 19,181 | 13,444 | 70% | 98.6% | 988 | 4,749 |
2026-09-02 15:55Z | 19,298 | 14,006 | 73% | 98.8% | 837 | 4,453 |
2026-09-02 14:25Z | 19,368 | 14,807 | 76% | 99.3% | 955 | 3,605 |
2026-09-02 12:58Z | 18,873 | 14,103 | 75% | 98.9% | 670 | 4,097 |
2026-09-02 11:38Z | 18,781 | 13,008 | 69% | 99.1% | 998 | 4,771 |
2026-09-02 10:05Z | 20,683 | 14,578 | 70% | 99.1% | 1,534 | 4,568 |
2026-09-02 08:30Z | 19,923 | 13,714 | 69% | 99.3% | 1,697 | 4,511 |
2026-09-02 06:37Z | 20,474 | 13,901 | 68% | 99.6% | 928 | 5,642 |
2026-09-02 04:17Z | 19,656 | 11,658 | 59% | 99.4% | 1,033 | 6,962 |
2026-09-02 01:29Z | 20,607 | 11,767 | 57% | 99.6% | 528 | 8,310 |
2026-09-01 22:35Z | 21,988 | 13,161 | 60% | 99.6% | 859 | 7,968 |
2026-09-01 20:38Z | 18,421 | 12,553 | 68% | 99.2% | 920 | 4,948 |
2026-09-01 18:48Z | 18,533 | 13,055 | 70% | 99.5% | 1,276 | 4,202 |
2026-09-01 17:11Z | 20,244 | 13,989 | 69% | 98.9% | 1,099 | 5,155 |
2026-09-01 15:45Z | 20,220 | 14,118 | 70% | 98.2% | 503 | 5,597 |
2026-09-01 14:15Z | 19,862 | 14,544 | 73% | 99.1% | 818 | 4,498 |
2026-09-01 12:50Z | 18,448 | 13,532 | 73% | 99.1% | 924 | 3,986 |
2026-09-01 11:17Z | 19,017 | 14,771 | 78% | 98.7% | 1,077 | 3,167 |
2026-09-01 09:46Z | 20,243 | 14,460 | 71% | 98.9% | 1,439 | 4,340 |
2026-09-01 08:10Z | 19,739 | 15,060 | 76% | 99.5% | 1,438 | 3,240 |
2026-09-01 06:22Z | 19,959 | 12,937 | 65% | 99.5% | 2,068 | 4,951 |
2026-09-01 05:05Z | 11,098 | 6,882 | 62% | 99.7% | 419 | 3,797 |
2026-09-01 02:20Z | 20,288 | 11,953 | 59% | 99.6% | 365 | 7,968 |
2026-08-31 23:10Z | 22,864 | 14,128 | 62% | 98.4% | 851 | 7,885 |
2026-08-31 21:01Z | 19,275 | 12,531 | 65% | 99.4% | 1,533 | 5,211 |
2026-08-31 18:55Z | 20,083 | 13,299 | 66% | 99.4% | 1,186 | 5,598 |
2026-08-31 17:11Z | 19,745 | 13,354 | 68% | 99.1% | 776 | 5,614 |
2026-08-31 15:35Z | 20,281 | 14,125 | 70% | 98.8% | 880 | 5,275 |
2026-08-31 14:01Z | 19,823 | 13,961 | 70% | 98.9% | 820 | 5,039 |
2026-08-31 12:23Z | 20,894 | 15,152 | 73% | 98.3% | 744 | 4,995 |
2026-08-31 10:48Z | 20,135 | 14,000 | 70% | 99.1% | 1,309 | 4,822 |
2026-08-31 09:10Z | 21,846 | 14,504 | 66% | 98.8% | 2,111 | 5,231 |
2026-08-31 07:22Z | 21,879 | 14,222 | 65% | 99.9% | 1,380 | 6,276 |
2026-08-31 05:12Z | 21,457 | 13,746 | 64% | 99.5% | 646 | 7,063 |
2026-08-31 02:04Z | 22,373 | 13,357 | 60% | 99.7% | 633 | 8,380 |
2026-08-30 22:29Z | 23,237 | 14,039 | 60% | 99.6% | 653 | 8,543 |
2026-08-30 20:00Z | 19,794 | 12,789 | 65% | 99.3% | 855 | 6,150 |
2026-08-30 17:42Z | 20,329 | 13,253 | 65% | 99.3% | 840 | 6,235 |
2026-08-30 15:18Z | 21,404 | 14,474 | 68% | 98.8% | 952 | 5,976 |
2026-08-30 13:06Z | 20,125 | 13,597 | 68% | 99.0% | 897 | 5,626 |
2026-08-30 10:56Z | 20,633 | 13,201 | 64% | 99.3% | 1,648 | 5,781 |
2026-08-30 08:29Z | 21,600 | 14,163 | 66% | 99.3% | 997 | 6,437 |
2026-08-30 05:42Z | 21,183 | 12,656 | 60% | 99.6% | 820 | 7,702 |
2026-08-30 01:44Z | 21,309 | 10,357 | 49% | 99.7% | 389 | 10,561 |
2026-08-29 22:06Z | 20,675 | 10,273 | 50% | 99.2% | 751 | 9,648 |
2026-08-29 19:29Z | 19,697 | 11,788 | 60% | 98.0% | 771 | 7,138 |
2026-08-29 17:09Z | 19,685 | 11,990 | 61% | 99.1% | 1,127 | 6,567 |
2026-08-29 14:47Z | 20,813 | 13,770 | 66% | 98.8% | 713 | 6,326 |
2026-08-29 12:39Z | 20,403 | 14,572 | 71% | 99.1% | 967 | 4,862 |
2026-08-29 10:20Z | 21,227 | 14,068 | 66% | 99.1% | 1,345 | 5,811 |
2026-08-29 08:02Z | 20,645 | 13,043 | 63% | 98.9% | 1,471 | 6,128 |
2026-08-29 05:16Z | 21,055 | 12,180 | 58% | 99.7% | 1,390 | 7,482 |
2026-08-29 02:08Z | 21,581 | 11,524 | 53% | 99.7% | 706 | 9,348 |
2026-08-28 22:31Z | 23,902 | 14,604 | 61% | 98.9% | 1,145 | 8,151 |
2026-08-28 20:31Z | 19,918 | 13,156 | 66% | 98.8% | 1,241 | 5,521 |
2026-08-28 18:30Z | 21,050 | 14,237 | 68% | 99.4% | 1,295 | 5,518 |
2026-08-28 16:37Z | 20,476 | 14,216 | 69% | 97.8% | 1,244 | 5,014 |
2026-08-28 14:54Z | 20,812 | 15,299 | 74% | 99.1% | 610 | 4,903 |
2026-08-28 13:14Z | 21,041 | 15,218 | 72% | 98.7% | 1,315 | 4,507 |
2026-08-28 11:37Z | 20,402 | 14,050 | 69% | 98.2% | 1,840 | 4,509 |
2026-08-28 10:02Z | 21,817 | 14,730 | 68% | 99.3% | 2,135 | 4,950 |
2026-08-28 08:15Z | 22,198 | 16,618 | 75% | 99.5% | 1,717 | 3,861 |
2026-08-28 06:25Z | 21,058 | 14,788 | 70% | 99.6% | 1,341 | 4,926 |
2026-08-28 04:01Z | 21,529 | 13,351 | 62% | 99.9% | 749 | 7,428 |
2026-08-28 00:49Z | 22,813 | 12,737 | 56% | 99.7% | 686 | 9,386 |
2026-08-27 21:58Z | 22,077 | 13,439 | 61% | 92.5% | 924 | 7,714 |
2026-08-27 19:58Z | 20,582 | 13,260 | 64% | 98.1% | 1,306 | 6,016 |
2026-08-27 18:07Z | 19,533 | 13,558 | 69% | 97.8% | 1,142 | 4,832 |
2026-08-27 16:29Z | 19,641 | 13,493 | 69% | 86.5% | 916 | 5,229 |
2026-08-27 14:58Z | 19,655 | 14,305 | 73% | 87.2% | 680 | 4,670 |
2026-08-27 13:28Z | 19,678 | 13,978 | 71% | 86.4% | 796 | 4,904 |
2026-08-27 11:51Z | 20,360 | 15,176 | 75% | 82.2% | 1,165 | 4,016 |
2026-08-27 10:32Z | 20,230 | 12,885 | 64% | 83.6% | 868 | 6,476 |
2026-08-27 09:11Z | 20,967 | 12,413 | 59% | 86.4% | 1,316 | 7,234 |
2026-08-27 07:48Z | 20,493 | 11,690 | 57% | 86.7% | 1,400 | 7,400 |
2026-08-27 06:06Z | 20,852 | 10,843 | 52% | 88.0% | 898 | 9,108 |
2026-08-27 04:09Z | 19,920 | 8,780 | 44% | 82.4% | 491 | 10,647 |
2026-08-27 02:03Z | 21,674 | 9,211 | 42% | 87.1% | 331 | 12,129 |
2026-08-26 23:52Z | 23,415 | 10,445 | 45% | 84.0% | 438 | 12,532 |
2026-08-26 22:08Z | 20,929 | 10,318 | 49% | 84.8% | 482 | 10,129 |
2026-08-26 20:38Z | 19,498 | 9,586 | 49% | 82.4% | 784 | 9,128 |
2026-08-26 19:15Z | 20,685 | 10,580 | 51% | 80.8% | 920 | 9,184 |
2026-08-26 17:55Z | 19,429 | 11,369 | 59% | 84.8% | 809 | 7,251 |
2026-08-26 16:40Z | 18,874 | 10,666 | 57% | 84.5% | 549 | 7,657 |
2026-08-26 15:33Z | 18,380 | 10,588 | 58% | 80.5% | 581 | 7,211 |
2026-08-26 14:34Z | 16,905 | 9,806 | 58% | 86.5% | 584 | 6,515 |
2026-08-26 13:26Z | 19,720 | 11,400 | 58% | 83.6% | 744 | 7,575 |
2026-08-26 12:19Z | 18,981 | 10,643 | 56% | 83.1% | 638 | 7,699 |
2026-08-26 11:11Z | 19,780 | 10,936 | 55% | 85.6% | 685 | 8,159 |
2026-08-26 10:04Z | 20,693 | 11,536 | 56% | 84.1% | 1,132 | 8,023 |
2026-08-26 08:56Z | 19,976 | 10,836 | 54% | 83.5% | 1,105 | 8,033 |
2026-08-26 07:45Z | 18,479 | 9,182 | 50% | 90.5% | 729 | 8,568 |
2026-08-26 06:28Z | 18,672 | 8,847 | 47% | 86.9% | 538 | 9,283 |
2026-08-26 04:57Z | 18,702 | 8,729 | 47% | 86.2% | 776 | 9,196 |
2026-08-26 03:23Z | 17,001 | 7,520 | 44% | 83.2% | 198 | 9,283 |
2026-08-26 01:30Z | 18,895 | 7,392 | 39% | 77.5% | 276 | 11,226 |
2026-08-25 23:50Z | 19,421 | 8,127 | 42% | 71.3% | 254 | 11,038 |
2026-08-25 22:27Z | 18,736 | 9,485 | 51% | 81.5% | 389 | 8,862 |
2026-08-25 20:59Z | 19,236 | 9,212 | 48% | 82.7% | 737 | 9,287 |
2026-08-25 19:39Z | 18,418 | 9,595 | 52% | 82.1% | 510 | 8,313 |
2026-08-25 18:28Z | 18,344 | 9,470 | 52% | 74.9% | 717 | 8,157 |
2026-08-25 17:13Z | 19,052 | 11,148 | 59% | 77.1% | 783 | 7,121 |
2026-08-25 16:04Z | 17,680 | 10,434 | 59% | 80.4% | 551 | 6,692 |
2026-08-25 15:05Z | 17,470 | 10,414 | 60% | 87.3% | 472 | 6,582 |
2026-08-25 14:02Z | 19,063 | 11,259 | 59% | 84.3% | 577 | 7,227 |
2026-08-25 12:55Z | 19,167 | 11,079 | 58% | 83.3% | 484 | 7,599 |
2026-08-25 11:52Z | 17,963 | 10,130 | 56% | 85.6% | 638 | 7,194 |
2026-08-25 10:53Z | 17,765 | 10,086 | 57% | 88.1% | 548 | 7,131 |
2026-08-25 09:48Z | 18,705 | 10,718 | 57% | 86.4% | 973 | 7,011 |
2026-08-25 08:42Z | 20,120 | 10,574 | 53% | 86.9% | 1,247 | 8,299 |
2026-08-25 07:30Z | 19,526 | 9,723 | 50% | 90.9% | 1,041 | 8,759 |
2026-08-25 06:04Z | 19,369 | 9,841 | 51% | 85.2% | 994 | 8,530 |
2026-08-25 04:29Z | 17,585 | 8,715 | 50% | 83.9% | 527 | 8,340 |
2026-08-25 02:42Z | 17,845 | 8,342 | 47% | 87.0% | 235 | 9,265 |
2026-08-25 00:39Z | 20,907 | 9,347 | 45% | 82.3% | 470 | 11,090 |
2026-08-24 22:53Z | 19,334 | 10,302 | 53% | 85.4% | 634 | 8,398 |
2026-08-24 21:05Z | 20,982 | 10,976 | 52% | 84.0% | 948 | 9,058 |
2026-08-24 19:30Z | 20,436 | 11,902 | 58% | 80.1% | 616 | 7,918 |
2026-08-24 18:04Z | 19,628 | 12,156 | 62% | 78.0% | 917 | 6,555 |
2026-08-24 16:48Z | 18,332 | 11,562 | 63% | 84.9% | 628 | 6,141 |
2026-08-24 15:42Z | 17,459 | 11,684 | 67% | 83.2% | 357 | 5,417 |
2026-08-24 14:26Z | 19,379 | 12,408 | 64% | 83.1% | 552 | 6,413 |
2026-08-24 13:06Z | 20,168 | 12,253 | 61% | 83.7% | 866 | 7,049 |
2026-08-24 11:47Z | 19,528 | 12,657 | 65% | 84.2% | 621 | 6,246 |
2026-08-24 10:27Z | 18,871 | 11,712 | 62% | 84.9% | 754 | 6,403 |
2026-08-24 09:06Z | 20,295 | 12,227 | 60% | 85.3% | 1,283 | 6,784 |
2026-08-24 07:37Z | 20,196 | 11,014 | 55% | 88.3% | 1,141 | 8,041 |
2026-08-24 06:04Z | 18,130 | 9,819 | 54% | 88.9% | 458 | 7,853 |
2026-08-24 04:08Z | 18,148 | 8,624 | 48% | 83.6% | 360 | 9,163 |
2026-08-24 01:50Z | 19,288 | 8,222 | 43% | 71.5% | 1,327 | 9,735 |
2026-08-23 23:22Z | 19,160 | 8,268 | 43% | 74.4% | 881 | 10,011 |
2026-08-23 21:02Z | 18,361 | 8,034 | 44% | 85.0% | 761 | 9,566 |
2026-08-23 19:12Z | 17,564 | 8,866 | 50% | 83.7% | 428 | 8,270 |
2026-08-23 17:23Z | 18,428 | 9,312 | 51% | 82.8% | 626 | 8,490 |
2026-08-23 15:41Z | 18,235 | 8,879 | 49% | 86.1% | 469 | 8,885 |
2026-08-23 13:55Z | 19,052 | 9,466 | 50% | 85.1% | 483 | 9,100 |
2026-08-23 12:10Z | 19,285 | 9,236 | 48% | 85.4% | 668 | 9,379 |
2026-08-19 00:08Z | 19,295 | 8,745 | 45% | 83.6% | 334 | 10,216 |
2026-08-23 10:31Z | 18,846 | 9,319 | 49% | 86.8% | 537 | 8,987 |
2026-08-21 05:53Z | 19,711 | 9,525 | 48% | 86.3% | 1,030 | 9,152 |
2026-08-18 22:20Z | 19,334 | 10,023 | 52% | 85.0% | 378 | 8,933 |
2026-08-23 08:43Z | 18,496 | 9,017 | 49% | 88.7% | 435 | 9,044 |
2026-08-23 07:04Z | 17,009 | 7,899 | 46% | 88.1% | 522 | 8,587 |
2026-08-23 05:02Z | 17,690 | 7,612 | 43% | 86.0% | 184 | 9,891 |
2026-08-23 02:50Z | 17,195 | 6,754 | 39% | 83.6% | 192 | 10,243 |
2026-08-23 00:07Z | 19,568 | 6,605 | 34% | 88.7% | 122 | 12,841 |
2026-08-22 21:44Z | 19,883 | 7,686 | 39% | 83.5% | 706 | 11,491 |
What the columns mean
| Column | Meaning |
|---|---|
| Pages | Records in the archive, including everything that is not an article. |
| Articles kept | What survived extraction and filtering and went into the index. |
| Keep rate | Typically 55–65%. The rest are section fronts, tag pages, paginated listings and pages with too little text. |
| Country known | Share of kept articles where the country cascade resolved. Around 82–85% is normal. See methodology. |
| Duplicates | Removed by body-text fingerprint within the batch — wire copy republished under several URLs. |
| Dropped | Everything discarded: not an article, no date, blocklisted host, or older than the freshness limit. |
Timing
A typical archive is about 1 GiB compressed and holds 15,000–20,000 pages. Download and extraction take roughly 87 seconds, indexing another 12 — about 99 seconds end to end. Articles become searchable in the same minute they are indexed.
Why publish this
Because "real-time" is the most abused word in this category, and a table of timestamps settles the question. Our data is hourly, with a typical lag of one to two hours from publication. Knowing that precisely is more useful to an agent than a badge claiming something better.
