ai-architecture
ranked
12 Sept 2026
Eight HTML to text extractors, ranked by how much of the page survives
Seven real pages, eight extractors, the same bytes into each one. One returned clean prose for a pricing page with every price deleted.
By Purvansh Parmar
Checked 2026-09-12
Read 8 min
Entries 8
You point an agent at a pricing page and it answers with a number. The question is whether that number came from the page or from the gaps the extractor left behind. I saved seven real pages on 12 September 2026, ran eight HTML to text tools over the identical bytes, and diffed each result against probe strings lifted from the source: prices, code identifiers, navigation junk that should not survive. One tool returned clean readable prose for the Cloudflare R2 pricing page with both pricing tables deleted and no marker where they had been. That is the failure that matters. Every tool here fails quietly.
How this was checkedOn 12 September 2026 I saved seven real pages to disk, from a 1.05 MB Wikipedia article down to a 6.8 KB client-rendered app shell, and ran eight extractors over the identical bytes. Each output was diffed against probe strings lifted from the source: four prices from the Cloudflare R2 tables, code identifiers from the Python asyncio docs, and navigation text that should not have survived. readability-lxml returned clean readable prose for the R2 pricing page with both pricing tables gone.
What I compared on
- Kept: how many of 19 probe strings taken from the seven source pages survived (prices, code identifiers, section text)
- Junk: how many of 13 navigation and chrome strings leaked into the output
- Structure: whether tables came out as rows and code came out as fenced blocks, or as loose text
- Cost: characters emitted across all seven pages, against 2.07 MB of input HTML
- Time: wall clock to process all seven pages on one machine
A newer JavaScript content extractor that runs against a DOM and can emit markdown directly.
Wins
Highest fidelity in the run: 18 of 19 probes kept, 1 of 13 junk strings leaked, the Cloudflare R2 pricing tables preserved as 37 pipe-delimited rows, and 34 fenced code blocks on the asyncio page, one for each of the 34 pre elements in the source. It was teh only tool that got the tables, the code fences and the row labels all correct at once.
Loses
It is slow enough to change your architecture. 24.9 seconds for the seven pages, of which 11.9 went to the single 1.05 MB Wikipedia article, against trafilatura's 0.42 seconds on the same file. It also needs a DOM, so you are carrying jsdom and paying its parse cost before Defuddle starts.
18/19 probes kept1/13 junk24.9 s for 7 pages11.9 s on one 1.05 MB page34 of 34 code blocks fenced37 table rowsv0.19.3
Python extractor built for corpus construction, with table and link retention behind explicit flags.
Wins
Best ratio of fidelity to cost by a wide margin. 17 of 19 probes, zero junk out of 13, clean pipe tables on the R2 pricing page with the Standard and Infrequent Access columns still attached to their prices, and it finished all seven pages in 1.13 seconds. That is 22 times faster than Defuddle for output that is nearly as good.
Loses
Code blocks nested inside list items get flattened onto one line. On the asyncio page it fenced 8 code blocks where the source has 34, and a three line function came back as a single run of text with the indentation gone. The characters are all there. The Python is not runnable.
17/19 probes kept0/13 junk1.13 s for 7 pages8 of 34 code blocks fenced283,040 chars outv2.2.0
The Firefox Reader View algorithm, paired with the usual HTML to markdown converter behind it.
Wins
18 of 19 probes and zero junk, which is the cleanest boilerplate removal in the set. On a plain article it is the reference implementation for a reason, and the two pages here that are just prose came back with nothing but prose.
Loses
Two structural failures. The R2 pricing table came out as a bare column of cells with zero pipe rows, so Standard storage, Infrequent Access storage, $0.015 and $0.01 all appear as separate paragraphs and the pairing is left for the model to guess. Then turndown escaped 210 underscores on the asyncio page, turning create_task into create\_task, because the code was not fenced.
18/19 probes kept0 pipe table rows0 of 34 code blocks fenced210 escaped underscores on one page3.66 s for 7 pagesreadability 0.6.0, turndown 7.2.4
Hosted endpoint: prefix any URL with r.jina.ai and get markdown back, no key required.
Wins
The only entry that saw a client-rendered page. Excalidraw's served HTML is a 6.8 KB shell whose visible text is one sentence telling you to enable JavaScript, and every local tool came back with at most 79 characters of readable text. Jina returned 669 characters of actual interface content because it renders first. If your corpus has single page apps in it, nothing else on this list competes.
Loses
It fetches on your behalf from a shared pool, so you inherit a reputation you do not control. My request for a GitHub repository page came back as an abuse block: anonymous access to github.com suspended for several hours over a DDoS suspicion triggered by a different URL entirely. Anonymous use is capped at 20 requests per minute, and it stripped every code fence on both documentation pages.
15/19 probes kept3/13 junk20 requests per minute anonymous669 chars from a page others read as 79 or fewer0 code blocks fenced19.5 s for 7 pages over the network
Converts a whole HTML document to markdown with no attempt to find the article inside it.
Wins
Fastest in the set and it never lies to you by omission. 628 milliseconds for all seven pages, 18 of 19 probes kept, and because it converts everything it cannot silently drop the section you needed. When you are going to chunk and embed anyway, and the retriever can out-vote the noise, this is enough.
Loses
All 13 junk strings survived, every one. The GitHub page came back with the full navigation tree, the product menu and the sign-in links ahead of the README, and the corpus total was 467 KB against trafilatura's 283 KB for strictly less usable content. It also fenced no code at all on either documentation page.
628 ms for 7 pages18/19 probes13/13 junk strings467,081 chars out0 code fencesv2025.4.15
A Python reimplementation of the Readability scoring algorithm on top of lxml.
Wins
Fast and quiet: 1.04 seconds for seven pages, zero junk out of 13, and on prose-only pages its output is indistinguishable from the JavaScript original. If every document you handle is an article and you want one pip install rather than a Node runtime, it does that job.
Loses
It deleted every table on the Cloudflare R2 page and kept 1 of the 4 probes there, so $0.015, $4.50 and the Infrequent Access column are simply absent. Overall it scored 13 of 19, the worst recall here, and the output reads perfectly, which is the problem. Nothing downstream can see that the prices were ever on the page.
13/19 probes kept1/4 on the pricing page0 pipe table rows1.04 s for 7 pagesv0.9
BeautifulSoup-based whole-document HTML to markdown converter with per-tag strip options.
Wins
Structure survives better than any other whole-page converter: 45 pipe rows on the R2 page and all 34 code blocks fenced on the asyncio docs, matching the tools that actually try to find the article. As a mechanical HTML to markdown step in front of your own cleaning pass, the markup it produces is correct.
Loses
It emitted 639,664 characters for 2.07 MB of input, 31 percent of the raw HTML and 2.3 times what trafilatura produced. Passing strip=['script'] does not help: the tag goes, the text inside it stays, so the Excalidraw shell came back as 2,692 characters of inline JavaScript. GitHub's import map landed in the output as markdown.
639,664 chars out31% of raw HTML12/13 junk strings2,692 chars of JS from a 46-char pagev1.2.3
The two-line baseline everyone writes first: parse, drop script and style, take the text.
Wins
640 milliseconds for seven pages and 267 KB out, cheaper than every whole-page converter, and it kept 18 of 19 probes. For pages that are one block of prose with no code and no tables, the more sophisticated tools are not buying you anything this does not already do.
Loses
The separator argument corrupts code. Syntax highlighters wrap every token in its own span, so get_text(' ') inserts a space at each boundary: asyncio.run came out as asyncio . run in 13 of 14 occurrences, and time.strftime as time . strftime in all 10. Drop the separator and words fuse together instead. There is no setting that is right for both prose and code.
640 ms for 7 pages267,351 chars out13/13 junk strings13 of 14 dotted identifiers brokenbs4 4.15.0
Side by side
| Kept /19 | Junk /13 | Chars out | Time, 7 pages |
|---|
| Defuddle | 18 | 1 | 301,636 | 24.9 s |
| trafilatura | 17 | 0 | 283,040 | 1.13 s |
| @mozilla/readability | 18 | 0 | 331,086 | 3.66 s |
| Jina Reader | 15 | 3 | 243,894 | 19.5 s |
| html2text | 18 | 13 | 467,081 | 0.63 s |
| readability-lxml | 13 | 0 | 241,153 | 1.04 s |
| markdownify | 18 | 12 | 639,664 | 1.57 s |
| BeautifulSoup get_text() | 18 | 13 | 267,351 | 0.64 s |
Verdict
trafilatura, with Defuddle held in reserve for anything built out of tables or code. trafilatura kept 17 of 19 probes, leaked none of the 13 junk strings, held the pricing table together with its column headers, and did all seven pages in 1.13 seconds. Defuddle beat it on code by fencing 34 blocks to trafilatura's 8, matched it on tables, then charged 24.9 seconds for the same seven pages. So run trafilatura by default. When the page you are about to hand a model is a price table or an API reference, the extra 24 seconds is the cheapest correctness you will buy that day.
Sources