Extracting Structured Data from HTML with LLMs

The case against selectors

Selectors encode a page's current DOM structure into your codebase. Every redesign, A/B test and localization variant is a potential silent break, and the failure usually shows up as nulls in a dashboard days later.

Language-model extraction reads the page the way a person does, so cosmetic changes do not matter. The trade-off is cost, latency and the need for validation.

Make it cheap: clean before you extract

Most of a page is navigation, scripts, tracking and footer boilerplate. Stripping those before extraction typically removes 80–95% of the tokens and improves accuracy, because the model has less to be distracted by.

Convert to a compact representation — cleaned HTML or markdown — and keep only the region likely to contain your fields when you can identify it cheaply.

const { data, confidence } = await irmu.extract({
  url: "https://example.com/product/42",
  schema: {
    name: "string",
    price: "number",
    currency: "string",
    availability: "string",
  },
});

if (confidence.price < 0.8) await queueForReview(data);

Make it reliable: schema and confidence

Always validate against a schema. A typed contract turns a plausible-looking wrong answer into a caught error, and it lets you fail loudly instead of writing garbage into your warehouse.

Per-field confidence lets you route the uncertain minority to human review while the rest flows through automatically. In practice, teams see 95–98% straight-through processing with a small review queue for the rest.

When selectors still win

If you scrape one stable page shape millions of times a day and own the monitoring for it, selectors are cheaper. The moment you are covering many sites, or a site that changes often, extraction wins on total cost of ownership even at a higher per-request price.