API Reference

Crawl a url

GET /crawl — bearer auth, JSON responses, billed per successful request.

GET/crawl

Crawl a url

Fetches a URL through a proxy pool, charging the calling organization and tallying the request against the URL's host for the day. Returns the page content with cost details in headers, or a JSON envelope when only an ai_query result is produced.

When to use it: Use for single-page retrieval when you need the rendered HTML, a screenshot, or an AI-extracted answer. Use js=false for known-static pages to reduce cost and latency. Use premium for targets that block standard proxy pools. Use country and city for geo-specific content, bearing in mind city-level targeting doubles the credit cost.

Notes:

images is off by default and costs 2 extra credits when enabled; image src attributes remain in the HTML regardless.
adblock is on by default and free; disable it only if you need ad or tracker payloads that the filter would remove.
A 202 response means credits are already spent and the crawl is in progress; collect results from /crawl/{job} without additional charge.

Parameter	In	Type	Description
url *	query	string<uri>	The page to crawl. Must be `http` or `https` and must have a host.
premium	query	string	Fetch through the premium proxy pool instead of the standard one. Costs more; use it for targets that refuse ordinary requests. Every crawl is proxied either way -- this chooses which pool, not whether there is one. `country` works on both. Sent as text in a query string, so the spellings a person would reach for all work: `true`, `false`, `1`, `0`, `yes`, `no`, `on`, `off`, in any case. Anything that is not a yes or a no is refused rather than quietly read as `false`.
js	query	string	Render javascript before reading the page. On by default, because it is what works on most of the web -- a caller who knows their target is static turns it off and pays less. Accepts the same spellings as `premium`. Off, the page comes back as the server delivered it: scripts are neither run nor even downloaded, along with anything they would have requested, which makes this the bluntest lever there is on how long a crawl takes. Only use it where what you came for is in the served html -- on a client-rendered site the page will be empty, and it is still charged for.
images	query	string	Download the images on the page. Off by default, and the one option here that costs more when you turn it on: 2 credits on top of the fetch, whatever the fetch itself came to. Off, image requests are aborted before they leave the browser and everything else -- HTML, CSS, scripts, XHR -- still loads, so the page behaves normally apart from having no pictures. An `<img>` tag is in the html either way, so its `src` is still there to be read; what you do not get is the file behind it. On a test page the difference measured 422 KB against 166 KB. Turn them on when the bytes are what you came for. Stacks with `adblock`, which blocks a different set of requests. Accepts the same spellings as `premium`.
ai_query	query	string	A question to answer about the page once it has been retrieved. Costs 5 credits on top of the fetch. An empty value is the same as asking nothing and costs nothing extra.
adblock	query	string	Drop ads and trackers before the page loads, using the EasyList filters. On by default, and free either way -- it changes nothing about the price. Ad and tracker payloads dominate a typical retail page, and blocking them cuts what has to be downloaded by around 80% while leaving the page's own assets untouched, so this is the right default for almost every target. What it also removes is anything an ad or analytics script would have written into the document -- a tracking pixel's payload, a `dataLayer` an ad tag populates. Send `adblock=false` if that is what you came for. Accepts the same spellings as `premium`.
wait_until	query	load \| domcontentloaded \| networkidle0 \| networkidle2	When the page counts as loaded. `networkidle0` by default -- it waits for the network to go quiet, which is the slowest of the four and the one most likely to return a finished page. Use `domcontentloaded` or `load` for a page that keeps a connection open (a socket, a poll, an analytics beacon), where waiting for silence means waiting until the crawl times out.
viewport	query	string	The window size, written `WIDTHxHEIGHT`. Both sides must be positive. Defaults to `1920x1080`. Required to use `device_scale_factor`, `mobile`, `landscape` or `touch`, which describe a viewport and are meaningless without one.
device_scale_factor	query	number	The device pixel ratio, as a retina screen reports it. `3` for a recent phone. Requires `viewport`.
mobile	query	string	Tell the page it is on a phone. Requires `viewport`, and accepts the same spellings as `premium`. This changes the window metrics only. A site that decides what to serve from the user agent will still send its desktop page, so pair it with `user_agent`. And a page with no `<meta name="viewport">` tag falls back to a 980px layout viewport whatever width you asked for -- standard browser behaviour rather than the setting being ignored.
landscape	query	string	Report the screen as landscape. Requires `viewport`, and accepts the same spellings as `premium`.
touch	query	string	Report the screen as touch capable. Requires `viewport`, and accepts the same spellings as `premium`.
user_agent	query	string	The user agent to send, replacing the one the crawler would otherwise present.
country	query	string	The country to appear to be browsing from, as a two-letter ISO 3166-1 alpha-2 code (`es`, `us`, `de`). Case does not matter. Free, and it works on either proxy pool, so it can be combined with `premium`. Omit it to take whatever exit the pool picks. A country name is not accepted -- `spain` is a `422`, `es` is not. The pool reads the code out of the credentials it is given, so a name would have to be mapped to a code, and that mapping is a list of wrong answers waiting to be picked. Required to use `city`, which narrows a country rather than standing on its own.
city	query	string	The city inside that country to appear to be browsing from, as the pool spells it. Case does not matter. Requires `country` -- a city on its own is a `422`, because the pool reads it as the segment after a country and not as a location of its own. A city doubles what the crawl costs, surcharges included: exits in one city are a far smaller pool than exits in a country. So `country=de&city=berlin` on the cheapest fetch is 2 credits rather than 1, and on a premium rendered fetch carrying a question it is 60 rather than 30. `credits_charged` is always the figure that was taken. Letters, digits, hyphens and underscores only. A multi-word city is written the way the pool writes it -- `new_york`, not `new york` -- and anything carrying a `.` or a `;` is refused: those are the separators the pool parses its own targeting with. The `city` field of the response echoes the city actually targeted, so it is what to check rather than what was sent.
key	query	string	The API key, for callers that cannot set a header. Accepted on this route only -- a key in a url is copied into access logs, proxy logs and `Referer` headers, which is a fair trade for a machine-to-machine endpoint and a bad one for anything else. Ignored when an `Authorization` header is present.

request.sh

curl -X GET "https://app.irmu.com/api/crawl?url=&premium=&js=&images=&ai_query=&adblock=&wait_until=&viewport=&device_scale_factor=&mobile=&landscape=&touch=&user_agent=&country=&city=&key=" \
  -H "Authorization: Bearer $IRMU_API_KEY"

200.json

"string"

Guides using this endpoint

Scraping API overview →

request.js

const res = await fetch("https://app.irmu.com/api/crawl", {
  method: "GET",
  headers: {
    Authorization: `Bearer ${process.env.IRMU_API_KEY}`,
    "Content-Type": "application/json",
  }
});

const data = await res.json();

request.py

import os, requests

res = requests.get(
    "https://app.irmu.com/api/crawl",
    headers={"Authorization": f"Bearer {os.environ['IRMU_API_KEY']}"}
)

print(res.json())

Status	Meaning
200	The crawl finished, and the body is the page itself -- `text/html`, with what it cost in headers, because a caller who asked for a url wants the document rather than a document escaped inside a field. A crawl that finished without producing a page -- which is what an `ai_query` does -- answers the JSON envelope instead, with the answer in `data`. Branch on the content type rather than on the status code. One more rule for the same branch: a result that carries both a page and a picture of one comes back as JSON, with the page in `content`, rather than as a bare document that would drop the picture in silence. Nothing produces that pair today -- it is stated so the rule does not have to be guessed at if something does.
202	Paid for and running. The runner gives up on waiting long before it gives up on crawling, so a slow page comes back as a job id rather than a result -- collect it from `/crawl/{job}`, which is free. The credits are already spent and the crawl already tallied: a `202` is a crawl that is happening, not one that might. With `debuga=1`, a timeout also shows you where the crawl had got to: `debug` carries the url the browser was on, a viewport screenshot and the page source at that moment. Null when there is nothing to show yet -- a crawl still in the queue has not opened a tab. It costs nothing.
401	No key, an unknown key, or a caller that is not an organization -- a signed-in dashboard user reaches this endpoint as a user and is refused here, since a user has no credits to spend.
402	The organization's balance will not cover the crawl. Nothing is charged and nothing is recorded.
422	The request did not validate.
500	The crawler did not deliver. The charge is handed back -- `credits_charged` is `0` and the balance is whole -- and no crawl is counted against the domain, though the attempt is recorded so a customer whose target has started refusing us can see that we tried. Reported as ours rather than as the caller's: the request was valid and paid for, and the crawler is part of this service rather than a third party they chose.
503	The crawl could not be routed. Every crawl goes out through a proxy pool, and the one this crawl's tier uses is not available -- a fault at our end rather than anything wrong with the request. Nothing is charged and nothing is recorded. The crawl is refused rather than sent out directly, because connecting without the pool would put our own address in front of the target under your name. Retrying is reasonable; the same request will work once the pool is back.

Full reference POST /crawl →

Start building with Irmu today

200 free credits every month, no card required. Every API, every integration, one key.

Start free Read the docs