From API key to production crawl
Short pages, runnable examples, no ceremony. Start at the quickstart and branch out.
Authenticate
Every request carries a bearer token. Keys are created in the dashboard and can be scoped per environment.
export IRMU_API_KEY="irmu_sk_live_..."
curl -G https://app.irmu.com/api/crawl \
-H "Authorization: Bearer $IRMU_API_KEY" \
--data-urlencode "url=https://example.com"Make your first request
Generated from the live OpenAPI spec, so it always matches the current surface.
curl -X GET "https://app.irmu.com/api/crawl?url=&premium=&js=&images=&ai_query=&adblock=&wait_until=&viewport=&device_scale_factor=&mobile=&landscape=&touch=&user_agent=&country=&city=&key=" \
-H "Authorization: Bearer $IRMU_API_KEY"Read the response
A page comes back as text/html with the credit headers set. A crawl that produces no page — an ai_query — answers a JSON envelope with the answer in data instead, so branch on the content type rather than the status code. A slow page returns 202 with a job_id you collect from /crawl/{job} for free.
"string"Handling errors
Errors are typed and stable. Retry 429 (you exceeded your plan concurrency) with exponential backoff, and retry 502 a couple of times — Irmu could not serve the request internally, so it was not billed. Note that a 404, 500 or anti-bot response from the target is a successful request and is billed.
Do not retry 400 or 401: the request or the key is wrong and retrying will fail identically. 402 means the credit pool is exhausted, so top up or wait for the reset instead of hammering the endpoint.
The ideas behind every parameter
The handful of ideas that explain every parameter and every bill.
Credits & billing
Everything is priced in credits, and only successful requests are billed. If a fetch fails after our internal retries, you are not charged for it.
Cost per request depends on what you asked for: plain fetches are cheapest, JavaScript rendering and premium geo-routing cost more. The response includes credits_used so you can attribute spend per job.
Retries & idempotency
Irmu retries transient failures server-side — blocked responses, timeouts, proxy hiccups — before returning anything to you. What reaches your code is either a usable response or a typed error.
Reads are idempotent: re-sending the same request returns the same shape and bills again only if it succeeds. Make your own retry loop bounded and jittered so a bad target does not burn your daily budget.
Rate limits & concurrency
Limits are expressed as concurrent requests, not requests per second: Free 1, Lite 5, Standard 50, Pro 100. Enterprise plans go higher.
Going over your concurrency returns 429. Cap your worker pool at your plan's number rather than relying on retries, and add a per-domain cap of your own so one slow target cannot starve the rest of the queue.
Sessions & cookies
Sticky sessions are not part of the API yet. Each crawl is independent, so treat every request as a fresh visitor rather than a step in a logged-in flow.
For paginated targets, request each page by URL and keep country constant so the results stay in one market.
Rendering JavaScript
JavaScript rendering is on by default: the page loads in a real browser and you get the DOM after execution instead of the initial HTML shell. Control when the capture happens with wait_until — networkidle0 by default, or domcontentloaded for pages that hold a connection open.
Rendering is slower and costs more credits, so send js=false for static pages. A quick test: fetch the page with js=false, and if the field you need is missing from the HTML, leave rendering on for that route.
Geo-targeting
Pass a country code to route the request through an IP in that country. Use it for localized pricing, regional search results and geo-fenced content. Geotargeting costs no extra credits.
Combine country with premium=true when a target only serves real residential visitors from that market.
Endpoints
Every operation the API exposes, with parameters, responses and a runnable example.
Crawl
Crawl a url
Fetch a single URL through a headless browser. The request is charged to the calling organization and tallied against the URL's host for the day. The response returns either the fetched document directly or a JSON envelope containing cost metadata and any AI-generated answer.
When to use it: Use this endpoint for one-off page retrieval when you need rendered content, proxy routing, or AI extraction. For bulk or asynchronous workloads, prefer the job-based flow. Use premium for hostile targets, disable js only when you know the page is server-rendered, and set wait_until to domcontentloaded or load for pages that keep persistent connections open.
Notes
- The response format branches on content type:
text/htmlfor a bare document,application/jsonwhenai_queryis present or a screenshot accompanies the page. Do not assume JSON from status alone. js=trueis the default; disabling it on a client-rendered site returns an empty page and still charges credits. Verify the target serves meaningful HTML before turning it off.countryrequires an ISO 3166-1 alpha-2 code (es, notspain);cityrequirescountry, doubles the cost, and must match the pool's spelling exactly (new_york, notnew york).
curl -X GET "https://app.irmu.com/api/crawl?url=&premium=&js=&images=&ai_query=&adblock=&wait_until=&viewport=&device_scale_factor=&mobile=&landscape=&touch=&user_agent=&country=&city=&key=" \
-H "Authorization: Bearer $IRMU_API_KEY"Run a recipe
Runs a custom recipe—a JSON array of blocks executed in order against a page. Unlike GET /crawl, you define the blocks, enabling form submission, pagination, field extraction, and other multi-step interactions. Query options (premium, country, js, etc.) control the browser session; the request body contains only the recipe.
When to use it: Use when GET /crawl is insufficient—any crawl requiring clicks, text input, multi-page navigation, or structured data extraction beyond a single page load.
- First block is overridden:
blocks[0].details.sourcesupplies the starting URL only;waitUntil,adblock,javascript,images,viewport,userAgent, and proxy settings are derived from query parameters, not your block definition. - Body fields are strict: Any field outside
blocks,name, orexpectedOutputreturns422; place all options in the query string. - Flat pricing: 30 credits regardless of block count,
premium,country, orcity—cost is per browser session, not per-request complexity.
curl -X POST "https://app.irmu.com/api/crawl?premium=&js=&images=&adblock=&wait_until=&viewport=&device_scale_factor=&mobile=&landscape=&touch=&user_agent=&country=&city=&key=" \
-H "Authorization: Bearer $IRMU_API_KEY" \
-H "Content-Type: application/json" \
-d '{ "name": "Products, page 2", "expectedOutput": "list", "blocks": [ { "type": "start", "details": { "type": "url", "source": "https://example.com/products" } }, { "id": "rows", "type": "extract", "details": { "selector": ".product .title", "name": "title", "property": "textContent" } }, { "type": "paginate", "details": { "selector": ".pagination .next", "startBlock": "rows" } } ] }'Collect a crawl
Retrieves the result of an asynchronous crawl job that previously returned 202 Accepted. Use this endpoint after /crawl indicates a job is still running and you need to poll for completion.
When to use it: Call when /crawl responds with 202 and you need the final result, or when you want to monitor a long-running crawl's progress without consuming credits. Suitable for any polling strategy; no rate limits apply.
- Authentication:
Authorizationheader only. Query parameter?key=is rejected. - Privacy by design: Other organizations' job IDs return
404, identical to nonexistent IDs. Job IDs are scoped to your organization prefix. - No credit headers:
200responses omitcredits_chargedand credit headers; collection is always free regardless of outcome.
curl -X GET "https://app.irmu.com/api/crawl/{job}" \
-H "Authorization: Bearer $IRMU_API_KEY"ChatGPT
Ask ChatGPT
Sends a prompt to ChatGPT through a browser automation and returns the response as structured JSON nodes—paragraphs, headings, lists, tables, product lists—each preserving citations. The crawler types the prompt into ChatGPT's web interface, waits for the streamed reply, and parses it. No OpenAI API key is required; this drives the public website directly.
When to use it. Use when you need ChatGPT's output with citation metadata intact, or when you lack API access to the underlying model. For simple text extraction from static pages, /crawl is cheaper and faster. This endpoint is priced flat at 30 credits regardless of proxy options.
Notes
- Prompts are typed character-by-character: the 1000-character limit is a hard ceiling, and long prompts consume significant wait time before generation begins.
- A 202 response means the job is still running; collect it from
/crawl/{job}, but expect the answer nested underdata[0].conversationrather than top-levelconversation. - Failed or unanswered prompts refund credits (
credits_charged: 0), though the attempt is still logged againstchatgpt.com.
curl -X GET "https://app.irmu.com/api/chatgpt?prompt=&premium=&country=&city=&key=" \
-H "Authorization: Bearer $IRMU_API_KEY"Screenshot
Screenshot a url
Captures a full-page screenshot of a URL and returns it as a base64-encoded JPEG. Uses the same request parameters as /crawl — same URL, same fetch options, same ai_query — but returns a photographic rendering of the entire scroll height rather than HTML.
When to use it: Use when you need to visually preserve or inspect a page exactly as a browser renders it, including full scroll height, rather than extracting its markup. Useful for archival, visual regression, compliance capture, or any workflow where the rendered appearance matters more than the underlying structure. The flat 30-credit pricing makes costs predictable regardless of proxy tier, geolocation, or AI query usage.
Notes
imagesdefaults totrue(opposite of/crawl), since pages photographed without images show empty boxes where assets belong.- Response payloads grow large: long pages run to several megabytes, and base64 encoding adds ~33% overhead; consider constraining
viewportand reviewing timeouts for tall targets. - Asynchronous jobs return
202— collect completed screenshots from/crawl/{job}underdata.screenshots[0], not the top-levelscreenshotfield used by synchronous200responses.
curl -X GET "https://app.irmu.com/api/screenshot?url=&premium=&js=&images=&ai_query=&adblock=&wait_until=&viewport=&device_scale_factor=&mobile=&landscape=&touch=&user_agent=&country=&city=&key=" \
-H "Authorization: Bearer $IRMU_API_KEY"Handle failures explicitly
Target responses — including 404s, 500s and anti-bot blocks — are returned to you and billed as successful requests. Typed errors below are Irmu-side or coverage failures and are not billed.
Operating safely
What to watch, what to cap and how to keep keys safe once you are past the first crawl.
Observability
The dashboard reports success rate and usage per API key, and credit consumption broken down by day, website and key. Tagging jobs with their own key is the cheapest way to get per-pipeline telemetry without building it yourself.
Watch success rate per site rather than in aggregate — a single target changing its defences is invisible in a global number.
Cost control
Set a daily credit limit in organization settings. It is the most your organization may spend in a day; a crawl that would go over it is refused until midnight. Leave it empty for no limit.
For crawls that ask a question about a page, plug in your own language model in settings — extraction then runs on your model and no additional credits are charged.
Security best practices
Keep keys in your secret manager, never in source control or client-side code. Rotate on a schedule and immediately after anyone with access leaves.
Because keys are unlimited and individually revocable, prefer many narrow keys over one shared key: revoking a compromised key should never require a coordinated redeploy.
What you get once you sign up
Unlimited API keys with per-key telemetry, team invites, plan and billing, credit usage by day, site and key, plus daily spend limits and your own extraction model.
Go deeper
Start building with Irmu today
200 free credits every month, no card required. Every API, every integration, one key.