Browser Recipe blocks

start

Opens the page and gets past bot challenges. Writes: –

• • Always the first block. Cloudflare, DDoS-Guard, EdgeOne, DataDome and PerimeterX challenges and captchas are handled automatically.
• • A 404 ends the crawl with no result.
• • A recipe with only a start block returns the raw HTML of the page instead of JSON.
• • Only source is read from the start block. Page-load options (wait_until, js, adblock, premium, country, city) go in the POST /crawl query string.

{
  "id": "b1",
  "type": "start",
  "details": {
    "type": "url",
    "source": "https://example.com/"
  }
}

wait

Pauses for a number of seconds. Writes: –

{
  "id": "w1",
  "type": "wait",
  "details": {
    "seconds": 2
  }
}

click

Clicks an element. Writes: –

• • If no element matches, the crawl fails. When the element might be missing, put a remove or wait before it.

{
  "id": "c1",
  "type": "click",
  "details": {
    "selector": "button.load-more"
  }
}

input

Types text into an element. Writes: –

• • If no element matches, the crawl fails.

{
  "id": "i1",
  "type": "input",
  "details": {
    "selector": "input[name=q]",
    "text": "running shoes"
  }
}

scroll-to-element

Scrolls an element into view. Writes: –

• • If nothing matches, the block is skipped. Useful for triggering lazy loading, for example by scrolling to the footer.

{
  "id": "s1",
  "type": "scroll-to-element",
  "details": {
    "selector": "footer"
  }
}

remove

Removes elements from the page. Writes: –

• • Use it to get rid of cookie banners and overlays before a click or screenshot.

{
  "id": "r1",
  "type": "remove",
  "details": {
    "selector": "#cookie-banner"
  }
}

paginate

Goes to the next page and runs earlier blocks again. Writes: –

• • Pagination stops when a full pass adds no new rows, so the loop must include an extract block.
• • Use expectedOutput: "list".

{
  "id": "next",
  "type": "paginate",
  "details": {
    "startBlock": "extract-title",
    "selector": "a.next"
  }
}

extract

Reads a value from every element a selector matches. Writes: Row by row

• • A field named price longer than 20 characters is trimmed to the first currency amount found.

{
  "id": "e1",
  "type": "extract",
  "details": {
    "name": "url",
    "parentSelector": ".card",
    "selector": "a.title",
    "property": "href"
  }
}

jsonschema

Reads a schema.org object from JSON-LD or microdata. Writes: Row 0 (replaces it)

• • Searches nested objects. If the JSON-LD object is sparse, missing fields are filled from microdata.

{
  "id": "j1",
  "type": "jsonschema",
  "details": {
    "type": "Product"
  }
}

datalayer

Reads the product from the GTM / GA4 dataLayer. Writes: Row 0

• • Takes the first product from ecommerce.detail.products (Universal Analytics) or ecommerce.items (GA4), plus currency.
• • For GA4 only single-item events count and view_item_list is skipped, so listings aren't mistaken for a product.

{
  "id": "d1",
  "type": "datalayer",
  "details": {
    "type": "ecommerce"
  }
}

extract-variable

Reads a JavaScript global. Writes: Row 0

• • Needs row 0 to exist already: place it after extract or jsonschema.

{
  "id": "v1",
  "type": "extract-variable",
  "details": {
    "variable": "__NEXT_DATA__.props.pageProps",
    "field": "page"
  }
}

add-field

Adds a fixed value. Writes: Row 0

• • Needs row 0 to exist already: place it after extract or jsonschema.

{
  "id": "a1",
  "type": "add-field",
  "details": {
    "field": "source",
    "value": "competitor-a"
  }
}

screenshot

Takes a full-page JPEG screenshot. Writes: data.screenshots

• • Takes no fields: "details": {}.
• • Make it the last block of an "item" recipe: every screenshot the recipe took is returned as base64 in data.screenshots.

{
  "id": "shot",
  "type": "screenshot",
  "details": {}
}

ai

Has your language model extract data or act on the page. Writes: Row 0

• • Uses the language model and key saved in your organization settings (https://irmu.com/docs/dashboard). No key goes in the recipe.
• • Expect POST /crawl to return a job id to poll with GET /crawl/{job} instead of the result.

{
  "id": "ai1",
  "type": "ai",
  "details": {
    "query": "Get the product name, price and availability"
  }
}

chatgpt

Asks ChatGPT and captures the answer. Writes: Row 0 (conversation)

• • Put it after a start block that opens https://chatgpt.com/ with JavaScript on. The answer is returned as structured nodes under conversation.

{
  "id": "g1",
  "type": "chatgpt",
  "details": {
    "prompt": "What are the best web scraping APIs?"
  }
}

booking

Reads a Booking.com hotel page. Writes: Row 0

• • Takes no fields: "details": {}. Returns name, address, rating, price, rooms, facilities and images.
• • Search result pages need extract blocks, or use GET /booking/search.

{
  "id": "bk",
  "type": "booking",
  "details": {}
}

google-ai-overview

Reads Google's AI Overview. Writes: Row 0

• • Accepts the consent dialog first if one appears. A page without an overview is left as it is.
• • Or call GET /google-ai-overview directly.

{
  "id": "aio",
  "type": "google-ai-overview",
  "details": {}
}