# Irmu — full content > Irmu is web data infrastructure for developers: reliable APIs to scrape websites, render JavaScript, capture screenshots, rotate proxies, bypass anti-bot protections and extract structured JSON with AI. Operated by Irmu LLC, 5437 Cove Cir, Naples, FL 34119, United States. Contact: info@irmu.com. Source: https://irmu.com ## Products ### Web Scraping API URL: https://irmu.com/products/scraping-api One endpoint. Any website. No blocks. The Scraping API is the fastest path from a URL to usable markup. Every request is routed through a healthy proxy, fingerprinted like a real browser, retried on failure and returned as clean HTML — with optional JavaScript rendering, geo-targeting and session persistence. Endpoint: `GET https://app.irmu.com/api/scrape` Features: - Rotating proxies: Residential, ISP and datacenter IPs selected automatically per target. - JavaScript rendering: Real Chromium rendering with wait-for-selector and network-idle conditions. - Anti-bot handling: Cloudflare, DataDome, PerimeterX and Akamai challenges solved inline. - Geo-targeting: Country, state and city-level targeting for localized content and pricing. - Sticky sessions: Keep the same IP and cookie jar across a multi-step crawl. - Automatic retries: Failed fetches are retried on fresh IPs and never billed. Pricing: - Standard fetch: 1 credit (Datacenter IP, no JavaScript execution) - Standard fetch + JavaScript: 5 credits (Page rendered in a real browser) - Premium fetch: 10 credits (Residential IP, rarely blocked) - Premium fetch + JavaScript: 25 credits (Residential IP and full rendering) - AI question about a page: 5 credits (Free with your own language model) - Country, region or city targeting: Free (Included on every request) Example request: ```http curl -G https://app.irmu.com/api/scrape \ -H "Authorization: Bearer irmu_sk_live_..." \ --data-urlencode "url=https://news.ycombinator.com" \ --data-urlencode "render=true" \ --data-urlencode "country=us" ``` Example response: ```json { "status": 200, "url": "https://news.ycombinator.com", "credits_used": 5, "resolved_ip_country": "us", "html": "..." } ``` FAQ: - **Do I need to manage proxies?** No. Irmu selects, rotates and retires IPs for you based on live success rates per domain. - **Can I POST forms?** Yes. Send a POST body with method, headers and payload and Irmu will forward it through the proxy layer. - **How do I keep a session?** Pass session_id with any string. Requests sharing the id reuse the same IP and cookies for up to 30 minutes. ### Browser API URL: https://irmu.com/products/browser-api Playwright power without the infrastructure. Run full browser sessions on demand: click, type, scroll, wait, intercept network traffic and evaluate JavaScript — all through JSON instructions or a remote CDP connection you can drive from Playwright and Puppeteer directly. Endpoint: `POST https://app.irmu.com/api/browser` Features: - Action sequences: Describe clicks, inputs, waits and scrolls as a JSON array — no runner to host. - CDP endpoint: Connect Playwright or Puppeteer over WebSocket to a managed Chromium. - Network interception: Capture XHR/fetch payloads to grab data before it reaches the DOM. - Stealth profile: Fingerprints, fonts, canvas and WebGL noise tuned to look human. - Session recording: Optional trace and video of every step for debugging. - Autoscaling: Thousands of concurrent browsers, cold start under 800ms. Pricing: - Standard fetch: 1 credit (Datacenter IP, no JavaScript execution) - Standard fetch + JavaScript: 5 credits (Page rendered in a real browser) - Premium fetch: 10 credits (Residential IP, rarely blocked) - Premium fetch + JavaScript: 25 credits (Residential IP and full rendering) - AI question about a page: 5 credits (Free with your own language model) - Country, region or city targeting: Free (Included on every request) Example request: ```http curl -X POST https://app.irmu.com/api/browser \ -H "Authorization: Bearer irmu_sk_live_..." \ -H "Content-Type: application/json" \ -d '{ "url": "https://app.example.com/login", "actions": [ { "type": "fill", "selector": "#email", "value": "demo@irmu.com" }, { "type": "click", "selector": "button[type=submit]" }, { "type": "waitForSelector", "selector": ".dashboard" } ], "return": ["html", "cookies"] }' ``` Example response: ```json { "status": "completed", "steps": 3, "duration_ms": 2841, "html": "
...", "cookies": [{ "name": "sid", "value": "..." }] } ``` FAQ: - **Can I use Playwright directly?** Yes — connect over CDP with your API key in the query string and drive the browser exactly as you would locally. - **Are downloads supported?** Yes. Files downloaded during a session are uploaded to temporary storage and returned as signed URLs. - **What about logged-in flows?** Provide cookies or storage state on session start, or perform the login as part of the action sequence. ### Screenshot API URL: https://irmu.com/products/screenshot-api Every viewport, every format, one URL. Capture full-page, viewport or element screenshots in PNG, JPEG, WebP or PDF. Block cookie banners and ads, inject CSS, set device emulation and get a CDN-hosted image back in under two seconds. Endpoint: `GET https://app.irmu.com/api/screenshot` Features: - Full-page capture: Lazy-loaded content scrolled and settled before capture. - Device emulation: Presets for iPhone, Pixel, iPad and desktop, or a custom viewport. - Banner blocking: Cookie notices, chat widgets and ads removed automatically. - Custom CSS & JS: Inject styles or scripts to hide, highlight or annotate elements. - PDF export: Print-quality PDFs with headers, footers and page ranges. - CDN delivery: Cached, signed URLs with configurable TTL. Pricing: - Standard fetch: 1 credit (Datacenter IP, no JavaScript execution) - Standard fetch + JavaScript: 5 credits (Page rendered in a real browser) - Premium fetch: 10 credits (Residential IP, rarely blocked) - Premium fetch + JavaScript: 25 credits (Residential IP and full rendering) - AI question about a page: 5 credits (Free with your own language model) - Country, region or city targeting: Free (Included on every request) Example request: ```http curl -G https://app.irmu.com/api/screenshot \ -H "Authorization: Bearer irmu_sk_live_..." \ --data-urlencode "url=https://stripe.com" \ --data-urlencode "full_page=true" \ --data-urlencode "format=webp" \ --data-urlencode "block_banners=true" \ --output stripe.webp ``` Example response: ```json { "url": "https://cdn.irmu.com/s/9f2a1c.webp", "width": 1440, "height": 8210, "format": "webp", "bytes": 412883, "expires_at": "2026-09-01T00:00:00Z" } ``` FAQ: - **Can I screenshot pages behind a login?** Yes. Pass cookies or a storage state blob, or chain the capture onto a Browser API session. - **Do you cache identical requests?** Yes — set cache_ttl to reuse a capture and avoid paying twice for the same view. - **Can I capture a single element?** Provide a selector and Irmu will crop to that element's bounding box. ### AI Extract API URL: https://irmu.com/products/ai-extract-api Describe the data. Skip the selectors. Send a URL and a schema — or just a sentence — and get validated JSON back. AI Extract cleans the DOM, strips boilerplate, runs extraction against a language model and validates the result against your schema before it leaves our infrastructure. Endpoint: `POST https://app.irmu.com/api/extract` Features: - Schema-first output: Provide a JSON Schema and every response is validated against it. - Prompt extraction: No schema? Describe the fields in plain language and Irmu infers one. - Selector-free: Layout changes stop breaking your pipeline overnight. - Boilerplate removal: Nav, footers, ads and scripts stripped before the model sees the page. - Batch mode: Submit thousands of URLs and collect results by webhook. - Confidence scores: Per-field confidence so you can route low-certainty rows to review. Pricing: - Standard fetch: 1 credit (Datacenter IP, no JavaScript execution) - Standard fetch + JavaScript: 5 credits (Page rendered in a real browser) - Premium fetch: 10 credits (Residential IP, rarely blocked) - Premium fetch + JavaScript: 25 credits (Residential IP and full rendering) - AI question about a page: 5 credits (Free with your own language model) - Country, region or city targeting: Free (Included on every request) Example request: ```http curl -X POST https://app.irmu.com/api/extract \ -H "Authorization: Bearer irmu_sk_live_..." \ -H "Content-Type: application/json" \ -d '{ "url": "https://www.amazon.com/dp/B0CHX1W1XY", "schema": { "title": "string", "price": "number", "currency": "string", "rating": "number", "in_stock": "boolean" } }' ``` Example response: ```json { "data": { "title": "Kindle Paperwhite (16 GB)", "price": 149.99, "currency": "USD", "rating": 4.6, "in_stock": true }, "confidence": { "title": 0.99, "price": 0.98, "rating": 0.95 }, "credits_used": 15 } ``` FAQ: - **Which model powers extraction?** Irmu runs a managed extraction stack and tunes the model per content type. You never provide or pay for model keys. - **What if a field is missing?** The field returns null with a zero confidence score rather than a hallucinated value. - **Can I extract from HTML I already have?** Yes — post the html field instead of url and Irmu skips the fetch step. ### Proxy API URL: https://irmu.com/products/proxy-api A clean IP for every request. When you want to keep your own crawler and only fix the network layer, point it at Irmu's proxy endpoint. Rotating or sticky, geo-targeted to city level, with per-request success telemetry you can actually act on. Endpoint: `http://proxy.irmu.com:8080` Features: - Three pool types: Residential, ISP and datacenter, selectable per request. - City-level targeting: 195+ countries with state and city granularity. - Sticky or rotating: Hold an IP for up to 30 minutes or rotate on every connection. - HTTP & SOCKS5: Drop-in support for any client or headless browser. - Unlimited concurrency: No hard connection cap on Pro and Enterprise plans. - Per-domain telemetry: Success rates and latency broken out by target domain. Pricing: - Standard fetch: 1 credit (Datacenter IP, no JavaScript execution) - Standard fetch + JavaScript: 5 credits (Page rendered in a real browser) - Premium fetch: 10 credits (Residential IP, rarely blocked) - Premium fetch + JavaScript: 25 credits (Residential IP and full rendering) - AI question about a page: 5 credits (Free with your own language model) - Country, region or city targeting: Free (Included on every request) Example request: ```http curl -x "http://customer-acme-country-us-session-42:irmu_sk_live_...@proxy.irmu.com:8080" \ https://httpbin.org/ip ``` Example response: ```json { "origin": "72.14.201.88" } ``` FAQ: - **Can I use the proxy with Playwright?** Yes — pass it in the launch options proxy config; the stealth layer still applies. - **How is bandwidth measured?** Compressed bytes transferred through the tunnel, reported live in the dashboard. - **Are there IP allowlists?** Yes. Restrict proxy credentials to specific source IPs from the dashboard. ## Solutions ### Lead Generation URL: https://irmu.com/solutions/lead-generation Problem: Buying lead lists means paying for stale records that half your competitors already emailed. Building them in-house means maintaining scrapers for a dozen sources that each block you differently, plus a normalization layer that nobody wants to own. Solution: Irmu turns lead sourcing into a single API surface. Pull business listings from Google Maps, company profiles from LinkedIn and Crunchbase, and review signals from Trustpilot and Yelp — then hand the raw pages to AI Extract with your CRM schema so records land already shaped for import. Workflow: - Define the territory: Query Google Maps by category and geography to enumerate every candidate business. - Enrich each record: Fan out to LinkedIn, Crunchbase and the company site for headcount, funding and tech signals. - Structure with AI: Run AI Extract with your CRM field schema so every record arrives normalized and validated. - Score and sync: Filter on confidence and freshness, then push to HubSpot, Salesforce or your warehouse via webhook. ### Market Research URL: https://irmu.com/solutions/market-research Problem: Market research decks go stale the week they ship. Analysts spend most of their time collecting and cleaning rather than interpreting, and one-off pulls make trend analysis impossible. Solution: Irmu gives research teams a repeatable collection layer: schedule crawls across marketplaces, app stores, review sites and social platforms, extract the same schema every run, and build a time series instead of a snapshot. Workflow: - Pick your sources: Marketplaces, app stores, review platforms and social channels relevant to the category. - Schedule collection: Cron-driven jobs pull the same URLs and search terms on a fixed cadence. - Extract a stable schema: AI Extract enforces the same fields run after run even as page layouts change. - Analyze the series: Load into your warehouse and chart share of shelf, pricing bands and sentiment over time. ### Price Monitoring URL: https://irmu.com/solutions/price-monitoring Problem: Pricing pages are the most heavily defended part of the web. Rate limits, geo-specific prices and personalized offers mean a naive crawler sees a price no real customer ever saw. Solution: Irmu fetches pricing exactly as a shopper in the target market would: residential IPs pinned to the right city, real browser rendering for dynamic prices, and AI extraction that handles promotions, tiered pricing and variant matrices. Workflow: - Map the SKUs: Resolve competitor product URLs per marketplace and region. - Fetch as a local shopper: Geo-targeted residential proxies plus rendering to trigger the same price logic. - Extract prices and offers: Base price, sale price, shipping, availability and variant-level breakdowns. - Alert on change: Webhook fires the moment a monitored SKU moves beyond your threshold. ### SEO Monitoring URL: https://irmu.com/solutions/seo-monitoring Problem: Rank tracking tools give you a number without the context. You cannot see which features occupied the SERP, what the competing page actually said, or how the answer changed in AI assistants. Solution: Irmu collects the whole result page — organic positions, ads, People Also Ask, local packs, shopping units — plus the competing pages themselves, and now the answers AI assistants give for the same query. Workflow: - Define keyword sets: Group by intent, market and language. - Collect SERPs: Geo-accurate result pages including every SERP feature, on your schedule. - Crawl the competition: Fetch ranking pages and extract headings, word counts, schema and internal links. - Watch AI answers: Query ChatGPT, Perplexity and Claude for the same prompts and track brand mentions. ### AI Training Data URL: https://irmu.com/solutions/ai-training Problem: Training and retrieval pipelines need volume, freshness and cleanliness at the same time. Raw crawls are full of boilerplate, duplicates and markup that quietly poison embeddings. Solution: Irmu handles collection and cleaning in one pass: high-throughput fetching with automatic unblocking, boilerplate stripping, markdown conversion and optional schema extraction, delivered straight to object storage. Workflow: - Seed the crawl: Provide domains, sitemaps or search queries as entry points. - Fetch at volume: High-volume crawling with automatic retry and dedupe. - Clean and convert: Boilerplate removal, markdown output and language detection. - Deliver to storage: Stream results to S3, GCS or your warehouse in JSONL. ### Competitive Intelligence URL: https://irmu.com/solutions/competitive-intelligence Problem: Competitive insight is scattered across changelogs, pricing pages, ad libraries, job posts and review sites. By the time someone compiles it manually, the interesting part already happened. Solution: Irmu monitors every competitor surface on a schedule and diffs the results, so your team gets a feed of actual changes instead of a folder of screenshots. Workflow: - List the surfaces: Pricing, product, changelog, careers, ad library, review profiles. - Snapshot on a cadence: Screenshot API for visual diffs, Scraping API for text. - Diff and summarize: AI Extract turns raw diffs into a readable change summary. - Route to the team: Slack or email digest whenever something material moves. ### Travel Data URL: https://irmu.com/solutions/travel-data Problem: Travel inventory is dynamic, geo-personalized and defended by some of the most aggressive anti-bot stacks on the web. Prices change by device, currency, session and point of sale. Solution: Irmu reproduces the exact search context — market, currency, device, dates — through geo-pinned residential IPs and real browsers, so the rates you collect match the rates travellers see. Workflow: - Define search grids: Property or route, date windows, occupancy and point of sale. - Fetch with real context: Locale-correct rendering with the right currency and device profile. - Extract rates: Nightly rate, taxes, fees, cancellation terms and availability. - Track and compare: Build rate parity dashboards and demand curves over time. ### Real Estate URL: https://irmu.com/solutions/real-estate Problem: Property portals expose rich data but restrict bulk access, and listing structures differ enough between portals that a single normalized model is real work to maintain. Solution: Irmu collects listings, agents, price changes and days-on-market from the major portals and normalizes them into one schema you control, updated as often as your model needs. Workflow: - Cover the market: Enumerate listings by geography, property type and price band. - Collect detail pages: Full listing pages including photos, features and agent details. - Normalize fields: One schema across portals with unit and currency normalization. - Track over time: Price cuts, relistings and time-to-sale as a continuous series. ### Ecommerce URL: https://irmu.com/solutions/ecommerce Problem: Retail teams need to know how their products appear on channels they do not control: wrong images, missing bullets, unauthorized sellers and out-of-stock listings that quietly kill conversion. Solution: Irmu audits your catalog across marketplaces and retailer sites, extracts every content element and flags divergence from your source of truth. Workflow: - Match your catalog: Map internal SKUs to marketplace listings. - Audit every listing: Title, bullets, images, A+ content, price, buy box and stock. - Diff against truth: Flag content mismatches and unauthorized sellers automatically. - Close the loop: Push exceptions into the channel team's queue with screenshots attached. ### Recruiting URL: https://irmu.com/solutions/recruiting Problem: Compensation benchmarks age fast and hiring signals hide in job boards, review sites and company pages that all block bulk access. Solution: Irmu collects postings, salaries and employer reviews continuously so talent teams can price roles against the live market instead of last year's survey. Workflow: - Track postings: Roles, locations, seniority and posted dates across major boards. - Extract compensation: Ranges, equity language and benefits parsed from free text. - Add employer signal: Review scores and themes from Glassdoor and Indeed. - Benchmark: Compare your offers against live market bands by role and geography. ### Brand Monitoring URL: https://irmu.com/solutions/brand-monitoring Problem: Brand conversation now spans social platforms, review sites, forums and AI assistants — and the assistants are increasingly the first answer a customer sees. Solution: Irmu collects mentions across social and review platforms and queries AI assistants directly, so you can measure share of voice on human and machine surfaces at once. Workflow: - Define the watchlist: Brand terms, products, executives and competitors. - Collect mentions: Social platforms, forums and review sites on a schedule. - Query the assistants: Ask ChatGPT, Claude and Perplexity your category questions and log the answers. - Measure and alert: Sentiment, share of voice and alerting on negative spikes. ## Integrations ### Google Maps API URL: https://irmu.com/integrations/google-maps-api Category: Search & Maps. Target: Google Maps. Local business listings, reviews, ratings and contact details at scale. Data points: - business_name: Listed name of the place - category: Primary Google category, e.g. Dental clinic - address: Full formatted address - phone: Public phone number when listed - website: Outbound website URL - rating: Average star rating - review_count: Total number of reviews - price_level: Reported price band - opening_hours: Weekly opening hours - coordinates: Latitude and longitude - reviews: Review text, author, rating and date - photos: Listing photo URLs Use cases: - Local lead generation: Enumerate every business in a category and geography with contact details attached. - Franchise and territory audits: Verify listing accuracy, hours and photos across hundreds of locations. - Local SEO tracking: Monitor local pack position, review velocity and rating movement week over week. - Site selection: Combine density, ratings and review volume to score candidate locations. ### Google Search API URL: https://irmu.com/integrations/google-search-api Category: Search & Maps. Target: Google Search. Full SERP data including organic results, ads, PAA and features. Data points: - organic_results: Position, title, URL and snippet - ads: Paid results with advertiser and copy - people_also_ask: Related questions and answers - featured_snippet: Answer box content and source - local_pack: Map results shown for local intent - shopping_results: Product listing ads - related_searches: Query refinements - total_results: Reported result count Use cases: - Rank tracking: Track positions by keyword, country, language and device. - SERP feature analysis: Measure how much of the fold you actually own. - Ad intelligence: Watch competitor paid copy and landing pages. - Content gap research: Mine PAA and related searches for topic clusters. ### Google Shopping API URL: https://irmu.com/integrations/google-shopping-api Category: Search & Maps. Target: Google Shopping. Product listings, merchant offers and price comparisons. Data points: - product_title: Listed product name - price: Offer price - currency: Offer currency - merchant: Selling merchant - rating: Product rating - shipping: Shipping cost or terms - product_id: Google product identifier - condition: New, used or refurbished Use cases: - Price benchmarking: Compare your price against every listed merchant per market. - Merchant discovery: Find unauthorized or grey-market sellers of your products. - Assortment analysis: See which competitor SKUs are listed in which countries. - Feed QA: Verify your own listings render with the right price and imagery. ### Google Images API URL: https://irmu.com/integrations/google-images-api Category: Search & Maps. Target: Google Images. Image results with source pages, dimensions and thumbnails. Data points: - image_url: Full-resolution image URL - thumbnail: Thumbnail URL - source_page: Page hosting the image - title: Image title text - dimensions: Width and height - domain: Source domain Use cases: - Counterfeit detection: Find unauthorized uses of your product imagery. - Visual dataset building: Collect labelled image sets for model training. - Brand asset audits: Check which images rank for your brand terms. - Creative research: Survey visual conventions in a category. ### Meta Ads Library API URL: https://irmu.com/integrations/meta-ads-library-api Category: Social. Target: the Meta Ads Library. Every active ad creative, spend range and targeting signal from Facebook and Instagram. Data points: - ad_id: Library identifier for the creative - page_name: Advertiser page running the ad - ad_creative_body: Primary ad copy - ad_creative_link_title: Headline of the linked destination - media_urls: Image and video creative URLs - start_date: First date the ad ran - platforms: Facebook, Instagram, Messenger, Audience Network - spend_range: Reported spend band where disclosed - impressions_range: Reported impression band where disclosed - landing_url: Destination URL including UTM parameters Use cases: - Creative intelligence: Track which hooks and formats competitors keep running — longevity signals performance. - Launch detection: Spot new positioning or products the moment ads go live. - Funnel teardown: Follow landing URLs to map competitor offers and page structure. - Category benchmarking: Measure advertiser volume and creative churn across a whole vertical. ### ChatGPT API URL: https://irmu.com/integrations/chatgpt-api Category: AI Assistants. Target: ChatGPT. Capture how ChatGPT answers your category questions, with citations. Data points: - prompt: The query submitted - answer: Full assistant response text - citations: Cited source URLs and titles - brands_mentioned: Brands named in the answer - position: Order in which each brand appears - sentiment: Tone of each brand mention - captured_at: Timestamp of the run Use cases: - AI visibility tracking: Measure whether you are mentioned for high-intent category prompts. - Citation monitoring: See which of your pages assistants actually cite. - Misinformation alerts: Catch wrong claims about pricing or features early. - Competitive share of answer: Track mention share against competitors over time. ### Claude API URL: https://irmu.com/integrations/claude-api Category: AI Assistants. Target: Claude. Track Claude's answers, citations and brand mentions over time. Data points: - prompt: The query submitted - answer: Full response text - citations: Referenced sources - brands_mentioned: Brands named - sentiment: Tone of mention - captured_at: Run timestamp Use cases: - Cross-assistant comparison: Diff answers between Claude, ChatGPT and Perplexity. - Source influence mapping: Find which third-party pages shape your category's answers. - Positioning research: See how assistants describe your differentiators unprompted. - Compliance review: Log answers about regulated products for the record. ### Perplexity API URL: https://irmu.com/integrations/perplexity-api Category: AI Assistants. Target: Perplexity. Answer engine results with the full citation graph. Data points: - prompt: Query submitted - answer: Synthesized answer - citations: Ordered source list - follow_ups: Suggested follow-up questions - brands_mentioned: Brands named in the answer Use cases: - Citation SEO: Identify the exact pages you need to influence or outrank. - Answer monitoring: Track answer drift for critical queries. - Content strategy: Mine follow-up questions for the next article. - Competitor tracking: See who gets cited when you do not. ### LinkedIn API URL: https://irmu.com/integrations/linkedin-api Category: Business & Careers. Target: LinkedIn. Public company pages, job posts and profile signals for B2B workflows. Data points: - company_name: Public company name - industry: Listed industry - headcount: Employee count band - headquarters: Primary location - description: Public about text - specialties: Listed specialties - job_postings: Open roles with title and location - website: Linked company website - follower_count: Page followers Use cases: - Account enrichment: Fill firmographic gaps in your CRM automatically. - Hiring-intent signals: Growing engineering headcount is a buying signal for developer tools. - Territory planning: Size an addressable market by industry and headcount band. - Competitor org tracking: Watch team growth in the functions that matter to you. ### Amazon API URL: https://irmu.com/integrations/amazon-api Category: Marketplaces. Target: Amazon. Product detail, pricing, buy box, reviews and search rankings. Data points: - title: Product title - price: Current price - list_price: Strikethrough price - currency: Marketplace currency - rating: Average star rating - review_count: Number of ratings - buybox_seller: Seller currently winning the buy box - in_stock: Availability state - bullets: Feature bullet points - images: Gallery image URLs - asin: Amazon identifier - best_seller_rank: Category rank - reviews: Review text, rating, date and verified flag - variants: Size, colour and style variations Use cases: - Competitive repricing: Track competitor prices per marketplace and react within the hour. - Buy box monitoring: Alert when you lose the buy box or an unauthorized seller appears. - Listing content audits: Verify titles, bullets and imagery match your source of truth. - Review mining: Extract themes and defects from thousands of reviews with AI. - Keyword rank tracking: Monitor organic and sponsored position for target search terms. ### Zillow API URL: https://irmu.com/integrations/zillow-api Category: Business & Careers. Target: Zillow. Property listings, price history and market metrics. Data points: - address: Full property address - price: List price - beds: Bedroom count - baths: Bathroom count - sqft: Interior square footage - lot_size: Lot area - year_built: Construction year - days_on_market: Time since listing - price_history: Prior listings and price changes - agent: Listing agent and brokerage Use cases: - Automated valuation: Train and refresh AVMs on current inventory. - Investor screening: Filter markets by price-per-sqft and days-on-market trends. - Market reporting: Publish monthly metrics by metro and property type. - Lead sourcing: Identify agents with growing listing volume. ### Airbnb API URL: https://irmu.com/integrations/airbnb-api Category: Travel & Places. Target: Airbnb. Short-term rental listings, nightly rates, availability and reviews. Data points: - listing_title: Listing headline - price_per_night: Nightly rate for the searched dates - currency: Displayed currency - rating: Guest rating - review_count: Number of reviews - bedrooms: Bedroom count - host_type: Superhost or standard - amenities: Listed amenities - availability: Available dates in the window Use cases: - Revenue management: Benchmark nightly rates against comparable supply. - Supply analysis: Track listing counts and professionalization by neighbourhood. - Investment screening: Estimate yield from occupancy and rate data. - Regulatory research: Monitor supply response to local policy changes. ### Booking.com API URL: https://irmu.com/integrations/booking-com-api Category: Travel & Places. Target: Booking.com. Hotel rates, availability, taxes and cancellation terms by market. Data points: - hotel_name: Property name - price_total: Total stay price - price_per_night: Nightly rate - currency: Point-of-sale currency - taxes_fees: Disclosed taxes and fees - rating: Guest review score - free_cancellation: Cancellation policy flag - rooms_left: Scarcity indicator - location: District and coordinates Use cases: - Rate parity monitoring: Detect where OTAs undercut your direct rate. - Compset benchmarking: Track competitor pricing by date and occupancy. - Demand forecasting: Use rate and scarcity movement as a demand proxy. - Channel audits: Verify inventory and content across markets. ### Tripadvisor API URL: https://irmu.com/integrations/tripadvisor-api Category: Travel & Places. Target: Tripadvisor. Reviews, rankings and attraction data for hospitality intelligence. Data points: - venue_name: Property or attraction name - category: Hotel, restaurant or attraction - rating: Average rating - review_count: Number of reviews - ranking: Rank within its city category - price_band: Reported price level - reviews: Review text, rating, language and date - amenities: Listed features Use cases: - Reputation benchmarking: Compare review velocity and sentiment against your compset. - Menu and amenity research: Extract structured detail from thousands of venues. - Destination analysis: Rank neighbourhoods by supply quality and demand. - Fake review detection: Feed patterns into your own detection models. ### Yelp API URL: https://irmu.com/integrations/yelp-api Category: Travel & Places. Target: Yelp. Local business profiles, reviews and category rankings. Data points: - business_name: Business name - categories: Yelp categories - rating: Average rating - review_count: Review total - address: Street address - phone: Listed phone - price_range: Price indicator - hours: Opening hours - reviews: Full review text with dates Use cases: - Local lead generation: Source businesses by category, rating and review volume. - Reputation monitoring: Track your locations and your competitors' daily. - Market density analysis: Measure supply saturation by neighbourhood. - Review theme extraction: Turn free text into structured complaint categories. ### Instagram API URL: https://irmu.com/integrations/instagram-api Category: Social. Target: Instagram. Public profiles, posts, engagement metrics and hashtag feeds. Data points: - username: Public handle - follower_count: Followers - post_count: Total public posts - bio: Profile description - posts: Caption, likes, comments and timestamp - engagement_rate: Computed engagement rate - hashtags: Hashtags used Use cases: - Influencer vetting: Verify real engagement before you sign a contract. - Campaign tracking: Measure branded hashtag reach across creators. - Trend detection: Spot rising formats and audio in your category. - Competitor content audit: Benchmark posting cadence and engagement. ### Facebook API URL: https://irmu.com/integrations/facebook-api Category: Social. Target: Facebook. Public pages, posts, events and engagement data. Data points: - page_name: Page name - category: Page category - likes: Page likes - followers: Page followers - posts: Post text, reactions and timestamps - contact_info: Public phone, email and site - events: Upcoming public events Use cases: - Local data enrichment: Fill contact and hours gaps for local businesses. - Brand monitoring: Track public posts and engagement across markets. - Event intelligence: Collect event listings by category and city. - Competitor messaging: Archive competitor announcements over time. ### X API URL: https://irmu.com/integrations/x-api Category: Social. Target: X (Twitter). Public posts, profiles and conversation threads. Data points: - handle: Account handle - display_name: Profile name - followers: Follower count - posts: Post text, timestamp and metrics - replies: Thread replies - likes: Like count - reposts: Repost count Use cases: - Social listening: Track brand and category conversation in real time. - Crisis detection: Alert on negative sentiment spikes. - Research datasets: Collect public discourse for academic and model work. - Competitor announcements: Archive launches and outage notices. ### Reddit API URL: https://irmu.com/integrations/reddit-api Category: Social. Target: Reddit. Posts, comments and subreddit activity for research and RAG. Data points: - title: Post title - subreddit: Community - score: Upvote score - author: Public username - created_at: Post timestamp - body: Post text - comments: Comment tree with scores - flair: Post flair Use cases: - Voice-of-customer research: Mine complaints and feature requests in your category. - RAG corpora: Build retrieval sets grounded in real discussion. - Trend spotting: Track emerging tools and terminology. - Brand monitoring: Catch mentions before they reach mainstream channels. ### TikTok API URL: https://irmu.com/integrations/tiktok-api Category: Social. Target: TikTok. Public videos, creator profiles, sounds and hashtag performance. Data points: - username: Creator handle - followers: Follower count - video_url: Public video URL - caption: Video caption - views: View count - likes: Like count - sound: Audio track used - hashtags: Hashtags on the post Use cases: - Creator vetting: Compare claimed reach to observed performance. - Trend detection: Track rising sounds and formats early. - Campaign measurement: Aggregate branded hashtag performance. - Product discovery: Spot products going viral before they hit marketplaces. ### YouTube API URL: https://irmu.com/integrations/youtube-api Category: Social. Target: YouTube. Videos, channels, transcripts and comment threads. Data points: - title: Video title - channel: Channel name - views: View count - likes: Like count - published_at: Publish date - description: Video description - transcript: Full caption text - comments: Top comments with authors Use cases: - Transcript mining: Turn hours of video into searchable text for RAG. - Competitor content analysis: Track topics, cadence and performance. - Sentiment analysis: Extract themes from comment threads. - Training corpora: Build domain-specific spoken-language datasets. ### GitHub API URL: https://irmu.com/integrations/github-api Category: Developer & Apps. Target: GitHub. Repositories, releases, contributors and dependency signals. Data points: - repo: Repository full name - stars: Star count - forks: Fork count - language: Primary language - last_commit: Most recent commit date - topics: Repository topics - contributors: Top contributors - dependencies: Declared dependencies Use cases: - Developer-tool market maps: Track adoption across an entire ecosystem. - Technographic enrichment: See which companies use which stacks publicly. - Release monitoring: Alert when a dependency ships a breaking change. - Talent sourcing: Identify active contributors in a domain. ### Shopify API URL: https://irmu.com/integrations/shopify-api Category: Marketplaces. Target: Shopify stores. Catalog, pricing and inventory data from any public storefront. Data points: - product_title: Product name - price: Current price - compare_at_price: Original price - variants: Variant options and prices - inventory: Availability by variant - images: Product imagery - collections: Collections the product belongs to - vendor: Brand or vendor field Use cases: - Assortment tracking: Watch competitor catalogs expand and contract. - Promotion detection: Catch discounts the moment compare-at prices change. - Inventory signals: Infer demand from stock movement. - DTC market research: Survey pricing across hundreds of brands in a category. ### Indeed API URL: https://irmu.com/integrations/indeed-api Category: Business & Careers. Target: Indeed. Job postings, salary ranges and hiring velocity. Data points: - title: Job title - company: Hiring company - location: Job location - salary_min: Lower salary bound - salary_max: Upper salary bound - remote: Remote flag - posted_at: Posting date - description: Full posting text Use cases: - Compensation benchmarking: Price roles against live market bands. - Hiring-intent signals: Score accounts by role-specific hiring activity. - Technographics: Extract tools and stacks named in postings. - Labour market research: Track demand by function and geography. ### Glassdoor API URL: https://irmu.com/integrations/glassdoor-api Category: Business & Careers. Target: Glassdoor. Employer reviews, ratings and salary reports. Data points: - company: Employer name - overall_rating: Average rating - review_count: Number of reviews - ceo_approval: CEO approval rate - pros: Common positive themes - cons: Common negative themes - salaries: Reported ranges by role Use cases: - Employer benchmarking: Compare your ratings against competing employers. - Retention risk analysis: Track sentiment shifts after reorgs or layoffs. - Compensation research: Cross-check posted ranges with reported pay. - Due diligence: Assess culture risk in acquisition targets. ### Crunchbase API URL: https://irmu.com/integrations/crunchbase-api Category: Business & Careers. Target: Crunchbase. Company profiles, funding rounds and investor networks. Data points: - company: Company name - description: Company summary - founded: Founding year - headcount: Employee band - total_funding: Total raised - last_round: Most recent round type and date - investors: Named investors - industries: Industry tags Use cases: - Funding-triggered outbound: Reach out the week a round is announced. - Market mapping: Chart a category by funding stage and geography. - Investor research: Track portfolio activity and co-investment patterns. - Account scoring: Weight accounts by capital raised and stage. ### Trustpilot API URL: https://irmu.com/integrations/trustpilot-api Category: Business & Careers. Target: Trustpilot. Business reviews, ratings and response behaviour. Data points: - business: Reviewed business - trust_score: Overall score - review_count: Total reviews - rating_distribution: Star breakdown - reviews: Review text, rating and date - response_rate: Business reply rate - verified: Verified review flag Use cases: - Reputation benchmarking: Compare scores and velocity across a category. - Churn theme analysis: Extract recurring complaints from competitor reviews. - Review response auditing: Measure how quickly competitors reply. - Conversion research: Correlate review signals with market share. ### Steam API URL: https://irmu.com/integrations/steam-api Category: Developer & Apps. Target: Steam. Game listings, pricing, player counts and reviews. Data points: - title: Game title - price: Current price - discount: Active discount percentage - review_score: Review summary - review_count: Number of reviews - tags: Community tags - release_date: Launch date - developer: Studio name Use cases: - Pricing strategy: Benchmark price and discount cadence by genre. - Launch analysis: Track review velocity in the first 30 days. - Genre research: Measure supply and sentiment by tag. - Sale monitoring: Detect seasonal discount patterns. ### App Store API URL: https://irmu.com/integrations/app-store-api Category: Developer & Apps. Target: the Apple App Store. App listings, rankings, ratings and review text. Data points: - app_name: App title - developer: Publisher - rating: Average rating - review_count: Rating count - category_rank: Rank in category - price: Price or in-app purchase range - last_updated: Latest release date - reviews: Review text and version - whats_new: Release notes Use cases: - ASO tracking: Monitor keyword rank and category position. - Competitor release monitoring: Diff release notes to see what shipped. - Review mining: Cluster complaints by app version. - Market sizing: Estimate category depth and quality. ### Google Play API URL: https://irmu.com/integrations/google-play-api Category: Developer & Apps. Target: Google Play. Android app metadata, installs, ratings and reviews. Data points: - app_name: App title - developer: Publisher - installs: Install band - rating: Average rating - review_count: Rating count - last_updated: Latest update - reviews: Review text and device - in_app_purchases: IAP price range Use cases: - Install-base estimation: Model category share from install bands. - Competitive ASO: Track ranking and metadata changes. - Quality monitoring: Detect rating drops after releases. - Localization research: Compare listings across country storefronts. ## Comparisons ### Irmu vs ScrapingDog URL: https://irmu.com/compare/irmu-vs-scrapingdog ScrapingDog is a straightforward scraping endpoint with attractive entry pricing and a set of prebuilt scrapers for popular targets. If all you need is HTML from moderately protected pages at low volume, it does the job. Irmu covers the same ground but adds the parts teams reach for next: a real browser API, screenshots, schema-validated AI extraction and a proxy endpoint you can point existing crawlers at — all behind one key and one quota. | Feature | Irmu | ScrapingDog | | --- | --- | --- | | Unified API surface | Scrape, browser, screenshot, extract, proxy | Scraping + prebuilt endpoints | | AI structured extraction | Built in, schema-validated | Not offered | | Headless browser control | REST actions + remote CDP | Rendering flag only | | Screenshot & PDF | Included | Basic | | Residential proxy pool | Residential, ISP and datacenter, city targeting | Country targeting | | Billing model | Successful requests only | Counts some failures | | Free tier | 200 credits/month, forever | Trial credits | | Support | Email, on every plan | Paid tiers only | Verdict: Choose ScrapingDog for cheap, simple HTML fetching at modest volume. Choose Irmu when you need rendering, browser control and structured extraction to live behind the same API as your fetching. ### Irmu vs ScrapingBee URL: https://irmu.com/compare/irmu-vs-scrapingbee ScrapingBee built its reputation on reliable JavaScript rendering and clear documentation, and it remains a solid choice for teams whose main problem is client-side rendered pages. Irmu matches the rendering story and extends it with remote browser control over CDP, batch AI extraction with confidence scores, screenshot and PDF output, and a standalone proxy endpoint for crawlers you already run. | Feature | Irmu | ScrapingBee | | --- | --- | --- | | Unified API surface | Scrape, browser, screenshot, extract, proxy | Scraping + screenshots | | AI structured extraction | Built in, schema-validated | Basic AI extraction | | Headless browser control | REST actions + remote CDP | JS scenarios, no CDP | | Screenshot & PDF | Included | Basic | | Residential proxy pool | Residential, ISP and datacenter, city targeting | Country targeting | | Billing model | Successful requests only | Counts some failures | | Free tier | 200 credits/month, forever | Trial credits | | Support | Email, on every plan | Paid tiers only | Verdict: ScrapingBee is a good rendering API. Irmu is the better fit when extraction, browser automation and proxying need to share one platform, quota and observability surface. ### Irmu vs ZenRows URL: https://irmu.com/compare/irmu-vs-zenrows ZenRows is strong on unblocking, with a well-engineered anti-bot layer and a growing set of prebuilt scrapers for major targets. Irmu treats unblocking as table stakes and focuses on what happens after the fetch: schema-validated extraction, screenshots, scheduling with webhook delivery, and a browser API for flows that need real interaction. | Feature | Irmu | ZenRows | | --- | --- | --- | | Unified API surface | Scrape, browser, screenshot, extract, proxy | Scraping + scrapers | | AI structured extraction | Built in, schema-validated | Auto-parse for select sites | | Headless browser control | REST actions + remote CDP | Rendering + limited actions | | Screenshot & PDF | Included | Basic | | Residential proxy pool | Residential, ISP and datacenter, city targeting | Country targeting | | Billing model | Successful requests only | Counts some failures | | Free tier | 200 credits/month, forever | Trial credits | | Support | Email, on every plan | Paid tiers only | Verdict: Both clear the same defences. Pick Irmu when structured output, scheduling and browser automation matter as much as getting past the wall. ### Irmu vs Bright Data URL: https://irmu.com/compare/irmu-vs-bright-data Bright Data operates one of the largest proxy networks in the world with an enormous product catalog, extensive compliance tooling and enterprise contracts to match. Irmu targets the team that wants that level of reliability without the procurement cycle, the dashboard complexity or the minimum commitments — a single API key, transparent credits and a free tier that never expires. | Feature | Irmu | Bright Data | | --- | --- | --- | | Unified API surface | Scrape, browser, screenshot, extract, proxy | Very broad, many separate products | | AI structured extraction | Built in, schema-validated | Available in select products | | Headless browser control | REST actions + remote CDP | Scraping Browser product | | Screenshot & PDF | Included | Basic | | Residential proxy pool | Residential, ISP and datacenter, city targeting | 150M+ IPs, city targeting | | Billing model | Successful requests only | Per GB, plan minimums | | Free tier | 200 credits/month, forever | Trial credits | | Support | Email, on every plan | Account-managed | Verdict: Bright Data suits large organizations that need bespoke contracts and the widest possible IP footprint. Irmu suits engineering teams that want to ship this week and scale without renegotiating. ### Irmu vs Apify URL: https://irmu.com/compare/irmu-vs-apify Apify's model is a marketplace of community-built actors plus a hosting platform to run them. It is flexible and covers a huge number of niche targets. That flexibility comes with variance: actors differ in quality, maintenance and output shape. Irmu is first-party infrastructure with one consistent request and response contract across every target and product. | Feature | Irmu | Apify | | --- | --- | --- | | Unified API surface | Scrape, browser, screenshot, extract, proxy | Platform + community actors | | AI structured extraction | Built in, schema-validated | Depends on the actor | | Headless browser control | REST actions + remote CDP | Full, self-authored | | Screenshot & PDF | Included | Basic | | Residential proxy pool | Residential, ISP and datacenter, city targeting | Country targeting | | Billing model | Successful requests only | Compute units + proxy | | Free tier | 200 credits/month, forever | Trial credits | | Support | Email, on every plan | Plan-dependent | Verdict: Apify wins when you want to write and host arbitrary crawler code. Irmu wins when you want a consistent, supported API contract you can build a product on. ### Irmu vs Oxylabs URL: https://irmu.com/compare/irmu-vs-oxylabs Oxylabs is a mature proxy provider with strong enterprise credentials, a large residential network and dedicated scraper APIs for major verticals. Irmu offers comparable unblocking with a materially simpler surface: five products, one key, credits instead of per-GB negotiation, and AI extraction included rather than sold as a separate scraper product per target. | Feature | Irmu | Oxylabs | | --- | --- | --- | | Unified API surface | Scrape, browser, screenshot, extract, proxy | Proxies + vertical scraper APIs | | AI structured extraction | Built in, schema-validated | Per-vertical parsers | | Headless browser control | REST actions + remote CDP | Web Unblocker | | Screenshot & PDF | Included | Basic | | Residential proxy pool | Residential, ISP and datacenter, city targeting | 100M+ IPs, city targeting | | Billing model | Successful requests only | Per GB, plan minimums | | Free tier | 200 credits/month, forever | Trial | | Support | Email, on every plan | Account-managed | Verdict: Oxylabs is a strong enterprise proxy vendor. Irmu is the better default when you want general-purpose extraction rather than a separate parser product per target site. ## Blog ### How to Scrape Google Maps (2026 Guide) URL: https://irmu.com/blog/how-to-scrape-google-maps Guides · 2026-08-12 · Irmu Engineering Collect business listings, ratings and reviews from Google Maps reliably — without maintaining a browser farm. #### Why Google Maps is hard to scrape Google Maps renders results progressively inside a virtualized list, loads detail panels over XHR, and applies aggressive rate limiting per IP and per session. A naive HTTP request returns a shell with no listings in it, and a naive headless browser gets challenged within a few dozen requests. The official Places API solves reliability but introduces different limits: capped result sets, a small number of returned reviews and pricing that grows quickly with coverage. For territory mapping or lead generation you usually need more rows than it will give you. #### The approach that works Render the search URL in a real browser, scroll the results panel until the list stops growing, then extract each listing from the settled DOM. Rotate residential IPs geographically close to the searched area so results match what a local user sees. With Irmu, all of that is one request: rendering, scrolling, proxy selection and extraction happen server-side, and you describe the output shape instead of writing selectors. ```python import requests res = requests.post( "https://app.irmu.com/api/extract", headers={"Authorization": "Bearer irmu_sk_live_..."}, json={ "url": "https://www.google.com/maps/search/dental+clinics+in+austin", "country": "us", "schema": { "business_name": "string", "address": "string", "phone": "string", "rating": "number", "review_count": "number", }, }, ) for row in res.json()["data"]: print(row["business_name"], row["rating"]) ``` #### Getting reviews as well as listings Review text lives behind a second interaction: opening the place panel and expanding the review list. Point the same extract call at the place URL and request a reviews array; Irmu performs the expansion before extraction. For continuous monitoring, schedule the job rather than looping client-side. Scheduled runs deliver by webhook and share the same retry semantics as ad-hoc requests. #### Staying on the right side of the rules Collect public business information, not personal data. Respect the volume you actually need, cache aggressively, and read Google's terms for your specific use case. Irmu's Acceptable Use Policy prohibits circumventing authentication and unlawful personal-data collection. ### Scrape Amazon Product Data with Python URL: https://irmu.com/blog/scrape-amazon-with-python Guides · 2026-08-05 · Irmu Engineering A practical walkthrough for pulling prices, buy box, stock and reviews from Amazon at scale. #### What makes Amazon different Amazon serves different markup to different visitors. Price, buy box winner and availability vary by delivery location, session history and device, so a request from a datacenter IP in the wrong country can return a price no customer would ever see. It also mixes server-rendered and client-rendered sections, and rotates layout variants for A/B tests. Selector-based parsers break constantly for this reason alone. #### Fetching the page correctly Use a residential IP in the target marketplace's country, render the page, and let the request retry on soft blocks. In Python that is a single call. ```python import requests API = "https://app.irmu.com/api" HEADERS = {"Authorization": "Bearer irmu_sk_live_..."} def product(asin: str, country: str = "us"): res = requests.post( f"{API}/extract", headers=HEADERS, json={ "url": f"https://www.amazon.com/dp/{asin}", "country": country, "render": True, "schema": { "title": "string", "price": "number", "currency": "string", "rating": "number", "review_count": "number", "buybox_seller": "string", "in_stock": "boolean", }, }, timeout=90, ) res.raise_for_status() return res.json()["data"] print(product("B0CHX1W1XY")) ``` #### Scaling to a catalog Once one ASIN works, submit the rest in batch. Batch extraction accepts thousands of URLs and posts results to your webhook as they complete, which avoids holding open connections and lets you process results as a stream. Track credits per successful record rather than per request — failed fetches are retried and never billed, so your unit economics stay stable even on bad days. ```python res = requests.post( f"{API}/extract/batch", headers=HEADERS, json={ "urls": [f"https://www.amazon.com/dp/{a}" for a in asins], "schema": {"title": "string", "price": "number", "in_stock": "boolean"}, "webhook": "https://hooks.acme.com/irmu", }, ) print(res.json()["batch_id"]) ``` #### Reviews and content audits Ask for a reviews array with text, rating, date and verified flag, then run your own clustering to surface defect themes. For content audits, extract bullets and image URLs and diff them against your PIM export on a schedule. ### Cloudflare Bypass: What Actually Works in 2026 URL: https://irmu.com/blog/cloudflare-bypass-guide Engineering · 2026-07-28 · Irmu Engineering A technical look at Turnstile, TLS fingerprinting and why most open-source bypasses stopped working. #### What Cloudflare actually checks Modern bot detection is a scoring system, not a single gate. TLS handshake fingerprints (JA3/JA4), HTTP/2 frame ordering, header casing and order, canvas and WebGL entropy, timing of input events, and IP reputation all feed a score that decides whether you see the page, a challenge, or a block. This is why swapping in a random user agent stopped working years ago: the user agent is one of the weakest signals in the stack, and a mismatch between it and everything else is itself a strong bot signal. #### Why open-source patches decay Patched browser builds and stealth plugins work until the detection vendor adds a check for the patch. The lifecycle of a public bypass is typically weeks, and the failure mode is silent: you keep getting 200 responses that contain a challenge page instead of content. Any serious pipeline needs response validation — assert that the content you expected is present, not merely that the status code was 200. ```javascript const { html, status } = await irmu.scrape({ url, render: true }); // Validate content, not just status if (status !== 200 || !html.includes('data-product-id')) { throw new Error("Challenge page or layout change detected"); } ``` #### The infrastructure answer Sustainable access comes from matching the whole profile: consistent TLS and HTTP fingerprints, residential IPs with clean reputation in the right geography, realistic interaction timing, and challenge solving when a challenge does appear. That is maintenance work with no end date, which is precisely the argument for buying it. Irmu runs the fingerprint and IP-reputation layer as a service and validates responses before returning them, so a challenge page is treated as a failure and retried rather than billed to you as a success. ### Playwright vs Puppeteer for Web Scraping URL: https://irmu.com/blog/playwright-vs-puppeteer Comparisons · 2026-07-19 · Irmu Engineering Two mature browser automation libraries, and how to decide which one your crawler should use. #### The short answer Use Playwright for new projects. It supports Chromium, Firefox and WebKit through one API, has better auto-waiting, first-class network interception and official bindings for Python, .NET and Java as well as Node. Puppeteer remains excellent if you are Chromium-only, already invested, and want the smallest possible dependency footprint. #### Where they differ in practice Auto-waiting is the biggest day-to-day difference: Playwright waits for actionability before interacting, which eliminates most of the arbitrary sleeps that make Puppeteer scripts flaky. Browser contexts are also cheaper to isolate, which matters when you run many parallel sessions with distinct cookie jars. For scraping specifically, Playwright's route interception makes it easy to capture the JSON a page fetches rather than parsing rendered HTML — usually the more stable extraction path. ```javascript await page.route("**/api/products*", async (route) => { const response = await route.fetch(); const json = await response.json(); products.push(...json.items); // structured data, no DOM parsing await route.fulfill({ response }); }); ``` #### The part neither one solves Both libraries drive a browser. Neither gives you clean IPs, fingerprint consistency, challenge solving, autoscaling or a retry budget — which is where nearly all scraping cost and on-call pain actually lives. Running your own farm is viable at small scale. Past a few hundred thousand sessions a month, connecting your existing Playwright code to managed browsers over CDP usually costs less than the engineering time it replaces. ### Extracting Structured Data from HTML with LLMs URL: https://irmu.com/blog/extract-structured-data-using-gpt AI · 2026-07-08 · Irmu Engineering When language-model extraction beats CSS selectors, when it doesn't, and how to keep it accurate and cheap. #### The case against selectors Selectors encode a page's current DOM structure into your codebase. Every redesign, A/B test and localization variant is a potential silent break, and the failure usually shows up as nulls in a dashboard days later. Language-model extraction reads the page the way a person does, so cosmetic changes do not matter. The trade-off is cost, latency and the need for validation. #### Make it cheap: clean before you extract Most of a page is navigation, scripts, tracking and footer boilerplate. Stripping those before extraction typically removes 80–95% of the tokens and improves accuracy, because the model has less to be distracted by. Convert to a compact representation — cleaned HTML or markdown — and keep only the region likely to contain your fields when you can identify it cheaply. ```javascript const { data, confidence } = await irmu.extract({ url: "https://example.com/product/42", schema: { name: "string", price: "number", currency: "string", availability: "string", }, }); if (confidence.price < 0.8) await queueForReview(data); ``` #### Make it reliable: schema and confidence Always validate against a schema. A typed contract turns a plausible-looking wrong answer into a caught error, and it lets you fail loudly instead of writing garbage into your warehouse. Per-field confidence lets you route the uncertain minority to human review while the rest flows through automatically. In practice, teams see 95–98% straight-through processing with a small review queue for the rest. #### When selectors still win If you scrape one stable page shape millions of times a day and own the monitoring for it, selectors are cheaper. The moment you are covering many sites, or a site that changes often, extraction wins on total cost of ownership even at a higher per-request price. ### How to Rotate Residential Proxies Properly URL: https://irmu.com/blog/rotate-residential-proxies Engineering · 2026-06-24 · Irmu Engineering Session strategy, geo-targeting and the mistakes that get clean IP pools burned. #### Rotation is not the goal — trust is Rotating every request looks like the safest default and often is not. Many sites treat a session that changes IP mid-flow as a stronger anomaly than one that keeps the same address for a plausible amount of time. The right unit of rotation is the logical session: one IP per user journey, rotated between journeys. #### Match the geography to the content Prices, availability, language and even layout vary by location. If you are collecting German prices, use a German IP with a German locale and Accept-Language header. Mismatched signals are both a data-quality problem and a detection signal. ```bash curl -x "http://customer-acme-country-de-city-berlin-session-a91:KEY@proxy.irmu.com:8080" \ -H "Accept-Language: de-DE,de;q=0.9" \ https://www.example.de/product/42 ``` #### How pools get burned Hammering one target from a narrow subnet, ignoring 429s, retrying instantly on failure and reusing an IP after it has been challenged all degrade pool quality — for you and for everyone else on it. Back off exponentially, cap concurrency per domain, and retire IPs that fail rather than retrying them. Irmu does this automatically and exposes per-domain success telemetry so you can see it working.