Free tool
URL extractor
Paste HTML, Markdown or plain text and pull out every URL. Deduplicated, resolved against a base URL, filtered by domain and file type, exportable as CSV.
Filters
HTML, Markdown or text input
0 URLs
0 URLs · 0 domains · deduplicatedRuns entirely in your browser — nothing is uploaded.
Code examples
Do the same thing in your own pipeline
The tool above parses in the browser. Here is the equivalent extraction with the standard parser in each language.
# pip install beautifulsoup4
from urllib.parse import urljoin, urlparse
from bs4 import BeautifulSoup
def extract_urls(html: str, base: str) -> list[str]:
soup = BeautifulSoup(html, "html.parser")
found = []
for tag, attr in (("a", "href"), ("img", "src"), ("script", "src"), ("link", "href")):
for el in soup.find_all(tag):
value = el.get(attr)
if value:
found.append(urljoin(base, value))
seen, out = set(), []
for url in found:
parsed = urlparse(url)
if parsed.scheme not in ("http", "https"):
continue
key = url.rstrip("/").lower()
if key not in seen:
seen.add(key)
out.append(url)
return out
for url in extract_urls('<a href="/docs">Docs</a><img src="a.png">', "https://example.com/"):
print(url)How it works
From messy input to a clean URL list
- Three input modes — HTML, Markdown and plain text — with auto-detection based on what you paste.
- Relative links are resolved against the base URL you provide, so /docs becomes an absolute, shareable URL.
- Deduplication collapses repeats, optionally ignoring trailing slashes and tracking query parameters.
- Filters narrow by domain include/exclude lists, internal versus external, and file-type groups or your own extensions.
- Export the result as a plain list, a CSV with domain, extension, source and anchor text, or JSON for a downstream job.
With Irmu
When you also need to fetch the page
Irmu returns the page HTML — rendering, proxies and retries included. Extract and export links from the response.
crawl_to_urls.py
# pip install requests beautifulsoup4
import csv, os, requests
from urllib.parse import urljoin
from bs4 import BeautifulSoup
target = "https://example.com/blog"
resp = requests.get(
"https://app.irmu.com/api/crawl",
headers={"Authorization": f"Bearer {os.environ['IRMU_API_KEY']}"},
params={"url": target, "render": "true"},
timeout=60,
)
resp.raise_for_status()
soup = BeautifulSoup(resp.text, "html.parser")
links = {urljoin(target, a["href"]) for a in soup.select("a[href]")}
with open("urls.csv", "w", newline="") as f:
writer = csv.writer(f)
writer.writerow(["url"])
writer.writerows([[u] for u in sorted(links)])FAQ
Questions about this tool
Turn any URL into clean data
Fetch, render and extract with one API. 1,000 free credits every month, no card required.