Site crawl

Site crawl

Crawl a whole site, audit every page with the WASM engine, and get site-wide fixes: broken links, duplicates, sitemap gaps, thin and orphaned pages, and issues to fix once in your template.

Same host only, robots.txt obeyed, identified as MetaInfoBot. Without a Scrapling service, pages are fetched one by one through this app (slower).

No crawl yet

Start with your homepage. Bigger crawls find more site-wide problems but take longer.