Use Image discovery Discover every image and detect missing ALT text, broken files, and assets larger than 500KB.
Use Robots-aware crawling Respect Allow and Disallow directives and report every URL skipped before request.
Use Complete internal-link crawling Follow the full discoverable link graph without page-count or depth limits.
Use XML sitemap generation Generate indexable XML sitemap files automatically after every successful crawl.
Use Status-code diagnostics Track successful pages, redirects, client errors, server errors, and response time.
Use Secure crawl protection Block private, reserved, local, and link-local targets before each request.