Use Image discoveryDiscover every image and detect missing ALT text, broken files, and assets larger than 500KB.
Use Robots-aware crawlingRespect Allow and Disallow directives and report every URL skipped before request.
Use Complete internal-link crawlingFollow the full discoverable link graph without page-count or depth limits.
Use XML sitemap generationGenerate indexable XML sitemap files automatically after every successful crawl.
Use Status-code diagnosticsTrack successful pages, redirects, client errors, server errors, and response time.
Use Secure crawl protectionBlock private, reserved, local, and link-local targets before each request.