Operational meaning
A crawler requests URLs, reads responses, and may follow links according to its purpose and policy. Technical crawls reveal observable structure, but the findings still require interpretation and a defined sample.
Common misunderstanding
Crawling does not authorize bypassing authentication, access controls, provider terms, or privacy expectations.
References
No external reference is claimed for this plain-language definition.