< View all glossary terms

crawler

A crawler, or web spider, is an automated program used by search engines to browse and index web pages, gathering information to help rank content in search engine results.