< View all glossary terms

robots.txt

A file used by websites to communicate with search engine crawlers, specifying which pages or sections should not be indexed or accessed by search engines.