Skip to content

About

This is a reference site on robots.txt for a small website: what the file does, how to write the rules, how to test it, and the mistakes that hide pages from search, kept deliberately narrow.

Facts on these pages come from published sources, such as makers' documentation, official pages and reference works, and each page links to the ones it used. Every page shows when it was last checked.

There are no ads here, and nothing is for sale.

What you read here is general information. Your own equipment's manual comes first, and where safety or the law calls for a qualified tradesperson, use one.