Website and technology
robots.txt
In short and in plain language
The robots.txt is a small file on your website.
It tells search engines which parts of the site to read and which not. It can also point to the list of your pages.
A page can still show up in the results. The file does not reliably stop that.
Definition
The robots.txt is a text file in the root directory of a website that tells crawlers which areas they should fetch and which not. It can also point to the sitemap. Blocking an address in robots.txt does not reliably keep it out of search results.