Robots.txt is for blocking search engine bots from accessing part (or whole) of a website.
As it is generally used for preventing search engines to sites, very few SEO experts make full use of it, thinking more pages in Google index from the domain is better, but it is entirely false.
One should block Google from indexing pages that are thin on content as well.