Where the file belongs
The file is normally placed at the root of the host and named robots.txt.
Basic directives
User-agent identifies the crawler. Disallow identifies paths that should not be crawled.
Do not block pages you want crawled
Check that important tools, guides, styles and scripts remain accessible.
Robots.txt and indexing are different
A crawl restriction is not the same as an indexing directive.