User-agent: * # directed to all spiders, Disallow: /cgi-bin/sources Disallow: /bin Disallow: /cgi-bin Disallow: /no_robots Disallow: /styles Disallow: /logs Disallow: .css