# For domain: http://www.zutubi.com # All robots will spider the domain User-agent: * Disallow: # Disallow directory /products/pulse/download/ User-agent: * Disallow: /products/pulse/download/ #Specifically disallow the BecomeBot User-agent: BecomeBot Disallow: / # Specifically disallow the Nutch Bot. User-agent: Nutch Disallow: /