# Tells Scanning Robots Where They Are And Are Not Welcome # # User-agent: can also specify by name; "*" is for everyone # Disallow: disallow if this matches first part of requested path # # For now disallow all we can modify this as needed to allow certain crawlers. # User-agent: * Disallow: /cgi-bin Disallow: /cgi_data Disallow: /adm Disallow: /404_error_page Disallow: /banner Disallow: /functions Disallow: /logs Disallow: /orders Disallow: /pictures Disallow: /secure Disallow: /style Disallow: /toppan