User-agent: * Disallow: /cgi-bin/ Disallow: /cgi/ User-Agent: DISCo Pump, Teleport Pro, WebSnake, Teleport Disallow: / User-Agent: Offline Explorer, MSIECrawler, NetVampire Disallow: / User-Agent: WebZIP, Web-By-Mail, Wget, Reget, GetRight Disallow: /