# robots.txt file for http://www.cybergrants.com, effective 11/15/99 # Updated to exclude evil spambots/email harvesters User-agent: * Disallow: /cybergrants/ Disallow: /image/ Disallow: /WMPro/ Disallow: /servlet/ User-agent: ExtractorPro User-agent: Crescent Internet ToolPak HTTP OLE Control v.1.0 # Extractor,WebWeasel,WebMole,Harvester User-agent: Mozilla/2.0 (compatible; NEWT ActiveX; Win32) # Bull'sEyeGold, MaverickII,WebCollector User-agent: CherryPicker/1.0 User-agent: CherryPickerSE/1.0 User-agent: CherryPickerElite/1.0 User-agent: Microsoft URL Control - 5.01.4511 # DynamicWebWizard User-agent: EmailCollector/1.0 User-agent: EmailWolf 1.00 User-agent: NICErsPRO User-agent: EmailSiphon User-agent: WebBandit/2.1 User-agent: WebBandit/3.50 User-agent: Kilroy # OCLC User-agent: Whacker # WebWhacker User-agent: FlashSite User-agent: NetJet User-agent: Snarf User-Agent: TestApp/1.0 libwww/4.1b5 User-Agent: GETWWW-ROBOT/2.0 User-Agent: MemoWeb User-Agent: net.Sweep/v1.2e-T1-CHI User-Agent: FFiNet32.DLL/3.1 libwww/3.1 User-Agent: Javachapman:05/19/19-15:35 User-Agent: LaSCoW-trace/1.0