#============================================================================= # Copyright(c) 2002-2002 by Interwoven Inc. All Rights Reserved. #============================================================================= #***************************************************************************** #>>> ROBOTS.TXT FILE #***************************************************************************** # # $Id$ # Synopsis: keeps things from crawling the site. # # # REFERENCES # ---------- # http://www.robotstxt.org/wc/robots.html -- good entry point # http://www.robotstxt.org/wc/norobots.html -- file syntax spec # # # LAWS OF ROBOTICS # ---------------- # 1. A robot may not injure a human being, or, through inaction, allow # a human to come to harm. # # 2. A robot must obey orders given to him by human beings except where # such orders would conflict with the First Law. # # 3. A robot must protect its own existence as long as such protection # does not conflict with the First or Second Law. # #***************************************************************************** User-Agent: * Allow: / Disallow: /*.css Disallow: /*.js Disallow: /cgi-bin Disallow: /components/ Disallow: /toolkit #*****************************************************************************