|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES All Classes | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||
java.lang.Object | +--websphinx.RobotExclusion
| Constructor Summary | |
RobotExclusion(java.lang.String userAgent)
Make a RobotExclusion object. |
|
| Method Summary | |
void |
clear()
Clear the cache of robots.txt entries. |
boolean |
disallowed(java.net.URL url)
Check whether a URL is disallowed by robots.txt. |
static void |
main(java.lang.String[] argv)
|
| Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
| Constructor Detail |
public RobotExclusion(java.lang.String userAgent)
userAgent - name of the robot using this object, as shown
in the User-Agent header fields of its HTTP requests. Use
null for anonymous robots.| Method Detail |
public boolean disallowed(java.net.URL url)
url - URL to testpublic void clear()
public static void main(java.lang.String[] argv)
throws java.lang.Exception
|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES All Classes | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||