NAME

WWW::RobotRules::DB_File - Parse robots.txt files using a disk cache

SYNOPSIS

require WWW::RobotRules::DB_File;
require LWP::RobotUA;
#Create a robot useragent that uses a disk caching RobotRules
$ua = WWW::RobotUA->new( 'my-robot/1.0', 'me@foo.com' ,
      WWW::RobotRules::DB_File->new( 'my-robot/1.0', '/path/cachefile' ));
#The just use $ua as usual
$res=$ua->request($req);

DESCRIPTION

This is a subclass of WWW::RobotRules that uses the DB_File package to implement disk caching of robots.txt.

METHODS

This is a subclass of WWW::RobotRules, so it implements the same methods

$rules = WWW::RobotRules::DB_File->new('my-robot/1.0', /path/cachefile)

This is the constructor. The only difference from the original constructor from WWW::RobotRules is that you here has to specify a cache file as well.

SEE ALSO

WWW::RobotRules

AUTHOR

Hakan Ardo <hakan@munin.ub2.lu.se>