WWW::RobotRules::DB_File(3)                User Contributed Perl Documentation


NAME
       WWW::RobotRules::DB_File - Parse robots.txt files using a disk cache

SYNOPSIS
           require WWW::RobotRules::DB_File;
           require LWP::RobotUA;

           #Create a robot useragent that uses a disk caching RobotRules
           $ua = WWW::RobotUA->new( 'my-robot/1.0', 'me@foo.com' ,
                 WWW::RobotRules::DB_File->new( 'my-robot/1.0', '/path/cachefile' ));

           #The just use $ua as usual
           $res=$ua->request($req);

DESCRIPTION
       This is a subclass of WWW::RobotRules that uses the DB_File package to
       implement disk caching of robots.txt.

METHODS
       This is a subclass of WWW::RobotRules, so it implements the same
       methods

       $rules = WWW::RobotRules::DB_File->new('my-robot/1.0', /path/cachefile)
           This is the constructor. The only difference from the original
           constructor from WWW::RobotRules is that you here has to specify a
           cache file as well.

SEE ALSO
       WWW::RobotRules

AUTHOR
       Hakan Ardo <hakan@munin.ub2.lu.se>

perl v5.42.2                      2026-05-23       WWW::RobotRules::DB_File(3)