RobotsParseHandler

Handler for directives found in robots.txt. These callbacks are called by ParseRobotsTxt() in the sequence they have been found in the file.

@safe abstract

class RobotsParseHandler {

void HandleRobotsStart();

void HandleRobotsEnd();

void HandleUserAgent(int line_num, string value);

void HandleAllow(int line_num, string value);

void HandleDisallow(int line_num, string value);

void HandleSitemap(int line_num, string value);

void HandleUnknownAction(int line_num, string action, string value);

}

Members

Functions

HandleAllow void HandleAllow(int line_num, string value): Undocumented in source.
HandleDisallow void HandleDisallow(int line_num, string value): Undocumented in source.
HandleRobotsEnd void HandleRobotsEnd(): Undocumented in source.
HandleRobotsStart void HandleRobotsStart(): Undocumented in source.
HandleSitemap void HandleSitemap(int line_num, string value): Undocumented in source.
HandleUnknownAction void HandleUnknownAction(int line_num, string action, string value): Any other unrecognized name/value pairs.
HandleUserAgent void HandleUserAgent(int line_num, string value): Undocumented in source.

Meta

Source

See Implementation

unrobotstxt modules

classes

functions

static functions

extractUserAgent

structs

variables

kHexDigits