RobotsTxtParser

Undocumented in source.

package @safe

struct RobotsTxtParser {

alias Key = ParsedRobotsKey;

void Parse();

}

Members

Aliases

Key alias Key = ParsedRobotsKey: Undocumented in source.

Functions

Parse void Parse(): Undocumented in source. Be warned that the author may not have intended to support it.

Meta

Source

See Implementation

unrobotstxt modules

classes

functions

static functions

extractUserAgent

structs

variables

kHexDigits