ParsedRobotsKey

A robots.txt has lines of key/value pairs. A ParsedRobotsKey represents a key. This class can parse a text-representation (including common typos) and represent them as an enumeration which allows for faster processing afterwards. For unparsable keys, the original string representation is kept.

package @safe

struct ParsedRobotsKey {

enum KeyType;

void Parse(string key);

KeyType type();

string GetUnknownText();

}

Members

Enums

KeyType enum KeyType: Undocumented in source.

Functions

GetUnknownText string GetUnknownText(): If this is an unknown key, get the text.
Parse void Parse(string key): Parse given key text. Does not copy the text, so the text_key must stay valid for the object's life-time or the next Parse() call.
type KeyType type(): Returns the type of key.

ParsedRobotsKey

Members

Enums

Functions

Meta

Source