unrobotstxt.test

Undocumented in source.

Members

Classes

RobotsStatsReporter
class RobotsStatsReporter
Undocumented in source.

Functions

IsUserAgentAllowed
bool IsUserAgentAllowed(string robotstxt, string useragent, string url)
Undocumented in source. Be warned that the author may not have intended to support it.
TestEscape
void TestEscape(string url, string expected)
Undocumented in source. Be warned that the author may not have intended to support it.
TestPath
void TestPath(string url, string expected_path)
Undocumented in source. Be warned that the author may not have intended to support it.

Meta