RobotsMatcher.AllowedByRobots

Returns true iff 'url' is allowed to be fetched by any member of the "user_agents" vector. 'url' must be %-encoded according to RFC3986.

class RobotsMatcher

bool

AllowedByRobots

(

string robots_body

,

const(string[]) user_agents

,

string url

)

Meta

Source

See Implementation