unrobotstxt ~master (2020-07-03T06:00:54Z)
Dub
Repo
RobotsMatcher.user_agents_
unrobotstxt
RobotsMatcher
Undocumented in source.
class
RobotsMatcher
package
const
(
string
)[]
user_agents_
;
Meta
Source
See Implementation
unrobotstxt
RobotsMatcher
constructors
this
functions
AllowedByRobots
HandleAllow
HandleDisallow
HandleRobotsEnd
HandleRobotsStart
HandleSitemap
HandleUnknownAction
HandleUserAgent
InitUserAgentsAndPath
OneAgentAllowedByRobots
disallow
disallow_ignore_global
ever_seen_specific_agent
matching_line
seen_any_agent
static functions
IsValidUserAgentToObey
structs
Match
MatchHierarchy
variables
allow_
disallow_
ever_seen_specific_agent_
match_strategy_
path_
seen_global_agent_
seen_separator_
seen_specific_agent_
user_agents_