For SEO, you must be sure to grant permission against search engine bots or crawlers such as google, yahoo and being while shut out the bad bots. This feature is possible to fulfill your wishes by giving a pair of “UA string” and “Qualification” separated by an applicable rule which can be “:” (pass) or “#” (block).
Syntax and Synopsis
UA string : [ ! ] QualificationUA string # [ ! ] Qualification
UA string
You can specify a part of user agent string (case sensitive). An asterisk “*” matches all user agents.
Qualification
Currently, you can obtain 7 types of qualification listed bellow:
| Qualification | Description |
|---|---|
| FEED | True if the request is the feed url. |
| HOST | True if the result of host name is available. |
| HOST=string | True if the host name includes string. |
| REF=string | True if the HTTP referer includes string. |
| Country code | True if the request comes from the specified country. |
| IP address (CIDR) | True if the IP address is within the specific range. |
| AS number | True if the AS number is matched. |
| * | Always true |
The host name HOST corresponding IP address will be retrieved via DNS reverse lookup which is disabled by default.
Negative operation
A negative operation “!” can be placed just before a qualification. It inverts the meaning of qualification.
DNS reverse lookup
DNS reverse lookup will attempt to fetch a dns PTR record for an IP address. Its cost against the server’s resources is relatively high. For example, the following picture shows the result of 1st and 2nd time which is available at “Installation information” in “Plugin settings” section. It means that the fetched result would be cached in your server.
If “DNS reverse lookup” is disabled, then HOST and HOST=... will always return true. For example, Google:HOST would be converted to Google:*. In this case, only user agent string would be checked. If the converted result is one of *:*, *:!*, *#* and *#!*, it would be eliminated.
Examples
| Sample | Description |
|---|---|
| *:2620:101:4000::/42 | Pass if an IP address is within the range. |
| Google:HOST | Pass if UA includes “Google” and host name is available. |
| Google#!HOST | Block if UA includes “Google” and host name is unavailable. |
| Yandex#* | Block if UA includes “Yandex”. |
| *#HOST=amazonaws | Block all UA if host name includes “amazonaws”. |
bot:HOST appears before xyz-bot#HOST, then all the bots will be passed whose user agent string includes “bot” and its host name is available.