󰣨 ymrtech 󰀄 login

Notes · crawlers

1 post tagged crawlers, newest first.

Don't ban the Googlebot
[2026-10-11] / security
Blocking crawlers by User-Agent stops only the crawlers that identify themselves. The durable control is verified source ranges, pinned by hash.
[ read ] crowdsec crawlers filtering false-positives
󰣨 ymrtech@ymrtech | 󰌠 NixOS | 󰍢 UTF-8