Block AI scrapers
This commit is contained in:
parent
1623189556
commit
9ed727bb6c
|
@ -4,6 +4,24 @@ Allow:
|
|||
Disallow: /static/
|
||||
Disallow: /.well-known/webfinger
|
||||
|
||||
User-agent: CCBot
|
||||
Disallow: /
|
||||
|
||||
User-agent: ChatGPT-User
|
||||
Disallow: /
|
||||
|
||||
User-agent: GPTBot
|
||||
Disallow: /
|
||||
|
||||
User-agent: Google-Extended
|
||||
Disallow: /
|
||||
|
||||
User-agent: Omgilibot
|
||||
Disallow: /
|
||||
|
||||
User-agent: FacebookBot
|
||||
Disallow: /
|
||||
|
||||
# Everybody else
|
||||
User-agent: *
|
||||
Allow: /
|
||||
|
|
Loading…
Reference in New Issue