Apply 1 suggestion(s) to 1 file(s)
This commit is contained in:
parent
e0dee833f2
commit
a4a2d38640
1 changed files with 1 additions and 1 deletions
|
@ -26,7 +26,7 @@ There's a mix tasks to [generate a new robot.txt](../../administration/CLI_tasks
|
|||
|
||||
For more complex things, you can write your own robots.txt to `$static_dir/robots.txt`.
|
||||
|
||||
E.g. if you want to block all crawerls except for [fediverse.newtork](https://fediverse.network/about) you can use
|
||||
E.g. if you want to block all crawlers except for [fediverse.network](https://fediverse.network/about) you can use
|
||||
|
||||
```
|
||||
User-Agent: *
|
||||
|
|
Loading…
Reference in a new issue