limepotato/akkoma
1
0
Fork 0
Code Pull requests Activity

Merge pull request 'Rename index for faster database restore' (#455) from ilja/akkoma:rename_index_for_faster_restore into develop

Browse source 
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/455
This commit is contained in:
floatingghost 2023-02-26 21:58:56 +00:00
commit f7211459ef
3 changed files with 29 additions and 22 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
CHANGELOG.md
View file

@ -7,9 +7,11 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
## Unreleased
## Fixed
- Allowed contentMap to be updated on edit
### Changed
- Restoring the database from a dump now goes much faster without need for work-arounds
## 2023.02
### Added

24
docs/docs/administration/backup.md
View file

@ -21,33 +21,15 @@
6. Restore the database schema and akkoma role using either of the following options
* You can use the original `setup_db.psql` if you have it[²]: `sudo -Hu postgres psql -f config/setup_db.psql`.
* Or recreate the database and user yourself (replace the password with the one you find in the config file) `sudo -Hu postgres psql -c "CREATE USER akkoma WITH ENCRYPTED PASSWORD '<database-password-wich-you-can-find-in-your-config-file>'; CREATE DATABASE akkoma OWNER akkoma;"`.
7. Now restore the Akkoma instance's data into the empty database schema[¹][³]: `sudo -Hu postgres pg_restore -d akkoma -v -1 </path/to/backup_location/akkoma.pgdump>`
8. If you installed a newer Akkoma version, you should run `MIX_ENV=prod mix ecto.migrate`[⁴]. This task performs database migrations, if there were any.
7. Now restore the Akkoma instance's data into the empty database schema[¹]: `sudo -Hu postgres pg_restore -d akkoma -v -1 </path/to/backup_location/akkoma.pgdump>`
8. If you installed a newer Akkoma version, you should run `MIX_ENV=prod mix ecto.migrate`[³]. This task performs database migrations, if there were any.
9. Restart the Akkoma service.
10. Run `sudo -Hu postgres vacuumdb --all --analyze-in-stages`. This will quickly generate the statistics so that postgres can properly plan queries.
11. If setting up on a new server configure Nginx by using the `installation/akkoma.nginx` config sample or reference the Akkoma installation guide for your OS which contains the Nginx configuration instructions.
[¹]: We assume the database name and user are both "akkoma". If not, you can find the correct name in your config files.
[²]: You can recreate the `config/setup_db.psql` by running the `mix pleroma.instance gen` task again. You can ignore most of the questions, but make the database user, name, and password the same as found in your backed up config file. This will also create a new `config/generated_config.exs` file which you may delete as it is not needed.
[³]: `pg_restore` will add data before adding indexes. The indexes are added in alphabetical order. There's one index, `activities_visibility_index` which may take a long time because it can't make use of an index that's only added later. You can significantly speed up restoration by skipping this index and add it afterwards. For that, you can do the following (we assume the akkoma.pgdump is in the directory you're running the commands):
```sh
pg_restore -l akkoma.pgdump > db.list
# Comment out the step for creating activities_visibility_index by adding a semi colon at the start of the line
sed -i -E 's/(.*activities_visibility_index.*)/;\1/' db.list
# We restore the database using the db.list list-file
sudo -Hu postgres pg_restore -L db.list -d akkoma -v -1 akkoma.pgdump
# You can see the sql statement with which to create the index using
grep -Eao 'CREATE INDEX activities_visibility_index.*' akkoma.pgdump
# Then create the index manually
# Make sure that the command to create is correct! You never know it has changed since writing this guide
sudo -Hu postgres psql -d pleroma_ynh -c "CREATE INDEX activities_visibility_index ON public.activities USING btree (public.activity_visibility(actor, recipients, data), id DESC NULLS LAST) WHERE ((data ->> 'type'::text) = 'Create'::text);"
```
[⁴]: Prefix with `MIX_ENV=prod` to run it using the production config file.
[³]: Prefix with `MIX_ENV=prod` to run it using the production config file.
## Remove

23
priv/repo/migrations/20230127143303_rename_index_users_ap_id_coalesce_follower_address_index.exs Normal file
View file

@ -0,0 +1,23 @@
defmodule Pleroma.Repo.Migrations.RenameIndexUsersApId_COALESCEFollowerAddressIndex do
alias Pleroma.Repo
use Ecto.Migration
def up do
# By default Postgresql first restores the data and then the indexes when dumping and restoring the database.
# Restoring index activities_visibility_index took a very long time.
# users_ap_id_COALESCE_follower_address_index was later added because having this could speed up the restoration tremendously.
# The problem now is that restoration apparently happens in alphabetical order, so this new index wasn't created yet
# by the time activities_visibility_index needed it.
# There were several work-arounds which included more complex steps during backup/restore.
# By renaming this index, it should be restored first and thus activities_visibility_index can make use of it.
# This speeds up restoration significantly without requiring more complex or unexpected steps from people.
Repo.query!("ALTER INDEX public.\"users_ap_id_COALESCE_follower_address_index\"
RENAME TO \"aa_users_ap_id_COALESCE_follower_address_index\";")
end
def down do
Repo.query!("ALTER INDEX public.\"aa_users_ap_id_COALESCE_follower_address_index\"
RENAME TO \"users_ap_id_COALESCE_follower_address_index\";")
end
end