limepotato/akkoma
1
0
Fork 0
Code Pull requests Activity
15612 commits 27 branches 107 tags 315 MiB
Commit graph

591 commits

Author SHA1 Message Date
Ekaterina Vaartis
cc3319ac1d Make chunk size configurable 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis
bac70a2bc1 Implement suggestions from the Meilisearch MR 
- Index unlisted posts
- Move version check outside of the streaming and only do it once
- Use a PUT request instead of checking manually if there is need to insert
- Add error handling, sort of
 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis
0769f06bd1 Style fixes 2022-06-29 20:49:00 +01:00
Ekaterina Vaartis
86971fceaa Support reindexing meilisearch >=0.24.0 
It has has a different error code key
 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis
9e7d7ebd48 Add a reindex option 
Signed-off-by: Ekaterina Vaartis <vaartis@kotobank.ch>
 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis
5ed1759091 Reorder ranking rules for (maybe) better results 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis
d1079f1aa3 Add the meilisearch.stats command 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
d5cc272a91 Add a message with a count of posts to index 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
dbf556cdcf Implement meilisearch auth 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
5360cc1097 Make indexing logs rewrite themselves 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
0cf3654907 Rework task indexing to share code with the main module 
The code in the main module now scrubs new posts too
 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
117f525fd6 Adjust content indexing to skip more unneeded stuff 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
14ef6ce80f Mark only content as searchable for meilisearch 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
52a872432d Make the chunk size smaller 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
a586ce0ddd Use content instead of source and scrub it 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
c3a04166a0 Tweak search ordering to hopefully return newer results 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
7b3701e6b9 Make meilisearch sort on publish date converted to unix time 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
abf82a63ec Make the indexing batch differently and more, show number indexed 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
e5ac2ffa07 Ensure only indexing public posts and implement clearing and delete 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
41db5c8653 Add logging to milisiearch index and make it use desc(id) 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
e961cf2689 Initial meilisearch implementation, doesn't delete posts yet 2022-06-29 20:48:28 +01:00
FloatingGhost
12a171844b Merge remote-tracking branch 'upstream/develop' into develop 2021-12-30 18:05:22 +00:00
FloatingGhost
415d9dd6f6 Merge remote-tracking branch 'origin/feature/elasticsearch' into develop 2021-12-16 16:13:41 +00:00
Alex Gleason
e1b89fe3aa
Merge remote-tracking branch 'origin/develop' into live-dashboard 2021-12-15 19:05:36 -05:00
Alex Gleason
29d80b39f2
Add Phoenix LiveDashboard 
Co-authored-by: Egor Kislitsyn <egor@kislitsyn.com>
 2021-12-15 19:05:27 -05:00
sadposter
8378c3a8bc extra cool 2021-12-14 14:48:24 +00:00
FloatingGhost
9d5d964bf8 merge 2021-12-14 14:28:15 +00:00
sadposter
cc4c5f22f4 fix inbound federation 2021-12-14 14:27:42 +00:00
sadposter
8f78cb3050 Merge branch 'develop' of gitssh.ihatebeinga.live:IHBAGang/pleroma into develop 2021-12-14 14:01:09 +00:00
sadposter
5d901c337e fix inbound federation 2021-12-14 13:58:16 +00:00
FloatingGhost
703f53c08f Search through users and hashtags as well 2021-12-14 13:55:14 +00:00
FloatingGhost
a85bf5929b add hashtag indexing 2021-12-14 13:55:14 +00:00
FloatingGhost
6ca70833f5 add user import 2021-12-14 13:55:14 +00:00
FloatingGhost
c03e8d46e8 Search through users and hashtags as well 2021-12-14 13:53:46 +00:00
FloatingGhost
8547cefaff add hashtag indexing 2021-12-14 12:37:10 +00:00
FloatingGhost
6bb19454fd add user import 2021-12-13 20:27:35 +00:00
sadposter
e835212a7d enforce visibility 2021-12-12 20:12:06 +00:00
sadposter
23c758bbe7 enforce visibility 2021-12-12 18:25:20 +00:00
FloatingGhost
de60b1f3c1 make search provider configurable 2021-12-12 17:35:35 +00:00
FloatingGhost
9002e5155b Add import functionality 2021-12-12 17:35:35 +00:00
FloatingGhost
ed3a866f94 make search provider configurable 2021-12-12 17:23:44 +00:00
FloatingGhost
c684265585 Add import functionality 2021-12-11 17:36:49 +00:00
sadposter
6a6260f64e re-enable deactivated check 2021-12-06 12:03:16 +00:00
sadposter
ca2bd8b72b remove deactivated_users call 2021-12-06 10:18:16 +00:00
sadposter
bb54720778 add debug 2021-12-05 21:56:11 +00:00
sadposter
eb1c7d75f4 Merge branch 'develop' of https://git.pleroma.social/pleroma/pleroma into develop 2021-10-16 14:08:04 +01:00
Haelwenn (lanodan) Monnier
a17910a6c6
CI: Bump lint stage to elixir-1.12 
Elixir 1.12 changed formatting rules, this allows to avoid having to rollback to run `mix format`
 2021-10-06 08:11:05 +02:00
sadposter
b202cf15d6 Merge branch 'develop' of https://git.pleroma.social/pleroma/pleroma into develop 2021-08-30 22:04:42 +01:00
someone
cc4f20b130
mix pleroma.database set_text_search_config now runs concurrently and infinitely 
Backport of: https://git.pleroma.social/pleroma/pleroma/-/merge_requests/3514
 2021-08-28 18:29:43 +02:00
someone
4b940e441a mix pleroma.database set_text_search_config now runs concurrently and infinitely 2021-08-15 13:49:12 -04:00
sadposter
e56c319e8f merge 2021-06-27 22:31:37 +01:00
feld
bc51dea425 Update lib/mix/tasks/pleroma/database.ex 2021-06-07 20:02:28 +00:00
faried nawaz
5be9d13981
a better query to delete from hashtags 
old query:

Delete on hashtags  (cost=5089.81..5521.63 rows=6160 width=18)
   ->  Hash Semi Join  (cost=5089.81..5521.63 rows=6160 width=18)
         Hash Cond: (hashtags.id = ht.id)
         ->  Seq Scan on hashtags  (cost=0.00..317.28 rows=17528 width=14)
         ->  Hash  (cost=5012.81..5012.81 rows=6160 width=20)
               ->  Merge Anti Join  (cost=0.70..5012.81 rows=6160 width=20)
                     Merge Cond: (ht.id = hto.hashtag_id)
                     ->  Index Scan using hashtags_pkey on hashtags ht  (cost=0.29..610.53 rows=17528 width=14)
                     ->  Index Scan using hashtags_objects_pkey on hashtags_objects hto  (cost=0.42..3506.48 rows=68158 width=14)

new query:

Delete on hashtags ht  (cost=0.70..5012.81 rows=6160 width=12)
   ->  Merge Anti Join  (cost=0.70..5012.81 rows=6160 width=12)
         Merge Cond: (ht.id = hto.hashtag_id)
         ->  Index Scan using hashtags_pkey on hashtags ht  (cost=0.29..610.53 rows=17528 width=14)
         ->  Index Scan using hashtags_objects_pkey on hashtags_objects hto  (cost=0.42..3506.48 rows=68158 width=14)
 2021-05-08 02:00:43 +05:00
faried nawaz
a0c9a2b4cc
mix prune_objects: remove unused hashtags after pruning remote objects 2021-05-08 02:00:42 +05:00
sadposter
5f62b55a6f Merge branch 'develop' of https://git.pleroma.social/pleroma/pleroma into develop 2021-03-23 23:01:07 +00:00
Alexander Strizhakov
4cd34d0197
suggestion 2021-03-23 17:27:02 +03:00
Alexander Strizhakov
03843a5386
migrating config to tmp folder 2021-03-23 14:29:02 +03:00
sadposter
292ddf730c Merge branch 'develop' of https://git.pleroma.social/pleroma/pleroma into develop 2021-03-03 13:51:07 +00:00
Ivan Tashkinov
882dd46843 Merge remote-tracking branch 'remotes/origin/develop' into feature/object-hashtags-rework 2021-03-02 08:26:30 +03:00
feld
2217b35bd4 Merge branch 'ecto-rollback-in-test-env' into 'develop' 
Possibility to run rollback in test env

See merge request pleroma/pleroma!3212
 2021-02-26 16:47:53 +00:00
Ivan Tashkinov
40d4362261 [#3213] mix pleroma.database rollback tweaks. 2021-02-23 18:11:25 +03:00
sadposter
5f33699169 Merge branch 'develop' of https://git.pleroma.social/pleroma/pleroma into develop 2021-02-20 15:27:56 +00:00
Ivan Tashkinov
5992382cf8 Merge remote-tracking branch 'remotes/origin/develop' into feature/object-hashtags-rework 
# Conflicts:
#	CHANGELOG.md
#	lib/mix/tasks/pleroma/database.ex
#	lib/pleroma/web/templates/feed/feed/_activity.rss.eex
 2021-02-11 19:31:57 +03:00
rinpatch
f13f5d9303 OpenAPI spec task: Load pleroma application to get version info 
For whatever reason it seems to pick up the version without loading the
app on my machine, but not on the CI.
 2021-02-09 22:10:09 +03:00
feld
593c0851d9 Merge branch 'develop' into 'new-user-emails' 2021-02-08 16:01:50 -06:00
Ivan Tashkinov
d1c6dd97aa [#3213] Partially addressed code review points. 
migration rollback task changes, hashtags-related config handling tweaks, `hashtags.data` deletion (unused).
 2021-02-07 22:24:12 +03:00
rinpatch
6e68058b63 Merge branch 'feat/allow_alt_text_search_config' into 'develop' 
allow user defined text search config in database

See merge request pleroma/pleroma!3275
 2021-02-06 09:42:20 +00:00
hyperion
8d4e0342e1 Update priv/repo/migrations/20190501125843_add_fts_index_to_objects.exs, priv/repo/optional_migrations/rum_indexing/20190510135645_add_fts_index_to_objects_two.exs files 2021-02-06 09:42:17 +00:00
Mark Felder
af37a5c51a Also make this maybe_ for consistency 2021-02-04 14:33:49 -06:00
rinpatch
e2d7c4fd57 Merge branch 'feat/openapi-spec-export' into 'develop' 
Improve OpenAPI spec and deploy it to api.pleroma.social

See merge request pleroma/pleroma!3296
 2021-02-03 13:21:19 +00:00
rinpatch
9fcff7851f Improve OpenAPI spec and deploy it to api.pleroma.social 
Too many changes in OpenAPI spec to describe each one, but
basically it is tag fixes, bringing consitency to operation summaries
and fixing some incorrect information.
 2021-02-03 16:07:44 +03:00
Ivan Tashkinov
4e14945670 Merge remote-tracking branch 'remotes/origin/develop' into feature/object-hashtags-rework 
# Conflicts:
#	CHANGELOG.md
#	lib/pleroma/web/activity_pub/activity_pub.ex
 2021-02-03 09:31:38 +03:00
Ivan Tashkinov
108e90b18e [#3213] Explicitly defined PKs in hashtags_objects and data_migration_failed_ids. Added "pleroma.database rollback" task to revert a single migration. 2021-01-31 22:03:59 +03:00
Alibek Omarov
aaceb042c5 fix format 2021-01-28 10:20:25 +00:00
Alibek Omarov
1547a2fda4 mix: instance: ensure all needed folders are created before generating config 2021-01-28 09:39:53 +00:00
Ivan Tashkinov
694d98be55 Merge remote-tracking branch 'remotes/origin/develop' into feature/object-hashtags-rework 2021-01-25 21:24:07 +03:00
Mark Felder
f868dcf3ac Ensure consistent behavior between pleroma.user activate/deactivate mix tasks 2021-01-25 10:48:28 -06:00
Mark Felder
5e8da27e14 Provide pleroma.user mix task for both activate and deactivate 2021-01-25 10:36:51 -06:00
sadposter
3aa25b008d Merge branch 'develop' of https://git.pleroma.social/pleroma/pleroma into develop 2021-01-23 22:38:07 +00:00
Ivan Tashkinov
2634a16b4c Merge remote-tracking branch 'remotes/origin/develop' into feature/object-hashtags-rework 
# Conflicts:
#	CHANGELOG.md
 2021-01-21 20:20:35 +03:00
Mark Felder
28581e03ad Merge branch 'develop' into refactor/deactivated_user_field 2021-01-18 14:58:21 -06:00
Mark Felder
2c0fe2ea9e Remove toggle_confirmation; require explicit state change 
Also cosmetic changes to make the code clearer
 2021-01-15 13:11:51 -06:00
Mark Felder
d36182c088 Change user.confirmation_pending field to user.is_confirmed 2021-01-15 12:44:41 -06:00
Mark Felder
ebd7d1365b Make setting user activation status explicit 2021-01-15 11:37:17 -06:00
Mark Felder
7516660753 Remove toggle_activation 2021-01-15 11:25:32 -06:00
Mark Felder
860b5c7804 Change user.deactivated field to user.is_active 2021-01-15 11:24:46 -06:00
Ivan Tashkinov
e350898828 Merge remote-tracking branch 'remotes/origin/develop' into feature/object-hashtags-rework 2021-01-13 22:11:16 +03:00
Ivan Tashkinov
3e4d84729a [#3213] Prototype of data migrations functionality / HashtagsTableMigrator. 2021-01-13 22:07:38 +03:00
Haelwenn (lanodan) Monnier
c4439c630f
Bump Copyright to 2021 
grep -rl '# Copyright © .* Pleroma' * | xargs sed -i 's;Copyright © .* Pleroma .*;Copyright © 2017-2021 Pleroma Authors <https://pleroma.social/>;'
 2021-01-13 07:49:50 +01:00
Ivan Tashkinov
8c972de045 [#3213] transfer_hashtags mix task refactoring. 2021-01-10 11:44:39 +03:00
Ivan Tashkinov
0d521022fe [#3213] Removed PK from hashtags_objects table. Improved hashtags_transfer mix task (logging of failed ids). 2021-01-07 12:20:29 +03:00
Ivan Tashkinov
367f0c31c3 [#3213] Added query options support for Repo.chunk_stream/4. 
Used infinite timeout in transfer_hashtags select query.
 2020-12-31 09:36:26 +03:00
Ivan Tashkinov
a25c1e8ec0 [#3213] Improved database.transfer_hashtags mix task: proper rollback, speedup. 2020-12-30 14:35:19 +03:00
sadposter
6cd0586ef9 Merge branch 'develop' of https://git.pleroma.social/pleroma/pleroma into develop 2020-12-29 13:16:14 +00:00
Haelwenn
3966add048 Revert "Merge branch 'features/hashtag-column' into 'develop'" 
This reverts merge request !2824
 2020-12-28 12:02:16 +00:00
Haelwenn (lanodan) Monnier
d0c2479710
pleroma.database fill_old_hashtags: Add month_limit argument 2020-12-28 11:05:25 +01:00
Ivan Tashkinov
cbb19d0e18 [#3213] Hashtag-filtering functions in ActivityPub. Mix task for migrating hashtags to hashtags table. 2020-12-26 22:20:55 +03:00
Alexander Strizhakov
538af14d52
possibility to run rollback in test env 2020-12-22 17:55:30 +03:00
Haelwenn (lanodan) Monnier
acb03d591b
Insert text representation of hashtags into object["hashtags"] 
Includes a new mix task: pleroma.database fill_old_hashtags
 2020-12-22 05:15:34 +01:00
Alex Gleason
80891e83d8
Merge remote-tracking branch 'upstream/develop' into registration-workflow 2020-12-17 09:05:36 -06:00