Merge branch 'develop' into feature/hide-follows-remote

CHANGELOG.md

@@ -11,6 +11,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Federation: Return 403 errors when trying to request pages from a user's follower/following collections if they have `hide_followers`/`hide_follows` set
 - NodeInfo: Return `skipThreadContainment` in `metadata` for the `skip_thread_containment` option
 - Mastodon API: Unsubscribe followers when they unfollow a user
+- AdminAPI: Add "godmode" while fetching user statuses (i.e. admin can see private statuses)
 
 ### Fixed
 - Not being able to pin unlisted posts
@@ -19,13 +20,20 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Mastodon API: Handling of search timeouts (`/api/v1/search` and `/api/v2/search`)
 - Mastodon API: Embedded relationships not being properly rendered in the Account entity of Status entity
 - Mastodon API: Add `account_id`, `type`, `offset`, and `limit` to search API (`/api/v1/search` and `/api/v2/search`)
+- Mastodon API, streaming: Fix filtering of notifications based on blocks/mutes/thread mutes
 - ActivityPub C2S: follower/following collection pages being inaccessible even when authentifucated if `hide_followers`/ `hide_follows` was set
 - Existing user id not being preserved on insert conflict
+- Rich Media: Parser failing when no TTL can be found by image TTL setters
+- Rich Media: The crawled URL is now spliced into the rich media data.
+- ActivityPub S2S: sharedInbox usage has been mostly aligned with the rules in the AP specification.
 
 ### Added
 - MRF: Support for priming the mediaproxy cache (`Pleroma.Web.ActivityPub.MRF.MediaProxyWarmingPolicy`)
 - MRF: Support for excluding specific domains from Transparency.
 - MRF: Support for filtering posts based on who they mention (`Pleroma.Web.ActivityPub.MRF.MentionPolicy`)
+- MRF (Simple Policy): Support for wildcard domains.
+- Support for wildcard domains in user domain blocks setting.
+- Configuration: `quarantined_instances` support wildcard domains.
 - Configuration: `federation_incoming_replies_max_depth` option
 - Mastodon API: Support for the [`tagged` filter](https://github.com/tootsuite/mastodon/pull/9755) in [`GET /api/v1/accounts/:id/statuses`](https://docs.joinmastodon.org/api/rest/accounts/#get-api-v1-accounts-id-statuses)
 - Mastodon API, streaming: Add support for passing the token in the `Sec-WebSocket-Protocol` header
@@ -33,6 +41,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Mastodon API: Add support for categories for custom emojis by reusing the group feature. <https://github.com/tootsuite/mastodon/pull/11196>
 - Mastodon API: Add support for muting/unmuting notifications
 - Mastodon API: Add support for the `blocked_by` attribute in the relationship API (`GET /api/v1/accounts/relationships`). <https://github.com/tootsuite/mastodon/pull/10373>
+- Mastodon API: Add support for the `domain_blocking` attribute in the relationship API (`GET /api/v1/accounts/relationships`).
 - Mastodon API: Add `pleroma.deactivated` to the Account entity
 - Mastodon API: added `/auth/password` endpoint for password reset with rate limit.
 - Mastodon API: /api/v1/accounts/:id/statuses now supports nicknames or user id
@@ -47,6 +56,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Twitter API: added rate limit for `/api/account/password_reset` endpoint.
 - ActivityPub: Add an internal service actor for fetching ActivityPub objects.
 - ActivityPub: Optional signing of ActivityPub object fetches.
+- Admin API: Endpoint for fetching latest user's statuses
 
 ### Changed
 - Configuration: Filter.AnonymizeFilename added ability to retain file extension with custom text

docs/api/admin_api.md

@@ -187,6 +187,19 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret
   - On failure: `Not found`
   - On success: JSON of the user
 
+## `/api/pleroma/admin/users/:nickname_or_id/statuses`
+
+### Retrive user's latest statuses
+
+- Method: `GET`
+- Params:
+  - `nickname` or `id`
+  - *optional* `page_size`: number of statuses to return (default is `20`)
+  - *optional* `godmode`: `true`/`false` – allows to see private statuses
+- Response:
+  - On failure: `Not found`
+  - On success: JSON array of user's latest statuses
+
 ## `/api/pleroma/admin/relay`
 
 ### Follow a Relay
@@ -564,6 +577,7 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret
 
 ## `/api/pleroma/admin/config`
 ### List config settings
+List config settings only works with `:pleroma => :instance => :dynamic_configuration` setting to `true`.
 - Method `GET`
 - Params: none
 - Response:
@@ -582,6 +596,7 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret
 
 ## `/api/pleroma/admin/config`
 ### Update config settings
+Updating config settings only works with `:pleroma => :instance => :dynamic_configuration` setting to `true`.
 Module name can be passed as string, which starts with `Pleroma`, e.g. `"Pleroma.Upload"`.
 Atom keys and values can be passed with `:` in the beginning, e.g. `":upload"`.
 Tuples can be passed as `{"tuple": ["first_val", Pleroma.Module, []]}`.

docs/config/small_customizations.md

@@ -1,35 +1,12 @@
 # Small customizations
-Replace `dev.secret.exs` with `prod.secret.exs` according to your setup.
 
-# Thumbnail
+See also static_dir.md for visual settings.
 
-Replace `priv/static/instance/thumbnail.jpeg` with your selfie or other neat picture. It will appear in [Pleroma Instances](http://distsn.org/pleroma-instances.html).
+## Theme
-
-# Instance-specific panel
-
-![instance-specific panel demo](/uploads/296b19ec806b130e0b49b16bfe29ce8a/image.png)
-
-To show the instance specific panel, set `show_instance_panel` to `true` in `config/dev.secret.exs`. You can modify its content by editing `priv/static/instance/panel.html`.
-
-# Background
-
-You can change the background of your Pleroma instance by uploading it to `priv/static/static`, and then changing `"background"` in `config/dev.secret.exs` accordingly.
-
-# Logo
-
-![logo modification demo](/uploads/c70b14de60fa74245e7f0dcfa695ebff/image.png)
-
-If you want to give a brand to your instance, look no further. You can change the logo of your instance by uploading it to `priv/static/static`, and then changing `logo` in `config/dev.secret.exs` accordingly.
-
-# Theme
 
 All users of your instance will be able to change the theme they use by going to the settings (the cog in the top-right hand corner). However, if you wish to change the default theme, you can do so by editing `theme` in `config/dev.secret.exs` accordingly.
 
-# Terms of Service
+## Message Visibility
-
-Terms of Service will be shown to all users on the registration page. It's the best place where to write down the rules for your instance. You can modify the rules by changing `priv/static/static/terms-of-service.html`.
-
-# Message Visibility
 
 To enable message visibility options when posting like in the Mastodon frontend, set
 `scope_options_enabled` to `true` in `config/dev.secret.exs`.

docs/config/static_dir.md

@@ -7,7 +7,13 @@ config :pleroma, :instance,
   static_dir: "instance/static/",
 ```
 
-You can overwrite this value in your configuration to use a different static instance directory.
+For example, edit `instance/static/instance/panel.html` .
+
+Alternatively, you can overwrite this value in your configuration to use a different static instance directory.
+
+This document is written assuming `instance/static/`.
+
+Or, if you want to manage your custom file in git repository, basically remove the `instance/` entry from `.gitignore`.
 
 ## robots.txt
 
@@ -18,3 +24,46 @@ If you want to generate a restrictive `robots.txt`, you can run the following mi
 ```
 mix pleroma.robots_txt disallow_all
 ```
+
+## Thumbnail
+
+Put on `instance/static/instance/thumbnail.jpeg` with your selfie or other neat picture. It will appear in [Pleroma Instances](http://distsn.org/pleroma-instances.html).
+
+## Instance-specific panel
+
+![instance-specific panel demo](/uploads/296b19ec806b130e0b49b16bfe29ce8a/image.png)
+
+Create and Edit your file on `instance/static/instance/panel.html`.
+
+## Background
+
+You can change the background of your Pleroma instance by uploading it to `instance/static/`, and then changing `background` in `config/prod.secret.exs` accordingly.
+
+If you put `instance/static/images/background.jpg`
+
+```
+config :pleroma, :frontend_configurations,
+  pleroma_fe: %{
+    background: "/images/background.jpg"
+  }
+```
+
+## Logo
+
+![logo modification demo](/uploads/c70b14de60fa74245e7f0dcfa695ebff/image.png)
+
+If you want to give a brand to your instance, You can change the logo of your instance by uploading it to `instance/static/`.
+
+Alternatively, you can specify the path with config.
+If you put `instance/static/static/mylogo-file.png`
+
+```
+config :pleroma, :frontend_configurations,
+  pleroma_fe: %{
+   logo: "/static/mylogo-file.png"
+  }
+```
+
+## Terms of Service
+
+Terms of Service will be shown to all users on the registration page. It's the best place where to write down the rules for your instance. You can modify the rules by changing `instance/static/static/terms-of-service.html`.

lib/pleroma/plugs/set_format_plug.ex

@@ -0,0 +1,24 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Plugs.SetFormatPlug do
+  import Plug.Conn, only: [assign: 3, fetch_query_params: 1]
+
+  def init(_), do: nil
+
+  def call(conn, _) do
+    case get_format(conn) do
+      nil -> conn
+      format -> assign(conn, :format, format)
+    end
+  end
+
+  defp get_format(conn) do
+    conn.private[:phoenix_format] ||
+      case fetch_query_params(conn) do
+        %{query_params: %{"_format" => format}} -> format
+        _ -> nil
+      end
+  end
+end

lib/pleroma/signature.ex

@@ -10,9 +10,18 @@ defmodule Pleroma.Signature do
   alias Pleroma.Web.ActivityPub.ActivityPub
 
   def key_id_to_actor_id(key_id) do
+    uri =
       URI.parse(key_id)
       |> Map.put(:fragment, nil)
-    |> URI.to_string()
+
+    uri =
+      if String.ends_with?(uri.path, "/publickey") do
+        Map.put(uri, :path, String.replace(uri.path, "/publickey", ""))
+      else
+        uri
+      end
+
+    URI.to_string(uri)
   end
 
   def fetch_public_key(conn) do

lib/pleroma/upload.ex

@@ -228,7 +228,14 @@ defmodule Pleroma.Upload do
           ""
         end
 
-    [base_url, "media", path]
+    prefix =
+      if is_nil(Pleroma.Config.get([__MODULE__, :base_url])) do
+        "media"
+      else
+        ""
+      end
+
+    [base_url, prefix, path]
     |> Path.join()
   end
 

lib/pleroma/user.ex

@@ -592,12 +592,23 @@ defmodule Pleroma.User do
   @spec get_followers_query(User.t()) :: Ecto.Query.t()
   def get_followers_query(user), do: get_followers_query(user, nil)
 
+  @spec get_followers(User.t(), pos_integer()) :: {:ok, list(User.t())}
   def get_followers(user, page \\ nil) do
     q = get_followers_query(user, page)
 
     {:ok, Repo.all(q)}
   end
 
+  @spec get_external_followers(User.t(), pos_integer()) :: {:ok, list(User.t())}
+  def get_external_followers(user, page \\ nil) do
+    q =
+      user
+      |> get_followers_query(page)
+      |> User.Query.build(%{external: true})
+
+    {:ok, Repo.all(q)}
+  end
+
   def get_followers_ids(user, page \\ nil) do
     q = get_followers_query(user, page)
 
@@ -918,14 +929,26 @@ defmodule Pleroma.User do
   def muted_notifications?(user, %{ap_id: ap_id}),
     do: Enum.member?(user.info.muted_notifications, ap_id)
 
-  def blocks?(%User{info: info} = _user, %{ap_id: ap_id}) do
+  def blocks?(%User{} = user, %User{} = target) do
-    blocks = info.blocks
+    blocks_ap_id?(user, target) || blocks_domain?(user, target)
-    domain_blocks = info.domain_blocks
-    %{host: host} = URI.parse(ap_id)
-
-    Enum.member?(blocks, ap_id) || Enum.any?(domain_blocks, &(&1 == host))
   end
 
+  def blocks?(nil, _), do: false
+
+  def blocks_ap_id?(%User{} = user, %User{} = target) do
+    Enum.member?(user.info.blocks, target.ap_id)
+  end
+
+  def blocks_ap_id?(_, _), do: false
+
+  def blocks_domain?(%User{} = user, %User{} = target) do
+    domain_blocks = Pleroma.Web.ActivityPub.MRF.subdomains_regex(user.info.domain_blocks)
+    %{host: host} = URI.parse(target.ap_id)
+    Pleroma.Web.ActivityPub.MRF.subdomain_match?(domain_blocks, host)
+  end
+
+  def blocks_domain?(_, _), do: false
+
   def subscribed_to?(user, %{ap_id: ap_id}) do
     with %User{} = target <- get_cached_by_ap_id(ap_id) do
       Enum.member?(target.info.subscribers, user.ap_id)

lib/pleroma/user_invite_token.ex

@@ -74,7 +74,7 @@ defmodule Pleroma.UserInviteToken do
 
   @spec find_by_token(token()) :: {:ok, UserInviteToken.t()} | nil
   def find_by_token(token) do
-    with invite <- Repo.get_by(UserInviteToken, token: token) do
+    with %UserInviteToken{} = invite <- Repo.get_by(UserInviteToken, token: token) do
       {:ok, invite}
     end
   end

lib/pleroma/web/activity_pub/activity_pub.ex

@@ -631,15 +631,26 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
       |> Map.put("pinned_activity_ids", user.info.pinned_activities)
 
     recipients =
+      user_activities_recipients(%{
+        "godmode" => params["godmode"],
+        "reading_user" => reading_user
+      })
+
+    fetch_activities(recipients, params)
+    |> Enum.reverse()
+  end
+
+  defp user_activities_recipients(%{"godmode" => true}) do
+    []
+  end
+
+  defp user_activities_recipients(%{"reading_user" => reading_user}) do
     if reading_user do
       ["https://www.w3.org/ns/activitystreams#Public"] ++
         [reading_user.ap_id | reading_user.following]
     else
       ["https://www.w3.org/ns/activitystreams#Public"]
     end
-
-    fetch_activities(recipients, params)
-    |> Enum.reverse()
   end
 
   defp restrict_since(query, %{"since_id" => ""}), do: query

lib/pleroma/web/activity_pub/mrf.ex

@@ -25,4 +25,14 @@ defmodule Pleroma.Web.ActivityPub.MRF do
   defp get_policies(policy) when is_atom(policy), do: [policy]
   defp get_policies(policies) when is_list(policies), do: policies
   defp get_policies(_), do: []
+
+  @spec subdomains_regex([String.t()]) :: [Regex.t()]
+  def subdomains_regex(domains) when is_list(domains) do
+    for domain <- domains, do: ~r(^#{String.replace(domain, "*.", "(.*\\.)*")}$)
+  end
+
+  @spec subdomain_match?([Regex.t()], String.t()) :: boolean()
+  def subdomain_match?(domains, host) do
+    Enum.any?(domains, fn domain -> Regex.match?(domain, host) end)
+  end
 end

lib/pleroma/web/activity_pub/mrf/simple_policy.ex

@@ -4,22 +4,29 @@
 
 defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicy do
   alias Pleroma.User
+  alias Pleroma.Web.ActivityPub.MRF
   @moduledoc "Filter activities depending on their origin instance"
-  @behaviour Pleroma.Web.ActivityPub.MRF
+  @behaviour MRF
 
   defp check_accept(%{host: actor_host} = _actor_info, object) do
-    accepts = Pleroma.Config.get([:mrf_simple, :accept])
+    accepts =
+      Pleroma.Config.get([:mrf_simple, :accept])
+      |> MRF.subdomains_regex()
 
     cond do
       accepts == [] -> {:ok, object}
       actor_host == Pleroma.Config.get([Pleroma.Web.Endpoint, :url, :host]) -> {:ok, object}
-      Enum.member?(accepts, actor_host) -> {:ok, object}
+      MRF.subdomain_match?(accepts, actor_host) -> {:ok, object}
       true -> {:reject, nil}
     end
   end
 
   defp check_reject(%{host: actor_host} = _actor_info, object) do
-    if Enum.member?(Pleroma.Config.get([:mrf_simple, :reject]), actor_host) do
+    rejects =
+      Pleroma.Config.get([:mrf_simple, :reject])
+      |> MRF.subdomains_regex()
+
+    if MRF.subdomain_match?(rejects, actor_host) do
       {:reject, nil}
     else
       {:ok, object}
@@ -31,8 +38,12 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicy do
          %{"type" => "Create", "object" => %{"attachment" => child_attachment}} = object
        )
        when length(child_attachment) > 0 do
+    media_removal =
+      Pleroma.Config.get([:mrf_simple, :media_removal])
+      |> MRF.subdomains_regex()
+
     object =
-      if Enum.member?(Pleroma.Config.get([:mrf_simple, :media_removal]), actor_host) do
+      if MRF.subdomain_match?(media_removal, actor_host) do
         child_object = Map.delete(object["object"], "attachment")
         Map.put(object, "object", child_object)
       else
@@ -51,8 +62,12 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicy do
            "object" => child_object
          } = object
        ) do
+    media_nsfw =
+      Pleroma.Config.get([:mrf_simple, :media_nsfw])
+      |> MRF.subdomains_regex()
+
     object =
-      if Enum.member?(Pleroma.Config.get([:mrf_simple, :media_nsfw]), actor_host) do
+      if MRF.subdomain_match?(media_nsfw, actor_host) do
         tags = (child_object["tag"] || []) ++ ["nsfw"]
         child_object = Map.put(child_object, "tag", tags)
         child_object = Map.put(child_object, "sensitive", true)
@@ -67,12 +82,12 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicy do
   defp check_media_nsfw(_actor_info, object), do: {:ok, object}
 
   defp check_ftl_removal(%{host: actor_host} = _actor_info, object) do
+    timeline_removal =
+      Pleroma.Config.get([:mrf_simple, :federated_timeline_removal])
+      |> MRF.subdomains_regex()
+
     object =
-      with true <-
+      with true <- MRF.subdomain_match?(timeline_removal, actor_host),
-             Enum.member?(
-               Pleroma.Config.get([:mrf_simple, :federated_timeline_removal]),
-               actor_host
-             ),
            user <- User.get_cached_by_ap_id(object["actor"]),
            true <- "https://www.w3.org/ns/activitystreams#Public" in object["to"] do
         to =
@@ -94,7 +109,11 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicy do
   end
 
   defp check_report_removal(%{host: actor_host} = _actor_info, %{"type" => "Flag"} = object) do
-    if actor_host in Pleroma.Config.get([:mrf_simple, :report_removal]) do
+    report_removal =
+      Pleroma.Config.get([:mrf_simple, :report_removal])
+      |> MRF.subdomains_regex()
+
+    if MRF.subdomain_match?(report_removal, actor_host) do
       {:reject, nil}
     else
       {:ok, object}
@@ -104,7 +123,11 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicy do
   defp check_report_removal(_actor_info, object), do: {:ok, object}
 
   defp check_avatar_removal(%{host: actor_host} = _actor_info, %{"icon" => _icon} = object) do
-    if actor_host in Pleroma.Config.get([:mrf_simple, :avatar_removal]) do
+    avatar_removal =
+      Pleroma.Config.get([:mrf_simple, :avatar_removal])
+      |> MRF.subdomains_regex()
+
+    if MRF.subdomain_match?(avatar_removal, actor_host) do
       {:ok, Map.delete(object, "icon")}
     else
       {:ok, object}
@@ -114,7 +137,11 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicy do
   defp check_avatar_removal(_actor_info, object), do: {:ok, object}
 
   defp check_banner_removal(%{host: actor_host} = _actor_info, %{"image" => _image} = object) do
-    if actor_host in Pleroma.Config.get([:mrf_simple, :banner_removal]) do
+    banner_removal =
+      Pleroma.Config.get([:mrf_simple, :banner_removal])
+      |> MRF.subdomains_regex()
+
+    if MRF.subdomain_match?(banner_removal, actor_host) do
       {:ok, Map.delete(object, "image")}
     else
       {:ok, object}

lib/pleroma/web/activity_pub/publisher.ex

@@ -87,18 +87,23 @@ defmodule Pleroma.Web.ActivityPub.Publisher do
     if public do
       true
     else
-      inbox_info = URI.parse(inbox)
+      %{host: host} = URI.parse(inbox)
-      !Enum.member?(Config.get([:instance, :quarantined_instances], []), inbox_info.host)
+
+      quarantined_instances =
+        Config.get([:instance, :quarantined_instances], [])
+        |> Pleroma.Web.ActivityPub.MRF.subdomains_regex()
+
+      !Pleroma.Web.ActivityPub.MRF.subdomain_match?(quarantined_instances, host)
     end
   end
 
+  @spec recipients(User.t(), Activity.t()) :: list(User.t()) | []
   defp recipients(actor, activity) do
-    followers =
+    {:ok, followers} =
       if actor.follower_address in activity.recipients do
-        {:ok, followers} = User.get_followers(actor)
+        User.get_external_followers(actor)
-        Enum.filter(followers, &(!&1.local))
       else
-        []
+        {:ok, []}
       end
 
     Pleroma.Web.Salmon.remote_users(actor, activity) ++ followers
@@ -112,6 +117,45 @@ defmodule Pleroma.Web.ActivityPub.Publisher do
     |> Enum.map(& &1.ap_id)
   end
 
+  @as_public "https://www.w3.org/ns/activitystreams#Public"
+
+  defp maybe_use_sharedinbox(%User{info: %{source_data: data}}),
+    do: (is_map(data["endpoints"]) && Map.get(data["endpoints"], "sharedInbox")) || data["inbox"]
+
+  @doc """
+  Determine a user inbox to use based on heuristics.  These heuristics
+  are based on an approximation of the ``sharedInbox`` rules in the
+  [ActivityPub specification][ap-sharedinbox].
+
+  Please do not edit this function (or its children) without reading
+  the spec, as editing the code is likely to introduce some breakage
+  without some familiarity.
+
+     [ap-sharedinbox]: https://www.w3.org/TR/activitypub/#shared-inbox-delivery
+  """
+  def determine_inbox(
+        %Activity{data: activity_data},
+        %User{info: %{source_data: data}} = user
+      ) do
+    to = activity_data["to"] || []
+    cc = activity_data["cc"] || []
+    type = activity_data["type"]
+
+    cond do
+      type == "Delete" ->
+        maybe_use_sharedinbox(user)
+
+      @as_public in to || @as_public in cc ->
+        maybe_use_sharedinbox(user)
+
+      length(to) + length(cc) > 1 ->
+        maybe_use_sharedinbox(user)
+
+      true ->
+        data["inbox"]
+    end
+  end
+
   @doc """
   Publishes an activity with BCC to all relevant peers.
   """
@@ -166,8 +210,8 @@ defmodule Pleroma.Web.ActivityPub.Publisher do
 
     recipients(actor, activity)
     |> Enum.filter(fn user -> User.ap_enabled?(user) end)
-    |> Enum.map(fn %{info: %{source_data: data}} ->
+    |> Enum.map(fn %User{} = user ->
-      (is_map(data["endpoints"]) && Map.get(data["endpoints"], "sharedInbox")) || data["inbox"]
+      determine_inbox(activity, user)
     end)
     |> Enum.uniq()
     |> Enum.filter(fn inbox -> should_federate?(inbox, public) end)

lib/pleroma/web/activity_pub/visibility.ex

@@ -8,14 +8,14 @@ defmodule Pleroma.Web.ActivityPub.Visibility do
   alias Pleroma.Repo
   alias Pleroma.User
 
+  @public "https://www.w3.org/ns/activitystreams#Public"
+
+  @spec is_public?(Object.t() | Activity.t() | map()) :: boolean()
   def is_public?(%Object{data: %{"type" => "Tombstone"}}), do: false
   def is_public?(%Object{data: data}), do: is_public?(data)
   def is_public?(%Activity{data: data}), do: is_public?(data)
   def is_public?(%{"directMessage" => true}), do: false
-
+  def is_public?(data), do: @public in (data["to"] ++ (data["cc"] || []))
-  def is_public?(data) do
-    "https://www.w3.org/ns/activitystreams#Public" in (data["to"] ++ (data["cc"] || []))
-  end
 
   def is_private?(activity) do
     with false <- is_public?(activity),
@@ -69,15 +69,14 @@ defmodule Pleroma.Web.ActivityPub.Visibility do
   end
 
   def get_visibility(object) do
-    public = "https://www.w3.org/ns/activitystreams#Public"
     to = object.data["to"] || []
     cc = object.data["cc"] || []
 
     cond do
-      public in to ->
+      @public in to ->
         "public"
 
-      public in cc ->
+      @public in cc ->
         "unlisted"
 
       # this should use the sql for the object's activity

lib/pleroma/web/admin_api/admin_api_controller.ex

@@ -82,6 +82,25 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
     end
   end
 
+  def list_user_statuses(conn, %{"nickname" => nickname} = params) do
+    godmode = params["godmode"] == "true" || params["godmode"] == true
+
+    with %User{} = user <- User.get_cached_by_nickname_or_id(nickname) do
+      {_, page_size} = page_params(params)
+
+      activities =
+        ActivityPub.fetch_user_activities(user, nil, %{
+          "limit" => page_size,
+          "godmode" => godmode
+        })
+
+      conn
+      |> json(StatusView.render("index.json", %{activities: activities, as: :activity}))
+    else
+      _ -> {:error, :not_found}
+    end
+  end
+
   def user_toggle_activation(conn, %{"nickname" => nickname}) do
     user = User.get_cached_by_nickname(nickname)
 
@@ -272,11 +291,13 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
 
   @doc "Revokes invite by token"
   def revoke_invite(conn, %{"token" => token}) do
-    invite = UserInviteToken.find_by_token!(token)
+    with {:ok, invite} <- UserInviteToken.find_by_token(token),
-    {:ok, updated_invite} = UserInviteToken.update_invite(invite, %{used: true})
+         {:ok, updated_invite} = UserInviteToken.update_invite(invite, %{used: true}) do
-
       conn
       |> json(AccountView.render("invite.json", %{invite: updated_invite}))
+    else
+      nil -> {:error, :not_found}
+    end
   end
 
   @doc "Get a password reset token (base64 string) for given nickname"

lib/pleroma/web/admin_api/config.ex

@@ -84,6 +84,7 @@ defmodule Pleroma.Web.AdminAPI.Config do
   end
 
   defp do_convert({:dispatch, [entity]}), do: %{"tuple" => [":dispatch", [inspect(entity)]]}
+  defp do_convert({:partial_chain, entity}), do: %{"tuple" => [":partial_chain", inspect(entity)]}
 
   defp do_convert(entity) when is_tuple(entity),
     do: %{"tuple" => do_convert(Tuple.to_list(entity))}
@@ -113,11 +114,15 @@ defmodule Pleroma.Web.AdminAPI.Config do
   defp do_transform(%Regex{} = entity) when is_map(entity), do: entity
 
   defp do_transform(%{"tuple" => [":dispatch", [entity]]}) do
-    cleaned_string = String.replace(entity, ~r/[^\w|^{:,[|^,|^[|^\]^}|^\/|^\.|^"]^\s/, "")
+    {dispatch_settings, []} = do_eval(entity)
-    {dispatch_settings, []} = Code.eval_string(cleaned_string, [], requires: [], macros: [])
     {:dispatch, [dispatch_settings]}
   end
 
+  defp do_transform(%{"tuple" => [":partial_chain", entity]}) do
+    {partial_chain, []} = do_eval(entity)
+    {:partial_chain, partial_chain}
+  end
+
   defp do_transform(%{"tuple" => entity}) do
     Enum.reduce(entity, {}, fn val, acc -> Tuple.append(acc, do_transform(val)) end)
   end
@@ -149,4 +154,9 @@ defmodule Pleroma.Web.AdminAPI.Config do
       do: String.to_existing_atom("Elixir." <> value),
       else: value
   end
+
+  defp do_eval(entity) do
+    cleaned_string = String.replace(entity, ~r/[^\w|^{:,[|^,|^[|^\]^}|^\/|^\.|^"]^\s/, "")
+    Code.eval_string(cleaned_string, [], requires: [], macros: [])
+  end
 end

lib/pleroma/web/common_api/utils.ex

@@ -439,6 +439,13 @@ defmodule Pleroma.Web.CommonAPI.Utils do
 
   def maybe_notify_mentioned_recipients(recipients, _), do: recipients
 
+  # Do not notify subscribers if author is making a reply
+  def maybe_notify_subscribers(recipients, %Activity{
+        object: %Object{data: %{"inReplyTo" => _ap_id}}
+      }) do
+    recipients
+  end
+
   def maybe_notify_subscribers(
         recipients,
         %Activity{data: %{"actor" => actor, "type" => type}} = activity

lib/pleroma/web/mastodon_api/mastodon_api_controller.ex

@@ -883,7 +883,10 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIController do
     with %Activity{data: %{"object" => object}} <- Activity.get_by_id(id),
          %Object{data: %{"likes" => likes}} <- Object.normalize(object) do
       q = from(u in User, where: u.ap_id in ^likes)
-      users = Repo.all(q)
+
+      users =
+        Repo.all(q)
+        |> Enum.filter(&(not User.blocks?(user, &1)))
 
       conn
       |> put_view(AccountView)
@@ -897,7 +900,10 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIController do
     with %Activity{data: %{"object" => object}} <- Activity.get_by_id(id),
          %Object{data: %{"announcements" => announces}} <- Object.normalize(object) do
       q = from(u in User, where: u.ap_id in ^announces)
-      users = Repo.all(q)
+
+      users =
+        Repo.all(q)
+        |> Enum.filter(&(not User.blocks?(user, &1)))
 
       conn
       |> put_view(AccountView)

lib/pleroma/web/mastodon_api/views/account_view.ex

@@ -50,13 +50,13 @@ defmodule Pleroma.Web.MastodonAPI.AccountView do
       id: to_string(target.id),
       following: User.following?(user, target),
       followed_by: User.following?(target, user),
-      blocking: User.blocks?(user, target),
+      blocking: User.blocks_ap_id?(user, target),
-      blocked_by: User.blocks?(target, user),
+      blocked_by: User.blocks_ap_id?(target, user),
       muting: User.mutes?(user, target),
       muting_notifications: User.muted_notifications?(user, target),
       subscribing: User.subscribed_to?(user, target),
       requested: requested,
-      domain_blocking: false,
+      domain_blocking: User.blocks_domain?(user, target),
       showing_reblogs: User.showing_reblogs?(user, target),
       endorsed: false
     }

lib/pleroma/web/mastodon_api/views/status_view.ex

@@ -222,7 +222,7 @@ defmodule Pleroma.Web.MastodonAPI.StatusView do
       if user.local do
         Pleroma.Web.Router.Helpers.o_status_url(Pleroma.Web.Endpoint, :notice, activity)
       else
-        object.data["external_url"] || object.data["id"]
+        object.data["url"] || object.data["external_url"] || object.data["id"]
       end
 
     %{

lib/pleroma/web/rich_media/parser.ex

@@ -55,8 +55,8 @@ defmodule Pleroma.Web.RichMedia.Parser do
         ttl_setters: [MyModule]
   """
   def set_ttl_based_on_image({:ok, data}, url) do
-    with {:ok, nil} <- Cachex.ttl(:rich_media_cache, url) do
+    with {:ok, nil} <- Cachex.ttl(:rich_media_cache, url),
-      ttl = get_ttl_from_image(data, url)
+         ttl when is_number(ttl) <- get_ttl_from_image(data, url) do
       Cachex.expire_at(:rich_media_cache, url, ttl * 1000)
       {:ok, data}
     else
@@ -82,6 +82,7 @@ defmodule Pleroma.Web.RichMedia.Parser do
 
       html
       |> maybe_parse()
+      |> Map.put(:url, url)
       |> clean_parsed_data()
       |> check_parsed_data()
     rescue

lib/pleroma/web/router.ex

@@ -154,22 +154,12 @@ defmodule Pleroma.Web.Router do
     post("/users/follow", AdminAPIController, :user_follow)
     post("/users/unfollow", AdminAPIController, :user_unfollow)
 
-    # TODO: to be removed at version 1.0
-    delete("/user", AdminAPIController, :user_delete)
-    post("/user", AdminAPIController, :user_create)
-
     delete("/users", AdminAPIController, :user_delete)
     post("/users", AdminAPIController, :user_create)
     patch("/users/:nickname/toggle_activation", AdminAPIController, :user_toggle_activation)
     put("/users/tag", AdminAPIController, :tag_users)
     delete("/users/tag", AdminAPIController, :untag_users)
 
-    # TODO: to be removed at version 1.0
-    get("/permission_group/:nickname", AdminAPIController, :right_get)
-    get("/permission_group/:nickname/:permission_group", AdminAPIController, :right_get)
-    post("/permission_group/:nickname/:permission_group", AdminAPIController, :right_add)
-    delete("/permission_group/:nickname/:permission_group", AdminAPIController, :right_delete)
-
     get("/users/:nickname/permission_group", AdminAPIController, :right_get)
     get("/users/:nickname/permission_group/:permission_group", AdminAPIController, :right_get)
     post("/users/:nickname/permission_group/:permission_group", AdminAPIController, :right_add)
@@ -190,13 +180,11 @@ defmodule Pleroma.Web.Router do
     post("/users/revoke_invite", AdminAPIController, :revoke_invite)
     post("/users/email_invite", AdminAPIController, :email_invite)
 
-    # TODO: to be removed at version 1.0
-    get("/password_reset", AdminAPIController, :get_password_reset)
-
     get("/users/:nickname/password_reset", AdminAPIController, :get_password_reset)
 
     get("/users", AdminAPIController, :list_users)
     get("/users/:nickname", AdminAPIController, :user_show)
+    get("/users/:nickname/statuses", AdminAPIController, :list_user_statuses)
 
     get("/reports", AdminAPIController, :list_reports)
     get("/reports/:id", AdminAPIController, :report_show)
@@ -663,6 +651,12 @@ defmodule Pleroma.Web.Router do
     end
   end
 
+  scope "/", Pleroma.Web.ActivityPub do
+    pipe_through(:activitypub)
+    post("/inbox", ActivityPubController, :inbox)
+    post("/users/:nickname/inbox", ActivityPubController, :inbox)
+  end
+
   scope "/relay", Pleroma.Web.ActivityPub do
     pipe_through(:ap_service_actor)
 
@@ -677,12 +671,6 @@ defmodule Pleroma.Web.Router do
     post("/inbox", ActivityPubController, :inbox)
   end
 
-  scope "/", Pleroma.Web.ActivityPub do
-    pipe_through(:activitypub)
-    post("/inbox", ActivityPubController, :inbox)
-    post("/users/:nickname/inbox", ActivityPubController, :inbox)
-  end
-
   scope "/.well-known", Pleroma.Web do
     pipe_through(:well_known)
 

lib/pleroma/web/streamer.ex

@@ -13,6 +13,7 @@ defmodule Pleroma.Web.Streamer do
   alias Pleroma.User
   alias Pleroma.Web.ActivityPub.ActivityPub
   alias Pleroma.Web.ActivityPub.Visibility
+  alias Pleroma.Web.CommonAPI
   alias Pleroma.Web.MastodonAPI.NotificationView
 
   @keepalive_interval :timer.seconds(30)
@@ -118,10 +119,14 @@ defmodule Pleroma.Web.Streamer do
     topics
     |> Map.get("#{topic}:#{item.user_id}", [])
     |> Enum.each(fn socket ->
+      with %User{} = user <- User.get_cached_by_ap_id(socket.assigns[:user].ap_id),
+           true <- should_send?(user, item),
+           false <- CommonAPI.thread_muted?(user, item.activity) do
         send(
           socket.transport_pid,
           {:text, represent_notification(socket.assigns[:user], item)}
         )
+      end
     end)
 
     {:noreply, topics}
@@ -225,19 +230,37 @@ defmodule Pleroma.Web.Streamer do
     |> Jason.encode!()
   end
 
+  defp should_send?(%User{} = user, %Activity{} = item) do
+    blocks = user.info.blocks || []
+    mutes = user.info.mutes || []
+    reblog_mutes = user.info.muted_reblogs || []
+    domain_blocks = Pleroma.Web.ActivityPub.MRF.subdomains_regex(user.info.domain_blocks)
+
+    with parent when not is_nil(parent) <- Object.normalize(item),
+         true <- Enum.all?([blocks, mutes, reblog_mutes], &(item.actor not in &1)),
+         true <- Enum.all?([blocks, mutes], &(parent.data["actor"] not in &1)),
+         %{host: item_host} <- URI.parse(item.actor),
+         %{host: parent_host} <- URI.parse(parent.data["actor"]),
+         false <- Pleroma.Web.ActivityPub.MRF.subdomain_match?(domain_blocks, item_host),
+         false <- Pleroma.Web.ActivityPub.MRF.subdomain_match?(domain_blocks, parent_host),
+         true <- thread_containment(item, user) do
+      true
+    else
+      _ -> false
+    end
+  end
+
+  defp should_send?(%User{} = user, %Notification{activity: activity}) do
+    should_send?(user, activity)
+  end
+
   def push_to_socket(topics, topic, %Activity{data: %{"type" => "Announce"}} = item) do
     Enum.each(topics[topic] || [], fn socket ->
       # Get the current user so we have up-to-date blocks etc.
       if socket.assigns[:user] do
         user = User.get_cached_by_ap_id(socket.assigns[:user].ap_id)
-        blocks = user.info.blocks || []
-        mutes = user.info.mutes || []
-        reblog_mutes = user.info.muted_reblogs || []
 
-        with parent when not is_nil(parent) <- Object.normalize(item),
+        if should_send?(user, item) do
-             true <- Enum.all?([blocks, mutes, reblog_mutes], &(item.actor not in &1)),
-             true <- Enum.all?([blocks, mutes], &(parent.data["actor"] not in &1)),
-             true <- thread_containment(item, user) do
           send(socket.transport_pid, {:text, represent_update(item, user)})
         end
       else

lib/pleroma/web/twitter_api/controllers/util_controller.ex

@@ -17,7 +17,6 @@ defmodule Pleroma.Web.TwitterAPI.UtilController do
   alias Pleroma.Web
   alias Pleroma.Web.ActivityPub.ActivityPub
   alias Pleroma.Web.CommonAPI
-  alias Pleroma.Web.OStatus
   alias Pleroma.Web.WebFinger
 
   def help_test(conn, _params) do
@@ -60,7 +59,7 @@ defmodule Pleroma.Web.TwitterAPI.UtilController do
       %Activity{id: activity_id} = Activity.get_create_by_object_ap_id(object.data["id"])
       redirect(conn, to: "/notice/#{activity_id}")
     else
-      {err, followee} = OStatus.find_or_make_user(acct)
+      {err, followee} = User.get_or_fetch(acct)
       avatar = User.avatar_url(followee)
       name = followee.nickname
       id = followee.id

lib/pleroma/web/web_finger/web_finger.ex

@@ -86,11 +86,17 @@ defmodule Pleroma.Web.WebFinger do
     |> XmlBuilder.to_doc()
   end
 
-  defp get_magic_key(magic_key) do
+  defp get_magic_key("data:application/magic-public-key," <> magic_key) do
-    "data:application/magic-public-key," <> magic_key = magic_key
     {:ok, magic_key}
-  rescue
+  end
-    MatchError -> {:error, "Missing magic key data."}
+
+  defp get_magic_key(nil) do
+    Logger.debug("Undefined magic key.")
+    {:ok, nil}
+  end
+
+  defp get_magic_key(_) do
+    {:error, "Missing magic key data."}
   end
 
   defp webfinger_from_xml(doc) do
@@ -187,6 +193,7 @@ defmodule Pleroma.Web.WebFinger do
     end
   end
 
+  @spec finger(String.t()) :: {:ok, map()} | {:error, any()}
   def finger(account) do
     account = String.trim_leading(account, "@")
 
@@ -220,8 +227,6 @@ defmodule Pleroma.Web.WebFinger do
       else
         with {:ok, doc} <- Jason.decode(body) do
           webfinger_from_json(doc)
-        else
-          {:error, e} -> e
         end
       end
     else

lib/pleroma/web/web_finger/web_finger_controller.ex

@@ -7,6 +7,7 @@ defmodule Pleroma.Web.WebFinger.WebFingerController do
 
   alias Pleroma.Web.WebFinger
 
+  plug(Pleroma.Plugs.SetFormatPlug)
   plug(Pleroma.Web.FederatingPlug)
 
   def host_meta(conn, _params) do
@@ -17,9 +18,8 @@ defmodule Pleroma.Web.WebFinger.WebFingerController do
     |> send_resp(200, xml)
   end
 
-  def webfinger(conn, %{"resource" => resource}) do
+  def webfinger(%{assigns: %{format: format}} = conn, %{"resource" => resource})
-    case get_format(conn) do
+      when format in ["xml", "xrd+xml"] do
-      n when n in ["xml", "xrd+xml"] ->
     with {:ok, response} <- WebFinger.webfinger(resource, "XML") do
       conn
       |> put_resp_content_type("application/xrd+xml")
@@ -27,20 +27,19 @@ defmodule Pleroma.Web.WebFinger.WebFingerController do
     else
       _e -> send_resp(conn, 404, "Couldn't find user")
     end
+  end
 
-      n when n in ["json", "jrd+json"] ->
+  def webfinger(%{assigns: %{format: format}} = conn, %{"resource" => resource})
+      when format in ["json", "jrd+json"] do
     with {:ok, response} <- WebFinger.webfinger(resource, "JSON") do
       json(conn, response)
     else
-          _e -> send_resp(conn, 404, "Couldn't find user")
+      _e ->
-        end
+        conn
-
+        |> put_status(404)
-      _ ->
+        |> json("Couldn't find user")
-        send_resp(conn, 404, "Unsupported format")
     end
   end
 
-  def webfinger(conn, _params) do
+  def webfinger(conn, _params), do: send_resp(conn, 400, "Bad Request")
-    send_resp(conn, 400, "Bad Request")
-  end
 end

mix.exs

@@ -143,7 +143,7 @@ defmodule Pleroma.Mixfile do
       {:telemetry, "~> 0.3"},
       {:prometheus_ex, "~> 3.0"},
       {:prometheus_plugs, "~> 1.1"},
-      {:prometheus_phoenix, "~> 1.2"},
+      {:prometheus_phoenix, "~> 1.3"},
       {:prometheus_ecto, "~> 1.4"},
       {:recon, github: "ferd/recon", tag: "2.4.0"},
       {:quack, "~> 0.1.1"},

mix.lock

@@ -56,22 +56,22 @@
   "nimble_parsec": {:hex, :nimble_parsec, "0.5.0", "90e2eca3d0266e5c53f8fbe0079694740b9c91b6747f2b7e3c5d21966bba8300", [:mix], [], "hexpm"},
   "parse_trans": {:hex, :parse_trans, "3.3.0", "09765507a3c7590a784615cfd421d101aec25098d50b89d7aa1d66646bc571c1", [:rebar3], [], "hexpm"},
   "pbkdf2_elixir": {:hex, :pbkdf2_elixir, "0.12.3", "6706a148809a29c306062862c803406e88f048277f6e85b68faf73291e820b84", [:mix], [], "hexpm"},
-  "phoenix": {:hex, :phoenix, "1.4.8", "c72dc3adeb49c70eb963a0ea24f7a064ec1588e651e84e1b7ad5ed8253c0b4a2", [:mix], [{:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}, {:phoenix_pubsub, "~> 1.1", [hex: :phoenix_pubsub, repo: "hexpm", optional: false]}, {:plug, "~> 1.8.1 or ~> 1.9", [hex: :plug, repo: "hexpm", optional: false]}, {:plug_cowboy, "~> 1.0 or ~> 2.0", [hex: :plug_cowboy, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm"},
+  "phoenix": {:hex, :phoenix, "1.4.9", "746d098e10741c334d88143d3c94cab1756435f94387a63441792e66ec0ee974", [:mix], [{:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}, {:phoenix_pubsub, "~> 1.1", [hex: :phoenix_pubsub, repo: "hexpm", optional: false]}, {:plug, "~> 1.8.1 or ~> 1.9", [hex: :plug, repo: "hexpm", optional: false]}, {:plug_cowboy, "~> 1.0 or ~> 2.0", [hex: :plug_cowboy, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm"},
   "phoenix_ecto": {:hex, :phoenix_ecto, "4.0.0", "c43117a136e7399ea04ecaac73f8f23ee0ffe3e07acfcb8062fe5f4c9f0f6531", [:mix], [{:ecto, "~> 3.0", [hex: :ecto, repo: "hexpm", optional: false]}, {:phoenix_html, "~> 2.9", [hex: :phoenix_html, repo: "hexpm", optional: true]}, {:plug, "~> 1.0", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"},
   "phoenix_html": {:hex, :phoenix_html, "2.13.1", "fa8f034b5328e2dfa0e4131b5569379003f34bc1fafdaa84985b0b9d2f12e68b", [:mix], [{:plug, "~> 1.5", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"},
   "phoenix_pubsub": {:hex, :phoenix_pubsub, "1.1.2", "496c303bdf1b2e98a9d26e89af5bba3ab487ba3a3735f74bf1f4064d2a845a3e", [:mix], [], "hexpm"},
   "pleroma_job_queue": {:hex, :pleroma_job_queue, "0.2.0", "879e660aa1cebe8dc6f0aaaa6aa48b4875e89cd961d4a585fd128e0773b31a18", [:mix], [], "hexpm"},
   "plug": {:hex, :plug, "1.8.2", "0bcce1daa420f189a6491f3940cc77ea7fb1919761175c9c3b59800d897440fc", [:mix], [{:mime, "~> 1.0", [hex: :mime, repo: "hexpm", optional: false]}, {:plug_crypto, "~> 1.0", [hex: :plug_crypto, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4", [hex: :telemetry, repo: "hexpm", optional: true]}], "hexpm"},
-  "plug_cowboy": {:hex, :plug_cowboy, "2.0.2", "6055f16868cc4882b24b6e1d63d2bada94fb4978413377a3b32ac16c18dffba2", [:mix], [{:cowboy, "~> 2.5", [hex: :cowboy, repo: "hexpm", optional: false]}, {:plug, "~> 1.7", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"},
+  "plug_cowboy": {:hex, :plug_cowboy, "2.1.0", "b75768153c3a8a9e8039d4b25bb9b14efbc58e9c4a6e6a270abff1cd30cbe320", [:mix], [{:cowboy, "~> 2.5", [hex: :cowboy, repo: "hexpm", optional: false]}, {:plug, "~> 1.7", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"},
   "plug_crypto": {:hex, :plug_crypto, "1.0.0", "18e49317d3fa343f24620ed22795ec29d4a5e602d52d1513ccea0b07d8ea7d4d", [:mix], [], "hexpm"},
   "plug_static_index_html": {:hex, :plug_static_index_html, "1.0.0", "840123d4d3975585133485ea86af73cb2600afd7f2a976f9f5fd8b3808e636a0", [:mix], [{:plug, "~> 1.0", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"},
   "poison": {:hex, :poison, "3.1.0", "d9eb636610e096f86f25d9a46f35a9facac35609a7591b3be3326e99a0484665", [:mix], [], "hexpm"},
   "poolboy": {:hex, :poolboy, "1.5.2", "392b007a1693a64540cead79830443abf5762f5d30cf50bc95cb2c1aaafa006b", [:rebar3], [], "hexpm"},
   "postgrex": {:hex, :postgrex, "0.14.3", "5754dee2fdf6e9e508cbf49ab138df964278700b764177e8f3871e658b345a1e", [:mix], [{:connection, "~> 1.0", [hex: :connection, repo: "hexpm", optional: false]}, {:db_connection, "~> 2.0", [hex: :db_connection, repo: "hexpm", optional: false]}, {:decimal, "~> 1.5", [hex: :decimal, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}], "hexpm"},
-  "prometheus": {:hex, :prometheus, "4.2.2", "a830e77b79dc6d28183f4db050a7cac926a6c58f1872f9ef94a35cd989aceef8", [:mix, :rebar3], [], "hexpm"},
+  "prometheus": {:hex, :prometheus, "4.4.1", "1e96073b3ed7788053768fea779cbc896ddc3bdd9ba60687f2ad50b252ac87d6", [:mix, :rebar3], [], "hexpm"},
   "prometheus_ecto": {:hex, :prometheus_ecto, "1.4.1", "6c768ea9654de871e5b32fab2eac348467b3021604ebebbcbd8bcbe806a65ed5", [:mix], [{:ecto, "~> 2.0 or ~> 3.0", [hex: :ecto, repo: "hexpm", optional: false]}, {:prometheus_ex, "~> 1.1 or ~> 2.0 or ~> 3.0", [hex: :prometheus_ex, repo: "hexpm", optional: false]}], "hexpm"},
   "prometheus_ex": {:hex, :prometheus_ex, "3.0.5", "fa58cfd983487fc5ead331e9a3e0aa622c67232b3ec71710ced122c4c453a02f", [:mix], [{:prometheus, "~> 4.0", [hex: :prometheus, repo: "hexpm", optional: false]}], "hexpm"},
-  "prometheus_phoenix": {:hex, :prometheus_phoenix, "1.2.1", "964a74dfbc055f781d3a75631e06ce3816a2913976d1df7830283aa3118a797a", [:mix], [{:phoenix, "~> 1.3", [hex: :phoenix, repo: "hexpm", optional: false]}, {:prometheus_ex, "~> 1.3 or ~> 2.0 or ~> 3.0", [hex: :prometheus_ex, repo: "hexpm", optional: false]}], "hexpm"},
+  "prometheus_phoenix": {:hex, :prometheus_phoenix, "1.3.0", "c4b527e0b3a9ef1af26bdcfbfad3998f37795b9185d475ca610fe4388fdd3bb5", [:mix], [{:phoenix, "~> 1.4", [hex: :phoenix, repo: "hexpm", optional: false]}, {:prometheus_ex, "~> 1.3 or ~> 2.0 or ~> 3.0", [hex: :prometheus_ex, repo: "hexpm", optional: false]}], "hexpm"},
   "prometheus_plugs": {:hex, :prometheus_plugs, "1.1.5", "25933d48f8af3a5941dd7b621c889749894d8a1082a6ff7c67cc99dec26377c5", [:mix], [{:accept, "~> 0.1", [hex: :accept, repo: "hexpm", optional: false]}, {:plug, "~> 1.0", [hex: :plug, repo: "hexpm", optional: false]}, {:prometheus_ex, "~> 1.1 or ~> 2.0 or ~> 3.0", [hex: :prometheus_ex, repo: "hexpm", optional: false]}, {:prometheus_process_collector, "~> 1.1", [hex: :prometheus_process_collector, repo: "hexpm", optional: true]}], "hexpm"},
   "prometheus_process_collector": {:hex, :prometheus_process_collector, "1.4.0", "6dbd39e3165b9ef1c94a7a820e9ffe08479f949dcdd431ed4aaea7b250eebfde", [:rebar3], [{:prometheus, "~> 4.0", [hex: :prometheus, repo: "hexpm", optional: false]}], "hexpm"},
   "quack": {:hex, :quack, "0.1.1", "cca7b4da1a233757fdb44b3334fce80c94785b3ad5a602053b7a002b5a8967bf", [:mix], [{:poison, ">= 1.0.0", [hex: :poison, repo: "hexpm", optional: false]}, {:tesla, "~> 1.2.0", [hex: :tesla, repo: "hexpm", optional: false]}], "hexpm"},

test/fixtures/tesla_mock/kpherox@mstdn.jp.xml

@@ -0,0 +1,10 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<XRD xmlns="http://docs.oasis-open.org/ns/xri/xrd-1.0">
+  <Subject>acct:kPherox@mstdn.jp</Subject>
+  <Alias>https://mstdn.jp/@kPherox</Alias>
+  <Alias>https://mstdn.jp/users/kPherox</Alias>
+  <Link rel="http://webfinger.net/rel/profile-page" type="text/html" href="https://mstdn.jp/@kPherox"/>
+  <Link rel="http://schemas.google.com/g/2010#updates-from" type="application/atom+xml" href="https://mstdn.jp/users/kPherox.atom"/>
+  <Link rel="self" type="application/activity+json" href="https://mstdn.jp/users/kPherox"/>
+  <Link rel="http://ostatus.org/schema/1.0/subscribe" template="https://mstdn.jp/authorize_interaction?acct={uri}"/>
+</XRD>

test/fixtures/tesla_mock/wedistribute-article.json

@@ -0,0 +1,18 @@
+{
+  "@context": [
+    "https://www.w3.org/ns/activitystreams"
+  ],
+  "type": "Article",
+  "name": "The end is near: Mastodon plans to drop OStatus support",
+  "content": "<!-- wp:paragraph {\"dropCap\":true} -->\n<p class=\"has-drop-cap\">The days of OStatus are numbered. The venerable protocol has served as a glue between many different types of servers since the early days of the Fediverse, connecting StatusNet (now GNU Social) to Friendica, Hubzilla, Mastodon, and Pleroma.</p>\n<!-- /wp:paragraph -->\n\n<!-- wp:paragraph -->\n<p>Now that many fediverse platforms support ActivityPub as a successor protocol, Mastodon appears to be drawing a line in the sand. In <a href=\"https://www.patreon.com/posts/mastodon-2-9-and-28121681\">a Patreon update</a>, Eugen Rochko writes:</p>\n<!-- /wp:paragraph -->\n\n<!-- wp:quote -->\n<blockquote class=\"wp-block-quote\"><p>...OStatus...has overstayed its welcome in the code...and now that most of the network uses ActivityPub, it's time for it to go. </p><cite>Eugen Rochko, Mastodon creator</cite></blockquote>\n<!-- /wp:quote -->\n\n<!-- wp:paragraph -->\n<p>The <a href=\"https://github.com/tootsuite/mastodon/pull/11205\">pull request</a> to remove Pubsubhubbub and Salmon, two of the main components of OStatus, has already been merged into Mastodon's master branch.</p>\n<!-- /wp:paragraph -->\n\n<!-- wp:paragraph -->\n<p>Some projects will be left in the dark as a side effect of this. GNU Social and PostActiv, for example, both only communicate using OStatus. While <a href=\"https://mastodon.social/@dansup/102076573310057902\">some discussion</a> exists regarding adopting ActivityPub for GNU Social, and <a href=\"https://notabug.org/diogo/gnu-social/src/activitypub/plugins/ActivityPub\">a plugin is in development</a>, it hasn't been formally adopted yet. We just hope that the <a href=\"https://status.fsf.org/main/public\">Free Software Foundation's instance</a> gets updated in time!</p>\n<!-- /wp:paragraph -->",
+  "summary": "One of the largest platforms in the federated social web is dropping the protocol that it started with.",
+  "attributedTo": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog",
+  "url": "https://wedistribute.org/2019/07/mastodon-drops-ostatus/",
+  "to": [
+    "https://www.w3.org/ns/activitystreams#Public",
+    "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog/followers"
+  ],
+  "id": "https://wedistribute.org/wp-json/pterotype/v1/object/85810",
+  "likes": "https://wedistribute.org/wp-json/pterotype/v1/object/85810/likes",
+  "shares": "https://wedistribute.org/wp-json/pterotype/v1/object/85810/shares"
+}

test/fixtures/tesla_mock/wedistribute-user.json

@@ -0,0 +1,31 @@
+{
+  "@context": [
+    "https://www.w3.org/ns/activitystreams",
+    "https://w3id.org/security/v1",
+    {
+      "manuallyApprovesFollowers": "as:manuallyApprovesFollowers"
+    }
+  ],
+  "type": "Organization",
+  "id": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog",
+  "following": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog/following",
+  "followers": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog/followers",
+  "liked": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog/liked",
+  "inbox": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog/inbox",
+  "outbox": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog/outbox",
+  "name": "We Distribute",
+  "preferredUsername": "blog",
+  "summary": "<p>Connecting many threads in the federated web. We Distribute is an independent publication dedicated to the fediverse, decentralization, P2P technologies, and Free Software!</p>",
+  "url": "https://wedistribute.org/",
+  "publicKey": {
+    "id": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog#publicKey",
+    "owner": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog",
+    "publicKeyPem": "-----BEGIN PUBLIC KEY-----\r\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA1bmUJ+y8PS8JFVi0KugN\r\nFl4pLvLog3V2lsV9ftmCXpveB/WJx66Tr1fQLsU3qYvQFc8UPGWD52zV4RENR1SN\r\nx0O6T2f97KUbRM+Ckow7Jyjtssgl+Mqq8UBZQ/+H8I/1Vpvt5E5hUykhFgwzx9qg\r\nzoIA3OK7alOpQbSoKXo0QcOh6yTRUnMSRMJAgUoZJzzXI/FmH/DtKr7ziQ1T2KWs\r\nVs8mWnTb/OlCxiheLuMlmJNMF+lPyVthvMIxF6Z5gV9d5QAmASSCI628e6uH2EUF\r\nDEEF5jo+Z5ffeNv28953lrnM+VB/wTjl3tYA+zCQeAmUPksX3E+YkXGxj+4rxBAY\r\n8wIDAQAB\r\n-----END PUBLIC KEY-----"
+  },
+  "manuallyApprovesFollowers": false,
+  "icon": {
+    "url": "https://wedistribute.org/wp-content/uploads/2019/02/b067de423757a538.png",
+    "type": "Image",
+    "mediaType": "image/png"
+  }
+}

test/notification_test.exs

@@ -42,6 +42,28 @@ defmodule Pleroma.NotificationTest do
 
       assert notification.user_id == subscriber.id
     end
+
+    test "does not create a notification for subscribed users if status is a reply" do
+      user = insert(:user)
+      other_user = insert(:user)
+      subscriber = insert(:user)
+
+      User.subscribe(subscriber, other_user)
+
+      {:ok, activity} = CommonAPI.post(user, %{"status" => "test post"})
+
+      {:ok, _reply_activity} =
+        CommonAPI.post(other_user, %{
+          "status" => "test reply",
+          "in_reply_to_status_id" => activity.id
+        })
+
+      user_notifications = Notification.for_user(user)
+      assert length(user_notifications) == 1
+
+      subscriber_notifications = Notification.for_user(subscriber)
+      assert Enum.empty?(subscriber_notifications)
+    end
   end
 
   describe "create_notification" do

test/object/fetcher_test.exs

@@ -110,6 +110,13 @@ defmodule Pleroma.Object.FetcherTest do
       assert object
     end
 
+    test "it can fetch wedistribute articles" do
+      {:ok, object} =
+        Fetcher.fetch_object_from_id("https://wedistribute.org/wp-json/pterotype/v1/object/85810")
+
+      assert object
+    end
+
     test "all objects with fake directions are rejected by the object fetcher" do
       assert {:error, _} =
                Fetcher.fetch_and_contain_remote_object_from_id(

test/plugs/authentication_plug_test.exs

@@ -9,7 +9,6 @@ defmodule Pleroma.Plugs.AuthenticationPlugTest do
   alias Pleroma.User
 
   import ExUnit.CaptureLog
-  import Mock
 
   setup %{conn: conn} do
     user = %User{
@@ -67,14 +66,13 @@ defmodule Pleroma.Plugs.AuthenticationPlugTest do
       refute AuthenticationPlug.checkpw("test-password1", hash)
     end
 
+    @tag :skip_on_mac
     test "check sha512-crypt hash" do
       hash =
         "$6$9psBWV8gxkGOZWBz$PmfCycChoxeJ3GgGzwvhlgacb9mUoZ.KUXNCssekER4SJ7bOK53uXrHNb2e4i8yPFgSKyzaW9CcmrDXWIEMtD1"
 
-      with_mock :crypt, crypt: fn _password, password_hash -> password_hash end do
       assert AuthenticationPlug.checkpw("password", hash)
     end
-    end
 
     test "it returns false when hash invalid" do
       hash =

test/plugs/legacy_authentication_plug_test.exs

@@ -5,19 +5,18 @@
 defmodule Pleroma.Plugs.LegacyAuthenticationPlugTest do
   use Pleroma.Web.ConnCase
 
+  import Pleroma.Factory
+
   alias Pleroma.Plugs.LegacyAuthenticationPlug
   alias Pleroma.User
 
-  import Mock
-
   setup do
-    # password is "password"
+    user =
-    user = %User{
+      insert(:user,
-      id: 1,
+        password: "password",
-      name: "dude",
         password_hash:
           "$6$9psBWV8gxkGOZWBz$PmfCycChoxeJ3GgGzwvhlgacb9mUoZ.KUXNCssekER4SJ7bOK53uXrHNb2e4i8yPFgSKyzaW9CcmrDXWIEMtD1"
-    }
+      )
 
     %{user: user}
   end
@@ -36,6 +35,7 @@ defmodule Pleroma.Plugs.LegacyAuthenticationPlugTest do
     assert ret_conn == conn
   end
 
+  @tag :skip_on_mac
   test "it authenticates the auth_user if present and password is correct and resets the password",
        %{
          conn: conn,
@@ -46,22 +46,12 @@ defmodule Pleroma.Plugs.LegacyAuthenticationPlugTest do
       |> assign(:auth_credentials, %{username: "dude", password: "password"})
       |> assign(:auth_user, user)
 
-    conn =
+    conn = LegacyAuthenticationPlug.call(conn, %{})
-      with_mocks([
+
-        {:crypt, [], [crypt: fn _password, password_hash -> password_hash end]},
+    assert conn.assigns.user.id == user.id
-        {User, [],
-         [
-           reset_password: fn user, %{password: password, password_confirmation: password} ->
-             {:ok, user}
-           end
-         ]}
-      ]) do
-        LegacyAuthenticationPlug.call(conn, %{})
-      end
-
-    assert conn.assigns.user == user
   end
 
+  @tag :skip_on_mac
   test "it does nothing if the password is wrong", %{
     conn: conn,
     user: user

test/plugs/set_format_plug_test.exs

@@ -0,0 +1,38 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2018 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Plugs.SetFormatPlugTest do
+  use ExUnit.Case, async: true
+  use Plug.Test
+
+  alias Pleroma.Plugs.SetFormatPlug
+
+  test "set format from params" do
+    conn =
+      :get
+      |> conn("/cofe?_format=json")
+      |> SetFormatPlug.call([])
+
+    assert %{format: "json"} == conn.assigns
+  end
+
+  test "set format from header" do
+    conn =
+      :get
+      |> conn("/cofe")
+      |> put_private(:phoenix_format, "xml")
+      |> SetFormatPlug.call([])
+
+    assert %{format: "xml"} == conn.assigns
+  end
+
+  test "doesn't set format" do
+    conn =
+      :get
+      |> conn("/cofe")
+      |> SetFormatPlug.call([])
+
+    refute conn.assigns[:format]
+  end
+end

test/signature_test.exs

@@ -48,16 +48,14 @@ defmodule Pleroma.SignatureTest do
 
     test "it returns error when not found user" do
       assert capture_log(fn ->
-               assert Signature.fetch_public_key(make_fake_conn("test-ap_id")) ==
+               assert Signature.fetch_public_key(make_fake_conn("test-ap_id")) == {:error, :error}
-                        {:error, :error}
              end) =~ "[error] Could not decode user"
     end
 
     test "it returns error if public key is empty" do
       user = insert(:user, %{info: %{source_data: %{"publicKey" => %{}}}})
 
-      assert Signature.fetch_public_key(make_fake_conn(user.ap_id)) ==
+      assert Signature.fetch_public_key(make_fake_conn(user.ap_id)) == {:error, :error}
-               {:error, :error}
     end
   end
 
@@ -65,8 +63,7 @@ defmodule Pleroma.SignatureTest do
     test "it returns key" do
       ap_id = "https://mastodon.social/users/lambadalambda"
 
-      assert Signature.refetch_public_key(make_fake_conn(ap_id)) ==
+      assert Signature.refetch_public_key(make_fake_conn(ap_id)) == {:ok, @rsa_public_key}
-               {:ok, @rsa_public_key}
     end
 
     test "it returns error when not found user" do
@@ -105,4 +102,16 @@ defmodule Pleroma.SignatureTest do
              ) == {:error, []}
     end
   end
+
+  describe "key_id_to_actor_id/1" do
+    test "it properly deduces the actor id for misskey" do
+      assert Signature.key_id_to_actor_id("https://example.com/users/1234/publickey") ==
+               "https://example.com/users/1234"
+    end
+
+    test "it properly deduces the actor id for mastodon and pleroma" do
+      assert Signature.key_id_to_actor_id("https://example.com/users/1234#main-key") ==
+               "https://example.com/users/1234"
+    end
+  end
 end

test/support/factory.ex

@@ -118,9 +118,12 @@ defmodule Pleroma.Factory do
   def note_activity_factory(attrs \\ %{}) do
     user = attrs[:user] || insert(:user)
     note = attrs[:note] || insert(:note, user: user)
-    attrs = Map.drop(attrs, [:user, :note])
 
-    data = %{
+    data_attrs = attrs[:data_attrs] || %{}
+    attrs = Map.drop(attrs, [:user, :note, :data_attrs])
+
+    data =
+      %{
         "id" => Pleroma.Web.ActivityPub.Utils.generate_activity_id(),
         "type" => "Create",
         "actor" => note.data["actor"],
@@ -129,6 +132,7 @@ defmodule Pleroma.Factory do
         "published" => DateTime.utc_now() |> DateTime.to_iso8601(),
         "context" => note.data["context"]
       }
+      |> Map.merge(data_attrs)
 
     %Pleroma.Activity{
       data: data,

test/support/http_request_mock.ex

@@ -301,6 +301,22 @@ defmodule HttpRequestMock do
      }}
   end
 
+  def get("https://wedistribute.org/wp-json/pterotype/v1/object/85810", _, _, _) do
+    {:ok,
+     %Tesla.Env{
+       status: 200,
+       body: File.read!("test/fixtures/tesla_mock/wedistribute-article.json")
+     }}
+  end
+
+  def get("https://wedistribute.org/wp-json/pterotype/v1/actor/-blog", _, _, _) do
+    {:ok,
+     %Tesla.Env{
+       status: 200,
+       body: File.read!("test/fixtures/tesla_mock/wedistribute-user.json")
+     }}
+  end
+
   def get("http://mastodon.example.org/users/admin", _, _, Accept: "application/activity+json") do
     {:ok,
      %Tesla.Env{
@@ -614,6 +630,15 @@ defmodule HttpRequestMock do
      }}
   end
 
+  def get(
+        "https://social.heldscal.la/.well-known/webfinger?resource=invalid_content@social.heldscal.la",
+        _,
+        _,
+        Accept: "application/xrd+xml,application/jrd+json"
+      ) do
+    {:ok, %Tesla.Env{status: 200, body: ""}}
+  end
+
   def get("http://framatube.org/.well-known/host-meta", _, _, _) do
     {:ok,
      %Tesla.Env{
@@ -915,6 +940,14 @@ defmodule HttpRequestMock do
     {:ok, %Tesla.Env{status: 404, body: ""}}
   end
 
+  def get("https://mstdn.jp/.well-known/webfinger?resource=acct:kpherox@mstdn.jp", _, _, _) do
+    {:ok,
+     %Tesla.Env{
+       status: 200,
+       body: File.read!("test/fixtures/tesla_mock/kpherox@mstdn.jp.xml")
+     }}
+  end
+
   def get(url, query, body, headers) do
     {:error,
      "Not implemented the mock response for get #{inspect(url)}, #{query}, #{inspect(body)}, #{

test/tasks/user_test.exs

@@ -5,6 +5,9 @@
 defmodule Mix.Tasks.Pleroma.UserTest do
   alias Pleroma.Repo
   alias Pleroma.User
+  alias Pleroma.Web.OAuth.Authorization
+  alias Pleroma.Web.OAuth.Token
+
   use Pleroma.DataCase
 
   import Pleroma.Factory
@@ -327,6 +330,13 @@ defmodule Mix.Tasks.Pleroma.UserTest do
       assert_received {:mix_shell, :info, [message]}
       assert message =~ "Invite for token #{invite.token} was revoked."
     end
+
+    test "it prints an error message when invite is not exist" do
+      Mix.Tasks.Pleroma.User.run(["revoke_invite", "foo"])
+
+      assert_received {:mix_shell, :error, [message]}
+      assert message =~ "No invite found"
+    end
   end
 
   describe "running delete_activities" do
@@ -337,6 +347,13 @@ defmodule Mix.Tasks.Pleroma.UserTest do
       assert_received {:mix_shell, :info, [message]}
       assert message == "User #{nickname} statuses deleted."
     end
+
+    test "it prints an error message when user is not exist" do
+      Mix.Tasks.Pleroma.User.run(["delete_activities", "foo"])
+
+      assert_received {:mix_shell, :error, [message]}
+      assert message =~ "No local user"
+    end
   end
 
   describe "running toggle_confirmed" do
@@ -364,6 +381,13 @@ defmodule Mix.Tasks.Pleroma.UserTest do
       refute user.info.confirmation_pending
       refute user.info.confirmation_token
     end
+
+    test "it prints an error message when user is not exist" do
+      Mix.Tasks.Pleroma.User.run(["toggle_confirmed", "foo"])
+
+      assert_received {:mix_shell, :error, [message]}
+      assert message =~ "No local user"
+    end
   end
 
   describe "search" do
@@ -386,4 +410,64 @@ defmodule Mix.Tasks.Pleroma.UserTest do
                User.Search.search("moon fediverse", for_user: user) |> Enum.map(& &1.id)
     end
   end
+
+  describe "signing out" do
+    test "it deletes all user's tokens and authorizations" do
+      user = insert(:user)
+      insert(:oauth_token, user: user)
+      insert(:oauth_authorization, user: user)
+
+      assert Repo.get_by(Token, user_id: user.id)
+      assert Repo.get_by(Authorization, user_id: user.id)
+
+      :ok = Mix.Tasks.Pleroma.User.run(["sign_out", user.nickname])
+
+      refute Repo.get_by(Token, user_id: user.id)
+      refute Repo.get_by(Authorization, user_id: user.id)
+    end
+
+    test "it prints an error message when user is not exist" do
+      Mix.Tasks.Pleroma.User.run(["sign_out", "foo"])
+
+      assert_received {:mix_shell, :error, [message]}
+      assert message =~ "No local user"
+    end
+  end
+
+  describe "tagging" do
+    test "it add tags to a user" do
+      user = insert(:user)
+
+      :ok = Mix.Tasks.Pleroma.User.run(["tag", user.nickname, "pleroma"])
+
+      user = User.get_cached_by_nickname(user.nickname)
+      assert "pleroma" in user.tags
+    end
+
+    test "it prints an error message when user is not exist" do
+      Mix.Tasks.Pleroma.User.run(["tag", "foo"])
+
+      assert_received {:mix_shell, :error, [message]}
+      assert message =~ "Could not change user tags"
+    end
+  end
+
+  describe "untagging" do
+    test "it deletes tags from a user" do
+      user = insert(:user, tags: ["pleroma"])
+      assert "pleroma" in user.tags
+
+      :ok = Mix.Tasks.Pleroma.User.run(["untag", user.nickname, "pleroma"])
+
+      user = User.get_cached_by_nickname(user.nickname)
+      assert Enum.empty?(user.tags)
+    end
+
+    test "it prints an error message when user is not exist" do
+      Mix.Tasks.Pleroma.User.run(["untag", "foo"])
+
+      assert_received {:mix_shell, :error, [message]}
+      assert message =~ "Could not change user tags"
+    end
+  end
 end

test/test_helper.exs

@@ -2,7 +2,8 @@
 # Copyright © 2017-2018 Pleroma Authors <https://pleroma.social/>
 # SPDX-License-Identifier: AGPL-3.0-only
 
-ExUnit.start()
+os_exclude = if :os.type() == {:unix, :darwin}, do: [skip_on_mac: true], else: []
+ExUnit.start(exclude: os_exclude)
 Ecto.Adapters.SQL.Sandbox.mode(Pleroma.Repo, :manual)
 Mox.defmock(Pleroma.ReverseProxy.ClientMock, for: Pleroma.ReverseProxy.Client)
 {:ok, _} = Application.ensure_all_started(:ex_machina)

test/upload/filter/dedupe_test.exs

@@ -25,7 +25,7 @@ defmodule Pleroma.Upload.Filter.DedupeTest do
 
     assert {
              :ok,
-             %Pleroma.Upload{id: @shasum, path: "#{@shasum}.jpg"}
+             %Pleroma.Upload{id: @shasum, path: @shasum <> ".jpg"}
            } = Dedupe.filter(upload)
   end
 end

test/upload_test.exs

@@ -122,24 +122,6 @@ defmodule Pleroma.UploadTest do
       assert String.starts_with?(url, Pleroma.Web.base_url() <> "/media/")
     end
 
-    test "returns a media url with configured base_url" do
-      base_url = "https://cache.pleroma.social"
-
-      File.cp!("test/fixtures/image.jpg", "test/fixtures/image_tmp.jpg")
-
-      file = %Plug.Upload{
-        content_type: "image/jpg",
-        path: Path.absname("test/fixtures/image_tmp.jpg"),
-        filename: "image.jpg"
-      }
-
-      {:ok, data} = Upload.store(file, base_url: base_url)
-
-      assert %{"url" => [%{"href" => url}]} = data
-
-      assert String.starts_with?(url, base_url <> "/media/")
-    end
-
     test "copies the file to the configured folder with deduping" do
       File.cp!("test/fixtures/image.jpg", "test/fixtures/image_tmp.jpg")
 
@@ -266,4 +248,32 @@ defmodule Pleroma.UploadTest do
                "%3A%3F%23%5B%5D%40%21%24%26%5C%27%28%29%2A%2B%2C%3B%3D.jpg"
     end
   end
+
+  describe "Setting a custom base_url for uploaded media" do
+    setup do
+      Pleroma.Config.put([Pleroma.Upload, :base_url], "https://cache.pleroma.social")
+
+      on_exit(fn ->
+        Pleroma.Config.put([Pleroma.Upload, :base_url], nil)
+      end)
+    end
+
+    test "returns a media url with configured base_url" do
+      base_url = Pleroma.Config.get([Pleroma.Upload, :base_url])
+
+      File.cp!("test/fixtures/image.jpg", "test/fixtures/image_tmp.jpg")
+
+      file = %Plug.Upload{
+        content_type: "image/jpg",
+        path: Path.absname("test/fixtures/image_tmp.jpg"),
+        filename: "image.jpg"
+      }
+
+      {:ok, data} = Upload.store(file, base_url: base_url)
+
+      assert %{"url" => [%{"href" => url}]} = data
+
+      refute String.starts_with?(url, base_url <> "/media/")
+    end
+  end
 end

test/user_test.exs

@@ -824,6 +824,48 @@ defmodule Pleroma.UserTest do
       assert User.blocks?(user, collateral_user)
     end
 
+    test "does not block domain with same end" do
+      user = insert(:user)
+
+      collateral_user =
+        insert(:user, %{ap_id: "https://another-awful-and-rude-instance.com/user/bully"})
+
+      {:ok, user} = User.block_domain(user, "awful-and-rude-instance.com")
+
+      refute User.blocks?(user, collateral_user)
+    end
+
+    test "does not block domain with same end if wildcard added" do
+      user = insert(:user)
+
+      collateral_user =
+        insert(:user, %{ap_id: "https://another-awful-and-rude-instance.com/user/bully"})
+
+      {:ok, user} = User.block_domain(user, "*.awful-and-rude-instance.com")
+
+      refute User.blocks?(user, collateral_user)
+    end
+
+    test "blocks domain with wildcard for subdomain" do
+      user = insert(:user)
+
+      user_from_subdomain =
+        insert(:user, %{ap_id: "https://subdomain.awful-and-rude-instance.com/user/bully"})
+
+      user_with_two_subdomains =
+        insert(:user, %{
+          ap_id: "https://subdomain.second_subdomain.awful-and-rude-instance.com/user/bully"
+        })
+
+      user_domain = insert(:user, %{ap_id: "https://awful-and-rude-instance.com/user/bully"})
+
+      {:ok, user} = User.block_domain(user, "*.awful-and-rude-instance.com")
+
+      assert User.blocks?(user, user_from_subdomain)
+      assert User.blocks?(user, user_with_two_subdomains)
+      assert User.blocks?(user, user_domain)
+    end
+
     test "unblocks domains" do
       user = insert(:user)
       collateral_user = insert(:user, %{ap_id: "https://awful-and-rude-instance.com/user/bully"})

test/web/activity_pub/activity_pub_test.exs

@@ -6,11 +6,9 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubTest do
   use Pleroma.DataCase
   alias Pleroma.Activity
   alias Pleroma.Builders.ActivityBuilder
-  alias Pleroma.Instances
   alias Pleroma.Object
   alias Pleroma.User
   alias Pleroma.Web.ActivityPub.ActivityPub
-  alias Pleroma.Web.ActivityPub.Publisher
   alias Pleroma.Web.ActivityPub.Utils
   alias Pleroma.Web.CommonAPI
 
@@ -1083,113 +1081,6 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubTest do
            } = activity
   end
 
-  describe "publish_one/1" do
-    test_with_mock "calls `Instances.set_reachable` on successful federation if `unreachable_since` is not specified",
-                   Instances,
-                   [:passthrough],
-                   [] do
-      actor = insert(:user)
-      inbox = "http://200.site/users/nick1/inbox"
-
-      assert {:ok, _} = Publisher.publish_one(%{inbox: inbox, json: "{}", actor: actor, id: 1})
-
-      assert called(Instances.set_reachable(inbox))
-    end
-
-    test_with_mock "calls `Instances.set_reachable` on successful federation if `unreachable_since` is set",
-                   Instances,
-                   [:passthrough],
-                   [] do
-      actor = insert(:user)
-      inbox = "http://200.site/users/nick1/inbox"
-
-      assert {:ok, _} =
-               Publisher.publish_one(%{
-                 inbox: inbox,
-                 json: "{}",
-                 actor: actor,
-                 id: 1,
-                 unreachable_since: NaiveDateTime.utc_now()
-               })
-
-      assert called(Instances.set_reachable(inbox))
-    end
-
-    test_with_mock "does NOT call `Instances.set_reachable` on successful federation if `unreachable_since` is nil",
-                   Instances,
-                   [:passthrough],
-                   [] do
-      actor = insert(:user)
-      inbox = "http://200.site/users/nick1/inbox"
-
-      assert {:ok, _} =
-               Publisher.publish_one(%{
-                 inbox: inbox,
-                 json: "{}",
-                 actor: actor,
-                 id: 1,
-                 unreachable_since: nil
-               })
-
-      refute called(Instances.set_reachable(inbox))
-    end
-
-    test_with_mock "calls `Instances.set_unreachable` on target inbox on non-2xx HTTP response code",
-                   Instances,
-                   [:passthrough],
-                   [] do
-      actor = insert(:user)
-      inbox = "http://404.site/users/nick1/inbox"
-
-      assert {:error, _} = Publisher.publish_one(%{inbox: inbox, json: "{}", actor: actor, id: 1})
-
-      assert called(Instances.set_unreachable(inbox))
-    end
-
-    test_with_mock "it calls `Instances.set_unreachable` on target inbox on request error of any kind",
-                   Instances,
-                   [:passthrough],
-                   [] do
-      actor = insert(:user)
-      inbox = "http://connrefused.site/users/nick1/inbox"
-
-      assert {:error, _} = Publisher.publish_one(%{inbox: inbox, json: "{}", actor: actor, id: 1})
-
-      assert called(Instances.set_unreachable(inbox))
-    end
-
-    test_with_mock "does NOT call `Instances.set_unreachable` if target is reachable",
-                   Instances,
-                   [:passthrough],
-                   [] do
-      actor = insert(:user)
-      inbox = "http://200.site/users/nick1/inbox"
-
-      assert {:ok, _} = Publisher.publish_one(%{inbox: inbox, json: "{}", actor: actor, id: 1})
-
-      refute called(Instances.set_unreachable(inbox))
-    end
-
-    test_with_mock "does NOT call `Instances.set_unreachable` if target instance has non-nil `unreachable_since`",
-                   Instances,
-                   [:passthrough],
-                   [] do
-      actor = insert(:user)
-      inbox = "http://connrefused.site/users/nick1/inbox"
-
-      assert {:error, _} =
-               Publisher.publish_one(%{
-                 inbox: inbox,
-                 json: "{}",
-                 actor: actor,
-                 id: 1,
-                 unreachable_since: NaiveDateTime.utc_now()
-               })
-
-      refute called(Instances.set_unreachable(inbox))
-    end
-  end
-
   test "fetch_activities/2 returns activities addressed to a list " do
     user = insert(:user)
     member = insert(:user)

test/web/activity_pub/mrf/mrf_test.exs

@@ -0,0 +1,46 @@
+defmodule Pleroma.Web.ActivityPub.MRFTest do
+  use ExUnit.Case, async: true
+  alias Pleroma.Web.ActivityPub.MRF
+
+  test "subdomains_regex/1" do
+    assert MRF.subdomains_regex(["unsafe.tld", "*.unsafe.tld"]) == [
+             ~r/^unsafe.tld$/,
+             ~r/^(.*\.)*unsafe.tld$/
+           ]
+  end
+
+  describe "subdomain_match/2" do
+    test "common domains" do
+      regexes = MRF.subdomains_regex(["unsafe.tld", "unsafe2.tld"])
+
+      assert regexes == [~r/^unsafe.tld$/, ~r/^unsafe2.tld$/]
+
+      assert MRF.subdomain_match?(regexes, "unsafe.tld")
+      assert MRF.subdomain_match?(regexes, "unsafe2.tld")
+
+      refute MRF.subdomain_match?(regexes, "example.com")
+    end
+
+    test "wildcard domains with one subdomain" do
+      regexes = MRF.subdomains_regex(["*.unsafe.tld"])
+
+      assert regexes == [~r/^(.*\.)*unsafe.tld$/]
+
+      assert MRF.subdomain_match?(regexes, "unsafe.tld")
+      assert MRF.subdomain_match?(regexes, "sub.unsafe.tld")
+      refute MRF.subdomain_match?(regexes, "anotherunsafe.tld")
+      refute MRF.subdomain_match?(regexes, "unsafe.tldanother")
+    end
+
+    test "wildcard domains with two subdomains" do
+      regexes = MRF.subdomains_regex(["*.unsafe.tld"])
+
+      assert regexes == [~r/^(.*\.)*unsafe.tld$/]
+
+      assert MRF.subdomain_match?(regexes, "unsafe.tld")
+      assert MRF.subdomain_match?(regexes, "sub.sub.unsafe.tld")
+      refute MRF.subdomain_match?(regexes, "sub.anotherunsafe.tld")
+      refute MRF.subdomain_match?(regexes, "sub.unsafe.tldanother")
+    end
+  end
+end

test/web/activity_pub/mrf/simple_policy_test.exs

@@ -49,6 +49,19 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do
 
       assert SimplePolicy.filter(local_message) == {:ok, local_message}
     end
+
+    test "match with wildcard domain" do
+      Config.put([:mrf_simple, :media_removal], ["*.remote.instance"])
+      media_message = build_media_message()
+      local_message = build_local_message()
+
+      assert SimplePolicy.filter(media_message) ==
+               {:ok,
+                media_message
+                |> Map.put("object", Map.delete(media_message["object"], "attachment"))}
+
+      assert SimplePolicy.filter(local_message) == {:ok, local_message}
+    end
   end
 
   describe "when :media_nsfw" do
@@ -74,6 +87,20 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do
 
       assert SimplePolicy.filter(local_message) == {:ok, local_message}
     end
+
+    test "match with wildcard domain" do
+      Config.put([:mrf_simple, :media_nsfw], ["*.remote.instance"])
+      media_message = build_media_message()
+      local_message = build_local_message()
+
+      assert SimplePolicy.filter(media_message) ==
+               {:ok,
+                media_message
+                |> put_in(["object", "tag"], ["foo", "nsfw"])
+                |> put_in(["object", "sensitive"], true)}
+
+      assert SimplePolicy.filter(local_message) == {:ok, local_message}
+    end
   end
 
   defp build_media_message do
@@ -106,6 +133,15 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do
       assert SimplePolicy.filter(report_message) == {:reject, nil}
       assert SimplePolicy.filter(local_message) == {:ok, local_message}
     end
+
+    test "match with wildcard domain" do
+      Config.put([:mrf_simple, :report_removal], ["*.remote.instance"])
+      report_message = build_report_message()
+      local_message = build_local_message()
+
+      assert SimplePolicy.filter(report_message) == {:reject, nil}
+      assert SimplePolicy.filter(local_message) == {:ok, local_message}
+    end
   end
 
   defp build_report_message do
@@ -146,6 +182,27 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do
       assert SimplePolicy.filter(local_message) == {:ok, local_message}
     end
 
+    test "match with wildcard domain" do
+      {actor, ftl_message} = build_ftl_actor_and_message()
+
+      ftl_message_actor_host =
+        ftl_message
+        |> Map.fetch!("actor")
+        |> URI.parse()
+        |> Map.fetch!(:host)
+
+      Config.put([:mrf_simple, :federated_timeline_removal], ["*." <> ftl_message_actor_host])
+      local_message = build_local_message()
+
+      assert {:ok, ftl_message} = SimplePolicy.filter(ftl_message)
+      assert actor.follower_address in ftl_message["to"]
+      refute actor.follower_address in ftl_message["cc"]
+      refute "https://www.w3.org/ns/activitystreams#Public" in ftl_message["to"]
+      assert "https://www.w3.org/ns/activitystreams#Public" in ftl_message["cc"]
+
+      assert SimplePolicy.filter(local_message) == {:ok, local_message}
+    end
+
     test "has a matching host but only as:Public in to" do
       {_actor, ftl_message} = build_ftl_actor_and_message()
 
@@ -192,6 +249,14 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do
 
       assert SimplePolicy.filter(remote_message) == {:reject, nil}
     end
+
+    test "match with wildcard domain" do
+      Config.put([:mrf_simple, :reject], ["*.remote.instance"])
+
+      remote_message = build_remote_message()
+
+      assert SimplePolicy.filter(remote_message) == {:reject, nil}
+    end
   end
 
   describe "when :accept" do
@@ -224,6 +289,16 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do
       assert SimplePolicy.filter(local_message) == {:ok, local_message}
       assert SimplePolicy.filter(remote_message) == {:ok, remote_message}
     end
+
+    test "match with wildcard domain" do
+      Config.put([:mrf_simple, :accept], ["*.remote.instance"])
+
+      local_message = build_local_message()
+      remote_message = build_remote_message()
+
+      assert SimplePolicy.filter(local_message) == {:ok, local_message}
+      assert SimplePolicy.filter(remote_message) == {:ok, remote_message}
+    end
   end
 
   describe "when :avatar_removal" do
@@ -251,6 +326,15 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do
 
       refute filtered["icon"]
     end
+
+    test "match with wildcard domain" do
+      Config.put([:mrf_simple, :avatar_removal], ["*.remote.instance"])
+
+      remote_user = build_remote_user()
+      {:ok, filtered} = SimplePolicy.filter(remote_user)
+
+      refute filtered["icon"]
+    end
   end
 
   describe "when :banner_removal" do
@@ -278,6 +362,15 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do
 
       refute filtered["image"]
     end
+
+    test "match with wildcard domain" do
+      Config.put([:mrf_simple, :banner_removal], ["*.remote.instance"])
+
+      remote_user = build_remote_user()
+      {:ok, filtered} = SimplePolicy.filter(remote_user)
+
+      refute filtered["image"]
+    end
   end
 
   defp build_local_message do

test/web/activity_pub/publisher_test.exs

@@ -0,0 +1,266 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ActivityPub.PublisherTest do
+  use Pleroma.DataCase
+
+  import Pleroma.Factory
+  import Tesla.Mock
+  import Mock
+
+  alias Pleroma.Activity
+  alias Pleroma.Instances
+  alias Pleroma.Web.ActivityPub.Publisher
+
+  @as_public "https://www.w3.org/ns/activitystreams#Public"
+
+  setup do
+    mock(fn env -> apply(HttpRequestMock, :request, [env]) end)
+    :ok
+  end
+
+  describe "determine_inbox/2" do
+    test "it returns sharedInbox for messages involving as:Public in to" do
+      user =
+        insert(:user, %{
+          info: %{source_data: %{"endpoints" => %{"sharedInbox" => "http://example.com/inbox"}}}
+        })
+
+      activity = %Activity{
+        data: %{"to" => [@as_public], "cc" => [user.follower_address]}
+      }
+
+      assert Publisher.determine_inbox(activity, user) == "http://example.com/inbox"
+    end
+
+    test "it returns sharedInbox for messages involving as:Public in cc" do
+      user =
+        insert(:user, %{
+          info: %{source_data: %{"endpoints" => %{"sharedInbox" => "http://example.com/inbox"}}}
+        })
+
+      activity = %Activity{
+        data: %{"cc" => [@as_public], "to" => [user.follower_address]}
+      }
+
+      assert Publisher.determine_inbox(activity, user) == "http://example.com/inbox"
+    end
+
+    test "it returns sharedInbox for messages involving multiple recipients in to" do
+      user =
+        insert(:user, %{
+          info: %{source_data: %{"endpoints" => %{"sharedInbox" => "http://example.com/inbox"}}}
+        })
+
+      user_two = insert(:user)
+      user_three = insert(:user)
+
+      activity = %Activity{
+        data: %{"cc" => [], "to" => [user.ap_id, user_two.ap_id, user_three.ap_id]}
+      }
+
+      assert Publisher.determine_inbox(activity, user) == "http://example.com/inbox"
+    end
+
+    test "it returns sharedInbox for messages involving multiple recipients in cc" do
+      user =
+        insert(:user, %{
+          info: %{source_data: %{"endpoints" => %{"sharedInbox" => "http://example.com/inbox"}}}
+        })
+
+      user_two = insert(:user)
+      user_three = insert(:user)
+
+      activity = %Activity{
+        data: %{"to" => [], "cc" => [user.ap_id, user_two.ap_id, user_three.ap_id]}
+      }
+
+      assert Publisher.determine_inbox(activity, user) == "http://example.com/inbox"
+    end
+
+    test "it returns sharedInbox for messages involving multiple recipients in total" do
+      user =
+        insert(:user, %{
+          info: %{
+            source_data: %{
+              "inbox" => "http://example.com/personal-inbox",
+              "endpoints" => %{"sharedInbox" => "http://example.com/inbox"}
+            }
+          }
+        })
+
+      user_two = insert(:user)
+
+      activity = %Activity{
+        data: %{"to" => [user_two.ap_id], "cc" => [user.ap_id]}
+      }
+
+      assert Publisher.determine_inbox(activity, user) == "http://example.com/inbox"
+    end
+
+    test "it returns inbox for messages involving single recipients in total" do
+      user =
+        insert(:user, %{
+          info: %{
+            source_data: %{
+              "inbox" => "http://example.com/personal-inbox",
+              "endpoints" => %{"sharedInbox" => "http://example.com/inbox"}
+            }
+          }
+        })
+
+      activity = %Activity{
+        data: %{"to" => [user.ap_id], "cc" => []}
+      }
+
+      assert Publisher.determine_inbox(activity, user) == "http://example.com/personal-inbox"
+    end
+  end
+
+  describe "publish_one/1" do
+    test_with_mock "calls `Instances.set_reachable` on successful federation if `unreachable_since` is not specified",
+                   Instances,
+                   [:passthrough],
+                   [] do
+      actor = insert(:user)
+      inbox = "http://200.site/users/nick1/inbox"
+
+      assert {:ok, _} = Publisher.publish_one(%{inbox: inbox, json: "{}", actor: actor, id: 1})
+
+      assert called(Instances.set_reachable(inbox))
+    end
+
+    test_with_mock "calls `Instances.set_reachable` on successful federation if `unreachable_since` is set",
+                   Instances,
+                   [:passthrough],
+                   [] do
+      actor = insert(:user)
+      inbox = "http://200.site/users/nick1/inbox"
+
+      assert {:ok, _} =
+               Publisher.publish_one(%{
+                 inbox: inbox,
+                 json: "{}",
+                 actor: actor,
+                 id: 1,
+                 unreachable_since: NaiveDateTime.utc_now()
+               })
+
+      assert called(Instances.set_reachable(inbox))
+    end
+
+    test_with_mock "does NOT call `Instances.set_reachable` on successful federation if `unreachable_since` is nil",
+                   Instances,
+                   [:passthrough],
+                   [] do
+      actor = insert(:user)
+      inbox = "http://200.site/users/nick1/inbox"
+
+      assert {:ok, _} =
+               Publisher.publish_one(%{
+                 inbox: inbox,
+                 json: "{}",
+                 actor: actor,
+                 id: 1,
+                 unreachable_since: nil
+               })
+
+      refute called(Instances.set_reachable(inbox))
+    end
+
+    test_with_mock "calls `Instances.set_unreachable` on target inbox on non-2xx HTTP response code",
+                   Instances,
+                   [:passthrough],
+                   [] do
+      actor = insert(:user)
+      inbox = "http://404.site/users/nick1/inbox"
+
+      assert {:error, _} = Publisher.publish_one(%{inbox: inbox, json: "{}", actor: actor, id: 1})
+
+      assert called(Instances.set_unreachable(inbox))
+    end
+
+    test_with_mock "it calls `Instances.set_unreachable` on target inbox on request error of any kind",
+                   Instances,
+                   [:passthrough],
+                   [] do
+      actor = insert(:user)
+      inbox = "http://connrefused.site/users/nick1/inbox"
+
+      assert {:error, _} = Publisher.publish_one(%{inbox: inbox, json: "{}", actor: actor, id: 1})
+
+      assert called(Instances.set_unreachable(inbox))
+    end
+
+    test_with_mock "does NOT call `Instances.set_unreachable` if target is reachable",
+                   Instances,
+                   [:passthrough],
+                   [] do
+      actor = insert(:user)
+      inbox = "http://200.site/users/nick1/inbox"
+
+      assert {:ok, _} = Publisher.publish_one(%{inbox: inbox, json: "{}", actor: actor, id: 1})
+
+      refute called(Instances.set_unreachable(inbox))
+    end
+
+    test_with_mock "does NOT call `Instances.set_unreachable` if target instance has non-nil `unreachable_since`",
+                   Instances,
+                   [:passthrough],
+                   [] do
+      actor = insert(:user)
+      inbox = "http://connrefused.site/users/nick1/inbox"
+
+      assert {:error, _} =
+               Publisher.publish_one(%{
+                 inbox: inbox,
+                 json: "{}",
+                 actor: actor,
+                 id: 1,
+                 unreachable_since: NaiveDateTime.utc_now()
+               })
+
+      refute called(Instances.set_unreachable(inbox))
+    end
+  end
+
+  describe "publish/2" do
+    test_with_mock "publishes an activity with BCC to all relevant peers.",
+                   Pleroma.Web.Federator.Publisher,
+                   [:passthrough],
+                   [] do
+      follower =
+        insert(:user,
+          local: false,
+          info: %{
+            ap_enabled: true,
+            source_data: %{"inbox" => "https://domain.com/users/nick1/inbox"}
+          }
+        )
+
+      actor = insert(:user, follower_address: follower.ap_id)
+      user = insert(:user)
+
+      {:ok, _follower_one} = Pleroma.User.follow(follower, actor)
+      actor = refresh_record(actor)
+
+      note_activity =
+        insert(:note_activity,
+          recipients: [follower.ap_id],
+          data_attrs: %{"bcc" => [user.ap_id]}
+        )
+
+      res = Publisher.publish(actor, note_activity)
+      assert res == :ok
+
+      assert called(
+               Pleroma.Web.Federator.Publisher.enqueue_one(Publisher, %{
+                 inbox: "https://domain.com/users/nick1/inbox",
+                 actor: actor,
+                 id: note_activity.data["id"]
+               })
+             )
+    end
+  end
+end

test/web/admin_api/admin_api_controller_test.exs

@@ -1010,6 +1010,17 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do
                "uses" => 0
              }
     end
+
+    test "with invalid token" do
+      admin = insert(:user, info: %{is_admin: true})
+
+      conn =
+        build_conn()
+        |> assign(:user, admin)
+        |> post("/api/pleroma/admin/users/revoke_invite", %{"token" => "foo"})
+
+      assert json_response(conn, :not_found) == "Not found"
+    end
   end
 
   describe "GET /api/pleroma/admin/reports/:id" do
@@ -1560,7 +1571,8 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do
                 %{"tuple" => [":method", "Pleroma.Captcha.Kocaptcha"]},
                 %{"tuple" => [":seconds_valid", 60]},
                 %{"tuple" => [":path", ""]},
-                %{"tuple" => [":key1", nil]}
+                %{"tuple" => [":key1", nil]},
+                %{"tuple" => [":partial_chain", "&:hackney_connect.partial_chain/1"]}
               ]
             }
           ]
@@ -1576,7 +1588,8 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do
                      %{"tuple" => [":method", "Pleroma.Captcha.Kocaptcha"]},
                      %{"tuple" => [":seconds_valid", 60]},
                      %{"tuple" => [":path", ""]},
-                     %{"tuple" => [":key1", nil]}
+                     %{"tuple" => [":key1", nil]},
+                     %{"tuple" => [":partial_chain", "&:hackney_connect.partial_chain/1"]}
                    ]
                  }
                ]
@@ -1902,6 +1915,63 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do
              }
     end
   end
+
+  describe "GET /api/pleroma/admin/users/:nickname/statuses" do
+    setup do
+      admin = insert(:user, info: %{is_admin: true})
+      user = insert(:user)
+
+      date1 = (DateTime.to_unix(DateTime.utc_now()) + 2000) |> DateTime.from_unix!()
+      date2 = (DateTime.to_unix(DateTime.utc_now()) + 1000) |> DateTime.from_unix!()
+      date3 = (DateTime.to_unix(DateTime.utc_now()) + 3000) |> DateTime.from_unix!()
+
+      insert(:note_activity, user: user, published: date1)
+      insert(:note_activity, user: user, published: date2)
+      insert(:note_activity, user: user, published: date3)
+
+      conn =
+        build_conn()
+        |> assign(:user, admin)
+
+      {:ok, conn: conn, user: user}
+    end
+
+    test "renders user's statuses", %{conn: conn, user: user} do
+      conn = get(conn, "/api/pleroma/admin/users/#{user.nickname}/statuses")
+
+      assert json_response(conn, 200) |> length() == 3
+    end
+
+    test "renders user's statuses with a limit", %{conn: conn, user: user} do
+      conn = get(conn, "/api/pleroma/admin/users/#{user.nickname}/statuses?page_size=2")
+
+      assert json_response(conn, 200) |> length() == 2
+    end
+
+    test "doesn't return private statuses by default", %{conn: conn, user: user} do
+      {:ok, _private_status} =
+        CommonAPI.post(user, %{"status" => "private", "visibility" => "private"})
+
+      {:ok, _public_status} =
+        CommonAPI.post(user, %{"status" => "public", "visibility" => "public"})
+
+      conn = get(conn, "/api/pleroma/admin/users/#{user.nickname}/statuses")
+
+      assert json_response(conn, 200) |> length() == 4
+    end
+
+    test "returns private statuses with godmode on", %{conn: conn, user: user} do
+      {:ok, _private_status} =
+        CommonAPI.post(user, %{"status" => "private", "visibility" => "private"})
+
+      {:ok, _public_status} =
+        CommonAPI.post(user, %{"status" => "public", "visibility" => "public"})
+
+      conn = get(conn, "/api/pleroma/admin/users/#{user.nickname}/statuses?godmode=true")
+
+      assert json_response(conn, 200) |> length() == 5
+    end
+  end
 end
 
 # Needed for testing

test/web/admin_api/config_test.exs

@@ -238,6 +238,14 @@ defmodule Pleroma.Web.AdminAPI.ConfigTest do
       assert Config.from_binary(binary) == [key: "value"]
     end
 
+    test "keyword with partial_chain key" do
+      binary =
+        Config.transform([%{"tuple" => [":partial_chain", "&:hackney_connect.partial_chain/1"]}])
+
+      assert binary == :erlang.term_to_binary(partial_chain: &:hackney_connect.partial_chain/1)
+      assert Config.from_binary(binary) == [partial_chain: &:hackney_connect.partial_chain/1]
+    end
+
     test "keyword" do
       binary =
         Config.transform([

test/web/federator_test.exs

@@ -22,6 +22,15 @@ defmodule Pleroma.Web.FederatorTest do
     :ok
   end
 
+  describe "Publisher.perform" do
+    test "call `perform` with unknown task" do
+      assert {
+               :error,
+               "Don't know what to do with this"
+             } = Pleroma.Web.Federator.Publisher.perform("test", :ok, :ok)
+    end
+  end
+
   describe "Publish an activity" do
     setup do
       user = insert(:user)

test/web/mastodon_api/account_view_test.exs

@@ -231,6 +231,16 @@ defmodule Pleroma.Web.MastodonAPI.AccountViewTest do
                AccountView.render("relationship.json", %{user: user, target: other_user})
     end
 
+    test "represent a relationship for the user blocking a domain" do
+      user = insert(:user)
+      other_user = insert(:user, ap_id: "https://bad.site/users/other_user")
+
+      {:ok, user} = User.block_domain(user, "bad.site")
+
+      assert %{domain_blocking: true, blocking: false} =
+               AccountView.render("relationship.json", %{user: user, target: other_user})
+    end
+
     test "represent a relationship for the user with a pending follow request" do
       user = insert(:user)
       other_user = insert(:user, %{info: %User.Info{locked: true}})

test/web/mastodon_api/mastodon_api_controller_test.exs

@@ -2815,11 +2815,11 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIControllerTest do
 
       card_data = %{
         "image" => "http://ia.media-imdb.com/images/rock.jpg",
-        "provider_name" => "www.imdb.com",
+        "provider_name" => "example.com",
-        "provider_url" => "http://www.imdb.com",
+        "provider_url" => "https://example.com",
         "title" => "The Rock",
         "type" => "link",
-        "url" => "http://www.imdb.com/title/tt0117500/",
+        "url" => "https://example.com/ogp",
         "description" =>
           "Directed by Michael Bay. With Sean Connery, Nicolas Cage, Ed Harris, John Spencer.",
         "pleroma" => %{
@@ -2827,7 +2827,7 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIControllerTest do
             "image" => "http://ia.media-imdb.com/images/rock.jpg",
             "title" => "The Rock",
             "type" => "video.movie",
-            "url" => "http://www.imdb.com/title/tt0117500/",
+            "url" => "https://example.com/ogp",
             "description" =>
               "Directed by Michael Bay. With Sean Connery, Nicolas Cage, Ed Harris, John Spencer."
           }
@@ -2868,14 +2868,14 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIControllerTest do
                "title" => "Pleroma",
                "description" => "",
                "image" => nil,
-               "provider_name" => "pleroma.social",
+               "provider_name" => "example.com",
-               "provider_url" => "https://pleroma.social",
+               "provider_url" => "https://example.com",
-               "url" => "https://pleroma.social/",
+               "url" => "https://example.com/ogp-missing-data",
                "pleroma" => %{
                  "opengraph" => %{
                    "title" => "Pleroma",
                    "type" => "website",
-                   "url" => "https://pleroma.social/"
+                   "url" => "https://example.com/ogp-missing-data"
                  }
                }
              }
@@ -3768,6 +3768,38 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIControllerTest do
 
       assert Enum.empty?(response)
     end
+
+    test "does not return users who have favorited the status but are blocked", %{
+      conn: %{assigns: %{user: user}} = conn,
+      activity: activity
+    } do
+      other_user = insert(:user)
+      {:ok, user} = User.block(user, other_user)
+
+      {:ok, _, _} = CommonAPI.favorite(activity.id, other_user)
+
+      response =
+        conn
+        |> assign(:user, user)
+        |> get("/api/v1/statuses/#{activity.id}/favourited_by")
+        |> json_response(:ok)
+
+      assert Enum.empty?(response)
+    end
+
+    test "does not fail on an unauthenticated request", %{conn: conn, activity: activity} do
+      other_user = insert(:user)
+      {:ok, _, _} = CommonAPI.favorite(activity.id, other_user)
+
+      response =
+        conn
+        |> assign(:user, nil)
+        |> get("/api/v1/statuses/#{activity.id}/favourited_by")
+        |> json_response(:ok)
+
+      [%{"id" => id}] = response
+      assert id == other_user.id
+    end
   end
 
   describe "GET /api/v1/statuses/:id/reblogged_by" do
@@ -3807,6 +3839,38 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIControllerTest do
 
       assert Enum.empty?(response)
     end
+
+    test "does not return users who have reblogged the status but are blocked", %{
+      conn: %{assigns: %{user: user}} = conn,
+      activity: activity
+    } do
+      other_user = insert(:user)
+      {:ok, user} = User.block(user, other_user)
+
+      {:ok, _, _} = CommonAPI.repeat(activity.id, other_user)
+
+      response =
+        conn
+        |> assign(:user, user)
+        |> get("/api/v1/statuses/#{activity.id}/reblogged_by")
+        |> json_response(:ok)
+
+      assert Enum.empty?(response)
+    end
+
+    test "does not fail on an unauthenticated request", %{conn: conn, activity: activity} do
+      other_user = insert(:user)
+      {:ok, _, _} = CommonAPI.repeat(activity.id, other_user)
+
+      response =
+        conn
+        |> assign(:user, nil)
+        |> get("/api/v1/statuses/#{activity.id}/reblogged_by")
+        |> json_response(:ok)
+
+      [%{"id" => id}] = response
+      assert id == other_user.id
+    end
   end
 
   describe "POST /auth/password, with valid parameters" do

test/web/mastodon_api/status_view_test.exs

@@ -300,6 +300,16 @@ defmodule Pleroma.Web.MastodonAPI.StatusViewTest do
     assert %{id: "2"} = StatusView.render("attachment.json", %{attachment: object})
   end
 
+  test "put the url advertised in the Activity in to the url attribute" do
+    id = "https://wedistribute.org/wp-json/pterotype/v1/object/85810"
+    [activity] = Activity.search(nil, id)
+
+    status = StatusView.render("status.json", %{activity: activity})
+
+    assert status.uri == id
+    assert status.url == "https://wedistribute.org/2019/07/mastodon-drops-ostatus/"
+  end
+
   test "a reblog" do
     user = insert(:user)
     activity = insert(:note_activity)

test/web/oauth/oauth_controller_test.exs

@@ -5,9 +5,7 @@
 defmodule Pleroma.Web.OAuth.OAuthControllerTest do
   use Pleroma.Web.ConnCase
   import Pleroma.Factory
-  import Mock
 
-  alias Pleroma.Registration
   alias Pleroma.Repo
   alias Pleroma.Web.OAuth.Authorization
   alias Pleroma.Web.OAuth.OAuthController
@@ -108,11 +106,10 @@ defmodule Pleroma.Web.OAuth.OAuthControllerTest do
         "state" => ""
       }
 
-      with_mock Pleroma.Web.Auth.Authenticator,
-        get_registration: fn _ -> {:ok, registration} end do
       conn =
-          get(
+        conn
-            conn,
+        |> assign(:ueberauth_auth, %{provider: registration.provider, uid: registration.uid})
+        |> get(
           "/oauth/twitter/callback",
           %{
             "oauth_token" => "G-5a3AAAAAAAwMH9AAABaektfSM",
@@ -125,11 +122,10 @@ defmodule Pleroma.Web.OAuth.OAuthControllerTest do
       assert response = html_response(conn, 302)
       assert redirected_to(conn) =~ ~r/#{redirect_uri}\?code=.+/
     end
-    end
 
     test "with user-unbound registration, GET /oauth/<provider>/callback renders registration_details page",
          %{app: app, conn: conn} do
-      registration = insert(:registration, user: nil)
+      user = insert(:user)
 
       state_params = %{
         "scope" => "read write",
@@ -138,11 +134,14 @@ defmodule Pleroma.Web.OAuth.OAuthControllerTest do
         "state" => "a_state"
       }
 
-      with_mock Pleroma.Web.Auth.Authenticator,
-        get_registration: fn _ -> {:ok, registration} end do
       conn =
-          get(
+        conn
-            conn,
+        |> assign(:ueberauth_auth, %{
+          provider: "twitter",
+          uid: "171799000",
+          info: %{nickname: user.nickname, email: user.email, name: user.name, description: nil}
+        })
+        |> get(
           "/oauth/twitter/callback",
           %{
             "oauth_token" => "G-5a3AAAAAAAwMH9AAABaektfSM",
@@ -155,9 +154,8 @@ defmodule Pleroma.Web.OAuth.OAuthControllerTest do
       assert response = html_response(conn, 200)
       assert response =~ ~r/name="op" type="submit" value="register"/
       assert response =~ ~r/name="op" type="submit" value="connect"/
-        assert response =~ Registration.email(registration)
+      assert response =~ user.email
-        assert response =~ Registration.nickname(registration)
+      assert response =~ user.nickname
-      end
     end
 
     test "on authentication error, GET /oauth/<provider>/callback redirects to `redirect_uri`", %{

test/web/rich_media/parser_test.exs

@@ -59,7 +59,8 @@ defmodule Pleroma.Web.RichMedia.ParserTest do
 
   test "doesn't just add a title" do
     assert Pleroma.Web.RichMedia.Parser.parse("http://example.com/non-ogp") ==
-             {:error, "Found metadata was invalid or incomplete: %{}"}
+             {:error,
+              "Found metadata was invalid or incomplete: %{url: \"http://example.com/non-ogp\"}"}
   end
 
   test "parses ogp" do
@@ -71,7 +72,7 @@ defmodule Pleroma.Web.RichMedia.ParserTest do
                 description:
                   "Directed by Michael Bay. With Sean Connery, Nicolas Cage, Ed Harris, John Spencer.",
                 type: "video.movie",
-                url: "http://www.imdb.com/title/tt0117500/"
+                url: "http://example.com/ogp"
               }}
   end
 
@@ -84,7 +85,7 @@ defmodule Pleroma.Web.RichMedia.ParserTest do
                 description:
                   "Directed by Michael Bay. With Sean Connery, Nicolas Cage, Ed Harris, John Spencer.",
                 type: "video.movie",
-                url: "http://www.imdb.com/title/tt0117500/"
+                url: "http://example.com/ogp-missing-title"
               }}
   end
 
@@ -96,7 +97,8 @@ defmodule Pleroma.Web.RichMedia.ParserTest do
                 site: "@flickr",
                 image: "https://farm6.staticflickr.com/5510/14338202952_93595258ff_z.jpg",
                 title: "Small Island Developing States Photo Submission",
-                description: "View the album on Flickr."
+                description: "View the album on Flickr.",
+                url: "http://example.com/twitter-card"
               }}
   end
 
@@ -120,7 +122,7 @@ defmodule Pleroma.Web.RichMedia.ParserTest do
                 thumbnail_width: 150,
                 title: "Bacon Lollys",
                 type: "photo",
-                url: "https://farm4.staticflickr.com/3040/2362225867_4a87ab8baf_b.jpg",
+                url: "http://example.com/oembed",
                 version: "1.0",
                 web_page: "https://www.flickr.com/photos/bees/2362225867/",
                 web_page_short_url: "https://flic.kr/p/4AK2sc",

test/web/streamer_test.exs

@@ -65,6 +65,63 @@ defmodule Pleroma.Web.StreamerTest do
       Streamer.stream("user:notification", notify)
       Task.await(task)
     end
+
+    test "it doesn't send notify to the 'user:notification' stream when a user is blocked", %{
+      user: user
+    } do
+      blocked = insert(:user)
+      {:ok, user} = User.block(user, blocked)
+
+      task = Task.async(fn -> refute_receive {:text, _}, 4_000 end)
+
+      Streamer.add_socket(
+        "user:notification",
+        %{transport_pid: task.pid, assigns: %{user: user}}
+      )
+
+      {:ok, activity} = CommonAPI.post(user, %{"status" => ":("})
+      {:ok, notif, _} = CommonAPI.favorite(activity.id, blocked)
+
+      Streamer.stream("user:notification", notif)
+      Task.await(task)
+    end
+
+    test "it doesn't send notify to the 'user:notification' stream when a thread is muted", %{
+      user: user
+    } do
+      user2 = insert(:user)
+      task = Task.async(fn -> refute_receive {:text, _}, 4_000 end)
+
+      Streamer.add_socket(
+        "user:notification",
+        %{transport_pid: task.pid, assigns: %{user: user}}
+      )
+
+      {:ok, activity} = CommonAPI.post(user, %{"status" => "super hot take"})
+      {:ok, activity} = CommonAPI.add_mute(user, activity)
+      {:ok, notif, _} = CommonAPI.favorite(activity.id, user2)
+      Streamer.stream("user:notification", notif)
+      Task.await(task)
+    end
+
+    test "it doesn't send notify to the 'user:notification' stream' when a domain is blocked", %{
+      user: user
+    } do
+      user2 = insert(:user, %{ap_id: "https://hecking-lewd-place.com/user/meanie"})
+      task = Task.async(fn -> refute_receive {:text, _}, 4_000 end)
+
+      Streamer.add_socket(
+        "user:notification",
+        %{transport_pid: task.pid, assigns: %{user: user}}
+      )
+
+      {:ok, user} = User.block_domain(user, "hecking-lewd-place.com")
+      {:ok, activity} = CommonAPI.post(user, %{"status" => "super hot take"})
+      {:ok, notif, _} = CommonAPI.favorite(activity.id, user2)
+
+      Streamer.stream("user:notification", notif)
+      Task.await(task)
+    end
   end
 
   test "it sends to public" do

test/web/web_finger/web_finger_controller_test.exs

@@ -19,6 +19,19 @@ defmodule Pleroma.Web.WebFinger.WebFingerControllerTest do
     :ok
   end
 
+  test "GET host-meta" do
+    response =
+      build_conn()
+      |> get("/.well-known/host-meta")
+
+    assert response.status == 200
+
+    assert response.resp_body ==
+             ~s(<?xml version="1.0" encoding="UTF-8"?><XRD xmlns="http://docs.oasis-open.org/ns/xri/xrd-1.0"><Link rel="lrdd" template="#{
+               Pleroma.Web.base_url()
+             }/.well-known/webfinger?resource={uri}" type="application/xrd+xml" /></XRD>)
+  end
+
   test "Webfinger JRD" do
     user = insert(:user)
 
@@ -30,6 +43,16 @@ defmodule Pleroma.Web.WebFinger.WebFingerControllerTest do
     assert json_response(response, 200)["subject"] == "acct:#{user.nickname}@localhost"
   end
 
+  test "it returns 404 when user isn't found (JSON)" do
+    result =
+      build_conn()
+      |> put_req_header("accept", "application/jrd+json")
+      |> get("/.well-known/webfinger?resource=acct:jimm@localhost")
+      |> json_response(404)
+
+    assert result == "Couldn't find user"
+  end
+
   test "Webfinger XML" do
     user = insert(:user)
 
@@ -41,6 +64,26 @@ defmodule Pleroma.Web.WebFinger.WebFingerControllerTest do
     assert response(response, 200)
   end
 
+  test "it returns 404 when user isn't found (XML)" do
+    result =
+      build_conn()
+      |> put_req_header("accept", "application/xrd+xml")
+      |> get("/.well-known/webfinger?resource=acct:jimm@localhost")
+      |> response(404)
+
+    assert result == "Couldn't find user"
+  end
+
+  test "Sends a 404 when invalid format" do
+    user = insert(:user)
+
+    assert_raise Phoenix.NotAcceptableError, fn ->
+      build_conn()
+      |> put_req_header("accept", "text/html")
+      |> get("/.well-known/webfinger?resource=acct:#{user.nickname}@localhost")
+    end
+  end
+
   test "Sends a 400 when resource param is missing" do
     response =
       build_conn()

test/web/web_finger/web_finger_test.exs

@@ -40,6 +40,11 @@ defmodule Pleroma.Web.WebFingerTest do
   end
 
   describe "fingering" do
+    test "returns error when fails parse xml or json" do
+      user = "invalid_content@social.heldscal.la"
+      assert {:error, %Jason.DecodeError{}} = WebFinger.finger(user)
+    end
+
     test "returns the info for an OStatus user" do
       user = "shp@social.heldscal.la"
 
@@ -81,6 +86,20 @@ defmodule Pleroma.Web.WebFingerTest do
       assert data["subscribe_address"] == "https://gnusocial.de/main/ostatussub?profile={uri}"
     end
 
+    test "it work for AP-only user" do
+      user = "kpherox@mstdn.jp"
+
+      {:ok, data} = WebFinger.finger(user)
+
+      assert data["magic_key"] == nil
+      assert data["salmon"] == nil
+
+      assert data["topic"] == "https://mstdn.jp/users/kPherox.atom"
+      assert data["subject"] == "acct:kPherox@mstdn.jp"
+      assert data["ap_id"] == "https://mstdn.jp/users/kPherox"
+      assert data["subscribe_address"] == "https://mstdn.jp/authorize_interaction?acct={uri}"
+    end
+
     test "it works for friendica" do
       user = "lain@squeet.me"