Move support for application service query parameter authorization behind a configuration option (#16017)

Adds three new configuration variables:

* destination_min_retry_interval is identical to before (10mn).
* destination_retry_multiplier is now 2 instead of 5, the maximum value will
  be reached slower.
* destination_max_retry_interval is one day instead of (essentially) infinity.

Capping this will cause destinations to continue to be retried sometimes instead
of being lost forever. The previous value was 2 ^ 62 milliseconds.

Allow configuring the set of workers to proxy outbound federation traffic through (`outbound_federation_restricted_to`).

This is useful when you have a worker setup with `federation_sender` instances responsible for sending outbound federation requests and want to make sure *all* outbound federation traffic goes through those instances. Before this change, the generic workers would still contact federation themselves for things like profile lookups, backfill, etc. This PR allows you to set more strict access controls/firewall for all workers and only allow the `federation_sender`'s to contact the outside world.

https://github.com/matrix-org/synapse/pull/15708 didn't quite make the cut for `1.88.0` this morning.

Unix socket support for `federation` and `client` Listeners has existed now for a little while(since [1.81.0](https://github.com/matrix-org/synapse/pull/15353)

), but there was one last hold out before it could be complete: HTTP Replication communication. This should finish it up. The Listeners would have always worked, but would have had no way to be talked to/at.

---------

Co-authored-by: Eric Eastwood <madlittlemods@gmail.com>
Co-authored-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
Co-authored-by: Eric Eastwood <erice@element.io>

Revert "Federation outbound proxy (#15773)"

This reverts commit b07b14b4.

Co-authored-by: reivilibre <oliverw@matrix.org>

Allow configuring the set of workers to proxy outbound federation traffic through (`outbound_federation_restricted_to`).

This is useful when you have a worker setup with `federation_sender` instances responsible for sending outbound federation requests and want to make sure *all* outbound federation traffic goes through those instances. Before this change, the generic workers would still contact federation themselves for things like profile lookups, backfill, etc. This PR allows you to set more strict access controls/firewall for all workers and only allow the `federation_sender`'s to contact the outside world.

The original code is from @erikjohnston's branches which I've gotten in-shape to merge.

Allow for the configuration of max request retries and min/max retry delays in the matrix federation client (#15783)

Revert "Allow for the configuration of max request retries and min/max retry delays in the matrix federation client (#12504)"

This reverts commit d84e6614.

Allow for the configuration of max request retries and min/max retry delays in the matrix federation client (#12504)


Co-authored-by: Mathieu Velten <mathieuv@matrix.org>
Co-authored-by: Erik Johnston <erik@matrix.org>

Spawning from https://github.com/matrix-org/synapse/pull/15731

Implement stable support for MSC3882 to allow an existing device/session to generate a login token for use on a new device/session (#15388)

Implements stable support for MSC3882; this involves updating Synapse's support to
match the MSC / the spec says.

Continue to support the unstable version to allow clients to transition.

Adds a new configuration setting to connect to Redis via a Unix
socket instead of over TCP. Disabled by default.

R30v2 has been out since 2021-07-19 (https://github.com/matrix-org/synapse/pull/10332)
and we started collecting stats on 2021-08-16. Since it's been over a year now
(almost 2 years), this is enough grace period for us to now rip it out.

* Add SSL options to redis config

* fix lint issues

* Add documentation and changelog file

* add missing . at the end of the changelog

* Move client context factory to new file

* Rename ssl to tls and fix typo

* fix lint issues

* Added when redis attributes were added

* Add master to the instance_map as part of Complement, have ReplicationEndpoint look at instance_map for master.

* Fix typo in drive by.

* Remove unnecessary worker_replication_* bits from unit tests and add master to instance_map(hopefully in the right place)

* Several updates:

1. Switch from master to main for naming the main process in the instance_map. Add useful constants for easier adjustment of names in the future.
2. Add backwards compatibility for worker_replication_* to allow time to transition to new style. Make sure to prioritize declaring main directly on the instance_map.
3. Clean up old comments/commented out code.
4. Adjust unit tests to match with new code.
5. Adjust Complement setup infrastructure to only add main to the instance_map if workers are used and remove now unused options from the worker.yaml template.

* Initial Docs upload

* Changelog

* Missed some commented out code that can go now

* Remove TODO comment that no longer holds true.

* Fix links in docs

* More docs

* Remove debug logging

* Apply suggestions from code review

Co-authored-by: reivilibre <olivier@librepush.net>

* Apply suggestions from code review

Co-authored-by: reivilibre <olivier@librepush.net>

* Update version to latest, include completeish before/after examples in upgrade notes.

* Fix up and docs too

---------

Co-authored-by: reivilibre <olivier@librepush.net>

This stops media (and thumbnails) from being accessed from the
listed domains. It does not delete any already locally cached media,
but will prevent accessing it.

Note that admin APIs are unaffected by this change.

This is largely based off the stats and user directory updater code.

Signed-off-by: Sean Quah <seanq@matrix.org>

This is to discourage timing based profiling on the push gateways.

Add a note to the config documentation that the 'delete_stale_devices_after' job always runs on the main process (#15452)

Signed-off-by: Warren Bailey <warren@warrenbailey.net>

* doc: fix account login requests ratelimit defaults typo

Signed-off-by: td <nirvejayesh@gmail.com>

* chore: changelog.d file

---------

Signed-off-by: td <nirvejayesh@gmail.com>

* Correct documentation about registration_shared_secret_path

* Create 15168.doc

* Update changelog.d/15168.doc

---------

Co-authored-by: David Robertson <david.m.robertson1@gmail.com>

* Update database_maintenance_tools.md

Included a blog post by Jackson Chen, which DID work when I followed it to perform Matrix Synapse Maintenance, versus the 2020 blog post by Victor Berger, which DID NOT work when performining maintenance.

* Update database_maintenance_tools.md

* Rephrasing

* Add account data to to export command

* newsfile

* remove not needed function

* update newsfile

* adopt #14973

* fix a typo in registration_shared_secret_path docs
Signed-off-by: Filip Rutar <filip.rutar@gmail.com>

* changelog

Note that this is only used for key-value store (cached values)
and not for the pub/sub replication used by Synapse.

* The user's profile information.
* The user's devices.
* The user's connections / IP address information.

You do not have to restart synapse to reload the log config.

* Change Documentation to have v10 as default room version

* Change Default Room version to 10

* Add changelog entry for default room version swap

* Add changelog entry for v10 default room version in docs

* Clarify doc changelog entry

Co-authored-by: David Robertson <david.m.robertson1@gmail.com>

* Improve Documentation changes.

Co-authored-by: David Robertson <david.m.robertson1@gmail.com>

* Update Changelog entry to have correct format

Co-authored-by: David Robertson <david.m.robertson1@gmail.com>

* Update Spec Version to 1.5

* Only need 1 changelog.

* Fix test.

* Update "Changed in" line

Co-authored-by: David Robertson <david.m.robertson1@gmail.com>
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
Co-authored-by: Patrick Cloke <patrickc@matrix.org>