* Add account data to to export command

* newsfile

* remove not needed function

* update newsfile

* adopt #14973

* Add Start Synapse with Poetry

* Create 14892.doc

* Apply suggestions from code review

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>

* Update docs/workers.md

---------

Co-authored-by: David Robertson <david.m.robertson1@gmail.com>
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>

* fix a typo in registration_shared_secret_path docs
Signed-off-by: Filip Rutar <filip.rutar@gmail.com>

* changelog

Note that this is only used for key-value store (cached values)
and not for the pub/sub replication used by Synapse.

Respond with a 405 error if a request is received on a known endpoint,
but to an unknown method, per MSC3743.

* Document how to run Synapse

* Changelog for 15022

* Update docs/development/contributing_guide.md

This PR just clarifies in the SRV DNS delegation document that there are
still cases a user may have to serve files from `.well-known` endpoints,
and this may not be a valid case for using SRV delegation. This has
caused some confusion in a few cases.

Signed-off-by: William Kray <github@williamkray.com>

Signed-off-by: Jason Little <realtyem@gmail.com>

* The user's profile information.
* The user's devices.
* The user's connections / IP address information.

Co-authored-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
Co-authored-by: Shay <hillerys@element.io>

* Request partial joins by default

This is a little sloppy, but we are trying to gain confidence in faster
joins in the upcoming RC.

Admins can still opt out by adding the following to their Synapse
config:

```yaml
experimental:
    faster_joins: false
```

We may revert this change before the release proper, depending on how
testing in the wild goes.

* Changelog

* Try to fix the backfill test failures

* Upgrade notes

* Postgres compat?

You do not have to restart synapse to reload the log config.

* Change Documentation to have v10 as default room version

* Change Default Room version to 10

* Add changelog entry for default room version swap

* Add changelog entry for v10 default room version in docs

* Clarify doc changelog entry

Co-authored-by: David Robertson <david.m.robertson1@gmail.com>

* Improve Documentation changes.

Co-authored-by: David Robertson <david.m.robertson1@gmail.com>

* Update Changelog entry to have correct format

Co-authored-by: David Robertson <david.m.robertson1@gmail.com>

* Update Spec Version to 1.5

* Only need 1 changelog.

* Fix test.

* Update "Changed in" line

Co-authored-by: David Robertson <david.m.robertson1@gmail.com>
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
Co-authored-by: Patrick Cloke <patrickc@matrix.org>

* Upgrade to new lockfile format

Now requires poetry >= 1.2.2 to read and poetry >= 1.3.0 to write.

Cheat sheet:

```
poetry --version
poetry show > scratch/before
pipx upgrade poetry
poetry --version
poetry show > scratch/after
diff scratch{before,after} && echo "no change!"
```

* Use Poetry 1.3.2 when reading or writing lockfile

* Remove unneeded(?) poetry dep for cibuildwheel

* Update docs

* Remove redundant call to setup-python

* Remove outdated comments related to Poetry 1.x

* Remove outdated docs line

was fixed in #13082

* Minor improvements to poetry cheat sheet

* Invoke setup-python-poetry with explicit version

Not sure about this. It's hardcoding versions everywhere.

* Changelog

* Check the lockfile is version 2.0

Might one day incorporate other checks like #14742

* Typo fixes, thanks Sean

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>

* Fix missing field in AS documentation

The [AS Configuration Snippet](https://matrix-org.github.io/synapse/latest/application_services.html) is missing `id` field, without it Synapse will fail to load:

```
synapse-synapse-main-0 synapse 2023-01-13 23:05:25,450 - synapse.storage.databases - 84 - INFO - main - [database config 'master']: Starting 'main' database
synapse-synapse-main-0 synapse 2023-01-13 23:05:25,452 - synapse.config.appservice - 79 - ERROR - main - Failed to load appservice from '/as/synapse-hookshot-as/registration.yaml'
synapse-synapse-main-0 synapse 2023-01-13 23:05:25,452 - synapse.config.appservice - 80 - ERROR - main - "Required string field: 'id' (/as/synapse-hookshot-as/registration.yaml)"
synapse-synapse-main-0 synapse Traceback (most recent call last):
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/config/appservice.py", line 57, in load_appservices
synapse-synapse-main-0 synapse     appservice = _load_appservice(hostname, yaml.safe_load(f), config_file)
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/config/appservice.py", line 91, in _load_appservice
synapse-synapse-main-0 synapse     raise KeyError(
synapse-synapse-main-0 synapse KeyError: "Required string field: 'id' (/as/synapse-hookshot-as/registration.yaml)"
synapse-synapse-main-0 synapse 2023-01-13 23:05:25,452 - synapse.app._base - 207 - ERROR - main - Exception during startup
synapse-synapse-main-0 synapse Traceback (most recent call last):
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/app/homeserver.py", line 340, in setup
synapse-synapse-main-0 synapse     hs.setup()
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/server.py", line 310, in setup
synapse-synapse-main-0 synapse     self.datastores = Databases(self.DATASTORE_CLASS, self)
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/__init__.py", line 93, in __init__
synapse-synapse-main-0 synapse     main = main_store_class(database, db_conn, hs)
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/__init__.py", line 139, in __init__
synapse-synapse-main-0 synapse     super().__init__(database, db_conn, hs)
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/events_bg_updates.py", line 98, in __init__
synapse-synapse-main-0 synapse     super().__init__(database, db_conn, hs)
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/devices.py", line 1584, in __init__
synapse-synapse-main-0 synapse     super().__init__(database, db_conn, hs)
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/devices.py", line 89, in __init__
synapse-synapse-main-0 synapse     super().__init__(database, db_conn, hs)
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/roommember.py", line 1494, in __init__
synapse-synapse-main-0 synapse     super().__init__(database, db_conn, hs)
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/room.py", line 1827, in __init__
synapse-synapse-main-0 synapse     super().__init__(database, db_conn, hs)
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/room.py", line 1365, in __init__
synapse-synapse-main-0 synapse     super().__init__(database, db_conn, hs)
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/room.py", line 119, in __init__
synapse-synapse-main-0 synapse     super().__init__(database, db_conn, hs)
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/registration.py", line 2158, in __init__
synapse-synapse-main-0 synapse     super().__init__(database, db_conn, hs)
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/presence.py", line 67, in __init__
synapse-synapse-main-0 synapse     super().__init__(database, db_conn, hs)
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/presence.py", line 48, in __init__
synapse-synapse-main-0 synapse     super().__init__(database, db_conn, hs)
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/transactions.py", line 73, in __init__
synapse-synapse-main-0 synapse     super().__init__(database, db_conn, hs)
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/state.py", line 666, in __init__
synapse-synapse-main-0 synapse     super().__init__(database, db_conn, hs)
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/state.py", line 82, in __init__
synapse-synapse-main-0 synapse     super().__init__(database, db_conn, hs)
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/state.py", line 470, in __init__
synapse-synapse-main-0 synapse     super().__init__(database, db_conn, hs)
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/event_federation.py", line 2007, in __init__
synapse-synapse-main-0 synapse     super().__init__(database, db_conn, hs)
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/media_repository.py", line 148, in __init__
synapse-synapse-main-0 synapse     super().__init__(database, db_conn, hs)
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/media_repository.py", line 68, in __init__
synapse-synapse-main-0 synapse     super().__init__(database, db_conn, hs)
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/push_rule.py", line 330, in __init__
synapse-synapse-main-0 synapse     super().__init__(database, db_conn, hs)
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/event_push_actions.py", line 1938, in __init__
synapse-synapse-main-0 synapse     super().__init__(database, db_conn, hs)
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/metrics.py", line 68, in __init__
synapse-synapse-main-0 synapse     super().__init__(database, db_conn, hs)
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/event_push_actions.py", line 249, in __init__
synapse-synapse-main-0 synapse     super().__init__(database, db_conn, hs)
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/end_to_end_keys.py", line 1181, in __init__
synapse-synapse-main-0 synapse     super().__init__(database, db_conn, hs)
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/search.py", line 426, in __init__
synapse-synapse-main-0 synapse     super().__init__(database, db_conn, hs)
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/search.py", line 137, in __init__
synapse-synapse-main-0 synapse     super().__init__(database, db_conn, hs)
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/account_data.py", line 64, in __init__
synapse-synapse-main-0 synapse     super().__init__(database, db_conn, hs)
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/push_rule.py", line 114, in __init__
synapse-synapse-main-0 synapse     super().__init__(database, db_conn, hs)
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/appservice.py", line 76, in __init__
synapse-synapse-main-0 synapse     self.services_cache = load_appservices(
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/config/appservice.py", line 57, in load_appservices
synapse-synapse-main-0 synapse     appservice = _load_appservice(hostname, yaml.safe_load(f), config_file)
synapse-synapse-main-0 synapse   File "/usr/local/lib/python3.9/site-packages/synapse/config/appservice.py", line 91, in _load_appservice
synapse-synapse-main-0 synapse     raise KeyError(
synapse-synapse-main-0 synapse KeyError: "Required string field: 'id' (/as/synapse-hookshot-as/registration.yaml)"
synapse-synapse-main-0 synapse ******************************************************************************
synapse-synapse-main-0 synapse  Error during initialisation:
synapse-synapse-main-0 synapse     "Required string field: 'id' (/as/synapse-hookshot-as/registration.yaml)"
synapse-synapse-main-0 synapse  There may be more information in the logs.
synapse-synapse-main-0 synapse ******************************************************************************
```

* Changelog

Fixes #13852

Closes: #13643

Fixes: #8780

* Add `tag` to `listeners` documentation

* newsfile

* Add missing worker settings to shared configuration

* newsfile

* update docs after review

* more update for doc

* This -> These

Co-authored-by: David Robertson <david.m.robertson1@gmail.com>

* Update link to towncrier in contribution guide

* newsfile

* Fix upgrade notes for installing ICU

As noticed in https://github.com/matrix-org/synapse/pull/14712/files#r1058433297

* Changelog

* Fix stale external links

* Fix some internal links

* Fix URLs without trailing / where needed

* Fix more links

* Newsfile

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>

* Reapply docs/openid.md fix after conflict

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>

This also alphabetizes the documentation for the various OpenID providers.

PKCE can protect against certain attacks and is enabled by default. Support
can be controlled manually by setting the pkce_method of each oidc_providers
entry to 'auto' (default), 'always', or 'never'.

This is required by Twitter OAuth 2.0 support.

OpenID specifies the format of the user info endpoint and some
OAuth 2.0 IdPs do not follow it, e.g. NextCloud and Twitter.

This adds subject_template and picture_template options to the
default mapping provider for more flexibility in matching those user
info responses.

* Broken link "request_id_header"

The link above leads to an ERROR 404

* Update docs/reverse_proxy.md

Co-authored-by: reivilibre <olivier@librepush.net>

* Move `email` to server in config file documentation

* changelog

Fixes #14704.

Signed-off-by: Mathieu Velten <mathieuv@matrix.org>

Fix `target_memory_usage` being used in the description for the actual `cache_autotune` sub-option `target_cache_memory_usage`.

Signed-off-by: Jeremy Kescher <jeremy@kescher.at>

Signed-off-by: Jeremy Kescher <jeremy@kescher.at>

* Declare support for Python 3.11

* Changelog

* Declare new config

* Parse new config

* Read new config

* Don't use trial/our TestCase where it's not needed

Before:

```
$ time trial tests/events/test_utils.py > /dev/null

real	0m2.277s
user	0m2.186s
sys	0m0.083s
```

After:
```
$ time trial tests/events/test_utils.py > /dev/null

real	0m0.566s
user	0m0.508s
sys	0m0.056s
```

* Helper to upsert to event fields

without exceeding size limits.

* Use helper when adding invite/knock state

Now that we allow admins to include events in prejoin room state with
arbitrary state keys, be a good Matrix citizen and ensure they don't
accidentally create an oversized event.

* Changelog

* Move StateFilter tests

should have done this in #14668

* Add extra methods to StateFilter

* Use StateFilter

* Ensure test file enforces typed defs; alphabetise

* Workaround surprising get_current_state_ids

* Whoops, fix mypy