This takes about 10 seconds in the best case; often more.

Fix unread counts making sync fail if the value of the `unread_count`
column in `event_push_summary` is `None`.

I'm hoping this will provide some pointers for debugging
https://github.com/matrix-org/synapse/issues/7968.

All the callers want this info in the same place, so let's reduce the
duplication by doing it here.

* Fixup `ALTER TABLE` database queries

Make the new columns nullable, because doing otherwise can wedge a
server with a big database, as setting a default value rewrites the
table.

* Switch back to using the notifications count in the push badge

Clients are likely to be confused if we send a push but the badge count
is the unread messages one, and not the notifications one.

* Changelog

Regressed in #8037.

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

* Revert "Add experimental support for sharding event persister. (#8170)"

This reverts commit 82c1ee1c.

* Changelog

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

I think this is simpler (and moves stuff out of the db threads)

This requires adding a mypy plugin to fiddle with the type signatures a bit.

We have three things which all call `_get_e2e_device_keys_and_signatures_txn`
with their own `runInteraction`. Factor out the common code.

We can use the existing `_get_e2e_device_keys_and_signatures_txn` instead of
creating our own txn function

This is *not* ready for production yet. Caveats:

1. We should write some tests...
2. The stream token that we use for events can get stalled at the minimum position of all writers. This means that new events may not be processed and e.g. sent down sync streams if a writer isn't writing or is slow.

* Add shared_rooms api

* Add changelog

* Add .

* Wrap response in {"rooms": }

* linting

* Add unstable_features key

* Remove options from isort that aren't part of 5.x

`-y` and `-rc` are now default behaviour and no longer exist.

`dont-skip` is no longer required

https://timothycrosley.github.io/isort/CHANGELOG/#500-penny-july-4-2020



* Update imports to make isort happy

* Add changelog

* Update tox.ini file with correct invocation

* fix linting again for isort

* Vendor prefix unstable API

* Fix to match spec

* import Codes

* import Codes

* Use FORBIDDEN

* Update changelog.d/7785.feature

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

* Implement get_shared_rooms_for_users

* a comma

* trailing whitespace

* Handle the easy feedback

* Switch to using runInteraction

* Add tests

* Feedback

* Seperate unstable endpoint from v2

* Add upgrade node

* a line

* Fix style by adding a blank line at EOF.

* Update synapse/storage/databases/main/user_directory.py

Co-authored-by: Tulir Asokan <tulir@maunium.net>

* Update synapse/storage/databases/main/user_directory.py

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

* Update UPGRADE.rst

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

* Fix UPGRADE/CHANGELOG unstable paths

unstable unstable unstable

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
Co-authored-by: Tulir Asokan <tulir@maunium.net>

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
Co-authored-by: Tulir Asokan <tulir@maunium.net>

this makes it a bit clearer what's going on.

Fixes https://github.com/matrix-org/synapse/issues/8185