Faster joins: omit partial rooms from eager syncs until the resync completes (#14870)
* Allow `AbstractSet` in `StrCollection` Or else frozensets are excluded. This will be useful in an upcoming commit where I plan to change a function that accepts `List[str]` to accept `StrCollection` instead. * `rooms_to_exclude` -> `rooms_to_exclude_globally` I am about to make use of this exclusion mechanism to exclude rooms for a specific user and a specific sync. This rename helps to clarify the distinction between the global config and the rooms to exclude for a specific sync. * Better function names for internal sync methods * Track a list of excluded rooms on SyncResultBuilder I plan to feed a list of partially stated rooms for this sync to ignore * Exclude partial state rooms during eager sync using the mechanism established in the previous commit * Track un-partial-state stream in sync tokens So that we can work out which rooms have become fully-stated during a given sync period. * Fix mutation of `@cached` return value This was fouling up a complement test added alongside this PR. Excluding a room would mean the set of forgotten rooms in the cache would be extended. This means that room could be erroneously considered forgotten in the future. Introduced in #12310, Synapse 1.57.0. I don't think this had any user-visible side effects (until now). * SyncResultBuilder: track rooms to force as newly joined Similar plan as before. We've omitted rooms from certain sync responses; now we establish the mechanism to reintroduce them into future syncs. * Read new field, to present rooms as newly joined * Force un-partial-stated rooms to be newly-joined for eager incremental syncs only, provided they're still fully stated * Notify user stream listeners to wake up long polling syncs * Changelog * Typo fix Co-authored-by:Sean Quah <8349537+squahtx@users.noreply.github.com> * Unnecessary list cast Co-authored-by:
Sean Quah <8349537+squahtx@users.noreply.github.com> * Rephrase comment Co-authored-by:
Sean Quah <8349537+squahtx@users.noreply.github.com> * Another comment Co-authored-by:
Sean Quah <8349537+squahtx@users.noreply.github.com> * Fixup merge(?) * Poke notifier when receiving un-partial-stated msg over replication * Fixup merge whoops Thanks MV :) Co-authored-by:
Mathieu Velen <mathieuv@matrix.org> Co-authored-by:
Mathieu Velten <mathieuv@matrix.org> Co-authored-by:
Sean Quah <8349537+squahtx@users.noreply.github.com>
Showing
- changelog.d/14870.feature 1 addition, 0 deletionschangelog.d/14870.feature
- synapse/handlers/federation.py 5 additions, 10 deletionssynapse/handlers/federation.py
- synapse/handlers/sync.py 55 additions, 10 deletionssynapse/handlers/sync.py
- synapse/notifier.py 26 additions, 0 deletionssynapse/notifier.py
- synapse/replication/tcp/client.py 1 addition, 0 deletionssynapse/replication/tcp/client.py
- synapse/storage/databases/main/relations.py 1 addition, 0 deletionssynapse/storage/databases/main/relations.py
- synapse/storage/databases/main/room.py 41 additions, 6 deletionssynapse/storage/databases/main/room.py
- synapse/storage/databases/main/roommember.py 14 additions, 5 deletionssynapse/storage/databases/main/roommember.py
- synapse/streams/events.py 6 additions, 0 deletionssynapse/streams/events.py
- synapse/types/__init__.py 10 additions, 5 deletionssynapse/types/__init__.py
- tests/rest/admin/test_room.py 2 additions, 2 deletionstests/rest/admin/test_room.py
- tests/rest/client/test_rooms.py 5 additions, 5 deletionstests/rest/client/test_rooms.py
- tests/rest/client/test_sync.py 3 additions, 1 deletiontests/rest/client/test_sync.py
Loading
Please register or sign in to comment