Adds an option to complement.sh to run Synapse in worker
mode (instead of the default monolith mode).

Fixes #9970

Signed-off-by: Sergio Miguéns Iglesias <lonyelon@lony.xyz>

Signed-off-by: Jerin J Titus <72017981+jerinjtitus@users.noreply.github.com>

fixes #9979

`keylen` seems to be a thing that is frequently incorrectly set, and we don't really need it.

The only time it was used was to figure out if we had removed a subtree in `del_multi`, which we can do better by changing `TreeCache.pop` to return a different type (`TreeCacheNode`).

Commits should be independently reviewable.

The previous spaces_enabled flag now defaults to true and
is exposed in the sample config.

It looks like a typo copy/paste from `get_state_for_event` above.

* Fix /upload 500'ing when presented a very large image

Catch DecompressionBombError and re-raise as ThumbnailErrors

* Set PIL's MAX_IMAGE_PIXELS to match homeserver.yaml

to get it to bomb out quicker, to load less into memory
in the case of super large images

* Add changelog entry for 10029

https://github.com/matrix-org/synapse/issues/9962 uncovered that we accidentally removed all but one of the presence updates that we store in the database when persisting multiple updates. This could cause users' presence state to be stale.

The bug was fixed in #10014, and this PR just adds a test that failed on the old code, and was used to initially verify the bug.

The test attempts to insert some presence into the database in a batch using `PresenceStore.update_presence`, and then simply pulls it out again.

Fixes: https://github.com/matrix-org/synapse/issues/9962



This is a fix for above problem.

I fixed it by swaping the order of insertion of new records and deletion of old ones. This ensures that we don't delete fresh database records as we do deletes before inserts.

Signed-off-by: Marek Matys <themarcq@gmail.com>

Also add support for giving a callback to generate the JSON object to
verify. This should reduce memory usage, as we no longer have the event
in memory in dict form (which has a large memory footprint) for extend
periods of time.

Instead of parsing the full response to `/send_join` into Python objects (which can be huge for large rooms) and *then* parsing that into events, we instead use ijson to stream parse the response directly into `EventBase` objects.

This finishes up the experimental implementation of MSC3083 by showing
the restricted rooms in the spaces summary (from MSC2946).

Signed-off-by: Javier Junquera Sánchez <javier@junquera.io>

Signed-off-by: Savyasachee Jha <savya.jha@hawkradius.com>

To be more consistent with similar code. The check now automatically
raises an AuthError instead of passing back a boolean. It also absorbs
some shared logic between callers.

Use a database table to hold the users that should have full presence sent to them, instead of something in-memory (#9823)

\x7F was meant to be \0x7E (~) this was originally incorrect
in MSC1772.

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

Turns out these scripts weren't getting linted.

- use a tuple rather than a list for the iterable that is passed into the
  wrapped function, for performance

- test that we can pass an iterable and that keys are correctly deduped.

It's not obvious that instances of SQLBaseStore each need their own
instances of random.SystemRandom(); let's just use random directly.

Introduced by 52839886

Signed-off-by: Dan Callahan <danc@element.io>

Our strtobool already returns a bool, so no need to re-cast here

Signed-off-by: Dan Callahan <danc@element.io>

We can get away with just catching UnicodeError here.

    ⋮
    +-- ValueError
    |    +-- UnicodeError
    |         +-- UnicodeDecodeError
    |         +-- UnicodeEncodeError
    |         +-- UnicodeTranslateError
    ⋮

https://docs.python.org/3/library/exceptions.html#exception-hierarchy



Signed-off-by: Dan Callahan <danc@element.io>

Functionally identical, but more obviously cryptographically secure.
...Explicit is better than implicit?

Avoids needing to know that SystemRandom() implies a CSPRNG, and
complies with the big scary red box on the documentation for random:

> Warning:
>   The pseudo-random generators of this module should not be used for
>   security purposes. For security or cryptographic uses, see the
>   secrets module.

https://docs.python.org/3/library/random.html



Signed-off-by: Dan Callahan <danc@element.io>

Add a config option which allows enabling opentracing by user id, eg for
debugging requests made by a test user.

* Correctly ratelimit invites when creating a room

Also allow ratelimiting for more than one action at a time.