MAP.rst

-Directory Structure
-===================
-
-Warning: this may be a bit stale...
-
-::
-
-    .
-    ├── cmdclient           Basic CLI python Matrix client
-    ├── demo                Scripts for running standalone Matrix demos
-    ├── docs                All doc, including the draft Matrix API spec
-    │   ├── client-server       The client-server Matrix API spec
-    │   ├── model               Domain-specific elements of the Matrix API spec
-    │   ├── server-server       The server-server model of the Matrix API spec
-    │   └── sphinx              The internal API doc of the Synapse homeserver
-    ├── experiments         Early experiments of using Synapse's internal APIs
-    ├── graph               Visualisation of Matrix's distributed message store 
-    ├── synapse             The reference Matrix homeserver implementation
-    │   ├── api                 Common building blocks for the APIs
-    │   │   ├── events              Definition of state representation Events 
-    │   │   └── streams             Definition of streamable Event objects
-    │   ├── app                 The __main__ entry point for the homeserver
-    │   ├── crypto              The PKI client/server used for secure federation
-    │   │   └── resource            PKI helper objects (e.g. keys)
-    │   ├── federation          Server-server state replication logic
-    │   ├── handlers            The main business logic of the homeserver
-    │   ├── http                Wrappers around Twisted's HTTP server & client
-    │   ├── rest                Servlet-style RESTful API
-    │   ├── storage             Persistence subsystem (currently only sqlite3)
-    │   │   └── schema              sqlite persistence schema
-    │   └── util                Synapse-specific utilities
-    ├── tests               Unit tests for the Synapse homeserver
-    └── webclient           Basic AngularJS Matrix web client
-
-

README.md

+# Maunium Synapse
+This is a fork of [Synapse] to remove dumb limits and fix bugs that the
+upstream devs don't want to fix.
+
+The only official distribution is the docker image in the [GitLab container
+registry], but you can also install from source ([upstream instructions]).
+
+The master branch and `:latest` docker tag are upgraded to each upstream
+release candidate very soon after release (usually within 10 minutes†). There
+are also docker tags for each release, e.g. `:1.75.0`. If you don't want RCs,
+use the specific release tags.
+
+†If there are merge conflicts, the update may be delayed for up to a few days
+after the full release.
+
+[Synapse]: https://github.com/matrix-org/synapse
+[GitLab container registry]: https://mau.dev/maunium/synapse/container_registry
+[upstream instructions]: https://github.com/matrix-org/synapse/blob/develop/INSTALL.md#installing-from-source
+
+## List of changes
+* Default power level for room creator is 9001 instead of 100.
+* Room creator can specify a custom room ID with the `room_id` param in the
+  request body. If the room ID is already in use, it will return `M_CONFLICT`.
+* ~~URL previewer user agent includes `Bot` so Twitter previews work properly.~~
+  Upstreamed after over 2 years 🎉
+* ~~Local event creation concurrency is disabled to avoid unnecessary state
+  resolution.~~ Upstreamed after over 3 years 🎉
+* Register admin API can register invalid user IDs.
+* Docker image with jemalloc enabled by default.
+* Config option to allow specific users to send events without unnecessary
+  validation.
+* Config option to allow specific users to receive events that are usually
+  filtered away (e.g. `org.matrix.dummy_event` and `m.room.aliases`).
+* Config option to allow specific users to use timestamp massaging without
+  being appservice users.
+* Removed bad pusher URL validation.
+* webp images are thumbnailed to webp instead of jpeg to avoid losing
+  transparency.
+* Media repo `Cache-Control` header says `immutable` and 1 year for all media
+  that exists, as media IDs in Matrix are immutable.
+* Allowed sending custom data with read receipts.
+
+You can view the full list of changes on the [meow-patchset] branch.
+Additionally, historical patch sets are saved as `meow-patchset-vX` [tags].
+
+[meow-patchset]: https://mau.dev/maunium/synapse/-/compare/patchset-base...meow-patchset
+[tags]: https://mau.dev/maunium/synapse/-/tags?search=meow-patchset&sort=updated_desc
+
+## Configuration reference
+```yaml
+meow:
+   # List of users who aren't subject to unnecessary validation in the C-S API.
+   validation_override:
+   - "@you:example.com"
+   # List of users who will get org.matrix.dummy_event and m.room.aliases events down /sync
+   filter_override:
+   - "@you:example.com"
+   # Whether or not the admin API should be able to register invalid user IDs.
+   admin_api_register_invalid: true
+   # List of users who can use timestamp massaging without being appservices
+   timestamp_override:
+   - "@you:example.com"
+```

book.toml

+# Documentation for possible options in this file is at
+# https://rust-lang.github.io/mdBook/format/config.html
+[book]
+title = "Synapse"
+authors = ["The Matrix.org Foundation C.I.C."]
+language = "en"
+multilingual = false
+
+# The directory that documentation files are stored in
+src = "docs"
+
+[build]
+# Prevent markdown pages from being automatically generated when they're
+# linked to in SUMMARY.md
+create-missing = false
+
+[output.html]
+# The URL visitors will be directed to when they try to edit a page
+edit-url-template = "https://github.com/element-hq/synapse/edit/develop/{path}"
+
+# Remove the numbers that appear before each item in the sidebar, as they can
+# get quite messy as we nest deeper
+no-section-label = true
+
+# The source code URL of the repository
+git-repository-url = "https://github.com/element-hq/synapse"
+
+# The path that the docs are hosted on
+site-url = "/synapse/"
+
+# Additional HTML, JS, CSS that's injected into each page of the book.
+# More information available in docs/website_files/README.md
+additional-css = [
+    "docs/website_files/table-of-contents.css",
+    "docs/website_files/remove-nav-buttons.css",
+    "docs/website_files/indent-section-headers.css",
+    "docs/website_files/version-picker.css",
+]
+additional-js = [
+    "docs/website_files/table-of-contents.js",
+    "docs/website_files/version-picker.js",
+    "docs/website_files/version.js",
+]
+theme = "docs/website_files/theme"
+
+[preprocessor.schema_versions]
+command = "./scripts-dev/schema_versions.py"

build_rust.py

+# A build script for poetry that adds the rust extension.
+
+import itertools
+import os
+from typing import Any, Dict
+
+from packaging.specifiers import SpecifierSet
+from setuptools_rust import Binding, RustExtension
+
+
+def build(setup_kwargs: Dict[str, Any]) -> None:
+    original_project_dir = os.path.dirname(os.path.realpath(__file__))
+    cargo_toml_path = os.path.join(original_project_dir, "rust", "Cargo.toml")
+
+    extension = RustExtension(
+        target="synapse.synapse_rust",
+        path=cargo_toml_path,
+        binding=Binding.PyO3,
+        # This flag is a no-op in the latest versions. Instead, we need to
+        # specify this in the `bdist_wheel` config below.
+        py_limited_api=True,
+        # We force always building in release mode, as we can't tell the
+        # difference between using `poetry` in development vs production.
+        debug=False,
+    )
+    setup_kwargs.setdefault("rust_extensions", []).append(extension)
+    setup_kwargs["zip_safe"] = False
+
+    # We lookup the minimum supported python version by looking at
+    # `python_requires` (e.g. ">=3.9.0,<4.0.0") and finding the first python
+    # version that matches. We then convert that into the `py_limited_api` form,
+    # e.g. cp39 for python 3.9.
+    py_limited_api: str
+    python_bounds = SpecifierSet(setup_kwargs["python_requires"])
+    for minor_version in itertools.count(start=8):
+        if f"3.{minor_version}.0" in python_bounds:
+            py_limited_api = f"cp3{minor_version}"
+            break
+
+    setup_kwargs.setdefault("options", {}).setdefault("bdist_wheel", {})[
+        "py_limited_api"
+    ] = py_limited_api

changelog.d/.gitignore

+!.gitignore

contrib/README.rst

+Community Contributions
+=======================
+
+Everything in this directory are projects submitted by the community that may be useful
+to others. As such, the project maintainers cannot guarantee support, stability
+or backwards compatibility of these projects. 
+
+Files in this directory should *not* be relied on directly, as they may not
+continue to work or exist in future. If you wish to use any of these files then
+they should be copied to avoid them breaking from underneath you.

contrib/datagrip/README.md

+# Schema symlinks
+
+This directory contains symlinks to the latest dump of the postgres full schema. This is useful to have, as it allows IDEs to understand our schema and provide autocomplete, linters, inspections, etc.
+
+In particular, the DataGrip functionality in IntelliJ's products seems to only consider files called `*.sql` when defining a schema from DDL; `*.sql.postgres` will be ignored. To get around this we symlink those files to ones ending in `.sql`. We've chosen to ignore the `.sql.sqlite` schema dumps here, as they're not intended for production use (and are much quicker to test against).
+
+## Example
+![](datagrip-aware-of-schema.png)
+
+## Caveats
+
+- Doesn't include temporary tables created ad-hoc by Synapse.
+- Postgres only. IDEs will likely be confused by SQLite-specific queries.
+- Will not include migrations created after the latest schema dump.
+- Symlinks might confuse checkouts on Windows systems.
+
+## Instructions
+
+### Jetbrains IDEs with DataGrip plugin
+
+- View -> Tool Windows -> Database
+- `+` Icon -> DDL Data Source
+- Pick a name, e.g. `Synapse schema dump`
+- Under sources, click `+`.
+- Add an entry with Path pointing to this directory, and dialect set to PostgreSQL.
+- OK, and OK.
+- IDE should now be aware of the schema.
+- Try control-clicking on a table name in a bit of SQL e.g. in `_get_forgotten_rooms_for_user_txn`.
\ No newline at end of file

contrib/datagrip/common.sql

+../../synapse/storage/schema/common/full_schemas/72/full.sql.postgres
\ No newline at end of file

contrib/datagrip/main.sql

+../../synapse/storage/schema/main/full_schemas/72/full.sql.postgres
\ No newline at end of file

contrib/datagrip/schema_version.sql

+../../synapse/storage/schema/common/schema_version.sql
\ No newline at end of file

contrib/datagrip/state.sql

+../../synapse/storage/schema/state/full_schemas/72/full.sql.postgres
\ No newline at end of file

contrib/docker/conf/log.config

-version: 1
-
-formatters:
-  precise:
-   format: '%(asctime)s - %(name)s - %(lineno)d - %(levelname)s - %(request)s- %(message)s'
-
-filters:
-  context:
-    (): synapse.util.logcontext.LoggingContextFilter
-    request: ""
-
-handlers:
-  file:
-    class: logging.handlers.RotatingFileHandler
-    formatter: precise
-    filename: /data/homeserver.log
-    maxBytes: 104857600
-    backupCount: 10
-    filters: [context]
-  console:
-    class: logging.StreamHandler
-    formatter: precise
-    filters: [context]
-
-loggers:
-    synapse:
-        level: INFO
-
-    synapse.storage.SQL:
-        # beware: increasing this to DEBUG will make synapse log sensitive
-        # information such as access tokens.
-        level: INFO
-
-root:
-    level: INFO
-    handlers: [file, console]

contrib/docker/docker-compose.yml

@@ -6,16 +6,24 @@ version: '3'
 services:
 
   synapse:
-    image: matrixdotorg/synapse:latest
-    # Since snyapse does not retry to connect to the database, restart upon
+    build:
+      context: ../..
+      dockerfile: docker/Dockerfile
+    image: docker.io/matrixdotorg/synapse:latest
+    # Since synapse does not retry to connect to the database, restart upon
     # failure
     restart: unless-stopped
     # See the readme for a full documentation of the environment settings
+    # NOTE: You must edit homeserver.yaml to use postgres, it defaults to sqlite
     environment:
-      - SYNAPSE_SERVER_NAME=my.matrix.host
-      - SYNAPSE_ENABLE_REGISTRATION=yes
+      - SYNAPSE_CONFIG_PATH=/data/homeserver.yaml
     volumes:
+      # You may either store all the files in a local folder
       - ./files:/data
+      # .. or you may split this between different storage points
+      # - ./files:/data
+      # - /path/to/ssd:/data/uploads
+      # - /path/to/large_hdd:/data/media
     depends_on:
       - db
     # In order to expose Synapse, remove one of the following, you might for
@@ -24,15 +32,35 @@ services:
       - 8448:8448/tcp
     # ... or use a reverse proxy, here is an example for traefik:
     labels:
+      # The following lines are valid for Traefik version 1.x:
       - traefik.enable=true
       - traefik.frontend.rule=Host:my.matrix.Host
-      - traefik.port=8448
+      - traefik.port=8008
+      # Alternatively, for Traefik version 2.0:
+      - traefik.enable=true
+      - traefik.http.routers.http-synapse.entryPoints=http
+      - traefik.http.routers.http-synapse.rule=Host(`my.matrix.host`)
+      - traefik.http.middlewares.https_redirect.redirectscheme.scheme=https
+      - traefik.http.middlewares.https_redirect.redirectscheme.permanent=true
+      - traefik.http.routers.http-synapse.middlewares=https_redirect
+      - traefik.http.routers.https-synapse.entryPoints=https
+      - traefik.http.routers.https-synapse.rule=Host(`my.matrix.host`)
+      - traefik.http.routers.https-synapse.service=synapse
+      - traefik.http.routers.https-synapse.tls=true
+      - traefik.http.services.synapse.loadbalancer.server.port=8008
+      - traefik.http.routers.https-synapse.tls.certResolver=le-ssl
 
   db:
-    image: postgres:latest
+    image: docker.io/postgres:15-alpine
     # Change that password, of course!
     environment:
-      - POSTGRES_USER=matrix
+      - POSTGRES_USER=synapse
       - POSTGRES_PASSWORD=changeme
+      # ensure the database gets created correctly
+      # https://element-hq.github.io/synapse/latest/postgres.html#set-up-database
+      - POSTGRES_INITDB_ARGS=--encoding=UTF-8 --lc-collate=C --lc-ctype=C
     volumes:
+      # You may store the database tables in a local folder..
       - ./schemas:/var/lib/postgresql/data
+      # .. or store them on some high performance storage for better results
+      # - /path/to/ssd/storage:/var/lib/postgresql/data