Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
synapse
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Container Registry
Model registry
Monitor
Service Desk
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
Maunium
synapse
Commits
079bc3c8
Unverified
Commit
079bc3c8
authored
4 years ago
by
Erik Johnston
Committed by
GitHub
4 years ago
Browse files
Options
Downloads
Patches
Plain Diff
Fixup worker doc (again) (#8000)
parent
a7bdf98d
No related branches found
Branches containing commit
No related tags found
Tags containing commit
No related merge requests found
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
changelog.d/8000.doc
+1
-0
1 addition, 0 deletions
changelog.d/8000.doc
docs/systemd-with-workers/workers/federation_reader.yaml
+1
-1
1 addition, 1 deletion
docs/systemd-with-workers/workers/federation_reader.yaml
docs/workers.md
+41
-13
41 additions, 13 deletions
docs/workers.md
with
43 additions
and
14 deletions
changelog.d/8000.doc
0 → 100644
+
1
−
0
View file @
079bc3c8
Improve workers docs.
This diff is collapsed.
Click to expand it.
docs/systemd-with-workers/workers/federation_reader.yaml
+
1
−
1
View file @
079bc3c8
worker_app
:
synapse.app.federation_reader
worker_app
:
synapse.app.federation_reader
worker_name
:
federation_reader1
worker_replication_host
:
127.0.0.1
worker_replication_host
:
127.0.0.1
worker_replication_port
:
9092
worker_replication_http_port
:
9093
worker_replication_http_port
:
9093
worker_listeners
:
worker_listeners
:
...
...
This diff is collapsed.
Click to expand it.
docs/workers.md
+
41
−
13
View file @
079bc3c8
...
@@ -23,7 +23,7 @@ The processes communicate with each other via a Synapse-specific protocol called
...
@@ -23,7 +23,7 @@ The processes communicate with each other via a Synapse-specific protocol called
feeds streams of newly written data between processes so they can be kept in
feeds streams of newly written data between processes so they can be kept in
sync with the database state.
sync with the database state.
When configured to do so, Synapse uses a
When configured to do so, Synapse uses a
[
Redis pub/sub channel
](
https://redis.io/topics/pubsub
)
to send the replication
[
Redis pub/sub channel
](
https://redis.io/topics/pubsub
)
to send the replication
stream between all configured Synapse processes. Additionally, processes may
stream between all configured Synapse processes. Additionally, processes may
make HTTP requests to each other, primarily for operations which need to wait
make HTTP requests to each other, primarily for operations which need to wait
...
@@ -66,23 +66,31 @@ https://hub.docker.com/r/matrixdotorg/synapse/.
...
@@ -66,23 +66,31 @@ https://hub.docker.com/r/matrixdotorg/synapse/.
To make effective use of the workers, you will need to configure an HTTP
To make effective use of the workers, you will need to configure an HTTP
reverse-proxy such as nginx or haproxy, which will direct incoming requests to
reverse-proxy such as nginx or haproxy, which will direct incoming requests to
the correct worker, or to the main synapse instance. See
the correct worker, or to the main synapse instance. See
[
reverse_proxy.md
](
reverse_proxy.md
)
for information on setting up a reverse
[
reverse_proxy.md
](
reverse_proxy.md
)
for information on setting up a reverse
proxy.
proxy.
To enable workers you should create a configuration file for each worker
When using workers, each worker process has its own configuration file which
process. Each worker configuration file inherits the configuration of the shared
contains settings specific to that worker, such as the HTTP listener that it
homeserver configuration file. You can then override configuration specific to
provides (if any), logging configuration, etc.
that worker, e.g. the HTTP listener that it provides (if any); logging
configuration; etc. You should minimise the number of overrides though to
maintain a usable config.
Normally, the worker processes are configured to read from a shared
configuration file as well as the worker-specific configuration files. This
makes it easier to keep common configuration settings synchronised across all
the processes.
### Shared Configuration
The main process is somewhat special in this respect: it does not normally
need its own configuration file and can take all of its configuration from the
shared configuration file.
### Shared configuration
Normally, only a couple of changes are needed to make an existing configuration
file suitable for use with workers. First, you need to enable an "HTTP replication
listener" for the main process; and secondly, you need to enable redis-based
replication. For example:
Next you need to add both a HTTP replication listener, used for HTTP requests
between processes, and redis config to the shared Synapse configuration file
(
`homeserver.yaml`
). For example:
```
yaml
```
yaml
# extend the existing `listeners` section. This defines the ports that the
# extend the existing `listeners` section. This defines the ports that the
...
@@ -105,7 +113,7 @@ Under **no circumstances** should the replication listener be exposed to the
...
@@ -105,7 +113,7 @@ Under **no circumstances** should the replication listener be exposed to the
public internet; it has no authentication and is unencrypted.
public internet; it has no authentication and is unencrypted.
### Worker
C
onfiguration
### Worker
c
onfiguration
In the config file for each worker, you must specify the type of worker
In the config file for each worker, you must specify the type of worker
application (
`worker_app`
), and you should specify a unqiue name for the worker
application (
`worker_app`
), and you should specify a unqiue name for the worker
...
@@ -145,6 +153,9 @@ plain HTTP endpoint on port 8083 separately serving various endpoints, e.g.
...
@@ -145,6 +153,9 @@ plain HTTP endpoint on port 8083 separately serving various endpoints, e.g.
Obviously you should configure your reverse-proxy to route the relevant
Obviously you should configure your reverse-proxy to route the relevant
endpoints to the worker (
`localhost:8083`
in the above example).
endpoints to the worker (
`localhost:8083`
in the above example).
### Running Synapse with workers
Finally, you need to start your worker processes. This can be done with either
Finally, you need to start your worker processes. This can be done with either
`synctl`
or your distribution's preferred service manager such as
`systemd`
. We
`synctl`
or your distribution's preferred service manager such as
`systemd`
. We
recommend the use of
`systemd`
where available: for information on setting up
recommend the use of
`systemd`
where available: for information on setting up
...
@@ -407,6 +418,23 @@ all these to be folded into the `generic_worker` app and to use config to define
...
@@ -407,6 +418,23 @@ all these to be folded into the `generic_worker` app and to use config to define
which processes handle the various proccessing such as push notifications.
which processes handle the various proccessing such as push notifications.
## Migration from old config
There are two main independent changes that have been made: introducing Redis
support and merging apps into
`synapse.app.generic_worker`
. Both these changes
are backwards compatible and so no changes to the config are required, however
server admins are encouraged to plan to migrate to Redis as the old style direct
TCP replication config is deprecated.
To migrate to Redis add the
`redis`
config as above, and optionally remove the
TCP
`replication`
listener from master and
`worker_replication_port`
from worker
config.
To migrate apps to use
`synapse.app.generic_worker`
simply update the
`worker_app`
option in the worker configs, and where worker are started (e.g.
in systemd service files, but not required for synctl).
## Architectural diagram
## Architectural diagram
The following shows an example setup using Redis and a reverse proxy:
The following shows an example setup using Redis and a reverse proxy:
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment