* make the OIDC bits of the test work at a higher level - via the REST api instead of poking the OIDCHandler directly.
* Move it to test_login.py, where I think it fits better.

Again in preparation for handling more than one OIDC provider, add a new caveat to the macaroon used as an OIDC session cookie, which remembers which OIDC provider we are talking to. In future, when we get a callback, we'll need it to make sure we talk to the right IdP.

As part of this, I'm adding an idp_id and idp_name field to the OIDC configuration object. They aren't yet documented, and we'll just use the old values by default.

Give the user a better error when they present bad SSO creds

We passed in a graph to `sorted_topologically` which didn't have an
entry for each node (as we dropped nodes with no edges).

t was doing a sequential scan on `destination_rooms`, which took
minutes.

The idea here is that we will have an instance of OidcProvider for each
configured IdP, with OidcHandler just doing the marshalling of them.

For now it's still hardcoded with a single provider.

A reactor was being passed instead of a whitelist for the BlacklistingAgentWrapper
used by the WellyKnownResolver. This coulld cause exceptions when attempting to
connect to IP addresses that are blacklisted, but in reality this did not have any
observable affect since this code is not used for IP literals.

since we're hacking on this code anyway, may as well move it out of the
cluttered AuthHandler.

If a user tries to do UI Auth via SSO, but uses the wrong account on the SSO
IdP, try to give them a better error.

Previously, the UIA would claim to be successful, but then the operation in
question would simply fail with "auth fail". Instead, serve up an error page
which explains the failure.

* Add complete test for UI-Auth-via-SSO.

* review comments

Enhancements to OIDC configuration handling

Removes a bare `except Exception` clause and replaces it with
catching a specific exception around the portion that might throw.

Collect all the config options which related to an OIDC provider into a single
object.

This checks that the domain given to `DomainSpecificString.is_valid` (e.g.
`UserID`, `RoomAlias`, etc.) is of a valid form. Previously some validation
was done on the localpart (e.g. the sigil), but not the domain portion.

Some light refactoring of OidcHandler, in preparation for bigger things:

  * remove inheritance from deprecated BaseHandler
  * add an object to hold the things that go into a session cookie
  * factor out a separate class for manipulating said cookies

This only applies if the user's data is to be erased.

Fixes #8782

If we have integrations with multiple identity providers, when the user does a UI Auth, we need to redirect them to the right one.

There are a few steps to this. First of all we actually need to store the userid of the user we are trying to validate in the UIA session, since the /auth/sso/fallback/web request is unauthenticated.

Then, once we get the /auth/sso/fallback/web request, we can fish the user id out of the session, and use it to look up the external id mappings, and hence pick an SSO provider for them.

It is expected that the proxy would be on a private IP address so the
configured proxy should be connected to regardless of the IP range
blacklist.

Debian package builds were failing for two reasons:

 1. Python versions prior to 3.7 throw exceptions when attempting to print
    Unicode characters under a "C" locale. (#9076)

 2. We depended on `dh-systemd` which no longer exists in Debian Bullseye, but
    is necessary in Ubuntu Xenial. (#9073)

Setting `LANG="C.UTF-8"` in the build environment fixes the first issue.
See also: https://bugs.python.org/issue19846



The second issue is a bit trickier. The dh-systemd package was merged into
debhelper version 9.20160709 and a transitional package left in its wake.

The transitional dh-systemd package was removed in Debian Bullseye.

However, Ubuntu Xenial ships an older debhelper, and still needs dh-systemd.

Thus, builds were failing on Bullseye since we depended on a package which had
ceased existing, but we couldn't remove it from the debian/control file and our
build scripts because we still needed it for Ubuntu Xenial.

We can fix the debian/control issue by listing dh-systemd as an alternative to
the newer versions of debhelper. Since dh-systemd declares that it depends on
debhelper, Ubuntu Xenial will select its older dh-systemd which will in turn
pull in its older debhelper, resulting in no change from the status quo. All
other supported releases will satisfy the debhelper dependency constraint and
skip the dh-systemd alternative.

Build scripts were fixed by unconditionally attempting to install dh-systemd on
all releases and suppressing failures.

Once we drop support for Ubuntu Xenial, we can revert most of this commit and
rely on the version constraint on debhelper in debian/control.

Fixes #9076
Fixes #9073

Signed-off-by: Dan Callahan <danc@element.io>

Homeserver.get_ip_from_request() used to be a bit more complicated, but now it is totally redundant. Let's get rid of it.

SynapseRequest is in danger of becoming a bit of a dumping-ground for "useful stuff relating to Requests",
which isn't really its intention (its purpose is to override render, finished and connectionLost to set up the 
LoggingContext and write the right entries to the request log).

Putting utility functions inside SynapseRequest means that lots of our code ends up requiring a
SynapseRequest when there is nothing synapse-specific about the Request at all, and any old
twisted.web.iweb.IRequest will do. This increases code coupling and makes testing more difficult.

In short: move get_user_agent out to a utility function.

For remote users, only the rooms which the server knows about are returned.
Local users have all of their joined rooms returned.