Make `synapse_port_db` correctly create indexes in the PostgreSQL database, by having it run the background updates on the database before migrating the data.

To ensure we're migrating the right data, also block the port if the SQLite3 database still has pending or ongoing background updates.

Fixes #4877 

Small fix to synapse_port_db to be able to convert from database schema v56.

Fixes #5306

Tables dropped: 
 * application_services, 
 * application_services_regex, 
 * transaction_id_to_pdu, 
 * stats_reporting
 * current_state_resets
 * event_content_hashes
 * event_destinations
 * event_edge_hashes
 * event_signatures
 * feedback
 * room_hosts
 * state_forward_extremities

Make synapse_port_db --curses work with Python 3.
Signed-off-by: Anders Jensen-Waud <anders@jensenwaud.com>

This column was considered as an int, crashing the whole
migration process

Signed-off-by: Eric <eric@pedr0.net>

Broadly three things here:

* disable W504 which seems a bit whacko
* remove a bunch of `as e` expressions from exception handlers that don't use
  them
* use `r""` for strings which include backslashes

Also, we don't use pep8 any more, so we can get rid of the duplicate config
there.

Signed-off-by: Adrian Tschira <nota@notafile.com>

Fixes https://github.com/matrix-org/synapse/issues/3050.

Improve logging and comments. Group all the stuff to do with inspecting tables
together rather than creating the port tables in the middle.

We really shouldn't spit out "Failed to create port table", it looks scary.

Postgres doesn't support NULs in strings so it makes the script
throw an exception and stop if any values contain \0. Drop them
with appropriate warning.

We changed _simple_update_one_txn to use _simple_update_txn but didn't
yank it out in the port script.

Fixes #2565

https://github.com/matrix-org/synapse/issues/2187 contains a report of a port
failing due to nul characters somewhere in the search table. Let's try dropping
the offending rows.

When fetching the list of tables from the postgres database during the
db port, it is assumed that the tables are in the public schema. This is
not always the case, so lets just rely on postgres to determine the
default schema to use.

This is what I get for not proofreading

See https://matrix.to/#/!cURbafjkfsMDVwdRDQ:matrix.org/$148686272020hCgRD:potatofrom.space



Signed-off-by: Kevin Liu <kevin@potatofrom.space>

This also fixes a bug where the port script would explode when it
encountered the newly added boolean column
`public_room_list_stream.visibility`