Fix stuck invites

If rejecting a remote invite fails with an error response don't fail
the entire request; instead mark the invite as locally rejected.

This fixes the bug where users can get stuck invites which they can
neither accept nor reject.