Fix a bug where we would discard a key result which the origin server is no
longer returning. Fixes #5305.