Pagination of /relations doesn't return correct next page

Passing a given next_batch to the from parameter of /relations doesn't return the next page, but returns the same page with 1 event removed. This can be repeated until there are no events left.

Requests

First page:

https://matrix.blazma.st/_matrix/client/v1/rooms/!A1R2YakaTm2rdoxTJu%3Ablazma.st/relations/%24wk4oYc_EcF7X9accDv8BD-Dv6ZGv1bof3dFMV9yRKTc?dir=b

First page response

{
    "chunk": [
        {
            "content": {
                "body": "11",
                "m.mentions": {},
                "m.relates_to": {
                    "event_id": "$wk4oYc_EcF7X9accDv8BD-Dv6ZGv1bof3dFMV9yRKTc",
                    "is_falling_back": true,
                    "m.in_reply_to": {
                        "event_id": "$wk4oYc_EcF7X9accDv8BD-Dv6ZGv1bof3dFMV9yRKTc"
                    },
                    "rel_type": "m.thread"
                },
                "msgtype": "m.text"
            },
            "event_id": "$US7vLqX5QM4hbi8Qd0sFt5Fma9IjsNusejD5qVkz3uI",
            "origin_server_ts": 1723878751443,
            "room_id": "!A1R2YakaTm2rdoxTJu:blazma.st",
            "sender": "@spur:blazma.st",
            "type": "m.room.message",
            "unsigned": {}
        },
        {
            "content": {
                "body": "10",
                "m.mentions": {},
                "m.relates_to": {
                    "event_id": "$wk4oYc_EcF7X9accDv8BD-Dv6ZGv1bof3dFMV9yRKTc",
                    "is_falling_back": true,
                    "m.in_reply_to": {
                        "event_id": "$KSPS3BJuJAzDV9l3STmK7TFrFSUUGwhrZhAKP8A-CNk"
                    },
                    "rel_type": "m.thread"
                },
                "msgtype": "m.text"
            },
            "event_id": "$zJP0km2Ovrq9qUx2cUXt1N5sFLQyEJij71Ee1cO8rbw",
            "origin_server_ts": 1723878722904,
            "room_id": "!A1R2YakaTm2rdoxTJu:blazma.st",
            "sender": "@test-bot:blazma.st",
            "type": "m.room.message",
            "unsigned": {
                "transaction_id": "m1723878714891.9"
            }
        },
        {
            "content": {
                "body": "9",
                "m.mentions": {},
                "m.relates_to": {
                    "event_id": "$wk4oYc_EcF7X9accDv8BD-Dv6ZGv1bof3dFMV9yRKTc",
                    "is_falling_back": true,
                    "m.in_reply_to": {
                        "event_id": "$hQNt9XxJ3-6X1YsiLn-8O515Q4YEr8F_nUbclAEPubE"
                    },
                    "rel_type": "m.thread"
                },
                "msgtype": "m.text"
            },
            "event_id": "$3d3X5a5WzRJPswQcMC3o9qCg-guoXvLek43LHZ--MsI",
            "origin_server_ts": 1723878720734,
            "room_id": "!A1R2YakaTm2rdoxTJu:blazma.st",
            "sender": "@test-bot:blazma.st",
            "type": "m.room.message",
            "unsigned": {
                "transaction_id": "m1723878712644.8"
            }
        },
        {
            "content": {
                "body": "8",
                "m.mentions": {},
                "m.relates_to": {
                    "event_id": "$wk4oYc_EcF7X9accDv8BD-Dv6ZGv1bof3dFMV9yRKTc",
                    "is_falling_back": true,
                    "m.in_reply_to": {
                        "event_id": "$hQNt9XxJ3-6X1YsiLn-8O515Q4YEr8F_nUbclAEPubE"
                    },
                    "rel_type": "m.thread"
                },
                "msgtype": "m.text"
            },
            "event_id": "$FRB54-s8lrVY3XMYhX-aIeVKZKbNN4i4mSYpODR9EPY",
            "origin_server_ts": 1723878719716,
            "room_id": "!A1R2YakaTm2rdoxTJu:blazma.st",
            "sender": "@test-bot:blazma.st",
            "type": "m.room.message",
            "unsigned": {
                "transaction_id": "m1723878711886.7"
            }
        },
        {
            "content": {
                "body": "7",
                "m.mentions": {},
                "m.relates_to": {
                    "event_id": "$wk4oYc_EcF7X9accDv8BD-Dv6ZGv1bof3dFMV9yRKTc",
                    "is_falling_back": true,
                    "m.in_reply_to": {
                        "event_id": "$SXenHKcWa3OTk5cuTo8MhVmsSSruhX8180Bj7weySKk"
                    },
                    "rel_type": "m.thread"
                },
                "msgtype": "m.text"
            },
            "event_id": "$dVLXzIEg3jUWesKXCGvsXMpOwH1-ArxONCWAjV6huGM",
            "origin_server_ts": 1723878718405,
            "room_id": "!A1R2YakaTm2rdoxTJu:blazma.st",
            "sender": "@test-bot:blazma.st",
            "type": "m.room.message",
            "unsigned": {
                "transaction_id": "m1723878711049.6"
            }
        },
        {
            "content": {
                "body": "6",
                "m.mentions": {},
                "m.relates_to": {
                    "event_id": "$wk4oYc_EcF7X9accDv8BD-Dv6ZGv1bof3dFMV9yRKTc",
                    "is_falling_back": true,
                    "m.in_reply_to": {
                        "event_id": "$SXenHKcWa3OTk5cuTo8MhVmsSSruhX8180Bj7weySKk"
                    },
                    "rel_type": "m.thread"
                },
                "msgtype": "m.text"
            },
            "event_id": "$KSPS3BJuJAzDV9l3STmK7TFrFSUUGwhrZhAKP8A-CNk",
            "origin_server_ts": 1723878712938,
            "room_id": "!A1R2YakaTm2rdoxTJu:blazma.st",
            "sender": "@test-bot:blazma.st",
            "type": "m.room.message",
            "unsigned": {
                "transaction_id": "m1723878710264.5"
            }
        },
        {
            "content": {
                "body": "5",
                "m.mentions": {},
                "m.relates_to": {
                    "event_id": "$wk4oYc_EcF7X9accDv8BD-Dv6ZGv1bof3dFMV9yRKTc",
                    "is_falling_back": true,
                    "m.in_reply_to": {
                        "event_id": "$V4GWW_j-mogfmWZil4Iofb3J_qcV3sSbMD7dlLZehGQ"
                    },
                    "rel_type": "m.thread"
                },
                "msgtype": "m.text"
            },
            "event_id": "$hQNt9XxJ3-6X1YsiLn-8O515Q4YEr8F_nUbclAEPubE",
            "origin_server_ts": 1723878711177,
            "room_id": "!A1R2YakaTm2rdoxTJu:blazma.st",
            "sender": "@test-bot:blazma.st",
            "type": "m.room.message",
            "unsigned": {
                "transaction_id": "m1723878709307.4"
            }
        },
        {
            "content": {
                "body": "4",
                "m.mentions": {},
                "m.relates_to": {
                    "event_id": "$wk4oYc_EcF7X9accDv8BD-Dv6ZGv1bof3dFMV9yRKTc",
                    "is_falling_back": true,
                    "m.in_reply_to": {
                        "event_id": "$3fR8jwqD7XZEFJHKgNoAR8rXA0ppZEs9_J_1hV2Ikyw"
                    },
                    "rel_type": "m.thread"
                },
                "msgtype": "m.text"
            },
            "event_id": "$SXenHKcWa3OTk5cuTo8MhVmsSSruhX8180Bj7weySKk",
            "origin_server_ts": 1723878708938,
            "room_id": "!A1R2YakaTm2rdoxTJu:blazma.st",
            "sender": "@test-bot:blazma.st",
            "type": "m.room.message",
            "unsigned": {
                "transaction_id": "m1723878708378.3"
            }
        },
        {
            "content": {
                "body": "3",
                "m.mentions": {},
                "m.relates_to": {
                    "event_id": "$wk4oYc_EcF7X9accDv8BD-Dv6ZGv1bof3dFMV9yRKTc",
                    "is_falling_back": true,
                    "m.in_reply_to": {
                        "event_id": "$3fR8jwqD7XZEFJHKgNoAR8rXA0ppZEs9_J_1hV2Ikyw"
                    },
                    "rel_type": "m.thread"
                },
                "msgtype": "m.text"
            },
            "event_id": "$V4GWW_j-mogfmWZil4Iofb3J_qcV3sSbMD7dlLZehGQ",
            "origin_server_ts": 1723878707206,
            "room_id": "!A1R2YakaTm2rdoxTJu:blazma.st",
            "sender": "@test-bot:blazma.st",
            "type": "m.room.message",
            "unsigned": {
                "transaction_id": "m1723878706291.2"
            }
        },
        {
            "content": {
                "body": "2",
                "m.mentions": {},
                "m.relates_to": {
                    "event_id": "$wk4oYc_EcF7X9accDv8BD-Dv6ZGv1bof3dFMV9yRKTc",
                    "is_falling_back": true,
                    "m.in_reply_to": {
                        "event_id": "$wk4oYc_EcF7X9accDv8BD-Dv6ZGv1bof3dFMV9yRKTc"
                    },
                    "rel_type": "m.thread"
                },
                "msgtype": "m.text"
            },
            "event_id": "$3fR8jwqD7XZEFJHKgNoAR8rXA0ppZEs9_J_1hV2Ikyw",
            "origin_server_ts": 1723878702695,
            "room_id": "!A1R2YakaTm2rdoxTJu:blazma.st",
            "sender": "@test-bot:blazma.st",
            "type": "m.room.message",
            "unsigned": {
                "transaction_id": "m1723878701016.1"
            }
        }
    ],
    "next_batch": "4750766",
    "prev_batch": "18446744073709551615"
}

Requesting the second page using the given next_batch:

https://matrix.blazma.st/_matrix/client/v1/rooms/!A1R2YakaTm2rdoxTJu%3Ablazma.st/relations/%24wk4oYc_EcF7X9accDv8BD-Dv6ZGv1bof3dFMV9yRKTc?dir=b&limit=50&from=4750766

Second page response

Notice the same events here as the previous page

{
    "chunk": [
        {
            "content": {
                "body": "10",
                "m.mentions": {},
                "m.relates_to": {
                    "event_id": "$wk4oYc_EcF7X9accDv8BD-Dv6ZGv1bof3dFMV9yRKTc",
                    "is_falling_back": true,
                    "m.in_reply_to": {
                        "event_id": "$KSPS3BJuJAzDV9l3STmK7TFrFSUUGwhrZhAKP8A-CNk"
                    },
                    "rel_type": "m.thread"
                },
                "msgtype": "m.text"
            },
            "event_id": "$zJP0km2Ovrq9qUx2cUXt1N5sFLQyEJij71Ee1cO8rbw",
            "origin_server_ts": 1723878722904,
            "room_id": "!A1R2YakaTm2rdoxTJu:blazma.st",
            "sender": "@test-bot:blazma.st",
            "type": "m.room.message",
            "unsigned": {
                "transaction_id": "m1723878714891.9"
            }
        },
        {
            "content": {
                "body": "9",
                "m.mentions": {},
                "m.relates_to": {
                    "event_id": "$wk4oYc_EcF7X9accDv8BD-Dv6ZGv1bof3dFMV9yRKTc",
                    "is_falling_back": true,
                    "m.in_reply_to": {
                        "event_id": "$hQNt9XxJ3-6X1YsiLn-8O515Q4YEr8F_nUbclAEPubE"
                    },
                    "rel_type": "m.thread"
                },
                "msgtype": "m.text"
            },
            "event_id": "$3d3X5a5WzRJPswQcMC3o9qCg-guoXvLek43LHZ--MsI",
            "origin_server_ts": 1723878720734,
            "room_id": "!A1R2YakaTm2rdoxTJu:blazma.st",
            "sender": "@test-bot:blazma.st",
            "type": "m.room.message",
            "unsigned": {
                "transaction_id": "m1723878712644.8"
            }
        },
        {
            "content": {
                "body": "8",
                "m.mentions": {},
                "m.relates_to": {
                    "event_id": "$wk4oYc_EcF7X9accDv8BD-Dv6ZGv1bof3dFMV9yRKTc",
                    "is_falling_back": true,
                    "m.in_reply_to": {
                        "event_id": "$hQNt9XxJ3-6X1YsiLn-8O515Q4YEr8F_nUbclAEPubE"
                    },
                    "rel_type": "m.thread"
                },
                "msgtype": "m.text"
            },
            "event_id": "$FRB54-s8lrVY3XMYhX-aIeVKZKbNN4i4mSYpODR9EPY",
            "origin_server_ts": 1723878719716,
            "room_id": "!A1R2YakaTm2rdoxTJu:blazma.st",
            "sender": "@test-bot:blazma.st",
            "type": "m.room.message",
            "unsigned": {
                "transaction_id": "m1723878711886.7"
            }
        },
        {
            "content": {
                "body": "7",
                "m.mentions": {},
                "m.relates_to": {
                    "event_id": "$wk4oYc_EcF7X9accDv8BD-Dv6ZGv1bof3dFMV9yRKTc",
                    "is_falling_back": true,
                    "m.in_reply_to": {
                        "event_id": "$SXenHKcWa3OTk5cuTo8MhVmsSSruhX8180Bj7weySKk"
                    },
                    "rel_type": "m.thread"
                },
                "msgtype": "m.text"
            },
            "event_id": "$dVLXzIEg3jUWesKXCGvsXMpOwH1-ArxONCWAjV6huGM",
            "origin_server_ts": 1723878718405,
            "room_id": "!A1R2YakaTm2rdoxTJu:blazma.st",
            "sender": "@test-bot:blazma.st",
            "type": "m.room.message",
            "unsigned": {
                "transaction_id": "m1723878711049.6"
            }
        },
        {
            "content": {
                "body": "6",
                "m.mentions": {},
                "m.relates_to": {
                    "event_id": "$wk4oYc_EcF7X9accDv8BD-Dv6ZGv1bof3dFMV9yRKTc",
                    "is_falling_back": true,
                    "m.in_reply_to": {
                        "event_id": "$SXenHKcWa3OTk5cuTo8MhVmsSSruhX8180Bj7weySKk"
                    },
                    "rel_type": "m.thread"
                },
                "msgtype": "m.text"
            },
            "event_id": "$KSPS3BJuJAzDV9l3STmK7TFrFSUUGwhrZhAKP8A-CNk",
            "origin_server_ts": 1723878712938,
            "room_id": "!A1R2YakaTm2rdoxTJu:blazma.st",
            "sender": "@test-bot:blazma.st",
            "type": "m.room.message",
            "unsigned": {
                "transaction_id": "m1723878710264.5"
            }
        },
        {
            "content": {
                "body": "5",
                "m.mentions": {},
                "m.relates_to": {
                    "event_id": "$wk4oYc_EcF7X9accDv8BD-Dv6ZGv1bof3dFMV9yRKTc",
                    "is_falling_back": true,
                    "m.in_reply_to": {
                        "event_id": "$V4GWW_j-mogfmWZil4Iofb3J_qcV3sSbMD7dlLZehGQ"
                    },
                    "rel_type": "m.thread"
                },
                "msgtype": "m.text"
            },
            "event_id": "$hQNt9XxJ3-6X1YsiLn-8O515Q4YEr8F_nUbclAEPubE",
            "origin_server_ts": 1723878711177,
            "room_id": "!A1R2YakaTm2rdoxTJu:blazma.st",
            "sender": "@test-bot:blazma.st",
            "type": "m.room.message",
            "unsigned": {
                "transaction_id": "m1723878709307.4"
            }
        },
        {
            "content": {
                "body": "4",
                "m.mentions": {},
                "m.relates_to": {
                    "event_id": "$wk4oYc_EcF7X9accDv8BD-Dv6ZGv1bof3dFMV9yRKTc",
                    "is_falling_back": true,
                    "m.in_reply_to": {
                        "event_id": "$3fR8jwqD7XZEFJHKgNoAR8rXA0ppZEs9_J_1hV2Ikyw"
                    },
                    "rel_type": "m.thread"
                },
                "msgtype": "m.text"
            },
            "event_id": "$SXenHKcWa3OTk5cuTo8MhVmsSSruhX8180Bj7weySKk",
            "origin_server_ts": 1723878708938,
            "room_id": "!A1R2YakaTm2rdoxTJu:blazma.st",
            "sender": "@test-bot:blazma.st",
            "type": "m.room.message",
            "unsigned": {
                "transaction_id": "m1723878708378.3"
            }
        },
        {
            "content": {
                "body": "3",
                "m.mentions": {},
                "m.relates_to": {
                    "event_id": "$wk4oYc_EcF7X9accDv8BD-Dv6ZGv1bof3dFMV9yRKTc",
                    "is_falling_back": true,
                    "m.in_reply_to": {
                        "event_id": "$3fR8jwqD7XZEFJHKgNoAR8rXA0ppZEs9_J_1hV2Ikyw"
                    },
                    "rel_type": "m.thread"
                },
                "msgtype": "m.text"
            },
            "event_id": "$V4GWW_j-mogfmWZil4Iofb3J_qcV3sSbMD7dlLZehGQ",
            "origin_server_ts": 1723878707206,
            "room_id": "!A1R2YakaTm2rdoxTJu:blazma.st",
            "sender": "@test-bot:blazma.st",
            "type": "m.room.message",
            "unsigned": {
                "transaction_id": "m1723878706291.2"
            }
        },
        {
            "content": {
                "body": "2",
                "m.mentions": {},
                "m.relates_to": {
                    "event_id": "$wk4oYc_EcF7X9accDv8BD-Dv6ZGv1bof3dFMV9yRKTc",
                    "is_falling_back": true,
                    "m.in_reply_to": {
                        "event_id": "$wk4oYc_EcF7X9accDv8BD-Dv6ZGv1bof3dFMV9yRKTc"
                    },
                    "rel_type": "m.thread"
                },
                "msgtype": "m.text"
            },
            "event_id": "$3fR8jwqD7XZEFJHKgNoAR8rXA0ppZEs9_J_1hV2Ikyw",
            "origin_server_ts": 1723878702695,
            "room_id": "!A1R2YakaTm2rdoxTJu:blazma.st",
            "sender": "@test-bot:blazma.st",
            "type": "m.room.message",
            "unsigned": {
                "transaction_id": "m1723878701016.1"
            }
        }
    ],
    "next_batch": "4750707",
    "prev_batch": "4750766"
}

Tested on 85c5b11e and 66dc631

added bug label

TODO: should be relations_after sounds relevant to this issue.

mentioned in merge request !609 (merged)

If you would like to, could you please test and see if the issue still persists on main branch?

Seems like all the events are there now, but (at least on Element) reaction events show up as the first events (with displaying hidden events enabled), so, wrong order for those(?), but the regular messages are correct. Seems like Element also sometimes tries loading events past the first thread message and gives a You can't see earlier messages. I really don't know if these issues are related though, so, if they're not, then #556 (closed) can probably be closed.

It's possible both of these are Element bugs. I'm not sure if there's another client with threading support, but if there is testing it there would be helpful.

I'll consider this resolved, if there are any further threads issues that are confirmed to not be client-related, please open new issues for those.

closed