Skip to content
Snippets Groups Projects
  • Erik Johnston's avatar
    c46d452c
    Fix bug where purging history could lead to increase in disk space usage (#18131) · c46d452c
    Erik Johnston authored
    When purging history, we try and delete any state groups that become
    unreferenced (i.e. there are no longer any events that directly
    reference them). When we delete a state group that is referenced by
    another state group, we "de-delta" that state group so that it no longer
    refers to the state group that is deleted.
    
    There are two bugs with this approach that we fix here:
    1. There is a common pattern where we end up storing two state groups
    when persisting a state event: the state before and after the new state
    event, where the latter is stored as a delta to the former. When
    deleting state groups we only deleted the "new" state and left (and
    potentially de-deltaed) the old state. This was due to a bug/typo when
    trying to find referenced state groups.
    2. There are times where we store unreferenced state groups in the DB,
    during the purging of history these would not get rechecked and instead
    always de-deltaed. Instead, we should check for this case and delete any
    unreferenced state groups rather than de-deltaing them.
    
    The effect of the above bugs is that when purging history we'd end up
    with lots of unreferenced state groups that had been de-deltaed (i.e.
    stored as the full state). This can lead to dramatic increases in
    storage space used.
    Fix bug where purging history could lead to increase in disk space usage (#18131)
    Erik Johnston authored
    When purging history, we try and delete any state groups that become
    unreferenced (i.e. there are no longer any events that directly
    reference them). When we delete a state group that is referenced by
    another state group, we "de-delta" that state group so that it no longer
    refers to the state group that is deleted.
    
    There are two bugs with this approach that we fix here:
    1. There is a common pattern where we end up storing two state groups
    when persisting a state event: the state before and after the new state
    event, where the latter is stored as a delta to the former. When
    deleting state groups we only deleted the "new" state and left (and
    potentially de-deltaed) the old state. This was due to a bug/typo when
    trying to find referenced state groups.
    2. There are times where we store unreferenced state groups in the DB,
    during the purging of history these would not get rechecked and instead
    always de-deltaed. Instead, we should check for this case and delete any
    unreferenced state groups rather than de-deltaing them.
    
    The effect of the above bugs is that when purging history we'd end up
    with lots of unreferenced state groups that had been de-deltaed (i.e.
    stored as the full state). This can lead to dramatic increases in
    storage space used.