Sliding Sync: Speed up incremental sync by avoiding extra work (#17665)

Speed up incremental sync by avoiding extra work. We first look at the
state delta changes and only fetch and calculate further derived things
if they have changed.