808 B
808 B
SQ-017: WAL Trimming
Status: [ ] TODO
Blocked by: SQ-016
Priority: Medium
Description
Garbage collect local WAL segments after they have been confirmed in object storage.
Files to Create/Modify
crates/sq-storage/src/wal/trimmer.rs- WalTrimmer
Behavior
- Periodically scan for segments marked as "shipped"
- Verify the segment exists in object storage (optional double-check)
- Delete the local WAL segment file
- Update the offset index to point to S3 location instead
Acceptance Criteria
- Segment marked as shipped -> trimmer deletes local file
- Segment NOT marked as shipped -> trimmer leaves it
- After trimming, index entries point to S3 location
- Trimmer respects a minimum retention period (keep recent segments locally even if shipped)