Files
sq/todos/SQ-017-wal-trimming.md
2026-02-26 21:52:50 +01:00

808 B

SQ-017: WAL Trimming

Status: [ ] TODO Blocked by: SQ-016 Priority: Medium

Description

Garbage collect local WAL segments after they have been confirmed in object storage.

Files to Create/Modify

  • crates/sq-storage/src/wal/trimmer.rs - WalTrimmer

Behavior

  1. Periodically scan for segments marked as "shipped"
  2. Verify the segment exists in object storage (optional double-check)
  3. Delete the local WAL segment file
  4. Update the offset index to point to S3 location instead

Acceptance Criteria

  • Segment marked as shipped -> trimmer deletes local file
  • Segment NOT marked as shipped -> trimmer leaves it
  • After trimming, index entries point to S3 location
  • Trimmer respects a minimum retention period (keep recent segments locally even if shipped)