27
todos/SQ-017-wal-trimming.md
Normal file
27
todos/SQ-017-wal-trimming.md
Normal file
@@ -0,0 +1,27 @@
|
||||
# SQ-017: WAL Trimming
|
||||
|
||||
**Status:** `[ ] TODO`
|
||||
**Blocked by:** SQ-016
|
||||
**Priority:** Medium
|
||||
|
||||
## Description
|
||||
|
||||
Garbage collect local WAL segments after they have been confirmed in object storage.
|
||||
|
||||
## Files to Create/Modify
|
||||
|
||||
- `crates/sq-storage/src/wal/trimmer.rs` - WalTrimmer
|
||||
|
||||
## Behavior
|
||||
|
||||
1. Periodically scan for segments marked as "shipped"
|
||||
2. Verify the segment exists in object storage (optional double-check)
|
||||
3. Delete the local WAL segment file
|
||||
4. Update the offset index to point to S3 location instead
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] Segment marked as shipped -> trimmer deletes local file
|
||||
- [ ] Segment NOT marked as shipped -> trimmer leaves it
|
||||
- [ ] After trimming, index entries point to S3 location
|
||||
- [ ] Trimmer respects a minimum retention period (keep recent segments locally even if shipped)
|
||||
Reference in New Issue
Block a user