28 lines
808 B
Markdown
28 lines
808 B
Markdown
# SQ-017: WAL Trimming
|
|
|
|
**Status:** `[ ] TODO`
|
|
**Blocked by:** SQ-016
|
|
**Priority:** Medium
|
|
|
|
## Description
|
|
|
|
Garbage collect local WAL segments after they have been confirmed in object storage.
|
|
|
|
## Files to Create/Modify
|
|
|
|
- `crates/sq-storage/src/wal/trimmer.rs` - WalTrimmer
|
|
|
|
## Behavior
|
|
|
|
1. Periodically scan for segments marked as "shipped"
|
|
2. Verify the segment exists in object storage (optional double-check)
|
|
3. Delete the local WAL segment file
|
|
4. Update the offset index to point to S3 location instead
|
|
|
|
## Acceptance Criteria
|
|
|
|
- [ ] Segment marked as shipped -> trimmer deletes local file
|
|
- [ ] Segment NOT marked as shipped -> trimmer leaves it
|
|
- [ ] After trimming, index entries point to S3 location
|
|
- [ ] Trimmer respects a minimum retention period (keep recent segments locally even if shipped)
|