30 lines
937 B
Markdown
30 lines
937 B
Markdown
# SQ-025: Compression & Performance Tuning
|
|
|
|
**Status:** `[x] DONE`
|
|
**Blocked by:** SQ-024
|
|
**Priority:** Low
|
|
|
|
## Description
|
|
|
|
Add zstd compression to S3 segment shipping and create a benchmark suite.
|
|
|
|
## Files to Create/Modify
|
|
|
|
- `crates/sq-storage/src/object_store/shipper.rs` - add zstd compression
|
|
- `crates/sq-storage/src/object_store/reader.rs` - add zstd decompression
|
|
- `crates/sq-storage/benches/` - throughput benchmarks
|
|
|
|
## Benchmarks
|
|
|
|
- Write throughput: messages/sec at various payload sizes
|
|
- Read throughput: messages/sec sequential scan
|
|
- Compression ratio: raw vs compressed segment size
|
|
- S3 round-trip: write, ship, trim, read from S3
|
|
|
|
## Acceptance Criteria
|
|
|
|
- [ ] Compressed segments round-trip correctly (write -> compress -> upload -> download -> decompress -> read)
|
|
- [ ] Compression ratio metrics are logged
|
|
- [ ] Benchmarks produce readable output
|
|
- [ ] No correctness regressions (all existing tests still pass)
|