42
todos/SQ-007-storage-engine-facade.md
Normal file
42
todos/SQ-007-storage-engine-facade.md
Normal file
@@ -0,0 +1,42 @@
|
||||
# SQ-007: Storage Engine Facade
|
||||
|
||||
**Status:** `[ ] TODO`
|
||||
**Blocked by:** SQ-006
|
||||
**Priority:** High
|
||||
|
||||
## Description
|
||||
|
||||
Unified read/write interface wrapping WAL writer + reader + offset index. This is the single entry point for all storage operations used by the server.
|
||||
|
||||
## Files to Create/Modify
|
||||
|
||||
- `crates/sq-storage/src/engine.rs` - StorageEngine
|
||||
|
||||
## StorageEngine API
|
||||
|
||||
```rust
|
||||
pub struct StorageEngine<F: FileSystem, C: Clock> {
|
||||
fs: Arc<F>,
|
||||
clock: Arc<C>,
|
||||
config: WalConfig,
|
||||
writers: HashMap<(String, u32), WalWriter<F>>,
|
||||
index: OffsetIndex,
|
||||
}
|
||||
|
||||
impl<F: FileSystem, C: Clock> StorageEngine<F, C> {
|
||||
pub async fn new(fs: Arc<F>, clock: Arc<C>, config: WalConfig) -> Result<Self>;
|
||||
pub async fn append(&mut self, topic: &str, partition: u32, key: Option<&[u8]>, value: &[u8], headers: &[Header]) -> Result<u64>;
|
||||
pub async fn read(&self, topic: &str, partition: u32, from_offset: u64, limit: usize) -> Result<Vec<Message>>;
|
||||
pub async fn recover(&mut self) -> Result<()>; // Rebuild state from existing WAL files on startup
|
||||
pub fn closed_segments(&self) -> Vec<ClosedSegment>; // For S3 shipper
|
||||
}
|
||||
```
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] Write 1000 messages, read from offset 0, verify all present
|
||||
- [ ] Write, read from offset 500, verify correct slice returned
|
||||
- [ ] Read with limit, verify at most N messages returned
|
||||
- [ ] Write to multiple topics/partitions, verify isolation (no cross-contamination)
|
||||
- [ ] Recovery: write messages, drop engine, create new engine, call recover(), read all messages back
|
||||
- [ ] Segment rotation happens transparently during append
|
||||
Reference in New Issue
Block a user