43 lines
1.6 KiB
Markdown
43 lines
1.6 KiB
Markdown
# SQ-007: Storage Engine Facade
|
|
|
|
**Status:** `[ ] TODO`
|
|
**Blocked by:** SQ-006
|
|
**Priority:** High
|
|
|
|
## Description
|
|
|
|
Unified read/write interface wrapping WAL writer + reader + offset index. This is the single entry point for all storage operations used by the server.
|
|
|
|
## Files to Create/Modify
|
|
|
|
- `crates/sq-storage/src/engine.rs` - StorageEngine
|
|
|
|
## StorageEngine API
|
|
|
|
```rust
|
|
pub struct StorageEngine<F: FileSystem, C: Clock> {
|
|
fs: Arc<F>,
|
|
clock: Arc<C>,
|
|
config: WalConfig,
|
|
writers: HashMap<(String, u32), WalWriter<F>>,
|
|
index: OffsetIndex,
|
|
}
|
|
|
|
impl<F: FileSystem, C: Clock> StorageEngine<F, C> {
|
|
pub async fn new(fs: Arc<F>, clock: Arc<C>, config: WalConfig) -> Result<Self>;
|
|
pub async fn append(&mut self, topic: &str, partition: u32, key: Option<&[u8]>, value: &[u8], headers: &[Header]) -> Result<u64>;
|
|
pub async fn read(&self, topic: &str, partition: u32, from_offset: u64, limit: usize) -> Result<Vec<Message>>;
|
|
pub async fn recover(&mut self) -> Result<()>; // Rebuild state from existing WAL files on startup
|
|
pub fn closed_segments(&self) -> Vec<ClosedSegment>; // For S3 shipper
|
|
}
|
|
```
|
|
|
|
## Acceptance Criteria
|
|
|
|
- [ ] Write 1000 messages, read from offset 0, verify all present
|
|
- [ ] Write, read from offset 500, verify correct slice returned
|
|
- [ ] Read with limit, verify at most N messages returned
|
|
- [ ] Write to multiple topics/partitions, verify isolation (no cross-contamination)
|
|
- [ ] Recovery: write messages, drop engine, create new engine, call recover(), read all messages back
|
|
- [ ] Segment rotation happens transparently during append
|