# SQ-004: WAL Segment Writer **Status:** `[x] DONE` **Blocked by:** SQ-002, SQ-003 **Priority:** High ## Description Implement the WAL segment writer that appends records to segment files with fsync for durability. Handles segment rotation when size or time thresholds are exceeded. ## Files to Create/Modify - `crates/sq-storage/src/wal/writer.rs` - WalWriter with append + fsync + rotation - `crates/sq-storage/src/wal/segment.rs` - segment header encoding/decoding ## Segment Header Format (32 bytes) ``` [magic: [u8; 4]] = b"SQWL" [version: u16] = 1 [topic_len: u16] [topic: [u8; 20]] (padded/truncated) [partition: u32] ``` ## WalWriter API ```rust pub struct WalWriter { fs: Arc, config: WalConfig, topic: TopicName, partition: u32, active_segment: Option>, segment_base_offset: u64, segment_position: u64, next_offset: u64, segment_opened_at: Instant, } impl WalWriter { pub async fn new(fs: Arc, config: WalConfig, topic: TopicName, partition: u32) -> Result; pub async fn append(&mut self, key: Option<&[u8]>, value: &[u8], headers: &[Header], timestamp_ms: u64) -> Result; // returns offset pub async fn close_active_segment(&mut self) -> Result>; pub fn next_offset(&self) -> u64; } ``` ## Acceptance Criteria (using InMemoryFileSystem) - [ ] Write 1 message, verify segment file exists with correct header + record - [ ] Write 100 messages, verify all offsets are monotonically increasing (0, 1, 2, ...) - [ ] Segment rotation: write until size > max_segment_bytes, verify new segment created - [ ] Segment rotation: advance clock past max_segment_age, verify rotation on next write - [ ] fsync failure: set fault on InMemoryFS, verify append() returns error - [ ] fsync failure: offset is NOT advanced (can retry the write) - [ ] Segment directory structure: `{data_dir}/{topic}/{partition}/{base_offset}.wal` ## Notes - sq-storage depends on sq-sim for the FileSystem trait - Writer must call fsync after every append (or batch of appends) - `ClosedSegment` contains the path and offset range of the completed segment