50 lines
1.6 KiB
Markdown
50 lines
1.6 KiB
Markdown
# SQ-002: WAL Record Encoding/Decoding
|
|
|
|
**Status:** `[ ] TODO`
|
|
**Blocked by:** SQ-001
|
|
**Priority:** High
|
|
|
|
## Description
|
|
|
|
Implement binary encoding and decoding of individual WAL records, independent of file I/O. Each record is CRC32-protected for corruption detection.
|
|
|
|
## Files to Create/Modify
|
|
|
|
- `crates/sq-storage/src/wal/mod.rs` - module declaration
|
|
- `crates/sq-storage/src/wal/record.rs` - encode_record, decode_record, CRC validation
|
|
- `crates/sq-storage/src/lib.rs` - re-export wal module
|
|
|
|
## Record Binary Format
|
|
|
|
```
|
|
[crc32: u32] - CRC32 over everything after this field
|
|
[length: u32] - total byte length of record (excluding crc32 and length)
|
|
[offset: u64] - monotonic offset
|
|
[timestamp_ms: u64] - wall clock millis
|
|
[key_len: u32] - 0 = no key
|
|
[key: [u8; key_len]]
|
|
[value_len: u32]
|
|
[value: [u8; value_len]]
|
|
[headers_count: u16]
|
|
[for each header:]
|
|
[hdr_key_len: u16]
|
|
[hdr_key: [u8; hdr_key_len]]
|
|
[hdr_val_len: u32]
|
|
[hdr_val: [u8; hdr_val_len]]
|
|
```
|
|
|
|
## Acceptance Criteria
|
|
|
|
- [ ] `encode_record(&Message) -> Vec<u8>` produces correct binary
|
|
- [ ] `decode_record(&[u8]) -> Result<(Message, usize)>` parses correctly (returns bytes consumed)
|
|
- [ ] Roundtrip: encode then decode, verify equality
|
|
- [ ] Corruption test: flip a byte, decode returns CRC error
|
|
- [ ] Edge cases: empty value, empty key, no headers, many headers, max-size value
|
|
- [ ] Uses `crc32fast` crate for CRC computation
|
|
|
|
## Notes
|
|
|
|
- Use little-endian byte order throughout
|
|
- The `length` field allows skipping records without full parsing
|
|
- CRC is over the bytes AFTER the CRC field itself
|