Initialize *gen to NO_GENERATION at the start of file_tree_read so that
error paths (FILETREE_BADPATH, FILETREE_NOENT, FILETREE_ISDIR) have a
well-defined value. Previously, when file_tree_read failed, gen was
uninitialized, causing the assertion in process_client_read to fire.
The assertion checking gen != NO_GENERATION before the error check was
removed since:
1. On error paths, gen is not used
2. On success paths, the assertion at line 482 already validates gen
Implements a new special generation counter value (UINT64_MAX) that
allows clients to assert that a file should NOT exist when performing
operations.
Primary use case: Atomic "create-if-not-exists" operations
When combined with TOASTY_WRITE_CREATE_IF_MISSING flag:
- File doesn't exist → server creates file and writes (SUCCESS)
- File exists → gen_match fails → BADGEN error (prevents overwrite)
This enables race-condition-free file creation where you want to
create a NEW file or fail if it already exists.
Key changes:
- Added MISSING_FILE_GENERATION constant to file_tree.h
- Updated gen_match() to return false when checking existing entities
against MISSING_FILE_GENERATION
- Modified file_tree_delete_entity() to succeed (no-op) when file is
missing and MISSING_FILE_GENERATION is specified (idempotent delete)
- Modified file_tree_write() to validate against the new value
- Added documentation in metadata_server.c explaining how
MISSING_FILE_GENERATION works WITH CREATE_IF_MISSING
- Updated protocol documentation in PROTOCOL.txt
- Added comprehensive feature documentation in MISSING_FILE_GENERATION.md
Additional use cases:
- Idempotent delete operations (delete only if not already gone)
- Detecting unexpected file creation in validation/testing scenarios
The implementation is backward compatible and type-safe. Files are
never assigned MISSING_FILE_GENERATION as their actual generation value.
Adds support for truncating files after write operations, enabling proper
HTTP PUT semantics where the entire file content should be replaced.
Implementation:
- Added TOASTY_WRITE_TRUNCATE_AFTER flag (0x02) to ToastyFS API
- Updated file_tree_write() to accept truncate_after parameter:
* When true, sets file size to exactly offset+length
* Removes chunks beyond the new file size
* Marks removed chunks for garbage collection
- Metadata server extracts and passes truncate flag to file_tree_write()
- WAL replay passes false for truncate_after (already handled in original write)
- Web proxy PUT now uses both CREATE_IF_MISSING and TRUNCATE_AFTER flags
- Updated documentation in PROTOCOL.txt, DESIGN.txt, and web/DESIGN.txt
Benefits:
- Proper HTTP PUT semantics (replace entire file content)
- Efficient: single write operation truncates and updates file
- Works with both sync and async APIs
- Garbage collection automatically removes orphaned chunks
Per PROTOCOL.txt specification, WRITE operations must provide a valid
generation counter and cannot use expect_gen=0 (unlike other operations).
This ensures the client has retrieved the file's metadata and knows the
correct chunk size before writing.
Changes:
- Added FILETREE_BADGEN error code (-8)
- Modified file_tree_write() to reject expect_gen=0
- Added error message for FILETREE_BADGEN
Previously, file_size was calculated as num_chunks * chunk_size, which
incorrectly treated the file size as the full capacity of all allocated
chunks. The actual file size should be the offset of the last byte
written plus 1.
Changes:
- Added file_size field to File structure to track actual file extent
- Initialize file_size to 0 when creating new files
- Update file_size in file_tree_write() based on write offset + length
- Modified file_tree_read() to use file_size instead of num_chunks * chunk_size
- Updated serialization/deserialization to handle file_size field
This ensures that actual_bytes calculations correctly reflect the true
file size, not just the allocated chunk capacity.
Add functionality to determine the actual number of bytes read during
read operations, making it possible to detect when a read was truncated
because it went past the end of the file.
Changes:
- Modified file_tree_read() to calculate and return actual_bytes via
new output parameter
- Updated metadata server to send actual_bytes in READ_SUCCESS messages
- Added bytes_read field to ToastyResult structure
- Modified client to parse, store, and report actual bytes read
- Updated toasty_read() to return the actual number of bytes read
instead of always returning 0
- Fixed web server to use bytes_read field instead of non-existent
count field
This allows clients to distinguish between:
- Reading zeros because the file is sparse (has holes)
- Reading past the end of the file (truncated read)
Fixed multiple uninitialized memory issues detected by valgrind:
1. SHA256 struct size mismatch: Changed from 64 bytes to 32 bytes to
match the actual output of sha256() function (which produces 256
bits = 32 bytes).
2. File tree chunk allocation: Fixed loop that wasn't initializing
newly allocated chunks because num_chunks was updated before the
initialization loop. Now saves old_num_chunks first.
3. Metadata server process_client_write: Initialized results[i].num_addrs
to 0 before use to prevent reading uninitialized value.
4. Address struct initialization: Zero-initialize Address structs before
setting fields to ensure the entire union is initialized (union
contains either IPv4 or IPv6, leaving unused bytes uninitialized).
5. Fixed bug in IPv6 address creation: Changed incorrect is_ipv4=true
to is_ipv4=false for IPv6 addresses.
6. Fixed all_chunk_servers_holding_chunk: Function was incrementing
counter for all chunk servers instead of only those containing the
hash, causing access to uninitialized array elements.
All valgrind errors resolved: ERROR SUMMARY: 0 errors from 0 contexts
Claude
cozis