Per PROTOCOL.txt specification, WRITE operations must provide a valid
generation counter and cannot use expect_gen=0 (unlike other operations).
This ensures the client has retrieved the file's metadata and knows the
correct chunk size before writing.
Changes:
- Added FILETREE_BADGEN error code (-8)
- Modified file_tree_write() to reject expect_gen=0
- Added error message for FILETREE_BADGEN
Updated both client.c and metadata_server.c to match the protocol
specification in misc/PROTOCOL.txt:
Client-to-Server Messages:
- Added expect_gen field to DELETE, LIST, READ, and WRITE messages
- Removed old_hash and chunk_size from WRITE message chunks
Server-to-Client Responses:
- Added generation counter to CREATE_SUCCESS response
- Added generation counter to LIST_SUCCESS response
- Fixed LIST_SUCCESS field order (gen, truncated, item_count)
- Added generation counter to READ_SUCCESS response
- Added generation counter to WRITE_SUCCESS response
All changes compile successfully and pass valgrind memory checks.
Previously, file_size was calculated as num_chunks * chunk_size, which
incorrectly treated the file size as the full capacity of all allocated
chunks. The actual file size should be the offset of the last byte
written plus 1.
Changes:
- Added file_size field to File structure to track actual file extent
- Initialize file_size to 0 when creating new files
- Update file_size in file_tree_write() based on write offset + length
- Modified file_tree_read() to use file_size instead of num_chunks * chunk_size
- Updated serialization/deserialization to handle file_size field
This ensures that actual_bytes calculations correctly reflect the true
file size, not just the allocated chunk capacity.
Add functionality to determine the actual number of bytes read during
read operations, making it possible to detect when a read was truncated
because it went past the end of the file.
Changes:
- Modified file_tree_read() to calculate and return actual_bytes via
new output parameter
- Updated metadata server to send actual_bytes in READ_SUCCESS messages
- Added bytes_read field to ToastyResult structure
- Modified client to parse, store, and report actual bytes read
- Updated toasty_read() to return the actual number of bytes read
instead of always returning 0
- Fixed web server to use bytes_read field instead of non-existent
count field
This allows clients to distinguish between:
- Reading zeros because the file is sparse (has holes)
- Reading past the end of the file (truncated read)
- Added mock_lseek declaration and implementation for Linux
- Added mock_SetFilePointer declaration and implementation for Windows
- Added sys_lseek and sys_SetFilePointer macros for both BUILD_TEST and production modes
- Mocks follow existing pattern: validate descriptors, check type, forward to real functions
- Both mocks properly handle error cases and set errno/SetLastError
Add mock implementation for MoveFileExW Windows API function following
the existing pattern in the codebase. The mock forwards calls to the
real Windows API, allowing for future interception in the simulation
framework if needed.
CRITICAL BUG FIX: The previous Windows implementation had a fatal flaw
where it deleted the old WAL file before renaming, creating a window
where all data could be lost if the rename failed.
Previous (BROKEN) Windows code:
remove_file_or_dir(wal->file_path); // Delete old file
rename(...); // <-- If this fails, we've lost all data!
New implementation uses MoveFileExW with MOVEFILE_REPLACE_EXISTING:
- Windows: MoveFileExW(..., MOVEFILE_REPLACE_EXISTING) atomically
replaces the destination file, matching Unix semantics
- Unix/Linux: rename() continues to atomically replace as before
- Both platforms now have atomic file replacement with no data loss window
This ensures durability on both platforms - if the operation fails at
any point, we still have either the old file or the new file, never
losing all data.
This commit addresses several important robustness issues in the WAL
implementation:
1. Fixed file_path lifetime issue:
- The file_path argument to wal_open() may not have the same lifetime
as the WAL structure
- Now allocate and copy the path string to ensure WAL owns its data
- Properly free the allocated path in wal_close()
- Added error cleanup path in wal_open() to prevent memory leaks
2. Replaced static arrays with dynamic allocation in next_entry():
- Static arrays were not thread-safe and had limited lifetimes
- Now dynamically allocate path buffer and hash buffers using sys_malloc
- Caller must free allocated fields after using the entry
- Added proper error cleanup to free allocations on failure
- Updated wal_open() to free entry fields after processing each entry
3. Improved swap_file() for cross-platform compatibility:
- On Unix/Linux: rename() atomically replaces the destination file
- On Windows: rename() doesn't overwrite, so delete old file first
- Added platform-specific handling with #ifdef _WIN32
- Ensures WAL rotation works correctly on both platforms
4. Added system.h include for sys_malloc/sys_free definitions
These changes ensure proper memory management, prevent leaks, and make
the WAL implementation more robust across different platforms.
This commit completes the Write-Ahead Log (WAL) implementation for the
metadata server, including all missing functionality and error fixes:
File System Changes (src/file_system.c):
- Implemented file_set_offset() using lseek (Linux) and SetFilePointer (Windows)
- Implemented file_get_offset() to get current file position
WAL Header Changes (src/wal.h):
- Added file_tree pointer to WAL structure for snapshot serialization
- Added file_path to WAL structure for rotation operations
WAL Implementation (src/wal.c):
- Fixed handle assignment bug in wal_open (was using wal->handle before assignment)
- Fixed missing return statement in append_begin()
- Implemented serialize_callback for writing file tree snapshots to WAL
- Implemented deserialize_callback for reading file tree snapshots from WAL
- Implemented next_entry() to read WAL entries from file during recovery
- Implemented wal_append_write() to write file modifications to WAL
- Implemented swap_file() for complete WAL file rotation:
* Creates temporary file with new snapshot
* Writes WAL header and serialized file tree
* Atomically replaces old WAL file
* Resets entry counter for new rotation cycle
- Added WAL file initialization for newly created files
- Added helper functions: read_exact, read_u8, read_u16, read_u32, read_u64, write_u32
- Fixed typo in comment (Not -> Now)
The WAL now supports:
- Full crash recovery by replaying logged operations
- Automatic file rotation when entry limit is reached
- Atomic file replacement to ensure durability
- Proper file locking throughout rotation process
This commit adds tooling to measure how many branches the random simulation
reaches during execution, which helps understand code coverage and identify
untested code paths.
Changes:
- Add coverage build target to Makefile with --coverage flags
- Create measure_coverage.sh script to build, run, and report branch coverage
- Add signal handlers (SIGINT/SIGTERM) to main_test.c for clean shutdown
- Update clean target to remove coverage files (*.gcda, *.gcno)
The script runs the simulation for a specified duration (default 5 seconds),
then uses gcov to analyze which branches were executed. In a 3-second run,
the simulation reaches approximately 32% of all branches (781/2431).
Usage:
./measure_coverage.sh [duration_in_seconds]
Example output shows per-file and total branch coverage statistics.
Fixed compilation errors:
- Added <dirent.h> include to file_system.h for DIR type on Linux
- Replaced xxx placeholder in chunk_server.c with actual chunks_dir path
using state->store.path
Fixed several bugs in the DirectoryScanner implementation:
Windows fixes:
- Fixed snprintf call missing buffer size parameter
- Fixed FindFirstFileA to use pattern (with wildcard) instead of path
- Changed sys_GetLastError() to GetLastError()
- Added scanner->first = false after using first result
- Removed TODO comment from cFileName usage (it's the correct field)
Linux fixes:
- Fixed typo: sys_reddir → sys_readdir
- Added missing 'done' field to DirectoryScanner struct
- Initialize scanner->done = false on successful opendir
- Fixed return value when scanner->done (should return 1, not -1)
Added mock implementations for file/directory search operations to
support both Windows and Linux in the simulation environment.
Windows mocks:
- FindFirstFileA: Wraps real search handle in descriptor
- FindNextFileA: Forwards to real API
- FindClose: Properly cleans up search handles
Linux mocks:
- opendir: Wraps real DIR* handle in descriptor
- readdir: Forwards to real API
- closedir: Properly cleans up directory handles
Common changes:
- Added DESC_DIRECTORY descriptor type to track directory handles
- Added real_d field to Descriptor for directory handles (HANDLE on Windows, DIR* on Linux)
- Updated close_desc to handle directory cleanup on both platforms
- Added sys_* macros for both BUILD_TEST and non-BUILD_TEST modes
- Added <dirent.h> include for Linux directory operations
Claude
cozis