Introduce a Makefile that builds libtoastyfs.a and libtoastyfs.so from the
client-side sources (client, tcp, byte_queue, message, basic) so external
programs can link against the ToastyFS client API in include/toastyfs.h.
Add cluster.sh, a convenience script that manages a local 3-node cluster
by spawning toastyfs server processes on ports 8081-8083. Sub-commands:
up, down, status, logs, and "run <source.c>" which builds the library,
compiles the source against it, starts the cluster, runs the binary, and
tears everything down.
Add examples/example.c demonstrating synchronous PUT, GET, DELETE, and a
NOT_FOUND verification.
Fix a stack overflow in the MAIN_SERVER entry point: ServerState is ~40 MB
(MetaStore holds 4096 ObjectMeta entries) and was declared on the stack.
Heap-allocate it instead.
https://claude.ai/code/session_019DVhmc25jfzcHnmdNxzib2
- Move the CEIL macro from config.h to basic.h alongside MIN/MAX, and
use it in deadline_to_timeout for the rounding-up division
- Remove the heartbeat reset in complete_view_change_and_become_primary
since it was not needed — heartbeat is already set when the node
enters STATUS_CHANGE_VIEW, which is close enough in time
https://claude.ai/code/session_0184gHjra7fsmSPZ4kppaGhC
The client library had several issues preventing operations from completing
in the simulation:
- toastyfs_register_events discarded the computed timeout instead of
returning it to callers, so clients never woke up on schedule
- random_client_tick ignored the timeout parameter entirely
- deadline_to_timeout truncated sub-millisecond values to 0 instead of
rounding up, causing busy-wait loops
- The server didn't reset the heartbeat timer after completing a view
change, so followers would immediately timeout again
- The client's DELETE handler didn't handle META_RESULT_NOT_FOUND,
causing an assertion failure when deleting non-existent keys
- Most critically, messages sent before TCP connections were established
were silently dropped with no retry mechanism — added timeout-based
retry logic in toastyfs_process_events
https://claude.ai/code/session_0184gHjra7fsmSPZ4kppaGhC
Add logging infrastructure to client.c mirroring the server's node_log
pattern, with trace lines for: init, async put/get/delete, begin
transfers, replay requests, all received message types (store ack, fetch
chunk response, redirect, reply, get blob response), disconnects, and
unexpected messages.
Add result/operation trace lines to random_client.c and fix a bug where
the random client never started its first operation (the condition
`result.type != TOASTYFS_RESULT_VOID` was always false on the first
tick since no operation was pending).
https://claude.ai/code/session_0184gHjra7fsmSPZ4kppaGhC
- Add #warning to remind about replacing compute_chunk_hash with proper SHA256
- Define MAX_TRANSFERS and CEIL macros in config.h and use them
- Add TODO noting server selection formula is temporary
- Add NOTE about sticky error pattern in async functions
- Add TODO for INVALID_TIME error handling at all get_current_time call sites
- Add TODO about determining timeout based on pending operation status
https://claude.ai/code/session_01TkNavR4qyZcTuXMPmrtWUJ
- Add REPLICATION_FACTOR and CHUNK_SIZE constants to config.h
- Add proper error codes (REJECTED, FULL, NOT_FOUND, TRANSFER_FAILED, etc.) to toastyfs.h
- Implement compute_chunk_hash() for deterministic chunk hashing
- Implement replay_request() to resend operations after leader redirect
- Implement send_message_to_server_ex() for sending messages with trailing data
- Implement choose_store_locations_for_chunk() to pick REPLICATION_FACTOR servers
- Fix begin_transfers() to handle both StoreChunk (PUT) and FetchChunk (GET)
- Complete toastyfs_async_put(): chunk splitting, hashing, data copy, metadata setup
- Complete toastyfs_async_get()/delete(): key setup, step/time initialization
- Fix get_result(): use actual transfer sizes instead of fixed chunk_size, free transfer data
- Replace all xxx placeholders and TOASTYFS_ERROR_XXX with proper values
- Add chunk_sizes[] tracking and put_data lifetime management
- Implement random_client.c: rng(), choose_random_oper(), proper API call arguments
https://claude.ai/code/session_01TkNavR4qyZcTuXMPmrtWUJ
The node_log macro used a 256-byte buffer for formatting log details,
but bucket names (64B) and keys (512B) can together require up to ~606
bytes, causing gcc to emit format-truncation warnings. Increase the
buffer to 1024 bytes to accommodate the worst case.
https://claude.ai/code/session_01HXKM6UjE3G4A29zTc7gFxo
- Rename phase_time to step_time for consistency with Step enum
- Unify blob_size and file_size into single blob_size field
- Merge toastyfs_alloc into toastyfs_init, returning malloced ToastyFS*
- Replace (MessageHeader*)&msg casts with &msg.base
- Replace !resp.size with explicit resp.size == 0
https://claude.ai/code/session_018MRQTdAZoBCXJZgdP4U6V6
1. Fix client not reading all server addresses in READ response
The metadata server sends address info for ALL chunk servers holding
a chunk, but the client only read info for ONE server. This caused
message parsing to become misaligned for subsequent chunks.
Added a loop to iterate through all num_servers and read each
server's address list (using the first one, skipping the rest).
2. Fix tag value conflict between metadata and chunk server responses
TAG_RETRIEVE_METADATA_FOR_READ was 1, which conflicted with chunk
download range indices (0, 1, 2, 3...). When the second chunk
download response came with range_idx=1, it was incorrectly treated
as a metadata response.
Changed TAG values from positive to negative integers since range
indices are always non-negative.
Also updated the test to properly exit on success instead of assert(0).
https://claude.ai/code/session_01QYDQCNiUj1cjrsS6qRgkvd
The upload tag is computed as TAG_UPLOAD_CHUNK_MIN + upload_index, and the
assertion tag <= TAG_UPLOAD_CHUNK_MAX was failing when the number of uploads
exceeded 1000. This can happen with small chunk sizes (e.g., 1 byte) and
large writes, where each chunk needs multiple upload operations for
replication.
Increase TAG_UPLOAD_CHUNK_MAX from 2000 to 100000 to accommodate up to 99000
concurrent upload operations per write.
When responding to a full chunk download request (full == 1), the code
was writing uninitialized target_len to the response message instead of
the actual data length. This caused chunk-to-chunk downloads to fail,
preventing proper replication and blocking coverage of download-related
code paths.
Two changes to improve code coverage in deterministic simulation testing:
1. Add 3 chunk servers (was 1) to match the replication factor
- This enables testing of chunk replication and chunk-to-chunk
server communication code paths in chunk_server.c
- Covers: process_chunk_server_download_*, start_download,
download_targets_* functions
2. Fix Quakey bug where events targeting a closed descriptor caused
assertion failures
- Added remove_events_targeting_desc() to clean up DISCONNECT and
DATA events when a descriptor is freed
- Modified desc_free() to accept Sim pointer and call cleanup
- This was exposed by running multiple chunk servers which create
more connection activity
Three related issues were causing the simulation to get stuck:
1. Network events (connect, disconnect, send_data) were scheduled with
relative times instead of absolute times, causing events to be
scheduled in the past once simulation time exceeded 10-100ms.
2. When DATA events couldn't be fully consumed (receiver buffer full),
they stayed at their original time, preventing time from advancing.
Now they are rescheduled to current_time + 10ms.
3. When poll_timeout was 0, WAKEUP events were created at current_time,
causing an infinite loop where time never advanced. Now timeout=0
sets timedout immediately without creating an event.
Initialize *gen to NO_GENERATION at the start of file_tree_read so that
error paths (FILETREE_BADPATH, FILETREE_NOENT, FILETREE_ISDIR) have a
well-defined value. Previously, when file_tree_read failed, gen was
uninitialized, causing the assertion in process_client_read to fire.
The assertion checking gen != NO_GENERATION before the error check was
removed since:
1. On error paths, gen is not used
2. On success paths, the assertion at line 482 already validates gen
Simplifies the script to automatically compile, run (60s), and generate
the coverage report in a single invocation. Removes the two-step --report
flag approach and adds quieter output with clear step markers.
Adds build_coverage.sh script that:
- Builds the simulation with GCC branch coverage instrumentation
- Supports --report flag to run simulation and generate lcov HTML report
- Uses 60 second timeout with SIGINT for graceful shutdown
Also adds SIGINT handler to simulation to exit cleanly and flush coverage data.
Implements a new special generation counter value (UINT64_MAX) that
allows clients to assert that a file should NOT exist when performing
operations.
Primary use case: Atomic "create-if-not-exists" operations
When combined with TOASTY_WRITE_CREATE_IF_MISSING flag:
- File doesn't exist → server creates file and writes (SUCCESS)
- File exists → gen_match fails → BADGEN error (prevents overwrite)
This enables race-condition-free file creation where you want to
create a NEW file or fail if it already exists.
Key changes:
- Added MISSING_FILE_GENERATION constant to file_tree.h
- Updated gen_match() to return false when checking existing entities
against MISSING_FILE_GENERATION
- Modified file_tree_delete_entity() to succeed (no-op) when file is
missing and MISSING_FILE_GENERATION is specified (idempotent delete)
- Modified file_tree_write() to validate against the new value
- Added documentation in metadata_server.c explaining how
MISSING_FILE_GENERATION works WITH CREATE_IF_MISSING
- Updated protocol documentation in PROTOCOL.txt
- Added comprehensive feature documentation in MISSING_FILE_GENERATION.md
Additional use cases:
- Idempotent delete operations (delete only if not already gone)
- Detecting unexpected file creation in validation/testing scenarios
The implementation is backward compatible and type-safe. Files are
never assigned MISSING_FILE_GENERATION as their actual generation value.
Adds support for truncating files after write operations, enabling proper
HTTP PUT semantics where the entire file content should be replaced.
Implementation:
- Added TOASTY_WRITE_TRUNCATE_AFTER flag (0x02) to ToastyFS API
- Updated file_tree_write() to accept truncate_after parameter:
* When true, sets file size to exactly offset+length
* Removes chunks beyond the new file size
* Marks removed chunks for garbage collection
- Metadata server extracts and passes truncate flag to file_tree_write()
- WAL replay passes false for truncate_after (already handled in original write)
- Web proxy PUT now uses both CREATE_IF_MISSING and TRUNCATE_AFTER flags
- Updated documentation in PROTOCOL.txt, DESIGN.txt, and web/DESIGN.txt
Benefits:
- Proper HTTP PUT semantics (replace entire file content)
- Efficient: single write operation truncates and updates file
- Works with both sync and async APIs
- Garbage collection automatically removes orphaned chunks
Adds support for automatically creating files when write operations target
non-existent files. This is essential for REST PUT operations which should
be able to create new resources.
Implementation:
- Added TOASTY_WRITE_CREATE_IF_MISSING flag to ToastyFS API header
- Updated toasty_write() and toasty_begin_write() to accept flags parameter
- Client sends write flags in MESSAGE_TYPE_WRITE message to metadata server
- Metadata server parses flags and handles file auto-creation atomically:
* When write fails with FILETREE_NOENT and flag is set
* Logs creation to WAL for crash consistency
* Creates file with default 4096-byte chunk size
* Retries write with newly created file's generation tag
- Updated web proxy PUT handler to use TOASTY_WRITE_CREATE_IF_MISSING
- Updated examples and simulation client for new API signature
Benefits:
- Works for both sync and async APIs
- Single round trip (atomic server-side operation)
- Prevents race conditions
- Proper WAL logging ensures crash consistency
Per PROTOCOL.txt specification, WRITE operations must provide a valid
generation counter and cannot use expect_gen=0 (unlike other operations).
This ensures the client has retrieved the file's metadata and knows the
correct chunk size before writing.
Changes:
- Added FILETREE_BADGEN error code (-8)
- Modified file_tree_write() to reject expect_gen=0
- Added error message for FILETREE_BADGEN
Updated both client.c and metadata_server.c to match the protocol
specification in misc/PROTOCOL.txt:
Client-to-Server Messages:
- Added expect_gen field to DELETE, LIST, READ, and WRITE messages
- Removed old_hash and chunk_size from WRITE message chunks
Server-to-Client Responses:
- Added generation counter to CREATE_SUCCESS response
- Added generation counter to LIST_SUCCESS response
- Fixed LIST_SUCCESS field order (gen, truncated, item_count)
- Added generation counter to READ_SUCCESS response
- Added generation counter to WRITE_SUCCESS response
All changes compile successfully and pass valgrind memory checks.
Fixed two bugs in the HTTP server that could cause hangs:
1. web/chttp.c:4060 - Fixed incorrect error buffer check
- Was checking output buffer for errors after a read operation
- Should check input buffer instead
- This could cause the server to miss read errors and not close
bad connections properly
2. web/main.c:334 - Added loop to process multiple queued requests
- Previously only processed one request per event loop iteration
- Now processes all queued requests before returning to poll()
- This prevents requests from being stuck in the queue when the
socket is in ESTABLISHED_READY state
These fixes improve the server's ability to handle multiple requests
on Keep-Alive connections.
- Add web server integration to cluster_demo.sh script
- Configure web server port (8090)
- Update build_if_needed() to check for toastyfs_web binary
- Add get_web_binary() function
- Start web server after chunk servers
- Update status display to show web server
- Update help text to show HTTP interface
- Fix bug in web/main.c where --local-port incorrectly set
upstream_port instead of local_port
The cluster demo now starts a full cluster with:
- Metadata server on port 8080 (ToastyFS protocol)
- N chunk servers starting from port 8081
- Web server on port 8090 (HTTP interface)
This script provides a convenient way to spawn and manage a ToastyFS cluster
for demo and testing purposes. Features include:
- Start a cluster with configurable number of chunk servers
- Automatic building if binary is not present
- Process management with PID tracking
- Status checking for all cluster nodes
- Easy cleanup of all cluster processes
- Separate log files for each server component
- Colorized output for better readability
Usage:
./scripts/cluster_demo.sh start [num_servers] - Start cluster
./scripts/cluster_demo.sh stop - Stop cluster
./scripts/cluster_demo.sh status - Show status
./scripts/cluster_demo.sh clean - Clean data/logs
Previously, file_size was calculated as num_chunks * chunk_size, which
incorrectly treated the file size as the full capacity of all allocated
chunks. The actual file size should be the offset of the last byte
written plus 1.
Changes:
- Added file_size field to File structure to track actual file extent
- Initialize file_size to 0 when creating new files
- Update file_size in file_tree_write() based on write offset + length
- Modified file_tree_read() to use file_size instead of num_chunks * chunk_size
- Updated serialization/deserialization to handle file_size field
This ensures that actual_bytes calculations correctly reflect the true
file size, not just the allocated chunk capacity.
Add functionality to determine the actual number of bytes read during
read operations, making it possible to detect when a read was truncated
because it went past the end of the file.
Changes:
- Modified file_tree_read() to calculate and return actual_bytes via
new output parameter
- Updated metadata server to send actual_bytes in READ_SUCCESS messages
- Added bytes_read field to ToastyResult structure
- Modified client to parse, store, and report actual bytes read
- Updated toasty_read() to return the actual number of bytes read
instead of always returning 0
- Fixed web server to use bytes_read field instead of non-existent
count field
This allows clients to distinguish between:
- Reading zeros because the file is sparse (has holes)
- Reading past the end of the file (truncated read)
- Added mock_lseek declaration and implementation for Linux
- Added mock_SetFilePointer declaration and implementation for Windows
- Added sys_lseek and sys_SetFilePointer macros for both BUILD_TEST and production modes
- Mocks follow existing pattern: validate descriptors, check type, forward to real functions
- Both mocks properly handle error cases and set errno/SetLastError
Add mock implementation for MoveFileExW Windows API function following
the existing pattern in the codebase. The mock forwards calls to the
real Windows API, allowing for future interception in the simulation
framework if needed.
CRITICAL BUG FIX: The previous Windows implementation had a fatal flaw
where it deleted the old WAL file before renaming, creating a window
where all data could be lost if the rename failed.
Previous (BROKEN) Windows code:
remove_file_or_dir(wal->file_path); // Delete old file
rename(...); // <-- If this fails, we've lost all data!
New implementation uses MoveFileExW with MOVEFILE_REPLACE_EXISTING:
- Windows: MoveFileExW(..., MOVEFILE_REPLACE_EXISTING) atomically
replaces the destination file, matching Unix semantics
- Unix/Linux: rename() continues to atomically replace as before
- Both platforms now have atomic file replacement with no data loss window
This ensures durability on both platforms - if the operation fails at
any point, we still have either the old file or the new file, never
losing all data.
This commit addresses several important robustness issues in the WAL
implementation:
1. Fixed file_path lifetime issue:
- The file_path argument to wal_open() may not have the same lifetime
as the WAL structure
- Now allocate and copy the path string to ensure WAL owns its data
- Properly free the allocated path in wal_close()
- Added error cleanup path in wal_open() to prevent memory leaks
2. Replaced static arrays with dynamic allocation in next_entry():
- Static arrays were not thread-safe and had limited lifetimes
- Now dynamically allocate path buffer and hash buffers using sys_malloc
- Caller must free allocated fields after using the entry
- Added proper error cleanup to free allocations on failure
- Updated wal_open() to free entry fields after processing each entry
3. Improved swap_file() for cross-platform compatibility:
- On Unix/Linux: rename() atomically replaces the destination file
- On Windows: rename() doesn't overwrite, so delete old file first
- Added platform-specific handling with #ifdef _WIN32
- Ensures WAL rotation works correctly on both platforms
4. Added system.h include for sys_malloc/sys_free definitions
These changes ensure proper memory management, prevent leaks, and make
the WAL implementation more robust across different platforms.
This commit completes the Write-Ahead Log (WAL) implementation for the
metadata server, including all missing functionality and error fixes:
File System Changes (src/file_system.c):
- Implemented file_set_offset() using lseek (Linux) and SetFilePointer (Windows)
- Implemented file_get_offset() to get current file position
WAL Header Changes (src/wal.h):
- Added file_tree pointer to WAL structure for snapshot serialization
- Added file_path to WAL structure for rotation operations
WAL Implementation (src/wal.c):
- Fixed handle assignment bug in wal_open (was using wal->handle before assignment)
- Fixed missing return statement in append_begin()
- Implemented serialize_callback for writing file tree snapshots to WAL
- Implemented deserialize_callback for reading file tree snapshots from WAL
- Implemented next_entry() to read WAL entries from file during recovery
- Implemented wal_append_write() to write file modifications to WAL
- Implemented swap_file() for complete WAL file rotation:
* Creates temporary file with new snapshot
* Writes WAL header and serialized file tree
* Atomically replaces old WAL file
* Resets entry counter for new rotation cycle
- Added WAL file initialization for newly created files
- Added helper functions: read_exact, read_u8, read_u16, read_u32, read_u64, write_u32
- Fixed typo in comment (Not -> Now)
The WAL now supports:
- Full crash recovery by replaying logged operations
- Automatic file rotation when entry limit is reached
- Atomic file replacement to ensure durability
- Proper file locking throughout rotation process
Use the standard lcov tools instead of custom HTML generation.
Changes:
- Completely rewrite generate_coverage_html.sh to use lcov
- Install lcov if not available
- Use lcov to capture coverage data from .gcda files
- Use genhtml to create HTML reports with branch coverage
- Add scripts/README.md with usage instructions
Benefits:
- Professional HTML reports with standard lcov styling
- Better branch coverage visualization
- Sortable tables by line/function/branch coverage
- Source code view with execution counts
- Much simpler script (30 lines vs 200+ lines)
The reports now show:
- 32.1% branch coverage (781/2431 branches)
- 43.5% line coverage
- 61.3% function coverage
Remove all custom styling and CSS to use only plain HTML with
browser default rendering.
Changes:
- Remove all <style> tags and CSS
- Remove CSS classes from generated HTML
- Simplify source code display to plain text format
- Use simple line number: execution count: source code format
- Browser will render with its own default styles
The reports are now pure HTML with no styling at all.
Remove fancy styling (rounded corners, shadows, progress bars, colors)
and use simple, clean HTML with basic tables and minimal CSS.
Changes:
- Use simple sans-serif font instead of Segoe UI
- Remove container divs, shadows, and rounded corners
- Replace progress bars with plain percentage text
- Simplify color scheme (light green/red for branch coverage)
- Remove emoji and decorative elements
- Clean up table styling to basic borders
The report is now much cleaner and follows standard HTML conventions.
This fixes issues when running scripts in WSL where CRLF line endings
cause 'No such file or directory' errors. Also add a fix_line_endings.sh
script for users who already have the wrong line endings.
Reorganize coverage tooling for better project structure and easier usage.
Changes:
- Move measure_coverage.sh and generate_coverage_html.sh to scripts/
- Add Makefile targets:
- make coverage-report: Generate text coverage summary (5s simulation)
- make coverage-html: Generate HTML coverage report (5s simulation)
- Update measure_coverage.sh to find generate_coverage_html.sh using relative path
Usage:
make coverage-report # Quick text summary
make coverage-html # Full HTML report with branch details
The HTML report provides interactive visualization of which branches
were taken during simulation execution, with color-coded source views.
Enhance the branch coverage measurement tool with HTML report generation
that shows exactly which branches were taken during simulation execution.
Changes:
- Add generate_coverage_html.sh script to create interactive HTML reports
- Update measure_coverage.sh with --html flag to generate reports
- Add *.gcov, *.gcda, *.gcno to .gitignore
- HTML reports show:
- Overall coverage summary with visual progress bars
- Per-file coverage breakdown with clickable links
- Source code view with color-coded branch coverage
- Green highlight: branches taken
- Red highlight: branches not taken
- Branch execution counts and percentages
Usage:
./measure_coverage.sh [duration] --html
The HTML report is generated in coverage_report/index.html and can be
viewed in any web browser. Each source file links to a detailed view
showing which specific branches were executed.
This commit adds tooling to measure how many branches the random simulation
reaches during execution, which helps understand code coverage and identify
untested code paths.
Changes:
- Add coverage build target to Makefile with --coverage flags
- Create measure_coverage.sh script to build, run, and report branch coverage
- Add signal handlers (SIGINT/SIGTERM) to main_test.c for clean shutdown
- Update clean target to remove coverage files (*.gcda, *.gcno)
The script runs the simulation for a specified duration (default 5 seconds),
then uses gcov to analyze which branches were executed. In a 3-second run,
the simulation reaches approximately 32% of all branches (781/2431).
Usage:
./measure_coverage.sh [duration_in_seconds]
Example output shows per-file and total branch coverage statistics.
Fixed compilation errors:
- Added <dirent.h> include to file_system.h for DIR type on Linux
- Replaced xxx placeholder in chunk_server.c with actual chunks_dir path
using state->store.path
Fixed several bugs in the DirectoryScanner implementation:
Windows fixes:
- Fixed snprintf call missing buffer size parameter
- Fixed FindFirstFileA to use pattern (with wildcard) instead of path
- Changed sys_GetLastError() to GetLastError()
- Added scanner->first = false after using first result
- Removed TODO comment from cFileName usage (it's the correct field)
Linux fixes:
- Fixed typo: sys_reddir → sys_readdir
- Added missing 'done' field to DirectoryScanner struct
- Initialize scanner->done = false on successful opendir
- Fixed return value when scanner->done (should return 1, not -1)
Added mock implementations for file/directory search operations to
support both Windows and Linux in the simulation environment.
Windows mocks:
- FindFirstFileA: Wraps real search handle in descriptor
- FindNextFileA: Forwards to real API
- FindClose: Properly cleans up search handles
Linux mocks:
- opendir: Wraps real DIR* handle in descriptor
- readdir: Forwards to real API
- closedir: Properly cleans up directory handles
Common changes:
- Added DESC_DIRECTORY descriptor type to track directory handles
- Added real_d field to Descriptor for directory handles (HANDLE on Windows, DIR* on Linux)
- Updated close_desc to handle directory cleanup on both platforms
- Added sys_* macros for both BUILD_TEST and non-BUILD_TEST modes
- Added <dirent.h> include for Linux directory operations
Fixed critical bugs in the DOWNLOAD_LOCATIONS message exchange between
metadata server and chunk server:
Metadata server changes:
- Fixed bug using wrong index (j instead of holders[j]) when writing
chunk server addresses
- Added missing hash field to the message (was only read, never written)
- Added trace output for STATE_UPDATE_ERROR messages
- Pre-read all missing chunks before building response
Chunk server changes:
- Updated message parser to match metadata server's message format
- Changed from group-based format to per-chunk format
- Updated type sizes (uint32_t for counts instead of uint8_t/uint16_t)
- Correctly accumulates addresses from all holders before adding to
pending download list
The message now follows this structure:
- num_missing (uint32_t)
- For each missing chunk:
- num_holders (uint32_t)
- For each holder: server addresses (num_ipv4, num_ipv6, addresses)
- hash (SHA256)
Tested successfully with mousefs_random_test under valgrind with no
memory errors detected.
- Add CLIENT_CFILES and CLIENT_OFILES variables to Makefile
- Add rules to build object files and create static library
- Include libmousefs_client.a in the 'all' target
- Update clean target to remove library and object files
- Update .gitignore to ignore *.o and *.a build artifacts
The library includes: client.c, basic.c, tcp.c, message.c
- Fix inverted binary_read error checks in process_metadata_server_download_locations
- Fix array indexing bugs (use correct loop variables j and k instead of i)
- Fix IPv4 port array type from uint8_t to uint16_t
- Fix reader cursor advancement in process_client_upload_chunk
- Enable periodic download retry mechanism by calling start_download_if_necessary
Fix how chunk servers handle state updates from the metadata server
based on the design in DESIGN.txt. The implementation now:
1. Checks if chunks in the add_list exist in the chunk store and
collects any missing chunks
2. Unmarks chunks in the add_list that were previously marked for removal
3. Marks chunks in the rem_list for removal with timestamps
4. Responds with SUCCESS if all chunks are present, or ERROR with the
list of missing chunks
Added:
- RemovalList data structure to track chunks marked for deletion with
timestamps
- chunk_store_exists() to check if a chunk file exists
- Helper functions for removal list management (init, free, add, remove, find)
This implements the garbage collection mechanism described in DESIGN.txt
where chunks are marked for removal and can be unmarked if they appear
in the add_list again.
This implements the periodic STATE_UPDATE mechanism described in DESIGN.txt
lines 80-99, where the metadata server periodically sends synchronization
messages to chunk servers.
Changes:
1. Added timing fields to ChunkServerPeer:
- last_sync_time: tracks when STATE_UPDATE was last sent
- last_response_time: tracks when chunk server last responded
2. Added health check configuration constants (config.h):
- HEALTH_CHECK_INTERVAL: 30 seconds between STATE_UPDATE messages
- HEALTH_CHECK_TIMEOUT: 90 seconds before marking server as unhealthy
3. Implemented send_state_update() function:
- Serializes and sends add_list and rem_list to chunk servers
- Updates last_sync_time when messages are sent
4. Added message handlers for chunk server responses:
- process_chunk_server_state_update_success(): merges add_list into old_list,
clears pending lists, updates last_response_time
- process_chunk_server_state_update_error(): handles missing chunks,
updates last_response_time, logs errors
5. Implemented periodic timer logic in metadata_server_step():
- Calculates when next STATE_UPDATE should be sent to each chunk server
- Sends updates when HEALTH_CHECK_INTERVAL has elapsed
- Logs warnings when chunk servers haven't responded within HEALTH_CHECK_TIMEOUT
- Sets appropriate poll() timeout to wake up for next health check
The implementation ensures that:
- Chunk servers receive regular synchronization messages with add_list/rem_list
- The metadata server tracks chunk server health based on response times
- STATE_UPDATE messages are sent periodically (every 30 seconds by default)
- Failed/missing chunks are detected and logged for recovery
This completes the health check mechanism described in the architecture design,
enabling the metadata server to maintain accurate state with chunk servers and
detect unhealthy servers.
Changes:
- Added is_nonblocking flag to Descriptor structure in system.c
- Implemented mock_fcntl (Linux) and mock_ioctlsocket (Windows) to handle setting sockets as non-blocking
- Set all sockets as non-blocking in tcp.c:
- Listen sockets in create_listen_socket()
- Accepted sockets in tcp_translate_events()
- Connecting sockets in tcp_connect()
- Added checks in mock_accept(), mock_recv(), and mock_send() to abort the simulation if a socket would block but is not configured as non-blocking
- This ensures proper non-blocking socket configuration and helps catch blocking socket errors during simulation
- Removed simulated_time struct timespec variable
- Updated mock_QueryPerformanceCounter to convert current_time (nanoseconds) to performance counter ticks
- Updated mock_clock_gettime to convert current_time (nanoseconds) to timespec format
- All time tracking now uses the unified current_time variable
Fixed two critical bugs in the TinyDFS client code:
1. Added bounds checking in alloc_operation() to prevent buffer overflow
when searching for a free operation slot. The while loop could go past
the end of the operations array, causing undefined behavior.
2. Fixed all tinydfs_submit_* functions to return the operation index
(opidx) instead of 0 on success. This was causing all operations to
be tracked with index 0, leading to type mismatches between operation
types and result types, which triggered the assertion:
"Assertion `pending.type == PENDING_OPERATION_DELETE' failed"
The test now runs successfully past the original assertion point.
Fixed multiple uninitialized memory issues detected by valgrind:
1. SHA256 struct size mismatch: Changed from 64 bytes to 32 bytes to
match the actual output of sha256() function (which produces 256
bits = 32 bytes).
2. File tree chunk allocation: Fixed loop that wasn't initializing
newly allocated chunks because num_chunks was updated before the
initialization loop. Now saves old_num_chunks first.
3. Metadata server process_client_write: Initialized results[i].num_addrs
to 0 before use to prevent reading uninitialized value.
4. Address struct initialization: Zero-initialize Address structs before
setting fields to ensure the entire union is initialized (union
contains either IPv4 or IPv6, leaving unused bytes uninitialized).
5. Fixed bug in IPv6 address creation: Changed incorrect is_ipv4=true
to is_ipv4=false for IPv6 addresses.
6. Fixed all_chunk_servers_holding_chunk: Function was incrementing
counter for all chunk servers instead of only those containing the
hash, causing access to uninitialized array elements.
All valgrind errors resolved: ERROR SUMMARY: 0 errors from 0 contexts
The RequestQueue for chunk servers should be initialized when the
connection is actually established in get_chunk_server_connection(),
not ahead of time during tinydfs_init().
The metadata server RequestQueue initialization remains in tinydfs_init()
since the metadata server connection is established during initialization.
Changes:
- Removed premature RequestQueue initialization for chunk servers
- Added clarifying comments about initialization timing
- RequestQueue is initialized in get_chunk_server_connection() (line 227)
This commit resolves all Valgrind errors reported in the test run:
1. Fixed uninitialized 'removed' array in tcp_translate_events (tcp.c:179)
- Added memset to initialize the array to zero before use
2. Fixed uninitialized RequestQueue in client initialization (client.c)
- Added initialization of metadata_server.reqs and chunk_servers[].reqs
- Added forward declaration of request_queue_init function
3. Fixed uninitialized descriptor generation field (system.c:324)
- Initialize generation to 0 when creating process descriptors
- Prevents uninitialized value errors in handle_to_desc
4. Fixed 34-byte memory leak in tcp_context_free (tcp.c:91-103)
- Added cleanup of connection byte queues when freeing TCP context
- Frees both input and output ByteQueues for all connections
All Valgrind errors have been resolved:
- ERROR SUMMARY: 0 errors from 0 contexts
- All heap blocks freed - no leaks possible
Simplified conditional compilation by wrapping the entire system.c file
with a single #ifdef BUILD_TEST guard, removing all nested conditionals
for simulation client code.
**Rationale:**
The entire system.c file contains simulation framework code (mock functions,
Process structure, spawn_simulated_process, update_simulation, etc.) that
is only used when BUILD_TEST is defined. Having nested #ifdef BUILD_TEST
conditionals within this file made no sense.
**Changes:**
**system.c:**
- Added #ifdef BUILD_TEST at the very beginning of the file (line 1)
- Added #endif // BUILD_TEST at the very end of the file (line 1486)
- Removed #ifdef BUILD_TEST around simulation_client.h include
- Removed #ifdef BUILD_TEST around PROCESS_TYPE_CLIENT enum value
- Removed #ifdef BUILD_TEST around SimulationClient union member
- Removed #ifdef BUILD_TEST around is_client() function
- Removed #ifdef BUILD_TEST around client detection in spawn_simulated_process()
- Removed #ifdef BUILD_TEST around PROCESS_TYPE_CLIENT cases in:
- spawn_simulated_process() init switch
- free_process() cleanup switch
- update_simulation() step switch
**Result:**
- Much cleaner code with single top-level BUILD_TEST guard
- No nested conditionals cluttering the code
- All three build targets still compile successfully:
- tinydfs_server.out (excludes system.c entirely)
- example_client.out (excludes system.c entirely)
- tinydfs_test.out (includes all of system.c)
- Simulation runs correctly with client support
The file structure now correctly reflects that system.c is purely test
infrastructure and not used in production builds.
Fixed uninitialized timeout variables in metadata_server_main() and
chunk_server_main() functions by setting default value to -1.
**Changes:**
- metadata_server_main(): Changed `int timeout;` to `int timeout = -1;`
- chunk_server_main(): Changed `int timeout;` to `int timeout = -1;`
This ensures the timeout parameter has a defined default value before
being passed to init functions, which is important for predictable
behavior even though the init functions set the value themselves.
All build targets (tinydfs_server.out, example_client.out,
tinydfs_test.out) compile and run successfully.
Fixed build errors in tinydfs_server and example_client targets caused
by missing timeout parameter and simulation client references.
**Changes:**
**main_server.c:**
- Added int timeout variable declaration in metadata_server_main()
- Added int timeout variable declaration in chunk_server_main()
- Pass &timeout to metadata_server_init() call
- Pass &timeout to metadata_server_step() call
- Pass &timeout to chunk_server_init() call
- Pass &timeout to chunk_server_step() call
**system.c:**
- Guarded simulation_client.h include with #ifdef BUILD_TEST
- Guarded PROCESS_TYPE_CLIENT enum value with #ifdef BUILD_TEST
- Guarded SimulationClient member in Process union with #ifdef BUILD_TEST
- Guarded is_client() function with #ifdef BUILD_TEST
- Guarded client detection logic in spawn_simulated_process()
- Guarded PROCESS_TYPE_CLIENT cases in all switch statements:
- spawn_simulated_process() init switch
- free_process() cleanup switch
- update_simulation() step switch
**Result:**
- tinydfs_server.out now compiles successfully
- example_client.out now compiles successfully
- tinydfs_test.out continues to work correctly
- Simulation client code only included when BUILD_TEST is defined
- All three build targets pass compilation with only pre-existing warnings
The simulation_client code is now properly isolated to test builds only,
preventing linker errors when building the real server and example client.
Unified the function signatures across all process types (metadata server,
chunk server, and simulation client) to accept an int *timeout parameter.
This provides consistency in the simulation framework and allows all
process types to control their wakeup times.
**Changes:**
**Headers (metadata_server.h, chunk_server.h):**
- Updated metadata_server_init() to accept int *timeout parameter
- Updated metadata_server_step() to accept int *timeout parameter
- Updated chunk_server_init() to accept int *timeout parameter
- Updated chunk_server_step() to accept int *timeout parameter
**Implementations (metadata_server.c, chunk_server.c):**
- Modified init functions to set *timeout = -1 (no timeout needed)
- Modified step functions to set *timeout = -1 (no timeout needed)
- Both server types currently don't require periodic wakeups, but the
parameter allows for future timeout-based behavior
**System Integration (system.c):**
- Updated spawn_simulated_process() to declare timeout variable once
and pass it to all process init functions uniformly
- Updated update_simulation() to declare timeout variable once and
pass it to all process step functions uniformly
- Simplified control flow by removing special case for client timeout
handling - now all process types use the same timeout logic
- Consolidated timeout-to-wakeup_time conversion after switch statements
**Benefits:**
- Consistent API across all process types
- Cleaner code with reduced duplication
- Future-proof for server timeout requirements
- All processes now managed uniformly by simulation framework
The simulation continues to run correctly with all process types
handling timeout parameter appropriately.
Added a simulation client that can run inside the TinyDFS simulation
framework, enabling fully deterministic end-to-end testing of the
distributed file system.
**New Files:**
- src/simulation_client.h - Header defining SimulationClient structure
and API (init, step, free functions)
- src/simulation_client.c - Implementation of simulation client with
test operations (create dir, create file, write, read, list, delete)
**Key Changes:**
**System Framework (src/system.c):**
- Added ProcessType enum (METADATA_SERVER, CHUNK_SERVER, CLIENT)
to distinguish different simulated process types
- Extended Process union to include SimulationClient
- Modified spawn_simulated_process() to detect --client flag and
initialize client processes using simulation_client_init()
- Updated update_simulation() to call simulation_client_step() for
client processes, handling timeout management
- Fixed cleanup_simulation() to properly set current_process before
freeing, preventing NULL pointer dereferences in mock_close()
- Added simulated_time static variable (struct timespec) for
deterministic clock mocking
- Added helper function is_client() to detect client processes
**API Updates (src/system.h):**
- Exported startup_simulation() function for proper initialization
- Added function declarations for simulation management
**Test Updates (src/main_test.c):**
- Added startup_simulation() call for proper initialization
- Spawns simulation client process with "--client" flag
- Increased iteration limit to 100,000 to allow operations to complete
- Added progress logging every 10,000 iterations
- Added explicit error checking and debug output
**Simulation Client Features:**
- Non-blocking operation model using tinydfs_isdone() and
tinydfs_process_events()
- State machine for sequential test operations
- Comprehensive test scenario:
1. Create directory (/test_dir)
2. Create file (/test_dir/test_file.txt)
3. Write data to file
4. Read data back
5. List directory contents
6. Delete file
- Proper integration with TinyDFS client library
- Returns poll descriptors from tinydfs_process_events()
- Timeout management for simulation scheduling
**Bug Fixes:**
- Fixed missing return statement in spawn_simulated_process()
- Fixed NULL pointer dereference in cleanup by setting current_process
- Added missing stdlib.h include for atoi()
The simulation client successfully initializes and integrates into the
simulation loop, demonstrating the framework's ability to run client
code deterministically alongside servers.
Implemented Windows high-resolution timing mocks for deterministic simulation:
**mock_QueryPerformanceCounter:**
- Returns deterministic counter based on simulated_time
- Uses fixed 10 MHz frequency (10,000,000 counts/second)
- Calculation: count = (tv_sec * freq) + (tv_nsec * freq / 1e9)
- Validates NULL pointer with ERROR_INVALID_PARAMETER
- Provides consistent, reproducible timing across test runs
**mock_QueryPerformanceFrequency:**
- Returns fixed frequency of 10 MHz (10,000,000 Hz)
- Common frequency on modern Windows systems
- Ensures deterministic behavior (no variation between runs)
- Validates NULL pointer with ERROR_INVALID_PARAMETER
**Integration:**
- Added function declarations to system.h
- Macros already existed (sys_QueryPerformanceCounter/Frequency)
- Complements existing mock_clock_gettime on Linux
- Both platforms now have deterministic high-resolution timing
This ensures Windows code using QueryPerformanceCounter for timing,
benchmarking, or rate limiting behaves deterministically in tests.
The fixed 10 MHz frequency provides nanosecond-level precision
(100ns per tick) suitable for most timing needs.
Enhanced the deterministic simulation framework with comprehensive error handling:
**Socket Operations:**
- mock_socket: Set EAFNOSUPPORT, EMFILE
- mock_bind: Set EBADF, ENOTSOCK, EINVAL, EADDRINUSE (with address collision detection)
- mock_listen: Set EBADF, ENOTSOCK, EDESTADDRREQ (check socket is bound)
- mock_accept: Set EBADF, EINVAL, EWOULDBLOCK, ECONNABORTED, EMFILE
- mock_connect: Set EBADF, EISCONN, EINVAL, EINPROGRESS
- mock_recv: Set EBADF, EOPNOTSUPP, ENOTCONN, ECONNRESET, EWOULDBLOCK
- mock_send: Set EBADF, EOPNOTSUPP, ENOTCONN, EPIPE, EWOULDBLOCK
- mock_getsockopt: Set EBADF, ENOPROTOOPT, EINVAL (implemented SO_ERROR)
- mock_setsockopt: Set EBADF, ENOPROTOOPT (no-op but validates input)
**File Operations (Linux):**
- mock_close: Set EBADF with bounds checking
- mock_flock: Set EBADF with validation
- mock_fsync: Set EBADF, EINVAL
- mock_read: Set EBADF with descriptor type checking
- mock_write: Set EBADF with descriptor type checking
- mock_fstat: Set EBADF with validation
- All file ops forward errno from real syscalls when appropriate
**File Operations (Windows):**
- mock_CloseHandle: SetLastError ERROR_INVALID_HANDLE
- mock_LockFile/UnlockFile: SetLastError ERROR_INVALID_HANDLE
- mock_FlushFileBuffers: SetLastError ERROR_INVALID_HANDLE
- mock_ReadFile/WriteFile: SetLastError ERROR_INVALID_HANDLE
- mock_GetFileSizeEx: SetLastError ERROR_INVALID_HANDLE
- All Windows file ops forward errors from real API calls
**Simulated Clock:**
- Implemented mock_clock_gettime with proper errno handling
- Returns simulated time for deterministic behavior
- Validates clock ID (CLOCK_REALTIME, CLOCK_MONOTONIC)
- Sets EFAULT for NULL pointer, EINVAL for invalid clock ID
**Improvements:**
- Added comprehensive bounds checking on all file descriptors
- Proper descriptor type validation before operations
- Peer connection validation in socket operations
- Address-in-use detection for bind operations
- Consistent error code semantics matching POSIX/Windows standards
All file I/O operations continue to forward to real OS as intended,
with mocking layer providing precise error simulation and validation.
Fixed 4 compilation errors in system.c:
- mock_CloseHandle: Changed undefined 'fd' to 'handle'
- mock_LockFile: Changed 'handle' to 'hFile' parameter
- mock_UnlockFile: Changed 'handle' to 'hFile' parameter
- mock_WriteFile: Added missing comma in WriteFile call
All mocks now use correct parameter names matching their function signatures.
- Fixed compilation errors in descriptor type checks and field references
- Implemented sys_accept with pending connection queue support
- Completed sys_connect with full connection establishment between processes
- Implemented sys_getsockopt for SO_ERROR option
- Added network transit logic to transfer data between connected sockets
- Added peer tracking to descriptors for connection management
- Initialized byte queues for socket I/O buffers
The simulation now supports:
- Socket creation and binding
- Connection establishment between processes
- Bidirectional data transfer via byte queues
- Accept queue management for listen sockets
- Deterministic network simulation
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Implement simple and correct approach for determining which chunks can
be safely removed:
When a write overwrites chunks:
1. Update the file tree with new chunk hashes
2. For each old hash, walk the entire tree to check if still in use
3. Only mark chunks for removal if NOT found anywhere in the tree
Changes:
- Modified file_tree_write() to return removed_hashes array
- After updating chunks, check each old_hash with entity_uses_hash()
- Walks entire tree from root to verify hash is truly unreferenced
- Caller receives array of hashes safe to remove
- Only those hashes are added to chunk servers' rem_list
This approach is:
- Simple: No complex reference counting needed
- Correct: Handles deduplication naturally (same content, same hash)
- Safe: Only removes chunks with zero references
- Easy to understand: Linear tree walk on each write
Performance can be optimized later if needed. Correctness first!
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit addresses the following TODO items:
1. Fixed proper error codes (lines 2063, 2068, 2071, 2081)
- Changed generic -1 returns to specific FILETREE_* error codes
2. Added overflow check for chunk_size (line 2659)
- Added validation that chunk_size fits in uint32_t before casting
3. Handled error event flags in connection polling (line 1559)
- Added checks for POLLERR, POLLHUP, and POLLNVAL events
4. Fixed absolute path edge case (line 1773)
- Properly handle ".." in absolute paths (stays at root)
- Reject ".." in relative paths when at the start
5. Documented authentication verification (line 2903)
- Added comment about production authentication requirements
6. Implemented chunk removal on overwrite (line 2790)
- Old chunks are now added to rem_list for garbage collection
- Zero hashes (new chunks) are properly handled
7. Completed download chunk implementation (lines 3247, 3253, 3256)
- Implemented full download request with hash, offset, and length
- Added error handling for failed connections
8. Implemented state update chunk management (line 3308)
- Move chunks between main and orphaned directories
- Validate chunk presence and report missing chunks
9. Implemented process_chunk_server_download_error (line 3450)
- Handle download failures and retry next pending download
10. Implemented process_chunk_server_download_success (line 3456)
- Store downloaded chunks and continue with pending downloads
11. Handled chunk server connection disconnect (line 3721)
- Reset downloading state on disconnect for retry
12. Implemented AUTH message on reconnect (line 3777)
- Send authentication message when reconnecting to metadata server
13. Implemented read list processing (line 4324)
- Parse and validate list response message format
14. Documented write operation stub (line 4631)
- Added comments explaining what full implementation would require
Remaining TODOs are architectural notes for future enhancements:
- Line 124: Chunk orphaning strategy (now implemented)
- Line 662: Test parent_path function (testing task)
- Line 3673: IPv6 support (future enhancement)
- Line 3766: Periodic chunk hash verification (future feature)
- Line 3768: Periodic download management (mostly implemented)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Claude
