The upload tag is computed as TAG_UPLOAD_CHUNK_MIN + upload_index, and the
assertion tag <= TAG_UPLOAD_CHUNK_MAX was failing when the number of uploads
exceeded 1000. This can happen with small chunk sizes (e.g., 1 byte) and
large writes, where each chunk needs multiple upload operations for
replication.
Increase TAG_UPLOAD_CHUNK_MAX from 2000 to 100000 to accommodate up to 99000
concurrent upload operations per write.
When responding to a full chunk download request (full == 1), the code
was writing uninitialized target_len to the response message instead of
the actual data length. This caused chunk-to-chunk downloads to fail,
preventing proper replication and blocking coverage of download-related
code paths.
Two changes to improve code coverage in deterministic simulation testing:
1. Add 3 chunk servers (was 1) to match the replication factor
- This enables testing of chunk replication and chunk-to-chunk
server communication code paths in chunk_server.c
- Covers: process_chunk_server_download_*, start_download,
download_targets_* functions
2. Fix Quakey bug where events targeting a closed descriptor caused
assertion failures
- Added remove_events_targeting_desc() to clean up DISCONNECT and
DATA events when a descriptor is freed
- Modified desc_free() to accept Sim pointer and call cleanup
- This was exposed by running multiple chunk servers which create
more connection activity
Three related issues were causing the simulation to get stuck:
1. Network events (connect, disconnect, send_data) were scheduled with
relative times instead of absolute times, causing events to be
scheduled in the past once simulation time exceeded 10-100ms.
2. When DATA events couldn't be fully consumed (receiver buffer full),
they stayed at their original time, preventing time from advancing.
Now they are rescheduled to current_time + 10ms.
3. When poll_timeout was 0, WAKEUP events were created at current_time,
causing an infinite loop where time never advanced. Now timeout=0
sets timedout immediately without creating an event.
Initialize *gen to NO_GENERATION at the start of file_tree_read so that
error paths (FILETREE_BADPATH, FILETREE_NOENT, FILETREE_ISDIR) have a
well-defined value. Previously, when file_tree_read failed, gen was
uninitialized, causing the assertion in process_client_read to fire.
The assertion checking gen != NO_GENERATION before the error check was
removed since:
1. On error paths, gen is not used
2. On success paths, the assertion at line 482 already validates gen
Simplifies the script to automatically compile, run (60s), and generate
the coverage report in a single invocation. Removes the two-step --report
flag approach and adds quieter output with clear step markers.
Adds build_coverage.sh script that:
- Builds the simulation with GCC branch coverage instrumentation
- Supports --report flag to run simulation and generate lcov HTML report
- Uses 60 second timeout with SIGINT for graceful shutdown
Also adds SIGINT handler to simulation to exit cleanly and flush coverage data.
Implements a new special generation counter value (UINT64_MAX) that
allows clients to assert that a file should NOT exist when performing
operations.
Primary use case: Atomic "create-if-not-exists" operations
When combined with TOASTY_WRITE_CREATE_IF_MISSING flag:
- File doesn't exist → server creates file and writes (SUCCESS)
- File exists → gen_match fails → BADGEN error (prevents overwrite)
This enables race-condition-free file creation where you want to
create a NEW file or fail if it already exists.
Key changes:
- Added MISSING_FILE_GENERATION constant to file_tree.h
- Updated gen_match() to return false when checking existing entities
against MISSING_FILE_GENERATION
- Modified file_tree_delete_entity() to succeed (no-op) when file is
missing and MISSING_FILE_GENERATION is specified (idempotent delete)
- Modified file_tree_write() to validate against the new value
- Added documentation in metadata_server.c explaining how
MISSING_FILE_GENERATION works WITH CREATE_IF_MISSING
- Updated protocol documentation in PROTOCOL.txt
- Added comprehensive feature documentation in MISSING_FILE_GENERATION.md
Additional use cases:
- Idempotent delete operations (delete only if not already gone)
- Detecting unexpected file creation in validation/testing scenarios
The implementation is backward compatible and type-safe. Files are
never assigned MISSING_FILE_GENERATION as their actual generation value.
Adds support for truncating files after write operations, enabling proper
HTTP PUT semantics where the entire file content should be replaced.
Implementation:
- Added TOASTY_WRITE_TRUNCATE_AFTER flag (0x02) to ToastyFS API
- Updated file_tree_write() to accept truncate_after parameter:
* When true, sets file size to exactly offset+length
* Removes chunks beyond the new file size
* Marks removed chunks for garbage collection
- Metadata server extracts and passes truncate flag to file_tree_write()
- WAL replay passes false for truncate_after (already handled in original write)
- Web proxy PUT now uses both CREATE_IF_MISSING and TRUNCATE_AFTER flags
- Updated documentation in PROTOCOL.txt, DESIGN.txt, and web/DESIGN.txt
Benefits:
- Proper HTTP PUT semantics (replace entire file content)
- Efficient: single write operation truncates and updates file
- Works with both sync and async APIs
- Garbage collection automatically removes orphaned chunks
Adds support for automatically creating files when write operations target
non-existent files. This is essential for REST PUT operations which should
be able to create new resources.
Implementation:
- Added TOASTY_WRITE_CREATE_IF_MISSING flag to ToastyFS API header
- Updated toasty_write() and toasty_begin_write() to accept flags parameter
- Client sends write flags in MESSAGE_TYPE_WRITE message to metadata server
- Metadata server parses flags and handles file auto-creation atomically:
* When write fails with FILETREE_NOENT and flag is set
* Logs creation to WAL for crash consistency
* Creates file with default 4096-byte chunk size
* Retries write with newly created file's generation tag
- Updated web proxy PUT handler to use TOASTY_WRITE_CREATE_IF_MISSING
- Updated examples and simulation client for new API signature
Benefits:
- Works for both sync and async APIs
- Single round trip (atomic server-side operation)
- Prevents race conditions
- Proper WAL logging ensures crash consistency
Per PROTOCOL.txt specification, WRITE operations must provide a valid
generation counter and cannot use expect_gen=0 (unlike other operations).
This ensures the client has retrieved the file's metadata and knows the
correct chunk size before writing.
Changes:
- Added FILETREE_BADGEN error code (-8)
- Modified file_tree_write() to reject expect_gen=0
- Added error message for FILETREE_BADGEN
Updated both client.c and metadata_server.c to match the protocol
specification in misc/PROTOCOL.txt:
Client-to-Server Messages:
- Added expect_gen field to DELETE, LIST, READ, and WRITE messages
- Removed old_hash and chunk_size from WRITE message chunks
Server-to-Client Responses:
- Added generation counter to CREATE_SUCCESS response
- Added generation counter to LIST_SUCCESS response
- Fixed LIST_SUCCESS field order (gen, truncated, item_count)
- Added generation counter to READ_SUCCESS response
- Added generation counter to WRITE_SUCCESS response
All changes compile successfully and pass valgrind memory checks.
Fixed two bugs in the HTTP server that could cause hangs:
1. web/chttp.c:4060 - Fixed incorrect error buffer check
- Was checking output buffer for errors after a read operation
- Should check input buffer instead
- This could cause the server to miss read errors and not close
bad connections properly
2. web/main.c:334 - Added loop to process multiple queued requests
- Previously only processed one request per event loop iteration
- Now processes all queued requests before returning to poll()
- This prevents requests from being stuck in the queue when the
socket is in ESTABLISHED_READY state
These fixes improve the server's ability to handle multiple requests
on Keep-Alive connections.
- Add web server integration to cluster_demo.sh script
- Configure web server port (8090)
- Update build_if_needed() to check for toastyfs_web binary
- Add get_web_binary() function
- Start web server after chunk servers
- Update status display to show web server
- Update help text to show HTTP interface
- Fix bug in web/main.c where --local-port incorrectly set
upstream_port instead of local_port
The cluster demo now starts a full cluster with:
- Metadata server on port 8080 (ToastyFS protocol)
- N chunk servers starting from port 8081
- Web server on port 8090 (HTTP interface)
This script provides a convenient way to spawn and manage a ToastyFS cluster
for demo and testing purposes. Features include:
- Start a cluster with configurable number of chunk servers
- Automatic building if binary is not present
- Process management with PID tracking
- Status checking for all cluster nodes
- Easy cleanup of all cluster processes
- Separate log files for each server component
- Colorized output for better readability
Usage:
./scripts/cluster_demo.sh start [num_servers] - Start cluster
./scripts/cluster_demo.sh stop - Stop cluster
./scripts/cluster_demo.sh status - Show status
./scripts/cluster_demo.sh clean - Clean data/logs
Previously, file_size was calculated as num_chunks * chunk_size, which
incorrectly treated the file size as the full capacity of all allocated
chunks. The actual file size should be the offset of the last byte
written plus 1.
Changes:
- Added file_size field to File structure to track actual file extent
- Initialize file_size to 0 when creating new files
- Update file_size in file_tree_write() based on write offset + length
- Modified file_tree_read() to use file_size instead of num_chunks * chunk_size
- Updated serialization/deserialization to handle file_size field
This ensures that actual_bytes calculations correctly reflect the true
file size, not just the allocated chunk capacity.
cozis
Claude