Adds support for truncating files after write operations, enabling proper
HTTP PUT semantics where the entire file content should be replaced.
Implementation:
- Added TOASTY_WRITE_TRUNCATE_AFTER flag (0x02) to ToastyFS API
- Updated file_tree_write() to accept truncate_after parameter:
* When true, sets file size to exactly offset+length
* Removes chunks beyond the new file size
* Marks removed chunks for garbage collection
- Metadata server extracts and passes truncate flag to file_tree_write()
- WAL replay passes false for truncate_after (already handled in original write)
- Web proxy PUT now uses both CREATE_IF_MISSING and TRUNCATE_AFTER flags
- Updated documentation in PROTOCOL.txt, DESIGN.txt, and web/DESIGN.txt
Benefits:
- Proper HTTP PUT semantics (replace entire file content)
- Efficient: single write operation truncates and updates file
- Works with both sync and async APIs
- Garbage collection automatically removes orphaned chunks
Adds support for automatically creating files when write operations target
non-existent files. This is essential for REST PUT operations which should
be able to create new resources.
Implementation:
- Added TOASTY_WRITE_CREATE_IF_MISSING flag to ToastyFS API header
- Updated toasty_write() and toasty_begin_write() to accept flags parameter
- Client sends write flags in MESSAGE_TYPE_WRITE message to metadata server
- Metadata server parses flags and handles file auto-creation atomically:
* When write fails with FILETREE_NOENT and flag is set
* Logs creation to WAL for crash consistency
* Creates file with default 4096-byte chunk size
* Retries write with newly created file's generation tag
- Updated web proxy PUT handler to use TOASTY_WRITE_CREATE_IF_MISSING
- Updated examples and simulation client for new API signature
Benefits:
- Works for both sync and async APIs
- Single round trip (atomic server-side operation)
- Prevents race conditions
- Proper WAL logging ensures crash consistency
Fixed two bugs in the HTTP server that could cause hangs:
1. web/chttp.c:4060 - Fixed incorrect error buffer check
- Was checking output buffer for errors after a read operation
- Should check input buffer instead
- This could cause the server to miss read errors and not close
bad connections properly
2. web/main.c:334 - Added loop to process multiple queued requests
- Previously only processed one request per event loop iteration
- Now processes all queued requests before returning to poll()
- This prevents requests from being stuck in the queue when the
socket is in ESTABLISHED_READY state
These fixes improve the server's ability to handle multiple requests
on Keep-Alive connections.
- Add web server integration to cluster_demo.sh script
- Configure web server port (8090)
- Update build_if_needed() to check for toastyfs_web binary
- Add get_web_binary() function
- Start web server after chunk servers
- Update status display to show web server
- Update help text to show HTTP interface
- Fix bug in web/main.c where --local-port incorrectly set
upstream_port instead of local_port
The cluster demo now starts a full cluster with:
- Metadata server on port 8080 (ToastyFS protocol)
- N chunk servers starting from port 8081
- Web server on port 8090 (HTTP interface)
Add functionality to determine the actual number of bytes read during
read operations, making it possible to detect when a read was truncated
because it went past the end of the file.
Changes:
- Modified file_tree_read() to calculate and return actual_bytes via
new output parameter
- Updated metadata server to send actual_bytes in READ_SUCCESS messages
- Added bytes_read field to ToastyResult structure
- Modified client to parse, store, and report actual bytes read
- Updated toasty_read() to return the actual number of bytes read
instead of always returning 0
- Fixed web server to use bytes_read field instead of non-existent
count field
This allows clients to distinguish between:
- Reading zeros because the file is sparse (has holes)
- Reading past the end of the file (truncated read)
Claude
cozis