Add TOASTY_WRITE_CREATE_IF_MISSING flag for server-side file auto-creation

Adds support for automatically creating files when write operations target
non-existent files. This is essential for REST PUT operations which should
be able to create new resources.

Implementation:
- Added TOASTY_WRITE_CREATE_IF_MISSING flag to ToastyFS API header
- Updated toasty_write() and toasty_begin_write() to accept flags parameter
- Client sends write flags in MESSAGE_TYPE_WRITE message to metadata server
- Metadata server parses flags and handles file auto-creation atomically:
  * When write fails with FILETREE_NOENT and flag is set
  * Logs creation to WAL for crash consistency
  * Creates file with default 4096-byte chunk size
  * Retries write with newly created file's generation tag
- Updated web proxy PUT handler to use TOASTY_WRITE_CREATE_IF_MISSING
- Updated examples and simulation client for new API signature

Benefits:
- Works for both sync and async APIs
- Single round trip (atomic server-side operation)
- Prevents race conditions
- Proper WAL logging ensures crash consistency
This commit is contained in:
Claude
2025-12-03 18:48:42 +00:00
parent a082f46b78
commit 8160c5ac51
9 changed files with 80 additions and 16 deletions
+13 -5
View File
@@ -41,11 +41,19 @@ Architecture
modifications are successful, the client holds the set of
old hashes and new hashes for that file range. It completes
the write by telling the metadata server to swap the old
hashes with the new ones. If the old hashes don't match,
another write succeded in the mean time and touched that
range, therefore the write fails. If the old hashes match,
the write succeded. If the client fails to modify any
chunks, it doesn't commit the write with the metadata server.
hashes with the new ones (optionally including write flags
such as TOASTY_WRITE_CREATE_IF_MISSING). If the old hashes
don't match, another write succeded in the mean time and
touched that range, therefore the write fails. If the old
hashes match, the write succeded. If the client fails to
modify any chunks, it doesn't commit the write with the
metadata server.
If the file doesn't exist and the TOASTY_WRITE_CREATE_IF_MISSING
flag is set, the metadata server atomically creates the file
with a default chunk size (4096 bytes) and retries the write.
This operation is logged to the WAL to ensure crash consistency.
Note that write failures may cause chunks to be orphaned
on chunk servers. This is solved by a garbage collection
algorithm implemented by the synchronization messages
+7
View File
@@ -106,6 +106,7 @@ the metadata server:
struct WriteMessage {
Header header; // type=WRITE
uint64_t expect_gen;
uint32_t flags;
uint16_t path_len;
char path[path_len];
uint32_t offset;
@@ -116,6 +117,12 @@ the metadata server:
If the expect_gen field doesn't match the generation of the target file, the operation fails. Note that unlike other operations, the expect_gen CAN'T be 0. This is due to the assumption that the chunk size hasn't change for that file since the writer originally retrieved the file's metadata.
The flags field contains write operation flags. Currently defined flags:
- TOASTY_WRITE_CREATE_IF_MISSING (0x01): If the file doesn't exist,
the metadata server will atomically create it with a default chunk
size of 4096 bytes and retry the write operation. The creation is
logged to the WAL for crash consistency.
The offset and length mark the region that is being written to.
Then comes an array of num_chunks sections each specifying where a given chunk was written to. Note that the number of chunks is equal to