Add TOASTY_WRITE_CREATE_IF_MISSING flag for server-side file auto-creation
Adds support for automatically creating files when write operations target non-existent files. This is essential for REST PUT operations which should be able to create new resources. Implementation: - Added TOASTY_WRITE_CREATE_IF_MISSING flag to ToastyFS API header - Updated toasty_write() and toasty_begin_write() to accept flags parameter - Client sends write flags in MESSAGE_TYPE_WRITE message to metadata server - Metadata server parses flags and handles file auto-creation atomically: * When write fails with FILETREE_NOENT and flag is set * Logs creation to WAL for crash consistency * Creates file with default 4096-byte chunk size * Retries write with newly created file's generation tag - Updated web proxy PUT handler to use TOASTY_WRITE_CREATE_IF_MISSING - Updated examples and simulation client for new API signature Benefits: - Works for both sync and async APIs - Single round trip (atomic server-side operation) - Prevents race conditions - Proper WAL logging ensures crash consistency
This commit is contained in:
+13
-5
@@ -41,11 +41,19 @@ Architecture
|
||||
modifications are successful, the client holds the set of
|
||||
old hashes and new hashes for that file range. It completes
|
||||
the write by telling the metadata server to swap the old
|
||||
hashes with the new ones. If the old hashes don't match,
|
||||
another write succeded in the mean time and touched that
|
||||
range, therefore the write fails. If the old hashes match,
|
||||
the write succeded. If the client fails to modify any
|
||||
chunks, it doesn't commit the write with the metadata server.
|
||||
hashes with the new ones (optionally including write flags
|
||||
such as TOASTY_WRITE_CREATE_IF_MISSING). If the old hashes
|
||||
don't match, another write succeded in the mean time and
|
||||
touched that range, therefore the write fails. If the old
|
||||
hashes match, the write succeded. If the client fails to
|
||||
modify any chunks, it doesn't commit the write with the
|
||||
metadata server.
|
||||
|
||||
If the file doesn't exist and the TOASTY_WRITE_CREATE_IF_MISSING
|
||||
flag is set, the metadata server atomically creates the file
|
||||
with a default chunk size (4096 bytes) and retries the write.
|
||||
This operation is logged to the WAL to ensure crash consistency.
|
||||
|
||||
Note that write failures may cause chunks to be orphaned
|
||||
on chunk servers. This is solved by a garbage collection
|
||||
algorithm implemented by the synchronization messages
|
||||
|
||||
@@ -106,6 +106,7 @@ the metadata server:
|
||||
struct WriteMessage {
|
||||
Header header; // type=WRITE
|
||||
uint64_t expect_gen;
|
||||
uint32_t flags;
|
||||
uint16_t path_len;
|
||||
char path[path_len];
|
||||
uint32_t offset;
|
||||
@@ -116,6 +117,12 @@ the metadata server:
|
||||
|
||||
If the expect_gen field doesn't match the generation of the target file, the operation fails. Note that unlike other operations, the expect_gen CAN'T be 0. This is due to the assumption that the chunk size hasn't change for that file since the writer originally retrieved the file's metadata.
|
||||
|
||||
The flags field contains write operation flags. Currently defined flags:
|
||||
- TOASTY_WRITE_CREATE_IF_MISSING (0x01): If the file doesn't exist,
|
||||
the metadata server will atomically create it with a default chunk
|
||||
size of 4096 bytes and retry the write operation. The creation is
|
||||
logged to the WAL for crash consistency.
|
||||
|
||||
The offset and length mark the region that is being written to.
|
||||
|
||||
Then comes an array of num_chunks sections each specifying where a given chunk was written to. Note that the number of chunks is equal to
|
||||
|
||||
Reference in New Issue
Block a user