Claude
8160c5ac51
Adds support for automatically creating files when write operations target non-existent files. This is essential for REST PUT operations which should be able to create new resources. Implementation: - Added TOASTY_WRITE_CREATE_IF_MISSING flag to ToastyFS API header - Updated toasty_write() and toasty_begin_write() to accept flags parameter - Client sends write flags in MESSAGE_TYPE_WRITE message to metadata server - Metadata server parses flags and handles file auto-creation atomically: * When write fails with FILETREE_NOENT and flag is set * Logs creation to WAL for crash consistency * Creates file with default 4096-byte chunk size * Retries write with newly created file's generation tag - Updated web proxy PUT handler to use TOASTY_WRITE_CREATE_IF_MISSING - Updated examples and simulation client for new API signature Benefits: - Works for both sync and async APIs - Single round trip (atomic server-side operation) - Prevents race conditions - Proper WAL logging ensures crash consistency
ToastyFS
ToastyFS is a distributed file system designed for self-hosting, so it aims to be pragmatic, understandable, and robust. You can use ToastyFS to store your files reliably over multiple machines knowing they will be automatically replicated and healed in case of failures.
To try it out, run the script ./scripts/cluster_demo.sh. It will spawn a local cluster with a web interface at https://127.0.0.1:8090/ which will allow you to PUT, GET, DELETE files.
⚠️ Note that ToastyFS is still in early development ⚠️
🎵 Now let's get toasty 🎵
Features
- Cross-platform (runs on Windows and Linux)
- Automatic Replication & Self-Healing
- Automatic content deduplication via internal content-addressing
- Configurable file chunk sizes
- Small and understandable
But ToastyFS is still in early development, so here are the missing features:
- No master replication
- No authentication or encryption
Testing
ToastyFS is tested by running an in-memory simulation of a cluster with many clients running hundreds of random operations in parallel. The test is run for long periods of times under valgrind or compiled with sanitizers.