Claudeandcozis
6746b0d5fb
Implements a new special generation counter value (UINT64_MAX) that allows clients to assert that a file should NOT exist when performing operations. Primary use case: Atomic "create-if-not-exists" operations When combined with TOASTY_WRITE_CREATE_IF_MISSING flag: - File doesn't exist → server creates file and writes (SUCCESS) - File exists → gen_match fails → BADGEN error (prevents overwrite) This enables race-condition-free file creation where you want to create a NEW file or fail if it already exists. Key changes: - Added MISSING_FILE_GENERATION constant to file_tree.h - Updated gen_match() to return false when checking existing entities against MISSING_FILE_GENERATION - Modified file_tree_delete_entity() to succeed (no-op) when file is missing and MISSING_FILE_GENERATION is specified (idempotent delete) - Modified file_tree_write() to validate against the new value - Added documentation in metadata_server.c explaining how MISSING_FILE_GENERATION works WITH CREATE_IF_MISSING - Updated protocol documentation in PROTOCOL.txt - Added comprehensive feature documentation in MISSING_FILE_GENERATION.md Additional use cases: - Idempotent delete operations (delete only if not already gone) - Detecting unexpected file creation in validation/testing scenarios The implementation is backward compatible and type-safe. Files are never assigned MISSING_FILE_GENERATION as their actual generation value.
ToastyFS
ToastyFS is a distributed file system designed for self-hosting, so it aims to be pragmatic, understandable, and robust. You can use ToastyFS to store your files reliably over multiple machines knowing they will be automatically replicated and healed in case of failures.
To try it out, run the script ./scripts/cluster_demo.sh. It will spawn a local cluster with a web interface at https://127.0.0.1:8090/ which will allow you to PUT, GET, DELETE files.
⚠️ Note that ToastyFS is still in early development ⚠️
🎵 Now let's get toasty 🎵
Features
- Cross-platform (runs on Windows and Linux)
- Automatic Replication & Self-Healing
- Automatic content deduplication via internal content-addressing
- Configurable file chunk sizes
- Small and understandable
But ToastyFS is still in early development, so here are the missing features:
- No master replication
- No authentication or encryption
Testing
ToastyFS is tested by running an in-memory simulation of a cluster with many clients running hundreds of random operations in parallel. The test is run for long periods of times under valgrind or compiled with sanitizers.