Claude
e8d1e059c3
This commit addresses several important robustness issues in the WAL
implementation:
1. Fixed file_path lifetime issue:
- The file_path argument to wal_open() may not have the same lifetime
as the WAL structure
- Now allocate and copy the path string to ensure WAL owns its data
- Properly free the allocated path in wal_close()
- Added error cleanup path in wal_open() to prevent memory leaks
2. Replaced static arrays with dynamic allocation in next_entry():
- Static arrays were not thread-safe and had limited lifetimes
- Now dynamically allocate path buffer and hash buffers using sys_malloc
- Caller must free allocated fields after using the entry
- Added proper error cleanup to free allocations on failure
- Updated wal_open() to free entry fields after processing each entry
3. Improved swap_file() for cross-platform compatibility:
- On Unix/Linux: rename() atomically replaces the destination file
- On Windows: rename() doesn't overwrite, so delete old file first
- Added platform-specific handling with #ifdef _WIN32
- Ensures WAL rotation works correctly on both platforms
4. Added system.h include for sys_malloc/sys_free definitions
These changes ensure proper memory management, prevent leaks, and make
the WAL implementation more robust across different platforms.
ToastyFS
ToastyFS is a distributed file system designed for self-hosting, so it aims to be pragmatic, understandable, and robust. You can use ToastyFS to store your files reliably over multiple machines knowing they will be automatically replicated and healed in case of hardware failure. ToastyFS works by running nodes on multiple machines. Clients using the ToastyFS C library can then send file operations to the cluster. Here's a quick example:
#include <ToastyFS.h>
int main(void)
{
ToastyString addr = TOASTY_STR("127.0.0.1");
int port = 8080;
ToastyString file = TOASTY_STR("/my_file.txt");
// Connect to cluster
ToastyFS *toasty = toasty_connect(addr, port);
// Create and write to a file
toasty_create_file(toasty, file, 4096);
toasty_write(toasty, file, 0, "Hello!", 6);
// Read it back
char buf[6];
toasty_read(toasty, file, 0, buf, 6);
// Done!
toasty_disconnect(toasty);
return 0;
}
⚠️ Note that ToastyFS is still in early development ⚠️
🎵 Now let's get toasty 🎵
Features
- Cross-platform (runs on Windows and Linux)
- Automatic Replication & Self-Healing
- Automatic content deduplication via internal content-addressing
- Configurable file chunk sizes
- Small and understandable
But ToastyFS is still in early development, so here are the missing features:
- No master replication
- No authentication or encryption
Testing
ToastyFS is tested by running an in-memory simulation of a cluster with many clients running hundreds of random operations in parallel. The test is run for long periods of times under valgrind or compiled with sanitizers.