The RequestQueue for chunk servers should be initialized when the
connection is actually established in get_chunk_server_connection(),
not ahead of time during tinydfs_init().
The metadata server RequestQueue initialization remains in tinydfs_init()
since the metadata server connection is established during initialization.
Changes:
- Removed premature RequestQueue initialization for chunk servers
- Added clarifying comments about initialization timing
- RequestQueue is initialized in get_chunk_server_connection() (line 227)
This commit resolves all Valgrind errors reported in the test run:
1. Fixed uninitialized 'removed' array in tcp_translate_events (tcp.c:179)
- Added memset to initialize the array to zero before use
2. Fixed uninitialized RequestQueue in client initialization (client.c)
- Added initialization of metadata_server.reqs and chunk_servers[].reqs
- Added forward declaration of request_queue_init function
3. Fixed uninitialized descriptor generation field (system.c:324)
- Initialize generation to 0 when creating process descriptors
- Prevents uninitialized value errors in handle_to_desc
4. Fixed 34-byte memory leak in tcp_context_free (tcp.c:91-103)
- Added cleanup of connection byte queues when freeing TCP context
- Frees both input and output ByteQueues for all connections
All Valgrind errors have been resolved:
- ERROR SUMMARY: 0 errors from 0 contexts
- All heap blocks freed - no leaks possible
Simplified conditional compilation by wrapping the entire system.c file
with a single #ifdef BUILD_TEST guard, removing all nested conditionals
for simulation client code.
**Rationale:**
The entire system.c file contains simulation framework code (mock functions,
Process structure, spawn_simulated_process, update_simulation, etc.) that
is only used when BUILD_TEST is defined. Having nested #ifdef BUILD_TEST
conditionals within this file made no sense.
**Changes:**
**system.c:**
- Added #ifdef BUILD_TEST at the very beginning of the file (line 1)
- Added #endif // BUILD_TEST at the very end of the file (line 1486)
- Removed #ifdef BUILD_TEST around simulation_client.h include
- Removed #ifdef BUILD_TEST around PROCESS_TYPE_CLIENT enum value
- Removed #ifdef BUILD_TEST around SimulationClient union member
- Removed #ifdef BUILD_TEST around is_client() function
- Removed #ifdef BUILD_TEST around client detection in spawn_simulated_process()
- Removed #ifdef BUILD_TEST around PROCESS_TYPE_CLIENT cases in:
- spawn_simulated_process() init switch
- free_process() cleanup switch
- update_simulation() step switch
**Result:**
- Much cleaner code with single top-level BUILD_TEST guard
- No nested conditionals cluttering the code
- All three build targets still compile successfully:
- tinydfs_server.out (excludes system.c entirely)
- example_client.out (excludes system.c entirely)
- tinydfs_test.out (includes all of system.c)
- Simulation runs correctly with client support
The file structure now correctly reflects that system.c is purely test
infrastructure and not used in production builds.
Fixed uninitialized timeout variables in metadata_server_main() and
chunk_server_main() functions by setting default value to -1.
**Changes:**
- metadata_server_main(): Changed `int timeout;` to `int timeout = -1;`
- chunk_server_main(): Changed `int timeout;` to `int timeout = -1;`
This ensures the timeout parameter has a defined default value before
being passed to init functions, which is important for predictable
behavior even though the init functions set the value themselves.
All build targets (tinydfs_server.out, example_client.out,
tinydfs_test.out) compile and run successfully.
Fixed build errors in tinydfs_server and example_client targets caused
by missing timeout parameter and simulation client references.
**Changes:**
**main_server.c:**
- Added int timeout variable declaration in metadata_server_main()
- Added int timeout variable declaration in chunk_server_main()
- Pass &timeout to metadata_server_init() call
- Pass &timeout to metadata_server_step() call
- Pass &timeout to chunk_server_init() call
- Pass &timeout to chunk_server_step() call
**system.c:**
- Guarded simulation_client.h include with #ifdef BUILD_TEST
- Guarded PROCESS_TYPE_CLIENT enum value with #ifdef BUILD_TEST
- Guarded SimulationClient member in Process union with #ifdef BUILD_TEST
- Guarded is_client() function with #ifdef BUILD_TEST
- Guarded client detection logic in spawn_simulated_process()
- Guarded PROCESS_TYPE_CLIENT cases in all switch statements:
- spawn_simulated_process() init switch
- free_process() cleanup switch
- update_simulation() step switch
**Result:**
- tinydfs_server.out now compiles successfully
- example_client.out now compiles successfully
- tinydfs_test.out continues to work correctly
- Simulation client code only included when BUILD_TEST is defined
- All three build targets pass compilation with only pre-existing warnings
The simulation_client code is now properly isolated to test builds only,
preventing linker errors when building the real server and example client.
Unified the function signatures across all process types (metadata server,
chunk server, and simulation client) to accept an int *timeout parameter.
This provides consistency in the simulation framework and allows all
process types to control their wakeup times.
**Changes:**
**Headers (metadata_server.h, chunk_server.h):**
- Updated metadata_server_init() to accept int *timeout parameter
- Updated metadata_server_step() to accept int *timeout parameter
- Updated chunk_server_init() to accept int *timeout parameter
- Updated chunk_server_step() to accept int *timeout parameter
**Implementations (metadata_server.c, chunk_server.c):**
- Modified init functions to set *timeout = -1 (no timeout needed)
- Modified step functions to set *timeout = -1 (no timeout needed)
- Both server types currently don't require periodic wakeups, but the
parameter allows for future timeout-based behavior
**System Integration (system.c):**
- Updated spawn_simulated_process() to declare timeout variable once
and pass it to all process init functions uniformly
- Updated update_simulation() to declare timeout variable once and
pass it to all process step functions uniformly
- Simplified control flow by removing special case for client timeout
handling - now all process types use the same timeout logic
- Consolidated timeout-to-wakeup_time conversion after switch statements
**Benefits:**
- Consistent API across all process types
- Cleaner code with reduced duplication
- Future-proof for server timeout requirements
- All processes now managed uniformly by simulation framework
The simulation continues to run correctly with all process types
handling timeout parameter appropriately.
Added a simulation client that can run inside the TinyDFS simulation
framework, enabling fully deterministic end-to-end testing of the
distributed file system.
**New Files:**
- src/simulation_client.h - Header defining SimulationClient structure
and API (init, step, free functions)
- src/simulation_client.c - Implementation of simulation client with
test operations (create dir, create file, write, read, list, delete)
**Key Changes:**
**System Framework (src/system.c):**
- Added ProcessType enum (METADATA_SERVER, CHUNK_SERVER, CLIENT)
to distinguish different simulated process types
- Extended Process union to include SimulationClient
- Modified spawn_simulated_process() to detect --client flag and
initialize client processes using simulation_client_init()
- Updated update_simulation() to call simulation_client_step() for
client processes, handling timeout management
- Fixed cleanup_simulation() to properly set current_process before
freeing, preventing NULL pointer dereferences in mock_close()
- Added simulated_time static variable (struct timespec) for
deterministic clock mocking
- Added helper function is_client() to detect client processes
**API Updates (src/system.h):**
- Exported startup_simulation() function for proper initialization
- Added function declarations for simulation management
**Test Updates (src/main_test.c):**
- Added startup_simulation() call for proper initialization
- Spawns simulation client process with "--client" flag
- Increased iteration limit to 100,000 to allow operations to complete
- Added progress logging every 10,000 iterations
- Added explicit error checking and debug output
**Simulation Client Features:**
- Non-blocking operation model using tinydfs_isdone() and
tinydfs_process_events()
- State machine for sequential test operations
- Comprehensive test scenario:
1. Create directory (/test_dir)
2. Create file (/test_dir/test_file.txt)
3. Write data to file
4. Read data back
5. List directory contents
6. Delete file
- Proper integration with TinyDFS client library
- Returns poll descriptors from tinydfs_process_events()
- Timeout management for simulation scheduling
**Bug Fixes:**
- Fixed missing return statement in spawn_simulated_process()
- Fixed NULL pointer dereference in cleanup by setting current_process
- Added missing stdlib.h include for atoi()
The simulation client successfully initializes and integrates into the
simulation loop, demonstrating the framework's ability to run client
code deterministically alongside servers.
Implemented Windows high-resolution timing mocks for deterministic simulation:
**mock_QueryPerformanceCounter:**
- Returns deterministic counter based on simulated_time
- Uses fixed 10 MHz frequency (10,000,000 counts/second)
- Calculation: count = (tv_sec * freq) + (tv_nsec * freq / 1e9)
- Validates NULL pointer with ERROR_INVALID_PARAMETER
- Provides consistent, reproducible timing across test runs
**mock_QueryPerformanceFrequency:**
- Returns fixed frequency of 10 MHz (10,000,000 Hz)
- Common frequency on modern Windows systems
- Ensures deterministic behavior (no variation between runs)
- Validates NULL pointer with ERROR_INVALID_PARAMETER
**Integration:**
- Added function declarations to system.h
- Macros already existed (sys_QueryPerformanceCounter/Frequency)
- Complements existing mock_clock_gettime on Linux
- Both platforms now have deterministic high-resolution timing
This ensures Windows code using QueryPerformanceCounter for timing,
benchmarking, or rate limiting behaves deterministically in tests.
The fixed 10 MHz frequency provides nanosecond-level precision
(100ns per tick) suitable for most timing needs.
Enhanced the deterministic simulation framework with comprehensive error handling:
**Socket Operations:**
- mock_socket: Set EAFNOSUPPORT, EMFILE
- mock_bind: Set EBADF, ENOTSOCK, EINVAL, EADDRINUSE (with address collision detection)
- mock_listen: Set EBADF, ENOTSOCK, EDESTADDRREQ (check socket is bound)
- mock_accept: Set EBADF, EINVAL, EWOULDBLOCK, ECONNABORTED, EMFILE
- mock_connect: Set EBADF, EISCONN, EINVAL, EINPROGRESS
- mock_recv: Set EBADF, EOPNOTSUPP, ENOTCONN, ECONNRESET, EWOULDBLOCK
- mock_send: Set EBADF, EOPNOTSUPP, ENOTCONN, EPIPE, EWOULDBLOCK
- mock_getsockopt: Set EBADF, ENOPROTOOPT, EINVAL (implemented SO_ERROR)
- mock_setsockopt: Set EBADF, ENOPROTOOPT (no-op but validates input)
**File Operations (Linux):**
- mock_close: Set EBADF with bounds checking
- mock_flock: Set EBADF with validation
- mock_fsync: Set EBADF, EINVAL
- mock_read: Set EBADF with descriptor type checking
- mock_write: Set EBADF with descriptor type checking
- mock_fstat: Set EBADF with validation
- All file ops forward errno from real syscalls when appropriate
**File Operations (Windows):**
- mock_CloseHandle: SetLastError ERROR_INVALID_HANDLE
- mock_LockFile/UnlockFile: SetLastError ERROR_INVALID_HANDLE
- mock_FlushFileBuffers: SetLastError ERROR_INVALID_HANDLE
- mock_ReadFile/WriteFile: SetLastError ERROR_INVALID_HANDLE
- mock_GetFileSizeEx: SetLastError ERROR_INVALID_HANDLE
- All Windows file ops forward errors from real API calls
**Simulated Clock:**
- Implemented mock_clock_gettime with proper errno handling
- Returns simulated time for deterministic behavior
- Validates clock ID (CLOCK_REALTIME, CLOCK_MONOTONIC)
- Sets EFAULT for NULL pointer, EINVAL for invalid clock ID
**Improvements:**
- Added comprehensive bounds checking on all file descriptors
- Proper descriptor type validation before operations
- Peer connection validation in socket operations
- Address-in-use detection for bind operations
- Consistent error code semantics matching POSIX/Windows standards
All file I/O operations continue to forward to real OS as intended,
with mocking layer providing precise error simulation and validation.
Fixed 4 compilation errors in system.c:
- mock_CloseHandle: Changed undefined 'fd' to 'handle'
- mock_LockFile: Changed 'handle' to 'hFile' parameter
- mock_UnlockFile: Changed 'handle' to 'hFile' parameter
- mock_WriteFile: Added missing comma in WriteFile call
All mocks now use correct parameter names matching their function signatures.
- Fixed compilation errors in descriptor type checks and field references
- Implemented sys_accept with pending connection queue support
- Completed sys_connect with full connection establishment between processes
- Implemented sys_getsockopt for SO_ERROR option
- Added network transit logic to transfer data between connected sockets
- Added peer tracking to descriptors for connection management
- Initialized byte queues for socket I/O buffers
The simulation now supports:
- Socket creation and binding
- Connection establishment between processes
- Bidirectional data transfer via byte queues
- Accept queue management for listen sockets
- Deterministic network simulation
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Implement simple and correct approach for determining which chunks can
be safely removed:
When a write overwrites chunks:
1. Update the file tree with new chunk hashes
2. For each old hash, walk the entire tree to check if still in use
3. Only mark chunks for removal if NOT found anywhere in the tree
Changes:
- Modified file_tree_write() to return removed_hashes array
- After updating chunks, check each old_hash with entity_uses_hash()
- Walks entire tree from root to verify hash is truly unreferenced
- Caller receives array of hashes safe to remove
- Only those hashes are added to chunk servers' rem_list
This approach is:
- Simple: No complex reference counting needed
- Correct: Handles deduplication naturally (same content, same hash)
- Safe: Only removes chunks with zero references
- Easy to understand: Linear tree walk on each write
Performance can be optimized later if needed. Correctness first!
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit addresses the following TODO items:
1. Fixed proper error codes (lines 2063, 2068, 2071, 2081)
- Changed generic -1 returns to specific FILETREE_* error codes
2. Added overflow check for chunk_size (line 2659)
- Added validation that chunk_size fits in uint32_t before casting
3. Handled error event flags in connection polling (line 1559)
- Added checks for POLLERR, POLLHUP, and POLLNVAL events
4. Fixed absolute path edge case (line 1773)
- Properly handle ".." in absolute paths (stays at root)
- Reject ".." in relative paths when at the start
5. Documented authentication verification (line 2903)
- Added comment about production authentication requirements
6. Implemented chunk removal on overwrite (line 2790)
- Old chunks are now added to rem_list for garbage collection
- Zero hashes (new chunks) are properly handled
7. Completed download chunk implementation (lines 3247, 3253, 3256)
- Implemented full download request with hash, offset, and length
- Added error handling for failed connections
8. Implemented state update chunk management (line 3308)
- Move chunks between main and orphaned directories
- Validate chunk presence and report missing chunks
9. Implemented process_chunk_server_download_error (line 3450)
- Handle download failures and retry next pending download
10. Implemented process_chunk_server_download_success (line 3456)
- Store downloaded chunks and continue with pending downloads
11. Handled chunk server connection disconnect (line 3721)
- Reset downloading state on disconnect for retry
12. Implemented AUTH message on reconnect (line 3777)
- Send authentication message when reconnecting to metadata server
13. Implemented read list processing (line 4324)
- Parse and validate list response message format
14. Documented write operation stub (line 4631)
- Added comments explaining what full implementation would require
Remaining TODOs are architectural notes for future enhancements:
- Line 124: Chunk orphaning strategy (now implemented)
- Line 662: Test parent_path function (testing task)
- Line 3673: IPv6 support (future enhancement)
- Line 3766: Periodic chunk hash verification (future feature)
- Line 3768: Periodic download management (mostly implemented)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
cozis
