Fixed critical bugs in the DOWNLOAD_LOCATIONS message exchange between
metadata server and chunk server:
Metadata server changes:
- Fixed bug using wrong index (j instead of holders[j]) when writing
chunk server addresses
- Added missing hash field to the message (was only read, never written)
- Added trace output for STATE_UPDATE_ERROR messages
- Pre-read all missing chunks before building response
Chunk server changes:
- Updated message parser to match metadata server's message format
- Changed from group-based format to per-chunk format
- Updated type sizes (uint32_t for counts instead of uint8_t/uint16_t)
- Correctly accumulates addresses from all holders before adding to
pending download list
The message now follows this structure:
- num_missing (uint32_t)
- For each missing chunk:
- num_holders (uint32_t)
- For each holder: server addresses (num_ipv4, num_ipv6, addresses)
- hash (SHA256)
Tested successfully with mousefs_random_test under valgrind with no
memory errors detected.
- Add CLIENT_CFILES and CLIENT_OFILES variables to Makefile
- Add rules to build object files and create static library
- Include libmousefs_client.a in the 'all' target
- Update clean target to remove library and object files
- Update .gitignore to ignore *.o and *.a build artifacts
The library includes: client.c, basic.c, tcp.c, message.c
- Fix inverted binary_read error checks in process_metadata_server_download_locations
- Fix array indexing bugs (use correct loop variables j and k instead of i)
- Fix IPv4 port array type from uint8_t to uint16_t
- Fix reader cursor advancement in process_client_upload_chunk
- Enable periodic download retry mechanism by calling start_download_if_necessary
Fix how chunk servers handle state updates from the metadata server
based on the design in DESIGN.txt. The implementation now:
1. Checks if chunks in the add_list exist in the chunk store and
collects any missing chunks
2. Unmarks chunks in the add_list that were previously marked for removal
3. Marks chunks in the rem_list for removal with timestamps
4. Responds with SUCCESS if all chunks are present, or ERROR with the
list of missing chunks
Added:
- RemovalList data structure to track chunks marked for deletion with
timestamps
- chunk_store_exists() to check if a chunk file exists
- Helper functions for removal list management (init, free, add, remove, find)
This implements the garbage collection mechanism described in DESIGN.txt
where chunks are marked for removal and can be unmarked if they appear
in the add_list again.
This implements the periodic STATE_UPDATE mechanism described in DESIGN.txt
lines 80-99, where the metadata server periodically sends synchronization
messages to chunk servers.
Changes:
1. Added timing fields to ChunkServerPeer:
- last_sync_time: tracks when STATE_UPDATE was last sent
- last_response_time: tracks when chunk server last responded
2. Added health check configuration constants (config.h):
- HEALTH_CHECK_INTERVAL: 30 seconds between STATE_UPDATE messages
- HEALTH_CHECK_TIMEOUT: 90 seconds before marking server as unhealthy
3. Implemented send_state_update() function:
- Serializes and sends add_list and rem_list to chunk servers
- Updates last_sync_time when messages are sent
4. Added message handlers for chunk server responses:
- process_chunk_server_state_update_success(): merges add_list into old_list,
clears pending lists, updates last_response_time
- process_chunk_server_state_update_error(): handles missing chunks,
updates last_response_time, logs errors
5. Implemented periodic timer logic in metadata_server_step():
- Calculates when next STATE_UPDATE should be sent to each chunk server
- Sends updates when HEALTH_CHECK_INTERVAL has elapsed
- Logs warnings when chunk servers haven't responded within HEALTH_CHECK_TIMEOUT
- Sets appropriate poll() timeout to wake up for next health check
The implementation ensures that:
- Chunk servers receive regular synchronization messages with add_list/rem_list
- The metadata server tracks chunk server health based on response times
- STATE_UPDATE messages are sent periodically (every 30 seconds by default)
- Failed/missing chunks are detected and logged for recovery
This completes the health check mechanism described in the architecture design,
enabling the metadata server to maintain accurate state with chunk servers and
detect unhealthy servers.
Changes:
- Added is_nonblocking flag to Descriptor structure in system.c
- Implemented mock_fcntl (Linux) and mock_ioctlsocket (Windows) to handle setting sockets as non-blocking
- Set all sockets as non-blocking in tcp.c:
- Listen sockets in create_listen_socket()
- Accepted sockets in tcp_translate_events()
- Connecting sockets in tcp_connect()
- Added checks in mock_accept(), mock_recv(), and mock_send() to abort the simulation if a socket would block but is not configured as non-blocking
- This ensures proper non-blocking socket configuration and helps catch blocking socket errors during simulation
- Removed simulated_time struct timespec variable
- Updated mock_QueryPerformanceCounter to convert current_time (nanoseconds) to performance counter ticks
- Updated mock_clock_gettime to convert current_time (nanoseconds) to timespec format
- All time tracking now uses the unified current_time variable
Fixed two critical bugs in the TinyDFS client code:
1. Added bounds checking in alloc_operation() to prevent buffer overflow
when searching for a free operation slot. The while loop could go past
the end of the operations array, causing undefined behavior.
2. Fixed all tinydfs_submit_* functions to return the operation index
(opidx) instead of 0 on success. This was causing all operations to
be tracked with index 0, leading to type mismatches between operation
types and result types, which triggered the assertion:
"Assertion `pending.type == PENDING_OPERATION_DELETE' failed"
The test now runs successfully past the original assertion point.
Fixed multiple uninitialized memory issues detected by valgrind:
1. SHA256 struct size mismatch: Changed from 64 bytes to 32 bytes to
match the actual output of sha256() function (which produces 256
bits = 32 bytes).
2. File tree chunk allocation: Fixed loop that wasn't initializing
newly allocated chunks because num_chunks was updated before the
initialization loop. Now saves old_num_chunks first.
3. Metadata server process_client_write: Initialized results[i].num_addrs
to 0 before use to prevent reading uninitialized value.
4. Address struct initialization: Zero-initialize Address structs before
setting fields to ensure the entire union is initialized (union
contains either IPv4 or IPv6, leaving unused bytes uninitialized).
5. Fixed bug in IPv6 address creation: Changed incorrect is_ipv4=true
to is_ipv4=false for IPv6 addresses.
6. Fixed all_chunk_servers_holding_chunk: Function was incrementing
counter for all chunk servers instead of only those containing the
hash, causing access to uninitialized array elements.
All valgrind errors resolved: ERROR SUMMARY: 0 errors from 0 contexts
Claude
cozis