Fixed several bugs in the DirectoryScanner implementation:
Windows fixes:
- Fixed snprintf call missing buffer size parameter
- Fixed FindFirstFileA to use pattern (with wildcard) instead of path
- Changed sys_GetLastError() to GetLastError()
- Added scanner->first = false after using first result
- Removed TODO comment from cFileName usage (it's the correct field)
Linux fixes:
- Fixed typo: sys_reddir → sys_readdir
- Added missing 'done' field to DirectoryScanner struct
- Initialize scanner->done = false on successful opendir
- Fixed return value when scanner->done (should return 1, not -1)
Added mock implementations for file/directory search operations to
support both Windows and Linux in the simulation environment.
Windows mocks:
- FindFirstFileA: Wraps real search handle in descriptor
- FindNextFileA: Forwards to real API
- FindClose: Properly cleans up search handles
Linux mocks:
- opendir: Wraps real DIR* handle in descriptor
- readdir: Forwards to real API
- closedir: Properly cleans up directory handles
Common changes:
- Added DESC_DIRECTORY descriptor type to track directory handles
- Added real_d field to Descriptor for directory handles (HANDLE on Windows, DIR* on Linux)
- Updated close_desc to handle directory cleanup on both platforms
- Added sys_* macros for both BUILD_TEST and non-BUILD_TEST modes
- Added <dirent.h> include for Linux directory operations
Fixed critical bugs in the DOWNLOAD_LOCATIONS message exchange between
metadata server and chunk server:
Metadata server changes:
- Fixed bug using wrong index (j instead of holders[j]) when writing
chunk server addresses
- Added missing hash field to the message (was only read, never written)
- Added trace output for STATE_UPDATE_ERROR messages
- Pre-read all missing chunks before building response
Chunk server changes:
- Updated message parser to match metadata server's message format
- Changed from group-based format to per-chunk format
- Updated type sizes (uint32_t for counts instead of uint8_t/uint16_t)
- Correctly accumulates addresses from all holders before adding to
pending download list
The message now follows this structure:
- num_missing (uint32_t)
- For each missing chunk:
- num_holders (uint32_t)
- For each holder: server addresses (num_ipv4, num_ipv6, addresses)
- hash (SHA256)
Tested successfully with mousefs_random_test under valgrind with no
memory errors detected.
- Add CLIENT_CFILES and CLIENT_OFILES variables to Makefile
- Add rules to build object files and create static library
- Include libmousefs_client.a in the 'all' target
- Update clean target to remove library and object files
- Update .gitignore to ignore *.o and *.a build artifacts
The library includes: client.c, basic.c, tcp.c, message.c
- Fix inverted binary_read error checks in process_metadata_server_download_locations
- Fix array indexing bugs (use correct loop variables j and k instead of i)
- Fix IPv4 port array type from uint8_t to uint16_t
- Fix reader cursor advancement in process_client_upload_chunk
- Enable periodic download retry mechanism by calling start_download_if_necessary
Fix how chunk servers handle state updates from the metadata server
based on the design in DESIGN.txt. The implementation now:
1. Checks if chunks in the add_list exist in the chunk store and
collects any missing chunks
2. Unmarks chunks in the add_list that were previously marked for removal
3. Marks chunks in the rem_list for removal with timestamps
4. Responds with SUCCESS if all chunks are present, or ERROR with the
list of missing chunks
Added:
- RemovalList data structure to track chunks marked for deletion with
timestamps
- chunk_store_exists() to check if a chunk file exists
- Helper functions for removal list management (init, free, add, remove, find)
This implements the garbage collection mechanism described in DESIGN.txt
where chunks are marked for removal and can be unmarked if they appear
in the add_list again.
This implements the periodic STATE_UPDATE mechanism described in DESIGN.txt
lines 80-99, where the metadata server periodically sends synchronization
messages to chunk servers.
Changes:
1. Added timing fields to ChunkServerPeer:
- last_sync_time: tracks when STATE_UPDATE was last sent
- last_response_time: tracks when chunk server last responded
2. Added health check configuration constants (config.h):
- HEALTH_CHECK_INTERVAL: 30 seconds between STATE_UPDATE messages
- HEALTH_CHECK_TIMEOUT: 90 seconds before marking server as unhealthy
3. Implemented send_state_update() function:
- Serializes and sends add_list and rem_list to chunk servers
- Updates last_sync_time when messages are sent
4. Added message handlers for chunk server responses:
- process_chunk_server_state_update_success(): merges add_list into old_list,
clears pending lists, updates last_response_time
- process_chunk_server_state_update_error(): handles missing chunks,
updates last_response_time, logs errors
5. Implemented periodic timer logic in metadata_server_step():
- Calculates when next STATE_UPDATE should be sent to each chunk server
- Sends updates when HEALTH_CHECK_INTERVAL has elapsed
- Logs warnings when chunk servers haven't responded within HEALTH_CHECK_TIMEOUT
- Sets appropriate poll() timeout to wake up for next health check
The implementation ensures that:
- Chunk servers receive regular synchronization messages with add_list/rem_list
- The metadata server tracks chunk server health based on response times
- STATE_UPDATE messages are sent periodically (every 30 seconds by default)
- Failed/missing chunks are detected and logged for recovery
This completes the health check mechanism described in the architecture design,
enabling the metadata server to maintain accurate state with chunk servers and
detect unhealthy servers.
Changes:
- Added is_nonblocking flag to Descriptor structure in system.c
- Implemented mock_fcntl (Linux) and mock_ioctlsocket (Windows) to handle setting sockets as non-blocking
- Set all sockets as non-blocking in tcp.c:
- Listen sockets in create_listen_socket()
- Accepted sockets in tcp_translate_events()
- Connecting sockets in tcp_connect()
- Added checks in mock_accept(), mock_recv(), and mock_send() to abort the simulation if a socket would block but is not configured as non-blocking
- This ensures proper non-blocking socket configuration and helps catch blocking socket errors during simulation
- Removed simulated_time struct timespec variable
- Updated mock_QueryPerformanceCounter to convert current_time (nanoseconds) to performance counter ticks
- Updated mock_clock_gettime to convert current_time (nanoseconds) to timespec format
- All time tracking now uses the unified current_time variable
Claude
cozis