Commit Graph
38 Commits
Author SHA1 Message Date
cozis 46db334a2e Resolve compilation errors and implement blocking API 2025-11-17 11:32:09 +01:00
cozis a4a37b7a36 Minor fixes 2025-11-16 01:18:05 +01:00
cozis 7af1bc27b5 Fix value of CHUNK_SERVER_RECONNECT_TIME by expressing it in seconds, also change it to a runtime-modifiable value 2025-11-15 12:10:26 +01:00
cozis 11054cae86 Add new chunk hashes to the cs_add_list when creating or updating them in the chunk server 2025-11-15 10:55:11 +01:00
cozis a366b46398 Bug fixes 2025-11-15 10:38:45 +01:00
cozis a11740ab41 Fix uninitialized BinaryReader struct when parsing auth response message 2025-11-14 21:14:33 +01:00
cozis bbae5fdb7f Fix compilation errors 2025-11-14 21:06:38 +01:00
cozis 3df1745d69 Add comments 2025-11-14 12:07:32 +01:00
cozis 2bbe1ac0e8 Clean up download logic for missing/lost chunks 2025-11-14 11:56:29 +01:00
cozis 9a944feb01 Rewrote simulation scheduler and fixed some bugs 2025-11-14 01:16:48 +01:00
cozis 3048f3d66b Bug fixes 2025-11-13 23:02:25 +01:00
cozis 7b01ac6384 Fix compilation errors 2025-11-13 20:21:26 +01:00
cozis cdce267768 Update DESIGN.txt and code accordingly 2025-11-13 17:27:00 +01:00
Claude 14cd2f1ac6 Fix compilation issues
Fixed compilation errors:
- Added <dirent.h> include to file_system.h for DIR type on Linux
- Replaced xxx placeholder in chunk_server.c with actual chunks_dir path
  using state->store.path
2025-11-13 00:47:56 +00:00
cozis 83c2ccb2ca Implement DirectoryScanner object 2025-11-13 01:35:15 +01:00
cozis 88189c6693 Draft of the chunk server endpoint that returns the list of all chunks 2025-11-13 01:09:01 +01:00
Claude e1869b4cf1 Fix DOWNLOAD_LOCATIONS message implementation
Fixed critical bugs in the DOWNLOAD_LOCATIONS message exchange between
metadata server and chunk server:

Metadata server changes:
- Fixed bug using wrong index (j instead of holders[j]) when writing
  chunk server addresses
- Added missing hash field to the message (was only read, never written)
- Added trace output for STATE_UPDATE_ERROR messages
- Pre-read all missing chunks before building response

Chunk server changes:
- Updated message parser to match metadata server's message format
- Changed from group-based format to per-chunk format
- Updated type sizes (uint32_t for counts instead of uint8_t/uint16_t)
- Correctly accumulates addresses from all holders before adding to
  pending download list

The message now follows this structure:
- num_missing (uint32_t)
- For each missing chunk:
  - num_holders (uint32_t)
  - For each holder: server addresses (num_ipv4, num_ipv6, addresses)
  - hash (SHA256)

Tested successfully with mousefs_random_test under valgrind with no
memory errors detected.
2025-11-10 22:01:24 +00:00
Claude 1fb07e0b4d Fix chunk download system bugs and implement periodic retry
- Fix inverted binary_read error checks in process_metadata_server_download_locations
- Fix array indexing bugs (use correct loop variables j and k instead of i)
- Fix IPv4 port array type from uint8_t to uint16_t
- Fix reader cursor advancement in process_client_upload_chunk
- Enable periodic download retry mechanism by calling start_download_if_necessary
2025-11-10 17:46:33 +00:00
cozis d25d79c1f1 Clean up state update logic in chunk server 2025-11-10 18:36:11 +01:00
Claude ade03ac6f3 Implement chunk server state update handling
Fix how chunk servers handle state updates from the metadata server
based on the design in DESIGN.txt. The implementation now:

1. Checks if chunks in the add_list exist in the chunk store and
   collects any missing chunks
2. Unmarks chunks in the add_list that were previously marked for removal
3. Marks chunks in the rem_list for removal with timestamps
4. Responds with SUCCESS if all chunks are present, or ERROR with the
   list of missing chunks

Added:
- RemovalList data structure to track chunks marked for deletion with
  timestamps
- chunk_store_exists() to check if a chunk file exists
- Helper functions for removal list management (init, free, add, remove, find)

This implements the garbage collection mechanism described in DESIGN.txt
where chunks are marked for removal and can be unmarked if they appear
in the add_list again.
2025-11-10 16:58:13 +00:00
cozis 81ecbf3a80 Remove outdated code 2025-11-10 17:48:43 +01:00
cozis 50dd662057 Add asserts 2025-11-10 17:24:15 +01:00
cozis 62d7439ddd Set sockets as non-blocking and fix chunk server timeout 2025-11-09 21:30:22 +01:00
cozis bab8558511 Progress 2025-11-09 20:36:08 +01:00
cozis 5f018c18b4 Progress 2025-11-09 16:49:15 +01:00
cozis 739d6a7aee Progress 2025-11-09 12:08:22 +01:00
cozis 88735d1595 Progress 2025-11-08 23:44:37 +01:00
cozis 279777fc8b Progress 2025-11-08 02:43:41 +01:00
cozis 463b1b4ba9 Progress 2025-11-08 02:16:46 +01:00
cozis d7cf946431 Progress 2025-11-07 20:26:15 +01:00
cozis 44fb1c4f36 Progress 2025-11-07 00:30:28 +01:00
cozis e124952fe2 Progress 2025-11-05 14:26:57 +01:00
cozis e135d871c1 Progress 2025-11-03 15:52:25 +01:00
Claude 88a3a6e4f0 Add timeout parameter to metadata_server and chunk_server init/step functions
Unified the function signatures across all process types (metadata server,
chunk server, and simulation client) to accept an int *timeout parameter.
This provides consistency in the simulation framework and allows all
process types to control their wakeup times.

**Changes:**

**Headers (metadata_server.h, chunk_server.h):**
- Updated metadata_server_init() to accept int *timeout parameter
- Updated metadata_server_step() to accept int *timeout parameter
- Updated chunk_server_init() to accept int *timeout parameter
- Updated chunk_server_step() to accept int *timeout parameter

**Implementations (metadata_server.c, chunk_server.c):**
- Modified init functions to set *timeout = -1 (no timeout needed)
- Modified step functions to set *timeout = -1 (no timeout needed)
- Both server types currently don't require periodic wakeups, but the
  parameter allows for future timeout-based behavior

**System Integration (system.c):**
- Updated spawn_simulated_process() to declare timeout variable once
  and pass it to all process init functions uniformly
- Updated update_simulation() to declare timeout variable once and
  pass it to all process step functions uniformly
- Simplified control flow by removing special case for client timeout
  handling - now all process types use the same timeout logic
- Consolidated timeout-to-wakeup_time conversion after switch statements

**Benefits:**
- Consistent API across all process types
- Cleaner code with reduced duplication
- Future-proof for server timeout requirements
- All processes now managed uniformly by simulation framework

The simulation continues to run correctly with all process types
handling timeout parameter appropriately.
2025-11-02 12:33:40 +00:00
cozis eb93541c84 Test framework cleanup 2025-11-01 22:44:04 +01:00
cozis 7525a93035 Progress 2025-10-29 23:48:45 +01:00
cozis 174f37c6c0 Continue work on DST framework 2025-10-29 21:46:17 +01:00
cozis c5a21608d7 Split project over multiple files 2025-10-28 17:24:25 +01:00