1. tcp_write infinite loop: When the output byte queue enters an error
state (buffer full), tcp_write would loop forever because
byte_queue_write_buf returns {NULL,0} without advancing. Break out
of the loop when the write buffer is empty.
2. Proxy ignores async return values: toastyfs_async_get/put/delete
can fail (return -1) but the proxy unconditionally set the operation
to STARTED, causing it to wait forever for a result that never comes.
Now checks return values and responds with 500 on failure.
3. Use-after-free on closed connections: When a client disconnects while
its request is queued as PENDING, the HTTP_Conn is freed but the proxy
still holds stale pointers. Added http_response_builder_is_valid() to
detect and discard operations for dead connections.
4. Wrong HTTP status codes: The proxy returned 500 for all errors. Now
returns 507 (Insufficient Storage) for FULL, 404 for NOT_FOUND, and
503 (Service Unavailable) when the operation queue is full.
5. TCP connection leak (root cause of proxy hang): When a peer closes a
connection, process_conn_events closes the socket and sets HUP, but
tcp_conn_free_maybe immediately frees the TCP_Conn before
tcp_next_event can deliver the HUP event. The HTTP server never learns
about the disconnect, so HTTP_Conn entries accumulate until max_conns
is reached and the proxy stops accepting new connections. Fixed by
only calling tcp_conn_free_maybe when the CLOSED flag is already set
(meaning the user called tcp_close), preserving the connection for
HUP event delivery otherwise.
Also adds wrk-based stress test suite (stress-test.sh + lua scripts).
https://claude.ai/code/session_01EHoWWnVgyCFxah5WaNV4kS
23 lines
573 B
Lua
23 lines
573 B
Lua
-- wrk script for mixed workload stress test
|
|
-- Mix of PUT, GET, and DELETE operations
|
|
|
|
counter = 0
|
|
|
|
request = function()
|
|
counter = counter + 1
|
|
local op = counter % 10
|
|
local key = "/mixed-key-" .. (counter % 200 + 1)
|
|
|
|
if op < 5 then
|
|
-- 50% PUTs
|
|
local body = "val-" .. counter .. "-" .. string.rep("y", 64)
|
|
return wrk.format("PUT", key, nil, body)
|
|
elseif op < 9 then
|
|
-- 40% GETs
|
|
return wrk.format("GET", key, nil, nil)
|
|
else
|
|
-- 10% DELETEs
|
|
return wrk.format("DELETE", key, nil, nil)
|
|
end
|
|
end
|