248 lines
16 KiB
Plaintext
248 lines
16 KiB
Plaintext
Consensus Protocol Simulation Results
|
|
Simulated time: 1200s per run, seeds 1-100
|
|
3 nodes, 2 clients, fault injection enabled, network partitioning enabled
|
|
|
|
Column descriptions:
|
|
Seed - RNG seed used for the simulation run
|
|
Proto - Consensus protocol (VSR or Raft)
|
|
Invocations - Number of client requests initiated
|
|
OK - Requests that completed successfully with a response
|
|
Timeouts - Requests that timed out without a response
|
|
Rejected - Requests explicitly rejected (e.g. sent to a non-leader)
|
|
Checks - Linearizability checks performed by the checker
|
|
Steps - Total simulation scheduling steps (ticks across all hosts)
|
|
|
|
Seed Proto Invocations OK Timeouts Rejected Checks Steps
|
|
-----------------------------------------------------------------------
|
|
1 VSR 2244 2206 33 5 2244 105648
|
|
1 Raft 3276 1700 368 1206 3274 363190
|
|
2 VSR 2883 2839 39 5 2883 126828
|
|
2 Raft 3394 2761 285 346 3392 171163
|
|
3 VSR 592 497 94 1 592 42548
|
|
3 Raft 2789 2358 330 99 2787 150699
|
|
4 VSR 1274 1231 37 6 1274 78222
|
|
4 Raft 2068 1362 464 240 2066 88436
|
|
5 VSR 1405 1372 31 1 1404 75712
|
|
5 Raft 2304 1630 416 256 2302 131192
|
|
6 VSR 2025 1975 49 0 2024 169653
|
|
6 Raft 2182 1244 447 489 2180 378921
|
|
7 VSR 955 856 98 1 955 56039
|
|
7 Raft 3433 1609 311 1511 3431 297830
|
|
8 VSR 720 636 79 4 719 46945
|
|
8 Raft 2498 1759 436 301 2496 123015
|
|
9 VSR 859 780 77 2 859 56003
|
|
9 Raft 2778 1816 362 599 2777 225590
|
|
10 VSR 1486 1401 84 1 1486 95773
|
|
10 Raft 3114 2093 321 698 3112 341220
|
|
11 VSR 456 302 153 1 456 33396
|
|
11 Raft 3441 1209 375 1855 3439 345485
|
|
12 VSR 1220 1128 88 2 1218 66620
|
|
12 Raft 2555 1064 458 1031 2553 370909
|
|
13 VSR 863 753 108 1 862 52065
|
|
13 Raft 1965 1210 494 259 1963 162897
|
|
14 VSR 645 524 115 5 644 44976
|
|
14 Raft 2498 1918 399 179 2496 138682
|
|
15 VSR 690 638 50 1 689 48727
|
|
15 Raft 2541 1871 381 287 2539 200612
|
|
16 VSR 944 835 106 1 942 52517
|
|
16 Raft 2369 1451 466 450 2367 465559
|
|
17 VSR 647 564 80 2 646 40272
|
|
17 Raft 2446 1846 399 199 2444 124084
|
|
18 VSR 392 310 76 4 390 35414
|
|
18 Raft 1955 923 527 503 1953 218171
|
|
19 VSR 1165 1068 97 0 1165 60283
|
|
19 Raft 2987 1849 363 773 2985 230511
|
|
20 VSR 2229 2164 62 2 2228 108081
|
|
20 Raft 2996 1742 381 871 2994 450695
|
|
21 VSR 1390 1341 44 5 1390 78704
|
|
21 Raft 2441 1485 412 542 2439 120767
|
|
22 VSR 1844 1758 81 5 1844 96250
|
|
22 Raft 3561 1897 306 1356 3559 212082
|
|
23 VSR 1393 1335 55 3 1393 79177
|
|
23 Raft 2524 1496 434 593 2523 222489
|
|
24 VSR 971 951 18 0 969 56647
|
|
24 Raft 2177 1606 412 157 2175 177816
|
|
25 VSR 458 376 77 5 458 35323
|
|
25 Raft 2507 1898 374 233 2505 139076
|
|
26 VSR 1177 1117 60 0 1177 70344
|
|
26 Raft 2583 1978 365 238 2581 186689
|
|
27 VSR 1042 937 104 1 1042 70667
|
|
27 Raft 2794 2237 355 200 2792 171007
|
|
28 VSR 1118 1067 43 6 1116 67368
|
|
28 Raft 2790 1870 357 561 2788 257457
|
|
29 VSR 683 610 73 0 683 45166
|
|
29 Raft 2557 1439 408 708 2555 110388
|
|
30 VSR 1202 1134 63 3 1200 64864
|
|
30 Raft 3312 2297 316 697 3310 406253
|
|
31 VSR 1466 1399 65 2 1466 83025
|
|
31 Raft 2818 1612 377 827 2816 311506
|
|
32 VSR 1207 1161 41 5 1207 68520
|
|
32 Raft 2676 1141 416 1117 2674 255671
|
|
33 VSR 1219 1134 83 1 1218 65923
|
|
33 Raft 3260 2844 274 140 3258 191445
|
|
34 VSR 1255 1229 24 1 1254 71120
|
|
34 Raft 3311 1372 405 1532 3309 428477
|
|
35 VSR 1197 1142 50 4 1196 73779
|
|
35 Raft 2435 1793 418 222 2433 147736
|
|
36 VSR 1560 1500 55 4 1559 89849
|
|
36 Raft 2110 1186 515 407 2108 310505
|
|
37 VSR 1487 1424 58 4 1486 148383
|
|
37 Raft 3223 1407 335 1479 3221 242814
|
|
38 VSR 1397 1349 42 5 1396 76827
|
|
38 Raft 2466 1260 441 763 2464 352797
|
|
39 VSR 1304 1249 53 1 1303 78321
|
|
39 Raft 3027 2062 343 620 3025 334793
|
|
40 VSR 1299 1270 25 2 1297 72474
|
|
40 Raft 2183 1213 479 489 2181 229529
|
|
41 VSR 1091 1040 41 8 1089 65193
|
|
41 Raft 2856 2333 339 182 2854 162726
|
|
42 VSR 496 410 84 1 495 36493
|
|
42 Raft 2421 1797 400 222 2419 163015
|
|
43 VSR 658 562 92 2 656 46387
|
|
43 Raft 2834 2134 362 336 2832 161248
|
|
44 VSR 1372 1318 49 4 1371 75563
|
|
44 Raft 2654 2076 370 206 2652 262333
|
|
45 VSR 257 127 129 0 256 33593
|
|
45 Raft 3146 1319 366 1459 3144 395077
|
|
46 VSR 1515 1458 53 2 1513 105709
|
|
46 Raft 3320 1456 382 1480 3318 444523
|
|
47 VSR 1057 1017 39 0 1056 61117
|
|
47 Raft 2243 1351 467 423 2241 210836
|
|
48 VSR 1195 1140 51 3 1194 69487
|
|
48 Raft 2378 1708 422 246 2376 148188
|
|
49 VSR 950 876 71 2 949 55394
|
|
49 Raft 2984 2444 309 229 2982 153760
|
|
50 VSR 964 875 88 1 964 57668
|
|
50 Raft 2889 2295 344 248 2887 182648
|
|
51 VSR 1302 1250 49 1 1300 74028
|
|
51 Raft 2363 1734 415 212 2361 127568
|
|
52 VSR 1205 1111 91 2 1204 64727
|
|
52 Raft 2905 2380 303 221 2904 170346
|
|
53 VSR 869 744 124 0 868 95545
|
|
53 Raft 2203 964 444 794 2202 306208
|
|
54 VSR 408 333 72 3 408 36481
|
|
54 Raft 2113 1409 434 268 2111 103004
|
|
55 VSR 494 443 50 1 494 40161
|
|
55 Raft 2618 1937 375 305 2617 239299
|
|
56 VSR 1300 1231 66 2 1299 70686
|
|
56 Raft 3401 2723 282 394 3399 194017
|
|
57 VSR 238 145 93 0 238 34783
|
|
57 Raft 1941 1255 451 233 1939 90352
|
|
58 VSR 922 854 63 5 922 54772
|
|
58 Raft 2173 1303 448 420 2171 225395
|
|
59 VSR 1941 1853 84 3 1940 103258
|
|
59 Raft 2501 1237 420 842 2499 214547
|
|
60 VSR 1766 1645 114 5 1764 86384
|
|
60 Raft 2909 1062 437 1408 2907 446530
|
|
61 VSR 1630 1557 68 4 1629 76775
|
|
61 Raft 2956 2258 333 363 2954 199725
|
|
62 VSR 1077 1023 51 2 1076 107542
|
|
62 Raft 3418 1993 303 1120 3416 178317
|
|
63 VSR 2158 2107 47 3 2157 107086
|
|
63 Raft 2245 1039 473 731 2243 331394
|
|
64 VSR 1804 1743 53 7 1803 101345
|
|
64 Raft 2804 2152 325 325 2802 206594
|
|
65 VSR 934 825 106 1 932 77299
|
|
65 Raft 2279 1090 486 701 2277 266669
|
|
66 VSR 798 726 69 2 797 59313
|
|
66 Raft 2105 1135 444 524 2103 246688
|
|
67 VSR 965 874 91 0 965 71117
|
|
67 Raft 2268 1598 439 229 2266 103555
|
|
68 VSR 438 327 109 1 437 54528
|
|
68 Raft 2787 1755 395 635 2785 347905
|
|
69 VSR 922 869 52 1 922 61234
|
|
69 Raft 2588 1752 425 409 2586 218784
|
|
70 VSR 720 607 110 3 720 47228
|
|
70 Raft 2424 1495 414 513 2422 255104
|
|
71 VSR 1067 982 81 4 1067 62847
|
|
71 Raft 2207 1583 398 224 2205 165271
|
|
72 VSR 673 561 110 0 671 48927
|
|
72 Raft 2927 2052 348 525 2925 145377
|
|
73 VSR 1361 1270 88 2 1360 70464
|
|
73 Raft 2798 1976 353 467 2796 131273
|
|
74 VSR 1891 1842 43 5 1890 104058
|
|
74 Raft 2176 1640 405 129 2174 123942
|
|
75 VSR 970 879 91 0 970 86026
|
|
75 Raft 2456 879 491 1084 2454 380910
|
|
76 VSR 851 737 111 2 850 55131
|
|
76 Raft 3226 1945 343 936 3224 158287
|
|
77 VSR 610 474 133 2 609 40458
|
|
77 Raft 2492 1882 387 221 2490 134293
|
|
78 VSR 1269 1232 31 6 1269 69782
|
|
78 Raft 3411 1575 329 1506 3410 238137
|
|
79 VSR 921 859 62 0 921 55960
|
|
79 Raft 3008 2516 335 155 3006 176544
|
|
80 VSR 1271 1230 36 5 1271 77391
|
|
80 Raft 2605 1033 426 1144 2603 287940
|
|
81 VSR 835 708 124 2 834 52299
|
|
81 Raft 2582 1845 361 374 2580 140619
|
|
82 VSR 1024 913 109 1 1023 58016
|
|
82 Raft 2921 2264 330 325 2919 143004
|
|
83 VSR 922 802 120 0 922 51388
|
|
83 Raft 2600 1885 412 301 2598 129191
|
|
84 VSR 1533 1480 52 1 1533 79724
|
|
84 Raft 2344 1753 440 149 2342 129605
|
|
85 VSR 1650 1588 58 4 1650 88562
|
|
85 Raft 3366 1419 355 1590 3364 379641
|
|
86 VSR 884 763 118 1 882 49372
|
|
86 Raft 2736 1258 457 1019 2734 552128
|
|
87 VSR 1072 1003 68 0 1071 59323
|
|
87 Raft 1875 968 506 399 1873 253771
|
|
88 VSR 1493 1437 52 4 1493 92016
|
|
88 Raft 1902 1285 445 170 1900 109923
|
|
89 VSR 1294 1239 53 0 1292 69379
|
|
89 Raft 1922 1204 471 245 1920 91616
|
|
90 VSR 1215 1119 93 2 1214 67086
|
|
90 Raft 3440 2057 296 1085 3438 284840
|
|
91 VSR 2176 2142 28 6 2176 108769
|
|
91 Raft 2606 2025 358 221 2604 136452
|
|
92 VSR 594 550 42 0 592 43714
|
|
92 Raft 2563 1504 432 625 2561 370793
|
|
93 VSR 775 699 76 0 775 67757
|
|
93 Raft 1990 826 509 653 1988 320932
|
|
94 VSR 546 455 88 3 546 35972
|
|
94 Raft 2449 1686 399 362 2447 216855
|
|
95 VSR 420 326 94 0 420 33058
|
|
95 Raft 2506 1891 379 234 2504 141706
|
|
96 VSR 583 509 70 4 583 40050
|
|
96 Raft 2307 1263 473 569 2305 325134
|
|
97 VSR 1991 1965 21 3 1989 95473
|
|
97 Raft 2679 1360 397 920 2677 281323
|
|
98 VSR 1198 1123 73 2 1198 78181
|
|
98 Raft 2550 1467 467 614 2548 416874
|
|
99 VSR 954 884 67 3 954 72020
|
|
99 Raft 2051 1356 481 212 2049 102556
|
|
100 VSR 1426 1319 105 1 1425 76369
|
|
100 Raft 2817 1402 440 973 2815 474134
|
|
-----------------------------------------------------------------------
|
|
|
|
Summary Statistics
|
|
|
|
VSR (100 completed runs, 0 violations):
|
|
Avg invocations: 1132.8
|
|
Avg ok: 1057.4
|
|
Avg timeouts: 72.3
|
|
Avg rejected: 2.3
|
|
Avg steps: 69089.2
|
|
Total invocations: 113278
|
|
Total ok: 105742
|
|
Total timeouts: 7226
|
|
Total rejected: 234
|
|
OK rate: 93.3%
|
|
Timeout rate: 6.4%
|
|
Rejection rate: 0.2%
|
|
|
|
Raft (100 completed runs, 0 violations):
|
|
Avg invocations: 2645.8
|
|
Avg ok: 1664.9
|
|
Avg timeouts: 397.6
|
|
Avg rejected: 581.4
|
|
Avg steps: 234875.8
|
|
Total invocations: 264582
|
|
Total ok: 166491
|
|
Total timeouts: 39755
|
|
Total rejected: 58142
|
|
OK rate: 62.9%
|
|
Timeout rate: 15.0%
|
|
Rejection rate: 22.0%
|