Phase M2: Remote fetch (3-node cluster)

USE_REMOTE_FETCH (getMappedRange) against SCAN_AND_FETCH on master (fa8725f1dcf6ff3ac66825ab6e3e819efbc0abcc), from one dedicated client host to three FoundationDB 7.3.77 server hosts (double redundancy, ssd-2, 9 storage servers). Both methods run in one master binary per benchmark. This is phase M2 (M2rl and M2skip) of fdbmn-20261020.

Cases and shard placement

• M2rl (BenchmarkIndexFetch): small (unsplit) and split (100,000-byte padding, two chunks) records × limit1, rows10, rows1000 × forward/reverse: 12 cases.
• M2skip (BenchmarkPlanSkip): Skip 3 × rows10, rows1000 × forward/reverse: 4 cases.
• Fixtures: 1,000 Orders records, VALUE index price_idx, no record versions. Store prefixes are 19 bytes (M2rl) and 22 bytes (M2skip).
• Placement: f is the fraction of fetched records whose storage-server team is disjoint from the index shard’s team. f = 1 means disjoint storage processes, not disjoint machines: with two replicas on three machines, any two teams share a machine.
  • Placement attempt 3 was selected by the preregistered rule (the first attempt where every cell passes, otherwise attempt 3).
  • Small and planskip stores: f = 1 on every cell.
  • Split store: f = 1 on every cell except rows1000, which is partially colocated at f = 0.820.
• Data distribution stayed off for all of M2. Placement receipts taken before and after each eligible case matched.

Exclusions

Four of 16 cases are excluded whole, each by the pre-arm client CPU gate (15% on any single benchmark CPU, 5% aggregate):

All twelve other cases have their complete six-block population and a clean R4 telemetry reconciliation. No case was rerun.

Results

Geometric mean of six paired block ratios (median of three testing.B repeats per process, 3 s benchtime), with nominal 95% block-bootstrap intervals (20,000 resamples, preregistered seed 492817). Below 1 means USE_REMOTE_FETCH is faster or allocates less. The metric is mean ns/op, not a latency percentile.

M2rl: BenchmarkIndexFetch (USE_REMOTE_FETCH / SCAN_AND_FETCH)

M2skip: BenchmarkPlanSkip (USE_REMOTE_FETCH / SCAN_AND_FETCH)

All rows/op and prefix-B ratios in both phases are 1.000 [1.000, 1.000]; they are audit quantities, not results.

• Every 1,000-row scan is slower with USE_REMOTE_FETCH: small rows1000 by about 21% in both directions, Skip rows1000 by 15–22%, and the partially colocated split rows1000 reverse at 2.304.
• On small rows1000, USE_REMOTE_FETCH allocates about 44% fewer Go bytes and 52% fewer Go objects per operation.
• One-row small fetches are 2–3% faster with USE_REMOTE_FETCH (0.976 and 0.974).

Absolute values

Medians of each process’s three testing.B means, per arm.

Comparison with the 2026-10-14 campaign

The 2026-10-14 M2 ran both methods on old Go (267a41434); this campaign runs both on master. No old-versus-master M2 arm was measured in either campaign, so the table compares two cohorts and is descriptive only: no blocks are pooled and no cross-campaign interval is computed.

• Placement changed for two cases. split-rows10-reverse moved from f = 0 to f = 1 and split-rows1000-reverse from f = 0.696 to f = 0.820 (seven record shards then, six now). Their ratios are not a comparison of code.
• Reverse Skip 10 rows is labelled cross-server in both cohorts, but the physical teams differ: index and record teams both occupied the node1/node2 pair on 2026-10-14, and span node1/node3 and node2/node3 now. Absolute ordinary/remote times were 31.127 ms / 4.873 ms then and 4.498 ms / 5.259 ms now.
• Matched properties: the same frozen fixture schema, key schema and store prefixes; the same 12-case and 4-case geometry, method orders, repeats and benchtime; GOMAXPROCS=8 on CPUs 8–15. Key counts match (2,000 small, 3,000 split and 2,000 planskip keys); serialized value bytes were not compared.
• Different: fresh hosts, freshly seeded data, different shard and replica layouts in all three stores, and the production revision (old Go then, master now).