Conductor Startup

↑ Back to index

This scenario measures Holochain conductor startup time when many cells are installed. The agent starts an empty conductor, installs a configurable number of apps without enabling them (disabled cells), restarts the conductor, and then enables cells one by one — timing each enable call and periodically restarting the conductor to measure startup time as the share of enabled cells grows.

Started
Thu, 01 Oct 2026 03:58:30 UTC
Duration
900s
Peer count
150
Behaviours
  • default (1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
conductor_startup_36809382471_1
Initial conductor startup timing
Startup time with an empty conductor before any cells are installed.
mean
4.265917
s
SD
2.017388
s
p50
3.943908
s
<
p95
8.091524
s
<
p99
8.743847
s
Disabled-cell conductor startup timing
Startup time after all configured cells are installed but remain disabled.
mean
3.422043
s
SD
1.499264
s
p50
3.509555
s
<
p95
6.237261
s
<
p99
9.928432
s
Progressive conductor startup timing
Startup time sampled periodically as more installed cells are enabled.
mean
16.089336
s
SD
7.742565
s
p50
15.532381
s
<
p95
28.909978
s
<
p99
29.796397
s
Cell enable timing
Time to enable one previously disabled cell via the admin interface.
mean
0.325243
s
SD
0.200348
s
p50
0.29828
s
<
p95
0.638132
s
<
p99
0.951547
s
Installed cells
Number of cells installed on the conductor during setup.
500
cells
Maximum enabled cells
Highest percentage of installed cells enabled during the run.
100
%
Holochain MetricsClick to toggle visibility.
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
1
over 594.840710094s
mean rate
0.09
/s
std
0.11
/s
p5
0
/s
p95
0.25
/s
peak
0.25
/s
WASM usage: 'callback_integrity::genesis_self_check_2'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 445.524736459s
Not enough time data to show a trend.
WASM call duration: 'callback_integrity::genesis_self_check_2'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.002133
s
SD
0.001094
s
p50
0.002172
s
<
p95
0.00414
s
<
p99
0.005794
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
1.76
s
<
mean
140.62
s
SD
178.88
s
<
p95
574.17
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
15
over 831.948182797s
mean rate
48.26
/s
std
395.96
/s
p5
0
/s
p95
190.81
/s
peak
11096.14
/s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
6.318209
s
SD
4.363827
s
p50
5.98529
s
<
p95
11.176537
s
<
p99
19.723983
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.013482
s
SD
0.0437
s
p50
0.001529
s
<
p95
0.009337
s
<
p99
0.174627
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
1e-06
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.079005
s
SD
0.068127
s
p50
0.001955
s
<
p95
0.169242
s
<
p99
0.224018
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.035042
s
SD
0.043218
s
p50
0.002281
s
<
p95
0.014109
s
<
p99
0.040292
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
1.17741
s
SD
1.138023
s
p50
0.002638
s
<
p95
3.336989
s
<
p99
5.01036
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
0.00816
s
SD
0.039086
s
p50
0.000705
s
<
p95
0.004892
s
<
p99
0.166945
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.00083
s
SD
0.000381
s
p50
0.000278
s
<
p95
0.00108
s
<
p99
0.001288
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.003792
s
SD
0.004496
s
p50
0.002176
s
<
p95
0.005647
s
<
p99
0.01975
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000565
s
SD
0.001218
s
p50
0.000497
s
<
p95
0.003608
s
<
p99
0.005345
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
0.071589
s
SD
0.057794
s
p50
0.045095
s
<
p95
0.141801
s
<
p99
0.229242
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.09793
s
SD
0.049292
s
p50
0.093797
s
<
p95
0.220382
s
<
p99
0.244065
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
42.58
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
2.50
KiB/s
<
mean
352.44
KiB/s
SD
644.22
KiB/s
<
p95
1.32
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
159
B/s
<
mean
46.37
KiB/s
SD
43.29
KiB/s
<
p95
119.82
KiB/s
Total bytes received
Total bytes received on primary network interface
count
5.51
GiB
mean
6.33
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
741.70
MiB
mean
853.37
KiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected

Warning CPU p99 reached 96.9%

Memory leak anomaly
Memory leak anomaly was detected
Detected

Warning Memory growing at 1.24 MiB/s

Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
Detected

Warning 11% of hosts overloaded (load5/ncpus > 1.0)

CPU usage
Total CPU usage and kernel CPU usage
Total
p5
5.64
%
<
mean
63.97
%
SD
28.69
%
<
p95
96.27
%
System
21.38
%
CPU percentiles
CPU usage percentiles
p50
75.45
%
p95
96.27
%
p99
96.90
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
34
hosts
mean time
321.24
s
Memory used percentage
Percentage of memory used
p5
6.07
%
<
mean
13.24
%
SD
5.75
%
<
p95
22.12
%
Memory available percentage
Percentage of available memory
p5
77.88
%
<
mean
86.76
%
SD
5.75
%
<
p95
93.93
%
Max host memory used
Maximum memory usage percentage across all hosts
max
36.18
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
9.29
%
Memory growth rate
Rate of memory growth over time
growth
1.24
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.09
MiB/s
Disk write throughput
Disk write throughput in MiB/s
57.45
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point /
0/49
hosts
Mount Point /etc/hostname
0/10
hosts
Mount Point /etc/hosts
0/10
hosts
Mount Point /etc/nomad.d/client-meta.json
0/3
hosts
Mount Point /etc/resolv.conf
0/10
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
2.92
5 min
1.75
15 min
0.95
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
11.29
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
1.78
%
<
mean
35.3084
%
SD
30.3211
%
<
p95
97.56
%
60 second average
29.4637
%
300 second average
17.9624
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
1.0643
%
SD
1.8651
%
<
p95
5.82
%
60 second average
0.9252
%
300 second average
0.5365
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
0.2377
%
SD
0.6356
%
<
p95
1.22
%
60 second average
0.2188
%
300 second average
0.1508
%
Holochain process CPU usage
CPU usage by Holochain process
p5
15.56
%
<
mean
50.29
%
SD
23.01
%
<
p95
89.38
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
23.12
KiB
<
mean
382.34
KiB
SD
343.69
KiB
<
p95
1.13
MiB
Holochain process threads
Number of threads in Holochain process
p5
5
threads
<
mean
623.08
threads
SD
683.11
threads
<
p95
1884
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
8
file descriptors
<
mean
1421.36
file descriptors
SD
1502.26
file descriptors
<
p95
4096
file descriptors

DHT Sync Lag

↑ Back to index

Measures the lag time between an agent publishing data and other peers being able to see it. This scenario has two roles:

  • write: Creates entries with a timestamp field, linked to a known base hash so that record_lag agents can discover them.
  • record_lag: Repeatedly queries for links from the known base hash. When a new entry is discovered, it calculates the time difference between the entry's timestamp and the current time, giving the sync lag for that entry.
Started
Thu, 01 Oct 2026 04:07:43 UTC
Duration
1800s
Peer count
150
Peer count at end
135
Behaviours
  • record_lag (1 agent)
  • write (1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
dht_sync_lag_36809382471_1
Create rate
The average number of created records per agent.
Mean
mean
14.15
/s
Max
Highest per-partition mean rate
max
38.26
/s
Min
Lowest per-partition mean rate
min
1.72
/s
Sync lag timing
The average time between when a record was created and when it is first seen by an agent.
Mean
mean
1.2666917437300001e+30
s
SD
5.03339309722e+30
s
Max
Highest per-partition mean latency
max mean
2.0267067899700002e+31
s
Min
Lowest per-partition mean latency
min mean
4.72852
s
Sync lag rate
The average number of created records discovered per agent.
Mean
mean
28.54
/s
Max
Highest per-partition mean rate
max
105
/s
Min
Lowest per-partition mean rate
min
0
/s
Delivery ratio
The proportion of created entries successfully received by all record_lag agents.
0.0015
Sent count
The total number of entries sent (created) by write agents, by agent.
Total
327876
Agents affected
123 / 123
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
2666
Max
Highest count in any single partition
6882
Min
Lowest count in any single partition
166
Received count
The total number of entries received by record_lag agents, by agent.
Total
10110
Agents affected
14 / 20
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
506
Max
Highest count in any single partition
1996
Min
Lowest count in any single partition
0
Error count
The number of errors encountered during the scenario.
35
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.002453
s
SD
0.001654
s
p50
0.001963
s
<
p95
0.005832
s
<
p99
0.007175
s
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
1
over 1691.447883996s
mean rate
61.36
/s
std
155.7
/s
p5
0
/s
p95
407.95
/s
peak
854.4
/s
WASM usage: 'timed::created_timed_entry'
Metered usage count of the WASM ribosome for this zome function.
total
123484800
over 1789.367980687s
mean rate
4.5840276929e+09
/s
std
4.8013720333e+10
/s
p5
3.491407473e+07
/s
p95
1.42178551884e+10
/s
peak
3.9565704287e+12
/s
WASM usage: 'timed::get_timed_entries_local'
Metered usage count of the WASM ribosome for this zome function.
total
324147684
over 1776.788212268s
mean rate
1.52446331457e+10
/s
std
1.77026729315e+10
/s
p5
1.1639512993e+08
/s
p95
5.39372438236e+10
/s
peak
6.34375475115e+10
/s
WASM usage: 'timed_integrity::entry_defs'
Metered usage count of the WASM ribosome for this zome function.
total
19824309
over 1789.396132682s
mean rate
1.06033343807e+09
/s
std
1.0618311063e+10
/s
p5
6.39791482e+06
/s
p95
2.16054247408e+09
/s
peak
5.33654855643e+11
/s
Zome call duration: 'timed::created_timed_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.023028
s
SD
0.07395
s
p50
0.020058
s
<
p95
0.080047
s
<
p99
0.124186
s
Zome call duration: 'timed::get_timed_entries_local'
Duration of this zome call as measured by Holochain internally.
mean
0.108512
s
SD
0.257563
s
p50
0.062441
s
<
p95
0.721791
s
<
p99
1.364878
s
WASM call duration: 'timed::created_timed_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.021579
s
SD
0.071397
s
p50
0.018152
s
<
p95
0.077086
s
<
p99
0.120955
s
WASM call duration: 'timed::get_timed_entries_local'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.106998
s
SD
0.257667
s
p50
0.059343
s
<
p95
0.720273
s
<
p99
1.36297
s
WASM call duration: 'timed_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.00198
s
SD
0.005564
s
p50
0.00199
s
<
p95
0.003872
s
<
p99
0.016242
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.005503
s
SD
0.023858
s
p50
0.003782
s
<
p95
0.0251
s
<
p99
0.042287
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.005349
s
SD
0.026493
s
p50
0.003633
s
<
p95
0.024752
s
<
p99
0.041776
s
Host function call duration: '__hc__get_1'
Duration of this host function call invoked from within WASM.
mean
0.002418
s
SD
0.00148
s
p50
0.002184
s
<
p95
0.005534
s
<
p99
0.006621
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.005718
s
SD
0.007361
s
p50
0.004454
s
<
p95
0.024385
s
<
p99
0.029422
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.004415
s
SD
0.008958
s
p50
0.004275
s
<
p95
0.012308
s
<
p99
0.030216
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.001775
s
SD
0.008027
s
p50
0.001281
s
<
p95
0.006912
s
<
p99
0.011789
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
86.14
s
<
mean
870.38
s
SD
504.3
s
<
p95
1657.52
s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
77.901172
s
SD
82.576626
s
p50
6.54915
s
<
p95
219.152689
s
<
p99
458.742562
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.230318
s
SD
1.743745
s
p50
0.176995
s
<
p95
2.451048
s
<
p99
5.145885
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.245311
s
SD
0.438533
s
p50
0.155916
s
<
p95
1.407898
s
<
p99
1.735436
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
2.247444
s
SD
3.015926
s
p50
1.195461
s
<
p95
8.408695
s
<
p99
10.896409
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
20.137184
s
SD
60.072066
s
p50
26.745916
s
<
p95
184.926564
s
<
p99
263.289723
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
17.086304
s
SD
41.14041
s
p50
0.550869
s
<
p95
109.476741
s
<
p99
208.826737
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.004356
s
SD
0.004469
s
p50
0.004862
s
<
p95
0.011686
s
<
p99
0.02168
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.007085
s
SD
0.009585
s
p50
0.004052
s
<
p95
0.019207
s
<
p99
0.042089
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.005324
s
SD
0.03466
s
p50
0.002965
s
<
p95
0.025906
s
<
p99
0.07117
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
4.845467
s
SD
27.88761
s
p50
3.4532
s
<
p95
60.042824
s
<
p99
60.274508
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
1.748904
s
SD
26.769152
s
p50
4.22399
s
<
p95
60.040293
s
<
p99
64.471982
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
59.61
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
82.10
KiB/s
<
mean
617.56
KiB/s
SD
513.86
KiB/s
<
p95
1.63
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
6.51
KiB/s
<
mean
139.24
KiB/s
SD
192.97
KiB/s
<
p95
475.81
KiB/s
Total bytes received
Total bytes received on primary network interface
count
148.03
GiB
mean
84.64
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
33.33
GiB
mean
19.06
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected

Warning CPU p99 reached 98.0%

Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected

Critical Heavy swap usage (74.7% swap used)

System overload anomaly
System overload anomaly was detected
Detected

Warning 92% of hosts overloaded (load5/ncpus > 1.0)

CPU usage
Total CPU usage and kernel CPU usage
Total
p5
41.09
%
<
mean
80.67
%
SD
18.23
%
<
p95
96.14
%
System
21.07
%
CPU percentiles
CPU usage percentiles
p50
86.70
%
p95
96.14
%
p99
98.05
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
138
hosts
mean time
1336.99
s
Memory used percentage
Percentage of memory used
p5
8.45
%
<
mean
13.56
%
SD
8.17
%
<
p95
26.89
%
Memory available percentage
Percentage of available memory
p5
73.11
%
<
mean
86.44
%
SD
8.17
%
<
p95
91.55
%
Max host memory used
Maximum memory usage percentage across all hosts
max
86.43
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
74.72
%
Memory growth rate
Rate of memory growth over time
growth
0.48
MiB/s
Disk read throughput
Disk read throughput in MiB/s
5.19
MiB/s
Disk write throughput
Disk write throughput in MiB/s
159.79
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point /
0/116
hosts
Mount Point /efi-boot
0/2
hosts
Mount Point /etc/hostname
0/14
hosts
Mount Point /etc/hosts
0/14
hosts
Mount Point /etc/nomad.d/client-meta.json
0/13
hosts
Mount Point /etc/resolv.conf
0/14
hosts
Mount Point /nix/store
0/2
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
4.04
5 min
3.63
15 min
2.64
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
91.61
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
1.71
%
<
mean
49.2287
%
SD
21.5854
%
<
p95
76.25
%
60 second average
48.0569
%
300 second average
42.9385
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0.0063
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0001
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0.0063
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
9.7127
%
SD
7.3721
%
<
p95
19.48
%
60 second average
9.4981
%
300 second average
8.3508
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
3.0145
%
SD
3.8504
%
<
p95
8.72
%
60 second average
2.9759
%
300 second average
2.7432
%
Holochain process CPU usage
CPU usage by Holochain process
p5
33.52
%
<
mean
74.09
%
SD
19.6
%
<
p95
92.74
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
234.82
KiB
<
mean
442.72
KiB
SD
162.40
KiB
<
p95
685.46
KiB
Holochain process threads
Number of threads in Holochain process
p5
23
threads
<
mean
44.28
threads
SD
129.23
threads
≤
p95
44
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
62
file descriptors
<
mean
103.57
file descriptors
SD
281.98
file descriptors
≤
p95
92
file descriptors

Full-Arc Create (Validated) / Zero-Arc Read

↑ Back to index

A full-arc/zero-arc mixed scenario where full-arc nodes create data that gets validated and zero-arc nodes read the data. The scenario has two roles:

  • full: A full-arc conductor that creates entries with a timestamp field. Those entries get validated and then retrieved by zero-arc nodes.
  • zero: A zero-arc conductor that reads the entries created by the full-arc node(s) and records the time lag between when the entry had been created and when it was first discovered.
Started
Thu, 01 Oct 2026 07:37:49 UTC
Duration
900s
Peer count
150
Peer count at end
126
Behaviours
  • full (1 agent)
  • zero (1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
full_arc_create_validated_zero_arc_read_36809382471_1
Create rate
The number of timed entries created by the full-arc node(s) per second.
Mean
mean
21.62
/s
Max
Highest per-partition mean rate
max
63.51
/s
Min
Lowest per-partition mean rate
min
8.05
/s
Fetch lag timing
For each entry, the time lag between when it was created by a full-arc node and when a zero-arc node fetched it from the network.
Mean
mean
176.509178
s
SD
7.85528
s
Max
Highest per-partition mean latency
max mean
1113.767209
s
Min
Lowest per-partition mean latency
min mean
0.273387
s
Fetch rate
The number of entries per second fetched by zero-arc nodes.
Mean
mean
35.65
/s
Max
Highest per-partition mean rate
max
220
/s
Min
Lowest per-partition mean rate
min
0
/s
Delivery ratio
The proportion of created entries successfully fetched by all zero-arc nodes.
0.0022
Entry created count
Total number of entries created by full-arc nodes, by agent.
Total
42184
Agents affected
21 / 21
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
2009
Max
Highest count in any single partition
5887
Min
Lowest count in any single partition
682
Received count
Total number of entries fetched by zero-arc nodes, by agent.
Total
11115
Agents affected
21 / 121
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
92
Max
Highest count in any single partition
3399
Min
Lowest count in any single partition
0
Open connections
The number of currently open connections to other conductors.
full-arc
p5
1
<
mean
80.81
SD
43.58
<
p95
133
zero-arc
p5
1
<
mean
12.48
SD
12.49
<
p95
42
Retrieval errors
Errors encountered by zero-arc nodes when attempting to retrieve entries created by full-arc nodes.
Total
1588
Agents affected
119 / 121
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
13
Max
Highest count in any single partition
15
Min
Lowest count in any single partition
0
Error count
The number of errors accumulated across all nodes.
1590
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.166231
s
SD
3.705332
s
p50
1.746265
s
<
p95
10.710938
s
<
p99
17.060799
s
Zome call duration: 'timed_and_validated::create_timed_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.017894
s
SD
0.020564
s
p50
0.017639
s
<
p95
0.069236
s
<
p99
0.097488
s
Zome call duration: 'timed_and_validated::get_timed_entries_network'
Duration of this zome call as measured by Holochain internally.
mean
2.353453
s
SD
135.356675
s
p50
68.777674
s
<
p95
378.6634
s
<
p99
540.421642
s
WASM call duration: 'timed_and_validated::create_timed_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.016763
s
SD
0.019962
s
p50
0.016082
s
<
p95
0.066973
s
<
p99
0.094587
s
WASM call duration: 'timed_and_validated::get_timed_entries_network'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
4.037797
s
SD
147.74239
s
p50
95.025728
s
<
p95
490.020501
s
<
p99
567.388115
s
WASM call duration: 'timed_and_validated_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001685
s
SD
0.000766
s
p50
0.001943
s
<
p95
0.002699
s
<
p99
0.003803
s
WASM call duration: 'timed_and_validated_integrity::validate'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.012092
s
SD
0.006256
s
p50
0.001849
s
<
p95
0.016114
s
<
p99
0.034166
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.00423
s
SD
0.007495
s
p50
0.003339
s
<
p95
0.023418
s
<
p99
0.034015
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.003936
s
SD
0.006834
s
p50
0.003174
s
<
p95
0.020991
s
<
p99
0.031289
s
Host function call duration: '__hc__get_1'
Duration of this host function call invoked from within WASM.
mean
0.184397
s
SD
4.213664
s
p50
2.229972
s
<
p95
12.075568
s
<
p99
19.447145
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.261689
s
SD
1.778367
s
p50
0.803335
s
<
p95
4.876474
s
<
p99
9.449561
s
Host function call duration: '__hc__must_get_valid_record_1'
Duration of this host function call invoked from within WASM.
mean
0.013043
s
SD
0.00752
s
p50
0.009965
s
<
p95
0.027886
s
<
p99
0.034978
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.003698
s
SD
0.001769
s
p50
0.003231
s
<
p95
0.006515
s
<
p99
0.009953
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.001178
s
SD
0.000968
s
p50
0.000226
s
<
p95
0.0025
s
<
p99
0.00491
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
44.55
s
<
mean
427.22
s
SD
249.31
s
<
p95
822.33
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 889.179603087s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
102.348011
s
SD
31.327805
s
p50
0
s
<
p95
0.14478
s
<
p99
136.496991
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
2.895207
s
SD
3.653965
s
p50
0.001688
s
<
p95
4.108393
s
<
p99
18.231438
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.372078
s
SD
0.208973
s
p50
0.001977
s
<
p95
0.521385
s
<
p99
1.139953
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.244314
s
SD
0.101864
s
p50
0.020274
s
<
p95
0.291169
s
<
p99
0.471134
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
11.018991
s
SD
13.495914
s
p50
0.003151
s
<
p95
32.570923
s
<
p99
67.666862
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
3.802631
s
SD
10.018597
s
p50
0.000957
s
<
p95
0.013643
s
<
p99
33.446605
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.002474
s
SD
0.001357
s
p50
0.000198
s
<
p95
0.004149
s
<
p99
0.006345
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.004327
s
SD
0.021939
s
p50
0.003134
s
<
p95
0.034703
s
<
p99
0.082505
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.003905
s
SD
0.003355
s
p50
0.000649
s
<
p95
0.00505
s
<
p99
0.022217
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
2.779087
s
SD
30.111302
s
p50
0.962057
s
<
p95
60.003529
s
<
p99
62.184085
s
P2P request roundtrip: 'get_links'
Time spent sending a get_links request and awaiting its response.
mean
0.429551
s
SD
6.905431
s
p50
0.560645
s
<
p95
3.500606
s
<
p99
60.000593
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.184276
s
SD
0.442168
s
p50
0.189107
s
<
p95
0.870581
s
<
p99
2.903702
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
14.87
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
3.83
KiB/s
<
mean
221.47
KiB/s
SD
433.40
KiB/s
<
p95
820.60
KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
473
B/s
<
mean
64.82
KiB/s
SD
213.51
KiB/s
<
p95
440.74
KiB/s
Total bytes received
Total bytes received on primary network interface
count
24.69
GiB
mean
28.41
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
7.18
GiB
mean
8.26
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected

Warning CPU p99 reached 98.2%

Memory leak anomaly
Memory leak anomaly was detected
Detected

Warning Memory growing at 1.39 MiB/s

Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected

Critical Heavy swap usage (74.7% swap used)

System overload anomaly
System overload anomaly was detected
Detected

Warning 13% of hosts overloaded (load5/ncpus > 1.0)

CPU usage
Total CPU usage and kernel CPU usage
Total
p5
1.51
%
<
mean
20.8
%
SD
32.26
%
<
p95
94.38
%
System
5.93
%
CPU percentiles
CPU usage percentiles
p50
4.09
%
p95
94.38
%
p99
98.20
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
88
hosts
mean time
185.32
s
Memory used percentage
Percentage of memory used
p5
7.18
%
<
mean
13.57
%
SD
8.17
%
<
p95
25.13
%
Memory available percentage
Percentage of available memory
p5
74.87
%
<
mean
86.43
%
SD
8.17
%
<
p95
92.82
%
Max host memory used
Maximum memory usage percentage across all hosts
max
87.47
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
74.71
%
Memory growth rate
Rate of memory growth over time
growth
1.39
MiB/s
Disk read throughput
Disk read throughput in MiB/s
2.96
MiB/s
Disk write throughput
Disk write throughput in MiB/s
84.08
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point /
0/104
hosts
Mount Point /efi-boot
0/2
hosts
Mount Point /etc/hostname
0/13
hosts
Mount Point /etc/hosts
0/13
hosts
Mount Point /etc/nomad.d/client-meta.json
0/17
hosts
Mount Point /etc/resolv.conf
0/13
hosts
Mount Point /nix/store
0/2
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
1.07
5 min
1.10
15 min
1.41
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
13.43
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
10.0046
%
SD
24.4397
%
<
p95
81.13
%
60 second average
9.4888
%
300 second average
10.1017
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
1.3124
%
SD
3.9316
%
<
p95
11.14
%
60 second average
1.2616
%
300 second average
1.3756
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
0.3204
%
SD
1.0738
%
<
p95
2.31
%
60 second average
0.3202
%
300 second average
0.4628
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.3
%
<
mean
16.03
%
SD
30.46
%
<
p95
89.21
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
269.98
KiB
<
mean
392.07
KiB
SD
90.41
KiB
<
p95
515.77
KiB
Holochain process threads
Number of threads in Holochain process
p5
14
threads
<
mean
29.89
threads
SD
10.75
threads
<
p95
39
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
44
file descriptors
<
mean
65.36
file descriptors
SD
16.48
file descriptors
<
p95
89
file descriptors

Mixed Arc get_agent_activity

↑ Back to index

A mixed-arc scenario that measures the DHT's ability to propagate agent activity and make it available via get_agent_activity calls. The scenario has three roles:

  • zero_write: A zero-arc conductor that creates entries on its source chain.
  • full_write: A full-arc conductor that creates entries on its source chain.
  • zero_read: A zero-arc conductor that picks a random writer and repeatedly calls get_agent_activity to observe new entries appearing on the writer's chain, measuring the lag between creation and discovery.
Started
Thu, 01 Oct 2026 08:13:36 UTC
Duration
900s
Peer count
150
Peer count at end
128
Behaviours
  • full_write (1 agent)
  • zero_read (1 agent)
  • zero_write (1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
mixed_arc_get_agent_activity_36809382471_1
Entry created count
The total number of entries created by write agents over the run, partitioned by agent and behaviour.
Total
437519
Agents affected
100 / 100
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
4375
Max
Highest count in any single partition
10650
Min
Lowest count in any single partition
717
Observed chain advancement
The maximum highest-observed action sequence per write agent, aggregated across all reading agents. The reading dimension is collapsed by taking the maximum observed value for each write agent (any reader successfully tracking that point counts as propagation). Captures how far readers tracked writers’ chains during the run.
Mean max (across write agents)
Mean of the highest chain head value observed per write agent. Each write agent contributes its maximum — the value seen by any reader.
2202.36
Max
Highest chain head value observed for any single write agent.
4722
Write agents observed
14
Chain head delay timing
The time lag between when a writer creates a new entry and when a reader discovers it via get_agent_activity. This is measured per new action in the chain.
Mean
mean
0.482459
s
SD
1.189655
s
Max
Highest per-partition mean latency
max mean
5.395322
s
Min
Lowest per-partition mean latency
min mean
-1.437973
s
Chain head delay rate
The rate at which readers detect new chain heads appearing on writers' chains.
Mean
mean
10.34
/s
Max
Highest per-partition mean rate
max
72.27
/s
Min
Lowest per-partition mean rate
min
1
/s
get_agent_activity_full zome call timing
The time taken to complete the get_agent_activity_full zome call that queries a writer's chain.
Mean
mean
0.917227
s
SD
2.226188
s
Max
Highest per-partition mean latency
max mean
8.238192
s
Min
Lowest per-partition mean latency
min mean
0.049427
s
Retrieval errors
A running accumulation of errors encountered by read agents when calling get_agent_activity.
Total
181
Agents affected
38 / 42
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
4
Max
Highest count in any single partition
11
Min
Lowest count in any single partition
0
Open connections
The number of currently open connections to other conductors, partitioned by agent behaviour.
  • behaviour: full_write
p5
2
<
mean
70.2
SD
40.3
<
p95
132
  • behaviour: zero_read
p5
4
<
mean
30.7
SD
16.08
<
p95
61
  • behaviour: zero_write
p5
4
<
mean
28.86
SD
15.24
<
p95
56
Final error count
The total number of errors accumulated across all agents over the run.
195
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.326747
s
SD
5.388763
s
p50
0.681816
s
<
p95
11.18455
s
<
p99
30.008963
s
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
0
over 826.546627498s
Not enough time data to show a trend.
Zome call duration: 'agent_activity::announce_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.008194
s
SD
0.00497
s
p50
0.007836
s
<
p95
0.015507
s
<
p99
0.037299
s
Zome call duration: 'agent_activity::create_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.008432
s
SD
0.014821
s
p50
0.006911
s
<
p95
0.041764
s
<
p99
0.078836
s
Zome call duration: 'agent_activity::get_agent_activity_full'
Duration of this zome call as measured by Holochain internally.
mean
0.3359
s
SD
2.772663
s
p50
0.712179
s
<
p95
7.312574
s
<
p99
15.348613
s
Zome call duration: 'agent_activity::get_random_agent_with_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.141758
s
SD
6.750538
s
p50
0.611912
s
<
p95
11.198285
s
<
p99
30.013181
s
WASM call duration: 'agent_activity::announce_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.006747
s
SD
0.00444
s
p50
0.006256
s
<
p95
0.013684
s
<
p99
0.035993
s
WASM call duration: 'agent_activity::create_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.007302
s
SD
0.01414
s
p50
0.00576
s
<
p95
0.038754
s
<
p99
0.07505
s
WASM call duration: 'agent_activity::get_agent_activity_full'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.334804
s
SD
2.772698
s
p50
0.710878
s
<
p95
7.31165
s
<
p99
15.348007
s
WASM call duration: 'agent_activity::get_random_agent_with_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.17508
s
SD
6.746675
s
p50
0.61149
s
<
p95
11.196896
s
<
p99
30.012541
s
WASM call duration: 'agent_activity_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001583
s
SD
0.00093
s
p50
0.001658
s
<
p95
0.003476
s
<
p99
0.005321
s
Host function call duration: '__hc__agent_info_1'
Duration of this host function call invoked from within WASM.
mean
5.5e-05
s
SD
3.7e-05
s
p50
4.6e-05
s
<
p95
0.000141
s
<
p99
0.000213
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.002577
s
SD
0.010163
s
p50
0.001115
s
<
p95
0.024879
s
<
p99
0.052448
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.001146
s
SD
0.002237
s
p50
0.000655
s
<
p95
0.004405
s
<
p99
0.015792
s
Host function call duration: '__hc__get_agent_activity_1'
Duration of this host function call invoked from within WASM.
mean
0.332406
s
SD
2.772557
s
p50
0.707118
s
<
p95
7.307172
s
<
p99
15.346923
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.171032
s
SD
6.746169
s
p50
0.609749
s
<
p95
11.19052
s
<
p99
30.008999
s
Host function call duration: '__hc__random_bytes_1'
Duration of this host function call invoked from within WASM.
mean
8e-06
s
SD
6e-06
s
p50
1.2e-05
s
<
p95
2.7e-05
s
<
p99
2.8e-05
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.003031
s
SD
0.002571
s
p50
0.002821
s
<
p95
0.009014
s
<
p99
0.012992
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.001148
s
SD
0.002983
s
p50
0.000584
s
<
p95
0.008013
s
<
p99
0.014868
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
45.03
s
<
mean
430.8
s
SD
249.72
s
<
p95
825.13
s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
17.861995
s
SD
26.728459
s
p50
0
s
<
p95
14.03728
s
<
p99
66.56191
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.134874
s
SD
0.460101
s
p50
0.001933
s
<
p95
1.02849
s
<
p99
2.621062
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.053731
s
SD
0.305809
s
p50
0.002185
s
<
p95
1.016947
s
<
p99
1.280094
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.134166
s
SD
0.087326
s
p50
0.094341
s
<
p95
0.262304
s
<
p99
0.326307
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
7.656407
s
SD
20.538094
s
p50
0.004368
s
<
p95
45.031946
s
<
p99
98.871745
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
0.533011
s
SD
2.226611
s
p50
0.001347
s
<
p95
0.170714
s
<
p99
16.68139
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.003554
s
SD
0.002705
s
p50
0.002264
s
<
p95
0.007467
s
<
p99
0.009293
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.006457
s
SD
0.003784
s
p50
0.00248
s
<
p95
0.012392
s
<
p99
0.017764
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.002525
s
SD
0.008628
s
p50
0.000682
s
<
p95
0.020965
s
<
p99
0.041876
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
5.95865
s
SD
27.098564
s
p50
2.703979
s
<
p95
60.033817
s
<
p99
60.046235
s
P2P request roundtrip: 'get_links'
Time spent sending a get_links request and awaiting its response.
mean
1.449366
s
SD
12.735812
s
p50
0.614111
s
<
p95
17.377437
s
<
p99
60.001832
s
P2P request roundtrip: 'get_agent_activity'
Time spent sending a get_agent_activity request and awaiting its response.
mean
0.270111
s
SD
29.255354
s
p50
0.583082
s
<
p95
60.001847
s
<
p99
60.003142
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.8371
s
SD
5.44253
s
p50
0.215646
s
<
p95
2.178053
s
<
p99
16.978294
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
42.38
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
9.17
KiB/s
<
mean
561.63
KiB/s
SD
455.02
KiB/s
<
p95
1.25
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
1.18
KiB/s
<
mean
146.86
KiB/s
SD
211.35
KiB/s
<
p95
627.48
KiB/s
Total bytes received
Total bytes received on primary network interface
count
63.36
GiB
mean
72.90
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
16.52
GiB
mean
19.01
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected

Warning CPU p99 reached 97.8%

Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected

Critical Heavy swap usage (73.7% swap used)

System overload anomaly
System overload anomaly was detected
Detected

Warning 43% of hosts overloaded (load5/ncpus > 1.0)

CPU usage
Total CPU usage and kernel CPU usage
Total
p5
1.94
%
<
mean
56.89
%
SD
37.79
%
<
p95
94.19
%
System
14.51
%
CPU percentiles
CPU usage percentiles
p50
81.25
%
p95
94.19
%
p99
97.75
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
92
hosts
mean time
682.36
s
Memory used percentage
Percentage of memory used
p5
6.77
%
<
mean
12.59
%
SD
8.35
%
<
p95
24.8
%
Memory available percentage
Percentage of available memory
p5
75.2
%
<
mean
87.41
%
SD
8.35
%
<
p95
93.23
%
Max host memory used
Maximum memory usage percentage across all hosts
max
86.44
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
73.71
%
Memory growth rate
Rate of memory growth over time
growth
0.45
MiB/s
Disk read throughput
Disk read throughput in MiB/s
2.64
MiB/s
Disk write throughput
Disk write throughput in MiB/s
112.25
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point /
0/107
hosts
Mount Point /efi-boot
0/2
hosts
Mount Point /etc/hostname
0/13
hosts
Mount Point /etc/hosts
0/13
hosts
Mount Point /etc/nomad.d/client-meta.json
0/18
hosts
Mount Point /etc/resolv.conf
0/13
hosts
Mount Point /nix/store
0/2
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
2.72
5 min
2.12
15 min
1.50
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
42.75
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
32.5578
%
SD
28.8342
%
<
p95
74.71
%
60 second average
30.9424
%
300 second average
23.9050
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
4.2357
%
SD
5.9983
%
<
p95
16.38
%
60 second average
4.0428
%
300 second average
3.1510
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
1.3725
%
SD
2.377
%
<
p95
5.32
%
60 second average
1.3117
%
300 second average
1.0203
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.55
%
<
mean
50.8
%
SD
36.4
%
<
p95
89.27
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
204.85
KiB
<
mean
288.55
KiB
SD
101.79
KiB
<
p95
443.96
KiB
Holochain process threads
Number of threads in Holochain process
p5
16
threads
<
mean
31.29
threads
SD
15.32
threads
<
p95
37
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
46
file descriptors
<
mean
76
file descriptors
SD
22.81
file descriptors
<
p95
89
file descriptors

Mixed Arc must_get_agent_activity

↑ Back to index

A scenario where write agents create entries in batches of 10, while must_get_agent_activity agents each pick a random write agent and repeatedly attempt to create an entry that references the chain top of their latest batch. This reference means that the entry's validation function needs to make a must_get_agent_activity call.

The purpose of this scenario is to measure the time it takes for published agent activity data to be gossiped among authorities and become available to peers that query it via must_get_agent_activity.

This test is similar to Write Validated must_get_agent_activity, but some write agents are zero-arc and all must_get_agent_activity agents are zero-arc. This forces the must_get_agent_activity host call to go to the network.

Started
Thu, 01 Oct 2026 08:31:34 UTC
Duration
900s
Peer count
150
Peer count at end
127
Behaviours
  • full_write (1 agent)
  • zero_must_get_agent_activity (1 agent)
  • zero_write (1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
mixed_arc_must_get_agent_activity_36809382471_1
Highest observed action_seq
The maximum chain length observed per write agent, aggregated across all reading agents. The reading dimension is collapsed by taking the maximum observed value for each write agent (any successful read counts as propagation). This reflects the DHT’s ability to propagate agent activity ops and make them available to querying peers.
Mean max (across write agents)
Mean of the highest chain head value observed per write agent. Each write agent contributes its maximum — the value seen by any reader.
446.1
Max
Highest chain head value observed for any single write agent.
1556
Write agents observed
10
Chain batch delay timing
The time between a write agent's creation of a batch and a must_get_agent_activity agent's successful discovery of the batch and creation/self-validation of a new entry that depends on it.
Mean
mean
191.177165
s
SD
107.207889
s
Max
Highest per-partition mean latency
max mean
334.696923
s
Min
Lowest per-partition mean latency
min mean
44.271
s
Chain batch delay rate
The rate at which a must_get_agent_activity agent was able to discover batches and create/self-validate new entries that depend on them.
Mean
mean
1.13
/s
Max
Highest per-partition mean rate
max
3.21
/s
Min
Lowest per-partition mean rate
min
0
/s
create_validated_sample_entry zome call timing
The time taken to complete the zome function call that creates the entry that depends on a write agent's source chain.
Mean
mean
0.561172
s
SD
0.621844
s
Max
Highest per-partition mean latency
max mean
2.063667
s
Min
Lowest per-partition mean latency
min mean
0.079789
s
Retrieval errors
A running accumulation of the errors encountered by an agent when attempting to self-validate actions that depend on must_get_agent_activity calls.
Total
122239
Agents affected
41 / 42
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
2910
Max
Highest count in any single partition
16392
Min
Lowest count in any single partition
0
Open connections
The average number of open connections maintained by an agent over the run, broken down by behaviour.
  • behaviour: full_write
p5
3
<
mean
68.91
SD
39.73
<
p95
136
  • behaviour: zero_must_get_agent_activity
p5
7
<
mean
27.59
SD
12.97
<
p95
56
  • behaviour: zero_write
p5
4
<
mean
30.64
SD
15.56
<
p95
61
Final error count
The total number of all types of error accumulated over the run by all agents.
132458
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.137663
s
SD
1.076824
s
p50
0.185335
s
<
p95
1.24301
s
<
p99
4.126518
s
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
4
over 795.945663346s
mean rate
34.65
/s
std
146.7
/s
p5
0
/s
p95
92.65
/s
peak
2331.07
/s
Zome call duration: 'validated_must_get_agent_activity::announce_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.008353
s
SD
0.003999
s
p50
0.00792
s
<
p95
0.015042
s
<
p99
0.028535
s
Zome call duration: 'validated_must_get_agent_activity::create_sample_entries_batch'
Duration of this zome call as measured by Holochain internally.
mean
0.146401
s
SD
0.140601
s
p50
0.078507
s
<
p95
0.404357
s
<
p99
0.72414
s
Zome call duration: 'validated_must_get_agent_activity::create_validated_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.005938
s
SD
0.002023
s
p50
0.006573
s
<
p95
0.008271
s
<
p99
0.010537
s
Zome call duration: 'validated_must_get_agent_activity::get_chain_top_for_batch'
Duration of this zome call as measured by Holochain internally.
mean
0.531832
s
SD
0.769658
s
p50
0.208851
s
<
p95
1.85757
s
<
p99
2.990137
s
Zome call duration: 'validated_must_get_agent_activity::get_random_agent_with_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.038845
s
SD
0.476535
s
p50
0.38507
s
<
p95
1.219974
s
<
p99
1.252635
s
WASM call duration: 'validated_must_get_agent_activity::announce_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.006773
s
SD
0.003394
s
p50
0.006395
s
<
p95
0.011356
s
<
p99
0.026532
s
WASM call duration: 'validated_must_get_agent_activity::create_sample_entries_batch'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.144784
s
SD
0.139997
s
p50
0.076962
s
<
p95
0.401434
s
<
p99
0.720756
s
WASM call duration: 'validated_must_get_agent_activity::create_validated_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.004878
s
SD
0.001733
s
p50
0.005384
s
<
p95
0.006776
s
<
p99
0.009022
s
WASM call duration: 'validated_must_get_agent_activity::get_chain_top_for_batch'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.655846
s
SD
1.837976
s
p50
0.442867
s
<
p95
2.031307
s
<
p99
11.236574
s
WASM call duration: 'validated_must_get_agent_activity::get_random_agent_with_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.037859
s
SD
0.476508
s
p50
0.384431
s
<
p95
1.218791
s
<
p99
1.251166
s
WASM call duration: 'validated_must_get_agent_activity_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001576
s
SD
0.000754
s
p50
0.00182
s
<
p95
0.002995
s
<
p99
0.004391
s
WASM call duration: 'validated_must_get_agent_activity_integrity::validate'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.025395
s
SD
0.09165
s
p50
0.005798
s
<
p95
0.116674
s
<
p99
0.271113
s
Host function call duration: '__hc__agent_info_1'
Duration of this host function call invoked from within WASM.
mean
4.8e-05
s
SD
5.5e-05
s
p50
4e-05
s
<
p95
0.000144
s
<
p99
0.000232
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.001494
s
SD
0.004599
s
p50
0.000553
s
<
p95
0.006932
s
<
p99
0.029232
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.002514
s
SD
0.00456
s
p50
0.000632
s
<
p95
0.006865
s
<
p99
0.030298
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.575833
s
SD
1.360704
s
p50
0.408474
s
<
p95
1.53452
s
<
p99
11.229389
s
Host function call duration: '__hc__must_get_agent_activity_1'
Duration of this host function call invoked from within WASM.
mean
0.110729
s
SD
0.468023
s
p50
0.095848
s
<
p95
0.568983
s
<
p99
1.978481
s
Host function call duration: '__hc__query_1'
Duration of this host function call invoked from within WASM.
mean
0.066701
s
SD
0.07029
s
p50
0.021376
s
<
p95
0.206583
s
<
p99
0.266892
s
Host function call duration: '__hc__random_bytes_1'
Duration of this host function call invoked from within WASM.
mean
1.1e-05
s
SD
9e-06
s
p50
1.1e-05
s
<
p95
3.2e-05
s
<
p99
4.6e-05
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.0028
s
SD
0.001659
s
p50
0.002945
s
<
p95
0.005623
s
<
p99
0.010141
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.001101
s
SD
0.001547
s
p50
0.000468
s
<
p95
0.003512
s
<
p99
0.008438
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
42.31
s
<
mean
424.79
s
SD
249.34
s
<
p95
818.19
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 888.856123657s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
84.253194
s
SD
37.358439
s
p50
0
s
<
p95
45.822155
s
<
p99
185.575355
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
2.304115
s
SD
1.301729
s
p50
0.002207
s
<
p95
3.0704
s
<
p99
6.643019
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.337706
s
SD
0.264065
s
p50
0.002404
s
<
p95
0.697339
s
<
p99
1.301143
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.186178
s
SD
0.209169
s
p50
0.029389
s
<
p95
0.539798
s
<
p99
0.929213
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
9.788357
s
SD
9.69167
s
p50
0.004079
s
<
p95
23.600059
s
<
p99
48.363986
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
14.270118
s
SD
19.135197
s
p50
0.001605
s
<
p95
64.173393
s
<
p99
82.619808
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.00171
s
SD
0.001427
s
p50
0.00018
s
<
p95
0.004004
s
<
p99
0.006164
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.002815
s
SD
0.005319
s
p50
0.002712
s
<
p95
0.015724
s
<
p99
0.029277
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.001799
s
SD
0.00429
s
p50
0.000485
s
<
p95
0.006647
s
<
p99
0.025381
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
2.30463
s
SD
29.552071
s
p50
0.478729
s
<
p95
60.006483
s
<
p99
60.01687
s
P2P request roundtrip: 'get_links'
Time spent sending a get_links request and awaiting its response.
mean
0.205769
s
SD
20.190043
s
p50
0.219612
s
<
p95
60.001477
s
<
p99
60.001821
s
P2P request roundtrip: 'must_get_agent_activity'
Time spent sending a must_get_agent_activity request and awaiting its response.
mean
0.148036
s
SD
28.932788
s
p50
0.146788
s
<
p95
60.001661
s
<
p99
60.00204
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.230524
s
SD
24.264639
s
p50
0.231337
s
<
p95
60.007396
s
<
p99
60.015265
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
23.48
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
6.97
KiB/s
<
mean
413.08
KiB/s
SD
394.57
KiB/s
<
p95
983.34
KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
770
B/s
<
mean
91.32
KiB/s
SD
118.78
KiB/s
<
p95
345.90
KiB/s
Total bytes received
Total bytes received on primary network interface
count
46.56
GiB
mean
53.57
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
10.16
GiB
mean
11.69
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected

Warning CPU p99 reached 96.6%

Memory leak anomaly
Memory leak anomaly was detected
Detected

Warning Memory growing at 1.21 MiB/s

Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected

Critical Heavy swap usage (72.6% swap used)

System overload anomaly
System overload anomaly was detected
Detected

Warning 27% of hosts overloaded (load5/ncpus > 1.0)

CPU usage
Total CPU usage and kernel CPU usage
Total
p5
2.08
%
<
mean
34.33
%
SD
34.01
%
<
p95
91.03
%
System
10.85
%
CPU percentiles
CPU usage percentiles
p50
15.17
%
p95
91.03
%
p99
96.63
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
98
hosts
mean time
269.83
s
Memory used percentage
Percentage of memory used
p5
7.19
%
<
mean
13.56
%
SD
8.14
%
<
p95
24.28
%
Memory available percentage
Percentage of available memory
p5
75.72
%
<
mean
86.44
%
SD
8.14
%
<
p95
92.81
%
Max host memory used
Maximum memory usage percentage across all hosts
max
87.36
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
72.62
%
Memory growth rate
Rate of memory growth over time
growth
1.21
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.51
MiB/s
Disk write throughput
Disk write throughput in MiB/s
123.40
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point /
0/111
hosts
Mount Point /efi-boot
0/2
hosts
Mount Point /etc/hostname
0/13
hosts
Mount Point /etc/hosts
0/13
hosts
Mount Point /etc/nomad.d/client-meta.json
0/20
hosts
Mount Point /etc/resolv.conf
0/13
hosts
Mount Point /nix/store
0/2
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
1.49
5 min
1.53
15 min
1.60
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
27.08
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
16.9861
%
SD
24.5491
%
<
p95
65.11
%
60 second average
16.0354
%
300 second average
16.6074
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
3.3876
%
SD
6.2024
%
<
p95
15.06
%
60 second average
3.2128
%
300 second average
2.9737
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
1.4192
%
SD
3.0785
%
<
p95
7.78
%
60 second average
1.3638
%
300 second average
1.2415
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.72
%
<
mean
29.4
%
SD
31.71
%
<
p95
83.78
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
265.89
KiB
<
mean
381.45
KiB
SD
83.98
KiB
<
p95
500.97
KiB
Holochain process threads
Number of threads in Holochain process
p5
20
threads
<
mean
30.88
threads
SD
10.86
threads
<
p95
38
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
57
file descriptors
<
mean
73.9
file descriptors
SD
17.69
file descriptors
<
p95
89
file descriptors

Remote Call Rate

↑ Back to index

Test the throughput of remote_call operations. Each agent in this scenario waits for a certain number of peers to be available or for up to two minutes, whichever happens first, before starting its behaviour.

Started
Thu, 01 Oct 2026 03:23:06 UTC
Duration
900s
Peer count
150
Behaviours
  • default (1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
remote_call_rate_36809382471_1
Dispatch timing
The time between sending a remote call and the remote handler being invoked.
Mean
mean
-5.076567
s
SD
52.550402
s
Max
Highest per-partition mean latency
max mean
1.486796
s
Min
Lowest per-partition mean latency
min mean
-54.86346
s
Round-trip timing
The total elapsed time to get a response to the client.
Mean
mean
0.509496
s
SD
1.398629
s
Max
Highest per-partition mean latency
max mean
4.820884
s
Min
Lowest per-partition mean latency
min mean
0.069342
s
Error count
The total number of errors accumulated during the run.
3149
Holochain MetricsClick to toggle visibility.
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
0
over 630.011386516s
Not enough time data to show a trend.
WASM usage: 'remote_call::call_echo_timestamp'
Metered usage count of the WASM ribosome for this zome function.
total
48094396
over 888.007834491s
mean rate
1.36081008358e+09
/s
std
7.23147995233e+09
/s
p5
1.041585099e+07
/s
p95
3.83501772759e+09
/s
peak
2.08522301166e+11
/s
WASM usage: 'remote_call::echo_timestamp'
Metered usage count of the WASM ribosome for this zome function.
total
10401314
over 888.007834491s
mean rate
2.6301811104e+08
/s
std
1.33820082816e+09
/s
p5
325107.68
/s
p95
8.1214793738e+08
/s
peak
3.69322213652e+10
/s
WASM usage: 'remote_call::init'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 888.007834491s
Not enough time data to show a trend.
Zome call duration: 'remote_call::call_echo_timestamp'
Duration of this zome call as measured by Holochain internally.
mean
0.196112
s
SD
1.895895
s
p50
0.191449
s
<
p95
3.523168
s
<
p99
7.01754
s
Zome call duration: 'remote_call::echo_timestamp'
Duration of this zome call as measured by Holochain internally.
mean
0.003289
s
SD
0.001426
s
p50
0.003766
s
<
p95
0.005499
s
<
p99
0.007132
s
WASM call duration: 'remote_call::call_echo_timestamp'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.305442
s
SD
2.911456
s
p50
0.291701
s
<
p95
7.498114
s
<
p99
13.048455
s
WASM call duration: 'remote_call::echo_timestamp'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.00207
s
SD
0.000954
s
p50
0.002389
s
<
p95
0.003707
s
<
p99
0.004912
s
WASM call duration: 'remote_call::init'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.004663
s
SD
0.003173
s
p50
0.004054
s
<
p95
0.009407
s
<
p99
0.021154
s
Host function call duration: '__hc__call_1'
Duration of this host function call invoked from within WASM.
mean
0.303014
s
SD
2.911158
s
p50
0.289233
s
<
p95
7.494459
s
<
p99
13.045426
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000725
s
SD
0.000764
s
p50
0.00059
s
<
p95
0.0014
s
<
p99
0.004838
s
Host function call duration: '__hc__sys_time_1'
Duration of this host function call invoked from within WASM.
mean
1.1e-05
s
SD
6e-06
s
p50
1e-05
s
<
p95
2.3e-05
s
<
p99
2.6e-05
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.000301
s
SD
0.000989
s
p50
0.000371
s
<
p95
0.001108
s
<
p99
0.001746
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.000531
s
SD
0.00109
s
p50
0.000197
s
<
p95
0.001919
s
<
p99
0.004928
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
44.41
s
<
mean
423.61
s
SD
246.09
s
<
p95
815.1
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
1863
over 888.007686647s
mean rate
16915.26
/s
std
801128.51
/s
p5
0
/s
p95
26821.67
/s
peak
6.766917293e+07
/s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
14.525772
s
SD
9.379357
s
p50
10.801758
s
<
p95
33.777511
s
<
p99
49.309098
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.009741
s
SD
0.014602
s
p50
0.010286
s
<
p95
0.040393
s
<
p99
0.079117
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.064783
s
SD
0.046489
s
p50
0.067631
s
<
p95
0.153904
s
<
p99
0.238611
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.018658
s
SD
0.013361
s
p50
0.015986
s
<
p95
0.038168
s
<
p99
0.051089
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
4.709793
s
SD
5.041449
s
p50
6.616765
s
<
p95
14.910899
s
<
p99
27.600153
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
2.60202
s
SD
7.884573
s
p50
1.273844
s
<
p95
17.454765
s
<
p99
36.04178
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.000391
s
SD
0.000192
s
p50
0.00039
s
<
p95
0.00072
s
<
p99
0.000834
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.003037
s
SD
0.001535
s
p50
0.002635
s
<
p95
0.005621
s
<
p99
0.007171
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000507
s
SD
0.000272
s
p50
0.000587
s
<
p95
0.000959
s
<
p99
0.0012
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
0.18833
s
SD
27.385628
s
p50
0.110708
s
<
p95
60.002713
s
<
p99
60.004621
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.323812
s
SD
23.137631
s
p50
0.201803
s
<
p95
60.001588
s
<
p99
60.002026
s
P2P request roundtrip: 'call_remote'
Time spent sending a call_remote request and awaiting its response.
mean
0.24188
s
SD
27.38838
s
p50
0.252724
s
<
p95
60.001502
s
<
p99
60.001969
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
5.43
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
12.60
KiB/s
<
mean
136.19
KiB/s
SD
321.43
KiB/s
<
p95
455.56
KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
6.07
KiB/s
<
mean
15.90
KiB/s
SD
13.90
KiB/s
<
p95
41.63
KiB/s
Total bytes received
Total bytes received on primary network interface
count
16.49
GiB
mean
18.97
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
1.93
GiB
mean
2.22
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
NotDetected
Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected

Critical Heavy swap usage (69.0% swap used)

System overload anomaly
System overload anomaly was detected
NotDetected
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
2.06
%
<
mean
9.57
%
SD
10.7
%
<
p95
31.31
%
System
4.15
%
CPU percentiles
CPU usage percentiles
p50
6.33
%
p95
31.31
%
p99
60.00
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
55
hosts
mean time
10.22
s
Memory used percentage
Percentage of memory used
p5
5.7
%
<
mean
10.43
%
SD
7.65
%
<
p95
21.47
%
Memory available percentage
Percentage of available memory
p5
78.53
%
<
mean
89.57
%
SD
7.65
%
<
p95
94.3
%
Max host memory used
Maximum memory usage percentage across all hosts
max
85.91
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
68.99
%
Memory growth rate
Rate of memory growth over time
growth
0.36
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.46
MiB/s
Disk write throughput
Disk write throughput in MiB/s
20.28
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point /
0/120
hosts
Mount Point /efi-boot
0/2
hosts
Mount Point /etc/hostname
0/13
hosts
Mount Point /etc/hosts
0/13
hosts
Mount Point /etc/nomad.d/client-meta.json
0/12
hosts
Mount Point /etc/resolv.conf
0/13
hosts
Mount Point /nix/store
0/2
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
0.31
5 min
0.26
15 min
0.15
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
0.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
2.0377
%
SD
2.5346
%
<
p95
7.14
%
60 second average
2.0204
%
300 second average
1.6637
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
0.296
%
SD
1.7192
%
<
p95
1.22
%
60 second average
0.3971
%
300 second average
0.4247
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
0.2272
%
SD
1.6366
%
<
p95
0.82
%
60 second average
0.3225
%
300 second average
0.3583
%
Holochain process CPU usage
CPU usage by Holochain process
p5
1.3
%
<
mean
6.72
%
SD
9.01
%
<
p95
18.99
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
216.35
KiB
<
mean
244.82
KiB
SD
34.40
KiB
<
p95
282.63
KiB
Holochain process threads
Number of threads in Holochain process
p5
29
threads
<
mean
33.69
threads
SD
13.28
threads
<
p95
39
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
76
file descriptors
<
mean
82.01
file descriptors
SD
14.13
file descriptors
<
p95
89
file descriptors

Remote Signals

↑ Back to index

This scenario tests the throughput of remote_signals operations.

Started
Thu, 01 Oct 2026 03:41:09 UTC
Duration
900s
Peer count
300
Behaviours
  • default (1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
remote_signals_36809382471_1
Round trip time
The time from origin signal dispatch to origin receive of the remote side's response signal.
mean
1.99228
s
SD
13.557231
s
p50
0.131713
s
<
p95
3.658678
s
<
p99
50.382294
s
Timeouts
The number of timeouts waiting for the remote side's response signal (default timeout is 20 seconds).
total
642
over 2485.828605869s
mean rate
119370.12
/s
std
4.81380025e+06
/s
p5
1.4
/s
p95
110058.72
/s
peak
6.1224489796e+08
/s
Timeout rate
The proportion of remote signal round-trips that timed out.
0.0036
Holochain MetricsClick to toggle visibility.
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
1
over 785.478022242s
mean rate
111.41
/s
std
1103.1
/s
p5
0
/s
p95
104.05
/s
peak
21693.39
/s
WASM usage: 'remote_signal::init'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 887.898526957s
Not enough time data to show a trend.
WASM usage: 'remote_signal::recv_remote_signal'
Metered usage count of the WASM ribosome for this zome function.
total
17642236
over 887.898526957s
mean rate
6.3957201229e+09
/s
std
7.97886953123e+10
/s
p5
4.49487152e+06
/s
p95
1.12187813536e+10
/s
peak
4.212444e+12
/s
WASM usage: 'remote_signal::signal_request'
Metered usage count of the WASM ribosome for this zome function.
total
12282840
over 887.898526957s
mean rate
2.4180678176e+08
/s
std
7.03727258559e+09
/s
p5
0
/s
p95
1.2666800583e+08
/s
peak
5.58409276249e+11
/s
Zome call duration: 'remote_signal::recv_remote_signal'
Duration of this zome call as measured by Holochain internally.
mean
0.003625
s
SD
0.001483
s
p50
0.004112
s
<
p95
0.005713
s
<
p99
0.006253
s
Zome call duration: 'remote_signal::signal_request'
Duration of this zome call as measured by Holochain internally.
mean
0.003499
s
SD
0.001405
s
p50
0.003908
s
<
p95
0.005594
s
<
p99
0.006199
s
WASM call duration: 'remote_signal::init'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.004369
s
SD
0.006974
s
p50
0.00357
s
<
p95
0.01037
s
<
p99
0.013038
s
WASM call duration: 'remote_signal::recv_remote_signal'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.00233
s
SD
0.001003
s
p50
0.002681
s
<
p95
0.003756
s
<
p99
0.004348
s
WASM call duration: 'remote_signal::signal_request'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.002178
s
SD
0.000914
s
p50
0.002474
s
<
p95
0.003551
s
<
p99
0.004188
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000666
s
SD
0.000584
s
p50
0.000535
s
<
p95
0.001409
s
<
p99
0.004191
s
Host function call duration: '__hc__emit_signal_1'
Duration of this host function call invoked from within WASM.
mean
6.8e-05
s
SD
5.9e-05
s
p50
7.7e-05
s
<
p95
0.000109
s
<
p99
0.000138
s
Host function call duration: '__hc__send_remote_signal_1'
Duration of this host function call invoked from within WASM.
mean
6.6e-05
s
SD
3.2e-05
s
p50
7.3e-05
s
<
p95
0.000112
s
<
p99
0.000155
s
Host function call duration: '__hc__sys_time_1'
Duration of this host function call invoked from within WASM.
mean
8e-06
s
SD
6e-06
s
p50
8e-06
s
<
p95
1.6e-05
s
<
p99
2.5e-05
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.000955
s
SD
0.006621
s
p50
0.000372
s
<
p95
0.001377
s
<
p99
0.007394
s
Emitted local signals
Count of local signals emitted via the emit_signal host function.
total
1009
over 887.898174924s
mean rate
1.25895098e+06
/s
std
5.024744369e+07
/s
p5
154.71
/s
p95
516846.88
/s
peak
4.8125e+09
/s
Sent remote signals
Count of remote signals sent via the send_remote_signal host function.
total
1231
over 887.898170087s
mean rate
1.75859486e+06
/s
std
5.053697167e+07
/s
p5
150.71
/s
p95
430873.52
/s
peak
4.31578947368e+09
/s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.000662
s
SD
0.001509
s
p50
0.000203
s
<
p95
0.002964
s
<
p99
0.009774
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
42.51
s
<
mean
426.43
s
SD
248.39
s
<
p95
817.03
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
3668
over 888.105574155s
mean rate
1.35539548e+06
/s
std
1.580924615e+07
/s
p5
120.99
/s
p95
2.8083174e+06
/s
peak
9.4848484848e+08
/s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
19.291945
s
SD
11.290488
s
p50
15.707906
s
<
p95
40.377724
s
<
p99
49.488284
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.017578
s
SD
0.03605
s
p50
0.01649
s
<
p95
0.074338
s
<
p99
0.179469
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.08913
s
SD
0.072984
s
p50
0.090302
s
<
p95
0.221213
s
<
p99
0.338405
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.024705
s
SD
0.015494
s
p50
0.021843
s
<
p95
0.052065
s
<
p99
0.077104
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
6.69219
s
SD
8.490253
s
p50
8.645603
s
<
p95
27.281913
s
<
p99
38.211616
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
3.84172
s
SD
9.065219
s
p50
2.013526
s
<
p95
22.651102
s
<
p99
43.374512
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.000594
s
SD
0.000418
s
p50
0.000502
s
<
p95
0.001344
s
<
p99
0.001773
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.003456
s
SD
0.003065
s
p50
0.0028
s
<
p95
0.007358
s
<
p99
0.010066
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000669
s
SD
0.005269
s
p50
0.000705
s
<
p95
0.001317
s
<
p99
0.002468
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
0.335465
s
SD
28.796192
s
p50
0.374269
s
<
p95
60.00321
s
<
p99
60.006683
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.319595
s
SD
23.315375
s
p50
0.223581
s
<
p95
60.001546
s
<
p99
60.00198
s
P2P received remote signals
Count of remote signals received.
total
1009
over 887.89845026s
mean rate
341607.51
/s
std
5.3501824e+06
/s
p5
136.68
/s
p95
431281.15
/s
peak
3.2246376812e+08
/s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
10.41
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
29.39
KiB/s
<
mean
251.10
KiB/s
SD
339.84
KiB/s
<
p95
599.08
KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
18.83
KiB/s
<
mean
38.39
KiB/s
SD
28.39
KiB/s
<
p95
86.60
KiB/s
Total bytes received
Total bytes received on primary network interface
count
29.68
GiB
mean
34.15
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
4.54
GiB
mean
5.22
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected

Warning CPU p99 reached 95.5%

Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected

Critical Heavy swap usage (71.0% swap used)

System overload anomaly
System overload anomaly was detected
NotDetected
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
3.42
%
<
mean
16.42
%
SD
15.49
%
<
p95
39.44
%
System
6.01
%
CPU percentiles
CPU usage percentiles
p50
12.80
%
p95
39.44
%
p99
95.50
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
117
hosts
mean time
44.65
s
Memory used percentage
Percentage of memory used
p5
6.33
%
<
mean
13.27
%
SD
8
%
<
p95
25.14
%
Memory available percentage
Percentage of available memory
p5
74.86
%
<
mean
86.73
%
SD
8
%
<
p95
93.67
%
Max host memory used
Maximum memory usage percentage across all hosts
max
88.56
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
70.97
%
Memory growth rate
Rate of memory growth over time
growth
0.59
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.68
MiB/s
Disk write throughput
Disk write throughput in MiB/s
33.29
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point /
0/119
hosts
Mount Point /efi-boot
0/2
hosts
Mount Point /etc/hostname
0/13
hosts
Mount Point /etc/hosts
0/13
hosts
Mount Point /etc/nomad.d/client-meta.json
0/14
hosts
Mount Point /etc/resolv.conf
0/13
hosts
Mount Point /nix/store
0/2
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
0.59
5 min
0.49
15 min
0.34
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
0.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
6.3512
%
SD
10.5654
%
<
p95
18.57
%
60 second average
6.1480
%
300 second average
5.1508
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
0.0003
%
SD
0.0174
%
≤
p95
0
%
60 second average
0.0005
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
0.4462
%
SD
1.326
%
<
p95
1.79
%
60 second average
0.4492
%
300 second average
0.3890
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
0.2641
%
SD
1.0222
%
<
p95
1.08
%
60 second average
0.2716
%
300 second average
0.2472
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.97
%
<
mean
6.45
%
SD
7.51
%
<
p95
18.07
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
191.07
KiB
<
mean
229.45
KiB
SD
44.90
KiB
<
p95
275.16
KiB
Holochain process threads
Number of threads in Holochain process
p5
27
threads
<
mean
33.04
threads
SD
13.9
threads
<
p95
42
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
72
file descriptors
<
mean
80.54
file descriptors
SD
15.8
file descriptors
<
p95
90
file descriptors

Validation Receipts

↑ Back to index

Creates an entry, wait for required validation receipts, then repeat. Records the amount of time it took to accumulate the required number of receipts for all DHT operations. This is measured to the nearest 20ms so that we don't keep the agent too busy checking for receipts.

Each agent in this scenario waits for a certain number of peers to be available or for up to two minutes, whichever happens first, before starting its behaviour.

By default, this scenario will wait for a complete set of validation receipts before committing the next record. If the NO_VALIDATION_COMPLETE environment variable is set, it will instead publish new records on every round, building up an ever-growing list of action hashes to check on.

Started
Thu, 01 Oct 2026 04:40:06 UTC
Duration
900s
Peer count
150
Peer count at end
133
Behaviours
  • default (1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
validation_receipts_36809382471_1
Receipts complete timing
The amount of time between publishing a record and receiving the required number of validation receipts.
Mean
mean
60.893305
s
SD
37.739333
s
Max
Highest per-partition mean latency
max mean
657.837856
s
Min
Lowest per-partition mean latency
min mean
13.159658
s
Receipts complete rate
The number of complete validation receipt sets collected per second.
Mean
mean
0.96
/s
Max
Highest per-partition mean rate
max
1.11
/s
Min
Lowest per-partition mean rate
min
0
/s
Error count
The total number of errors accumulated during the run.
0
Holochain MetricsClick to toggle visibility.
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
0
over 499.452026422s
Not enough time data to show a trend.
WASM usage: 'crud::create_sample_entry'
Metered usage count of the WASM ribosome for this zome function.
total
308745
over 889.870301453s
mean rate
1.313563256e+07
/s
std
5.653226642e+07
/s
p5
0
/s
p95
5.140950262e+07
/s
peak
2.98115479634e+09
/s
WASM usage: 'crud::get_sample_entry_validation_receipts'
Metered usage count of the WASM ribosome for this zome function.
total
10905810
over 889.870301453s
mean rate
1.2854134865e+08
/s
std
4.1442520592e+08
/s
p5
374247.91
/s
p95
5.2232156893e+08
/s
peak
8.10230785465e+09
/s
WASM usage: 'crud::init'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 889.870301453s
Not enough time data to show a trend.
WASM usage: 'crud_integrity::entry_defs'
Metered usage count of the WASM ribosome for this zome function.
total
38304
over 889.870301453s
mean rate
1.61935205e+06
/s
std
6.97345484e+06
/s
p5
0
/s
p95
6.37174259e+06
/s
peak
3.6985263994e+08
/s
Zome call duration: 'crud::create_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.005845
s
SD
0.0029
s
p50
0.007169
s
<
p95
0.010143
s
<
p99
0.014942
s
Zome call duration: 'crud::get_sample_entry_validation_receipts'
Duration of this zome call as measured by Holochain internally.
mean
0.013436
s
SD
0.007653
s
p50
0.009011
s
<
p95
0.024848
s
<
p99
0.041258
s
WASM call duration: 'crud::create_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.004789
s
SD
0.002275
s
p50
0.005935
s
<
p95
0.008356
s
<
p99
0.011055
s
WASM call duration: 'crud::get_sample_entry_validation_receipts'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.012038
s
SD
0.00744
s
p50
0.007418
s
<
p95
0.023122
s
<
p99
0.039121
s
WASM call duration: 'crud::init'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.003263
s
SD
0.002195
s
p50
0.003042
s
<
p95
0.006558
s
<
p99
0.014262
s
WASM call duration: 'crud_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001563
s
SD
0.000674
s
p50
0.00191
s
<
p95
0.0025
s
<
p99
0.003016
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000556
s
SD
0.000982
s
p50
0.000619
s
<
p95
0.001495
s
<
p99
0.003035
s
Host function call duration: '__hc__get_validation_receipts_1'
Duration of this host function call invoked from within WASM.
mean
0.009925
s
SD
0.007233
s
p50
0.00527
s
<
p95
0.020618
s
<
p99
0.036211
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.002443
s
SD
0.001216
s
p50
0.002988
s
<
p95
0.004272
s
<
p99
0.007086
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.000563
s
SD
0.000598
s
p50
0.000499
s
<
p95
0.00154
s
<
p99
0.003242
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
44.68
s
<
mean
428.66
s
SD
248.77
s
<
p95
817.66
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
8183
over 889.870426879s
mean rate
78096.45
/s
std
1.00196249e+06
/s
p5
455.9
/s
p95
327294.6
/s
peak
7.020387273e+07
/s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
18.978479
s
SD
10.259487
s
p50
17.040974
s
<
p95
35.601581
s
<
p99
45.680869
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.005069
s
SD
0.041536
s
p50
0.010326
s
<
p95
0.106353
s
<
p99
0.222748
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.036876
s
SD
0.096143
s
p50
0.067029
s
<
p95
0.296131
s
<
p99
0.429906
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.050132
s
SD
0.032638
s
p50
0.032679
s
<
p95
0.105209
s
<
p99
0.165814
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.639733
s
SD
9.11049
s
p50
2.367315
s
<
p95
27.044589
s
<
p99
38.506356
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
13.7053
s
SD
20.964779
s
p50
17.238357
s
<
p95
66.29006
s
<
p99
94.814822
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.000963
s
SD
0.000513
s
p50
0.00076
s
<
p95
0.00182
s
<
p99
0.002322
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.004104
s
SD
0.002641
s
p50
0.002782
s
<
p95
0.006966
s
<
p99
0.009345
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000517
s
SD
0.000287
s
p50
0.000584
s
<
p95
0.001084
s
<
p99
0.00135
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
0.54238
s
SD
29.110454
s
p50
0.439473
s
<
p95
60.002827
s
<
p99
60.00388
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.204619
s
SD
26.239627
s
p50
0.217405
s
<
p95
60.001615
s
<
p99
60.001958
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
11.13
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
19.34
KiB/s
<
mean
151.18
KiB/s
SD
326.49
KiB/s
<
p95
366.24
KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
5.70
KiB/s
<
mean
19.39
KiB/s
SD
19.41
KiB/s
<
p95
56.98
KiB/s
Total bytes received
Total bytes received on primary network interface
count
18.21
GiB
mean
20.95
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
2.33
GiB
mean
2.68
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
NotDetected
Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected

Critical Heavy swap usage (74.3% swap used)

System overload anomaly
System overload anomaly was detected
Detected

Warning 3% of hosts overloaded (load5/ncpus > 1.0)

CPU usage
Total CPU usage and kernel CPU usage
Total
p5
3.02
%
<
mean
16.88
%
SD
13.64
%
<
p95
40.22
%
System
5.75
%
CPU percentiles
CPU usage percentiles
p50
13.63
%
p95
40.22
%
p99
78.08
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
83
hosts
mean time
14.45
s
Memory used percentage
Percentage of memory used
p5
6.2
%
<
mean
11.27
%
SD
7.4
%
<
p95
21.94
%
Memory available percentage
Percentage of available memory
p5
78.06
%
<
mean
88.73
%
SD
7.4
%
<
p95
93.8
%
Max host memory used
Maximum memory usage percentage across all hosts
max
82.03
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
74.26
%
Memory growth rate
Rate of memory growth over time
growth
0.30
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.85
MiB/s
Disk write throughput
Disk write throughput in MiB/s
31.30
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point /
0/121
hosts
Mount Point /efi-boot
0/2
hosts
Mount Point /etc/hostname
0/13
hosts
Mount Point /etc/hosts
0/13
hosts
Mount Point /etc/nomad.d/client-meta.json
0/12
hosts
Mount Point /etc/resolv.conf
0/13
hosts
Mount Point /nix/store
0/2
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
0.59
5 min
1.31
15 min
2.28
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
2.74
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
4.4027
%
SD
5.5645
%
<
p95
11.71
%
60 second average
5.0346
%
300 second average
14.3350
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
0.0001
%
SD
0.0104
%
≤
p95
0
%
60 second average
0.0001
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
0.0001
%
SD
0.0104
%
≤
p95
0
%
60 second average
0.0001
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
1.4133
%
SD
4.6608
%
<
p95
3.48
%
60 second average
1.5256
%
300 second average
3.2416
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
1.1093
%
SD
4.3496
%
<
p95
2.43
%
60 second average
1.1107
%
300 second average
1.4306
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.82
%
<
mean
13.4
%
SD
11.89
%
<
p95
32.84
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
237.76
KiB
<
mean
280.69
KiB
SD
42.93
KiB
<
p95
316.10
KiB
Holochain process threads
Number of threads in Holochain process
p5
12
threads
<
mean
31.15
threads
SD
14.79
threads
<
p95
38
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
44
file descriptors
<
mean
77.41
file descriptors
SD
21.49
file descriptors
<
p95
87
file descriptors

Write/get_agent_activity

↑ Back to index

A scenario where write peers write entries, while get_agent_activity peers each query a single write agent's activity with get_agent_activity.

Before a target write peer and the requesting get_agent_activity peer are in sync, this will measure the get_agent_activity call performance over a network. Once a write peer reaches sync with a get_agent_activity peer, the write peer will publish their actions and entries, and so the get_agent_activity calls will likely have most of the data they need locally. At that point this measures the database query performance and code paths through host functions.

Started
Thu, 01 Oct 2026 04:57:17 UTC
Duration
900s
Peer count
150
Peer count at end
135
Behaviours
  • get_agent_activity (1 agent)
  • write (1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
write_get_agent_activity_36809382471_1
Observed chain advancement
The maximum highest-observed action sequence per write agent, aggregated across all reading agents. The reading dimension is collapsed by taking the maximum observed value for each write agent (any reader successfully tracking that point counts as propagation). Captures how far readers tracked writers’ chains during the run.
Mean max (across write agents)
Mean of the highest chain head value observed per write agent. Each write agent contributes its maximum — the value seen by any reader.
2043.65
Max
Highest chain head value observed for any single write agent.
5830
Write agents observed
20
get_agent_activity_full zome call timing
The time taken to call the zome function that retrieves information on a write peer's source chain.
Mean
mean
0.804465
s
SD
1.795147
s
Max
Highest per-partition mean latency
max mean
6.319846
s
Min
Lowest per-partition mean latency
min mean
0.053717
s
Error count
The number of errors accumulated.
256
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.083199
s
SD
2.655636
s
p50
0.09328
s
<
p95
3.73345
s
<
p99
15.483146
s
Zome call duration: 'agent_activity::announce_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.006134
s
SD
0.002173
s
p50
0.006642
s
<
p95
0.009398
s
<
p99
0.012088
s
Zome call duration: 'agent_activity::create_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.009055
s
SD
0.011166
s
p50
0.009431
s
<
p95
0.030735
s
<
p99
0.064735
s
Zome call duration: 'agent_activity::get_agent_activity_full'
Duration of this zome call as measured by Holochain internally.
mean
0.173252
s
SD
2.516347
s
p50
0.230118
s
<
p95
4.122983
s
<
p99
15.831243
s
Zome call duration: 'agent_activity::get_random_agent_with_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.00547
s
SD
1.846444
s
p50
0.007352
s
<
p95
3.156951
s
<
p99
8.001462
s
WASM call duration: 'agent_activity::announce_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.005025
s
SD
0.001814
s
p50
0.005539
s
<
p95
0.007368
s
<
p99
0.009331
s
WASM call duration: 'agent_activity::create_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.007783
s
SD
0.01034
s
p50
0.00787
s
<
p95
0.027885
s
<
p99
0.059516
s
WASM call duration: 'agent_activity::get_agent_activity_full'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.172635
s
SD
2.516259
s
p50
0.22842
s
<
p95
4.119137
s
<
p99
15.830367
s
WASM call duration: 'agent_activity::get_random_agent_with_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.004778
s
SD
2.77628
s
p50
0.006032
s
<
p95
3.156213
s
<
p99
15.487056
s
WASM call duration: 'agent_activity_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001422
s
SD
0.000764
s
p50
0.001554
s
<
p95
0.00262
s
<
p99
0.003725
s
Host function call duration: '__hc__agent_info_1'
Duration of this host function call invoked from within WASM.
mean
5.3e-05
s
SD
3.7e-05
s
p50
4.1e-05
s
<
p95
0.000143
s
<
p99
0.000168
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.003067
s
SD
0.006716
s
p50
0.002668
s
<
p95
0.015292
s
<
p99
0.037135
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.000611
s
SD
0.000434
s
p50
0.000539
s
<
p95
0.001049
s
<
p99
0.002841
s
Host function call duration: '__hc__get_agent_activity_1'
Duration of this host function call invoked from within WASM.
mean
0.170646
s
SD
2.51599
s
p50
0.225912
s
<
p95
4.115324
s
<
p99
15.829557
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.000994
s
SD
2.776085
s
p50
0.000646
s
<
p95
3.152456
s
<
p99
15.48322
s
Host function call duration: '__hc__random_bytes_1'
Duration of this host function call invoked from within WASM.
mean
1e-05
s
SD
6e-06
s
p50
1e-05
s
<
p95
2.2e-05
s
<
p99
3e-05
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.002522
s
SD
0.001627
s
p50
0.002536
s
<
p95
0.005094
s
<
p99
0.009344
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.00132
s
SD
0.001625
s
p50
0.000281
s
<
p95
0.003714
s
<
p99
0.007075
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
42.73
s
<
mean
426.41
s
SD
247.83
s
<
p95
817.35
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
4298
over 890.001077325s
mean rate
707042.06
/s
std
2.916624408e+07
/s
p5
976.62
/s
p95
654342.7
/s
peak
2.28270412643e+09
/s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
89.538468
s
SD
64.052217
s
p50
23.366833
s
<
p95
183.950943
s
<
p99
331.976932
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.097916
s
SD
0.305307
s
p50
0.081803
s
<
p95
0.659893
s
<
p99
1.176519
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.106123
s
SD
0.42895
s
p50
0.223789
s
<
p95
1.216032
s
<
p99
1.441075
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.799371
s
SD
0.787662
s
p50
0.109699
s
<
p95
2.179072
s
<
p99
3.873219
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
11.732739
s
SD
21.010758
s
p50
18.845561
s
<
p95
61.763977
s
<
p99
95.08845
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
10.922973
s
SD
19.037184
s
p50
0.25475
s
<
p95
50.450146
s
<
p99
75.403731
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.004195
s
SD
0.002848
s
p50
0.003674
s
<
p95
0.008303
s
<
p99
0.012281
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.004588
s
SD
0.004334
s
p50
0.003147
s
<
p95
0.009754
s
<
p99
0.017658
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.002621
s
SD
0.004841
s
p50
0.000993
s
<
p95
0.007948
s
<
p99
0.025389
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
1.766931
s
SD
29.339084
s
p50
1.558481
s
<
p95
60.012225
s
<
p99
60.076265
s
P2P request roundtrip: 'get_links'
Time spent sending a get_links request and awaiting its response.
mean
1.499328
s
SD
2.061491
s
p50
1.034474
s
<
p95
7.982054
s
<
p99
9.471732
s
P2P request roundtrip: 'get_agent_activity'
Time spent sending a get_agent_activity request and awaiting its response.
mean
0.150669
s
SD
28.902215
s
p50
0.340002
s
<
p95
60.005947
s
<
p99
60.01383
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.650784
s
SD
26.529379
s
p50
2.886808
s
<
p95
60.006871
s
<
p99
60.027803
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
50.33
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
32.79
KiB/s
<
mean
666.50
KiB/s
SD
560.61
KiB/s
<
p95
1.71
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
2.60
KiB/s
<
mean
137.12
KiB/s
SD
189.58
KiB/s
<
p95
435.73
KiB/s
Total bytes received
Total bytes received on primary network interface
count
80.09
GiB
mean
92.15
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
16.47
GiB
mean
18.95
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected

Warning CPU p99 reached 98.0%

Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected

Critical Heavy swap usage (72.8% swap used)

System overload anomaly
System overload anomaly was detected
Detected

Warning 61% of hosts overloaded (load5/ncpus > 1.0)

CPU usage
Total CPU usage and kernel CPU usage
Total
p5
5.83
%
<
mean
68.65
%
SD
27.56
%
<
p95
94.73
%
System
18.32
%
CPU percentiles
CPU usage percentiles
p50
80.43
%
p95
94.73
%
p99
98.00
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
134
hosts
mean time
500.60
s
Memory used percentage
Percentage of memory used
p5
6.36
%
<
mean
12.1
%
SD
7.85
%
<
p95
24.11
%
Memory available percentage
Percentage of available memory
p5
75.89
%
<
mean
87.9
%
SD
7.85
%
<
p95
93.64
%
Max host memory used
Maximum memory usage percentage across all hosts
max
85.80
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
72.77
%
Memory growth rate
Rate of memory growth over time
growth
0.70
MiB/s
Disk read throughput
Disk read throughput in MiB/s
1.06
MiB/s
Disk write throughput
Disk write throughput in MiB/s
127.03
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point /
0/121
hosts
Mount Point /efi-boot
0/2
hosts
Mount Point /etc/hostname
0/13
hosts
Mount Point /etc/hosts
0/13
hosts
Mount Point /etc/nomad.d/client-meta.json
0/13
hosts
Mount Point /etc/resolv.conf
0/13
hosts
Mount Point /nix/store
0/2
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
3.26
5 min
2.43
15 min
2.03
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
61.22
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
1
%
<
mean
41.8627
%
SD
27.589
%
<
p95
83.31
%
60 second average
39.1470
%
300 second average
28.7464
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
8.1425
%
SD
6.7799
%
<
p95
17.53
%
60 second average
7.6525
%
300 second average
5.7833
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
3.1334
%
SD
3.6498
%
<
p95
9.74
%
60 second average
2.9848
%
300 second average
2.4520
%
Holochain process CPU usage
CPU usage by Holochain process
p5
1.45
%
<
mean
63.1
%
SD
27.32
%
<
p95
90.45
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
226.19
KiB
<
mean
341.87
KiB
SD
96.18
KiB
<
p95
479.71
KiB
Holochain process threads
Number of threads in Holochain process
p5
8
threads
<
mean
32.31
threads
SD
14.92
threads
<
p95
37
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
12
file descriptors
<
mean
78.94
file descriptors
SD
22.53
file descriptors
<
p95
89
file descriptors

Write/get_agent_activity with volatile conductors

↑ Back to index

A scenario where write peers write entries, while get_agent_activity_volatile peers each query a single write agents activity with get_agent_activity but shutdown and restart their conductors at semi-random intervals.

Before a target write peer and the requesting get_agent_activity_volatile peer are in sync, this will measure the get_agent_activity call performance over a network. Once a write peer reaches sync with a get_agent_activity peer, the write peer will publish their actions and entries, and so the get_agent_activity calls will likely have most of the data they need locally. At that point this measures the database query performance and code paths through host functions.

Started
Thu, 01 Oct 2026 05:15:27 UTC
Duration
1800s
Peer count
150
Peer count at end
114
Behaviours
  • get_agent_activity_volatile (1 agent)
  • write (1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
write_get_agent_activity_volatile_36809382471_1
Highest observed action_seq
The rate at which zero-arc readers observe new chain heads on the writers' chains via get_agent_activity. This reflects the DHT's ability to propagate agent activity ops and make them available to querying peers.
Mean
mean
252.23
/s
Max
Highest per-partition mean rate
max
738.68
/s
Min
Lowest per-partition mean rate
min
23.5
/s
get_agent_activity_full zome call timing
The time taken to call the zome function that retrieves information on a write peer's source chain.
Mean
mean
0.035598
s
SD
0.27565
s
Max
Highest per-partition mean latency
max mean
0.39046
s
Min
Lowest per-partition mean latency
min mean
0.008543
s
Running volatile conductors count
The number of conductors running by get_agent_activity_volatile peers.
p5
63
conductors
<
mean
77.04
conductors
SD
10.03
conductors
<
p95
90
conductors
Volatile conductor total running duration
The total running duration of a get_agent_activity_volatile conductor.
Mean
mean
903.649415
s
SD
557.527371
s
Max
Highest per-partition mean latency
max mean
1438.041667
s
Min
Lowest per-partition mean latency
min mean
35.333333
s
Volatile conductor running duration
The duration that a get_agent_activity_volatile conductor was running before being stopped.
Mean
mean
56.369393
s
SD
48.096803
s
Max
Highest per-partition mean latency
max mean
98.172711
s
Min
Lowest per-partition mean latency
min mean
18.839177
s
Volatile conductor stopped duration
The duration that a get_agent_activity_volatile conductor was stopped before being started again.
Mean
mean
48.282911
s
SD
51.003461
s
Max
Highest per-partition mean latency
max mean
99.132416
s
Min
Lowest per-partition mean latency
min mean
13.630229
s
Reached target arc
Did get_agent_activity_volatile peer reach their target arc in the moment before they were shutdown.
  • get_agent_activity_volatile_agent: uhCAk0ROzFj9kvTGNNJfjVL8xnyrrIkde1Rru49KfTUMGFHllp9vK
p5
0
<
mean
0.08
SD
0.27
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk0Rdyx27adSW-mlTVLSQ8ZcbLyCPkdVa9EoVKJCJc-RCv2DKI
p5
0
<
mean
0.28
SD
0.45
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk0kUp_6nLKRLfP04B1xukAo4JpoidkNcT4PVrhigw1zCRnTKz
p5
0
<
mean
0.43
SD
0.5
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk1N24mx-Yw3JTEmRWjBfpvIQIMq5xBvR8pWmup4xe0Z979qcH
p5
0
<
mean
0.33
SD
0.47
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk29Mm_vjiytL2x591vRznMQ1Xeo8jnt_WEfuXclRYBntK2Qsh
p5
0
<
mean
0.29
SD
0.45
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk2EdAFuanePrOsN-uAhDEPBymOdvEAFW4jvM6CZI62YUuNt4r
p5
0
<
mean
0.3
SD
0.46
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk2pA6SFWggqygI2Q16OTExMjHzoSOEGAnt9o64_CY_GnxoVST
p5
0
<
mean
0.44
SD
0.5
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk35obtsSNdefmFDgxnJdG_A0jTbOsWh14n3f2o9WnOf7cd4ol
p5
0
<
mean
0.41
SD
0.49
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk3ELVm-MzsYLT2kTT79F4OWFPlFIG5hwEqAxN0m9racyj6-_0
p5
0
<
mean
0.23
SD
0.42
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk3WIDrnNfmUiheOgDN6QjLCRscRQRxn6UdoTEuIST3b0qn-C3
p5
0
<
mean
0.38
SD
0.49
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk3eReGmeT2YSyt9JHUBrYzkbdee0sWwSC-m1O941yQdjNIyH-
p5
0
<
mean
0.41
SD
0.49
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk5FRyyuTwLwDbsZTJRzwdYeWoqaJaa4W00U2cfwddASDZWOHt
p5
0
<
mean
0.28
SD
0.45
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk5ePBe8Cb57SCSDcbLr71ZdjO3ieeafWltmnp2cFWUhwpY9ao
p5
0
<
mean
0.27
SD
0.45
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk5gu_NK5HmGLQUU4p_C1z_zQJpjpQrPbHDdwWH7pw3bqSBzTe
p5
0
<
mean
0.32
SD
0.47
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk66Wu2dd21ZXG1zi9ZyblyVkW2K_XG_Uu-xZdVTuFXHX_LDQV
p5
0
<
mean
0.24
SD
0.42
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk6DkU3BTVs8a9sMd_01AzDtISZqsM8QVgMdVxiIVgImFs53Xr
p5
0
<
mean
0.22
SD
0.42
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk6QEt3B1glCIxhkCE_UJulY90JW5ObF_ucFjlndUjKhk1xDza
p5
0
<
mean
0.2
SD
0.4
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk75Z1LOuDxZ8CxEfjlM2VSvKd8jQiv70nstKXOivSGZm61khN
p5
0
<
mean
0.29
SD
0.45
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk8w3oqRS4-m3DLxNBXvNF4nin9wtWxx2s7Le6psZBSBVzHCvH
p5
0
<
mean
0.5
SD
0.5
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkA5CqSlF4yP2AJP43ETZjAWxdrNwDFWkT0vJjcYUkU92WtzYL
p5
0
<
mean
0.02
SD
0.15
≤
p95
0
  • get_agent_activity_volatile_agent: uhCAkAIfgFQydCJ8Hm6SLsu9SaVJZGdIYxi9leeRA92Q4m3cjus1Y
p5
0
<
mean
0.38
SD
0.49
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkAepzTjb05drypVR_ZUDvD6MuYc4zpD0ysPhC33HxZAg6B-sT
p5
0
<
mean
0.35
SD
0.48
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkAoOG4w83KJs3suFoE89L1szXu9c9xAZYjzoKNjgguDIAf7Ph
p5
0
<
mean
0.24
SD
0.43
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkC8_OtYCNHkrKBBxJ29jm14SNgm-2kag3qlmtntptuiXhpzR5
p5
0
<
mean
0.38
SD
0.49
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkCArd-0GiO86XhcPyHS-fNdH6I--1QtR-Zf8GBN22okJatBsB
p5
0
<
mean
0.3
SD
0.46
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkCWF5xiFUO8A7hdswfmf0ZHDYMz8BgH3aPDHafvMrxJ-zh9F9
p5
0
<
mean
0.23
SD
0.42
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkDT53290FQ6s7ppGJ-xXlHKL_Fb-5z1L6iDv0in--D7YvzcvL
p5
0
<
mean
0.07
SD
0.26
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkDpbpC-zE7fmmQQvuJ6_Q6shxcB3e3S-SH_-_pQ9E-qDo4xGU
p5
0
<
mean
0.29
SD
0.45
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkEBcJ75K8aMoPkyj9TEl1eh1VvuZepoLybOSO5-Bgb_KvZEQ4
p5
0
<
mean
0.07
SD
0.26
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkEc3uf9vF4vMQkJyLwDZoc0adalpc1W-gp3Jk8gEyC-Y4UQqC
p5
0
<
mean
0.42
SD
0.49
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkF1TNfIKdV7Ai-7KFRfR7_N9EZKcBtA5Ddwr354ou2AbJwyMP
p5
0
<
mean
0.38
SD
0.49
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkFUrj2Lx_dnKWQmG0shEvsXw2zYP9ml97ql4wo1eVkyPyQsIb
p5
0
<
mean
0.7
SD
0.46
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkG7m2Y2fGxvkj4F4AU0LwQ_akdcYe0tF4YMcvEj7rcrHUprrq
p5
0
<
mean
0.28
SD
0.45
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkGEeD6SCk3nTLrt7YsTXGO7Sf-IAaIqETpU1DUoexxhSl0UQr
p5
0
<
mean
0.41
SD
0.49
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkGcs9Q31TncSEfBPZUDfr3sDOeeTW6wg5XjbgRJtQgl-ajjsg
p5
0
<
mean
0.3
SD
0.46
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkHoNplojnkWcZrUNct1zlFuIbNsGtl-3VaqRrXoxhDhNK5sdy
p5
0
<
mean
0.22
SD
0.42
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkHw1mEDWz5U9BQAtZ-ZJSmhl7vxYNf6sUGoOsxbOEIvla8--R
p5
0
<
mean
0.35
SD
0.48
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkIFetnmRpEPq7C5RI35LdJWL3Rv3MVZ3xAUA4YpkhPeipv5sb
p5
0
<
mean
0.03
SD
0.17
≤
p95
0
  • get_agent_activity_volatile_agent: uhCAkIzqe0SBnYe3JjJYU2SBOiV_4MvoFeRNz-b7pccPfZC8So-8w
p5
0
<
mean
0.25
SD
0.43
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkJbNe-cQUSN6E0exAmS70jXLrK4yx6lGktzi6Iz8GOcfj5l_X
p5
0
<
mean
0.33
SD
0.47
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkJv2LS6qayRaNPas9JR995_tpIJG4MXwHXTIRX2m9XB17wstj
p5
0
<
mean
0.45
SD
0.5
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkKCI0jf5wZs32HnwUAAyLbVxWqMUXYgJZXCESaYPo-QhNokpS
p5
0
<
mean
0.35
SD
0.48
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkKcDcL4gMsSLn2ecLuGAPxA4HFPAScJBt9io1PJm3OIMDPu8F
p5
0
<
mean
0.29
SD
0.45
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkLTasSPnlOYtIN3joL5fwZhX6Z_m7xLbN4bR2GF2ryz97GNhR
p5
0
<
mean
0.26
SD
0.44
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkLpW9uMXCJ8ioCBzkEeKMMwM6CUwxzMZ09aZzR0WrNwgyz4z-
p5
0
<
mean
0.35
SD
0.48
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkM2HrctYJ-uFERdq3d9XnYoNR7NcEVvC8VEsynnT-IUD6Yi_M
p5
0
<
mean
0.29
SD
0.45
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkMKIwh2T15YYIaJjOsUwW42yiy93gxO8ef928q7Wcwb8eZInr
p5
0
<
mean
0.41
SD
0.49
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkMkkxXIX5Q79HNO5F3ViJz9o7dRNAaRUi5QP2KL1CfscZ4Lgj
p5
0
<
mean
0.32
SD
0.47
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkN245Z4zdT-dA4BL4o1zTLx8fP8hmiRyDwm6xV6YYuFB_YQF5
p5
0
<
mean
0.28
SD
0.45
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkNaZXM8HJYmurtuDCc7ardStk1mNaPzEl6pOBvi-zakuGJJof
p5
0
<
mean
0.29
SD
0.46
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkPHiKZyP3GipVfRsIVZoJbQ570Sgf1_hxHUeb_SEsNswhsGQ1
p5
0
<
mean
0.19
SD
0.39
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkQNAK0yIFtrHo_cDz-f1fmUCuaeUlzt_UH43IFm7aVBifGCc8
p5
0
<
mean
0.35
SD
0.48
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkQtOZ3PrBeIhvAgtuL-p_2aeUpea0if1Wvs-hzFhQ-Wbs0IuQ
p5
0
<
mean
0.28
SD
0.45
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkSYzVcLRxIdLb2HbCpKY6NNLf1rQGHTaSrcbt_q4nrNlYfx_X
p5
0
<
mean
0.5
SD
0.5
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkUAenSN4i1CWn6xVVNXBxKHlF5KuElvIYn3_3DcacV8s-fPmF
p5
0
<
mean
0.32
SD
0.47
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkVLXnrjlulRnVLLhHscsuvxrMsNicHHFjtlxQiEoz0D7EsglM
p5
0
<
mean
0.3
SD
0.46
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkVYzDL4PRlDmqfWCzfmaqOdfm6D_R4Ca9ycuEGWb4NoMUjXoP
p5
0
<
mean
0.29
SD
0.45
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkWGLFkShzAwMpZnQ2KKY8-Y3TxtSBhz8n4EG9KHByOU5JmCHq
p5
0
<
mean
0.5
SD
0.5
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkXVY4S2T-a9lGW8rIre5yYcNrlJ7walU6y5xb6U63sc47L1sm
p5
0
<
mean
0.26
SD
0.44
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkYuJJd6oMmdS-vC6S9NhzyGUqpsvGeUYbFtQLsKcOcd4MMx3i
p5
0
<
mean
0.37
SD
0.48
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk_Jv3a4g0RdvDDNX82uWIehPzgBUJO0KnTArpB5qdBFgsz7QX
p5
0
<
mean
0.5
SD
0.5
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk_PHPqFeZ1UoD7MQZoLFEH8nji6p0GyNAsxpuKc4ZTdcJMZp-
p5
0
<
mean
0.43
SD
0.49
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk_Ugv-sBUfs-3RtLK9dgIIbVpemY9eli9AKaY2SvyeEQhFhRW
p5
0
<
mean
0.27
SD
0.44
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk_hZjHBKehMMZQ3n0N-nay0AtK_5XI_RpqKaGKbzcZa8ksP2K
p5
0
<
mean
0.21
SD
0.41
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkah5VYttbYFp4lIQeHpJqsJZWbr7mjqpYxDTpTKjb-sB9vr1x
p5
0
<
mean
0.38
SD
0.49
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkbOlCP1dqN_-DX2_5BxZvuwBr6rL4daucVj014BoSH3tLRWma
p5
0
<
mean
0.55
SD
0.5
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkbi1WGppujM8w2DaiQL3DsFKPaqbtLMKUhPPWFhho6Zrxfp2w
p5
0
<
mean
0.17
SD
0.37
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkcQ9YRm1Z45I1wHPhdOA-NJmexlyCCdSYJtBjm2iVdS8PPD2G
p5
0
<
mean
0.21
SD
0.41
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkdex3VPnKcKa_79Rp9r9C0__jqRiCCvqVWBI2Zt2x6z75s0Gq
p5
0
<
mean
0.21
SD
0.41
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkdxzEvAa6K7kpRU6T9PXuJuV8lv2-KJ7OidOqNVtFWBhsWSFq
p5
0
<
mean
0.32
SD
0.46
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkfsME1GQLP9i5AlMoMlfz_vXbdanfAX4TfeMtsX5c7BGEYvgL
p5
0
<
mean
0.02
SD
0.15
≤
p95
0
  • get_agent_activity_volatile_agent: uhCAkg7ED7AsRl8F0gepJQx1CWYxTBMoUWgMzQNrWwYVHXW8R8wlI
p5
0
<
mean
0.33
SD
0.47
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkhOHqkiEND5VFhSugdRcoFDVfqp9LOj0tfxyC87AAEiCbVLUH
p5
0
<
mean
0.38
SD
0.49
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkhmtoussSeyhE8cQKj-CnSP2tv2piQp0-6BPBs4F--gP-KuMK
p5
0
<
mean
0.36
SD
0.48
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkiYLsOGj3rj7r-LhPMaC7NukrqEHPzJmpDWHC7oA3YL9dwnL7
p5
0
<
mean
0.34
SD
0.48
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkigKkqs7Xqc-3zGB9_ry5EHpAD4CNEwdSCJTDUUX6MSBDauDe
p5
0
<
mean
0.36
SD
0.48
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkjGShCPLSqfe18kTYxmF540IEvLzDs-GH-CU8XNa2NJe9R8Hn
p5
0
<
mean
0.32
SD
0.47
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkjmGuSGK8HUiH8e8kLDswUym7QKN2pvTOqwNQ7TUBTvMP3REi
p5
0
<
mean
0.16
SD
0.37
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkjw-q77BkMiRbgUx0mH91ADPLOD5bTN0zna5BWRwBzPi6XgEg
p5
0
<
mean
0.39
SD
0.49
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkm1kD3xj_f3VS0SYj2GO47wBj5M_zAdxuHBc3THiz2b7mVWjS
p5
0
<
mean
0.37
SD
0.48
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkmD38zRkokyHQfjm9r2SphoqKQMKFOkJOF3e_Z-yMO1xlLRCW
p5
0
<
mean
0.37
SD
0.48
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkmYdit6mZ3Z188i2vjIvooWJkAYT_I5SD-wrFTops_cY9HczV
p5
0
<
mean
0.21
SD
0.41
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkn__GaTQVnw7JWoCcPNvex7uB-JvXgdbMrJVzr03v_UeNq6KY
p5
0
<
mean
0.18
SD
0.38
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkoUnxHwkfq4z6i8fNAGSsKQPmf2afE1ojLqfokrwnqFzPsG8M
p5
0
<
mean
0.35
SD
0.48
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkovJkBll43azCZ0rZBfSzSUUgPW4h4qivJbhuioH2K2rgbxSN
p5
0
<
mean
0.34
SD
0.48
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkoxiK2H2MGd-rE_i11_BZt3XQxvXcFtJC2_HCgFPjQAUCMaRP
p5
0
≤
mean
0
SD
0
≤
p95
0
  • get_agent_activity_volatile_agent: uhCAkq8Y4lQSDB76hV3QzWBRT4zVEIZVjzjzzOLYhqsDzLvBA649W
p5
0
<
mean
0.35
SD
0.48
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkqqNqszQAua0fYy4AFzgQtrP27lCvuth7gHM1Im6jAVyiRb-r
p5
0
<
mean
0.05
SD
0.22
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkrWA8fY4Crcq4x1C4taaxmx_O0B218dTEJ1re0QqeARHtno9F
p5
0
<
mean
0.19
SD
0.39
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkrp01PB0qwYEyo2-VTCbBbRFvTREjARXZKBeIYOsOCX7IjlNZ
p5
0
<
mean
0.3
SD
0.46
<
p95
1
  • get_agent_activity_volatile_agent: uhCAksVgyFhQtqb0m9x8MSvEYOVunFN5hto3IOKsLhhaOHu-aJb9i
p5
0
<
mean
0.45
SD
0.5
<
p95
1
  • get_agent_activity_volatile_agent: uhCAksXyPw-hbWEd2ipBmlwj4yuf7jpHr7U2tnjMkRSd46Ymm7ahh
p5
0
<
mean
0.79
SD
0.4
<
p95
1
  • get_agent_activity_volatile_agent: uhCAktiklkgdtdEkcGoZ_vDX86qIiX6CQX0ytdQqbSrOZSxSezGKj
p5
0
<
mean
0.2
SD
0.4
<
p95
1
  • get_agent_activity_volatile_agent: uhCAktooVEnoAqdfJdMwe4UR3BfEuD4a8_-FjEGEzyexiz9rxafoc
p5
0
<
mean
0.25
SD
0.43
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkuf7rzcjhxHLB0KIga1p2PflgMmGKNF1e4ypo0HmdGp1sZLm-
p5
0
<
mean
0.33
SD
0.47
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkuotZavSRUtoViZmSsTbJuP1cN-6iba2GAHCxpa7Sl2v4OOoS
p5
0
<
mean
0.32
SD
0.47
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkvVRGGR9dvCOKLuJmLbVLJauOT-K8K85Z5kmFENGwYeGtoTV9
p5
0
<
mean
0.25
SD
0.43
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkw12m65hqetf6wuHJs5CBaZPtHeW9GN5qIdMEMfI05PxEPWPI
p5
0
<
mean
0.21
SD
0.41
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkw8caSvbZpXe0XfV9v-66kuLBxMnWe5JCmhYGhrqFN06PQ2oq
p5
0
<
mean
0.42
SD
0.49
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkwitXvUbx17k2fPg2pOcdBeXSxjq0acNrsUjUC8MK_xvxTYog
p5
0
<
mean
0.39
SD
0.49
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkxCkBQgGMl4Ys2VWbD2ohv8ZlaOkbS_J-ENQifyph37yeGuKw
p5
0
<
mean
0.32
SD
0.47
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkxENyNgo9UT1humNtd032qI5gh7is9zf_T6k-gn5bzaqZqGVa
p5
0
<
mean
0.29
SD
0.45
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkxyLmP2eh1HP2R_UW8Ru5h9a4PX3Kz4qy6nvuVYIEtvbCy7IA
p5
0
<
mean
0.47
SD
0.5
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkyEL_7Ob7XyYLIRT4P4xRrjKwLtl89U8x61KMgyhWF7_vDtx2
p5
0
<
mean
0.32
SD
0.47
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkyzjNBpwuP-BD4pzCBaF0zvW8nnI16jLkLqFsIaAT4yIl1eJq
p5
0
<
mean
0.26
SD
0.44
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkzgGznMmPLDh-WWOMFGmrY7anT-i3UY1W3emKYRgcdZWjgLYw
p5
0
<
mean
0.33
SD
0.47
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkzlyhOBZ8npSHuuD_wN7rDkYOGx7_2z15a0kvr3W5GiY88hWh
p5
0
<
mean
0.18
SD
0.38
<
p95
1
Error count
The number of errors accumulated.
1092
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.047178
s
SD
28.240412
s
p50
0.0863
s
<
p95
22.024015
s
<
p99
175.419195
s
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
19
over 1720.467059594s
mean rate
177.81
/s
std
894.4
/s
p5
0
/s
p95
691.96
/s
peak
46687.28
/s
Zome call duration: 'agent_activity::announce_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.006595
s
SD
0.002225
s
p50
0.006464
s
<
p95
0.010347
s
<
p99
0.012143
s
Zome call duration: 'agent_activity::create_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.005946
s
SD
0.001863
s
p50
0.006975
s
<
p95
0.008678
s
<
p99
0.009235
s
Zome call duration: 'agent_activity::get_agent_activity_full'
Duration of this zome call as measured by Holochain internally.
mean
0.023914
s
SD
4.81698
s
p50
0.075326
s
<
p95
2.470193
s
<
p99
14.504451
s
Zome call duration: 'agent_activity::get_random_agent_with_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.006098
s
SD
1.376537
s
p50
0.00902
s
<
p95
4.003646
s
<
p99
6.349029
s
WASM call duration: 'agent_activity::announce_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.005509
s
SD
0.001917
s
p50
0.005496
s
<
p95
0.008943
s
<
p99
0.010358
s
WASM call duration: 'agent_activity::create_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.004828
s
SD
0.001531
s
p50
0.005665
s
<
p95
0.006945
s
<
p99
0.007501
s
WASM call duration: 'agent_activity::get_agent_activity_full'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.051216
s
SD
28.902907
s
p50
0.090692
s
<
p95
22.032781
s
<
p99
175.957491
s
WASM call duration: 'agent_activity::get_random_agent_with_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.005229
s
SD
1.33818
s
p50
0.007638
s
<
p95
4.002413
s
<
p99
6.347248
s
WASM call duration: 'agent_activity_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001629
s
SD
0.000545
s
p50
0.001917
s
<
p95
0.002334
s
<
p99
0.002553
s
Host function call duration: '__hc__agent_info_1'
Duration of this host function call invoked from within WASM.
mean
5.3e-05
s
SD
4.4e-05
s
p50
3.8e-05
s
<
p95
0.00018
s
<
p99
0.000199
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000514
s
SD
0.000169
s
p50
0.000596
s
<
p95
0.000792
s
<
p99
0.000922
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.000542
s
SD
0.00024
s
p50
0.000527
s
<
p95
0.000948
s
<
p99
0.001443
s
Host function call duration: '__hc__get_agent_activity_1'
Duration of this host function call invoked from within WASM.
mean
0.049785
s
SD
28.902934
s
p50
0.088981
s
<
p95
22.0304
s
<
p99
175.956281
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.002342
s
SD
1.337641
s
p50
0.00101
s
<
p95
3.997718
s
<
p99
6.339576
s
Host function call duration: '__hc__random_bytes_1'
Duration of this host function call invoked from within WASM.
mean
7e-06
s
SD
3.2e-05
s
p50
9e-06
s
<
p95
2.4e-05
s
<
p99
0.000312
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.002501
s
SD
0.000908
s
p50
0.002851
s
<
p95
0.003925
s
<
p99
0.005097
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.000313
s
SD
0.00036
s
p50
0.000376
s
<
p95
0.000592
s
<
p99
0.000856
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
4.96
s
<
mean
335.37
s
SD
488.96
s
<
p95
1492.2
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
2066
over 1788.871266937s
mean rate
294843.93
/s
std
3.88252491e+06
/s
p5
0
/s
p95
1.0495379e+06
/s
peak
3.7013888889e+08
/s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
92.682692
s
SD
145.747252
s
p50
60.253267
s
<
p95
405.955308
s
<
p99
680.788514
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.11676
s
SD
0.126297
s
p50
0.027387
s
<
p95
0.33106
s
<
p99
0.499081
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.016707
s
SD
0.377381
s
p50
0.013655
s
<
p95
0.361526
s
<
p99
1.369835
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.797489
s
SD
1.050717
s
p50
0.081787
s
<
p95
2.985103
s
<
p99
5.351183
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
49.078278
s
SD
44.54934
s
p50
31.487482
s
<
p95
108.851777
s
<
p99
252.035751
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
19.611553
s
SD
35.106842
s
p50
0.021751
s
<
p95
58.828837
s
<
p99
170.166494
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.004167
s
SD
0.002378
s
p50
0.000661
s
<
p95
0.00711
s
<
p99
0.009921
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.006422
s
SD
0.120354
s
p50
0.003005
s
<
p95
0.026447
s
<
p99
0.258105
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000434
s
SD
0.001394
s
p50
0.00065
s
<
p95
0.003386
s
<
p99
0.007041
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
5.881742
s
SD
28.131016
s
p50
0.933634
s
<
p95
60.002652
s
<
p99
60.003366
s
P2P request roundtrip: 'get_links'
Time spent sending a get_links request and awaiting its response.
mean
1.42518
s
SD
2.105011
s
p50
0.723154
s
<
p95
5.312392
s
<
p99
13.031903
s
P2P request roundtrip: 'get_agent_activity'
Time spent sending a get_agent_activity request and awaiting its response.
mean
0.153119
s
SD
19.587815
s
p50
0.240084
s
<
p95
60.001228
s
<
p99
60.001796
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.350938
s
SD
27.502059
s
p50
0.580297
s
<
p95
60.001744
s
<
p99
60.002623
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
20.25
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
4.20
KiB/s
<
mean
126.24
KiB/s
SD
222.89
KiB/s
<
p95
434.82
KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
134
B/s
<
mean
23.73
KiB/s
SD
69.43
KiB/s
<
p95
114.56
KiB/s
Total bytes received
Total bytes received on primary network interface
count
29.91
GiB
mean
17.11
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
5.61
GiB
mean
3.21
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected

Warning CPU p99 reached 93.4%

Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected

Critical Heavy swap usage (73.8% swap used)

System overload anomaly
System overload anomaly was detected
Detected

Warning 3% of hosts overloaded (load5/ncpus > 1.0)

CPU usage
Total CPU usage and kernel CPU usage
Total
p5
1.67
%
<
mean
27.6
%
SD
25.75
%
<
p95
79.12
%
System
7.35
%
CPU percentiles
CPU usage percentiles
p50
21.19
%
p95
79.12
%
p99
93.44
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
103
hosts
mean time
108.80
s
Memory used percentage
Percentage of memory used
p5
6.44
%
<
mean
10.19
%
SD
7.56
%
<
p95
22.32
%
Memory available percentage
Percentage of available memory
p5
77.68
%
<
mean
89.81
%
SD
7.56
%
<
p95
93.56
%
Max host memory used
Maximum memory usage percentage across all hosts
max
80.66
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
73.75
%
Memory growth rate
Rate of memory growth over time
growth
0.27
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.22
MiB/s
Disk write throughput
Disk write throughput in MiB/s
39.23
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point /
0/118
hosts
Mount Point /efi-boot
0/2
hosts
Mount Point /etc/hostname
0/14
hosts
Mount Point /etc/hosts
0/14
hosts
Mount Point /etc/nomad.d/client-meta.json
0/12
hosts
Mount Point /etc/resolv.conf
0/14
hosts
Mount Point /nix/store
0/2
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
0.88
5 min
1.07
15 min
1.48
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
2.78
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
7.6723
%
SD
12.0024
%
<
p95
29.72
%
60 second average
7.4269
%
300 second average
10.0062
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0.0007
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0.0007
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
0.9138
%
SD
3.4892
%
<
p95
3.84
%
60 second average
0.9153
%
300 second average
1.5956
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
0.592
%
SD
2.6626
%
<
p95
2.16
%
60 second average
0.5886
%
300 second average
0.8047
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0
%
<
mean
23.2
%
SD
26.2
%
<
p95
76.73
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
91.48
KiB
<
mean
186.25
KiB
SD
115.88
KiB
<
p95
350.15
KiB
Holochain process threads
Number of threads in Holochain process
p5
1
threads
<
mean
28.3
threads
SD
15.97
threads
<
p95
36
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
0
file descriptors
<
mean
74.39
file descriptors
SD
25.84
file descriptors
<
p95
89
file descriptors

Write Validated must_get_agent_activity

↑ Back to index

A scenario where write agents create entries in batches of 10, while must_get_agent_activity agents each pick a random write agent and repeatedly attempt to create an entry that references the chain top of their latest batch. This reference means that the entry's validation function needs to make a must_get_agent_activity call.

The purpose of this scenario is to measure the time it takes for published agent activity data to be gossiped among authorities and become available to peers that query it via must_get_agent_activity.

This test is similar to Mixed-Arc must_get_agent_activity, but all agents are full-arc.

Started
Thu, 01 Oct 2026 05:48:50 UTC
Duration
900s
Peer count
150
Peer count at end
129
Behaviours
  • must_get_agent_activity (1 agent)
  • write (1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
write_validated_must_get_agent_activity_36809382471_1
Highest observed action_seq
The maximum chain length observed per write agent, aggregated across all reading agents. The reading dimension is collapsed by taking the maximum observed value for each write agent (any successful read counts as propagation). This reflects the DHT’s ability to propagate agent activity ops and make them available to querying peers.
Mean max (across write agents)
Mean of the highest chain head value observed per write agent. Each write agent contributes its maximum — the value seen by any reader.
441.33
Max
Highest chain head value observed for any single write agent.
852
Write agents observed
6
Chain batch delay timing
The time between a write agent's creation of a batch and a must_get_agent_activity agent's successful discovery of the batch and creation/self-validation of a new entry that depends on it.
Mean
mean
275.785939
s
SD
38.882258
s
Max
Highest per-partition mean latency
max mean
1903.536
s
Min
Lowest per-partition mean latency
min mean
7.397667
s
Chain batch delay rate
The rate at which a must_get_agent_activity agent was able to discover batches and create/self-validate new entries that depend on them.
Mean
mean
1.26
/s
Max
Highest per-partition mean rate
max
7.33
/s
Min
Lowest per-partition mean rate
min
0
/s
create_validated_sample_entry zome call timing
The time taken to complete the zome function call that creates the entry that depends on a write agent's source chain.
Mean
mean
0.330085
s
SD
0.150205
s
Max
Highest per-partition mean latency
max mean
4.443026
s
Min
Lowest per-partition mean latency
min mean
0.016496
s
Retrieval errors
A running accumulation of the errors encountered by an agent when attempting to self-validate actions that depend on must_get_agent_activity calls.
Total
47652
Agents affected
87 / 87
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
548
Max
Highest count in any single partition
763
Min
Lowest count in any single partition
153
Final error count
The total number of all types of error accumulated over the run by all agents.
86439
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.027062
s
SD
0.81063
s
p50
0.001689
s
<
p95
0.079185
s
<
p99
0.766727
s
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
0
over 782.956942141s
Not enough time data to show a trend.
Zome call duration: 'validated_must_get_agent_activity::announce_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.0073
s
SD
0.005321
s
p50
0.006027
s
<
p95
0.016081
s
<
p99
0.030088
s
Zome call duration: 'validated_must_get_agent_activity::create_sample_entries_batch'
Duration of this zome call as measured by Holochain internally.
mean
0.224035
s
SD
0.178979
s
p50
0.142063
s
<
p95
0.553391
s
<
p99
0.922286
s
Zome call duration: 'validated_must_get_agent_activity::create_validated_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.008629
s
SD
0.008499
s
p50
0.008132
s
<
p95
0.01701
s
<
p99
0.047784
s
Zome call duration: 'validated_must_get_agent_activity::get_chain_top_for_batch'
Duration of this zome call as measured by Holochain internally.
mean
0.037155
s
SD
1.016878
s
p50
0.007365
s
<
p95
0.020229
s
<
p99
1.400715
s
Zome call duration: 'validated_must_get_agent_activity::get_random_agent_with_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.037728
s
SD
0.891834
s
p50
0.007322
s
<
p95
0.453545
s
<
p99
3.611202
s
WASM call duration: 'validated_must_get_agent_activity::announce_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.006017
s
SD
0.004211
s
p50
0.0051
s
<
p95
0.014728
s
<
p99
0.022472
s
WASM call duration: 'validated_must_get_agent_activity::create_sample_entries_batch'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.222012
s
SD
0.177893
s
p50
0.140603
s
<
p95
0.548823
s
<
p99
0.915384
s
WASM call duration: 'validated_must_get_agent_activity::create_validated_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.006943
s
SD
0.007881
s
p50
0.006359
s
<
p95
0.014111
s
<
p99
0.042969
s
WASM call duration: 'validated_must_get_agent_activity::get_chain_top_for_batch'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.006586
s
SD
0.981895
s
p50
0.006604
s
<
p95
0.016109
s
<
p99
1.399239
s
WASM call duration: 'validated_must_get_agent_activity::get_random_agent_with_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.038337
s
SD
0.987727
s
p50
0.006033
s
<
p95
0.452379
s
<
p99
3.610345
s
WASM call duration: 'validated_must_get_agent_activity_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001402
s
SD
0.000875
s
p50
0.001708
s
<
p95
0.003663
s
<
p99
0.005192
s
WASM call duration: 'validated_must_get_agent_activity_integrity::validate'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.013811
s
SD
0.082201
s
p50
0.008737
s
<
p95
0.03907
s
<
p99
0.06152
s
Host function call duration: '__hc__agent_info_1'
Duration of this host function call invoked from within WASM.
mean
6.6e-05
s
SD
5.9e-05
s
p50
3.9e-05
s
<
p95
0.000157
s
<
p99
0.000294
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.003023
s
SD
0.005871
s
p50
0.000923
s
<
p95
0.010783
s
<
p99
0.037791
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.004521
s
SD
0.00635
s
p50
0.000712
s
<
p95
0.01279
s
<
p99
0.038148
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.001865
s
SD
0.985221
s
p50
0.000786
s
<
p95
0.052757
s
<
p99
1.393146
s
Host function call duration: '__hc__must_get_agent_activity_1'
Duration of this host function call invoked from within WASM.
mean
0.028486
s
SD
0.465769
s
p50
0.020475
s
<
p95
0.0801
s
<
p99
0.112729
s
Host function call duration: '__hc__query_1'
Duration of this host function call invoked from within WASM.
mean
0.110644
s
SD
0.073168
s
p50
0.070263
s
<
p95
0.227969
s
<
p99
0.287732
s
Host function call duration: '__hc__random_bytes_1'
Duration of this host function call invoked from within WASM.
mean
1.9e-05
s
SD
1.4e-05
s
p50
1e-05
s
<
p95
2.3e-05
s
<
p99
0.000132
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.002623
s
SD
0.002248
s
p50
0.002982
s
<
p95
0.006671
s
<
p99
0.014227
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.001838
s
SD
0.00191
s
p50
0.000293
s
<
p95
0.004134
s
<
p99
0.009926
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
42.09
s
<
mean
427.88
s
SD
250.99
s
<
p95
824.8
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
13434
over 889.361651033s
mean rate
358195.25
/s
std
2.49171869e+06
/s
p5
4078.64
/s
p95
1.15899113e+06
/s
peak
1.2590983161e+08
/s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
140.594128
s
SD
106.646166
s
p50
22.206525
s
<
p95
330.555463
s
<
p99
467.420436
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
2.091559
s
SD
1.097362
s
p50
2.223609
s
<
p95
3.35308
s
<
p99
7.41105
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.864824
s
SD
0.44657
s
p50
0.47773
s
<
p95
1.381712
s
<
p99
1.598782
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.535427
s
SD
0.358149
s
p50
0.10124
s
<
p95
0.792892
s
<
p99
1.661366
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
12.766823
s
SD
15.65898
s
p50
13.733154
s
<
p95
52.57585
s
<
p99
76.028795
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
15.92938
s
SD
35.087321
s
p50
0.165374
s
<
p95
105.316174
s
<
p99
150.053525
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.003523
s
SD
0.002627
s
p50
0.003477
s
<
p95
0.00868
s
<
p99
0.010831
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.003474
s
SD
0.002774
s
p50
0.002658
s
<
p95
0.007221
s
<
p99
0.014483
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.00265
s
SD
0.005412
s
p50
0.00084
s
<
p95
0.009439
s
<
p99
0.035509
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
1.309596
s
SD
29.205237
s
p50
0.829641
s
<
p95
60.002379
s
<
p99
60.009204
s
P2P request roundtrip: 'get_links'
Time spent sending a get_links request and awaiting its response.
mean
1.130168
s
SD
3.062909
s
p50
0.615813
s
<
p95
3.57596
s
<
p99
17.608728
s
P2P request roundtrip: 'must_get_agent_activity'
Time spent sending a must_get_agent_activity request and awaiting its response.
mean
1.417744
s
SD
5.986373
s
p50
0.014135
s
<
p95
4.65361
s
<
p99
13.849521
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.703491
s
SD
20.178537
s
p50
1.214528
s
<
p95
60.001471
s
<
p99
60.00464
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
47.28
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
53.55
KiB/s
<
mean
517.45
KiB/s
SD
477.74
KiB/s
<
p95
1.41
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
4.05
KiB/s
<
mean
105.11
KiB/s
SD
144.59
KiB/s
<
p95
413.39
KiB/s
Total bytes received
Total bytes received on primary network interface
count
57.53
GiB
mean
66.19
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
11.66
GiB
mean
13.42
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected

Warning CPU p99 reached 98.2%

Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected

Critical Heavy swap usage (73.9% swap used)

System overload anomaly
System overload anomaly was detected
Detected

Warning 56% of hosts overloaded (load5/ncpus > 1.0)

CPU usage
Total CPU usage and kernel CPU usage
Total
p5
11.86
%
<
mean
67.54
%
SD
23.59
%
<
p95
93.94
%
System
20.26
%
CPU percentiles
CPU usage percentiles
p50
73.75
%
p95
93.94
%
p99
98.20
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
125
hosts
mean time
365.20
s
Memory used percentage
Percentage of memory used
p5
6.83
%
<
mean
13.32
%
SD
8.36
%
<
p95
26.28
%
Memory available percentage
Percentage of available memory
p5
73.72
%
<
mean
86.68
%
SD
8.36
%
<
p95
93.17
%
Max host memory used
Maximum memory usage percentage across all hosts
max
87.27
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
73.92
%
Memory growth rate
Rate of memory growth over time
growth
0.95
MiB/s
Disk read throughput
Disk read throughput in MiB/s
2.66
MiB/s
Disk write throughput
Disk write throughput in MiB/s
146.19
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point /
0/106
hosts
Mount Point /efi-boot
0/2
hosts
Mount Point /etc/hostname
0/13
hosts
Mount Point /etc/hosts
0/13
hosts
Mount Point /etc/nomad.d/client-meta.json
0/17
hosts
Mount Point /etc/resolv.conf
0/13
hosts
Mount Point /nix/store
0/2
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
3.27
5 min
2.49
15 min
1.80
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
55.88
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0.32
%
<
mean
41.128
%
SD
26.3683
%
<
p95
81.35
%
60 second average
38.1949
%
300 second average
28.5019
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
0.0016
%
SD
0.1208
%
≤
p95
0
%
60 second average
0.0009
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
0.0013
%
SD
0.0993
%
≤
p95
0
%
60 second average
0.0007
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
9.256
%
SD
7.3163
%
<
p95
19.76
%
60 second average
8.6830
%
300 second average
6.3643
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
4.6178
%
SD
4.7052
%
<
p95
13.53
%
60 second average
4.3873
%
300 second average
3.3379
%
Holochain process CPU usage
CPU usage by Holochain process
p5
1.34
%
<
mean
60.71
%
SD
24.2
%
<
p95
89.96
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
110.11
KiB
<
mean
402.12
KiB
SD
109.23
KiB
<
p95
541.60
KiB
Holochain process threads
Number of threads in Holochain process
p5
6
threads
<
mean
33.77
threads
SD
10.42
threads
<
p95
45
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
12
file descriptors
<
mean
78.34
file descriptors
SD
19.93
file descriptors
<
p95
89
file descriptors

Zero-Arc Create and Read

↑ Back to index

A zero-arc/full-arc mixed scenario with two types of zero-arc nodes -- ones that create data and ones that read data -- as well as full arc nodes to "relay" the data. The scenario has three roles:

  • zero_write: A zero-arc conductor that just creates entries with a timestamp field. Those entries are linked to a known base hash so that zero_read nodes can retrieve them.
  • zero_read: A zero-arc conductor that reads the entries created by the zero-arc node(s) and records the time lag between when the entry had been created and when it was first discovered.
  • full: A full-arc conductor that is just here to serve entries to zero arc nodes.
Started
Thu, 01 Oct 2026 07:19:24 UTC
Duration
900s
Peer count
150
Peer count at end
136
Behaviours
  • full (1 agent)
  • zero_read (1 agent)
  • zero_write (1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
zero_arc_create_and_read_36809382471_1
Create rate
The number of timed entries created by the zero-arc write node(s) per second.
Mean
mean
51.8
/s
Max
Highest per-partition mean rate
max
79.59
/s
Min
Lowest per-partition mean rate
min
22.19
/s
Fetch lag timing
For each entry, the time lag between when it was created and when a zero-arc read node fetched it from the network via the get_timed_entries_network zome function.
Mean
mean
4.39535987317e+31
s
SD
3.63049518828e+31
s
Max
Highest per-partition mean latency
max mean
3.09347606292e+32
s
Min
Lowest per-partition mean latency
min mean
9.281844
s
Fetch lag rate
The number of entries per second fetched by zero-arc read nodes.
Mean
mean
31.74
/s
Max
Highest per-partition mean rate
max
326
/s
Min
Lowest per-partition mean rate
min
0
/s
Delivery ratio
The proportion of created entries successfully fetched by all zero-arc read nodes.
0.0004
Entry created count
Total number of entries created by zero-arc write nodes, by agent.
Total
282953
Agents affected
60 / 60
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
4716
Max
Highest count in any single partition
7565
Min
Lowest count in any single partition
1947
Received count
Total number of entries fetched by zero-arc read nodes, by agent.
Total
7056
Agents affected
46 / 57
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
124
Max
Highest count in any single partition
767
Min
Lowest count in any single partition
0
Open connections
The number of currently open connections to other conductors.
full-arc
p5
2
<
mean
37.11
SD
45.53
<
p95
142
zero-arc
p5
2
<
mean
16.76
SD
11.92
<
p95
41
Error count
The number of errors accumulated across all nodes.
736
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.330256
s
SD
1.1622
s
p50
0.465652
s
<
p95
3.535682
s
<
p99
5.683939
s
Zome call duration: 'timed::created_timed_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.007938
s
SD
0.003384
s
p50
0.009369
s
<
p95
0.013067
s
<
p99
0.019457
s
Zome call duration: 'timed::get_timed_entries_network'
Duration of this zome call as measured by Holochain internally.
mean
4.875374
s
SD
49.91935
s
p50
12.903704
s
<
p95
129.607771
s
<
p99
230.982942
s
WASM call duration: 'timed::created_timed_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.006965
s
SD
0.003
s
p50
0.008176
s
<
p95
0.011505
s
<
p99
0.017276
s
WASM call duration: 'timed::get_timed_entries_network'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
4.900896
s
SD
49.906977
s
p50
12.92802
s
<
p95
129.607129
s
<
p99
230.981742
s
WASM call duration: 'timed_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001265
s
SD
0.000791
s
p50
0.001581
s
<
p95
0.002434
s
<
p99
0.003012
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000683
s
SD
0.000557
s
p50
0.000699
s
<
p95
0.001773
s
<
p99
0.002997
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.000627
s
SD
0.000481
s
p50
0.00068
s
<
p95
0.001527
s
<
p99
0.002317
s
Host function call duration: '__hc__get_1'
Duration of this host function call invoked from within WASM.
mean
0.333014
s
SD
1.00445
s
p50
0.447259
s
<
p95
3.3583
s
<
p99
5.367827
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.250516
s
SD
2.372648
s
p50
0.481483
s
<
p95
7.026957
s
<
p99
10.514027
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.002056
s
SD
0.001582
s
p50
0.002542
s
<
p95
0.003845
s
<
p99
0.00595
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.000476
s
SD
0.003188
s
p50
0.000327
s
<
p95
0.001895
s
<
p99
0.013594
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
44.95
s
<
mean
427.92
s
SD
248.55
s
<
p95
822.82
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 889.262364758s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
1.008935
s
SD
4.916697
s
p50
0
s
<
p95
3.014638
s
<
p99
29.651274
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.251913
s
SD
0.331169
s
p50
0.001812
s
<
p95
0.627517
s
<
p99
1.556619
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.043882
s
SD
0.401741
s
p50
0.00192
s
<
p95
1.259869
s
<
p99
1.395335
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.157683
s
SD
0.078798
s
p50
0.026016
s
<
p95
0.226104
s
<
p99
0.286352
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
11.897156
s
SD
13.22828
s
p50
0.003081
s
<
p95
28.229792
s
<
p99
69.549011
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
1.75808
s
SD
9.107566
s
p50
0.001256
s
<
p95
0.084809
s
<
p99
1.107903
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.00283
s
SD
0.002179
s
p50
0.001411
s
<
p95
0.005883
s
<
p99
0.008957
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.004613
s
SD
0.0041
s
p50
0.00277
s
<
p95
0.009105
s
<
p99
0.012986
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000842
s
SD
0.004227
s
p50
0.000645
s
<
p95
0.001888
s
<
p99
0.025578
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
0.375874
s
SD
29.734248
s
p50
1.000868
s
<
p95
60.013607
s
<
p99
60.055758
s
P2P request roundtrip: 'get_links'
Time spent sending a get_links request and awaiting its response.
mean
0.197439
s
SD
22.18525
s
p50
0.274348
s
<
p95
60.001188
s
<
p99
60.002659
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.292859
s
SD
1.160369
s
p50
0.206701
s
<
p95
4.397342
s
<
p99
4.544886
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
34.01
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
6.15
KiB/s
<
mean
439.12
KiB/s
SD
445.11
KiB/s
<
p95
1.17
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
929
B/s
<
mean
89.32
KiB/s
SD
145.81
KiB/s
<
p95
356.41
KiB/s
Total bytes received
Total bytes received on primary network interface
count
51.34
GiB
mean
59.07
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
10.44
GiB
mean
12.02
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected

Warning CPU p99 reached 95.5%

Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected

Critical Heavy swap usage (72.4% swap used)

System overload anomaly
System overload anomaly was detected
Detected

Warning 45% of hosts overloaded (load5/ncpus > 1.0)

CPU usage
Total CPU usage and kernel CPU usage
Total
p5
1.66
%
<
mean
45.87
%
SD
37.49
%
<
p95
90.26
%
System
11.86
%
CPU percentiles
CPU usage percentiles
p50
59.26
%
p95
90.26
%
p99
95.55
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
85
hosts
mean time
509.70
s
Memory used percentage
Percentage of memory used
p5
6.87
%
<
mean
11.97
%
SD
7.79
%
<
p95
22.81
%
Memory available percentage
Percentage of available memory
p5
77.19
%
<
mean
88.03
%
SD
7.79
%
<
p95
93.13
%
Max host memory used
Maximum memory usage percentage across all hosts
max
85.52
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
72.38
%
Memory growth rate
Rate of memory growth over time
growth
0.52
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.84
MiB/s
Disk write throughput
Disk write throughput in MiB/s
101.90
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point /
0/114
hosts
Mount Point /efi-boot
0/2
hosts
Mount Point /etc/hostname
0/13
hosts
Mount Point /etc/hosts
0/13
hosts
Mount Point /etc/nomad.d/client-meta.json
0/15
hosts
Mount Point /etc/resolv.conf
0/13
hosts
Mount Point /nix/store
0/2
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
1.76
5 min
1.81
15 min
1.96
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
45.07
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
19.8477
%
SD
21.7708
%
<
p95
54.73
%
60 second average
19.1626
%
300 second average
19.4028
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
0.0001
%
SD
0.0089
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
0.0001
%
SD
0.0089
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
2.6349
%
SD
4.3386
%
<
p95
13.47
%
60 second average
2.5423
%
300 second average
2.4082
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
1.181
%
SD
1.9814
%
<
p95
6.07
%
60 second average
1.1360
%
300 second average
1.1174
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.37
%
<
mean
40.84
%
SD
36.1
%
<
p95
86.67
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
208.76
KiB
<
mean
280.99
KiB
SD
85.09
KiB
<
p95
418.88
KiB
Holochain process threads
Number of threads in Holochain process
p5
9
threads
<
mean
29.77
threads
SD
9.75
threads
<
p95
36
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
23
file descriptors
<
mean
71.97
file descriptors
SD
18.89
file descriptors
<
p95
89
file descriptors

Zero-Arc Create Data

↑ Back to index
A zero-arc/full-arc mixed scenario where zero-arc nodes create data and full-arc nodes read the data. The scenario has two roles:
  • zero: A zero-arc conductor that just creates entries with a timestamp field. Those entries are linked to a known base hash so that full-arc nodes can retrieve them.
  • full: A full-arc conductor that reads the entries created by the zero-arc node(s) and records the time lag between when the entry had been created and when it was first discovered.
Started
Thu, 01 Oct 2026 06:25:57 UTC
Duration
900s
Peer count
150
Peer count at end
140
Behaviours
  • full (1 agent)
  • zero (1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
zero_arc_create_data--high_zero_36809382471_1
Create rate
The number of timed entries created by the zero-arc node(s) per second.
Mean
mean
50.95
/s
Max
Highest per-partition mean rate
max
81.98
/s
Min
Lowest per-partition mean rate
min
25.61
/s
Sync lag timing
For each entry, the time lag between when it was created and when the full-arc node could read it via the get_timed_local_entries zome function.
Mean
mean
8.79737246434e+28
s
SD
2.4452897729000002e+30
s
Max
Highest per-partition mean latency
max mean
4.39868623217e+29
s
Min
Lowest per-partition mean latency
min mean
0.117043
s
Sync lag rate
The number of entries per second received by full nodes.
Mean
mean
332.8
/s
Max
Highest per-partition mean rate
max
1664
/s
Min
Lowest per-partition mean rate
min
0
/s
Delivery ratio
The proportion of created entries successfully received by all full-arc nodes.
0.0008
Entry created count
Total number of entries created by zero-arc nodes, by agent.
Total
644197
Agents affected
140 / 140
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
4601
Max
Highest count in any single partition
7696
Min
Lowest count in any single partition
2200
Received count
Total number of entries received by full-arc nodes, by agent.
Total
3871
Agents affected
3 / 8
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
484
Max
Highest count in any single partition
3868
Min
Lowest count in any single partition
0
Open connections
The number of currently open connections to other conductors.
full-arc
p5
6
<
mean
50.67
SD
38.94
<
p95
139
zero-arc
p5
2
<
mean
13.4
SD
11.28
<
p95
38
Error count
The number of errors accumulated across all nodes.
0
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.000334
s
SD
0.000955
s
p50
0.001137
s
<
p95
0.003412
s
<
p99
0.003838
s
WASM usage: 'timed::created_timed_entry'
Metered usage count of the WASM ribosome for this zome function.
total
272251488
over 876.228503187s
mean rate
2.56468179742e+10
/s
std
4.522472957e+11
/s
p5
1.915171812e+07
/s
p95
1.31203892035e+10
/s
peak
2.15308118742e+13
/s
WASM usage: 'timed::get_timed_entries_local'
Metered usage count of the WASM ribosome for this zome function.
total
254060019
over 884.162955049s
mean rate
7.7100156887e+10
/s
std
1.90481125375e+11
/s
p5
1.015848588e+07
/s
p95
6.08021128188e+11
/s
peak
8.31295861191e+11
/s
WASM usage: 'timed_integrity::entry_defs'
Metered usage count of the WASM ribosome for this zome function.
total
37246734
over 889.601744743s
mean rate
3.10742079187e+09
/s
std
5.71281544824e+10
/s
p5
1.58178141e+06
/s
p95
1.71588803194e+09
/s
peak
2.90403583348e+12
/s
Zome call duration: 'timed::created_timed_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.007251
s
SD
0.003548
s
p50
0.007608
s
<
p95
0.014047
s
<
p99
0.023414
s
Zome call duration: 'timed::get_timed_entries_local'
Duration of this zome call as measured by Holochain internally.
mean
0.026686
s
SD
0.019025
s
p50
0.016672
s
<
p95
0.065598
s
<
p99
0.100224
s
WASM call duration: 'timed::created_timed_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.006342
s
SD
0.003162
s
p50
0.006633
s
<
p95
0.012219
s
<
p99
0.021125
s
WASM call duration: 'timed::get_timed_entries_local'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.025395
s
SD
0.018966
s
p50
0.014763
s
<
p95
0.064973
s
<
p99
0.099593
s
WASM call duration: 'timed_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001293
s
SD
0.000739
s
p50
0.001372
s
<
p95
0.002815
s
<
p99
0.004162
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000426
s
SD
0.000247
s
p50
0.000438
s
<
p95
0.000828
s
<
p99
0.001751
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.00041
s
SD
0.000237
s
p50
0.000407
s
<
p95
0.000813
s
<
p99
0.001597
s
Host function call duration: '__hc__get_1'
Duration of this host function call invoked from within WASM.
mean
0.000314
s
SD
0.001244
s
p50
0.001911
s
<
p95
0.003957
s
<
p99
0.004282
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.001821
s
SD
0.001346
s
p50
0.001251
s
<
p95
0.004328
s
<
p99
0.006427
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.002065
s
SD
0.001921
s
p50
0.002053
s
<
p95
0.00567
s
<
p99
0.012422
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.000394
s
SD
0.000436
s
p50
0.000477
s
<
p95
0.001099
s
<
p99
0.003216
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
42.78
s
<
mean
424.96
s
SD
248.64
s
<
p95
822.39
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 889.601770395s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
9.258562
s
SD
19.428706
s
p50
0
s
≤
p95
0
s
<
p99
1.836056
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.124746
s
SD
0.169639
s
p50
0.001655
s
<
p95
0.149172
s
<
p99
0.777417
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.009015
s
SD
0.264121
s
p50
0.00164
s
<
p95
0.908852
s
<
p99
1.292176
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.133832
s
SD
0.062553
s
p50
0.118471
s
<
p95
0.222542
s
<
p99
0.286979
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
6.716777
s
SD
8.306309
s
p50
0.002602
s
<
p95
2.196992
s
<
p99
30.794956
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
1.173617
s
SD
12.776497
s
p50
0.001202
s
<
p95
0.014618
s
<
p99
0.088779
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.006234
s
SD
0.004197
s
p50
0.008114
s
<
p95
0.014857
s
<
p99
0.017636
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.005974
s
SD
0.004508
s
p50
0.003181
s
<
p95
0.012529
s
<
p99
0.025114
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000334
s
SD
0.001845
s
p50
0.000346
s
<
p95
0.000919
s
<
p99
0.004689
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
3.439355
s
SD
28.010443
s
p50
10.739129
s
<
p95
60.11087
s
≤
p99
60.11087
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.280854
s
SD
0.901041
s
p50
0.272461
s
<
p95
2.439589
s
≤
p99
2.439589
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
55.95
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
12.45
KiB/s
<
mean
177.20
KiB/s
SD
349.77
KiB/s
<
p95
646.37
KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
742
B/s
<
mean
47.19
KiB/s
SD
66.34
KiB/s
<
p95
178.46
KiB/s
Total bytes received
Total bytes received on primary network interface
count
21.40
GiB
mean
24.62
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
5.70
GiB
mean
6.56
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected

Warning CPU p99 reached 96.6%

Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected

Critical Heavy swap usage (72.9% swap used)

System overload anomaly
System overload anomaly was detected
Detected

Warning 53% of hosts overloaded (load5/ncpus > 1.0)

CPU usage
Total CPU usage and kernel CPU usage
Total
p5
14.93
%
<
mean
71.85
%
SD
22.06
%
<
p95
93.68
%
System
15.90
%
CPU percentiles
CPU usage percentiles
p50
80.29
%
p95
93.68
%
p99
96.61
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
127
hosts
mean time
535.47
s
Memory used percentage
Percentage of memory used
p5
6.71
%
<
mean
12.01
%
SD
8.08
%
<
p95
23.54
%
Memory available percentage
Percentage of available memory
p5
76.46
%
<
mean
87.99
%
SD
8.08
%
<
p95
93.29
%
Max host memory used
Maximum memory usage percentage across all hosts
max
87.33
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
72.89
%
Memory growth rate
Rate of memory growth over time
growth
0.72
MiB/s
Disk read throughput
Disk read throughput in MiB/s
1.08
MiB/s
Disk write throughput
Disk write throughput in MiB/s
107.89
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point /
0/119
hosts
Mount Point /efi-boot
0/2
hosts
Mount Point /etc/hostname
0/13
hosts
Mount Point /etc/hosts
0/13
hosts
Mount Point /etc/nomad.d/client-meta.json
0/16
hosts
Mount Point /etc/resolv.conf
0/13
hosts
Mount Point /nix/store
0/2
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
2.46
5 min
2.24
15 min
2.15
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
53.38
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0.71
%
<
mean
26.4489
%
SD
17.9304
%
<
p95
58.22
%
60 second average
25.6137
%
300 second average
23.6710
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
3.1625
%
SD
3.7898
%
<
p95
8.24
%
60 second average
3.0718
%
300 second average
2.8904
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
1.8408
%
SD
2.8216
%
<
p95
4.36
%
60 second average
1.7822
%
300 second average
1.6359
%
Holochain process CPU usage
CPU usage by Holochain process
p5
5.43
%
<
mean
65.17
%
SD
22.66
%
<
p95
86.4
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
204.88
KiB
<
mean
285.66
KiB
SD
68.39
KiB
<
p95
373.87
KiB
Holochain process threads
Number of threads in Holochain process
p5
19
threads
<
mean
31.56
threads
SD
9.57
threads
<
p95
37
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
60
file descriptors
<
mean
76.39
file descriptors
SD
17.27
file descriptors
<
p95
87
file descriptors

Zero-arc Create Data (Validated)

↑ Back to index

A zero-arc/full-arc mixed scenario where zero-arc nodes create data that gets validated and full-arc nodes read the data. The scenario has two roles:

  • zero: A zero-arc conductor that just creates entries with a timestamp field. Those entries are linked to a known base hash so that full-arc nodes can retrieve them.
  • full: A full-arc conductor that reads the entries created by the zero-arc node(s) and records the time lag between when the entry had been created and when it was first discovered.
Started
Thu, 01 Oct 2026 07:02:07 UTC
Duration
900s
Peer count
150
Peer count at end
139
Behaviours
  • full (1 agent)
  • zero (1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
zero_arc_create_data_validated--high_zero_36809382471_1
Create rate
The number of timed entries created by the zero-arc node(s) per second.
Mean
mean
46.25
/s
Max
Highest per-partition mean rate
max
78.37
/s
Min
Lowest per-partition mean rate
min
25.28
/s
Sync lag timing
For each entry, the time lag between when it was created and when the full-arc node could read it via the get_timed_local_entries zome function.
Mean
mean
114.630008
s
SD
44.024984
s
Max
Highest per-partition mean latency
max mean
419.537739
s
Min
Lowest per-partition mean latency
min mean
8.266052
s
Sync lag rate
The number of entries per second received by full nodes.
Mean
mean
95
/s
Max
Highest per-partition mean rate
max
475
/s
Min
Lowest per-partition mean rate
min
0
/s
Delivery ratio
The proportion of created entries successfully received by all full-arc nodes.
0.0004
Entry created count
Total number of entries created by zero-arc nodes, by agent.
Total
559625
Agents affected
138 / 138
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
4055
Max
Highest count in any single partition
6930
Min
Lowest count in any single partition
2066
Received count
Total number of entries received by full-arc nodes, by agent.
Total
2110
Agents affected
4 / 9
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
234
Max
Highest count in any single partition
1781
Min
Lowest count in any single partition
0
Open connections
The number of currently open connections to other conductors.
full-arc
p5
7
<
mean
59.83
SD
39.48
<
p95
129
zero-arc
p5
2
<
mean
14.04
SD
11.48
<
p95
39
Error count
The number of errors accumulated across all nodes.
0
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.008197
s
SD
0.005212
s
p50
0.009023
s
<
p95
0.018512
s
<
p99
0.023947
s
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
12
over 783.194350069s
mean rate
12043.43
/s
std
14777.37
/s
p5
4.47
/s
p95
44344.27
/s
peak
48806.17
/s
WASM usage: 'timed_and_validated::create_timed_entry'
Metered usage count of the WASM ribosome for this zome function.
total
254858712
over 862.626281602s
mean rate
1.29803408294e+10
/s
std
4.29918009615e+11
/s
p5
2.127255145e+07
/s
p95
1.55384159472e+10
/s
peak
2.97511070853e+13
/s
WASM usage: 'timed_and_validated::get_timed_entries_local'
Metered usage count of the WASM ribosome for this zome function.
total
3177101882
over 871.143718415s
mean rate
7.4628551873e+08
/s
std
8.3887858424e+08
/s
p5
7.20462706e+06
/s
p95
2.63869989367e+09
/s
peak
2.90379568132e+09
/s
WASM usage: 'timed_and_validated_integrity::entry_defs'
Metered usage count of the WASM ribosome for this zome function.
total
149290308
over 888.223980038s
mean rate
7.03547368579e+09
/s
std
2.38902737674e+11
/s
p5
6.91376804e+06
/s
p95
8.12730714192e+09
/s
peak
1.73254830918e+13
/s
WASM usage: 'timed_and_validated_integrity::validate'
Metered usage count of the WASM ribosome for this zome function.
total
1180701616
over 888.223980038s
mean rate
5.78025850629e+10
/s
std
1.91010739407e+12
/s
p5
7.328824882e+07
/s
p95
8.4777478353e+10
/s
peak
1.37023349436e+14
/s
Zome call duration: 'timed_and_validated::create_timed_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.007669
s
SD
0.003997
s
p50
0.008312
s
<
p95
0.018389
s
<
p99
0.021687
s
Zome call duration: 'timed_and_validated::get_timed_entries_local'
Duration of this zome call as measured by Holochain internally.
mean
0.047193
s
SD
0.169367
s
p50
0.021039
s
<
p95
0.54318
s
<
p99
0.669256
s
WASM call duration: 'timed_and_validated::create_timed_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.0067
s
SD
0.003565
s
p50
0.007216
s
<
p95
0.016386
s
<
p99
0.019397
s
WASM call duration: 'timed_and_validated::get_timed_entries_local'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.045522
s
SD
0.169382
s
p50
0.018517
s
<
p95
0.541172
s
<
p99
0.667444
s
WASM call duration: 'timed_and_validated_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001358
s
SD
0.000827
s
p50
0.001405
s
<
p95
0.003352
s
<
p99
0.004086
s
WASM call duration: 'timed_and_validated_integrity::validate'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.006329
s
SD
0.004077
s
p50
0.005468
s
<
p95
0.014134
s
<
p99
0.022908
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000429
s
SD
0.000268
s
p50
0.000449
s
<
p95
0.001174
s
<
p99
0.001492
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.000415
s
SD
0.000249
s
p50
0.000417
s
<
p95
0.001123
s
<
p99
0.001379
s
Host function call duration: '__hc__get_1'
Duration of this host function call invoked from within WASM.
mean
0.004594
s
SD
0.001045
s
p50
0.003851
s
<
p95
0.005758
s
<
p99
0.006084
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.001937
s
SD
0.004434
s
p50
0.001506
s
<
p95
0.015183
s
<
p99
0.020135
s
Host function call duration: '__hc__must_get_valid_record_1'
Duration of this host function call invoked from within WASM.
mean
0.009412
s
SD
0.005038
s
p50
0.009476
s
<
p95
0.0188
s
<
p99
0.024125
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.002378
s
SD
0.002118
s
p50
0.002227
s
<
p95
0.006686
s
<
p99
0.01245
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.000493
s
SD
0.001552
s
p50
0.000607
s
<
p95
0.001605
s
<
p99
0.009923
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
44.62
s
<
mean
427.29
s
SD
248.04
s
<
p95
817.21
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 888.224102086s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
2.934268
s
SD
15.998105
s
p50
0
s
<
p95
0.833801
s
<
p99
75.021677
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
5.181698
s
SD
2.025154
s
p50
0.0018
s
<
p95
3.477488
s
<
p99
11.02726
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
1e-06
s
SD
4e-06
s
p50
0
s
≤
p95
0
s
<
p99
3.2e-05
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.011187
s
SD
0.262633
s
p50
0.002084
s
<
p95
1.009245
s
<
p99
1.221869
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.123505
s
SD
0.060737
s
p50
0.105246
s
<
p95
0.215886
s
<
p99
0.265533
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
5.649856
s
SD
4.688632
s
p50
0.002865
s
<
p95
6.95092
s
<
p99
13.071956
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
1.03756
s
SD
6.168782
s
p50
0.001242
s
<
p95
0.017874
s
<
p99
0.282537
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.006161
s
SD
0.003361
s
p50
0.00618
s
<
p95
0.012033
s
<
p99
0.014669
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.005723
s
SD
0.004184
s
p50
0.003352
s
<
p95
0.012605
s
<
p99
0.019061
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000368
s
SD
0.003342
s
p50
0.000353
s
<
p95
0.001205
s
<
p99
0.015566
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
10.178624
s
SD
29.585213
s
p50
4.40825
s
<
p95
60.041001
s
<
p99
60.045182
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.389943
s
SD
17.166193
s
p50
0.324173
s
<
p95
60.055471
s
<
p99
60.066069
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
52.38
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
4.99
KiB/s
<
mean
176.14
KiB/s
SD
306.56
KiB/s
<
p95
558.38
KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
555
B/s
<
mean
44.49
KiB/s
SD
53.63
KiB/s
<
p95
169.33
KiB/s
Total bytes received
Total bytes received on primary network interface
count
21.06
GiB
mean
24.23
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
5.32
GiB
mean
6.12
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected

Warning CPU p99 reached 97.2%

Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected

Critical Heavy swap usage (74.1% swap used)

System overload anomaly
System overload anomaly was detected
Detected

Warning 37% of hosts overloaded (load5/ncpus > 1.0)

CPU usage
Total CPU usage and kernel CPU usage
Total
p5
23.23
%
<
mean
69.88
%
SD
20.62
%
<
p95
92.44
%
System
17.49
%
CPU percentiles
CPU usage percentiles
p50
76.49
%
p95
92.44
%
p99
97.25
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
124
hosts
mean time
374.79
s
Memory used percentage
Percentage of memory used
p5
7.83
%
<
mean
13.74
%
SD
7.98
%
<
p95
25.74
%
Memory available percentage
Percentage of available memory
p5
74.26
%
<
mean
86.26
%
SD
7.98
%
<
p95
92.17
%
Max host memory used
Maximum memory usage percentage across all hosts
max
86.79
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
74.08
%
Memory growth rate
Rate of memory growth over time
growth
0.50
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.21
MiB/s
Disk write throughput
Disk write throughput in MiB/s
96.72
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point /
0/117
hosts
Mount Point /efi-boot
0/2
hosts
Mount Point /etc/hostname
0/13
hosts
Mount Point /etc/hosts
0/13
hosts
Mount Point /etc/nomad.d/client-meta.json
0/16
hosts
Mount Point /etc/resolv.conf
0/13
hosts
Mount Point /nix/store
0/2
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
2.40
5 min
2.21
15 min
2.13
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
36.99
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0.38
%
<
mean
27.1568
%
SD
18.3489
%
<
p95
60.74
%
60 second average
26.1921
%
300 second average
23.9148
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
2.6779
%
SD
3.3244
%
<
p95
7.94
%
60 second average
2.5970
%
300 second average
2.3474
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
1.4
%
SD
2.3104
%
<
p95
3.31
%
60 second average
1.3558
%
300 second average
1.2476
%
Holochain process CPU usage
CPU usage by Holochain process
p5
17.61
%
<
mean
63.18
%
SD
20.96
%
<
p95
84.82
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
181.36
KiB
<
mean
384.91
KiB
SD
103.51
KiB
<
p95
512.16
KiB
Holochain process threads
Number of threads in Holochain process
p5
14
threads
<
mean
32.96
threads
SD
9.05
threads
<
p95
38
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
44
file descriptors
<
mean
77.06
file descriptors
SD
16.4
file descriptors
<
p95
89
file descriptors