Conductor Startup

↑ Back to index

This scenario measures Holochain conductor startup time when many cells are installed. The agent starts an empty conductor, installs a configurable number of apps without enabling them (disabled cells), restarts the conductor, and then enables cells one by one — timing each enable call and periodically restarting the conductor to measure startup time as the share of enabled cells grows.

Started
Thu, 22 Oct 2026 03:19:13 UTC
Duration
900s
Peer count
150
Behaviours
  • default (1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
conductor_startup_39711995436_1
Initial conductor startup timing
Startup time with an empty conductor before any cells are installed.
mean
5.087028
s
SD
2.090883
s
p50
5.227981
s
<
p95
7.967734
s
<
p99
8.492191
s
Disabled-cell conductor startup timing
Startup time after all configured cells are installed but remain disabled.
mean
3.863299
s
SD
1.536745
s
p50
3.996676
s
<
p95
6.158189
s
<
p99
7.008262
s
Progressive conductor startup timing
Startup time sampled periodically as more installed cells are enabled.
mean
16.123439
s
SD
7.671196
s
p50
15.355465
s
<
p95
28.656392
s
<
p99
29.838187
s
Cell enable timing
Time to enable one previously disabled cell via the admin interface.
mean
0.326115
s
SD
0.226403
s
p50
0.219168
s
<
p95
0.738999
s
<
p99
0.944258
s
Installed cells
Number of cells installed on the conductor during setup.
500
cells
Maximum enabled cells
Highest percentage of installed cells enabled during the run.
84
%
Holochain MetricsClick to toggle visibility.
WASM usage: 'callback_integrity::genesis_self_check_2'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 208.619052132s
Not enough time data to show a trend.
WASM call duration: 'callback_integrity::genesis_self_check_2'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001959
s
SD
0.000909
s
p50
0.001763
s
<
p95
0.003836
s
<
p99
0.005074
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
2.25
s
<
mean
43.66
s
SD
31.95
s
<
p95
95.2
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 393.193050915s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
5.00415
s
SD
6.162198
s
p50
1.006258
s
<
p95
18.166437
s
<
p99
22.146239
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.001359
s
SD
0.005196
s
p50
0.001065
s
<
p95
0.002504
s
<
p99
0.003681
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.012034
s
SD
0.017253
s
p50
0.001466
s
<
p95
0.003535
s
<
p99
0.063368
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.002434
s
SD
0.009843
s
p50
0.001669
s
<
p95
0.005224
s
<
p99
0.009594
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.009919
s
SD
0.062289
s
p50
0.001698
s
<
p95
0.004664
s
<
p99
0.0092
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
0.00138
s
SD
0.009742
s
p50
0.000673
s
<
p95
0.003359
s
<
p99
0.005757
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.000102
s
SD
0.000273
s
p50
0.000199
s
<
p95
0.000401
s
<
p99
0.000542
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.011314
s
SD
0.002684
s
p50
0.00221
s
<
p95
0.005812
s
<
p99
0.015031
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000288
s
SD
0.000161
s
p50
0.000306
s
<
p95
0.000574
s
<
p99
0.001111
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
0.188318
s
SD
0.153451
s
p50
0.156325
s
<
p95
0.45673
s
<
p99
0.87058
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.309145
s
SD
0.07226
s
p50
0.369544
s
<
p95
0.387558
s
≤
p99
0.387558
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
21.52
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
1.32
KiB/s
<
mean
522.33
KiB/s
SD
845.02
KiB/s
<
p95
2.38
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
182
B/s
<
mean
49.49
KiB/s
SD
59.37
KiB/s
<
p95
157.88
KiB/s
Total bytes received
Total bytes received on primary network interface
count
1.21
GiB
mean
2.95
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
117.44
MiB
mean
286.34
KiB/s
CPU spike anomaly
CPU spike anomaly was detected
NotDetected
Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected

Warning Moderate swap usage (19.4% swap used)

System overload anomaly
System overload anomaly was detected
NotDetected
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
0.93
%
<
mean
34.68
%
SD
29.77
%
<
p95
77.28
%
System
13.15
%
CPU percentiles
CPU usage percentiles
p50
35.35
%
p95
77.28
%
p99
81.61
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
2
hosts
mean time
66.96
s
Memory used percentage
Percentage of memory used
p5
5.7
%
<
mean
13.56
%
SD
7.88
%
<
p95
26.21
%
Memory available percentage
Percentage of available memory
p5
73.79
%
<
mean
86.44
%
SD
7.88
%
<
p95
94.3
%
Max host memory used
Maximum memory usage percentage across all hosts
max
31.54
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
19.42
%
Memory growth rate
Rate of memory growth over time
growth
0.78
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.06
MiB/s
Disk write throughput
Disk write throughput in MiB/s
65.76
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point /
0/13
hosts
Mount Point /etc/hostname
0/9
hosts
Mount Point /etc/hosts
0/9
hosts
Mount Point /etc/nomad.d/client-meta.json
0/4
hosts
Mount Point /etc/resolv.conf
0/9
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
1.46
5 min
0.79
15 min
0.56
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
0.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0.02
%
<
mean
13.7019
%
SD
17.0114
%
<
p95
45.87
%
60 second average
10.4304
%
300 second average
6.3906
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
0.6571
%
SD
0.9586
%
<
p95
2.65
%
60 second average
0.5332
%
300 second average
0.3398
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
0.3941
%
SD
0.5656
%
<
p95
1.63
%
60 second average
0.3048
%
300 second average
0.1854
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0
%
<
mean
30.18
%
SD
29.16
%
<
p95
72.82
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
57.97
KiB
<
mean
323.94
KiB
SD
280.35
KiB
<
p95
1.05
MiB
Holochain process threads
Number of threads in Holochain process
p5
1
threads
<
mean
471.04
threads
SD
595.29
threads
<
p95
1830
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
0
file descriptors
<
mean
1089.02
file descriptors
SD
1335.73
file descriptors
<
p95
4043
file descriptors

DHT Sync Lag

↑ Back to index

Measures the lag time between an agent publishing data and other peers being able to see it. This scenario has two roles:

  • write: Creates entries with a timestamp field, linked to a known base hash so that record_lag agents can discover them.
  • record_lag: Repeatedly queries for links from the known base hash. When a new entry is discovered, it calculates the time difference between the entry's timestamp and the current time, giving the sync lag for that entry.
Started
Thu, 22 Oct 2026 03:29:46 UTC
Duration
1800s
Peer count
150
Behaviours
  • record_lag (1 agent)
  • write (1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
dht_sync_lag_39711995436_1
Create rate
The average number of created records per agent.
Mean
mean
27.17
/s
Max
Highest per-partition mean rate
max
55.76
/s
Min
Lowest per-partition mean rate
min
11.42
/s
Sync lag timing
The average time between when a record was created and when it is first seen by an agent.
Mean
mean
381.055906
s
SD
220.129952
s
Max
Highest per-partition mean latency
max mean
1006.463614
s
Min
Lowest per-partition mean latency
min mean
9.920376
s
Sync lag rate
The average number of created records discovered per agent.
Mean
mean
76.41
/s
Max
Highest per-partition mean rate
max
280
/s
Min
Lowest per-partition mean rate
min
0
/s
Delivery ratio
The proportion of created entries successfully received by all record_lag agents.
0.0036
Sent count
The total number of entries sent (created) by write agents, by agent.
Total
587907
Agents affected
121 / 121
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
4859
Max
Highest count in any single partition
10040
Min
Lowest count in any single partition
2132
Received count
The total number of entries received by record_lag agents, by agent.
Total
42867
Agents affected
20 / 20
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
2143
Max
Highest count in any single partition
5135
Min
Lowest count in any single partition
1
Error count
The number of errors encountered during the scenario.
208
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.000419
s
SD
0.00026
s
p50
0.000458
s
<
p95
0.001055
s
<
p99
0.001206
s
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
4
over 1731.91699044s
mean rate
29.98
/s
std
296.43
/s
p5
0
/s
p95
127.93
/s
peak
13455.69
/s
WASM usage: 'timed::created_timed_entry'
Metered usage count of the WASM ribosome for this zome function.
total
249720780
over 1789.871773388s
mean rate
5.65738868178e+09
/s
std
3.20378052264e+10
/s
p5
3.982585781e+07
/s
p95
1.76537129693e+10
/s
peak
9.88977390301e+11
/s
WASM usage: 'timed::get_timed_entries_local'
Metered usage count of the WASM ribosome for this zome function.
total
66204060183
over 1789.075502457s
mean rate
1.62619141677e+11
/s
std
3.00001167223e+11
/s
p5
1.12148886171e+09
/s
p95
8.56077488899e+11
/s
peak
1.68577334454e+12
/s
WASM usage: 'timed_integrity::entry_defs'
Metered usage count of the WASM ribosome for this zome function.
total
39329196
over 1789.871773388s
mean rate
6.28395248782e+09
/s
std
1.66683909882e+11
/s
p5
7.73383603e+06
/s
p95
6.38840748782e+09
/s
peak
1.27985988665e+13
/s
Zome call duration: 'timed::created_timed_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.009969
s
SD
0.004834
s
p50
0.010743
s
<
p95
0.016794
s
<
p99
0.023551
s
Zome call duration: 'timed::get_timed_entries_local'
Duration of this zome call as measured by Holochain internally.
mean
0.111608
s
SD
0.147722
s
p50
0.100761
s
<
p95
0.479746
s
<
p99
0.641388
s
WASM call duration: 'timed::created_timed_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.008712
s
SD
0.004533
s
p50
0.009328
s
<
p95
0.014638
s
<
p99
0.022782
s
WASM call duration: 'timed::get_timed_entries_local'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.110605
s
SD
0.147425
s
p50
0.099849
s
<
p95
0.47793
s
<
p99
0.639506
s
WASM call duration: 'timed_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001607
s
SD
0.000534
s
p50
0.001768
s
<
p95
0.002599
s
<
p99
0.003006
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000751
s
SD
0.001288
s
p50
0.000759
s
<
p95
0.00131
s
<
p99
0.004828
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.000743
s
SD
0.001238
s
p50
0.000757
s
<
p95
0.00132
s
<
p99
0.004565
s
Host function call duration: '__hc__get_1'
Duration of this host function call invoked from within WASM.
mean
0.000425
s
SD
0.000275
s
p50
0.000492
s
<
p95
0.00108
s
<
p99
0.001228
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.005024
s
SD
0.005396
s
p50
0.005158
s
<
p95
0.017955
s
<
p99
0.023313
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.002581
s
SD
0.000991
s
p50
0.002802
s
<
p95
0.004406
s
<
p99
0.005337
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.000484
s
SD
0.000345
s
p50
0.000566
s
<
p95
0.001076
s
<
p99
0.001902
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
86.97
s
<
mean
881.46
s
SD
509.43
s
<
p95
1674.78
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
8392
over 1789.871786968s
mean rate
666300.81
/s
std
1.207707688e+07
/s
p5
37.22
/s
p95
1.36418475e+06
/s
peak
1.07532084999e+09
/s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
127.879778
s
SD
110.063621
s
p50
46.339651
s
<
p95
301.965816
s
<
p99
498.569122
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.228157
s
SD
0.34816
s
p50
0.200863
s
<
p95
0.851253
s
<
p99
1.6202
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.068952
s
SD
0.407463
s
p50
0.033722
s
<
p95
1.140755
s
<
p99
1.756889
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.456561
s
SD
0.385245
s
p50
0.373624
s
<
p95
0.849101
s
<
p99
1.930486
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
48.465127
s
SD
51.951769
s
p50
50.996237
s
<
p95
162.369568
s
<
p99
237.230349
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
1.876515
s
SD
26.402683
s
p50
0.046253
s
<
p95
0.177175
s
<
p99
147.552469
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.002002
s
SD
0.005097
s
p50
0.006753
s
<
p95
0.016122
s
<
p99
0.018991
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.008897
s
SD
0.007898
s
p50
0.004401
s
<
p95
0.024582
s
<
p99
0.032789
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.00061
s
SD
0.001089
s
p50
0.000626
s
<
p95
0.001299
s
<
p99
0.00369
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
5.311038
s
SD
29.411484
s
p50
2.423155
s
<
p95
60.138542
s
<
p99
61.1004
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.200776
s
SD
23.581223
s
p50
0.135248
s
<
p95
60.001172
s
<
p99
60.001373
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
51.05
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
27.57
KiB/s
<
mean
961.86
KiB/s
SD
972.92
KiB/s
<
p95
2.77
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
3.40
KiB/s
<
mean
97.27
KiB/s
SD
157.75
KiB/s
<
p95
365.47
KiB/s
Total bytes received
Total bytes received on primary network interface
count
231.29
GiB
mean
132.31
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
23.39
GiB
mean
13.38
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected

Warning CPU p99 reached 97.9%

Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected

Warning Moderate swap usage (19.4% swap used)

System overload anomaly
System overload anomaly was detected
Detected

Warning 53% of hosts overloaded (load5/ncpus > 1.0)

CPU usage
Total CPU usage and kernel CPU usage
Total
p5
30.29
%
<
mean
67.03
%
SD
19.23
%
<
p95
94.53
%
System
15.98
%
CPU percentiles
CPU usage percentiles
p50
68.25
%
p95
94.53
%
p99
97.90
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
129
hosts
mean time
538.04
s
Memory used percentage
Percentage of memory used
p5
7.24
%
<
mean
10.29
%
SD
2.52
%
<
p95
14.77
%
Memory available percentage
Percentage of available memory
p5
85.23
%
<
mean
89.71
%
SD
2.52
%
<
p95
92.76
%
Max host memory used
Maximum memory usage percentage across all hosts
max
31.75
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
19.42
%
Memory growth rate
Rate of memory growth over time
growth
0.35
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.11
MiB/s
Disk write throughput
Disk write throughput in MiB/s
171.78
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point /
0/119
hosts
Mount Point /efi-boot
0/6
hosts
Mount Point /etc/hostname
0/9
hosts
Mount Point /etc/hosts
0/9
hosts
Mount Point /etc/nomad.d/client-meta.json
0/20
hosts
Mount Point /etc/resolv.conf
0/9
hosts
Mount Point /nix/store
0/11
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
2.79
5 min
2.54
15 min
1.93
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
53.38
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0.79
%
<
mean
28.0685
%
SD
20.4136
%
<
p95
66.57
%
60 second average
27.4965
%
300 second average
25.1649
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
5.2778
%
SD
8.034
%
<
p95
16.83
%
60 second average
5.2783
%
300 second average
5.0718
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
3.1382
%
SD
5.5738
%
<
p95
10.08
%
60 second average
3.1631
%
300 second average
3.1589
%
Holochain process CPU usage
CPU usage by Holochain process
p5
31.82
%
<
mean
65.54
%
SD
18.06
%
<
p95
92.45
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
230.45
KiB
<
mean
348.27
KiB
SD
134.91
KiB
<
p95
639.79
KiB
Holochain process threads
Number of threads in Holochain process
p5
29
threads
<
mean
35.08
threads
SD
43.26
threads
<
p95
48
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
76
file descriptors
<
mean
84.99
file descriptors
SD
95.55
file descriptors
<
p95
91
file descriptors

Full-Arc Create (Validated) / Zero-Arc Read

↑ Back to index

A full-arc/zero-arc mixed scenario where full-arc nodes create data that gets validated and zero-arc nodes read the data. The scenario has two roles:

  • full: A full-arc conductor that creates entries with a timestamp field. Those entries get validated and then retrieved by zero-arc nodes.
  • zero: A zero-arc conductor that reads the entries created by the full-arc node(s) and records the time lag between when the entry had been created and when it was first discovered.
Started
Thu, 22 Oct 2026 07:10:14 UTC
Duration
900s
Peer count
150
Peer count at end
134
Behaviours
  • full (1 agent)
  • zero (1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
full_arc_create_validated_zero_arc_read--high_zero_39711995436_1
Create rate
The number of timed entries created by the full-arc node(s) per second.
Mean
mean
16.73
/s
Max
Highest per-partition mean rate
max
27.95
/s
Min
Lowest per-partition mean rate
min
7.19
/s
Fetch lag timing
For each entry, the time lag between when it was created by a full-arc node and when a zero-arc node fetched it from the network.
Mean
mean
49.121621
s
SD
10.103247
s
Max
Highest per-partition mean latency
max mean
1251.42369
s
Min
Lowest per-partition mean latency
min mean
1.189622
s
Fetch rate
The number of entries per second fetched by zero-arc nodes.
Mean
mean
15.73
/s
Max
Highest per-partition mean rate
max
530.5
/s
Min
Lowest per-partition mean rate
min
0
/s
Delivery ratio
The proportion of created entries successfully fetched by all zero-arc nodes.
0.0048
Entry created count
Total number of entries created by full-arc nodes, by agent.
Total
12058
Agents affected
8 / 8
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
1507
Max
Highest count in any single partition
2566
Min
Lowest count in any single partition
605
Received count
Total number of entries fetched by zero-arc nodes, by agent.
Total
7250
Agents affected
31 / 126
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
58
Max
Highest count in any single partition
3773
Min
Lowest count in any single partition
0
Open connections
The number of currently open connections to other conductors.
full-arc
p5
26
<
mean
105.07
SD
31.82
<
p95
132
zero-arc
p5
0
<
mean
11.84
SD
12.29
<
p95
40
Retrieval errors
Errors encountered by zero-arc nodes when attempting to retrieve entries created by full-arc nodes.
Total
1711
Agents affected
124 / 126
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
14
Max
Highest count in any single partition
14
Min
Lowest count in any single partition
0
Error count
The number of errors accumulated across all nodes.
1712
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.367933
s
SD
1.513778
s
p50
0.518299
s
<
p95
3.835214
s
<
p99
7.598109
s
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
0
over 780.167957648s
Not enough time data to show a trend.
Zome call duration: 'timed_and_validated::create_timed_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.012673
s
SD
0.006319
s
p50
0.0124
s
<
p95
0.030711
s
<
p99
0.031475
s
Zome call duration: 'timed_and_validated::get_timed_entries_network'
Duration of this zome call as measured by Holochain internally.
mean
0.059297
s
SD
55.973138
s
p50
0.046352
s
<
p95
126.087555
s
<
p99
274.098018
s
WASM call duration: 'timed_and_validated::create_timed_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.011149
s
SD
0.005422
s
p50
0.010894
s
<
p95
0.026842
s
<
p99
0.027353
s
WASM call duration: 'timed_and_validated::get_timed_entries_network'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.15575
s
SD
81.987446
s
p50
0.38825
s
<
p95
210.004245
s
<
p99
410.209542
s
WASM call duration: 'timed_and_validated_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001216
s
SD
0.000519
s
p50
0.001532
s
<
p95
0.002678
s
<
p99
0.003041
s
WASM call duration: 'timed_and_validated_integrity::validate'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.008887
s
SD
0.002505
s
p50
0.001655
s
<
p95
0.008967
s
<
p99
0.012279
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.001516
s
SD
0.001393
s
p50
0.001529
s
<
p95
0.005774
s
<
p99
0.00593
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.00127
s
SD
0.00107
s
p50
0.001251
s
<
p95
0.004508
s
<
p99
0.004606
s
Host function call duration: '__hc__get_1'
Duration of this host function call invoked from within WASM.
mean
0.621803
s
SD
1.848731
s
p50
1.223845
s
<
p95
5.585861
s
<
p99
9.309659
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.015043
s
SD
1.695383
s
p50
0.191609
s
<
p95
3.856784
s
<
p99
9.421035
s
Host function call duration: '__hc__must_get_valid_record_1'
Duration of this host function call invoked from within WASM.
mean
0.012064
s
SD
0.00325
s
p50
0.009127
s
<
p95
0.015098
s
<
p99
0.015846
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.002013
s
SD
0.001008
s
p50
0.002399
s
<
p95
0.004526
s
<
p99
0.005874
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.001064
s
SD
0.000558
s
p50
0.000224
s
<
p95
0.001573
s
<
p99
0.002545
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
43.12
s
<
mean
430.44
s
SD
249.64
s
<
p95
822.99
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 889.277948372s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
121.335084
s
SD
20.281076
s
p50
0
s
<
p95
0.199237
s
<
p99
19.381777
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
1.834871
s
SD
0.740302
s
p50
0.001738
s
<
p95
0.873848
s
<
p99
2.352616
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.157882
s
SD
0.097657
s
p50
0.002121
s
<
p95
0.043554
s
<
p99
0.340876
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.295661
s
SD
0.074905
s
p50
0.019012
s
<
p95
0.104525
s
<
p99
0.462865
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
4.265867
s
SD
4.833763
s
p50
0.003578
s
<
p95
6.215857
s
<
p99
23.796655
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
0.041648
s
SD
0.041387
s
p50
0.001338
s
<
p95
0.007529
s
<
p99
0.356362
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.001944
s
SD
0.001524
s
p50
0.000151
s
<
p95
0.003019
s
<
p99
0.008542
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.008219
s
SD
0.005599
s
p50
0.003922
s
<
p95
0.016664
s
<
p99
0.027
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.001218
s
SD
0.00059
s
p50
0.000801
s
<
p95
0.00206
s
<
p99
0.003163
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
4.99742
s
SD
29.744951
s
p50
2.102043
s
<
p95
60.001663
s
<
p99
60.003782
s
P2P request roundtrip: 'get_links'
Time spent sending a get_links request and awaiting its response.
mean
1.051991
s
SD
18.722719
s
p50
0.696929
s
<
p95
60.001268
s
<
p99
60.00176
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
3.230109
s
SD
17.95806
s
p50
0.258767
s
<
p95
60.00059
s
<
p99
60.000769
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
7.69
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
1.17
KiB/s
<
mean
320.80
KiB/s
SD
458.29
KiB/s
<
p95
1.27
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
0
B/s
<
mean
19.55
KiB/s
SD
90.25
KiB/s
<
p95
86.33
KiB/s
Total bytes received
Total bytes received on primary network interface
count
35.69
GiB
mean
41.06
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
2.18
GiB
mean
2.50
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected

Warning CPU p99 reached 94.7%

Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
Detected

Critical Disk /etc/resolv.conf: 1/12 hosts above 90% capacity

Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected

Critical Heavy swap usage (81.6% swap used)

System overload anomaly
System overload anomaly was detected
Detected

Warning 6% of hosts overloaded (load5/ncpus > 1.0)

CPU usage
Total CPU usage and kernel CPU usage
Total
p5
1.15
%
<
mean
11.25
%
SD
23.08
%
<
p95
87.74
%
System
3.56
%
CPU percentiles
CPU usage percentiles
p50
3.63
%
p95
87.74
%
p99
94.66
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
81
hosts
mean time
89.52
s
Memory used percentage
Percentage of memory used
p5
7.61
%
<
mean
12.65
%
SD
8.15
%
<
p95
15.68
%
Memory available percentage
Percentage of available memory
p5
84.32
%
<
mean
87.35
%
SD
8.15
%
<
p95
92.39
%
Max host memory used
Maximum memory usage percentage across all hosts
max
89.18
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
81.64
%
Memory growth rate
Rate of memory growth over time
growth
0.57
MiB/s
Disk read throughput
Disk read throughput in MiB/s
1.39
MiB/s
Disk write throughput
Disk write throughput in MiB/s
72.29
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point /
0/116
hosts
Mount Point /efi-boot
0/6
hosts
Mount Point /etc/hostname
1/12
hosts
Mount Point /etc/hosts
1/12
hosts
Mount Point /etc/nomad.d/client-meta.json
0/13
hosts
Mount Point /etc/nomad.d/meta.json
1/1
hosts
Mount Point /etc/resolv.conf
1/12
hosts
Mount Point /nix/store
0/11
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
0.49
5 min
0.56
15 min
0.83
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
5.67
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
4.8923
%
SD
17.6666
%
<
p95
62.41
%
60 second average
4.7270
%
300 second average
5.6694
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0004
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0004
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
0.8025
%
SD
2.5946
%
<
p95
7.25
%
60 second average
0.7831
%
300 second average
0.8868
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
0.3558
%
SD
1.35
%
<
p95
2.15
%
60 second average
0.3498
%
300 second average
0.3596
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.3
%
<
mean
8.8
%
SD
22.55
%
<
p95
82.92
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
309.47
KiB
<
mean
381.77
KiB
SD
58.02
KiB
<
p95
474.77
KiB
Holochain process threads
Number of threads in Holochain process
p5
21
threads
<
mean
28.65
threads
SD
6.83
threads
<
p95
37
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
55
file descriptors
<
mean
64.24
file descriptors
SD
10.62
file descriptors
<
p95
81
file descriptors

Mixed Arc get_agent_activity

↑ Back to index

A mixed-arc scenario that measures the DHT's ability to propagate agent activity and make it available via get_agent_activity calls. The scenario has three roles:

  • zero_write: A zero-arc conductor that creates entries on its source chain.
  • full_write: A full-arc conductor that creates entries on its source chain.
  • zero_read: A zero-arc conductor that picks a random writer and repeatedly calls get_agent_activity to observe new entries appearing on the writer's chain, measuring the lag between creation and discovery.
Started
Thu, 22 Oct 2026 07:27:13 UTC
Duration
900s
Peer count
150
Peer count at end
135
Behaviours
  • full_write (1 agent)
  • zero_read (1 agent)
  • zero_write (1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
mixed_arc_get_agent_activity_39711995436_1
Entry created count
The total number of entries created by write agents over the run, partitioned by agent and behaviour.
Total
372040
Agents affected
92 / 92
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
4044
Max
Highest count in any single partition
9964
Min
Lowest count in any single partition
865
Observed chain advancement
The maximum highest-observed action sequence per write agent, aggregated across all reading agents. The reading dimension is collapsed by taking the maximum observed value for each write agent (any reader successfully tracking that point counts as propagation). Captures how far readers tracked writers’ chains during the run.
Mean max (across write agents)
Mean of the highest chain head value observed per write agent. Each write agent contributes its maximum — the value seen by any reader.
2224.58
Max
Highest chain head value observed for any single write agent.
5321
Write agents observed
24
Chain head delay timing
The time lag between when a writer creates a new entry and when a reader discovers it via get_agent_activity. This is measured per new action in the chain.
Mean
mean
1.260479
s
SD
2.018063
s
Max
Highest per-partition mean latency
max mean
37.02306
s
Min
Lowest per-partition mean latency
min mean
-0.280975
s
Chain head delay rate
The rate at which readers detect new chain heads appearing on writers' chains.
Mean
mean
3.61
/s
Max
Highest per-partition mean rate
max
22.19
/s
Min
Lowest per-partition mean rate
min
0
/s
get_agent_activity_full zome call timing
The time taken to complete the get_agent_activity_full zome call that queries a writer's chain.
Mean
mean
1.108737
s
SD
2.710634
s
Max
Highest per-partition mean latency
max mean
8.200946
s
Min
Lowest per-partition mean latency
min mean
0.010947
s
Retrieval errors
A running accumulation of errors encountered by read agents when calling get_agent_activity.
Total
194
Agents affected
40 / 43
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
5
Max
Highest count in any single partition
8
Min
Lowest count in any single partition
0
Open connections
The number of currently open connections to other conductors, partitioned by agent behaviour.
  • behaviour: full_write
p5
10
<
mean
72.23
SD
33.96
<
p95
133
  • behaviour: zero_read
p5
6
<
mean
32.36
SD
15.8
<
p95
64
  • behaviour: zero_write
p5
3
<
mean
28.69
SD
17.45
<
p95
61
Final error count
The total number of errors accumulated across all agents over the run.
208
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.558294
s
SD
8.467832
s
p50
1.276767
s
<
p95
22.328861
s
<
p99
33.124984
s
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
0
over 833.820205979s
Not enough time data to show a trend.
Zome call duration: 'agent_activity::announce_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.008311
s
SD
0.00232
s
p50
0.008411
s
<
p95
0.012532
s
<
p99
0.01507
s
Zome call duration: 'agent_activity::create_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.008526
s
SD
0.010696
s
p50
0.007959
s
<
p95
0.037345
s
<
p99
0.050303
s
Zome call duration: 'agent_activity::get_agent_activity_full'
Duration of this zome call as measured by Holochain internally.
mean
0.721442
s
SD
2.399261
s
p50
1.119161
s
<
p95
6.658839
s
<
p99
9.604349
s
Zome call duration: 'agent_activity::get_random_agent_with_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.046828
s
SD
7.906664
s
p50
0.714848
s
<
p95
21.995274
s
<
p99
26.224515
s
WASM call duration: 'agent_activity::announce_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.006944
s
SD
0.001983
s
p50
0.007108
s
<
p95
0.010395
s
<
p99
0.012783
s
WASM call duration: 'agent_activity::create_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.007254
s
SD
0.010028
s
p50
0.00642
s
<
p95
0.034229
s
<
p99
0.04694
s
WASM call duration: 'agent_activity::get_agent_activity_full'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.72716
s
SD
2.690952
s
p50
1.126023
s
<
p95
6.743516
s
<
p99
11.14106
s
WASM call duration: 'agent_activity::get_random_agent_with_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.104795
s
SD
10.260064
s
p50
3.968648
s
<
p95
26.111496
s
<
p99
33.13299
s
WASM call duration: 'agent_activity_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001687
s
SD
0.000704
s
p50
0.001882
s
<
p95
0.003304
s
<
p99
0.004133
s
Host function call duration: '__hc__agent_info_1'
Duration of this host function call invoked from within WASM.
mean
6.4e-05
s
SD
4.2e-05
s
p50
4.8e-05
s
<
p95
0.000154
s
<
p99
0.000224
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.00221
s
SD
0.006641
s
p50
0.000842
s
<
p95
0.019745
s
<
p99
0.029485
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.000708
s
SD
0.000305
s
p50
0.000655
s
<
p95
0.00125
s
<
p99
0.002146
s
Host function call duration: '__hc__get_agent_activity_1'
Duration of this host function call invoked from within WASM.
mean
0.724153
s
SD
2.691005
s
p50
1.124026
s
<
p95
6.73958
s
<
p99
11.137254
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.10261
s
SD
10.259282
s
p50
3.965244
s
<
p95
26.104014
s
<
p99
33.12638
s
Host function call duration: '__hc__random_bytes_1'
Duration of this host function call invoked from within WASM.
mean
4e-06
s
SD
1.4e-05
s
p50
1.6e-05
s
<
p95
2.6e-05
s
<
p99
9.7e-05
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.003104
s
SD
0.002048
s
p50
0.003327
s
<
p95
0.007878
s
<
p99
0.012352
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.001023
s
SD
0.001915
s
p50
0.000644
s
<
p95
0.005671
s
<
p99
0.009095
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
42.86
s
<
mean
429.55
s
SD
249.45
s
<
p95
821.66
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
485
over 888.820886407s
mean rate
50680.69
/s
std
744315.57
/s
p5
0
/s
p95
107771.21
/s
peak
2.487783948e+07
/s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
23.698443
s
SD
25.272146
s
p50
0
s
<
p95
22.895364
s
<
p99
93.041757
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.237322
s
SD
0.561447
s
p50
0.002397
s
<
p95
1.393051
s
<
p99
2.334068
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
1e-06
s
SD
7e-06
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.039784
s
SD
0.149234
s
p50
0.002618
s
<
p95
0.369608
s
<
p99
0.751226
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.262772
s
SD
0.17894
s
p50
0.149052
s
<
p95
0.484867
s
<
p99
0.61733
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
16.628822
s
SD
23.39162
s
p50
0.004337
s
<
p95
57.654125
s
<
p99
110.582112
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
1.954472
s
SD
10.782077
s
p50
0.002037
s
<
p95
2.940396
s
<
p99
13.456656
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.004503
s
SD
0.002917
s
p50
0.002759
s
<
p95
0.008789
s
<
p99
0.010337
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.007127
s
SD
0.004177
s
p50
0.003343
s
<
p95
0.013965
s
<
p99
0.020949
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.002203
s
SD
0.005732
s
p50
0.000791
s
<
p95
0.016168
s
<
p99
0.028894
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
7.065856
s
SD
26.27356
s
p50
4.516283
s
<
p95
60.039239
s
<
p99
60.110925
s
P2P request roundtrip: 'get_links'
Time spent sending a get_links request and awaiting its response.
mean
1.586233
s
SD
15.207225
s
p50
0.690219
s
<
p95
60.000484
s
<
p99
60.002098
s
P2P request roundtrip: 'get_agent_activity'
Time spent sending a get_agent_activity request and awaiting its response.
mean
0.41738
s
SD
29.091457
s
p50
0.868908
s
<
p95
60.001948
s
<
p99
60.00284
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
8.187316
s
SD
25.147174
s
p50
6.1286
s
<
p95
60.018967
s
<
p99
67.191786
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
37.17
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
3.59
KiB/s
<
mean
890.66
KiB/s
SD
844.18
KiB/s
<
p95
2.35
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
0
B/s
<
mean
88.97
KiB/s
SD
147.12
KiB/s
<
p95
351.69
KiB/s
Total bytes received
Total bytes received on primary network interface
count
101.00
GiB
mean
116.21
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
10.09
GiB
mean
11.61
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected

Warning CPU p99 reached 94.5%

Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
Detected

Critical Disk /etc/resolv.conf: 1/12 hosts above 90% capacity

Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected

Critical Heavy swap usage (80.6% swap used)

System overload anomaly
System overload anomaly was detected
Detected

Warning 29% of hosts overloaded (load5/ncpus > 1.0)

CPU usage
Total CPU usage and kernel CPU usage
Total
p5
1.9
%
<
mean
50.93
%
SD
35.87
%
<
p95
92.47
%
System
13.76
%
CPU percentiles
CPU usage percentiles
p50
62.95
%
p95
92.47
%
p99
94.52
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
99
hosts
mean time
413.92
s
Memory used percentage
Percentage of memory used
p5
7.32
%
<
mean
11.74
%
SD
8.11
%
<
p95
14.1
%
Memory available percentage
Percentage of available memory
p5
85.9
%
<
mean
88.26
%
SD
8.11
%
<
p95
92.68
%
Max host memory used
Maximum memory usage percentage across all hosts
max
88.36
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
80.63
%
Memory growth rate
Rate of memory growth over time
growth
0.57
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.72
MiB/s
Disk write throughput
Disk write throughput in MiB/s
114.75
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point /
0/119
hosts
Mount Point /efi-boot
0/6
hosts
Mount Point /etc/hostname
1/12
hosts
Mount Point /etc/hosts
1/12
hosts
Mount Point /etc/nomad.d/client-meta.json
0/13
hosts
Mount Point /etc/nomad.d/meta.json
1/1
hosts
Mount Point /etc/resolv.conf
1/12
hosts
Mount Point /nix/store
0/11
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
2.21
5 min
1.71
15 min
1.21
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
29.17
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
26.7008
%
SD
28.2608
%
<
p95
76.02
%
60 second average
25.3386
%
300 second average
19.5336
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
5.1647
%
SD
7.2102
%
<
p95
19.92
%
60 second average
4.9458
%
300 second average
3.8176
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
1.559
%
SD
2.4665
%
<
p95
4.87
%
60 second average
1.5117
%
300 second average
1.2112
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.55
%
<
mean
48.11
%
SD
34.55
%
<
p95
86.77
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
206.04
KiB
<
mean
292.86
KiB
SD
84.77
KiB
<
p95
437.80
KiB
Holochain process threads
Number of threads in Holochain process
p5
21
threads
<
mean
30.39
threads
SD
7.06
threads
<
p95
36
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
57
file descriptors
<
mean
75.59
file descriptors
SD
14.73
file descriptors
<
p95
89
file descriptors

Mixed Arc must_get_agent_activity

↑ Back to index

A scenario where write agents create entries in batches of 10, while must_get_agent_activity agents each pick a random write agent and repeatedly attempt to create an entry that references the chain top of their latest batch. This reference means that the entry's validation function needs to make a must_get_agent_activity call.

The purpose of this scenario is to measure the time it takes for published agent activity data to be gossiped among authorities and become available to peers that query it via must_get_agent_activity.

This test is similar to Write Validated must_get_agent_activity, but some write agents are zero-arc and all must_get_agent_activity agents are zero-arc. This forces the must_get_agent_activity host call to go to the network.

Started
Thu, 22 Oct 2026 07:44:10 UTC
Duration
900s
Peer count
150
Peer count at end
135
Behaviours
  • full_write (1 agent)
  • zero_must_get_agent_activity (1 agent)
  • zero_write (1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
mixed_arc_must_get_agent_activity_39711995436_1
Highest observed action_seq
The maximum chain length observed per write agent, aggregated across all reading agents. The reading dimension is collapsed by taking the maximum observed value for each write agent (any successful read counts as propagation). This reflects the DHT’s ability to propagate agent activity ops and make them available to querying peers.
Mean max (across write agents)
Mean of the highest chain head value observed per write agent. Each write agent contributes its maximum — the value seen by any reader.
406
Max
Highest chain head value observed for any single write agent.
1358
Write agents observed
11
Chain batch delay timing
The time between a write agent's creation of a batch and a must_get_agent_activity agent's successful discovery of the batch and creation/self-validation of a new entry that depends on it.
Mean
mean
358.86135
s
SD
157.357573
s
Max
Highest per-partition mean latency
max mean
1039.74613
s
Min
Lowest per-partition mean latency
min mean
98.430472
s
Chain batch delay rate
The rate at which a must_get_agent_activity agent was able to discover batches and create/self-validate new entries that depend on them.
Mean
mean
1.43
/s
Max
Highest per-partition mean rate
max
3.27
/s
Min
Lowest per-partition mean rate
min
0
/s
create_validated_sample_entry zome call timing
The time taken to complete the zome function call that creates the entry that depends on a write agent's source chain.
Mean
mean
6.654035
s
SD
6.924078
s
Max
Highest per-partition mean latency
max mean
16.978342
s
Min
Lowest per-partition mean latency
min mean
0.328749
s
Retrieval errors
A running accumulation of the errors encountered by an agent when attempting to self-validate actions that depend on must_get_agent_activity calls.
Total
44596
Agents affected
40 / 40
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
1115
Max
Highest count in any single partition
6378
Min
Lowest count in any single partition
87
Open connections
The average number of open connections maintained by an agent over the run, broken down by behaviour.
  • behaviour: full_write
p5
5
<
mean
39.97
SD
30.11
<
p95
102
  • behaviour: zero_must_get_agent_activity
p5
1
<
mean
19.05
SD
13.54
<
p95
51
  • behaviour: zero_write
p5
3
<
mean
18.76
SD
10.94
<
p95
44
Final error count
The total number of all types of error accumulated over the run by all agents.
53158
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.173783
s
SD
4.224824
s
p50
0.481317
s
<
p95
7.786539
s
<
p99
23.312209
s
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
2
over 805.456387156s
mean rate
54.3
/s
std
220.03
/s
p5
0
/s
p95
225.02
/s
peak
3546.39
/s
Zome call duration: 'validated_must_get_agent_activity::announce_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.008144
s
SD
0.002919
s
p50
0.007693
s
<
p95
0.013216
s
<
p99
0.016966
s
Zome call duration: 'validated_must_get_agent_activity::create_sample_entries_batch'
Duration of this zome call as measured by Holochain internally.
mean
0.106275
s
SD
0.083643
s
p50
0.065142
s
<
p95
0.304052
s
<
p99
0.406294
s
Zome call duration: 'validated_must_get_agent_activity::create_validated_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.006281
s
SD
0.002123
s
p50
0.007228
s
<
p95
0.010688
s
<
p99
0.011176
s
Zome call duration: 'validated_must_get_agent_activity::get_chain_top_for_batch'
Duration of this zome call as measured by Holochain internally.
mean
1.491538
s
SD
5.917936
s
p50
0.758353
s
<
p95
10.46322
s
<
p99
35.016456
s
Zome call duration: 'validated_must_get_agent_activity::get_random_agent_with_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.022704
s
SD
1.777284
s
p50
0.688504
s
<
p95
6.312435
s
<
p99
7.48421
s
WASM call duration: 'validated_must_get_agent_activity::announce_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.006728
s
SD
0.002502
s
p50
0.006233
s
<
p95
0.011099
s
<
p99
0.015302
s
WASM call duration: 'validated_must_get_agent_activity::create_sample_entries_batch'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.104786
s
SD
0.083264
s
p50
0.063845
s
<
p95
0.30176
s
<
p99
0.403603
s
WASM call duration: 'validated_must_get_agent_activity::create_validated_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.005201
s
SD
0.001766
s
p50
0.005946
s
<
p95
0.008912
s
<
p99
0.009393
s
WASM call duration: 'validated_must_get_agent_activity::get_chain_top_for_batch'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
1.559009
s
SD
6.631734
s
p50
1.941991
s
<
p95
21.413363
s
<
p99
35.013052
s
WASM call duration: 'validated_must_get_agent_activity::get_random_agent_with_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.044858
s
SD
4.422351
s
p50
0.763157
s
<
p95
11.959297
s
<
p99
21.988671
s
WASM call duration: 'validated_must_get_agent_activity_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001515
s
SD
0.000508
s
p50
0.001657
s
<
p95
0.002594
s
<
p99
0.002784
s
WASM call duration: 'validated_must_get_agent_activity_integrity::validate'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.024139
s
SD
0.249598
s
p50
0.005393
s
<
p95
0.559728
s
<
p99
1.174884
s
Host function call duration: '__hc__agent_info_1'
Duration of this host function call invoked from within WASM.
mean
4e-05
s
SD
4.5e-05
s
p50
3.8e-05
s
<
p95
0.000163
s
<
p99
0.000266
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000842
s
SD
0.0016
s
p50
0.000553
s
<
p95
0.001773
s
<
p99
0.006643
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.001097
s
SD
0.001569
s
p50
0.000583
s
<
p95
0.002244
s
<
p99
0.00589
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.625605
s
SD
5.659828
s
p50
1.151772
s
<
p95
12.213761
s
<
p99
35.005821
s
Host function call duration: '__hc__must_get_agent_activity_1'
Duration of this host function call invoked from within WASM.
mean
0.137028
s
SD
1.305578
s
p50
0.055712
s
<
p95
2.535716
s
<
p99
5.174744
s
Host function call duration: '__hc__query_1'
Duration of this host function call invoked from within WASM.
mean
0.04737
s
SD
0.061512
s
p50
0.015726
s
<
p95
0.196665
s
<
p99
0.272997
s
Host function call duration: '__hc__random_bytes_1'
Duration of this host function call invoked from within WASM.
mean
5e-06
s
SD
8e-06
s
p50
1.6e-05
s
<
p95
3.3e-05
s
<
p99
3.4e-05
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.002498
s
SD
0.001122
s
p50
0.002876
s
<
p95
0.004704
s
<
p99
0.006844
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.000642
s
SD
0.000639
s
p50
0.000471
s
<
p95
0.001581
s
<
p99
0.003113
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
43.06
s
<
mean
429.63
s
SD
248.99
s
<
p95
818.18
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 889.510219725s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
66.772936
s
SD
32.818836
s
p50
0
s
<
p95
60.439664
s
<
p99
109.916129
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
2.337661
s
SD
1.123932
s
p50
0.003434
s
<
p95
3.114412
s
<
p99
3.731753
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.18086
s
SD
0.197789
s
p50
0.002606
s
<
p95
0.453436
s
<
p99
1.138956
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.153278
s
SD
0.192209
s
p50
0.03034
s
<
p95
0.535143
s
<
p99
0.983321
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
8.292996
s
SD
10.454402
s
p50
0.005813
s
<
p95
28.968325
s
<
p99
46.310585
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
1.73099
s
SD
10.960274
s
p50
0.001885
s
<
p95
2.632584
s
<
p99
64.013581
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.001894
s
SD
0.001207
s
p50
0.000243
s
<
p95
0.003754
s
<
p99
0.005681
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.0034
s
SD
0.004871
s
p50
0.003641
s
<
p95
0.015587
s
<
p99
0.026062
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000898
s
SD
0.001816
s
p50
0.000494
s
<
p95
0.001833
s
<
p99
0.005938
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
4.661936
s
SD
28.505364
s
p50
3.726291
s
<
p95
60.003113
s
<
p99
60.496991
s
P2P request roundtrip: 'get_links'
Time spent sending a get_links request and awaiting its response.
mean
0.531457
s
SD
17.894923
s
p50
0.768993
s
<
p95
60.001236
s
<
p99
60.001715
s
P2P request roundtrip: 'must_get_agent_activity'
Time spent sending a must_get_agent_activity request and awaiting its response.
mean
0.371825
s
SD
29.238487
s
p50
1.297832
s
<
p95
60.001776
s
<
p99
60.004831
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
3.03324
s
SD
25.097964
s
p50
3.121712
s
<
p95
60.002017
s
<
p99
60.011821
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
15.92
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
5.33
KiB/s
<
mean
386.06
KiB/s
SD
407.05
KiB/s
<
p95
1.01
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
57
B/s
<
mean
45.04
KiB/s
SD
62.28
KiB/s
<
p95
170.70
KiB/s
Total bytes received
Total bytes received on primary network interface
count
44.32
GiB
mean
50.99
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
5.17
GiB
mean
5.95
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected

Warning CPU p99 reached 94.3%

Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
Detected

Critical Disk /etc/resolv.conf: 1/13 hosts above 90% capacity

Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected

Critical Heavy swap usage (81.4% swap used)

System overload anomaly
System overload anomaly was detected
Detected

Warning 14% of hosts overloaded (load5/ncpus > 1.0)

CPU usage
Total CPU usage and kernel CPU usage
Total
p5
1.96
%
<
mean
23.05
%
SD
28.83
%
<
p95
88.17
%
System
7.13
%
CPU percentiles
CPU usage percentiles
p50
9.39
%
p95
88.17
%
p99
94.29
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
96
hosts
mean time
136.46
s
Memory used percentage
Percentage of memory used
p5
7.7
%
<
mean
12.7
%
SD
8.08
%
<
p95
16.19
%
Memory available percentage
Percentage of available memory
p5
83.81
%
<
mean
87.3
%
SD
8.08
%
<
p95
92.3
%
Max host memory used
Maximum memory usage percentage across all hosts
max
89.66
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
81.42
%
Memory growth rate
Rate of memory growth over time
growth
0.44
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.99
MiB/s
Disk write throughput
Disk write throughput in MiB/s
130.52
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point /
0/119
hosts
Mount Point /efi-boot
0/6
hosts
Mount Point /etc/hostname
1/13
hosts
Mount Point /etc/hosts
1/13
hosts
Mount Point /etc/nomad.d/client-meta.json
0/13
hosts
Mount Point /etc/nomad.d/meta.json
1/1
hosts
Mount Point /etc/resolv.conf
1/13
hosts
Mount Point /nix/store
0/11
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
1.00
5 min
1.23
15 min
1.36
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
13.79
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
10.2019
%
SD
20.363
%
<
p95
65.25
%
60 second average
9.9787
%
300 second average
12.8306
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
0.0003
%
SD
0.0222
%
≤
p95
0
%
60 second average
0.0008
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
0.0003
%
SD
0.0222
%
≤
p95
0
%
60 second average
0.0008
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
2.6218
%
SD
6.1842
%
<
p95
15.77
%
60 second average
2.5568
%
300 second average
2.9306
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
1.349
%
SD
3.6815
%
<
p95
8.35
%
60 second average
1.3154
%
300 second average
1.2926
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.66
%
<
mean
19.65
%
SD
27.31
%
<
p95
81.87
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
263.36
KiB
<
mean
374.12
KiB
SD
85.26
KiB
<
p95
504.49
KiB
Holochain process threads
Number of threads in Holochain process
p5
20
threads
<
mean
29.88
threads
SD
7.46
threads
<
p95
37
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
58
file descriptors
<
mean
72.5
file descriptors
SD
15.17
file descriptors
<
p95
85
file descriptors

Remote Call Rate

↑ Back to index

Test the throughput of remote_call operations. Each agent in this scenario waits for a certain number of peers to be available or for up to two minutes, whichever happens first, before starting its behaviour.

Started
Thu, 22 Oct 2026 02:41:22 UTC
Duration
900s
Peer count
150
Behaviours
  • default (1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
remote_call_rate_39711995436_1
Dispatch timing
The time between sending a remote call and the remote handler being invoked.
Mean
mean
-2.268199
s
SD
30.56255
s
Max
Highest per-partition mean latency
max mean
0.488806
s
Min
Lowest per-partition mean latency
min mean
-33.912924
s
Round-trip timing
The total elapsed time to get a response to the client.
Mean
mean
0.24621
s
SD
0.843385
s
Max
Highest per-partition mean latency
max mean
1.294751
s
Min
Lowest per-partition mean latency
min mean
0.058657
s
Error count
The total number of errors accumulated during the run.
2157
Holochain MetricsClick to toggle visibility.
WASM usage: 'remote_call::call_echo_timestamp'
Metered usage count of the WASM ribosome for this zome function.
total
77002632
over 889.312277265s
mean rate
5.50618562208e+09
/s
std
2.01688946776e+11
/s
p5
5.58740064e+06
/s
p95
8.55267025386e+09
/s
peak
1.5379761596e+13
/s
WASM usage: 'remote_call::echo_timestamp'
Metered usage count of the WASM ribosome for this zome function.
total
25599561
over 889.312277265s
mean rate
6.5173863381e+08
/s
std
2.04934780056e+10
/s
p5
834732.13
/s
p95
1.16007090868e+09
/s
peak
1.45910801828e+12
/s
WASM usage: 'remote_call::init'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 889.312277265s
Not enough time data to show a trend.
Zome call duration: 'remote_call::call_echo_timestamp'
Duration of this zome call as measured by Holochain internally.
mean
0.273717
s
SD
1.054938
s
p50
0.359439
s
<
p95
1.589879
s
<
p99
3.273742
s
Zome call duration: 'remote_call::echo_timestamp'
Duration of this zome call as measured by Holochain internally.
mean
0.003743
s
SD
0.001226
s
p50
0.003779
s
<
p95
0.006667
s
<
p99
0.00694
s
WASM call duration: 'remote_call::call_echo_timestamp'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.298964
s
SD
1.159914
s
p50
0.423673
s
<
p95
1.724958
s
<
p99
4.092565
s
WASM call duration: 'remote_call::echo_timestamp'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.00239
s
SD
0.00083
s
p50
0.002433
s
<
p95
0.00455
s
<
p99
0.004719
s
WASM call duration: 'remote_call::init'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.00501
s
SD
0.002449
s
p50
0.004788
s
<
p95
0.009259
s
<
p99
0.012051
s
Host function call duration: '__hc__call_1'
Duration of this host function call invoked from within WASM.
mean
0.296045
s
SD
1.159875
s
p50
0.420889
s
<
p95
1.72154
s
<
p99
4.089319
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.00072
s
SD
0.000346
s
p50
0.000678
s
<
p95
0.001227
s
<
p99
0.002341
s
Host function call duration: '__hc__sys_time_1'
Duration of this host function call invoked from within WASM.
mean
1.2e-05
s
SD
6e-06
s
p50
1.2e-05
s
<
p95
2.4e-05
s
<
p99
2.7e-05
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.000335
s
SD
0.000784
s
p50
0.000428
s
<
p95
0.001
s
<
p99
0.001627
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.00057
s
SD
0.000871
s
p50
0.00024
s
<
p95
0.002566
s
<
p99
0.003966
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
43.65
s
<
mean
430.8
s
SD
249.82
s
<
p95
823.48
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
1654
over 889.312195585s
mean rate
8257.1
/s
std
114205.73
/s
p5
0
/s
p95
29039.55
/s
peak
9.40901755e+06
/s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
13.254769
s
SD
7.314243
s
p50
10.428787
s
<
p95
28.947539
s
<
p99
38.241076
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.009178
s
SD
0.011043
s
p50
0.010864
s
<
p95
0.033566
s
<
p99
0.053947
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.067702
s
SD
0.040696
s
p50
0.070866
s
<
p95
0.147181
s
<
p99
0.217795
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.018071
s
SD
0.008836
s
p50
0.01766
s
<
p95
0.03168
s
<
p99
0.040184
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
3.493791
s
SD
3.725091
s
p50
4.64744
s
<
p95
12.248929
s
<
p99
15.710079
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
3.868486
s
SD
7.507775
s
p50
2.71846
s
<
p95
18.015529
s
<
p99
42.433375
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.000403
s
SD
0.000158
s
p50
0.000361
s
<
p95
0.000716
s
<
p99
0.000783
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.003359
s
SD
0.001551
s
p50
0.003171
s
<
p95
0.006141
s
<
p99
0.00753
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000535
s
SD
0.000211
s
p50
0.000534
s
<
p95
0.000877
s
<
p99
0.001069
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
0.165253
s
SD
25.397964
s
p50
0.154498
s
<
p95
60.002641
s
<
p99
60.00424
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.427201
s
SD
17.874859
s
p50
0.344192
s
<
p95
60.001019
s
<
p99
60.001482
s
P2P request roundtrip: 'call_remote'
Time spent sending a call_remote request and awaiting its response.
mean
0.278952
s
SD
22.101311
s
p50
0.446038
s
<
p95
60.0013
s
<
p99
60.002108
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
6.72
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
12.99
KiB/s
<
mean
229.63
KiB/s
SD
451.56
KiB/s
<
p95
613.02
KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
5.69
KiB/s
<
mean
15.90
KiB/s
SD
13.90
KiB/s
<
p95
46.88
KiB/s
Total bytes received
Total bytes received on primary network interface
count
26.35
GiB
mean
30.32
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
1.82
GiB
mean
2.10
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
NotDetected
Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
Detected

Critical Disk /etc/resolv.conf: 1/13 hosts above 90% capacity

Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected

Critical Heavy swap usage (76.9% swap used)

System overload anomaly
System overload anomaly was detected
NotDetected
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
2.77
%
<
mean
10.85
%
SD
9.66
%
<
p95
21.2
%
System
4.13
%
CPU percentiles
CPU usage percentiles
p50
9.41
%
p95
21.20
%
p99
67.34
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
59
hosts
mean time
10.12
s
Memory used percentage
Percentage of memory used
p5
5.89
%
<
mean
9.74
%
SD
8.04
%
<
p95
12.16
%
Memory available percentage
Percentage of available memory
p5
87.84
%
<
mean
90.26
%
SD
8.04
%
<
p95
94.11
%
Max host memory used
Maximum memory usage percentage across all hosts
max
88.57
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
76.90
%
Memory growth rate
Rate of memory growth over time
growth
0.27
MiB/s
Disk read throughput
Disk read throughput in MiB/s
1.29
MiB/s
Disk write throughput
Disk write throughput in MiB/s
61.16
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point /
0/119
hosts
Mount Point /efi-boot
0/6
hosts
Mount Point /etc/hostname
1/13
hosts
Mount Point /etc/hosts
1/13
hosts
Mount Point /etc/nomad.d/client-meta.json
0/13
hosts
Mount Point /etc/nomad.d/meta.json
1/1
hosts
Mount Point /etc/resolv.conf
1/13
hosts
Mount Point /nix/store
0/11
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
0.34
5 min
0.26
15 min
0.14
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
0.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
2.7237
%
SD
2.2861
%
<
p95
7.33
%
60 second average
2.6123
%
300 second average
1.9917
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
0.0024
%
SD
0.1191
%
≤
p95
0
%
60 second average
0.0015
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
0.0023
%
SD
0.1181
%
≤
p95
0
%
60 second average
0.0015
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
0.809
%
SD
3.7009
%
<
p95
3
%
60 second average
0.7762
%
300 second average
0.5608
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
0.7061
%
SD
3.5273
%
<
p95
2.69
%
60 second average
0.6712
%
300 second average
0.4750
%
Holochain process CPU usage
CPU usage by Holochain process
p5
1.72
%
<
mean
8.75
%
SD
9.17
%
<
p95
17.4
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
211.41
KiB
<
mean
238.62
KiB
SD
31.40
KiB
<
p95
282.73
KiB
Holochain process threads
Number of threads in Holochain process
p5
28
threads
<
mean
32.39
threads
SD
7.55
threads
<
p95
48
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
75
file descriptors
<
mean
80.03
file descriptors
SD
10.13
file descriptors
<
p95
92
file descriptors

Remote Signals

↑ Back to index

This scenario tests the throughput of remote_signals operations.

Started
Thu, 22 Oct 2026 02:58:20 UTC
Duration
900s
Peer count
300
Behaviours
  • default (1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
remote_signals_39711995436_1
Round trip time
The time from origin signal dispatch to origin receive of the remote side's response signal.
mean
2.668614
s
SD
12.461066
s
p50
0.207258
s
<
p95
12.148493
s
<
p99
62.813185
s
Timeouts
The number of timeouts waiting for the remote side's response signal (default timeout is 20 seconds).
total
552
over 1852.005503112s
mean rate
6911.32
/s
std
75904.2
/s
p5
4.91
/s
p95
16968.83
/s
peak
5.81468051e+06
/s
Timeout rate
The proportion of remote signal round-trips that timed out.
0.0031
Holochain MetricsClick to toggle visibility.
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
7
over 817.86086237s
mean rate
987.17
/s
std
2099.54
/s
p5
0
/s
p95
5425.53
/s
peak
9502.45
/s
WASM usage: 'remote_signal::init'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 889.890553713s
Not enough time data to show a trend.
WASM usage: 'remote_signal::recv_remote_signal'
Metered usage count of the WASM ribosome for this zome function.
total
24510904
over 889.890553713s
mean rate
5.1030543839e+09
/s
std
9.27803339432e+10
/s
p5
1.56157374e+06
/s
p95
6.78950937128e+09
/s
peak
6.20281702128e+12
/s
WASM usage: 'remote_signal::signal_request'
Metered usage count of the WASM ribosome for this zome function.
total
12994691
over 889.890553713s
mean rate
2.7339047675e+08
/s
std
1.13017273491e+10
/s
p5
0
/s
p95
1.3684021213e+08
/s
peak
9.0421559633e+11
/s
Zome call duration: 'remote_signal::recv_remote_signal'
Duration of this zome call as measured by Holochain internally.
mean
0.004222
s
SD
0.001079
s
p50
0.004242
s
<
p95
0.005857
s
<
p99
0.00642
s
Zome call duration: 'remote_signal::signal_request'
Duration of this zome call as measured by Holochain internally.
mean
0.004076
s
SD
0.000914
s
p50
0.004103
s
<
p95
0.00546
s
<
p99
0.0062
s
WASM call duration: 'remote_signal::init'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.007834
s
SD
0.021336
s
p50
0.003702
s
<
p95
0.020459
s
<
p99
0.183907
s
WASM call duration: 'remote_signal::recv_remote_signal'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.002727
s
SD
0.000715
s
p50
0.002718
s
<
p95
0.003989
s
<
p99
0.004352
s
WASM call duration: 'remote_signal::signal_request'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.002561
s
SD
0.000572
s
p50
0.002544
s
<
p95
0.003479
s
<
p99
0.003968
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.001036
s
SD
0.002475
s
p50
0.000594
s
<
p95
0.001827
s
<
p99
0.017917
s
Host function call duration: '__hc__emit_signal_1'
Duration of this host function call invoked from within WASM.
mean
7.5e-05
s
SD
2.3e-05
s
p50
7.7e-05
s
<
p95
0.00011
s
<
p99
0.000123
s
Host function call duration: '__hc__send_remote_signal_1'
Duration of this host function call invoked from within WASM.
mean
7.3e-05
s
SD
2.9e-05
s
p50
7.4e-05
s
<
p95
0.000115
s
<
p99
0.000145
s
Host function call duration: '__hc__sys_time_1'
Duration of this host function call invoked from within WASM.
mean
9e-06
s
SD
5e-06
s
p50
9e-06
s
<
p95
1.8e-05
s
<
p99
2.5e-05
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.003338
s
SD
0.019243
s
p50
0.000421
s
<
p95
0.003623
s
<
p99
0.159143
s
Emitted local signals
Count of local signals emitted via the emit_signal host function.
total
1386
over 889.890625317s
mean rate
342393.96
/s
std
3.51421852e+06
/s
p5
25.24
/s
p95
465356.77
/s
peak
1.7363751584e+08
/s
Sent remote signals
Count of remote signals sent via the send_remote_signal host function.
total
1468
over 889.89062703s
mean rate
280880.06
/s
std
2.34066985e+06
/s
p5
22.09
/s
p95
402706.98
/s
peak
1.0810810811e+08
/s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.000818
s
SD
0.001952
s
p50
0.00022
s
<
p95
0.003698
s
<
p99
0.011608
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
43.09
s
<
mean
429.84
s
SD
249.14
s
<
p95
821.78
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
3381
over 889.890501647s
mean rate
4.56779295e+06
/s
std
1.6731982669e+08
/s
p5
120.55
/s
p95
2.18384967e+06
/s
peak
1.484e+10
/s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
19.950931
s
SD
11.681296
s
p50
15.829312
s
<
p95
41.265563
s
<
p99
55.190201
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.015921
s
SD
0.028456
s
p50
0.017746
s
<
p95
0.086957
s
<
p99
0.141884
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
<
p99
1e-06
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.081348
s
SD
0.073395
s
p50
0.089307
s
<
p95
0.244223
s
<
p99
0.375784
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.027485
s
SD
0.020805
s
p50
0.024135
s
<
p95
0.056467
s
<
p99
0.112753
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
4.705531
s
SD
7.972529
s
p50
7.242
s
<
p95
24.633134
s
<
p99
38.069933
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
5.889747
s
SD
18.837854
s
p50
2.35224
s
<
p95
52.874091
s
<
p99
88.312117
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.000663
s
SD
0.000274
s
p50
0.000592
s
<
p95
0.001109
s
<
p99
0.00125
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.004221
s
SD
0.002294
s
p50
0.003588
s
<
p95
0.007487
s
<
p99
0.011401
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000802
s
SD
0.000774
s
p50
0.00075
s
<
p95
0.00156
s
<
p99
0.003089
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
0.307257
s
SD
28.664846
s
p50
0.455707
s
<
p95
60.003742
s
<
p99
60.008963
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.580341
s
SD
25.32326
s
p50
0.572843
s
<
p95
60.001665
s
<
p99
60.002087
s
P2P received remote signals
Count of remote signals received.
total
1386
over 889.890574996s
mean rate
583061.85
/s
std
7.85646568e+06
/s
p5
31.96
/s
p95
341856.97
/s
peak
3.3999344907e+08
/s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
11.62
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
21.23
KiB/s
<
mean
310.51
KiB/s
SD
451.68
KiB/s
<
p95
1.03
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
13.10
KiB/s
<
mean
39.68
KiB/s
SD
33.68
KiB/s
<
p95
110.34
KiB/s
Total bytes received
Total bytes received on primary network interface
count
25.91
GiB
mean
29.81
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
3.31
GiB
mean
3.81
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected

Warning CPU p99 reached 94.8%

Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
Detected

Critical Disk /etc/resolv.conf: 1/13 hosts above 90% capacity

Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected

Critical Heavy swap usage (81.1% swap used)

System overload anomaly
System overload anomaly was detected
NotDetected
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
3.48
%
<
mean
17.65
%
SD
15.33
%
<
p95
43.64
%
System
6.03
%
CPU percentiles
CPU usage percentiles
p50
14.56
%
p95
43.64
%
p99
94.77
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
75
hosts
mean time
24.42
s
Memory used percentage
Percentage of memory used
p5
7.25
%
<
mean
12.79
%
SD
9.29
%
<
p95
15.44
%
Memory available percentage
Percentage of available memory
p5
84.56
%
<
mean
87.21
%
SD
9.29
%
<
p95
92.75
%
Max host memory used
Maximum memory usage percentage across all hosts
max
88.63
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
81.13
%
Memory growth rate
Rate of memory growth over time
growth
0.43
MiB/s
Disk read throughput
Disk read throughput in MiB/s
1.06
MiB/s
Disk write throughput
Disk write throughput in MiB/s
72.63
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point /
0/82
hosts
Mount Point /efi-boot
0/6
hosts
Mount Point /etc/hostname
1/13
hosts
Mount Point /etc/hosts
1/13
hosts
Mount Point /etc/nomad.d/client-meta.json
0/13
hosts
Mount Point /etc/nomad.d/meta.json
1/1
hosts
Mount Point /etc/resolv.conf
1/13
hosts
Mount Point /nix/store
0/11
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
0.65
5 min
0.59
15 min
0.41
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
0.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
7.3619
%
SD
11.2067
%
<
p95
22.52
%
60 second average
7.2258
%
300 second average
6.4346
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
0.0011
%
SD
0.0328
%
≤
p95
0
%
60 second average
0.0013
%
300 second average
0.0011
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
0.0011
%
SD
0.0327
%
≤
p95
0
%
60 second average
0.0013
%
300 second average
0.0011
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
0.7003
%
SD
1.6356
%
<
p95
2.73
%
60 second average
0.7310
%
300 second average
0.8165
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
0.4789
%
SD
1.4126
%
<
p95
1.96
%
60 second average
0.5020
%
300 second average
0.6068
%
Holochain process CPU usage
CPU usage by Holochain process
p5
1.15
%
<
mean
7.51
%
SD
7.45
%
<
p95
20.55
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
189.21
KiB
<
mean
227.34
KiB
SD
44.30
KiB
<
p95
283.93
KiB
Holochain process threads
Number of threads in Holochain process
p5
27
threads
<
mean
33.25
threads
SD
9.21
threads
<
p95
51
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
72
file descriptors
<
mean
80.48
file descriptors
SD
14.26
file descriptors
<
p95
95
file descriptors

Validation Receipts

↑ Back to index

Creates an entry, wait for required validation receipts, then repeat. Records the amount of time it took to accumulate the required number of receipts for all DHT operations. This is measured to the nearest 20ms so that we don't keep the agent too busy checking for receipts.

Each agent in this scenario waits for a certain number of peers to be available or for up to two minutes, whichever happens first, before starting its behaviour.

By default, this scenario will wait for a complete set of validation receipts before committing the next record. If the NO_VALIDATION_COMPLETE environment variable is set, it will instead publish new records on every round, building up an ever-growing list of action hashes to check on.

Started
Thu, 22 Oct 2026 04:02:23 UTC
Duration
900s
Peer count
150
Peer count at end
103
Behaviours
  • default (1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
validation_receipts_39711995436_1
Receipts complete timing
The amount of time between publishing a record and receiving the required number of validation receipts.
Mean
mean
33.493757
s
SD
17.756045
s
Max
Highest per-partition mean latency
max mean
633.200107
s
Min
Lowest per-partition mean latency
min mean
17.659241
s
Receipts complete rate
The number of complete validation receipt sets collected per second.
Mean
mean
1
/s
Max
Highest per-partition mean rate
max
1.25
/s
Min
Lowest per-partition mean rate
min
0
/s
Error count
The total number of errors accumulated during the run.
0
Holochain MetricsClick to toggle visibility.
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
0
over 725.330213305s
Not enough time data to show a trend.
WASM usage: 'crud::create_sample_entry'
Metered usage count of the WASM ribosome for this zome function.
total
1022820
over 889.618830246s
mean rate
1.032862605e+07
/s
std
3.534878062e+07
/s
p5
0
/s
p95
5.024891592e+07
/s
peak
4.9519764187e+08
/s
WASM usage: 'crud::get_sample_entry_validation_receipts'
Metered usage count of the WASM ribosome for this zome function.
total
9210924
over 889.618830246s
mean rate
8.49122685e+07
/s
std
2.3934681305e+08
/s
p5
145210.81
/s
p95
4.0781460248e+08
/s
peak
3.54546640657e+09
/s
WASM usage: 'crud::init'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 889.618830246s
Not enough time data to show a trend.
WASM usage: 'crud_integrity::entry_defs'
Metered usage count of the WASM ribosome for this zome function.
total
132120
over 889.618830246s
mean rate
1.32723673e+06
/s
std
4.54034595e+06
/s
p5
0
/s
p95
6.36340551e+06
/s
peak
6.39658126e+07
/s
Zome call duration: 'crud::create_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.007482
s
SD
0.001889
s
p50
0.007818
s
<
p95
0.009967
s
<
p99
0.011446
s
Zome call duration: 'crud::get_sample_entry_validation_receipts'
Duration of this zome call as measured by Holochain internally.
mean
0.020539
s
SD
0.011326
s
p50
0.014158
s
<
p95
0.040962
s
<
p99
0.050539
s
WASM call duration: 'crud::create_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.006138
s
SD
0.001526
s
p50
0.006448
s
<
p95
0.008119
s
<
p99
0.009453
s
WASM call duration: 'crud::get_sample_entry_validation_receipts'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.018974
s
SD
0.011001
s
p50
0.012699
s
<
p95
0.038751
s
<
p99
0.048312
s
WASM call duration: 'crud::init'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.004148
s
SD
0.005785
s
p50
0.003184
s
<
p95
0.006072
s
<
p99
0.048424
s
WASM call duration: 'crud_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.002013
s
SD
0.000436
s
p50
0.002117
s
<
p95
0.002535
s
<
p99
0.002918
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.00067
s
SD
0.000487
s
p50
0.000695
s
<
p95
0.001564
s
<
p99
0.003089
s
Host function call duration: '__hc__get_validation_receipts_1'
Duration of this host function call invoked from within WASM.
mean
0.016548
s
SD
0.010787
s
p50
0.010238
s
<
p95
0.035923
s
<
p99
0.045433
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.003141
s
SD
0.000794
s
p50
0.003296
s
<
p95
0.004183
s
<
p99
0.004894
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.000748
s
SD
0.000616
s
p50
0.000636
s
<
p95
0.001773
s
<
p99
0.002931
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
44.42
s
<
mean
430.53
s
SD
249.33
s
<
p95
822.29
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
8912
over 889.618784459s
mean rate
19129.93
/s
std
60168.39
/s
p5
98.1
/s
p95
102702.08
/s
peak
1.27482253e+06
/s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
14.100518
s
SD
8.973563
s
p50
11.213755
s
<
p95
31.546607
s
<
p99
36.720259
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.00537
s
SD
0.039938
s
p50
0.008222
s
<
p95
0.107025
s
<
p99
0.191253
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.043582
s
SD
0.08804
s
p50
0.070154
s
<
p95
0.279103
s
<
p99
0.378827
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.07012
s
SD
0.045792
s
p50
0.044105
s
<
p95
0.154601
s
<
p99
0.20107
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.360811
s
SD
6.346054
s
p50
0.855026
s
<
p95
17.876523
s
<
p99
26.497467
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
19.634754
s
SD
22.761567
s
p50
24.889954
s
<
p95
71.007433
s
<
p99
98.145106
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.001182
s
SD
0.000577
s
p50
0.000886
s
<
p95
0.002127
s
<
p99
0.00262
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.005704
s
SD
0.005037
s
p50
0.003408
s
<
p95
0.007823
s
<
p99
0.032529
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.00065
s
SD
0.000222
s
p50
0.000623
s
<
p95
0.000981
s
<
p99
0.001123
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
0.323238
s
SD
28.980598
s
p50
0.164759
s
<
p95
60.003397
s
<
p99
60.004607
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.184183
s
SD
0.15018
s
p50
0.18329
s
<
p95
0.397473
s
<
p99
0.901526
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
13.48
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
16.32
KiB/s
<
mean
145.34
KiB/s
SD
318.80
KiB/s
<
p95
408.78
KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
5.61
KiB/s
<
mean
19.14
KiB/s
SD
16.54
KiB/s
<
p95
50.86
KiB/s
Total bytes received
Total bytes received on primary network interface
count
12.49
GiB
mean
14.37
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
1.65
GiB
mean
1.89
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
NotDetected
Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
Detected

Critical Disk /etc/resolv.conf: 1/14 hosts above 90% capacity

Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected

Critical Heavy swap usage (79.0% swap used)

System overload anomaly
System overload anomaly was detected
NotDetected
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
2.97
%
<
mean
19.7
%
SD
13.39
%
<
p95
41.72
%
System
6.23
%
CPU percentiles
CPU usage percentiles
p50
18.20
%
p95
41.72
%
p99
68.79
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
33
hosts
mean time
10.76
s
Memory used percentage
Percentage of memory used
p5
6.16
%
<
mean
10.94
%
SD
9.28
%
<
p95
13
%
Memory available percentage
Percentage of available memory
p5
87
%
<
mean
89.06
%
SD
9.28
%
<
p95
93.84
%
Max host memory used
Maximum memory usage percentage across all hosts
max
87.90
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
79.00
%
Memory growth rate
Rate of memory growth over time
growth
0.36
MiB/s
Disk read throughput
Disk read throughput in MiB/s
1.15
MiB/s
Disk write throughput
Disk write throughput in MiB/s
69.94
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point /
0/85
hosts
Mount Point /efi-boot
0/6
hosts
Mount Point /etc/hostname
1/14
hosts
Mount Point /etc/hosts
1/14
hosts
Mount Point /etc/nomad.d/client-meta.json
0/13
hosts
Mount Point /etc/nomad.d/meta.json
1/1
hosts
Mount Point /etc/resolv.conf
1/14
hosts
Mount Point /nix/store
0/11
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
0.74
5 min
1.03
15 min
1.56
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
0.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
5.8315
%
SD
5.0892
%
<
p95
15.39
%
60 second average
5.7395
%
300 second average
8.4753
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
0.0022
%
SD
0.0971
%
≤
p95
0
%
60 second average
0.0013
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
0.0022
%
SD
0.0971
%
≤
p95
0
%
60 second average
0.0012
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
3.9722
%
SD
9.4619
%
<
p95
22.47
%
60 second average
3.7873
%
300 second average
3.4365
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
3.3649
%
SD
8.9471
%
<
p95
21.11
%
60 second average
3.1995
%
300 second average
2.7146
%
Holochain process CPU usage
CPU usage by Holochain process
p5
1.76
%
<
mean
17.4
%
SD
12.58
%
<
p95
37.84
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
234.50
KiB
<
mean
274.43
KiB
SD
36.03
KiB
<
p95
313.79
KiB
Holochain process threads
Number of threads in Holochain process
p5
29
threads
<
mean
32.53
threads
SD
8.75
threads
<
p95
50
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
75
file descriptors
<
mean
80.02
file descriptors
SD
13.11
file descriptors
<
p95
95
file descriptors

Write/get_agent_activity

↑ Back to index

A scenario where write peers write entries, while get_agent_activity peers each query a single write agent's activity with get_agent_activity.

Before a target write peer and the requesting get_agent_activity peer are in sync, this will measure the get_agent_activity call performance over a network. Once a write peer reaches sync with a get_agent_activity peer, the write peer will publish their actions and entries, and so the get_agent_activity calls will likely have most of the data they need locally. At that point this measures the database query performance and code paths through host functions.

Started
Thu, 22 Oct 2026 04:19:34 UTC
Duration
900s
Peer count
150
Peer count at end
137
Behaviours
  • get_agent_activity (1 agent)
  • write (1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
write_get_agent_activity_39711995436_1
Observed chain advancement
The maximum highest-observed action sequence per write agent, aggregated across all reading agents. The reading dimension is collapsed by taking the maximum observed value for each write agent (any reader successfully tracking that point counts as propagation). Captures how far readers tracked writers’ chains during the run.
Mean max (across write agents)
Mean of the highest chain head value observed per write agent. Each write agent contributes its maximum — the value seen by any reader.
2277.9
Max
Highest chain head value observed for any single write agent.
5883
Write agents observed
21
get_agent_activity_full zome call timing
The time taken to call the zome function that retrieves information on a write peer's source chain.
Mean
mean
0.80387
s
SD
1.92101
s
Max
Highest per-partition mean latency
max mean
3.293385
s
Min
Lowest per-partition mean latency
min mean
0.01853
s
Error count
The number of errors accumulated.
261
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.075713
s
SD
2.99087
s
p50
0.193754
s
<
p95
4.531535
s
<
p99
17.696057
s
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
0
over 781.316576887s
Not enough time data to show a trend.
Zome call duration: 'agent_activity::announce_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.007144
s
SD
0.001911
s
p50
0.007276
s
<
p95
0.010333
s
<
p99
0.011698
s
Zome call duration: 'agent_activity::create_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.01121
s
SD
0.009029
s
p50
0.014549
s
<
p95
0.030387
s
<
p99
0.036185
s
Zome call duration: 'agent_activity::get_agent_activity_full'
Duration of this zome call as measured by Holochain internally.
mean
0.236695
s
SD
1.243954
s
p50
0.392694
s
<
p95
3.567911
s
<
p99
5.498801
s
Zome call duration: 'agent_activity::get_random_agent_with_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.006835
s
SD
3.908415
s
p50
0.007777
s
<
p95
7.181836
s
<
p99
30.004651
s
WASM call duration: 'agent_activity::announce_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.005931
s
SD
0.001587
s
p50
0.005914
s
<
p95
0.008327
s
<
p99
0.009684
s
WASM call duration: 'agent_activity::create_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.009775
s
SD
0.008236
s
p50
0.012509
s
<
p95
0.027493
s
<
p99
0.032681
s
WASM call duration: 'agent_activity::get_agent_activity_full'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.2358
s
SD
1.243755
s
p50
0.390162
s
<
p95
3.566826
s
<
p99
5.498094
s
WASM call duration: 'agent_activity::get_random_agent_with_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.006009
s
SD
3.908397
s
p50
0.006483
s
<
p95
7.180243
s
<
p99
30.003996
s
WASM call duration: 'agent_activity_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001641
s
SD
0.000605
s
p50
0.001813
s
<
p95
0.002946
s
<
p99
0.003451
s
Host function call duration: '__hc__agent_info_1'
Duration of this host function call invoked from within WASM.
mean
5.6e-05
s
SD
3.9e-05
s
p50
4.3e-05
s
<
p95
0.000165
s
<
p99
0.000177
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.004157
s
SD
0.005131
s
p50
0.005155
s
<
p95
0.015325
s
<
p99
0.018553
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.000652
s
SD
0.000373
s
p50
0.000604
s
<
p95
0.001084
s
<
p99
0.0026
s
Host function call duration: '__hc__get_agent_activity_1'
Duration of this host function call invoked from within WASM.
mean
0.233577
s
SD
1.243443
s
p50
0.386954
s
<
p95
3.56463
s
<
p99
5.494894
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.001563
s
SD
3.90835
s
p50
0.000604
s
<
p95
7.1738
s
<
p99
30.000566
s
Host function call duration: '__hc__random_bytes_1'
Duration of this host function call invoked from within WASM.
mean
1.1e-05
s
SD
6e-06
s
p50
1.4e-05
s
<
p95
2.4e-05
s
<
p99
3.2e-05
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.002893
s
SD
0.001688
s
p50
0.002962
s
<
p95
0.007068
s
<
p99
0.009107
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.001496
s
SD
0.001386
s
p50
0.000316
s
<
p95
0.004187
s
<
p99
0.006075
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
43.25
s
<
mean
430.93
s
SD
249.74
s
<
p95
823.12
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
25105
over 890.010614842s
mean rate
605819.2
/s
std
4.7403155e+06
/s
p5
188.52
/s
p95
1.02356415e+06
/s
peak
1.3107201966e+08
/s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
97.053568
s
SD
76.746258
s
p50
26.670343
s
<
p95
219.922415
s
<
p99
412.533606
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.144573
s
SD
0.378133
s
p50
0.115127
s
<
p95
0.783311
s
<
p99
1.679897
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
2e-06
s
SD
2.8e-05
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.114618
s
SD
0.401896
s
p50
0.293757
s
<
p95
1.21891
s
<
p99
1.435319
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.747357
s
SD
0.67568
s
p50
0.108035
s
<
p95
1.944293
s
<
p99
3.250965
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
17.066606
s
SD
23.02452
s
p50
20.767858
s
<
p95
64.879786
s
<
p99
123.413271
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
14.610084
s
SD
23.027216
s
p50
2.74044
s
<
p95
62.800968
s
<
p99
118.95354
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.004518
s
SD
0.002965
s
p50
0.003823
s
<
p95
0.009165
s
<
p99
0.010508
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.006912
s
SD
0.005027
s
p50
0.003865
s
<
p95
0.017145
s
<
p99
0.022393
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.003805
s
SD
0.004833
s
p50
0.001012
s
<
p95
0.014388
s
<
p99
0.021142
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
1.537216
s
SD
29.3511
s
p50
1.843967
s
<
p95
60.018365
s
<
p99
60.044362
s
P2P request roundtrip: 'get_links'
Time spent sending a get_links request and awaiting its response.
mean
2.486041
s
SD
10.490826
s
p50
0.88967
s
<
p95
17.692303
s
<
p99
60.00028
s
P2P request roundtrip: 'get_agent_activity'
Time spent sending a get_agent_activity request and awaiting its response.
mean
0.181706
s
SD
29.183275
s
p50
0.498504
s
<
p95
60.006537
s
<
p99
60.031981
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.734516
s
SD
24.505394
s
p50
2.763149
s
<
p95
60.008234
s
<
p99
60.047638
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
42.94
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
12.81
KiB/s
<
mean
810.05
KiB/s
SD
677.32
KiB/s
<
p95
2.00
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
0
B/s
<
mean
107.03
KiB/s
SD
180.89
KiB/s
<
p95
345.54
KiB/s
Total bytes received
Total bytes received on primary network interface
count
92.98
GiB
mean
106.98
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
12.29
GiB
mean
14.14
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected

Warning CPU p99 reached 96.7%

Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
Detected

Critical Disk /etc/resolv.conf: 1/14 hosts above 90% capacity

Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected

Critical Heavy swap usage (79.1% swap used)

System overload anomaly
System overload anomaly was detected
Detected

Warning 54% of hosts overloaded (load5/ncpus > 1.0)

CPU usage
Total CPU usage and kernel CPU usage
Total
p5
10.97
%
<
mean
60.59
%
SD
24.29
%
<
p95
93.2
%
System
17.65
%
CPU percentiles
CPU usage percentiles
p50
65.05
%
p95
93.20
%
p99
96.70
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
111
hosts
mean time
269.24
s
Memory used percentage
Percentage of memory used
p5
6.75
%
<
mean
11.28
%
SD
8.13
%
<
p95
14.76
%
Memory available percentage
Percentage of available memory
p5
85.24
%
<
mean
88.72
%
SD
8.13
%
<
p95
93.25
%
Max host memory used
Maximum memory usage percentage across all hosts
max
89.22
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
79.08
%
Memory growth rate
Rate of memory growth over time
growth
0.49
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.94
MiB/s
Disk write throughput
Disk write throughput in MiB/s
143.91
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point /
0/119
hosts
Mount Point /efi-boot
0/6
hosts
Mount Point /etc/hostname
1/14
hosts
Mount Point /etc/hosts
1/14
hosts
Mount Point /etc/nomad.d/client-meta.json
0/13
hosts
Mount Point /etc/nomad.d/meta.json
1/1
hosts
Mount Point /etc/resolv.conf
1/14
hosts
Mount Point /nix/store
0/11
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
3.22
5 min
2.40
15 min
1.77
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
54.11
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0.12
%
<
mean
39.7957
%
SD
26.6865
%
<
p95
78.88
%
60 second average
37.1330
%
300 second average
27.1339
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
11.6916
%
SD
9.0597
%
<
p95
25.39
%
60 second average
11.0059
%
300 second average
8.4675
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
4.8404
%
SD
4.8062
%
<
p95
13.78
%
60 second average
4.6001
%
300 second average
3.9084
%
Holochain process CPU usage
CPU usage by Holochain process
p5
13.64
%
<
mean
58.71
%
SD
22.11
%
<
p95
88.02
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
228.24
KiB
<
mean
332.03
KiB
SD
78.65
KiB
<
p95
451.58
KiB
Holochain process threads
Number of threads in Holochain process
p5
27
threads
<
mean
33.07
threads
SD
7.92
threads
<
p95
49
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
70
file descriptors
<
mean
81.4
file descriptors
SD
12.65
file descriptors
<
p95
93
file descriptors

Write/get_agent_activity with volatile conductors

↑ Back to index

A scenario where write peers write entries, while get_agent_activity_volatile peers each query a single write agents activity with get_agent_activity but shutdown and restart their conductors at semi-random intervals.

Before a target write peer and the requesting get_agent_activity_volatile peer are in sync, this will measure the get_agent_activity call performance over a network. Once a write peer reaches sync with a get_agent_activity peer, the write peer will publish their actions and entries, and so the get_agent_activity calls will likely have most of the data they need locally. At that point this measures the database query performance and code paths through host functions.

Started
Thu, 22 Oct 2026 04:36:34 UTC
Duration
1800s
Peer count
150
Peer count at end
114
Behaviours
  • get_agent_activity_volatile (1 agent)
  • write (1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
write_get_agent_activity_volatile_39711995436_1
Highest observed action_seq
The rate at which zero-arc readers observe new chain heads on the writers' chains via get_agent_activity. This reflects the DHT's ability to propagate agent activity ops and make them available to querying peers.
Mean
mean
270.04
/s
Max
Highest per-partition mean rate
max
1166.42
/s
Min
Lowest per-partition mean rate
min
106.83
/s
get_agent_activity_full zome call timing
The time taken to call the zome function that retrieves information on a write peer's source chain.
Mean
mean
0.031534
s
SD
0.249201
s
Max
Highest per-partition mean latency
max mean
0.070772
s
Min
Lowest per-partition mean latency
min mean
0.006166
s
Running volatile conductors count
The number of conductors running by get_agent_activity_volatile peers.
p5
50
conductors
<
mean
66.4
conductors
SD
11.91
conductors
<
p95
83
conductors
Volatile conductor total running duration
The total running duration of a get_agent_activity_volatile conductor.
Mean
mean
962.148467
s
SD
494.627798
s
Max
Highest per-partition mean latency
max mean
1445.481481
s
Min
Lowest per-partition mean latency
min mean
508.741935
s
Volatile conductor running duration
The duration that a get_agent_activity_volatile conductor was running before being stopped.
Mean
mean
47.852322
s
SD
45.743515
s
Max
Highest per-partition mean latency
max mean
141.509238
s
Min
Lowest per-partition mean latency
min mean
24.663022
s
Volatile conductor stopped duration
The duration that a get_agent_activity_volatile conductor was stopped before being started again.
Mean
mean
37.315785
s
SD
43.961202
s
Max
Highest per-partition mean latency
max mean
68.359546
s
Min
Lowest per-partition mean latency
min mean
7.941791
s
Reached target arc
Did get_agent_activity_volatile peer reach their target arc in the moment before they were shutdown.
  • get_agent_activity_volatile_agent: uhCAk--zMviWklX6utoJPtoxFi0xMFbuEs3W4nOzE3nSSv50cgCKu
p5
0
<
mean
0.18
SD
0.38
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk-Fn8rNC2yj2Hp0TCCrJ-MQcpxVX5WomiQz6oSRCprU63yOek
p5
0
<
mean
0.15
SD
0.36
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk-tAUIMXtxUtALxxGEFY57U4uYSM34UB96eRDmeaGg87KnNe0
p5
0
<
mean
0.17
SD
0.38
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk0WcqFcHRXiUiBjLBtfT6cR1wWQXkUgCoGedXGKwEIV4GbBGv
p5
0
<
mean
0.17
SD
0.38
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk1aj4GzfnEoPhVNpNwB2A-jCKpxJYAFURQQXhA7ctO9cooFZ5
p5
0
<
mean
0.15
SD
0.36
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk2aBiJZqsv-ceHHr646NQ8ZYUUHoskoUCfCS75f2rYj4XRQzR
p5
0
<
mean
0.13
SD
0.34
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk2yBuU1oMZfnK4smej2fuQjdyeWmLbU29mjopUQ_yKvBf434S
p5
0
<
mean
0.21
SD
0.41
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk3c0Ona5r_4DFZ6dhiCCt9jjaGPw36aU7fVbMV73u4aocmiFu
p5
0
<
mean
0.22
SD
0.42
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk3gP1DM1G1buZdVU6OnulyHpAe212sDq_qhR7SV-nOs-Ki4Cu
p5
0
≤
mean
0
SD
0
≤
p95
0
  • get_agent_activity_volatile_agent: uhCAk48n8qO86ragVtHYActgGVcik1vB0I-SHa8hLxOK6WTWOR8LW
p5
0
<
mean
0.17
SD
0.37
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk4Ptq5urA6K49IkCfoYiuQIHMhWwgh5qdSM6fuTCxeCpNM5oY
p5
0
<
mean
0.18
SD
0.38
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk6UrpMpiJw2XaA2s2qHpvfWmsB2Q1AArPB-6TOK9swk7fPiaU
p5
0
<
mean
0.18
SD
0.38
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk8q46nrjC5QYQ9Hf-vQcQQNAhF7ZJ9IksOY4jqxrBTbEneqj6
p5
0
<
mean
0.17
SD
0.37
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk9NlY8bi0p8Ikqus8SwyUozun9IMherIBA7vv1S2oy38yCa3P
p5
0
<
mean
0.21
SD
0.41
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkAEk_jl_qRplzx-167XxA9610G8UWPVjlTdDfMHtfQx53TIFj
p5
0
<
mean
0.16
SD
0.37
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkAgrmWX9OyeKz5BuRw4Xr4DQPnLcgX_7ETWSGcQpSHxZ8gKsz
p5
0
<
mean
0.14
SD
0.35
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkBV2kAtwIBHbG9qcb69_hLvg4qpXnjLsjnRY5Oy-5AG2JaFox
p5
0
<
mean
0.16
SD
0.36
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkBrYLakxj6Y5FypeR4DxdIKOE9MG3hpHvRvF5fEDT6oh4mPhz
p5
0
<
mean
0.47
SD
0.5
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkC5rwvZEjw1rYVxwOWFVvGUXrkdA3YXs1Jmbfa10fNrD3bvvU
p5
0
<
mean
0.12
SD
0.32
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkCadGi4XJCGcKNlK_zV30m9R9TBb-LB5s5ku6r3_qA8QkIJkQ
p5
0
<
mean
0.23
SD
0.42
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkCcEF3HfnLNxfkhDvSAjiDfjRrKELEHPrBIeZiWMHqyw4q_Uo
p5
0
<
mean
0.15
SD
0.36
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkDANEousFy5ISfxdmBnE6DySyDvBERdpyEMrmsTrXWn2lRGAV
p5
0
<
mean
0.12
SD
0.33
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkDAlm-_vBcxPn98wh3L8kL-M3vUr1t7WuY8wnVlYpov5zYLQe
p5
0
<
mean
0.18
SD
0.38
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkDnABMysS2tUq5JsGDBPDdQTBSccRSneHrwujyd76EwwLZ2_g
p5
0
<
mean
0.21
SD
0.41
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkDswk8tMdvAVcrUg6Ia5BaIrpmCEyd-TBfD3_7U_0I3QWo-Gp
p5
0
<
mean
0.11
SD
0.31
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkE4Zo09yfkAkIeOEjG7_WDy7_fanzIQL-8DirX2ytYjOQ3Bua
p5
0
<
mean
0.23
SD
0.42
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkEHufb_pw6tjj99-H-_JGOSuxtLM-ZY74h6JEEFsGlgYLKbTT
p5
0
<
mean
0.18
SD
0.39
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkFQoKAhcgJmw_QeDh9_84Ygzn_2ThMEkqs-uE4sgIamLowGYB
p5
0
<
mean
0.06
SD
0.24
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkFZ_sWuxxpoEY57jwCeUJEAGdfWUvjQlpXANMUcB6DRWYm2L2
p5
0
<
mean
0.25
SD
0.43
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkFp-9O__UZW95KznG-pgq1Fh6DLwWxSfR55TkCsE60YXsw63H
p5
0
<
mean
0.19
SD
0.4
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkH4gO03OZydqGK8h5DSHbq2E1hdoCK2yTigpcHir4trdSQl1S
p5
0
<
mean
0.11
SD
0.32
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkHfJrWqNr4bXfcy5gktLfPfBpOFHE5FP7KrdPsKGUV4xHS6I8
p5
0
<
mean
0.09
SD
0.29
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkIBQbnwyUP_CeqNN3RhCwrq4sK6LQunL0FU9qfWko62CaP7P2
p5
0
<
mean
0.24
SD
0.43
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkINgIL2aySq7UypAzdVHWSwnor1fbQwkZUYeyhK8hJbiJ13Zb
p5
0
<
mean
0.16
SD
0.37
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkIgSaqcqzAnw2JvXuxqEw2AowhnkQcymteVI5m2CsuBShp1fA
p5
0
<
mean
0.17
SD
0.37
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkJKG_Yn2kxCVcd3A8bieXgX4JpAF66io_d7BshMsDfnZaTBZa
p5
0
<
mean
0.02
SD
0.15
≤
p95
0
  • get_agent_activity_volatile_agent: uhCAkJjYtsVos82HJgPajG3kQF_UOazuL9mIztvuGLfEBNnFGitgs
p5
0
<
mean
0.18
SD
0.38
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkJmc8d5QP5iQ7Hunfx-CVxuM75BEbU3hSZiLpaQI8JbqXEp17
p5
0
<
mean
0.2
SD
0.4
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkKhpdKGQveDuoLfVttFsz5lwGP5SrGzOu92gHtZeXVX59Le2b
p5
0
<
mean
0.19
SD
0.39
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkLZZ-Yhcr-CIIRk1Z354F1sLtDaf3ImcNh-7jKhVYjRJ79MvG
p5
0
<
mean
0.17
SD
0.38
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkLcTd0JPTnR8-LR9caN93fVqZrF849XmpH5jWqsrE2wD5mMYs
p5
0
≤
mean
0
SD
0
≤
p95
0
  • get_agent_activity_volatile_agent: uhCAkN1_V0_sAxAxhIKHfNtFNtpFjbs2TbkGILEKpBYQHOds8eFch
p5
0
<
mean
0.21
SD
0.41
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkNAuwf6CySw2EWbFKiOBpQFwWIbCAGb3cpYor7T-xrt2ta6ta
p5
0
<
mean
0.25
SD
0.43
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkNYztdbHGI10vMeLZgO9bbwdsE_gRsjqGNP9twizZtdoBLU1p
p5
0
<
mean
0.14
SD
0.34
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkNhldOkRpURrUw3B93-tBKcssUgy0lXFt-OCYjraXILZVk_cw
p5
0
<
mean
0.24
SD
0.43
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkOSQu_krn4WvWSnh0sTtPExxwY-zIVrwfvKM8gomqnvb9-W4F
p5
0
<
mean
0.19
SD
0.4
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkOuGqXKh6SS81htlUe-8HwoNjqWR4fKWAvmVZHZYl9RNFGmOo
p5
0
<
mean
0.2
SD
0.4
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkQmz2dXLOd_Y12aKAl4E3Gm_FxJSeQzGqd8ScbzRNUqaDEcdb
p5
0
<
mean
0.11
SD
0.31
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkR3ytRPNwia1j_DcIn_Hz6Hpa_MWtHumGbTzyz1BOH21G1a8L
p5
0
<
mean
0.21
SD
0.41
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkR4GO15O14_yxsTAb3c3DPV5DgbOclCoWQbPSJ91qXF1nTakW
p5
0
<
mean
0.19
SD
0.39
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkREgbAAXVWpewTZ6X4QYnp9MluI1yIULf5k2fBjLYNTwIGRrK
p5
0
<
mean
0.17
SD
0.38
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkSMD5lNU5nvIjrA8uJgBLVlH7AbQmMK3rr5qHAkRyYyXqvdnc
p5
0
<
mean
0.26
SD
0.44
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkTDFmbSbtjjlMCW6irFVV8zctWGoc--plC_HHYno4iku_MLH_
p5
0
<
mean
0.17
SD
0.38
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkTskhX4JKbeF4OUmpxhHyzBxbpEVyXgwh-E_bGTKpGu5Or3Iu
p5
0
<
mean
0.15
SD
0.35
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkUFAs8u86DY4SmMUHiVD5YygZyUstm05urhUe7UTO3SbVfxe1
p5
0
<
mean
0.15
SD
0.36
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkUYTWldfmQQaa0_B1vXoRTvk5Q2oVTBJFUL6hyw_CcxGVwO07
p5
0
<
mean
0.21
SD
0.41
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkVnFRZQ_JZmM41xMtNKIJuMeXpN-sqIGMWBhxOk-2XWPZvZr3
p5
0
<
mean
0.13
SD
0.33
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkVqFOPfvqX5tO_bvw_8QNCJF3cVpl5f9PWOzQUgkrX5j5V6Uz
p5
0
<
mean
0.17
SD
0.38
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkX3e2keFfv2QbisBLtV_2agKbB0G0yS3W0BSm3LUYAgyv3TIU
p5
0
<
mean
0.21
SD
0.41
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkXbSiAmPGdIBb5HThY3PV3M8cNNRuZapOlDBGq8r1FqsWSOjZ
p5
0
<
mean
0.16
SD
0.37
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkXrEQc52TnfFfxfFATcfFFfPCHzO7rl7tm2b_aiyYU6g7V0Ru
p5
0
<
mean
0.24
SD
0.43
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkY4cg_-kz6hIfJxycl-H2W26kZkTvXV--7b9G6x_GxFCv4cpF
p5
0
<
mean
0.23
SD
0.42
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkZ77Tre1s9-N9VvlXPuU29mDCqznWPtsGLnwd9TQGIwSMqjfH
p5
0
<
mean
0.09
SD
0.28
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk_C7QBeOzusv3d06OrppdzF0vznSA6ZnoNhPiC6ZBPhmXg6X2
p5
0
<
mean
0.22
SD
0.41
<
p95
1
  • get_agent_activity_volatile_agent: uhCAk_gTXOEObwrkOvCMVmP8Qi7sOfNM-DSZl01-qDHPJJ0cpmN9G
p5
0
<
mean
0.17
SD
0.38
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkclEEWmMEXMFG3NJ3vLukPRh22ROZ0OaYPLndCNFnilNraauH
p5
0
<
mean
0.21
SD
0.41
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkd4btQeg-P97n6YFyh-WlQ-yT2QBFj5RGazPtoBc-K7NWbOb8
p5
0
<
mean
0.21
SD
0.41
<
p95
1
  • get_agent_activity_volatile_agent: uhCAken1ydimLSkTKH3GYu2thea1E2fltoVlJVVAM1Gxt8E5D0pVH
p5
0
<
mean
0.18
SD
0.38
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkfF_266GVjS_LiIRtCQ_0Xa47t67X7V2RGP2J_8VaQbyX0nbu
p5
0
<
mean
0.25
SD
0.43
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkfO6vInmQK131pdFxJXe5pBgQiJcSijZYO-YlIgV_2ApimhOB
p5
0
<
mean
0.12
SD
0.32
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkfcaOIpNNH0w3408DkhL96N-2YVOprDJqZFv3X2ZVQ7_BK0Nc
p5
0
<
mean
0.25
SD
0.43
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkgv8LR_AfLU19yvauuU5pMF1THwn2xPx-60pTiMX2ywlsPGNd
p5
0
<
mean
0.18
SD
0.38
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkk1QnIjoIr4wEWDFAC0n5KoByj4yH5vf_pFUJYRuNuCTsshfy
p5
0
<
mean
0.21
SD
0.41
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkkZmO2YPYE2o1xD6pMNG-eJI097NY9fEHwMt-1gBVfSoZBiai
p5
0
<
mean
0.2
SD
0.4
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkkpbazZqB_zFVF8EWl77mfq7GTTNLXEDzBUurpFMhJMfQ9T6S
p5
0
<
mean
0.13
SD
0.34
<
p95
1
  • get_agent_activity_volatile_agent: uhCAklIoT52zS3MTfpqoJZD7UFBOo5xYgXgO_CQB2q4DVLhtkqROc
p5
0
<
mean
0.25
SD
0.43
<
p95
1
  • get_agent_activity_volatile_agent: uhCAklMJ7CiWXH7-cfiVCjl5qiCkA-WiGven4TjoBTrD7uyY39YPq
p5
0
<
mean
0.23
SD
0.42
<
p95
1
  • get_agent_activity_volatile_agent: uhCAklfts8u-k_cBR9Q1fxp-nqCf0gsKGm_Hp8x5UhgMIB0_3zIkA
p5
0
<
mean
0.35
SD
0.48
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkoGlrTdnZiJLsdBWt8SJhwGUKMb3iUHoyJ2F1Jv3iXFAjGdnU
p5
0
<
mean
0.13
SD
0.33
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkpIuh9UhtnlIMt_7y6RMiJ_P7V0gL85httaR6mJv54LOx5N5w
p5
0
<
mean
0.14
SD
0.34
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkr7TtvWwkT-SFIJ9kUIqU4SXtWCO3rm0BCU7mUJPJ_rODtzOE
p5
0
<
mean
0.18
SD
0.39
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkr8IHTrSw3D3iXztPZllxoOcsgxa7ROsd7ZCGJxCjPufGFfR_
p5
0
<
mean
0.23
SD
0.42
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkrdwoPbF5BKTaEDNQxjU4cnwmA4sCcNfWfCz52KyWUTlrdye6
p5
0
<
mean
0.17
SD
0.37
<
p95
1
  • get_agent_activity_volatile_agent: uhCAksaK3To8GVrPeRYN6OqW4K9v049_Xp9EIGoSfbF3vJzNYzFsx
p5
0
<
mean
0.18
SD
0.38
<
p95
1
  • get_agent_activity_volatile_agent: uhCAksz88dQH1U01z1kT5i-44VIukJI4kVCgLtv0gKFG8ZnPkgKko
p5
0
<
mean
0.14
SD
0.35
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkuzSFoO97mG4kz9WqP8mLQ_Lku2IrI8UB5ElMNFgsvvl_o6_L
p5
0
<
mean
0.12
SD
0.32
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkvH7QpRd1KNqyAFNbBGI2yweAcQ-de-tYjiHiGYOT-66BlUrx
p5
0
<
mean
0.11
SD
0.31
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkvjfJ2rVDJJHmCl9tO95X5HoKudj3mVDYzk7mBluGiA-lW2Bc
p5
0
<
mean
0.2
SD
0.4
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkw4agAx_A6QnDUkLKWcDlcQ34uSwvP11zhbgNsBHUSMrSIZ_-
p5
0
<
mean
0.19
SD
0.4
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkwEjoHGQr0LbG8RM_JJmiX0u9UWVs3NDfXpDI2nL2nh9pHbO0
p5
0
<
mean
0.22
SD
0.41
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkxDNRWMF6SSHHvXxuesoNmio_SUZgmC5n-BTmQAljgsjhwx7N
p5
0
<
mean
0.25
SD
0.43
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkxvFlHS49O90n6fOMyGRJSLTi2lq9X0SKfwblqJRLjW8N-99l
p5
0
<
mean
0.28
SD
0.45
<
p95
1
  • get_agent_activity_volatile_agent: uhCAky4VIDjUevekXqFaP9wms9EQteiIxTv70-0l4No5yvoj7HGwQ
p5
0
<
mean
0.22
SD
0.42
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkyJ_O97LnJ7x3wzTFKhNbmLSzZEzM9Anc_W1CvDBt6vTfScjo
p5
0
<
mean
0.2
SD
0.4
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkyNsfCXZj-Q1ChWHhGgXEi3yU6H6bXkoZhwAWKN_bakemMgWy
p5
0
<
mean
0.22
SD
0.41
<
p95
1
  • get_agent_activity_volatile_agent: uhCAkzo_foZ2adsRGxgqE4valLfb8Zk1hDA4sTCNOc4RZMrp2jhEd
p5
0
<
mean
0.21
SD
0.41
<
p95
1
Error count
The number of errors accumulated.
790
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.02703
s
SD
28.327885
s
p50
0.013484
s
<
p95
22.773964
s
<
p99
110.002175
s
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
14
over 1734.868914136s
mean rate
138.19
/s
std
900.64
/s
p5
0
/s
p95
282.42
/s
peak
26105.91
/s
Zome call duration: 'agent_activity::announce_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.007809
s
SD
0.003064
s
p50
0.007298
s
<
p95
0.015175
s
<
p99
0.017438
s
Zome call duration: 'agent_activity::create_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.006025
s
SD
0.001694
s
p50
0.006266
s
<
p95
0.008853
s
<
p99
0.009708
s
Zome call duration: 'agent_activity::get_agent_activity_full'
Duration of this zome call as measured by Holochain internally.
mean
0.013704
s
SD
4.155691
s
p50
0.01015
s
<
p95
2.083559
s
<
p99
22.942741
s
Zome call duration: 'agent_activity::get_random_agent_with_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.009196
s
SD
1.324973
s
p50
0.379578
s
<
p95
2.139293
s
<
p99
10.713644
s
WASM call duration: 'agent_activity::announce_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.006493
s
SD
0.002595
s
p50
0.005911
s
<
p95
0.012841
s
<
p99
0.014639
s
WASM call duration: 'agent_activity::create_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.004869
s
SD
0.001354
s
p50
0.005067
s
<
p95
0.007131
s
<
p99
0.007865
s
WASM call duration: 'agent_activity::get_agent_activity_full'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.028949
s
SD
28.983153
s
p50
0.015217
s
<
p95
26.315352
s
<
p99
241.854273
s
WASM call duration: 'agent_activity::get_random_agent_with_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.00981
s
SD
2.519885
s
p50
0.314673
s
<
p95
2.046836
s
<
p99
22.017353
s
WASM call duration: 'agent_activity_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001625
s
SD
0.000466
s
p50
0.001736
s
<
p95
0.002358
s
<
p99
0.002777
s
Host function call duration: '__hc__agent_info_1'
Duration of this host function call invoked from within WASM.
mean
7.9e-05
s
SD
0.000113
s
p50
4.5e-05
s
<
p95
0.00023
s
<
p99
0.000703
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000529
s
SD
0.000181
s
p50
0.000543
s
<
p95
0.000854
s
<
p99
0.000933
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.000642
s
SD
0.000327
s
p50
0.000588
s
<
p95
0.001137
s
<
p99
0.002081
s
Host function call duration: '__hc__get_agent_activity_1'
Duration of this host function call invoked from within WASM.
mean
0.027461
s
SD
28.983113
s
p50
0.013214
s
<
p95
26.312467
s
<
p99
241.851064
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.006388
s
SD
2.519556
s
p50
0.308126
s
<
p95
2.040354
s
<
p99
22.010795
s
Host function call duration: '__hc__random_bytes_1'
Duration of this host function call invoked from within WASM.
mean
8e-06
s
SD
7e-06
s
p50
1.4e-05
s
<
p95
2.7e-05
s
<
p99
4.4e-05
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.002535
s
SD
0.00109
s
p50
0.002778
s
<
p95
0.004898
s
<
p99
0.007747
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.000332
s
SD
0.000269
s
p50
0.000377
s
<
p95
0.00066
s
<
p99
0.001006
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
3.55
s
<
mean
349.75
s
SD
505.08
s
<
p95
1513.56
s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
151.820295
s
SD
122.364206
s
p50
66.231551
s
<
p95
333.023625
s
<
p99
610.294307
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.176408
s
SD
0.23229
s
p50
0.041001
s
<
p95
0.680669
s
<
p99
0.948754
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
5e-06
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.041638
s
SD
0.468517
s
p50
0.026243
s
<
p95
0.457277
s
<
p99
1.538782
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.6829
s
SD
1.134873
s
p50
0.142995
s
<
p95
2.638263
s
<
p99
5.465694
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
41.324524
s
SD
48.621588
s
p50
25.37894
s
<
p95
138.048118
s
<
p99
224.354395
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
35.873236
s
SD
51.612559
s
p50
0.023702
s
<
p95
147.714821
s
<
p99
263.12794
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.004879
s
SD
0.003249
s
p50
0.000954
s
<
p95
0.009897
s
<
p99
0.014432
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.00878
s
SD
0.146009
s
p50
0.003277
s
<
p95
0.039126
s
<
p99
0.320544
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000432
s
SD
0.001219
s
p50
0.000763
s
<
p95
0.003215
s
<
p99
0.006797
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
5.673006
s
SD
28.787807
s
p50
1.016214
s
<
p95
60.002658
s
<
p99
60.004544
s
P2P request roundtrip: 'get_links'
Time spent sending a get_links request and awaiting its response.
mean
0.84835
s
SD
0.765658
s
p50
0.612943
s
<
p95
2.123413
s
<
p99
4.070785
s
P2P request roundtrip: 'get_agent_activity'
Time spent sending a get_agent_activity request and awaiting its response.
mean
0.145123
s
SD
18.363545
s
p50
0.246708
s
<
p95
60.001245
s
<
p99
60.001704
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.637711
s
SD
25.592037
s
p50
1.065223
s
<
p95
60.001648
s
<
p99
60.002097
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
28.62
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
4.64
KiB/s
<
mean
231.56
KiB/s
SD
387.11
KiB/s
<
p95
817.93
KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
179
B/s
<
mean
26.63
KiB/s
SD
81.04
KiB/s
<
p95
121.65
KiB/s
Total bytes received
Total bytes received on primary network interface
count
52.98
GiB
mean
30.31
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
6.09
GiB
mean
3.49
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected

Warning CPU p99 reached 95.8%

Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
Detected

Critical Disk /etc/resolv.conf: 1/14 hosts above 90% capacity

Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected

Critical Heavy swap usage (80.6% swap used)

System overload anomaly
System overload anomaly was detected
Detected

Warning 21% of hosts overloaded (load5/ncpus > 1.0)

CPU usage
Total CPU usage and kernel CPU usage
Total
p5
2.01
%
<
mean
37.43
%
SD
30.26
%
<
p95
90.12
%
System
8.81
%
CPU percentiles
CPU usage percentiles
p50
34.27
%
p95
90.12
%
p99
95.79
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
112
hosts
mean time
275.97
s
Memory used percentage
Percentage of memory used
p5
6.44
%
<
mean
9.77
%
SD
8.33
%
<
p95
13.64
%
Memory available percentage
Percentage of available memory
p5
86.36
%
<
mean
90.23
%
SD
8.33
%
<
p95
93.56
%
Max host memory used
Maximum memory usage percentage across all hosts
max
88.92
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
80.59
%
Memory growth rate
Rate of memory growth over time
growth
0.13
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.81
MiB/s
Disk write throughput
Disk write throughput in MiB/s
86.62
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point /
0/118
hosts
Mount Point /efi-boot
0/6
hosts
Mount Point /etc/hostname
1/14
hosts
Mount Point /etc/hosts
1/14
hosts
Mount Point /etc/nomad.d/client-meta.json
0/13
hosts
Mount Point /etc/nomad.d/meta.json
1/1
hosts
Mount Point /etc/resolv.conf
1/14
hosts
Mount Point /nix/store
0/11
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
1.39
5 min
1.58
15 min
1.78
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
21.38
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
13.2111
%
SD
16.7849
%
<
p95
50.13
%
60 second average
13.0551
%
300 second average
15.6056
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
0.0003
%
SD
0.0158
%
≤
p95
0
%
60 second average
0.0003
%
300 second average
0.0001
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
0.0003
%
SD
0.0156
%
≤
p95
0
%
60 second average
0.0003
%
300 second average
0.0001
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
2.8171
%
SD
7.1731
%
<
p95
12.9
%
60 second average
2.8779
%
300 second average
3.9845
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
1.6611
%
SD
4.6568
%
<
p95
7.59
%
60 second average
1.6865
%
300 second average
2.0983
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0
%
<
mean
35.56
%
SD
31.4
%
<
p95
87.23
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
50.12
KiB
<
mean
192.61
KiB
SD
116.34
KiB
<
p95
393.73
KiB
Holochain process threads
Number of threads in Holochain process
p5
5
threads
<
mean
27.84
threads
SD
10.06
threads
<
p95
39
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
12
file descriptors
<
mean
74.18
file descriptors
SD
21.78
file descriptors
<
p95
91
file descriptors

Write Validated must_get_agent_activity

↑ Back to index

A scenario where write agents create entries in batches of 10, while must_get_agent_activity agents each pick a random write agent and repeatedly attempt to create an entry that references the chain top of their latest batch. This reference means that the entry's validation function needs to make a must_get_agent_activity call.

The purpose of this scenario is to measure the time it takes for published agent activity data to be gossiped among authorities and become available to peers that query it via must_get_agent_activity.

This test is similar to Mixed-Arc must_get_agent_activity, but all agents are full-arc.

Started
Thu, 22 Oct 2026 05:08:39 UTC
Duration
900s
Peer count
150
Peer count at end
137
Behaviours
  • must_get_agent_activity (1 agent)
  • write (1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
write_validated_must_get_agent_activity_39711995436_1
Highest observed action_seq
The maximum chain length observed per write agent, aggregated across all reading agents. The reading dimension is collapsed by taking the maximum observed value for each write agent (any successful read counts as propagation). This reflects the DHT’s ability to propagate agent activity ops and make them available to querying peers.
Mean max (across write agents)
Mean of the highest chain head value observed per write agent. Each write agent contributes its maximum — the value seen by any reader.
432.43
Max
Highest chain head value observed for any single write agent.
1182
Write agents observed
7
Chain batch delay timing
The time between a write agent's creation of a batch and a must_get_agent_activity agent's successful discovery of the batch and creation/self-validation of a new entry that depends on it.
Mean
mean
240.968152
s
SD
54.142223
s
Max
Highest per-partition mean latency
max mean
1285.867173
s
Min
Lowest per-partition mean latency
min mean
7.35
s
Chain batch delay rate
The rate at which a must_get_agent_activity agent was able to discover batches and create/self-validate new entries that depend on them.
Mean
mean
1.51
/s
Max
Highest per-partition mean rate
max
6.67
/s
Min
Lowest per-partition mean rate
min
0
/s
create_validated_sample_entry zome call timing
The time taken to complete the zome function call that creates the entry that depends on a write agent's source chain.
Mean
mean
0.288574
s
SD
0.126109
s
Max
Highest per-partition mean latency
max mean
3.354924
s
Min
Lowest per-partition mean latency
min mean
0.039811
s
Retrieval errors
A running accumulation of the errors encountered by an agent when attempting to self-validate actions that depend on must_get_agent_activity calls.
Total
34801
Agents affected
74 / 75
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
464
Max
Highest count in any single partition
795
Min
Lowest count in any single partition
0
Final error count
The total number of all types of error accumulated over the run by all agents.
96841
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.018601
s
SD
3.360606
s
p50
0.001994
s
<
p95
0.38217
s
<
p99
16.599089
s
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
7
over 807.232369704s
mean rate
2.38
/s
std
5.16
/s
p5
0
/s
p95
14.3
/s
peak
35.08
/s
Zome call duration: 'validated_must_get_agent_activity::announce_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.008183
s
SD
0.002058
s
p50
0.007805
s
<
p95
0.011247
s
<
p99
0.015985
s
Zome call duration: 'validated_must_get_agent_activity::create_sample_entries_batch'
Duration of this zome call as measured by Holochain internally.
mean
0.293273
s
SD
0.224843
s
p50
0.204777
s
<
p95
0.709478
s
<
p99
0.906795
s
Zome call duration: 'validated_must_get_agent_activity::create_validated_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.013308
s
SD
0.007255
s
p50
0.010899
s
<
p95
0.025291
s
<
p99
0.035851
s
Zome call duration: 'validated_must_get_agent_activity::get_chain_top_for_batch'
Duration of this zome call as measured by Holochain internally.
mean
0.297637
s
SD
7.380088
s
p50
0.008725
s
<
p95
0.257814
s
<
p99
49.965879
s
Zome call duration: 'validated_must_get_agent_activity::get_random_agent_with_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.063733
s
SD
1.663238
s
p50
0.008858
s
<
p95
0.702549
s
<
p99
11.574858
s
WASM call duration: 'validated_must_get_agent_activity::announce_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.006812
s
SD
0.001697
s
p50
0.006521
s
<
p95
0.010172
s
<
p99
0.012405
s
WASM call duration: 'validated_must_get_agent_activity::create_sample_entries_batch'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.290606
s
SD
0.223548
s
p50
0.202548
s
<
p95
0.704242
s
<
p99
0.901415
s
WASM call duration: 'validated_must_get_agent_activity::create_validated_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.010879
s
SD
0.006186
s
p50
0.008643
s
<
p95
0.02123
s
<
p99
0.031343
s
WASM call duration: 'validated_must_get_agent_activity::get_chain_top_for_batch'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.012113
s
SD
5.945752
s
p50
0.007495
s
<
p95
2.108578
s
<
p99
49.964118
s
WASM call duration: 'validated_must_get_agent_activity::get_random_agent_with_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.061457
s
SD
1.663218
s
p50
0.007467
s
<
p95
0.700794
s
<
p99
11.573064
s
WASM call duration: 'validated_must_get_agent_activity_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001816
s
SD
0.000951
s
p50
0.002173
s
<
p95
0.003991
s
<
p99
0.004976
s
WASM call duration: 'validated_must_get_agent_activity_integrity::validate'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.015429
s
SD
0.118938
s
p50
0.011492
s
<
p95
0.056055
s
<
p99
0.090764
s
Host function call duration: '__hc__agent_info_1'
Duration of this host function call invoked from within WASM.
mean
9.4e-05
s
SD
6.3e-05
s
p50
4.7e-05
s
<
p95
0.000195
s
<
p99
0.000311
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.005394
s
SD
0.005663
s
p50
0.001709
s
<
p95
0.015079
s
<
p99
0.029893
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.008309
s
SD
0.008083
s
p50
0.000758
s
<
p95
0.021064
s
<
p99
0.041648
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.007416
s
SD
4.358494
s
p50
0.000939
s
<
p95
0.693583
s
<
p99
21.589928
s
Host function call duration: '__hc__must_get_agent_activity_1'
Duration of this host function call invoked from within WASM.
mean
0.019251
s
SD
0.496582
s
p50
0.013583
s
<
p95
0.070504
s
<
p99
0.126295
s
Host function call duration: '__hc__query_1'
Duration of this host function call invoked from within WASM.
mean
0.116775
s
SD
0.085864
s
p50
0.083429
s
<
p95
0.250566
s
<
p99
0.302124
s
Host function call duration: '__hc__random_bytes_1'
Duration of this host function call invoked from within WASM.
mean
2.4e-05
s
SD
9e-06
s
p50
1.2e-05
s
<
p95
3.7e-05
s
<
p99
5.3e-05
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.003652
s
SD
0.002378
s
p50
0.003555
s
<
p95
0.008967
s
<
p99
0.012846
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.002266
s
SD
0.001893
s
p50
0.000334
s
<
p95
0.00559
s
<
p99
0.009024
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
43.19
s
<
mean
429.07
s
SD
248.43
s
<
p95
815.12
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
1808
over 889.265927063s
mean rate
360730.13
/s
std
2.34842323e+06
/s
p5
768.55
/s
p95
1.45053036e+06
/s
peak
9.715138169e+07
/s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
84.851067
s
SD
57.904556
s
p50
24.841006
s
<
p95
167.236926
s
<
p99
274.057175
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
1.988136
s
SD
1.224379
s
p50
2.064984
s
<
p95
4.32364
s
<
p99
7.194312
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.924473
s
SD
0.499052
s
p50
0.421948
s
<
p95
1.491238
s
<
p99
1.757759
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.662459
s
SD
0.54978
s
p50
0.120769
s
<
p95
1.163176
s
<
p99
2.504643
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
14.032328
s
SD
17.920035
s
p50
16.278412
s
<
p95
51.735056
s
<
p99
83.367009
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
10.6247
s
SD
22.243011
s
p50
0.021388
s
<
p95
48.492186
s
<
p99
112.624686
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.004842
s
SD
0.003965
s
p50
0.004425
s
<
p95
0.012815
s
<
p99
0.016946
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.00644
s
SD
0.004594
s
p50
0.00428
s
<
p95
0.01153
s
<
p99
0.022895
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.004011
s
SD
0.005476
s
p50
0.001549
s
<
p95
0.013754
s
<
p99
0.029246
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
1.629323
s
SD
28.783537
s
p50
1.06641
s
<
p95
60.005283
s
<
p99
60.022751
s
P2P request roundtrip: 'get_links'
Time spent sending a get_links request and awaiting its response.
mean
1.486576
s
SD
2.866077
s
p50
0.436383
s
<
p95
5.778556
s
<
p99
18.639559
s
P2P request roundtrip: 'must_get_agent_activity'
Time spent sending a must_get_agent_activity request and awaiting its response.
mean
3.444706
s
SD
13.798496
s
p50
0.709542
s
<
p95
60.000727
s
<
p99
60.001164
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
1.592903
s
SD
27.099977
s
p50
4.296001
s
<
p95
60.010227
s
<
p99
60.035661
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
41.56
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
116
B/s
<
mean
728.82
KiB/s
SD
542.07
KiB/s
<
p95
1.47
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
0
B/s
<
mean
79.44
KiB/s
SD
106.42
KiB/s
<
p95
324.05
KiB/s
Total bytes received
Total bytes received on primary network interface
count
78.24
GiB
mean
90.02
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
8.52
GiB
mean
9.80
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected

Warning CPU p99 reached 98.0%

Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
Detected

Critical Disk /etc/resolv.conf: 1/13 hosts above 90% capacity

Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected

Critical Heavy swap usage (83.3% swap used)

System overload anomaly
System overload anomaly was detected
Detected

Warning 77% of hosts overloaded (load5/ncpus > 1.0)

CPU usage
Total CPU usage and kernel CPU usage
Total
p5
12.48
%
<
mean
61.97
%
SD
23.82
%
<
p95
92.94
%
System
20.41
%
CPU percentiles
CPU usage percentiles
p50
65.41
%
p95
92.94
%
p99
98.00
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
112
hosts
mean time
257.24
s
Memory used percentage
Percentage of memory used
p5
7.65
%
<
mean
12.59
%
SD
8.22
%
<
p95
16.26
%
Memory available percentage
Percentage of available memory
p5
83.74
%
<
mean
87.41
%
SD
8.22
%
<
p95
92.35
%
Max host memory used
Maximum memory usage percentage across all hosts
max
87.84
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
83.33
%
Memory growth rate
Rate of memory growth over time
growth
0.35
MiB/s
Disk read throughput
Disk read throughput in MiB/s
3.54
MiB/s
Disk write throughput
Disk write throughput in MiB/s
126.48
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point /
0/111
hosts
Mount Point /efi-boot
0/6
hosts
Mount Point /etc/hostname
1/13
hosts
Mount Point /etc/hosts
1/13
hosts
Mount Point /etc/nomad.d/client-meta.json
0/13
hosts
Mount Point /etc/nomad.d/meta.json
1/1
hosts
Mount Point /etc/resolv.conf
1/13
hosts
Mount Point /nix/store
0/11
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
3.55
5 min
2.84
15 min
2.20
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
76.64
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0.12
%
<
mean
45.3196
%
SD
29.1932
%
<
p95
83.54
%
60 second average
42.3330
%
300 second average
33.0264
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
0.0035
%
SD
0.1367
%
≤
p95
0
%
60 second average
0.0035
%
300 second average
0.0029
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
0.0033
%
SD
0.129
%
≤
p95
0
%
60 second average
0.0033
%
300 second average
0.0027
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
12.7355
%
SD
9.7288
%
<
p95
27.35
%
60 second average
12.0112
%
300 second average
9.0860
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
5.5589
%
SD
5.5084
%
<
p95
15.52
%
60 second average
5.3162
%
300 second average
4.2400
%
Holochain process CPU usage
CPU usage by Holochain process
p5
13.05
%
<
mean
59.75
%
SD
22.34
%
<
p95
88.58
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
94.55
KiB
<
mean
384.17
KiB
SD
125.39
KiB
<
p95
544.97
KiB
Holochain process threads
Number of threads in Holochain process
p5
9
threads
<
mean
34.6
threads
SD
9.93
threads
<
p95
53
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
14
file descriptors
<
mean
79.68
file descriptors
SD
19.12
file descriptors
<
p95
97
file descriptors

Zero-Arc Create and Read

↑ Back to index

A zero-arc/full-arc mixed scenario with two types of zero-arc nodes -- ones that create data and ones that read data -- as well as full arc nodes to "relay" the data. The scenario has three roles:

  • zero_write: A zero-arc conductor that just creates entries with a timestamp field. Those entries are linked to a known base hash so that zero_read nodes can retrieve them.
  • zero_read: A zero-arc conductor that reads the entries created by the zero-arc node(s) and records the time lag between when the entry had been created and when it was first discovered.
  • full: A full-arc conductor that is just here to serve entries to zero arc nodes.
Started
Thu, 22 Oct 2026 06:34:36 UTC
Duration
900s
Peer count
150
Peer count at end
135
Behaviours
  • full (1 agent)
  • zero_read (1 agent)
  • zero_write (1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
zero_arc_create_and_read_39711995436_1
Create rate
The number of timed entries created by the zero-arc write node(s) per second.
Mean
mean
46.88
/s
Max
Highest per-partition mean rate
max
84.95
/s
Min
Lowest per-partition mean rate
min
31.62
/s
Fetch lag timing
For each entry, the time lag between when it was created and when a zero-arc read node fetched it from the network via the get_timed_entries_network zome function.
Mean
mean
247.772451
s
SD
66.250396
s
Max
Highest per-partition mean latency
max mean
1336.769436
s
Min
Lowest per-partition mean latency
min mean
18.049053
s
Fetch lag rate
The number of entries per second fetched by zero-arc read nodes.
Mean
mean
22.28
/s
Max
Highest per-partition mean rate
max
334.5
/s
Min
Lowest per-partition mean rate
min
0
/s
Delivery ratio
The proportion of created entries successfully fetched by all zero-arc read nodes.
0.0005
Entry created count
Total number of entries created by zero-arc write nodes, by agent.
Total
232625
Agents affected
55 / 55
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
4230
Max
Highest count in any single partition
7543
Min
Lowest count in any single partition
2779
Received count
Total number of entries fetched by zero-arc read nodes, by agent.
Total
5973
Agents affected
42 / 49
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
122
Max
Highest count in any single partition
762
Min
Lowest count in any single partition
0
Open connections
The number of currently open connections to other conductors.
full-arc
p5
11
<
mean
54.59
SD
39.65
<
p95
132
zero-arc
p5
1
<
mean
16.84
SD
12.19
<
p95
43
Error count
The number of errors accumulated across all nodes.
630
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.032487
s
SD
2.879557
s
p50
0.957581
s
<
p95
5.233575
s
<
p99
13.05287
s
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
0
over 830.009389374s
Not enough time data to show a trend.
WASM usage: 'timed::created_timed_entry'
Metered usage count of the WASM ribosome for this zome function.
total
243097920
over 879.733359859s
mean rate
1.00020265189e+09
/s
std
2.21917309221e+09
/s
p5
4.15857808e+06
/s
p95
4.6141300662e+09
/s
peak
3.46911778529e+10
/s
WASM usage: 'timed::get_timed_entries_network'
Metered usage count of the WASM ribosome for this zome function.
total
53475359
over 884.056885449s
mean rate
1.78639629675e+11
/s
std
7.2612132233e+11
/s
p5
2.49642908e+06
/s
p95
1.46121626348e+12
/s
peak
4.94126174041e+12
/s
WASM usage: 'timed_integrity::entry_defs'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 889.882506531s
Not enough time data to show a trend.
Zome call duration: 'timed::created_timed_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.008523
s
SD
0.002943
s
p50
0.008821
s
<
p95
0.013754
s
<
p99
0.015838
s
Zome call duration: 'timed::get_timed_entries_network'
Duration of this zome call as measured by Holochain internally.
mean
3.204208
s
SD
50.155571
s
p50
22.014255
s
<
p95
139.918769
s
<
p99
245.40042
s
WASM call duration: 'timed::created_timed_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.007444
s
SD
0.002565
s
p50
0.00775
s
<
p95
0.012048
s
<
p99
0.014045
s
WASM call duration: 'timed::get_timed_entries_network'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
3.218523
s
SD
50.011306
s
p50
22.014498
s
<
p95
139.916876
s
<
p99
245.396607
s
WASM call duration: 'timed_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001432
s
SD
0.000467
s
p50
0.001756
s
<
p95
0.002386
s
<
p99
0.002879
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000587
s
SD
0.000368
s
p50
0.000581
s
<
p95
0.001477
s
<
p99
0.002024
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.00058
s
SD
0.00034
s
p50
0.000568
s
<
p95
0.001403
s
<
p99
0.001797
s
Host function call duration: '__hc__get_1'
Duration of this host function call invoked from within WASM.
mean
0.031285
s
SD
2.273029
s
p50
0.883404
s
<
p95
4.716057
s
<
p99
6.651586
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.392513
s
SD
4.345363
s
p50
2.35801
s
<
p95
11.372491
s
<
p99
20.667438
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.002235
s
SD
0.000757
s
p50
0.002786
s
<
p95
0.003745
s
<
p99
0.004334
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.000456
s
SD
0.000926
s
p50
0.000317
s
<
p95
0.001788
s
<
p99
0.005341
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
43.23
s
<
mean
429.52
s
SD
248.94
s
<
p95
818.18
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
28
over 889.882518925s
mean rate
11024.86
/s
std
59621.52
/s
p5
0
/s
p95
32192.34
/s
peak
659151.19
/s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
29.040916
s
SD
28.600341
s
p50
0
s
<
p95
17.366327
s
<
p99
73.435403
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.115786
s
SD
0.263712
s
p50
0.00228
s
<
p95
0.241987
s
<
p99
1.054501
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.016124
s
SD
0.276225
s
p50
0.00222
s
<
p95
0.81878
s
<
p99
1.322392
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.28983
s
SD
0.162155
s
p50
0.027989
s
<
p95
0.451865
s
<
p99
0.600726
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
9.545099
s
SD
12.921795
s
p50
0.003919
s
<
p95
16.825944
s
<
p99
50.63604
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
1.686047
s
SD
6.298264
s
p50
0.001525
s
<
p95
0.476954
s
<
p99
40.396253
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.002153
s
SD
0.002128
s
p50
0.001344
s
<
p95
0.006374
s
<
p99
0.008367
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.005322
s
SD
0.005035
s
p50
0.003789
s
<
p95
0.011253
s
<
p99
0.032205
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.0007
s
SD
0.001782
s
p50
0.00062
s
<
p95
0.001758
s
<
p99
0.008023
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
0.547014
s
SD
28.819272
s
p50
1.415808
s
<
p95
60.002246
s
<
p99
60.012456
s
P2P request roundtrip: 'get_links'
Time spent sending a get_links request and awaiting its response.
mean
0.193555
s
SD
16.716691
s
p50
0.496564
s
<
p95
60.001201
s
<
p99
60.001601
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
1.457133
s
SD
17.867499
s
p50
1.191676
s
<
p95
60.000575
s
<
p99
60.001095
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
32.57
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
4.03
KiB/s
<
mean
594.73
KiB/s
SD
623.93
KiB/s
<
p95
1.76
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
0
B/s
<
mean
88.10
KiB/s
SD
273.70
KiB/s
<
p95
349.45
KiB/s
Total bytes received
Total bytes received on primary network interface
count
67.58
GiB
mean
77.75
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
10.01
GiB
mean
11.52
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected

Warning CPU p99 reached 95.4%

Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
Detected

Critical Disk /etc/resolv.conf: 1/13 hosts above 90% capacity

Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected

Critical Heavy swap usage (81.7% swap used)

System overload anomaly
System overload anomaly was detected
Detected

Warning 30% of hosts overloaded (load5/ncpus > 1.0)

CPU usage
Total CPU usage and kernel CPU usage
Total
p5
2.25
%
<
mean
44.19
%
SD
35.69
%
<
p95
89.94
%
System
11.62
%
CPU percentiles
CPU usage percentiles
p50
52.56
%
p95
89.94
%
p99
95.40
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
95
hosts
mean time
374.68
s
Memory used percentage
Percentage of memory used
p5
7.07
%
<
mean
11.2
%
SD
7.91
%
<
p95
13.98
%
Memory available percentage
Percentage of available memory
p5
86.02
%
<
mean
88.8
%
SD
7.91
%
<
p95
92.93
%
Max host memory used
Maximum memory usage percentage across all hosts
max
86.07
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
81.71
%
Memory growth rate
Rate of memory growth over time
growth
0.44
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.96
MiB/s
Disk write throughput
Disk write throughput in MiB/s
120.08
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point /
0/119
hosts
Mount Point /efi-boot
0/6
hosts
Mount Point /etc/hostname
1/13
hosts
Mount Point /etc/hosts
1/13
hosts
Mount Point /etc/nomad.d/client-meta.json
0/13
hosts
Mount Point /etc/nomad.d/meta.json
1/1
hosts
Mount Point /etc/resolv.conf
1/13
hosts
Mount Point /nix/store
0/11
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
1.62
5 min
1.69
15 min
1.90
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
30.34
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
16.9221
%
SD
19.7952
%
<
p95
54.94
%
60 second average
16.3589
%
300 second average
17.2246
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
0.0006
%
SD
0.0183
%
≤
p95
0
%
60 second average
0.0005
%
300 second average
0.0003
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
0.0006
%
SD
0.0183
%
≤
p95
0
%
60 second average
0.0005
%
300 second average
0.0003
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
3.4363
%
SD
5.8803
%
<
p95
14.71
%
60 second average
3.3255
%
300 second average
3.4158
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
1.9278
%
SD
3.9488
%
<
p95
8.45
%
60 second average
1.8724
%
300 second average
2.0060
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.8
%
<
mean
40.36
%
SD
33.77
%
<
p95
85.94
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
201.45
KiB
<
mean
270.47
KiB
SD
72.87
KiB
<
p95
401.24
KiB
Holochain process threads
Number of threads in Holochain process
p5
21
threads
<
mean
30.23
threads
SD
6.97
threads
<
p95
38
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
58
file descriptors
<
mean
73.66
file descriptors
SD
14.99
file descriptors
<
p95
89
file descriptors

Zero-Arc Create Data

↑ Back to index
A zero-arc/full-arc mixed scenario where zero-arc nodes create data and full-arc nodes read the data. The scenario has two roles:
  • zero: A zero-arc conductor that just creates entries with a timestamp field. Those entries are linked to a known base hash so that full-arc nodes can retrieve them.
  • full: A full-arc conductor that reads the entries created by the zero-arc node(s) and records the time lag between when the entry had been created and when it was first discovered.
Started
Thu, 22 Oct 2026 05:43:10 UTC
Duration
900s
Peer count
150
Peer count at end
135
Behaviours
  • full (1 agent)
  • zero (1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
zero_arc_create_data--high_zero_39711995436_1
Create rate
The number of timed entries created by the zero-arc node(s) per second.
Mean
mean
46.32
/s
Max
Highest per-partition mean rate
max
81.69
/s
Min
Lowest per-partition mean rate
min
30.84
/s
Sync lag timing
For each entry, the time lag between when it was created and when the full-arc node could read it via the get_timed_local_entries zome function.
Mean
mean
3.27047749809e+30
s
SD
1.21773423241e+31
s
Max
Highest per-partition mean latency
max mean
2.28933424866e+31
s
Min
Lowest per-partition mean latency
min mean
13.522863
s
Sync lag rate
The number of entries per second received by full nodes.
Mean
mean
35.95
/s
Max
Highest per-partition mean rate
max
191.47
/s
Min
Lowest per-partition mean rate
min
0
/s
Delivery ratio
The proportion of created entries successfully received by all full-arc nodes.
0.0011
Entry created count
Total number of entries created by zero-arc nodes, by agent.
Total
530891
Agents affected
128 / 128
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
4148
Max
Highest count in any single partition
7315
Min
Lowest count in any single partition
2769
Received count
Total number of entries received by full-arc nodes, by agent.
Total
4557
Agents affected
7 / 8
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
570
Max
Highest count in any single partition
3820
Min
Lowest count in any single partition
0
Open connections
The number of currently open connections to other conductors.
full-arc
p5
1
<
mean
15.86
SD
19.4
<
p95
50
zero-arc
p5
0
<
mean
11.02
SD
10.92
<
p95
37
Error count
The number of errors accumulated across all nodes.
0
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.00045
s
SD
0.00083
s
p50
0.000417
s
<
p95
0.003045
s
<
p99
0.003862
s
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
2
over 700.502957631s
mean rate
0.2
/s
std
0.6
/s
p5
0
/s
p95
0.49
/s
peak
4.04
/s
WASM usage: 'timed::created_timed_entry'
Metered usage count of the WASM ribosome for this zome function.
total
416266230
over 884.96444557s
mean rate
6.26454230667e+09
/s
std
1.50651741053e+11
/s
p5
8.05482103e+06
/s
p95
1.3105000932e+10
/s
peak
7.04827476038e+12
/s
WASM usage: 'timed::get_timed_entries_local'
Metered usage count of the WASM ribosome for this zome function.
total
58590055571
over 889.306782522s
mean rate
2.06098987076e+10
/s
std
2.41303099363e+10
/s
p5
1.9744027787e+08
/s
p95
6.88295189092e+10
/s
peak
8.03416055663e+10
/s
WASM usage: 'timed_integrity::entry_defs'
Metered usage count of the WASM ribosome for this zome function.
total
66489452
over 890.378335878s
mean rate
1.12319614794e+09
/s
std
2.46984380724e+10
/s
p5
797175.03
/s
p95
2.33091205039e+09
/s
peak
1.11005171261e+12
/s
Zome call duration: 'timed::created_timed_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.008479
s
SD
0.002794
s
p50
0.010046
s
<
p95
0.014285
s
<
p99
0.014544
s
Zome call duration: 'timed::get_timed_entries_local'
Duration of this zome call as measured by Holochain internally.
mean
0.033416
s
SD
0.066983
s
p50
0.033188
s
<
p95
0.228979
s
<
p99
0.332012
s
WASM call duration: 'timed::created_timed_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.007398
s
SD
0.002402
s
p50
0.008785
s
<
p95
0.012411
s
<
p99
0.012654
s
WASM call duration: 'timed::get_timed_entries_local'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.0326
s
SD
0.066943
s
p50
0.032231
s
<
p95
0.228234
s
<
p99
0.331228
s
WASM call duration: 'timed_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001479
s
SD
0.000454
s
p50
0.001743
s
<
p95
0.00238
s
<
p99
0.00244
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000478
s
SD
0.000168
s
p50
0.000569
s
<
p95
0.00081
s
<
p99
0.000837
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.000466
s
SD
0.000176
s
p50
0.00056
s
<
p95
0.000837
s
<
p99
0.000865
s
Host function call duration: '__hc__get_1'
Duration of this host function call invoked from within WASM.
mean
0.000455
s
SD
0.000975
s
p50
0.000413
s
<
p95
0.003499
s
<
p99
0.00411
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.001547
s
SD
0.002289
s
p50
0.001551
s
<
p95
0.008071
s
<
p99
0.011353
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.002279
s
SD
0.000764
s
p50
0.002712
s
<
p95
0.00387
s
<
p99
0.003955
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.000421
s
SD
0.000286
s
p50
0.000499
s
<
p95
0.000947
s
<
p99
0.001236
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
43.41
s
<
mean
431.2
s
SD
249.91
s
<
p95
823.4
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 890.378394523s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
1.395219
s
SD
9.217718
s
p50
0
s
<
p95
0.446724
s
<
p99
53.334581
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.048287
s
SD
0.097254
s
p50
0.001895
s
<
p95
0.014378
s
<
p99
0.360635
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.004707
s
SD
0.171166
s
p50
0.002156
s
<
p95
0.180217
s
<
p99
1.077686
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.153792
s
SD
0.068216
s
p50
0.135762
s
<
p95
0.242585
s
<
p99
0.274984
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
7.195224
s
SD
8.283968
s
p50
0.003341
s
<
p95
3.58579
s
<
p99
32.688294
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
0.992883
s
SD
1.913238
s
p50
0.001347
s
<
p95
0.012948
s
<
p99
4.458437
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.003909
s
SD
0.005477
s
p50
0.009077
s
<
p95
0.01713
s
<
p99
0.018872
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.00786
s
SD
0.00585
s
p50
0.004251
s
<
p95
0.015655
s
<
p99
0.032619
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000385
s
SD
0.001376
s
p50
0.000452
s
<
p95
0.000672
s
<
p99
0.002804
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
5.665339
s
SD
27.956679
s
p50
9.788276
s
<
p95
60.001604
s
<
p99
60.052828
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.838861
s
SD
26.448616
s
p50
2.041351
s
<
p95
60.00103
s
<
p99
60.001114
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
53.96
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
6.07
KiB/s
<
mean
196.27
KiB/s
SD
354.00
KiB/s
<
p95
578.15
KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
263
B/s
<
mean
37.66
KiB/s
SD
72.09
KiB/s
<
p95
178.45
KiB/s
Total bytes received
Total bytes received on primary network interface
count
22.55
GiB
mean
25.66
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
4.33
GiB
mean
4.92
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected

Warning CPU p99 reached 94.5%

Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
Detected

Critical Disk /etc/resolv.conf: 1/13 hosts above 90% capacity

Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected

Critical Heavy swap usage (81.1% swap used)

System overload anomaly
System overload anomaly was detected
Detected

Warning 59% of hosts overloaded (load5/ncpus > 1.0)

CPU usage
Total CPU usage and kernel CPU usage
Total
p5
21.49
%
<
mean
68.91
%
SD
20.26
%
<
p95
88.12
%
System
14.94
%
CPU percentiles
CPU usage percentiles
p50
75.30
%
p95
88.12
%
p99
94.50
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
110
hosts
mean time
328.19
s
Memory used percentage
Percentage of memory used
p5
7.17
%
<
mean
10.94
%
SD
8
%
<
p95
13.97
%
Memory available percentage
Percentage of available memory
p5
86.03
%
<
mean
89.06
%
SD
8
%
<
p95
92.83
%
Max host memory used
Maximum memory usage percentage across all hosts
max
87.13
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
81.08
%
Memory growth rate
Rate of memory growth over time
growth
0.37
MiB/s
Disk read throughput
Disk read throughput in MiB/s
1.10
MiB/s
Disk write throughput
Disk write throughput in MiB/s
141.41
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point /
0/119
hosts
Mount Point /efi-boot
0/6
hosts
Mount Point /etc/hostname
1/13
hosts
Mount Point /etc/hosts
1/13
hosts
Mount Point /etc/nomad.d/client-meta.json
0/13
hosts
Mount Point /etc/nomad.d/meta.json
1/1
hosts
Mount Point /etc/resolv.conf
1/13
hosts
Mount Point /nix/store
0/11
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
2.29
5 min
2.16
15 min
2.29
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
59.31
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0.04
%
<
mean
23.7064
%
SD
11.475
%
<
p95
34.84
%
60 second average
22.7184
%
300 second average
21.6951
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
≤
mean
0
%
SD
0
%
≤
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
5.5139
%
SD
5.132
%
<
p95
12.46
%
60 second average
5.2930
%
300 second average
5.0017
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
3.9999
%
SD
4.3596
%
<
p95
9.24
%
60 second average
3.8257
%
300 second average
3.4693
%
Holochain process CPU usage
CPU usage by Holochain process
p5
25.58
%
<
mean
67.05
%
SD
18.56
%
<
p95
84.97
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
202.08
KiB
<
mean
272.38
KiB
SD
69.83
KiB
<
p95
386.14
KiB
Holochain process threads
Number of threads in Holochain process
p5
26
threads
<
mean
32.06
threads
SD
7.1
threads
<
p95
38
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
67
file descriptors
<
mean
78.16
file descriptors
SD
13.05
file descriptors
<
p95
85
file descriptors

Zero-arc Create Data (Validated)

↑ Back to index

A zero-arc/full-arc mixed scenario where zero-arc nodes create data that gets validated and full-arc nodes read the data. The scenario has two roles:

  • zero: A zero-arc conductor that just creates entries with a timestamp field. Those entries are linked to a known base hash so that full-arc nodes can retrieve them.
  • full: A full-arc conductor that reads the entries created by the zero-arc node(s) and records the time lag between when the entry had been created and when it was first discovered.
Started
Thu, 22 Oct 2026 06:17:33 UTC
Duration
900s
Peer count
150
Peer count at end
135
Behaviours
  • full (1 agent)
  • zero (1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
zero_arc_create_data_validated--high_zero_39711995436_1
Create rate
The number of timed entries created by the zero-arc node(s) per second.
Mean
mean
41.99
/s
Max
Highest per-partition mean rate
max
88.1
/s
Min
Lowest per-partition mean rate
min
27.05
/s
Sync lag timing
For each entry, the time lag between when it was created and when the full-arc node could read it via the get_timed_local_entries zome function.
Mean
mean
202.256411
s
SD
11.678488
s
Max
Highest per-partition mean latency
max mean
670.520979
s
Min
Lowest per-partition mean latency
min mean
2.835546
s
Sync lag rate
The number of entries per second received by full nodes.
Mean
mean
0
/s
Max
Highest per-partition mean rate
max
0
/s
Min
Lowest per-partition mean rate
min
0
/s
Delivery ratio
The proportion of created entries successfully received by all full-arc nodes.
0.0001
Entry created count
Total number of entries created by zero-arc nodes, by agent.
Total
457750
Agents affected
126 / 126
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
3633
Max
Highest count in any single partition
7495
Min
Lowest count in any single partition
2325
Received count
Total number of entries received by full-arc nodes, by agent.
Total
525
Agents affected
6 / 9
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
58
Max
Highest count in any single partition
284
Min
Lowest count in any single partition
0
Open connections
The number of currently open connections to other conductors.
full-arc
p5
8
<
mean
62.21
SD
33.16
<
p95
127
zero-arc
p5
2
<
mean
15.33
SD
11.34
<
p95
40
Error count
The number of errors accumulated across all nodes.
0
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.009458
s
SD
0.005326
s
p50
0.009284
s
<
p95
0.018957
s
<
p99
0.021621
s
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
6627
over 805.708112199s
mean rate
4798.64
/s
std
4102.53
/s
p5
155.39
/s
p95
14043.06
/s
peak
15884
/s
WASM usage: 'timed_and_validated::create_timed_entry'
Metered usage count of the WASM ribosome for this zome function.
total
284286240
over 845.881827693s
mean rate
2.82960419057e+09
/s
std
9.98503766183e+09
/s
p5
2.518761742e+07
/s
p95
1.02097564366e+10
/s
peak
2.56139833859e+11
/s
WASM usage: 'timed_and_validated::get_timed_entries_local'
Metered usage count of the WASM ribosome for this zome function.
total
588386142
over 845.708443739s
mean rate
8.077550852e+07
/s
std
9.300087185e+07
/s
p5
1.88096593e+06
/s
p95
1.9105044362e+08
/s
peak
6.7241074771e+08
/s
WASM usage: 'timed_and_validated_integrity::entry_defs'
Metered usage count of the WASM ribosome for this zome function.
total
167429106
over 890.15098874s
mean rate
2.37014394744e+09
/s
std
1.65287716569e+10
/s
p5
0
/s
p95
7.14765071865e+09
/s
peak
9.62139912984e+11
/s
WASM usage: 'timed_and_validated_integrity::validate'
Metered usage count of the WASM ribosome for this zome function.
total
1310027603
over 890.15098874s
mean rate
2.38160830735e+10
/s
std
1.6737146546e+11
/s
p5
9.270515337e+07
/s
p95
6.16959525245e+10
/s
peak
5.53348394207e+12
/s
Zome call duration: 'timed_and_validated::create_timed_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.00897
s
SD
0.003431
s
p50
0.011167
s
<
p95
0.01576
s
<
p99
0.01634
s
Zome call duration: 'timed_and_validated::get_timed_entries_local'
Duration of this zome call as measured by Holochain internally.
mean
0.022139
s
SD
0.019494
s
p50
0.020648
s
<
p95
0.05101
s
<
p99
0.130857
s
WASM call duration: 'timed_and_validated::create_timed_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.007825
s
SD
0.002963
s
p50
0.00972
s
<
p95
0.013752
s
<
p99
0.014227
s
WASM call duration: 'timed_and_validated::get_timed_entries_local'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.0199
s
SD
0.018668
s
p50
0.018517
s
<
p95
0.047086
s
<
p99
0.125763
s
WASM call duration: 'timed_and_validated_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001579
s
SD
0.000832
s
p50
0.001838
s
<
p95
0.003178
s
<
p99
0.005184
s
WASM call duration: 'timed_and_validated_integrity::validate'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.007183
s
SD
0.004508
s
p50
0.006745
s
<
p95
0.011759
s
<
p99
0.029547
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000488
s
SD
0.0002
s
p50
0.000607
s
<
p95
0.000865
s
<
p99
0.000911
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.000485
s
SD
0.000212
s
p50
0.000621
s
<
p95
0.000911
s
<
p99
0.00095
s
Host function call duration: '__hc__get_1'
Duration of this host function call invoked from within WASM.
mean
0.004896
s
SD
0.001995
s
p50
0.003244
s
<
p95
0.007836
s
<
p99
0.009931
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.001427
s
SD
0.000965
s
p50
0.001418
s
<
p95
0.003419
s
<
p99
0.00568
s
Host function call duration: '__hc__must_get_valid_record_1'
Duration of this host function call invoked from within WASM.
mean
0.010021
s
SD
0.005107
s
p50
0.009794
s
<
p95
0.019152
s
<
p99
0.021798
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.002778
s
SD
0.002735
s
p50
0.003063
s
<
p95
0.008076
s
<
p99
0.017677
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.000523
s
SD
0.000738
s
p50
0.000647
s
<
p95
0.001266
s
<
p99
0.005218
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
43.05
s
<
mean
431.01
s
SD
249.87
s
<
p95
823.09
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 890.151082268s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
1.16336
s
SD
32.193522
s
p50
0
s
<
p95
1.22634
s
<
p99
3.560825
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
5.081925
s
SD
1.690897
s
p50
0.001975
s
<
p95
2.51892
s
<
p99
9.400454
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
≤
p95
0
s
≤
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.012444
s
SD
0.265582
s
p50
0.002408
s
<
p95
0.901153
s
<
p99
1.236138
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.138223
s
SD
0.066484
s
p50
0.119673
s
<
p95
0.22915
s
<
p99
0.257554
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
4.804521
s
SD
4.075459
s
p50
0.003476
s
<
p95
2.813902
s
<
p99
12.546203
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
0.47994
s
SD
1.223413
s
p50
0.001238
s
<
p95
0.018239
s
<
p99
3.194748
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.006565
s
SD
0.003501
s
p50
0.006446
s
<
p95
0.01223
s
<
p99
0.014001
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.008804
s
SD
0.005199
s
p50
0.004278
s
<
p95
0.014263
s
<
p99
0.030233
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000389
s
SD
0.004108
s
p50
0.000503
s
<
p95
0.001302
s
<
p99
0.021969
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
15.474936
s
SD
29.409006
s
p50
16.098917
s
<
p95
64.235734
s
<
p99
76.717401
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
6.212068
s
SD
18.186576
s
p50
0.679098
s
<
p95
60.116322
s
≤
p99
60.116322
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
46.73
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
4.17
KiB/s
<
mean
325.43
KiB/s
SD
366.35
KiB/s
<
p95
894.48
KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
311
B/s
<
mean
38.99
KiB/s
SD
45.55
KiB/s
<
p95
136.85
KiB/s
Total bytes received
Total bytes received on primary network interface
count
37.06
GiB
mean
42.64
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
4.44
GiB
mean
5.11
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected

Warning CPU p99 reached 92.0%

Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected

Critical Heavy swap usage (83.0% swap used)

System overload anomaly
System overload anomaly was detected
Detected

Warning 54% of hosts overloaded (load5/ncpus > 1.0)

CPU usage
Total CPU usage and kernel CPU usage
Total
p5
15.77
%
<
mean
63.63
%
SD
20.75
%
<
p95
87.36
%
System
16.90
%
CPU percentiles
CPU usage percentiles
p50
70.22
%
p95
87.36
%
p99
91.95
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
112
hosts
mean time
167.71
s
Memory used percentage
Percentage of memory used
p5
8.08
%
<
mean
12.32
%
SD
7.02
%
<
p95
16.11
%
Memory available percentage
Percentage of available memory
p5
83.89
%
<
mean
87.68
%
SD
7.02
%
<
p95
91.92
%
Max host memory used
Maximum memory usage percentage across all hosts
max
89.65
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
82.95
%
Memory growth rate
Rate of memory growth over time
growth
0.49
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.68
MiB/s
Disk write throughput
Disk write throughput in MiB/s
116.89
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point /
0/119
hosts
Mount Point /efi-boot
0/6
hosts
Mount Point /etc/hostname
0/12
hosts
Mount Point /etc/hosts
0/12
hosts
Mount Point /etc/nomad.d/client-meta.json
0/13
hosts
Mount Point /etc/resolv.conf
0/12
hosts
Mount Point /nix/store
0/11
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
2.20
5 min
2.12
15 min
2.14
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
54.17
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
24.2049
%
SD
15.9953
%
<
p95
55.2
%
60 second average
23.5415
%
300 second average
23.0010
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
0.0006
%
SD
0.0328
%
≤
p95
0
%
60 second average
0.0009
%
300 second average
0.0009
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
<
mean
0.0005
%
SD
0.0315
%
≤
p95
0
%
60 second average
0.0009
%
300 second average
0.0009
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
4.4912
%
SD
4.8148
%
<
p95
14.22
%
60 second average
4.3742
%
300 second average
4.2415
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
<
mean
2.8558
%
SD
3.8578
%
<
p95
9.59
%
60 second average
2.7691
%
300 second average
2.5852
%
Holochain process CPU usage
CPU usage by Holochain process
p5
19.98
%
<
mean
61.75
%
SD
18.46
%
<
p95
81.38
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
160.27
KiB
<
mean
378.74
KiB
SD
102.54
KiB
<
p95
540.70
KiB
Holochain process threads
Number of threads in Holochain process
p5
24
threads
<
mean
33.25
threads
SD
5.92
threads
<
p95
38
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
64
file descriptors
<
mean
78.19
file descriptors
SD
12.76
file descriptors
<
p95
89
file descriptors