App Install
↑ Back to indexMeasure app installation speed. This scenario has two roles:
minimal: Installs a simple app which implements initialisation callbacks but otherwise doesn't contain a lot of code.large: Installs a larger app that contains some dependencies and some generated data as padding to make the bundle larger.
These are intended to catch a compilation slowdown or issues with copying around large WASMs (e.g. accidental cloning or tracing). In each case, the behaviour will uninstall the app it installed so that it can re-install it on the next iteration.
Started
Thu, 22 Oct 2026 09:57:08 UTC
Duration
300s
Peer count
1
Behaviours
-
large(1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
app_install_large_39722287437_1
First install time
The time from conductor startup to completion of the first app installation attempt.
0.112176092
s
Install app timing
The time taken to install an app.
mean
0.058527
s
SD
0.011235
s
p50
0.056929
s
p95
0.085924
s
p99
0.106398
s
Holochain MetricsClick to toggle visibility.
WASM usage: 'large::init'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 0s
Not enough time data to show a trend.
WASM call duration: 'large::init'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.003368
s
SD
0.000851
s
p50
0.003033
s
p95
0.00418
s
p99
0.006882
s
Not enough time data to show a trend.
Post-commit duration
Time spent executing post-commit workflows.
mean
8.9e-05
s
SD
9.2e-05
s
p50
6.1e-05
s
p95
0.000157
s
p99
0.000485
s
Not enough time data to show a trend.
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
6.69
s
mean
11.69
s
SD
5
s
p95
16.69
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 9.99998991s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Not enough time data to show a trend.
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Not enough time data to show a trend.
App validation workflow duration
Time spent running the app validation workflow.
mean
0.000935
s
SD
5.7e-05
s
p50
0.001043
s
p95
0.001043
s
p99
0.001043
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.001279
s
SD
5.3e-05
s
p50
0.001281
s
p95
0.001281
s
p99
0.001281
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.004382
s
SD
0.001201
s
p50
0.006729
s
p95
0.006729
s
p99
0.006729
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.002586
s
SD
0.000454
s
p50
0.002628
s
p95
0.002628
s
p99
0.002628
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
0.00175
s
SD
0.000413
s
p50
0.001788
s
p95
0.001788
s
p99
0.001788
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
3.9e-05
s
SD
8e-06
s
p50
5.4e-05
s
p95
5.4e-05
s
p99
5.4e-05
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.001033
s
SD
0.000263
s
p50
0.001167
s
p95
0.001393
s
p99
0.001393
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000279
s
SD
2e-06
s
p50
0.000279
s
p95
0.000279
s
p99
0.000279
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
47.59
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
54.15
KiB/s
mean
3.12
MiB/s
SD
2.19
MiB/s
p95
5.02
MiB/s
Not enough time data to show a trend.
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
30.39
KiB/s
mean
124.00
KiB/s
SD
67.80
KiB/s
p95
188.77
KiB/s
Not enough time data to show a trend.
Total bytes received
Total bytes received on primary network interface
count
93.71
MiB
mean
3.12
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
3.63
MiB
mean
124.00
KiB/s
CPU spike anomaly
CPU spike anomaly was detected
NotDetected
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 7.85 MiB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
Detected
Warning 100% of hosts overloaded (load5/ncpus > 1.0)
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
48.73
%
mean
60.24
%
SD
8.61
%
p95
69.41
%
Not enough time data to show a trend.
System
12.65
%
CPU percentiles
CPU usage percentiles
p50
62.59
%
p95
69.41
%
p99
69.41
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
0
hosts
mean time
0.00
s
Memory used percentage
Percentage of memory used
p5
21.08
%
mean
28.71
%
SD
9.13
%
p95
43.44
%
Not enough time data to show a trend.
Memory available percentage
Percentage of available memory
p5
56.56
%
mean
71.29
%
SD
9.13
%
p95
78.92
%
Not enough time data to show a trend.
Max host memory used
Maximum memory usage percentage across all hosts
max
28.71
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00
%
Memory growth rate
Rate of memory growth over time
growth
7.85
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.01
MiB/s
Disk write throughput
Disk write throughput in MiB/s
15.82
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/etc/nomad.d/client-meta.json0/1
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
1.87
5 min
2.06
15 min
1.27
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
100.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
11.68
%
mean
17.9975
%
SD
4.6212
%
p95
24.73
%
Not enough time data to show a trend.
60 second average
22.2125
%
300 second average
23.6100
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
Not enough time data to show a trend.
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
Not enough time data to show a trend.
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0.05
%
mean
1.04
%
SD
0.8764
%
p95
2.18
%
Not enough time data to show a trend.
60 second average
0.5825
%
300 second average
0.5175
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
0.5375
%
SD
0.4548
%
p95
1.18
%
Not enough time data to show a trend.
60 second average
0.2475
%
300 second average
0.2275
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0
%
mean
32.61
%
SD
23.45
%
p95
54.13
%
Not enough time data to show a trend.
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
15.52
KiB
mean
644.52
KiB
SD
748.79
KiB
p95
1.96
MiB
Not enough time data to show a trend.
Holochain process threads
Number of threads in Holochain process
p5
1
threads
mean
99
threads
SD
124.93
threads
p95
327
threads
Not enough time data to show a trend.
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
0
file descriptors
mean
220.4
file descriptors
SD
272.56
file descriptors
p95
714
file descriptors
Not enough time data to show a trend.
App Install
↑ Back to indexMeasure app installation speed. This scenario has two roles:
minimal: Installs a simple app which implements initialisation callbacks but otherwise doesn't contain a lot of code.large: Installs a larger app that contains some dependencies and some generated data as padding to make the bundle larger.
These are intended to catch a compilation slowdown or issues with copying around large WASMs (e.g. accidental cloning or tracing). In each case, the behaviour will uninstall the app it installed so that it can re-install it on the next iteration.
Started
Thu, 22 Oct 2026 09:59:04 UTC
Duration
300s
Peer count
1
Behaviours
-
minimal(1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
app_install_minimal_39722287437_1
First install time
The time from conductor startup to completion of the first app installation attempt.
0.451684362
s
Install app timing
The time taken to install an app.
mean
0.080878
s
SD
0.190271
s
p50
0.049328
s
p95
0.071753
s
p99
1.312886
s
Holochain MetricsClick to toggle visibility.
WASM usage: 'callback::init'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 10.000196228s
Not enough time data to show a trend.
WASM usage: 'callback_integrity::genesis_self_check_2'
Metered usage count of the WASM ribosome for this zome function.
total
173712
over 20.000283229s
mean rate
8685.48
/s
std
0.05
/s
p5
8685.43
/s
p95
8685.52
/s
peak
8685.52
/s
WASM call duration: 'callback::init'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001197
s
SD
0.000558
s
p50
0.000975
s
p95
0.002303
s
p99
0.003443
s
WASM call duration: 'callback_integrity::genesis_self_check_2'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001223
s
SD
0.000164
s
p50
0.001217
s
p95
0.00125
s
p99
0.00125
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.000117
s
SD
0.000116
s
p50
6.9e-05
s
p95
0.000399
s
p99
0.000549
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
5.93
s
mean
15.93
s
SD
8.17
s
p95
25.93
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 20.000370874s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.00167
s
SD
0.000411
s
p50
0.001674
s
p95
0.001702
s
p99
0.001702
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.002105
s
SD
0.000112
s
p50
0.002233
s
p95
0.002276
s
p99
0.002276
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.007555
s
SD
0.00088
s
p50
0.007323
s
p95
0.008069
s
p99
0.008069
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.002928
s
SD
0.000433
s
p50
0.002787
s
p95
0.003251
s
p99
0.003251
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
0.001973
s
SD
0.000272
s
p50
0.001929
s
p95
0.002086
s
p99
0.002086
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
4.9e-05
s
SD
1.1e-05
s
p50
5e-05
s
p95
7.3e-05
s
p99
7.3e-05
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.001609
s
SD
0.000579
s
p50
0.001819
s
p95
0.002496
s
p99
0.002496
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000418
s
SD
8.9e-05
s
p50
0.000412
s
p95
0.000436
s
p99
0.000436
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
69.82
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
3.98
MiB/s
mean
4.16
MiB/s
SD
181.99
KiB/s
p95
4.33
MiB/s
Not enough time data to show a trend.
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
60.79
KiB/s
mean
94.39
KiB/s
SD
33.60
KiB/s
p95
127.99
KiB/s
Not enough time data to show a trend.
Total bytes received
Total bytes received on primary network interface
count
83.14
MiB
mean
4.16
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
1.84
MiB
mean
94.39
KiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 96.6%
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 2.35 MiB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
Detected
Warning 100% of hosts overloaded (load5/ncpus > 1.0)
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
87.43
%
mean
92.02
%
SD
4.58
%
p95
96.6
%
Not enough time data to show a trend.
System
22.20
%
CPU percentiles
CPU usage percentiles
p50
96.60
%
p95
96.60
%
p99
96.60
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
1
hosts
mean time
300.00
s
Memory used percentage
Percentage of memory used
p5
20.07
%
mean
25.47
%
SD
4.53
%
p95
31.16
%
Not enough time data to show a trend.
Memory available percentage
Percentage of available memory
p5
68.84
%
mean
74.53
%
SD
4.53
%
p95
79.93
%
Not enough time data to show a trend.
Max host memory used
Maximum memory usage percentage across all hosts
max
25.47
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00
%
Memory growth rate
Rate of memory growth over time
growth
2.35
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.00
MiB/s
Disk write throughput
Disk write throughput in MiB/s
10.71
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/etc/nomad.d/client-meta.json0/1
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
4.99
5 min
4.00
15 min
2.36
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
100.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
36.83
%
mean
48.93
%
SD
8.8233
%
p95
57.62
%
Not enough time data to show a trend.
60 second average
36.1833
%
300 second average
29.2133
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
Not enough time data to show a trend.
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
Not enough time data to show a trend.
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
1.02
%
mean
2.6033
%
SD
1.6563
%
p95
4.89
%
Not enough time data to show a trend.
60 second average
2.6100
%
300 second average
1.5233
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0.54
%
mean
1.74
%
SD
1.2238
%
p95
3.42
%
Not enough time data to show a trend.
60 second average
1.6067
%
300 second average
0.8567
%
Holochain process CPU usage
CPU usage by Holochain process
p5
36.7
%
mean
47.79
%
SD
11.09
%
p95
58.88
%
Not enough time data to show a trend.
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
273.49
KiB
mean
594.39
KiB
SD
449.13
KiB
p95
1.20
MiB
Not enough time data to show a trend.
Holochain process threads
Number of threads in Holochain process
p5
5
threads
mean
104
threads
SD
117.45
threads
p95
269
threads
Not enough time data to show a trend.
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
12
file descriptors
mean
230
file descriptors
SD
250.61
file descriptors
p95
581
file descriptors
Not enough time data to show a trend.
Conductor Startup
↑ Back to indexThis scenario measures Holochain conductor startup time when many cells are installed. The agent starts an empty conductor, installs a configurable number of apps without enabling them (disabled cells), restarts the conductor, and then enables cells one by one — timing each enable call and periodically restarting the conductor to measure startup time as the share of enabled cells grows.
Started
Thu, 22 Oct 2026 09:45:54 UTC
Duration
900s
Peer count
30
Peer count at end
19
Behaviours
-
default(1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
conductor_startup_39722287437_1
Initial conductor startup timing
Startup time with an empty conductor before any cells are installed.
mean
3.369546
s
SD
2.509633
s
p50
1.945254
s
p95
8.838662
s
p99
11.311801
s
Disabled-cell conductor startup timing
Startup time after all configured cells are installed but remain disabled.
mean
1.882192
s
SD
0.503343
s
p50
1.555524
s
p95
2.534551
s
p99
2.534551
s
Progressive conductor startup timing
Startup time sampled periodically as more installed cells are enabled.
mean
11.001148
s
SD
5.88736
s
p50
10.074734
s
p95
22.237924
s
p99
26.420433
s
Cell enable timing
Time to enable one previously disabled cell via the admin interface.
mean
0.132756
s
SD
0.067726
s
p50
0.117164
s
p95
0.216307
s
p99
0.356755
s
Installed cells
Number of cells installed on the conductor during setup.
200
cells
Maximum enabled cells
Highest percentage of installed cells enabled during the run.
100
%
Holochain MetricsClick to toggle visibility.
WASM usage: 'callback_integrity::genesis_self_check_2'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 726.65469187s
Not enough time data to show a trend.
WASM call duration: 'callback_integrity::genesis_self_check_2'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001889
s
SD
0.00073
s
p50
0.002147
s
p95
0.002849
s
p99
0.003165
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
0.79
s
mean
262.84
s
SD
226.07
s
p95
653.28
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 869.977546128s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
3.855467
s
SD
4.87275
s
p50
1.003884
s
p95
14.429428
s
p99
18.508715
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.001659
s
SD
0.006394
s
p50
0.001283
s
p95
0.003212
s
p99
0.004728
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
1e-06
s
p50
0
s
p95
0
s
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.009889
s
SD
0.015539
s
p50
0.001761
s
p95
0.003838
s
p99
0.010108
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.031133
s
SD
0.211367
s
p50
0.002167
s
p95
0.005949
s
p99
1.342091
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.011788
s
SD
0.108429
s
p50
0.00214
s
p95
0.004616
s
p99
0.009458
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
0.00146
s
SD
0.006172
s
p50
0.000912
s
p95
0.003411
s
p99
0.005931
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.000115
s
SD
0.000104
s
p50
0.000172
s
p95
0.00032
s
p99
0.000406
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.011423
s
SD
0.067825
s
p50
0.001749
s
p95
0.003358
s
p99
0.086688
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000302
s
SD
0.000124
s
p50
0.00034
s
p95
0.000572
s
p99
0.000747
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
0.149094
s
SD
0.094106
s
p50
0.157958
s
p95
0.3204
s
p99
0.327303
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.31775
s
SD
0
s
p50
0.31775
s
p95
0.31775
s
p99
0.31775
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
17.18
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
5.18
KiB/s
mean
338.15
KiB/s
SD
791.60
KiB/s
p95
1.80
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
3.84
KiB/s
mean
57.48
KiB/s
SD
52.26
KiB/s
p95
142.78
KiB/s
Total bytes received
Total bytes received on primary network interface
count
1.66
GiB
mean
1.94
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
289.63
MiB
mean
337.02
KiB/s
CPU spike anomaly
CPU spike anomaly was detected
NotDetected
Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
NotDetected
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
3.24
%
mean
27.18
%
SD
23.83
%
p95
72.26
%
System
10.00
%
CPU percentiles
CPU usage percentiles
p50
16.60
%
p95
72.26
%
p99
76.80
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
1
hosts
mean time
50.00
s
Memory used percentage
Percentage of memory used
p5
7.35
%
mean
16.92
%
SD
6.73
%
p95
27.32
%
Memory available percentage
Percentage of available memory
p5
72.68
%
mean
83.08
%
SD
6.73
%
p95
92.65
%
Max host memory used
Maximum memory usage percentage across all hosts
max
32.94
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
8.49
%
Memory growth rate
Rate of memory growth over time
growth
0.54
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.02
MiB/s
Disk write throughput
Disk write throughput in MiB/s
131.15
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/0/8
hosts
Mount Point
/efi-boot0/4
hosts
Mount Point
/etc/hostname0/4
hosts
Mount Point
/etc/hosts0/4
hosts
Mount Point
/etc/nomad.d/client-meta.json0/13
hosts
Mount Point
/etc/resolv.conf0/4
hosts
Mount Point
/nix/store0/8
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
2.76
5 min
1.75
15 min
0.91
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
0.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
7.9017
%
SD
13.2952
%
p95
41.35
%
60 second average
5.5625
%
300 second average
2.4057
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0002
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0001
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
26
%
SD
24.831
%
p95
58.44
%
60 second average
25.6824
%
300 second average
20.2129
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
23.2872
%
SD
23.3757
%
p95
55.62
%
60 second average
23.2924
%
300 second average
18.6014
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0
%
mean
10.73
%
SD
21.28
%
p95
65.08
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
69.53
KiB
mean
278.93
KiB
SD
137.80
KiB
p95
526.58
KiB
Holochain process threads
Number of threads in Holochain process
p5
5
threads
mean
325.54
threads
SD
446.43
threads
p95
1380
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
12
file descriptors
mean
741.45
file descriptors
SD
980.06
file descriptors
p95
3043
file descriptors
DHT Sync Lag
↑ Back to indexMeasures the lag time between an agent publishing data and other peers being able to see it. This scenario has two roles:
write: Creates entries with a timestamp field, linked to a known base hash so thatrecord_lagagents can discover them.record_lag: Repeatedly queries for links from the known base hash. When a new entry is discovered, it calculates the time difference between the entry's timestamp and the current time, giving the sync lag for that entry.
Started
Thu, 22 Oct 2026 10:01:03 UTC
Duration
1800s
Peer count
30
Peer count at end
23
Behaviours
-
record_lag(1 agent) -
write(1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
dht_sync_lag_39722287437_1
Create rate
The average number of created records per agent.
Mean
mean
19.45
/s
Max
Highest per-partition mean rate
max
33.94
/s
Min
Lowest per-partition mean rate
min
10.52
/s
Sync lag timing
The average time between when a record was created and when it is first seen by an agent.
Mean
mean
396.184705
s
SD
336.878667
s
Max
Highest per-partition mean latency
max mean
701.527712
s
Min
Lowest per-partition mean latency
min mean
20.411471
s
Sync lag rate
The average number of created records discovered per agent.
Mean
mean
35.69
/s
Max
Highest per-partition mean rate
max
89.76
/s
Min
Lowest per-partition mean rate
min
0
/s
Delivery ratio
The proportion of created entries successfully received by all record_lag agents.
0.0431
Sent count
The total number of entries sent (created) by write agents, by agent.
Total
65283
Agents affected
18 / 18
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
3627
Max
Highest count in any single partition
6101
Min
Lowest count in any single partition
2146
Received count
The total number of entries received by record_lag agents, by agent.
Total
14071
Agents affected
5 / 5
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
2814
Max
Highest count in any single partition
4791
Min
Lowest count in any single partition
2
Error count
The number of errors encountered during the scenario.
10
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.000592
s
SD
0.001024
s
p50
0.000519
s
p95
0.00355
s
p99
0.003617
s
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
0
over 1722.650703139s
Not enough time data to show a trend.
WASM usage: 'timed::created_timed_entry'
Metered usage count of the WASM ribosome for this zome function.
total
269069920
over 1788.800747226s
mean rate
1.2499961722e+08
/s
std
1.7472985923e+08
/s
p5
3.69988921e+06
/s
p95
5.682201348e+08
/s
peak
9.3519742554e+08
/s
WASM usage: 'timed::get_timed_entries_local'
Metered usage count of the WASM ribosome for this zome function.
total
85888693095
over 1784.948553s
mean rate
1.02535174372e+11
/s
std
7.40847777167e+10
/s
p5
2.8771119434e+08
/s
p95
2.56565393573e+11
/s
peak
2.93325817723e+11
/s
WASM usage: 'timed_integrity::entry_defs'
Metered usage count of the WASM ribosome for this zome function.
total
42376544
over 1788.800747226s
mean rate
2.585596917e+07
/s
std
3.036607616e+07
/s
p5
806635.84
/s
p95
1.0073400056e+08
/s
peak
1.5616964859e+08
/s
Zome call duration: 'timed::created_timed_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.012244
s
SD
0.011757
s
p50
0.009373
s
p95
0.04667
s
p99
0.053708
s
Zome call duration: 'timed::get_timed_entries_local'
Duration of this zome call as measured by Holochain internally.
mean
0.110341
s
SD
0.145852
s
p50
0.157603
s
p95
0.452376
s
p99
0.456365
s
WASM call duration: 'timed::created_timed_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.011413
s
SD
0.011592
s
p50
0.008527
s
p95
0.045327
s
p99
0.052192
s
WASM call duration: 'timed::get_timed_entries_local'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.109743
s
SD
0.145731
s
p50
0.156991
s
p95
0.451429
s
p99
0.45542
s
WASM call duration: 'timed_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.0017
s
SD
0.001182
s
p50
0.001494
s
p95
0.004755
s
p99
0.00611
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.001768
s
SD
0.0034
s
p50
0.000774
s
p95
0.011857
s
p99
0.014665
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.001718
s
SD
0.003264
s
p50
0.000751
s
p95
0.011303
s
p99
0.014206
s
Host function call duration: '__hc__get_1'
Duration of this host function call invoked from within WASM.
mean
0.000595
s
SD
0.001
s
p50
0.000532
s
p95
0.003498
s
p99
0.003565
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.003726
s
SD
0.004237
s
p50
0.005559
s
p95
0.012796
s
p99
0.012912
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.002746
s
SD
0.002204
s
p50
0.002292
s
p95
0.008694
s
p99
0.01076
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.000268
s
SD
0.000485
s
p50
0.000175
s
p95
0.001705
s
p99
0.002089
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
88.13
s
mean
888.23
s
SD
511.86
s
p95
1686.86
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
19397
over 1788.800509876s
mean rate
36097.49
/s
std
40840.27
/s
p5
5.6
/s
p95
124404.23
/s
peak
199966.94
/s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
84.2238
s
SD
76.341658
s
p50
5.438946
s
p95
169.102121
s
p99
382.282813
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.091706
s
SD
0.729589
s
p50
0.225439
s
p95
1.18966
s
p99
4.785483
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.202498
s
SD
0.50662
s
p50
0.121336
s
p95
1.555268
s
p99
1.799877
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.49215
s
SD
0.357431
s
p50
0.326292
s
p95
1.15568
s
p99
1.605031
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
13.905797
s
SD
68.644913
s
p50
36.634183
s
p95
237.216915
s
p99
284.002459
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
43.189084
s
SD
57.733594
s
p50
0.057081
s
p95
170.998952
s
p99
229.083463
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.001683
s
SD
0.001765
s
p50
0.002077
s
p95
0.005607
s
p99
0.007767
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.007407
s
SD
0.013996
s
p50
0.001794
s
p95
0.045043
s
p99
0.058453
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.001163
s
SD
0.002751
s
p50
0.000471
s
p95
0.008444
s
p99
0.013178
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
1.792299
s
SD
28.769629
s
p50
7.282974
s
p95
60.01258
s
p99
60.025189
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.209795
s
SD
21.40782
s
p50
1.995463
s
p95
60.00531
s
p99
60.011738
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
70.94
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
212.97
KiB/s
mean
946.22
KiB/s
SD
1.32
MiB/s
p95
2.73
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
94.63
KiB/s
mean
588.16
KiB/s
SD
458.84
KiB/s
p95
1.45
MiB/s
Total bytes received
Total bytes received on primary network interface
count
36.98
GiB
mean
21.15
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
22.99
GiB
mean
13.15
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 98.9%
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 1.05 MiB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
Detected
Warning 81% of hosts overloaded (load5/ncpus > 1.0)
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
43.58
%
mean
83.52
%
SD
19.52
%
p95
98.75
%
System
12.58
%
CPU percentiles
CPU usage percentiles
p50
94.73
%
p95
98.75
%
p99
98.93
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
22
hosts
mean time
1254.89
s
Memory used percentage
Percentage of memory used
p5
8.96
%
mean
19.4
%
SD
5.51
%
p95
27.08
%
Memory available percentage
Percentage of available memory
p5
72.92
%
mean
80.6
%
SD
5.51
%
p95
91.04
%
Max host memory used
Maximum memory usage percentage across all hosts
max
33.00
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
8.49
%
Memory growth rate
Rate of memory growth over time
growth
1.05
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.03
MiB/s
Disk write throughput
Disk write throughput in MiB/s
195.06
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/0/11
hosts
Mount Point
/efi-boot0/6
hosts
Mount Point
/etc/hostname0/4
hosts
Mount Point
/etc/hosts0/4
hosts
Mount Point
/etc/nomad.d/client-meta.json0/12
hosts
Mount Point
/etc/resolv.conf0/4
hosts
Mount Point
/nix/store0/11
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
11.86
5 min
10.69
15 min
7.39
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
81.48
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
2.23
%
mean
55.2857
%
SD
31.5297
%
p95
97.38
%
60 second average
54.0741
%
300 second average
47.9930
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0020
%
300 second average
0.0011
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0007
%
300 second average
0.0005
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
16.8198
%
SD
24.6243
%
p95
65.26
%
60 second average
16.6238
%
300 second average
16.0390
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
5.6673
%
SD
9.9103
%
p95
27.38
%
60 second average
5.6940
%
300 second average
6.2612
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.04
%
mean
22.71
%
SD
24.24
%
p95
75.96
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
24.29
KiB
mean
274.53
KiB
SD
141.90
KiB
p95
517.61
KiB
Holochain process threads
Number of threads in Holochain process
p5
1
threads
mean
32.57
threads
SD
30.83
threads
p95
53
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
7
file descriptors
mean
72.71
file descriptors
SD
66.73
file descriptors
p95
98
file descriptors
First Call
↑ Back to indexThis scenario installs a simple app which implements the init callback, and measures the time elapsed between start of install and calling init. It will uninstall the app it installed so that it can re-install it on the next iteration. This is required to re-run the initialisation callback.
Started
Thu, 22 Oct 2026 09:38:32 UTC
Duration
300s
Peer count
1
Behaviours
-
default(1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
first_call_39722287437_1
Zome call timing
The time elapsed between start of install and calling
init.mean
0.012113
s
SD
0.006553
s
p50
0.010909
s
p95
0.0165
s
p99
0.051499
s
Holochain MetricsClick to toggle visibility.
WASM usage: 'crud::create_sample_entry'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 10.000260374s
Not enough time data to show a trend.
WASM usage: 'crud::init'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 10.000260374s
Not enough time data to show a trend.
WASM usage: 'crud_integrity::entry_defs'
Metered usage count of the WASM ribosome for this zome function.
total
255432
over 10.000260374s
mean rate
25542.53
/s
std
0
/s
p5
25542.53
/s
p95
25542.53
/s
peak
25542.53
/s
Not enough time data to show a trend.
Zome call duration: 'crud::create_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.004769
s
SD
0.000166
s
p50
0.005058
s
p95
0.005058
s
p99
0.005058
s
WASM call duration: 'crud::create_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.004016
s
SD
0.000741
s
p50
0.004006
s
p95
0.005588
s
p99
0.005597
s
WASM call duration: 'crud::init'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001273
s
SD
0.000169
s
p50
0.001251
s
p95
0.001304
s
p99
0.002297
s
WASM call duration: 'crud_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001135
s
SD
1.1e-05
s
p50
0.001153
s
p95
0.001153
s
p99
0.001153
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000313
s
SD
5.5e-05
s
p50
0.000356
s
p95
0.000386
s
p99
0.000386
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.002088
s
SD
5.9e-05
s
p50
0.00219
s
p95
0.00219
s
p99
0.00219
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.000111
s
SD
7.6e-05
s
p50
8.2e-05
s
p95
0.000349
s
p99
0.000369
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
6.89
s
mean
11.89
s
SD
5
s
p95
16.89
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 10.000096029s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.001042
s
SD
0.000108
s
p50
0.001231
s
p95
0.001231
s
p99
0.001231
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.001277
s
SD
5.6e-05
s
p50
0.001292
s
p95
0.001292
s
p99
0.001292
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.004073
s
SD
0.000263
s
p50
0.004532
s
p95
0.004532
s
p99
0.004532
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.002975
s
SD
0.000595
s
p50
0.004016
s
p95
0.004016
s
p99
0.004016
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
0.002277
s
SD
0.000519
s
p50
0.003186
s
p95
0.003186
s
p99
0.003186
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
3.6e-05
s
SD
5e-06
s
p50
4.4e-05
s
p95
4.4e-05
s
p99
4.4e-05
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.001088
s
SD
0.000459
s
p50
0.001791
s
p95
0.001848
s
p99
0.001848
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000265
s
SD
9e-06
s
p50
0.000281
s
p95
0.000281
s
p99
0.000281
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
48.45
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
35.45
KiB/s
mean
45.98
KiB/s
SD
10.53
KiB/s
p95
56.52
KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
19.39
KiB/s
mean
24.67
KiB/s
SD
5.27
KiB/s
p95
29.94
KiB/s
Total bytes received
Total bytes received on primary network interface
count
919.68
KiB
mean
45.98
KiB/s
Total bytes sent
Total bytes sent on primary network interface
count
493.38
KiB
mean
24.67
KiB/s
CPU spike anomaly
CPU spike anomaly was detected
NotDetected
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 7.71 MiB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
NotDetected
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
47.22
%
mean
58.15
%
SD
10.93
%
p95
69.07
%
System
9.69
%
CPU percentiles
CPU usage percentiles
p50
69.07
%
p95
69.07
%
p99
69.07
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
0
hosts
mean time
0.00
s
Memory used percentage
Percentage of memory used
p5
17.36
%
mean
25.59
%
SD
8.34
%
p95
37.02
%
Memory available percentage
Percentage of available memory
p5
62.98
%
mean
74.41
%
SD
8.34
%
p95
82.64
%
Max host memory used
Maximum memory usage percentage across all hosts
max
25.59
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00
%
Memory growth rate
Rate of memory growth over time
growth
7.71
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.03
MiB/s
Disk write throughput
Disk write throughput in MiB/s
15.84
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/etc/nomad.d/client-meta.json0/1
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
0.24
5 min
0.07
15 min
0.02
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
0.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
4.23
%
mean
7.7267
%
SD
2.9881
%
p95
11.53
%
60 second average
2.3300
%
300 second average
0.5367
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
1.2433
%
SD
1.0045
%
p95
2.46
%
60 second average
0.3067
%
300 second average
0.0800
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
0.7967
%
SD
0.5698
%
p95
1.3
%
60 second average
0.2200
%
300 second average
0.0600
%
Holochain process CPU usage
CPU usage by Holochain process
p5
44.86
%
mean
56.28
%
SD
11.42
%
p95
67.69
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
73.46
KiB
mean
1008.39
KiB
SD
794.86
KiB
p95
1.97
MiB
Holochain process threads
Number of threads in Holochain process
p5
23
threads
mean
191
threads
SD
152.44
threads
p95
392
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
68
file descriptors
mean
423.33
file descriptors
SD
323.21
file descriptors
p95
850
file descriptors
Full-Arc Create (Validated) / Zero-Arc Read
↑ Back to indexA full-arc/zero-arc mixed scenario where full-arc nodes create data that gets validated and zero-arc nodes read the data. The scenario has two roles:
full: A full-arc conductor that creates entries with a timestamp field. Those entries get validated and then retrieved by zero-arc nodes.zero: A zero-arc conductor that reads the entries created by the full-arc node(s) and records the time lag between when the entry had been created and when it was first discovered.
Started
Thu, 22 Oct 2026 10:36:50 UTC
Duration
900s
Peer count
30
Peer count at end
16
Behaviours
-
full(1 agent) -
zero(1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
full_arc_create_validated_zero_arc_read--high_zero_39722287437_1
Create rate
The number of timed entries created by the full-arc node(s) per second.
Mean
mean
29.06
/s
Max
Highest per-partition mean rate
max
36.45
/s
Min
Lowest per-partition mean rate
min
21.66
/s
Fetch lag timing
For each entry, the time lag between when it was created by a full-arc node and when a zero-arc node fetched it from the network.
Mean
mean
292.939003
s
SD
141.202429
s
Max
Highest per-partition mean latency
max mean
765.809596
s
Min
Lowest per-partition mean latency
min mean
108.884344
s
Fetch rate
The number of entries per second fetched by zero-arc nodes.
Mean
mean
193.76
/s
Max
Highest per-partition mean rate
max
556.75
/s
Min
Lowest per-partition mean rate
min
0
/s
Delivery ratio
The proportion of created entries successfully fetched by all zero-arc nodes.
0.7536
Entry created count
Total number of entries created by full-arc nodes, by agent.
Total
5317
Agents affected
2 / 2
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
2659
Max
Highest count in any single partition
3364
Min
Lowest count in any single partition
1953
Received count
Total number of entries fetched by zero-arc nodes, by agent.
Total
56096
Agents affected
13 / 14
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
4007
Max
Highest count in any single partition
5044
Min
Lowest count in any single partition
0
Open connections
The number of currently open connections to other conductors.
full-arc
p5
9
mean
14.4
SD
2.03
p95
15
zero-arc
p5
1
mean
8.2
SD
4.25
p95
15
Retrieval errors
Errors encountered by zero-arc nodes when attempting to retrieve entries created by full-arc nodes.
Total
142
Agents affected
14 / 14
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
10
Max
Highest count in any single partition
14
Min
Lowest count in any single partition
6
Error count
The number of errors accumulated across all nodes.
143
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.015919
s
SD
0.089642
s
p50
0.028632
s
p95
0.120974
s
p99
0.309669
s
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
0
over 817.317220982s
Not enough time data to show a trend.
Zome call duration: 'timed_and_validated::create_timed_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.01155
s
SD
0.006493
s
p50
0.00784
s
p95
0.022173
s
p99
0.022409
s
Zome call duration: 'timed_and_validated::get_timed_entries_network'
Duration of this zome call as measured by Holochain internally.
mean
1.463315
s
SD
95.828778
s
p50
20.537872
s
p95
247.846674
s
p99
397.818895
s
WASM call duration: 'timed_and_validated::create_timed_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.010684
s
SD
0.006372
s
p50
0.00698
s
p95
0.021142
s
p99
0.021356
s
WASM call duration: 'timed_and_validated::get_timed_entries_network'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
1.579758
s
SD
88.968688
s
p50
49.739724
s
p95
225.569107
s
p99
397.817676
s
WASM call duration: 'timed_and_validated_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.002152
s
SD
0.000793
s
p50
0.001241
s
p95
0.003398
s
p99
0.004532
s
WASM call duration: 'timed_and_validated_integrity::validate'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.01241
s
SD
0.005717
s
p50
0.001375
s
p95
0.016126
s
p99
0.030147
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000927
s
SD
0.000711
s
p50
0.000508
s
p95
0.002102
s
p99
0.002119
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.000902
s
SD
0.000674
s
p50
0.000515
s
p95
0.002017
s
p99
0.002036
s
Host function call duration: '__hc__get_1'
Duration of this host function call invoked from within WASM.
mean
0.015344
s
SD
0.074023
s
p50
0.033687
s
p95
0.117684
s
p99
0.249005
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.076044
s
SD
5.152128
s
p50
1.446016
s
p95
12.882187
s
p99
30.470731
s
Host function call duration: '__hc__must_get_valid_record_1'
Duration of this host function call invoked from within WASM.
mean
0.023839
s
SD
0.014345
s
p50
0.016224
s
p95
0.046416
s
p99
0.048534
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.003581
s
SD
0.001913
s
p50
0.002034
s
p95
0.007218
s
p99
0.011444
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.000175
s
SD
0.000131
s
p50
0.00012
s
p95
0.000489
s
p99
0.000489
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
47.33
s
mean
435.95
s
SD
251.43
s
p95
828.18
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 887.180106605s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
142.393826
s
SD
57.339988
s
p50
0
s
p95
63.97328
s
p99
331.748611
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
5.066049
s
SD
3.544258
s
p50
0.00191
s
p95
4.423896
s
p99
23.04486
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.038816
s
SD
0.021506
s
p50
0.001897
s
p95
0.022677
s
p99
0.138551
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.209285
s
SD
0.066938
s
p50
0.02225
s
p95
0.226482
s
p99
0.321118
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
15.940786
s
SD
9.7565
s
p50
0.002525
s
p95
26.769487
s
p99
53.309174
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
8.656764
s
SD
5.745192
s
p50
0.002088
s
p95
15.157816
s
p99
30.973955
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.002548
s
SD
0.003952
s
p50
0.000107
s
p95
0.012868
s
p99
0.017734
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.008624
s
SD
0.010974
s
p50
0.001689
s
p95
0.009305
s
p99
0.072517
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000489
s
SD
0.000232
s
p50
0.0003
s
p95
0.000925
s
p99
0.001282
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
1.30272
s
SD
29.108793
s
p50
0.20855
s
p95
60.001527
s
p99
60.002403
s
P2P request roundtrip: 'get_links'
Time spent sending a get_links request and awaiting its response.
mean
0.664786
s
SD
22.422098
s
p50
0.71137
s
p95
60.001939
s
p99
60.001939
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
2.21597
s
SD
4.484063
s
p50
2.788147
s
p95
5.282449
s
p99
12.279813
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
46.56
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
0
B/s
mean
744.07
KiB/s
SD
1.17
MiB/s
p95
2.34
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
0
B/s
mean
477.72
KiB/s
SD
701.38
KiB/s
p95
1.86
MiB/s
Total bytes received
Total bytes received on primary network interface
count
10.23
GiB
mean
11.63
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
6.57
GiB
mean
7.47
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 98.7%
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 1.60 MiB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected
Critical Heavy swap usage (94.5% swap used)
System overload anomaly
System overload anomaly was detected
Detected
Warning 25% of hosts overloaded (load5/ncpus > 1.0)
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
2.23
%
mean
57.95
%
SD
33.93
%
p95
98.48
%
System
11.38
%
CPU percentiles
CPU usage percentiles
p50
65.15
%
p95
98.48
%
p99
98.65
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
13
hosts
mean time
387.04
s
Memory used percentage
Percentage of memory used
p5
10.78
%
mean
24.63
%
SD
17.48
%
p95
88.33
%
Memory available percentage
Percentage of available memory
p5
11.67
%
mean
75.37
%
SD
17.48
%
p95
89.22
%
Max host memory used
Maximum memory usage percentage across all hosts
max
89.10
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
94.53
%
Memory growth rate
Rate of memory growth over time
growth
1.60
MiB/s
Disk read throughput
Disk read throughput in MiB/s
7.96
MiB/s
Disk write throughput
Disk write throughput in MiB/s
191.17
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/0/5
hosts
Mount Point
/efi-boot0/1
hosts
Mount Point
/etc/hostname0/11
hosts
Mount Point
/etc/hosts0/11
hosts
Mount Point
/etc/nomad.d/client-meta.json0/12
hosts
Mount Point
/etc/resolv.conf0/11
hosts
Mount Point
/nix/store0/5
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
7.70
5 min
7.77
15 min
8.09
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
25.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
27.0866
%
SD
31.9242
%
p95
88.44
%
60 second average
28.0355
%
300 second average
28.7297
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0.0583
%
SD
0.544
%
p95
0.01
%
60 second average
0.0568
%
300 second average
0.0521
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0.0563
%
SD
0.5266
%
p95
0.01
%
60 second average
0.0545
%
300 second average
0.0500
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
2.7171
%
SD
10.2544
%
p95
28.3
%
60 second average
2.5963
%
300 second average
2.3724
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
2.0255
%
SD
7.8253
%
p95
20.6
%
60 second average
1.9115
%
300 second average
1.7395
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.07
%
mean
14.63
%
SD
19.61
%
p95
52.53
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
24.47
KiB
mean
322.24
KiB
SD
135.13
KiB
p95
490.33
KiB
Holochain process threads
Number of threads in Holochain process
p5
9
threads
mean
33.69
threads
SD
12.07
threads
p95
57
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
8
file descriptors
mean
64.19
file descriptors
SD
22.58
file descriptors
p95
93
file descriptors
Local Signals
↑ Back to indexThis scenario is for testing the speed and reliability of local signals. It uses a zome with a function that will emit 10,000 signals. The scenario measures how long it takes to send the signals and how many have been received by the time the zome call ends.
Started
Thu, 22 Oct 2026 09:40:16 UTC
Duration
300s
Peer count
1
Peer count at end
1
Behaviours
-
default(1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
local_signals_39722287437_1
Send timing
The time taken to emit a batch of 10,000 signals.
mean
1.352356
s
SD
0.7946
s
p50
1.348331
s
p95
2.651498
s
p99
2.651498
s
Receive timing
The time taken to receive the completed batch, to the nearest 250ms.
mean
27.84428
s
SD
6.47052
s
p50
30.000969
s
p95
30.001929
s
p99
30.001929
s
Success ratio
The proportion of signals successfully received in a batch.
min
0.4424000084400177
mean
0.6359
SD
0.0896
max
0.7716000080108643
Holochain MetricsClick to toggle visibility.
WASM usage: 'signal::emit_10k_signals'
Metered usage count of the WASM ribosome for this zome function.
total
279680004
over 280.002823168s
mean rate
998846.78
/s
std
1.45129073e+06
/s
p5
0
/s
p95
3.1075343e+06
/s
peak
3.10754855e+06
/s
Zome call duration: 'signal::emit_10k_signals'
Duration of this zome call as measured by Holochain internally.
mean
0.16774
s
SD
0.00867
s
p50
0.168199
s
p95
0.175549
s
p99
0.177515
s
WASM call duration: 'signal::emit_10k_signals'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.166771
s
SD
0.008648
s
p50
0.167412
s
p95
0.174589
s
p99
0.176556
s
Host function call duration: '__hc__emit_signal_1'
Duration of this host function call invoked from within WASM.
mean
1.1e-05
s
SD
1e-06
s
p50
1.1e-05
s
p95
1.2e-05
s
p99
1.2e-05
s
Emitted local signals
Count of local signals emitted via the emit_signal host function.
total
90000
over 280.002831575s
mean rate
321.43
/s
std
467.02
/s
p5
0
/s
p95
999.99
/s
peak
1000
/s
Post-commit duration
Time spent executing post-commit workflows.
mean
8.1e-05
s
SD
0
s
p50
8.1e-05
s
p95
8.1e-05
s
p99
8.1e-05
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
16.91
s
mean
146.91
s
SD
83.67
s
p95
276.91
s
Dropped WebSocket signals
Count of signals dropped from the app WebSocket due to channel overload.
total
3385
over 280.002760107s
mean rate
12.09
/s
std
21.65
/s
p5
0
/s
p95
56
/s
peak
86.2
/s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 280.00277139s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.001646
s
SD
0
s
p50
0.001646
s
p95
0.001646
s
p99
0.001646
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.001273
s
SD
0
s
p50
0.001273
s
p95
0.001273
s
p99
0.001273
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.002633
s
SD
0.000264
s
p50
0.002814
s
p95
0.003042
s
p99
0.003042
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.00165
s
SD
0
s
p50
0.00165
s
p95
0.00165
s
p99
0.00165
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
0.000543
s
SD
0
s
p50
0.000543
s
p95
0.000543
s
p99
0.000543
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
9.6e-05
s
SD
2.3e-05
s
p50
8.8e-05
s
p95
0.000129
s
p99
0.00013
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.000824
s
SD
0.000163
s
p50
0.001048
s
p95
0.001048
s
p99
0.001048
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000208
s
SD
0
s
p50
0.000208
s
p95
0.000208
s
p99
0.000208
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
14.54
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
338
B/s
mean
888.50
KiB/s
SD
2.53
MiB/s
p95
9.07
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
35
B/s
mean
24.35
KiB/s
SD
38.42
KiB/s
p95
143.62
KiB/s
Total bytes received
Total bytes received on primary network interface
count
251.62
MiB
mean
888.50
KiB/s
Total bytes sent
Total bytes sent on primary network interface
count
6.90
MiB
mean
24.35
KiB/s
CPU spike anomaly
CPU spike anomaly was detected
NotDetected
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 3.26 MiB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
NotDetected
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
1.89
%
mean
21.25
%
SD
22.52
%
p95
70.12
%
System
6.71
%
CPU percentiles
CPU usage percentiles
p50
12.49
%
p95
70.12
%
p99
89.32
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
1
hosts
mean time
10.34
s
Memory used percentage
Percentage of memory used
p5
12.76
%
mean
17.11
%
SD
4.04
%
p95
24.28
%
Memory available percentage
Percentage of available memory
p5
75.72
%
mean
82.89
%
SD
4.04
%
p95
87.24
%
Max host memory used
Maximum memory usage percentage across all hosts
max
17.11
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00
%
Memory growth rate
Rate of memory growth over time
growth
3.26
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.00
MiB/s
Disk write throughput
Disk write throughput in MiB/s
3.89
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/etc/nomad.d/client-meta.json0/1
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
1.04
5 min
0.47
15 min
0.18
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
0.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0.3
%
mean
8.8627
%
SD
11.7746
%
p95
37.52
%
60 second average
6.4937
%
300 second average
2.8533
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0487
%
300 second average
0.0790
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0450
%
300 second average
0.0717
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
0.768
%
SD
1.1008
%
p95
2.73
%
60 second average
0.9520
%
300 second average
0.7300
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
0.6717
%
SD
1.0937
%
p95
2.73
%
60 second average
0.8563
%
300 second average
0.6550
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.25
%
mean
5.56
%
SD
8.48
%
p95
17.82
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
109.03
KiB
mean
213.64
KiB
SD
44.38
KiB
p95
254.99
KiB
Holochain process threads
Number of threads in Holochain process
p5
18
threads
mean
24
threads
SD
6.18
threads
p95
30
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
57
file descriptors
mean
66.2
file descriptors
SD
14.38
file descriptors
p95
78
file descriptors
Mixed Arc get_agent_activity
↑ Back to index
A mixed-arc scenario that measures the DHT's ability to propagate agent activity and make it available via get_agent_activity calls. The scenario has three roles:
zero_write: A zero-arc conductor that creates entries on its source chain.full_write: A full-arc conductor that creates entries on its source chain.zero_read: A zero-arc conductor that picks a random writer and repeatedly callsget_agent_activityto observe new entries appearing on the writer's chain, measuring the lag between creation and discovery.
Started
Thu, 22 Oct 2026 10:39:01 UTC
Duration
900s
Peer count
30
Peer count at end
19
Behaviours
-
full_write(1 agent) -
zero_read(1 agent) -
zero_write(1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
mixed_arc_get_agent_activity_39722287437_1
Entry created count
The total number of entries created by write agents over the run, partitioned by agent and behaviour.
Total
68466
Agents affected
12 / 12
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
5706
Max
Highest count in any single partition
9532
Min
Lowest count in any single partition
2358
Observed chain advancement
The maximum highest-observed action sequence per write agent, aggregated across all reading agents. The reading dimension is collapsed by taking the maximum observed value for each write agent (any reader successfully tracking that point counts as propagation). Captures how far readers tracked writers’ chains during the run.
Mean max (across write agents)
Mean of the highest chain head value observed per write agent. Each write agent contributes its maximum — the value seen by any reader.
2182
Max
Highest chain head value observed for any single write agent.
3218
Write agents observed
3
Chain head delay timing
The time lag between when a writer creates a new entry and when a reader discovers it via
get_agent_activity. This is measured per new action in the chain.Mean
mean
0.346018
s
SD
0.271632
s
Max
Highest per-partition mean latency
max mean
0.82012
s
Min
Lowest per-partition mean latency
min mean
0.024232
s
Chain head delay rate
The rate at which readers detect new chain heads appearing on writers' chains.
Mean
mean
11.07
/s
Max
Highest per-partition mean rate
max
22.01
/s
Min
Lowest per-partition mean rate
min
0
/s
get_agent_activity_full zome call timingThe time taken to complete the
get_agent_activity_full zome call that queries a writer's chain.Mean
mean
1.230567
s
SD
2.361354
s
Max
Highest per-partition mean latency
max mean
4.7417
s
Min
Lowest per-partition mean latency
min mean
0.365285
s
Retrieval errors
A running accumulation of errors encountered by read agents when calling
get_agent_activity.Total
33
Agents affected
7 / 7
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
5
Max
Highest count in any single partition
7
Min
Lowest count in any single partition
1
Open connections
The number of currently open connections to other conductors, partitioned by agent behaviour.
behaviour:full_write
p5
15
mean
17.36
SD
1.56
p95
18
behaviour:zero_read
p5
5
mean
9.65
SD
2.91
p95
14
behaviour:zero_write
p5
0
mean
9.36
SD
4.79
p95
17
Final error count
The total number of errors accumulated across all agents over the run.
33
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.574429
s
SD
1.826452
s
p50
0.516216
s
p95
5.15853
s
p99
8.603741
s
Zome call duration: 'agent_activity::announce_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.005139
s
SD
0.002042
s
p50
0.004794
s
p95
0.008881
s
p99
0.008881
s
Zome call duration: 'agent_activity::create_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.005897
s
SD
0.003322
s
p50
0.006091
s
p95
0.012917
s
p99
0.014096
s
Zome call duration: 'agent_activity::get_agent_activity_full'
Duration of this zome call as measured by Holochain internally.
mean
0.730321
s
SD
2.156085
s
p50
0.778095
s
p95
7.418746
s
p99
9.011196
s
Zome call duration: 'agent_activity::get_random_agent_with_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.013491
s
SD
1.37625
s
p50
0.388245
s
p95
4.254856
s
p99
4.254856
s
WASM call duration: 'agent_activity::announce_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.004427
s
SD
0.001887
s
p50
0.003884
s
p95
0.008041
s
p99
0.008041
s
WASM call duration: 'agent_activity::create_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.005145
s
SD
0.003059
s
p50
0.005396
s
p95
0.011557
s
p99
0.012648
s
WASM call duration: 'agent_activity::get_agent_activity_full'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.729307
s
SD
2.156166
s
p50
0.77579
s
p95
7.418039
s
p99
9.010417
s
WASM call duration: 'agent_activity::get_random_agent_with_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.013094
s
SD
1.376111
s
p50
0.387714
s
p95
4.254155
s
p99
4.254155
s
WASM call duration: 'agent_activity_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001702
s
SD
0.00096
s
p50
0.001269
s
p95
0.00349
s
p99
0.004365
s
Host function call duration: '__hc__agent_info_1'
Duration of this host function call invoked from within WASM.
mean
4e-05
s
SD
3.4e-05
s
p50
1.9e-05
s
p95
0.000129
s
p99
0.000129
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000687
s
SD
0.000564
s
p50
0.00071
s
p95
0.00195
s
p99
0.002025
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.000586
s
SD
0.000583
s
p50
0.000394
s
p95
0.002309
s
p99
0.002309
s
Host function call duration: '__hc__get_agent_activity_1'
Duration of this host function call invoked from within WASM.
mean
0.72585
s
SD
2.156901
s
p50
0.772224
s
p95
7.416891
s
p99
9.008997
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.011495
s
SD
1.37603
s
p50
0.384444
s
p95
4.25141
s
p99
4.25141
s
Host function call duration: '__hc__random_bytes_1'
Duration of this host function call invoked from within WASM.
mean
4e-06
s
SD
1e-06
s
p50
7e-06
s
p95
8e-06
s
p99
8e-06
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.00272
s
SD
0.001378
s
p50
0.00176
s
p95
0.005693
s
p99
0.006699
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.00022
s
SD
0.000162
s
p50
0.000127
s
p95
0.000583
s
p99
0.000632
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
47.04
s
mean
440.87
s
SD
253.96
s
p95
837.62
s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
45.953193
s
SD
46.197104
s
p50
0
s
p95
0.955908
s
p99
333.079682
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.044909
s
SD
0.066637
s
p50
0.001703
s
p95
0.132689
s
p99
0.475254
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.059821
s
SD
0.334425
s
p50
0.002344
s
p95
1.010271
s
p99
1.226999
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.099149
s
SD
0.061369
s
p50
0.049399
s
p95
0.189699
s
p99
0.217495
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
6.732964
s
SD
18.333957
s
p50
0.003902
s
p95
20.200283
s
p99
32.923806
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
0.09716
s
SD
0.094235
s
p50
0.001326
s
p95
0.412115
s
p99
0.412115
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.003651
s
SD
0.001944
s
p50
0.001756
s
p95
0.005461
s
p99
0.005672
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.002751
s
SD
0.004097
s
p50
0.001475
s
p95
0.003892
s
p99
0.030888
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000637
s
SD
0.000514
s
p50
0.000413
s
p95
0.001765
s
p99
0.001974
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
11.954403
s
SD
27.56345
s
p50
13.344432
s
p95
60.003206
s
p99
60.003206
s
P2P request roundtrip: 'get_links'
Time spent sending a get_links request and awaiting its response.
mean
0.548447
s
SD
0.501739
s
p50
0.381412
s
p95
1.673922
s
p99
1.673922
s
P2P request roundtrip: 'get_agent_activity'
Time spent sending a get_agent_activity request and awaiting its response.
mean
0.713886
s
SD
28.913023
s
p50
0.859784
s
p95
60.001608
s
p99
60.001995
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.721431
s
SD
0.755098
s
p50
0.764345
s
p95
3.852445
s
p99
3.852445
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
49.37
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
3.48
KiB/s
mean
573.63
KiB/s
SD
855.31
KiB/s
p95
1.96
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
2.77
KiB/s
mean
402.76
KiB/s
SD
649.18
KiB/s
p95
1.49
MiB/s
Total bytes received
Total bytes received on primary network interface
count
9.19
GiB
mean
10.57
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
6.45
GiB
mean
7.42
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 98.7%
Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
Detected
Critical Disk /etc/resolv.conf: 1/9 hosts above 90% capacity
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected
Critical Heavy swap usage (94.2% swap used)
System overload anomaly
System overload anomaly was detected
Detected
Warning 44% of hosts overloaded (load5/ncpus > 1.0)
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
2.42
%
mean
59.97
%
SD
36.67
%
p95
98.55
%
System
10.60
%
CPU percentiles
CPU usage percentiles
p50
74.30
%
p95
98.55
%
p99
98.70
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
14
hosts
mean time
535.46
s
Memory used percentage
Percentage of memory used
p5
8.09
%
mean
24.83
%
SD
18.53
%
p95
83.2
%
Memory available percentage
Percentage of available memory
p5
16.8
%
mean
75.17
%
SD
18.53
%
p95
91.91
%
Max host memory used
Maximum memory usage percentage across all hosts
max
88.07
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
94.15
%
Memory growth rate
Rate of memory growth over time
growth
0.42
MiB/s
Disk read throughput
Disk read throughput in MiB/s
7.86
MiB/s
Disk write throughput
Disk write throughput in MiB/s
159.42
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/0/5
hosts
Mount Point
/efi-boot0/1
hosts
Mount Point
/etc/hostname1/9
hosts
Mount Point
/etc/hosts1/9
hosts
Mount Point
/etc/nomad.d/client-meta.json0/11
hosts
Mount Point
/etc/nomad.d/meta.json1/1
hosts
Mount Point
/etc/resolv.conf1/9
hosts
Mount Point
/nix/store0/5
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
6.84
5 min
7.17
15 min
7.58
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
44.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
31.3548
%
SD
33.8113
%
p95
89.76
%
60 second average
31.9955
%
300 second average
33.2213
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0.0324
%
SD
0.384
%
p95
0
%
60 second average
0.0344
%
300 second average
0.0404
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0.0311
%
SD
0.3699
%
p95
0
%
60 second average
0.0329
%
300 second average
0.0388
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
2.4567
%
SD
9.488
%
p95
11.79
%
60 second average
2.4635
%
300 second average
2.3291
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
1.7027
%
SD
7.2446
%
p95
9.63
%
60 second average
1.7119
%
300 second average
1.6092
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0
%
mean
26.34
%
SD
30.66
%
p95
92.03
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
23.63
KiB
mean
289.68
KiB
SD
133.18
KiB
p95
484.47
KiB
Holochain process threads
Number of threads in Holochain process
p5
1
threads
mean
32.08
threads
SD
13.08
threads
p95
52
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
0
file descriptors
mean
63.08
file descriptors
SD
27.24
file descriptors
p95
88
file descriptors
Mixed Arc must_get_agent_activity
↑ Back to index
A scenario where write agents create entries in batches of 10, while must_get_agent_activity agents each pick a random write agent and repeatedly attempt to create an entry that references the chain top of their latest batch. This reference means that the entry's validation function needs to make a must_get_agent_activity call.
The purpose of this scenario is to measure the time it takes for published agent activity data to be gossiped among authorities and become available to peers that query it via must_get_agent_activity.
This test is similar to Write Validated must_get_agent_activity, but some write agents are zero-arc and all must_get_agent_activity agents are zero-arc. This forces the must_get_agent_activity host call to go to the network.
Started
Thu, 22 Oct 2026 10:40:54 UTC
Duration
900s
Peer count
30
Peer count at end
13
Behaviours
-
full_write(1 agent) -
zero_must_get_agent_activity(1 agent) -
zero_write(1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
mixed_arc_must_get_agent_activity_39722287437_1
Highest observed
action_seqThe maximum chain length observed per
write agent, aggregated across all reading agents. The reading dimension is collapsed by taking the maximum observed value for each write agent (any successful read counts as propagation). This reflects the DHT’s ability to propagate agent activity ops and make them available to querying peers.Mean max (across write agents)
Mean of the highest chain head value observed per write agent. Each write agent contributes its maximum — the value seen by any reader.
1853
Max
Highest chain head value observed for any single write agent.
1886
Write agents observed
3
Chain batch delay timing
The time between a
write agent's creation of a batch and a must_get_agent_activity agent's successful discovery of the batch and creation/self-validation of a new entry that depends on it.Mean
mean
92.574116
s
SD
98.867867
s
Max
Highest per-partition mean latency
max mean
139.509614
s
Min
Lowest per-partition mean latency
min mean
6.73803
s
Chain batch delay rate
The rate at which a
must_get_agent_activity agent was able to discover batches and create/self-validate new entries that depend on them.Mean
mean
3.19
/s
Max
Highest per-partition mean rate
max
3.73
/s
Min
Lowest per-partition mean rate
min
2.14
/s
create_validated_sample_entry zome call timingThe time taken to complete the zome function call that creates the entry that depends on a
write agent's source chain.Mean
mean
0.908853
s
SD
1.072821
s
Max
Highest per-partition mean latency
max mean
1.623492
s
Min
Lowest per-partition mean latency
min mean
0.32041
s
Retrieval errors
A running accumulation of the errors encountered by an agent when attempting to self-validate actions that depend on
must_get_agent_activity calls.Total
6273
Agents affected
4 / 4
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
1568
Max
Highest count in any single partition
2543
Min
Lowest count in any single partition
593
Open connections
The average number of open connections maintained by an agent over the run, broken down by behaviour.
behaviour:full_write
p5
10
mean
11.45
SD
1.04
p95
12
behaviour:zero_must_get_agent_activity
p5
5
mean
7.86
SD
2.16
p95
11
behaviour:zero_write
p5
5
mean
8.16
SD
2.68
p95
12
Final error count
The total number of all types of error accumulated over the run by all agents.
10130
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.154297
s
SD
0.169176
s
p50
0.122864
s
p95
0.619965
s
p99
0.619965
s
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
1
over 784.834476351s
mean rate
0.71
/s
std
1.43
/s
p5
0
/s
p95
4.83
/s
peak
4.84
/s
Zome call duration: 'validated_must_get_agent_activity::announce_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.007254
s
SD
0.003146
s
p50
0.006652
s
p95
0.011282
s
p99
0.011282
s
Zome call duration: 'validated_must_get_agent_activity::create_sample_entries_batch'
Duration of this zome call as measured by Holochain internally.
mean
0.05537
s
SD
0.02085
s
p50
0.047015
s
p95
0.10078
s
p99
0.140084
s
Zome call duration: 'validated_must_get_agent_activity::create_validated_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.006392
s
SD
0.001018
s
p50
0.007199
s
p95
0.008133
s
p99
0.00819
s
Zome call duration: 'validated_must_get_agent_activity::get_chain_top_for_batch'
Duration of this zome call as measured by Holochain internally.
mean
0.115374
s
SD
0.051189
s
p50
0.115817
s
p95
0.22821
s
p99
0.22821
s
Zome call duration: 'validated_must_get_agent_activity::get_random_agent_with_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.01623
s
SD
0.224313
s
p50
0.31037
s
p95
0.625901
s
p99
0.625901
s
WASM call duration: 'validated_must_get_agent_activity::announce_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.006083
s
SD
0.002634
s
p50
0.005695
s
p95
0.009332
s
p99
0.009332
s
WASM call duration: 'validated_must_get_agent_activity::create_sample_entries_batch'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.054497
s
SD
0.020641
s
p50
0.046164
s
p95
0.099321
s
p99
0.138573
s
WASM call duration: 'validated_must_get_agent_activity::create_validated_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.005306
s
SD
0.000872
s
p50
0.00592
s
p95
0.006936
s
p99
0.006985
s
WASM call duration: 'validated_must_get_agent_activity::get_chain_top_for_batch'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.239807
s
SD
0.084866
s
p50
0.182845
s
p95
0.342935
s
p99
0.352638
s
WASM call duration: 'validated_must_get_agent_activity::get_random_agent_with_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.015342
s
SD
0.224245
s
p50
0.308864
s
p95
0.624805
s
p99
0.624805
s
WASM call duration: 'validated_must_get_agent_activity_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001141
s
SD
0.000463
s
p50
0.001274
s
p95
0.002265
s
p99
0.002361
s
WASM call duration: 'validated_must_get_agent_activity_integrity::validate'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.00976
s
SD
0.032069
s
p50
0.003091
s
p95
0.062395
s
p99
0.176301
s
Host function call duration: '__hc__agent_info_1'
Duration of this host function call invoked from within WASM.
mean
2.1e-05
s
SD
4.2e-05
s
p50
2.4e-05
s
p95
0.000201
s
p99
0.000201
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000437
s
SD
0.000125
s
p50
0.000442
s
p95
0.00061
s
p99
0.000722
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.000424
s
SD
0.000189
s
p50
0.000469
s
p95
0.000834
s
p99
0.000834
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.195782
s
SD
0.182302
s
p50
0.246625
s
p95
0.620024
s
p99
0.620024
s
Host function call duration: '__hc__must_get_agent_activity_1'
Duration of this host function call invoked from within WASM.
mean
0.141602
s
SD
0.113212
s
p50
0.05395
s
p95
0.401163
s
p99
0.519768
s
Host function call duration: '__hc__query_1'
Duration of this host function call invoked from within WASM.
mean
0.016866
s
SD
0.01311
s
p50
0.009918
s
p95
0.042503
s
p99
0.07304
s
Host function call duration: '__hc__random_bytes_1'
Duration of this host function call invoked from within WASM.
mean
9e-06
s
SD
1e-06
s
p50
9e-06
s
p95
1.1e-05
s
p99
1.1e-05
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.001765
s
SD
0.000906
s
p50
0.002528
s
p95
0.004418
s
p99
0.004734
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.000117
s
SD
6.8e-05
s
p50
0.000108
s
p95
0.000308
s
p99
0.000345
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
47.08
s
mean
435.21
s
SD
252.15
s
p95
838.18
s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
36.710003
s
SD
18.64395
s
p50
0
s
p95
45.22089
s
p99
60.72617
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
1.131214
s
SD
0.877753
s
p50
0.002561
s
p95
2.497377
s
p99
2.754967
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.096443
s
SD
0.078925
s
p50
0.001288
s
p95
0.231898
s
p99
0.301537
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.058637
s
SD
0.063597
s
p50
0.014399
s
p95
0.170883
s
p99
0.327439
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
2.711469
s
SD
3.066292
s
p50
0.004221
s
p95
7.941659
s
p99
12.521126
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
16.994471
s
SD
11.761959
s
p50
0.003497
s
p95
26.701155
s
p99
43.975791
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.00166
s
SD
0.001215
s
p50
0.000339
s
p95
0.003521
s
p99
0.005822
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.005021
s
SD
0.044184
s
p50
0.002143
s
p95
0.082969
s
p99
0.250549
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000364
s
SD
0.000112
s
p50
0.000398
s
p95
0.000564
s
p99
0.000674
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
2.895947
s
SD
29.540037
s
p50
0.178055
s
p95
60.001285
s
p99
60.001285
s
P2P request roundtrip: 'get_links'
Time spent sending a get_links request and awaiting its response.
mean
0.146829
s
SD
17.486124
s
p50
0.219842
s
p95
60.001575
s
p99
60.001903
s
P2P request roundtrip: 'must_get_agent_activity'
Time spent sending a must_get_agent_activity request and awaiting its response.
mean
0.155047
s
SD
26.960171
s
p50
0.135327
s
p95
60.001586
s
p99
60.002101
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.198433
s
SD
5.080058
s
p50
0.239252
s
p95
15.242172
s
p99
15.242172
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
17.76
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
7.50
KiB/s
mean
379.07
KiB/s
SD
1.02
MiB/s
p95
1.75
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
8.61
KiB/s
mean
89.64
KiB/s
SD
119.64
KiB/s
p95
340.46
KiB/s
Total bytes received
Total bytes received on primary network interface
count
4.37
GiB
mean
5.03
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
1.03
GiB
mean
1.19
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 94.3%
Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected
Warning Moderate swap usage (19.4% swap used)
System overload anomaly
System overload anomaly was detected
Detected
Warning 7% of hosts overloaded (load5/ncpus > 1.0)
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
2.17
%
mean
23.1
%
SD
23.9
%
p95
81.25
%
System
5.34
%
CPU percentiles
CPU usage percentiles
p50
12.60
%
p95
81.25
%
p99
94.34
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
5
hosts
mean time
132.46
s
Memory used percentage
Percentage of memory used
p5
6.86
%
mean
17.99
%
SD
18.37
%
p95
78.62
%
Memory available percentage
Percentage of available memory
p5
21.38
%
mean
82.01
%
SD
18.37
%
p95
93.14
%
Max host memory used
Maximum memory usage percentage across all hosts
max
78.74
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
19.42
%
Memory growth rate
Rate of memory growth over time
growth
0.35
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.27
MiB/s
Disk write throughput
Disk write throughput in MiB/s
113.74
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/0/2
hosts
Mount Point
/efi-boot0/1
hosts
Mount Point
/etc/hostname0/11
hosts
Mount Point
/etc/hosts0/11
hosts
Mount Point
/etc/nomad.d/client-meta.json0/15
hosts
Mount Point
/etc/nomad.d/meta.json0/1
hosts
Mount Point
/etc/resolv.conf0/11
hosts
Mount Point
/nix/store0/2
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
1.71
5 min
1.80
15 min
2.04
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
7.14
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
4.7938
%
SD
11.2862
%
p95
28.07
%
60 second average
4.9425
%
300 second average
5.5986
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
4.2631
%
SD
13.7393
%
p95
42.34
%
60 second average
4.6620
%
300 second average
5.9489
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
3.365
%
SD
10.6251
%
p95
32.85
%
60 second average
3.6791
%
300 second average
4.5009
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.35
%
mean
12.93
%
SD
15.52
%
p95
45.88
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
278.68
KiB
mean
423.92
KiB
SD
124.24
KiB
p95
496.29
KiB
Holochain process threads
Number of threads in Holochain process
p5
9
threads
mean
30.05
threads
SD
8.37
threads
p95
37
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
8
file descriptors
mean
65.96
file descriptors
SD
17.9
file descriptors
p95
79
file descriptors
Remote Call Rate
↑ Back to indexTest the throughput of remote_call operations. Each agent in this scenario waits for a certain number of peers to be available or for up to two minutes, whichever happens first, before starting its behaviour.
Started
Thu, 22 Oct 2026 09:42:08 UTC
Duration
900s
Peer count
10
Peer count at end
10
Behaviours
-
default(1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
remote_call_rate_39722287437_1
Dispatch timing
The time between sending a remote call and the remote handler being invoked.
Mean
mean
0.078579
s
SD
0.165266
s
Max
Highest per-partition mean latency
max mean
0.102542
s
Min
Lowest per-partition mean latency
min mean
0.053541
s
Round-trip timing
The total elapsed time to get a response to the client.
Mean
mean
0.183613
s
SD
0.336222
s
Max
Highest per-partition mean latency
max mean
0.235045
s
Min
Lowest per-partition mean latency
min mean
0.162221
s
Error count
The total number of errors accumulated during the run.
3
Holochain MetricsClick to toggle visibility.
WASM usage: 'remote_call::call_echo_timestamp'
Metered usage count of the WASM ribosome for this zome function.
total
132312348
over 883.898749988s
mean rate
3.953362716e+07
/s
std
6.015976179e+07
/s
p5
1.35469181e+06
/s
p95
1.4830508568e+08
/s
peak
4.8650410547e+08
/s
WASM usage: 'remote_call::echo_timestamp'
Metered usage count of the WASM ribosome for this zome function.
total
32393629
over 883.898749988s
mean rate
3.245233292e+07
/s
std
7.135073624e+07
/s
p5
57852.5
/s
p95
1.9703177543e+08
/s
peak
2.012499961e+08
/s
WASM usage: 'remote_call::init'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 885.745012081s
Not enough time data to show a trend.
Zome call duration: 'remote_call::call_echo_timestamp'
Duration of this zome call as measured by Holochain internally.
mean
0.115774
s
SD
0.041357
s
p50
0.091386
s
p95
0.198941
s
p99
0.202957
s
Zome call duration: 'remote_call::echo_timestamp'
Duration of this zome call as measured by Holochain internally.
mean
0.003162
s
SD
0.000529
s
p50
0.00342
s
p95
0.003897
s
p99
0.004395
s
WASM call duration: 'remote_call::call_echo_timestamp'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.114672
s
SD
0.041419
s
p50
0.090104
s
p95
0.197992
s
p99
0.202172
s
WASM call duration: 'remote_call::echo_timestamp'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.002095
s
SD
0.000357
s
p50
0.002247
s
p95
0.002592
s
p99
0.003022
s
WASM call duration: 'remote_call::init'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.002274
s
SD
0.000574
s
p50
0.002532
s
p95
0.002831
s
p99
0.002831
s
Host function call duration: '__hc__call_1'
Duration of this host function call invoked from within WASM.
mean
0.112228
s
SD
0.041458
s
p50
0.087502
s
p95
0.195631
s
p99
0.200204
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000362
s
SD
8.9e-05
s
p50
0.000373
s
p95
0.000525
s
p99
0.000525
s
Host function call duration: '__hc__sys_time_1'
Duration of this host function call invoked from within WASM.
mean
1e-05
s
SD
3e-06
s
p50
1e-05
s
p95
1.5e-05
s
p99
1.5e-05
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.000277
s
SD
5.5e-05
s
p50
0.000241
s
p95
0.000347
s
p99
0.000363
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.000141
s
SD
5.8e-05
s
p50
0.000135
s
p95
0.00024
s
p99
0.00024
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
46.45
s
mean
438.1
s
SD
252.3
s
p95
836.4
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
110
over 885.744951245s
mean rate
2.54
/s
std
25.47
/s
p5
0
/s
p95
13.12
/s
peak
687.99
/s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
1.083842
s
SD
0.71618
s
p50
0.811392
s
p95
3.057488
s
p99
3.057488
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.002316
s
SD
0.000524
s
p50
0.002391
s
p95
0.003607
s
p99
0.003607
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.102866
s
SD
0.034185
s
p50
0.11169
s
p95
0.193625
s
p99
0.193625
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.005941
s
SD
0.0017
s
p50
0.006037
s
p95
0.008934
s
p99
0.011748
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.296292
s
SD
0.33472
s
p50
0.216249
s
p95
1.015911
s
p99
1.10313
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
0.0811
s
SD
0.064691
s
p50
0.033723
s
p95
0.175148
s
p99
0.179742
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.000141
s
SD
2.8e-05
s
p50
0.000142
s
p95
0.000196
s
p99
0.000224
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.001088
s
SD
0.00036
s
p50
0.001245
s
p95
0.002049
s
p99
0.002049
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000414
s
SD
6.7e-05
s
p50
0.000431
s
p95
0.000512
s
p99
0.000587
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
0.061411
s
SD
0.043583
s
p50
0.08628
s
p95
0.134524
s
p99
0.151729
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.090467
s
SD
0.061493
s
p50
0.037677
s
p95
0.177571
s
p99
0.19265
s
P2P request roundtrip: 'call_remote'
Time spent sending a call_remote request and awaiting its response.
mean
0.111693
s
SD
0.041458
s
p50
0.086908
s
p95
0.195191
s
p99
0.199762
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
14.77
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
6.12
KiB/s
mean
376.69
KiB/s
SD
1.36
MiB/s
p95
1.68
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
4.97
KiB/s
mean
25.13
KiB/s
SD
35.13
KiB/s
p95
114.45
KiB/s
Total bytes received
Total bytes received on primary network interface
count
3.12
GiB
mean
3.63
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
213.29
MiB
mean
248.19
KiB/s
CPU spike anomaly
CPU spike anomaly was detected
NotDetected
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 1.18 MiB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
NotDetected
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
4.02
%
mean
21.6
%
SD
24.82
%
p95
76.14
%
System
6.84
%
CPU percentiles
CPU usage percentiles
p50
9.91
%
p95
76.14
%
p99
80.50
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
3
hosts
mean time
40.91
s
Memory used percentage
Percentage of memory used
p5
4.07
%
mean
18.61
%
SD
6.51
%
p95
27.2
%
Memory available percentage
Percentage of available memory
p5
72.8
%
mean
81.39
%
SD
6.51
%
p95
95.93
%
Max host memory used
Maximum memory usage percentage across all hosts
max
24.00
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00
%
Memory growth rate
Rate of memory growth over time
growth
1.18
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.00
MiB/s
Disk write throughput
Disk write throughput in MiB/s
26.32
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/0/6
hosts
Mount Point
/efi-boot0/5
hosts
Mount Point
/etc/nomad.d/client-meta.json0/4
hosts
Mount Point
/nix/store0/6
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
2.42
5 min
1.69
15 min
0.86
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
0.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
7.3104
%
SD
13.0018
%
p95
37.52
%
60 second average
6.8896
%
300 second average
4.9759
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0004
%
300 second average
0.0014
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0003
%
300 second average
0.0012
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
26.8113
%
SD
25.6327
%
p95
60.34
%
60 second average
25.1052
%
300 second average
17.5768
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
24.645
%
SD
23.8128
%
p95
56.59
%
60 second average
23.0500
%
300 second average
16.1410
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0
%
mean
11.38
%
SD
22.07
%
p95
66.47
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
72.60
KiB
mean
239.52
KiB
SD
98.00
KiB
p95
414.57
KiB
Holochain process threads
Number of threads in Holochain process
p5
1
threads
mean
86.14
threads
SD
258.95
threads
p95
208
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
0
file descriptors
mean
202.2
file descriptors
SD
566.33
file descriptors
p95
479
file descriptors
Remote Signals
↑ Back to indexThis scenario tests the throughput of remote_signals operations.
Started
Thu, 22 Oct 2026 09:44:10 UTC
Duration
900s
Peer count
10
Peer count at end
10
Behaviours
-
default(1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
remote_signals_39722287437_1
Round trip time
The time from origin signal dispatch to origin receive of the remote side's response signal.
mean
0.157963
s
SD
0.298395
s
p50
0.126592
s
p95
0.271963
s
p99
1.419947
s
Timeout rate
The proportion of remote signal round-trips that timed out.
0
Holochain MetricsClick to toggle visibility.
WASM usage: 'remote_signal::init'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 875.742528111s
Not enough time data to show a trend.
WASM usage: 'remote_signal::recv_remote_signal'
Metered usage count of the WASM ribosome for this zome function.
total
30333170
over 875.742528111s
mean rate
3.387378421e+07
/s
std
1.2322044865e+08
/s
p5
7185.54
/s
p95
1.2928648535e+08
/s
peak
2.29128653373e+09
/s
WASM usage: 'remote_signal::signal_request'
Metered usage count of the WASM ribosome for this zome function.
total
12615616
over 875.742528111s
mean rate
9.100996951e+07
/s
std
9.1456856166e+08
/s
p5
0
/s
p95
1.4417009847e+08
/s
peak
1.87197530864e+10
/s
Zome call duration: 'remote_signal::recv_remote_signal'
Duration of this zome call as measured by Holochain internally.
mean
0.003198
s
SD
0.000628
s
p50
0.00359
s
p95
0.003914
s
p99
0.005047
s
Zome call duration: 'remote_signal::signal_request'
Duration of this zome call as measured by Holochain internally.
mean
0.00317
s
SD
0.000637
s
p50
0.003548
s
p95
0.003925
s
p99
0.005228
s
WASM call duration: 'remote_signal::init'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.002753
s
SD
0.000384
s
p50
0.002839
s
p95
0.003214
s
p99
0.003214
s
WASM call duration: 'remote_signal::recv_remote_signal'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.002187
s
SD
0.000411
s
p50
0.002442
s
p95
0.002758
s
p99
0.003599
s
WASM call duration: 'remote_signal::signal_request'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.00216
s
SD
0.000421
s
p50
0.002385
s
p95
0.002736
s
p99
0.003708
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000355
s
SD
0.00012
s
p50
0.000309
s
p95
0.00058
s
p99
0.00058
s
Host function call duration: '__hc__emit_signal_1'
Duration of this host function call invoked from within WASM.
mean
8.1e-05
s
SD
2.2e-05
s
p50
8e-05
s
p95
0.000111
s
p99
0.000121
s
Host function call duration: '__hc__send_remote_signal_1'
Duration of this host function call invoked from within WASM.
mean
0.000104
s
SD
4.6e-05
s
p50
0.000108
s
p95
0.000181
s
p99
0.000204
s
Host function call duration: '__hc__sys_time_1'
Duration of this host function call invoked from within WASM.
mean
6e-06
s
SD
1e-06
s
p50
6e-06
s
p95
8e-06
s
p99
9e-06
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.000416
s
SD
0.000384
s
p50
0.000222
s
p95
0.001197
s
p99
0.001197
s
Emitted local signals
Count of local signals emitted via the emit_signal host function.
total
1679
over 875.742523224s
mean rate
1782.73
/s
std
3473.34
/s
p5
0
/s
p95
8524.11
/s
peak
33975.28
/s
Sent remote signals
Count of remote signals sent via the send_remote_signal host function.
total
1679
over 875.742528476s
mean rate
1723.33
/s
std
3248.96
/s
p5
0
/s
p95
8371.43
/s
peak
33876.3
/s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.000192
s
SD
0.000122
s
p50
0.000146
s
p95
0.000534
s
p99
0.000534
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
45.67
s
mean
441.75
s
SD
254.66
s
p95
836.03
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
103
over 885.742798685s
mean rate
471.44
/s
std
2141.34
/s
p5
0
/s
p95
2009.43
/s
peak
14999.15
/s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
1.114792
s
SD
0.72248
s
p50
0.811772
s
p95
2.997006
s
p99
3.405852
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.003233
s
SD
0.000655
s
p50
0.003317
s
p95
0.004309
s
p99
0.006465
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.106219
s
SD
0.039622
s
p50
0.107555
s
p95
0.169146
s
p99
0.187377
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.007317
s
SD
0.002189
s
p50
0.007221
s
p95
0.010791
s
p99
0.018595
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.089833
s
SD
0.126742
s
p50
0.020778
s
p95
0.351438
s
p99
0.435811
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
0.100287
s
SD
0.040113
s
p50
0.083872
s
p95
0.151152
s
p99
0.195469
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.000152
s
SD
3.9e-05
s
p50
0.000154
s
p95
0.000252
s
p99
0.000294
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.001501
s
SD
0.000808
s
p50
0.001546
s
p95
0.004366
s
p99
0.004373
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000552
s
SD
8.3e-05
s
p50
0.000586
s
p95
0.000647
s
p99
0.000721
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
0.053628
s
SD
0.029804
s
p50
0.053081
s
p95
0.109353
s
p99
0.124602
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.10223
s
SD
0.03259
s
p50
0.080847
s
p95
0.149078
s
p99
0.149078
s
P2P received remote signals
Count of remote signals received.
total
1679
over 875.742592874s
mean rate
1703.5
/s
std
3250.51
/s
p5
0
/s
p95
7554.02
/s
peak
37471.19
/s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
25.66
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
7.31
KiB/s
mean
505.17
KiB/s
SD
1.61
MiB/s
p95
2.92
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
6.78
KiB/s
mean
34.35
KiB/s
SD
42.10
KiB/s
p95
124.17
KiB/s
Total bytes received
Total bytes received on primary network interface
count
2.13
GiB
mean
2.46
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
148.59
MiB
mean
170.96
KiB/s
CPU spike anomaly
CPU spike anomaly was detected
NotDetected
Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
Detected
Warning 20% of hosts overloaded (load5/ncpus > 1.0)
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
8.85
%
mean
36.85
%
SD
28.97
%
p95
79.51
%
System
11.19
%
CPU percentiles
CPU usage percentiles
p50
18.21
%
p95
79.51
%
p99
82.82
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
3
hosts
mean time
60.67
s
Memory used percentage
Percentage of memory used
p5
19.33
%
mean
23.73
%
SD
3.09
%
p95
27.58
%
Memory available percentage
Percentage of available memory
p5
72.42
%
mean
76.27
%
SD
3.09
%
p95
80.67
%
Max host memory used
Maximum memory usage percentage across all hosts
max
24.48
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00
%
Memory growth rate
Rate of memory growth over time
growth
0.48
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.02
MiB/s
Disk write throughput
Disk write throughput in MiB/s
14.42
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/0/2
hosts
Mount Point
/efi-boot0/2
hosts
Mount Point
/etc/nomad.d/client-meta.json0/3
hosts
Mount Point
/nix/store0/2
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
2.68
5 min
1.99
15 min
1.06
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
20.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0.41
%
mean
15.2273
%
SD
15.6865
%
p95
42.97
%
60 second average
14.8625
%
300 second average
11.3282
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0.0045
%
SD
0.0944
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0037
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0.0045
%
SD
0.0944
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0031
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
18.4521
%
SD
23.0593
%
p95
54.42
%
60 second average
17.6330
%
300 second average
13.1567
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
16.2153
%
SD
20.8017
%
p95
48.95
%
60 second average
15.5147
%
300 second average
11.5883
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0
%
mean
21.48
%
SD
29.43
%
p95
72.59
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
88.33
KiB
mean
266.34
KiB
SD
143.15
KiB
p95
486.78
KiB
Holochain process threads
Number of threads in Holochain process
p5
1
threads
mean
121.81
threads
SD
333.14
threads
p95
792
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
0
file descriptors
mean
281.61
file descriptors
SD
727.25
file descriptors
p95
1744
file descriptors
Single-write, many-read
↑ Back to indexCreates an entry in the agent setup, then read one's own record back. This tests the maximum read performance for reading back local data when the chain is short.
Started
Thu, 22 Oct 2026 09:48:04 UTC
Duration
300s
Peer count
1
Peer count at end
1
Behaviours
-
default(1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
single_write_many_read_39722287437_1
read call timingThe time it takes to complete the zome call that retrieves the record.
mean
0.018918
s
SD
0.223607
s
p50
0.003142
s
p95
0.004619
s
p99
0.011214
s
read call rateThe number of read zome calls completed per second.
mean
516.39
/s
Error count
The number of errors collected.
0
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.000263
s
SD
1e-06
s
p50
0.000263
s
p95
0.000264
s
p99
0.000264
s
WASM usage: 'crud::create_sample_entry'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 200.002009235s
Not enough time data to show a trend.
WASM usage: 'crud::get_sample_entry'
Metered usage count of the WASM ribosome for this zome function.
total
369016560
over 200.002009235s
mean rate
1.84506423e+06
/s
std
499288.14
/s
p5
917340.55
/s
p95
2.76542995e+06
/s
peak
2.76542995e+06
/s
WASM usage: 'crud::init'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 200.002009235s
Not enough time data to show a trend.
WASM usage: 'crud_integrity::entry_defs'
Metered usage count of the WASM ribosome for this zome function.
total
4404
over 210.002028803s
mean rate
20.97
/s
std
93.79
/s
p5
0
/s
p95
0
/s
peak
440.4
/s
Zome call duration: 'crud::create_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.00457
s
SD
0
s
p50
0.00457
s
p95
0.00457
s
p99
0.00457
s
Zome call duration: 'crud::get_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.00179
s
SD
7e-06
s
p50
0.001791
s
p95
0.001797
s
p99
0.001798
s
WASM call duration: 'crud::create_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.003855
s
SD
0
s
p50
0.003855
s
p95
0.003855
s
p99
0.003855
s
WASM call duration: 'crud::get_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001247
s
SD
5e-06
s
p50
0.001247
s
p95
0.001252
s
p99
0.001253
s
WASM call duration: 'crud::init'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.003148
s
SD
0
s
p50
0.003148
s
p95
0.003148
s
p99
0.003148
s
WASM call duration: 'crud_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.002164
s
SD
0.000178
s
p50
0.002144
s
p95
0.002144
s
p99
0.002997
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000461
s
SD
0.000138
s
p50
0.000599
s
p95
0.000599
s
p99
0.000599
s
Host function call duration: '__hc__get_1'
Duration of this host function call invoked from within WASM.
mean
0.000295
s
SD
1e-06
s
p50
0.000295
s
p95
0.000296
s
p99
0.000296
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.001996
s
SD
0
s
p50
0.001996
s
p95
0.001996
s
p99
0.001996
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.000173
s
SD
0
s
p50
0.000173
s
p95
0.000173
s
p99
0.000173
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
13.27
s
mean
108.27
s
SD
63.44
s
p95
203.28
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 200.0020101s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.001611
s
SD
0
s
p50
0.001611
s
p95
0.001611
s
p99
0.001611
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.001805
s
SD
0
s
p50
0.001805
s
p95
0.001805
s
p99
0.001805
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.004576
s
SD
0.000575
s
p50
0.004483
s
p95
0.005496
s
p99
0.005496
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.002719
s
SD
0
s
p50
0.002719
s
p95
0.002719
s
p99
0.002719
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
0.003194
s
SD
0
s
p50
0.003194
s
p95
0.003194
s
p99
0.003194
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
5.1e-05
s
SD
0
s
p50
5e-05
s
p95
5.1e-05
s
p99
5.1e-05
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.001832
s
SD
0.000185
s
p50
0.00174
s
p95
0.002095
s
p99
0.002095
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.00054
s
SD
0
s
p50
0.00054
s
p95
0.00054
s
p99
0.00054
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
9.60
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
6.25
KiB/s
mean
224.48
KiB/s
SD
552.93
KiB/s
p95
391.87
KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
4.42
KiB/s
mean
13.15
KiB/s
SD
15.66
KiB/s
p95
21.87
KiB/s
Total bytes received
Total bytes received on primary network interface
count
50.42
MiB
mean
224.48
KiB/s
Total bytes sent
Total bytes sent on primary network interface
count
2.95
MiB
mean
13.15
KiB/s
CPU spike anomaly
CPU spike anomaly was detected
NotDetected
Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
NotDetected
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
10.7
%
mean
15.39
%
SD
7.08
%
p95
18.06
%
System
5.80
%
CPU percentiles
CPU usage percentiles
p50
14.01
%
p95
18.06
%
p99
47.38
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
0
hosts
mean time
0.00
s
Memory used percentage
Percentage of memory used
p5
17.29
%
mean
21.31
%
SD
1.36
%
p95
22.29
%
Memory available percentage
Percentage of available memory
p5
77.71
%
mean
78.69
%
SD
1.36
%
p95
82.71
%
Max host memory used
Maximum memory usage percentage across all hosts
max
21.31
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00
%
Memory growth rate
Rate of memory growth over time
growth
0.76
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.00
MiB/s
Disk write throughput
Disk write throughput in MiB/s
4.53
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/0/1
hosts
Mount Point
/nix/store0/1
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
5.18
5 min
3.18
15 min
1.45
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
0.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0.52
%
mean
1.0554
%
SD
0.5165
%
p95
1.78
%
60 second average
1.0588
%
300 second average
0.6138
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
45.86
%
mean
56.7404
%
SD
5.5169
%
p95
66
%
60 second average
54.4508
%
300 second average
35.5354
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
40.29
%
mean
48.8075
%
SD
5.5876
%
p95
58.91
%
60 second average
47.0567
%
300 second average
31.3400
%
Holochain process CPU usage
CPU usage by Holochain process
p5
1.28
%
mean
6.91
%
SD
6.94
%
p95
8.56
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
72.60
KiB
mean
220.06
KiB
SD
60.73
KiB
p95
281.43
KiB
Holochain process threads
Number of threads in Holochain process
p5
1
threads
mean
26.19
threads
SD
17.78
threads
p95
80
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
7
file descriptors
mean
64.81
file descriptors
SD
47.72
file descriptors
p95
212
file descriptors
Unyt Proposal
↑ Back to indexThis scenario tests the performance of the Unyt negotiated transaction flow.
Agents create proposals, exchange counter-proposals, commit to agreed terms, and
finalize or reject transactions. There are three roles:
initiate (progenitor that bootstraps the network),
user (full-arc agents), and
zero_user (zero-arc agents).
Negotiation metrics are broken down by arc type to compare full-arc and zero-arc behaviour.
Started
Thu, 22 Oct 2026 10:54:16 UTC
Duration
900s
Peer count
30
Peer count at end
26
Behaviours
-
initiate(1 agent) -
user(1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
unyt_proposal_39722287437_1
Global definition propagation time
Time (seconds since session start) at which each agent first detects network initialization, partitioned by agent and arc type.
Mean
mean
18.36
s
SD
0
s
Max
Highest per-partition mean latency
max mean
103
s
Min
Lowest per-partition mean latency
min mean
0
s
Proposal round-trip time (accepted)
Duration from proposal creation to final acceptance/receipt, partitioned by arc type and agent.
Mean
mean
474.353003
s
SD
0.054875
s
Max
Highest per-partition mean latency
max mean
696.042869
s
Min
Lowest per-partition mean latency
min mean
252.663137
s
Proposal round-trip time (rejected)
Duration from proposal creation to rejection/reclaim, partitioned by arc type and agent.
Mean
mean
0
s
SD
0
s
Max
Highest per-partition mean latency
max mean
0
s
Min
Lowest per-partition mean latency
min mean
0
s
Not enough time data to show a trend.
Negotiation rounds
Number of counter-proposal rounds before a proposal reaches commitment, partitioned by agent and arc type. Lower values indicate faster convergence.
Mean
mean
0
s
SD
0
s
Max
Highest per-partition mean latency
max mean
0
s
Min
Lowest per-partition mean latency
min mean
0
s
Sync lag — proposals
Delay between proposal publish and first observation by the counterparty, partitioned by arc type and agent.
Mean
mean
326.220604
s
SD
166.176284
s
Max
Highest per-partition mean latency
max mean
460.197948
s
Min
Lowest per-partition mean latency
min mean
180.032754
s
Sync lag — commitments
Delay between commitment publish and first observation by the counterparty, partitioned by arc type and agent.
Mean
mean
373.969709
s
SD
0.122222
s
Max
Highest per-partition mean latency
max mean
538.488803
s
Min
Lowest per-partition mean latency
min mean
209.450614
s
Sync lag — accepts
Delay between accept publish and first observation by the counterparty, partitioned by arc type and agent.
Mean
mean
0
s
SD
0
s
Max
Highest per-partition mean latency
max mean
0
s
Min
Lowest per-partition mean latency
min mean
0
s
Not enough time data to show a trend.
Sync lag — rejects
Delay between reject publish and first observation by the counterparty, partitioned by arc type and agent.
Mean
mean
0
s
SD
0
s
Max
Highest per-partition mean latency
max mean
0
s
Min
Lowest per-partition mean latency
min mean
0
s
Not enough time data to show a trend.
Zome call — create_proposal
Duration of create_proposal zome calls per agent.
Mean
mean
4.761912
s
SD
3.751515
s
Max
Highest per-partition mean latency
max mean
11.647889
s
Min
Lowest per-partition mean latency
min mean
2.668865
s
Zome call — create_counter_proposal
Duration of create_counter_proposal zome calls per agent.
Mean
mean
7.351361
s
SD
1.134556
s
Max
Highest per-partition mean latency
max mean
11.586956
s
Min
Lowest per-partition mean latency
min mean
4.093813
s
Zome call — create_commit_to_proposal
Duration of create_commit_to_proposal zome calls per agent.
Mean
mean
7.144599
s
SD
2.818335
s
Max
Highest per-partition mean latency
max mean
13.745563
s
Min
Lowest per-partition mean latency
min mean
3.742411
s
Zome call — create_accept
Duration of create_accept zome calls per agent.
Mean
mean
2.856196
s
SD
0.100703
s
Max
Highest per-partition mean latency
max mean
3.145651
s
Min
Lowest per-partition mean latency
min mean
2.56674
s
Zome call — create_reject_proposal
Duration of create_reject_proposal zome calls per agent.
Mean
mean
3.578873
s
SD
0.269687
s
Max
Highest per-partition mean latency
max mean
7.19236
s
Min
Lowest per-partition mean latency
min mean
0.877471
s
Zome call — get_actionable_transactions
Duration of get_actionable_transactions zome calls per agent. Called every iteration.
Mean
mean
0.151306
s
SD
0.228409
s
Max
Highest per-partition mean latency
max mean
1.730384
s
Min
Lowest per-partition mean latency
min mean
0.021661
s
UI action list refresh: Duration
Duration of the UI action list refresh per behaviour iteration (seconds).
mean
0.316225
s
SD
1.124674
s
p50
0.14593
s
p95
0.812185
s
p99
2.119368
s
UI action list refresh: Failed calls
UI action list refresh sub-calls that failed, as a per-agent counter. Total is the run-wide count across all agents.
Total
0
Agents affected
0 / 25
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
0
Max
Highest count in any single partition
0
Min
Lowest count in any single partition
0
UI transaction detail item refresh: Duration
Duration of refreshing a single transaction detail item per behaviour iteration (seconds).
mean
2.629165
s
SD
3.607343
s
p50
2.085726
s
p95
5.782131
s
p99
10.618671
s
UI transaction detail refresh: Duration
Duration of refreshing all watched transaction details per behaviour iteration (seconds).
mean
12.84103
s
SD
12.678337
s
p50
8.949611
s
p95
40.245647
s
p99
70.215555
s
UI transaction detail refresh: Transactions processed
Transactions processed during the full detail refresh, as a per-agent counter.
Total
313
Agents affected
15 / 16
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
20
Max
Highest count in any single partition
60
Min
Lowest count in any single partition
0
UI transaction detail refresh: Primary transaction total calls
Primary transaction fetches made during the full detail refresh, as a per-agent counter.
Total
626
Agents affected
15 / 16
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
39
Max
Highest count in any single partition
120
Min
Lowest count in any single partition
0
UI transaction detail refresh: Primary transaction failed calls
Primary transaction fetches that failed during the full detail refresh, as a per-agent counter.
Total
2
Agents affected
2 / 16
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
0
Max
Highest count in any single partition
1
Min
Lowest count in any single partition
0
UI transaction detail refresh: Related transaction total calls
Related transaction fetches made during the full detail refresh, as a per-agent counter.
Total
0
Agents affected
0 / 16
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
0
Max
Highest count in any single partition
0
Min
Lowest count in any single partition
0
UI transaction detail refresh: Related transaction failed calls
Related transaction fetches that failed during the full detail refresh, as a per-agent counter.
Total
0
Agents affected
0 / 16
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
0
Max
Highest count in any single partition
0
Min
Lowest count in any single partition
0
Zome call errors
Total number of zome call errors observed during the run.
781
Completed transaction accepts
Completed transaction accepts at teardown.
sum
3
mean
0.1
min
0
max
2
n
30
Completed transaction spends
Completed transaction spends at teardown.
sum
140
mean
4.67
min
0
max
27
n
30
Completed transaction RAVEs
Completed transaction RAVEs at teardown.
sum
3882
mean
129.4
min
0
max
222
n
30
Actionable transaction proposals
Remaining actionable transaction proposals at teardown. Non-zero sum indicates unresolved negotiations.
sum
138
mean
4.6
min
0
max
18
n
30
Actionable transaction commitments
Remaining actionable transaction commitments at teardown. Non-zero sum indicates unresolved negotiations.
sum
2
mean
0.07
min
0
max
1
n
30
Actionable transaction accepts
Remaining actionable transaction accepts at teardown. Non-zero sum indicates unresolved negotiations.
sum
0
mean
0
min
0
max
0
n
30
Actionable transaction rejects
Remaining actionable transaction rejects at teardown. Non-zero sum indicates unresolved negotiations.
sum
0
mean
0
min
0
max
0
n
30
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.003107
s
SD
0.003115
s
p50
0.002077
s
p95
0.008246
s
p99
0.015873
s
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
18
over 647.246467112s
mean rate
1415.27
/s
std
5022.22
/s
p5
0
/s
p95
10156.63
/s
peak
108057.79
/s
Zome call duration: 'transactor::create_accept'
Duration of this zome call as measured by Holochain internally.
mean
0.35176
s
SD
0.015234
s
p50
0.353364
s
p95
0.353364
s
p99
0.353364
s
Zome call duration: 'transactor::create_code_template'
Duration of this zome call as measured by Holochain internally.
mean
0.025088
s
SD
0.020959
s
p50
0.021053
s
p95
0.036027
s
p99
0.118432
s
Zome call duration: 'transactor::create_commit_to_proposal'
Duration of this zome call as measured by Holochain internally.
mean
0.307552
s
SD
0.198349
s
p50
0.288477
s
p95
0.783285
s
p99
0.930383
s
Zome call duration: 'transactor::create_counter_proposal'
Duration of this zome call as measured by Holochain internally.
mean
0.306126
s
SD
0.216199
s
p50
0.318654
s
p95
0.822174
s
p99
0.995567
s
Zome call duration: 'transactor::create_proposal'
Duration of this zome call as measured by Holochain internally.
mean
0.287116
s
SD
0.211068
s
p50
0.245547
s
p95
0.58285
s
p99
1.329108
s
Zome call duration: 'transactor::create_reject_proposal'
Duration of this zome call as measured by Holochain internally.
mean
0.160604
s
SD
0.156717
s
p50
0.141277
s
p95
0.345819
s
p99
0.921623
s
Zome call duration: 'transactor::create_smart_agreement'
Duration of this zome call as measured by Holochain internally.
mean
0.024898
s
SD
0
s
p50
0.024898
s
p95
0.024898
s
p99
0.024898
s
Zome call duration: 'transactor::get_actionable_transactions'
Duration of this zome call as measured by Holochain internally.
mean
0.03104
s
SD
0.028849
s
p50
0.024071
s
p95
0.06983
s
p99
0.169241
s
Zome call duration: 'transactor::get_all_notification_links'
Duration of this zome call as measured by Holochain internally.
mean
0.034105
s
SD
0.030837
s
p50
0.026289
s
p95
0.111377
s
p99
0.141343
s
Zome call duration: 'transactor::get_code_templates_lib'
Duration of this zome call as measured by Holochain internally.
mean
0.040404
s
SD
0.031256
s
p50
0.03256
s
p95
0.072513
s
p99
0.173392
s
Zome call duration: 'transactor::get_current_global_definition'
Duration of this zome call as measured by Holochain internally.
mean
0.01347
s
SD
0.012179
s
p50
0.011154
s
p95
0.038025
s
p99
0.060758
s
Zome call duration: 'transactor::get_history'
Duration of this zome call as measured by Holochain internally.
mean
0.14025
s
SD
0.09183
s
p50
0.122029
s
p95
0.274955
s
p99
0.589251
s
Zome call duration: 'transactor::get_incoming_raves'
Duration of this zome call as measured by Holochain internally.
mean
0.007942
s
SD
0.005013
s
p50
0.006863
s
p95
0.013967
s
p99
0.029749
s
Zome call duration: 'transactor::get_ledger'
Duration of this zome call as measured by Holochain internally.
mean
0.075663
s
SD
0.04381
s
p50
0.069136
s
p95
0.173649
s
p99
0.277811
s
Zome call duration: 'transactor::get_my_current_applied_credit_limit'
Duration of this zome call as measured by Holochain internally.
mean
0.047278
s
SD
0.03079
s
p50
0.04109
s
p95
0.103509
s
p99
0.201348
s
Zome call duration: 'transactor::get_requests_to_execute_agreements'
Duration of this zome call as measured by Holochain internally.
mean
0.008485
s
SD
0.007744
s
p50
0.00689
s
p95
0.01721
s
p99
0.04426
s
Zome call duration: 'transactor::get_smart_agreements_for_code_template'
Duration of this zome call as measured by Holochain internally.
mean
0.01207
s
SD
0.013577
s
p50
0.007489
s
p95
0.021064
s
p99
0.074618
s
Zome call duration: 'transactor::get_sorted_requests_to_spend'
Duration of this zome call as measured by Holochain internally.
mean
0.008267
s
SD
0.006679
s
p50
0.007029
s
p95
0.016649
s
p99
0.03734
s
Zome call duration: 'transactor::get_status'
Duration of this zome call as measured by Holochain internally.
mean
1.909531
s
SD
1.229588
s
p50
1.813195
s
p95
3.878357
s
p99
6.723332
s
Zome call duration: 'transactor::get_transaction'
Duration of this zome call as measured by Holochain internally.
mean
0.020932
s
SD
0.015047
s
p50
0.020752
s
p95
0.060982
s
p99
0.068935
s
Zome call duration: 'transactor::initialize_global_definition'
Duration of this zome call as measured by Holochain internally.
mean
0.066588
s
SD
0
s
p50
0.066588
s
p95
0.066588
s
p99
0.066588
s
Zome call duration: 'transactor::recv_remote_signal'
Duration of this zome call as measured by Holochain internally.
mean
0.015717
s
SD
0.011641
s
p50
0.013086
s
p95
0.029825
s
p99
0.07049
s
WASM call duration: 'transactor::create_accept'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.350264
s
SD
0.015106
s
p50
0.351854
s
p95
0.351854
s
p99
0.351854
s
WASM call duration: 'transactor::create_code_template'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.022752
s
SD
0.019767
s
p50
0.018827
s
p95
0.032754
s
p99
0.111491
s
WASM call duration: 'transactor::create_commit_to_proposal'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.306296
s
SD
0.198103
s
p50
0.287208
s
p95
0.781003
s
p99
0.928836
s
WASM call duration: 'transactor::create_counter_proposal'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.304946
s
SD
0.215912
s
p50
0.317466
s
p95
0.821028
s
p99
0.993856
s
WASM call duration: 'transactor::create_proposal'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.285355
s
SD
0.210578
s
p50
0.244186
s
p95
0.580349
s
p99
1.325438
s
WASM call duration: 'transactor::create_reject_proposal'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.159406
s
SD
0.156662
s
p50
0.140207
s
p95
0.344637
s
p99
0.920335
s
WASM call duration: 'transactor::create_smart_agreement'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.022824
s
SD
0
s
p50
0.022824
s
p95
0.022824
s
p99
0.022824
s
WASM call duration: 'transactor::get_actionable_transactions'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.029651
s
SD
0.028712
s
p50
0.022047
s
p95
0.068688
s
p99
0.167613
s
WASM call duration: 'transactor::get_all_notification_links'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.032645
s
SD
0.030913
s
p50
0.024782
s
p95
0.110337
s
p99
0.140308
s
WASM call duration: 'transactor::get_code_templates_lib'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.038972
s
SD
0.030549
s
p50
0.031313
s
p95
0.071021
s
p99
0.168888
s
WASM call duration: 'transactor::get_current_global_definition'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.008434
s
SD
0.010337
s
p50
0.009135
s
p95
0.022122
s
p99
0.059093
s
WASM call duration: 'transactor::get_history'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.138523
s
SD
0.091399
s
p50
0.120697
s
p95
0.272215
s
p99
0.585934
s
WASM call duration: 'transactor::get_incoming_raves'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.0066
s
SD
0.004204
s
p50
0.005746
s
p95
0.012338
s
p99
0.02472
s
WASM call duration: 'transactor::get_ledger'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.074199
s
SD
0.043332
s
p50
0.067532
s
p95
0.171412
s
p99
0.274528
s
WASM call duration: 'transactor::get_my_current_applied_credit_limit'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.045938
s
SD
0.030369
s
p50
0.039886
s
p95
0.101352
s
p99
0.198072
s
WASM call duration: 'transactor::get_requests_to_execute_agreements'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.007118
s
SD
0.006861
s
p50
0.005664
s
p95
0.015556
s
p99
0.038336
s
WASM call duration: 'transactor::get_smart_agreements_for_code_template'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.010451
s
SD
0.012835
s
p50
0.006285
s
p95
0.020003
s
p99
0.069851
s
WASM call duration: 'transactor::get_sorted_requests_to_spend'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.006828
s
SD
0.005854
s
p50
0.005831
s
p95
0.014979
s
p99
0.032915
s
WASM call duration: 'transactor::get_status'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
1.974111
s
SD
1.231447
s
p50
1.811872
s
p95
3.876405
s
p99
6.721078
s
WASM call duration: 'transactor::get_transaction'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.019569
s
SD
0.014455
s
p50
0.019412
s
p95
0.057332
s
p99
0.065313
s
WASM call duration: 'transactor::init'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.018341
s
SD
0.005906
s
p50
0.017499
s
p95
0.035009
s
p99
0.041228
s
WASM call duration: 'transactor::initialize_global_definition'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.065014
s
SD
0
s
p50
0.065014
s
p95
0.065014
s
p99
0.065014
s
WASM call duration: 'transactor::post_commit'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.063097
s
SD
0.042663
s
p50
0.04753
s
p95
0.133457
s
p99
0.230867
s
WASM call duration: 'transactor::recv_remote_signal'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.014144
s
SD
0.01105
s
p50
0.011541
s
p95
0.027635
s
p99
0.066097
s
WASM call duration: 'transactor_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001702
s
SD
0.001489
s
p50
0.001697
s
p95
0.00376
s
p99
0.009564
s
WASM call duration: 'transactor_integrity::genesis_self_check_2'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001756
s
SD
0.000957
s
p50
0.00139
s
p95
0.004004
s
p99
0.004194
s
WASM call duration: 'transactor_integrity::validate'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.030737
s
SD
0.118014
s
p50
0.020014
s
p95
0.319265
s
p99
0.445574
s
Host function call duration: '__hc__agent_info_1'
Duration of this host function call invoked from within WASM.
mean
1.5e-05
s
SD
7.9e-05
s
p50
1.2e-05
s
p95
3.1e-05
s
p99
0.000388
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000699
s
SD
0.001206
s
p50
0.000307
s
p95
0.001635
s
p99
0.007496
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.000677
s
SD
0.00106
s
p50
0.000308
s
p95
0.001159
s
p99
0.003737
s
Host function call duration: '__hc__delete_link_1'
Duration of this host function call invoked from within WASM.
mean
0.000703
s
SD
0.000335
s
p50
0.000598
s
p95
0.001516
s
p99
0.001876
s
Host function call duration: '__hc__dna_info_2'
Duration of this host function call invoked from within WASM.
mean
6e-06
s
SD
6e-06
s
p50
3e-06
s
p95
1.1e-05
s
p99
5e-05
s
Host function call duration: '__hc__emit_signal_1'
Duration of this host function call invoked from within WASM.
mean
8.5e-05
s
SD
0.000111
s
p50
3.7e-05
s
p95
0.000273
s
p99
0.000567
s
Host function call duration: '__hc__get_1'
Duration of this host function call invoked from within WASM.
mean
0.000841
s
SD
0.000757
s
p50
0.000487
s
p95
0.002886
s
p99
0.003761
s
Host function call duration: '__hc__get_agent_activity_1'
Duration of this host function call invoked from within WASM.
mean
0.135028
s
SD
0.219841
s
p50
0.054298
s
p95
0.344479
s
p99
0.741688
s
Host function call duration: '__hc__get_details_1'
Duration of this host function call invoked from within WASM.
mean
0.003635
s
SD
0.003565
s
p50
0.003005
s
p95
0.009969
s
p99
0.01953
s
Host function call duration: '__hc__get_init_properties_1'
Duration of this host function call invoked from within WASM.
mean
0.001666
s
SD
0.004493
s
p50
0.000734
s
p95
0.001638
s
p99
0.023945
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.001079
s
SD
0.001439
s
p50
0.000931
s
p95
0.002903
s
p99
0.009438
s
Host function call duration: '__hc__must_get_agent_activity_1'
Duration of this host function call invoked from within WASM.
mean
0.013759
s
SD
0.008166
s
p50
0.011498
s
p95
0.031246
s
p99
0.037516
s
Host function call duration: '__hc__must_get_valid_record_1'
Duration of this host function call invoked from within WASM.
mean
0.002528
s
SD
0.003578
s
p50
0.003041
s
p95
0.010359
s
p99
0.020052
s
Host function call duration: '__hc__query_1'
Duration of this host function call invoked from within WASM.
mean
0.018041
s
SD
0.01527
s
p50
0.016285
s
p95
0.050507
s
p99
0.074051
s
Host function call duration: '__hc__send_remote_signal_1'
Duration of this host function call invoked from within WASM.
mean
4.8e-05
s
SD
3.7e-05
s
p50
4e-05
s
p95
0.000118
s
p99
0.000187
s
Host function call duration: '__hc__sys_time_1'
Duration of this host function call invoked from within WASM.
mean
4e-06
s
SD
3.4e-05
s
p50
5e-06
s
p95
1.2e-05
s
p99
0.000152
s
Host function call duration: '__hc__trace_1'
Duration of this host function call invoked from within WASM.
mean
5.7e-05
s
SD
0.00023
s
p50
6e-05
s
p95
0.000153
s
p99
0.000347
s
Host function call duration: '__hc__update_1'
Duration of this host function call invoked from within WASM.
mean
0.001236
s
SD
0
s
p50
0.001236
s
p95
0.001236
s
p99
0.001236
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.002974
s
SD
0.00328
s
p50
0.002849
s
p95
0.008278
s
p99
0.019317
s
Emitted local signals
Count of local signals emitted via the emit_signal host function.
total
144
over 713.287894055s
mean rate
846.43
/s
std
4475.12
/s
p5
7.61
/s
p95
3765.38
/s
peak
82792.53
/s
Sent remote signals
Count of remote signals sent via the send_remote_signal host function.
total
136
over 713.287891013s
mean rate
678.31
/s
std
4461.36
/s
p5
3.06
/s
p95
1670.34
/s
peak
74403.84
/s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.064895
s
SD
0.043409
s
p50
0.049052
s
p95
0.136233
s
p99
0.236693
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
46.44
s
mean
435.37
s
SD
253.3
s
p95
835.53
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
4851
over 870.526316712s
mean rate
26270.58
/s
std
216469.46
/s
p5
34.41
/s
p95
63977.29
/s
peak
4.45183882e+06
/s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
114.481444
s
SD
65.174441
s
p50
71.235583
s
p95
210.222055
s
p99
245.562475
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
4.444245
s
SD
4.887784
s
p50
3.456693
s
p95
14.734585
s
p99
21.860219
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.031443
s
SD
0.041257
s
p50
0.032665
s
p95
0.13941
s
p99
0.205245
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.091515
s
SD
0.112706
s
p50
0.051292
s
p95
0.1786
s
p99
0.804163
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.452592
s
SD
2.323487
s
p50
0.710383
s
p95
5.988063
s
p99
17.327279
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
4.979778
s
SD
5.26459
s
p50
3.002901
s
p95
14.941192
s
p99
25.63884
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.001012
s
SD
0.000729
s
p50
0.000756
s
p95
0.002312
s
p99
0.004071
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.005645
s
SD
0.007575
s
p50
0.001498
s
p95
0.025266
s
p99
0.035337
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000529
s
SD
0.001035
s
p50
0.000384
s
p95
0.000808
s
p99
0.006989
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
0.740804
s
SD
29.442395
s
p50
1.385052
s
p95
60.001711
s
p99
60.006462
s
P2P request roundtrip: 'get_agent_activity'
Time spent sending a get_agent_activity request and awaiting its response.
mean
0.307697
s
SD
8.314978
s
p50
0.074128
s
p95
0.682347
s
p99
60.001602
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.23499
s
SD
7.905964
s
p50
0.264084
s
p95
0.490803
s
p99
60.000202
s
P2P received remote signals
Count of remote signals received.
total
144
over 713.287585276s
mean rate
816.81
/s
std
3867.93
/s
p5
7.61
/s
p95
3734.7
/s
peak
64581.9
/s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
46.06
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
11.86
KiB/s
mean
291.74
KiB/s
SD
1003.08
KiB/s
p95
647.08
KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
5.97
KiB/s
mean
112.76
KiB/s
SD
105.63
KiB/s
p95
276.15
KiB/s
Total bytes received
Total bytes received on primary network interface
count
6.44
GiB
mean
7.41
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
2.49
GiB
mean
2.87
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 99.3%
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 2.05 MiB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
Detected
Warning 56% of hosts overloaded (load5/ncpus > 1.0)
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
2.94
%
mean
59.13
%
SD
30.68
%
p95
98.7
%
System
13.07
%
CPU percentiles
CPU usage percentiles
p50
66.24
%
p95
98.70
%
p99
99.30
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
22
hosts
mean time
269.93
s
Memory used percentage
Percentage of memory used
p5
9.17
%
mean
28.15
%
SD
11.18
%
p95
42.85
%
Memory available percentage
Percentage of available memory
p5
57.15
%
mean
71.85
%
SD
11.18
%
p95
90.83
%
Max host memory used
Maximum memory usage percentage across all hosts
max
40.10
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00
%
Memory growth rate
Rate of memory growth over time
growth
2.05
MiB/s
Disk read throughput
Disk read throughput in MiB/s
1.41
MiB/s
Disk write throughput
Disk write throughput in MiB/s
137.64
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/0/11
hosts
Mount Point
/efi-boot0/6
hosts
Mount Point
/etc/hostname0/3
hosts
Mount Point
/etc/hosts0/3
hosts
Mount Point
/etc/nomad.d/client-meta.json0/13
hosts
Mount Point
/etc/resolv.conf0/3
hosts
Mount Point
/nix/store0/11
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
4.53
5 min
4.18
15 min
5.22
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
55.56
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
25.7585
%
SD
30.0361
%
p95
86.42
%
60 second average
24.4498
%
300 second average
21.9527
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
7.6865
%
SD
12.81
%
p95
35.19
%
60 second average
7.1961
%
300 second average
6.2091
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
2.9816
%
SD
5.1884
%
p95
13.82
%
60 second average
2.8507
%
300 second average
2.9813
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.2
%
mean
15.76
%
SD
20.54
%
p95
56.14
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
243.34
KiB
mean
1.00
MiB
SD
295.72
KiB
p95
1.31
MiB
Holochain process threads
Number of threads in Holochain process
p5
13
threads
mean
32.84
threads
SD
9.73
threads
p95
49
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
39
file descriptors
mean
69.25
file descriptors
SD
16.98
file descriptors
p95
87
file descriptors
Validation Receipts
↑ Back to indexCreates an entry, wait for required validation receipts, then repeat. Records the amount of time it took to accumulate the required number of receipts for all DHT operations. This is measured to the nearest 20ms so that we don't keep the agent too busy checking for receipts.
Each agent in this scenario waits for a certain number of peers to be available or for up to two minutes, whichever happens first, before starting its behaviour.
By default, this scenario will wait for a complete set of validation receipts before committing the next record. If the NO_VALIDATION_COMPLETE environment variable is set, it will instead publish new records on every round, building up an ever-growing list of action hashes to check on.
Started
Thu, 22 Oct 2026 10:02:46 UTC
Duration
900s
Peer count
20
Peer count at end
19
Behaviours
-
default(1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
validation_receipts_39722287437_1
Receipts complete timing
The amount of time between publishing a record and receiving the required number of validation receipts.
Mean
mean
6.225661
s
SD
2.101297
s
Max
Highest per-partition mean latency
max mean
7.413058
s
Min
Lowest per-partition mean latency
min mean
5.600791
s
Receipts complete rate
The number of complete validation receipt sets collected per second.
Mean
mean
1.52
/s
Max
Highest per-partition mean rate
max
1.67
/s
Min
Lowest per-partition mean rate
min
1.29
/s
Error count
The total number of errors accumulated during the run.
0
Holochain MetricsClick to toggle visibility.
WASM usage: 'crud::create_sample_entry'
Metered usage count of the WASM ribosome for this zome function.
total
3716246
over 887.92486036s
mean rate
1.386673039e+07
/s
std
6.502086814e+07
/s
p5
0
/s
p95
2.753564623e+07
/s
peak
6.485945596e+08
/s
WASM usage: 'crud::get_sample_entry_validation_receipts'
Metered usage count of the WASM ribosome for this zome function.
total
15060164
over 887.92486036s
mean rate
8.079670619e+07
/s
std
2.1374743761e+08
/s
p5
192949.09
/s
p95
6.7828823859e+08
/s
peak
1.22277468501e+09
/s
WASM usage: 'crud::init'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 887.92486036s
Not enough time data to show a trend.
WASM usage: 'crud_integrity::entry_defs'
Metered usage count of the WASM ribosome for this zome function.
total
480036
over 887.92486036s
mean rate
1.78277503e+06
/s
std
8.37901464e+06
/s
p5
0
/s
p95
3.55684243e+06
/s
peak
8.378044349e+07
/s
Zome call duration: 'crud::create_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.006482
s
SD
0.001819
s
p50
0.005322
s
p95
0.008246
s
p99
0.009884
s
Zome call duration: 'crud::get_sample_entry_validation_receipts'
Duration of this zome call as measured by Holochain internally.
mean
0.019515
s
SD
0.011325
s
p50
0.010946
s
p95
0.035468
s
p99
0.04194
s
WASM call duration: 'crud::create_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.005611
s
SD
0.001668
s
p50
0.004487
s
p95
0.007298
s
p99
0.008894
s
WASM call duration: 'crud::get_sample_entry_validation_receipts'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.018534
s
SD
0.011124
s
p50
0.010041
s
p95
0.034343
s
p99
0.040685
s
WASM call duration: 'crud::init'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001558
s
SD
0.000911
s
p50
0.001333
s
p95
0.005362
s
p99
0.005362
s
WASM call duration: 'crud_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001833
s
SD
0.000471
s
p50
0.001536
s
p95
0.002392
s
p99
0.002529
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000643
s
SD
0.000227
s
p50
0.000347
s
p95
0.000877
s
p99
0.001056
s
Host function call duration: '__hc__get_validation_receipts_1'
Duration of this host function call invoked from within WASM.
mean
0.016266
s
SD
0.010497
s
p50
0.008157
s
p95
0.031178
s
p99
0.037323
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.002989
s
SD
0.000943
s
p50
0.002285
s
p95
0.004003
s
p99
0.004454
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.000196
s
SD
7.9e-05
s
p50
0.000154
s
p95
0.000275
s
p99
0.00034
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
43.63
s
mean
434.65
s
SD
251.38
s
p95
827.81
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
5446
over 887.924843687s
mean rate
2410.24
/s
std
8729.71
/s
p5
0
/s
p95
10731.07
/s
peak
95320.41
/s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
1.782434
s
SD
1.433824
s
p50
1.87585
s
p95
5.190825
s
p99
7.443565
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.00665
s
SD
0.013542
s
p50
0.0028
s
p95
0.031226
s
p99
0.061858
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.045723
s
SD
0.029065
s
p50
0.050182
s
p95
0.088991
s
p99
0.173909
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.042477
s
SD
0.021528
s
p50
0.021312
s
p95
0.067972
s
p99
0.093307
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.076461
s
SD
1.03112
s
p50
0.080775
s
p95
2.396854
s
p99
5.780256
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
1.726085
s
SD
4.606828
s
p50
2.189246
s
p95
14.868301
s
p99
21.753834
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.000919
s
SD
0.000498
s
p50
0.000567
s
p95
0.001705
s
p99
0.001983
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.009008
s
SD
0.009821
s
p50
0.001449
s
p95
0.030311
s
p99
0.036063
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000329
s
SD
8.8e-05
s
p50
0.000288
s
p95
0.000482
s
p99
0.000526
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
0.05933
s
SD
0.050986
s
p50
0.039989
s
p95
0.17676
s
p99
0.200794
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.133967
s
SD
0.051907
s
p50
0.106927
s
p95
0.22432
s
p99
0.239083
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
68.83
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
266.00
KiB/s
mean
1.03
MiB/s
SD
1.45
MiB/s
p95
3.01
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
142.07
KiB/s
mean
672.83
KiB/s
SD
431.76
KiB/s
p95
1.45
MiB/s
Total bytes received
Total bytes received on primary network interface
count
17.49
GiB
mean
20.12
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
11.15
GiB
mean
12.82
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 98.8%
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 3.42 MiB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
Detected
Warning 90% of hosts overloaded (load5/ncpus > 1.0)
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
47.67
%
mean
82.42
%
SD
17.8
%
p95
98.7
%
System
13.58
%
CPU percentiles
CPU usage percentiles
p50
88.14
%
p95
98.70
%
p99
98.83
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
19
hosts
mean time
574.63
s
Memory used percentage
Percentage of memory used
p5
10.01
%
mean
20.87
%
SD
5.82
%
p95
27.62
%
Memory available percentage
Percentage of available memory
p5
72.38
%
mean
79.13
%
SD
5.82
%
p95
89.99
%
Max host memory used
Maximum memory usage percentage across all hosts
max
25.34
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00
%
Memory growth rate
Rate of memory growth over time
growth
3.42
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.04
MiB/s
Disk write throughput
Disk write throughput in MiB/s
166.02
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/0/10
hosts
Mount Point
/efi-boot0/5
hosts
Mount Point
/etc/nomad.d/client-meta.json0/10
hosts
Mount Point
/nix/store0/10
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
13.14
5 min
9.52
15 min
5.36
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
90.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
4.83
%
mean
55.3166
%
SD
31.144
%
p95
97.24
%
60 second average
52.6597
%
300 second average
38.9699
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0022
%
300 second average
0.0012
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0006
%
300 second average
0.0003
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
19.7803
%
SD
26.777
%
p95
66.79
%
60 second average
19.1404
%
300 second average
17.0264
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
6.3235
%
SD
9.9903
%
p95
27.38
%
60 second average
6.3168
%
300 second average
7.2774
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.25
%
mean
16.3
%
SD
14.02
%
p95
42.46
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
1.49
KiB
mean
261.88
KiB
SD
108.20
KiB
p95
405.26
KiB
Holochain process threads
Number of threads in Holochain process
p5
1
threads
mean
34.51
threads
SD
13.21
threads
p95
53
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
8
file descriptors
mean
74.85
file descriptors
SD
22.64
file descriptors
p95
99
file descriptors
Write/get_agent_activity
↑ Back to index
A scenario where write peers write entries, while get_agent_activity peers each query a single write agent's activity with get_agent_activity.
Before a target write peer and the requesting get_agent_activity peer are in sync, this will measure the get_agent_activity call performance over a network. Once a write peer reaches sync with a get_agent_activity peer, the write peer will publish their actions and entries, and so the get_agent_activity calls will likely have most of the data they need locally. At that point this measures the database query performance and code paths through host functions.
Started
Thu, 22 Oct 2026 10:04:39 UTC
Duration
900s
Peer count
30
Peer count at end
23
Behaviours
-
get_agent_activity(1 agent) -
write(1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
write_get_agent_activity_39722287437_1
Observed chain advancement
The maximum highest-observed action sequence per write agent, aggregated across all reading agents. The reading dimension is collapsed by taking the maximum observed value for each write agent (any reader successfully tracking that point counts as propagation). Captures how far readers tracked writers’ chains during the run.
Mean max (across write agents)
Mean of the highest chain head value observed per write agent. Each write agent contributes its maximum — the value seen by any reader.
2418.75
Max
Highest chain head value observed for any single write agent.
3158
Write agents observed
4
get_agent_activity_full zome call timingThe time taken to call the zome function that retrieves information on a
write peer's source chain.Mean
mean
0.308114
s
SD
0.53553
s
Max
Highest per-partition mean latency
max mean
1.059698
s
Min
Lowest per-partition mean latency
min mean
0.07678
s
Error count
The number of errors accumulated.
22
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.02313
s
SD
0.426524
s
p50
0.00103
s
p95
0.410165
s
p99
1.389465
s
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
0
over 590.039085559s
Not enough time data to show a trend.
Zome call duration: 'agent_activity::announce_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.006065
s
SD
0.002845
s
p50
0.005216
s
p95
0.012098
s
p99
0.012098
s
Zome call duration: 'agent_activity::create_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.007164
s
SD
0.003258
s
p50
0.008264
s
p95
0.012587
s
p99
0.012745
s
Zome call duration: 'agent_activity::get_agent_activity_full'
Duration of this zome call as measured by Holochain internally.
mean
0.059555
s
SD
0.62456
s
p50
0.075882
s
p95
1.013275
s
p99
3.982187
s
Zome call duration: 'agent_activity::get_random_agent_with_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.005198
s
SD
0.04678
s
p50
0.005494
s
p95
0.014855
s
p99
0.244716
s
WASM call duration: 'agent_activity::announce_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.005512
s
SD
0.002865
s
p50
0.004756
s
p95
0.011568
s
p99
0.011568
s
WASM call duration: 'agent_activity::create_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.006335
s
SD
0.003031
s
p50
0.007325
s
p95
0.011367
s
p99
0.011614
s
WASM call duration: 'agent_activity::get_agent_activity_full'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.059192
s
SD
0.624393
s
p50
0.075173
s
p95
1.011016
s
p99
3.980787
s
WASM call duration: 'agent_activity::get_random_agent_with_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.004478
s
SD
0.046832
s
p50
0.004561
s
p95
0.014042
s
p99
0.244228
s
WASM call duration: 'agent_activity_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001642
s
SD
0.001181
s
p50
0.001473
s
p95
0.003644
s
p99
0.006013
s
Host function call duration: '__hc__agent_info_1'
Duration of this host function call invoked from within WASM.
mean
1.7e-05
s
SD
4e-06
s
p50
1.5e-05
s
p95
2.4e-05
s
p99
2.4e-05
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.00104
s
SD
0.000728
s
p50
0.001115
s
p95
0.002264
s
p99
0.002342
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.000701
s
SD
0.000623
s
p50
0.000499
s
p95
0.002098
s
p99
0.002098
s
Host function call duration: '__hc__get_agent_activity_1'
Duration of this host function call invoked from within WASM.
mean
0.056626
s
SD
0.623864
s
p50
0.072777
s
p95
1.002065
s
p99
3.97383
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.000526
s
SD
0.047249
s
p50
0.000477
s
p95
0.001774
s
p99
0.242245
s
Host function call duration: '__hc__random_bytes_1'
Duration of this host function call invoked from within WASM.
mean
1.4e-05
s
SD
8e-06
s
p50
7e-06
s
p95
3e-05
s
p99
3.3e-05
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.002652
s
SD
0.001845
s
p50
0.002452
s
p95
0.006095
s
p99
0.009162
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.000298
s
SD
0.001125
s
p50
0.000131
s
p95
0.001017
s
p99
0.005618
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
43.13
s
mean
426.42
s
SD
248.78
s
p95
825.85
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
1373
over 886.858041403s
mean rate
66524.89
/s
std
635952.12
/s
p5
209.61
/s
p95
239229.59
/s
peak
2.096662432e+07
/s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
135.047846
s
SD
110.734814
s
p50
38.011236
s
p95
321.062746
s
p99
438.496859
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.01348
s
SD
0.179763
s
p50
0.08469
s
p95
0.556984
s
p99
0.651886
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.163962
s
SD
0.55757
s
p50
0.5077
s
p95
1.907613
s
p99
2.081748
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.189907
s
SD
0.824285
s
p50
0.074279
s
p95
0.53927
s
p99
4.292304
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
1.390371
s
SD
25.443153
s
p50
14.209432
s
p95
73.360833
s
p99
90.456812
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
20.181462
s
SD
33.389643
s
p50
0.108273
s
p95
95.836742
s
p99
127.081286
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.004608
s
SD
0.003013
s
p50
0.003001
s
p95
0.010267
s
p99
0.015688
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.006285
s
SD
0.016549
s
p50
0.002249
s
p95
0.041
s
p99
0.107148
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000824
s
SD
0.000531
s
p50
0.000568
s
p95
0.001914
s
p99
0.002074
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
2.522038
s
SD
27.404297
s
p50
2.679876
s
p95
60.014992
s
p99
60.041366
s
P2P request roundtrip: 'get_links'
Time spent sending a get_links request and awaiting its response.
mean
0.19794
s
SD
0.031508
s
p50
0.233787
s
p95
0.233787
s
p99
0.233787
s
P2P request roundtrip: 'get_agent_activity'
Time spent sending a get_agent_activity request and awaiting its response.
mean
0.057844
s
SD
27.595777
s
p50
0.076438
s
p95
60.003132
s
p99
60.016624
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.832545
s
SD
23.510192
s
p50
4.000337
s
p95
60.002156
s
p99
60.002923
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
72.60
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
253.87
KiB/s
mean
963.26
KiB/s
SD
1.26
MiB/s
p95
2.73
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
137.41
KiB/s
mean
673.03
KiB/s
SD
459.26
KiB/s
p95
1.53
MiB/s
Total bytes received
Total bytes received on primary network interface
count
19.00
GiB
mean
21.86
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
13.27
GiB
mean
15.27
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 98.9%
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 2.94 MiB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
Detected
Warning 79% of hosts overloaded (load5/ncpus > 1.0)
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
53.16
%
mean
86.47
%
SD
16.78
%
p95
98.75
%
System
13.87
%
CPU percentiles
CPU usage percentiles
p50
96.78
%
p95
98.75
%
p99
98.92
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
23
hosts
mean time
658.95
s
Memory used percentage
Percentage of memory used
p5
11.99
%
mean
22.1
%
SD
5.01
%
p95
27.69
%
Memory available percentage
Percentage of available memory
p5
72.31
%
mean
77.9
%
SD
5.01
%
p95
88.02
%
Max host memory used
Maximum memory usage percentage across all hosts
max
33.19
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
8.49
%
Memory growth rate
Rate of memory growth over time
growth
2.94
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.04
MiB/s
Disk write throughput
Disk write throughput in MiB/s
216.90
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/0/11
hosts
Mount Point
/efi-boot0/6
hosts
Mount Point
/etc/hostname0/4
hosts
Mount Point
/etc/hosts0/4
hosts
Mount Point
/etc/nomad.d/client-meta.json0/13
hosts
Mount Point
/etc/resolv.conf0/4
hosts
Mount Point
/nix/store0/11
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
14.22
5 min
10.78
15 min
6.16
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
78.57
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
6.88
%
mean
64.1051
%
SD
30.6932
%
p95
97.93
%
60 second average
61.8960
%
300 second average
48.6115
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0008
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
17.3823
%
SD
26.2186
%
p95
66.79
%
60 second average
17.0115
%
300 second average
15.0509
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
5.1671
%
SD
9.4124
%
p95
26.25
%
60 second average
5.1670
%
300 second average
5.5911
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.18
%
mean
18.19
%
SD
14.21
%
p95
43.61
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
16.32
KiB
mean
248.95
KiB
SD
116.37
KiB
p95
396.56
KiB
Holochain process threads
Number of threads in Holochain process
p5
1
threads
mean
32.98
threads
SD
13.2
threads
p95
53
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
7
file descriptors
mean
73.56
file descriptors
SD
23.67
file descriptors
p95
97
file descriptors
Write/get_agent_activity with volatile conductors
↑ Back to index
A scenario where write peers write entries, while get_agent_activity_volatile peers each query a single write agents activity with get_agent_activity but shutdown and restart their conductors at semi-random intervals.
Before a target write peer and the requesting get_agent_activity_volatile peer are in sync, this will measure the get_agent_activity call performance over a network. Once a write peer reaches sync with a get_agent_activity peer, the write peer will publish their actions and entries, and so the get_agent_activity calls will likely have most of the data they need locally. At that point this measures the database query performance and code paths through host functions.
Started
Thu, 22 Oct 2026 10:06:37 UTC
Duration
1800s
Peer count
30
Peer count at end
14
Behaviours
-
get_agent_activity_volatile(1 agent) -
write(1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
write_get_agent_activity_volatile_39722287437_1
Highest observed
action_seqThe rate at which zero-arc readers observe new chain heads on the writers' chains via
get_agent_activity. This reflects the DHT's ability to propagate agent activity ops and make them available to querying peers.Mean
mean
48.64
/s
Max
Highest per-partition mean rate
max
313.24
/s
Min
Lowest per-partition mean rate
min
9.47
/s
get_agent_activity_full zome call timingThe time taken to call the zome function that retrieves information on a
write peer's source chain.Mean
mean
0.389063
s
SD
0.501746
s
Max
Highest per-partition mean latency
max mean
0.724753
s
Min
Lowest per-partition mean latency
min mean
0.022194
s
Running volatile conductors count
The number of conductors running by
get_agent_activity_volatile peers.p5
4
conductors
mean
6.66
conductors
SD
1.57
conductors
p95
9
conductors
Volatile conductor total running duration
The total running duration of a
get_agent_activity_volatile conductor.Mean
mean
789.964065
s
SD
399.185612
s
Max
Highest per-partition mean latency
max mean
961.166667
s
Min
Lowest per-partition mean latency
min mean
627.5
s
Volatile conductor running duration
The duration that a
get_agent_activity_volatile conductor was running before being stopped.Mean
mean
48.541535
s
SD
22.713783
s
Max
Highest per-partition mean latency
max mean
140.266694
s
Min
Lowest per-partition mean latency
min mean
22.623385
s
Volatile conductor stopped duration
The duration that a
get_agent_activity_volatile conductor was stopped before being started again.Mean
mean
21.22628
s
SD
24.892463
s
Max
Highest per-partition mean latency
max mean
29.709321
s
Min
Lowest per-partition mean latency
min mean
9.597344
s
Reached target arc
Did
get_agent_activity_volatile peer reach their target arc in the moment before they were shutdown.get_agent_activity_volatile_agent:uhCAk4gQtka-mO5JFkeDEtkLmWeIzHYsNa9E5-Sk0Dmt2-dd1yzG9
p5
0
mean
0.24
SD
0.43
p95
1
get_agent_activity_volatile_agent:uhCAk6OSypsX7EEJqtKpjMPdUET17qNUXo9nalJBB_PXsbKlCi83N
p5
0
mean
0
SD
0
p95
0
get_agent_activity_volatile_agent:uhCAkF_OpWtF98gpCX0lh2QBdg8uPFt71f1ZyHOzeMovt9m3mBGbW
p5
0
mean
0.16
SD
0.37
p95
1
get_agent_activity_volatile_agent:uhCAkFkQo4nIsPAf13sRulU8t0obXLgUSRWNkwsFvEVAaFlZfszfn
p5
0
mean
0.16
SD
0.37
p95
1
get_agent_activity_volatile_agent:uhCAkN_enkJk67tElbGbbV7eigLaV8OCLq_rY8cCEmpyamodAW1bH
p5
0
mean
0.11
SD
0.31
p95
1
get_agent_activity_volatile_agent:uhCAkPzhiEWahWGi3goeJs8yze6vC2pGs4EbyeGSCrQzqGiFJMEAK
p5
0
mean
0.1
SD
0.3
p95
1
get_agent_activity_volatile_agent:uhCAkTenITlfYnqbMrYhLeVQHkB78XKzxsfyYwG847ihY7IzdC2Ka
p5
0
mean
0.19
SD
0.4
p95
1
get_agent_activity_volatile_agent:uhCAkbxj4dWl952fRdoOC9G06DQXjFWotOfmtUaf2ZvN8BCrlNtUX
p5
0
mean
0.09
SD
0.28
p95
1
get_agent_activity_volatile_agent:uhCAkdQPH01t6JCax0WuqSr5_z49AsPh5M8wbx1RgU52CLTllSwLo
p5
0
mean
0.11
SD
0.31
p95
1
get_agent_activity_volatile_agent:uhCAkfggp2AA3ZU3rtyBI7Pf0YXyIF-_lq6b0wXp_FRHrjHGdMOiL
p5
0
mean
0.1
SD
0.31
p95
1
get_agent_activity_volatile_agent:uhCAkmfbbhC6HUx2S3m4TETQ6AcvUax7VejZT2bbH2Km4CNI2Eq64
p5
0
mean
0
SD
0
p95
0
get_agent_activity_volatile_agent:uhCAkrgoFvUq32sptInRC6SacQJn6pvFrs2HTJUdRACjoUNQ9wQtO
p5
0
mean
0.07
SD
0.26
p95
1
Error count
The number of errors accumulated.
50
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.212053
s
SD
14.512316
s
p50
0.418464
s
p95
1.485919
s
p99
5.958362
s
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
0
over 1125.453882189s
Not enough time data to show a trend.
WASM usage: 'agent_activity::announce_write_behaviour'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 1786.993421471s
Not enough time data to show a trend.
WASM usage: 'agent_activity::create_sample_entry'
Metered usage count of the WASM ribosome for this zome function.
total
133596540
over 1786.993421471s
mean rate
8.7032190118e+08
/s
std
4.61358433437e+09
/s
p5
550022.67
/s
p95
4.0576054768e+09
/s
peak
6.27084371663e+10
/s
WASM usage: 'agent_activity::get_agent_activity_full'
Metered usage count of the WASM ribosome for this zome function.
total
4227713
over 1787.227769839s
mean rate
5.99071441e+07
/s
std
1.7280108934e+08
/s
p5
551852.87
/s
p95
2.7858534081e+08
/s
peak
2.12166620996e+09
/s
WASM usage: 'agent_activity::get_random_agent_with_write_behaviour'
Metered usage count of the WASM ribosome for this zome function.
total
481168
over 1024.819320498s
mean rate
2.76617861e+06
/s
std
7.23930142e+06
/s
p5
0
/s
p95
1.87965796e+07
/s
peak
3.037167411e+07
/s
WASM usage: 'agent_activity_integrity::entry_defs'
Metered usage count of the WASM ribosome for this zome function.
total
17223822
over 1786.993421471s
mean rate
1.0147460548e+08
/s
std
5.6556781285e+08
/s
p5
45726.87
/s
p95
4.9001075208e+08
/s
peak
8.08463272814e+09
/s
Zome call duration: 'agent_activity::announce_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.005324
s
SD
0.001959
s
p50
0.004457
s
p95
0.009047
s
p99
0.009047
s
Zome call duration: 'agent_activity::create_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.01027
s
SD
0.002125
s
p50
0.01003
s
p95
0.013062
s
p99
0.013204
s
Zome call duration: 'agent_activity::get_agent_activity_full'
Duration of this zome call as measured by Holochain internally.
mean
0.193941
s
SD
0.412184
s
p50
0.443811
s
p95
1.358001
s
p99
1.890306
s
Zome call duration: 'agent_activity::get_random_agent_with_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.016239
s
SD
1.534597
s
p50
0.011829
s
p95
4.415619
s
p99
4.415619
s
WASM call duration: 'agent_activity::announce_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.004315
s
SD
0.002083
s
p50
0.003286
s
p95
0.008376
s
p99
0.008376
s
WASM call duration: 'agent_activity::create_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.00919
s
SD
0.001969
s
p50
0.008918
s
p95
0.011743
s
p99
0.011867
s
WASM call duration: 'agent_activity::get_agent_activity_full'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.225108
s
SD
14.816818
s
p50
0.43446
s
p95
1.41128
s
p99
5.960097
s
WASM call duration: 'agent_activity::get_random_agent_with_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.011633
s
SD
0.967792
s
p50
0.010802
s
p95
2.7649
s
p99
2.7649
s
WASM call duration: 'agent_activity_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.002968
s
SD
0.000702
s
p50
0.002916
s
p95
0.003698
s
p99
0.004754
s
Host function call duration: '__hc__agent_info_1'
Duration of this host function call invoked from within WASM.
mean
2.3e-05
s
SD
5e-06
s
p50
2.1e-05
s
p95
3.2e-05
s
p99
3.2e-05
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.001331
s
SD
0.000404
s
p50
0.001314
s
p95
0.002077
s
p99
0.002123
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.000333
s
SD
0.00013
s
p50
0.000274
s
p95
0.000591
s
p99
0.000591
s
Host function call duration: '__hc__get_agent_activity_1'
Duration of this host function call invoked from within WASM.
mean
0.222939
s
SD
14.816911
s
p50
0.432342
s
p95
1.405534
s
p99
5.958512
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.00702
s
SD
0.962384
s
p50
0.001856
s
p95
2.744272
s
p99
2.744272
s
Host function call duration: '__hc__random_bytes_1'
Duration of this host function call invoked from within WASM.
mean
1.2e-05
s
SD
4e-06
s
p50
1e-05
s
p95
1.7e-05
s
p99
1.7e-05
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.005003
s
SD
0.001985
s
p50
0.003875
s
p95
0.006526
s
p99
0.006526
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.00031
s
SD
0.000139
s
p50
0.000307
s
p95
0.000417
s
p99
0.000423
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
4.03
s
mean
397.24
s
SD
544.23
s
p95
1577.56
s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
327.888636
s
SD
236.026437
s
p50
86.91293
s
p95
673.680146
s
p99
789.868366
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.039024
s
SD
0.209977
s
p50
0.0147
s
p95
0.431
s
p99
0.928809
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
1e-06
s
SD
1.2e-05
s
p50
0
s
p95
0
s
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.191916
s
SD
0.329702
s
p50
0.031926
s
p95
0.481877
s
p99
1.618013
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.306815
s
SD
0.141014
s
p50
0.13878
s
p95
0.427366
s
p99
0.522319
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
4.844232
s
SD
32.021296
s
p50
8.900989
s
p95
99.002737
s
p99
125.603613
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
73.014504
s
SD
57.813575
s
p50
54.000253
s
p95
178.399156
s
p99
212.863176
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.012446
s
SD
0.006129
s
p50
0.00156
s
p95
0.017921
s
p99
0.021953
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.007719
s
SD
0.075015
s
p50
0.001763
s
p95
0.013422
s
p99
0.160577
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.001102
s
SD
0.000695
s
p50
0.000823
s
p95
0.001737
s
p99
0.00292
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
4.730738
s
SD
28.288204
s
p50
0.792904
s
p95
60.002474
s
p99
60.002524
s
P2P request roundtrip: 'get_links'
Time spent sending a get_links request and awaiting its response.
mean
0.318945
s
SD
0.267976
s
p50
0.559509
s
p95
0.579623
s
p99
0.579623
s
P2P request roundtrip: 'get_agent_activity'
Time spent sending a get_agent_activity request and awaiting its response.
mean
0.324423
s
SD
10.734826
s
p50
0.450594
s
p95
1.603059
s
p99
60.001703
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.544266
s
SD
21.630725
s
p50
0.582057
s
p95
60.001015
s
p99
60.001653
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
70.44
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
0
B/s
mean
849.87
KiB/s
SD
1.43
MiB/s
p95
2.72
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
0
B/s
mean
529.50
KiB/s
SD
489.56
KiB/s
p95
1.49
MiB/s
Total bytes received
Total bytes received on primary network interface
count
21.48
GiB
mean
12.29
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
13.38
GiB
mean
7.66
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 99.0%
Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected
Warning Moderate swap usage (19.4% swap used)
System overload anomaly
System overload anomaly was detected
Detected
Warning 56% of hosts overloaded (load5/ncpus > 1.0)
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
17.84
%
mean
83
%
SD
25.28
%
p95
98.8
%
System
12.56
%
CPU percentiles
CPU usage percentiles
p50
96.81
%
p95
98.80
%
p99
98.95
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
16
hosts
mean time
1272.05
s
Memory used percentage
Percentage of memory used
p5
10.33
%
mean
19.3
%
SD
5.03
%
p95
26.85
%
Memory available percentage
Percentage of available memory
p5
73.15
%
mean
80.7
%
SD
5.03
%
p95
89.67
%
Max host memory used
Maximum memory usage percentage across all hosts
max
32.69
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
19.42
%
Memory growth rate
Rate of memory growth over time
growth
0.73
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.05
MiB/s
Disk write throughput
Disk write throughput in MiB/s
179.37
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/0/6
hosts
Mount Point
/efi-boot0/2
hosts
Mount Point
/etc/hostname0/9
hosts
Mount Point
/etc/hosts0/9
hosts
Mount Point
/etc/nomad.d/client-meta.json0/10
hosts
Mount Point
/etc/resolv.conf0/9
hosts
Mount Point
/nix/store0/6
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
12.79
5 min
12.12
15 min
8.85
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
56.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0.45
%
mean
56.5451
%
SD
34.1962
%
p95
97.64
%
60 second average
56.5128
%
300 second average
53.7261
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
2.6649
%
SD
9.7654
%
p95
12.4
%
60 second average
2.6043
%
300 second average
2.2331
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
0.8926
%
SD
4.5689
%
p95
1.96
%
60 second average
0.8820
%
300 second average
0.7684
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.08
%
mean
11.98
%
SD
15.25
%
p95
41.04
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
39.68
KiB
mean
221.23
KiB
SD
143.81
KiB
p95
487.63
KiB
Holochain process threads
Number of threads in Holochain process
p5
1
threads
mean
30.21
threads
SD
13.6
threads
p95
53
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
8
file descriptors
mean
68.53
file descriptors
SD
24.71
file descriptors
p95
99
file descriptors
Write/Query
↑ Back to indexCreates an entry, then queries the source chain and performs a simple operation on the entries, then repeats.
Started
Thu, 22 Oct 2026 09:49:51 UTC
Duration
300s
Peer count
1
Peer count at end
1
Behaviours
-
default(1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
write_query_39722287437_1
Write timing
The time it takes to complete the zome function call that writes the entry.
mean
0.031079
s
SD
0.015542
s
p50
0.031071
s
p95
0.052968
s
p99
0.059177
s
Write rate
The number of writes completed per second.
mean
90.96
/s
Query timing
The time it takes to complete the
chain_query_count_len zome call that queries the source chain.mean
0.071774
s
SD
0.038893
s
p50
0.070672
s
p95
0.132636
s
p99
0.144847
s
Query rate
The number of chain queries completed per second.
mean
90.96
/s
Error count
The number of errors accumulated during the run.
0
Holochain MetricsClick to toggle visibility.
WASM usage: 'crud::chain_query_count_len'
Metered usage count of the WASM ribosome for this zome function.
total
84871823838
over 280.003409221s
mean rate
3.031099852e+08
/s
std
3.017547735e+07
/s
p5
2.5085809481e+08
/s
p95
3.2608519746e+08
/s
peak
3.2700084279e+08
/s
WASM usage: 'crud::create_sample_entry'
Metered usage count of the WASM ribosome for this zome function.
total
92394740
over 280.003409221s
mean rate
329977.21
/s
std
207357.22
/s
p5
187515.85
/s
p95
750063.25
/s
peak
1.1694013e+06
/s
WASM usage: 'crud::init'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 280.003409221s
Not enough time data to show a trend.
WASM usage: 'crud_integrity::entry_defs'
Metered usage count of the WASM ribosome for this zome function.
total
23869680
over 280.003409221s
mean rate
85247.82
/s
std
53544.14
/s
p5
48443.7
/s
p95
193334.37
/s
peak
302108.48
/s
Zome call duration: 'crud::chain_query_count_len'
Duration of this zome call as measured by Holochain internally.
mean
0.029701
s
SD
0.009624
s
p50
0.02704
s
p95
0.038785
s
p99
0.039614
s
Zome call duration: 'crud::create_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.003263
s
SD
2.1e-05
s
p50
0.003265
s
p95
0.003273
s
p99
0.003275
s
WASM call duration: 'crud::chain_query_count_len'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.02909
s
SD
0.009593
s
p50
0.026435
s
p95
0.038149
s
p99
0.038975
s
WASM call duration: 'crud::create_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.002716
s
SD
1.1e-05
s
p50
0.002717
s
p95
0.002721
s
p99
0.002722
s
WASM call duration: 'crud::init'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.00265
s
SD
0
s
p50
0.00265
s
p95
0.00265
s
p99
0.00265
s
WASM call duration: 'crud_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.000954
s
SD
9e-06
s
p50
0.000954
s
p95
0.000963
s
p99
0.000965
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000252
s
SD
0.000121
s
p50
0.000494
s
p95
0.000494
s
p99
0.000494
s
Host function call duration: '__hc__query_1'
Duration of this host function call invoked from within WASM.
mean
0.012325
s
SD
0.004212
s
p50
0.011287
s
p95
0.016116
s
p99
0.016458
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.001462
s
SD
4.1e-05
s
p50
0.00143
s
p95
0.001525
s
p99
0.001538
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.000109
s
SD
7e-06
s
p50
0.000108
s
p95
0.000111
s
p99
0.000111
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
16.82
s
mean
146.82
s
SD
83.67
s
p95
276.82
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 280.003415215s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.001982
s
SD
0
s
p50
0.001982
s
p95
0.001982
s
p99
0.001982
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.001816
s
SD
0.000162
s
p50
0.001806
s
p95
0.001892
s
p99
0.001895
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.024613
s
SD
0.007871
s
p50
0.022948
s
p95
0.031445
s
p99
0.031981
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.002029
s
SD
0
s
p50
0.002029
s
p95
0.002029
s
p99
0.002029
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
0.004531
s
SD
0
s
p50
0.004531
s
p95
0.004531
s
p99
0.004531
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.00233
s
SD
0.000785
s
p50
0.002173
s
p95
0.002996
s
p99
0.003048
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.001035
s
SD
0.00042
s
p50
0.001832
s
p95
0.001832
s
p99
0.001832
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000224
s
SD
2e-06
s
p50
0.000223
s
p95
0.000225
s
p99
0.000226
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
54.61
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
10.05
KiB/s
mean
13.03
KiB/s
SD
1.94
KiB/s
p95
15.93
KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
10.22
KiB/s
mean
14.09
KiB/s
SD
2.32
KiB/s
p95
17.80
KiB/s
Total bytes received
Total bytes received on primary network interface
count
3.69
MiB
mean
13.03
KiB/s
Total bytes sent
Total bytes sent on primary network interface
count
3.99
MiB
mean
14.09
KiB/s
CPU spike anomaly
CPU spike anomaly was detected
NotDetected
Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
Detected
Warning 100% of hosts overloaded (load5/ncpus > 1.0)
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
64.14
%
mean
70.06
%
SD
2.35
%
p95
72.4
%
System
15.45
%
CPU percentiles
CPU usage percentiles
p50
70.99
%
p95
72.40
%
p99
75.27
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
0
hosts
mean time
0.00
s
Memory used percentage
Percentage of memory used
p5
24.62
%
mean
25.33
%
SD
0.46
%
p95
26.3
%
Memory available percentage
Percentage of available memory
p5
73.7
%
mean
74.67
%
SD
0.46
%
p95
75.38
%
Max host memory used
Maximum memory usage percentage across all hosts
max
25.33
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00
%
Memory growth rate
Rate of memory growth over time
growth
-0.24
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.00
MiB/s
Disk write throughput
Disk write throughput in MiB/s
2.72
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/etc/nomad.d/client-meta.json0/1
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
1.99
5 min
2.06
15 min
1.21
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
100.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
21.93
%
mean
24.988
%
SD
3.1935
%
p95
29.29
%
60 second average
25.6307
%
300 second average
23.5647
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
0.327
%
SD
0.4464
%
p95
1.15
%
60 second average
0.4260
%
300 second average
0.6457
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
0.1307
%
SD
0.2668
%
p95
0.58
%
60 second average
0.1917
%
300 second average
0.3607
%
Holochain process CPU usage
CPU usage by Holochain process
p5
56.81
%
mean
62.51
%
SD
2.31
%
p95
64.62
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
254.85
KiB
mean
261.69
KiB
SD
29.22
KiB
p95
279.81
KiB
Holochain process threads
Number of threads in Holochain process
p5
23
threads
mean
23.37
threads
SD
0.84
threads
p95
25
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
63
file descriptors
mean
63.77
file descriptors
SD
1.93
file descriptors
p95
68
file descriptors
Write/Read
↑ Back to indexCreates an entry and reads it back, then repeats.
Started
Thu, 22 Oct 2026 09:51:32 UTC
Duration
300s
Peer count
1
Peer count at end
1
Behaviours
-
default(1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
write_read_39722287437_1
Write timing
The time taken to write the entry.
mean
0.037159
s
SD
0.018522
s
p50
0.036606
s
p95
0.064884
s
p99
0.075144
s
Write rate
The number of writes completed per second.
mean
125.11
/s
Read timing
The time taken to read the entry.
mean
0.041067
s
SD
0.021104
s
p50
0.041618
s
p95
0.072145
s
p99
0.084946
s
Read rate
The number of reads completed per second.
mean
125.11
/s
Error count
The number of errors accumulated over the run.
0
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.000607
s
SD
8.5e-05
s
p50
0.000591
s
p95
0.000674
s
p99
0.000676
s
WASM usage: 'crud::create_sample_entry'
Metered usage count of the WASM ribosome for this zome function.
total
119022154
over 270.00546035s
mean rate
440815.18
/s
std
255893.58
/s
p5
255703.89
/s
p95
917120.63
/s
peak
1.46603479e+06
/s
WASM usage: 'crud::get_sample_entry'
Metered usage count of the WASM ribosome for this zome function.
total
117113591
over 270.00546035s
mean rate
433746.55
/s
std
251005.09
/s
p5
255139.11
/s
p95
903159.65
/s
peak
1.43740203e+06
/s
WASM usage: 'crud::init'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 270.00546035s
Not enough time data to show a trend.
WASM usage: 'crud_integrity::entry_defs'
Metered usage count of the WASM ribosome for this zome function.
total
15374364
over 270.00546035s
mean rate
56941.11
/s
std
33054.36
/s
p5
33029.86
/s
p95
118466.57
/s
peak
189371.07
/s
Zome call duration: 'crud::create_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.003177
s
SD
2.2e-05
s
p50
0.003177
s
p95
0.003193
s
p99
0.003198
s
Zome call duration: 'crud::get_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.002298
s
SD
0.000128
s
p50
0.002279
s
p95
0.002396
s
p99
0.002398
s
WASM call duration: 'crud::create_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.002639
s
SD
1.4e-05
s
p50
0.002638
s
p95
0.00265
s
p99
0.002654
s
WASM call duration: 'crud::get_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001749
s
SD
0.000118
s
p50
0.00173
s
p95
0.001841
s
p99
0.001843
s
WASM call duration: 'crud::init'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.002418
s
SD
0
s
p50
0.002418
s
p95
0.002418
s
p99
0.002418
s
WASM call duration: 'crud_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.000912
s
SD
3e-06
s
p50
0.000912
s
p95
0.000914
s
p99
0.000922
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000251
s
SD
0.000104
s
p50
0.000458
s
p95
0.000458
s
p99
0.000458
s
Host function call duration: '__hc__get_1'
Duration of this host function call invoked from within WASM.
mean
0.000638
s
SD
8.6e-05
s
p50
0.000623
s
p95
0.000706
s
p99
0.000708
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.001365
s
SD
8e-06
s
p50
0.001366
s
p95
0.001371
s
p99
0.001373
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.000106
s
SD
6e-06
s
p50
0.000106
s
p95
0.000109
s
p99
0.00011
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
16.66
s
mean
141.66
s
SD
80.78
s
p95
266.67
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 270.005461857s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.001187
s
SD
0
s
p50
0.001187
s
p95
0.001187
s
p99
0.001187
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.001749
s
SD
0.000173
s
p50
0.001763
s
p95
0.001811
s
p99
0.001818
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.030573
s
SD
0.010035
s
p50
0.028756
s
p95
0.03927
s
p99
0.040024
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.002833
s
SD
0
s
p50
0.002833
s
p95
0.002833
s
p99
0.002833
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
0.000681
s
SD
0
s
p50
0.000681
s
p95
0.000681
s
p99
0.000681
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.002391
s
SD
0.000814
s
p50
0.002254
s
p95
0.003083
s
p99
0.003143
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.001044
s
SD
0.000232
s
p50
0.001455
s
p95
0.001455
s
p99
0.001455
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000223
s
SD
2e-06
s
p50
0.000221
s
p95
0.000226
s
p99
0.000226
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
60.72
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
12.11
KiB/s
mean
302.50
KiB/s
SD
1.28
MiB/s
p95
1.14
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
11.33
KiB/s
mean
20.61
KiB/s
SD
34.19
KiB/s
p95
45.45
KiB/s
Total bytes received
Total bytes received on primary network interface
count
85.67
MiB
mean
302.50
KiB/s
Total bytes sent
Total bytes sent on primary network interface
count
5.84
MiB
mean
20.61
KiB/s
CPU spike anomaly
CPU spike anomaly was detected
NotDetected
Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
NotDetected
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
66.05
%
mean
74.73
%
SD
8.11
%
p95
79.77
%
System
14.01
%
CPU percentiles
CPU usage percentiles
p50
76.81
%
p95
79.77
%
p99
80.26
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
1
hosts
mean time
10.34
s
Memory used percentage
Percentage of memory used
p5
24.31
%
mean
24.89
%
SD
0.72
%
p95
25.92
%
Memory available percentage
Percentage of available memory
p5
74.08
%
mean
75.11
%
SD
0.72
%
p95
75.69
%
Max host memory used
Maximum memory usage percentage across all hosts
max
24.89
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00
%
Memory growth rate
Rate of memory growth over time
growth
0.05
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.00
MiB/s
Disk write throughput
Disk write throughput in MiB/s
2.79
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/etc/nomad.d/client-meta.json0/1
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
2.27
5 min
1.80
15 min
0.98
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
0.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
10.74
%
mean
29.097
%
SD
6.3567
%
p95
34.12
%
60 second average
25.7010
%
300 second average
20.6667
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0.03
%
mean
0.3687
%
SD
0.4223
%
p95
1.1
%
60 second average
0.3963
%
300 second average
0.5927
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
0.1583
%
SD
0.2685
%
p95
0.92
%
60 second average
0.1903
%
300 second average
0.3153
%
Holochain process CPU usage
CPU usage by Holochain process
p5
58.37
%
mean
67.16
%
SD
13.2
%
p95
73.73
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
262.16
KiB
mean
276.09
KiB
SD
18.13
KiB
p95
288.23
KiB
Holochain process threads
Number of threads in Holochain process
p5
5
threads
mean
21.1
threads
SD
4.92
threads
p95
24
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
12
file descriptors
mean
58.33
file descriptors
SD
14.19
file descriptors
p95
66
file descriptors
Write validated
↑ Back to indexCreates an entry, updates it and links the create to the update, then repeats. Each of the three actions is validated by the included hApp.
Started
Thu, 22 Oct 2026 09:53:14 UTC
Duration
300s
Peer count
1
Peer count at end
1
Behaviours
-
default(1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
write_validated_39722287437_1
Write timing
The time it takes to complete the zome function call that writes the entry, updates it, and links to the update.
mean
0.070896
s
SD
0.029958
s
p50
0.072608
s
p95
0.113813
s
p99
0.141361
s
Write rate
The number of write zome function calls that complete per second.
mean
55.93
/s
Read timing
The time it takes to complete the zome function call that reads the data.
mean
0.108914
s
SD
0.042037
s
p50
0.108786
s
p95
0.171567
s
p99
0.204276
s
Read rate
The number of read zome function calls that complete per second.
mean
55.93
/s
Error count
The number of errors accumulated over the run.
0
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.005167
s
SD
0.002006
s
p50
0.004722
s
p95
0.008226
s
p99
0.008568
s
WASM usage: 'validated::create_sample_entry'
Metered usage count of the WASM ribosome for this zome function.
total
48142729
over 270.006253008s
mean rate
178301.74
/s
std
68099.04
/s
p5
119443.43
/s
p95
325130.54
/s
peak
401462.8
/s
WASM usage: 'validated::update_sample_entry'
Metered usage count of the WASM ribosome for this zome function.
total
56887850
over 270.006253008s
mean rate
210690.24
/s
std
80391.72
/s
p5
137314.67
/s
p95
380532.41
/s
peak
474715.63
/s
WASM usage: 'validated_integrity::entry_defs'
Metered usage count of the WASM ribosome for this zome function.
total
97130904
over 280.006303206s
mean rate
346887.52
/s
std
139457.53
/s
p5
220223.87
/s
p95
661563.88
/s
peak
755179.37
/s
WASM usage: 'validated_integrity::validate'
Metered usage count of the WASM ribosome for this zome function.
total
844031913
over 270.006253008s
mean rate
3.12596249e+06
/s
std
1.192852e+06
/s
p5
2.05046189e+06
/s
p95
5.65845014e+06
/s
peak
7.0421214e+06
/s
Zome call duration: 'validated::create_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.00373
s
SD
0.000156
s
p50
0.003738
s
p95
0.003794
s
p99
0.003796
s
Zome call duration: 'validated::update_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.010333
s
SD
0.002248
s
p50
0.009852
s
p95
0.012347
s
p99
0.012501
s
WASM call duration: 'validated::create_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.003169
s
SD
0.000146
s
p50
0.003179
s
p95
0.003226
s
p99
0.003228
s
WASM call duration: 'validated::update_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.009768
s
SD
0.002238
s
p50
0.009288
s
p95
0.011774
s
p99
0.011927
s
WASM call duration: 'validated_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.000957
s
SD
0.000207
s
p50
0.000955
s
p95
0.000971
s
p99
0.002079
s
WASM call duration: 'validated_integrity::validate'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.00349
s
SD
0.000314
s
p50
0.003407
s
p95
0.003803
s
p99
0.003831
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.00028
s
SD
1.5e-05
s
p50
0.000282
s
p95
0.000284
s
p99
0.000284
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.000273
s
SD
3e-06
s
p50
0.000271
s
p95
0.000276
s
p99
0.000276
s
Host function call duration: '__hc__must_get_valid_record_1'
Duration of this host function call invoked from within WASM.
mean
0.004538
s
SD
0.001455
s
p50
0.004109
s
p95
0.006054
s
p99
0.006209
s
Host function call duration: '__hc__update_1'
Duration of this host function call invoked from within WASM.
mean
0.006768
s
SD
0.002151
s
p50
0.0063
s
p95
0.008723
s
p99
0.008883
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.001508
s
SD
0.000103
s
p50
0.00151
s
p95
0.001732
s
p99
0.001736
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.00014
s
SD
4e-06
s
p50
0.00014
s
p95
0.000146
s
p99
0.000159
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
16.73
s
mean
146.73
s
SD
83.67
s
p95
276.73
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 280.006299691s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.000504
s
SD
0
s
p50
0.000504
s
p95
0.000504
s
p99
0.000504
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.001676
s
SD
0.00011
s
p50
0.001673
s
p95
0.00171
s
p99
0.001715
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.046678
s
SD
0.016342
s
p50
0.041658
s
p95
0.060695
s
p99
0.061869
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.003004
s
SD
0
s
p50
0.003004
s
p95
0.003004
s
p99
0.003004
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
0.000372
s
SD
0
s
p50
0.000372
s
p95
0.000372
s
p99
0.000372
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.003302
s
SD
0.001235
s
p50
0.002954
s
p95
0.004326
s
p99
0.004411
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.001483
s
SD
0.000104
s
p50
0.00145
s
p95
0.001545
s
p99
0.001559
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000226
s
SD
4.3e-05
s
p50
0.000225
s
p95
0.00023
s
p99
0.000452
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
63.26
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
7.03
KiB/s
mean
216.17
KiB/s
SD
828.65
KiB/s
p95
1.43
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
5.31
KiB/s
mean
17.67
KiB/s
SD
14.39
KiB/s
p95
58.82
KiB/s
Total bytes received
Total bytes received on primary network interface
count
61.22
MiB
mean
216.17
KiB/s
Total bytes sent
Total bytes sent on primary network interface
count
5.01
MiB
mean
17.67
KiB/s
CPU spike anomaly
CPU spike anomaly was detected
NotDetected
Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
NotDetected
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
63.88
%
mean
77.2
%
SD
6.41
%
p95
83.1
%
System
13.94
%
CPU percentiles
CPU usage percentiles
p50
79.40
%
p95
83.10
%
p99
87.76
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
1
hosts
mean time
134.48
s
Memory used percentage
Percentage of memory used
p5
22.74
%
mean
26.41
%
SD
1.78
%
p95
27.68
%
Memory available percentage
Percentage of available memory
p5
72.32
%
mean
73.59
%
SD
1.78
%
p95
77.26
%
Max host memory used
Maximum memory usage percentage across all hosts
max
26.41
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00
%
Memory growth rate
Rate of memory growth over time
growth
-1.03
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.00
MiB/s
Disk write throughput
Disk write throughput in MiB/s
3.22
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/etc/nomad.d/client-meta.json0/1
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
2.00
5 min
1.41
15 min
0.83
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
0.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
15.15
%
mean
30.86
%
SD
4.8294
%
p95
34.26
%
60 second average
26.9463
%
300 second average
19.8987
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
0.3193
%
SD
0.4457
%
p95
0.91
%
60 second average
0.2437
%
300 second average
0.3910
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
0.1717
%
SD
0.3769
%
p95
0.74
%
60 second average
0.1470
%
300 second average
0.2140
%
Holochain process CPU usage
CPU usage by Holochain process
p5
57.11
%
mean
70.96
%
SD
6.76
%
p95
80.05
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
94.80
KiB
mean
383.31
KiB
SD
85.39
KiB
p95
415.73
KiB
Holochain process threads
Number of threads in Holochain process
p5
15
threads
mean
26.29
threads
SD
4.11
threads
p95
28
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
47
file descriptors
mean
64.97
file descriptors
SD
11.15
file descriptors
p95
72
file descriptors
Write Validated must_get_agent_activity
↑ Back to index
A scenario where write agents create entries in batches of 10, while must_get_agent_activity agents each pick a random write agent and repeatedly attempt to create an entry that references the chain top of their latest batch. This reference means that the entry's validation function needs to make a must_get_agent_activity call.
The purpose of this scenario is to measure the time it takes for published agent activity data to be gossiped among authorities and become available to peers that query it via must_get_agent_activity.
This test is similar to Mixed-Arc must_get_agent_activity, but all agents are full-arc.
Started
Thu, 22 Oct 2026 10:08:38 UTC
Duration
900s
Peer count
30
Peer count at end
17
Behaviours
-
must_get_agent_activity(1 agent) -
write(1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
write_validated_must_get_agent_activity_39722287437_1
Highest observed
action_seqThe maximum chain length observed per
write agent, aggregated across all reading agents. The reading dimension is collapsed by taking the maximum observed value for each write agent (any successful read counts as propagation). This reflects the DHT’s ability to propagate agent activity ops and make them available to querying peers.Mean max (across write agents)
Mean of the highest chain head value observed per write agent. Each write agent contributes its maximum — the value seen by any reader.
1327.75
Max
Highest chain head value observed for any single write agent.
1798
Write agents observed
4
Chain batch delay timing
The time between a
write agent's creation of a batch and a must_get_agent_activity agent's successful discovery of the batch and creation/self-validation of a new entry that depends on it.Mean
mean
323.691486
s
SD
51.223175
s
Max
Highest per-partition mean latency
max mean
1249.710046
s
Min
Lowest per-partition mean latency
min mean
13.006879
s
Chain batch delay rate
The rate at which a
must_get_agent_activity agent was able to discover batches and create/self-validate new entries that depend on them.Mean
mean
3.16
/s
Max
Highest per-partition mean rate
max
5.6
/s
Min
Lowest per-partition mean rate
min
1.38
/s
create_validated_sample_entry zome call timingThe time taken to complete the zome function call that creates the entry that depends on a
write agent's source chain.Mean
mean
0.999726
s
SD
0.893397
s
Max
Highest per-partition mean latency
max mean
5.768522
s
Min
Lowest per-partition mean latency
min mean
0.171797
s
Retrieval errors
A running accumulation of the errors encountered by an agent when attempting to self-validate actions that depend on
must_get_agent_activity calls.Total
1798
Agents affected
9 / 9
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
200
Max
Highest count in any single partition
487
Min
Lowest count in any single partition
8
Final error count
The total number of all types of error accumulated over the run by all agents.
19491
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.028322
s
SD
0.852076
s
p50
0.006375
s
p95
0.511298
s
p99
4.485059
s
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
0
over 200.001325278s
Not enough time data to show a trend.
WASM usage: 'validated_must_get_agent_activity::announce_write_behaviour'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 881.830275995s
Not enough time data to show a trend.
WASM usage: 'validated_must_get_agent_activity::create_sample_entries_batch'
Metered usage count of the WASM ribosome for this zome function.
total
2124527304
over 881.830275995s
mean rate
5.910341787e+07
/s
std
5.020112563e+07
/s
p5
1.31047636e+06
/s
p95
1.6477828353e+08
/s
peak
1.9265179817e+08
/s
WASM usage: 'validated_must_get_agent_activity::create_validated_sample_entry'
Metered usage count of the WASM ribosome for this zome function.
total
1817680
over 877.097916772s
mean rate
9.72087811e+06
/s
std
1.149879428e+07
/s
p5
43422.99
/s
p95
3.328274289e+07
/s
peak
7.058502969e+07
/s
WASM usage: 'validated_must_get_agent_activity::get_chain_top_for_batch'
Metered usage count of the WASM ribosome for this zome function.
total
5559279
over 877.097916772s
mean rate
6.3286814726e+08
/s
std
6.0982988911e+08
/s
p5
1.18217621e+06
/s
p95
1.80249971051e+09
/s
peak
2.1468860277e+09
/s
WASM usage: 'validated_must_get_agent_activity::get_random_agent_with_write_behaviour'
Metered usage count of the WASM ribosome for this zome function.
total
1444009
over 886.818228626s
mean rate
4.53154629e+06
/s
std
3.27903324e+06
/s
p5
115613.41
/s
p95
9.03597039e+06
/s
peak
1.574257372e+07
/s
WASM usage: 'validated_must_get_agent_activity_integrity::entry_defs'
Metered usage count of the WASM ribosome for this zome function.
total
263449278
over 887.100251357s
mean rate
8.6862046959e+08
/s
std
8.8761987559e+09
/s
p5
288142.56
/s
p95
4.5249383607e+08
/s
peak
1.04010170508e+11
/s
WASM usage: 'validated_must_get_agent_activity_integrity::validate'
Metered usage count of the WASM ribosome for this zome function.
total
236307898
over 886.820516598s
mean rate
3.3410750743e+10
/s
std
3.72659858495e+11
/s
p5
7.69835239e+06
/s
p95
7.04863233482e+10
/s
peak
6.41952839327e+12
/s
Zome call duration: 'validated_must_get_agent_activity::announce_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.006045
s
SD
0.001249
s
p50
0.006457
s
p95
0.007532
s
p99
0.007532
s
Zome call duration: 'validated_must_get_agent_activity::create_sample_entries_batch'
Duration of this zome call as measured by Holochain internally.
mean
0.125449
s
SD
0.060603
s
p50
0.099696
s
p95
0.267106
s
p99
0.273843
s
Zome call duration: 'validated_must_get_agent_activity::create_validated_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.009292
s
SD
0.003547
s
p50
0.008235
s
p95
0.014498
s
p99
0.014882
s
Zome call duration: 'validated_must_get_agent_activity::get_chain_top_for_batch'
Duration of this zome call as measured by Holochain internally.
mean
0.051374
s
SD
0.525381
s
p50
0.009517
s
p95
1.619845
s
p99
2.017999
s
Zome call duration: 'validated_must_get_agent_activity::get_random_agent_with_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.168333
s
SD
0.132979
s
p50
0.006678
s
p95
0.51744
s
p99
0.51744
s
WASM call duration: 'validated_must_get_agent_activity::announce_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.005025
s
SD
0.00098
s
p50
0.005735
s
p95
0.005861
s
p99
0.005861
s
WASM call duration: 'validated_must_get_agent_activity::create_sample_entries_batch'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.124326
s
SD
0.060397
s
p50
0.098527
s
p95
0.265516
s
p99
0.272219
s
WASM call duration: 'validated_must_get_agent_activity::create_validated_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.008322
s
SD
0.003374
s
p50
0.007202
s
p95
0.013226
s
p99
0.013584
s
WASM call duration: 'validated_must_get_agent_activity::get_chain_top_for_batch'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.010043
s
SD
0.524996
s
p50
0.008489
s
p95
1.618754
s
p99
2.016981
s
WASM call duration: 'validated_must_get_agent_activity::get_random_agent_with_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.167448
s
SD
0.132922
s
p50
0.00592
s
p95
0.516371
s
p99
0.516371
s
WASM call duration: 'validated_must_get_agent_activity_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.002486
s
SD
0.000946
s
p50
0.002331
s
p95
0.003978
s
p99
0.004111
s
WASM call duration: 'validated_must_get_agent_activity_integrity::validate'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.012004
s
SD
0.259884
s
p50
0.009568
s
p95
0.040411
s
p99
1.452514
s
Host function call duration: '__hc__agent_info_1'
Duration of this host function call invoked from within WASM.
mean
4.7e-05
s
SD
3.9e-05
s
p50
2.4e-05
s
p95
0.000148
s
p99
0.000148
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.001099
s
SD
0.000595
s
p50
0.000895
s
p95
0.002346
s
p99
0.002714
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.001107
s
SD
0.000535
s
p50
0.00051
s
p95
0.001914
s
p99
0.002551
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.003889
s
SD
0.385649
s
p50
0.000703
s
p95
0.511342
s
p99
1.936397
s
Host function call duration: '__hc__must_get_agent_activity_1'
Duration of this host function call invoked from within WASM.
mean
0.03458
s
SD
1.15222
s
p50
0.029693
s
p95
0.104271
s
p99
7.748022
s
Host function call duration: '__hc__query_1'
Duration of this host function call invoked from within WASM.
mean
0.038182
s
SD
0.024316
s
p50
0.026244
s
p95
0.096725
s
p99
0.106842
s
Host function call duration: '__hc__random_bytes_1'
Duration of this host function call invoked from within WASM.
mean
7e-06
s
SD
2e-06
s
p50
6e-06
s
p95
1e-05
s
p99
1e-05
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.004128
s
SD
0.001861
s
p50
0.003789
s
p95
0.007143
s
p99
0.009447
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.000255
s
SD
0.000126
s
p50
0.000205
s
p95
0.000526
s
p99
0.000627
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
37.5
s
mean
426.73
s
SD
253.49
s
p95
836.88
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
30646
over 887.100230756s
mean rate
94862.88
/s
std
931265.21
/s
p5
53.65
/s
p95
79379.5
/s
peak
1.136137142e+07
/s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
24.177927
s
SD
25.642273
s
p50
12.220383
s
p95
63.394596
s
p99
116.340141
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.815194
s
SD
1.954009
s
p50
2.474973
s
p95
5.659999
s
p99
8.57367
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.078297
s
SD
0.096498
s
p50
0.067524
s
p95
0.311392
s
p99
0.401572
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.210428
s
SD
0.135489
s
p50
0.112017
s
p95
0.41525
s
p99
0.595178
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.306052
s
SD
1.096806
s
p50
0.385905
s
p95
2.93501
s
p99
5.889184
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
15.713222
s
SD
17.944221
s
p50
15.318389
s
p95
42.534412
s
p99
71.826581
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.003902
s
SD
0.002128
s
p50
0.002828
s
p95
0.007126
s
p99
0.007935
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.007875
s
SD
0.004698
s
p50
0.002961
s
p95
0.012229
s
p99
0.016523
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000648
s
SD
0.000338
s
p50
0.000549
s
p95
0.001441
s
p99
0.001695
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
0.274454
s
SD
0.433229
s
p50
0.134286
s
p95
0.421435
s
p99
2.96883
s
P2P request roundtrip: 'get_links'
Time spent sending a get_links request and awaiting its response.
mean
1.995747
s
SD
5.129023
s
p50
2.009757
s
p95
11.744153
s
p99
11.744153
s
P2P request roundtrip: 'must_get_agent_activity'
Time spent sending a must_get_agent_activity request and awaiting its response.
mean
5.817935
s
SD
25.316123
s
p50
5.482486
s
p95
60.001717
s
p99
60.001717
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.329023
s
SD
15.818261
s
p50
0.392686
s
p95
60.000792
s
p99
60.002287
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
65.30
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
0
B/s
mean
801.41
KiB/s
SD
1.14
MiB/s
p95
2.32
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
0
B/s
mean
641.88
KiB/s
SD
521.89
KiB/s
p95
1.58
MiB/s
Total bytes received
Total bytes received on primary network interface
count
9.17
GiB
mean
10.55
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
7.35
GiB
mean
8.45
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 98.9%
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 1.30 MiB/s
Disk full anomaly
Disk full anomaly was detected
Detected
Critical Disk /etc/resolv.conf: 1/9 hosts above 90% capacity
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected
Critical Heavy swap usage (88.0% swap used)
System overload anomaly
System overload anomaly was detected
Detected
Warning 48% of hosts overloaded (load5/ncpus > 1.0)
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
5.2
%
mean
77.9
%
SD
31.86
%
p95
98.83
%
System
12.60
%
CPU percentiles
CPU usage percentiles
p50
96.98
%
p95
98.83
%
p99
98.93
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
12
hosts
mean time
736.72
s
Memory used percentage
Percentage of memory used
p5
11.37
%
mean
28.66
%
SD
21.3
%
p95
91.36
%
Memory available percentage
Percentage of available memory
p5
8.64
%
mean
71.34
%
SD
21.3
%
p95
88.63
%
Max host memory used
Maximum memory usage percentage across all hosts
max
92.01
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
88.03
%
Memory growth rate
Rate of memory growth over time
growth
1.30
MiB/s
Disk read throughput
Disk read throughput in MiB/s
3.56
MiB/s
Disk write throughput
Disk write throughput in MiB/s
182.58
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/0/6
hosts
Mount Point
/efi-boot0/1
hosts
Mount Point
/etc/hostname1/9
hosts
Mount Point
/etc/hosts1/9
hosts
Mount Point
/etc/nomad.d/client-meta.json0/8
hosts
Mount Point
/etc/nomad.d/meta.json1/1
hosts
Mount Point
/etc/resolv.conf1/9
hosts
Mount Point
/nix/store0/6
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
16.29
5 min
13.35
15 min
7.76
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
47.83
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
54.7699
%
SD
37.6991
%
p95
97.72
%
60 second average
53.9974
%
300 second average
45.1517
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0.1038
%
SD
1.3516
%
p95
0
%
60 second average
0.1031
%
300 second average
0.0851
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0.0968
%
SD
1.2753
%
p95
0
%
60 second average
0.0960
%
300 second average
0.0793
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
2.5624
%
SD
10.9845
%
p95
7.67
%
60 second average
2.6092
%
300 second average
2.4913
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
1.4201
%
SD
6.4059
%
p95
6.88
%
60 second average
1.4230
%
300 second average
1.3402
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.06
%
mean
8.75
%
SD
8.9
%
p95
24.13
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
24.11
KiB
mean
241.87
KiB
SD
141.86
KiB
p95
501.36
KiB
Holochain process threads
Number of threads in Holochain process
p5
1
threads
mean
33.42
threads
SD
15.12
threads
p95
55
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
4
file descriptors
mean
69.56
file descriptors
SD
27.04
file descriptors
p95
100
file descriptors
Zero-Arc Create and Read
↑ Back to indexA zero-arc/full-arc mixed scenario with two types of zero-arc nodes -- ones that create data and ones that read data -- as well as full arc nodes to "relay" the data. The scenario has three roles:
zero_write: A zero-arc conductor that just creates entries with a timestamp field. Those entries are linked to a known base hash so thatzero_readnodes can retrieve them.zero_read: A zero-arc conductor that reads the entries created by the zero-arc node(s) and records the time lag between when the entry had been created and when it was first discovered.full: A full-arc conductor that is just here to serve entries to zero arc nodes.
Started
Thu, 22 Oct 2026 10:33:07 UTC
Duration
900s
Peer count
30
Peer count at end
22
Behaviours
-
full(1 agent) -
zero_read(1 agent) -
zero_write(1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
zero_arc_create_and_read_39722287437_1
Create rate
The number of timed entries created by the zero-arc write node(s) per second.
Mean
mean
53.05
/s
Max
Highest per-partition mean rate
max
67.82
/s
Min
Lowest per-partition mean rate
min
34.59
/s
Fetch lag timing
For each entry, the time lag between when it was created and when a zero-arc read node fetched it from the network via the
get_timed_entries_network zome function.Mean
mean
236.683938
s
SD
76.506269
s
Max
Highest per-partition mean latency
max mean
400.494488
s
Min
Lowest per-partition mean latency
min mean
49.132149
s
Fetch lag rate
The number of entries per second fetched by zero-arc read nodes.
Mean
mean
311.57
/s
Max
Highest per-partition mean rate
max
631
/s
Min
Lowest per-partition mean rate
min
0
/s
Delivery ratio
The proportion of created entries successfully fetched by all zero-arc read nodes.
0.0092
Entry created count
Total number of entries created by zero-arc write nodes, by agent.
Total
48004
Agents affected
10 / 10
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
4800
Max
Highest count in any single partition
6355
Min
Lowest count in any single partition
2804
Received count
Total number of entries fetched by zero-arc read nodes, by agent.
Total
3089
Agents affected
6 / 7
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
441
Max
Highest count in any single partition
637
Min
Lowest count in any single partition
0
Open connections
The number of currently open connections to other conductors.
full-arc
p5
5
mean
18.59
SD
4.79
p95
21
zero-arc
p5
3
mean
9.38
SD
3.88
p95
15
Error count
The number of errors accumulated across all nodes.
35
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.002868
s
SD
0.132397
s
p50
0.003866
s
p95
0.254423
s
p99
0.326469
s
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
0
over 770.009662616s
Not enough time data to show a trend.
WASM usage: 'timed::created_timed_entry'
Metered usage count of the WASM ribosome for this zome function.
total
217645360
over 878.156610185s
mean rate
1.3981385269e+08
/s
std
2.0934431894e+08
/s
p5
4.03327964e+06
/s
p95
6.2150180982e+08
/s
peak
7.2053580933e+08
/s
WASM usage: 'timed::get_timed_entries_network'
Metered usage count of the WASM ribosome for this zome function.
total
25296698588
over 877.044065021s
mean rate
2.18463932213e+10
/s
std
2.55965561239e+10
/s
p5
794784.77
/s
p95
7.32495671125e+10
/s
peak
7.74121519614e+10
/s
WASM usage: 'timed_integrity::entry_defs'
Metered usage count of the WASM ribosome for this zome function.
total
35228570
over 888.15662774s
mean rate
2.5865415353e+08
/s
std
6.3978014484e+08
/s
p5
27549.43
/s
p95
2.25068282159e+09
/s
peak
7.41449553133e+09
/s
Zome call duration: 'timed::created_timed_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.007457
s
SD
0.001938
s
p50
0.007487
s
p95
0.009966
s
p99
0.010973
s
Zome call duration: 'timed::get_timed_entries_network'
Duration of this zome call as measured by Holochain internally.
mean
1.348401
s
SD
16.635623
s
p50
1.214978
s
p95
26.118848
s
p99
65.673238
s
WASM call duration: 'timed::created_timed_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.006776
s
SD
0.001866
s
p50
0.006826
s
p95
0.009239
s
p99
0.01006
s
WASM call duration: 'timed::get_timed_entries_network'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
1.37202
s
SD
16.618335
s
p50
1.352999
s
p95
26.117886
s
p99
65.672335
s
WASM call duration: 'timed_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001364
s
SD
0.000732
s
p50
0.001148
s
p95
0.003217
s
p99
0.004091
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000499
s
SD
0.000204
s
p50
0.000509
s
p95
0.000827
s
p99
0.000969
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.000478
s
SD
0.000193
s
p50
0.000454
s
p95
0.000805
s
p99
0.000926
s
Host function call duration: '__hc__get_1'
Duration of this host function call invoked from within WASM.
mean
0.002505
s
SD
0.070896
s
p50
0.003497
s
p95
0.19777
s
p99
0.273742
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.224858
s
SD
0.822641
s
p50
0.329716
s
p95
2.10424
s
p99
2.907931
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.002146
s
SD
0.000713
s
p50
0.001853
s
p95
0.003314
s
p99
0.003906
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.000157
s
SD
5.8e-05
s
p50
0.000136
s
p95
0.000242
s
p99
0.000281
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
45.97
s
mean
432.84
s
SD
250.63
s
p95
828.43
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 888.156631967s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
6.586622
s
SD
8.157689
s
p50
0
s
p95
20.195465
s
p99
31.239403
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.080003
s
SD
0.068461
s
p50
0.00138
s
p95
0.093328
s
p99
0.328127
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.019463
s
SD
0.321328
s
p50
0.002212
s
p95
1.092982
s
p99
1.460461
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.123869
s
SD
0.059544
s
p50
0.049584
s
p95
0.170689
s
p99
0.204273
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
9.978728
s
SD
7.437201
s
p50
0.003837
s
p95
13.538497
s
p99
31.111501
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
3.983819
s
SD
6.261169
s
p50
0.002744
s
p95
15.962083
s
p99
28.693864
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
0.001038
s
SD
0.001323
s
p50
0.0015
s
p95
0.003826
s
p99
0.004187
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.003397
s
SD
0.015772
s
p50
0.001887
s
p95
0.044438
s
p99
0.078108
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000447
s
SD
0.000582
s
p50
0.000412
s
p95
0.000692
s
p99
0.003122
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
0.996562
s
SD
29.225699
s
p50
1.468572
s
p95
60.002135
s
p99
60.010496
s
P2P request roundtrip: 'get_links'
Time spent sending a get_links request and awaiting its response.
mean
0.136571
s
SD
19.210534
s
p50
0.185989
s
p95
60.000194
s
p99
60.000194
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.234704
s
SD
20.874032
s
p50
0.64208
s
p95
60.000942
s
p99
60.00099
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
55.13
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
0
B/s
mean
746.29
KiB/s
SD
1.32
MiB/s
p95
2.55
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
0
B/s
mean
443.87
KiB/s
SD
620.39
KiB/s
p95
1.54
MiB/s
Total bytes received
Total bytes received on primary network interface
count
14.17
GiB
mean
16.30
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
8.43
GiB
mean
9.70
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 98.8%
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 1.64 MiB/s
Disk full anomaly
Disk full anomaly was detected
Detected
Critical Disk /etc/resolv.conf: 1/6 hosts above 90% capacity
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected
Critical Heavy swap usage (63.9% swap used)
System overload anomaly
System overload anomaly was detected
Detected
Warning 59% of hosts overloaded (load5/ncpus > 1.0)
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
15.94
%
mean
68.52
%
SD
29.04
%
p95
98.6
%
System
13.39
%
CPU percentiles
CPU usage percentiles
p50
75.20
%
p95
98.60
%
p99
98.80
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
18
hosts
mean time
512.49
s
Memory used percentage
Percentage of memory used
p5
14.4
%
mean
25.22
%
SD
9.02
%
p95
32.74
%
Memory available percentage
Percentage of available memory
p5
67.26
%
mean
74.78
%
SD
9.02
%
p95
85.6
%
Max host memory used
Maximum memory usage percentage across all hosts
max
63.80
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
63.86
%
Memory growth rate
Rate of memory growth over time
growth
1.64
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.24
MiB/s
Disk write throughput
Disk write throughput in MiB/s
188.53
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/0/10
hosts
Mount Point
/efi-boot0/6
hosts
Mount Point
/etc/hostname1/6
hosts
Mount Point
/etc/hosts1/6
hosts
Mount Point
/etc/nomad.d/client-meta.json0/11
hosts
Mount Point
/etc/nomad.d/meta.json1/1
hosts
Mount Point
/etc/resolv.conf1/6
hosts
Mount Point
/nix/store0/10
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
7.51
5 min
7.52
15 min
8.28
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
59.26
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0.34
%
mean
36.9479
%
SD
34.2055
%
p95
90.57
%
60 second average
36.0713
%
300 second average
36.0154
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0001
%
300 second average
0.0002
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
9.1137
%
SD
17.7835
%
p95
50.5
%
60 second average
9.2012
%
300 second average
10.3349
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
6.3965
%
SD
13.5209
%
p95
37.92
%
60 second average
6.4842
%
300 second average
6.5046
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.08
%
mean
19.31
%
SD
21.89
%
p95
57.33
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
24.51
KiB
mean
286.68
KiB
SD
115.50
KiB
p95
470.54
KiB
Holochain process threads
Number of threads in Holochain process
p5
1
threads
mean
30.93
threads
SD
11.61
threads
p95
51
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
7
file descriptors
mean
65.84
file descriptors
SD
22.19
file descriptors
p95
87
file descriptors
Zome Call, Single Value
↑ Back to indexCalls a zome function that returns a fixed value. This tests the maximum performance of zome calls when the zome function is not calling into Holochain other otherwise doing any work.
Started
Thu, 22 Oct 2026 09:55:06 UTC
Duration
300s
Peer count
1
Peer count at end
1
Behaviours
-
default(1 agent)
Holochain version
0.7.0
Wind Tunnel version
0.7.0
Run ID
zome_call_single_value_39722287437_1
Call timing
The time taken for the call to complete.
mean
0.003192
s
SD
0.006169
s
p50
0.002822
s
p95
0.004703
s
p99
0.007381
s
Call rate
The number of zome calls completed per second.
mean
3106.1
/s
Error count
The number of errors accumulated during the run.
0
Holochain MetricsClick to toggle visibility.
WASM usage: 'return_single_value::get_value'
Metered usage count of the WASM ribosome for this zome function.
total
355641190
over 280.004123064s
mean rate
1.27013e+06
/s
std
175528.27
/s
p5
912755.21
/s
p95
1.41204514e+06
/s
peak
1.42313144e+06
/s
Zome call duration: 'return_single_value::get_value'
Duration of this zome call as measured by Holochain internally.
mean
0.001341
s
SD
1.1e-05
s
p50
0.001338
s
p95
0.001356
s
p99
0.001381
s
WASM call duration: 'return_single_value::get_value'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.000874
s
SD
9e-06
s
p50
0.000874
s
p95
0.000884
s
p99
0.000914
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.000152
s
SD
0
s
p50
0.000152
s
p95
0.000152
s
p99
0.000152
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
16.66
s
mean
146.66
s
SD
83.67
s
p95
276.66
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 280.004110866s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.000606
s
SD
0
s
p50
0.000606
s
p95
0.000606
s
p99
0.000606
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.001518
s
SD
0
s
p50
0.001518
s
p95
0.001518
s
p99
0.001518
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.00524
s
SD
0.001197
s
p50
0.005575
s
p95
0.007594
s
p99
0.007594
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.003543
s
SD
0
s
p50
0.003543
s
p95
0.003543
s
p99
0.003543
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
0.000607
s
SD
0
s
p50
0.000607
s
p95
0.000607
s
p99
0.000607
s
DHT DB connection use time
Time spent holding DHT database connections.
mean
4.6e-05
s
SD
2e-06
s
p50
4.5e-05
s
p95
4.7e-05
s
p99
5.4e-05
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.001147
s
SD
0.000113
s
p50
0.001299
s
p95
0.001299
s
p99
0.001299
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.00031
s
SD
0
s
p50
0.00031
s
p95
0.00031
s
p99
0.00031
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
40.02
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
5.35
KiB/s
mean
1.21
MiB/s
SD
2.31
MiB/s
p95
5.87
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
4.92
KiB/s
mean
32.07
KiB/s
SD
40.64
KiB/s
p95
127.99
KiB/s
Total bytes received
Total bytes received on primary network interface
count
351.17
MiB
mean
1.21
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
9.08
MiB
mean
32.07
KiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 96.6%
Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
Detected
Warning 100% of hosts overloaded (load5/ncpus > 1.0)
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
55.3
%
mean
62.77
%
SD
9.23
%
p95
87.45
%
System
22.74
%
CPU percentiles
CPU usage percentiles
p50
60.02
%
p95
87.45
%
p99
96.60
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
1
hosts
mean time
20.69
s
Memory used percentage
Percentage of memory used
p5
15.99
%
mean
23.43
%
SD
4.79
%
p95
31.16
%
Memory available percentage
Percentage of available memory
p5
68.84
%
mean
76.57
%
SD
4.79
%
p95
84.01
%
Max host memory used
Maximum memory usage percentage across all hosts
max
23.43
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00
%
Memory growth rate
Rate of memory growth over time
growth
-0.67
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.01
MiB/s
Disk write throughput
Disk write throughput in MiB/s
4.12
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/etc/nomad.d/client-meta.json0/1
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
4.95
5 min
3.46
15 min
2.01
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
100.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
20.32
%
mean
32.9247
%
SD
7.7727
%
p95
52.34
%
60 second average
30.6203
%
300 second average
26.7177
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0.1407
%
SD
0.4527
%
p95
1.63
%
60 second average
0.0527
%
300 second average
0.0123
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0.069
%
SD
0.2225
%
p95
0.72
%
60 second average
0.0267
%
300 second average
0.0060
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0.08
%
mean
4.0893
%
SD
6.5107
%
p95
26.33
%
60 second average
2.4773
%
300 second average
1.2403
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
2.1813
%
SD
3.2594
%
p95
12.93
%
60 second average
1.3017
%
300 second average
0.6387
%
Holochain process CPU usage
CPU usage by Holochain process
p5
32.33
%
mean
45.45
%
SD
9.88
%
p95
51.23
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
217.54
KiB
mean
312.03
KiB
SD
361.20
KiB
p95
1.20
MiB
Holochain process threads
Number of threads in Holochain process
p5
5
threads
mean
44.39
threads
SD
90.8
threads
p95
269
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
8
file descriptors
mean
106.65
file descriptors
SD
191.42
file descriptors
p95
581
file descriptors