App Install
↑ Back to indexMeasure app installation speed. This scenario has two roles:
minimal: Installs a simple app which implements initialisation callbacks but otherwise doesn't contain a lot of code.large: Installs a larger app that contains some dependencies and some generated data as padding to make the bundle larger.
These are intended to catch a compilation slowdown or issues with copying around large WASMs (e.g. accidental cloning or tracing). In each case, the behaviour will uninstall the app it installed so that it can re-install it on the next iteration.
Started
Thu, 02 Jul 2026 01:33:55 UTC
Peer count
1
Behaviours
-
large(1 agent)
Holochain version
0.7.0-dev.31
Wind Tunnel version
0.7.0
Run ID
app_install_large_28558451397_1
First install time
The time from conductor startup to completion of the first app installation attempt.
0.171636825
s
Install app timing
The time taken to install an app.
mean
0.060179
s
SD
0.026184
s
p50
0.051186
s
p95
0.092324
s
p99
0.216
s
Holochain MetricsClick to toggle visibility.
WASM usage: 'large::init'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 270.672360027s
Not enough time data to show a trend.
WASM call duration: 'large::init'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.0033
s
SD
0.000966
s
p50
0.002978
s
p95
0.004405
s
p99
0.008592
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.023693
s
SD
0.004186
s
p50
0.022145
s
p95
0.033582
s
p99
0.036997
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
15.96
s
mean
146.04
s
SD
83.78
s
p95
276.64
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 280.671658784s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.00123
s
SD
1.2e-05
s
p50
0.001231
s
p95
0.001231
s
p99
0.001262
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0.003086
s
SD
0.000115
s
p50
0.003077
s
p95
0.003077
s
p99
0.003565
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.002214
s
SD
0.000227
s
p50
0.002212
s
p95
0.002337
s
p99
0.003456
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.003452
s
SD
0.000435
s
p50
0.003447
s
p95
0.003691
s
p99
0.005826
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.002931
s
SD
0.000133
s
p50
0.002932
s
p95
0.002932
s
p99
0.003654
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
0.001953
s
SD
6e-06
s
p50
0.001953
s
p95
0.001953
s
p99
0.001981
s
Authored DB connection use time
Time spent holding authored database connections.
mean
0.000301
s
SD
0.000318
s
p50
0.00022
s
p95
0.000557
s
p99
0.002299
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.000341
s
SD
7.8e-05
s
p50
0.000362
s
p95
0.000515
s
p99
0.000627
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000289
s
SD
9e-06
s
p50
0.000289
s
p95
0.000289
s
p99
0.000318
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
64.43
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
345
B/s
mean
389.73
KiB/s
SD
1.22
MiB/s
p95
2.30
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
6
B/s
mean
66.14
KiB/s
SD
127.45
KiB/s
p95
405.45
KiB/s
Total bytes received
Total bytes received on primary network interface
count
114.30
MiB
mean
390.14
KiB/s
Total bytes sent
Total bytes sent on primary network interface
count
19.45
MiB
mean
66.39
KiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 96.8%
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 287.77 MiB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
Detected
Warning 100% of hosts overloaded (load5/ncpus > 1.0)
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
83.92
%
mean
87.3
%
SD
5.96
%
p95
96
%
System
22.86
%
CPU percentiles
CPU usage percentiles
p50
85.86
%
p95
96.00
%
p99
96.84
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
1
hosts
mean time
0.97
s
Memory used percentage
Percentage of memory used
p5
14.69
%
mean
17.73
%
SD
2.15
%
p95
20.65
%
Memory available percentage
Percentage of available memory
p5
79.35
%
mean
82.27
%
SD
2.15
%
p95
85.31
%
Max host memory used
Maximum memory usage percentage across all hosts
max
17.73
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00
%
Memory growth rate
Rate of memory growth over time
growth
287.77
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.00
MiB/s
Disk write throughput
Disk write throughput in MiB/s
3.98
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/etc/nomad.d/client-meta.json0/1
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
4.74
5 min
3.65
15 min
1.94
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
100.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
40.26
%
mean
66.4123
%
SD
15.483
%
p95
95.94
%
60 second average
58.5713
%
300 second average
46.9461
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
0.7374
%
SD
1.5335
%
p95
3.63
%
60 second average
1.2284
%
300 second average
2.4384
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
0.2423
%
SD
0.8087
%
p95
1.6
%
60 second average
0.4303
%
300 second average
0.6452
%
Holochain process CPU usage
CPU usage by Holochain process
p5
40.87
%
mean
56.3
%
SD
6.54
%
p95
60.98
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
1.84
KiB
mean
360.70
KiB
SD
169.15
KiB
p95
581.22
KiB
Holochain process threads
Number of threads in Holochain process
p5
1
threads
mean
95.32
threads
SD
152.77
threads
p95
440
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
6
file descriptors
mean
242.91
file descriptors
SD
386.1
file descriptors
p95
1111
file descriptors
DHT Sync Lag
↑ Back to indexMeasures the lag time between an agent publishing data and other peers being able to see it. This scenario has two roles:
write: Creates entries with a timestamp field, linked to a known base hash so thatrecord_lagagents can discover them.record_lag: Repeatedly queries for links from the known base hash. When a new entry is discovered, it calculates the time difference between the entry's timestamp and the current time, giving the sync lag for that entry.
Started
Thu, 02 Jul 2026 01:37:52 UTC
Peer count
30
Peer count at end
26
Behaviours
-
record_lag(1 agent) -
write(1 agent)
Holochain version
0.7.0-dev.31
Wind Tunnel version
0.7.0
Run ID
dht_sync_lag_28558451397_1
Create rate
The average number of created records per agent.
Mean
mean
44.17
/s
Max
Highest per-partition mean rate
max
79.79
/s
Min
Lowest per-partition mean rate
min
9.07
/s
Sync lag timing
The average time between when a record was created and when it is first seen by an agent.
Mean
mean
5.54703363354e+30
s
SD
3.81027802845e+31
s
Max
Highest per-partition mean latency
max mean
9.303494704550001e+30
s
Min
Lowest per-partition mean latency
min mean
248.2212
s
Sync lag rate
The average number of created records discovered per agent.
Mean
mean
110.79
/s
Max
Highest per-partition mean rate
max
184.92
/s
Min
Lowest per-partition mean rate
min
46.96
/s
Delivery ratio
The proportion of created entries successfully received by all record_lag agents.
0.0182
Sent count
The total number of entries sent (created) by write agents, by agent.
Total
162455
Agents affected
23 / 23
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
7063
Max
Highest count in any single partition
14154
Min
Lowest count in any single partition
1426
Received count
The total number of entries received by record_lag agents, by agent.
Total
14777
Agents affected
5 / 5
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
2955
Max
Highest count in any single partition
4840
Min
Lowest count in any single partition
1416
Error count
The number of errors encountered during the scenario.
7240
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.001233
s
SD
0.001326
s
p50
0.002147
s
p95
0.004539
s
p99
0.004858
s
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
4
over 2743.857089576s
mean rate
1.36
/s
std
1.17
/s
p5
0
/s
p95
2.69
/s
peak
2.71
/s
WASM usage: 'timed::created_timed_entry'
Metered usage count of the WASM ribosome for this zome function.
total
218246868
over 33983.608257596s
mean rate
6.4541808894e+08
/s
std
2.09290928737e+09
/s
p5
40869.75
/s
p95
2.38561645071e+09
/s
peak
4.35564902107e+10
/s
WASM usage: 'timed::get_timed_entries_local'
Metered usage count of the WASM ribosome for this zome function.
total
16788122879
over 1802.676155391s
mean rate
2.36356440211e+10
/s
std
3.22349671689e+10
/s
p5
2.0229321486e+08
/s
p95
9.10066722931e+10
/s
peak
9.89360063638e+10
/s
WASM usage: 'timed_integrity::entry_defs'
Metered usage count of the WASM ribosome for this zome function.
total
27832008
over 33983.608257596s
mean rate
8.097196353e+07
/s
std
2.3030647707e+08
/s
p5
8687.9
/s
p95
2.8079420033e+08
/s
peak
5.55542356461e+09
/s
Zome call duration: 'timed::created_timed_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.044645
s
SD
0.0294
s
p50
0.036918
s
p95
0.130864
s
p99
0.140765
s
Zome call duration: 'timed::get_timed_entries_local'
Duration of this zome call as measured by Holochain internally.
mean
0.265264
s
SD
0.54599
s
p50
0.359001
s
p95
1.599165
s
p99
1.757824
s
WASM call duration: 'timed::created_timed_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.031701
s
SD
0.022623
s
p50
0.026981
s
p95
0.099296
s
p99
0.107407
s
WASM call duration: 'timed::get_timed_entries_local'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.258655
s
SD
0.541444
s
p50
0.349545
s
p95
1.584388
s
p99
1.742466
s
WASM call duration: 'timed_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001486
s
SD
0.000981
s
p50
0.001512
s
p95
0.004181
s
p99
0.004993
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.001353
s
SD
0.003516
s
p50
0.000859
s
p95
0.013155
s
p99
0.014867
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.001411
s
SD
0.003669
s
p50
0.000911
s
p95
0.013917
s
p99
0.015314
s
Host function call duration: '__hc__get_1'
Duration of this host function call invoked from within WASM.
mean
0.001246
s
SD
0.00131
s
p50
0.002237
s
p95
0.004545
s
p99
0.004861
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.00698
s
SD
0.013728
s
p50
0.009887
s
p95
0.045996
s
p99
0.056513
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.013352
s
SD
0.007618
s
p50
0.01206
s
p95
0.033206
s
p99
0.037106
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.013024
s
SD
0.015391
s
p50
0.012774
s
p95
0.061318
s
p99
0.062536
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
93.28
s
mean
903.5
s
SD
520.35
s
p95
1716.58
s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
46.10867
s
SD
108.340862
s
p50
8.605009
s
p95
158.780046
s
p99
364.882021
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
1.224665
s
SD
10.538004
s
p50
2.539907
s
p95
34.012241
s
p99
44.424017
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0.001662
s
SD
0.001462
s
p50
0.001022
s
p95
0.004272
s
p99
0.007127
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.131964
s
SD
0.376841
s
p50
0.116556
s
p95
1.177479
s
p99
1.583728
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.330349
s
SD
4.624351
s
p50
0.218329
s
p95
1.338574
s
p99
37.088924
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
7.949482
s
SD
45.198845
s
p50
22.467175
s
p95
157.134075
s
p99
175.282428
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
35.95938
s
SD
67.480049
s
p50
0.115136
s
p95
166.410325
s
p99
397.891687
s
Authored DB connection use time
Time spent holding authored database connections.
mean
0.007576
s
SD
0.004057
s
p50
0.00288
s
p95
0.012254
s
p99
0.013837
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.004453
s
SD
0.011401
s
p50
0.001321
s
p95
0.034589
s
p99
0.054845
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.001248
s
SD
0.002967
s
p50
0.000723
s
p95
0.011074
s
p99
0.011832
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
0.77772
s
SD
29.689536
s
p50
0.221241
s
p95
60.013133
s
p99
60.14748
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.475963
s
SD
4.53536
s
p50
0.740176
s
p95
10.323155
s
p99
11.949113
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
53.50
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
4.59
KiB/s
mean
874.69
KiB/s
SD
593.68
KiB/s
p95
1.65
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
3.85
KiB/s
mean
801.52
KiB/s
SD
753.48
KiB/s
p95
2.27
MiB/s
Total bytes received
Total bytes received on primary network interface
count
40.61
GiB
mean
22.98
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
38.57
GiB
mean
21.82
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 98.5%
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 829.96 MiB/s
Disk full anomaly
Disk full anomaly was detected
Detected
Critical Disk /etc/resolv.conf: 1/6 hosts above 90% capacity
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected
Critical Heavy swap usage (23.4% swap used)
System overload anomaly
System overload anomaly was detected
Detected
Warning 64% of hosts overloaded (load5/ncpus > 1.0)
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
8.65
%
mean
65.02
%
SD
27.52
%
p95
95.19
%
System
11.52
%
CPU percentiles
CPU usage percentiles
p50
75.00
%
p95
95.19
%
p99
98.45
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
18
hosts
mean time
0.66
s
Memory used percentage
Percentage of memory used
p5
7.86
%
mean
18.59
%
SD
13.77
%
p95
40.61
%
Memory available percentage
Percentage of available memory
p5
59.39
%
mean
81.41
%
SD
13.77
%
p95
92.14
%
Max host memory used
Maximum memory usage percentage across all hosts
max
80.00
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
23.40
%
Memory growth rate
Rate of memory growth over time
growth
829.96
MiB/s
Disk read throughput
Disk read throughput in MiB/s
2.54
MiB/s
Disk write throughput
Disk write throughput in MiB/s
265.25
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/0/7
hosts
Mount Point
/efi-boot0/5
hosts
Mount Point
/etc/hostname1/6
hosts
Mount Point
/etc/hosts1/6
hosts
Mount Point
/etc/nomad.d/client-meta.json0/6
hosts
Mount Point
/etc/resolv.conf1/6
hosts
Mount Point
/nix/store0/7
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
5.71
5 min
5.01
15 min
3.50
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
64.29
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0.43
%
mean
28.9407
%
SD
21.0071
%
p95
63.81
%
60 second average
28.2176
%
300 second average
24.8800
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0.0043
%
SD
0.1612
%
p95
0
%
60 second average
0.0034
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0.0032
%
SD
0.1174
%
p95
0
%
60 second average
0.0020
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
15.1838
%
SD
22.5253
%
p95
63.12
%
60 second average
14.7672
%
300 second average
12.7841
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
9.8401
%
SD
17.6058
%
p95
54.31
%
60 second average
9.5722
%
300 second average
8.3138
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0
%
mean
42.46
%
SD
36.08
%
p95
92.19
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
107.67
KiB
mean
516.67
KiB
SD
272.21
KiB
p95
963.53
KiB
Holochain process threads
Number of threads in Holochain process
p5
8
threads
mean
42.34
threads
SD
14.57
threads
p95
61
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
8
file descriptors
mean
95.17
file descriptors
SD
26.44
file descriptors
p95
121
file descriptors
First Call
↑ Back to indexThis scenario installs a simple app which implements the init callback, and measures the time elapsed between start of install and calling init. It will uninstall the app it installed so that it can re-install it on the next iteration. This is required to re-run the initialisation callback.
Started
Thu, 02 Jul 2026 01:13:57 UTC
Peer count
1
Behaviours
-
default(1 agent)
Holochain version
0.7.0-dev.31
Wind Tunnel version
0.7.0
Run ID
first_call_28558451397_1
Zome call timing
The time elapsed between start of install and calling
init.mean
0.033535
s
SD
0.012078
s
p50
0.028094
s
p95
0.060872
s
p99
0.085749
s
Holochain MetricsClick to toggle visibility.
WASM usage: 'crud::create_sample_entry'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 280.001383513s
Not enough time data to show a trend.
WASM usage: 'crud::init'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 280.001383513s
Not enough time data to show a trend.
WASM usage: 'crud_integrity::entry_defs'
Metered usage count of the WASM ribosome for this zome function.
total
189286
over 280.001383513s
mean rate
676.04
/s
std
3512.8
/s
p5
0
/s
p95
0
/s
peak
18929.1
/s
Zome call duration: 'crud::create_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.026219
s
SD
0.000149
s
p50
0.026235
s
p95
0.026235
s
p99
0.026235
s
WASM call duration: 'crud::create_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.013112
s
SD
0.004954
s
p50
0.012241
s
p95
0.020173
s
p99
0.035857
s
WASM call duration: 'crud::init'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.00116
s
SD
0.000227
s
p50
0.001118
s
p95
0.001303
s
p99
0.002625
s
WASM call duration: 'crud_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.000966
s
SD
8e-06
s
p50
0.000965
s
p95
0.000965
s
p99
0.001008
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000273
s
SD
1.2e-05
s
p50
0.000285
s
p95
0.000285
s
p99
0.000289
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.01164
s
SD
0.000215
s
p50
0.011662
s
p95
0.011662
s
p99
0.011662
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.009653
s
SD
0.002071
s
p50
0.009934
s
p95
0.012832
s
p99
0.014377
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
18.21
s
mean
148.21
s
SD
83.67
s
p95
278.21
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 280.00152895s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.000892
s
SD
2e-06
s
p50
0.000892
s
p95
0.000892
s
p99
0.000902
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0.001377
s
SD
3.4e-05
s
p50
0.00138
s
p95
0.00138
s
p99
0.00138
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.001206
s
SD
6e-06
s
p50
0.001206
s
p95
0.001206
s
p99
0.001206
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.00213
s
SD
9e-06
s
p50
0.002131
s
p95
0.002131
s
p99
0.002131
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.002006
s
SD
1.8e-05
s
p50
0.002004
s
p95
0.002004
s
p99
0.002104
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
0.001907
s
SD
3.8e-05
s
p50
0.001903
s
p95
0.001903
s
p99
0.00211
s
Authored DB connection use time
Time spent holding authored database connections.
mean
0.000151
s
SD
5.4e-05
s
p50
0.000128
s
p95
0.000277
s
p99
0.000303
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.00033
s
SD
6.3e-05
s
p50
0.000377
s
p95
0.000467
s
p99
0.000544
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000229
s
SD
1e-06
s
p50
0.000229
s
p95
0.000229
s
p99
0.000235
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
32.33
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
59
B/s
mean
334.90
KiB/s
SD
1.27
MiB/s
p95
2.78
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
21
B/s
mean
6.53
KiB/s
SD
13.75
KiB/s
p95
40.26
KiB/s
Total bytes received
Total bytes received on primary network interface
count
94.84
MiB
mean
334.90
KiB/s
Total bytes sent
Total bytes sent on primary network interface
count
1.85
MiB
mean
6.53
KiB/s
CPU spike anomaly
CPU spike anomaly was detected
NotDetected
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 245.62 MiB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
NotDetected
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
44.06
%
mean
53.53
%
SD
3.76
%
p95
56.22
%
System
21.20
%
CPU percentiles
CPU usage percentiles
p50
53.82
%
p95
56.22
%
p99
64.88
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
0
hosts
mean time
0.00
s
Memory used percentage
Percentage of memory used
p5
6.53
%
mean
6.96
%
SD
0.96
%
p95
9.15
%
Memory available percentage
Percentage of available memory
p5
90.85
%
mean
93.04
%
SD
0.96
%
p95
93.47
%
Max host memory used
Maximum memory usage percentage across all hosts
max
6.96
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00
%
Memory growth rate
Rate of memory growth over time
growth
245.62
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.00
MiB/s
Disk write throughput
Disk write throughput in MiB/s
6.43
MiB/s
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
2.56
5 min
1.25
15 min
0.52
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
0.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
1.33
%
mean
11.3847
%
SD
3.3939
%
p95
15.41
%
60 second average
9.1423
%
300 second average
4.1317
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
0.012
%
SD
0.045
%
p95
0.08
%
60 second average
0.0127
%
300 second average
0.0013
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
0.0033
%
SD
0.0147
%
p95
0.02
%
60 second average
0.0023
%
300 second average
0.0000
%
Holochain process CPU usage
CPU usage by Holochain process
p5
2.35
%
mean
27.47
%
SD
8.95
%
p95
30.75
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
214.14
KiB
mean
341.34
KiB
SD
47.42
KiB
p95
370.45
KiB
Holochain process threads
Number of threads in Holochain process
p5
9
threads
mean
97.67
threads
SD
144.91
threads
p95
403
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
23
file descriptors
mean
245.13
file descriptors
SD
374.05
file descriptors
p95
1027
file descriptors
Full-Arc Create (Validated) / Zero-Arc Read
↑ Back to indexA full-arc/zero-arc mixed scenario where full-arc nodes create data that gets validated and zero-arc nodes read the data. The scenario has two roles:
full: A full-arc conductor that creates entries with a timestamp field. Those entries get validated and then retrieved by zero-arc nodes.zero: A zero-arc conductor that reads the entries created by the full-arc node(s) and records the time lag between when the entry had been created and when it was first discovered.
Started
Thu, 02 Jul 2026 04:44:42 UTC
Peer count
30
Peer count at end
26
Behaviours
-
full(1 agent) -
zero(1 agent)
Holochain version
0.7.0-dev.31
Wind Tunnel version
0.7.0
Run ID
full_arc_create_validated_zero_arc_read--high_zero_28558451397_1
Create rate
The number of timed entries created by the full-arc node(s) per second.
Mean
mean
21.9
/s
Max
Highest per-partition mean rate
max
31.47
/s
Min
Lowest per-partition mean rate
min
12.32
/s
Fetch lag timing
For each entry, the time lag between when it was created by a full-arc node and when a zero-arc node fetched it from the network.
Mean
mean
398.339589
s
SD
94.343392
s
Max
Highest per-partition mean latency
max mean
1343.852345
s
Min
Lowest per-partition mean latency
min mean
122.502293
s
Fetch rate
The number of entries per second fetched by zero-arc nodes.
Mean
mean
52.18
/s
Max
Highest per-partition mean rate
max
220.25
/s
Min
Lowest per-partition mean rate
min
0
/s
Delivery ratio
The proportion of created entries successfully fetched by all zero-arc nodes.
0.1391
Entry created count
Total number of entries created by full-arc nodes, by agent.
Total
3889
Agents affected
2 / 2
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
1945
Max
Highest count in any single partition
2818
Min
Lowest count in any single partition
1071
Received count
Total number of entries fetched by zero-arc nodes, by agent.
Total
13526
Agents affected
10 / 25
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
541
Max
Highest count in any single partition
3175
Min
Lowest count in any single partition
0
Open connections
The number of currently open connections to other conductors.
full-arc
p5
1
mean
19.9
SD
7.68
p95
24
zero-arc
p5
0
mean
5.15
SD
5.91
p95
17
Retrieval errors
Errors encountered by zero-arc nodes when attempting to retrieve entries created by full-arc nodes.
Total
251
Agents affected
22 / 25
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
10
Max
Highest count in any single partition
12
Min
Lowest count in any single partition
0
Error count
The number of errors accumulated across all nodes.
251
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.012223
s
SD
0.024391
s
p50
0.011039
s
p95
0.073739
s
p99
0.092041
s
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
4
over 668.281021018s
mean rate
10
/s
std
25.67
/s
p5
0
/s
p95
106.83
/s
peak
122.52
/s
WASM usage: 'timed_and_validated::create_timed_entry'
Metered usage count of the WASM ribosome for this zome function.
total
76361620
over 891.269207318s
mean rate
6.92880464e+06
/s
std
4.56720785e+06
/s
p5
481351.82
/s
p95
1.386124627e+07
/s
peak
1.429244803e+07
/s
WASM usage: 'timed_and_validated::get_timed_entries_network'
Metered usage count of the WASM ribosome for this zome function.
total
2715729185
over 37458.654142808s
mean rate
1.76889189639e+09
/s
std
5.08992083927e+09
/s
p5
2.27813479e+06
/s
p95
4.70144913562e+09
/s
peak
3.93393649889e+10
/s
WASM usage: 'timed_and_validated_integrity::entry_defs'
Metered usage count of the WASM ribosome for this zome function.
total
278422144
over 37460.289801544s
mean rate
2.6753886944e+08
/s
std
7.5162358371e+08
/s
p5
15645.54
/s
p95
2.56283868463e+09
/s
peak
3.79033483344e+09
/s
WASM usage: 'timed_and_validated_integrity::validate'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 37458.654142808s
Not enough time data to show a trend.
Zome call duration: 'timed_and_validated::create_timed_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.048006
s
SD
0.015748
s
p50
0.035344
s
p95
0.069291
s
p99
0.072708
s
Zome call duration: 'timed_and_validated::get_timed_entries_network'
Duration of this zome call as measured by Holochain internally.
mean
0.023008
s
SD
0.024587
s
p50
0.015122
s
p95
0.062336
s
p99
0.114239
s
WASM call duration: 'timed_and_validated::create_timed_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.035232
s
SD
0.011008
s
p50
0.027205
s
p95
0.050088
s
p99
0.052557
s
WASM call duration: 'timed_and_validated::get_timed_entries_network'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.031459
s
SD
0.037549
s
p50
0.020399
s
p95
0.104111
s
p99
0.144725
s
WASM call duration: 'timed_and_validated_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001027
s
SD
0.000561
s
p50
0.000869
s
p95
0.001941
s
p99
0.003185
s
WASM call duration: 'timed_and_validated_integrity::validate'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.011431
s
SD
0.003547
s
p50
0.00084
s
p95
0.012422
s
p99
0.01391
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000505
s
SD
0.000104
s
p50
0.000507
s
p95
0.000677
s
p99
0.000689
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.000722
s
SD
0.000369
s
p50
0.000601
s
p95
0.001616
s
p99
0.00163
s
Host function call duration: '__hc__get_1'
Duration of this host function call invoked from within WASM.
mean
0.016617
s
SD
0.090784
s
p50
0.08212
s
p95
0.289564
s
p99
0.358175
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.004043
s
SD
0.006874
s
p50
0.000997
s
p95
0.019365
s
p99
0.027156
s
Host function call duration: '__hc__must_get_valid_record_1'
Duration of this host function call invoked from within WASM.
mean
0.007418
s
SD
0.002779
s
p50
0.005772
s
p95
0.010515
s
p99
0.011117
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.005287
s
SD
0.003277
s
p50
0.00515
s
p95
0.012962
s
p99
0.021346
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.014406
s
SD
0.010582
s
p50
0.018073
s
p95
0.035897
s
p99
0.064824
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
47
s
mean
450.09
s
SD
258.64
s
p95
855.87
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 37460.289906145s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
18.241519
s
SD
4.455794
s
p50
0
s
p95
3.885201
s
p99
23.862155
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
3.428817
s
SD
2.172217
s
p50
0.001129
s
p95
2.639553
s
p99
15.60792
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0.001997
s
SD
0.001962
s
p50
0.001235
s
p95
0.005328
s
p99
0.009549
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.023765
s
SD
0.008926
s
p50
0.001957
s
p95
0.012733
s
p99
0.05744
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.10179
s
SD
0.022794
s
p50
0.007711
s
p95
0.064085
s
p99
0.136622
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
1.075206
s
SD
0.982164
s
p50
0.004572
s
p95
0.662809
s
p99
4.788804
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
3.076298
s
SD
3.273693
s
p50
0.004206
s
p95
0.025388
s
p99
12.734415
s
Authored DB connection use time
Time spent holding authored database connections.
mean
0.000137
s
SD
0.000601
s
p50
9.7e-05
s
p95
0.001462
s
p99
0.003578
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.000804
s
SD
0.006136
s
p50
0.000199
s
p95
0.010273
s
p99
0.034633
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000359
s
SD
0.000116
s
p50
0.000259
s
p95
0.000483
s
p99
0.000736
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
0.476689
s
SD
29.812955
s
p50
0.160273
s
p95
60.001451
s
p99
60.002226
s
P2P request roundtrip: 'get_links'
Time spent sending a get_links request and awaiting its response.
mean
11.607447
s
SD
28.442202
s
p50
1.067952
s
p95
60.001704
s
p99
60.002719
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
1.496605
s
SD
12.466449
s
p50
1.558181
s
p95
53.773845
s
p99
53.773845
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
8.94
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
427
B/s
mean
41.56
KiB/s
SD
183.42
KiB/s
p95
173.93
KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
182
B/s
mean
28.92
KiB/s
SD
160.71
KiB/s
p95
83.94
KiB/s
Total bytes received
Total bytes received on primary network interface
count
984.28
MiB
mean
1.08
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
684.87
MiB
mean
770.66
KiB/s
CPU spike anomaly
CPU spike anomaly was detected
NotDetected
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 428.04 MiB/s
Disk full anomaly
Disk full anomaly was detected
Detected
Critical Disk /etc/resolv.conf: 1/7 hosts above 90% capacity
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected
Critical Heavy swap usage (24.3% swap used)
System overload anomaly
System overload anomaly was detected
Detected
Warning 4% of hosts overloaded (load5/ncpus > 1.0)
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
0.5
%
mean
12.83
%
SD
16.44
%
p95
52.24
%
System
3.89
%
CPU percentiles
CPU usage percentiles
p50
4.20
%
p95
52.24
%
p99
66.09
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
3
hosts
mean time
0.01
s
Memory used percentage
Percentage of memory used
p5
7.07
%
mean
16.57
%
SD
13.62
%
p95
39.56
%
Memory available percentage
Percentage of available memory
p5
60.44
%
mean
83.43
%
SD
13.62
%
p95
92.93
%
Max host memory used
Maximum memory usage percentage across all hosts
max
78.53
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
24.30
%
Memory growth rate
Rate of memory growth over time
growth
428.04
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.06
MiB/s
Disk write throughput
Disk write throughput in MiB/s
51.82
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/0/7
hosts
Mount Point
/efi-boot0/4
hosts
Mount Point
/etc/hostname1/7
hosts
Mount Point
/etc/hosts1/7
hosts
Mount Point
/etc/nomad.d/client-meta.json0/5
hosts
Mount Point
/etc/resolv.conf1/7
hosts
Mount Point
/nix/store0/7
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
0.90
5 min
0.95
15 min
1.04
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
3.57
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
1.4224
%
SD
3.7992
%
p95
10.13
%
60 second average
1.5611
%
300 second average
2.5308
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
4.9392
%
SD
10.7773
%
p95
29.81
%
60 second average
4.9842
%
300 second average
4.8462
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
4.4073
%
SD
9.2821
%
p95
28.08
%
60 second average
4.4569
%
300 second average
4.3348
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.13
%
mean
9.95
%
SD
14.7
%
p95
48.65
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
342.02
KiB
mean
426.17
KiB
SD
110.20
KiB
p95
575.37
KiB
Holochain process threads
Number of threads in Holochain process
p5
22
threads
mean
31.5
threads
SD
8.48
threads
p95
46
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
56
file descriptors
mean
71.37
file descriptors
SD
16.95
file descriptors
p95
100
file descriptors
Mixed Arc get_agent_activity
↑ Back to index
A mixed-arc scenario that measures the DHT's ability to propagate agent activity and make it available via get_agent_activity calls. The scenario has three roles:
zero_write: A zero-arc conductor that creates entries on its source chain.full_write: A full-arc conductor that creates entries on its source chain.zero_read: A zero-arc conductor that picks a random writer and repeatedly callsget_agent_activityto observe new entries appearing on the writer's chain, measuring the lag between creation and discovery.
Started
Thu, 02 Jul 2026 05:00:23 UTC
Peer count
30
Peer count at end
26
Behaviours
-
full_write(1 agent) -
zero_read(1 agent) -
zero_write(1 agent)
Holochain version
0.7.0-dev.31
Wind Tunnel version
0.7.0
Run ID
mixed_arc_get_agent_activity_28558451397_1
Entry created count
The total number of entries created by write agents over the run, partitioned by agent and behaviour.
Total
310728
Agents affected
19 / 19
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
16354
Max
Highest count in any single partition
38109
Min
Lowest count in any single partition
1555
Observed chain advancement
The maximum highest-observed action sequence per write agent, aggregated across all reading agents. The reading dimension is collapsed by taking the maximum observed value for each write agent (any reader successfully tracking that point counts as propagation). Captures how far readers tracked writers’ chains during the run.
Mean max (across write agents)
Mean of the highest chain head value observed per write agent. Each write agent contributes its maximum — the value seen by any reader.
7424.5
Max
Highest chain head value observed for any single write agent.
12178
Write agents observed
2
Chain head delay timing
The time lag between when a writer creates a new entry and when a reader discovers it via
get_agent_activity. This is measured per new action in the chain.Mean
mean
0.000243
s
SD
0.021902
s
Max
Highest per-partition mean latency
max mean
0.000735
s
Min
Lowest per-partition mean latency
min mean
-0.000457
s
Chain head delay rate
The rate at which readers detect new chain heads appearing on writers' chains.
Mean
mean
79.18
/s
Max
Highest per-partition mean rate
max
133.07
/s
Min
Lowest per-partition mean rate
min
18.63
/s
get_agent_activity_full zome call timingThe time taken to complete the
get_agent_activity_full zome call that queries a writer's chain.Mean
mean
0.070275
s
SD
0.065011
s
Max
Highest per-partition mean latency
max mean
0.141625
s
Min
Lowest per-partition mean latency
min mean
0.022404
s
Retrieval errors
A running accumulation of errors encountered by read agents when calling
get_agent_activity.Total
7
Agents affected
4 / 6
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
1
Max
Highest count in any single partition
3
Min
Lowest count in any single partition
0
Open connections
The number of currently open connections to other conductors, partitioned by agent behaviour.
behaviour:full_write
p5
6
mean
22.42
SD
5.34
p95
24
behaviour:zero_read
p5
0
mean
11.39
SD
7.43
p95
22
behaviour:zero_write
p5
9
mean
13.75
SD
2.89
p95
18
Final error count
The total number of errors accumulated across all agents over the run.
42
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.014003
s
SD
0.089931
s
p50
0.040561
s
p95
0.311707
s
p99
0.311707
s
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
1
over 809.703970964s
mean rate
0.19
/s
std
0.6
/s
p5
0
/s
p95
0.61
/s
peak
2.94
/s
WASM usage: 'agent_activity::announce_write_behaviour'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 1899.202457377s
Not enough time data to show a trend.
WASM usage: 'agent_activity::create_sample_entry'
Metered usage count of the WASM ribosome for this zome function.
total
61615680
over 1899.202457377s
mean rate
1.68497482359e+10
/s
std
7.00777637781e+10
/s
p5
64108.32
/s
p95
6.21923039027e+10
/s
peak
6.19874554143e+11
/s
WASM usage: 'agent_activity::get_agent_activity_full'
Metered usage count of the WASM ribosome for this zome function.
total
1091853603
over 895.971997581s
mean rate
4.1223593653e+08
/s
std
4.489635013e+08
/s
p5
1.030508991e+07
/s
p95
1.46243576799e+09
/s
peak
1.54102216479e+09
/s
WASM usage: 'agent_activity::get_random_agent_with_write_behaviour'
Metered usage count of the WASM ribosome for this zome function.
total
5030829882
over 37459.180653173s
mean rate
2.238185053e+07
/s
std
3.292347338e+07
/s
p5
432332.26
/s
p95
8.764356478e+07
/s
peak
8.767857625e+07
/s
WASM usage: 'agent_activity_integrity::entry_defs'
Metered usage count of the WASM ribosome for this zome function.
total
357438775
over 37459.180653173s
mean rate
1.3343811369e+09
/s
std
6.85580001469e+09
/s
p5
0
/s
p95
4.62605903729e+09
/s
peak
7.59389016446e+10
/s
Zome call duration: 'agent_activity::announce_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.028447
s
SD
0.014255
s
p50
0.022403
s
p95
0.065223
s
p99
0.065223
s
Zome call duration: 'agent_activity::create_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.012636
s
SD
0.017132
s
p50
0.014376
s
p95
0.062711
s
p99
0.071824
s
Zome call duration: 'agent_activity::get_agent_activity_full'
Duration of this zome call as measured by Holochain internally.
mean
0.034623
s
SD
0.033325
s
p50
0.047003
s
p95
0.11831
s
p99
0.130816
s
Zome call duration: 'agent_activity::get_random_agent_with_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.006752
s
SD
0.114768
s
p50
0.124909
s
p95
0.326951
s
p99
0.326951
s
WASM call duration: 'agent_activity::announce_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.008689
s
SD
0.004181
s
p50
0.006246
s
p95
0.016925
s
p99
0.016925
s
WASM call duration: 'agent_activity::create_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.00751
s
SD
0.009873
s
p50
0.008749
s
p95
0.038285
s
p99
0.040094
s
WASM call duration: 'agent_activity::get_agent_activity_full'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.032595
s
SD
0.039526
s
p50
0.043223
s
p95
0.144165
s
p99
0.164397
s
WASM call duration: 'agent_activity::get_random_agent_with_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.004634
s
SD
0.108977
s
p50
0.103556
s
p95
0.316604
s
p99
0.316604
s
WASM call duration: 'agent_activity_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.000976
s
SD
0.000502
s
p50
0.000944
s
p95
0.002214
s
p99
0.00271
s
Host function call duration: '__hc__agent_info_1'
Duration of this host function call invoked from within WASM.
mean
3.7e-05
s
SD
1.6e-05
s
p50
3.4e-05
s
p95
9e-05
s
p99
9e-05
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000465
s
SD
0.001442
s
p50
0.000516
s
p95
0.00154
s
p99
0.008039
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.000502
s
SD
0.000351
s
p50
0.000378
s
p95
0.001865
s
p99
0.001865
s
Host function call duration: '__hc__get_agent_activity_1'
Duration of this host function call invoked from within WASM.
mean
0.029933
s
SD
0.039506
s
p50
0.040618
s
p95
0.141079
s
p99
0.16131
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.000495
s
SD
0.108599
s
p50
0.097531
s
p95
0.311778
s
p99
0.311778
s
Host function call duration: '__hc__random_bytes_1'
Duration of this host function call invoked from within WASM.
mean
8e-06
s
SD
6e-06
s
p50
1.2e-05
s
p95
2.1e-05
s
p99
2.1e-05
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.005131
s
SD
0.006326
s
p50
0.004867
s
p95
0.021424
s
p99
0.033288
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.004216
s
SD
0.00732
s
p50
0.008866
s
p95
0.022625
s
p99
0.030649
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
47
s
mean
449.51
s
SD
258.33
s
p95
853.23
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 37459.180733689s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
18.452199
s
SD
13.061067
s
p50
0
s
p95
9.968141
s
p99
90.27122
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.09084
s
SD
0.248158
s
p50
0.002027
s
p95
0.288116
s
p99
0.455901
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0.001632
s
SD
0.001326
s
p50
0.00111
s
p95
0.004969
s
p99
0.005475
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.019612
s
SD
0.255573
s
p50
0.002596
s
p95
0.891503
s
p99
1.069408
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.221816
s
SD
3.510304
s
p50
0.113837
s
p95
8.561044
s
p99
16.294617
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.325218
s
SD
0.287222
s
p50
0.008442
s
p95
0.686522
s
p99
0.953762
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
4.012416
s
SD
10.546145
s
p50
0.009979
s
p95
41.376128
s
p99
43.542123
s
Authored DB connection use time
Time spent holding authored database connections.
mean
0.001304
s
SD
0.001239
s
p50
0.00083
s
p95
0.003907
s
p99
0.005584
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.002142
s
SD
0.009807
s
p50
0.000503
s
p95
0.0216
s
p99
0.05332
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000466
s
SD
0.001349
s
p50
0.000332
s
p95
0.000982
s
p99
0.009173
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
4.074738
s
SD
29.664725
s
p50
0.156609
s
p95
60.014536
s
p99
60.307725
s
P2P request roundtrip: 'get_links'
Time spent sending a get_links request and awaiting its response.
mean
0.034419
s
SD
0.069882
s
p50
0.131894
s
p95
0.234185
s
p99
0.234185
s
P2P request roundtrip: 'get_agent_activity'
Time spent sending a get_agent_activity request and awaiting its response.
mean
0.032667
s
SD
22.002997
s
p50
0.046417
s
p95
60.001504
s
p99
60.002041
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.592078
s
SD
1.298941
s
p50
0.432597
s
p95
0.94656
s
p99
10.050246
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
29.79
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
196
B/s
mean
386.38
KiB/s
SD
608.22
KiB/s
p95
1.56
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
163
B/s
mean
361.69
KiB/s
SD
607.53
KiB/s
p95
1.47
MiB/s
Total bytes received
Total bytes received on primary network interface
count
9.74
GiB
mean
3.99
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
9.01
GiB
mean
3.69
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 97.3%
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 503.74 MiB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
Detected
Warning 14% of hosts overloaded (load5/ncpus > 1.0)
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
1.31
%
mean
37.87
%
SD
30.45
%
p95
94.44
%
System
8.08
%
CPU percentiles
CPU usage percentiles
p50
28.85
%
p95
94.44
%
p99
97.29
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
8
hosts
mean time
0.55
s
Memory used percentage
Percentage of memory used
p5
6.22
%
mean
13.99
%
SD
12.95
%
p95
29.42
%
Memory available percentage
Percentage of available memory
p5
70.58
%
mean
86.01
%
SD
12.95
%
p95
93.78
%
Max host memory used
Maximum memory usage percentage across all hosts
max
77.80
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00
%
Memory growth rate
Rate of memory growth over time
growth
503.74
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.13
MiB/s
Disk write throughput
Disk write throughput in MiB/s
168.16
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/0/7
hosts
Mount Point
/efi-boot0/4
hosts
Mount Point
/etc/hostname0/6
hosts
Mount Point
/etc/hosts0/6
hosts
Mount Point
/etc/nomad.d/client-meta.json0/4
hosts
Mount Point
/etc/resolv.conf0/6
hosts
Mount Point
/nix/store0/7
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
2.90
5 min
2.24
15 min
1.60
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
14.29
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
15.6881
%
SD
23.566
%
p95
71.46
%
60 second average
14.6431
%
300 second average
10.8627
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
9.4124
%
SD
16.0971
%
p95
45.97
%
60 second average
9.0277
%
300 second average
7.3397
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
6.8008
%
SD
12.4646
%
p95
37.88
%
60 second average
6.5732
%
300 second average
5.5407
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.27
%
mean
32.59
%
SD
29.65
%
p95
92.16
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
187.02
KiB
mean
347.01
KiB
SD
170.70
KiB
p95
697.54
KiB
Holochain process threads
Number of threads in Holochain process
p5
23
threads
mean
36.71
threads
SD
10.94
threads
p95
51
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
60
file descriptors
mean
83.17
file descriptors
SD
22.27
file descriptors
p95
108
file descriptors
Mixed Arc must_get_agent_activity
↑ Back to index
A scenario where write agents create entries in batches of 10, while must_get_agent_activity agents each pick a random write agent and repeatedly attempt to create an entry that references the chain top of their latest batch. This reference means that the entry's validation function needs to make a must_get_agent_activity call.
The purpose of this scenario is to measure the time it takes for published agent activity data to be gossiped among authorities and become available to peers that query it via must_get_agent_activity.
This test is similar to Write Validated must_get_agent_activity, but some write agents are zero-arc and all must_get_agent_activity agents are zero-arc. This forces the must_get_agent_activity host call to go to the network.
Started
Thu, 02 Jul 2026 05:16:13 UTC
Peer count
30
Peer count at end
23
Behaviours
-
full_write(1 agent) -
zero_must_get_agent_activity(1 agent) -
zero_write(1 agent)
Holochain version
0.7.0-dev.31
Wind Tunnel version
0.7.0
Run ID
mixed_arc_must_get_agent_activity_28558451397_1
Highest observed
action_seqThe maximum chain length observed per
write agent, aggregated across all reading agents. The reading dimension is collapsed by taking the maximum observed value for each write agent (any successful read counts as propagation). This reflects the DHT’s ability to propagate agent activity ops and make them available to querying peers.Mean max (across write agents)
Mean of the highest chain head value observed per write agent. Each write agent contributes its maximum — the value seen by any reader.
1685.25
Max
Highest chain head value observed for any single write agent.
1776
Write agents observed
4
Chain batch delay timing
The time between a
write agent's creation of a batch and a must_get_agent_activity agent's successful discovery of the batch and creation/self-validation of a new entry that depends on it.Mean
mean
34.02719
s
SD
29.683707
s
Max
Highest per-partition mean latency
max mean
150.067914
s
Min
Lowest per-partition mean latency
min mean
3.22943
s
Chain batch delay rate
The rate at which a
must_get_agent_activity agent was able to discover batches and create/self-validate new entries that depend on them.Mean
mean
2.17
/s
Max
Highest per-partition mean rate
max
2.57
/s
Min
Lowest per-partition mean rate
min
1.91
/s
create_validated_sample_entry zome call timingThe time taken to complete the zome function call that creates the entry that depends on a
write agent's source chain.Mean
mean
0.468044
s
SD
0.283346
s
Max
Highest per-partition mean latency
max mean
0.830774
s
Min
Lowest per-partition mean latency
min mean
0.158883
s
Retrieval errors
A running accumulation of the errors encountered by an agent when attempting to self-validate actions that depend on
must_get_agent_activity calls.Total
13248
Agents affected
8 / 8
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
1656
Max
Highest count in any single partition
6199
Min
Lowest count in any single partition
365
Open connections
The average number of open connections maintained by an agent over the run, broken down by behaviour.
behaviour:full_write
p5
18
mean
22.56
SD
1.65
p95
23
behaviour:zero_must_get_agent_activity
p5
9
mean
14.96
SD
3.11
p95
19
behaviour:zero_write
p5
8
mean
12.3
SD
3.54
p95
17
Final error count
The total number of all types of error accumulated over the run by all agents.
28707
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.068849
s
SD
0.14866
s
p50
0.052182
s
p95
0.51456
s
p99
0.533415
s
Zome call duration: 'validated_must_get_agent_activity::announce_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.05453
s
SD
0.028977
s
p50
0.053001
s
p95
0.112853
s
p99
0.112853
s
Zome call duration: 'validated_must_get_agent_activity::create_sample_entries_batch'
Duration of this zome call as measured by Holochain internally.
mean
0.165272
s
SD
0.07072
s
p50
0.130596
s
p95
0.290357
s
p99
0.360088
s
Zome call duration: 'validated_must_get_agent_activity::create_validated_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.017073
s
SD
0.00821
s
p50
0.018591
s
p95
0.029427
s
p99
0.050128
s
Zome call duration: 'validated_must_get_agent_activity::get_chain_top_for_batch'
Duration of this zome call as measured by Holochain internally.
mean
0.13783
s
SD
0.107046
s
p50
0.068374
s
p95
0.317322
s
p99
0.333157
s
Zome call duration: 'validated_must_get_agent_activity::get_random_agent_with_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.036275
s
SD
0.228885
s
p50
0.045253
s
p95
0.604081
s
p99
0.604081
s
WASM call duration: 'validated_must_get_agent_activity::announce_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.021031
s
SD
0.012068
s
p50
0.017467
s
p95
0.056553
s
p99
0.056553
s
WASM call duration: 'validated_must_get_agent_activity::create_sample_entries_batch'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.150605
s
SD
0.065919
s
p50
0.121662
s
p95
0.263386
s
p99
0.326894
s
WASM call duration: 'validated_must_get_agent_activity::create_validated_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.011989
s
SD
0.003681
s
p50
0.0121
s
p95
0.017717
s
p99
0.022751
s
WASM call duration: 'validated_must_get_agent_activity::get_chain_top_for_batch'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.175877
s
SD
0.098822
s
p50
0.225735
s
p95
0.348013
s
p99
0.435838
s
WASM call duration: 'validated_must_get_agent_activity::get_random_agent_with_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.020324
s
SD
0.220504
s
p50
0.024331
s
p95
0.560816
s
p99
0.560816
s
WASM call duration: 'validated_must_get_agent_activity_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.00127
s
SD
0.000576
s
p50
0.001146
s
p95
0.002769
s
p99
0.002986
s
WASM call duration: 'validated_must_get_agent_activity_integrity::validate'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.010776
s
SD
0.02579
s
p50
0.006854
s
p95
0.08709
s
p99
0.132551
s
Host function call duration: '__hc__agent_info_1'
Duration of this host function call invoked from within WASM.
mean
3.1e-05
s
SD
1.5e-05
s
p50
2.9e-05
s
p95
5.9e-05
s
p99
9.7e-05
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000498
s
SD
0.000135
s
p50
0.000463
s
p95
0.000707
s
p99
0.000768
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.000426
s
SD
0.000415
s
p50
0.000343
s
p95
0.000987
s
p99
0.002633
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.15622
s
SD
0.165176
s
p50
0.19414
s
p95
0.533488
s
p99
0.533488
s
Host function call duration: '__hc__must_get_agent_activity_1'
Duration of this host function call invoked from within WASM.
mean
0.034683
s
SD
0.097589
s
p50
0.01955
s
p95
0.294427
s
p99
0.419232
s
Host function call duration: '__hc__query_1'
Duration of this host function call invoked from within WASM.
mean
0.007262
s
SD
0.003582
s
p50
0.005121
s
p95
0.012029
s
p99
0.014913
s
Host function call duration: '__hc__random_bytes_1'
Duration of this host function call invoked from within WASM.
mean
1e-05
s
SD
1.3e-05
s
p50
1.2e-05
s
p95
5e-05
s
p99
5e-05
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.006007
s
SD
0.011613
s
p50
0.009431
s
p95
0.029656
s
p99
0.084575
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.008727
s
SD
0.007378
s
p50
0.007255
s
p95
0.023339
s
p99
0.02866
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
47.03
s
mean
451.13
s
SD
259.3
s
p95
856.65
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 37471.80327123s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
69.532056
s
SD
31.347263
s
p50
0
s
p95
89.297491
s
p99
131.431308
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
1.170091
s
SD
0.868212
s
p50
0.001469
s
p95
2.055428
s
p99
4.31633
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0.001778
s
SD
0.001523
s
p50
0.001083
s
p95
0.005869
s
p99
0.006666
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.049703
s
SD
0.041747
s
p50
0.001833
s
p95
0.095088
s
p99
0.245179
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.105722
s
SD
0.053906
s
p50
0.02472
s
p95
0.16301
s
p99
0.271579
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.609491
s
SD
0.40071
s
p50
0.005426
s
p95
1.20989
s
p99
2.025545
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
21.70281
s
SD
20.918698
s
p50
0.005369
s
p95
48.219691
s
p99
90.459674
s
Authored DB connection use time
Time spent holding authored database connections.
mean
0.00039
s
SD
0.00024
s
p50
0.000373
s
p95
0.000969
s
p99
0.000994
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.000642
s
SD
0.001528
s
p50
0.000201
s
p95
0.003183
s
p99
0.009066
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000374
s
SD
0.000137
s
p50
0.000383
s
p95
0.000643
s
p99
0.000826
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
5.072582
s
SD
29.887533
s
p50
0.254544
s
p95
60.005159
s
p99
60.006487
s
P2P request roundtrip: 'get_links'
Time spent sending a get_links request and awaiting its response.
mean
0.093914
s
SD
26.315813
s
p50
0.18891
s
p95
60.001781
s
p99
60.001818
s
P2P request roundtrip: 'must_get_agent_activity'
Time spent sending a must_get_agent_activity request and awaiting its response.
mean
0.071168
s
SD
19.830203
s
p50
0.056884
s
p95
60.001057
s
p99
60.001659
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.105683
s
SD
0.056169
s
p50
0.10301
s
p95
0.24859
s
p99
0.254427
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
14.87
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
914
B/s
mean
84.79
KiB/s
SD
265.87
KiB/s
p95
272.51
KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
166
B/s
mean
65.38
KiB/s
SD
164.11
KiB/s
p95
299.09
KiB/s
Total bytes received
Total bytes received on primary network interface
count
2.04
GiB
mean
830.38
KiB/s
Total bytes sent
Total bytes sent on primary network interface
count
1.52
GiB
mean
618.92
KiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 93.6%
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 476.19 MiB/s
Disk full anomaly
Disk full anomaly was detected
Detected
Critical Disk /etc/resolv.conf: 1/6 hosts above 90% capacity
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected
Critical Heavy swap usage (24.3% swap used)
System overload anomaly
System overload anomaly was detected
NotDetected
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
0.83
%
mean
20.37
%
SD
25.08
%
p95
77.52
%
System
5.49
%
CPU percentiles
CPU usage percentiles
p50
7.68
%
p95
77.52
%
p99
93.59
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
6
hosts
mean time
0.19
s
Memory used percentage
Percentage of memory used
p5
7.11
%
mean
17.02
%
SD
13.76
%
p95
38.42
%
Memory available percentage
Percentage of available memory
p5
61.58
%
mean
82.98
%
SD
13.76
%
p95
92.89
%
Max host memory used
Maximum memory usage percentage across all hosts
max
78.98
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
24.30
%
Memory growth rate
Rate of memory growth over time
growth
476.19
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.03
MiB/s
Disk write throughput
Disk write throughput in MiB/s
59.98
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/0/5
hosts
Mount Point
/efi-boot0/4
hosts
Mount Point
/etc/hostname1/6
hosts
Mount Point
/etc/hosts1/6
hosts
Mount Point
/etc/nomad.d/client-meta.json0/5
hosts
Mount Point
/etc/resolv.conf1/6
hosts
Mount Point
/nix/store0/5
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
1.14
5 min
1.44
15 min
1.58
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
0.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
5.0969
%
SD
12.4336
%
p95
34.31
%
60 second average
5.2464
%
300 second average
7.2960
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
1.1826
%
SD
2.8716
%
p95
5.38
%
60 second average
1.3684
%
300 second average
2.4073
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
0.9531
%
SD
2.6237
%
p95
4.09
%
60 second average
1.0906
%
300 second average
1.7583
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0
%
mean
15.63
%
SD
22.69
%
p95
67.2
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
14.91
KiB
mean
426.88
KiB
SD
155.42
KiB
p95
677.72
KiB
Holochain process threads
Number of threads in Holochain process
p5
9
threads
mean
33.89
threads
SD
12.24
threads
p95
58
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
8
file descriptors
mean
78.87
file descriptors
SD
25.06
file descriptors
p95
110
file descriptors
Remote Call Rate
↑ Back to indexTest the throughput of remote_call operations. Each agent in this scenario waits for a certain number of peers to be available or for up to two minutes, whichever happens first, before starting its behaviour.
Started
Thu, 02 Jul 2026 01:18:16 UTC
Peer count
10
Peer count at end
9
Behaviours
-
default(1 agent)
Holochain version
0.7.0-dev.31
Wind Tunnel version
0.7.0
Run ID
remote_call_rate_28558451397_1
Dispatch timing
The time between sending a remote call and the remote handler being invoked.
Mean
mean
0.069559
s
SD
0.039529
s
Max
Highest per-partition mean latency
max mean
0.119832
s
Min
Lowest per-partition mean latency
min mean
0.045405
s
Round-trip timing
The total elapsed time to get a response to the client.
Mean
mean
0.143896
s
SD
0.071257
s
Max
Highest per-partition mean latency
max mean
0.256356
s
Min
Lowest per-partition mean latency
min mean
0.094314
s
Error count
The total number of errors accumulated during the run.
0
Holochain MetricsClick to toggle visibility.
WASM usage: 'remote_call::call_echo_timestamp'
Metered usage count of the WASM ribosome for this zome function.
total
115376144
over 885.0169765s
mean rate
6.0276974848e+08
/s
std
1.09775710507e+09
/s
p5
1.34470797e+06
/s
p95
3.46830647321e+09
/s
peak
4.34918652779e+09
/s
WASM usage: 'remote_call::echo_timestamp'
Metered usage count of the WASM ribosome for this zome function.
total
43290683
over 885.0169765s
mean rate
3.06005739e+06
/s
std
4.75661817e+06
/s
p5
31104.17
/s
p95
1.547305788e+07
/s
peak
2.017011338e+07
/s
WASM usage: 'remote_call::init'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 33082.323660438s
Not enough time data to show a trend.
Zome call duration: 'remote_call::call_echo_timestamp'
Duration of this zome call as measured by Holochain internally.
mean
0.141203
s
SD
0.048319
s
p50
0.126229
s
p95
0.252026
s
p99
0.25453
s
Zome call duration: 'remote_call::echo_timestamp'
Duration of this zome call as measured by Holochain internally.
mean
0.005715
s
SD
0.001965
s
p50
0.005017
s
p95
0.008688
s
p99
0.009037
s
WASM call duration: 'remote_call::call_echo_timestamp'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.137614
s
SD
0.048689
s
p50
0.121486
s
p95
0.24988
s
p99
0.252001
s
WASM call duration: 'remote_call::echo_timestamp'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.002241
s
SD
0.000859
s
p50
0.001851
s
p95
0.0034
s
p99
0.003403
s
WASM call duration: 'remote_call::init'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.005457
s
SD
0.003722
s
p50
0.004335
s
p95
0.015498
s
p99
0.015498
s
Host function call duration: '__hc__call_1'
Duration of this host function call invoked from within WASM.
mean
0.132327
s
SD
0.04936
s
p50
0.114747
s
p95
0.247121
s
p99
0.249254
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000308
s
SD
0.000101
s
p50
0.000269
s
p95
0.000544
s
p99
0.000544
s
Host function call duration: '__hc__sys_time_1'
Duration of this host function call invoked from within WASM.
mean
2.3e-05
s
SD
9e-06
s
p50
2.3e-05
s
p95
4.2e-05
s
p99
4.2e-05
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.002931
s
SD
0.002686
s
p50
0.003024
s
p95
0.013508
s
p99
0.013508
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.004747
s
SD
0.005994
s
p50
0.003009
s
p95
0.021577
s
p99
0.021577
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
47.77
s
mean
448.2
s
SD
257.24
s
p95
847.92
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 33082.323718549s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
2.632342
s
SD
2.237469
s
p50
0.98393
s
p95
5.391558
s
p99
7.048283
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.023944
s
SD
0.050684
s
p50
0.007136
s
p95
0.12775
s
p99
0.283996
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0.001377
s
SD
0.000601
s
p50
0.001301
s
p95
0.002504
s
p99
0.002504
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.035874
s
SD
0.031294
s
p50
0.037367
s
p95
0.091247
s
p99
0.091247
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.004309
s
SD
0.001925
s
p50
0.003769
s
p95
0.007308
s
p99
0.010186
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.453389
s
SD
0.7733
s
p50
0.365811
s
p95
1.526928
s
p99
5.077415
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
0.206246
s
SD
0.230399
s
p50
0.150279
s
p95
0.575479
s
p99
0.933472
s
Authored DB connection use time
Time spent holding authored database connections.
mean
0.000251
s
SD
7.6e-05
s
p50
0.000247
s
p95
0.000366
s
p99
0.000367
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.000132
s
SD
0.000468
s
p50
0.000361
s
p95
0.001737
s
p99
0.001798
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000447
s
SD
0.000149
s
p50
0.000383
s
p95
0.000673
s
p99
0.000673
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
0.063129
s
SD
0.073153
s
p50
0.036602
s
p95
0.235421
s
p99
0.236459
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.169537
s
SD
0.033089
s
p50
0.189724
s
p95
0.236865
s
p99
0.236865
s
P2P request roundtrip: 'call_remote'
Time spent sending a call_remote request and awaiting its response.
mean
0.13172
s
SD
0.049449
s
p50
0.113982
s
p95
0.246821
s
p99
0.24896
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
7.92
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
1.24
KiB/s
mean
15.53
KiB/s
SD
114.77
KiB/s
p95
16.04
KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
157
B/s
mean
10.68
KiB/s
SD
14.46
KiB/s
p95
12.44
KiB/s
Total bytes received
Total bytes received on primary network interface
count
121.63
MiB
mean
138.39
KiB/s
Total bytes sent
Total bytes sent on primary network interface
count
83.63
MiB
mean
95.16
KiB/s
CPU spike anomaly
CPU spike anomaly was detected
NotDetected
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 71.37 MiB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
NotDetected
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
0.76
%
mean
15.21
%
SD
15.19
%
p95
47.44
%
System
7.29
%
CPU percentiles
CPU usage percentiles
p50
9.15
%
p95
47.44
%
p99
48.02
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
0
hosts
mean time
0.00
s
Memory used percentage
Percentage of memory used
p5
5.9
%
mean
11.98
%
SD
4.14
%
p95
22.49
%
Memory available percentage
Percentage of available memory
p5
77.51
%
mean
88.02
%
SD
4.14
%
p95
94.1
%
Max host memory used
Maximum memory usage percentage across all hosts
max
22.34
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00
%
Memory growth rate
Rate of memory growth over time
growth
71.37
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.04
MiB/s
Disk write throughput
Disk write throughput in MiB/s
9.67
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/etc/hostname0/2
hosts
Mount Point
/etc/hosts0/2
hosts
Mount Point
/etc/nomad.d/client-meta.json0/3
hosts
Mount Point
/etc/resolv.conf0/2
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
0.68
5 min
0.60
15 min
0.42
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
0.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
1.9074
%
SD
2.2389
%
p95
6
%
60 second average
1.8831
%
300 second average
1.5911
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
0.2076
%
SD
0.9625
%
p95
0.96
%
60 second average
0.2027
%
300 second average
0.1651
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
0.1543
%
SD
0.8665
%
p95
0.54
%
60 second average
0.1477
%
300 second average
0.1227
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.67
%
mean
7.29
%
SD
5.98
%
p95
17.8
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
197.96
KiB
mean
233.81
KiB
SD
41.58
KiB
p95
326.46
KiB
Holochain process threads
Number of threads in Holochain process
p5
23
threads
mean
36.11
threads
SD
8.8
threads
p95
59
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
54
file descriptors
mean
81.42
file descriptors
SD
13.92
file descriptors
p95
98
file descriptors
Remote Signals
↑ Back to indexThis scenario tests the throughput of remote_signals operations.
Started
Thu, 02 Jul 2026 01:20:24 UTC
Peer count
10
Peer count at end
8
Behaviours
-
default(1 agent)
Holochain version
0.7.0-dev.31
Wind Tunnel version
0.7.0
Run ID
remote_signals_28558451397_1
Round trip time
The time from origin signal dispatch to origin receive of the remote side's response signal.
mean
0.16577
s
SD
0.301319
s
p50
0.145677
s
p95
0.243193
s
p99
0.474178
s
Timeout rate
The proportion of remote signal round-trips that timed out.
0
Holochain MetricsClick to toggle visibility.
WASM usage: 'remote_signal::init'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 896.896822707s
Not enough time data to show a trend.
WASM usage: 'remote_signal::recv_remote_signal'
Metered usage count of the WASM ribosome for this zome function.
total
31324531
over 896.896822707s
mean rate
4.2050991164e+08
/s
std
1.67493180123e+09
/s
p5
0
/s
p95
2.70523197215e+09
/s
peak
2.05360537909e+10
/s
WASM usage: 'remote_signal::signal_request'
Metered usage count of the WASM ribosome for this zome function.
total
13330002
over 896.896822707s
mean rate
1.926409814e+07
/s
std
1.2220107349e+08
/s
p5
0
/s
p95
6.164959045e+07
/s
peak
2.08525826655e+09
/s
Zome call duration: 'remote_signal::recv_remote_signal'
Duration of this zome call as measured by Holochain internally.
mean
0.008912
s
SD
0.008048
s
p50
0.009081
s
p95
0.024021
s
p99
0.044184
s
Zome call duration: 'remote_signal::signal_request'
Duration of this zome call as measured by Holochain internally.
mean
0.008784
s
SD
0.005734
s
p50
0.009289
s
p95
0.02089
s
p99
0.037417
s
WASM call duration: 'remote_signal::init'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.028322
s
SD
0.019566
s
p50
0.017832
s
p95
0.064411
s
p99
0.064411
s
WASM call duration: 'remote_signal::recv_remote_signal'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.002623
s
SD
0.0007
s
p50
0.002405
s
p95
0.003695
s
p99
0.003703
s
WASM call duration: 'remote_signal::signal_request'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.00261
s
SD
0.000761
s
p50
0.002437
s
p95
0.003706
s
p99
0.003721
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000551
s
SD
0.000352
s
p50
0.000385
s
p95
0.00107
s
p99
0.00107
s
Host function call duration: '__hc__emit_signal_1'
Duration of this host function call invoked from within WASM.
mean
9.9e-05
s
SD
1.6e-05
s
p50
9.7e-05
s
p95
0.000118
s
p99
0.000118
s
Host function call duration: '__hc__send_remote_signal_1'
Duration of this host function call invoked from within WASM.
mean
0.000104
s
SD
2.1e-05
s
p50
0.000102
s
p95
0.000144
s
p99
0.000146
s
Host function call duration: '__hc__sys_time_1'
Duration of this host function call invoked from within WASM.
mean
1.7e-05
s
SD
4e-06
s
p50
1.7e-05
s
p95
2.2e-05
s
p99
2.2e-05
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.025968
s
SD
0.018031
s
p50
0.016442
s
p95
0.058893
s
p99
0.058893
s
Emitted local signals
Count of local signals emitted via the emit_signal host function.
total
1724
over 896.896943294s
mean rate
18780.19
/s
std
165694.92
/s
p5
0
/s
p95
44126.73
/s
peak
3.27868852e+06
/s
Sent remote signals
Count of remote signals sent via the send_remote_signal host function.
total
1723
over 896.896937495s
mean rate
23507.78
/s
std
302653.43
/s
p5
0
/s
p95
48722.85
/s
peak
6.62251656e+06
/s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.01964
s
SD
0.027165
s
p50
0.008961
s
p95
0.090288
s
p99
0.090288
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
45.64
s
mean
450.45
s
SD
259.8
s
p95
855.65
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
91
over 897.198783444s
mean rate
27077.47
/s
std
269821.37
/s
p5
0
/s
p95
23.18
/s
peak
4.63278376e+06
/s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
1.994038
s
SD
0.93062
s
p50
1.982553
s
p95
3.298455
s
p99
3.554302
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.140303
s
SD
0.361712
s
p50
0.112506
s
p95
1.056551
s
p99
1.314049
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0.001891
s
SD
0.001145
s
p50
0.001578
s
p95
0.004396
s
p99
0.004396
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.043112
s
SD
0.054107
s
p50
0.041209
s
p95
0.151718
s
p99
0.153536
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.010404
s
SD
0.011118
s
p50
0.007899
s
p95
0.046148
s
p99
0.051949
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.134819
s
SD
0.182925
s
p50
0.047986
s
p95
0.475718
s
p99
0.772864
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
0.212063
s
SD
0.261039
s
p50
0.183365
s
p95
0.889879
s
p99
1.112108
s
Authored DB connection use time
Time spent holding authored database connections.
mean
0.000293
s
SD
9.6e-05
s
p50
0.000326
s
p95
0.000375
s
p99
0.000528
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.000191
s
SD
0.006559
s
p50
0.000698
s
p95
0.013283
s
p99
0.034126
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000659
s
SD
0.000113
s
p50
0.000607
s
p95
0.000778
s
p99
0.000783
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
0.056148
s
SD
0.051684
s
p50
0.031028
s
p95
0.136032
s
p99
0.150695
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.132444
s
SD
0.032694
s
p50
0.129675
s
p95
0.174033
s
p99
0.174033
s
P2P received remote signals
Count of remote signals received.
total
1725
over 896.896812329s
mean rate
19499.93
/s
std
149986.76
/s
p5
0
/s
p95
62802.24
/s
peak
3.02114804e+06
/s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
4.04
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
4.84
KiB/s
mean
30.40
KiB/s
SD
355.89
KiB/s
p95
12.83
KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
4.03
KiB/s
mean
6.21
KiB/s
SD
6.38
KiB/s
p95
7.94
KiB/s
Total bytes received
Total bytes received on primary network interface
count
105.97
MiB
mean
120.57
KiB/s
Total bytes sent
Total bytes sent on primary network interface
count
21.63
MiB
mean
24.62
KiB/s
CPU spike anomaly
CPU spike anomaly was detected
NotDetected
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 14.17 MiB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
NotDetected
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
2.15
%
mean
5.85
%
SD
7.01
%
p95
16.14
%
System
1.81
%
CPU percentiles
CPU usage percentiles
p50
4.30
%
p95
16.14
%
p99
41.06
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
0
hosts
mean time
0.00
s
Memory used percentage
Percentage of memory used
p5
6.85
%
mean
29.01
%
SD
28.76
%
p95
78.53
%
Memory available percentage
Percentage of available memory
p5
21.47
%
mean
70.99
%
SD
28.76
%
p95
93.15
%
Max host memory used
Maximum memory usage percentage across all hosts
max
78.50
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00
%
Memory growth rate
Rate of memory growth over time
growth
14.17
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.05
MiB/s
Disk write throughput
Disk write throughput in MiB/s
4.38
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/0/1
hosts
Mount Point
/etc/hostname0/1
hosts
Mount Point
/etc/hosts0/1
hosts
Mount Point
/etc/nomad.d/client-meta.json0/1
hosts
Mount Point
/etc/resolv.conf0/1
hosts
Mount Point
/nix/store0/1
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
0.55
5 min
0.46
15 min
0.28
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
0.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0.8169
%
SD
4.2316
%
p95
1.67
%
60 second average
0.8153
%
300 second average
0.7473
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
3.0347
%
SD
8.3426
%
p95
25.39
%
60 second average
3.0173
%
300 second average
2.5909
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
2.8106
%
SD
7.7209
%
p95
23.64
%
60 second average
2.7877
%
300 second average
2.3861
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.57
%
mean
2.68
%
SD
4.44
%
p95
10.67
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
188.16
KiB
mean
206.78
KiB
SD
24.11
KiB
p95
232.18
KiB
Holochain process threads
Number of threads in Holochain process
p5
28
threads
mean
36.7
threads
SD
5.5
threads
p95
43
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
67
file descriptors
mean
86.21
file descriptors
SD
11.41
file descriptors
p95
98
file descriptors
Single-write, many-read
↑ Back to indexCreates an entry in the agent setup, then read one's own record back. This tests the maximum read performance for reading back local data when the chain is short.
Started
Thu, 02 Jul 2026 01:22:40 UTC
Peer count
1
Behaviours
-
default(1 agent)
Holochain version
0.7.0-dev.31
Wind Tunnel version
0.7.0
Run ID
single_write_many_read_28558451397_1
read call timingThe time it takes to complete the zome call that retrieves the record.
mean
0.011816
s
SD
0.100237
s
p50
0.004652
s
p95
0.006318
s
p99
0.013223
s
read call rateThe number of read zome calls completed per second.
mean
838.75
/s
Error count
The number of errors collected.
0
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.000332
s
SD
2e-06
s
p50
0.000332
s
p95
0.000336
s
p99
0.000339
s
WASM usage: 'crud::create_sample_entry'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 280.002785177s
Not enough time data to show a trend.
WASM usage: 'crud::get_sample_entry'
Metered usage count of the WASM ribosome for this zome function.
total
875177124
over 280.002785177s
mean rate
3.12560157e+06
/s
std
460194.1
/s
p5
2.11864886e+06
/s
p95
3.93568864e+06
/s
peak
4.14564119e+06
/s
WASM usage: 'crud::init'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 280.002785177s
Not enough time data to show a trend.
WASM usage: 'crud_integrity::entry_defs'
Metered usage count of the WASM ribosome for this zome function.
total
4402
over 290.002835007s
mean rate
15.18
/s
std
80.32
/s
p5
0
/s
p95
0
/s
peak
440.2
/s
Zome call duration: 'crud::create_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.027484
s
SD
0
s
p50
0.027484
s
p95
0.027484
s
p99
0.027484
s
Zome call duration: 'crud::get_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.003481
s
SD
4.3e-05
s
p50
0.003479
s
p95
0.003582
s
p99
0.003675
s
WASM call duration: 'crud::create_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.005991
s
SD
0
s
p50
0.005991
s
p95
0.005991
s
p99
0.005991
s
WASM call duration: 'crud::get_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001344
s
SD
3e-06
s
p50
0.001344
s
p95
0.001353
s
p99
0.001353
s
WASM call duration: 'crud::init'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001195
s
SD
0
s
p50
0.001195
s
p95
0.001195
s
p99
0.001195
s
WASM call duration: 'crud_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.000848
s
SD
1.1e-05
s
p50
0.00085
s
p95
0.00085
s
p99
0.00085
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000289
s
SD
4e-06
s
p50
0.000292
s
p95
0.000292
s
p99
0.000292
s
Host function call duration: '__hc__get_1'
Duration of this host function call invoked from within WASM.
mean
0.000381
s
SD
2e-06
s
p50
0.000381
s
p95
0.000386
s
p99
0.000388
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.004621
s
SD
0
s
p50
0.004621
s
p95
0.004621
s
p99
0.004621
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.009819
s
SD
0
s
p50
0.009819
s
p95
0.009819
s
p99
0.009819
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
15.87
s
mean
150.87
s
SD
86.56
s
p95
285.87
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 290.002865021s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.001533
s
SD
0
s
p50
0.001533
s
p95
0.001533
s
p99
0.001533
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0.000975
s
SD
0
s
p50
0.000975
s
p95
0.000975
s
p99
0.000975
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.001356
s
SD
4.5e-05
s
p50
0.001354
s
p95
0.001354
s
p99
0.001604
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.002361
s
SD
0.000396
s
p50
0.002337
s
p95
0.002697
s
p99
0.004329
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.006354
s
SD
0
s
p50
0.006354
s
p95
0.006354
s
p99
0.006354
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
0.006443
s
SD
0
s
p50
0.006443
s
p95
0.006443
s
p99
0.006443
s
Authored DB connection use time
Time spent holding authored database connections.
mean
9.7e-05
s
SD
4.8e-05
s
p50
9.7e-05
s
p95
9.8e-05
s
p99
0.000363
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.000121
s
SD
0.000212
s
p50
0.000505
s
p95
0.000505
s
p99
0.001
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000402
s
SD
3.2e-05
s
p50
0.000399
s
p95
0.000399
s
p99
0.000576
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
12.25
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
4.77
KiB/s
mean
6.43
KiB/s
SD
1.64
KiB/s
p95
9.02
KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
3.99
KiB/s
mean
5.18
KiB/s
SD
1.21
KiB/s
p95
8.92
KiB/s
Total bytes received
Total bytes received on primary network interface
count
1.88
MiB
mean
6.43
KiB/s
Total bytes sent
Total bytes sent on primary network interface
count
1.52
MiB
mean
5.18
KiB/s
CPU spike anomaly
CPU spike anomaly was detected
NotDetected
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 15.73 MiB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
NotDetected
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
13.92
%
mean
17.14
%
SD
6.12
%
p95
23.76
%
System
4.89
%
CPU percentiles
CPU usage percentiles
p50
15.96
%
p95
23.76
%
p99
48.47
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
0
hosts
mean time
0.00
s
Memory used percentage
Percentage of memory used
p5
17.16
%
mean
17.49
%
SD
0.32
%
p95
17.93
%
Memory available percentage
Percentage of available memory
p5
82.07
%
mean
82.51
%
SD
0.32
%
p95
82.84
%
Max host memory used
Maximum memory usage percentage across all hosts
max
17.49
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00
%
Memory growth rate
Rate of memory growth over time
growth
15.73
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.00
MiB/s
Disk write throughput
Disk write throughput in MiB/s
3.88
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/0/1
hosts
Mount Point
/nix/store0/1
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
1.50
5 min
0.81
15 min
0.35
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
0.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0.16
%
mean
0.4316
%
SD
0.2199
%
p95
0.98
%
60 second average
0.7210
%
300 second average
1.1574
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
13.67
%
mean
26.0961
%
SD
7.0882
%
p95
38.75
%
60 second average
22.4948
%
300 second average
11.3635
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
12.49
%
mean
24.0235
%
SD
6.3641
%
p95
34.92
%
60 second average
20.6261
%
300 second average
10.4084
%
Holochain process CPU usage
CPU usage by Holochain process
p5
8.7
%
mean
11.96
%
SD
6.33
%
p95
18.9
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
217.52
KiB
mean
215.39
KiB
SD
35.91
KiB
p95
225.00
KiB
Holochain process threads
Number of threads in Holochain process
p5
28
threads
mean
27.32
threads
SD
3.71
threads
p95
28
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
67
file descriptors
mean
66.81
file descriptors
SD
10.01
file descriptors
p95
73
file descriptors
Validation Receipts
↑ Back to indexCreates an entry, wait for required validation receipts, then repeat. Records the amount of time it took to accumulate the required number of receipts for all DHT operations. This is measured to the nearest 20ms so that we don't keep the agent too busy checking for receipts.
Each agent in this scenario waits for a certain number of peers to be available or for up to two minutes, whichever happens first, before starting its behaviour.
By default, this scenario will wait for a complete set of validation receipts before committing the next record. If the NO_VALIDATION_COMPLETE environment variable is set, it will instead publish new records on every round, building up an ever-growing list of action hashes to check on.
Started
Thu, 02 Jul 2026 01:40:03 UTC
Peer count
20
Peer count at end
18
Behaviours
-
default(1 agent)
Holochain version
0.7.0-dev.31
Wind Tunnel version
0.7.0
Run ID
validation_receipts_28558451397_1
Receipts complete timing
The amount of time between publishing a record and receiving the required number of validation receipts.
Mean
mean
10.890579
s
SD
7.449229
s
Max
Highest per-partition mean latency
max mean
15.524004
s
Min
Lowest per-partition mean latency
min mean
8.41821
s
Receipts complete rate
The number of complete validation receipt sets collected per second.
Mean
mean
1.21
/s
Max
Highest per-partition mean rate
max
1.34
/s
Min
Lowest per-partition mean rate
min
1.02
/s
Error count
The total number of errors accumulated during the run.
1
Holochain MetricsClick to toggle visibility.
WASM usage: 'crud::create_sample_entry'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 37468.918635738s
Not enough time data to show a trend.
WASM usage: 'crud::get_sample_entry_validation_receipts'
Metered usage count of the WASM ribosome for this zome function.
total
994500
over 37468.918635738s
mean rate
9.82000838e+06
/s
std
2.483333077e+07
/s
p5
1299.98
/s
p95
8.072112997e+07
/s
peak
1.3026190119e+08
/s
WASM usage: 'crud::init'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 37468.918635738s
Not enough time data to show a trend.
WASM usage: 'crud_integrity::entry_defs'
Metered usage count of the WASM ribosome for this zome function.
total
4402
over 37470.204850285s
mean rate
452314.36
/s
std
1.30755081e+06
/s
p5
0
/s
p95
3.69967315e+06
/s
peak
7.70537713e+06
/s
Zome call duration: 'crud::create_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.016116
s
SD
0.010674
s
p50
0.019038
s
p95
0.039956
s
p99
0.04715
s
Zome call duration: 'crud::get_sample_entry_validation_receipts'
Duration of this zome call as measured by Holochain internally.
mean
0.01622
s
SD
0.007718
s
p50
0.013847
s
p95
0.032077
s
p99
0.037052
s
WASM call duration: 'crud::create_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.009247
s
SD
0.004236
s
p50
0.009694
s
p95
0.020788
s
p99
0.020788
s
WASM call duration: 'crud::get_sample_entry_validation_receipts'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.009964
s
SD
0.006567
s
p50
0.005889
s
p95
0.021894
s
p99
0.027327
s
WASM call duration: 'crud::init'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.002041
s
SD
0.001539
s
p50
0.001428
s
p95
0.007903
s
p99
0.007903
s
WASM call duration: 'crud_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001681
s
SD
0.000639
s
p50
0.001261
s
p95
0.002628
s
p99
0.0029
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000519
s
SD
0.0005
s
p50
0.000376
s
p95
0.001191
s
p99
0.003229
s
Host function call duration: '__hc__get_validation_receipts_1'
Duration of this host function call invoked from within WASM.
mean
0.00782
s
SD
0.006014
s
p50
0.004094
s
p95
0.018987
s
p99
0.024803
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.006724
s
SD
0.00417
s
p50
0.006783
s
p95
0.01907
s
p99
0.01907
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.00794
s
SD
0.007577
s
p50
0.008061
s
p95
0.023607
s
p99
0.033937
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
46.57
s
mean
449.69
s
SD
258.76
s
p95
854.31
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 37470.204942349s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
1.871942
s
SD
1.451187
s
p50
1.619807
s
p95
4.684267
s
p99
6.396894
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.029958
s
SD
0.225678
s
p50
0.012305
s
p95
0.622646
s
p99
1.146291
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0.003937
s
SD
0.00723
s
p50
0.001628
s
p95
0.033535
s
p99
0.033535
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.056204
s
SD
0.042691
s
p50
0.049665
s
p95
0.146313
s
p99
0.172736
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.03785
s
SD
0.049094
s
p50
0.012857
s
p95
0.136628
s
p99
0.256767
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.111915
s
SD
0.442642
s
p50
0.111021
s
p95
0.700644
s
p99
2.602833
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
2.768558
s
SD
8.460904
s
p50
2.229549
s
p95
25.266482
s
p99
39.037606
s
Authored DB connection use time
Time spent holding authored database connections.
mean
0.000367
s
SD
0.000307
s
p50
0.000304
s
p95
0.000648
s
p99
0.000858
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.002859
s
SD
0.016325
s
p50
0.000553
s
p95
0.050498
s
p99
0.08723
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000357
s
SD
0.00023
s
p50
0.000339
s
p95
0.000613
s
p99
0.00078
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
0.04624
s
SD
0.04881
s
p50
0.033884
s
p95
0.157616
s
p99
0.163656
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.311506
s
SD
4.117618
s
p50
0.300863
s
p95
10.157197
s
p99
10.669791
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
38.78
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
832
B/s
mean
646.50
KiB/s
SD
513.20
KiB/s
p95
1.42
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
182
B/s
mean
539.71
KiB/s
SD
561.11
KiB/s
p95
1.44
MiB/s
Total bytes received
Total bytes received on primary network interface
count
10.63
GiB
mean
11.46
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
8.87
GiB
mean
9.56
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 96.6%
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 623.10 MiB/s
Disk full anomaly
Disk full anomaly was detected
Detected
Critical Disk /etc/resolv.conf: 1/5 hosts above 90% capacity
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected
Critical Heavy swap usage (23.2% swap used)
System overload anomaly
System overload anomaly was detected
Detected
Warning 53% of hosts overloaded (load5/ncpus > 1.0)
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
1.2
%
mean
48.53
%
SD
33.11
%
p95
94.57
%
System
9.75
%
CPU percentiles
CPU usage percentiles
p50
46.39
%
p95
94.57
%
p99
96.59
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
8
hosts
mean time
0.64
s
Memory used percentage
Percentage of memory used
p5
5.96
%
mean
21.27
%
SD
15.73
%
p95
80.05
%
Memory available percentage
Percentage of available memory
p5
19.95
%
mean
78.73
%
SD
15.73
%
p95
94.04
%
Max host memory used
Maximum memory usage percentage across all hosts
max
80.80
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
23.16
%
Memory growth rate
Rate of memory growth over time
growth
623.10
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.42
MiB/s
Disk write throughput
Disk write throughput in MiB/s
129.96
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/0/6
hosts
Mount Point
/efi-boot0/4
hosts
Mount Point
/etc/hostname1/5
hosts
Mount Point
/etc/hosts1/5
hosts
Mount Point
/etc/nomad.d/client-meta.json0/3
hosts
Mount Point
/etc/resolv.conf1/5
hosts
Mount Point
/nix/store0/6
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
5.19
5 min
4.29
15 min
2.58
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
52.63
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
22.5899
%
SD
21.2892
%
p95
60.45
%
60 second average
22.7112
%
300 second average
19.4929
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0.0101
%
SD
0.1675
%
p95
0
%
60 second average
0.0187
%
300 second average
0.0167
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0.0071
%
SD
0.1256
%
p95
0
%
60 second average
0.0129
%
300 second average
0.0110
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
21.3374
%
SD
25.3828
%
p95
65.88
%
60 second average
20.9055
%
300 second average
16.5700
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
14.493
%
SD
19.4412
%
p95
58.3
%
60 second average
14.2325
%
300 second average
11.3233
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0
%
mean
2.33
%
SD
5.93
%
p95
4.1
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
17.48
KiB
mean
229.54
KiB
SD
79.46
KiB
p95
303.08
KiB
Holochain process threads
Number of threads in Holochain process
p5
8
threads
mean
35.56
threads
SD
11.61
threads
p95
51
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
8
file descriptors
mean
80.85
file descriptors
SD
26.71
file descriptors
p95
100
file descriptors
Write/get_agent_activity
↑ Back to index
A scenario where write peers write entries, while get_agent_activity peers each query a single write agent's activity with get_agent_activity.
Before a target write peer and the requesting get_agent_activity peer are in sync, this will measure the get_agent_activity call performance over a network. Once a write peer reaches sync with a get_agent_activity peer, the write peer will publish their actions and entries, and so the get_agent_activity calls will likely have most of the data they need locally. At that point this measures the database query performance and code paths through host functions.
Started
Thu, 02 Jul 2026 02:08:38 UTC
Peer count
30
Peer count at end
26
Behaviours
-
get_agent_activity(1 agent) -
write(1 agent)
Holochain version
0.7.0-dev.31
Wind Tunnel version
0.7.0
Run ID
write_get_agent_activity_28558451397_1
Observed chain advancement
The maximum highest-observed action sequence per write agent, aggregated across all reading agents. The reading dimension is collapsed by taking the maximum observed value for each write agent (any reader successfully tracking that point counts as propagation). Captures how far readers tracked writers’ chains during the run.
Mean max (across write agents)
Mean of the highest chain head value observed per write agent. Each write agent contributes its maximum — the value seen by any reader.
3174
Max
Highest chain head value observed for any single write agent.
4688
Write agents observed
5
get_agent_activity_full zome call timingThe time taken to call the zome function that retrieves information on a
write peer's source chain.Mean
mean
0.092637
s
SD
0.119297
s
Max
Highest per-partition mean latency
max mean
0.23775
s
Min
Lowest per-partition mean latency
min mean
0.025316
s
Error count
The number of errors accumulated.
27
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.030878
s
SD
0.226594
s
p50
0.021611
s
p95
0.560072
s
p99
1.14126
s
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
0
over 1034.15890739s
Not enough time data to show a trend.
Zome call duration: 'agent_activity::announce_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.035794
s
SD
0.020338
s
p50
0.024394
s
p95
0.065335
s
p99
0.065335
s
Zome call duration: 'agent_activity::create_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.048221
s
SD
0.031173
s
p50
0.034574
s
p95
0.115006
s
p99
0.130654
s
Zome call duration: 'agent_activity::get_agent_activity_full'
Duration of this zome call as measured by Holochain internally.
mean
0.039607
s
SD
0.068201
s
p50
0.039258
s
p95
0.213372
s
p99
0.226002
s
Zome call duration: 'agent_activity::get_random_agent_with_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.007227
s
SD
0.30325
s
p50
0.02068
s
p95
1.163569
s
p99
1.163569
s
WASM call duration: 'agent_activity::announce_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.014357
s
SD
0.008466
s
p50
0.014117
s
p95
0.029439
s
p99
0.029439
s
WASM call duration: 'agent_activity::create_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.027376
s
SD
0.016919
s
p50
0.020552
s
p95
0.061903
s
p99
0.07027
s
WASM call duration: 'agent_activity::get_agent_activity_full'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.035958
s
SD
0.0631
s
p50
0.036115
s
p95
0.18116
s
p99
0.233011
s
WASM call duration: 'agent_activity::get_random_agent_with_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.004972
s
SD
0.300418
s
p50
0.009489
s
p95
1.148452
s
p99
1.148452
s
WASM call duration: 'agent_activity_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001438
s
SD
0.000601
s
p50
0.000959
s
p95
0.00264
s
p99
0.002806
s
Host function call duration: '__hc__agent_info_1'
Duration of this host function call invoked from within WASM.
mean
3.1e-05
s
SD
1e-05
s
p50
2.8e-05
s
p95
5.4e-05
s
p99
5.4e-05
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.001235
s
SD
0.000709
s
p50
0.001
s
p95
0.002895
s
p99
0.003164
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.000333
s
SD
0.000121
s
p50
0.000294
s
p95
0.000649
s
p99
0.000649
s
Host function call duration: '__hc__get_agent_activity_1'
Duration of this host function call invoked from within WASM.
mean
0.03435
s
SD
0.062903
s
p50
0.034692
s
p95
0.179776
s
p99
0.231118
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.000527
s
SD
0.300315
s
p50
0.000466
s
p95
1.141304
s
p99
1.141304
s
Host function call duration: '__hc__random_bytes_1'
Duration of this host function call invoked from within WASM.
mean
1e-05
s
SD
9e-06
s
p50
1.2e-05
s
p95
4e-05
s
p99
4e-05
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.015143
s
SD
0.011273
s
p50
0.004826
s
p95
0.03554
s
p99
0.05881
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.018647
s
SD
0.015058
s
p50
0.016871
s
p95
0.058011
s
p99
0.061389
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
47.02
s
mean
452.26
s
SD
259.92
s
p95
857.75
s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
123.674229
s
SD
101.315526
s
p50
35.154207
s
p95
279.652904
s
p99
322.02737
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.563744
s
SD
5.771926
s
p50
0.660622
s
p95
14.148819
s
p99
34.42235
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0.001675
s
SD
0.001333
s
p50
0.001224
s
p95
0.004493
s
p99
0.005528
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.107466
s
SD
0.320448
s
p50
0.291066
s
p95
0.984358
s
p99
1.013239
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.194135
s
SD
0.183779
s
p50
0.029121
s
p95
0.562085
s
p99
0.971672
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
2.473125
s
SD
18.675739
s
p50
3.488056
s
p95
50.608838
s
p99
82.052516
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
21.85292
s
SD
47.27372
s
p50
0.587972
s
p95
138.814736
s
p99
206.608006
s
Authored DB connection use time
Time spent holding authored database connections.
mean
0.000795
s
SD
0.001495
s
p50
0.000229
s
p95
0.004882
s
p99
0.006714
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.00323
s
SD
0.01769
s
p50
0.000681
s
p95
0.034568
s
p99
0.089235
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000763
s
SD
0.000474
s
p50
0.000365
s
p95
0.001505
s
p99
0.002415
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
4.486214
s
SD
29.848523
s
p50
0.199925
s
p95
60.002579
s
p99
60.242297
s
P2P request roundtrip: 'get_links'
Time spent sending a get_links request and awaiting its response.
mean
0.109409
s
SD
0.301227
s
p50
0.400744
s
p95
1.13869
s
p99
1.13869
s
P2P request roundtrip: 'get_agent_activity'
Time spent sending a get_agent_activity request and awaiting its response.
mean
0.04385
s
SD
25.167689
s
p50
0.066879
s
p95
60.001676
s
p99
60.002529
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.519346
s
SD
2.66822
s
p50
0.501601
s
p95
10.110025
s
p99
10.149889
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
40.70
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
76.45
KiB/s
mean
534.96
KiB/s
SD
565.11
KiB/s
p95
1.25
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
87.06
KiB/s
mean
478.62
KiB/s
SD
353.12
KiB/s
p95
1.12
MiB/s
Total bytes received
Total bytes received on primary network interface
count
12.39
GiB
mean
13.94
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
11.09
GiB
mean
12.48
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 98.0%
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 336.42 MiB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
Detected
Warning 56% of hosts overloaded (load5/ncpus > 1.0)
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
12.52
%
mean
51.2
%
SD
28.63
%
p95
97.15
%
System
10.49
%
CPU percentiles
CPU usage percentiles
p50
43.72
%
p95
97.15
%
p99
98.05
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
11
hosts
mean time
0.59
s
Memory used percentage
Percentage of memory used
p5
6.39
%
mean
13.99
%
SD
13.26
%
p95
25.37
%
Memory available percentage
Percentage of available memory
p5
74.63
%
mean
86.01
%
SD
13.26
%
p95
93.61
%
Max host memory used
Maximum memory usage percentage across all hosts
max
78.42
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00
%
Memory growth rate
Rate of memory growth over time
growth
336.42
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.15
MiB/s
Disk write throughput
Disk write throughput in MiB/s
169.38
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/0/6
hosts
Mount Point
/efi-boot0/5
hosts
Mount Point
/etc/hostname0/6
hosts
Mount Point
/etc/hosts0/6
hosts
Mount Point
/etc/nomad.d/client-meta.json0/6
hosts
Mount Point
/etc/resolv.conf0/6
hosts
Mount Point
/nix/store0/6
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
4.36
5 min
4.31
15 min
4.06
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
55.56
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
24.4008
%
SD
28.8954
%
p95
92.29
%
60 second average
24.0006
%
300 second average
23.5430
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0.0001
%
SD
0.0031
%
p95
0
%
60 second average
0.0003
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0.0001
%
SD
0.0031
%
p95
0
%
60 second average
0.0003
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
13.7373
%
SD
18.6843
%
p95
50.13
%
60 second average
13.5530
%
300 second average
13.0849
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
9.9073
%
SD
14.7109
%
p95
38.57
%
60 second average
9.7539
%
300 second average
9.0542
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0
%
mean
43.44
%
SD
28.08
%
p95
94.32
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
20.99
KiB
mean
342.43
KiB
SD
120.69
KiB
p95
531.11
KiB
Holochain process threads
Number of threads in Holochain process
p5
9
threads
mean
40.35
threads
SD
12.38
threads
p95
57
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
8
file descriptors
mean
90.89
file descriptors
SD
26.26
file descriptors
p95
113
file descriptors
Write/get_agent_activity with volatile conductors
↑ Back to index
A scenario where write peers write entries, while get_agent_activity_volatile peers each query a single write agents activity with get_agent_activity but shutdown and restart their conductors at semi-random intervals.
Before a target write peer and the requesting get_agent_activity_volatile peer are in sync, this will measure the get_agent_activity call performance over a network. Once a write peer reaches sync with a get_agent_activity peer, the write peer will publish their actions and entries, and so the get_agent_activity calls will likely have most of the data they need locally. At that point this measures the database query performance and code paths through host functions.
Started
Thu, 02 Jul 2026 02:24:27 UTC
Peer count
30
Peer count at end
22
Behaviours
-
get_agent_activity_volatile(1 agent) -
write(1 agent)
Holochain version
0.7.0-dev.31
Wind Tunnel version
0.7.0
Run ID
write_get_agent_activity_volatile_28558451397_1
Highest observed
action_seqThe rate at which zero-arc readers observe new chain heads on the writers' chains via
get_agent_activity. This reflects the DHT's ability to propagate agent activity ops and make them available to querying peers.Mean
mean
102.28
/s
Max
Highest per-partition mean rate
max
508.66
/s
Min
Lowest per-partition mean rate
min
16.51
/s
get_agent_activity_full zome call timingThe time taken to call the zome function that retrieves information on a
write peer's source chain.Mean
mean
0.140762
s
SD
0.239906
s
Max
Highest per-partition mean latency
max mean
0.405399
s
Min
Lowest per-partition mean latency
min mean
0.013307
s
Running volatile conductors count
The number of conductors running by
get_agent_activity_volatile peers.p5
10
conductors
mean
12.91
conductors
SD
1.96
conductors
p95
16
conductors
Volatile conductor total running duration
The total running duration of a
get_agent_activity_volatile conductor.Mean
mean
953.869606
s
SD
571.484569
s
Max
Highest per-partition mean latency
max mean
1583.961538
s
Min
Lowest per-partition mean latency
min mean
489.034483
s
Volatile conductor running duration
The duration that a
get_agent_activity_volatile conductor was running before being stopped.Mean
mean
68.973481
s
SD
57.177235
s
Max
Highest per-partition mean latency
max mean
172.752179
s
Min
Lowest per-partition mean latency
min mean
36.676848
s
Volatile conductor stopped duration
The duration that a
get_agent_activity_volatile conductor was stopped before being started again.Mean
mean
39.683954
s
SD
56.900154
s
Max
Highest per-partition mean latency
max mean
68.896224
s
Min
Lowest per-partition mean latency
min mean
8.406474
s
Reached target arc
Did
get_agent_activity_volatile peer reach their target arc in the moment before they were shutdown.get_agent_activity_volatile_agent:uhCAk3zwCU0clYuUfFh0CpnnK2swjcV8CxTnTpx_H2N2_Cl_b0FCn
p5
0
mean
0
SD
0
p95
0
get_agent_activity_volatile_agent:uhCAk51__rNJLFkus8xhMRNy84RV89NIwZny3KcCqI1JRHHI_UbAH
p5
0
mean
0.08
SD
0.27
p95
1
get_agent_activity_volatile_agent:uhCAk6u-ywmfl5vw8zbhgNPzwb4Mw8LMUI0iVRcI9lnXWXu0wHiJl
p5
0
mean
0.09
SD
0.29
p95
1
get_agent_activity_volatile_agent:uhCAkGziaVHX7oIvnvfrjubmAIh-zxIIPrTFs-I1vRawktDCcHaXm
p5
0
mean
0.08
SD
0.27
p95
1
get_agent_activity_volatile_agent:uhCAkHS5-ADe-uA8K5EmGQiDBV8Y1eQN3jL7-W8I05bOyyMMg6yrB
p5
0
mean
0
SD
0
p95
0
get_agent_activity_volatile_agent:uhCAkKidQKv5LT_nVd8onYCUitX36cgSh8-77uJA2ReQtkK_KNjsA
p5
0
mean
0.29
SD
0.46
p95
1
get_agent_activity_volatile_agent:uhCAkM3iU1EvERW8fty_IldqN5YdDGo4T3hTU6SlSBM-P8Hu2S5-Y
p5
0
mean
0.33
SD
0.47
p95
1
get_agent_activity_volatile_agent:uhCAkOIyP6DNRtVSjjIEItE1hW5DJNw6BG0XhnMdWDuFsL9kNgb0p
p5
0
mean
0.28
SD
0.45
p95
1
get_agent_activity_volatile_agent:uhCAkTNyzuMhqp1p5olRhpLaYljo_QsnkR706D2soguw9fNYW01FG
p5
0
mean
0.15
SD
0.36
p95
1
get_agent_activity_volatile_agent:uhCAkY-C984CbNDD78iMnPzzb3s_NYMbTbwOGBTBh36B3OEZIZMNn
p5
0
mean
0.15
SD
0.35
p95
1
get_agent_activity_volatile_agent:uhCAkY9K_duO5SUCowl4UBs3Pajvri8Wx8IikXGndYaQ-8Pv3j6OL
p5
0
mean
0.23
SD
0.42
p95
1
get_agent_activity_volatile_agent:uhCAkdRq-Us8Ivp2ZKVemz8BY_jm7g_8js_72cOHM9g3QU11orMcd
p5
0
mean
0.36
SD
0.48
p95
1
get_agent_activity_volatile_agent:uhCAkdjWcj1q1OqLywQ425UKdYVn7dIG4XBDBoXaL9S3Qf0T2rePy
p5
0
mean
0.17
SD
0.38
p95
1
get_agent_activity_volatile_agent:uhCAkeQ-SXBHxqnLN9BUuPBUV-ehp0XtuDvKhjoxwe4YiPESvGdbi
p5
0
mean
0.12
SD
0.32
p95
1
get_agent_activity_volatile_agent:uhCAknNcAjMjw-WHsCKU60VFz5Ff3QD0PQ3oVeLCo4fdW8jqL-orf
p5
0
mean
0.14
SD
0.35
p95
1
get_agent_activity_volatile_agent:uhCAkqzcD3wPK8XxKzzyEVco9OAbtCLYId4Qb8PlnQ4a2F4WB96bm
p5
0
mean
0.27
SD
0.44
p95
1
get_agent_activity_volatile_agent:uhCAksBXQKDFEz7tIOiuft5g8FMIvRf6zaHy0usDRZeJHGb4NX88w
p5
0
mean
0
SD
0
p95
0
get_agent_activity_volatile_agent:uhCAkzFkpuqOHcWsliqaqjdfHeligXvNQurlPLJ0PgIolNrHvPZBu
p5
0
mean
0.13
SD
0.34
p95
1
get_agent_activity_volatile_agent:uhCAkzgGnhcm0ck65hU5sgY4gnKPaYhHwuDZ8Qi76qTGkzLOM3dHi
p5
0
mean
0.17
SD
0.38
p95
1
Error count
The number of errors accumulated.
149
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.05332
s
SD
9.152341
s
p50
0.164998
s
p95
1.127993
s
p99
15.654871
s
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
8
over 3166.763815299s
mean rate
1.2
/s
std
3.9
/s
p5
0
/s
p95
16.24
/s
peak
20.27
/s
WASM usage: 'agent_activity::announce_write_behaviour'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 3320.963042121s
Not enough time data to show a trend.
WASM usage: 'agent_activity::create_sample_entry'
Metered usage count of the WASM ribosome for this zome function.
total
410189472
over 3320.963042121s
mean rate
9.192895346e+07
/s
std
1.3845857109e+08
/s
p5
203010.29
/s
p95
3.8331692557e+08
/s
peak
3.9811689181e+08
/s
WASM usage: 'agent_activity::get_agent_activity_full'
Metered usage count of the WASM ribosome for this zome function.
total
11056755
over 2796.720441742s
mean rate
7.984584448e+08
/s
std
9.93190289738e+09
/s
p5
742839.07
/s
p95
2.35275712515e+09
/s
peak
3.09741092729e+11
/s
WASM usage: 'agent_activity::get_random_agent_with_write_behaviour'
Metered usage count of the WASM ribosome for this zome function.
total
67854735
over 38349.681521116s
mean rate
4.582063737e+07
/s
std
1.8263839665e+08
/s
p5
0
/s
p95
1.0880733183e+08
/s
peak
1.43439745487e+09
/s
WASM usage: 'agent_activity_integrity::entry_defs'
Metered usage count of the WASM ribosome for this zome function.
total
5663174
over 38349.707935715s
mean rate
3.268747878e+07
/s
std
1.8010802268e+08
/s
p5
10804.91
/s
p95
4.801262992e+07
/s
peak
1.63000923027e+09
/s
Zome call duration: 'agent_activity::announce_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.025808
s
SD
0.009384
s
p50
0.022987
s
p95
0.044697
s
p99
0.044697
s
Zome call duration: 'agent_activity::create_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.049116
s
SD
0.030136
s
p50
0.040363
s
p95
0.099579
s
p99
0.104329
s
Zome call duration: 'agent_activity::get_agent_activity_full'
Duration of this zome call as measured by Holochain internally.
mean
0.045226
s
SD
0.3234
s
p50
0.174887
s
p95
0.865932
s
p99
1.364784
s
Zome call duration: 'agent_activity::get_random_agent_with_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.008376
s
SD
0.20072
s
p50
0.00936
s
p95
0.684679
s
p99
0.730301
s
WASM call duration: 'agent_activity::announce_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.00594
s
SD
0.001577
s
p50
0.00472
s
p95
0.008083
s
p99
0.008083
s
WASM call duration: 'agent_activity::create_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.025369
s
SD
0.01735
s
p50
0.021406
s
p95
0.054788
s
p99
0.057516
s
WASM call duration: 'agent_activity::get_agent_activity_full'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.063172
s
SD
9.385922
s
p50
0.179268
s
p95
1.132417
s
p99
15.657371
s
WASM call duration: 'agent_activity::get_random_agent_with_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.005358
s
SD
0.182796
s
p50
0.005677
s
p95
0.637957
s
p99
0.637957
s
WASM call duration: 'agent_activity_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001682
s
SD
0.000702
s
p50
0.001595
s
p95
0.002665
s
p99
0.002898
s
Host function call duration: '__hc__agent_info_1'
Duration of this host function call invoked from within WASM.
mean
2.9e-05
s
SD
5e-06
s
p50
2.9e-05
s
p95
3.8e-05
s
p99
3.8e-05
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000809
s
SD
0.000488
s
p50
0.000751
s
p95
0.001933
s
p99
0.001974
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.000336
s
SD
9e-05
s
p50
0.000296
s
p95
0.000526
s
p99
0.000526
s
Host function call duration: '__hc__get_agent_activity_1'
Duration of this host function call invoked from within WASM.
mean
0.061097
s
SD
9.386018
s
p50
0.174196
s
p95
1.128063
s
p99
15.654957
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.0004
s
SD
0.179769
s
p50
0.000304
s
p95
0.630885
s
p99
0.630885
s
Host function call duration: '__hc__random_bytes_1'
Duration of this host function call invoked from within WASM.
mean
1e-05
s
SD
5e-06
s
p50
9e-06
s
p95
2e-05
s
p99
2.1e-05
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.022433
s
SD
0.014653
s
p50
0.005696
s
p95
0.047815
s
p99
0.052024
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.018032
s
SD
0.015055
s
p50
0.016875
s
p95
0.043587
s
p99
0.047121
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
5.86
s
mean
308.42
s
SD
478.07
s
p95
1498.22
s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
98.664349
s
SD
99.734311
s
p50
3.017893
s
p95
222.272903
s
p99
682.162257
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
1.197654
s
SD
6.135744
s
p50
0.016037
s
p95
11.427881
s
p99
15.408293
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0.002524
s
SD
0.002449
s
p50
0.001725
s
p95
0.007217
s
p99
0.014937
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.056782
s
SD
0.081811
s
p50
0.005011
s
p95
0.141126
s
p99
0.402534
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.214843
s
SD
0.1377
s
p50
0.062262
s
p95
0.458063
s
p99
0.595595
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
9.035712
s
SD
55.54685
s
p50
10.825738
s
p95
100.310431
s
p99
201.184484
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
37.963064
s
SD
67.117451
s
p50
0.01525
s
p95
194.477299
s
p99
276.109836
s
Authored DB connection use time
Time spent holding authored database connections.
mean
0.003275
s
SD
0.001529
s
p50
0.000286
s
p95
0.00499
s
p99
0.006594
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.002922
s
SD
0.030128
s
p50
0.000599
s
p95
0.029701
s
p99
0.125622
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000665
s
SD
0.000335
s
p50
0.000511
s
p95
0.001358
s
p99
0.001773
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
8.387134
s
SD
27.201954
s
p50
0.462902
s
p95
60.002799
s
p99
60.00496
s
P2P request roundtrip: 'get_links'
Time spent sending a get_links request and awaiting its response.
mean
0.193822
s
SD
0.168586
s
p50
0.055839
s
p95
0.397218
s
p99
0.839334
s
P2P request roundtrip: 'get_agent_activity'
Time spent sending a get_agent_activity request and awaiting its response.
mean
0.153473
s
SD
17.577926
s
p50
0.216251
s
p95
60.001729
s
p99
60.002062
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.391645
s
SD
25.56901
s
p50
0.338136
s
p95
60.001602
s
p99
60.066036
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
20.87
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
818
B/s
mean
178.22
KiB/s
SD
415.02
KiB/s
p95
809.03
KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
435
B/s
mean
145.09
KiB/s
SD
273.78
KiB/s
p95
778.67
KiB/s
Total bytes received
Total bytes received on primary network interface
count
7.63
GiB
mean
4.27
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
6.21
GiB
mean
3.47
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 98.8%
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 436.10 MiB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
Detected
Warning 16% of hosts overloaded (load5/ncpus > 1.0)
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
0.73
%
mean
26.95
%
SD
33.34
%
p95
98.5
%
System
6.08
%
CPU percentiles
CPU usage percentiles
p50
10.68
%
p95
98.50
%
p99
98.85
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
7
hosts
mean time
0.52
s
Memory used percentage
Percentage of memory used
p5
5.65
%
mean
13.79
%
SD
12.99
%
p95
20.22
%
Memory available percentage
Percentage of available memory
p5
79.78
%
mean
86.21
%
SD
12.99
%
p95
94.35
%
Max host memory used
Maximum memory usage percentage across all hosts
max
75.39
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00
%
Memory growth rate
Rate of memory growth over time
growth
436.10
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.07
MiB/s
Disk write throughput
Disk write throughput in MiB/s
48.20
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/0/7
hosts
Mount Point
/efi-boot0/4
hosts
Mount Point
/etc/hostname0/6
hosts
Mount Point
/etc/hosts0/6
hosts
Mount Point
/etc/nomad.d/client-meta.json0/5
hosts
Mount Point
/etc/resolv.conf0/6
hosts
Mount Point
/nix/store0/7
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
2.37
5 min
2.47
15 min
2.81
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
16.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
13.8543
%
SD
29.0962
%
p95
95.9
%
60 second average
13.7583
%
300 second average
14.2050
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
4.5298
%
SD
10.2184
%
p95
34.43
%
60 second average
4.7470
%
300 second average
6.0095
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
3.1283
%
SD
7.6156
%
p95
23.01
%
60 second average
3.3037
%
300 second average
4.2012
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0
%
mean
21.05
%
SD
31.79
%
p95
97.36
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
16.87
KiB
mean
203.68
KiB
SD
151.56
KiB
p95
506.93
KiB
Holochain process threads
Number of threads in Holochain process
p5
6
threads
mean
29.04
threads
SD
12.76
threads
p95
47
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
8
file descriptors
mean
76.03
file descriptors
SD
29
file descriptors
p95
105
file descriptors
Write/Query
↑ Back to indexCreates an entry, then queries the source chain and performs a simple operation on the entries, then repeats.
Started
Thu, 02 Jul 2026 01:24:48 UTC
Peer count
1
Behaviours
-
default(1 agent)
Holochain version
0.7.0-dev.31
Wind Tunnel version
0.7.0
Run ID
write_query_28558451397_1
Write timing
The time it takes to complete the zome function call that writes the entry.
mean
0.051676
s
SD
0.008013
s
p50
0.053192
s
p95
0.060124
s
p99
0.069841
s
Write rate
The number of writes completed per second.
mean
62.71
/s
Query timing
The time it takes to complete the
chain_query_count_len zome call that queries the source chain.mean
0.105834
s
SD
0.037946
s
p50
0.109355
s
p95
0.163874
s
p99
0.177829
s
Query rate
The number of chain queries completed per second.
mean
62.71
/s
Error count
The number of errors accumulated during the run.
0
Holochain MetricsClick to toggle visibility.
WASM usage: 'crud::chain_query_count_len'
Metered usage count of the WASM ribosome for this zome function.
total
44099512097
over 280.003235031s
mean rate
1.5749640044e+08
/s
std
5.695442815e+07
/s
p5
5.167288153e+07
/s
p95
2.392951805e+08
/s
peak
2.4179465192e+08
/s
WASM usage: 'crud::create_sample_entry'
Metered usage count of the WASM ribosome for this zome function.
total
62845552
over 280.003235031s
mean rate
224445.82
/s
std
47734.22
/s
p5
175066.25
/s
p95
332268.86
/s
peak
357279.17
/s
WASM usage: 'crud::init'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 280.003235031s
Not enough time data to show a trend.
WASM usage: 'crud_integrity::entry_defs'
Metered usage count of the WASM ribosome for this zome function.
total
15481834
over 280.003235031s
mean rate
55291.63
/s
std
11762.62
/s
p5
43139.37
/s
p95
81876.82
/s
peak
88039.8
/s
Zome call duration: 'crud::chain_query_count_len'
Duration of this zome call as measured by Holochain internally.
mean
0.072103
s
SD
0.017665
s
p50
0.066111
s
p95
0.087082
s
p99
0.088275
s
Zome call duration: 'crud::create_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.023129
s
SD
0.000609
s
p50
0.02336
s
p95
0.024122
s
p99
0.024131
s
WASM call duration: 'crud::chain_query_count_len'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.062998
s
SD
0.017867
s
p50
0.056855
s
p95
0.078244
s
p99
0.079475
s
WASM call duration: 'crud::create_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.015698
s
SD
0.000434
s
p50
0.015796
s
p95
0.016267
s
p99
0.016272
s
WASM call duration: 'crud::init'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001421
s
SD
0
s
p50
0.001421
s
p95
0.001421
s
p99
0.001421
s
WASM call duration: 'crud_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.003139
s
SD
6.7e-05
s
p50
0.003147
s
p95
0.00321
s
p99
0.003211
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000957
s
SD
0.000331
s
p50
0.000857
s
p95
0.000971
s
p99
0.000972
s
Host function call duration: '__hc__query_1'
Duration of this host function call invoked from within WASM.
mean
0.020339
s
SD
0.007807
s
p50
0.017352
s
p95
0.027432
s
p99
0.028038
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.011592
s
SD
0.000528
s
p50
0.011475
s
p95
0.01224
s
p99
0.012245
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.011069
s
SD
0.000296
s
p50
0.011205
s
p95
0.011395
s
p99
0.011406
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
17.92
s
mean
147.92
s
SD
83.67
s
p95
277.92
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 280.003255182s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.000828
s
SD
0
s
p50
0.000828
s
p95
0.000828
s
p99
0.000828
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0.001174
s
SD
0
s
p50
0.001174
s
p95
0.001174
s
p99
0.001174
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.005835
s
SD
0.000198
s
p50
0.005873
s
p95
0.005938
s
p99
0.005941
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.043714
s
SD
0.014442
s
p50
0.03823
s
p95
0.056623
s
p99
0.057771
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.004025
s
SD
0
s
p50
0.004025
s
p95
0.004025
s
p99
0.004025
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
0.003665
s
SD
0
s
p50
0.003665
s
p95
0.003665
s
p99
0.003665
s
Authored DB connection use time
Time spent holding authored database connections.
mean
0.002718
s
SD
0.000836
s
p50
0.002403
s
p95
0.003458
s
p99
0.003524
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.001691
s
SD
0.000131
s
p50
0.001705
s
p95
0.001805
s
p99
0.001807
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000812
s
SD
2.4e-05
s
p50
0.000814
s
p95
0.000823
s
p99
0.000824
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
33.11
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
965
B/s
mean
6.59
KiB/s
SD
21.21
KiB/s
p95
7.85
KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
0
B/s
mean
697
B/s
SD
1.36
KiB/s
p95
1.27
KiB/s
Total bytes received
Total bytes received on primary network interface
count
1.87
MiB
mean
6.59
KiB/s
Total bytes sent
Total bytes sent on primary network interface
count
197.36
KiB
mean
697
B/s
CPU spike anomaly
CPU spike anomaly was detected
NotDetected
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 9.04 MiB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
NotDetected
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
36.63
%
mean
39.53
%
SD
2.17
%
p95
40.94
%
System
6.42
%
CPU percentiles
CPU usage percentiles
p50
39.78
%
p95
40.94
%
p99
48.59
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
0
hosts
mean time
0.00
s
Memory used percentage
Percentage of memory used
p5
10.86
%
mean
11.17
%
SD
0.29
%
p95
11.66
%
Memory available percentage
Percentage of available memory
p5
88.34
%
mean
88.83
%
SD
0.29
%
p95
89.14
%
Max host memory used
Maximum memory usage percentage across all hosts
max
11.17
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00
%
Memory growth rate
Rate of memory growth over time
growth
9.04
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.00
MiB/s
Disk write throughput
Disk write throughput in MiB/s
1.98
MiB/s
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
1.48
5 min
0.68
15 min
0.26
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
0.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
1.07
%
mean
2.0633
%
SD
0.5962
%
p95
3
%
60 second average
1.7893
%
300 second average
0.8933
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
0.0087
%
SD
0.0337
%
p95
0.06
%
60 second average
0.0033
%
300 second average
0.0000
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
0.0087
%
SD
0.0337
%
p95
0.06
%
60 second average
0.0033
%
300 second average
0.0000
%
Holochain process CPU usage
CPU usage by Holochain process
p5
34.65
%
mean
38
%
SD
2.33
%
p95
39.83
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
222.65
KiB
mean
235.05
KiB
SD
9.63
KiB
p95
246.19
KiB
Holochain process threads
Number of threads in Holochain process
p5
30
threads
mean
29.67
threads
SD
4.24
threads
p95
31
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
74
file descriptors
mean
72.57
file descriptors
SD
11.32
file descriptors
p95
77
file descriptors
Write/Read
↑ Back to indexCreates an entry and reads it back, then repeats.
Started
Thu, 02 Jul 2026 01:27:00 UTC
Peer count
1
Behaviours
-
default(1 agent)
Holochain version
0.7.0-dev.31
Wind Tunnel version
0.7.0
Run ID
write_read_28558451397_1
Write timing
The time taken to write the entry.
mean
0.040223
s
SD
0.039217
s
p50
0.024209
s
p95
0.107831
s
p99
0.154521
s
Write rate
The number of writes completed per second.
mean
149.31
/s
Read timing
The time taken to read the entry.
mean
0.026141
s
SD
0.021216
s
p50
0.017298
s
p95
0.067433
s
p99
0.098628
s
Read rate
The number of reads completed per second.
mean
149.31
/s
Error count
The number of errors accumulated over the run.
0
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.001013
s
SD
0.000137
s
p50
0.000951
s
p95
0.001139
s
p99
0.001157
s
WASM usage: 'crud::create_sample_entry'
Metered usage count of the WASM ribosome for this zome function.
total
153630400
over 290.06804995s
mean rate
529685.59
/s
std
354037.06
/s
p5
239376.34
/s
p95
1.2183129e+06
/s
peak
1.48615293e+06
/s
WASM usage: 'crud::get_sample_entry'
Metered usage count of the WASM ribosome for this zome function.
total
166101379
over 290.06804995s
mean rate
572682.71
/s
std
381998.82
/s
p5
258906.74
/s
p95
1.32157769e+06
/s
peak
1.59719068e+06
/s
WASM usage: 'crud::init'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 290.06804995s
Not enough time data to show a trend.
WASM usage: 'crud_integrity::entry_defs'
Metered usage count of the WASM ribosome for this zome function.
total
18928600
over 290.06804995s
mean rate
65261.87
/s
std
43627.18
/s
p5
29493.24
/s
p95
150546.93
/s
peak
183106.95
/s
Zome call duration: 'crud::create_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.015273
s
SD
0.004868
s
p50
0.01369
s
p95
0.020906
s
p99
0.021381
s
Zome call duration: 'crud::get_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.010091
s
SD
0.003036
s
p50
0.009345
s
p95
0.01321
s
p99
0.013445
s
WASM call duration: 'crud::create_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.00791
s
SD
0.00205
s
p50
0.007112
s
p95
0.010442
s
p99
0.010687
s
WASM call duration: 'crud::get_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.002279
s
SD
0.000181
s
p50
0.002195
s
p95
0.002442
s
p99
0.002464
s
WASM call duration: 'crud::init'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001308
s
SD
0
s
p50
0.001308
s
p95
0.001308
s
p99
0.001308
s
WASM call duration: 'crud_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001026
s
SD
6.2e-05
s
p50
0.001013
s
p95
0.001078
s
p99
0.001079
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000357
s
SD
4.4e-05
s
p50
0.000278
s
p95
0.000398
s
p99
0.0004
s
Host function call duration: '__hc__get_1'
Duration of this host function call invoked from within WASM.
mean
0.001069
s
SD
0.00014
s
p50
0.001005
s
p95
0.001198
s
p99
0.001218
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.006376
s
SD
0.00195
s
p50
0.0056
s
p95
0.008816
s
p99
0.009059
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.004076
s
SD
0.000655
s
p50
0.003991
s
p95
0.004646
s
p99
0.004668
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
17
s
mean
152.02
s
SD
86.58
s
p95
287.07
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 290.068071959s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.001853
s
SD
0
s
p50
0.001853
s
p95
0.001853
s
p99
0.001853
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0.001314
s
SD
0
s
p50
0.001314
s
p95
0.001314
s
p99
0.001314
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.002979
s
SD
0.000357
s
p50
0.00289
s
p95
0.00331
s
p99
0.00333
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.044166
s
SD
0.016985
s
p50
0.039597
s
p95
0.062317
s
p99
0.063846
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.00517
s
SD
0
s
p50
0.00517
s
p95
0.00517
s
p99
0.00517
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
0.003566
s
SD
0
s
p50
0.003566
s
p95
0.003566
s
p99
0.003566
s
Authored DB connection use time
Time spent holding authored database connections.
mean
0.001117
s
SD
0.000366
s
p50
0.000995
s
p95
0.001517
s
p99
0.001553
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.000784
s
SD
0.000171
s
p50
0.000765
s
p95
0.000993
s
p99
0.000996
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000313
s
SD
4.1e-05
s
p50
0.000302
s
p95
0.000354
s
p99
0.000355
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
77.80
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
343
B/s
mean
236.80
KiB/s
SD
1.10
MiB/s
p95
422.29
KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
34
B/s
mean
5.41
KiB/s
SD
22.66
KiB/s
p95
14.02
KiB/s
Total bytes received
Total bytes received on primary network interface
count
69.38
MiB
mean
236.80
KiB/s
Total bytes sent
Total bytes sent on primary network interface
count
1.59
MiB
mean
5.41
KiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 99.2%
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 784.29 MiB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
NotDetected
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
63.69
%
mean
88.99
%
SD
11.49
%
p95
99.15
%
System
11.19
%
CPU percentiles
CPU usage percentiles
p50
97.31
%
p95
99.15
%
p99
99.20
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
1
hosts
mean time
0.83
s
Memory used percentage
Percentage of memory used
p5
10.67
%
mean
14.43
%
SD
3.17
%
p95
17.65
%
Memory available percentage
Percentage of available memory
p5
82.35
%
mean
85.57
%
SD
3.17
%
p95
89.33
%
Max host memory used
Maximum memory usage percentage across all hosts
max
14.43
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00
%
Memory growth rate
Rate of memory growth over time
growth
784.29
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.07
MiB/s
Disk write throughput
Disk write throughput in MiB/s
3.91
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/etc/nomad.d/client-meta.json0/1
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
3.55
5 min
1.54
15 min
0.60
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
0.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
3.59
%
mean
57.6632
%
SD
28.1445
%
p95
89.41
%
60 second average
44.5842
%
300 second average
18.9500
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0.63
%
mean
5.0823
%
SD
4.2407
%
p95
13.65
%
60 second average
4.1677
%
300 second average
1.8748
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0.04
%
mean
1.2474
%
SD
1.7616
%
p95
6.47
%
60 second average
1.1761
%
300 second average
0.6610
%
Holochain process CPU usage
CPU usage by Holochain process
p5
44.91
%
mean
59.7
%
SD
15.19
%
p95
82.6
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
1.83
KiB
mean
282.74
KiB
SD
99.33
KiB
p95
384.19
KiB
Holochain process threads
Number of threads in Holochain process
p5
1
threads
mean
29.34
threads
SD
8.4
threads
p95
35
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
6
file descriptors
mean
78.03
file descriptors
SD
20.79
file descriptors
p95
91
file descriptors
Write validated
↑ Back to indexCreates an entry, updates it and links the create to the update, then repeats. Each of the three actions is validated by the included hApp.
Started
Thu, 02 Jul 2026 01:29:13 UTC
Peer count
1
Behaviours
-
default(1 agent)
Holochain version
0.7.0-dev.31
Wind Tunnel version
0.7.0
Run ID
write_validated_28558451397_1
Write timing
The time it takes to complete the zome function call that writes the entry, updates it, and links to the update.
mean
0.090577
s
SD
0.047971
s
p50
0.084989
s
p95
0.145744
s
p99
0.224167
s
Write rate
The number of write zome function calls that complete per second.
mean
39.56
/s
Read timing
The time it takes to complete the zome function call that reads the data.
mean
0.157567
s
SD
0.051024
s
p50
0.161424
s
p95
0.230328
s
p99
0.368116
s
Read rate
The number of read zome function calls that complete per second.
mean
39.56
/s
Error count
The number of errors accumulated over the run.
0
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.004825
s
SD
0.002342
s
p50
0.004029
s
p95
0.008532
s
p99
0.008818
s
WASM usage: 'validated::create_sample_entry'
Metered usage count of the WASM ribosome for this zome function.
total
35907088
over 260.226947888s
mean rate
138009.88
/s
std
37182.08
/s
p5
96117.39
/s
p95
205966.83
/s
peak
233428.94
/s
WASM usage: 'validated::update_sample_entry'
Metered usage count of the WASM ribosome for this zome function.
total
42279954
over 260.226947888s
mean rate
162503.53
/s
std
43870.7
/s
p5
117104
/s
p95
242290.63
/s
peak
274595.88
/s
WASM usage: 'validated_integrity::entry_defs'
Metered usage count of the WASM ribosome for this zome function.
total
67380384
over 280.23071028s
mean rate
240486.76
/s
std
82289.83
/s
p5
136464.02
/s
p95
372857.13
/s
peak
413302.02
/s
WASM usage: 'validated_integrity::validate'
Metered usage count of the WASM ribosome for this zome function.
total
655835028
over 260.226947888s
mean rate
2.52070549e+06
/s
std
681391.31
/s
p5
1.79086879e+06
/s
p95
3.7396191e+06
/s
peak
4.25819037e+06
/s
Zome call duration: 'validated::create_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.022253
s
SD
0.003863
s
p50
0.021203
s
p95
0.02595
s
p99
0.026611
s
Zome call duration: 'validated::update_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.034143
s
SD
0.007779
s
p50
0.032224
s
p95
0.040632
s
p99
0.041484
s
WASM call duration: 'validated::create_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.015469
s
SD
0.003528
s
p50
0.0143
s
p95
0.018363
s
p99
0.018621
s
WASM call duration: 'validated::update_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.022634
s
SD
0.005719
s
p50
0.021977
s
p95
0.026623
s
p99
0.027154
s
WASM call duration: 'validated_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001008
s
SD
5.5e-05
s
p50
0.001011
s
p95
0.001121
s
p99
0.001121
s
WASM call duration: 'validated_integrity::validate'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.007458
s
SD
0.000531
s
p50
0.007525
s
p95
0.007724
s
p99
0.007789
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000488
s
SD
6.8e-05
s
p50
0.000483
s
p95
0.00055
s
p99
0.000551
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.000327
s
SD
2.4e-05
s
p50
0.000328
s
p95
0.000337
s
p99
0.000337
s
Host function call duration: '__hc__must_get_valid_record_1'
Duration of this host function call invoked from within WASM.
mean
0.003584
s
SD
0.000919
s
p50
0.003406
s
p95
0.004407
s
p99
0.004532
s
Host function call duration: '__hc__update_1'
Duration of this host function call invoked from within WASM.
mean
0.007797
s
SD
0.002164
s
p50
0.007773
s
p95
0.009038
s
p99
0.00923
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.006433
s
SD
0.004284
s
p50
0.0082
s
p95
0.016233
s
p99
0.016866
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.014351
s
SD
0.00423
s
p50
0.014458
s
p95
0.017023
s
p99
0.01731
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
16.36
s
mean
146.47
s
SD
83.76
s
p95
276.58
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 270.227735405s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.006685
s
SD
0
s
p50
0.006685
s
p95
0.006685
s
p99
0.006685
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0.001864
s
SD
0
s
p50
0.001864
s
p95
0.001864
s
p99
0.001864
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.004442
s
SD
0.000321
s
p50
0.004443
s
p95
0.004648
s
p99
0.004672
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.052809
s
SD
0.017744
s
p50
0.054045
s
p95
0.061362
s
p99
0.062756
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.001649
s
SD
0
s
p50
0.001649
s
p95
0.001649
s
p99
0.001649
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
0.004633
s
SD
0
s
p50
0.004633
s
p95
0.004633
s
p99
0.004633
s
Authored DB connection use time
Time spent holding authored database connections.
mean
0.001484
s
SD
0.000434
s
p50
0.001351
s
p95
0.001854
s
p99
0.001904
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.000897
s
SD
0.000428
s
p50
0.000747
s
p95
0.0011
s
p99
0.001202
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000314
s
SD
2.7e-05
s
p50
0.000313
s
p95
0.000336
s
p99
0.000368
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
76.50
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
367
B/s
mean
241.04
KiB/s
SD
877.16
KiB/s
p95
2.32
MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
49
B/s
mean
6.63
KiB/s
SD
18.28
KiB/s
p95
50.83
KiB/s
Total bytes received
Total bytes received on primary network interface
count
70.62
MiB
mean
241.04
KiB/s
Total bytes sent
Total bytes sent on primary network interface
count
1.94
MiB
mean
6.63
KiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 99.2%
Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
Detected
Warning 100% of hosts overloaded (load5/ncpus > 1.0)
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
68.57
%
mean
87.36
%
SD
13.29
%
p95
99.15
%
System
10.86
%
CPU percentiles
CPU usage percentiles
p50
97.31
%
p95
99.15
%
p99
99.20
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
1
hosts
mean time
0.63
s
Memory used percentage
Percentage of memory used
p5
13.12
%
mean
15.83
%
SD
1.99
%
p95
17.78
%
Memory available percentage
Percentage of available memory
p5
82.22
%
mean
84.17
%
SD
1.99
%
p95
86.88
%
Max host memory used
Maximum memory usage percentage across all hosts
max
15.83
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00
%
Memory growth rate
Rate of memory growth over time
growth
-182.90
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.30
MiB/s
Disk write throughput
Disk write throughput in MiB/s
3.80
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/etc/nomad.d/client-meta.json0/1
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
4.31
5 min
2.57
15 min
1.10
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
100.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
21.07
%
mean
57.7955
%
SD
27.8662
%
p95
89.41
%
60 second average
56.3268
%
300 second average
33.1297
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
1.05
%
mean
5.2642
%
SD
4.2094
%
p95
13.65
%
60 second average
5.0410
%
300 second average
3.0306
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0.04
%
mean
1.1116
%
SD
1.3696
%
p95
4.35
%
60 second average
0.9648
%
300 second average
0.7348
%
Holochain process CPU usage
CPU usage by Holochain process
p5
45
%
mean
55.66
%
SD
9.92
%
p95
70.27
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
1.89
KiB
mean
317.27
KiB
SD
120.51
KiB
p95
396.33
KiB
Holochain process threads
Number of threads in Holochain process
p5
1
threads
mean
32.91
threads
SD
16.86
threads
p95
35
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
7
file descriptors
mean
86.74
file descriptors
SD
44.59
file descriptors
p95
94
file descriptors
Write Validated must_get_agent_activity
↑ Back to index
A scenario where write agents create entries in batches of 10, while must_get_agent_activity agents each pick a random write agent and repeatedly attempt to create an entry that references the chain top of their latest batch. This reference means that the entry's validation function needs to make a must_get_agent_activity call.
The purpose of this scenario is to measure the time it takes for published agent activity data to be gossiped among authorities and become available to peers that query it via must_get_agent_activity.
This test is similar to Mixed-Arc must_get_agent_activity, but all agents are full-arc.
Started
Thu, 02 Jul 2026 02:55:14 UTC
Peer count
30
Peer count at end
27
Behaviours
-
must_get_agent_activity(1 agent) -
write(1 agent)
Holochain version
0.7.0-dev.31
Wind Tunnel version
0.7.0
Run ID
write_validated_must_get_agent_activity_28558451397_1
Highest observed
action_seqThe maximum chain length observed per
write agent, aggregated across all reading agents. The reading dimension is collapsed by taking the maximum observed value for each write agent (any successful read counts as propagation). This reflects the DHT’s ability to propagate agent activity ops and make them available to querying peers.Mean max (across write agents)
Mean of the highest chain head value observed per write agent. Each write agent contributes its maximum — the value seen by any reader.
984
Max
Highest chain head value observed for any single write agent.
1776
Write agents observed
4
Chain batch delay timing
The time between a
write agent's creation of a batch and a must_get_agent_activity agent's successful discovery of the batch and creation/self-validation of a new entry that depends on it.Mean
mean
170.885125
s
SD
46.137152
s
Max
Highest per-partition mean latency
max mean
1028.360686
s
Min
Lowest per-partition mean latency
min mean
8.908427
s
Chain batch delay rate
The rate at which a
must_get_agent_activity agent was able to discover batches and create/self-validate new entries that depend on them.Mean
mean
2.5
/s
Max
Highest per-partition mean rate
max
6.53
/s
Min
Lowest per-partition mean rate
min
0
/s
create_validated_sample_entry zome call timingThe time taken to complete the zome function call that creates the entry that depends on a
write agent's source chain.Mean
mean
0.364154
s
SD
0.285455
s
Max
Highest per-partition mean latency
max mean
1.853945
s
Min
Lowest per-partition mean latency
min mean
0.068942
s
Retrieval errors
A running accumulation of the errors encountered by an agent when attempting to self-validate actions that depend on
must_get_agent_activity calls.Total
3413
Agents affected
16 / 16
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
213
Max
Highest count in any single partition
801
Min
Lowest count in any single partition
11
Final error count
The total number of all types of error accumulated over the run by all agents.
64411
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.016376
s
SD
0.143611
s
p50
0.004336
s
p95
0.256957
s
p99
1.012715
s
Zome call duration: 'validated_must_get_agent_activity::announce_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.055331
s
SD
0.031845
s
p50
0.041846
s
p95
0.133768
s
p99
0.133768
s
Zome call duration: 'validated_must_get_agent_activity::create_sample_entries_batch'
Duration of this zome call as measured by Holochain internally.
mean
0.305688
s
SD
0.22751
s
p50
0.207014
s
p95
0.891198
s
p99
1.182746
s
Zome call duration: 'validated_must_get_agent_activity::create_validated_sample_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.031193
s
SD
0.012102
s
p50
0.021866
s
p95
0.049316
s
p99
0.063772
s
Zome call duration: 'validated_must_get_agent_activity::get_chain_top_for_batch'
Duration of this zome call as measured by Holochain internally.
mean
0.037858
s
SD
0.034579
s
p50
0.028703
s
p95
0.158006
s
p99
0.158006
s
Zome call duration: 'validated_must_get_agent_activity::get_random_agent_with_write_behaviour'
Duration of this zome call as measured by Holochain internally.
mean
0.02119
s
SD
0.242253
s
p50
0.059155
s
p95
1.051358
s
p99
1.051358
s
WASM call duration: 'validated_must_get_agent_activity::announce_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.033003
s
SD
0.025189
s
p50
0.024629
s
p95
0.094164
s
p99
0.094164
s
WASM call duration: 'validated_must_get_agent_activity::create_sample_entries_batch'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.279267
s
SD
0.214379
s
p50
0.188528
s
p95
0.826898
s
p99
1.105071
s
WASM call duration: 'validated_must_get_agent_activity::create_validated_sample_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.016838
s
SD
0.006751
s
p50
0.01155
s
p95
0.027959
s
p99
0.03595
s
WASM call duration: 'validated_must_get_agent_activity::get_chain_top_for_batch'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.014249
s
SD
0.017728
s
p50
0.013259
s
p95
0.042711
s
p99
0.140562
s
WASM call duration: 'validated_must_get_agent_activity::get_random_agent_with_write_behaviour'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.014747
s
SD
0.243137
s
p50
0.029515
s
p95
1.0318
s
p99
1.0318
s
WASM call duration: 'validated_must_get_agent_activity_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001038
s
SD
0.000377
s
p50
0.001077
s
p95
0.001719
s
p99
0.002535
s
WASM call duration: 'validated_must_get_agent_activity_integrity::validate'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.009483
s
SD
0.008071
s
p50
0.008752
s
p95
0.026636
s
p99
0.04752
s
Host function call duration: '__hc__agent_info_1'
Duration of this host function call invoked from within WASM.
mean
3.2e-05
s
SD
1.3e-05
s
p50
2.7e-05
s
p95
7.1e-05
s
p99
7.6e-05
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000585
s
SD
0.000256
s
p50
0.000417
s
p95
0.000872
s
p99
0.001638
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.000654
s
SD
0.000495
s
p50
0.000384
s
p95
0.002241
s
p99
0.002241
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.001262
s
SD
0.186989
s
p50
0.000552
s
p95
0.355082
s
p99
1.012752
s
Host function call duration: '__hc__must_get_agent_activity_1'
Duration of this host function call invoked from within WASM.
mean
0.023177
s
SD
0.022144
s
p50
0.018963
s
p95
0.074045
s
p99
0.11557
s
Host function call duration: '__hc__query_1'
Duration of this host function call invoked from within WASM.
mean
0.009995
s
SD
0.006696
s
p50
0.006537
s
p95
0.020297
s
p99
0.034224
s
Host function call duration: '__hc__random_bytes_1'
Duration of this host function call invoked from within WASM.
mean
1.6e-05
s
SD
5e-06
s
p50
1.3e-05
s
p95
2.9e-05
s
p99
2.9e-05
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.006056
s
SD
0.013715
s
p50
0.01008
s
p95
0.038424
s
p99
0.089709
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.018794
s
SD
0.014491
s
p50
0.013522
s
p95
0.043796
s
p99
0.084731
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
47.76
s
mean
454.76
s
SD
261.12
s
p95
858.49
s
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
24.643111
s
SD
17.53107
s
p50
14.97493
s
p95
54.476421
s
p99
71.868658
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
1.669356
s
SD
7.465048
s
p50
3.410062
s
p95
20.230554
s
p99
39.810471
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0.001319
s
SD
0.00097
s
p50
0.001121
s
p95
0.00269
s
p99
0.00535
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.02179
s
SD
0.05314
s
p50
0.022759
s
p95
0.15845
s
p99
0.292367
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.11737
s
SD
0.090471
s
p50
0.057126
s
p95
0.277902
s
p99
0.381723
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.291703
s
SD
3.729072
s
p50
0.31958
s
p95
10.016813
s
p99
19.49932
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
20.992026
s
SD
25.794825
s
p50
16.889318
s
p95
80.411473
s
p99
111.317237
s
Authored DB connection use time
Time spent holding authored database connections.
mean
0.000427
s
SD
0.000399
s
p50
0.000255
s
p95
0.000986
s
p99
0.002621
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.001522
s
SD
0.006373
s
p50
0.000342
s
p95
0.011726
s
p99
0.037112
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000363
s
SD
0.000162
s
p50
0.000334
s
p95
0.000569
s
p99
0.00114
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
0.676737
s
SD
28.154902
s
p50
0.069132
s
p95
60.001845
s
p99
60.002439
s
P2P request roundtrip: 'get_links'
Time spent sending a get_links request and awaiting its response.
mean
0.096506
s
SD
0.302851
s
p50
0.251579
s
p95
1.010783
s
p99
1.010783
s
P2P request roundtrip: 'must_get_agent_activity'
Time spent sending a must_get_agent_activity request and awaiting its response.
mean
0.113342
s
SD
0.006099
s
p50
0.109088
s
p95
0.118375
s
p99
0.126146
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.135501
s
SD
0.043115
s
p50
0.128229
s
p95
0.229085
s
p99
0.271327
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
30.40
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
1.20
KiB/s
mean
104.93
KiB/s
SD
249.37
KiB/s
p95
163.21
KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
303
B/s
mean
81.72
KiB/s
SD
69.13
KiB/s
p95
208.76
KiB/s
Total bytes received
Total bytes received on primary network interface
count
2.44
GiB
mean
2.72
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
1.90
GiB
mean
2.12
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 97.8%
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 217.15 MiB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
Detected
Warning 19% of hosts overloaded (load5/ncpus > 1.0)
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
1.16
%
mean
38.14
%
SD
30.17
%
p95
95.01
%
System
7.74
%
CPU percentiles
CPU usage percentiles
p50
26.20
%
p95
95.01
%
p99
97.80
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
8
hosts
mean time
0.51
s
Memory used percentage
Percentage of memory used
p5
7.37
%
mean
12.89
%
SD
4.93
%
p95
20.19
%
Memory available percentage
Percentage of available memory
p5
79.81
%
mean
87.11
%
SD
4.93
%
p95
92.63
%
Max host memory used
Maximum memory usage percentage across all hosts
max
28.60
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00
%
Memory growth rate
Rate of memory growth over time
growth
217.15
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.01
MiB/s
Disk write throughput
Disk write throughput in MiB/s
151.55
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/0/8
hosts
Mount Point
/efi-boot0/3
hosts
Mount Point
/etc/hostname0/5
hosts
Mount Point
/etc/hosts0/5
hosts
Mount Point
/etc/nomad.d/client-meta.json0/5
hosts
Mount Point
/etc/resolv.conf0/5
hosts
Mount Point
/nix/store0/8
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
2.55
5 min
2.32
15 min
2.10
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
18.52
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
12.9263
%
SD
22.1601
%
p95
62.8
%
60 second average
12.8165
%
300 second average
12.4908
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
6.2253
%
SD
13.1679
%
p95
41.4
%
60 second average
5.9275
%
300 second average
4.6331
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
5.0905
%
SD
11.6246
%
p95
36.47
%
60 second average
4.8596
%
300 second average
3.7989
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.3
%
mean
31.42
%
SD
27.66
%
p95
93.38
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
231.32
KiB
mean
500.39
KiB
SD
140.47
KiB
p95
682.73
KiB
Holochain process threads
Number of threads in Holochain process
p5
25
threads
mean
44.85
threads
SD
13.27
threads
p95
58
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
61
file descriptors
mean
96.1
file descriptors
SD
23.23
file descriptors
p95
114
file descriptors
Zero-Arc Create and Read
↑ Back to indexA zero-arc/full-arc mixed scenario with two types of zero-arc nodes -- ones that create data and ones that read data -- as well as full arc nodes to "relay" the data. The scenario has three roles:
zero_write: A zero-arc conductor that just creates entries with a timestamp field. Those entries are linked to a known base hash so thatzero_readnodes can retrieve them.zero_read: A zero-arc conductor that reads the entries created by the zero-arc node(s) and records the time lag between when the entry had been created and when it was first discovered.full: A full-arc conductor that is just here to serve entries to zero arc nodes.
Started
Thu, 02 Jul 2026 04:13:30 UTC
Peer count
30
Peer count at end
23
Behaviours
-
full(1 agent) -
zero_read(1 agent) -
zero_write(1 agent)
Holochain version
0.7.0-dev.31
Wind Tunnel version
0.7.0
Run ID
zero_arc_create_and_read_28558451397_1
Create rate
The number of timed entries created by the zero-arc write node(s) per second.
Mean
mean
192.81
/s
Max
Highest per-partition mean rate
max
273
/s
Min
Lowest per-partition mean rate
min
37.99
/s
Fetch lag timing
For each entry, the time lag between when it was created and when a zero-arc read node fetched it from the network via the
get_timed_entries_network zome function.Mean
mean
461.608426
s
SD
58.286095
s
Max
Highest per-partition mean latency
max mean
1212.559472
s
Min
Lowest per-partition mean latency
min mean
81.975175
s
Fetch lag rate
The number of entries per second fetched by zero-arc read nodes.
Mean
mean
0.89
/s
Max
Highest per-partition mean rate
max
6
/s
Min
Lowest per-partition mean rate
min
0
/s
Delivery ratio
The proportion of created entries successfully fetched by all zero-arc read nodes.
0.0007
Entry created count
Total number of entries created by zero-arc write nodes, by agent.
Total
163771
Agents affected
10 / 10
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
16377
Max
Highest count in any single partition
23990
Min
Lowest count in any single partition
3371
Received count
Total number of entries fetched by zero-arc read nodes, by agent.
Total
1041
Agents affected
9 / 9
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
116
Max
Highest count in any single partition
425
Min
Lowest count in any single partition
3
Open connections
The number of currently open connections to other conductors.
full-arc
p5
0
mean
14.12
SD
10.37
p95
22
zero-arc
p5
7
mean
13.01
SD
3.66
p95
18
Error count
The number of errors accumulated across all nodes.
239
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.01273
s
SD
0.152484
s
p50
0.229363
s
p95
0.545766
s
p99
0.616927
s
WASM usage: 'timed::created_timed_entry'
Metered usage count of the WASM ribosome for this zome function.
total
222061420
over 874.722526627s
mean rate
1.12803097488e+09
/s
std
1.74868158877e+09
/s
p5
7.99385602e+06
/s
p95
5.1008140123e+09
/s
peak
5.7034467074e+09
/s
WASM usage: 'timed::get_timed_entries_network'
Metered usage count of the WASM ribosome for this zome function.
total
28933758110
over 1898.918023213s
mean rate
7.21268008e+08
/s
std
1.62613185818e+09
/s
p5
163639.32
/s
p95
4.59410942313e+09
/s
peak
5.95351309124e+09
/s
WASM usage: 'timed_integrity::entry_defs'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 37475.087510815s
Not enough time data to show a trend.
Zome call duration: 'timed::created_timed_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.01219
s
SD
0.004926
s
p50
0.011343
s
p95
0.024137
s
p99
0.024837
s
Zome call duration: 'timed::get_timed_entries_network'
Duration of this zome call as measured by Holochain internally.
mean
0.62214
s
SD
0.976235
s
p50
0.631865
s
p95
2.891228
s
p99
4.472531
s
WASM call duration: 'timed::created_timed_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.009596
s
SD
0.003863
s
p50
0.008857
s
p95
0.019326
s
p99
0.0199
s
WASM call duration: 'timed::get_timed_entries_network'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.617917
s
SD
0.97555
s
p50
0.627
s
p95
2.884954
s
p99
4.466187
s
WASM call duration: 'timed_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001106
s
SD
0.000647
s
p50
0.001051
s
p95
0.002598
s
p99
0.003499
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000347
s
SD
0.000139
s
p50
0.000329
s
p95
0.000708
s
p99
0.000727
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.000378
s
SD
0.000148
s
p50
0.000354
s
p95
0.000767
s
p99
0.000789
s
Host function call duration: '__hc__get_1'
Duration of this host function call invoked from within WASM.
mean
0.012105
s
SD
0.162806
s
p50
0.259079
s
p95
0.57728
s
p99
0.654119
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.10823
s
SD
0.07047
s
p50
0.13383
s
p95
0.282716
s
p99
0.295287
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.003805
s
SD
0.001594
s
p50
0.004562
s
p95
0.008557
s
p99
0.00867
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.002407
s
SD
0.009137
s
p50
0.015709
s
p95
0.020835
s
p99
0.039033
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
46.99
s
mean
448.67
s
SD
258.09
s
p95
848.21
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 37475.087497705s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
94.372109
s
SD
50.791576
s
p50
0
s
p95
57.947705
s
p99
451.666425
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.049907
s
SD
0.029586
s
p50
0.001187
s
p95
0.061044
s
p99
0.089715
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0.001625
s
SD
0.001174
s
p50
0.001275
s
p95
0.004357
s
p99
0.005046
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.005071
s
SD
0.177271
s
p50
0.001907
s
p95
0.111641
s
p99
0.982568
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
1.469721
s
SD
2.776207
s
p50
0.014148
s
p95
6.828302
s
p99
14.403356
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.229267
s
SD
0.220936
s
p50
0.004108
s
p95
0.279577
s
p99
1.284913
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
2.464907
s
SD
1.319812
s
p50
0.003671
s
p95
4.496571
s
p99
5.123743
s
Authored DB connection use time
Time spent holding authored database connections.
mean
0.002283
s
SD
0.001056
s
p50
0.000221
s
p95
0.003114
s
p99
0.00403
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.000902
s
SD
0.003574
s
p50
0.000193
s
p95
0.008042
s
p99
0.021015
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.00028
s
SD
0.000102
s
p50
0.000275
s
p95
0.00047
s
p99
0.000587
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
0.247302
s
SD
29.785538
s
p50
0.17338
s
p95
60.002397
s
p99
60.003208
s
P2P request roundtrip: 'get_links'
Time spent sending a get_links request and awaiting its response.
mean
0.092732
s
SD
0.056493
s
p50
0.081922
s
p95
0.205704
s
p99
0.209153
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.081911
s
SD
0.021444
s
p50
0.082544
s
p95
0.111166
s
p99
0.115978
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
20.08
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
568
B/s
mean
170.73
KiB/s
SD
597.35
KiB/s
p95
1010.67
KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
182
B/s
mean
145.60
KiB/s
SD
443.36
KiB/s
p95
1.04
MiB/s
Total bytes received
Total bytes received on primary network interface
count
3.67
GiB
mean
4.13
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
2.89
GiB
mean
3.25
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
NotDetected
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 451.33 MiB/s
Disk full anomaly
Disk full anomaly was detected
Detected
Critical Disk /etc/resolv.conf: 1/7 hosts above 90% capacity
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected
Critical Heavy swap usage (23.8% swap used)
System overload anomaly
System overload anomaly was detected
Detected
Warning 12% of hosts overloaded (load5/ncpus > 1.0)
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
0.64
%
mean
27.99
%
SD
25.23
%
p95
72.02
%
System
7.91
%
CPU percentiles
CPU usage percentiles
p50
26.84
%
p95
72.02
%
p99
81.55
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
7
hosts
mean time
0.06
s
Memory used percentage
Percentage of memory used
p5
6.28
%
mean
15.02
%
SD
14.16
%
p95
38.65
%
Memory available percentage
Percentage of available memory
p5
61.35
%
mean
84.98
%
SD
14.16
%
p95
93.72
%
Max host memory used
Maximum memory usage percentage across all hosts
max
77.67
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
23.80
%
Memory growth rate
Rate of memory growth over time
growth
451.33
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.25
MiB/s
Disk write throughput
Disk write throughput in MiB/s
120.50
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/0/7
hosts
Mount Point
/efi-boot0/4
hosts
Mount Point
/etc/hostname1/7
hosts
Mount Point
/etc/hosts1/7
hosts
Mount Point
/etc/nomad.d/client-meta.json0/3
hosts
Mount Point
/etc/resolv.conf1/7
hosts
Mount Point
/nix/store0/7
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
1.89
5 min
1.72
15 min
1.48
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
12.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
7.058
%
SD
11.7768
%
p95
36.26
%
60 second average
6.8619
%
300 second average
6.2311
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
6.0558
%
SD
12.5885
%
p95
35.07
%
60 second average
6.1564
%
300 second average
6.4372
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
4.2985
%
SD
10.5229
%
p95
21.14
%
60 second average
4.4937
%
300 second average
5.1607
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.32
%
mean
21.75
%
SD
20.7
%
p95
61.71
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
208.13
KiB
mean
300.08
KiB
SD
92.65
KiB
p95
478.80
KiB
Holochain process threads
Number of threads in Holochain process
p5
25
threads
mean
34.78
threads
SD
7.58
threads
p95
49
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
60
file descriptors
mean
80.32
file descriptors
SD
14.83
file descriptors
p95
103
file descriptors
Zero-arc Create Data (Validated)
↑ Back to indexA zero-arc/full-arc mixed scenario where zero-arc nodes create data that gets validated and full-arc nodes read the data. The scenario has two roles:
zero: A zero-arc conductor that just creates entries with a timestamp field. Those entries are linked to a known base hash so that full-arc nodes can retrieve them.full: A full-arc conductor that reads the entries created by the zero-arc node(s) and records the time lag between when the entry had been created and when it was first discovered.
Started
Thu, 02 Jul 2026 03:57:47 UTC
Peer count
30
Peer count at end
24
Behaviours
-
full(1 agent) -
zero(1 agent)
Holochain version
0.7.0-dev.31
Wind Tunnel version
0.7.0
Run ID
zero_arc_create_data_validated--high_zero_28558451397_1
Create rate
The number of timed entries created by the zero-arc node(s) per second.
Mean
mean
64.99
/s
Max
Highest per-partition mean rate
max
103.72
/s
Min
Lowest per-partition mean rate
min
33.98
/s
Sync lag timing
For each entry, the time lag between when it was created and when the full-arc node could read it via the
get_timed_local_entries zome function.Mean
mean
294.120299
s
SD
75.457037
s
Max
Highest per-partition mean latency
max mean
363.363379
s
Min
Lowest per-partition mean latency
min mean
224.877219
s
Sync lag rate
The number of entries per second received by full nodes.
Mean
mean
124.75
/s
Max
Highest per-partition mean rate
max
247.5
/s
Min
Lowest per-partition mean rate
min
2
/s
Delivery ratio
The proportion of created entries successfully received by all full-arc nodes.
0.0101
Entry created count
Total number of entries created by zero-arc nodes, by agent.
Total
126449
Agents affected
22 / 22
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
5748
Max
Highest count in any single partition
9274
Min
Lowest count in any single partition
3009
Received count
Total number of entries received by full-arc nodes, by agent.
Total
2557
Agents affected
2 / 2
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
1279
Max
Highest count in any single partition
1288
Min
Lowest count in any single partition
1269
Open connections
The number of currently open connections to other conductors.
full-arc
p5
14
mean
17.55
SD
3.77
p95
23
zero-arc
p5
2
mean
8.45
SD
4.56
p95
17
Error count
The number of errors accumulated across all nodes.
0
Holochain MetricsClick to toggle visibility.
Cascade duration
Time taken to execute a cascade (get) query inside Holochain.
mean
0.002405
s
SD
0.005812
s
p50
0.007558
s
p95
0.019976
s
p99
0.025639
s
Cascade fetch errors
Count of network fetch errors during cascade calls.
total
0
over 0s
Not enough time data to show a trend.
WASM usage: 'timed_and_validated::create_timed_entry'
Metered usage count of the WASM ribosome for this zome function.
total
253920228
over 1881.115584862s
mean rate
4.0552199133e+08
/s
std
4.9481464613e+08
/s
p5
285461.02
/s
p95
1.53482934925e+09
/s
peak
1.90438341389e+09
/s
WASM usage: 'timed_and_validated::get_timed_entries_local'
Metered usage count of the WASM ribosome for this zome function.
total
23781968981
over 891.098319342s
mean rate
1.64587243358e+09
/s
std
1.09678295258e+09
/s
p5
4.92096993e+06
/s
p95
2.62418943581e+09
/s
peak
2.78587437504e+09
/s
WASM usage: 'timed_and_validated_integrity::entry_defs'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 33084.067836006s
Not enough time data to show a trend.
WASM usage: 'timed_and_validated_integrity::validate'
Metered usage count of the WASM ribosome for this zome function.
total
0
over 33084.067836006s
Not enough time data to show a trend.
Zome call duration: 'timed_and_validated::create_timed_entry'
Duration of this zome call as measured by Holochain internally.
mean
0.022296
s
SD
0.010228
s
p50
0.01967
s
p95
0.049042
s
p99
0.054115
s
Zome call duration: 'timed_and_validated::get_timed_entries_local'
Duration of this zome call as measured by Holochain internally.
mean
0.069739
s
SD
0.035764
s
p50
0.063453
s
p95
0.124597
s
p99
0.135084
s
WASM call duration: 'timed_and_validated::create_timed_entry'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.01667
s
SD
0.008552
s
p50
0.014649
s
p95
0.040981
s
p99
0.042003
s
WASM call duration: 'timed_and_validated::get_timed_entries_local'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.061352
s
SD
0.034383
s
p50
0.0544
s
p95
0.116825
s
p99
0.127196
s
WASM call duration: 'timed_and_validated_integrity::entry_defs'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.001123
s
SD
0.000658
s
p50
0.000947
s
p95
0.003165
s
p99
0.003292
s
WASM call duration: 'timed_and_validated_integrity::validate'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.008917
s
SD
0.005189
s
p50
0.007617
s
p95
0.020148
s
p99
0.024166
s
Host function call duration: '__hc__create_1'
Duration of this host function call invoked from within WASM.
mean
0.000348
s
SD
0.000184
s
p50
0.0003
s
p95
0.00086
s
p99
0.000999
s
Host function call duration: '__hc__create_link_1'
Duration of this host function call invoked from within WASM.
mean
0.000392
s
SD
0.000198
s
p50
0.000343
s
p95
0.000934
s
p99
0.001096
s
Host function call duration: '__hc__get_1'
Duration of this host function call invoked from within WASM.
mean
0.001084
s
SD
6.5e-05
s
p50
0.001064
s
p95
0.001123
s
p99
0.001145
s
Host function call duration: '__hc__get_links_1'
Duration of this host function call invoked from within WASM.
mean
0.002667
s
SD
0.001396
s
p50
0.00251
s
p95
0.004791
s
p99
0.005233
s
Host function call duration: '__hc__must_get_valid_record_1'
Duration of this host function call invoked from within WASM.
mean
0.010546
s
SD
0.005628
s
p50
0.008209
s
p95
0.02053
s
p99
0.02595
s
Host function call duration: '__hc__zome_info_1'
Duration of this host function call invoked from within WASM.
mean
0.005555
s
SD
0.003732
s
p50
0.005374
s
p95
0.015115
s
p99
0.017816
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.005729
s
SD
0.006545
s
p50
0.004773
s
p95
0.019101
s
p99
0.027679
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
47.01
s
mean
448.08
s
SD
257.73
s
p95
848.2
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 33084.06789574s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
221.10645
s
SD
54.198548
s
p50
0
s
p95
84.165117
s
p99
317.301955
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
6.136621
s
SD
2.598187
s
p50
0.001053
s
p95
9.61616
s
p99
13.182958
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0.001462
s
SD
0.001082
s
p50
0.001098
s
p95
0.003497
s
p99
0.004954
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.006557
s
SD
0.112346
s
p50
0.00114
s
p95
0.332498
s
p99
0.581925
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.432469
s
SD
0.41799
s
p50
0.27446
s
p95
1.309005
s
p99
2.057768
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.664872
s
SD
0.26738
s
p50
0.005648
s
p95
0.340171
s
p99
1.786891
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
4.130898
s
SD
4.899808
s
p50
0.005118
s
p95
0.184434
s
p99
36.627124
s
Authored DB connection use time
Time spent holding authored database connections.
mean
0.001917
s
SD
0.001183
s
p50
0.000904
s
p95
0.003673
s
p99
0.004419
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
0.002474
s
SD
0.005093
s
p50
0.000296
s
p95
0.017461
s
p99
0.018583
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000285
s
SD
0.000142
s
p50
0.000255
s
p95
0.00069
s
p99
0.000803
s
P2P request roundtrip: 'get'
Time spent sending a get request and awaiting its response.
mean
0.006364
s
SD
0.001885
s
p50
0.004522
s
p95
0.008295
s
p99
0.008793
s
P2P request roundtrip: 'send_validation_receipts'
Time spent sending a send_validation_receipts request and awaiting its response.
mean
0.140279
s
SD
0.063758
s
p50
0.210613
s
p95
0.328938
s
p99
0.35585
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
21.15
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
561
B/s
mean
92.26
KiB/s
SD
497.65
KiB/s
p95
234.48
KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
353
B/s
mean
49.72
KiB/s
SD
155.17
KiB/s
p95
391.51
KiB/s
Total bytes received
Total bytes received on primary network interface
count
1.98
GiB
mean
2.20
MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
1.07
GiB
mean
1.19
MiB/s
CPU spike anomaly
CPU spike anomaly was detected
NotDetected
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 381.13 MiB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
Detected
Warning 8% of hosts overloaded (load5/ncpus > 1.0)
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
6.07
%
mean
28.19
%
SD
22.09
%
p95
80.99
%
System
7.04
%
CPU percentiles
CPU usage percentiles
p50
25.66
%
p95
80.99
%
p99
86.55
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
5
hosts
mean time
0.26
s
Memory used percentage
Percentage of memory used
p5
7.44
%
mean
15.93
%
SD
13.54
%
p95
28.12
%
Memory available percentage
Percentage of available memory
p5
71.88
%
mean
84.07
%
SD
13.54
%
p95
92.56
%
Max host memory used
Maximum memory usage percentage across all hosts
max
79.09
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00
%
Memory growth rate
Rate of memory growth over time
growth
381.13
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.08
MiB/s
Disk write throughput
Disk write throughput in MiB/s
96.37
MiB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/0/5
hosts
Mount Point
/efi-boot0/4
hosts
Mount Point
/etc/hostname0/6
hosts
Mount Point
/etc/hosts0/6
hosts
Mount Point
/etc/nomad.d/client-meta.json0/5
hosts
Mount Point
/etc/resolv.conf0/6
hosts
Mount Point
/nix/store0/5
hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
1.66
5 min
1.62
15 min
1.43
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
8.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
5.7232
%
SD
12.9354
%
p95
43.54
%
60 second average
5.5870
%
300 second average
5.2124
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
7.8153
%
SD
13.4289
%
p95
36.32
%
60 second average
7.8634
%
300 second average
7.8042
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
7.5105
%
SD
13.2625
%
p95
35.77
%
60 second average
7.5363
%
300 second average
7.3671
%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.27
%
mean
23.9
%
SD
17.72
%
p95
54.74
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
273.20
KiB
mean
448.16
KiB
SD
124.95
KiB
p95
647.79
KiB
Holochain process threads
Number of threads in Holochain process
p5
10
threads
mean
34.4
threads
SD
7.58
threads
p95
44
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
28
file descriptors
mean
79.8
file descriptors
SD
18.95
file descriptors
p95
96
file descriptors
Zome Call, Single Value
↑ Back to indexCalls a zome function that returns a fixed value. This tests the maximum performance of zome calls when the zome function is not calling into Holochain other otherwise doing any work.
Started
Thu, 02 Jul 2026 01:31:33 UTC
Peer count
1
Behaviours
-
default(1 agent)
Holochain version
0.7.0-dev.31
Wind Tunnel version
0.7.0
Run ID
zome_call_single_value_28558451397_1
Call timing
The time taken for the call to complete.
mean
0.004302
s
SD
0.001087
s
p50
0.003986
s
p95
0.006341
s
p99
0.008454
s
Call rate
The number of zome calls completed per second.
mean
2321.96
/s
Error count
The number of errors accumulated during the run.
0
Holochain MetricsClick to toggle visibility.
WASM usage: 'return_single_value::get_value'
Metered usage count of the WASM ribosome for this zome function.
total
259941964
over 280.001734831s
mean rate
928358.42
/s
std
22653.51
/s
p5
867960.56
/s
p95
952316.68
/s
peak
960313.45
/s
Zome call duration: 'return_single_value::get_value'
Duration of this zome call as measured by Holochain internally.
mean
0.002604
s
SD
0.0003
s
p50
0.002602
s
p95
0.003015
s
p99
0.004208
s
WASM call duration: 'return_single_value::get_value'
Duration of inner WASM calls for this function, excluding Holochain overhead.
mean
0.000906
s
SD
1e-06
s
p50
0.000906
s
p95
0.000907
s
p99
0.000908
s
Post-commit duration
Time spent executing post-commit workflows.
mean
0.018026
s
SD
0
s
p50
0.018026
s
p95
0.018026
s
p99
0.018026
s
Conductor uptime
Conductor uptime gauge. A drop in the trend indicates a restart during the run.
p5
17.76
s
mean
147.76
s
SD
83.67
s
p95
277.76
s
Integrated ops
Count of DHT ops integrated since the conductor started. Resets on restart.
total
0
over 280.001745268s
Not enough time data to show a trend.
Integration delay
Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
Validation attempts per op
Number of validation attempts required per op. Values consistently above 1 indicate retries.
mean
0
s
SD
0
s
p50
0
s
p95
0
s
p99
0
s
App validation workflow duration
Time spent running the app validation workflow.
mean
0.002083
s
SD
0
s
p50
0.002083
s
p95
0.002083
s
p99
0.002083
s
Countersigning workflow duration
Time spent running the countersigning workflow.
mean
0.001795
s
SD
0
s
p50
0.001795
s
p95
0.001795
s
p99
0.001795
s
Integrate DHT ops workflow duration
Time spent running the integration workflow.
mean
0.003397
s
SD
0
s
p50
0.003397
s
p95
0.003397
s
p99
0.003397
s
Publish DHT ops workflow duration
Time spent running the publish workflow.
mean
0.003103
s
SD
0.000334
s
p50
0.00322
s
p95
0.003684
s
p99
0.003684
s
System validation workflow duration
Time spent running the sys validation workflow.
mean
0.001285
s
SD
0
s
p50
0.001285
s
p95
0.001285
s
p99
0.001285
s
Validation receipt workflow duration
Time spent running the validation receipt workflow.
mean
0.00425
s
SD
0
s
p50
0.00425
s
p95
0.00425
s
p99
0.00425
s
Authored DB connection use time
Time spent holding authored database connections.
mean
9.8e-05
s
SD
0
s
p50
9.8e-05
s
p95
9.9e-05
s
p99
9.9e-05
s
Write transaction duration
Duration of exclusive write transactions across all databases.
mean
7.9e-05
s
SD
0.000276
s
p50
0.000632
s
p95
0.000632
s
p99
0.000632
s
Lair keystore: signing
Duration of Ed25519 signing requests to the Lair keystore.
mean
0.000532
s
SD
0
s
p50
0.000532
s
p95
0.000532
s
p99
0.000532
s
Host MetricsClick to toggle visibility.
User CPU usage
CPU usage by user space
17.76
%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5
951
B/s
mean
2.50
KiB/s
SD
1.54
KiB/s
p95
4.53
KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5
22
B/s
mean
425
B/s
SD
354
B/s
p95
981
B/s
Total bytes received
Total bytes received on primary network interface
count
726.37
KiB
mean
2.50
KiB/s
Total bytes sent
Total bytes sent on primary network interface
count
120.35
KiB
mean
425
B/s
CPU spike anomaly
CPU spike anomaly was detected
NotDetected
Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
NotDetected
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
55.84
%
mean
56.46
%
SD
0.62
%
p95
56.91
%
System
38.70
%
CPU percentiles
CPU usage percentiles
p50
56.50
%
p95
56.91
%
p99
59.17
%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
0
hosts
mean time
0.00
s
Memory used percentage
Percentage of memory used
p5
11.02
%
mean
11.33
%
SD
0.31
%
p95
11.67
%
Memory available percentage
Percentage of available memory
p5
88.33
%
mean
88.67
%
SD
0.31
%
p95
88.98
%
Max host memory used
Maximum memory usage percentage across all hosts
max
11.33
%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00
%
Memory growth rate
Rate of memory growth over time
growth
0.05
MiB/s
Disk read throughput
Disk read throughput in MiB/s
0.00
MiB/s
Disk write throughput
Disk write throughput in MiB/s
1.37
MiB/s
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
2.66
5 min
2.14
15 min
1.81
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
0.00
%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
8.76
%
mean
10.163
%
SD
1.0495
%
p95
11.21
%
60 second average
8.3840
%
300 second average
3.9340
%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
0.0057
%
SD
0.0225
%
p95
0.04
%
60 second average
0.0027
%
300 second average
0.0000
%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0
%
mean
0
%
SD
0
%
p95
0
%
60 second average
0.0000
%
300 second average
0.0000
%
Holochain process CPU usage
CPU usage by Holochain process
p5
21.45
%
mean
22.31
%
SD
0.97
%
p95
22.71
%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5
196.97
KiB
mean
200.34
KiB
SD
14.64
KiB
p95
205.66
KiB
Holochain process threads
Number of threads in Holochain process
p5
27
threads
mean
27
threads
SD
0
threads
p95
27
threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
67
file descriptors
mean
68.13
file descriptors
SD
2.29
file descriptors
p95
73
file descriptors