App Install
Measure app installation speed. This scenario has two roles:
minimal: Installs a simple app which implements initialisation callbacks but otherwise doesn't contain a lot of code.large: Installs a larger app that contains some dependencies and some generated data as padding to make the bundle larger.
These are intended to catch a compilation slowdown or issues with copying around large WASMs (e.g. accidental cloning or tracing). In each case, the behaviour will uninstall the app it installed so that it can re-install it on the next iteration.
Started
Wed, 18 Mar 2026 17:24:02 UTC
Peer count
1
Peer count at end
1
Behaviours
-
minimal(1 agent)
Holochain version
0.6.1-rc.3
Wind Tunnel version
0.6.0
Run ID
app_install_minimal_23256907830
First install time
The time from conductor startup to completion of the first app installation attempt.
0.865063753s
Install app timing
The time taken to install an app.
mean
1.069826s
(SD =
1.020513s)
p50
0.488824s
p95
3.198056s
p99
3.947885s
Host Metrics
User CPU usage
CPU usage by user space
26.98%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5 23.45 KiB/s
<
mean 782.27 KiB/s
(SD = 1.12 MiB/s)
<
p951.52 MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5 9.31 KiB/s
<
mean 423.16 KiB/s
(SD = 392.87 KiB/s)
<
p95945.96 KiB/s
Total bytes received
Total bytes received on primary network interface
count
229.18 MiB
mean
782.27 KiB/s
Total bytes sent
Total bytes sent on primary network interface
count
123.97 MiB
mean
423.16 KiB/s
CPU spike anomaly
CPU spike anomaly was detected
NotDetected
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 1171.88 MB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
NotDetected
Additional Host Metrics
Additional Host Metrics
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
10.2%
<
mean
35.17%
(SD =
20.86%)
<
p95
55.19%
System
8.19%
CPU percentiles
CPU usage percentiles
p50
49.22%
p95
55.19%
p99
78.47%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
0 hosts
mean time
0.00s
Memory used percentage
Percentage of memory used
p5
16.48%
<
mean
23.72%
(SD =
5.13%)
<
p95
29.78%
Memory available percentage
Percentage of available memory
p5
70.22%
<
mean
76.28%
(SD =
5.13%)
<
p95
83.52%
Max host memory used
Maximum memory usage percentage across all hosts
max
23.72%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00%
Memory growth rate
Rate of memory growth over time
growth
1171.88MB/s
Disk read throughput
Disk read throughput in MB/s
0.00MB/s
Disk write throughput
Disk write throughput in MB/s
5.51MB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/
0/1hosts
Mount Point
/nix/store
0/1hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
3.31
5 min
1.42
15 min
1.17
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
0.00%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0%
<
mean
6.2816%
(SD =
6.0102%)
<
p95
14.03%
60 second average
4.5552%
300 second average
1.6548%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
8.26%
<
mean
34.4532%
(SD =
8.5166%)
<
p95
42.13%
60 second average
28.4797%
300 second average
14.9626%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
7.59%
<
mean
24.22%
(SD =
9.2723%)
<
p95
37.84%
60 second average
21.1381%
300 second average
12.1674%
Holochain process CPU usage
CPU usage by Holochain process
p5
7.16%
<
mean
28.75%
(SD =
17.54%)
<
p95
50.76%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5 107.85 KiB
<
mean 391.86 KiB
(SD = 177.14 KiB)
<
p95737.85 KiB
Holochain process threads
Number of threads in Holochain process
p5
17 threads
<
mean
27.55 threads
(SD =
9.51 threads)
<
p95
38 threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
67 file descriptors
<
mean
261.77 file descriptors
(SD =
246.01 file descriptors)
<
p95
656 file descriptors
DHT Sync Lag
Measure lag time between an agent publishing data and other peers being able to see it. This scenario has two roles:
write: A simple job that just creates entries with a timestamp field. Those entries are linked to a known base hash. For each write, the metricws.custom.dht_sync_sent_countis incremented.record_lag: A job that repeatedly queries for links from the known base hash. It keeps track of records that it has seen and when a new record is found, and calculates the time difference between the timestamp of the new record and the current time. That time difference is then recorded as a custom metric calledwt.custom.dht_sync_lag.
After each behaviour loop the metric ws.custom.dht_sync_recv_count is incremented.
Started
Wed, 18 Mar 2026 17:26:07 UTC
Peer count
65
Peer count at end
4225
Behaviours
-
record_lag(1 agent) -
write(1 agent)
Holochain version
0.6.1-rc.3
Wind Tunnel version
0.6.0
Run ID
dht_sync_lag_23256907830
Create rate
The average number of created records per agent.
Mean
mean
45.99/s
Max
Highest per-partition mean rate
max
159.4/s
Min
Lowest per-partition mean rate
min
8.93/s
Sync lag timing
The average time between when a record was created and when it is first seen by an agent.
Mean
mean
2.8152112796750185e+31s
(SD =
8.051514868571862e+31s)
Max
Highest per-partition mean latency
max mean
1.0190747967684355e+32s
Min
Lowest per-partition mean latency
min mean
1.7414655420723564e+30s
Sync lag rate
The average number of created records discovered per agent.
Mean
mean
181.55/s
Max
Highest per-partition mean rate
max
507/s
Min
Lowest per-partition mean rate
min
40.85/s
Error count
The number of errors encountered during the scenario.
60
Host Metrics
User CPU usage
CPU usage by user space
37.49%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5 14.73 KiB/s
<
mean 790.36 KiB/s
(SD = 731.69 KiB/s)
<
p952.01 MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5 16.53 KiB/s
<
mean 759.87 KiB/s
(SD = 923.19 KiB/s)
<
p952.65 MiB/s
Total bytes received
Total bytes received on primary network interface
count
83.64 GiB
mean
44.38 MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
80.42 GiB
mean
42.67 MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 95.7%
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 1025.19 MB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected
Critical Heavy swap usage (39.9% swap used)
System overload anomaly
System overload anomaly was detected
Detected
Warning 24% of hosts overloaded (load5/ncpus > 1.0)
Additional Host Metrics
Additional Host Metrics
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
7.77%
<
mean
44.36%
(SD =
25.19%)
<
p95
88.36%
System
6.87%
CPU percentiles
CPU usage percentiles
p50
41.88%
p95
88.36%
p99
95.74%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
29 hosts
mean time
0.24s
Memory used percentage
Percentage of memory used
p5
8.38%
<
mean
20.63%
(SD =
13.09%)
<
p95
48.58%
Memory available percentage
Percentage of available memory
p5
51.42%
<
mean
79.37%
(SD =
13.09%)
<
p95
91.62%
Max host memory used
Maximum memory usage percentage across all hosts
max
75.39%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
39.85%
Memory growth rate
Rate of memory growth over time
growth
1025.19MB/s
Disk read throughput
Disk read throughput in MB/s
0.69MB/s
Disk write throughput
Disk write throughput in MB/s
743.02MB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/
0/26hosts
Mount Point
/boot
0/1hosts
Mount Point
/efi-boot
0/20hosts
Mount Point
/etc/hostname
0/35hosts
Mount Point
/etc/hosts
0/35hosts
Mount Point
/etc/resolv.conf
0/35hosts
Mount Point
/nix/store
0/26hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
3.98
5 min
3.56
15 min
2.61
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
24.19%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0%
<
mean
8.9283%
(SD =
13.4642%)
<
p95
39.74%
60 second average
8.8560%
300 second average
8.2680%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
<
mean
0.0001%
(SD =
0.0095%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
<
mean
0.0001%
(SD =
0.0095%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0%
<
mean
17.2858%
(SD =
16.7237%)
<
p95
49.5%
60 second average
16.8536%
300 second average
14.4623%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0%
<
mean
12.6326%
(SD =
14.6686%)
<
p95
45.67%
60 second average
12.2345%
300 second average
10.2266%
Holochain process CPU usage
CPU usage by Holochain process
p5
2.4%
<
mean
38.92%
(SD =
24.22%)
<
p95
81.2%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5 245.15 KiB
<
mean 494.19 KiB
(SD = 144.63 KiB)
<
p95692.83 KiB
Holochain process threads
Number of threads in Holochain process
p5
12 threads
<
mean
37.59 threads
(SD =
11.82 threads)
<
p95
52 threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
54 file descriptors
<
mean
88.05 file descriptors
(SD =
25.13 file descriptors)
<
p95
129 file descriptors
First Call
This scenario installs a simple app which implements the init callback, and measures the time elapsed between start of install and calling init. It will uninstall the app it installed so that it can re-install it on the next iteration. This is required to re-run the initialisation callback.
Started
Wed, 18 Mar 2026 17:02:41 UTC
Peer count
1
Peer count at end
1
Behaviours
-
default(1 agent)
Holochain version
0.6.1-rc.3
Wind Tunnel version
0.6.0
Run ID
first_call_23256907830
Zome call timing
The time elapsed between start of install and calling
init.
mean
0.013103s
(SD =
0.007417s)
p50
0.008919s
p95
0.025375s
p99
0.036672s
Host Metrics
User CPU usage
CPU usage by user space
12.68%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5 7.07 KiB/s
<
mean 75.65 KiB/s
(SD = 122.86 KiB/s)
<
p95329.52 KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5 8.14 KiB/s
<
mean 63.57 KiB/s
(SD = 130.80 KiB/s)
<
p95415.09 KiB/s
Total bytes received
Total bytes received on primary network interface
count
22.16 MiB
mean
75.65 KiB/s
Total bytes sent
Total bytes sent on primary network interface
count
18.63 MiB
mean
63.57 KiB/s
CPU spike anomaly
CPU spike anomaly was detected
NotDetected
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 178.77 MB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
NotDetected
Additional Host Metrics
Additional Host Metrics
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
10.14%
<
mean
16.67%
(SD =
9.01%)
<
p95
34.67%
System
3.99%
CPU percentiles
CPU usage percentiles
p50
13.15%
p95
34.67%
p99
48.34%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
0 hosts
mean time
0.00s
Memory used percentage
Percentage of memory used
p5
20.89%
<
mean
23.77%
(SD =
1.45%)
<
p95
24.85%
Memory available percentage
Percentage of available memory
p5
75.15%
<
mean
76.23%
(SD =
1.45%)
<
p95
79.11%
Max host memory used
Maximum memory usage percentage across all hosts
max
23.77%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00%
Memory growth rate
Rate of memory growth over time
growth
178.77MB/s
Disk read throughput
Disk read throughput in MB/s
0.00MB/s
Disk write throughput
Disk write throughput in MB/s
14.30MB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/
0/1hosts
Mount Point
/efi-boot
0/1hosts
Mount Point
/nix/store
0/1hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
2.95
5 min
2.17
15 min
2.11
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
0.00%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0.49%
<
mean
1.4419%
(SD =
1.1366%)
<
p95
3.02%
60 second average
1.1858%
300 second average
0.6045%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
24.95%
<
mean
44.3455%
(SD =
8.0601%)
<
p95
55.48%
60 second average
38.7258%
300 second average
28.0477%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
23.52%
<
mean
39.4923%
(SD =
7.6069%)
<
p95
49.96%
60 second average
34.6984%
300 second average
25.5281%
Holochain process CPU usage
CPU usage by Holochain process
p5
3.05%
<
mean
6.34%
(SD =
7.27%)
<
p95
18.64%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5 273.49 KiB
<
mean 447.99 KiB
(SD = 78.23 KiB)
<
p95491.43 KiB
Holochain process threads
Number of threads in Holochain process
p5
17 threads
<
mean
19.06 threads
(SD =
3.61 threads)
≤
p95
19 threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
53 file descriptors
<
mean
150.1 file descriptors
(SD =
138.61 file descriptors)
<
p95
435 file descriptors
Full-Arc Create (Validated) / Zero-Arc Read
A full-arc/zero-arc mixed scenario where full-arc nodes create data that gets validated and zero-arc nodes read the data. The scenario has two roles:
full: A full-arc conductor that creates entries with a timestamp field. Those entries get validated and then retrieved by zero-arc nodes.zero: A zero-arc conductor that reads the entries created by the full-arc node(s) and records the time lag between when the entry had been created and when it was first discovered.
Started
Wed, 18 Mar 2026 18:20:03 UTC
Peer count
65
Peer count at end
4033
Behaviours
-
full(1 agent) -
zero(1 agent)
Holochain version
0.6.1-rc.3
Wind Tunnel version
0.6.0
Run ID
full_arc_create_validated_zero_arc_read--high_zero_23256907830
Create rate
The number of timed entries created by the full-arc node(s) per second.
Mean
mean
27.31/s
Max
Highest per-partition mean rate
max
44.56/s
Min
Lowest per-partition mean rate
min
5.64/s
Fetch lag timing
For each entry, the time lag between when it was created by a full-arc node and when a zero-arc node fetched it from the network.
Mean
mean
68.510366s
(SD =
26.941268s)
Max
Highest per-partition mean latency
max mean
680.492043s
Min
Lowest per-partition mean latency
min mean
3.626516s
Fetch rate
The number of entries per second fetched by zero-arc nodes.
Mean
mean
9.23/s
Max
Highest per-partition mean rate
max
98/s
Min
Lowest per-partition mean rate
min
0/s
Open connections
The number of currently open connections to other conductors.
full-arc
p5
15
<
mean
56.69
(SD =
13.92)
<
p95
64
zero-arc
p5
0
<
mean
12.86
(SD =
10.85)
<
p95
34
Retrieval errors
Errors encountered by zero-arc nodes when attempting to retrieve entries created by full-arc nodes.
Total
827
Agents affected
61 / 61
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
14
Max
Highest count in any single partition
14
Min
Lowest count in any single partition
1
Error count
The number of errors accumulated across all nodes.
827
Local Signals
This scenario is for testing the speed and reliability of local signals. It uses a zome with a function that will emit 10,000 signals. The scenario measures how long it takes to send the signals and how many have been received by the time the zome call ends.
Started
Wed, 18 Mar 2026 17:04:43 UTC
Peer count
1
Peer count at end
1
Behaviours
-
default(1 agent)
Holochain version
0.6.1-rc.3
Wind Tunnel version
0.6.0
Run ID
local_signals_23256907830
Send timing
The time taken to emit a batch of 10,000 signals.
mean
1.221409s
(SD =
0.60353s)
p50
1.324455s
p95
2.251098s
p99
2.251098s
Receive timing
The time taken to receive the completed batch, to the nearest 250ms.
mean
28.021797s
(SD =
5.937778s)
p50
30.001029s
p95
30.001841s
p99
30.001841s
Success ratio
The proportion of signals successfully received in a batch.
min
0.8120999932289124
<
mean
0.8965
(SD =
0.047)
<
max
0.9700000286102296
Host Metrics
User CPU usage
CPU usage by user space
3.42%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5 197 B/s
<
mean 4.73 KiB/s
(SD = 4.99 KiB/s)
<
p9516.58 KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5 30 B/s
<
mean 8.83 KiB/s
(SD = 12.65 KiB/s)
<
p9548.13 KiB/s
Total bytes received
Total bytes received on primary network interface
count
1.39 MiB
mean
4.73 KiB/s
Total bytes sent
Total bytes sent on primary network interface
count
2.59 MiB
mean
8.83 KiB/s
CPU spike anomaly
CPU spike anomaly was detected
NotDetected
Memory leak anomaly
Memory leak anomaly was detected
NotDetected
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
NotDetected
Additional Host Metrics
Additional Host Metrics
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
1.03%
<
mean
6.68%
(SD =
9.63%)
<
p95
17.27%
System
3.26%
CPU percentiles
CPU usage percentiles
p50
2.78%
p95
17.27%
p99
52.46%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
0 hosts
mean time
0.00s
Memory used percentage
Percentage of memory used
p5
25.76%
<
mean
27.76%
(SD =
1.55%)
<
p95
29.04%
Memory available percentage
Percentage of available memory
p5
70.96%
<
mean
72.24%
(SD =
1.55%)
<
p95
74.24%
Max host memory used
Maximum memory usage percentage across all hosts
max
27.76%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00%
Memory growth rate
Rate of memory growth over time
growth
-331.32MB/s
Disk read throughput
Disk read throughput in MB/s
0.00MB/s
Disk write throughput
Disk write throughput in MB/s
0.98MB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/etc/hostname
0/1hosts
Mount Point
/etc/hosts
0/1hosts
Mount Point
/etc/resolv.conf
0/1hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
0.97
5 min
0.53
15 min
0.69
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
0.00%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0.22%
<
mean
1.7765%
(SD =
1.8617%)
<
p95
6.57%
60 second average
1.2758%
300 second average
0.5332%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0.75%
<
mean
2.1513%
(SD =
1.8117%)
<
p95
6%
60 second average
2.2758%
300 second average
2.1952%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0.72%
<
mean
2.0223%
(SD =
1.7231%)
<
p95
5.88%
60 second average
2.1406%
300 second average
2.0916%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.15%
<
mean
1.9%
(SD =
8.51%)
≤
p95
0.77%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5 26.93 KiB
<
mean 235.22 KiB
(SD = 54.18 KiB)
<
p95248.94 KiB
Holochain process threads
Number of threads in Holochain process
p5
7 threads
<
mean
12.24 threads
(SD =
2.37 threads)
<
p95
13 threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
8 file descriptors
<
mean
47 file descriptors
(SD =
17.18 file descriptors)
<
p95
70 file descriptors
Mixed Arc get_agent_activity
A mixed-arc scenario that measures the DHT's ability to propagate agent activity and make it available via get_agent_activity calls. The scenario has three roles:
zero_write: A zero-arc conductor that creates entries on its source chain.full_write: A full-arc conductor that creates entries on its source chain.zero_read: A zero-arc conductor that picks a random writer and repeatedly callsget_agent_activityto observe new entries appearing on the writer's chain, measuring the lag between creation and discovery.
Started
Wed, 18 Mar 2026 18:22:03 UTC
Peer count
65
Peer count at end
4225
Behaviours
-
full_write(1 agent) -
zero_read(1 agent) -
zero_write(1 agent)
Holochain version
0.6.1-rc.3
Wind Tunnel version
0.6.0
Run ID
mixed_arc_get_agent_activity_23256907830
Entry created count
The total number of entries created by write agents over the run, partitioned by agent and behaviour.
Total
346796
Agents affected
44 / 44
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
7882
Max
Highest count in any single partition
25850
Min
Lowest count in any single partition
1462
Observed chain advancement
The maximum highest-observed action sequence per write agent, aggregated across all reading agents. The reading dimension is collapsed by taking the maximum observed value for each write agent (any reader successfully tracking that point counts as propagation). Captures how far readers tracked writers’ chains during the run.
Mean max (across write agents)
Mean of the highest chain head value observed per write agent. Each write agent contributes its maximum — the value seen by any reader.
4256.25
Max
Highest chain head value observed for any single write agent.
6260
Write agents observed
8
Chain head delay timing
The time lag between when a writer creates a new entry and when a reader discovers it via
get_agent_activity. This is measured per new action in the chain.Mean
mean
0.008592s
(SD =
0.09761s)
Max
Highest per-partition mean latency
max mean
0.024724s
Min
Lowest per-partition mean latency
min mean
0.00182s
Chain head delay rate
The rate at which readers detect new chain heads appearing on writers' chains.
Mean
mean
21.48/s
Max
Highest per-partition mean rate
max
30.21/s
Min
Lowest per-partition mean rate
min
12.14/s
get_agent_activity_full zome call timingThe time taken to complete the
get_agent_activity_full zome call that queries a writer's chain.Mean
mean
0.385267s
(SD =
0.442548s)
Max
Highest per-partition mean latency
max mean
0.503405s
Min
Lowest per-partition mean latency
min mean
0.286905s
Retrieval errors
A running accumulation of errors encountered by read agents when calling
get_agent_activity.Total
16
Agents affected
11 / 18
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
1
Max
Highest count in any single partition
3
Min
Lowest count in any single partition
0
Open connections
The number of currently open connections to other conductors, partitioned by agent behaviour.
behaviour:full_write
p5
2
<
mean
47.32
(SD =
17.4)
<
p95
61
behaviour:zero_read
p5
26
<
mean
41.22
(SD =
7.91)
<
p95
50
behaviour:zero_write
p5
19
<
mean
34.85
(SD =
10.11)
<
p95
50
Final error count
The total number of errors accumulated across all agents over the run.
17
Host Metrics
User CPU usage
CPU usage by user space
52.36%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5 22.08 KiB/s
<
mean 309.27 KiB/s
(SD = 602.52 KiB/s)
<
p95829.50 KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5 15.54 KiB/s
<
mean 247.83 KiB/s
(SD = 326.45 KiB/s)
<
p95760.32 KiB/s
Total bytes received
Total bytes received on primary network interface
count
16.57 GiB
mean
17.67 MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
13.28 GiB
mean
14.16 MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 97.9%
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 1960.28 MB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
Detected
Warning 80% of hosts overloaded (load5/ncpus > 1.0)
Additional Host Metrics
Additional Host Metrics
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
6.05%
<
mean
70.6%
(SD =
31.95%)
<
p95
96.91%
System
18.23%
CPU percentiles
CPU usage percentiles
p50
88.12%
p95
96.91%
p99
97.94%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
61 hosts
mean time
0.68s
Memory used percentage
Percentage of memory used
p5
8.56%
<
mean
18.77%
(SD =
8.56%)
<
p95
35.73%
Memory available percentage
Percentage of available memory
p5
64.27%
<
mean
81.23%
(SD =
8.56%)
<
p95
91.44%
Max host memory used
Maximum memory usage percentage across all hosts
max
55.34%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
3.37%
Memory growth rate
Rate of memory growth over time
growth
1960.28MB/s
Disk read throughput
Disk read throughput in MB/s
0.21MB/s
Disk write throughput
Disk write throughput in MB/s
143.92MB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/
0/58hosts
Mount Point
/boot
0/1hosts
Mount Point
/efi-boot
0/2hosts
Mount Point
/etc/hostname
0/6hosts
Mount Point
/etc/hosts
0/6hosts
Mount Point
/etc/resolv.conf
0/6hosts
Mount Point
/nix/store
0/5hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
4.66
5 min
4.22
15 min
3.24
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
79.69%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0.05%
<
mean
52.5736%
(SD =
35.098%)
<
p95
95.48%
60 second average
51.5807%
300 second average
46.7041%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0%
<
mean
5.8379%
(SD =
9.5142%)
<
p95
19.97%
60 second average
5.8671%
300 second average
5.7265%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0%
<
mean
2.3584%
(SD =
7.7149%)
<
p95
13.44%
60 second average
2.4037%
300 second average
2.5013%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.4%
<
mean
31.78%
(SD =
29.84%)
<
p95
86.19%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5 2.23 KiB
<
mean 331.10 KiB
(SD = 243.39 KiB)
<
p95898.72 KiB
Holochain process threads
Number of threads in Holochain process
p5
1 threads
<
mean
24.65 threads
(SD =
17.82 threads)
<
p95
56 threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
0 file descriptors
<
mean
59.1 file descriptors
(SD =
30.05 file descriptors)
<
p95
93 file descriptors
Mixed Arc must_get_agent_activity
A scenario where write agents create entries in batches of 10, while must_get_agent_activity agents each pick a random write agent and repeatedly attempt to create an entry that references the chain top of their latest batch. This reference means that the entry's validation function needs to make a must_get_agent_activity call.
The purpose of this scenario is to measure the time it takes for published agent activity data to be gossiped among authorities and become available to peers that query it via must_get_agent_activity.
This test is similar to Write Validated must_get_agent_activity, but some write agents are zero-arc and all must_get_agent_activity agents are zero-arc. This forces the must_get_agent_activity host call to go to the network.
Started
Wed, 18 Mar 2026 18:24:09 UTC
Peer count
65
Peer count at end
4225
Behaviours
-
full_write(1 agent) -
zero_must_get_agent_activity(1 agent) -
zero_write(1 agent)
Holochain version
0.6.1-rc.3
Wind Tunnel version
0.6.0
Run ID
mixed_arc_must_get_agent_activity_23256907830
Highest observed
action_seqThe maximum chain length observed per
write agent, aggregated across all reading agents. The reading dimension is collapsed by taking the maximum observed value for each write agent (any successful read counts as propagation). This reflects the DHT’s ability to propagate agent activity ops and make them available to querying peers.Mean max (across write agents)
Mean of the highest chain head value observed per write agent. Each write agent contributes its maximum — the value seen by any reader.
595.7
Max
Highest chain head value observed for any single write agent.
1050
Write agents observed
10
Chain batch delay timing
The time between a
write agent's creation of a batch and a must_get_agent_activity agent's successful discovery of the batch and creation/self-validation of a new entry that depends on it.Mean
mean
212.55597s
(SD =
109.119685s)
Max
Highest per-partition mean latency
max mean
738.005143s
Min
Lowest per-partition mean latency
min mean
4.8131s
Chain batch delay rate
The rate at which a
must_get_agent_activity agent was able to discover batches and create/self-validate new entries that depend on them.Mean
mean
2.04/s
Max
Highest per-partition mean rate
max
2.54/s
Min
Lowest per-partition mean rate
min
1.5/s
create_validated_sample_entry zome call timingThe time taken to complete the zome function call that creates the entry that depends on a
write agent's source chain.Mean
mean
0.660187s
(SD =
0.642684s)
Max
Highest per-partition mean latency
max mean
0.983354s
Min
Lowest per-partition mean latency
min mean
0.175362s
Retrieval errors
A running accumulation of the errors encountered by an agent when attempting to self-validate actions that depend on
must_get_agent_activity calls.Total
20579
Agents affected
16 / 16
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
1286
Max
Highest count in any single partition
16238
Min
Lowest count in any single partition
54
Open connections
The average number of open connections maintained by an agent over the run, broken down by behaviour.
behaviour:full_write
p5
0
<
mean
42.88
(SD =
15.78)
<
p95
56
behaviour:zero_must_get_agent_activity
p5
3
<
mean
20.97
(SD =
10.34)
<
p95
34
behaviour:zero_write
p5
2
<
mean
21.55
(SD =
10.47)
<
p95
36
Final error count
The total number of all types of error accumulated over the run by all agents.
24650
Host Metrics
User CPU usage
CPU usage by user space
33.47%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5 656 B/s
<
mean 182.36 KiB/s
(SD = 504.84 KiB/s)
<
p95496.00 KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5 566 B/s
<
mean 160.03 KiB/s
(SD = 277.38 KiB/s)
<
p95554.56 KiB/s
Total bytes received
Total bytes received on primary network interface
count
9.78 GiB
mean
4.96 MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
8.47 GiB
mean
4.30 MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 98.0%
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 2085.10 MB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected
Critical Heavy swap usage (22.3% swap used)
System overload anomaly
System overload anomaly was detected
Detected
Warning 51% of hosts overloaded (load5/ncpus > 1.0)
Additional Host Metrics
Additional Host Metrics
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
2.15%
<
mean
46.77%
(SD =
38.16%)
<
p95
96.4%
System
13.30%
CPU percentiles
CPU usage percentiles
p50
37.67%
p95
96.40%
p99
97.98%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
55 hosts
mean time
0.43s
Memory used percentage
Percentage of memory used
p5
10.63%
<
mean
19.61%
(SD =
9.24%)
<
p95
37.57%
Memory available percentage
Percentage of available memory
p5
62.43%
<
mean
80.39%
(SD =
9.24%)
<
p95
89.37%
Max host memory used
Maximum memory usage percentage across all hosts
max
56.07%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
22.27%
Memory growth rate
Rate of memory growth over time
growth
2085.10MB/s
Disk read throughput
Disk read throughput in MB/s
1.21MB/s
Disk write throughput
Disk write throughput in MB/s
141.73MB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/
0/55hosts
Mount Point
/boot
0/1hosts
Mount Point
/efi-boot
0/5hosts
Mount Point
/etc/hostname
0/8hosts
Mount Point
/etc/hosts
0/8hosts
Mount Point
/etc/resolv.conf
0/8hosts
Mount Point
/nix/store
0/7hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
2.95
5 min
2.79
15 min
2.28
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
50.79%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0%
<
mean
31.0185%
(SD =
34.0544%)
<
p95
93.26%
60 second average
31.1387%
300 second average
28.9731%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0%
<
mean
5.1531%
(SD =
10.3427%)
<
p95
18.76%
60 second average
5.1308%
300 second average
5.1462%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0%
<
mean
3.0137%
(SD =
8.9232%)
<
p95
16.53%
60 second average
2.9792%
300 second average
3.1151%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.18%
<
mean
22.7%
(SD =
27.49%)
<
p95
81.19%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5 2.74 KiB
<
mean 437.33 KiB
(SD = 222.06 KiB)
<
p95909.63 KiB
Holochain process threads
Number of threads in Holochain process
p5
1 threads
<
mean
27.26 threads
(SD =
18.84 threads)
<
p95
71 threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
5 file descriptors
<
mean
67.55 file descriptors
(SD =
27.44 file descriptors)
<
p95
99 file descriptors
Remote Call Rate
Test the throughput of remote_call operations. Each agent in this scenario waits for a certain number of peers to be available or for up to two minutes, whichever happens first, before starting its behaviour.
Started
Wed, 18 Mar 2026 17:06:46 UTC
Peer count
10
Peer count at end
100
Behaviours
-
default(1 agent)
Holochain version
0.6.1-rc.3
Wind Tunnel version
0.6.0
Run ID
remote_call_rate_23256907830
Dispatch timing
The time between sending a remote call and the remote handler being invoked.
Mean
mean
11.684445s
(SD =
61.471496s)
Max
Highest per-partition mean latency
max mean
247.030078s
Min
Lowest per-partition mean latency
min mean
-24.006879s
Round-trip timing
The total elapsed time to get a response to the client.
Mean
mean
0.067229s
(SD =
0.057295s)
Max
Highest per-partition mean latency
max mean
0.116309s
Min
Lowest per-partition mean latency
min mean
0.043024s
Host Metrics
User CPU usage
CPU usage by user space
11.03%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5 15.66 KiB/s
<
mean 172.33 KiB/s
(SD = 486.47 KiB/s)
<
p951.15 MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5 14.50 KiB/s
<
mean 151.35 KiB/s
(SD = 394.14 KiB/s)
<
p951.19 MiB/s
Total bytes received
Total bytes received on primary network interface
count
1.33 GiB
mean
1.48 MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
1.17 GiB
mean
1.30 MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 92.9%
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 259.06 MB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected
Warning Moderate swap usage (16.5% swap used)
System overload anomaly
System overload anomaly was detected
Detected
Warning 11% of hosts overloaded (load5/ncpus > 1.0)
Additional Host Metrics
Additional Host Metrics
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
3.9%
<
mean
16.15%
(SD =
24.55%)
<
p95
91.84%
System
5.12%
CPU percentiles
CPU usage percentiles
p50
7.27%
p95
91.84%
p99
92.87%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
1 hosts
mean time
0.74s
Memory used percentage
Percentage of memory used
p5
7.82%
<
mean
17.06%
(SD =
9.16%)
<
p95
35.49%
Memory available percentage
Percentage of available memory
p5
64.51%
<
mean
82.94%
(SD =
9.16%)
<
p95
92.18%
Max host memory used
Maximum memory usage percentage across all hosts
max
32.52%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
16.46%
Memory growth rate
Rate of memory growth over time
growth
259.06MB/s
Disk read throughput
Disk read throughput in MB/s
0.69MB/s
Disk write throughput
Disk write throughput in MB/s
22.67MB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/
0/5hosts
Mount Point
/boot
0/1hosts
Mount Point
/efi-boot
0/4hosts
Mount Point
/etc/hostname
0/4hosts
Mount Point
/etc/hosts
0/4hosts
Mount Point
/etc/resolv.conf
0/4hosts
Mount Point
/nix/store
0/5hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
1.37
5 min
1.16
15 min
1.12
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
11.11%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0%
<
mean
5.3407%
(SD =
16.3689%)
<
p95
59.59%
60 second average
4.9914%
300 second average
3.4686%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0%
<
mean
2.5843%
(SD =
6.2591%)
<
p95
17.28%
60 second average
2.7155%
300 second average
3.1499%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0%
<
mean
0.9989%
(SD =
3.0565%)
<
p95
4.8%
60 second average
1.1530%
300 second average
1.8553%
Holochain process CPU usage
CPU usage by Holochain process
p5
1.25%
<
mean
5.35%
(SD =
4.24%)
<
p95
10.01%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5 200.72 KiB
<
mean 218.82 KiB
(SD = 39.10 KiB)
<
p95261.52 KiB
Holochain process threads
Number of threads in Holochain process
p5
13 threads
<
mean
18.24 threads
(SD =
3.48 threads)
<
p95
22 threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
45 file descriptors
<
mean
55.97 file descriptors
(SD =
12.95 file descriptors)
<
p95
85 file descriptors
Remote Signals
This scenario tests the throughput of remote_signals operations.
Started
Wed, 18 Mar 2026 17:09:33 UTC
Peer count
10
Peer count at end
50
Behaviours
-
default(1 agent)
Holochain version
0.6.1-rc.3
Wind Tunnel version
0.6.0
Run ID
remote_signals_23256907830
Round trip time
The time from origin signal dispatch to origin receive of the remote side's response signal.
mean
0.096656s
(SD =
0.258222s)
p50
0.052656s
p95
0.201098s
p99
0.2268s
Timeouts
The number of timeouts waiting for the remote side's response signal (default timeout is 20 seconds).
0
Host Metrics
User CPU usage
CPU usage by user space
3.22%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5 4.70 KiB/s
<
mean 166.32 KiB/s
(SD = 537.43 KiB/s)
<
p951.70 MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5 4.62 KiB/s
<
mean 176.55 KiB/s
(SD = 587.82 KiB/s)
<
p951.67 MiB/s
Total bytes received
Total bytes received on primary network interface
count
581.47 MiB
mean
661.58 KiB/s
Total bytes sent
Total bytes sent on primary network interface
count
617.23 MiB
mean
702.27 KiB/s
CPU spike anomaly
CPU spike anomaly was detected
NotDetected
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 34.99 MB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected
Critical Heavy swap usage (21.2% swap used)
System overload anomaly
System overload anomaly was detected
NotDetected
Additional Host Metrics
Additional Host Metrics
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
2.18%
<
mean
4.93%
(SD =
7.24%)
<
p95
9.2%
System
1.71%
CPU percentiles
CPU usage percentiles
p50
2.89%
p95
9.20%
p99
45.50%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
1 hosts
mean time
0.01s
Memory used percentage
Percentage of memory used
p5
6.91%
<
mean
22.38%
(SD =
11.19%)
<
p95
37.14%
Memory available percentage
Percentage of available memory
p5
62.86%
<
mean
77.62%
(SD =
11.19%)
<
p95
93.09%
Max host memory used
Maximum memory usage percentage across all hosts
max
36.60%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
21.17%
Memory growth rate
Rate of memory growth over time
growth
34.99MB/s
Disk read throughput
Disk read throughput in MB/s
0.00MB/s
Disk write throughput
Disk write throughput in MB/s
2.23MB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/
0/1hosts
Mount Point
/efi-boot
0/1hosts
Mount Point
/etc/hostname
0/2hosts
Mount Point
/etc/hosts
0/2hosts
Mount Point
/etc/resolv.conf
0/2hosts
Mount Point
/nix/store
0/1hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
0.32
5 min
0.52
15 min
0.91
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
0.00%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0%
<
mean
0.6878%
(SD =
3.7472%)
<
p95
0.9%
60 second average
0.6982%
300 second average
0.8229%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0%
<
mean
1.0057%
(SD =
2.135%)
<
p95
3.36%
60 second average
1.3370%
300 second average
4.2486%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0%
<
mean
0.9552%
(SD =
2.0799%)
<
p95
3.2%
60 second average
1.2774%
300 second average
4.0658%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.62%
<
mean
1.25%
(SD =
3.13%)
<
p95
1.44%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5 175.61 KiB
<
mean 197.21 KiB
(SD = 17.71 KiB)
<
p95216.01 KiB
Holochain process threads
Number of threads in Holochain process
p5
16 threads
<
mean
18.69 threads
(SD =
2.24 threads)
<
p95
21 threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
48 file descriptors
<
mean
55.54 file descriptors
(SD =
11.27 file descriptors)
<
p95
86 file descriptors
Single-write, many-read
Creates an entry in the agent setup, then read one's own record back. This tests the maximum read performance for reading back local data when the chain is short.
Started
Wed, 18 Mar 2026 17:11:35 UTC
Peer count
1
Peer count at end
1
Behaviours
-
default(1 agent)
Holochain version
0.6.1-rc.3
Wind Tunnel version
0.6.0
Run ID
single_write_many_read_23256907830
read call timingThe time it takes to complete the zome call that retrieves the record.
mean
0.001922s
(SD =
0.00055s)
p50
0.001841s
p95
0.002252s
p99
0.002721s
read call rateThe number of read zome calls completed per second.
mean
5133.36/s
Error count
The number of errors collected.
0
Host Metrics
User CPU usage
CPU usage by user space
10.20%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5 18.02 KiB/s
<
mean 22.36 KiB/s
(SD = 2.77 KiB/s)
<
p9527.07 KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5 18.90 KiB/s
<
mean 25.81 KiB/s
(SD = 10.27 KiB/s)
<
p9548.08 KiB/s
Total bytes received
Total bytes received on primary network interface
count
6.33 MiB
mean
22.36 KiB/s
Total bytes sent
Total bytes sent on primary network interface
count
7.31 MiB
mean
25.81 KiB/s
CPU spike anomaly
CPU spike anomaly was detected
NotDetected
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 31.07 MB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected
Warning Moderate swap usage (16.5% swap used)
System overload anomaly
System overload anomaly was detected
NotDetected
Additional Host Metrics
Additional Host Metrics
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
18.31%
<
mean
18.69%
(SD =
0.96%)
<
p95
18.8%
System
8.49%
CPU percentiles
CPU usage percentiles
p50
18.52%
p95
18.80%
p99
23.71%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
0 hosts
mean time
0.00s
Memory used percentage
Percentage of memory used
p5
35.49%
<
mean
36.34%
(SD =
0.81%)
<
p95
37.23%
Memory available percentage
Percentage of available memory
p5
62.77%
<
mean
63.66%
(SD =
0.81%)
<
p95
64.51%
Max host memory used
Maximum memory usage percentage across all hosts
max
36.34%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
16.46%
Memory growth rate
Rate of memory growth over time
growth
31.07MB/s
Disk read throughput
Disk read throughput in MB/s
0.00MB/s
Disk write throughput
Disk write throughput in MB/s
9.91MB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/etc/hostname
0/1hosts
Mount Point
/etc/hosts
0/1hosts
Mount Point
/etc/resolv.conf
0/1hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
1.23
5 min
1.22
15 min
1.36
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
0.00%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0%
<
mean
0.062%
(SD =
0.1834%)
<
p95
0.54%
60 second average
0.0620%
300 second average
0.0630%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
Holochain process CPU usage
CPU usage by Holochain process
p5
12.3%
<
mean
12.47%
(SD =
0.84%)
≤
p95
12.34%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5 248.35 KiB
<
mean 249.30 KiB
(SD = 9.30 KiB)
<
p95257.89 KiB
Holochain process threads
Number of threads in Holochain process
p5
16 threads
<
mean
16.27 threads
(SD =
1.44 threads)
≤
p95
16 threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
55 file descriptors
<
mean
57.27 file descriptors
(SD =
3.45 file descriptors)
<
p95
64 file descriptors
Two-party countersigning
This scenario tests the performance of countersigning operations. There are two roles: initiate and participate. The participants commit an entry to advertise that they are willing to participate in sessions. They listen for sessions and participate in one at a time.
Started
Wed, 18 Mar 2026 18:05:18 UTC
Peer count
3
Peer count at end
9
Behaviours
-
initiate(1 agent) -
participate(1 agent)
Holochain version
0.6.1-rc.3
Wind Tunnel version
0.6.0
Run ID
two_party_countersigning_23256907830
Session accepted -- timing
The duration of the session from acceptance to completion.
Mean
mean
1.020921s
(SD =
0.727687s)
Max
Highest per-partition mean latency
max mean
1.083344s
Min
Lowest per-partition mean latency
min mean
0.958497s
Session accepted -- success rate
The number of accepted sessions that completed per second.
Mean
mean
4.25/s
Max
Highest per-partition mean rate
max
5.04/s
Min
Lowest per-partition mean rate
min
3.46/s
Session accepted -- success ratio
Fraction of accepted sessions that completed successfully (0–1).
1
Session initiated -- timing
The duration of the session from initiation to completion.
Mean
mean
1.414448s
(SD =
0.633261s)
Max
Highest per-partition mean latency
max mean
1.414448s
Min
Lowest per-partition mean latency
min mean
1.414448s
Session initiated -- success rate
The number of initiated sessions that completed per second.
Mean
mean
7.08/s
Max
Highest per-partition mean rate
max
7.08/s
Min
Lowest per-partition mean rate
min
7.08/s
Session initiated -- success ratio
Fraction of initiated sessions that completed successfully (0–1).
1
Session initiated -- failures
Initiated sessions that failed to complete.
Total
2
Agents affected
1 / 1
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
2
Max
Highest count in any single partition
2
Min
Lowest count in any single partition
2
Host Metrics
User CPU usage
CPU usage by user space
61.76%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5 39.97 KiB/s
<
mean 344.90 KiB/s
(SD = 1.04 MiB/s)
<
p95676.59 KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5 26.50 KiB/s
<
mean 129.63 KiB/s
(SD = 106.18 KiB/s)
<
p95345.73 KiB/s
Total bytes received
Total bytes received on primary network interface
count
303.14 MiB
mean
1001.33 KiB/s
Total bytes sent
Total bytes sent on primary network interface
count
113.93 MiB
mean
376.34 KiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 98.0%
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 324.99 MB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
Detected
Warning 100% of hosts overloaded (load5/ncpus > 1.0)
Additional Host Metrics
Additional Host Metrics
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
61.19%
<
mean
88.13%
(SD =
11.66%)
<
p95
97.4%
System
26.37%
CPU percentiles
CPU usage percentiles
p50
93.51%
p95
97.40%
p99
97.95%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
3 hosts
mean time
0.83s
Memory used percentage
Percentage of memory used
p5
11.86%
<
mean
15.06%
(SD =
1.97%)
<
p95
19%
Memory available percentage
Percentage of available memory
p5
81%
<
mean
84.94%
(SD =
1.97%)
<
p95
88.14%
Max host memory used
Maximum memory usage percentage across all hosts
max
16.37%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00%
Memory growth rate
Rate of memory growth over time
growth
324.99MB/s
Disk read throughput
Disk read throughput in MB/s
0.00MB/s
Disk write throughput
Disk write throughput in MB/s
0.00MB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/
0/3hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
5.40
5 min
4.15
15 min
2.14
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
100.00%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
37.5%
<
mean
76.9772%
(SD =
17.7461%)
<
p95
94.08%
60 second average
74.7969%
300 second average
56.8224%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
3.39%
<
mean
6.9142%
(SD =
4.9225%)
<
p95
11.33%
60 second average
7.1428%
300 second average
7.9055%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0.01%
<
mean
2.0029%
(SD =
3.8886%)
<
p95
5.86%
60 second average
2.4737%
300 second average
4.4451%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.9%
<
mean
13.87%
(SD =
18.43%)
<
p95
56.1%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5 1.88 KiB
<
mean 154.32 KiB
(SD = 96.07 KiB)
<
p95251.81 KiB
Holochain process threads
Number of threads in Holochain process
p5
1 threads
<
mean
12.33 threads
(SD =
7.82 threads)
<
p95
31 threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
4 file descriptors
<
mean
53.42 file descriptors
(SD =
29.49 file descriptors)
<
p95
92 file descriptors
Validation Receipts
Creates an entry, wait for required validation receipts, then repeat. Records the amount of time it took to accumulate the required number of receipts for all DHT operations. This is measured to the nearest 20ms so that we don't keep the agent too busy checking for receipts.
Each agent in this scenario waits for a certain number of peers to be available or for up to two minutes, whichever happens first, before starting its behaviour.
By default, this scenario will wait for a complete set of validation receipts before committing the next record. If the NO_VALIDATION_COMPLETE environment variable is set, it will instead publish new records on every round, building up an ever-growing list of action hashes to check on.
Started
Wed, 18 Mar 2026 17:55:51 UTC
Peer count
20
Peer count at end
400
Behaviours
-
default(1 agent)
Holochain version
0.6.1-rc.3
Wind Tunnel version
0.6.0
Run ID
validation_receipts_23256907830
Receipts complete timing
The amount of time between publishing a record and receiving the required number of validation receipts.
Mean
mean
84.136269s
(SD =
50.110921s)
Max
Highest per-partition mean latency
max mean
330.939809s
Min
Lowest per-partition mean latency
min mean
8.664752s
Receipts complete rate
The number of complete validation receipt sets collected per second.
Mean
mean
1.45/s
Max
Highest per-partition mean rate
max
1.9/s
Min
Lowest per-partition mean rate
min
0/s
Host Metrics
User CPU usage
CPU usage by user space
36.13%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5 710 B/s
<
mean 340.26 KiB/s
(SD = 846.05 KiB/s)
<
p95761.27 KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5 484 B/s
<
mean 203.03 KiB/s
(SD = 249.86 KiB/s)
<
p95724.14 KiB/s
Total bytes received
Total bytes received on primary network interface
count
5.85 GiB
mean
6.45 MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
3.49 GiB
mean
3.85 MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 96.8%
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 1233.26 MB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected
Critical Heavy swap usage (39.8% swap used)
System overload anomaly
System overload anomaly was detected
Detected
Warning 55% of hosts overloaded (load5/ncpus > 1.0)
Additional Host Metrics
Additional Host Metrics
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
2.11%
<
mean
50.07%
(SD =
35.21%)
<
p95
94.84%
System
13.94%
CPU percentiles
CPU usage percentiles
p50
49.26%
p95
94.84%
p99
96.84%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
13 hosts
mean time
0.53s
Memory used percentage
Percentage of memory used
p5
8.13%
<
mean
23.98%
(SD =
14.33%)
<
p95
56.21%
Memory available percentage
Percentage of available memory
p5
43.79%
<
mean
76.02%
(SD =
14.33%)
<
p95
91.87%
Max host memory used
Maximum memory usage percentage across all hosts
max
68.47%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
39.83%
Memory growth rate
Rate of memory growth over time
growth
1233.26MB/s
Disk read throughput
Disk read throughput in MB/s
0.46MB/s
Disk write throughput
Disk write throughput in MB/s
88.48MB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/
0/14hosts
Mount Point
/efi-boot
0/3hosts
Mount Point
/etc/hostname
0/6hosts
Mount Point
/etc/hosts
0/6hosts
Mount Point
/etc/resolv.conf
0/6hosts
Mount Point
/nix/store
0/4hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
3.74
5 min
2.89
15 min
2.07
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
55.00%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0%
<
mean
32.5116%
(SD =
36.2219%)
<
p95
92.9%
60 second average
30.1931%
300 second average
20.9767%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0.06%
<
mean
13.3787%
(SD =
16.0898%)
<
p95
49.95%
60 second average
13.3854%
300 second average
12.8563%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0.02%
<
mean
8.9961%
(SD =
13.0713%)
<
p95
38.06%
60 second average
9.2108%
300 second average
9.5885%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.15%
<
mean
20.84%
(SD =
23.04%)
<
p95
71.23%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5 2.16 KiB
<
mean 228.55 KiB
(SD = 138.46 KiB)
<
p95484.10 KiB
Holochain process threads
Number of threads in Holochain process
p5
1 threads
<
mean
22.65 threads
(SD =
14.95 threads)
<
p95
52 threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
5 file descriptors
<
mean
67.81 file descriptors
(SD =
34.65 file descriptors)
<
p95
118 file descriptors
Write/get_agent_activity
A scenario where write peers write entries, while get_agent_activity peers each query a single write agent's activity with get_agent_activity.
Before a target write peer and the requesting get_agent_activity peer are in sync, this will measure the get_agent_activity call performance over a network. Once a write peer reaches sync with a get_agent_activity peer, the write peer will publish their actions and entries, and so the get_agent_activity calls will likely have most of the data they need locally. At that point this measures the database query performance and code paths through host functions.
Started
Wed, 18 Mar 2026 17:57:57 UTC
Peer count
65
Peer count at end
3393
Behaviours
-
get_agent_activity(1 agent) -
write(1 agent)
Holochain version
0.6.1-rc.3
Wind Tunnel version
0.6.0
Run ID
write_get_agent_activity_23256907830
Observed chain advancement
The maximum highest-observed action sequence per write agent, aggregated across all reading agents. The reading dimension is collapsed by taking the maximum observed value for each write agent (any reader successfully tracking that point counts as propagation). Captures how far readers tracked writers’ chains during the run.
Mean max (across write agents)
Mean of the highest chain head value observed per write agent. Each write agent contributes its maximum — the value seen by any reader.
3671.33
Max
Highest chain head value observed for any single write agent.
5134
Write agents observed
12
get_agent_activity_full zome call timingThe time taken to call the zome function that retrieves information on a
write peer's source chain.Mean
mean
0.395977s
(SD =
0.351177s)
Max
Highest per-partition mean latency
max mean
0.673377s
Min
Lowest per-partition mean latency
min mean
0.014513s
Error count
The number of errors accumulated.
103
Host Metrics
User CPU usage
CPU usage by user space
48.55%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5 1.08 KiB/s
<
mean 393.83 KiB/s
(SD = 871.51 KiB/s)
<
p95945.67 KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5 583 B/s
<
mean 234.34 KiB/s
(SD = 253.36 KiB/s)
<
p95740.11 KiB/s
Total bytes received
Total bytes received on primary network interface
count
21.76 GiB
mean
6.17 MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
12.79 GiB
mean
3.63 MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 97.5%
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 1561.03 MB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected
Critical Heavy swap usage (39.8% swap used)
System overload anomaly
System overload anomaly was detected
Detected
Warning 79% of hosts overloaded (load5/ncpus > 1.0)
Additional Host Metrics
Additional Host Metrics
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
12.14%
<
mean
67.65%
(SD =
30.24%)
<
p95
96.21%
System
19.09%
CPU percentiles
CPU usage percentiles
p50
82.77%
p95
96.21%
p99
97.50%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
51 hosts
mean time
0.65s
Memory used percentage
Percentage of memory used
p5
8.98%
<
mean
18.99%
(SD =
9.77%)
<
p95
31.95%
Memory available percentage
Percentage of available memory
p5
68.05%
<
mean
81.01%
(SD =
9.77%)
<
p95
91.02%
Max host memory used
Maximum memory usage percentage across all hosts
max
68.54%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
39.83%
Memory growth rate
Rate of memory growth over time
growth
1561.03MB/s
Disk read throughput
Disk read throughput in MB/s
0.49MB/s
Disk write throughput
Disk write throughput in MB/s
180.89MB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/
0/53hosts
Mount Point
/efi-boot
0/8hosts
Mount Point
/etc/hostname
0/9hosts
Mount Point
/etc/hosts
0/9hosts
Mount Point
/etc/resolv.conf
0/9hosts
Mount Point
/nix/store
0/8hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
4.90
5 min
3.61
15 min
2.19
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
79.03%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0%
<
mean
49.9095%
(SD =
36.8924%)
<
p95
95.58%
60 second average
46.6977%
300 second average
33.4376%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
<
mean
0.0003%
(SD =
0.0186%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
1.08%
<
mean
14.2629%
(SD =
17.754%)
<
p95
58.83%
60 second average
13.8665%
300 second average
11.8735%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0.04%
<
mean
8.5221%
(SD =
14.2435%)
<
p95
44.54%
60 second average
8.4456%
300 second average
7.8218%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.2%
<
mean
26.57%
(SD =
24.84%)
<
p95
78.29%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5 1.78 KiB
<
mean 220.35 KiB
(SD = 147.10 KiB)
<
p95492.49 KiB
Holochain process threads
Number of threads in Holochain process
p5
1 threads
<
mean
24 threads
(SD =
18.62 threads)
<
p95
56 threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
3 file descriptors
<
mean
62.75 file descriptors
(SD =
32.67 file descriptors)
<
p95
95 file descriptors
Write/get_agent_activity with volatile conductors
A scenario where write peers write entries, while get_agent_activity_volatile peers each query a single write agents activity with get_agent_activity but shutdown and restart their conductors at semi-random intervals.
Before a target write peer and the requesting get_agent_activity_volatile peer are in sync, this will measure the get_agent_activity call performance over a network. Once a write peer reaches sync with a get_agent_activity peer, the write peer will publish their actions and entries, and so the get_agent_activity calls will likely have most of the data they need locally. At that point this measures the database query performance and code paths through host functions.
Started
Wed, 18 Mar 2026 18:00:57 UTC
Peer count
65
Peer count at end
3521
Behaviours
-
get_agent_activity_volatile(1 agent) -
write(1 agent)
Holochain version
0.6.1-rc.3
Wind Tunnel version
0.6.0
Run ID
write_get_agent_activity_volatile_23256907830
Highest observed
action_seqThe rate at which zero-arc readers observe new chain heads on the writers' chains via
get_agent_activity. This reflects the DHT's ability to propagate agent activity ops and make them available to querying peers.Mean
mean
21.2/s
Max
Highest per-partition mean rate
max
210.27/s
Min
Lowest per-partition mean rate
min
6.6/s
get_agent_activity_full zome call timingThe time taken to call the zome function that retrieves information on a
write peer's source chain.Mean
mean
0.464894s
(SD =
1.377753s)
Max
Highest per-partition mean latency
max mean
0.984737s
Min
Lowest per-partition mean latency
min mean
0.033412s
Running volatile conductors count
The number of conductors running by
get_agent_activity_volatile peers.
p5
27 conductors
<
mean
34.77 conductors
(SD =
4.57 conductors)
<
p95
40 conductors
Volatile conductor total running duration
The total running duration of a
get_agent_activity_volatile conductor.Mean
mean
677.1174s
(SD =
554.3946s)
Max
Highest per-partition mean latency
max mean
1144.454545s
Min
Lowest per-partition mean latency
min mean
286.833333s
Volatile conductor running duration
The duration that a
get_agent_activity_volatile conductor was running before being stopped.Mean
mean
82.390823s
(SD =
61.566198s)
Max
Highest per-partition mean latency
max mean
141.640398s
Min
Lowest per-partition mean latency
min mean
31.33099s
Volatile conductor stopped duration
The duration that a
get_agent_activity_volatile conductor was stopped before being started again.Mean
mean
62.013581s
(SD =
61.980057s)
Max
Highest per-partition mean latency
max mean
99.195568s
Min
Lowest per-partition mean latency
min mean
7.976504s
Reached target arc
Did
get_agent_activity_volatile peer reach their target arc in the moment before they were shutdown.get_agent_activity_volatile_agent:uhCAk-sey53EI_sCh7Gl3Rjm3dSpup5ur2xNYxnP8Pji2C8JdOVjK
p5
0
<
mean
0.05
(SD =
0.22)
<
p95
1
get_agent_activity_volatile_agent:uhCAk0LAnKQJW4oSvKpTuWp3O3Hxjz85u19EtQsk3hV49y2uGg6wv
p5
0
<
mean
0.38
(SD =
0.48)
<
p95
1
get_agent_activity_volatile_agent:uhCAk1LhmhWb_RkhpzESvsS6-ojfefwF8QUDyouBGKDz8Hnx1O03F
p5
0
<
mean
0.52
(SD =
0.5)
<
p95
1
get_agent_activity_volatile_agent:uhCAk4RUBQ2Nr7_6VDbSTyv8c66HolMdWw4XqFfev3bbjahVjAquX
p5
0
<
mean
0.64
(SD =
0.48)
<
p95
1
get_agent_activity_volatile_agent:uhCAk5dgj0pazcA95UxCi5k3y19hhZQvj-__99OBPqdb06z8HayhU
p5
0
<
mean
0.62
(SD =
0.49)
<
p95
1
get_agent_activity_volatile_agent:uhCAk7FaonuX_tGSd6xK58ejoMAJb4zDo1LrAQ3h6BIIuP65EKwGM
p5
0
<
mean
0.63
(SD =
0.48)
<
p95
1
get_agent_activity_volatile_agent:uhCAk7vuIB1pEY_D4MegWg7fvmvor-MSJBnqQnjh-w4l9m_A_k3qX
p5
0
<
mean
0.5
(SD =
0.5)
<
p95
1
get_agent_activity_volatile_agent:uhCAk8TmsZmH_rmLMP58NsQ8OaCmC4ANzquUEtiZK8LpwUdLzOiO3
p5
0
<
mean
0.59
(SD =
0.49)
<
p95
1
get_agent_activity_volatile_agent:uhCAk8XG_Fd_dY7x_d2hOJ0y9aYJSkGrdj2-ZBC7WJawgZTMcrs3B
p5
0
<
mean
0.61
(SD =
0.49)
<
p95
1
get_agent_activity_volatile_agent:uhCAk9AR4YxN-RLmNapjWjBstynGNM38WP0o4iP9Bnk7rGskN51cK
p5
0
<
mean
0.59
(SD =
0.49)
<
p95
1
get_agent_activity_volatile_agent:uhCAkA_ItLy8EK3pu-v3vrM9XybXHeZDiOsUnQJnilVyXduCwBMdy
p5
0
<
mean
0.6
(SD =
0.49)
<
p95
1
get_agent_activity_volatile_agent:uhCAkBLpuMlXHomnCinRlpTTpYZ3cxR1qxg10Xp9R5kYpSrKUYhPG
p5
0
<
mean
0.67
(SD =
0.47)
<
p95
1
get_agent_activity_volatile_agent:uhCAkBWUGNB-rTgV2IelCjM1SbPShXFjAh5MuBsXmVwqudW4Xeval
p5
0
<
mean
0.75
(SD =
0.43)
<
p95
1
get_agent_activity_volatile_agent:uhCAkEbsgJSz8K5_-arh9bkZQunZxsgu1uOgcZIfFl6m0hhjqJKb1
p5
0
<
mean
0.4
(SD =
0.49)
<
p95
1
get_agent_activity_volatile_agent:uhCAkFRIXZ__S7xmbtSYMBiUBUqe7URDwlxy-Ht-vHkCl-xiTLTfn
p5
0
<
mean
0.53
(SD =
0.5)
<
p95
1
get_agent_activity_volatile_agent:uhCAkIA4UzkBnCH7rM1bxlGg1qDRlqo3Lwduc-3ZwgaJaTN6eu2jB
p5
0
<
mean
0.87
(SD =
0.34)
<
p95
1
get_agent_activity_volatile_agent:uhCAkJJqUv6aF8jGGC_Zo5Y4N5nhANCdm4JukbCZHHWyxRWLpg4lo
p5
0
<
mean
0.42
(SD =
0.49)
<
p95
1
get_agent_activity_volatile_agent:uhCAkLYXhoEbWxEHEcD3Zn2oXnouW4PWMLcgW__JZ8lbajGEalr7R
p5
0
<
mean
0.92
(SD =
0.28)
<
p95
1
get_agent_activity_volatile_agent:uhCAkOUhZ4L6NTAV5_KSy-pnH62RMh9_Fr9zoRhXNvCLzF_556dbl
p5
0
<
mean
0.62
(SD =
0.49)
<
p95
1
get_agent_activity_volatile_agent:uhCAkQywM8mdYGGTikgNVkE37Wnb81U2ulxWt9dndtKnRVKb-QhzE
p5
0
<
mean
0.44
(SD =
0.5)
<
p95
1
get_agent_activity_volatile_agent:uhCAkRYdQxh6HA-yixXT2DLQPLVPkp3V02qNeZYt-sqoq_riOTCjC
p5
0
<
mean
0.41
(SD =
0.49)
<
p95
1
get_agent_activity_volatile_agent:uhCAkS0_AsD3XjNqL5t5WCi1yYBb2tmQWiT2Lhfm_GzOAJ_LnopEq
p5
0
<
mean
0.58
(SD =
0.49)
<
p95
1
get_agent_activity_volatile_agent:uhCAkT8VfleGBG9q7mOEVQvoGlfrnG23kRH5XQuD04ae8iPaSuiKb
p5
0
<
mean
0.59
(SD =
0.49)
<
p95
1
get_agent_activity_volatile_agent:uhCAkTaceVnE3O-Hc0DcaqvZ7X1OIyyTVfJwFsvXkv8y2MFTXAjIh
p5
0
<
mean
0.65
(SD =
0.48)
<
p95
1
get_agent_activity_volatile_agent:uhCAkTfaVzpltNNTkmREtfZOeKiQekQohIQod2JtWaxGPdHdwpMJH
p5
0
<
mean
0.56
(SD =
0.5)
<
p95
1
get_agent_activity_volatile_agent:uhCAkUriV95ybkm9W4_juLGZp-3Ed0ZvU-Itw-ITcrBdpwd4U4pSU
p5
0
<
mean
0.43
(SD =
0.49)
<
p95
1
get_agent_activity_volatile_agent:uhCAkVHnyQ3KPWqgez0jG6hB6FLLAKDoKysLAzIkeJiiuKon278Z4
p5
0
<
mean
0.47
(SD =
0.5)
<
p95
1
get_agent_activity_volatile_agent:uhCAkVjzeUylYssR4Y1Qr2XUaNXHIOLDebxeyCWnJTxGUT8SuAN54
p5
0
<
mean
0.39
(SD =
0.49)
<
p95
1
get_agent_activity_volatile_agent:uhCAkWkLB4dYj8SW4LQGdJytM4S8ivxKWNP0Lwr5Uou4zPl6N8uTm
p5
0
≤
mean
0
(SD =
0)
≤
p95
0
get_agent_activity_volatile_agent:uhCAkWmKii7d5QYdsWuUL_SfXZLJTQ4KxexlH2XBTifQig9_nd3vJ
p5
0
<
mean
0.44
(SD =
0.5)
<
p95
1
get_agent_activity_volatile_agent:uhCAkXkWfXe6-eg7On_eCcB4BEHGB44-Hc-QmPBHm09zEN6cMzehv
p5
0
<
mean
0.27
(SD =
0.44)
<
p95
1
get_agent_activity_volatile_agent:uhCAkcszdt6glzh4kYlq0OB_LHOKheLZrLdYYGID4wzxSuzYOWCUg
p5
0
≤
mean
0
(SD =
0)
≤
p95
0
get_agent_activity_volatile_agent:uhCAkekXi6YLZZOdmP7rDwxLqJy-nPkP3ABpPxlMC9DV76O3IG_Ts
p5
0
<
mean
0.32
(SD =
0.46)
<
p95
1
get_agent_activity_volatile_agent:uhCAkhqwCeY3pGfDQxX9_4Oncw-mBWWhviwBF760V8doakx7z6ngs
p5
0
<
mean
0.57
(SD =
0.49)
<
p95
1
get_agent_activity_volatile_agent:uhCAkhx8edmO3ErFUq4Zq0PPLl-0okVr3YbcOV2jDfKNghfm7Evnq
p5
0
<
mean
0.55
(SD =
0.5)
<
p95
1
get_agent_activity_volatile_agent:uhCAkja-jXby3ALpxST-MzqG8vP1rqPeqiyysWD3wVmMdlXI9GjjM
p5
0
<
mean
0.25
(SD =
0.43)
<
p95
1
get_agent_activity_volatile_agent:uhCAkkycWbrz6Ki5Wcbelo4404JuESHRu7_07lPs-AuPfFY-PUjLJ
p5
0
<
mean
0.71
(SD =
0.46)
<
p95
1
get_agent_activity_volatile_agent:uhCAknGYE7GG2Ic6U8OBRdEDkj44otNU6py50cvfx4vxKW67ZLx9G
p5
0
<
mean
0.5
(SD =
0.5)
<
p95
1
get_agent_activity_volatile_agent:uhCAkpusfr9WrsuKBEdhpzjZ3iStdB_rKt0HJm75VyTUtazSmMQ9v
p5
0
<
mean
0.68
(SD =
0.46)
<
p95
1
get_agent_activity_volatile_agent:uhCAkqAT5MDGesVQayVWz5VJXoCvt7F_UbhQXY086n5cN4Je_h1rQ
p5
0
<
mean
0.46
(SD =
0.5)
<
p95
1
get_agent_activity_volatile_agent:uhCAkt2E54nD2jP58RFeVbn2w8JGm1NAbEF60x30CEp4hRRnOPXHL
p5
0
<
mean
0.41
(SD =
0.49)
<
p95
1
get_agent_activity_volatile_agent:uhCAkt42MJngdQPopV0pyjkQGD1xqRYrcoPTLVsbZYK_SOih7y1ab
p5
0
≤
mean
0
(SD =
0)
≤
p95
0
get_agent_activity_volatile_agent:uhCAktxvhGAO9pY2k6oLDC1yC3wJGISCQhMkSPefYBqCwTx9hNyig
p5
0
<
mean
0.57
(SD =
0.49)
<
p95
1
get_agent_activity_volatile_agent:uhCAkwWa1kWtS7YM9cJD8EoqjVn8c3EcYBmuWCcFsWlZMmhUJJnlk
p5
0
<
mean
0.35
(SD =
0.48)
<
p95
1
get_agent_activity_volatile_agent:uhCAkxXUFEB_2G8TUAx6zjZbTQJvvTUoE81YK1ajaK6ZXpiSNR-0e
p5
0
<
mean
0.48
(SD =
0.5)
<
p95
1
Error count
The number of errors accumulated.
682
Write/Query
Creates an entry, then queries the source chain and performs a simple operation on the entries, then repeats.
Started
Wed, 18 Mar 2026 17:14:21 UTC
Peer count
1
Peer count at end
1
Behaviours
-
default(1 agent)
Holochain version
0.6.1-rc.3
Wind Tunnel version
0.6.0
Run ID
write_query_23256907830
Write timing
The time it takes to complete the zome function call that writes the entry.
mean
0.035468s
(SD =
0.082598s)
p50
0.012369s
p95
0.248803s
p99
0.374277s
Write rate
The number of writes completed per second.
mean
106/s
Query timing
The time it takes to complete the
chain_query_count_len zome call that queries the source chain.
mean
0.054715s
(SD =
0.031436s)
p50
0.052274s
p95
0.105122s
p99
0.116908s
Query rate
The number of chain queries completed per second.
mean
105.96/s
Error count
The number of errors accumulated during the run.
0
Host Metrics
User CPU usage
CPU usage by user space
24.22%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5 61 B/s
<
mean 110.70 KiB/s
(SD = 506.84 KiB/s)
<
p95601.27 KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5 49 B/s
<
mean 2.35 KiB/s
(SD = 9.30 KiB/s)
<
p9515.31 KiB/s
Total bytes received
Total bytes received on primary network interface
count
33.51 MiB
mean
110.70 KiB/s
Total bytes sent
Total bytes sent on primary network interface
count
728.75 KiB
mean
2.35 KiB/s
CPU spike anomaly
CPU spike anomaly was detected
NotDetected
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 96.01 MB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected
Warning Moderate swap usage (14.1% swap used)
System overload anomaly
System overload anomaly was detected
NotDetected
Additional Host Metrics
Additional Host Metrics
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
22.56%
<
mean
29.38%
(SD =
6.17%)
<
p95
35.57%
System
5.16%
CPU percentiles
CPU usage percentiles
p50
30.41%
p95
35.57%
p99
41.41%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
0 hosts
mean time
0.00s
Memory used percentage
Percentage of memory used
p5
41.89%
<
mean
45.16%
(SD =
0.9%)
<
p95
45.6%
Memory available percentage
Percentage of available memory
p5
54.4%
<
mean
54.84%
(SD =
0.9%)
<
p95
58.11%
Max host memory used
Maximum memory usage percentage across all hosts
max
45.16%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
14.10%
Memory growth rate
Rate of memory growth over time
growth
96.01MB/s
Disk read throughput
Disk read throughput in MB/s
0.00MB/s
Disk write throughput
Disk write throughput in MB/s
9.28MB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/etc/hostname
0/1hosts
Mount Point
/etc/hosts
0/1hosts
Mount Point
/etc/resolv.conf
0/1hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
1.23
5 min
0.88
15 min
1.20
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
0.00%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0%
<
mean
0.2594%
(SD =
0.3688%)
<
p95
1.16%
60 second average
0.2678%
300 second average
0.2266%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
6.55%
<
mean
9.2225%
(SD =
2.6901%)
<
p95
14.41%
60 second average
8.2331%
300 second average
8.1512%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
6.16%
<
mean
8.8653%
(SD =
2.5794%)
<
p95
14.03%
60 second average
7.9222%
300 second average
7.7922%
Holochain process CPU usage
CPU usage by Holochain process
p5
0%
<
mean
26.43%
(SD =
7.52%)
<
p95
31.87%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5 235.41 KiB
<
mean 243.52 KiB
(SD = 15.14 KiB)
<
p95257.22 KiB
Holochain process threads
Number of threads in Holochain process
p5
1 threads
<
mean
13.19 threads
(SD =
3.21 threads)
<
p95
14 threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
0 file descriptors
<
mean
57.47 file descriptors
(SD =
15.04 file descriptors)
<
p95
66 file descriptors
Write/Read
Creates an entry and reads it back, then repeats.
Started
Wed, 18 Mar 2026 17:16:20 UTC
Peer count
1
Peer count at end
1
Behaviours
-
default(1 agent)
Holochain version
0.6.1-rc.3
Wind Tunnel version
0.6.0
Run ID
write_read_23256907830
Write timing
The time taken to write the entry.
mean
0.025762s
(SD =
0.033012s)
p50
0.024091s
p95
0.043558s
p99
0.059892s
Write rate
The number of writes completed per second.
mean
161.74/s
Read timing
The time taken to read the entry.
mean
0.032461s
(SD =
0.020629s)
p50
0.028002s
p95
0.069975s
p99
0.081341s
Read rate
The number of reads completed per second.
mean
161.74/s
Error count
The number of errors accumulated over the run.
0
Host Metrics
User CPU usage
CPU usage by user space
38.23%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5 809 B/s
<
mean 1.10 KiB/s
(SD = 661 B/s)
<
p951.70 KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5 1.71 KiB/s
<
mean 2.14 KiB/s
(SD = 776 B/s)
<
p953.45 KiB/s
Total bytes received
Total bytes received on primary network interface
count
331.27 KiB
mean
1.10 KiB/s
Total bytes sent
Total bytes sent on primary network interface
count
643.15 KiB
mean
2.14 KiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 98.7%
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 4.58 MB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
NotDetected
Additional Host Metrics
Additional Host Metrics
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
39.74%
<
mean
48.8%
(SD =
9.84%)
<
p95
52.76%
System
10.57%
CPU percentiles
CPU usage percentiles
p50
48.49%
p95
52.76%
p99
98.70%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
1 hosts
mean time
0.03s
Memory used percentage
Percentage of memory used
p5
14.15%
<
mean
15.42%
(SD =
0.6%)
<
p95
16.13%
Memory available percentage
Percentage of available memory
p5
83.87%
<
mean
84.58%
(SD =
0.6%)
<
p95
85.85%
Max host memory used
Maximum memory usage percentage across all hosts
max
15.42%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.05%
Memory growth rate
Rate of memory growth over time
growth
4.58MB/s
Disk read throughput
Disk read throughput in MB/s
0.00MB/s
Disk write throughput
Disk write throughput in MB/s
6.70MB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/etc/hostname
0/1hosts
Mount Point
/etc/hosts
0/1hosts
Mount Point
/etc/resolv.conf
0/1hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
2.03
5 min
1.23
15 min
1.48
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
0.00%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
3.1%
<
mean
6.6194%
(SD =
6.9772%)
<
p95
19.11%
60 second average
6.2071%
300 second average
4.6819%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0%
<
mean
0.721%
(SD =
1.0489%)
<
p95
2.76%
60 second average
0.5774%
300 second average
0.4597%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0%
<
mean
0.6432%
(SD =
1.0126%)
<
p95
2.64%
60 second average
0.5019%
300 second average
0.3716%
Holochain process CPU usage
CPU usage by Holochain process
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5 192.62 KiB
<
mean 231.73 KiB
(SD = 17.31 KiB)
<
p95260.41 KiB
Holochain process threads
Number of threads in Holochain process
p5
1 threads
≤
mean
1 threads
(SD =
0 threads)
≤
p95
1 threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
0 file descriptors
≤
mean
0 file descriptors
(SD =
0 file descriptors)
≤
p95
0 file descriptors
Write validated
Creates an entry, updates it and links the create to the update, then repeats. Each of the three actions is validated by the included hApp.
Started
Wed, 18 Mar 2026 17:18:14 UTC
Peer count
1
Peer count at end
1
Behaviours
-
default(1 agent)
Holochain version
0.6.1-rc.3
Wind Tunnel version
0.6.0
Run ID
write_validated_23256907830
Write timing
The time it takes to complete the zome function call that writes the entry, updates it, and links to the update.
mean
0.092547s
(SD =
0.241182s)
p50
0.034235s
p95
0.353858s
p99
1.572912s
Write rate
The number of write zome function calls that complete per second.
mean
39.39/s
Read timing
The time it takes to complete the zome function call that reads the data.
mean
0.153357s
(SD =
0.340327s)
p50
0.057141s
p95
0.447799s
p99
2.000735s
Read rate
The number of read zome function calls that complete per second.
mean
39/s
Error count
The number of errors accumulated over the run.
0
Host Metrics
User CPU usage
CPU usage by user space
18.34%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5 36 B/s
<
mean 366 B/s
(SD = 826 B/s)
<
p95952 B/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5 21 B/s
<
mean 386 B/s
(SD = 824 B/s)
<
p95845 B/s
Total bytes received
Total bytes received on primary network interface
count
107.25 KiB
mean
366 B/s
Total bytes sent
Total bytes sent on primary network interface
count
112.95 KiB
mean
386 B/s
CPU spike anomaly
CPU spike anomaly was detected
NotDetected
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 130.70 MB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
NotDetected
System overload anomaly
System overload anomaly was detected
NotDetected
Additional Host Metrics
Additional Host Metrics
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
12.83%
<
mean
22.14%
(SD =
10.51%)
<
p95
49.21%
System
3.80%
CPU percentiles
CPU usage percentiles
p50
19.14%
p95
49.21%
p99
61.13%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
0 hosts
mean time
0.00s
Memory used percentage
Percentage of memory used
p5
19.15%
<
mean
21.45%
(SD =
1.24%)
<
p95
22.1%
Memory available percentage
Percentage of available memory
p5
77.9%
<
mean
78.55%
(SD =
1.24%)
<
p95
80.85%
Max host memory used
Maximum memory usage percentage across all hosts
max
21.45%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
0.00%
Memory growth rate
Rate of memory growth over time
growth
130.70MB/s
Disk read throughput
Disk read throughput in MB/s
0.00MB/s
Disk write throughput
Disk write throughput in MB/s
9.72MB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/etc/hostname
0/1hosts
Mount Point
/etc/hosts
0/1hosts
Mount Point
/etc/resolv.conf
0/1hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
1.78
5 min
0.97
15 min
0.90
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
0.00%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0.14%
<
mean
0.4458%
(SD =
0.1585%)
<
p95
0.77%
60 second average
0.3603%
300 second average
0.1642%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
11.68%
<
mean
27.9923%
(SD =
8.2016%)
<
p95
39.99%
60 second average
24.1384%
300 second average
13.6513%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
11.39%
<
mean
27.4535%
(SD =
8.1043%)
<
p95
39.47%
60 second average
23.6745%
300 second average
13.4061%
Holochain process CPU usage
CPU usage by Holochain process
p5
11.39%
<
mean
19.39%
(SD =
8.37%)
<
p95
35.48%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5 471.06 KiB
≤
mean 469.86 KiB
(SD = 29.36 KiB)
<
p95477.28 KiB
Holochain process threads
Number of threads in Holochain process
p5
13 threads
<
mean
14 threads
(SD =
2.53 threads)
<
p95
15 threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
60 file descriptors
<
mean
60.74 file descriptors
(SD =
11.2 file descriptors)
<
p95
66 file descriptors
Write Validated must_get_agent_activity
A scenario where write agents create entries in batches of 10, while must_get_agent_activity agents each pick a random write agent and repeatedly attempt to create an entry that references the chain top of their latest batch. This reference means that the entry's validation function needs to make a must_get_agent_activity call.
The purpose of this scenario is to measure the time it takes for published agent activity data to be gossiped among authorities and become available to peers that query it via must_get_agent_activity.
This test is similar to Mixed-Arc must_get_agent_activity, but all agents are full-arc.
Started
Wed, 18 Mar 2026 18:02:55 UTC
Peer count
65
Peer count at end
4225
Behaviours
-
must_get_agent_activity(1 agent) -
write(1 agent)
Holochain version
0.6.1-rc.3
Wind Tunnel version
0.6.0
Run ID
write_validated_must_get_agent_activity_23256907830
Highest observed
action_seqThe maximum chain length observed per
write agent, aggregated across all reading agents. The reading dimension is collapsed by taking the maximum observed value for each write agent (any successful read counts as propagation). This reflects the DHT’s ability to propagate agent activity ops and make them available to querying peers.Mean max (across write agents)
Mean of the highest chain head value observed per write agent. Each write agent contributes its maximum — the value seen by any reader.
778.3
Max
Highest chain head value observed for any single write agent.
1413
Write agents observed
10
Chain batch delay timing
The time between a
write agent's creation of a batch and a must_get_agent_activity agent's successful discovery of the batch and creation/self-validation of a new entry that depends on it.Mean
mean
374.230388s
(SD =
74.076455s)
Max
Highest per-partition mean latency
max mean
855.810714s
Min
Lowest per-partition mean latency
min mean
63.18901s
Chain batch delay rate
The rate at which a
must_get_agent_activity agent was able to discover batches and create/self-validate new entries that depend on them.Mean
mean
3.56/s
Max
Highest per-partition mean rate
max
7.73/s
Min
Lowest per-partition mean rate
min
0/s
create_validated_sample_entry zome call timingThe time taken to complete the zome function call that creates the entry that depends on a
write agent's source chain.Mean
mean
0.636488s
(SD =
0.280366s)
Max
Highest per-partition mean latency
max mean
2.065181s
Min
Lowest per-partition mean latency
min mean
0.039252s
Retrieval errors
A running accumulation of the errors encountered by an agent when attempting to self-validate actions that depend on
must_get_agent_activity calls.Total
14243
Agents affected
39 / 39
Mean per agent
Total count divided by number of partitions, rounded to nearest whole number
365
Max
Highest count in any single partition
805
Min
Lowest count in any single partition
8
Final error count
The total number of all types of error accumulated over the run by all agents.
61281
Host Metrics
User CPU usage
CPU usage by user space
47.81%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5 37.27 KiB/s
<
mean 390.63 KiB/s
(SD = 873.01 KiB/s)
<
p951.25 MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5 31.13 KiB/s
<
mean 228.41 KiB/s
(SD = 284.52 KiB/s)
<
p95734.98 KiB/s
Total bytes received
Total bytes received on primary network interface
count
20.20 GiB
mean
20.89 MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
11.80 GiB
mean
12.20 MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 97.7%
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 1097.10 MB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected
Critical Heavy swap usage (21.2% swap used)
System overload anomaly
System overload anomaly was detected
Detected
Warning 81% of hosts overloaded (load5/ncpus > 1.0)
Additional Host Metrics
Additional Host Metrics
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
11.42%
<
mean
68.52%
(SD =
29.71%)
<
p95
96.35%
System
20.71%
CPU percentiles
CPU usage percentiles
p50
82.44%
p95
96.35%
p99
97.66%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
53 hosts
mean time
0.64s
Memory used percentage
Percentage of memory used
p5
11.15%
<
mean
19.34%
(SD =
6.29%)
<
p95
29.84%
Memory available percentage
Percentage of available memory
p5
70.16%
<
mean
80.66%
(SD =
6.29%)
<
p95
88.86%
Max host memory used
Maximum memory usage percentage across all hosts
max
44.30%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
21.17%
Memory growth rate
Rate of memory growth over time
growth
1097.10MB/s
Disk read throughput
Disk read throughput in MB/s
0.03MB/s
Disk write throughput
Disk write throughput in MB/s
238.02MB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/
0/60hosts
Mount Point
/boot
0/1hosts
Mount Point
/efi-boot
0/12hosts
Mount Point
/etc/hostname
0/3hosts
Mount Point
/etc/hosts
0/3hosts
Mount Point
/etc/resolv.conf
0/3hosts
Mount Point
/nix/store
0/15hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
5.56
5 min
4.42
15 min
2.83
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
80.95%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0.07%
<
mean
50.7344%
(SD =
36.4137%)
<
p95
95.69%
60 second average
48.8748%
300 second average
38.3669%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
<
mean
0.0004%
(SD =
0.0191%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0.01%
<
mean
16.9408%
(SD =
19.722%)
<
p95
62.23%
60 second average
16.8198%
300 second average
14.7463%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0%
<
mean
10.1841%
(SD =
16.1091%)
<
p95
49.55%
60 second average
10.2632%
300 second average
9.5428%
Holochain process CPU usage
CPU usage by Holochain process
p5
0%
<
mean
20.41%
(SD =
20.33%)
<
p95
61.56%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5 2.21 KiB
<
mean 258.58 KiB
(SD = 187.51 KiB)
<
p95575.80 KiB
Holochain process threads
Number of threads in Holochain process
p5
1 threads
<
mean
23.51 threads
(SD =
18.78 threads)
<
p95
56 threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
0 file descriptors
<
mean
60.82 file descriptors
(SD =
34.42 file descriptors)
<
p95
97 file descriptors
Zero-Arc Create and Read
A zero-arc/full-arc mixed scenario with two types of zero-arc nodes -- ones that create data and ones that read data -- as well as full arc nodes to "relay" the data. The scenario has three roles:
zero_write: A zero-arc conductor that just creates entries with a timestamp field. Those entries are linked to a known base hash so thatzero_readnodes can retrieve them.zero_read: A zero-arc conductor that reads the entries created by the zero-arc node(s) and records the time lag between when the entry had been created and when it was first discovered.full: A full-arc conductor that is just here to serve entries to zero arc nodes.
Started
Wed, 18 Mar 2026 18:15:34 UTC
Peer count
65
Peer count at end
4225
Behaviours
-
full(1 agent) -
zero_read(1 agent) -
zero_write(1 agent)
Holochain version
0.6.1-rc.3
Wind Tunnel version
0.6.0
Run ID
zero_arc_create_and_read_23256907830
Create rate
The number of timed entries created by the zero-arc write node(s) per second.
Mean
mean
45.62/s
Max
Highest per-partition mean rate
max
109.6/s
Min
Lowest per-partition mean rate
min
10.61/s
Fetch lag timing
For each entry, the time lag between when it was created and when a zero-arc read node fetched it from the network via the
get_timed_entries_network zome function.Mean
mean
277.240077s
(SD =
13.248703s)
Max
Highest per-partition mean latency
max mean
903.090865s
Min
Lowest per-partition mean latency
min mean
169.386786s
Fetch lag rate
The number of entries per second fetched by zero-arc read nodes.
Mean
mean
0.18/s
Max
Highest per-partition mean rate
max
1/s
Min
Lowest per-partition mean rate
min
0/s
Open connections
The number of currently open connections to other conductors.
full-arc
p5
32
<
mean
56.1
(SD =
11.89)
<
p95
63
zero-arc
p5
12
<
mean
34.54
(SD =
10.51)
<
p95
45
Error count
The number of errors accumulated across all nodes.
325
Zero-Arc Create Data
A zero-arc/full-arc mixed scenario where zero-arc nodes create data and full-arc nodes read the data. The scenario has two roles:
zero: A zero-arc conductor that just creates entries with a timestamp field. Those entries are linked to a known base hash so that full-arc nodes can retrieve them.full: A full-arc conductor that reads the entries created by the zero-arc node(s) and records the time lag between when the entry had been created and when it was first discovered.
Started
Wed, 18 Mar 2026 18:09:24 UTC
Peer count
65
Peer count at end
4225
Behaviours
-
full(1 agent) -
zero(1 agent)
Holochain version
0.6.1-rc.3
Wind Tunnel version
0.6.0
Run ID
zero_arc_create_data--high_zero_23256907830
Create rate
The number of timed entries created by the zero-arc node(s) per second.
Mean
mean
55.59/s
Max
Highest per-partition mean rate
max
106.64/s
Min
Lowest per-partition mean rate
min
16.06/s
Sync lag timing
For each entry, the time lag between when it was created and when the full-arc node could read it via the
get_timed_local_entries zome function.Mean
mean
498.259173s
(SD =
63.955209s)
Max
Highest per-partition mean latency
max mean
605.143647s
Min
Lowest per-partition mean latency
min mean
320.320155s
Sync lag rate
The number of entries per second received by full nodes.
Mean
mean
5.75/s
Max
Highest per-partition mean rate
max
17/s
Min
Lowest per-partition mean rate
min
0/s
Open connections
The number of currently open connections to other conductors.
full-arc
p5
10
<
mean
50.23
(SD =
17.37)
<
p95
63
zero-arc
p5
6
<
mean
27.47
(SD =
11.59)
<
p95
41
Error count
The number of errors accumulated across all nodes.
3
Host Metrics
User CPU usage
CPU usage by user space
57.47%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5 22.99 KiB/s
<
mean 366.11 KiB/s
(SD = 976.56 KiB/s)
<
p951.59 MiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5 30.31 KiB/s
<
mean 174.16 KiB/s
(SD = 198.23 KiB/s)
<
p95514.30 KiB/s
Total bytes received
Total bytes received on primary network interface
count
19.63 GiB
mean
21.16 MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
9.34 GiB
mean
10.07 MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 98.0%
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 1378.40 MB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected
Critical Heavy swap usage (21.2% swap used)
System overload anomaly
System overload anomaly was detected
Detected
Warning 87% of hosts overloaded (load5/ncpus > 1.0)
Additional Host Metrics
Additional Host Metrics
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
20.13%
<
mean
78.15%
(SD =
24.61%)
<
p95
97%
System
20.68%
CPU percentiles
CPU usage percentiles
p50
88.72%
p95
97.00%
p99
98.05%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
56 hosts
mean time
0.77s
Memory used percentage
Percentage of memory used
p5
8.65%
<
mean
17.64%
(SD =
7.71%)
<
p95
28.68%
Memory available percentage
Percentage of available memory
p5
71.32%
<
mean
82.36%
(SD =
7.71%)
<
p95
91.35%
Max host memory used
Maximum memory usage percentage across all hosts
max
52.06%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
21.17%
Memory growth rate
Rate of memory growth over time
growth
1378.40MB/s
Disk read throughput
Disk read throughput in MB/s
0.21MB/s
Disk write throughput
Disk write throughput in MB/s
221.16MB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/
0/57hosts
Mount Point
/boot
0/1hosts
Mount Point
/efi-boot
0/6hosts
Mount Point
/etc/hostname
0/5hosts
Mount Point
/etc/hosts
0/5hosts
Mount Point
/etc/resolv.conf
0/5hosts
Mount Point
/nix/store
0/9hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
5.44
5 min
4.41
15 min
2.82
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
87.10%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
1.81%
<
mean
55.7217%
(SD =
33.2407%)
<
p95
96.01%
60 second average
53.5124%
300 second average
43.0023%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0.14%
<
mean
10.2401%
(SD =
13.9857%)
<
p95
46.14%
60 second average
10.2949%
300 second average
9.0929%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0%
<
mean
5.6126%
(SD =
11.9927%)
<
p95
35.29%
60 second average
5.7780%
300 second average
5.3913%
Holochain process CPU usage
CPU usage by Holochain process
p5
0%
<
mean
24.46%
(SD =
24.21%)
<
p95
76.08%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5 2.21 KiB
<
mean 231.75 KiB
(SD = 134.35 KiB)
<
p95444.38 KiB
Holochain process threads
Number of threads in Holochain process
p5
1 threads
<
mean
23.61 threads
(SD =
24.8 threads)
<
p95
60 threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
0 file descriptors
<
mean
54.61 file descriptors
(SD =
29.89 file descriptors)
<
p95
89 file descriptors
Zero-arc Create Data (Validated)
A zero-arc/full-arc mixed scenario where zero-arc nodes create data that gets validated and full-arc nodes read the data. The scenario has two roles:
zero: A zero-arc conductor that just creates entries with a timestamp field. Those entries are linked to a known base hash so that full-arc nodes can retrieve them.full: A full-arc conductor that reads the entries created by the zero-arc node(s) and records the time lag between when the entry had been created and when it was first discovered.
Started
Wed, 18 Mar 2026 18:13:37 UTC
Peer count
65
Peer count at end
4225
Behaviours
-
full(1 agent) -
zero(1 agent)
Holochain version
0.6.1-rc.3
Wind Tunnel version
0.6.0
Run ID
zero_arc_create_data_validated--high_zero_23256907830
Create rate
The number of timed entries created by the zero-arc node(s) per second.
Mean
mean
42.03/s
Max
Highest per-partition mean rate
max
103.4/s
Min
Lowest per-partition mean rate
min
9.21/s
Sync lag timing
For each entry, the time lag between when it was created and when the full-arc node could read it via the
get_timed_local_entries zome function.Mean
mean
628.237668s
(SD =
39.653472s)
Max
Highest per-partition mean latency
max mean
755.199226s
Min
Lowest per-partition mean latency
min mean
436.505782s
Sync lag rate
The number of entries per second received by full nodes.
Mean
mean
4.88/s
Max
Highest per-partition mean rate
max
17/s
Min
Lowest per-partition mean rate
min
0/s
Open connections
The number of currently open connections to other conductors.
full-arc
p5
4
<
mean
22.19
(SD =
15.69)
<
p95
60
zero-arc
p5
4
<
mean
16.47
(SD =
10.14)
<
p95
37
Error count
The number of errors accumulated across all nodes.
5
Host Metrics
User CPU usage
CPU usage by user space
56.54%
Network receive rate (primary)
Rate of bytes received on primary network interface
p5 3.78 KiB/s
<
mean 246.95 KiB/s
(SD = 837.71 KiB/s)
<
p95649.72 KiB/s
Network send rate (primary)
Rate of bytes sent on primary network interface
p5 3.60 KiB/s
<
mean 120.67 KiB/s
(SD = 140.91 KiB/s)
<
p95393.54 KiB/s
Total bytes received
Total bytes received on primary network interface
count
13.21 GiB
mean
13.95 MiB/s
Total bytes sent
Total bytes sent on primary network interface
count
6.45 GiB
mean
6.81 MiB/s
CPU spike anomaly
CPU spike anomaly was detected
Detected
Warning CPU p99 reached 98.3%
Memory leak anomaly
Memory leak anomaly was detected
Detected
Warning Memory growing at 1515.18 MB/s
Disk full anomaly
Disk full anomaly was detected
NotDetected
Swap thrashing anomaly
Swap thrashing anomaly was detected
Detected
Warning Moderate swap usage (14.1% swap used)
System overload anomaly
System overload anomaly was detected
Detected
Warning 85% of hosts overloaded (load5/ncpus > 1.0)
Additional Host Metrics
Additional Host Metrics
CPU usage
Total CPU usage and kernel CPU usage
Total
p5
15.01%
<
mean
78.74%
(SD =
25.64%)
<
p95
97.35%
System
22.20%
CPU percentiles
CPU usage percentiles
p50
90.08%
p95
97.35%
p99
98.32%
CPU usage above 80%
Number of hosts above 80% CPU and mean time spent above threshold for those hosts
count
57 hosts
mean time
0.79s
Memory used percentage
Percentage of memory used
p5
10.96%
<
mean
19.39%
(SD =
7.4%)
<
p95
31.39%
Memory available percentage
Percentage of available memory
p5
68.61%
<
mean
80.61%
(SD =
7.4%)
<
p95
89.04%
Max host memory used
Maximum memory usage percentage across all hosts
max
55.78%
Max host swap used percentage
Maximum swap space usage percentage across all hosts
max
14.10%
Memory growth rate
Rate of memory growth over time
growth
1515.18MB/s
Disk read throughput
Disk read throughput in MB/s
0.02MB/s
Disk write throughput
Disk write throughput in MB/s
161.89MB/s
Disk space utilization risk
Number of hosts nearing disk space capacity by mount point
Mount Point
/
0/56hosts
Mount Point
/efi-boot
0/6hosts
Mount Point
/etc/hostname
0/6hosts
Mount Point
/etc/hosts
0/6hosts
Mount Point
/etc/resolv.conf
0/6hosts
Mount Point
/nix/store
0/7hosts
System load average
System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.
1 min
5.72
5 min
4.79
15 min
3.15
CPU overloaded hosts
Percentage of hosts that experienced CPU overload
85.48%
CPU pressure
CPU pressure over 10, 60, and 300 second averages
10 second average
p5
0.27%
<
mean
60.8101%
(SD =
33.0272%)
<
p95
95.79%
60 second average
59.0395%
300 second average
49.2402%
Memory pressure some
Memory pressure (some tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
Memory pressure full
Memory pressure (all tasks stalled) over 10, 60, and 300 second averages
10 second average
p5
0%
≤
mean
0%
(SD =
0%)
≤
p95
0%
60 second average
0.0000%
300 second average
0.0000%
I/O pressure some
I/O pressure (some tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0.15%
<
mean
9.8838%
(SD =
14.6036%)
<
p95
51.03%
60 second average
9.9892%
300 second average
9.7729%
I/O pressure full
I/O pressure (all tasks stalled) over 10, 60 and 300 second averages
10 second average
p5
0%
<
mean
5.3768%
(SD =
12.8856%)
<
p95
41.7%
60 second average
5.5096%
300 second average
5.7199%
Holochain process CPU usage
CPU usage by Holochain process
p5
0.18%
<
mean
28.33%
(SD =
26.76%)
<
p95
83.11%
Holochain process memory (PSS)
Proportional Set Size memory of Holochain process
p5 2.17 KiB
<
mean 274.48 KiB
(SD = 178.16 KiB)
<
p95546.60 KiB
Holochain process threads
Number of threads in Holochain process
p5
1 threads
<
mean
22.63 threads
(SD =
21.64 threads)
<
p95
49 threads
Holochain process file descriptors
Number of file descriptors used by Holochain process
p5
4 file descriptors
<
mean
57.15 file descriptors
(SD =
28.41 file descriptors)
<
p95
88 file descriptors