App Install

Measure app installation speed. This scenario has two roles:

minimal: Installs a simple app which implements initialisation callbacks but otherwise doesn't contain a lot of code.
large: Installs a larger app that contains some dependencies and some generated data as padding to make the bundle larger.

These are intended to catch a compilation slowdown or issues with copying around large WASMs (e.g. accidental cloning or tracing). In each case, the behaviour will uninstall the app it installed so that it can re-install it on the next iteration.

Started

Thu, 03 Sep 2026 09:01:42 UTC

Duration

300s

Peer count

1

Behaviours

large (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

app_install_large_33713946045_1

First install time

The time from conductor startup to completion of the first app installation attempt.

0.130775004

s

Install app timing

The time taken to install an app.

mean

0.067554

s

SD

0.004297

s

p50

0.067105

s

<

p95

0.075067

s

<

p99

0.076814

s

Holochain MetricsClick to toggle visibility.

WASM usage: 'large::init'

Metered usage count of the WASM ribosome for this zome function.

total

0

over 10.002175164s

Not enough time data to show a trend.

WASM call duration: 'large::init'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.004758

s

SD

0.001727

s

p50

0.004359

s

<

p95

0.008727

s

<

p99

0.009081

s

Post-commit duration

Time spent executing post-commit workflows.

mean

0.000186

s

SD

0.000258

s

p50

8.3e-05

s

<

p95

0.000833

s

<

p99

0.001049

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

7.18

s

<

mean

17.18

s

SD

8.17

s

<

p95

27.18

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

0

over 20.002722064s

Not enough time data to show a trend.

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.001529

s

SD

0.000414

s

p50

0.001539

s

<

p95

0.002373

s

≤

p99

0.002373

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.001764

s

SD

0.000203

s

p50

0.001711

s

<

p95

0.001795

s

≤

p99

0.001795

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.005756

s

SD

0.002714

s

p50

0.005752

s

<

p95

0.011468

s

≤

p99

0.011468

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

0.002788

s

SD

0.000229

s

p50

0.0026

s

<

p95

0.002894

s

≤

p99

0.002894

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

0.001235

s

SD

0.000273

s

p50

0.001235

s

<

p95

0.001272

s

≤

p99

0.001272

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

7.9e-05

s

SD

1.4e-05

s

p50

7.9e-05

s

<

p95

0.000108

s

≤

p99

0.000108

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.001618

s

SD

0.000371

s

p50

0.001724

s

<

p95

0.002268

s

≤

p99

0.002268

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000461

s

SD

1.8e-05

s

p50

0.000457

s

<

p95

0.000463

s

≤

p99

0.000463

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

67.92

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

86.75

KiB/s

<

mean

2.69

MiB/s

SD

3.33

MiB/s

<

p95

7.40

MiB/s

Not enough time data to show a trend.

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

35.52

KiB/s

<

mean

75.67

KiB/s

SD

55.88

KiB/s

<

p95

154.70

KiB/s

Not enough time data to show a trend.

Total bytes received

Total bytes received on primary network interface

count

80.83

MiB

mean

2.69

MiB/s

Total bytes sent

Total bytes sent on primary network interface

count

2.22

MiB

mean

75.67

KiB/s

CPU spike anomaly

CPU spike anomaly was detected

NotDetected

Memory leak anomaly

Memory leak anomaly was detected

Detected

Warning Memory growing at 7.19 MiB/s

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

NotDetected

System overload anomaly

System overload anomaly was detected

Detected

Warning 100% of hosts overloaded (load5/ncpus > 1.0)

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

85.32

%

<

mean

87.76

%

SD

1.74

%

<

p95

89.33

%

Not enough time data to show a trend.

System

19.83

%

CPU percentiles

CPU usage percentiles

p50

88.61

%

p95

89.33

%

p99

89.33

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

1

hosts

mean time

300.00

s

Memory used percentage

Percentage of memory used

p5

9.45

%

<

mean

11.78

%

SD

1.93

%

<

p95

14.7

%

Memory available percentage

Percentage of available memory

p5

85.3

%

<

mean

88.22

%

SD

1.93

%

<

p95

90.55

%

Max host memory used

Maximum memory usage percentage across all hosts

max

11.78

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

0.00

%

Memory growth rate

Rate of memory growth over time

growth

7.19

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.00

MiB/s

Disk write throughput

Disk write throughput in MiB/s

7.16

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /etc/nomad.d/client-meta.json

0/1

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

5.74

5 min

4.04

15 min

2.00

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

100.00

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

25.27

%

<

mean

39.66

%

SD

8.6935

%

<

p95

47.96

%

60 second average

38.3775

%

300 second average

24.7625

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

Holochain process CPU usage

CPU usage by Holochain process

p5

14.23

%

<

mean

22.78

%

SD

6.74

%

<

p95

30.7

%

Not enough time data to show a trend.

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

138.15

KiB

<

mean

681.95

KiB

SD

589.14

KiB

<

p95

1.58

MiB

Holochain process threads

Number of threads in Holochain process

p5

25

threads

<

mean

116

threads

SD

105.93

threads

<

p95

285

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

61

file descriptors

<

mean

253.5

file descriptors

SD

224.24

file descriptors

<

p95

610

file descriptors

App Install

↑ Back to index

Measure app installation speed. This scenario has two roles:

minimal: Installs a simple app which implements initialisation callbacks but otherwise doesn't contain a lot of code.
large: Installs a larger app that contains some dependencies and some generated data as padding to make the bundle larger.

These are intended to catch a compilation slowdown or issues with copying around large WASMs (e.g. accidental cloning or tracing). In each case, the behaviour will uninstall the app it installed so that it can re-install it on the next iteration.

Started

Thu, 03 Sep 2026 09:03:27 UTC

Duration

300s

Peer count

1

Behaviours

minimal (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

app_install_minimal_33713946045_1

First install time

The time from conductor startup to completion of the first app installation attempt.

0.410306385

s

Install app timing

The time taken to install an app.

mean

0.05499

s

SD

0.081058

s

p50

0.041522

s

<

p95

0.058226

s

<

p99

0.597235

s

Holochain MetricsClick to toggle visibility.

WASM usage: 'callback::init'

Metered usage count of the WASM ribosome for this zome function.

total

0

over 10.001126255s

Not enough time data to show a trend.

WASM usage: 'callback_integrity::genesis_self_check_2'

Metered usage count of the WASM ribosome for this zome function.

total

115209

over 10.001126255s

mean rate

11519.6

/s

std

0

/s

p5

11519.6

/s

p95

11519.6

/s

peak

11519.6

/s

Not enough time data to show a trend.

WASM call duration: 'callback::init'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.001016

s

SD

0.000249

s

p50

0.000946

s

<

p95

0.001923

s

≤

p99

0.001923

s

WASM call duration: 'callback_integrity::genesis_self_check_2'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.001011

s

SD

3.4e-05

s

p50

0.001018

s

≤

p95

0.001018

s

≤

p99

0.001018

s

Post-commit duration

Time spent executing post-commit workflows.

mean

7.2e-05

s

SD

2.7e-05

s

p50

6.4e-05

s

<

p95

0.000174

s

≤

p99

0.000174

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

6.62

s

<

mean

11.62

s

SD

5

s

<

p95

16.62

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

0

over 10.001018468s

Not enough time data to show a trend.

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.000954

s

SD

0.000117

s

p50

0.000975

s

≤

p95

0.000975

s

≤

p99

0.000975

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.001227

s

SD

0.000119

s

p50

0.001443

s

≤

p95

0.001443

s

≤

p99

0.001443

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.004349

s

SD

7.1e-05

s

p50

0.004362

s

≤

p95

0.004362

s

≤

p99

0.004362

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

0.00239

s

SD

0.000189

s

p50

0.002735

s

≤

p95

0.002735

s

≤

p99

0.002735

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

0.00114

s

SD

0.000358

s

p50

0.001203

s

≤

p95

0.001203

s

≤

p99

0.001203

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

3.6e-05

s

SD

5e-06

s

p50

4.5e-05

s

≤

p95

4.5e-05

s

≤

p99

4.5e-05

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.001077

s

SD

0.00027

s

p50

0.001352

s

<

p95

0.00154

s

≤

p99

0.00154

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000305

s

SD

6e-06

s

p50

0.000315

s

≤

p95

0.000315

s

≤

p99

0.000315

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

53.07

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

61.49

KiB/s

<

mean

4.18

MiB/s

SD

4.12

MiB/s

<

p95

8.30

MiB/s

Not enough time data to show a trend.

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

44.68

KiB/s

<

mean

92.11

KiB/s

SD

47.43

KiB/s

<

p95

139.54

KiB/s

Not enough time data to show a trend.

Total bytes received

Total bytes received on primary network interface

count

83.63

MiB

mean

4.18

MiB/s

Total bytes sent

Total bytes sent on primary network interface

count

1.80

MiB

mean

92.11

KiB/s

CPU spike anomaly

CPU spike anomaly was detected

NotDetected

Memory leak anomaly

Memory leak anomaly was detected

Detected

Warning Memory growing at 5.88 MiB/s

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

NotDetected

System overload anomaly

System overload anomaly was detected

NotDetected

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

61.06

%

<

mean

69.41

%

SD

8.35

%

<

p95

77.75

%

Not enough time data to show a trend.

System

16.33

%

CPU percentiles

CPU usage percentiles

p50

77.75

%

p95

77.75

%

p99

77.75

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

0

hosts

mean time

0.00

s

Memory used percentage

Percentage of memory used

p5

22.4

%

<

mean

28.41

%

SD

6.47

%

<

p95

37.39

%

Not enough time data to show a trend.

Memory available percentage

Percentage of available memory

p5

62.61

%

<

mean

71.59

%

SD

6.47

%

<

p95

77.6

%

Not enough time data to show a trend.

Max host memory used

Maximum memory usage percentage across all hosts

max

28.41

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

0.00

%

Memory growth rate

Rate of memory growth over time

growth

5.88

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.00

MiB/s

Disk write throughput

Disk write throughput in MiB/s

11.93

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /etc/nomad.d/client-meta.json

0/1

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

1.55

5 min

1.53

15 min

1.12

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

0.00

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

9.03

%

<

mean

18.2633

%

SD

6.5296

%

<

p95

22.99

%

Not enough time data to show a trend.

60 second average

10.6567

%

300 second average

14.6533

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

Not enough time data to show a trend.

60 second average

0.0000

%

300 second average

0.0000

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

Not enough time data to show a trend.

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0.06

%

<

mean

0.37

%

SD

0.3569

%

<

p95

0.87

%

Not enough time data to show a trend.

60 second average

0.4900

%

300 second average

0.5633

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0.05

%

<

mean

0.1233

%

SD

0.0544

%

<

p95

0.18

%

Not enough time data to show a trend.

60 second average

0.3133

%

300 second average

0.3233

%

Holochain process CPU usage

CPU usage by Holochain process

p5

1.06

%

<

mean

23.74

%

SD

22.68

%

<

p95

46.42

%

Not enough time data to show a trend.

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

125.45

KiB

<

mean

490.14

KiB

SD

460.03

KiB

<

p95

1.11

MiB

Not enough time data to show a trend.

Holochain process threads

Number of threads in Holochain process

p5

27

threads

<

mean

96.33

threads

SD

97.35

threads

<

p95

234

threads

Not enough time data to show a trend.

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

63

file descriptors

<

mean

214

file descriptors

SD

206.52

file descriptors

<

p95

506

file descriptors

Not enough time data to show a trend.

DHT Sync Lag

↑ Back to index

Measures the lag time between an agent publishing data and other peers being able to see it. This scenario has two roles:

write: Creates entries with a timestamp field, linked to a known base hash so that record_lag agents can discover them.
record_lag: Repeatedly queries for links from the known base hash. When a new entry is discovered, it calculates the time difference between the entry's timestamp and the current time, giving the sync lag for that entry.

Started

Thu, 03 Sep 2026 09:05:16 UTC

Duration

1800s

Peer count

30

Peer count at end

29

Behaviours

record_lag (1 agent)
write (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

dht_sync_lag_33713946045_1

Create rate

The average number of created records per agent.

Mean

mean

21.96

/s

Max

Highest per-partition mean rate

max

38.2

/s

Min

Lowest per-partition mean rate

min

9.86

/s

Sync lag timing

The average time between when a record was created and when it is first seen by an agent.

Mean

mean

1.97330884333e+30

s

SD

1.1419391095100002e+31

s

Max

Highest per-partition mean latency

max mean

9.86654421667e+30

s

Min

Lowest per-partition mean latency

min mean

20.419211

s

Sync lag rate

The average number of created records discovered per agent.

Mean

mean

56.98

/s

Max

Highest per-partition mean rate

max

218.4

/s

Min

Lowest per-partition mean rate

min

0

/s

Delivery ratio

The proportion of created entries successfully received by all record_lag agents.

0.0147

Sent count

The total number of entries sent (created) by write agents, by agent.

Total

98326

Agents affected

24 / 24

Mean per agent

Total count divided by number of partitions, rounded to nearest whole number

4097

Max

Highest count in any single partition

7018

Min

Lowest count in any single partition

1794

Received count

The total number of entries received by record_lag agents, by agent.

Total

7237

Agents affected

5 / 5

Mean per agent

Total count divided by number of partitions, rounded to nearest whole number

1447

Max

Highest count in any single partition

6001

Min

Lowest count in any single partition

2

Error count

The number of errors encountered during the scenario.

13

Holochain MetricsClick to toggle visibility.

Cascade duration

Time taken to execute a cascade (get) query inside Holochain.

mean

0.000428

s

SD

0.000355

s

p50

0.00074

s

<

p95

0.0012

s

<

p99

0.001263

s

Cascade fetch errors

Count of network fetch errors during cascade calls.

total

1

over 519.89026774s

WASM usage: 'timed::created_timed_entry'

Metered usage count of the WASM ribosome for this zome function.

total

372368705

over 1786.829317051s

mean rate

1.99809898981e+09

/s

std

1.77726981188e+10

/s

p5

6.65900843e+06

/s

p95

3.51709715815e+09

/s

peak

4.74467640968e+11

/s

WASM usage: 'timed::get_timed_entries_local'

Metered usage count of the WASM ribosome for this zome function.

total

598933746

over 1772.261111171s

mean rate

1.10036424096e+11

/s

std

1.79206764362e+11

/s

p5

1.1719917665e+09

/s

p95

5.30106651937e+11

/s

peak

6.28362523492e+11

/s

WASM usage: 'timed_integrity::entry_defs'

Metered usage count of the WASM ribosome for this zome function.

total

47479920

over 1786.829317051s

mean rate

2.60729142e+08

/s

std

2.15705385408e+09

/s

p5

975957.96

/s

p95

4.2182493252e+08

/s

peak

6.04983322531e+10

/s

Zome call duration: 'timed::created_timed_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.014057

s

SD

0.017293

s

p50

0.009176

s

<

p95

0.050638

s

<

p99

0.084572

s

Zome call duration: 'timed::get_timed_entries_local'

Duration of this zome call as measured by Holochain internally.

mean

0.053784

s

SD

0.123101

s

p50

0.018523

s

<

p95

0.398621

s

<

p99

0.49058

s

WASM call duration: 'timed::created_timed_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.013209

s

SD

0.01702

s

p50

0.008299

s

<

p95

0.048221

s

<

p99

0.082818

s

WASM call duration: 'timed::get_timed_entries_local'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.053103

s

SD

0.123086

s

p50

0.01817

s

<

p95

0.39788

s

<

p99

0.489793

s

WASM call duration: 'timed_integrity::entry_defs'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.001905

s

SD

0.002166

s

p50

0.001415

s

<

p95

0.00753

s

<

p99

0.012385

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.002245

s

SD

0.004785

s

p50

0.000748

s

<

p95

0.012139

s

<

p99

0.023639

s

Host function call duration: '__hc__create_link_1'

Duration of this host function call invoked from within WASM.

mean

0.002176

s

SD

0.004453

s

p50

0.000747

s

<

p95

0.011761

s

<

p99

0.02112

s

Host function call duration: '__hc__get_1'

Duration of this host function call invoked from within WASM.

mean

0.000423

s

SD

0.000398

s

p50

0.000846

s

<

p95

0.001304

s

<

p99

0.001335

s

Host function call duration: '__hc__get_links_1'

Duration of this host function call invoked from within WASM.

mean

0.002842

s

SD

0.004641

s

p50

0.001576

s

<

p95

0.015263

s

<

p99

0.018912

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.003037

s

SD

0.003322

s

p50

0.002256

s

<

p95

0.012477

s

<

p99

0.015882

s

Post-commit duration

Time spent executing post-commit workflows.

mean

0.000292

s

SD

0.000445

s

p50

0.000153

s

<

p95

0.00094

s

<

p99

0.002607

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

87.74

s

<

mean

884.98

s

SD

514.84

s

<

p95

1695.79

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

18208

over 1786.829387352s

mean rate

47978.22

/s

std

102492.04

/s

p5

0

/s

p95

280831.78

/s

peak

630394.97

/s

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

145.4581

s

SD

150.820375

s

p50

0.905658

s

<

p95

280.832491

s

<

p99

715.496316

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.425277

s

SD

0.71161

s

p50

0.297435

s

<

p95

2.929107

s

<

p99

3.130011

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

<

p99

1e-06

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.314802

s

SD

0.510201

s

p50

0.350378

s

<

p95

1.405822

s

<

p99

1.935377

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.555542

s

SD

0.824884

s

p50

0.389201

s

<

p95

2.091317

s

<

p99

4.443938

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

62.166655

s

SD

72.749781

s

p50

51.823326

s

<

p95

232.353757

s

<

p99

327.025159

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

33.506398

s

SD

63.936054

s

p50

0.018361

s

<

p95

95.193044

s

<

p99

331.730218

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

0.001832

s

SD

0.00119

s

p50

0.00201

s

<

p95

0.004086

s

<

p99

0.00574

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.003706

s

SD

0.008661

s

p50

0.001668

s

<

p95

0.026009

s

<

p99

0.04914

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.001899

s

SD

0.003937

s

p50

0.000521

s

<

p95

0.010451

s

<

p99

0.01992

s

P2P request roundtrip: 'get'

Time spent sending a get request and awaiting its response.

mean

5.624319

s

SD

26.909529

s

p50

11.573553

s

<

p95

60.007513

s

<

p99

60.041118

s

P2P request roundtrip: 'send_validation_receipts'

Time spent sending a send_validation_receipts request and awaiting its response.

mean

1.516943

s

SD

23.161754

s

p50

5.001807

s

<

p95

60.007518

s

<

p99

60.008553

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

65.73

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

153.57

KiB/s

<

mean

510.43

KiB/s

SD

754.56

KiB/s

<

p95

818.89

KiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

57.55

KiB/s

<

mean

378.81

KiB/s

SD

303.99

KiB/s

<

p95

995.88

KiB/s

Total bytes received

Total bytes received on primary network interface

count

22.96

GiB

mean

13.13

MiB/s

Total bytes sent

Total bytes sent on primary network interface

count

17.04

GiB

mean

9.75

MiB/s

CPU spike anomaly

CPU spike anomaly was detected

Detected

Warning CPU p99 reached 98.5%

Memory leak anomaly

Memory leak anomaly was detected

NotDetected

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

NotDetected

System overload anomaly

System overload anomaly was detected

Detected

Warning 79% of hosts overloaded (load5/ncpus > 1.0)

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

36.34

%

<

mean

79.8

%

SD

20.54

%

<

p95

98.1

%

System

14.06

%

CPU percentiles

CPU usage percentiles

p50

86.88

%

p95

98.10

%

p99

98.50

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

25

hosts

mean time

1308.51

s

Memory used percentage

Percentage of memory used

p5

7.48

%

<

mean

22.45

%

SD

10.02

%

<

p95

42.02

%

Memory available percentage

Percentage of available memory

p5

57.98

%

<

mean

77.55

%

SD

10.02

%

<

p95

92.52

%

Max host memory used

Maximum memory usage percentage across all hosts

max

44.46

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

4.46

%

Memory growth rate

Rate of memory growth over time

growth

0.32

MiB/s

Disk read throughput

Disk read throughput in MiB/s

3.90

MiB/s

Disk write throughput

Disk write throughput in MiB/s

163.46

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /

0/3

hosts

Mount Point /boot

0/1

hosts

Mount Point /efi-boot

0/2

hosts

Mount Point /etc/hostname

0/14

hosts

Mount Point /etc/hosts

0/14

hosts

Mount Point /etc/nomad.d/client-meta.json

0/11

hosts

Mount Point /etc/resolv.conf

0/14

hosts

Mount Point /nix/store

0/3

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

5.75

5 min

5.02

15 min

3.50

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

78.57

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

0.12

%

<

mean

39.8906

%

SD

23.6638

%

<

p95

74.33

%

60 second average

38.8659

%

300 second average

33.8718

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0007

%

300 second average

0.0014

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0005

%

300 second average

0.0009

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

5.219

%

SD

10.7067

%

<

p95

30.48

%

60 second average

5.0953

%

300 second average

4.4227

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

1.466

%

SD

4.1422

%

<

p95

9.77

%

60 second average

1.4440

%

300 second average

1.2969

%

Holochain process CPU usage

CPU usage by Holochain process

p5

0

%

<

mean

40.21

%

SD

36.2

%

<

p95

95.71

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

227.40

KiB

<

mean

420.53

KiB

SD

170.53

KiB

<

p95

679.90

KiB

Holochain process threads

Number of threads in Holochain process

p5

1

threads

<

mean

34.21

threads

SD

12.96

threads

<

p95

52

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

0

file descriptors

<

mean

76.03

file descriptors

SD

25.22

file descriptors

<

p95

98

file descriptors

First Call

↑ Back to index

This scenario installs a simple app which implements the init callback, and measures the time elapsed between start of install and calling init. It will uninstall the app it installed so that it can re-install it on the next iteration. This is required to re-run the initialisation callback.

Started

Thu, 03 Sep 2026 08:46:09 UTC

Duration

300s

Peer count

1

Behaviours

default (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

first_call_33713946045_1

Zome call timing

The time elapsed between start of install and calling init.

mean

0.01159

s

SD

0.00303

s

p50

0.011037

s

<

p95

0.015711

s

<

p99

0.0262

s

Holochain MetricsClick to toggle visibility.

WASM usage: 'crud::create_sample_entry'

Metered usage count of the WASM ribosome for this zome function.

total

0

over 10.000022597s

Not enough time data to show a trend.

WASM usage: 'crud::init'

Metered usage count of the WASM ribosome for this zome function.

total

0

over 10.000022597s

Not enough time data to show a trend.

WASM usage: 'crud_integrity::entry_defs'

Metered usage count of the WASM ribosome for this zome function.

total

255316

over 10.000022597s

mean rate

25531.54

/s

std

0

/s

p5

25531.54

/s

p95

25531.54

/s

peak

25531.54

/s

Not enough time data to show a trend.

Zome call duration: 'crud::create_sample_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.00507

s

SD

0.000121

s

p50

0.005281

s

≤

p95

0.005281

s

≤

p99

0.005281

s

WASM call duration: 'crud::create_sample_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.004284

s

SD

0.00069

s

p50

0.004201

s

<

p95

0.005507

s

<

p99

0.005598

s

WASM call duration: 'crud::init'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.001322

s

SD

0.000285

s

p50

0.001261

s

<

p95

0.00254

s

≤

p99

0.00254

s

WASM call duration: 'crud_integrity::entry_defs'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.001187

s

SD

7e-06

s

p50

0.001199

s

≤

p95

0.001199

s

≤

p99

0.001199

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.000338

s

SD

5.1e-05

s

p50

0.000389

s

<

p95

0.000401

s

≤

p99

0.000401

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.002212

s

SD

2.2e-05

s

p50

0.002251

s

≤

p95

0.002251

s

≤

p99

0.002251

s

Post-commit duration

Time spent executing post-commit workflows.

mean

8.8e-05

s

SD

2.1e-05

s

p50

8.1e-05

s

<

p95

0.000148

s

≤

p99

0.000148

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

6.98

s

<

mean

11.98

s

SD

5

s

<

p95

16.98

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

0

over 10.000026945s

Not enough time data to show a trend.

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.000953

s

SD

4.5e-05

s

p50

0.000964

s

≤

p95

0.000964

s

≤

p99

0.000964

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.001298

s

SD

5.9e-05

s

p50

0.001401

s

≤

p95

0.001401

s

≤

p99

0.001401

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.004113

s

SD

0.000139

s

p50

0.004358

s

≤

p95

0.004358

s

≤

p99

0.004358

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

0.002365

s

SD

0.000289

s

p50

0.002869

s

≤

p95

0.002869

s

≤

p99

0.002869

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

0.001245

s

SD

0.000209

s

p50

0.001609

s

≤

p95

0.001609

s

≤

p99

0.001609

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

3.1e-05

s

SD

2e-06

s

p50

3.3e-05

s

≤

p95

3.3e-05

s

≤

p99

3.3e-05

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.001011

s

SD

0.000275

s

p50

0.001337

s

<

p95

0.001514

s

≤

p99

0.001514

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000276

s

SD

1.8e-05

s

p50

0.000307

s

≤

p95

0.000307

s

≤

p99

0.000307

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

41.15

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

43.16

KiB/s

<

mean

123.39

KiB/s

SD

80.23

KiB/s

<

p95

203.62

KiB/s

Not enough time data to show a trend.

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

23.00

KiB/s

<

mean

26.75

KiB/s

SD

3.75

KiB/s

<

p95

30.50

KiB/s

Not enough time data to show a trend.

Total bytes received

Total bytes received on primary network interface

count

2.41

MiB

mean

123.39

KiB/s

Total bytes sent

Total bytes sent on primary network interface

count

534.99

KiB

mean

26.75

KiB/s

CPU spike anomaly

CPU spike anomaly was detected

NotDetected

Memory leak anomaly

Memory leak anomaly was detected

NotDetected

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

NotDetected

System overload anomaly

System overload anomaly was detected

NotDetected

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

44.8

%

<

mean

52.97

%

SD

8.17

%

<

p95

61.14

%

Not enough time data to show a trend.

System

11.82

%

CPU percentiles

CPU usage percentiles

p50

61.14

%

p95

61.14

%

p99

61.14

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

0

hosts

mean time

0.00

s

Memory used percentage

Percentage of memory used

p5

15.44

%

<

mean

19.88

%

SD

4.99

%

<

p95

26.85

%

Not enough time data to show a trend.

Memory available percentage

Percentage of available memory

p5

73.15

%

<

mean

80.12

%

SD

4.99

%

<

p95

84.56

%

Not enough time data to show a trend.

Max host memory used

Maximum memory usage percentage across all hosts

max

19.88

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

0.00

%

Memory growth rate

Rate of memory growth over time

growth

0.75

MiB/s

Disk read throughput

Disk read throughput in MiB/s

4.14

MiB/s

Disk write throughput

Disk write throughput in MiB/s

16.82

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /etc/nomad.d/client-meta.json

0/1

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

0.61

5 min

0.19

15 min

0.12

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

0.00

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

7.04

%

<

mean

9.5033

%

SD

1.8319

%

<

p95

11.43

%

Not enough time data to show a trend.

60 second average

3.0533

%

300 second average

0.7200

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

<

mean

0.7233

%

SD

1.0229

%

<

p95

2.17

%

Not enough time data to show a trend.

60 second average

0.1300

%

300 second average

0.0267

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

<

mean

0.6633

%

SD

0.9381

%

<

p95

1.99

%

Not enough time data to show a trend.

60 second average

0.1167

%

300 second average

0.0233

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

3.56

%

SD

4.4539

%

<

p95

9.84

%

Not enough time data to show a trend.

60 second average

0.7567

%

300 second average

0.1633

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

2.51

%

SD

3.162

%

<

p95

6.97

%

Not enough time data to show a trend.

60 second average

0.5133

%

300 second average

0.1067

%

Holochain process CPU usage

CPU usage by Holochain process

p5

0

%

<

mean

29.01

%

SD

29.01

%

<

p95

58.01

%

Not enough time data to show a trend.

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

103.59

KiB

<

mean

610.74

KiB

SD

507.15

KiB

<

p95

1.09

MiB

Not enough time data to show a trend.

Holochain process threads

Number of threads in Holochain process

p5

1

threads

<

mean

77.67

threads

SD

91.32

threads

<

p95

206

threads

Not enough time data to show a trend.

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

0

file descriptors

<

mean

174.67

file descriptors

SD

202.15

file descriptors

<

p95

458

file descriptors

Not enough time data to show a trend.

Local Signals

↑ Back to index

This scenario is for testing the speed and reliability of local signals. It uses a zome with a function that will emit 10,000 signals. The scenario measures how long it takes to send the signals and how many have been received by the time the zome call ends.

Started

Thu, 03 Sep 2026 08:47:50 UTC

Duration

300s

Peer count

1

Behaviours

default (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

local_signals_33713946045_1

Send timing

The time taken to emit a batch of 10,000 signals.

mean

1.781547

s

SD

1.165847

s

p50

2.106682

s

<

p95

4.257057

s

≤

p99

4.257057

s

Receive timing

The time taken to receive the completed batch, to the nearest 250ms.

mean

27.391008

s

SD

7.831073

s

p50

30.001043

s

<

p95

30.003364

s

≤

p99

30.003364

s

Success ratio

The proportion of signals successfully received in a batch.

min

0.5245000123977661

<

mean

0.6174

SD

0.0787

<

max

0.7705000042915344

Holochain MetricsClick to toggle visibility.

WASM usage: 'signal::emit_10k_signals'

Metered usage count of the WASM ribosome for this zome function.

total

246763944

over 280.002731576s

mean rate

881291.83

/s

std

1.39344473e+06

/s

p5

0

/s

p95

3.08453225e+06

/s

peak

3.08453838e+06

/s

Zome call duration: 'signal::emit_10k_signals'

Duration of this zome call as measured by Holochain internally.

mean

0.26844

s

SD

0.069282

s

p50

0.268398

s

<

p95

0.314553

s

≤

p99

0.314553

s

WASM call duration: 'signal::emit_10k_signals'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.267548

s

SD

0.069169

s

p50

0.26765

s

<

p95

0.313721

s

≤

p99

0.313721

s

Host function call duration: '__hc__emit_signal_1'

Duration of this host function call invoked from within WASM.

mean

1.9e-05

s

SD

5e-06

s

p50

2e-05

s

<

p95

2.3e-05

s

≤

p99

2.3e-05

s

Emitted local signals

Count of local signals emitted via the emit_signal host function.

total

80000

over 280.002771594s

mean rate

285.71

/s

std

431.98

/s

p5

0

/s

p95

999.99

/s

peak

999.99

/s

Post-commit duration

Time spent executing post-commit workflows.

mean

0.000298

s

SD

0

s

p50

0.000298

s

≤

p95

0.000298

s

≤

p99

0.000298

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

16.88

s

<

mean

146.88

s

SD

83.67

s

<

p95

276.88

s

Dropped WebSocket signals

Count of signals dropped from the app WebSocket due to channel overload.

total

2613

over 280.002739283s

mean rate

9.33

/s

std

17.61

/s

p5

0

/s

p95

52.7

/s

peak

59

/s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

0

over 280.002739084s

Not enough time data to show a trend.

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.002476

s

SD

0

s

p50

0.002476

s

≤

p95

0.002476

s

≤

p99

0.002476

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.002355

s

SD

0

s

p50

0.002355

s

≤

p95

0.002355

s

≤

p99

0.002355

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.00896

s

SD

0.002056

s

p50

0.009495

s

<

p95

0.013032

s

≤

p99

0.013032

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

0.006435

s

SD

0

s

p50

0.006435

s

≤

p95

0.006435

s

≤

p99

0.006435

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

0.002593

s

SD

0

s

p50

0.002593

s

≤

p95

0.002593

s

≤

p99

0.002593

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

0.00014

s

SD

2.9e-05

s

p50

0.000132

s

<

p95

0.000162

s

<

p99

0.000163

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.002593

s

SD

0.000163

s

p50

0.002629

s

<

p95

0.002904

s

≤

p99

0.002904

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000372

s

SD

0

s

p50

0.000372

s

≤

p95

0.000372

s

≤

p99

0.000372

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

16.88

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

406

B/s

<

mean

869.71

KiB/s

SD

2.72

MiB/s

<

p95

8.92

MiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

171

B/s

<

mean

28.39

KiB/s

SD

42.69

KiB/s

<

p95

139.92

KiB/s

Total bytes received

Total bytes received on primary network interface

count

246.31

MiB

mean

869.71

KiB/s

Total bytes sent

Total bytes sent on primary network interface

count

8.04

MiB

mean

28.39

KiB/s

CPU spike anomaly

CPU spike anomaly was detected

Detected

Warning CPU p99 reached 98.2%

Memory leak anomaly

Memory leak anomaly was detected

Detected

Warning Memory growing at 3.86 MiB/s

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

NotDetected

System overload anomaly

System overload anomaly was detected

NotDetected

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

2.86

%

<

mean

26.28

%

SD

22.26

%

<

p95

81

%

System

9.40

%

CPU percentiles

CPU usage percentiles

p50

24.15

%

p95

81.00

%

p99

98.25

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

1

hosts

mean time

20.69

s

Memory used percentage

Percentage of memory used

p5

14.99

%

<

mean

21.85

%

SD

5

%

<

p95

30.03

%

Memory available percentage

Percentage of available memory

p5

69.97

%

<

mean

78.15

%

SD

5

%

<

p95

85.01

%

Max host memory used

Maximum memory usage percentage across all hosts

max

21.85

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

0.00

%

Memory growth rate

Rate of memory growth over time

growth

3.86

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.00

MiB/s

Disk write throughput

Disk write throughput in MiB/s

4.34

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /etc/nomad.d/client-meta.json

0/1

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

1.36

5 min

0.62

15 min

0.29

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

0.00

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

0.75

%

<

mean

12.5203

%

SD

13.7231

%

<

p95

45.74

%

60 second average

10.1277

%

300 second average

4.5267

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0220

%

300 second average

0.0303

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0207

%

300 second average

0.0280

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

0.793

%

SD

1.2777

%

<

p95

3.01

%

60 second average

0.9517

%

300 second average

0.7390

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

0.651

%

SD

1.0566

%

<

p95

2.45

%

60 second average

0.7950

%

300 second average

0.6147

%

Holochain process CPU usage

CPU usage by Holochain process

p5

0.4

%

<

mean

8.61

%

SD

13.41

%

<

p95

36.97

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

88.16

KiB

<

mean

207.37

KiB

SD

37.87

KiB

<

p95

269.79

KiB

Holochain process threads

Number of threads in Holochain process

p5

6

threads

<

mean

25.63

threads

SD

6.56

threads

<

p95

31

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

23

file descriptors

<

mean

69

file descriptors

SD

15.23

file descriptors

<

p95

80

file descriptors

Mixed Arc `get_agent_activity`

↑ Back to index

A mixed-arc scenario that measures the DHT's ability to propagate agent activity and make it available via get_agent_activity calls. The scenario has three roles:

zero_write: A zero-arc conductor that creates entries on its source chain.
full_write: A full-arc conductor that creates entries on its source chain.
zero_read: A zero-arc conductor that picks a random writer and repeatedly calls get_agent_activity to observe new entries appearing on the writer's chain, measuring the lag between creation and discovery.

Started

Thu, 03 Sep 2026 12:27:44 UTC

Duration

900s

Peer count

30

Peer count at end

28

Behaviours

full_write (1 agent)
zero_read (1 agent)
zero_write (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

mixed_arc_get_agent_activity_33713946045_1

Entry created count

The total number of entries created by write agents over the run, partitioned by agent and behaviour.

Total

114189

Agents affected

19 / 19

Mean per agent

Total count divided by number of partitions, rounded to nearest whole number

6010

Max

Highest count in any single partition

10099

Min

Lowest count in any single partition

1827

Observed chain advancement

The maximum highest-observed action sequence per write agent, aggregated across all reading agents. The reading dimension is collapsed by taking the maximum observed value for each write agent (any reader successfully tracking that point counts as propagation). Captures how far readers tracked writers’ chains during the run.

Mean max (across write agents)

Mean of the highest chain head value observed per write agent. Each write agent contributes its maximum — the value seen by any reader.

3452

Max

Highest chain head value observed for any single write agent.

4683

Write agents observed

3

Chain head delay timing

The time lag between when a writer creates a new entry and when a reader discovers it via get_agent_activity. This is measured per new action in the chain.

Mean

mean

-0.014961

s

SD

0.710426

s

Max

Highest per-partition mean latency

max mean

0.128557

s

Min

Lowest per-partition mean latency

min mean

-0.205623

s

Chain head delay rate

The rate at which readers detect new chain heads appearing on writers' chains.

Mean

mean

14.8

/s

Max

Highest per-partition mean rate

max

30.14

/s

Min

Lowest per-partition mean rate

min

2.94

/s

get_agent_activity_full zome call timing

The time taken to complete the get_agent_activity_full zome call that queries a writer's chain.

Mean

mean

0.280306

s

SD

0.567821

s

Max

Highest per-partition mean latency

max mean

0.643685

s

Min

Lowest per-partition mean latency

min mean

0.011211

s

Retrieval errors

A running accumulation of errors encountered by read agents when calling get_agent_activity.

Total

15

Agents affected

2 / 9

Mean per agent

Total count divided by number of partitions, rounded to nearest whole number

2

Max

Highest count in any single partition

8

Min

Lowest count in any single partition

0

Open connections

The number of currently open connections to other conductors, partitioned by agent behaviour.

behaviour: full_write

p5

27

≤

mean

26.86

SD

1.13

<

p95

27

behaviour: zero_read

p5

0

<

mean

16.52

SD

7.3

<

p95

25

behaviour: zero_write

p5

12

<

mean

16.88

SD

3.93

<

p95

26

Final error count

The total number of errors accumulated across all agents over the run.

15

Holochain MetricsClick to toggle visibility.

Cascade duration

Time taken to execute a cascade (get) query inside Holochain.

mean

0.09481

s

SD

0.497591

s

p50

0.100405

s

<

p95

1.23685

s

<

p99

2.670981

s

Cascade fetch errors

Count of network fetch errors during cascade calls.

total

2

over 513.405395757s

mean rate

2.13

/s

std

3.99

/s

p5

0

/s

p95

7.46

/s

peak

14.73

/s

Zome call duration: 'agent_activity::announce_write_behaviour'

Duration of this zome call as measured by Holochain internally.

mean

0.006459

s

SD

0.00273

s

p50

0.005869

s

<

p95

0.011512

s

≤

p99

0.011512

s

Zome call duration: 'agent_activity::create_sample_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.004508

s

SD

0.004688

s

p50

0.00387

s

<

p95

0.017487

s

<

p99

0.024647

s

Zome call duration: 'agent_activity::get_agent_activity_full'

Duration of this zome call as measured by Holochain internally.

mean

0.121838

s

SD

0.590156

s

p50

0.26023

s

<

p95

1.413404

s

<

p99

3.159165

s

Zome call duration: 'agent_activity::get_random_agent_with_write_behaviour'

Duration of this zome call as measured by Holochain internally.

mean

0.004982

s

SD

0.363069

s

p50

0.008565

s

<

p95

0.940799

s

≤

p99

0.940799

s

WASM call duration: 'agent_activity::announce_write_behaviour'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.005382

s

SD

0.00239

s

p50

0.004822

s

<

p95

0.009847

s

≤

p99

0.009847

s

WASM call duration: 'agent_activity::create_sample_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.003825

s

SD

0.004391

s

p50

0.003206

s

<

p95

0.015614

s

<

p99

0.023176

s

WASM call duration: 'agent_activity::get_agent_activity_full'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.121893

s

SD

0.58959

s

p50

0.279664

s

<

p95

1.412003

s

<

p99

3.157562

s

WASM call duration: 'agent_activity::get_random_agent_with_write_behaviour'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.004602

s

SD

0.362701

s

p50

0.008076

s

<

p95

0.93914

s

≤

p99

0.93914

s

WASM call duration: 'agent_activity_integrity::entry_defs'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.001226

s

SD

0.001145

s

p50

0.001054

s

<

p95

0.00418

s

<

p99

0.006594

s

Host function call duration: '__hc__agent_info_1'

Duration of this host function call invoked from within WASM.

mean

3.1e-05

s

SD

1.3e-05

s

p50

2.7e-05

s

<

p95

5.2e-05

s

≤

p99

5.2e-05

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.000522

s

SD

0.001247

s

p50

0.000336

s

<

p95

0.002639

s

<

p99

0.007258

s

Host function call duration: '__hc__create_link_1'

Duration of this host function call invoked from within WASM.

mean

0.000528

s

SD

0.000249

s

p50

0.00047

s

<

p95

0.000985

s

≤

p99

0.000985

s

Host function call duration: '__hc__get_agent_activity_1'

Duration of this host function call invoked from within WASM.

mean

0.120431

s

SD

0.587429

s

p50

0.278383

s

<

p95

1.402862

s

<

p99

3.148353

s

Host function call duration: '__hc__get_links_1'

Duration of this host function call invoked from within WASM.

mean

0.002849

s

SD

0.360908

s

p50

0.005764

s

<

p95

0.93169

s

≤

p99

0.93169

s

Host function call duration: '__hc__random_bytes_1'

Duration of this host function call invoked from within WASM.

mean

4e-06

s

SD

4e-06

s

p50

7e-06

s

<

p95

1.5e-05

s

≤

p99

1.5e-05

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.00192

s

SD

0.001804

s

p50

0.001709

s

<

p95

0.005936

s

<

p99

0.010693

s

Post-commit duration

Time spent executing post-commit workflows.

mean

0.000161

s

SD

0.000267

s

p50

0.000129

s

<

p95

0.000716

s

<

p99

0.001594

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

47

s

<

mean

445.91

s

SD

256.21

s

<

p95

847

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

0

over 888.200756908s

Not enough time data to show a trend.

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

19.463113

s

SD

25.83784

s

p50

0

s

<

p95

61.389471

s

<

p99

145.762729

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.041604

s

SD

0.068867

s

p50

0.001473

s

<

p95

0.247107

s

<

p99

0.301722

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

<

p99

1e-06

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.038678

s

SD

0.304874

s

p50

0.001494

s

<

p95

1.022339

s

<

p99

1.260401

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.090611

s

SD

0.067581

s

p50

0.056028

s

<

p95

0.209055

s

<

p99

0.267035

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

4.719095

s

SD

8.026923

s

p50

0.003917

s

<

p95

14.856095

s

<

p99

45.037287

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

5.838769

s

SD

14.805632

s

p50

0.002438

s

<

p95

4.166873

s

<

p99

58.825899

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

0.002804

s

SD

0.00162

s

p50

0.001812

s

<

p95

0.005072

s

<

p99

0.006395

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.004207

s

SD

0.004224

s

p50

0.001115

s

<

p95

0.006874

s

<

p99

0.027206

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000492

s

SD

0.00104

s

p50

0.000294

s

<

p95

0.002242

s

<

p99

0.006559

s

P2P request roundtrip: 'get'

Time spent sending a get request and awaiting its response.

mean

18.52475

s

SD

26.580187

s

p50

7.185942

s

<

p95

60.013687

s

≤

p99

60.013687

s

P2P request roundtrip: 'get_links'

Time spent sending a get_links request and awaiting its response.

mean

0.024834

s

SD

0.295364

s

p50

0.124135

s

<

p95

0.926587

s

≤

p99

0.926587

s

P2P request roundtrip: 'get_agent_activity'

Time spent sending a get_agent_activity request and awaiting its response.

mean

0.121827

s

SD

23.105628

s

p50

0.448299

s

<

p95

60.001134

s

<

p99

60.001189

s

P2P request roundtrip: 'send_validation_receipts'

Time spent sending a send_validation_receipts request and awaiting its response.

mean

0.572407

s

SD

3.434404

s

p50

0.608655

s

<

p95

10.690161

s

<

p99

13.214553

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

42.92

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

2.74

KiB/s

<

mean

273.13

KiB/s

SD

345.57

KiB/s

<

p95

763.58

KiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

1.78

KiB/s

<

mean

227.35

KiB/s

SD

219.60

KiB/s

<

p95

683.28

KiB/s

Total bytes received

Total bytes received on primary network interface

count

6.03

GiB

mean

6.93

MiB/s

Total bytes sent

Total bytes sent on primary network interface

count

5.02

GiB

mean

5.77

MiB/s

CPU spike anomaly

CPU spike anomaly was detected

Detected

Warning CPU p99 reached 98.5%

Memory leak anomaly

Memory leak anomaly was detected

NotDetected

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

NotDetected

System overload anomaly

System overload anomaly was detected

Detected

Warning 37% of hosts overloaded (load5/ncpus > 1.0)

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

0.81

%

<

mean

55.51

%

SD

33.94

%

<

p95

97.74

%

System

12.59

%

CPU percentiles

CPU usage percentiles

p50

61.86

%

p95

97.74

%

p99

98.49

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

10

hosts

mean time

749.33

s

Memory used percentage

Percentage of memory used

p5

6.53

%

<

mean

17.91

%

SD

8.61

%

<

p95

36.12

%

Memory available percentage

Percentage of available memory

p5

63.88

%

<

mean

82.09

%

SD

8.61

%

<

p95

93.47

%

Max host memory used

Maximum memory usage percentage across all hosts

max

36.28

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

5.13

%

Memory growth rate

Rate of memory growth over time

growth

0.28

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.13

MiB/s

Disk write throughput

Disk write throughput in MiB/s

70.33

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /

0/3

hosts

Mount Point /boot

0/1

hosts

Mount Point /efi-boot

0/2

hosts

Mount Point /etc/hostname

0/13

hosts

Mount Point /etc/hosts

0/13

hosts

Mount Point /etc/nomad.d/client-meta.json

0/11

hosts

Mount Point /etc/resolv.conf

0/13

hosts

Mount Point /nix/store

0/3

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

3.59

5 min

3.02

15 min

2.38

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

37.04

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

0

%

<

mean

22.9965

%

SD

26.9667

%

<

p95

80.47

%

60 second average

21.7711

%

300 second average

17.3969

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

2.6771

%

SD

7.406

%

<

p95

25.41

%

60 second average

2.6420

%

300 second average

2.6364

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

0.5928

%

SD

1.726

%

<

p95

4.37

%

60 second average

0.6904

%

300 second average

1.1800

%

Holochain process CPU usage

CPU usage by Holochain process

p5

0.5

%

<

mean

49.09

%

SD

35.41

%

<

p95

95.69

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

225.00

KiB

<

mean

308.89

KiB

SD

78.61

KiB

<

p95

455.97

KiB

Holochain process threads

Number of threads in Holochain process

p5

23

threads

<

mean

32.7

threads

SD

7.05

threads

<

p95

47

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

57

file descriptors

<

mean

73.58

file descriptors

SD

12.68

file descriptors

<

p95

85

file descriptors

Mixed Arc `must_get_agent_activity`

↑ Back to index

A scenario where write agents create entries in batches of 10, while must_get_agent_activity agents each pick a random write agent and repeatedly attempt to create an entry that references the chain top of their latest batch. This reference means that the entry's validation function needs to make a must_get_agent_activity call.

The purpose of this scenario is to measure the time it takes for published agent activity data to be gossiped among authorities and become available to peers that query it via must_get_agent_activity.

This test is similar to Write Validated must_get_agent_activity, but some write agents are zero-arc and all must_get_agent_activity agents are zero-arc. This forces the must_get_agent_activity host call to go to the network.

Started

Thu, 03 Sep 2026 12:43:32 UTC

Duration

900s

Peer count

30

Peer count at end

28

Behaviours

full_write (1 agent)
zero_must_get_agent_activity (1 agent)
zero_write (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

mixed_arc_must_get_agent_activity_33713946045_1

Highest observed action_seq

The maximum chain length observed per write agent, aggregated across all reading agents. The reading dimension is collapsed by taking the maximum observed value for each write agent (any successful read counts as propagation). This reflects the DHT’s ability to propagate agent activity ops and make them available to querying peers.

Mean max (across write agents)

Mean of the highest chain head value observed per write agent. Each write agent contributes its maximum — the value seen by any reader.

976.67

Max

Highest chain head value observed for any single write agent.

1743

Write agents observed

3

Chain batch delay timing

The time between a write agent's creation of a batch and a must_get_agent_activity agent's successful discovery of the batch and creation/self-validation of a new entry that depends on it.

Mean

mean

2.107334

s

SD

86.187972

s

Max

Highest per-partition mean latency

max mean

218.755164

s

Min

Lowest per-partition mean latency

min mean

-772.461059

s

Chain batch delay rate

The rate at which a must_get_agent_activity agent was able to discover batches and create/self-validate new entries that depend on them.

Mean

mean

2.1

/s

Max

Highest per-partition mean rate

max

3.86

/s

Min

Lowest per-partition mean rate

min

1.34

/s

create_validated_sample_entry zome call timing

The time taken to complete the zome function call that creates the entry that depends on a write agent's source chain.

Mean

mean

1.091353

s

SD

1.184192

s

Max

Highest per-partition mean latency

max mean

1.87219

s

Min

Lowest per-partition mean latency

min mean

0.267419

s

Retrieval errors

A running accumulation of the errors encountered by an agent when attempting to self-validate actions that depend on must_get_agent_activity calls.

Total

15124

Agents affected

9 / 9

Mean per agent

Total count divided by number of partitions, rounded to nearest whole number

1680

Max

Highest count in any single partition

5362

Min

Lowest count in any single partition

418

Open connections

The average number of open connections maintained by an agent over the run, broken down by behaviour.

behaviour: full_write

p5

0

<

mean

24.72

SD

7.24

<

p95

27

behaviour: zero_must_get_agent_activity

p5

12

<

mean

17.48

SD

4.51

<

p95

26

behaviour: zero_write

p5

12

<

mean

16.31

SD

3.87

<

p95

25

Final error count

The total number of all types of error accumulated over the run by all agents.

25078

Holochain MetricsClick to toggle visibility.

Cascade duration

Time taken to execute a cascade (get) query inside Holochain.

mean

0.071093

s

SD

0.18768

s

p50

0.05982

s

<

p95

0.534111

s

<

p99

0.708043

s

Cascade fetch errors

Count of network fetch errors during cascade calls.

total

2

over 790.98623132s

mean rate

3.97

/s

std

7.84

/s

p5

0

/s

p95

24.22

/s

peak

24.43

/s

Zome call duration: 'validated_must_get_agent_activity::announce_write_behaviour'

Duration of this zome call as measured by Holochain internally.

mean

0.006756

s

SD

0.003031

s

p50

0.00643

s

<

p95

0.011786

s

≤

p99

0.011786

s

Zome call duration: 'validated_must_get_agent_activity::create_sample_entries_batch'

Duration of this zome call as measured by Holochain internally.

mean

0.052966

s

SD

0.020964

s

p50

0.04306

s

<

p95

0.092004

s

<

p99

0.107961

s

Zome call duration: 'validated_must_get_agent_activity::create_validated_sample_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.009439

s

SD

0.002705

s

p50

0.00605

s

<

p95

0.011804

s

<

p99

0.011858

s

Zome call duration: 'validated_must_get_agent_activity::get_chain_top_for_batch'

Duration of this zome call as measured by Holochain internally.

mean

0.329743

s

SD

0.217888

s

p50

0.256586

s

<

p95

0.723965

s

<

p99

0.860263

s

Zome call duration: 'validated_must_get_agent_activity::get_random_agent_with_write_behaviour'

Duration of this zome call as measured by Holochain internally.

mean

0.017289

s

SD

0.158099

s

p50

0.017965

s

<

p95

0.455772

s

≤

p99

0.455772

s

WASM call duration: 'validated_must_get_agent_activity::announce_write_behaviour'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.005669

s

SD

0.002619

s

p50

0.005213

s

<

p95

0.01

s

≤

p99

0.01

s

WASM call duration: 'validated_must_get_agent_activity::create_sample_entries_batch'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.052189

s

SD

0.020869

s

p50

0.042278

s

<

p95

0.09116

s

<

p99

0.107185

s

WASM call duration: 'validated_must_get_agent_activity::create_validated_sample_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.007911

s

SD

0.002305

s

p50

0.005117

s

<

p95

0.009905

s

<

p99

0.009942

s

WASM call duration: 'validated_must_get_agent_activity::get_chain_top_for_batch'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.199998

s

SD

0.207509

s

p50

0.250821

s

<

p95

0.69016

s

<

p99

0.926212

s

WASM call duration: 'validated_must_get_agent_activity::get_random_agent_with_write_behaviour'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.016392

s

SD

0.155246

s

p50

0.01686

s

<

p95

0.454913

s

≤

p99

0.454913

s

WASM call duration: 'validated_must_get_agent_activity_integrity::entry_defs'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.001238

s

SD

0.000704

s

p50

0.001125

s

<

p95

0.003335

s

<

p99

0.003385

s

WASM call duration: 'validated_must_get_agent_activity_integrity::validate'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.012878

s

SD

0.044432

s

p50

0.003572

s

<

p95

0.152603

s

<

p99

0.174695

s

Host function call duration: '__hc__agent_info_1'

Duration of this host function call invoked from within WASM.

mean

2e-05

s

SD

2.3e-05

s

p50

2.1e-05

s

<

p95

6.8e-05

s

<

p99

0.00014

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.000458

s

SD

0.000185

s

p50

0.000382

s

<

p95

0.000864

s

<

p99

0.000888

s

Host function call duration: '__hc__create_link_1'

Duration of this host function call invoked from within WASM.

mean

0.000362

s

SD

0.000194

s

p50

0.000375

s

<

p95

0.000875

s

<

p99

0.000915

s

Host function call duration: '__hc__get_links_1'

Duration of this host function call invoked from within WASM.

mean

0.153561

s

SD

0.203365

s

p50

0.168224

s

<

p95

0.589959

s

<

p99

0.719133

s

Host function call duration: '__hc__must_get_agent_activity_1'

Duration of this host function call invoked from within WASM.

mean

0.056261

s

SD

0.162633

s

p50

0.046926

s

<

p95

0.481953

s

<

p99

0.530747

s

Host function call duration: '__hc__query_1'

Duration of this host function call invoked from within WASM.

mean

0.020418

s

SD

0.014609

s

p50

0.010184

s

<

p95

0.048271

s

<

p99

0.061227

s

Host function call duration: '__hc__random_bytes_1'

Duration of this host function call invoked from within WASM.

mean

9e-06

s

SD

5e-06

s

p50

1.1e-05

s

<

p95

2.2e-05

s

≤

p99

2.2e-05

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.001902

s

SD

0.00154

s

p50

0.002094

s

<

p95

0.004996

s

<

p99

0.011272

s

Post-commit duration

Time spent executing post-commit workflows.

mean

0.000147

s

SD

8.9e-05

s

p50

0.000142

s

<

p95

0.000375

s

<

p99

0.000409

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

47.06

s

<

mean

446.06

s

SD

256.2

s

<

p95

847

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

42719

over 889.128325902s

mean rate

82910.32

/s

std

319308.94

/s

p5

0

/s

p95

239875.95

/s

peak

2.69947379e+06

/s

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

135.478686

s

SD

68.909111

s

p50

0

s

<

p95

192.362805

s

<

p99

260.944071

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

2.607102

s

SD

1.225983

s

p50

0.001941

s

<

p95

3.578237

s

<

p99

4.016017

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

<

p99

1e-06

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.124279

s

SD

0.136636

s

p50

0.001521

s

<

p95

0.376827

s

<

p99

0.553712

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.098421

s

SD

0.116822

s

p50

0.013784

s

<

p95

0.332749

s

<

p99

0.559069

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

2.951538

s

SD

11.261581

s

p50

0.003397

s

<

p95

28.424777

s

<

p99

57.45507

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

16.719567

s

SD

18.863191

s

p50

0.002275

s

<

p95

51.421638

s

<

p99

84.323848

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

0.002214

s

SD

0.001466

s

p50

0.000261

s

<

p95

0.004215

s

<

p99

0.006068

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.002962

s

SD

0.004156

s

p50

0.001577

s

<

p95

0.011006

s

<

p99

0.023068

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000371

s

SD

0.000157

s

p50

0.000341

s

<

p95

0.000759

s

<

p99

0.000781

s

P2P request roundtrip: 'get'

Time spent sending a get request and awaiting its response.

mean

3.731466

s

SD

29.779713

s

p50

0.202011

s

<

p95

60.001634

s

≤

p99

60.001634

s

P2P request roundtrip: 'get_links'

Time spent sending a get_links request and awaiting its response.

mean

0.099851

s

SD

23.845626

s

p50

0.174555

s

<

p95

60.001832

s

<

p99

60.002666

s

P2P request roundtrip: 'must_get_agent_activity'

Time spent sending a must_get_agent_activity request and awaiting its response.

mean

0.173449

s

SD

27.707503

s

p50

0.29352

s

<

p95

60.001438

s

<

p99

60.001592

s

P2P request roundtrip: 'send_validation_receipts'

Time spent sending a send_validation_receipts request and awaiting its response.

mean

0.183287

s

SD

6.896868

s

p50

0.231878

s

<

p95

16.640389

s

<

p99

17.512075

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

17.76

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

2.85

KiB/s

<

mean

84.88

KiB/s

SD

234.24

KiB/s

<

p95

166.84

KiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

1.71

KiB/s

<

mean

70.97

KiB/s

SD

113.91

KiB/s

<

p95

133.55

KiB/s

Total bytes received

Total bytes received on primary network interface

count

1.87

GiB

mean

2.16

MiB/s

Total bytes sent

Total bytes sent on primary network interface

count

1.57

GiB

mean

1.80

MiB/s

CPU spike anomaly

CPU spike anomaly was detected

Detected

Warning CPU p99 reached 92.6%

Memory leak anomaly

Memory leak anomaly was detected

NotDetected

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

NotDetected

System overload anomaly

System overload anomaly was detected

Detected

Warning 11% of hosts overloaded (load5/ncpus > 1.0)

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

1.29

%

<

mean

27.13

%

SD

27.83

%

<

p95

81.59

%

System

9.37

%

CPU percentiles

CPU usage percentiles

p50

12.77

%

p95

81.59

%

p99

92.58

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

9

hosts

mean time

156.18

s

Memory used percentage

Percentage of memory used

p5

7.72

%

<

mean

19.46

%

SD

9.07

%

<

p95

38.3

%

Memory available percentage

Percentage of available memory

p5

61.7

%

<

mean

80.54

%

SD

9.07

%

<

p95

92.28

%

Max host memory used

Maximum memory usage percentage across all hosts

max

38.91

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

5.13

%

Memory growth rate

Rate of memory growth over time

growth

0.73

MiB/s

Disk read throughput

Disk read throughput in MiB/s

870.65

MiB/s

Disk write throughput

Disk write throughput in MiB/s

54.61

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /

0/3

hosts

Mount Point /boot

0/1

hosts

Mount Point /efi-boot

0/2

hosts

Mount Point /etc/hostname

0/13

hosts

Mount Point /etc/hosts

0/13

hosts

Mount Point /etc/nomad.d/client-meta.json

0/11

hosts

Mount Point /etc/resolv.conf

0/13

hosts

Mount Point /nix/store

0/3

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

1.48

5 min

1.97

15 min

2.29

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

11.11

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

0

%

<

mean

6.14

%

SD

11.6076

%

<

p95

26.62

%

60 second average

6.4575

%

300 second average

10.0720

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

<

mean

0.2765

%

SD

1.7766

%

≤

p95

0

%

60 second average

0.2325

%

300 second average

0.1284

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

<

mean

0.2222

%

SD

1.6997

%

≤

p95

0

%

60 second average

0.1856

%

300 second average

0.0947

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

2.4698

%

SD

8.3448

%

<

p95

18.05

%

60 second average

2.3719

%

300 second average

2.2534

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

1.0809

%

SD

3.3976

%

<

p95

8.19

%

60 second average

1.0337

%

300 second average

0.8703

%

Holochain process CPU usage

CPU usage by Holochain process

p5

0.72

%

<

mean

20.14

%

SD

25.91

%

<

p95

78.46

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

339.76

KiB

<

mean

426.80

KiB

SD

78.92

KiB

<

p95

544.16

KiB

Holochain process threads

Number of threads in Holochain process

p5

26

threads

<

mean

33.24

threads

SD

6.12

threads

<

p95

44

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

63

file descriptors

<

mean

73.3

file descriptors

SD

9.96

file descriptors

<

p95

87

file descriptors

Remote Call Rate

↑ Back to index

Test the throughput of remote_call operations. Each agent in this scenario waits for a certain number of peers to be available or for up to two minutes, whichever happens first, before starting its behaviour.

Started

Thu, 03 Sep 2026 08:49:24 UTC

Duration

900s

Peer count

10

Peer count at end

10

Behaviours

default (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

remote_call_rate_33713946045_1

Dispatch timing

The time between sending a remote call and the remote handler being invoked.

Mean

mean

0.077023

s

SD

0.054782

s

Max

Highest per-partition mean latency

max mean

0.091969

s

Min

Lowest per-partition mean latency

min mean

0.03634

s

Round-trip timing

The total elapsed time to get a response to the client.

Mean

mean

0.155669

s

SD

0.083704

s

Max

Highest per-partition mean latency

max mean

0.190717

s

Min

Lowest per-partition mean latency

min mean

0.067773

s

Error count

The total number of errors accumulated during the run.

0

Holochain MetricsClick to toggle visibility.

WASM usage: 'remote_call::call_echo_timestamp'

Metered usage count of the WASM ribosome for this zome function.

total

162070839

over 886.218275057s

mean rate

9.230246858e+07

/s

std

1.667086009e+08

/s

p5

438552.68

/s

p95

5.5073760511e+08

/s

peak

7.6630182588e+08

/s

WASM usage: 'remote_call::echo_timestamp'

Metered usage count of the WASM ribosome for this zome function.

total

38388958

over 886.218275057s

mean rate

1.1790713e+06

/s

std

1.86336837e+06

/s

p5

38400.51

/s

p95

5.65779452e+06

/s

peak

9.90094772e+06

/s

WASM usage: 'remote_call::init'

Metered usage count of the WASM ribosome for this zome function.

total

0

over 886.218275057s

Not enough time data to show a trend.

Zome call duration: 'remote_call::call_echo_timestamp'

Duration of this zome call as measured by Holochain internally.

mean

0.152925

s

SD

0.049186

s

p50

0.178187

s

<

p95

0.212343

s

<

p99

0.283878

s

Zome call duration: 'remote_call::echo_timestamp'

Duration of this zome call as measured by Holochain internally.

mean

0.002871

s

SD

0.000967

s

p50

0.002674

s

<

p95

0.004328

s

<

p99

0.00434

s

WASM call duration: 'remote_call::call_echo_timestamp'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.151954

s

SD

0.049191

s

p50

0.177027

s

<

p95

0.211423

s

<

p99

0.283534

s

WASM call duration: 'remote_call::echo_timestamp'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.001934

s

SD

0.000696

s

p50

0.001846

s

<

p95

0.00299

s

<

p99

0.003003

s

WASM call duration: 'remote_call::init'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.002535

s

SD

0.000828

s

p50

0.002695

s

<

p95

0.003623

s

≤

p99

0.003623

s

Host function call duration: '__hc__call_1'

Duration of this host function call invoked from within WASM.

mean

0.149684

s

SD

0.049243

s

p50

0.174495

s

<

p95

0.20925

s

<

p99

0.2827

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.000409

s

SD

0.00012

s

p50

0.00044

s

<

p95

0.000573

s

≤

p99

0.000573

s

Host function call duration: '__hc__sys_time_1'

Duration of this host function call invoked from within WASM.

mean

9e-06

s

SD

4e-06

s

p50

9e-06

s

<

p95

1.5e-05

s

<

p99

1.6e-05

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.00028

s

SD

0.000218

s

p50

0.000234

s

<

p95

0.001183

s

≤

p99

0.001183

s

Post-commit duration

Time spent executing post-commit workflows.

mean

0.000168

s

SD

0.000103

s

p50

0.000143

s

<

p95

0.00042

s

≤

p99

0.00042

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

46.93

s

<

mean

445.11

s

SD

255.8

s

<

p95

846.79

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

98

over 886.218413558s

mean rate

28.18

/s

std

110.81

/s

p5

0

/s

p95

256.98

/s

peak

803.14

/s

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

1.417

s

SD

1.20098

s

p50

0.788223

s

<

p95

3.804893

s

<

p99

5.110736

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.002936

s

SD

0.002965

s

p50

0.002684

s

<

p95

0.009484

s

<

p99

0.013491

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.075851

s

SD

0.039285

s

p50

0.084404

s

<

p95

0.13348

s

≤

p99

0.13348

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.006579

s

SD

0.003511

s

p50

0.005614

s

<

p95

0.013432

s

<

p99

0.019292

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

0.357192

s

SD

0.247343

s

p50

0.273656

s

<

p95

0.843627

s

<

p99

0.948788

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

0.18014

s

SD

0.084665

s

p50

0.155067

s

<

p95

0.294195

s

<

p99

0.369198

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

0.000132

s

SD

3.7e-05

s

p50

0.000138

s

<

p95

0.000189

s

<

p99

0.000215

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.001124

s

SD

0.000632

s

p50

0.001058

s

<

p95

0.003411

s

≤

p99

0.003411

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000406

s

SD

0.000127

s

p50

0.000379

s

<

p95

0.000581

s

<

p99

0.000584

s

P2P request roundtrip: 'get'

Time spent sending a get request and awaiting its response.

mean

0.132311

s

SD

0.04593

s

p50

0.1487

s

<

p95

0.172629

s

<

p99

0.244513

s

P2P request roundtrip: 'send_validation_receipts'

Time spent sending a send_validation_receipts request and awaiting its response.

mean

0.155623

s

SD

0.035551

s

p50

0.158173

s

<

p95

0.187776

s

≤

p99

0.187776

s

P2P request roundtrip: 'call_remote'

Time spent sending a call_remote request and awaiting its response.

mean

0.149164

s

SD

0.049257

s

p50

0.173994

s

<

p95

0.208773

s

<

p99

0.282513

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

11.70

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

9.57

KiB/s

<

mean

233.79

KiB/s

SD

1.18

MiB/s

<

p95

987.31

KiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

8.10

KiB/s

<

mean

16.67

KiB/s

SD

23.32

KiB/s

<

p95

41.29

KiB/s

Total bytes received

Total bytes received on primary network interface

count

1.78

GiB

mean

2.05

MiB/s

Total bytes sent

Total bytes sent on primary network interface

count

129.93

MiB

mean

149.49

KiB/s

CPU spike anomaly

CPU spike anomaly was detected

NotDetected

Memory leak anomaly

Memory leak anomaly was detected

Detected

Warning Memory growing at 1.59 MiB/s

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

NotDetected

System overload anomaly

System overload anomaly was detected

NotDetected

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

1.58

%

<

mean

17.85

%

SD

20.64

%

<

p95

73.68

%

System

6.15

%

CPU percentiles

CPU usage percentiles

p50

10.69

%

p95

73.68

%

p99

89.33

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

3

hosts

mean time

78.22

s

Memory used percentage

Percentage of memory used

p5

7.55

%

<

mean

18.29

%

SD

10.35

%

<

p95

46.52

%

Memory available percentage

Percentage of available memory

p5

53.48

%

<

mean

81.71

%

SD

10.35

%

<

p95

92.45

%

Max host memory used

Maximum memory usage percentage across all hosts

max

45.32

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

3.23

%

Memory growth rate

Rate of memory growth over time

growth

1.59

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.15

MiB/s

Disk write throughput

Disk write throughput in MiB/s

20.10

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /etc/hostname

0/2

hosts

Mount Point /etc/hosts

0/2

hosts

Mount Point /etc/nomad.d/client-meta.json

0/8

hosts

Mount Point /etc/resolv.conf

0/2

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

0.90

5 min

0.67

15 min

0.36

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

0.00

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

0

%

<

mean

5.5303

%

SD

9.4382

%

<

p95

28.19

%

60 second average

5.3971

%

300 second average

3.9941

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

<

mean

0.0145

%

SD

0.3003

%

≤

p95

0

%

60 second average

0.0055

%

300 second average

0.0017

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

<

mean

0.0112

%

SD

0.2263

%

≤

p95

0

%

60 second average

0.0039

%

300 second average

0.0013

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

0.1358

%

SD

0.6666

%

<

p95

0.7

%

60 second average

0.1132

%

300 second average

0.0895

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

0.0926

%

SD

0.4619

%

<

p95

0.48

%

60 second average

0.0763

%

300 second average

0.0608

%

Holochain process CPU usage

CPU usage by Holochain process

p5

0.26

%

<

mean

8.65

%

SD

12.65

%

<

p95

37.59

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

201.96

KiB

<

mean

227.35

KiB

SD

91.02

KiB

<

p95

251.39

KiB

Holochain process threads

Number of threads in Holochain process

p5

25

threads

<

mean

30.81

threads

SD

17.82

threads

<

p95

36

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

63

file descriptors

<

mean

74.3

file descriptors

SD

37.82

file descriptors

<

p95

80

file descriptors

Remote Signals

↑ Back to index

This scenario tests the throughput of remote_signals operations.

Started

Thu, 03 Sep 2026 08:51:00 UTC

Duration

900s

Peer count

10

Peer count at end

10

Behaviours

default (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

remote_signals_33713946045_1

Round trip time

The time from origin signal dispatch to origin receive of the remote side's response signal.

mean

0.165437

s

SD

0.458553

s

p50

0.150873

s

<

p95

0.23763

s

<

p99

0.361254

s

Timeout rate

The proportion of remote signal round-trips that timed out.

0

Holochain MetricsClick to toggle visibility.

WASM usage: 'remote_signal::init'

Metered usage count of the WASM ribosome for this zome function.

total

0

over 884.424932267s

Not enough time data to show a trend.

WASM usage: 'remote_signal::recv_remote_signal'

Metered usage count of the WASM ribosome for this zome function.

total

31187677

over 874.424859873s

mean rate

1.9607744615e+08

/s

std

1.09271109945e+09

/s

p5

0

/s

p95

5.7581269652e+08

/s

peak

1.49925575101e+10

/s

WASM usage: 'remote_signal::signal_request'

Metered usage count of the WASM ribosome for this zome function.

total

13235400

over 884.424932267s

mean rate

9.97690832e+06

/s

std

1.3400052277e+08

/s

p5

0

/s

p95

2.495598847e+07

/s

peak

3.16953316953e+09

/s

Zome call duration: 'remote_signal::recv_remote_signal'

Duration of this zome call as measured by Holochain internally.

mean

0.002226

s

SD

0.00072

s

p50

0.002459

s

<

p95

0.003401

s

<

p99

0.003784

s

Zome call duration: 'remote_signal::signal_request'

Duration of this zome call as measured by Holochain internally.

mean

0.002193

s

SD

0.000682

s

p50

0.002458

s

<

p95

0.003467

s

<

p99

0.00373

s

WASM call duration: 'remote_signal::init'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.00182

s

SD

0.000731

s

p50

0.001824

s

<

p95

0.003372

s

≤

p99

0.003372

s

WASM call duration: 'remote_signal::recv_remote_signal'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.001484

s

SD

0.000537

s

p50

0.001497

s

<

p95

0.002369

s

<

p99

0.002577

s

WASM call duration: 'remote_signal::signal_request'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.001436

s

SD

0.000485

s

p50

0.001479

s

<

p95

0.002369

s

<

p99

0.002485

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.000304

s

SD

0.000107

s

p50

0.000236

s

<

p95

0.000466

s

≤

p99

0.000466

s

Host function call duration: '__hc__emit_signal_1'

Duration of this host function call invoked from within WASM.

mean

6e-05

s

SD

2.7e-05

s

p50

5e-05

s

<

p95

0.000111

s

<

p99

0.000121

s

Host function call duration: '__hc__send_remote_signal_1'

Duration of this host function call invoked from within WASM.

mean

7e-05

s

SD

4.4e-05

s

p50

5.3e-05

s

<

p95

0.000151

s

<

p99

0.000172

s

Host function call duration: '__hc__sys_time_1'

Duration of this host function call invoked from within WASM.

mean

4e-06

s

SD

1e-06

s

p50

4e-06

s

<

p95

5e-06

s

<

p99

7e-06

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.000163

s

SD

4e-05

s

p50

0.000148

s

<

p95

0.000224

s

≤

p99

0.000224

s

Emitted local signals

Count of local signals emitted via the emit_signal host function.

total

1714

over 874.424796335s

mean rate

8605.12

/s

std

44572.08

/s

p5

0

/s

p95

32124.9

/s

peak

554631.17

/s

Sent remote signals

Count of remote signals sent via the send_remote_signal host function.

total

1725

over 884.424918854s

mean rate

12212.44

/s

std

100595.19

/s

p5

0

/s

p95

26944.74

/s

peak

1.9193858e+06

/s

Post-commit duration

Time spent executing post-commit workflows.

mean

0.000155

s

SD

0.000147

s

p50

7.5e-05

s

<

p95

0.00053

s

≤

p99

0.00053

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

46.68

s

<

mean

445.26

s

SD

256.35

s

<

p95

845.94

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

103

over 884.424914721s

mean rate

3509.34

/s

std

30950.72

/s

p5

0

/s

p95

3647.28

/s

peak

555159.01

/s

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

2.264314

s

SD

1.824696

s

p50

2.000118

s

<

p95

5.922808

s

<

p99

6.950634

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.002297

s

SD

0.001432

s

p50

0.002336

s

<

p95

0.0045

s

<

p99

0.006243

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.069892

s

SD

0.045311

s

p50

0.070042

s

<

p95

0.183982

s

≤

p99

0.183982

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.006223

s

SD

0.004417

s

p50

0.004765

s

<

p95

0.015241

s

<

p99

0.021552

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

0.27382

s

SD

0.42628

s

p50

0.203541

s

<

p95

0.842852

s

<

p99

1.488733

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

0.154413

s

SD

0.064565

s

p50

0.158385

s

<

p95

0.245801

s

<

p99

0.368473

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

0.000127

s

SD

5.6e-05

s

p50

0.000136

s

<

p95

0.000239

s

<

p99

0.000329

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.00098

s

SD

0.000772

s

p50

0.001281

s

<

p95

0.003693

s

≤

p99

0.003693

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000405

s

SD

0.000149

s

p50

0.000426

s

<

p95

0.000607

s

<

p99

0.000835

s

P2P request roundtrip: 'get'

Time spent sending a get request and awaiting its response.

mean

0.073069

s

SD

0.035235

s

p50

0.092173

s

<

p95

0.114655

s

<

p99

0.147825

s

P2P request roundtrip: 'send_validation_receipts'

Time spent sending a send_validation_receipts request and awaiting its response.

mean

0.123859

s

SD

0.033864

s

p50

0.120096

s

<

p95

0.1664

s

<

p99

0.182222

s

P2P received remote signals

Count of remote signals received.

total

1715

over 874.424791018s

mean rate

5502.4

/s

std

24889.31

/s

p5

0

/s

p95

20966.38

/s

peak

279955.21

/s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

19.08

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

14.16

KiB/s

<

mean

588.45

KiB/s

SD

1.66

MiB/s

<

p95

3.74

MiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

12.72

KiB/s

<

mean

52.32

KiB/s

SD

144.96

KiB/s

<

p95

175.29

KiB/s

Total bytes received

Total bytes received on primary network interface

count

2.00

GiB

mean

2.30

MiB/s

Total bytes sent

Total bytes sent on primary network interface

count

181.91

MiB

mean

209.30

KiB/s

CPU spike anomaly

CPU spike anomaly was detected

Detected

Warning CPU p99 reached 96.0%

Memory leak anomaly

Memory leak anomaly was detected

NotDetected

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

NotDetected

System overload anomaly

System overload anomaly was detected

NotDetected

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

1.28

%

<

mean

27.78

%

SD

27.62

%

<

p95

81.98

%

System

8.70

%

CPU percentiles

CPU usage percentiles

p50

20.85

%

p95

81.98

%

p99

96.02

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

3

hosts

mean time

94.38

s

Memory used percentage

Percentage of memory used

p5

7.28

%

<

mean

22.93

%

SD

13.41

%

<

p95

46.6

%

Memory available percentage

Percentage of available memory

p5

53.4

%

<

mean

77.07

%

SD

13.41

%

<

p95

92.72

%

Max host memory used

Maximum memory usage percentage across all hosts

max

46.04

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

3.23

%

Memory growth rate

Rate of memory growth over time

growth

0.50

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.15

MiB/s

Disk write throughput

Disk write throughput in MiB/s

16.31

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /etc/hostname

0/2

hosts

Mount Point /etc/hosts

0/2

hosts

Mount Point /etc/nomad.d/client-meta.json

0/3

hosts

Mount Point /etc/resolv.conf

0/2

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

1.52

5 min

1.18

15 min

0.65

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

0.00

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

0

%

<

mean

9.8481

%

SD

14.1886

%

<

p95

40.73

%

60 second average

9.1972

%

300 second average

7.1585

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

<

mean

0.0289

%

SD

0.4233

%

≤

p95

0

%

60 second average

0.0111

%

300 second average

0.0100

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

<

mean

0.0222

%

SD

0.3189

%

≤

p95

0

%

60 second average

0.0067

%

300 second average

0.0070

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

0.191

%

SD

0.8432

%

<

p95

1.12

%

60 second average

0.1793

%

300 second average

0.1498

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

0.1064

%

SD

0.5444

%

<

p95

0.5

%

60 second average

0.1016

%

300 second average

0.0910

%

Holochain process CPU usage

CPU usage by Holochain process

p5

0.21

%

<

mean

11.97

%

SD

19.95

%

<

p95

59.95

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

135.63

KiB

<

mean

235.43

KiB

SD

128.41

KiB

<

p95

294.40

KiB

Holochain process threads

Number of threads in Holochain process

p5

23

threads

<

mean

32.19

threads

SD

24.84

threads

<

p95

37

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

59

file descriptors

<

mean

72.89

file descriptors

SD

53.03

file descriptors

<

p95

80

file descriptors

Single-write, many-read

↑ Back to index

Creates an entry in the agent setup, then read one's own record back. This tests the maximum read performance for reading back local data when the chain is short.

Started

Thu, 03 Sep 2026 08:52:38 UTC

Duration

300s

Peer count

1

Behaviours

default (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

single_write_many_read_33713946045_1

read call timing

The time it takes to complete the zome call that retrieves the record.

mean

0.002944

s

SD

0.001329

s

p50

0.00268

s

<

p95

0.004456

s

<

p99

0.005993

s

read call rate

The number of read zome calls completed per second.

mean

3361.39

/s

Error count

The number of errors collected.

0

Holochain MetricsClick to toggle visibility.

Cascade duration

Time taken to execute a cascade (get) query inside Holochain.

mean

0.000203

s

SD

1.2e-05

s

p50

0.000198

s

<

p95

0.000216

s

<

p99

0.000218

s

WASM usage: 'crud::create_sample_entry'

Metered usage count of the WASM ribosome for this zome function.

total

0

over 270.001742524s

Not enough time data to show a trend.

WASM usage: 'crud::get_sample_entry'

Metered usage count of the WASM ribosome for this zome function.

total

3056728752

over 270.001742524s

mean rate

1.132114527e+07

/s

std

2.06045685e+06

/s

p5

8.42239356e+06

/s

p95

1.441988141e+07

/s

peak

1.45130027e+07

/s

WASM usage: 'crud::init'

Metered usage count of the WASM ribosome for this zome function.

total

0

over 270.001742524s

Not enough time data to show a trend.

WASM usage: 'crud_integrity::entry_defs'

Metered usage count of the WASM ribosome for this zome function.

total

0

over 270.001742524s

Not enough time data to show a trend.

Zome call duration: 'crud::create_sample_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.006564

s

SD

0

s

p50

0.006564

s

≤

p95

0.006564

s

≤

p99

0.006564

s

Zome call duration: 'crud::get_sample_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.001324

s

SD

6.2e-05

s

p50

0.001301

s

<

p95

0.001391

s

<

p99

0.001401

s

WASM call duration: 'crud::create_sample_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.00537

s

SD

0

s

p50

0.00537

s

≤

p95

0.00537

s

≤

p99

0.00537

s

WASM call duration: 'crud::get_sample_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.000916

s

SD

4.3e-05

s

p50

0.0009

s

<

p95

0.000964

s

<

p99

0.000971

s

WASM call duration: 'crud::init'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.000828

s

SD

0

s

p50

0.000828

s

≤

p95

0.000828

s

≤

p99

0.000828

s

WASM call duration: 'crud_integrity::entry_defs'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.001865

s

SD

0

s

p50

0.001865

s

≤

p95

0.001865

s

≤

p99

0.001865

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.000392

s

SD

0.000124

s

p50

0.000516

s

≤

p95

0.000516

s

≤

p99

0.000516

s

Host function call duration: '__hc__get_1'

Duration of this host function call invoked from within WASM.

mean

0.000224

s

SD

1.3e-05

s

p50

0.000219

s

<

p95

0.000239

s

<

p99

0.000241

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.0028

s

SD

0

s

p50

0.0028

s

≤

p95

0.0028

s

≤

p99

0.0028

s

Post-commit duration

Time spent executing post-commit workflows.

mean

9.5e-05

s

SD

0

s

p50

9.5e-05

s

≤

p95

9.5e-05

s

≤

p99

9.5e-05

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

18.06

s

<

mean

143.06

s

SD

80.78

s

<

p95

268.06

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

0

over 270.00173991s

Not enough time data to show a trend.

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.002779

s

SD

0

s

p50

0.002779

s

≤

p95

0.002779

s

≤

p99

0.002779

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.001265

s

SD

0

s

p50

0.001265

s

≤

p95

0.001265

s

≤

p99

0.001265

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.003345

s

SD

0.000565

s

p50

0.003422

s

<

p95

0.004426

s

≤

p99

0.004426

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

0.002011

s

SD

0

s

p50

0.002011

s

≤

p95

0.002011

s

≤

p99

0.002011

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

0.000443

s

SD

0

s

p50

0.000443

s

≤

p95

0.000443

s

≤

p99

0.000443

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

3.9e-05

s

SD

2e-06

s

p50

3.8e-05

s

<

p95

4.2e-05

s

<

p99

4.3e-05

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.001047

s

SD

4.3e-05

s

p50

0.001109

s

≤

p95

0.001109

s

≤

p99

0.001109

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000363

s

SD

0

s

p50

0.000363

s

≤

p95

0.000363

s

≤

p99

0.000363

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

30.20

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

18.66

KiB/s

<

mean

483.32

KiB/s

SD

1.56

MiB/s

<

p95

2.02

MiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

14.69

KiB/s

<

mean

25.09

KiB/s

SD

31.22

KiB/s

<

p95

55.96

KiB/s

Total bytes received

Total bytes received on primary network interface

count

132.16

MiB

mean

483.32

KiB/s

Total bytes sent

Total bytes sent on primary network interface

count

6.86

MiB

mean

25.09

KiB/s

CPU spike anomaly

CPU spike anomaly was detected

NotDetected

Memory leak anomaly

Memory leak anomaly was detected

Detected

Warning Memory growing at 1.13 MiB/s

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

NotDetected

System overload anomaly

System overload anomaly was detected

NotDetected

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

34.37

%

<

mean

45.3

%

SD

15.68

%

<

p95

71.14

%

System

15.10

%

CPU percentiles

CPU usage percentiles

p50

34.99

%

p95

71.14

%

p99

71.40

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

0

hosts

mean time

0.00

s

Memory used percentage

Percentage of memory used

p5

7.26

%

<

mean

7.67

%

SD

0.47

%

<

p95

8.4

%

Memory available percentage

Percentage of available memory

p5

91.6

%

<

mean

92.33

%

SD

0.47

%

<

p95

92.74

%

Max host memory used

Maximum memory usage percentage across all hosts

max

7.67

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

0.00

%

Memory growth rate

Rate of memory growth over time

growth

1.13

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.00

MiB/s

Disk write throughput

Disk write throughput in MiB/s

4.17

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /etc/nomad.d/client-meta.json

0/1

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

1.91

5 min

1.04

15 min

0.45

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

0.00

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

2.19

%

<

mean

7.5445

%

SD

8.3979

%

<

p95

22.45

%

60 second average

5.3652

%

300 second average

2.7103

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

Holochain process CPU usage

CPU usage by Holochain process

p5

27.18

%

<

mean

30.1

%

SD

5.19

%

<

p95

37.97

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

135.63

KiB

<

mean

222.12

KiB

SD

39.87

KiB

<

p95

236.76

KiB

Holochain process threads

Number of threads in Holochain process

p5

25

threads

<

mean

25.52

threads

SD

3

threads

<

p95

28

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

59

file descriptors

<

mean

60.41

file descriptors

SD

5.12

file descriptors

<

p95

66

file descriptors

Validation Receipts

↑ Back to index

Creates an entry, wait for required validation receipts, then repeat. Records the amount of time it took to accumulate the required number of receipts for all DHT operations. This is measured to the nearest 20ms so that we don't keep the agent too busy checking for receipts.

Each agent in this scenario waits for a certain number of peers to be available or for up to two minutes, whichever happens first, before starting its behaviour.

By default, this scenario will wait for a complete set of validation receipts before committing the next record. If the NO_VALIDATION_COMPLETE environment variable is set, it will instead publish new records on every round, building up an ever-growing list of action hashes to check on.

Started

Thu, 03 Sep 2026 09:07:17 UTC

Duration

900s

Peer count

20

Peer count at end

20

Behaviours

default (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

validation_receipts_33713946045_1

Receipts complete timing

The amount of time between publishing a record and receiving the required number of validation receipts.

Mean

mean

7.369875

s

SD

3.008722

s

Max

Highest per-partition mean latency

max mean

11.093387

s

Min

Lowest per-partition mean latency

min mean

5.889724

s

Receipts complete rate

The number of complete validation receipt sets collected per second.

Mean

mean

1.36

/s

Max

Highest per-partition mean rate

max

1.58

/s

Min

Lowest per-partition mean rate

min

1.06

/s

Error count

The total number of errors accumulated during the run.

0

Holochain MetricsClick to toggle visibility.

WASM usage: 'crud::create_sample_entry'

Metered usage count of the WASM ribosome for this zome function.

total

4217910

over 887.406117777s

mean rate

1.94545277e+06

/s

std

3.00633672e+06

/s

p5

0

/s

p95

8.56730799e+06

/s

peak

1.702584608e+07

/s

WASM usage: 'crud::get_sample_entry_validation_receipts'

Metered usage count of the WASM ribosome for this zome function.

total

14112789

over 887.406117777s

mean rate

4.12160608e+06

/s

std

8.77142193e+06

/s

p5

97394.03

/s

p95

2.442429173e+07

/s

peak

5.158528494e+07

/s

WASM usage: 'crud::init'

Metered usage count of the WASM ribosome for this zome function.

total

0

over 887.406117777s

Not enough time data to show a trend.

WASM usage: 'crud_integrity::entry_defs'

Metered usage count of the WASM ribosome for this zome function.

total

519436

over 887.406117777s

mean rate

237721.94

/s

std

368816.37

/s

p5

0

/s

p95

1.03710473e+06

/s

peak

2.09673449e+06

/s

Zome call duration: 'crud::create_sample_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.007637

s

SD

0.008771

s

p50

0.004959

s

<

p95

0.010948

s

<

p99

0.057921

s

Zome call duration: 'crud::get_sample_entry_validation_receipts'

Duration of this zome call as measured by Holochain internally.

mean

0.018035

s

SD

0.012793

s

p50

0.01112

s

<

p95

0.036561

s

<

p99

0.07429

s

WASM call duration: 'crud::create_sample_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.006743

s

SD

0.008243

s

p50

0.004274

s

<

p95

0.009643

s

<

p99

0.054047

s

WASM call duration: 'crud::get_sample_entry_validation_receipts'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.016922

s

SD

0.011973

s

p50

0.010128

s

<

p95

0.034479

s

<

p99

0.068404

s

WASM call duration: 'crud::init'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.002295

s

SD

0.001322

s

p50

0.002118

s

<

p95

0.006609

s

≤

p99

0.006609

s

WASM call duration: 'crud_integrity::entry_defs'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.002347

s

SD

0.003344

s

p50

0.001394

s

<

p95

0.003509

s

<

p99

0.021572

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.000617

s

SD

0.000638

s

p50

0.000468

s

<

p95

0.001937

s

<

p99

0.003467

s

Host function call duration: '__hc__get_validation_receipts_1'

Duration of this host function call invoked from within WASM.

mean

0.013869

s

SD

0.008663

s

p50

0.008129

s

<

p95

0.027665

s

<

p99

0.04155

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.003414

s

SD

0.003855

s

p50

0.002285

s

<

p95

0.005257

s

<

p99

0.025561

s

Post-commit duration

Time spent executing post-commit workflows.

mean

0.000247

s

SD

0.000223

s

p50

0.000174

s

<

p95

0.00069

s

<

p99

0.001225

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

45.95

s

<

mean

441.98

s

SD

254.93

s

<

p95

838.41

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

5948

over 887.406100796s

mean rate

250.85

/s

std

330.65

/s

p5

14.74

/s

p95

918.8

/s

peak

2971.89

/s

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

1.832626

s

SD

2.9307

s

p50

1.539753

s

<

p95

6.18031

s

<

p99

18.286251

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.003703

s

SD

0.04012

s

p50

0.001791

s

<

p95

0.022084

s

<

p99

0.040183

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.037345

s

SD

0.035724

s

p50

0.039599

s

<

p95

0.137666

s

<

p99

0.157442

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.040114

s

SD

0.027997

s

p50

0.023271

s

<

p95

0.08593

s

<

p99

0.161189

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

0.051452

s

SD

1.105358

s

p50

0.055665

s

<

p95

2.368484

s

<

p99

6.819528

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

3.129617

s

SD

12.274944

s

p50

7.716223

s

<

p95

34.166382

s

<

p99

65.937784

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

0.000738

s

SD

0.00053

s

p50

0.000495

s

<

p95

0.001479

s

<

p99

0.003188

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.003448

s

SD

0.00597

s

p50

0.001597

s

<

p95

0.012348

s

<

p99

0.031991

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000327

s

SD

0.00023

s

p50

0.000255

s

<

p95

0.000796

s

<

p99

0.001223

s

P2P request roundtrip: 'get'

Time spent sending a get request and awaiting its response.

mean

0.190384

s

SD

16.658894

s

p50

0.105601

s

<

p95

60.001287

s

<

p99

60.001522

s

P2P request roundtrip: 'send_validation_receipts'

Time spent sending a send_validation_receipts request and awaiting its response.

mean

0.168836

s

SD

0.067157

s

p50

0.15045

s

<

p95

0.25553

s

<

p99

0.407407

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

69.20

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

0

B/s

<

mean

456.40

KiB/s

SD

643.88

KiB/s

<

p95

641.40

KiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

0

B/s

<

mean

352.59

KiB/s

SD

234.77

KiB/s

<

p95

840.38

KiB/s

Total bytes received

Total bytes received on primary network interface

count

7.32

GiB

mean

8.42

MiB/s

Total bytes sent

Total bytes sent on primary network interface

count

5.65

GiB

mean

6.50

MiB/s

CPU spike anomaly

CPU spike anomaly was detected

Detected

Warning CPU p99 reached 98.5%

Memory leak anomaly

Memory leak anomaly was detected

NotDetected

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

NotDetected

System overload anomaly

System overload anomaly was detected

Detected

Warning 85% of hosts overloaded (load5/ncpus > 1.0)

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

44.54

%

<

mean

83.5

%

SD

17.66

%

<

p95

98.2

%

System

14.30

%

CPU percentiles

CPU usage percentiles

p50

91.05

%

p95

98.20

%

p99

98.45

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

18

hosts

mean time

719.50

s

Memory used percentage

Percentage of memory used

p5

11.62

%

<

mean

25.87

%

SD

9.89

%

<

p95

45.68

%

Memory available percentage

Percentage of available memory

p5

54.32

%

<

mean

74.13

%

SD

9.89

%

<

p95

88.38

%

Max host memory used

Maximum memory usage percentage across all hosts

max

46.40

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

4.10

%

Memory growth rate

Rate of memory growth over time

growth

0.40

MiB/s

Disk read throughput

Disk read throughput in MiB/s

1.59

MiB/s

Disk write throughput

Disk write throughput in MiB/s

123.19

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /

0/1

hosts

Mount Point /efi-boot

0/1

hosts

Mount Point /etc/hostname

0/10

hosts

Mount Point /etc/hosts

0/10

hosts

Mount Point /etc/nomad.d/client-meta.json

0/9

hosts

Mount Point /etc/resolv.conf

0/10

hosts

Mount Point /nix/store

0/1

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

6.31

5 min

5.06

15 min

2.97

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

85.00

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

4.06

%

<

mean

47.2854

%

SD

23.1457

%

<

p95

77.91

%

60 second average

46.7134

%

300 second average

37.4226

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0002

%

300 second average

0.0022

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0001

%

300 second average

0.0013

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

5.5866

%

SD

11.7089

%

<

p95

37.36

%

60 second average

5.4278

%

300 second average

4.1842

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

1.6968

%

SD

5.1354

%

<

p95

11

%

60 second average

1.6698

%

300 second average

1.3217

%

Holochain process CPU usage

CPU usage by Holochain process

p5

0.87

%

<

mean

5.75

%

SD

6.75

%

<

p95

12.49

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

219.31

KiB

<

mean

253.84

KiB

SD

37.06

KiB

<

p95

303.95

KiB

Holochain process threads

Number of threads in Holochain process

p5

30

threads

<

mean

33.18

threads

SD

6.08

threads

<

p95

45

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

73

file descriptors

<

mean

77.32

file descriptors

SD

10.46

file descriptors

<

p95

85

file descriptors

Write/`get_agent_activity`

↑ Back to index

A scenario where write peers write entries, while get_agent_activity peers each query a single write agent's activity with get_agent_activity.

Before a target write peer and the requesting get_agent_activity peer are in sync, this will measure the get_agent_activity call performance over a network. Once a write peer reaches sync with a get_agent_activity peer, the write peer will publish their actions and entries, and so the get_agent_activity calls will likely have most of the data they need locally. At that point this measures the database query performance and code paths through host functions.

Started

Thu, 03 Sep 2026 09:35:55 UTC

Duration

900s

Peer count

30

Peer count at end

29

Behaviours

get_agent_activity (1 agent)
write (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

write_get_agent_activity_33713946045_1

Observed chain advancement

The maximum highest-observed action sequence per write agent, aggregated across all reading agents. The reading dimension is collapsed by taking the maximum observed value for each write agent (any reader successfully tracking that point counts as propagation). Captures how far readers tracked writers’ chains during the run.

Mean max (across write agents)

Mean of the highest chain head value observed per write agent. Each write agent contributes its maximum — the value seen by any reader.

4174.4

Max

Highest chain head value observed for any single write agent.

5891

Write agents observed

5

get_agent_activity_full zome call timing

The time taken to call the zome function that retrieves information on a write peer's source chain.

Mean

mean

0.384559

s

SD

0.727337

s

Max

Highest per-partition mean latency

max mean

0.807437

s

Min

Lowest per-partition mean latency

min mean

0.014598

s

Error count

The number of errors accumulated.

20

Holochain MetricsClick to toggle visibility.

Cascade duration

Time taken to execute a cascade (get) query inside Holochain.

mean

0.058687

s

SD

0.32405

s

p50

0.000896

s

<

p95

0.880379

s

<

p99

1.454337

s

Cascade fetch errors

Count of network fetch errors during cascade calls.

total

0

over 670.005775755s

Not enough time data to show a trend.

WASM usage: 'agent_activity::announce_write_behaviour'

Metered usage count of the WASM ribosome for this zome function.

total

0

over 874.826079613s

Not enough time data to show a trend.

WASM usage: 'agent_activity::create_sample_entry'

Metered usage count of the WASM ribosome for this zome function.

total

130291596

over 874.826079613s

mean rate

7.748631225e+07

/s

std

5.824066265e+07

/s

p5

6.26652855e+06

/s

p95

1.9016913582e+08

/s

peak

2.2053940663e+08

/s

WASM usage: 'agent_activity::get_agent_activity_full'

Metered usage count of the WASM ribosome for this zome function.

total

823786393

over 877.066848373s

mean rate

4.87944197567e+09

/s

std

1.34623907184e+10

/s

p5

1.346462526e+07

/s

p95

4.91541847266e+10

/s

peak

6.03422838287e+10

/s

WASM usage: 'agent_activity::get_random_agent_with_write_behaviour'

Metered usage count of the WASM ribosome for this zome function.

total

188865008

over 887.476215271s

mean rate

7.3175446624e+08

/s

std

8.887906479e+08

/s

p5

1.025691408e+07

/s

p95

2.99467413899e+09

/s

peak

3.80438517057e+09

/s

WASM usage: 'agent_activity_integrity::entry_defs'

Metered usage count of the WASM ribosome for this zome function.

total

13407499

over 887.476215271s

mean rate

4.841676338e+07

/s

std

5.894617718e+07

/s

p5

728296.41

/s

p95

1.9865590075e+08

/s

peak

4.3057587459e+08

/s

Zome call duration: 'agent_activity::announce_write_behaviour'

Duration of this zome call as measured by Holochain internally.

mean

0.004896

s

SD

0.002696

s

p50

0.003712

s

<

p95

0.010402

s

≤

p99

0.010402

s

Zome call duration: 'agent_activity::create_sample_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.004672

s

SD

0.002217

s

p50

0.004821

s

<

p95

0.010193

s

<

p99

0.01095

s

Zome call duration: 'agent_activity::get_agent_activity_full'

Duration of this zome call as measured by Holochain internally.

mean

0.174183

s

SD

0.391368

s

p50

0.206025

s

<

p95

1.232488

s

<

p99

1.592721

s

Zome call duration: 'agent_activity::get_random_agent_with_write_behaviour'

Duration of this zome call as measured by Holochain internally.

mean

0.004032

s

SD

0.002593

s

p50

0.003887

s

<

p95

0.012171

s

≤

p99

0.012171

s

WASM call duration: 'agent_activity::announce_write_behaviour'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.004139

s

SD

0.002324

s

p50

0.003053

s

<

p95

0.008762

s

≤

p99

0.008762

s

WASM call duration: 'agent_activity::create_sample_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.003925

s

SD

0.001913

s

p50

0.004036

s

<

p95

0.00854

s

<

p99

0.009205

s

WASM call duration: 'agent_activity::get_agent_activity_full'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.175296

s

SD

0.390555

s

p50

0.20533

s

<

p95

1.231835

s

<

p99

1.592315

s

WASM call duration: 'agent_activity::get_random_agent_with_write_behaviour'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.003428

s

SD

0.002443

s

p50

0.003267

s

<

p95

0.01148

s

≤

p99

0.01148

s

WASM call duration: 'agent_activity_integrity::entry_defs'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.001199

s

SD

0.000615

s

p50

0.001136

s

<

p95

0.002607

s

<

p99

0.003113

s

Host function call duration: '__hc__agent_info_1'

Duration of this host function call invoked from within WASM.

mean

3.6e-05

s

SD

2.5e-05

s

p50

2.9e-05

s

<

p95

9.6e-05

s

≤

p99

9.6e-05

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.000567

s

SD

0.000423

s

p50

0.000534

s

<

p95

0.001649

s

<

p99

0.001782

s

Host function call duration: '__hc__create_link_1'

Duration of this host function call invoked from within WASM.

mean

0.000369

s

SD

0.0002

s

p50

0.000334

s

<

p95

0.000811

s

≤

p99

0.000811

s

Host function call duration: '__hc__get_agent_activity_1'

Duration of this host function call invoked from within WASM.

mean

0.173036

s

SD

0.390765

s

p50

0.201925

s

<

p95

1.229917

s

<

p99

1.591457

s

Host function call duration: '__hc__get_links_1'

Duration of this host function call invoked from within WASM.

mean

0.000395

s

SD

0.00035

s

p50

0.000265

s

<

p95

0.001468

s

≤

p99

0.001468

s

Host function call duration: '__hc__random_bytes_1'

Duration of this host function call invoked from within WASM.

mean

1e-05

s

SD

5e-06

s

p50

7e-06

s

<

p95

1.9e-05

s

≤

p99

1.9e-05

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.001877

s

SD

0.001134

s

p50

0.001696

s

<

p95

0.004773

s

<

p99

0.005242

s

Post-commit duration

Time spent executing post-commit workflows.

mean

0.000163

s

SD

7.2e-05

s

p50

0.000151

s

<

p95

0.000331

s

<

p99

0.000349

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

46.94

s

<

mean

442.3

s

SD

254.28

s

<

p95

837.73

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

28682

over 887.476283967s

mean rate

36960.3

/s

std

50350.75

/s

p5

814.93

/s

p95

155356.22

/s

peak

256314.76

/s

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

218.078467

s

SD

160.649012

s

p50

55.418197

s

<

p95

453.694898

s

<

p99

520.190183

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.017928

s

SD

0.181279

s

p50

0.079145

s

<

p95

0.535612

s

<

p99

0.976018

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

<

p99

2e-06

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.306445

s

SD

0.514367

s

p50

0.945488

s

<

p95

1.314506

s

<

p99

2.448987

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.168839

s

SD

0.151764

s

p50

0.060423

s

<

p95

0.286874

s

<

p99

0.44264

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

2.386628

s

SD

20.537786

s

p50

8.916398

s

<

p95

70.43093

s

<

p99

93.47042

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

20.954244

s

SD

34.365098

s

p50

2.158011

s

<

p95

91.736672

s

<

p99

151.112717

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

0.002858

s

SD

0.001297

s

p50

0.002493

s

<

p95

0.004931

s

<

p99

0.005631

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.002649

s

SD

0.003449

s

p50

0.001403

s

<

p95

0.009486

s

<

p99

0.020309

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000384

s

SD

0.000299

s

p50

0.000269

s

<

p95

0.00109

s

<

p99

0.001602

s

P2P request roundtrip: 'get'

Time spent sending a get request and awaiting its response.

mean

2.005176

s

SD

25.778916

s

p50

0.744756

s

<

p95

60.001453

s

<

p99

60.002009

s

P2P request roundtrip: 'get_links'

Time spent sending a get_links request and awaiting its response.

mean

0.552039

s

SD

0.206933

s

p50

0.423347

s

<

p95

0.786118

s

≤

p99

0.786118

s

P2P request roundtrip: 'get_agent_activity'

Time spent sending a get_agent_activity request and awaiting its response.

mean

0.154

s

SD

27.671776

s

p50

0.413031

s

<

p95

60.001897

s

<

p99

60.00367

s

P2P request roundtrip: 'send_validation_receipts'

Time spent sending a send_validation_receipts request and awaiting its response.

mean

0.344789

s

SD

4.468909

s

p50

0.376261

s

<

p95

15.151741

s

<

p99

17.753529

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

54.80

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

91.83

KiB/s

<

mean

349.00

KiB/s

SD

600.17

KiB/s

<

p95

631.44

KiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

39.60

KiB/s

<

mean

258.07

KiB/s

SD

282.81

KiB/s

<

p95

820.97

KiB/s

Total bytes received

Total bytes received on primary network interface

count

7.67

GiB

mean

8.82

MiB/s

Total bytes sent

Total bytes sent on primary network interface

count

5.67

GiB

mean

6.52

MiB/s

CPU spike anomaly

CPU spike anomaly was detected

Detected

Warning CPU p99 reached 98.6%

Memory leak anomaly

Memory leak anomaly was detected

NotDetected

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

NotDetected

System overload anomaly

System overload anomaly was detected

Detected

Warning 59% of hosts overloaded (load5/ncpus > 1.0)

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

17.66

%

<

mean

67.57

%

SD

25.43

%

<

p95

98.25

%

System

12.76

%

CPU percentiles

CPU usage percentiles

p50

76.61

%

p95

98.25

%

p99

98.60

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

17

hosts

mean time

609.86

s

Memory used percentage

Percentage of memory used

p5

6.43

%

<

mean

19.19

%

SD

8.73

%

<

p95

36.51

%

Memory available percentage

Percentage of available memory

p5

63.49

%

<

mean

80.81

%

SD

8.73

%

<

p95

93.57

%

Max host memory used

Maximum memory usage percentage across all hosts

max

37.70

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

5.13

%

Memory growth rate

Rate of memory growth over time

growth

0.28

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.03

MiB/s

Disk write throughput

Disk write throughput in MiB/s

135.43

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /

0/3

hosts

Mount Point /boot

0/1

hosts

Mount Point /efi-boot

0/2

hosts

Mount Point /etc/hostname

0/13

hosts

Mount Point /etc/hosts

0/13

hosts

Mount Point /etc/nomad.d/client-meta.json

0/11

hosts

Mount Point /etc/resolv.conf

0/13

hosts

Mount Point /nix/store

0/3

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

4.41

5 min

4.73

15 min

4.78

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

59.26

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

0

%

<

mean

27.4381

%

SD

25.9034

%

<

p95

79.26

%

60 second average

27.5180

%

300 second average

30.3708

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

3.9867

%

SD

8.1802

%

<

p95

26.38

%

60 second average

3.9880

%

300 second average

4.0794

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

1.6394

%

SD

4.4852

%

<

p95

10.19

%

60 second average

1.6697

%

300 second average

1.5527

%

Holochain process CPU usage

CPU usage by Holochain process

p5

15.35

%

<

mean

61.54

%

SD

24.94

%

<

p95

96.43

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

264.56

KiB

<

mean

350.04

KiB

SD

69.49

KiB

<

p95

488.29

KiB

Holochain process threads

Number of threads in Holochain process

p5

29

threads

<

mean

35.47

threads

SD

8.9

threads

<

p95

53

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

71

file descriptors

<

mean

79.58

file descriptors

SD

13.09

file descriptors

<

p95

99

file descriptors

Write/`get_agent_activity` with volatile conductors

↑ Back to index

A scenario where write peers write entries, while get_agent_activity_volatile peers each query a single write agents activity with get_agent_activity but shutdown and restart their conductors at semi-random intervals.

Before a target write peer and the requesting get_agent_activity_volatile peer are in sync, this will measure the get_agent_activity call performance over a network. Once a write peer reaches sync with a get_agent_activity peer, the write peer will publish their actions and entries, and so the get_agent_activity calls will likely have most of the data they need locally. At that point this measures the database query performance and code paths through host functions.

Started

Thu, 03 Sep 2026 09:51:38 UTC

Duration

1800s

Peer count

30

Peer count at end

26

Behaviours

get_agent_activity_volatile (1 agent)
write (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

write_get_agent_activity_volatile_33713946045_1

Highest observed action_seq

The rate at which zero-arc readers observe new chain heads on the writers' chains via get_agent_activity. This reflects the DHT's ability to propagate agent activity ops and make them available to querying peers.

Mean

mean

40.6

/s

Max

Highest per-partition mean rate

max

375.79

/s

Min

Lowest per-partition mean rate

min

5.98

/s

get_agent_activity_full zome call timing

The time taken to call the zome function that retrieves information on a write peer's source chain.

Mean

mean

0.504529

s

SD

0.593883

s

Max

Highest per-partition mean latency

max mean

1.188908

s

Min

Lowest per-partition mean latency

min mean

0.017807

s

Running volatile conductors count

The number of conductors running by get_agent_activity_volatile peers.

p5

10

conductors

<

mean

13.52

conductors

SD

2.18

conductors

<

p95

17

conductors

Volatile conductor total running duration

The total running duration of a get_agent_activity_volatile conductor.

Mean

mean

746.035616

s

SD

381.740022

s

Max

Highest per-partition mean latency

max mean

1011.742857

s

Min

Lowest per-partition mean latency

min mean

610.976744

s

Volatile conductor running duration

The duration that a get_agent_activity_volatile conductor was running before being stopped.

Mean

mean

34.405915

s

SD

20.259833

s

Max

Highest per-partition mean latency

max mean

139.170811

s

Min

Lowest per-partition mean latency

min mean

24.095757

s

Volatile conductor stopped duration

The duration that a get_agent_activity_volatile conductor was stopped before being started again.

Mean

mean

20.596135

s

SD

22.097954

s

Max

Highest per-partition mean latency

max mean

30.66641

s

Min

Lowest per-partition mean latency

min mean

8.307177

s

Reached target arc

Did get_agent_activity_volatile peer reach their target arc in the moment before they were shutdown.

get_agent_activity_volatile_agent: uhCAk-taLrSLz0VPDa3aSSCSZsPAc7tP3L8ZGGuppHRtivu5FO6g9

p5

0

<

mean

0.29

SD

0.45

<

p95

1

get_agent_activity_volatile_agent: uhCAk-wL7yjFe_3_442hd_46rpOrPRyf1rpqGc37B0HvZGAvSvidA

p5

0

<

mean

0.21

SD

0.41

<

p95

1

get_agent_activity_volatile_agent: uhCAk07vPKpggp8d8lO89hypFrTIjOtC9LOv7ErpXETDHdBJU29ds

p5

0

<

mean

0.21

SD

0.41

<

p95

1

get_agent_activity_volatile_agent: uhCAkAl4YxzEffupn-xv_bMusEDG3F3c0zl3Ouhq3ptVBqq2sCUXK

p5

0

≤

mean

0

SD

0

≤

p95

0

get_agent_activity_volatile_agent: uhCAkC7PmZxccZuwGdFOr5AZ-EGb6nnnY9jvX6b2OBR2sF2uIcpGf

p5

0

<

mean

0.09

SD

0.28

<

p95

1

get_agent_activity_volatile_agent: uhCAkCiXpN3uLrYBowsKLrogrOqLvKg0UP6VL9dzeTrJMuHsxAL_U

p5

0

<

mean

0.22

SD

0.41

<

p95

1

get_agent_activity_volatile_agent: uhCAkDlS_iJINrcHlbONrL6lKtGyHLGbJYqEgKfTywAz7l8vQHnLb

p5

0

<

mean

0.16

SD

0.37

<

p95

1

get_agent_activity_volatile_agent: uhCAkH97plVG2bqtb9w4tNn69zLGuYtSVz7VH_pQKjj160n6tIMgY

p5

0

<

mean

0.2

SD

0.4

<

p95

1

get_agent_activity_volatile_agent: uhCAkHbLkUDBwnX9LqkaeGfKlXO4Zaw2rAhP0ulJaoWs4MWUhzex1

p5

0

<

mean

0.13

SD

0.34

<

p95

1

get_agent_activity_volatile_agent: uhCAkJbXeJO_28zVnXSzVgqOxpunzwWloFwGcjYxGh_I4Cs3Lqrgj

p5

0

<

mean

0.22

SD

0.42

<

p95

1

get_agent_activity_volatile_agent: uhCAkLO2oiTmZNvi_xYBa6O-NoGEAitYzDZ52ZHlDpRe5eCg3y7a4

p5

0

<

mean

0.24

SD

0.43

<

p95

1

get_agent_activity_volatile_agent: uhCAkPVDZuUbbqLDME_MJxsW53GYmtv66d1w1MJELrgMkpD5Usmw6

p5

0

<

mean

0.2

SD

0.4

<

p95

1

get_agent_activity_volatile_agent: uhCAkTBJUxspaepVI1ikWmccL7cU2uEuwOPFNxY1CZvfUoOizX14F

p5

0

≤

mean

0

SD

0

≤

p95

0

get_agent_activity_volatile_agent: uhCAkUwIacif10nlCASmEaRIEXeEZsGO1wH3MhLphHr9gW099K4HR

p5

0

<

mean

0.15

SD

0.35

<

p95

1

get_agent_activity_volatile_agent: uhCAkXpYpSXQtKrM4XbgMOmZFWSq_QKiti1mOt1gTADCuyeuZEYNx

p5

0

<

mean

0.24

SD

0.43

<

p95

1

get_agent_activity_volatile_agent: uhCAkeymqI1wSnzNjTPZ_FlseHIuEN8QCYFlJSXyFiBrtzpWciYYO

p5

0

<

mean

0.14

SD

0.35

<

p95

1

get_agent_activity_volatile_agent: uhCAkfC4CX5wnyeFlvEtwJ6IAyu0Wnph3C9qPXVMtFsQZeC779CBZ

p5

0

<

mean

0.29

SD

0.45

<

p95

1

get_agent_activity_volatile_agent: uhCAkv5jS7RU_JD2g7C0RkPq9e4AesVFDgVb9eQ2OwVGMHegZCqmH

p5

0

<

mean

0.23

SD

0.42

<

p95

1

get_agent_activity_volatile_agent: uhCAkxDP0-LOFAeSn8zMy8dbB5gXLxbC9U59LRsbuasiL_R2cXZ-9

p5

0

<

mean

0.21

SD

0.41

<

p95

1

get_agent_activity_volatile_agent: uhCAkzJywv_oJXoSj9W4MXWLj4NUfPjN9ZvuiuI2R1La6aWhJrmMt

p5

0

<

mean

0.28

SD

0.45

<

p95

1

Error count

The number of errors accumulated.

132

Holochain MetricsClick to toggle visibility.

Cascade duration

Time taken to execute a cascade (get) query inside Holochain.

mean

0.095609

s

SD

2.114689

s

p50

0.503941

s

<

p95

2.199832

s

<

p99

3.818146

s

Cascade fetch errors

Count of network fetch errors during cascade calls.

total

10

over 1675.765032303s

mean rate

20.12

/s

std

36.24

/s

p5

0

/s

p95

101.41

/s

peak

108.55

/s

Zome call duration: 'agent_activity::announce_write_behaviour'

Duration of this zome call as measured by Holochain internally.

mean

0.004928

s

SD

0.001898

s

p50

0.00426

s

<

p95

0.008123

s

≤

p99

0.008123

s

Zome call duration: 'agent_activity::create_sample_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.005272

s

SD

0.001197

s

p50

0.00495

s

<

p95

0.008007

s

<

p99

0.008353

s

Zome call duration: 'agent_activity::get_agent_activity_full'

Duration of this zome call as measured by Holochain internally.

mean

0.185405

s

SD

0.786256

s

p50

0.544077

s

<

p95

2.21081

s

<

p99

3.423199

s

Zome call duration: 'agent_activity::get_random_agent_with_write_behaviour'

Duration of this zome call as measured by Holochain internally.

mean

0.003296

s

SD

0.396987

s

p50

0.003834

s

<

p95

0.78185

s

<

p99

2.020081

s

WASM call duration: 'agent_activity::announce_write_behaviour'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.004082

s

SD

0.001488

s

p50

0.003571

s

<

p95

0.006227

s

≤

p99

0.006227

s

WASM call duration: 'agent_activity::create_sample_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.004436

s

SD

0.001052

s

p50

0.004167

s

<

p95

0.006804

s

<

p99

0.007147

s

WASM call duration: 'agent_activity::get_agent_activity_full'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.189524

s

SD

2.156715

s

p50

0.537727

s

<

p95

2.207711

s

<

p99

4.185477

s

WASM call duration: 'agent_activity::get_random_agent_with_write_behaviour'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.002775

s

SD

0.302541

s

p50

0.003228

s

<

p95

0.595467

s

<

p99

1.493233

s

WASM call duration: 'agent_activity_integrity::entry_defs'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.00146

s

SD

0.000391

s

p50

0.001394

s

<

p95

0.002325

s

<

p99

0.002435

s

Host function call duration: '__hc__agent_info_1'

Duration of this host function call invoked from within WASM.

mean

2.5e-05

s

SD

9e-06

s

p50

2.1e-05

s

<

p95

4.3e-05

s

≤

p99

4.3e-05

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.000523

s

SD

0.000142

s

p50

0.000485

s

<

p95

0.000838

s

<

p99

0.000909

s

Host function call duration: '__hc__create_link_1'

Duration of this host function call invoked from within WASM.

mean

0.000494

s

SD

0.000254

s

p50

0.000379

s

<

p95

0.000962

s

≤

p99

0.000962

s

Host function call duration: '__hc__get_agent_activity_1'

Duration of this host function call invoked from within WASM.

mean

0.185637

s

SD

2.156444

s

p50

0.528996

s

<

p95

2.199914

s

<

p99

4.176294

s

Host function call duration: '__hc__get_links_1'

Duration of this host function call invoked from within WASM.

mean

0.000382

s

SD

0.301543

s

p50

0.000274

s

<

p95

0.592

s

<

p99

1.484161

s

Host function call duration: '__hc__random_bytes_1'

Duration of this host function call invoked from within WASM.

mean

5e-06

s

SD

4e-06

s

p50

6e-06

s

<

p95

1.3e-05

s

<

p99

1.9e-05

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.002298

s

SD

0.000715

s

p50

0.00214

s

<

p95

0.003241

s

<

p99

0.003743

s

Post-commit duration

Time spent executing post-commit workflows.

mean

0.000151

s

SD

3.4e-05

s

p50

0.000143

s

<

p95

0.000218

s

<

p99

0.000228

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

5.19

s

<

mean

372.08

s

SD

533.39

s

<

p95

1557.04

s

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

128.499891

s

SD

170.186033

s

p50

78.308311

s

<

p95

524.964836

s

<

p99

918.696578

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.016024

s

SD

0.097723

s

p50

0.008248

s

<

p95

0.264312

s

<

p99

0.372238

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

2e-06

s

p50

0

s

≤

p95

0

s

<

p99

2e-06

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.048183

s

SD

0.097751

s

p50

0.022141

s

<

p95

0.199827

s

<

p99

0.460766

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.324319

s

SD

0.573266

s

p50

0.069436

s

<

p95

1.822315

s

<

p99

1.916112

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

4.003204

s

SD

32.341754

s

p50

11.910575

s

<

p95

88.595521

s

<

p99

100.03

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

81.335126

s

SD

83.369727

s

p50

37.005386

s

<

p95

243.422054

s

<

p99

343.804229

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

0.009103

s

SD

0.005074

s

p50

0.000916

s

<

p95

0.016537

s

<

p99

0.019012

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.009005

s

SD

0.020215

s

p50

0.001157

s

<

p95

0.018175

s

<

p99

0.053206

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000422

s

SD

0.000326

s

p50

0.000404

s

<

p95

0.000912

s

<

p99

0.001734

s

P2P request roundtrip: 'get'

Time spent sending a get request and awaiting its response.

mean

12.313896

s

SD

27.585406

s

p50

0.91502

s

<

p95

60.001518

s

<

p99

60.001596

s

P2P request roundtrip: 'get_links'

Time spent sending a get_links request and awaiting its response.

mean

0.438054

s

SD

0.195494

s

p50

0.393765

s

<

p95

0.950803

s

≤

p99

0.950803

s

P2P request roundtrip: 'get_agent_activity'

Time spent sending a get_agent_activity request and awaiting its response.

mean

0.254022

s

SD

6.251801

s

p50

0.53572

s

<

p95

2.33098

s

<

p99

60.000774

s

P2P request roundtrip: 'send_validation_receipts'

Time spent sending a send_validation_receipts request and awaiting its response.

mean

0.363132

s

SD

25.806462

s

p50

0.547385

s

<

p95

60.001471

s

<

p99

60.002076

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

25.63

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

1.98

KiB/s

<

mean

209.45

KiB/s

SD

421.68

KiB/s

<

p95

795.41

KiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

1.92

KiB/s

<

mean

203.34

KiB/s

SD

283.21

KiB/s

<

p95

759.40

KiB/s

Total bytes received

Total bytes received on primary network interface

count

9.38

GiB

mean

5.37

MiB/s

Total bytes sent

Total bytes sent on primary network interface

count

9.11

GiB

mean

5.21

MiB/s

CPU spike anomaly

CPU spike anomaly was detected

Detected

Warning CPU p99 reached 98.8%

Memory leak anomaly

Memory leak anomaly was detected

NotDetected

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

NotDetected

System overload anomaly

System overload anomaly was detected

Detected

Warning 25% of hosts overloaded (load5/ncpus > 1.0)

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

1.05

%

<

mean

34.09

%

SD

35.38

%

<

p95

97.84

%

System

8.46

%

CPU percentiles

CPU usage percentiles

p50

15.89

%

p95

97.84

%

p99

98.85

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

11

hosts

mean time

905.80

s

Memory used percentage

Percentage of memory used

p5

5.73

%

<

mean

17.95

%

SD

8.21

%

<

p95

32.66

%

Memory available percentage

Percentage of available memory

p5

67.34

%

<

mean

82.05

%

SD

8.21

%

<

p95

94.27

%

Max host memory used

Maximum memory usage percentage across all hosts

max

34.62

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

5.13

%

Memory growth rate

Rate of memory growth over time

growth

0.14

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.15

MiB/s

Disk write throughput

Disk write throughput in MiB/s

41.56

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /

0/3

hosts

Mount Point /boot

0/1

hosts

Mount Point /efi-boot

0/2

hosts

Mount Point /etc/hostname

0/14

hosts

Mount Point /etc/hosts

0/14

hosts

Mount Point /etc/nomad.d/client-meta.json

0/11

hosts

Mount Point /etc/resolv.conf

0/14

hosts

Mount Point /nix/store

0/3

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

2.42

5 min

2.62

15 min

3.20

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

25.00

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

0

%

<

mean

16.481

%

SD

28.4587

%

<

p95

83.51

%

60 second average

16.2775

%

300 second average

16.9263

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

1.8084

%

SD

6.0361

%

<

p95

16.83

%

60 second average

1.9235

%

300 second average

2.4079

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

0.759

%

SD

3.3096

%

<

p95

3.53

%

60 second average

0.8534

%

300 second average

1.1422

%

Holochain process CPU usage

CPU usage by Holochain process

p5

0

%

<

mean

26.48

%

SD

34.16

%

<

p95

96.8

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

96.96

KiB

<

mean

205.20

KiB

SD

118.56

KiB

<

p95

408.11

KiB

Holochain process threads

Number of threads in Holochain process

p5

1

threads

<

mean

25.64

threads

SD

10.25

threads

<

p95

36

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

0

file descriptors

<

mean

64.78

file descriptors

SD

24.11

file descriptors

<

p95

84

file descriptors

Write/Query

↑ Back to index

Creates an entry, then queries the source chain and performs a simple operation on the entries, then repeats.

Started

Thu, 03 Sep 2026 08:54:30 UTC

Duration

300s

Peer count

1

Behaviours

default (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

write_query_33713946045_1

Write timing

The time it takes to complete the zome function call that writes the entry.

mean

0.032204

s

SD

0.01617

s

p50

0.031297

s

<

p95

0.054587

s

<

p99

0.082102

s

Write rate

The number of writes completed per second.

mean

89.68

/s

Query timing

The time it takes to complete the chain_query_count_len zome call that queries the source chain.

mean

0.073238

s

SD

0.040547

s

p50

0.07111

s

<

p95

0.134502

s

<

p99

0.15239

s

Query rate

The number of chain queries completed per second.

mean

89.68

/s

Error count

The number of errors accumulated during the run.

0

Holochain MetricsClick to toggle visibility.

WASM usage: 'crud::chain_query_count_len'

Metered usage count of the WASM ribosome for this zome function.

total

80512243020

over 280.005669192s

mean rate

2.8753783863e+08

/s

std

3.116817935e+07

/s

p5

2.2599130038e+08

/s

p95

3.1294769239e+08

/s

peak

3.1617757239e+08

/s

WASM usage: 'crud::create_sample_entry'

Metered usage count of the WASM ribosome for this zome function.

total

95367660

over 280.005669192s

mean rate

340592.18

/s

std

219038.79

/s

p5

189447.36

/s

p95

757789.71

/s

peak

1.24034668e+06

/s

WASM usage: 'crud::init'

Metered usage count of the WASM ribosome for this zome function.

total

0

over 280.005669192s

Not enough time data to show a trend.

WASM usage: 'crud_integrity::entry_defs'

Metered usage count of the WASM ribosome for this zome function.

total

23489072

over 280.005669192s

mean rate

83887.92

/s

std

53925.81

/s

p5

46660.92

/s

p95

186203.55

/s

peak

305497.61

/s

Zome call duration: 'crud::chain_query_count_len'

Duration of this zome call as measured by Holochain internally.

mean

0.030073

s

SD

0.009575

s

p50

0.027441

s

<

p95

0.039017

s

<

p99

0.039822

s

Zome call duration: 'crud::create_sample_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.00342

s

SD

3.3e-05

s

p50

0.003417

s

<

p95

0.003438

s

<

p99

0.003507

s

WASM call duration: 'crud::chain_query_count_len'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.029444

s

SD

0.009542

s

p50

0.026819

s

<

p95

0.03836

s

<

p99

0.039163

s

WASM call duration: 'crud::create_sample_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.002841

s

SD

2.5e-05

s

p50

0.00284

s

<

p95

0.002855

s

<

p99

0.002924

s

WASM call duration: 'crud::init'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.002853

s

SD

0

s

p50

0.002853

s

≤

p95

0.002853

s

≤

p99

0.002853

s

WASM call duration: 'crud_integrity::entry_defs'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.001007

s

SD

1.3e-05

s

p50

0.001

s

<

p95

0.001017

s

≤

p99

0.001017

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.000279

s

SD

0.000114

s

p50

0.000505

s

≤

p95

0.000505

s

≤

p99

0.000505

s

Host function call duration: '__hc__query_1'

Duration of this host function call invoked from within WASM.

mean

0.012405

s

SD

0.004105

s

p50

0.011368

s

<

p95

0.016037

s

<

p99

0.016352

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.001522

s

SD

4.8e-05

s

p50

0.001488

s

<

p95

0.001598

s

<

p99

0.001601

s

Post-commit duration

Time spent executing post-commit workflows.

mean

0.000119

s

SD

4e-06

s

p50

0.000119

s

<

p95

0.000122

s

≤

p99

0.000122

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

16.66

s

<

mean

146.67

s

SD

83.67

s

<

p95

276.67

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

0

over 280.005620374s

Not enough time data to show a trend.

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.001343

s

SD

0

s

p50

0.001343

s

≤

p95

0.001343

s

≤

p99

0.001343

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.001834

s

SD

0.000183

s

p50

0.001818

s

<

p95

0.001917

s

<

p99

0.001919

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.025423

s

SD

0.008219

s

p50

0.023538

s

<

p95

0.032578

s

<

p99

0.033143

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

0.001972

s

SD

0

s

p50

0.001972

s

≤

p95

0.001972

s

≤

p99

0.001972

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

0.002913

s

SD

0

s

p50

0.002913

s

≤

p95

0.002913

s

≤

p99

0.002913

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

0.002385

s

SD

0.000807

s

p50

0.002218

s

<

p95

0.003065

s

<

p99

0.003119

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.001095

s

SD

8.9e-05

s

p50

0.001167

s

≤

p95

0.001167

s

≤

p99

0.001167

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000248

s

SD

3e-06

s

p50

0.000247

s

<

p95

0.00025

s

<

p99

0.000255

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

55.25

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

13.92

KiB/s

<

mean

16.82

KiB/s

SD

2.55

KiB/s

<

p95

21.42

KiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

17.41

KiB/s

<

mean

20.38

KiB/s

SD

3.62

KiB/s

<

p95

28.35

KiB/s

Total bytes received

Total bytes received on primary network interface

count

4.76

MiB

mean

16.82

KiB/s

Total bytes sent

Total bytes sent on primary network interface

count

5.77

MiB

mean

20.38

KiB/s

CPU spike anomaly

CPU spike anomaly was detected

NotDetected

Memory leak anomaly

Memory leak anomaly was detected

NotDetected

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

NotDetected

System overload anomaly

System overload anomaly was detected

NotDetected

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

69.41

%

<

mean

72.8

%

SD

1.85

%

<

p95

74.7

%

System

17.56

%

CPU percentiles

CPU usage percentiles

p50

73.12

%

p95

74.70

%

p99

78.30

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

0

hosts

mean time

0.00

s

Memory used percentage

Percentage of memory used

p5

25.58

%

<

mean

25.82

%

SD

0.21

%

<

p95

26.15

%

Memory available percentage

Percentage of available memory

p5

73.85

%

<

mean

74.18

%

SD

0.21

%

<

p95

74.42

%

Max host memory used

Maximum memory usage percentage across all hosts

max

25.82

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

0.00

%

Memory growth rate

Rate of memory growth over time

growth

-0.03

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.00

MiB/s

Disk write throughput

Disk write throughput in MiB/s

2.69

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /etc/nomad.d/client-meta.json

0/1

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

2.22

5 min

1.67

15 min

0.89

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

0.00

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

24.09

%

<

mean

29.0623

%

SD

3.0955

%

<

p95

33

%

60 second average

26.3893

%

300 second average

17.6327

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0.07

%

<

mean

0.7163

%

SD

0.6411

%

<

p95

2.06

%

60 second average

0.6540

%

300 second average

0.5960

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0.01

%

<

mean

0.2553

%

SD

0.3459

%

<

p95

0.98

%

60 second average

0.2440

%

300 second average

0.3477

%

Holochain process CPU usage

CPU usage by Holochain process

p5

55.72

%

<

mean

59.26

%

SD

2.08

%

<

p95

61.51

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

205.96

KiB

<

mean

216.47

KiB

SD

23.85

KiB

<

p95

227.45

KiB

Holochain process threads

Number of threads in Holochain process

p5

23

threads

<

mean

23.23

threads

SD

0.5

threads

<

p95

24

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

63

file descriptors

<

mean

63.6

file descriptors

SD

1.28

file descriptors

<

p95

67

file descriptors

Write/Read

↑ Back to index

Creates an entry and reads it back, then repeats.

Started

Thu, 03 Sep 2026 08:56:05 UTC

Duration

300s

Peer count

1

Behaviours

default (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

write_read_33713946045_1

Write timing

The time taken to write the entry.

mean

0.044411

s

SD

0.027459

s

p50

0.038839

s

<

p95

0.093169

s

<

p99

0.120562

s

Write rate

The number of writes completed per second.

mean

99.07

/s

Read timing

The time taken to read the entry.

mean

0.046765

s

SD

0.030904

s

p50

0.039556

s

<

p95

0.099859

s

<

p99

0.137344

s

Read rate

The number of reads completed per second.

mean

99.04

/s

Error count

The number of errors accumulated over the run.

0

Holochain MetricsClick to toggle visibility.

Cascade duration

Time taken to execute a cascade (get) query inside Holochain.

mean

0.000564

s

SD

9.3e-05

s

p50

0.000543

s

<

p95

0.000663

s

<

p99

0.000672

s

WASM usage: 'crud::create_sample_entry'

Metered usage count of the WASM ribosome for this zome function.

total

106198395

over 280.00668923s

mean rate

379273.4

/s

std

253281.24

/s

p5

178724.08

/s

p95

904343.09

/s

peak

1.33685675e+06

/s

WASM usage: 'crud::get_sample_entry'

Metered usage count of the WASM ribosome for this zome function.

total

99075134

over 280.00668923s

mean rate

353833.65

/s

std

236109.41

/s

p5

166789.14

/s

p95

843952.35

/s

peak

1.24443499e+06

/s

WASM usage: 'crud::init'

Metered usage count of the WASM ribosome for this zome function.

total

0

over 280.00668923s

Not enough time data to show a trend.

WASM usage: 'crud_integrity::entry_defs'

Metered usage count of the WASM ribosome for this zome function.

total

13078342

over 280.00668923s

mean rate

46707.55

/s

std

31191.61

/s

p5

22009.89

/s

p95

111369.93

/s

peak

164634.03

/s

Zome call duration: 'crud::create_sample_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.003092

s

SD

0.000127

s

p50

0.003053

s

<

p95

0.00325

s

<

p99

0.003265

s

Zome call duration: 'crud::get_sample_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.002112

s

SD

0.00015

s

p50

0.00207

s

<

p95

0.002289

s

<

p99

0.002304

s

WASM call duration: 'crud::create_sample_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.002515

s

SD

0.000111

s

p50

0.00248

s

<

p95

0.002656

s

<

p99

0.002669

s

WASM call duration: 'crud::get_sample_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.00154

s

SD

0.000131

s

p50

0.001506

s

<

p95

0.001693

s

<

p99

0.001707

s

WASM call duration: 'crud::init'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.001119

s

SD

0

s

p50

0.001119

s

≤

p95

0.001119

s

≤

p99

0.001119

s

WASM call duration: 'crud_integrity::entry_defs'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.000804

s

SD

3.4e-05

s

p50

0.000793

s

<

p95

0.000847

s

<

p99

0.000853

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.000292

s

SD

1.5e-05

s

p50

0.000275

s

<

p95

0.000313

s

<

p99

0.000318

s

Host function call duration: '__hc__get_1'

Duration of this host function call invoked from within WASM.

mean

0.000597

s

SD

9.4e-05

s

p50

0.000575

s

<

p95

0.000697

s

<

p99

0.000706

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.001289

s

SD

5.9e-05

s

p50

0.00127

s

<

p95

0.001363

s

<

p99

0.001372

s

Post-commit duration

Time spent executing post-commit workflows.

mean

8.8e-05

s

SD

7e-06

s

p50

8.6e-05

s

<

p95

9.6e-05

s

<

p99

9.7e-05

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

17.52

s

<

mean

147.52

s

SD

83.67

s

<

p95

277.53

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

0

over 280.006678987s

Not enough time data to show a trend.

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.002969

s

SD

0

s

p50

0.002969

s

≤

p95

0.002969

s

≤

p99

0.002969

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.001206

s

SD

0.000149

s

p50

0.001157

s

<

p95

0.001366

s

<

p99

0.001381

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.029764

s

SD

0.010529

s

p50

0.026962

s

<

p95

0.040849

s

<

p99

0.041754

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

0.004127

s

SD

0

s

p50

0.004127

s

≤

p95

0.004127

s

≤

p99

0.004127

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

0.001651

s

SD

0

s

p50

0.001651

s

≤

p95

0.001651

s

≤

p99

0.001651

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

0.002618

s

SD

0.000929

s

p50

0.002384

s

<

p95

0.003552

s

<

p99

0.003622

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.00095

s

SD

0.000327

s

p50

0.001541

s

≤

p95

0.001541

s

≤

p99

0.001541

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000259

s

SD

1.9e-05

s

p50

0.000254

s

<

p95

0.000282

s

<

p99

0.000284

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

60.29

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

19.04

KiB/s

<

mean

901.53

KiB/s

SD

2.01

MiB/s

<

p95

6.73

MiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

14.74

KiB/s

<

mean

36.70

KiB/s

SD

43.86

KiB/s

<

p95

126.09

KiB/s

Total bytes received

Total bytes received on primary network interface

count

246.51

MiB

mean

901.53

KiB/s

Total bytes sent

Total bytes sent on primary network interface

count

10.03

MiB

mean

36.70

KiB/s

CPU spike anomaly

CPU spike anomaly was detected

Detected

Warning CPU p99 reached 95.8%

Memory leak anomaly

Memory leak anomaly was detected

Detected

Warning Memory growing at 2.30 MiB/s

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

NotDetected

System overload anomaly

System overload anomaly was detected

NotDetected

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

69.43

%

<

mean

79.21

%

SD

8.21

%

<

p95

92.89

%

System

18.91

%

CPU percentiles

CPU usage percentiles

p50

79.52

%

p95

92.89

%

p99

95.75

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

1

hosts

mean time

128.57

s

Memory used percentage

Percentage of memory used

p5

8.32

%

<

mean

9.06

%

SD

0.72

%

<

p95

10.24

%

Memory available percentage

Percentage of available memory

p5

89.76

%

<

mean

90.94

%

SD

0.72

%

<

p95

91.68

%

Max host memory used

Maximum memory usage percentage across all hosts

max

9.06

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

0.00

%

Memory growth rate

Rate of memory growth over time

growth

2.30

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.00

MiB/s

Disk write throughput

Disk write throughput in MiB/s

6.43

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /etc/nomad.d/client-meta.json

0/1

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

4.18

5 min

2.41

15 min

1.10

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

0.00

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

18.66

%

<

mean

29.5969

%

SD

12.2371

%

<

p95

53.66

%

60 second average

23.1207

%

300 second average

11.1686

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

Holochain process CPU usage

CPU usage by Holochain process

p5

16.15

%

<

mean

31.83

%

SD

9.19

%

<

p95

38.88

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

26.84

KiB

<

mean

273.85

KiB

SD

124.93

KiB

<

p95

433.95

KiB

Holochain process threads

Number of threads in Holochain process

p5

9

threads

<

mean

26.34

threads

SD

7.12

threads

<

p95

31

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

8

file descriptors

<

mean

59.25

file descriptors

SD

17.76

file descriptors

<

p95

68

file descriptors

Write validated

↑ Back to index

Creates an entry, updates it and links the create to the update, then repeats. Each of the three actions is validated by the included hApp.

Started

Thu, 03 Sep 2026 08:57:45 UTC

Duration

300s

Peer count

1

Behaviours

default (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

write_validated_33713946045_1

Write timing

The time it takes to complete the zome function call that writes the entry, updates it, and links to the update.

mean

0.082201

s

SD

0.04202

s

p50

0.075254

s

<

p95

0.142997

s

<

p99

0.198631

s

Write rate

The number of write zome function calls that complete per second.

mean

47.61

/s

Read timing

The time it takes to complete the zome function call that reads the data.

mean

0.123469

s

SD

0.060639

s

p50

0.112362

s

<

p95

0.211385

s

<

p99

0.332607

s

Read rate

The number of read zome function calls that complete per second.

mean

47.61

/s

Error count

The number of errors accumulated over the run.

0

Holochain MetricsClick to toggle visibility.

Cascade duration

Time taken to execute a cascade (get) query inside Holochain.

mean

0.00561

s

SD

0.001974

s

p50

0.00527

s

<

p95

0.008434

s

<

p99

0.008813

s

WASM usage: 'validated::create_sample_entry'

Metered usage count of the WASM ribosome for this zome function.

total

41551400

over 260.006475648s

mean rate

159809.46

/s

std

74302.38

/s

p5

85849.5

/s

p95

315922.83

/s

peak

384606

/s

WASM usage: 'validated::update_sample_entry'

Metered usage count of the WASM ribosome for this zome function.

total

48886420

over 260.006475648s

mean rate

188020.44

/s

std

87762.95

/s

p5

101004.42

/s

p95

367652.19

/s

peak

456540.23

/s

WASM usage: 'validated_integrity::entry_defs'

Metered usage count of the WASM ribosome for this zome function.

total

81000748

over 270.006559614s

mean rate

299996.38

/s

std

155398.53

/s

p5

151239.93

/s

p95

684926.04

/s

peak

738379.2

/s

WASM usage: 'validated_integrity::validate'

Metered usage count of the WASM ribosome for this zome function.

total

731052960

over 260.006475648s

mean rate

2.81167873e+06

/s

std

1.31018133e+06

/s

p5

1.50601471e+06

/s

p95

5.52352429e+06

/s

peak

6.80154043e+06

/s

Zome call duration: 'validated::create_sample_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.003731

s

SD

0.000251

s

p50

0.00372

s

<

p95

0.003909

s

<

p99

0.003918

s

Zome call duration: 'validated::update_sample_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.010494

s

SD

0.002423

s

p50

0.010006

s

<

p95

0.012837

s

<

p99

0.012993

s

WASM call duration: 'validated::create_sample_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.00308

s

SD

0.000218

s

p50

0.00307

s

<

p95

0.003236

s

<

p99

0.003244

s

WASM call duration: 'validated::update_sample_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.009849

s

SD

0.002395

s

p50

0.009362

s

<

p95

0.012168

s

<

p99

0.012326

s

WASM call duration: 'validated_integrity::entry_defs'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.000895

s

SD

5.2e-05

s

p50

0.000898

s

<

p95

0.000931

s

<

p99

0.000933

s

WASM call duration: 'validated_integrity::validate'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.003519

s

SD

0.000422

s

p50

0.003441

s

<

p95

0.003911

s

<

p99

0.003925

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.000385

s

SD

4.3e-05

s

p50

0.000383

s

<

p95

0.000418

s

<

p99

0.000419

s

Host function call duration: '__hc__create_link_1'

Duration of this host function call invoked from within WASM.

mean

0.000344

s

SD

2.5e-05

s

p50

0.000347

s

<

p95

0.000362

s

<

p99

0.000363

s

Host function call duration: '__hc__must_get_valid_record_1'

Duration of this host function call invoked from within WASM.

mean

0.005196

s

SD

0.001548

s

p50

0.004728

s

<

p95

0.006721

s

<

p99

0.006839

s

Host function call duration: '__hc__update_1'

Duration of this host function call invoked from within WASM.

mean

0.006877

s

SD

0.002227

s

p50

0.006378

s

<

p95

0.009066

s

<

p99

0.009226

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.001473

s

SD

0.00011

s

p50

0.001499

s

<

p95

0.001674

s

<

p99

0.001686

s

Post-commit duration

Time spent executing post-commit workflows.

mean

0.000109

s

SD

9e-06

s

p50

0.00011

s

<

p95

0.000116

s

≤

p99

0.000116

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

17.5

s

<

mean

142.51

s

SD

80.78

s

<

p95

267.51

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

0

over 270.006556785s

Not enough time data to show a trend.

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.001684

s

SD

0

s

p50

0.001684

s

≤

p95

0.001684

s

≤

p99

0.001684

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.001535

s

SD

0.000111

s

p50

0.001559

s

<

p95

0.001603

s

<

p99

0.001824

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.048499

s

SD

0.017748

s

p50

0.044846

s

<

p95

0.065423

s

<

p99

0.066644

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

0.002186

s

SD

0

s

p50

0.002186

s

≤

p95

0.002186

s

≤

p99

0.002186

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

0.002253

s

SD

0

s

p50

0.002253

s

≤

p95

0.002253

s

≤

p99

0.002253

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

0.003678

s

SD

0.001435

s

p50

0.003474

s

<

p95

0.004912

s

<

p99

0.004999

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.00156

s

SD

0.000193

s

p50

0.00177

s

≤

p95

0.00177

s

≤

p99

0.00177

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000303

s

SD

2.6e-05

s

p50

0.000303

s

<

p95

0.000323

s

≤

p99

0.000323

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

64.36

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

19.45

KiB/s

<

mean

1.15

MiB/s

SD

1.88

MiB/s

<

p95

6.70

MiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

14.68

KiB/s

<

mean

43.34

KiB/s

SD

37.55

KiB/s

<

p95

125.68

KiB/s

Total bytes received

Total bytes received on primary network interface

count

321.17

MiB

mean

1.15

MiB/s

Total bytes sent

Total bytes sent on primary network interface

count

11.85

MiB

mean

43.34

KiB/s

CPU spike anomaly

CPU spike anomaly was detected

Detected

Warning CPU p99 reached 95.8%

Memory leak anomaly

Memory leak anomaly was detected

Detected

Warning Memory growing at 2.19 MiB/s

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

NotDetected

System overload anomaly

System overload anomaly was detected

NotDetected

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

75.29

%

<

mean

83.35

%

SD

6.03

%

<

p95

92.9

%

System

18.98

%

CPU percentiles

CPU usage percentiles

p50

81.97

%

p95

92.90

%

p99

95.75

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

1

hosts

mean time

192.86

s

Memory used percentage

Percentage of memory used

p5

8.94

%

<

mean

9.76

%

SD

1.18

%

<

p95

12.08

%

Memory available percentage

Percentage of available memory

p5

87.92

%

<

mean

90.24

%

SD

1.18

%

<

p95

91.06

%

Max host memory used

Maximum memory usage percentage across all hosts

max

9.76

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

0.00

%

Memory growth rate

Rate of memory growth over time

growth

2.19

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.09

MiB/s

Disk write throughput

Disk write throughput in MiB/s

6.50

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /etc/nomad.d/client-meta.json

0/1

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

5.02

5 min

3.20

15 min

1.52

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

0.00

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

23.67

%

<

mean

35.2883

%

SD

11.3113

%

<

p95

53.66

%

60 second average

32.1117

%

300 second average

17.7286

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

Holochain process CPU usage

CPU usage by Holochain process

p5

15.99

%

<

mean

26.16

%

SD

9.45

%

<

p95

37.58

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

26.84

KiB

<

mean

330.59

KiB

SD

279.88

KiB

<

p95

763.41

KiB

Holochain process threads

Number of threads in Holochain process

p5

9

threads

<

mean

38.55

threads

SD

48.65

threads

<

p95

127

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

8

file descriptors

<

mean

85.55

file descriptors

SD

104.33

file descriptors

<

p95

280

file descriptors

Write Validated `must_get_agent_activity`

↑ Back to index

A scenario where write agents create entries in batches of 10, while must_get_agent_activity agents each pick a random write agent and repeatedly attempt to create an entry that references the chain top of their latest batch. This reference means that the entry's validation function needs to make a must_get_agent_activity call.

The purpose of this scenario is to measure the time it takes for published agent activity data to be gossiped among authorities and become available to peers that query it via must_get_agent_activity.

This test is similar to Mixed-Arc must_get_agent_activity, but all agents are full-arc.

Started

Thu, 03 Sep 2026 10:22:18 UTC

Duration

900s

Peer count

30

Peer count at end

29

Behaviours

must_get_agent_activity (1 agent)
write (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

write_validated_must_get_agent_activity_33713946045_1

Highest observed action_seq

The maximum chain length observed per write agent, aggregated across all reading agents. The reading dimension is collapsed by taking the maximum observed value for each write agent (any successful read counts as propagation). This reflects the DHT’s ability to propagate agent activity ops and make them available to querying peers.

Mean max (across write agents)

Mean of the highest chain head value observed per write agent. Each write agent contributes its maximum — the value seen by any reader.

1611

Max

Highest chain head value observed for any single write agent.

1897

Write agents observed

5

Chain batch delay timing

The time between a write agent's creation of a batch and a must_get_agent_activity agent's successful discovery of the batch and creation/self-validation of a new entry that depends on it.

Mean

mean

162.435543

s

SD

38.800356

s

Max

Highest per-partition mean latency

max mean

1020.715075

s

Min

Lowest per-partition mean latency

min mean

4.607971

s

Chain batch delay rate

The rate at which a must_get_agent_activity agent was able to discover batches and create/self-validate new entries that depend on them.

Mean

mean

2.77

/s

Max

Highest per-partition mean rate

max

6.94

/s

Min

Lowest per-partition mean rate

min

0

/s

create_validated_sample_entry zome call timing

The time taken to complete the zome function call that creates the entry that depends on a write agent's source chain.

Mean

mean

0.273991

s

SD

0.328618

s

Max

Highest per-partition mean latency

max mean

0.560615

s

Min

Lowest per-partition mean latency

min mean

0.067199

s

Retrieval errors

A running accumulation of the errors encountered by an agent when attempting to self-validate actions that depend on must_get_agent_activity calls.

Total

5577

Agents affected

19 / 19

Mean per agent

Total count divided by number of partitions, rounded to nearest whole number

294

Max

Highest count in any single partition

803

Min

Lowest count in any single partition

8

Final error count

The total number of all types of error accumulated over the run by all agents.

52738

Holochain MetricsClick to toggle visibility.

Cascade duration

Time taken to execute a cascade (get) query inside Holochain.

mean

0.01886

s

SD

0.146964

s

p50

0.001084

s

<

p95

0.32549

s

<

p99

0.774932

s

Zome call duration: 'validated_must_get_agent_activity::announce_write_behaviour'

Duration of this zome call as measured by Holochain internally.

mean

0.009375

s

SD

0.01402

s

p50

0.005508

s

<

p95

0.050325

s

≤

p99

0.050325

s

Zome call duration: 'validated_must_get_agent_activity::create_sample_entries_batch'

Duration of this zome call as measured by Holochain internally.

mean

0.076307

s

SD

0.051906

s

p50

0.05138

s

<

p95

0.189077

s

<

p99

0.26247

s

Zome call duration: 'validated_must_get_agent_activity::create_validated_sample_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.004851

s

SD

0.001547

s

p50

0.004952

s

<

p95

0.007311

s

<

p99

0.010281

s

Zome call duration: 'validated_must_get_agent_activity::get_chain_top_for_batch'

Duration of this zome call as measured by Holochain internally.

mean

0.005537

s

SD

0.001985

s

p50

0.005465

s

<

p95

0.008725

s

<

p99

0.012253

s

Zome call duration: 'validated_must_get_agent_activity::get_random_agent_with_write_behaviour'

Duration of this zome call as measured by Holochain internally.

mean

0.07633

s

SD

0.253886

s

p50

0.006531

s

<

p95

0.779348

s

≤

p99

0.779348

s

WASM call duration: 'validated_must_get_agent_activity::announce_write_behaviour'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.008637

s

SD

0.01395

s

p50

0.004552

s

<

p95

0.049414

s

≤

p99

0.049414

s

WASM call duration: 'validated_must_get_agent_activity::create_sample_entries_batch'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.075417

s

SD

0.051594

s

p50

0.050681

s

<

p95

0.1876

s

<

p99

0.260377

s

WASM call duration: 'validated_must_get_agent_activity::create_validated_sample_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.004046

s

SD

0.001317

s

p50

0.004192

s

<

p95

0.006171

s

<

p99

0.008593

s

WASM call duration: 'validated_must_get_agent_activity::get_chain_top_for_batch'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.004562

s

SD

0.001542

s

p50

0.004833

s

<

p95

0.00734

s

<

p99

0.010606

s

WASM call duration: 'validated_must_get_agent_activity::get_random_agent_with_write_behaviour'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.075478

s

SD

0.253947

s

p50

0.005576

s

<

p95

0.778579

s

≤

p99

0.778579

s

WASM call duration: 'validated_must_get_agent_activity_integrity::entry_defs'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.001189

s

SD

0.0007

s

p50

0.001145

s

<

p95

0.002258

s

<

p99

0.004632

s

WASM call duration: 'validated_must_get_agent_activity_integrity::validate'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.008331

s

SD

0.007379

s

p50

0.007159

s

<

p95

0.024542

s

<

p99

0.035671

s

Host function call duration: '__hc__agent_info_1'

Duration of this host function call invoked from within WASM.

mean

2.2e-05

s

SD

1e-05

s

p50

2e-05

s

<

p95

4.8e-05

s

≤

p99

4.8e-05

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.000522

s

SD

0.000343

s

p50

0.000439

s

<

p95

0.0009

s

<

p99

0.002267

s

Host function call duration: '__hc__create_link_1'

Duration of this host function call invoked from within WASM.

mean

0.000693

s

SD

0.000621

s

p50

0.000412

s

<

p95

0.002328

s

<

p99

0.003383

s

Host function call duration: '__hc__get_links_1'

Duration of this host function call invoked from within WASM.

mean

0.000719

s

SD

0.19152

s

p50

0.000658

s

<

p95

0.636743

s

<

p99

0.774963

s

Host function call duration: '__hc__must_get_agent_activity_1'

Duration of this host function call invoked from within WASM.

mean

0.02128

s

SD

0.020866

s

p50

0.017796

s

<

p95

0.056118

s

<

p99

0.129729

s

Host function call duration: '__hc__query_1'

Duration of this host function call invoked from within WASM.

mean

0.027432

s

SD

0.019477

s

p50

0.016014

s

<

p95

0.071595

s

<

p99

0.085692

s

Host function call duration: '__hc__random_bytes_1'

Duration of this host function call invoked from within WASM.

mean

7e-06

s

SD

3e-06

s

p50

9e-06

s

<

p95

1.4e-05

s

≤

p99

1.4e-05

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.001893

s

SD

0.001243

s

p50

0.002014

s

<

p95

0.003837

s

<

p99

0.008144

s

Post-commit duration

Time spent executing post-commit workflows.

mean

0.000189

s

SD

0.000112

s

p50

0.000154

s

<

p95

0.00041

s

<

p99

0.000561

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

46.92

s

<

mean

442.53

s

SD

254.34

s

<

p95

837.72

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

21196

over 887.993758369s

mean rate

90831.61

/s

std

217515.89

/s

p5

454.55

/s

p95

424582.75

/s

peak

1.25759762e+06

/s

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

23.147736

s

SD

18.871996

s

p50

14.722101

s

<

p95

66.228405

s

<

p99

89.593968

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

1.191126

s

SD

2.841026

s

p50

2.136711

s

<

p95

5.99205

s

<

p99

18.305834

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

<

p99

2e-06

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.101618

s

SD

0.113657

s

p50

0.113835

s

<

p95

0.405841

s

<

p99

0.503533

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.157449

s

SD

0.101211

s

p50

0.076925

s

<

p95

0.310655

s

<

p99

0.495512

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

0.485015

s

SD

2.657131

s

p50

0.703395

s

<

p95

7.30177

s

<

p99

15.329521

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

20.381102

s

SD

23.086617

s

p50

17.096868

s

<

p95

51.823254

s

<

p99

141.974212

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

0.00312

s

SD

0.001676

s

p50

0.002468

s

<

p95

0.005756

s

<

p99

0.006706

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.007716

s

SD

0.005187

s

p50

0.001967

s

<

p95

0.015187

s

<

p99

0.026973

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000396

s

SD

0.000286

s

p50

0.000354

s

<

p95

0.000749

s

<

p99

0.001897

s

P2P request roundtrip: 'get'

Time spent sending a get request and awaiting its response.

mean

0.428561

s

SD

25.12943

s

p50

0.216778

s

<

p95

60.001275

s

<

p99

60.002126

s

P2P request roundtrip: 'get_links'

Time spent sending a get_links request and awaiting its response.

mean

0.565195

s

SD

0.130308

s

p50

0.63452

s

<

p95

0.638843

s

≤

p99

0.638843

s

P2P request roundtrip: 'send_validation_receipts'

Time spent sending a send_validation_receipts request and awaiting its response.

mean

0.356792

s

SD

0.623782

s

p50

0.406839

s

<

p95

0.713247

s

<

p99

0.925068

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

43.48

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

51.13

KiB/s

<

mean

157.31

KiB/s

SD

629.83

KiB/s

≤

p95

152.32

KiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

21.43

KiB/s

<

mean

78.12

KiB/s

SD

62.19

KiB/s

<

p95

190.51

KiB/s

Total bytes received

Total bytes received on primary network interface

count

3.44

GiB

mean

3.96

MiB/s

Total bytes sent

Total bytes sent on primary network interface

count

1.71

GiB

mean

1.96

MiB/s

CPU spike anomaly

CPU spike anomaly was detected

Detected

Warning CPU p99 reached 97.2%

Memory leak anomaly

Memory leak anomaly was detected

NotDetected

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

NotDetected

System overload anomaly

System overload anomaly was detected

Detected

Warning 30% of hosts overloaded (load5/ncpus > 1.0)

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

14.35

%

<

mean

57.17

%

SD

24.17

%

<

p95

92.04

%

System

13.69

%

CPU percentiles

CPU usage percentiles

p50

58.19

%

p95

92.04

%

p99

97.24

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

16

hosts

mean time

326.53

s

Memory used percentage

Percentage of memory used

p5

7.67

%

<

mean

20.75

%

SD

9.24

%

<

p95

39.83

%

Memory available percentage

Percentage of available memory

p5

60.17

%

<

mean

79.25

%

SD

9.24

%

<

p95

92.33

%

Max host memory used

Maximum memory usage percentage across all hosts

max

40.71

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

5.13

%

Memory growth rate

Rate of memory growth over time

growth

0.20

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.13

MiB/s

Disk write throughput

Disk write throughput in MiB/s

88.30

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /

0/3

hosts

Mount Point /boot

0/1

hosts

Mount Point /efi-boot

0/2

hosts

Mount Point /etc/hostname

0/13

hosts

Mount Point /etc/hosts

0/13

hosts

Mount Point /etc/nomad.d/client-meta.json

0/11

hosts

Mount Point /etc/resolv.conf

0/13

hosts

Mount Point /nix/store

0/3

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

3.12

5 min

2.97

15 min

2.91

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

29.63

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

0

%

<

mean

17.0572

%

SD

16.4931

%

<

p95

47.08

%

60 second average

17.0926

%

300 second average

17.8740

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

3.4639

%

SD

8.0197

%

<

p95

26.04

%

60 second average

3.2972

%

300 second average

2.6305

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

1.2625

%

SD

3.2799

%

<

p95

10.5

%

60 second average

1.2012

%

300 second average

0.9199

%

Holochain process CPU usage

CPU usage by Holochain process

p5

11.16

%

<

mean

51.35

%

SD

23.11

%

<

p95

89.06

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

349.59

KiB

<

mean

448.55

KiB

SD

78.25

KiB

<

p95

605.02

KiB

Holochain process threads

Number of threads in Holochain process

p5

33

threads

<

mean

38.23

threads

SD

8.41

threads

<

p95

52

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

73

file descriptors

<

mean

80.53

file descriptors

SD

10.9

file descriptors

<

p95

95

file descriptors

Zero-Arc Create and Read

↑ Back to index

A zero-arc/full-arc mixed scenario with two types of zero-arc nodes -- ones that create data and ones that read data -- as well as full arc nodes to "relay" the data. The scenario has three roles:

zero_write: A zero-arc conductor that just creates entries with a timestamp field. Those entries are linked to a known base hash so that zero_read nodes can retrieve them.
zero_read: A zero-arc conductor that reads the entries created by the zero-arc node(s) and records the time lag between when the entry had been created and when it was first discovered.
full: A full-arc conductor that is just here to serve entries to zero arc nodes.

Started

Thu, 03 Sep 2026 11:40:22 UTC

Duration

900s

Peer count

30

Peer count at end

28

Behaviours

full (1 agent)
zero_read (1 agent)
zero_write (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

zero_arc_create_and_read_33713946045_1

Create rate

The number of timed entries created by the zero-arc write node(s) per second.

Mean

mean

61.13

/s

Max

Highest per-partition mean rate

max

70.36

/s

Min

Lowest per-partition mean rate

min

49.24

/s

Fetch lag timing

For each entry, the time lag between when it was created and when a zero-arc read node fetched it from the network via the get_timed_entries_network zome function.

Mean

mean

520.923456

s

SD

37.441718

s

Max

Highest per-partition mean latency

max mean

805.533839

s

Min

Lowest per-partition mean latency

min mean

14.478024

s

Fetch lag rate

The number of entries per second fetched by zero-arc read nodes.

Mean

mean

309.8

/s

Max

Highest per-partition mean rate

max

837

/s

Min

Lowest per-partition mean rate

min

0

/s

Delivery ratio

The proportion of created entries successfully fetched by all zero-arc read nodes.

0.0086

Entry created count

Total number of entries created by zero-arc write nodes, by agent.

Total

62463

Agents affected

11 / 11

Mean per agent

Total count divided by number of partitions, rounded to nearest whole number

5678

Max

Highest count in any single partition

6625

Min

Lowest count in any single partition

4342

Received count

Total number of entries fetched by zero-arc read nodes, by agent.

Total

5891

Agents affected

10 / 11

Mean per agent

Total count divided by number of partitions, rounded to nearest whole number

536

Max

Highest count in any single partition

853

Min

Lowest count in any single partition

0

Open connections

The number of currently open connections to other conductors.

full-arc

p5

23

<

mean

26.17

SD

3.23

<

p95

27

zero-arc

p5

2

<

mean

9.75

SD

4.29

<

p95

17

Error count

The number of errors accumulated across all nodes.

144

Holochain MetricsClick to toggle visibility.

Cascade duration

Time taken to execute a cascade (get) query inside Holochain.

mean

0.442232

s

SD

0.415041

s

p50

0.835414

s

<

p95

1.583228

s

<

p99

1.936623

s

WASM usage: 'timed::created_timed_entry'

Metered usage count of the WASM ribosome for this zome function.

total

441868445

over 875.632932791s

mean rate

2.53120807029e+09

/s

std

2.64958539905e+10

/s

p5

2.367233455e+07

/s

p95

4.7045713661e+08

/s

peak

4.36775641026e+11

/s

WASM usage: 'timed::get_timed_entries_network'

Metered usage count of the WASM ribosome for this zome function.

total

6986131

over 878.02791222s

mean rate

2.5714276427e+08

/s

std

9.5632948682e+08

/s

p5

167092.21

/s

p95

2.07430901467e+09

/s

peak

9.21594241967e+09

/s

WASM usage: 'timed_integrity::entry_defs'

Metered usage count of the WASM ribosome for this zome function.

total

0

over 888.336106716s

Not enough time data to show a trend.

Zome call duration: 'timed::created_timed_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.005436

s

SD

0.000643

s

p50

0.005717

s

<

p95

0.006254

s

<

p99

0.006398

s

Zome call duration: 'timed::get_timed_entries_network'

Duration of this zome call as measured by Holochain internally.

mean

40.03441

s

SD

55.203052

s

p50

1.885327

s

<

p95

162.209137

s

<

p99

220.839704

s

WASM call duration: 'timed::created_timed_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.004812

s

SD

0.000589

s

p50

0.005083

s

<

p95

0.005523

s

<

p99

0.005646

s

WASM call duration: 'timed::get_timed_entries_network'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

40.033259

s

SD

55.203111

s

p50

1.884516

s

<

p95

162.208444

s

<

p99

220.838466

s

WASM call duration: 'timed_integrity::entry_defs'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.000995

s

SD

0.000519

s

p50

0.001077

s

<

p95

0.002214

s

<

p99

0.002824

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.000304

s

SD

3.5e-05

s

p50

0.000308

s

<

p95

0.000348

s

<

p99

0.000353

s

Host function call duration: '__hc__create_link_1'

Duration of this host function call invoked from within WASM.

mean

0.0003

s

SD

2.8e-05

s

p50

0.000299

s

<

p95

0.000334

s

<

p99

0.000339

s

Host function call duration: '__hc__get_1'

Duration of this host function call invoked from within WASM.

mean

0.433296

s

SD

0.403075

s

p50

0.801299

s

<

p95

1.471484

s

<

p99

1.80594

s

Host function call duration: '__hc__get_links_1'

Duration of this host function call invoked from within WASM.

mean

2.543219

s

SD

1.237251

s

p50

2.282148

s

<

p95

5.225934

s

<

p99

5.948944

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.001514

s

SD

0.000771

s

p50

0.001679

s

<

p95

0.003539

s

<

p99

0.00428

s

Post-commit duration

Time spent executing post-commit workflows.

mean

0.000108

s

SD

9.1e-05

s

p50

0.000107

s

<

p95

0.00027

s

<

p99

0.000521

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

47.11

s

<

mean

442.64

s

SD

254.44

s

<

p95

837.85

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

33

over 888.336051729s

mean rate

81591.7

/s

std

500598.83

/s

p5

0

/s

p95

1210.51

/s

peak

5.53737998e+06

/s

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

28.185895

s

SD

36.962582

s

p50

0

s

<

p95

3.486087

s

<

p99

7.90851

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.125253

s

SD

0.161129

s

p50

0.001402

s

<

p95

0.369856

s

<

p99

0.400817

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

<

p99

1e-06

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.047075

s

SD

0.401849

s

p50

0.001309

s

<

p95

1.159235

s

<

p99

1.249135

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.09583

s

SD

0.04077

s

p50

0.013013

s

<

p95

0.121542

s

<

p99

0.136412

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

13.819889

s

SD

15.74577

s

p50

0.003249

s

<

p95

22.482978

s

<

p99

58.094877

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

0.084473

s

SD

0.188603

s

p50

0.001242

s

<

p95

0.420965

s

<

p99

0.935643

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

0.001736

s

SD

0.000864

s

p50

0.000966

s

<

p95

0.002451

s

<

p99

0.002656

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.001251

s

SD

0.004034

s

p50

0.001541

s

<

p95

0.007858

s

<

p99

0.025455

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000246

s

SD

8e-05

s

p50

0.000275

s

<

p95

0.000454

s

<

p99

0.000457

s

P2P request roundtrip: 'get'

Time spent sending a get request and awaiting its response.

mean

0.550493

s

SD

28.6351

s

p50

1.41468

s

<

p95

60.001594

s

<

p99

60.005952

s

P2P request roundtrip: 'get_links'

Time spent sending a get_links request and awaiting its response.

mean

1.014835

s

SD

17.318503

s

p50

0.972053

s

<

p95

60.000492

s

<

p99

60.00072

s

P2P request roundtrip: 'send_validation_receipts'

Time spent sending a send_validation_receipts request and awaiting its response.

mean

3.165657

s

SD

6.978059

s

p50

0.838206

s

<

p95

15.686683

s

<

p99

22.406683

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

32.14

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

2.43

KiB/s

<

mean

170.58

KiB/s

SD

304.48

KiB/s

<

p95

569.45

KiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

1.59

KiB/s

<

mean

137.37

KiB/s

SD

128.64

KiB/s

<

p95

348.28

KiB/s

Total bytes received

Total bytes received on primary network interface

count

3.75

GiB

mean

4.32

MiB/s

Total bytes sent

Total bytes sent on primary network interface

count

3.02

GiB

mean

3.48

MiB/s

CPU spike anomaly

CPU spike anomaly was detected

Detected

Warning CPU p99 reached 96.6%

Memory leak anomaly

Memory leak anomaly was detected

NotDetected

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

NotDetected

System overload anomaly

System overload anomaly was detected

Detected

Warning 33% of hosts overloaded (load5/ncpus > 1.0)

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

1.46

%

<

mean

42.98

%

SD

35.63

%

<

p95

95.92

%

System

10.85

%

CPU percentiles

CPU usage percentiles

p50

37.26

%

p95

95.92

%

p99

96.59

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

8

hosts

mean time

619.38

s

Memory used percentage

Percentage of memory used

p5

6.06

%

<

mean

17.62

%

SD

8.33

%

<

p95

34.39

%

Memory available percentage

Percentage of available memory

p5

65.61

%

<

mean

82.38

%

SD

8.33

%

<

p95

93.94

%

Max host memory used

Maximum memory usage percentage across all hosts

max

35.18

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

5.13

%

Memory growth rate

Rate of memory growth over time

growth

0.22

MiB/s

Disk read throughput

Disk read throughput in MiB/s

2.56

MiB/s

Disk write throughput

Disk write throughput in MiB/s

84.91

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /

0/3

hosts

Mount Point /boot

0/1

hosts

Mount Point /efi-boot

0/2

hosts

Mount Point /etc/hostname

0/13

hosts

Mount Point /etc/hosts

0/13

hosts

Mount Point /etc/nomad.d/client-meta.json

0/11

hosts

Mount Point /etc/resolv.conf

0/13

hosts

Mount Point /nix/store

0/3

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

2.27

5 min

2.39

15 min

2.57

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

33.33

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

0

%

<

mean

12.6916

%

SD

16.2922

%

<

p95

42.7

%

60 second average

12.6395

%

300 second average

12.8030

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

1.5948

%

SD

5.7711

%

<

p95

7.6

%

60 second average

1.6558

%

300 second average

1.7530

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

1.0148

%

SD

4.8571

%

<

p95

2.1

%

60 second average

1.0943

%

300 second average

1.2419

%

Holochain process CPU usage

CPU usage by Holochain process

p5

0.46

%

<

mean

35.31

%

SD

33.78

%

<

p95

92.79

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

220.56

KiB

<

mean

291.40

KiB

SD

67.93

KiB

<

p95

418.43

KiB

Holochain process threads

Number of threads in Holochain process

p5

24

threads

<

mean

33.51

threads

SD

6.94

threads

<

p95

46

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

57

file descriptors

<

mean

72.59

file descriptors

SD

13.25

file descriptors

<

p95

89

file descriptors

Zero-Arc Create Data

↑ Back to index

A zero-arc/full-arc mixed scenario where zero-arc nodes create data and full-arc nodes read the data. The scenario has two roles:

zero: A zero-arc conductor that just creates entries with a timestamp field. Those entries are linked to a known base hash so that full-arc nodes can retrieve them.
full: A full-arc conductor that reads the entries created by the zero-arc node(s) and records the time lag between when the entry had been created and when it was first discovered.

Started

Thu, 03 Sep 2026 10:53:37 UTC

Duration

900s

Peer count

30

Peer count at end

28

Behaviours

full (1 agent)
zero (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

zero_arc_create_data--high_zero_33713946045_1

Create rate

The number of timed entries created by the zero-arc node(s) per second.

Mean

mean

68.6

/s

Max

Highest per-partition mean rate

max

88.66

/s

Min

Lowest per-partition mean rate

min

50.03

/s

Sync lag timing

For each entry, the time lag between when it was created and when the full-arc node could read it via the get_timed_local_entries zome function.

Mean

mean

331.193676

s

SD

254.481151

s

Max

Highest per-partition mean latency

max mean

331.193676

s

Min

Lowest per-partition mean latency

min mean

331.193676

s

Sync lag rate

The number of entries per second received by full nodes.

Mean

mean

126.1

/s

Max

Highest per-partition mean rate

max

126.1

/s

Min

Lowest per-partition mean rate

min

126.1

/s

Delivery ratio

The proportion of created entries successfully received by all full-arc nodes.

0.0092

Entry created count

Total number of entries created by zero-arc nodes, by agent.

Total

157477

Agents affected

26 / 26

Mean per agent

Total count divided by number of partitions, rounded to nearest whole number

6057

Max

Highest count in any single partition

7756

Min

Lowest count in any single partition

4455

Received count

Total number of entries received by full-arc nodes, by agent.

Total

2887

Agents affected

1 / 2

Mean per agent

Total count divided by number of partitions, rounded to nearest whole number

1444

Max

Highest count in any single partition

2887

Min

Lowest count in any single partition

0

Open connections

The number of currently open connections to other conductors.

full-arc

p5

5

<

mean

21.95

SD

7.52

<

p95

27

zero-arc

p5

3

<

mean

9.39

SD

5.76

<

p95

24

Error count

The number of errors accumulated across all nodes.

0

Holochain MetricsClick to toggle visibility.

Cascade duration

Time taken to execute a cascade (get) query inside Holochain.

mean

0.000337

s

SD

5.4e-05

s

p50

0.000339

s

<

p95

0.0004

s

<

p99

0.000421

s

Cascade fetch errors

Count of network fetch errors during cascade calls.

total

44

over 770.006298588s

mean rate

21.5

/s

std

15.55

/s

p5

0

/s

p95

45.27

/s

peak

45.28

/s

WASM usage: 'timed::created_timed_entry'

Metered usage count of the WASM ribosome for this zome function.

total

416725892

over 840.073623337s

mean rate

2.123131333e+08

/s

std

3.0867870858e+08

/s

p5

1.01490529e+06

/s

p95

1.01014306781e+09

/s

peak

1.15814267409e+09

/s

WASM usage: 'timed::get_timed_entries_local'

Metered usage count of the WASM ribosome for this zome function.

total

62052623829

over 870.880795477s

mean rate

3.46578997097e+10

/s

std

2.2228499857e+10

/s

p5

2.8863866953e+08

/s

p95

6.81036144148e+10

/s

peak

7.15939285987e+10

/s

WASM usage: 'timed_integrity::entry_defs'

Metered usage count of the WASM ribosome for this zome function.

total

53413824

over 888.709330798s

mean rate

2.696897297e+07

/s

std

3.839640121e+07

/s

p5

0

/s

p95

1.2191319228e+08

/s

peak

1.5702338091e+08

/s

Zome call duration: 'timed::created_timed_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.005041

s

SD

0.001269

s

p50

0.005182

s

<

p95

0.008575

s

<

p99

0.008827

s

Zome call duration: 'timed::get_timed_entries_local'

Duration of this zome call as measured by Holochain internally.

mean

0.03886

s

SD

0.068398

s

p50

0.006111

s

<

p95

0.195072

s

<

p99

0.210197

s

WASM call duration: 'timed::created_timed_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.00448

s

SD

0.001156

s

p50

0.004635

s

<

p95

0.007712

s

<

p99

0.007931

s

WASM call duration: 'timed::get_timed_entries_local'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.037911

s

SD

0.068537

s

p50

0.005014

s

<

p95

0.194367

s

<

p99

0.209485

s

WASM call duration: 'timed_integrity::entry_defs'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.000963

s

SD

0.000267

s

p50

0.000993

s

<

p95

0.001572

s

<

p99

0.001652

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.000272

s

SD

7.3e-05

s

p50

0.000273

s

<

p95

0.000475

s

<

p99

0.000495

s

Host function call duration: '__hc__create_link_1'

Duration of this host function call invoked from within WASM.

mean

0.000269

s

SD

8.9e-05

s

p50

0.000274

s

<

p95

0.000485

s

<

p99

0.000596

s

Host function call duration: '__hc__get_1'

Duration of this host function call invoked from within WASM.

mean

0.000342

s

SD

2.7e-05

s

p50

0.00034

s

<

p95

0.000412

s

<

p99

0.000438

s

Host function call duration: '__hc__get_links_1'

Duration of this host function call invoked from within WASM.

mean

0.001635

s

SD

0.002523

s

p50

0.000451

s

<

p95

0.007239

s

<

p99

0.007795

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.001454

s

SD

0.000405

s

p50

0.001473

s

<

p95

0.002449

s

<

p99

0.002638

s

Post-commit duration

Time spent executing post-commit workflows.

mean

0.000102

s

SD

5.6e-05

s

p50

0.000104

s

<

p95

0.000224

s

<

p99

0.000267

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

46.9

s

<

mean

441.94

s

SD

253.99

s

<

p95

837.35

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

0

over 888.709350313s

Not enough time data to show a trend.

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

4.492774

s

SD

8.162289

s

p50

0

s

≤

p95

0

s

<

p99

46.601094

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.085746

s

SD

0.164542

s

p50

0.001136

s

<

p95

0.148204

s

<

p99

0.325218

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

<

p99

1e-06

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.005289

s

SD

0.224609

s

p50

0.001027

s

<

p95

0.620231

s

<

p99

1.118654

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.088277

s

SD

0.037439

s

p50

0.074223

s

<

p95

0.127463

s

<

p99

0.149473

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

13.238678

s

SD

15.091668

s

p50

0.002625

s

<

p95

26.435119

s

<

p99

37.715302

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

4.071356

s

SD

6.518347

s

p50

0.001769

s

<

p95

0.007579

s

<

p99

39.080126

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

0.002507

s

SD

0.001672

s

p50

0.004084

s

<

p95

0.005548

s

<

p99

0.00689

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.003577

s

SD

0.004576

s

p50

0.001591

s

<

p95

0.005864

s

<

p99

0.027303

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000217

s

SD

7.6e-05

s

p50

0.000221

s

<

p95

0.000418

s

<

p99

0.000436

s

P2P request roundtrip: 'get'

Time spent sending a get request and awaiting its response.

mean

10.353964

s

SD

29.272958

s

p50

5.875425

s

<

p95

60.001235

s

≤

p99

60.001235

s

P2P request roundtrip: 'send_validation_receipts'

Time spent sending a send_validation_receipts request and awaiting its response.

mean

0.204187

s

SD

0.191402

s

p50

0.201695

s

<

p95

0.585156

s

<

p99

1.352945

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

48.21

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

5.44

KiB/s

<

mean

162.91

KiB/s

SD

671.03

KiB/s

<

p95

724.20

KiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

2.72

KiB/s

<

mean

107.66

KiB/s

SD

101.64

KiB/s

<

p95

193.92

KiB/s

Total bytes received

Total bytes received on primary network interface

count

3.57

GiB

mean

4.10

MiB/s

Total bytes sent

Total bytes sent on primary network interface

count

2.36

GiB

mean

2.71

MiB/s

CPU spike anomaly

CPU spike anomaly was detected

Detected

Warning CPU p99 reached 95.8%

Memory leak anomaly

Memory leak anomaly was detected

NotDetected

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

NotDetected

System overload anomaly

System overload anomaly was detected

Detected

Warning 33% of hosts overloaded (load5/ncpus > 1.0)

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

17.45

%

<

mean

61.4

%

SD

24.82

%

<

p95

94.87

%

System

13.19

%

CPU percentiles

CPU usage percentiles

p50

59.88

%

p95

94.87

%

p99

95.77

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

13

hosts

mean time

489.02

s

Memory used percentage

Percentage of memory used

p5

6.66

%

<

mean

18.82

%

SD

8.67

%

<

p95

36.24

%

Memory available percentage

Percentage of available memory

p5

63.76

%

<

mean

81.18

%

SD

8.67

%

<

p95

93.34

%

Max host memory used

Maximum memory usage percentage across all hosts

max

38.46

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

5.13

%

Memory growth rate

Rate of memory growth over time

growth

0.29

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.03

MiB/s

Disk write throughput

Disk write throughput in MiB/s

96.17

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /

0/3

hosts

Mount Point /boot

0/1

hosts

Mount Point /efi-boot

0/2

hosts

Mount Point /etc/hostname

0/13

hosts

Mount Point /etc/hosts

0/13

hosts

Mount Point /etc/nomad.d/client-meta.json

0/11

hosts

Mount Point /etc/resolv.conf

0/13

hosts

Mount Point /nix/store

0/3

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

2.82

5 min

2.78

15 min

2.84

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

33.33

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

0

%

<

mean

13.6761

%

SD

13.2096

%

<

p95

36.14

%

60 second average

13.4870

%

300 second average

13.5293

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

1.8483

%

SD

5.7235

%

<

p95

16.91

%

60 second average

1.8912

%

300 second average

2.0624

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

1.1148

%

SD

4.3052

%

<

p95

6.63

%

60 second average

1.1852

%

300 second average

1.3699

%

Holochain process CPU usage

CPU usage by Holochain process

p5

16.74

%

<

mean

54.53

%

SD

25.06

%

<

p95

93.03

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

233.71

KiB

<

mean

316.89

KiB

SD

58.33

KiB

<

p95

437.92

KiB

Holochain process threads

Number of threads in Holochain process

p5

29

threads

<

mean

33.76

threads

SD

5.26

threads

<

p95

40

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

66

file descriptors

<

mean

75.62

file descriptors

SD

9.88

file descriptors

<

p95

83

file descriptors

Zome Call, Single Value

↑ Back to index

Calls a zome function that returns a fixed value. This tests the maximum performance of zome calls when the zome function is not calling into Holochain other otherwise doing any work.

Started

Thu, 03 Sep 2026 08:59:41 UTC

Duration

300s

Peer count

1

Peer count at end

1

Behaviours

default (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

zome_call_single_value_33713946045_1

Call timing

The time taken for the call to complete.

mean

0.003394

s

SD

0.001935

s

p50

0.002673

s

<

p95

0.006784

s

<

p99

0.010429

s

Call rate

The number of zome calls completed per second.

mean

2955.04

/s

Error count

The number of errors accumulated during the run.

0

Holochain MetricsClick to toggle visibility.

WASM usage: 'return_single_value::get_value'

Metered usage count of the WASM ribosome for this zome function.

total

306778534

over 270.003008587s

mean rate

1.13620576e+06

/s

std

414944.56

/s

p5

615290.91

/s

p95

1.68354804e+06

/s

peak

1.68714044e+06

/s

Zome call duration: 'return_single_value::get_value'

Duration of this zome call as measured by Holochain internally.

mean

0.001395

s

SD

0.000201

s

p50

0.001543

s

<

p95

0.001802

s

<

p99

0.002178

s

WASM call duration: 'return_single_value::get_value'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.000876

s

SD

0.000149

s

p50

0.000971

s

<

p95

0.001188

s

<

p99

0.001493

s

Post-commit duration

Time spent executing post-commit workflows.

mean

9e-05

s

SD

0

s

p50

9e-05

s

≤

p95

9e-05

s

≤

p99

9e-05

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

17.14

s

<

mean

142.15

s

SD

80.78

s

<

p95

267.15

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

0

over 270.003017712s

Not enough time data to show a trend.

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.002284

s

SD

0

s

p50

0.002284

s

≤

p95

0.002284

s

≤

p99

0.002284

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.002894

s

SD

0

s

p50

0.002894

s

≤

p95

0.002894

s

≤

p99

0.002894

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.005144

s

SD

0.000937

s

p50

0.005412

s

<

p95

0.006926

s

≤

p99

0.006926

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

0.004748

s

SD

0

s

p50

0.004748

s

≤

p95

0.004748

s

≤

p99

0.004748

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

0.000817

s

SD

0

s

p50

0.000817

s

≤

p95

0.000817

s

≤

p99

0.000817

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

7.8e-05

s

SD

1.7e-05

s

p50

9.1e-05

s

<

p95

0.000119

s

≤

p99

0.000119

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.00183

s

SD

0.001068

s

p50

0.003298

s

≤

p95

0.003298

s

≤

p99

0.003298

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.001056

s

SD

0

s

p50

0.001056

s

≤

p95

0.001056

s

≤

p99

0.001056

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

49.25

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

17.08

KiB/s

<

mean

1.15

MiB/s

SD

1.82

MiB/s

<

p95

6.73

MiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

14.51

KiB/s

<

mean

40.62

KiB/s

SD

35.05

KiB/s

<

p95

126.09

KiB/s

Total bytes received

Total bytes received on primary network interface

count

333.93

MiB

mean

1.15

MiB/s

Total bytes sent

Total bytes sent on primary network interface

count

11.50

MiB

mean

40.62

KiB/s

CPU spike anomaly

CPU spike anomaly was detected

Detected

Warning CPU p99 reached 95.8%

Memory leak anomaly

Memory leak anomaly was detected

NotDetected

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

NotDetected

System overload anomaly

System overload anomaly was detected

NotDetected

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

34.45

%

<

mean

66.86

%

SD

24.16

%

<

p95

92.9

%

System

17.60

%

CPU percentiles

CPU usage percentiles

p50

76.34

%

p95

92.90

%

p99

95.76

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

1

hosts

mean time

134.48

s

Memory used percentage

Percentage of memory used

p5

7.46

%

<

mean

9.17

%

SD

1.66

%

<

p95

12.08

%

Memory available percentage

Percentage of available memory

p5

87.92

%

<

mean

90.83

%

SD

1.66

%

<

p95

92.54

%

Max host memory used

Maximum memory usage percentage across all hosts

max

9.17

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

0.00

%

Memory growth rate

Rate of memory growth over time

growth

-3.71

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.09

MiB/s

Disk write throughput

Disk write throughput in MiB/s

5.35

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /etc/nomad.d/client-meta.json

0/1

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

4.76

5 min

3.72

15 min

1.92

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

0.00

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

2.29

%

<

mean

27.6617

%

SD

19.2808

%

<

p95

53.66

%

60 second average

30.4770

%

300 second average

22.1430

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

Holochain process CPU usage

CPU usage by Holochain process

p5

15.95

%

<

mean

22.34

%

SD

4.57

%

<

p95

26.83

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

138.15

KiB

<

mean

280.61

KiB

SD

270.09

KiB

<

p95

763.39

KiB

Holochain process threads

Number of threads in Holochain process

p5

27

threads

<

mean

39.33

threads

SD

48.93

threads

<

p95

126

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

63

file descriptors

<

mean

89.47

file descriptors

SD

104.08

file descriptors

<

p95

279

file descriptors

App Install

App Install

DHT Sync Lag

First Call

Local Signals

Mixed Arc get_agent_activity

Mixed Arc must_get_agent_activity

Remote Call Rate

Remote Signals

Single-write, many-read

Validation Receipts

Write/get_agent_activity

Write/get_agent_activity with volatile conductors

Write/Query

Write/Read

Write validated

Write Validated must_get_agent_activity

Zero-Arc Create and Read

Zero-Arc Create Data

Zome Call, Single Value

Mixed Arc `get_agent_activity`

Mixed Arc `must_get_agent_activity`

Write/`get_agent_activity`

Write/`get_agent_activity` with volatile conductors

Write Validated `must_get_agent_activity`