App Install

Measure app installation speed. This scenario has two roles:

minimal: Installs a simple app which implements initialisation callbacks but otherwise doesn't contain a lot of code.
large: Installs a larger app that contains some dependencies and some generated data as padding to make the bundle larger.

These are intended to catch a compilation slowdown or issues with copying around large WASMs (e.g. accidental cloning or tracing). In each case, the behaviour will uninstall the app it installed so that it can re-install it on the next iteration.

Started

Thu, 17 Sep 2026 09:15:11 UTC

Duration

300s

Peer count

1

Behaviours

large (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

app_install_large_35182846570_1

First install time

The time from conductor startup to completion of the first app installation attempt.

0.125329156

s

Install app timing

The time taken to install an app.

mean

0.100558

s

SD

0.191471

s

p50

0.068121

s

<

p95

0.08059

s

<

p99

1.249036

s

Holochain MetricsClick to toggle visibility.

WASM usage: 'large::init'

Metered usage count of the WASM ribosome for this zome function.

total

0

over 10.00018032s

Not enough time data to show a trend.

WASM call duration: 'large::init'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.004757

s

SD

0.00461

s

p50

0.003483

s

<

p95

0.006909

s

<

p99

0.034362

s

Post-commit duration

Time spent executing post-commit workflows.

mean

0.000159

s

SD

0.000138

s

p50

9.6e-05

s

<

p95

0.000495

s

<

p99

0.000559

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

6.98

s

<

mean

16.98

s

SD

8.17

s

<

p95

26.98

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

0

over 20.000122215s

Not enough time data to show a trend.

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.001232

s

SD

9.5e-05

s

p50

0.001289

s

<

p95

0.001319

s

≤

p99

0.001319

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.001741

s

SD

0.000588

s

p50

0.001794

s

<

p95

0.002935

s

≤

p99

0.002935

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.005888

s

SD

0.003531

s

p50

0.005867

s

<

p95

0.013266

s

≤

p99

0.013266

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

0.002599

s

SD

0.000182

s

p50

0.002701

s

<

p95

0.002789

s

≤

p99

0.002789

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

0.00145

s

SD

0.00033

s

p50

0.001565

s

<

p95

0.001979

s

≤

p99

0.001979

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

7.4e-05

s

SD

9e-06

s

p50

7.4e-05

s

<

p95

9.2e-05

s

≤

p99

9.2e-05

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.001645

s

SD

0.000557

s

p50

0.001742

s

<

p95

0.002476

s

≤

p99

0.002476

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000457

s

SD

4.6e-05

s

p50

0.000475

s

<

p95

0.000528

s

≤

p99

0.000528

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

71.06

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

2.52

MiB/s

<

mean

2.60

MiB/s

SD

63.30

KiB/s

<

p95

2.67

MiB/s

Not enough time data to show a trend.

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

60.24

KiB/s

<

mean

74.53

KiB/s

SD

12.19

KiB/s

<

p95

90.02

KiB/s

Not enough time data to show a trend.

Total bytes received

Total bytes received on primary network interface

count

78.12

MiB

mean

2.60

MiB/s

Total bytes sent

Total bytes sent on primary network interface

count

2.18

MiB

mean

74.53

KiB/s

CPU spike anomaly

CPU spike anomaly was detected

Detected

Warning CPU p99 reached 97.0%

Memory leak anomaly

Memory leak anomaly was detected

Detected

Warning Memory growing at 6.24 MiB/s

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

NotDetected

System overload anomaly

System overload anomaly was detected

Detected

Warning 100% of hosts overloaded (load5/ncpus > 1.0)

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

85.14

%

<

mean

89.71

%

SD

5.24

%

<

p95

97.04

%

Not enough time data to show a trend.

System

18.65

%

CPU percentiles

CPU usage percentiles

p50

86.96

%

p95

97.04

%

p99

97.04

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

1

hosts

mean time

300.00

s

Memory used percentage

Percentage of memory used

p5

10.25

%

<

mean

11.76

%

SD

1.78

%

<

p95

14.61

%

Not enough time data to show a trend.

Memory available percentage

Percentage of available memory

p5

85.39

%

<

mean

88.24

%

SD

1.78

%

<

p95

89.75

%

Not enough time data to show a trend.

Max host memory used

Maximum memory usage percentage across all hosts

max

11.76

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

0.00

%

Memory growth rate

Rate of memory growth over time

growth

6.24

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.01

MiB/s

Disk write throughput

Disk write throughput in MiB/s

10.22

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /etc/nomad.d/client-meta.json

0/1

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

7.33

5 min

4.19

15 min

1.94

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

100.00

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

48.38

%

<

mean

56.71

%

SD

6.3113

%

<

p95

66

%

Not enough time data to show a trend.

60 second average

49.3675

%

300 second average

25.9775

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

Not enough time data to show a trend.

60 second average

0.0000

%

300 second average

0.0000

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

Not enough time data to show a trend.

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

Not enough time data to show a trend.

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

Not enough time data to show a trend.

60 second average

0.0000

%

300 second average

0.0000

%

Holochain process CPU usage

CPU usage by Holochain process

p5

8.82

%

<

mean

19.98

%

SD

8.08

%

<

p95

27.69

%

Not enough time data to show a trend.

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

139.19

KiB

<

mean

731.73

KiB

SD

632.56

KiB

<

p95

1.67

MiB

Not enough time data to show a trend.

Holochain process threads

Number of threads in Holochain process

p5

26

threads

<

mean

128.25

threads

SD

118.07

threads

<

p95

314

threads

Not enough time data to show a trend.

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

61

file descriptors

<

mean

278.5

file descriptors

SD

249.37

file descriptors

<

p95

670

file descriptors

Not enough time data to show a trend.

App Install

↑ Back to index

Measure app installation speed. This scenario has two roles:

minimal: Installs a simple app which implements initialisation callbacks but otherwise doesn't contain a lot of code.
large: Installs a larger app that contains some dependencies and some generated data as padding to make the bundle larger.

These are intended to catch a compilation slowdown or issues with copying around large WASMs (e.g. accidental cloning or tracing). In each case, the behaviour will uninstall the app it installed so that it can re-install it on the next iteration.

Started

Thu, 17 Sep 2026 09:17:02 UTC

Duration

300s

Peer count

1

Behaviours

minimal (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

app_install_minimal_35182846570_1

First install time

The time from conductor startup to completion of the first app installation attempt.

0.326416768

s

Install app timing

The time taken to install an app.

mean

0.03941

s

SD

0.002435

s

p50

0.039193

s

<

p95

0.043097

s

<

p99

0.050456

s

Holochain MetricsClick to toggle visibility.

WASM usage: 'callback::init'

Metered usage count of the WASM ribosome for this zome function.

total

0

over 10.000124198s

Not enough time data to show a trend.

WASM usage: 'callback_integrity::genesis_self_check_2'

Metered usage count of the WASM ribosome for this zome function.

total

115209

over 10.000124198s

mean rate

11520.76

/s

std

0

/s

p5

11520.76

/s

p95

11520.76

/s

peak

11520.76

/s

Not enough time data to show a trend.

WASM call duration: 'callback::init'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.000721

s

SD

5e-05

s

p50

0.000712

s

<

p95

0.000794

s

<

p99

0.000987

s

WASM call duration: 'callback_integrity::genesis_self_check_2'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.000724

s

SD

1.4e-05

s

p50

0.000747

s

≤

p95

0.000747

s

≤

p99

0.000747

s

Post-commit duration

Time spent executing post-commit workflows.

mean

7e-05

s

SD

1.8e-05

s

p50

6.5e-05

s

<

p95

9.8e-05

s

<

p99

0.000139

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

7.51

s

<

mean

12.51

s

SD

5

s

<

p95

17.51

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

0

over 10.000035217s

Not enough time data to show a trend.

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.00117

s

SD

3.9e-05

s

p50

0.001233

s

≤

p95

0.001233

s

≤

p99

0.001233

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.001182

s

SD

0.000125

s

p50

0.001384

s

≤

p95

0.001384

s

≤

p99

0.001384

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.003878

s

SD

0.000484

s

p50

0.004657

s

≤

p95

0.004657

s

≤

p99

0.004657

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

0.002207

s

SD

0.000175

s

p50

0.002485

s

≤

p95

0.002485

s

≤

p99

0.002485

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

0.001684

s

SD

9.6e-05

s

p50

0.001838

s

≤

p95

0.001838

s

≤

p99

0.001838

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

5.6e-05

s

SD

1e-06

s

p50

5.7e-05

s

≤

p95

5.7e-05

s

≤

p99

5.7e-05

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.000986

s

SD

0.000275

s

p50

0.001328

s

<

p95

0.001479

s

≤

p99

0.001479

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000273

s

SD

6e-06

s

p50

0.000283

s

≤

p95

0.000283

s

≤

p99

0.000283

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

45.82

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

120.15

KiB/s

<

mean

756.10

KiB/s

SD

635.96

KiB/s

<

p95

1.36

MiB/s

Not enough time data to show a trend.

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

42.55

KiB/s

<

mean

51.81

KiB/s

SD

9.25

KiB/s

<

p95

61.06

KiB/s

Not enough time data to show a trend.

Total bytes received

Total bytes received on primary network interface

count

14.77

MiB

mean

756.10

KiB/s

Total bytes sent

Total bytes sent on primary network interface

count

1.01

MiB

mean

51.81

KiB/s

CPU spike anomaly

CPU spike anomaly was detected

NotDetected

Memory leak anomaly

Memory leak anomaly was detected

NotDetected

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

NotDetected

System overload anomaly

System overload anomaly was detected

Detected

Warning 100% of hosts overloaded (load5/ncpus > 1.0)

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

62.7

%

<

mean

64.71

%

SD

2

%

<

p95

66.71

%

Not enough time data to show a trend.

System

18.89

%

CPU percentiles

CPU usage percentiles

p50

66.71

%

p95

66.71

%

p99

66.71

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

0

hosts

mean time

0.00

s

Memory used percentage

Percentage of memory used

p5

8.39

%

<

mean

9.19

%

SD

1.09

%

<

p95

10.73

%

Not enough time data to show a trend.

Memory available percentage

Percentage of available memory

p5

89.27

%

<

mean

90.81

%

SD

1.09

%

<

p95

91.61

%

Not enough time data to show a trend.

Max host memory used

Maximum memory usage percentage across all hosts

max

9.19

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

0.00

%

Memory growth rate

Rate of memory growth over time

growth

-0.09

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.03

MiB/s

Disk write throughput

Disk write throughput in MiB/s

20.35

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /etc/nomad.d/client-meta.json

0/1

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

4.90

5 min

4.38

15 min

2.29

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

100.00

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

14.56

%

<

mean

17.21

%

SD

1.8738

%

<

p95

18.54

%

Not enough time data to show a trend.

60 second average

24.8367

%

300 second average

25.3167

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

Not enough time data to show a trend.

60 second average

0.0000

%

300 second average

0.0000

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

Not enough time data to show a trend.

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

Not enough time data to show a trend.

60 second average

0.0833

%

300 second average

0.0567

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

Not enough time data to show a trend.

60 second average

0.0167

%

300 second average

0.0000

%

Holochain process CPU usage

CPU usage by Holochain process

p5

28.79

%

<

mean

31.42

%

SD

2.64

%

<

p95

34.06

%

Not enough time data to show a trend.

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

143.64

KiB

<

mean

471.72

KiB

SD

418.09

KiB

<

p95

1.04

MiB

Not enough time data to show a trend.

Holochain process threads

Number of threads in Holochain process

p5

1

threads

<

mean

73.67

threads

SD

82.47

threads

<

p95

189

threads

Not enough time data to show a trend.

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

0

file descriptors

<

mean

159.33

file descriptors

SD

181.32

file descriptors

<

p95

413

file descriptors

Not enough time data to show a trend.

DHT Sync Lag

↑ Back to index

Measures the lag time between an agent publishing data and other peers being able to see it. This scenario has two roles:

write: Creates entries with a timestamp field, linked to a known base hash so that record_lag agents can discover them.
record_lag: Repeatedly queries for links from the known base hash. When a new entry is discovered, it calculates the time difference between the entry's timestamp and the current time, giving the sync lag for that entry.

Started

Thu, 17 Sep 2026 09:18:38 UTC

Duration

1800s

Peer count

30

Peer count at end

28

Behaviours

record_lag (1 agent)
write (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

dht_sync_lag_35182846570_1

Create rate

The average number of created records per agent.

Mean

mean

23.28

/s

Max

Highest per-partition mean rate

max

42.55

/s

Min

Lowest per-partition mean rate

min

9.73

/s

Sync lag timing

The average time between when a record was created and when it is first seen by an agent.

Mean

mean

2.4321548992e+30

s

SD

1.97855819011e+31

s

Max

Highest per-partition mean latency

max mean

6.7516342643e+30

s

Min

Lowest per-partition mean latency

min mean

497.468299

s

Sync lag rate

The average number of created records discovered per agent.

Mean

mean

114.11

/s

Max

Highest per-partition mean rate

max

360.25

/s

Min

Lowest per-partition mean rate

min

1

/s

Delivery ratio

The proportion of created entries successfully received by all record_lag agents.

0.0294

Sent count

The total number of entries sent (created) by write agents, by agent.

Total

108702

Agents affected

25 / 25

Mean per agent

Total count divided by number of partitions, rounded to nearest whole number

4348

Max

Highest count in any single partition

7646

Min

Lowest count in any single partition

1750

Received count

The total number of entries received by record_lag agents, by agent.

Total

12780

Agents affected

4 / 4

Mean per agent

Total count divided by number of partitions, rounded to nearest whole number

3195

Max

Highest count in any single partition

6048

Min

Lowest count in any single partition

3

Error count

The number of errors encountered during the scenario.

3

Holochain MetricsClick to toggle visibility.

Cascade duration

Time taken to execute a cascade (get) query inside Holochain.

mean

0.000298

s

SD

0.000632

s

p50

0.000299

s

<

p95

0.001933

s

<

p99

0.00205

s

Cascade fetch errors

Count of network fetch errors during cascade calls.

total

0

over 1460.069352572s

Not enough time data to show a trend.

WASM usage: 'timed::created_timed_entry'

Metered usage count of the WASM ribosome for this zome function.

total

236230979

over 1786.847547001s

mean rate

1.28501606354e+09

/s

std

4.61601094556e+09

/s

p5

8.41202842e+06

/s

p95

6.17935909718e+09

/s

peak

1.01141588761e+11

/s

WASM usage: 'timed::get_timed_entries_local'

Metered usage count of the WASM ribosome for this zome function.

total

873577439

over 1770.058349948s

mean rate

3.66903227913e+10

/s

std

7.40823230627e+10

/s

p5

1.1599083575e+08

/s

p95

2.33765912071e+11

/s

peak

2.52143254346e+11

/s

WASM usage: 'timed_integrity::entry_defs'

Metered usage count of the WASM ribosome for this zome function.

total

30112608

over 1786.847547001s

mean rate

1.6741384439e+08

/s

std

5.8985271262e+08

/s

p5

942158.84

/s

p95

7.932681499e+08

/s

peak

1.28963430025e+10

/s

Zome call duration: 'timed::created_timed_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.011965

s

SD

0.014866

s

p50

0.008969

s

<

p95

0.043868

s

<

p99

0.08172

s

Zome call duration: 'timed::get_timed_entries_local'

Duration of this zome call as measured by Holochain internally.

mean

0.083622

s

SD

0.107467

s

p50

0.065751

s

<

p95

0.301906

s

<

p99

0.338124

s

WASM call duration: 'timed::created_timed_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.011193

s

SD

0.014662

s

p50

0.008185

s

<

p95

0.042711

s

<

p99

0.079953

s

WASM call duration: 'timed::get_timed_entries_local'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.082884

s

SD

0.107481

s

p50

0.065164

s

<

p95

0.301228

s

<

p99

0.337431

s

WASM call duration: 'timed_integrity::entry_defs'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.001737

s

SD

0.001485

s

p50

0.001485

s

<

p95

0.004336

s

<

p99

0.008713

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.001856

s

SD

0.004138

s

p50

0.000692

s

<

p95

0.011448

s

<

p99

0.020964

s

Host function call duration: '__hc__create_link_1'

Duration of this host function call invoked from within WASM.

mean

0.001781

s

SD

0.003947

s

p50

0.000669

s

<

p95

0.010599

s

<

p99

0.020302

s

Host function call duration: '__hc__get_1'

Duration of this host function call invoked from within WASM.

mean

0.000299

s

SD

0.000823

s

p50

0.000313

s

<

p95

0.00243

s

<

p99

0.002555

s

Host function call duration: '__hc__get_links_1'

Duration of this host function call invoked from within WASM.

mean

0.003636

s

SD

0.004396

s

p50

0.003112

s

<

p95

0.012632

s

<

p99

0.014486

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.002818

s

SD

0.002681

s

p50

0.002321

s

<

p95

0.007889

s

<

p99

0.015523

s

Post-commit duration

Time spent executing post-commit workflows.

mean

0.000243

s

SD

0.000431

s

p50

0.000143

s

<

p95

0.000918

s

<

p99

0.00267

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

87.09

s

<

mean

879.03

s

SD

516.93

s

<

p95

1695.85

s

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

78.863536

s

SD

75.0616

s

p50

0.593138

s

<

p95

276.403242

s

<

p99

320.9374

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.049922

s

SD

1.19819

s

p50

0.205708

s

<

p95

1.713022

s

<

p99

8.287601

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.305213

s

SD

0.515693

s

p50

0.311243

s

<

p95

1.534896

s

<

p99

2.138795

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.504847

s

SD

0.661926

s

p50

0.402093

s

<

p95

1.31362

s

<

p99

4.261686

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

6.773009

s

SD

56.714664

s

p50

29.308917

s

<

p95

173.607076

s

<

p99

196.493861

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

24.33096

s

SD

74.192909

s

p50

0.00255

s

<

p95

264.536896

s

<

p99

304.299781

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

0.001595

s

SD

0.001175

s

p50

0.001895

s

<

p95

0.004266

s

<

p99

0.005468

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.003575

s

SD

0.006671

s

p50

0.001538

s

<

p95

0.021703

s

<

p99

0.03685

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.001585

s

SD

0.003611

s

p50

0.000492

s

<

p95

0.00939

s

<

p99

0.019805

s

P2P request roundtrip: 'get'

Time spent sending a get request and awaiting its response.

mean

5.124095

s

SD

27.026505

s

p50

8.726765

s

<

p95

60.0027

s

<

p99

60.037771

s

P2P request roundtrip: 'send_validation_receipts'

Time spent sending a send_validation_receipts request and awaiting its response.

mean

2.50085

s

SD

26.504177

s

p50

6.884335

s

<

p95

60.010048

s

<

p99

60.013637

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

69.63

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

29.79

KiB/s

<

mean

563.28

KiB/s

SD

785.62

KiB/s

<

p95

892.62

KiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

39.13

KiB/s

<

mean

450.58

KiB/s

SD

308.57

KiB/s

<

p95

999.93

KiB/s

Total bytes received

Total bytes received on primary network interface

count

24.30

GiB

mean

13.90

MiB/s

Total bytes sent

Total bytes sent on primary network interface

count

19.44

GiB

mean

11.12

MiB/s

CPU spike anomaly

CPU spike anomaly was detected

Detected

Warning CPU p99 reached 98.5%

Memory leak anomaly

Memory leak anomaly was detected

NotDetected

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

Detected

Warning Moderate swap usage (12.9% swap used)

System overload anomaly

System overload anomaly was detected

Detected

Warning 81% of hosts overloaded (load5/ncpus > 1.0)

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

44.81

%

<

mean

83.39

%

SD

18.25

%

<

p95

98.1

%

System

13.77

%

CPU percentiles

CPU usage percentiles

p50

92.65

%

p95

98.10

%

p99

98.45

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

23

hosts

mean time

1471.61

s

Memory used percentage

Percentage of memory used

p5

6.97

%

<

mean

20.41

%

SD

8.8

%

<

p95

37.3

%

Memory available percentage

Percentage of available memory

p5

62.7

%

<

mean

79.59

%

SD

8.8

%

<

p95

93.03

%

Max host memory used

Maximum memory usage percentage across all hosts

max

39.18

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

12.91

%

Memory growth rate

Rate of memory growth over time

growth

0.26

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.85

MiB/s

Disk write throughput

Disk write throughput in MiB/s

157.09

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /

0/2

hosts

Mount Point /efi-boot

0/2

hosts

Mount Point /etc/hostname

0/13

hosts

Mount Point /etc/hosts

0/13

hosts

Mount Point /etc/nomad.d/client-meta.json

0/12

hosts

Mount Point /etc/resolv.conf

0/13

hosts

Mount Point /nix/store

0/2

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

5.63

5 min

4.94

15 min

3.44

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

81.48

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

1.44

%

<

mean

42.0179

%

SD

24.0534

%

<

p95

73.81

%

60 second average

40.9245

%

300 second average

35.8638

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

<

mean

0.0002

%

SD

0.0148

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

4.3422

%

SD

10.0447

%

<

p95

27.67

%

60 second average

4.2590

%

300 second average

3.7713

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

1.6364

%

SD

5.1616

%

<

p95

15.51

%

60 second average

1.6171

%

300 second average

1.4550

%

Holochain process CPU usage

CPU usage by Holochain process

p5

0

%

<

mean

45.78

%

SD

37.37

%

<

p95

96.33

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

228.11

KiB

<

mean

418.87

KiB

SD

163.74

KiB

<

p95

645.24

KiB

Holochain process threads

Number of threads in Holochain process

p5

1

threads

<

mean

34.05

threads

SD

10.2

threads

<

p95

50

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

0

file descriptors

<

mean

77.14

file descriptors

SD

20.99

file descriptors

<

p95

93

file descriptors

First Call

↑ Back to index

This scenario installs a simple app which implements the init callback, and measures the time elapsed between start of install and calling init. It will uninstall the app it installed so that it can re-install it on the next iteration. This is required to re-run the initialisation callback.

Started

Thu, 17 Sep 2026 08:59:39 UTC

Duration

300s

Peer count

1

Behaviours

default (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

first_call_35182846570_1

Zome call timing

The time elapsed between start of install and calling init.

mean

0.011624

s

SD

0.00518

s

p50

0.010541

s

<

p95

0.020377

s

<

p99

0.039112

s

Holochain MetricsClick to toggle visibility.

WASM usage: 'crud::create_sample_entry'

Metered usage count of the WASM ribosome for this zome function.

total

0

over 10.000505866s

Not enough time data to show a trend.

WASM usage: 'crud::init'

Metered usage count of the WASM ribosome for this zome function.

total

0

over 10.000505866s

Not enough time data to show a trend.

WASM usage: 'crud_integrity::entry_defs'

Metered usage count of the WASM ribosome for this zome function.

total

255316

over 10.000505866s

mean rate

25530.31

/s

std

0

/s

p5

25530.31

/s

p95

25530.31

/s

peak

25530.31

/s

Not enough time data to show a trend.

Zome call duration: 'crud::create_sample_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.005108

s

SD

0.000345

s

p50

0.00571

s

≤

p95

0.00571

s

≤

p99

0.00571

s

WASM call duration: 'crud::create_sample_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.004314

s

SD

0.001144

s

p50

0.004005

s

<

p95

0.007253

s

<

p99

0.008033

s

WASM call duration: 'crud::init'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.001373

s

SD

0.000362

s

p50

0.001262

s

<

p95

0.002613

s

≤

p99

0.002613

s

WASM call duration: 'crud_integrity::entry_defs'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.001173

s

SD

3.9e-05

s

p50

0.00124

s

≤

p95

0.00124

s

≤

p99

0.00124

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.000333

s

SD

6.2e-05

s

p50

0.000379

s

<

p95

0.000435

s

≤

p99

0.000435

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.002241

s

SD

0.000143

s

p50

0.00249

s

≤

p95

0.00249

s

≤

p99

0.00249

s

Post-commit duration

Time spent executing post-commit workflows.

mean

0.00011

s

SD

4.6e-05

s

p50

9.2e-05

s

<

p95

0.000229

s

≤

p99

0.000229

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

6.9

s

<

mean

11.9

s

SD

5

s

<

p95

16.9

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

0

over 10.000323156s

Not enough time data to show a trend.

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.001034

s

SD

0.000151

s

p50

0.001072

s

≤

p95

0.001072

s

≤

p99

0.001072

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.001256

s

SD

1.8e-05

s

p50

0.001287

s

≤

p95

0.001287

s

≤

p99

0.001287

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.004226

s

SD

0.000522

s

p50

0.005136

s

≤

p95

0.005136

s

≤

p99

0.005136

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

0.003068

s

SD

0.000383

s

p50

0.003739

s

≤

p95

0.003739

s

≤

p99

0.003739

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

0.001528

s

SD

0.000253

s

p50

0.001592

s

≤

p95

0.001592

s

≤

p99

0.001592

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

3.1e-05

s

SD

3e-06

s

p50

3.6e-05

s

≤

p95

3.6e-05

s

≤

p99

3.6e-05

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.001083

s

SD

0.000403

s

p50

0.001737

s

<

p95

0.001774

s

≤

p99

0.001774

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000262

s

SD

8e-06

s

p50

0.000277

s

≤

p95

0.000277

s

≤

p99

0.000277

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

46.84

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

1.20

MiB/s

<

mean

4.25

MiB/s

SD

3.05

MiB/s

<

p95

7.31

MiB/s

Not enough time data to show a trend.

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

32.38

KiB/s

<

mean

55.64

KiB/s

SD

23.26

KiB/s

<

p95

78.90

KiB/s

Not enough time data to show a trend.

Total bytes received

Total bytes received on primary network interface

count

85.10

MiB

mean

4.25

MiB/s

Total bytes sent

Total bytes sent on primary network interface

count

1.09

MiB

mean

55.64

KiB/s

CPU spike anomaly

CPU spike anomaly was detected

NotDetected

Memory leak anomaly

Memory leak anomaly was detected

Detected

Warning Memory growing at 5.55 MiB/s

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

NotDetected

System overload anomaly

System overload anomaly was detected

NotDetected

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

48.39

%

<

mean

54.91

%

SD

6.52

%

<

p95

61.43

%

Not enough time data to show a trend.

System

8.07

%

CPU percentiles

CPU usage percentiles

p50

61.43

%

p95

61.43

%

p99

61.43

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

0

hosts

mean time

0.00

s

Memory used percentage

Percentage of memory used

p5

10.39

%

<

mean

16.15

%

SD

6.07

%

<

p95

24.54

%

Memory available percentage

Percentage of available memory

p5

75.46

%

<

mean

83.85

%

SD

6.07

%

<

p95

89.61

%

Max host memory used

Maximum memory usage percentage across all hosts

max

16.15

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

0.00

%

Memory growth rate

Rate of memory growth over time

growth

5.55

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.00

MiB/s

Disk write throughput

Disk write throughput in MiB/s

10.96

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /etc/nomad.d/client-meta.json

0/1

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

0.40

5 min

0.11

15 min

0.03

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

0.00

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

1.81

%

<

mean

5.2033

%

SD

3.1983

%

<

p95

9.49

%

60 second average

1.4400

%

300 second average

0.3267

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

0.23

%

SD

0.3253

%

<

p95

0.69

%

60 second average

0.5200

%

300 second average

0.2867

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

0.23

%

SD

0.3253

%

<

p95

0.69

%

60 second average

0.5200

%

300 second average

0.2867

%

Holochain process CPU usage

CPU usage by Holochain process

p5

0

%

<

mean

28.8

%

SD

28.8

%

<

p95

57.6

%

Not enough time data to show a trend.

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

97.12

KiB

<

mean

593.12

KiB

SD

495.99

KiB

<

p95

1.06

MiB

Not enough time data to show a trend.

Holochain process threads

Number of threads in Holochain process

p5

1

threads

<

mean

80.67

threads

SD

95.53

threads

<

p95

215

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

0

file descriptors

<

mean

180

file descriptors

SD

208.98

file descriptors

<

p95

473

file descriptors

Full-Arc Create (Validated) / Zero-Arc Read

↑ Back to index

A full-arc/zero-arc mixed scenario where full-arc nodes create data that gets validated and zero-arc nodes read the data. The scenario has two roles:

full: A full-arc conductor that creates entries with a timestamp field. Those entries get validated and then retrieved by zero-arc nodes.
zero: A zero-arc conductor that reads the entries created by the full-arc node(s) and records the time lag between when the entry had been created and when it was first discovered.

Started

Thu, 17 Sep 2026 12:12:22 UTC

Duration

900s

Peer count

30

Peer count at end

29

Behaviours

full (1 agent)
zero (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

full_arc_create_validated_zero_arc_read--high_zero_35182846570_1

Create rate

The number of timed entries created by the full-arc node(s) per second.

Mean

mean

40.67

/s

Max

Highest per-partition mean rate

max

43.8

/s

Min

Lowest per-partition mean rate

min

37.55

/s

Fetch lag timing

For each entry, the time lag between when it was created by a full-arc node and when a zero-arc node fetched it from the network.

Mean

mean

87.287477

s

SD

18.52574

s

Max

Highest per-partition mean latency

max mean

1144.343347

s

Min

Lowest per-partition mean latency

min mean

0.626402

s

Fetch rate

The number of entries per second fetched by zero-arc nodes.

Mean

mean

58.15

/s

Max

Highest per-partition mean rate

max

237

/s

Min

Lowest per-partition mean rate

min

0

/s

Delivery ratio

The proportion of created entries successfully fetched by all zero-arc nodes.

0.0579

Entry created count

Total number of entries created by full-arc nodes, by agent.

Total

7332

Agents affected

2 / 2

Mean per agent

Total count divided by number of partitions, rounded to nearest whole number

3666

Max

Highest count in any single partition

4023

Min

Lowest count in any single partition

3309

Received count

Total number of entries fetched by zero-arc nodes, by agent.

Total

11456

Agents affected

18 / 27

Mean per agent

Total count divided by number of partitions, rounded to nearest whole number

424

Max

Highest count in any single partition

3255

Min

Lowest count in any single partition

0

Open connections

The number of currently open connections to other conductors.

full-arc

p5

15

<

mean

24.9

SD

3.41

<

p95

26

zero-arc

p5

0

<

mean

4.49

SD

6.51

<

p95

19

Retrieval errors

Errors encountered by zero-arc nodes when attempting to retrieve entries created by full-arc nodes.

Total

336

Agents affected

25 / 27

Mean per agent

Total count divided by number of partitions, rounded to nearest whole number

12

Max

Highest count in any single partition

14

Min

Lowest count in any single partition

0

Error count

The number of errors accumulated across all nodes.

337

Holochain MetricsClick to toggle visibility.

Cascade duration

Time taken to execute a cascade (get) query inside Holochain.

mean

0.032101

s

SD

0.274443

s

p50

0.060331

s

<

p95

0.799492

s

<

p99

1.146741

s

Cascade fetch errors

Count of network fetch errors during cascade calls.

total

9

over 812.840136862s

mean rate

77.08

/s

std

181.3

/s

p5

0

/s

p95

403.03

/s

peak

993.49

/s

WASM usage: 'timed_and_validated::create_timed_entry'

Metered usage count of the WASM ribosome for this zome function.

total

233717764

over 875.435475028s

mean rate

4.71582636e+06

/s

std

3.20892469e+06

/s

p5

434580.71

/s

p95

9.9039816e+06

/s

peak

1.077104932e+07

/s

WASM usage: 'timed_and_validated::get_timed_entries_network'

Metered usage count of the WASM ribosome for this zome function.

total

9720815904

over 885.846861365s

mean rate

1.21481676255e+10

/s

std

2.74910487009e+10

/s

p5

3.0698827e+07

/s

p95

6.88221276391e+10

/s

peak

2.21711388605e+11

/s

WASM usage: 'timed_and_validated_integrity::entry_defs'

Metered usage count of the WASM ribosome for this zome function.

total

8665924

over 888.498348184s

mean rate

2.3868614006e+08

/s

std

3.4504952262e+08

/s

p5

6.34688166e+06

/s

p95

1.04189632789e+09

/s

peak

1.82533591318e+09

/s

WASM usage: 'timed_and_validated_integrity::validate'

Metered usage count of the WASM ribosome for this zome function.

total

0

over 885.846861365s

Not enough time data to show a trend.

Zome call duration: 'timed_and_validated::create_timed_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.008171

s

SD

0.001195

s

p50

0.007689

s

<

p95

0.010002

s

<

p99

0.010188

s

Zome call duration: 'timed_and_validated::get_timed_entries_network'

Duration of this zome call as measured by Holochain internally.

mean

0.026022

s

SD

0.066761

s

p50

0.0131

s

<

p95

0.114836

s

<

p99

0.579865

s

WASM call duration: 'timed_and_validated::create_timed_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.007404

s

SD

0.001101

s

p50

0.006992

s

<

p95

0.00907

s

<

p99

0.009244

s

WASM call duration: 'timed_and_validated::get_timed_entries_network'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.425182

s

SD

20.992748

s

p50

0.361783

s

<

p95

29.407462

s

<

p99

115.881176

s

WASM call duration: 'timed_and_validated_integrity::entry_defs'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.001292

s

SD

0.000651

s

p50

0.000953

s

<

p95

0.003141

s

<

p99

0.003316

s

WASM call duration: 'timed_and_validated_integrity::validate'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.007589

s

SD

0.002523

s

p50

0.000991

s

<

p95

0.007303

s

<

p99

0.014294

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.000514

s

SD

7.6e-05

s

p50

0.000515

s

<

p95

0.000583

s

<

p99

0.000591

s

Host function call duration: '__hc__create_link_1'

Duration of this host function call invoked from within WASM.

mean

0.000479

s

SD

6.8e-05

s

p50

0.000492

s

<

p95

0.000532

s

<

p99

0.000556

s

Host function call duration: '__hc__get_1'

Duration of this host function call invoked from within WASM.

mean

0.030762

s

SD

0.454753

s

p50

0.267936

s

<

p95

0.905063

s

<

p99

2.161215

s

Host function call duration: '__hc__get_links_1'

Duration of this host function call invoked from within WASM.

mean

0.060021

s

SD

4.816837

s

p50

0.038808

s

<

p95

4.123282

s

<

p99

30.824691

s

Host function call duration: '__hc__must_get_valid_record_1'

Duration of this host function call invoked from within WASM.

mean

0.013109

s

SD

0.006435

s

p50

0.011865

s

<

p95

0.024047

s

<

p99

0.025645

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.002002

s

SD

0.000936

s

p50

0.001454

s

<

p95

0.004651

s

<

p99

0.004918

s

Post-commit duration

Time spent executing post-commit workflows.

mean

0.000137

s

SD

7.5e-05

s

p50

9.4e-05

s

<

p95

0.000225

s

<

p99

0.000434

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

47.2

s

<

mean

445.09

s

SD

255.6

s

<

p95

846.53

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

0

over 888.498287657s

Not enough time data to show a trend.

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

81.133929

s

SD

30.345523

s

p50

0

s

<

p95

19.024433

s

<

p99

253.337389

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

3.057377

s

SD

1.540878

s

p50

0.000995

s

<

p95

3.350555

s

<

p99

10.459534

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.049605

s

SD

0.014678

s

p50

0.001183

s

<

p95

0.036876

s

<

p99

0.093905

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.128798

s

SD

0.031876

s

p50

0.014725

s

<

p95

0.089076

s

<

p99

0.175558

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

13.411395

s

SD

7.57856

s

p50

0.002281

s

<

p95

18.698848

s

<

p99

39.928301

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

7.843189

s

SD

8.498315

s

p50

0.00204

s

<

p95

0.206817

s

<

p99

55.513887

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

0.001106

s

SD

0.001598

s

p50

0.00011

s

<

p95

0.005058

s

<

p99

0.007942

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.008385

s

SD

0.003328

s

p50

0.001623

s

<

p95

0.010583

s

<

p99

0.016745

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000291

s

SD

9.9e-05

s

p50

0.000342

s

<

p95

0.000485

s

<

p99

0.000494

s

P2P request roundtrip: 'get'

Time spent sending a get request and awaiting its response.

mean

1.718505

s

SD

29.720286

s

p50

1.395369

s

<

p95

60.001571

s

<

p99

60.002547

s

P2P request roundtrip: 'get_links'

Time spent sending a get_links request and awaiting its response.

mean

1.716101

s

SD

23.526513

s

p50

7.52947

s

<

p95

60.001333

s

<

p99

60.001735

s

P2P request roundtrip: 'send_validation_receipts'

Time spent sending a send_validation_receipts request and awaiting its response.

mean

3.917559

s

SD

21.094897

s

p50

6.517765

s

<

p95

60.000763

s

≤

p99

60.000763

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

6.86

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

1007

B/s

<

mean

67.15

KiB/s

SD

221.38

KiB/s

<

p95

250.40

KiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

517

B/s

<

mean

46.27

KiB/s

SD

196.48

KiB/s

<

p95

339.20

KiB/s

Total bytes received

Total bytes received on primary network interface

count

1.42

GiB

mean

1.64

MiB/s

Total bytes sent

Total bytes sent on primary network interface

count

1005.42

MiB

mean

1.13

MiB/s

CPU spike anomaly

CPU spike anomaly was detected

Detected

Warning CPU p99 reached 91.4%

Memory leak anomaly

Memory leak anomaly was detected

NotDetected

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

Detected

Warning Moderate swap usage (13.9% swap used)

System overload anomaly

System overload anomaly was detected

Detected

Warning 8% of hosts overloaded (load5/ncpus > 1.0)

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

0.73

%

<

mean

13.88

%

SD

23.62

%

<

p95

80.18

%

System

7.02

%

CPU percentiles

CPU usage percentiles

p50

2.55

%

p95

80.18

%

p99

91.43

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

4

hosts

mean time

293.26

s

Memory used percentage

Percentage of memory used

p5

6.85

%

<

mean

17.67

%

SD

8.29

%

<

p95

33.51

%

Memory available percentage

Percentage of available memory

p5

66.5

%

<

mean

82.33

%

SD

8.29

%

<

p95

93.15

%

Max host memory used

Maximum memory usage percentage across all hosts

max

38.13

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

13.87

%

Memory growth rate

Rate of memory growth over time

growth

0.17

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.01

MiB/s

Disk write throughput

Disk write throughput in MiB/s

27.29

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /

0/2

hosts

Mount Point /efi-boot

0/2

hosts

Mount Point /etc/hostname

0/12

hosts

Mount Point /etc/hosts

0/12

hosts

Mount Point /etc/nomad.d/client-meta.json

0/12

hosts

Mount Point /etc/resolv.conf

0/12

hosts

Mount Point /nix/store

0/2

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

0.84

5 min

0.98

15 min

1.36

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

7.69

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

0

%

<

mean

3.4096

%

SD

11.3822

%

<

p95

28.67

%

60 second average

3.6404

%

300 second average

4.9046

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

0.8189

%

SD

3.0206

%

<

p95

4.23

%

60 second average

0.7966

%

300 second average

0.7519

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

0.3696

%

SD

1.21

%

<

p95

2.32

%

60 second average

0.3579

%

300 second average

0.3705

%

Holochain process CPU usage

CPU usage by Holochain process

p5

0.2

%

<

mean

7.11

%

SD

18.81

%

<

p95

62.98

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

330.27

KiB

<

mean

402.67

KiB

SD

56.52

KiB

<

p95

490.63

KiB

Holochain process threads

Number of threads in Holochain process

p5

21

threads

<

mean

26.44

threads

SD

5.2

threads

<

p95

37

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

52

file descriptors

<

mean

60.89

file descriptors

SD

8.92

file descriptors

<

p95

77

file descriptors

Local Signals

↑ Back to index

This scenario is for testing the speed and reliability of local signals. It uses a zome with a function that will emit 10,000 signals. The scenario measures how long it takes to send the signals and how many have been received by the time the zome call ends.

Started

Thu, 17 Sep 2026 09:01:32 UTC

Duration

300s

Peer count

1

Peer count at end

1

Behaviours

default (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

local_signals_35182846570_1

Send timing

The time taken to emit a batch of 10,000 signals.

mean

1.454631

s

SD

0.923911

s

p50

1.321386

s

<

p95

3.422648

s

≤

p99

3.422648

s

Receive timing

The time taken to receive the completed batch, to the nearest 250ms.

mean

27.737343

s

SD

6.791929

s

p50

30.001327

s

<

p95

30.002119

s

≤

p99

30.002119

s

Success ratio

The proportion of signals successfully received in a batch.

min

0.4970000088214874

<

mean

0.5997

SD

0.0682

<

max

0.7243000268936157

Holochain MetricsClick to toggle visibility.

WASM usage: 'signal::emit_10k_signals'

Metered usage count of the WASM ribosome for this zome function.

total

246763944

over 270.003332761s

mean rate

913929.09

/s

std

1.40845931e+06

/s

p5

0

/s

p95

3.08452895e+06

/s

peak

3.08454259e+06

/s

Zome call duration: 'signal::emit_10k_signals'

Duration of this zome call as measured by Holochain internally.

mean

0.189646

s

SD

0.017795

s

p50

0.177727

s

<

p95

0.21433

s

≤

p99

0.21433

s

WASM call duration: 'signal::emit_10k_signals'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.18849

s

SD

0.017796

s

p50

0.176593

s

<

p95

0.213286

s

≤

p99

0.213286

s

Host function call duration: '__hc__emit_signal_1'

Duration of this host function call invoked from within WASM.

mean

1.3e-05

s

SD

1e-06

s

p50

1.2e-05

s

<

p95

1.5e-05

s

≤

p99

1.5e-05

s

Emitted local signals

Count of local signals emitted via the emit_signal host function.

total

80000

over 270.003329926s

mean rate

296.29

/s

std

456.62

/s

p5

0

/s

p95

999.99

/s

peak

1000

/s

Post-commit duration

Time spent executing post-commit workflows.

mean

0.000151

s

SD

0

s

p50

0.000151

s

≤

p95

0.000151

s

≤

p99

0.000151

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

16.92

s

<

mean

141.92

s

SD

80.78

s

<

p95

266.92

s

Dropped WebSocket signals

Count of signals dropped from the app WebSocket due to channel overload.

total

3426

over 270.003328355s

mean rate

12.69

/s

std

21.66

/s

p5

0

/s

p95

60.4

/s

peak

67.3

/s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

0

over 270.003328735s

Not enough time data to show a trend.

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.001017

s

SD

0

s

p50

0.001017

s

≤

p95

0.001017

s

≤

p99

0.001017

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.001773

s

SD

0

s

p50

0.001773

s

≤

p95

0.001773

s

≤

p99

0.001773

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.003861

s

SD

0.00045

s

p50

0.004043

s

<

p95

0.00467

s

≤

p99

0.00467

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

0.001887

s

SD

0

s

p50

0.001887

s

≤

p95

0.001887

s

≤

p99

0.001887

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

0.000287

s

SD

0

s

p50

0.000287

s

≤

p95

0.000287

s

≤

p99

0.000287

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

0.000145

s

SD

3.4e-05

s

p50

0.000143

s

<

p95

0.000171

s

≤

p99

0.000171

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.001052

s

SD

6.5e-05

s

p50

0.001077

s

<

p95

0.001144

s

≤

p99

0.001144

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000307

s

SD

0

s

p50

0.000307

s

≤

p95

0.000307

s

≤

p99

0.000307

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

15.13

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

3.37

KiB/s

<

mean

856.06

KiB/s

SD

1.99

MiB/s

<

p95

7.71

MiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

196

B/s

<

mean

19.02

KiB/s

SD

26.02

KiB/s

<

p95

93.64

KiB/s

Total bytes received

Total bytes received on primary network interface

count

234.08

MiB

mean

856.06

KiB/s

Total bytes sent

Total bytes sent on primary network interface

count

5.20

MiB

mean

19.02

KiB/s

CPU spike anomaly

CPU spike anomaly was detected

Detected

Warning CPU p99 reached 98.2%

Memory leak anomaly

Memory leak anomaly was detected

Detected

Warning Memory growing at 4.71 MiB/s

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

NotDetected

System overload anomaly

System overload anomaly was detected

NotDetected

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

1.91

%

<

mean

21.63

%

SD

23.3

%

<

p95

69.85

%

System

6.50

%

CPU percentiles

CPU usage percentiles

p50

14.81

%

p95

69.85

%

p99

98.25

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

1

hosts

mean time

10.71

s

Memory used percentage

Percentage of memory used

p5

12.82

%

<

mean

19.38

%

SD

5.7

%

<

p95

29.59

%

Memory available percentage

Percentage of available memory

p5

70.41

%

<

mean

80.62

%

SD

5.7

%

<

p95

87.18

%

Max host memory used

Maximum memory usage percentage across all hosts

max

19.38

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

0.00

%

Memory growth rate

Rate of memory growth over time

growth

4.71

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.00

MiB/s

Disk write throughput

Disk write throughput in MiB/s

4.42

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /etc/nomad.d/client-meta.json

0/1

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

0.55

5 min

0.33

15 min

0.14

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

0.00

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

0.26

%

<

mean

8.7831

%

SD

12.4779

%

<

p95

34.77

%

60 second average

6.6662

%

300 second average

2.8934

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0628

%

300 second average

0.1207

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0555

%

300 second average

0.1066

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

0.4783

%

SD

0.6841

%

<

p95

1.78

%

60 second average

0.6300

%

300 second average

0.6624

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

0.379

%

SD

0.6316

%

<

p95

1.63

%

60 second average

0.5234

%

300 second average

0.5762

%

Holochain process CPU usage

CPU usage by Holochain process

p5

0.55

%

<

mean

6.96

%

SD

12

%

<

p95

40.83

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

103.79

KiB

<

mean

207.64

KiB

SD

46.19

KiB

<

p95

250.29

KiB

Holochain process threads

Number of threads in Holochain process

p5

5

threads

<

mean

24.1

threads

SD

8.2

threads

<

p95

33

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

11

file descriptors

<

mean

66.77

file descriptors

SD

19.66

file descriptors

<

p95

87

file descriptors

Mixed Arc `get_agent_activity`

↑ Back to index

A mixed-arc scenario that measures the DHT's ability to propagate agent activity and make it available via get_agent_activity calls. The scenario has three roles:

zero_write: A zero-arc conductor that creates entries on its source chain.
full_write: A full-arc conductor that creates entries on its source chain.
zero_read: A zero-arc conductor that picks a random writer and repeatedly calls get_agent_activity to observe new entries appearing on the writer's chain, measuring the lag between creation and discovery.

Started

Thu, 17 Sep 2026 12:28:03 UTC

Duration

900s

Peer count

30

Peer count at end

27

Behaviours

full_write (1 agent)
zero_read (1 agent)
zero_write (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

mixed_arc_get_agent_activity_35182846570_1

Entry created count

The total number of entries created by write agents over the run, partitioned by agent and behaviour.

Total

107151

Agents affected

19 / 19

Mean per agent

Total count divided by number of partitions, rounded to nearest whole number

5640

Max

Highest count in any single partition

10076

Min

Lowest count in any single partition

2378

Observed chain advancement

The maximum highest-observed action sequence per write agent, aggregated across all reading agents. The reading dimension is collapsed by taking the maximum observed value for each write agent (any reader successfully tracking that point counts as propagation). Captures how far readers tracked writers’ chains during the run.

Mean max (across write agents)

Mean of the highest chain head value observed per write agent. Each write agent contributes its maximum — the value seen by any reader.

2344.75

Max

Highest chain head value observed for any single write agent.

3814

Write agents observed

4

Chain head delay timing

The time lag between when a writer creates a new entry and when a reader discovers it via get_agent_activity. This is measured per new action in the chain.

Mean

mean

0.066893

s

SD

0.275566

s

Max

Highest per-partition mean latency

max mean

0.289868

s

Min

Lowest per-partition mean latency

min mean

-0.002384

s

Chain head delay rate

The rate at which readers detect new chain heads appearing on writers' chains.

Mean

mean

5.36

/s

Max

Highest per-partition mean rate

max

12.39

/s

Min

Lowest per-partition mean rate

min

1.8

/s

get_agent_activity_full zome call timing

The time taken to complete the get_agent_activity_full zome call that queries a writer's chain.

Mean

mean

0.21565

s

SD

0.537304

s

Max

Highest per-partition mean latency

max mean

0.638818

s

Min

Lowest per-partition mean latency

min mean

0.069527

s

Retrieval errors

A running accumulation of errors encountered by read agents when calling get_agent_activity.

Total

51

Agents affected

8 / 8

Mean per agent

Total count divided by number of partitions, rounded to nearest whole number

6

Max

Highest count in any single partition

12

Min

Lowest count in any single partition

1

Open connections

The number of currently open connections to other conductors, partitioned by agent behaviour.

behaviour: full_write

p5

26

≤

mean

25.74

SD

1.58

<

p95

26

behaviour: zero_read

p5

10

<

mean

14.21

SD

3.14

<

p95

21

behaviour: zero_write

p5

0

<

mean

14.53

SD

5.5

<

p95

21

Final error count

The total number of errors accumulated across all agents over the run.

51

Holochain MetricsClick to toggle visibility.

Cascade duration

Time taken to execute a cascade (get) query inside Holochain.

mean

0.130834

s

SD

0.980996

s

p50

0.384074

s

<

p95

2.819919

s

<

p99

4.7855

s

Cascade fetch errors

Count of network fetch errors during cascade calls.

total

1

over 774.416916953s

mean rate

10.12

/s

std

21.18

/s

p5

0.23

/s

p95

65.64

/s

peak

71.04

/s

Zome call duration: 'agent_activity::announce_write_behaviour'

Duration of this zome call as measured by Holochain internally.

mean

0.007527

s

SD

0.003316

s

p50

0.00634

s

<

p95

0.016036

s

≤

p99

0.016036

s

Zome call duration: 'agent_activity::create_sample_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.004708

s

SD

0.002888

s

p50

0.004483

s

<

p95

0.011329

s

<

p99

0.015047

s

Zome call duration: 'agent_activity::get_agent_activity_full'

Duration of this zome call as measured by Holochain internally.

mean

0.134046

s

SD

1.319376

s

p50

0.234419

s

<

p95

4.228014

s

<

p99

5.366859

s

Zome call duration: 'agent_activity::get_random_agent_with_write_behaviour'

Duration of this zome call as measured by Holochain internally.

mean

0.034958

s

SD

0.296446

s

p50

0.3886

s

<

p95

1.100656

s

≤

p99

1.100656

s

WASM call duration: 'agent_activity::announce_write_behaviour'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.00633

s

SD

0.002978

s

p50

0.005514

s

<

p95

0.014149

s

≤

p99

0.014149

s

WASM call duration: 'agent_activity::create_sample_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.004052

s

SD

0.002778

s

p50

0.003811

s

<

p95

0.010546

s

<

p99

0.014206

s

WASM call duration: 'agent_activity::get_agent_activity_full'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.133865

s

SD

1.318863

s

p50

0.233776

s

<

p95

4.226633

s

<

p99

5.365909

s

WASM call duration: 'agent_activity::get_random_agent_with_write_behaviour'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.033927

s

SD

0.296181

s

p50

0.387771

s

<

p95

1.098621

s

≤

p99

1.098621

s

WASM call duration: 'agent_activity_integrity::entry_defs'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.001322

s

SD

0.000665

s

p50

0.001253

s

<

p95

0.002788

s

<

p99

0.003441

s

Host function call duration: '__hc__agent_info_1'

Duration of this host function call invoked from within WASM.

mean

4.4e-05

s

SD

2.7e-05

s

p50

3.8e-05

s

<

p95

0.000129

s

≤

p99

0.000129

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.000663

s

SD

0.001098

s

p50

0.000458

s

<

p95

0.003212

s

<

p99

0.005372

s

Host function call duration: '__hc__create_link_1'

Duration of this host function call invoked from within WASM.

mean

0.000681

s

SD

0.000352

s

p50

0.000635

s

<

p95

0.001514

s

≤

p99

0.001514

s

Host function call duration: '__hc__get_agent_activity_1'

Duration of this host function call invoked from within WASM.

mean

0.132556

s

SD

1.318449

s

p50

0.232633

s

<

p95

4.2233

s

<

p99

5.3639

s

Host function call duration: '__hc__get_links_1'

Duration of this host function call invoked from within WASM.

mean

0.029592

s

SD

0.29621

s

p50

0.384136

s

<

p95

1.095037

s

≤

p99

1.095037

s

Host function call duration: '__hc__random_bytes_1'

Duration of this host function call invoked from within WASM.

mean

1.1e-05

s

SD

1.5e-05

s

p50

1.1e-05

s

<

p95

5.5e-05

s

≤

p99

5.5e-05

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.002091

s

SD

0.001368

s

p50

0.002133

s

<

p95

0.005167

s

<

p99

0.006448

s

Post-commit duration

Time spent executing post-commit workflows.

mean

0.000175

s

SD

0.000175

s

p50

0.000118

s

<

p95

0.000635

s

<

p99

0.00075

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

47.2

s

<

mean

445.24

s

SD

255.72

s

<

p95

846.38

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

0

over 888.346312636s

Not enough time data to show a trend.

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

15.344641

s

SD

24.666043

s

p50

0

s

<

p95

16.115236

s

<

p99

135.949045

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.069007

s

SD

0.156655

s

p50

0.00139

s

<

p95

0.250275

s

<

p99

0.989217

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.034064

s

SD

0.160735

s

p50

0.001606

s

<

p95

0.523044

s

<

p99

0.69917

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.09547

s

SD

0.07102

s

p50

0.056012

s

<

p95

0.21218

s

<

p99

0.284835

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

12.476379

s

SD

27.872407

s

p50

0.003822

s

<

p95

35.376075

s

<

p99

164.646304

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

3.109829

s

SD

6.898017

s

p50

0.001741

s

<

p95

0.326233

s

<

p99

8.828082

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

0.002661

s

SD

0.001502

s

p50

0.001579

s

<

p95

0.00456

s

<

p99

0.005968

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.003089

s

SD

0.003061

s

p50

0.001138

s

<

p95

0.003587

s

<

p99

0.022637

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000654

s

SD

0.000971

s

p50

0.00033

s

<

p95

0.003093

s

<

p99

0.004936

s

P2P request roundtrip: 'get'

Time spent sending a get request and awaiting its response.

mean

17.130511

s

SD

27.636623

s

p50

11.723302

s

<

p95

60.006068

s

<

p99

60.008161

s

P2P request roundtrip: 'get_links'

Time spent sending a get_links request and awaiting its response.

mean

0.210426

s

SD

0.216847

s

p50

0.258523

s

<

p95

0.805986

s

≤

p99

0.805986

s

P2P request roundtrip: 'get_agent_activity'

Time spent sending a get_agent_activity request and awaiting its response.

mean

0.117037

s

SD

29.684311

s

p50

0.674199

s

<

p95

60.001431

s

<

p99

60.002055

s

P2P request roundtrip: 'send_validation_receipts'

Time spent sending a send_validation_receipts request and awaiting its response.

mean

1.191025

s

SD

18.232769

s

p50

1.035354

s

<

p95

60.001663

s

≤

p99

60.001663

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

46.44

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

1.63

KiB/s

<

mean

248.78

KiB/s

SD

319.15

KiB/s

<

p95

699.17

KiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

1.16

KiB/s

<

mean

215.11

KiB/s

SD

190.94

KiB/s

<

p95

519.75

KiB/s

Total bytes received

Total bytes received on primary network interface

count

5.28

GiB

mean

6.07

MiB/s

Total bytes sent

Total bytes sent on primary network interface

count

4.56

GiB

mean

5.25

MiB/s

CPU spike anomaly

CPU spike anomaly was detected

Detected

Warning CPU p99 reached 98.5%

Memory leak anomaly

Memory leak anomaly was detected

NotDetected

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

Detected

Warning Moderate swap usage (13.9% swap used)

System overload anomaly

System overload anomaly was detected

Detected

Warning 38% of hosts overloaded (load5/ncpus > 1.0)

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

1.38

%

<

mean

58.33

%

SD

37.08

%

<

p95

98.25

%

System

11.89

%

CPU percentiles

CPU usage percentiles

p50

66.99

%

p95

98.25

%

p99

98.45

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

11

hosts

mean time

820.94

s

Memory used percentage

Percentage of memory used

p5

6.12

%

<

mean

16.52

%

SD

7.73

%

<

p95

30.48

%

Memory available percentage

Percentage of available memory

p5

69.53

%

<

mean

83.48

%

SD

7.73

%

<

p95

93.88

%

Max host memory used

Maximum memory usage percentage across all hosts

max

36.76

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

13.87

%

Memory growth rate

Rate of memory growth over time

growth

0.26

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.03

MiB/s

Disk write throughput

Disk write throughput in MiB/s

72.37

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /

0/2

hosts

Mount Point /efi-boot

0/2

hosts

Mount Point /etc/hostname

0/12

hosts

Mount Point /etc/hosts

0/12

hosts

Mount Point /etc/nomad.d/client-meta.json

0/12

hosts

Mount Point /etc/resolv.conf

0/12

hosts

Mount Point /nix/store

0/2

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

3.48

5 min

2.74

15 min

1.99

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

38.46

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

0

%

<

mean

27.8853

%

SD

29.4178

%

<

p95

79.15

%

60 second average

26.4625

%

300 second average

20.4069

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

1.6536

%

SD

5.6064

%

<

p95

7.77

%

60 second average

1.5601

%

300 second average

1.2586

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

0.4975

%

SD

1.9242

%

<

p95

4.51

%

60 second average

0.4746

%

300 second average

0.4121

%

Holochain process CPU usage

CPU usage by Holochain process

p5

0.55

%

<

mean

51.89

%

SD

37.31

%

<

p95

97.02

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

224.10

KiB

<

mean

307.78

KiB

SD

76.01

KiB

<

p95

454.15

KiB

Holochain process threads

Number of threads in Holochain process

p5

21

threads

<

mean

32.01

threads

SD

6.57

threads

<

p95

38

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

57

file descriptors

<

mean

73.76

file descriptors

SD

12.44

file descriptors

<

p95

86

file descriptors

Mixed Arc `must_get_agent_activity`

↑ Back to index

A scenario where write agents create entries in batches of 10, while must_get_agent_activity agents each pick a random write agent and repeatedly attempt to create an entry that references the chain top of their latest batch. This reference means that the entry's validation function needs to make a must_get_agent_activity call.

The purpose of this scenario is to measure the time it takes for published agent activity data to be gossiped among authorities and become available to peers that query it via must_get_agent_activity.

This test is similar to Write Validated must_get_agent_activity, but some write agents are zero-arc and all must_get_agent_activity agents are zero-arc. This forces the must_get_agent_activity host call to go to the network.

Started

Thu, 17 Sep 2026 12:43:38 UTC

Duration

900s

Peer count

30

Peer count at end

28

Behaviours

full_write (1 agent)
zero_must_get_agent_activity (1 agent)
zero_write (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

mixed_arc_must_get_agent_activity_35182846570_1

Highest observed action_seq

The maximum chain length observed per write agent, aggregated across all reading agents. The reading dimension is collapsed by taking the maximum observed value for each write agent (any successful read counts as propagation). This reflects the DHT’s ability to propagate agent activity ops and make them available to querying peers.

Mean max (across write agents)

Mean of the highest chain head value observed per write agent. Each write agent contributes its maximum — the value seen by any reader.

1369

Max

Highest chain head value observed for any single write agent.

1545

Write agents observed

2

Chain batch delay timing

The time between a write agent's creation of a batch and a must_get_agent_activity agent's successful discovery of the batch and creation/self-validation of a new entry that depends on it.

Mean

mean

150.709108

s

SD

110.93153

s

Max

Highest per-partition mean latency

max mean

262.173829

s

Min

Lowest per-partition mean latency

min mean

62.144205

s

Chain batch delay rate

The rate at which a must_get_agent_activity agent was able to discover batches and create/self-validate new entries that depend on them.

Mean

mean

1.88

/s

Max

Highest per-partition mean rate

max

2.47

/s

Min

Lowest per-partition mean rate

min

1.47

/s

create_validated_sample_entry zome call timing

The time taken to complete the zome function call that creates the entry that depends on a write agent's source chain.

Mean

mean

0.835699

s

SD

0.948621

s

Max

Highest per-partition mean latency

max mean

1.428568

s

Min

Lowest per-partition mean latency

min mean

0.355929

s

Retrieval errors

A running accumulation of the errors encountered by an agent when attempting to self-validate actions that depend on must_get_agent_activity calls.

Total

21909

Agents affected

8 / 8

Mean per agent

Total count divided by number of partitions, rounded to nearest whole number

2739

Max

Highest count in any single partition

4359

Min

Lowest count in any single partition

582

Open connections

The average number of open connections maintained by an agent over the run, broken down by behaviour.

behaviour: full_write

p5

26

≤

mean

25.84

SD

1.35

<

p95

26

behaviour: zero_must_get_agent_activity

p5

0

<

mean

12.17

SD

6.01

<

p95

19

behaviour: zero_write

p5

0

<

mean

13.24

SD

5.03

<

p95

20

Final error count

The total number of all types of error accumulated over the run by all agents.

29772

Holochain MetricsClick to toggle visibility.

Cascade duration

Time taken to execute a cascade (get) query inside Holochain.

mean

0.054572

s

SD

0.192246

s

p50

0.084364

s

<

p95

0.580344

s

<

p99

0.691676

s

Cascade fetch errors

Count of network fetch errors during cascade calls.

total

2

over 780.009407642s

mean rate

3.67

/s

std

12.34

/s

p5

0

/s

p95

30.26

/s

peak

61.14

/s

WASM usage: 'validated_must_get_agent_activity::announce_write_behaviour'

Metered usage count of the WASM ribosome for this zome function.

total

0

over 878.211746276s

Not enough time data to show a trend.

WASM usage: 'validated_must_get_agent_activity::create_sample_entries_batch'

Metered usage count of the WASM ribosome for this zome function.

total

2356294212

over 878.211746276s

mean rate

9.249735813e+08

/s

std

1.78262310397e+09

/s

p5

0

/s

p95

4.40780161972e+09

/s

peak

1.17762733227e+10

/s

WASM usage: 'validated_must_get_agent_activity::create_validated_sample_entry'

Metered usage count of the WASM ribosome for this zome function.

total

148429503

over 881.575351288s

mean rate

3.6374864288e+08

/s

std

5.9953449407e+08

/s

p5

577606.51

/s

p95

1.73140546791e+09

/s

peak

2.47326064145e+09

/s

WASM usage: 'validated_must_get_agent_activity::get_chain_top_for_batch'

Metered usage count of the WASM ribosome for this zome function.

total

145956970

over 881.575351288s

mean rate

1.1685572078e+08

/s

std

1.8698495308e+08

/s

p5

351048.72

/s

p95

4.3204700118e+08

/s

peak

9.7606295454e+08

/s

WASM usage: 'validated_must_get_agent_activity::get_random_agent_with_write_behaviour'

Metered usage count of the WASM ribosome for this zome function.

total

20163

over 881.575351288s

mean rate

11829.79

/s

std

16814.92

/s

p5

0

/s

p95

44194.04

/s

peak

44227.97

/s

WASM usage: 'validated_must_get_agent_activity_integrity::entry_defs'

Metered usage count of the WASM ribosome for this zome function.

total

212911344

over 888.211785118s

mean rate

9.7165509954e+08

/s

std

2.28214573027e+09

/s

p5

4.55563381e+06

/s

p95

6.53979667096e+09

/s

peak

1.22275605958e+10

/s

WASM usage: 'validated_must_get_agent_activity_integrity::validate'

Metered usage count of the WASM ribosome for this zome function.

total

2143066348

over 888.211785118s

mean rate

7.80251044278e+09

/s

std

2.23524797086e+10

/s

p5

0

/s

p95

3.68891249183e+10

/s

peak

1.40319956685e+11

/s

Zome call duration: 'validated_must_get_agent_activity::announce_write_behaviour'

Duration of this zome call as measured by Holochain internally.

mean

0.007194

s

SD

0.002911

s

p50

0.008312

s

<

p95

0.011622

s

≤

p99

0.011622

s

Zome call duration: 'validated_must_get_agent_activity::create_sample_entries_batch'

Duration of this zome call as measured by Holochain internally.

mean

0.059686

s

SD

0.026554

s

p50

0.044716

s

<

p95

0.114348

s

<

p99

0.128955

s

Zome call duration: 'validated_must_get_agent_activity::create_validated_sample_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.006336

s

SD

0.002649

s

p50

0.005574

s

<

p95

0.011839

s

<

p99

0.011884

s

Zome call duration: 'validated_must_get_agent_activity::get_chain_top_for_batch'

Duration of this zome call as measured by Holochain internally.

mean

0.271521

s

SD

0.126571

s

p50

0.20522

s

<

p95

0.483162

s

<

p99

0.575565

s

Zome call duration: 'validated_must_get_agent_activity::get_random_agent_with_write_behaviour'

Duration of this zome call as measured by Holochain internally.

mean

0.262201

s

SD

0.156384

s

p50

0.354278

s

<

p95

0.414793

s

≤

p99

0.414793

s

WASM call duration: 'validated_must_get_agent_activity::announce_write_behaviour'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.00599

s

SD

0.002514

s

p50

0.007183

s

<

p95

0.009845

s

≤

p99

0.009845

s

WASM call duration: 'validated_must_get_agent_activity::create_sample_entries_batch'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.058849

s

SD

0.026351

s

p50

0.043924

s

<

p95

0.112811

s

<

p99

0.127344

s

WASM call duration: 'validated_must_get_agent_activity::create_validated_sample_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.005268

s

SD

0.002268

s

p50

0.004627

s

<

p95

0.009901

s

<

p99

0.009941

s

WASM call duration: 'validated_must_get_agent_activity::get_chain_top_for_batch'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.223594

s

SD

0.201344

s

p50

0.279627

s

<

p95

0.627606

s

<

p99

0.761838

s

WASM call duration: 'validated_must_get_agent_activity::get_random_agent_with_write_behaviour'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.261221

s

SD

0.156148

s

p50

0.353098

s

<

p95

0.413194

s

≤

p99

0.413194

s

WASM call duration: 'validated_must_get_agent_activity_integrity::entry_defs'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.001127

s

SD

0.000695

s

p50

0.00111

s

<

p95

0.003198

s

<

p99

0.00339

s

WASM call duration: 'validated_must_get_agent_activity_integrity::validate'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.011209

s

SD

0.042944

s

p50

0.003768

s

<

p95

0.125437

s

<

p99

0.205096

s

Host function call duration: '__hc__agent_info_1'

Duration of this host function call invoked from within WASM.

mean

2.3e-05

s

SD

1.4e-05

s

p50

2.2e-05

s

<

p95

5e-05

s

<

p99

5.3e-05

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.000418

s

SD

0.000182

s

p50

0.000356

s

<

p95

0.000805

s

<

p99

0.00091

s

Host function call duration: '__hc__create_link_1'

Duration of this host function call invoked from within WASM.

mean

0.000401

s

SD

0.000259

s

p50

0.000408

s

<

p95

0.000991

s

<

p99

0.001289

s

Host function call duration: '__hc__get_links_1'

Duration of this host function call invoked from within WASM.

mean

0.219168

s

SD

0.180505

s

p50

0.32318

s

<

p95

0.595629

s

<

p99

0.751931

s

Host function call duration: '__hc__must_get_agent_activity_1'

Duration of this host function call invoked from within WASM.

mean

0.040748

s

SD

0.165643

s

p50

0.035987

s

<

p95

0.501713

s

<

p99

0.669329

s

Host function call duration: '__hc__query_1'

Duration of this host function call invoked from within WASM.

mean

0.022991

s

SD

0.016388

s

p50

0.010957

s

<

p95

0.055915

s

<

p99

0.06559

s

Host function call duration: '__hc__random_bytes_1'

Duration of this host function call invoked from within WASM.

mean

1.3e-05

s

SD

7e-06

s

p50

1e-05

s

<

p95

2.6e-05

s

≤

p99

2.6e-05

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.001758

s

SD

0.001111

s

p50

0.002079

s

<

p95

0.004798

s

<

p99

0.005034

s

Post-commit duration

Time spent executing post-commit workflows.

mean

0.00016

s

SD

9.4e-05

s

p50

0.000119

s

<

p95

0.000391

s

<

p99

0.000417

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

47.16

s

<

mean

445.18

s

SD

255.72

s

<

p95

846.21

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

20096

over 888.211913133s

mean rate

79066.54

/s

std

236470.27

/s

p5

0

/s

p95

348839.05

/s

peak

1.45385559e+06

/s

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

137.578245

s

SD

73.729251

s

p50

0

s

<

p95

197.87928

s

<

p99

318.721049

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

1.81488

s

SD

1.084764

s

p50

0.001472

s

<

p95

3.062484

s

<

p99

4.042109

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.156073

s

SD

0.157332

s

p50

0.001687

s

<

p95

0.465927

s

<

p99

0.550236

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.097613

s

SD

0.109355

s

p50

0.015556

s

<

p95

0.348464

s

<

p99

0.456727

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

3.461496

s

SD

11.741225

s

p50

0.004909

s

<

p95

31.188749

s

<

p99

74.248531

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

20.543975

s

SD

23.36217

s

p50

0.003142

s

<

p95

57.784668

s

<

p99

93.997516

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

0.002193

s

SD

0.001494

s

p50

0.000279

s

<

p95

0.004763

s

<

p99

0.005667

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.003099

s

SD

0.003788

s

p50

0.001535

s

<

p95

0.00852

s

<

p99

0.022869

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000355

s

SD

0.000146

s

p50

0.000329

s

<

p95

0.000696

s

<

p99

0.000799

s

P2P request roundtrip: 'get'

Time spent sending a get request and awaiting its response.

mean

2.519115

s

SD

29.897823

s

p50

0.256354

s

<

p95

60.001612

s

<

p99

60.001674

s

P2P request roundtrip: 'get_links'

Time spent sending a get_links request and awaiting its response.

mean

0.116503

s

SD

23.03247

s

p50

0.281244

s

<

p95

60.001857

s

<

p99

60.001869

s

P2P request roundtrip: 'must_get_agent_activity'

Time spent sending a must_get_agent_activity request and awaiting its response.

mean

0.102985

s

SD

28.778894

s

p50

0.325381

s

<

p95

60.001649

s

<

p99

60.001964

s

P2P request roundtrip: 'send_validation_receipts'

Time spent sending a send_validation_receipts request and awaiting its response.

mean

0.210139

s

SD

7.047292

s

p50

0.235877

s

<

p95

15.837731

s

<

p99

17.075596

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

18.00

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

1.80

KiB/s

<

mean

119.94

KiB/s

SD

499.94

KiB/s

<

p95

175.13

KiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

1.35

KiB/s

<

mean

66.14

KiB/s

SD

70.66

KiB/s

<

p95

175.75

KiB/s

Total bytes received

Total bytes received on primary network interface

count

2.54

GiB

mean

2.92

MiB/s

Total bytes sent

Total bytes sent on primary network interface

count

1.40

GiB

mean

1.61

MiB/s

CPU spike anomaly

CPU spike anomaly was detected

Detected

Warning CPU p99 reached 97.6%

Memory leak anomaly

Memory leak anomaly was detected

NotDetected

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

Detected

Warning Moderate swap usage (13.9% swap used)

System overload anomaly

System overload anomaly was detected

Detected

Warning 19% of hosts overloaded (load5/ncpus > 1.0)

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

1.96

%

<

mean

27.77

%

SD

25.83

%

<

p95

84.21

%

System

9.77

%

CPU percentiles

CPU usage percentiles

p50

17.47

%

p95

84.21

%

p99

97.64

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

8

hosts

mean time

175.70

s

Memory used percentage

Percentage of memory used

p5

6.92

%

<

mean

18.19

%

SD

8.34

%

<

p95

35.59

%

Memory available percentage

Percentage of available memory

p5

64.41

%

<

mean

81.81

%

SD

8.34

%

<

p95

93.08

%

Max host memory used

Maximum memory usage percentage across all hosts

max

39.29

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

13.87

%

Memory growth rate

Rate of memory growth over time

growth

0.52

MiB/s

Disk read throughput

Disk read throughput in MiB/s

899.71

MiB/s

Disk write throughput

Disk write throughput in MiB/s

57.80

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /

0/2

hosts

Mount Point /efi-boot

0/2

hosts

Mount Point /etc/hostname

0/12

hosts

Mount Point /etc/hosts

0/12

hosts

Mount Point /etc/nomad.d/client-meta.json

0/12

hosts

Mount Point /etc/resolv.conf

0/12

hosts

Mount Point /nix/store

0/2

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

1.54

5 min

1.99

15 min

2.13

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

19.23

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

0

%

<

mean

6.0907

%

SD

12.6757

%

<

p95

39.13

%

60 second average

7.0504

%

300 second average

12.0148

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

<

mean

0.2847

%

SD

1.7051

%

≤

p95

0

%

60 second average

0.2671

%

300 second average

0.1794

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

<

mean

0.2744

%

SD

1.6416

%

≤

p95

0

%

60 second average

0.2594

%

300 second average

0.1729

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

2.3583

%

SD

8.3246

%

<

p95

17.72

%

60 second average

2.3711

%

300 second average

2.2227

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

1.652

%

SD

6.2142

%

<

p95

11.92

%

60 second average

1.6383

%

300 second average

1.3605

%

Holochain process CPU usage

CPU usage by Holochain process

p5

0.7

%

<

mean

20.3

%

SD

24.5

%

<

p95

69.64

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

323.48

KiB

<

mean

424.32

KiB

SD

88.07

KiB

<

p95

595.70

KiB

Holochain process threads

Number of threads in Holochain process

p5

25

threads

<

mean

32.67

threads

SD

7.63

threads

<

p95

52

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

62

file descriptors

<

mean

73.53

file descriptors

SD

12.5

file descriptors

<

p95

95

file descriptors

Remote Call Rate

↑ Back to index

Test the throughput of remote_call operations. Each agent in this scenario waits for a certain number of peers to be available or for up to two minutes, whichever happens first, before starting its behaviour.

Started

Thu, 17 Sep 2026 09:03:25 UTC

Duration

900s

Peer count

10

Peer count at end

9

Behaviours

default (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

remote_call_rate_35182846570_1

Dispatch timing

The time between sending a remote call and the remote handler being invoked.

Mean

mean

0.097881

s

SD

0.06851

s

Max

Highest per-partition mean latency

max mean

0.114697

s

Min

Lowest per-partition mean latency

min mean

0.079233

s

Round-trip timing

The total elapsed time to get a response to the client.

Mean

mean

0.196094

s

SD

0.086619

s

Max

Highest per-partition mean latency

max mean

0.233884

s

Min

Lowest per-partition mean latency

min mean

0.160063

s

Error count

The total number of errors accumulated during the run.

0

Holochain MetricsClick to toggle visibility.

WASM usage: 'remote_call::call_echo_timestamp'

Metered usage count of the WASM ribosome for this zome function.

total

146469906

over 886.034743008s

mean rate

1.345914744e+07

/s

std

1.46991164e+07

/s

p5

165273.18

/s

p95

4.724403649e+07

/s

peak

5.926871516e+07

/s

WASM usage: 'remote_call::echo_timestamp'

Metered usage count of the WASM ribosome for this zome function.

total

29430633

over 886.034743008s

mean rate

1.53641969e+06

/s

std

4.50877847e+06

/s

p5

14456.14

/s

p95

1.001189241e+07

/s

peak

3.162181308e+07

/s

WASM usage: 'remote_call::init'

Metered usage count of the WASM ribosome for this zome function.

total

0

over 886.034743008s

Not enough time data to show a trend.

Zome call duration: 'remote_call::call_echo_timestamp'

Duration of this zome call as measured by Holochain internally.

mean

0.191226

s

SD

0.036202

s

p50

0.197082

s

<

p95

0.237921

s

<

p99

0.34347

s

Zome call duration: 'remote_call::echo_timestamp'

Duration of this zome call as measured by Holochain internally.

mean

0.0032

s

SD

0.000774

s

p50

0.002856

s

<

p95

0.004337

s

<

p99

0.004347

s

WASM call duration: 'remote_call::call_echo_timestamp'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.190122

s

SD

0.036181

s

p50

0.195991

s

<

p95

0.236986

s

<

p99

0.342416

s

WASM call duration: 'remote_call::echo_timestamp'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.002149

s

SD

0.000581

s

p50

0.001926

s

<

p95

0.002976

s

<

p99

0.002982

s

WASM call duration: 'remote_call::init'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.00286

s

SD

0.000802

s

p50

0.003016

s

<

p95

0.004095

s

≤

p99

0.004095

s

Host function call duration: '__hc__call_1'

Duration of this host function call invoked from within WASM.

mean

0.187598

s

SD

0.036123

s

p50

0.193827

s

<

p95

0.234792

s

<

p99

0.339858

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.000549

s

SD

0.000248

s

p50

0.000517

s

<

p95

0.001197

s

≤

p99

0.001197

s

Host function call duration: '__hc__sys_time_1'

Duration of this host function call invoked from within WASM.

mean

1e-05

s

SD

4e-06

s

p50

9e-06

s

<

p95

1.6e-05

s

≤

p99

1.6e-05

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.0003

s

SD

0.000455

s

p50

0.000293

s

<

p95

0.002248

s

≤

p99

0.002248

s

Post-commit duration

Time spent executing post-commit workflows.

mean

0.000168

s

SD

7.4e-05

s

p50

0.000159

s

<

p95

0.000344

s

≤

p99

0.000344

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

46.89

s

<

mean

443.63

s

SD

255.07

s

<

p95

838.11

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

86

over 886.034683656s

mean rate

14.8

/s

std

104.07

/s

p5

0

/s

p95

24.88

/s

peak

1093.6

/s

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

1.497203

s

SD

1.1884

s

p50

1.052849

s

<

p95

3.426379

s

<

p99

4.710734

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.002449

s

SD

0.000495

s

p50

0.002521

s

<

p95

0.003061

s

≤

p99

0.003061

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.067286

s

SD

0.039348

s

p50

0.076086

s

<

p95

0.129278

s

<

p99

0.158331

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.006552

s

SD

0.001549

s

p50

0.006612

s

<

p95

0.009453

s

<

p99

0.012675

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

0.196071

s

SD

0.251204

s

p50

0.205065

s

<

p95

0.70929

s

<

p99

0.882108

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

0.178644

s

SD

0.077997

s

p50

0.172486

s

<

p95

0.279166

s

<

p99

0.376337

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

0.000147

s

SD

2.8e-05

s

p50

0.000161

s

<

p95

0.000187

s

<

p99

0.000217

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.001273

s

SD

0.000505

s

p50

0.001143

s

<

p95

0.00244

s

<

p99

0.002449

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000462

s

SD

8.9e-05

s

p50

0.000451

s

<

p95

0.000587

s

<

p99

0.000588

s

P2P request roundtrip: 'get'

Time spent sending a get request and awaiting its response.

mean

0.14659

s

SD

0.055669

s

p50

0.153278

s

<

p95

0.236204

s

≤

p99

0.236204

s

P2P request roundtrip: 'send_validation_receipts'

Time spent sending a send_validation_receipts request and awaiting its response.

mean

0.190562

s

SD

0.023798

s

p50

0.189471

s

<

p95

0.214326

s

<

p99

0.21746

s

P2P request roundtrip: 'call_remote'

Time spent sending a call_remote request and awaiting its response.

mean

0.187009

s

SD

0.036122

s

p50

0.193317

s

<

p95

0.234307

s

<

p99

0.339296

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

12.81

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

8.76

KiB/s

<

mean

268.44

KiB/s

SD

1.17

MiB/s

<

p95

1.56

MiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

7.58

KiB/s

<

mean

16.09

KiB/s

SD

21.39

KiB/s

<

p95

44.59

KiB/s

Total bytes received

Total bytes received on primary network interface

count

1.81

GiB

mean

2.08

MiB/s

Total bytes sent

Total bytes sent on primary network interface

count

111.09

MiB

mean

127.82

KiB/s

CPU spike anomaly

CPU spike anomaly was detected

Detected

Warning CPU p99 reached 92.1%

Memory leak anomaly

Memory leak anomaly was detected

Detected

Warning Memory growing at 1.84 MiB/s

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

NotDetected

System overload anomaly

System overload anomaly was detected

NotDetected

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

5.33

%

<

mean

19.21

%

SD

21.24

%

<

p95

73.04

%

System

6.40

%

CPU percentiles

CPU usage percentiles

p50

9.84

%

p95

73.04

%

p99

92.14

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

5

hosts

mean time

46.98

s

Memory used percentage

Percentage of memory used

p5

7.16

%

<

mean

18.78

%

SD

10.57

%

<

p95

45.92

%

Memory available percentage

Percentage of available memory

p5

54.08

%

<

mean

81.22

%

SD

10.57

%

<

p95

92.84

%

Max host memory used

Maximum memory usage percentage across all hosts

max

44.82

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

5.29

%

Memory growth rate

Rate of memory growth over time

growth

1.84

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.03

MiB/s

Disk write throughput

Disk write throughput in MiB/s

18.91

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /etc/hostname

0/2

hosts

Mount Point /etc/hosts

0/2

hosts

Mount Point /etc/nomad.d/client-meta.json

0/7

hosts

Mount Point /etc/resolv.conf

0/2

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

0.77

5 min

0.56

15 min

0.29

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

0.00

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

0.12

%

<

mean

6.1759

%

SD

9.8063

%

<

p95

25.23

%

60 second average

6.0418

%

300 second average

4.4401

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0002

%

300 second average

0.0026

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0001

%

300 second average

0.0022

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

0.1046

%

SD

0.3474

%

<

p95

0.65

%

60 second average

0.1028

%

300 second average

0.0962

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

0.0757

%

SD

0.2569

%

<

p95

0.48

%

60 second average

0.0746

%

300 second average

0.0715

%

Holochain process CPU usage

CPU usage by Holochain process

p5

0.88

%

<

mean

8.69

%

SD

13.5

%

<

p95

38.28

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

195.75

KiB

<

mean

223.84

KiB

SD

75.57

KiB

<

p95

252.55

KiB

Holochain process threads

Number of threads in Holochain process

p5

23

threads

<

mean

28.86

threads

SD

13

threads

<

p95

32

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

61

file descriptors

<

mean

73.09

file descriptors

SD

28.04

file descriptors

<

p95

79

file descriptors

Remote Signals

↑ Back to index

This scenario tests the throughput of remote_signals operations.

Started

Thu, 17 Sep 2026 09:05:11 UTC

Duration

900s

Peer count

10

Peer count at end

10

Behaviours

default (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

remote_signals_35182846570_1

Round trip time

The time from origin signal dispatch to origin receive of the remote side's response signal.

mean

0.202524

s

SD

0.486917

s

p50

0.176151

s

<

p95

0.258595

s

<

p99

0.446838

s

Timeout rate

The proportion of remote signal round-trips that timed out.

0

Holochain MetricsClick to toggle visibility.

WASM usage: 'remote_signal::init'

Metered usage count of the WASM ribosome for this zome function.

total

0

over 880.628409804s

Not enough time data to show a trend.

WASM usage: 'remote_signal::recv_remote_signal'

Metered usage count of the WASM ribosome for this zome function.

total

31628419

over 880.628409804s

mean rate

1.172244186e+08

/s

std

2.1908000522e+09

/s

p5

0

/s

p95

9.153812471e+07

/s

peak

5.18946135831e+10

/s

WASM usage: 'remote_signal::signal_request'

Metered usage count of the WASM ribosome for this zome function.

total

13560480

over 880.628409804s

mean rate

595416.83

/s

std

3.81424606e+06

/s

p5

0

/s

p95

4.64273439e+06

/s

peak

7.961693351e+07

/s

Zome call duration: 'remote_signal::recv_remote_signal'

Duration of this zome call as measured by Holochain internally.

mean

0.003102

s

SD

0.001034

s

p50

0.003002

s

<

p95

0.004786

s

<

p99

0.005391

s

Zome call duration: 'remote_signal::signal_request'

Duration of this zome call as measured by Holochain internally.

mean

0.003016

s

SD

0.000873

s

p50

0.002889

s

<

p95

0.004491

s

<

p99

0.004513

s

WASM call duration: 'remote_signal::init'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.002875

s

SD

0.002298

s

p50

0.002199

s

<

p95

0.009302

s

≤

p99

0.009302

s

WASM call duration: 'remote_signal::recv_remote_signal'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.002107

s

SD

0.000818

s

p50

0.002146

s

<

p95

0.003373

s

<

p99

0.003962

s

WASM call duration: 'remote_signal::signal_request'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.002017

s

SD

0.000669

s

p50

0.00202

s

<

p95

0.003121

s

<

p99

0.003142

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.000457

s

SD

0.000213

s

p50

0.000413

s

<

p95

0.000846

s

≤

p99

0.000846

s

Host function call duration: '__hc__emit_signal_1'

Duration of this host function call invoked from within WASM.

mean

8e-05

s

SD

3e-05

s

p50

8.6e-05

s

<

p95

0.00012

s

<

p99

0.000146

s

Host function call duration: '__hc__send_remote_signal_1'

Duration of this host function call invoked from within WASM.

mean

0.000101

s

SD

5.8e-05

s

p50

0.000101

s

<

p95

0.000195

s

<

p99

0.000271

s

Host function call duration: '__hc__sys_time_1'

Duration of this host function call invoked from within WASM.

mean

5e-06

s

SD

1e-06

s

p50

5e-06

s

<

p95

7e-06

s

<

p99

8e-06

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.000462

s

SD

0.00053

s

p50

0.000199

s

<

p95

0.001692

s

≤

p99

0.001692

s

Emitted local signals

Count of local signals emitted via the emit_signal host function.

total

1744

over 880.00761209s

mean rate

1106.69

/s

std

2532.8

/s

p5

0

/s

p95

6458.26

/s

peak

19710.36

/s

Sent remote signals

Count of remote signals sent via the send_remote_signal host function.

total

1743

over 880.628382309s

mean rate

487.8

/s

std

1745.9

/s

p5

0

/s

p95

3030.77

/s

peak

19727.95

/s

Post-commit duration

Time spent executing post-commit workflows.

mean

0.000128

s

SD

5.4e-05

s

p50

0.000126

s

<

p95

0.000214

s

≤

p99

0.000214

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

46.3

s

<

mean

445.04

s

SD

256.35

s

<

p95

846.15

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

115

over 886.608383458s

mean rate

1891.16

/s

std

9381.46

/s

p5

0

/s

p95

13036.35

/s

peak

117994.1

/s

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

1.435916

s

SD

0.951188

s

p50

0.883317

s

<

p95

2.753989

s

<

p99

3.613092

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.002812

s

SD

0.001129

s

p50

0.00296

s

<

p95

0.00457

s

<

p99

0.006756

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.073922

s

SD

0.030521

s

p50

0.070587

s

<

p95

0.133207

s

<

p99

0.159106

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.006952

s

SD

0.004952

s

p50

0.00537

s

<

p95

0.017646

s

<

p99

0.024701

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

0.1747

s

SD

0.245395

s

p50

0.044634

s

<

p95

0.515756

s

<

p99

1.040409

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

0.271466

s

SD

0.177862

s

p50

0.255754

s

<

p95

0.580953

s

<

p99

0.895083

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

0.000164

s

SD

5.3e-05

s

p50

0.000177

s

<

p95

0.000283

s

<

p99

0.000349

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.001278

s

SD

0.002004

s

p50

0.001342

s

<

p95

0.01005

s

≤

p99

0.01005

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000568

s

SD

0.000175

s

p50

0.000584

s

<

p95

0.000797

s

<

p99

0.001112

s

P2P request roundtrip: 'get'

Time spent sending a get request and awaiting its response.

mean

0.10733

s

SD

0.040642

s

p50

0.101456

s

<

p95

0.150835

s

<

p99

0.179117

s

P2P request roundtrip: 'send_validation_receipts'

Time spent sending a send_validation_receipts request and awaiting its response.

mean

0.163189

s

SD

0.024483

s

p50

0.166209

s

<

p95

0.184933

s

<

p99

0.193529

s

P2P received remote signals

Count of remote signals received.

total

1744

over 880.628427613s

mean rate

593.91

/s

std

2413.3

/s

p5

0

/s

p95

3396.28

/s

peak

29182.6

/s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

22.78

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

14.55

KiB/s

<

mean

519.62

KiB/s

SD

1.56

MiB/s

<

p95

2.55

MiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

12.89

KiB/s

<

mean

53.58

KiB/s

SD

133.90

KiB/s

<

p95

270.26

KiB/s

Total bytes received

Total bytes received on primary network interface

count

1.76

GiB

mean

2.02

MiB/s

Total bytes sent

Total bytes sent on primary network interface

count

185.74

MiB

mean

213.71

KiB/s

CPU spike anomaly

CPU spike anomaly was detected

Detected

Warning CPU p99 reached 97.3%

Memory leak anomaly

Memory leak anomaly was detected

NotDetected

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

NotDetected

System overload anomaly

System overload anomaly was detected

NotDetected

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

7.04

%

<

mean

32.64

%

SD

27.96

%

<

p95

89.65

%

System

9.86

%

CPU percentiles

CPU usage percentiles

p50

21.62

%

p95

89.65

%

p99

97.29

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

5

hosts

mean time

91.13

s

Memory used percentage

Percentage of memory used

p5

6.92

%

<

mean

23.76

%

SD

12.35

%

<

p95

46.8

%

Memory available percentage

Percentage of available memory

p5

53.2

%

<

mean

76.24

%

SD

12.35

%

<

p95

93.08

%

Max host memory used

Maximum memory usage percentage across all hosts

max

45.36

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

5.29

%

Memory growth rate

Rate of memory growth over time

growth

0.64

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.03

MiB/s

Disk write throughput

Disk write throughput in MiB/s

15.83

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /etc/hostname

0/2

hosts

Mount Point /etc/hosts

0/2

hosts

Mount Point /etc/nomad.d/client-meta.json

0/3

hosts

Mount Point /etc/resolv.conf

0/2

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

1.35

5 min

0.99

15 min

0.52

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

0.00

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

0.09

%

<

mean

12.9622

%

SD

17.3186

%

<

p95

54.97

%

60 second average

11.3901

%

300 second average

8.2898

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0024

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0020

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

0.1253

%

SD

0.3778

%

<

p95

0.8

%

60 second average

0.1075

%

300 second average

0.1026

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

0.0555

%

SD

0.1848

%

<

p95

0.38

%

60 second average

0.0530

%

300 second average

0.0602

%

Holochain process CPU usage

CPU usage by Holochain process

p5

0.83

%

<

mean

15.31

%

SD

23.94

%

<

p95

69.55

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

195.69

KiB

<

mean

228.71

KiB

SD

100.65

KiB

<

p95

269.35

KiB

Holochain process threads

Number of threads in Holochain process

p5

23

threads

<

mean

29.57

threads

SD

17.1

threads

<

p95

34

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

59

file descriptors

<

mean

72.84

file descriptors

SD

37.01

file descriptors

<

p95

84

file descriptors

Single-write, many-read

↑ Back to index

Creates an entry in the agent setup, then read one's own record back. This tests the maximum read performance for reading back local data when the chain is short.

Started

Thu, 17 Sep 2026 09:06:57 UTC

Duration

300s

Peer count

1

Peer count at end

1

Behaviours

default (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

single_write_many_read_35182846570_1

read call timing

The time it takes to complete the zome call that retrieves the record.

mean

0.002915

s

SD

0.00155

s

p50

0.002593

s

<

p95

0.004364

s

<

p99

0.006339

s

read call rate

The number of read zome calls completed per second.

mean

3449.36

/s

Error count

The number of errors collected.

0

Holochain MetricsClick to toggle visibility.

Cascade duration

Time taken to execute a cascade (get) query inside Holochain.

mean

0.000194

s

SD

1.1e-05

s

p50

0.000188

s

<

p95

0.000209

s

<

p99

0.000211

s

WASM usage: 'crud::create_sample_entry'

Metered usage count of the WASM ribosome for this zome function.

total

0

over 270.001557933s

Not enough time data to show a trend.

WASM usage: 'crud::get_sample_entry'

Metered usage count of the WASM ribosome for this zome function.

total

3141685008

over 270.001557933s

mean rate

1.163580314e+07

/s

std

2.2280068e+06

/s

p5

8.34920692e+06

/s

p95

1.42901727e+07

/s

peak

1.51017691e+07

/s

WASM usage: 'crud::init'

Metered usage count of the WASM ribosome for this zome function.

total

0

over 270.001557933s

Not enough time data to show a trend.

WASM usage: 'crud_integrity::entry_defs'

Metered usage count of the WASM ribosome for this zome function.

total

0

over 270.001557933s

Not enough time data to show a trend.

Zome call duration: 'crud::create_sample_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.003722

s

SD

0

s

p50

0.003722

s

≤

p95

0.003722

s

≤

p99

0.003722

s

Zome call duration: 'crud::get_sample_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.001291

s

SD

5.1e-05

s

p50

0.001265

s

<

p95

0.00136

s

<

p99

0.00137

s

WASM call duration: 'crud::create_sample_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.00304

s

SD

0

s

p50

0.00304

s

≤

p95

0.00304

s

≤

p99

0.00304

s

WASM call duration: 'crud::get_sample_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.000892

s

SD

3.6e-05

s

p50

0.000873

s

<

p95

0.000941

s

<

p99

0.000949

s

WASM call duration: 'crud::init'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.000856

s

SD

0

s

p50

0.000856

s

≤

p95

0.000856

s

≤

p99

0.000856

s

WASM call duration: 'crud_integrity::entry_defs'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.000758

s

SD

0

s

p50

0.000758

s

≤

p95

0.000758

s

≤

p99

0.000758

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.000289

s

SD

6e-05

s

p50

0.000349

s

≤

p95

0.000349

s

≤

p99

0.000349

s

Host function call duration: '__hc__get_1'

Duration of this host function call invoked from within WASM.

mean

0.000215

s

SD

1.2e-05

s

p50

0.000209

s

<

p95

0.000231

s

<

p99

0.000234

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.001569

s

SD

0

s

p50

0.001569

s

≤

p95

0.001569

s

≤

p99

0.001569

s

Post-commit duration

Time spent executing post-commit workflows.

mean

6.2e-05

s

SD

0

s

p50

6.2e-05

s

≤

p95

6.2e-05

s

≤

p99

6.2e-05

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

18.09

s

<

mean

143.09

s

SD

80.78

s

<

p95

268.09

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

0

over 270.001562253s

Not enough time data to show a trend.

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.001489

s

SD

0

s

p50

0.001489

s

≤

p95

0.001489

s

≤

p99

0.001489

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.000681

s

SD

0

s

p50

0.000681

s

≤

p95

0.000681

s

≤

p99

0.000681

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.002462

s

SD

0.000418

s

p50

0.002539

s

<

p95

0.003248

s

≤

p99

0.003248

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

0.002183

s

SD

0

s

p50

0.002183

s

≤

p95

0.002183

s

≤

p99

0.002183

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

0.002166

s

SD

0

s

p50

0.002166

s

≤

p95

0.002166

s

≤

p99

0.002166

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

3.8e-05

s

SD

2e-06

s

p50

3.6e-05

s

<

p95

4.1e-05

s

≤

p99

4.1e-05

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.00061

s

SD

8.8e-05

s

p50

0.000739

s

≤

p95

0.000739

s

≤

p99

0.000739

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000185

s

SD

0

s

p50

0.000185

s

≤

p95

0.000185

s

≤

p99

0.000185

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

30.75

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

17.06

KiB/s

<

mean

495.51

KiB/s

SD

1.62

MiB/s

<

p95

2.36

MiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

14.66

KiB/s

<

mean

24.62

KiB/s

SD

31.00

KiB/s

<

p95

57.33

KiB/s

Total bytes received

Total bytes received on primary network interface

count

135.49

MiB

mean

495.51

KiB/s

Total bytes sent

Total bytes sent on primary network interface

count

6.73

MiB

mean

24.62

KiB/s

CPU spike anomaly

CPU spike anomaly was detected

NotDetected

Memory leak anomaly

Memory leak anomaly was detected

Detected

Warning Memory growing at 1.14 MiB/s

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

NotDetected

System overload anomaly

System overload anomaly was detected

NotDetected

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

33.64

%

<

mean

45.37

%

SD

16.53

%

<

p95

70.69

%

System

14.62

%

CPU percentiles

CPU usage percentiles

p50

34.09

%

p95

70.69

%

p99

78.01

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

0

hosts

mean time

0.00

s

Memory used percentage

Percentage of memory used

p5

6.92

%

<

mean

7.37

%

SD

0.48

%

<

p95

8.14

%

Memory available percentage

Percentage of available memory

p5

91.86

%

<

mean

92.63

%

SD

0.48

%

<

p95

93.08

%

Max host memory used

Maximum memory usage percentage across all hosts

max

7.37

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

0.00

%

Memory growth rate

Rate of memory growth over time

growth

1.14

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.01

MiB/s

Disk write throughput

Disk write throughput in MiB/s

3.91

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /etc/nomad.d/client-meta.json

0/1

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

2.06

5 min

1.07

15 min

0.44

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

0.00

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

2.03

%

<

mean

7.5934

%

SD

8.4016

%

<

p95

21.55

%

60 second average

5.2579

%

300 second average

2.6103

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

Holochain process CPU usage

CPU usage by Holochain process

p5

27.38

%

<

mean

31.27

%

SD

6.04

%

<

p95

42.52

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

221.96

KiB

<

mean

225.78

KiB

SD

27.07

KiB

<

p95

237.24

KiB

Holochain process threads

Number of threads in Holochain process

p5

25

threads

<

mean

25.66

threads

SD

3.66

threads

<

p95

28

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

59

file descriptors

<

mean

60.52

file descriptors

SD

7.74

file descriptors

<

p95

66

file descriptors

Validation Receipts

↑ Back to index

Creates an entry, wait for required validation receipts, then repeat. Records the amount of time it took to accumulate the required number of receipts for all DHT operations. This is measured to the nearest 20ms so that we don't keep the agent too busy checking for receipts.

Each agent in this scenario waits for a certain number of peers to be available or for up to two minutes, whichever happens first, before starting its behaviour.

By default, this scenario will wait for a complete set of validation receipts before committing the next record. If the NO_VALIDATION_COMPLETE environment variable is set, it will instead publish new records on every round, building up an ever-growing list of action hashes to check on.

Started

Thu, 17 Sep 2026 09:20:09 UTC

Duration

900s

Peer count

20

Peer count at end

20

Behaviours

default (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

validation_receipts_35182846570_1

Receipts complete timing

The amount of time between publishing a record and receiving the required number of validation receipts.

Mean

mean

6.400298

s

SD

2.390423

s

Max

Highest per-partition mean latency

max mean

7.228101

s

Min

Lowest per-partition mean latency

min mean

5.81946

s

Receipts complete rate

The number of complete validation receipt sets collected per second.

Mean

mean

1.45

/s

Max

Highest per-partition mean rate

max

1.62

/s

Min

Lowest per-partition mean rate

min

1.31

/s

Error count

The total number of errors accumulated during the run.

0

Holochain MetricsClick to toggle visibility.

WASM usage: 'crud::create_sample_entry'

Metered usage count of the WASM ribosome for this zome function.

total

4253655

over 882.725616071s

mean rate

5.24325682e+06

/s

std

1.069087852e+07

/s

p5

0

/s

p95

2.749149439e+07

/s

peak

7.905801274e+07

/s

WASM usage: 'crud::get_sample_entry_validation_receipts'

Metered usage count of the WASM ribosome for this zome function.

total

15085169

over 882.725616071s

mean rate

5.2018889e+06

/s

std

7.17914477e+06

/s

p5

31687.29

/s

p95

2.109772489e+07

/s

peak

3.006982388e+07

/s

WASM usage: 'crud::init'

Metered usage count of the WASM ribosome for this zome function.

total

0

over 882.725616071s

Not enough time data to show a trend.

WASM usage: 'crud_integrity::entry_defs'

Metered usage count of the WASM ribosome for this zome function.

total

528240

over 886.700931578s

mean rate

639038.38

/s

std

1.31083474e+06

/s

p5

0

/s

p95

3.38558003e+06

/s

peak

9.73600146e+06

/s

Zome call duration: 'crud::create_sample_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.005263

s

SD

0.001559

s

p50

0.004784

s

<

p95

0.00978

s

<

p99

0.010988

s

Zome call duration: 'crud::get_sample_entry_validation_receipts'

Duration of this zome call as measured by Holochain internally.

mean

0.015691

s

SD

0.008107

s

p50

0.010903

s

<

p95

0.024542

s

<

p99

0.047426

s

WASM call duration: 'crud::create_sample_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.00453

s

SD

0.001432

s

p50

0.004049

s

<

p95

0.008645

s

<

p99

0.009711

s

WASM call duration: 'crud::get_sample_entry_validation_receipts'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.01484

s

SD

0.007975

s

p50

0.010074

s

<

p95

0.023518

s

<

p99

0.045967

s

WASM call duration: 'crud::init'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.002281

s

SD

0.000694

s

p50

0.002205

s

<

p95

0.003743

s

≤

p99

0.003743

s

WASM call duration: 'crud_integrity::entry_defs'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.001551

s

SD

0.000563

s

p50

0.001332

s

<

p95

0.003014

s

<

p99

0.003315

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.000483

s

SD

0.000375

s

p50

0.000464

s

<

p95

0.001966

s

<

p99

0.002

s

Host function call duration: '__hc__get_validation_receipts_1'

Duration of this host function call invoked from within WASM.

mean

0.012917

s

SD

0.007524

s

p50

0.008276

s

<

p95

0.021145

s

<

p99

0.041839

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.002438

s

SD

0.000818

s

p50

0.002135

s

<

p95

0.004792

s

<

p99

0.005364

s

Post-commit duration

Time spent executing post-commit workflows.

mean

0.000182

s

SD

6.8e-05

s

p50

0.00016

s

<

p95

0.000311

s

<

p99

0.000389

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

45.88

s

<

mean

438.88

s

SD

253.45

s

<

p95

837.11

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

6248

over 886.700906767s

mean rate

412.45

/s

std

1795.71

/s

p5

1.98

/s

p95

1767.7

/s

peak

34446.11

/s

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

1.242584

s

SD

1.154169

s

p50

1.276925

s

<

p95

4.312975

s

<

p99

5.691224

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.003171

s

SD

0.009014

s

p50

0.001689

s

<

p95

0.024708

s

<

p99

0.038088

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.035728

s

SD

0.030276

s

p50

0.039516

s

<

p95

0.076257

s

<

p99

0.161293

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.033477

s

SD

0.016955

s

p50

0.022328

s

<

p95

0.052734

s

<

p99

0.091896

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

0.033103

s

SD

0.481869

s

p50

0.034991

s

<

p95

0.435798

s

<

p99

3.032231

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

3.17194

s

SD

12.65199

s

p50

4.198086

s

<

p95

40.129722

s

<

p99

48.690923

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

0.000648

s

SD

0.000302

s

p50

0.000473

s

<

p95

0.00116

s

<

p99

0.001701

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.003338

s

SD

0.005756

s

p50

0.001422

s

<

p95

0.008036

s

<

p99

0.029963

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.00029

s

SD

0.000101

s

p50

0.000254

s

<

p95

0.000534

s

<

p99

0.000606

s

P2P request roundtrip: 'get'

Time spent sending a get request and awaiting its response.

mean

0.104812

s

SD

0.095823

s

p50

0.094998

s

<

p95

0.274614

s

<

p99

0.278464

s

P2P request roundtrip: 'send_validation_receipts'

Time spent sending a send_validation_receipts request and awaiting its response.

mean

0.165947

s

SD

0.045282

s

p50

0.144177

s

<

p95

0.255694

s

<

p99

0.26512

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

69.59

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

0

B/s

<

mean

501.28

KiB/s

SD

445.42

KiB/s

<

p95

790.04

KiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

0

B/s

<

mean

410.16

KiB/s

SD

273.62

KiB/s

<

p95

913.96

KiB/s

Total bytes received

Total bytes received on primary network interface

count

7.52

GiB

mean

8.75

MiB/s

Total bytes sent

Total bytes sent on primary network interface

count

6.15

GiB

mean

7.16

MiB/s

CPU spike anomaly

CPU spike anomaly was detected

Detected

Warning CPU p99 reached 98.4%

Memory leak anomaly

Memory leak anomaly was detected

NotDetected

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

Detected

Warning Moderate swap usage (12.6% swap used)

System overload anomaly

System overload anomaly was detected

Detected

Warning 84% of hosts overloaded (load5/ncpus > 1.0)

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

11.55

%

<

mean

82.66

%

SD

22.86

%

<

p95

98.05

%

System

13.07

%

CPU percentiles

CPU usage percentiles

p50

93.15

%

p95

98.05

%

p99

98.40

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

16

hosts

mean time

792.65

s

Memory used percentage

Percentage of memory used

p5

6.4

%

<

mean

25.25

%

SD

8.54

%

<

p95

39.36

%

Memory available percentage

Percentage of available memory

p5

60.65

%

<

mean

74.75

%

SD

8.54

%

<

p95

93.64

%

Max host memory used

Maximum memory usage percentage across all hosts

max

40.87

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

12.61

%

Memory growth rate

Rate of memory growth over time

growth

0.38

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.25

MiB/s

Disk write throughput

Disk write throughput in MiB/s

111.46

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /

0/2

hosts

Mount Point /efi-boot

0/2

hosts

Mount Point /etc/hostname

0/9

hosts

Mount Point /etc/hosts

0/9

hosts

Mount Point /etc/nomad.d/client-meta.json

0/8

hosts

Mount Point /etc/resolv.conf

0/9

hosts

Mount Point /nix/store

0/2

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

5.74

5 min

4.49

15 min

2.60

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

84.21

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

0.45

%

<

mean

46.935

%

SD

24.2952

%

<

p95

78.27

%

60 second average

45.9481

%

300 second average

35.9205

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

<

mean

0.0024

%

SD

0.0527

%

≤

p95

0

%

60 second average

0.0018

%

300 second average

0.0009

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

<

mean

0.0006

%

SD

0.0143

%

≤

p95

0

%

60 second average

0.0004

%

300 second average

0.0001

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

6.2696

%

SD

13.2648

%

<

p95

43.11

%

60 second average

6.1816

%

300 second average

4.8136

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

2.3171

%

SD

6.5029

%

<

p95

21.54

%

60 second average

2.3282

%

300 second average

1.8970

%

Holochain process CPU usage

CPU usage by Holochain process

p5

1.25

%

<

mean

5.73

%

SD

5.95

%

<

p95

11.89

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

221.19

KiB

<

mean

250.12

KiB

SD

41.99

KiB

<

p95

299.59

KiB

Holochain process threads

Number of threads in Holochain process

p5

30

threads

<

mean

32.2

threads

SD

5.76

threads

<

p95

44

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

71

file descriptors

<

mean

76.53

file descriptors

SD

11.7

file descriptors

<

p95

83

file descriptors

Write/`get_agent_activity`

↑ Back to index

A scenario where write peers write entries, while get_agent_activity peers each query a single write agent's activity with get_agent_activity.

Before a target write peer and the requesting get_agent_activity peer are in sync, this will measure the get_agent_activity call performance over a network. Once a write peer reaches sync with a get_agent_activity peer, the write peer will publish their actions and entries, and so the get_agent_activity calls will likely have most of the data they need locally. At that point this measures the database query performance and code paths through host functions.

Started

Thu, 17 Sep 2026 09:49:12 UTC

Duration

900s

Peer count

30

Peer count at end

29

Behaviours

get_agent_activity (1 agent)
write (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

write_get_agent_activity_35182846570_1

Observed chain advancement

The maximum highest-observed action sequence per write agent, aggregated across all reading agents. The reading dimension is collapsed by taking the maximum observed value for each write agent (any reader successfully tracking that point counts as propagation). Captures how far readers tracked writers’ chains during the run.

Mean max (across write agents)

Mean of the highest chain head value observed per write agent. Each write agent contributes its maximum — the value seen by any reader.

3191.75

Max

Highest chain head value observed for any single write agent.

3871

Write agents observed

4

get_agent_activity_full zome call timing

The time taken to call the zome function that retrieves information on a write peer's source chain.

Mean

mean

0.304052

s

SD

0.439952

s

Max

Highest per-partition mean latency

max mean

0.8618

s

Min

Lowest per-partition mean latency

min mean

0.08221

s

Error count

The number of errors accumulated.

20

Holochain MetricsClick to toggle visibility.

Cascade duration

Time taken to execute a cascade (get) query inside Holochain.

mean

0.044929

s

SD

0.445986

s

p50

0.045421

s

<

p95

0.848101

s

<

p99

2.247026

s

Cascade fetch errors

Count of network fetch errors during cascade calls.

total

0

over 760.015527461s

Not enough time data to show a trend.

WASM usage: 'agent_activity::announce_write_behaviour'

Metered usage count of the WASM ribosome for this zome function.

total

0

over 887.004632432s

Not enough time data to show a trend.

WASM usage: 'agent_activity::create_sample_entry'

Metered usage count of the WASM ribosome for this zome function.

total

173650872

over 887.004632432s

mean rate

4.523274823e+07

/s

std

3.485209016e+07

/s

p5

2.84983183e+06

/s

p95

1.1692958924e+08

/s

peak

1.4663285992e+08

/s

WASM usage: 'agent_activity::get_agent_activity_full'

Metered usage count of the WASM ribosome for this zome function.

total

5731345956

over 881.942885882s

mean rate

7.29686499554e+09

/s

std

1.50974816272e+10

/s

p5

2.342687115e+07

/s

p95

2.83898132387e+10

/s

peak

1.98033735496e+11

/s

WASM usage: 'agent_activity::get_random_agent_with_write_behaviour'

Metered usage count of the WASM ribosome for this zome function.

total

595431120

over 887.450952941s

mean rate

1.53942178512e+09

/s

std

2.87001470864e+09

/s

p5

0

/s

p95

8.14364947217e+09

/s

peak

8.15811587671e+09

/s

WASM usage: 'agent_activity_integrity::entry_defs'

Metered usage count of the WASM ribosome for this zome function.

total

42277470

over 887.450952941s

mean rate

1.5253765035e+08

/s

std

7.0606564174e+08

/s

p5

0

/s

p95

5.776090814e+08

/s

peak

2.09545181893e+10

/s

Zome call duration: 'agent_activity::announce_write_behaviour'

Duration of this zome call as measured by Holochain internally.

mean

0.00668

s

SD

0.002839

s

p50

0.006171

s

<

p95

0.011213

s

≤

p99

0.011213

s

Zome call duration: 'agent_activity::create_sample_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.004962

s

SD

0.001561

s

p50

0.004605

s

<

p95

0.007676

s

<

p99

0.007934

s

Zome call duration: 'agent_activity::get_agent_activity_full'

Duration of this zome call as measured by Holochain internally.

mean

0.092356

s

SD

0.344512

s

p50

0.089462

s

<

p95

0.805149

s

<

p99

1.712119

s

Zome call duration: 'agent_activity::get_random_agent_with_write_behaviour'

Duration of this zome call as measured by Holochain internally.

mean

0.003885

s

SD

0.519815

s

p50

0.005755

s

<

p95

2.253131

s

≤

p99

2.253131

s

WASM call duration: 'agent_activity::announce_write_behaviour'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.005659

s

SD

0.002416

s

p50

0.005268

s

<

p95

0.009409

s

≤

p99

0.009409

s

WASM call duration: 'agent_activity::create_sample_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.004233

s

SD

0.001372

s

p50

0.003848

s

<

p95

0.006554

s

<

p99

0.006768

s

WASM call duration: 'agent_activity::get_agent_activity_full'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.091643

s

SD

0.344477

s

p50

0.08886

s

<

p95

0.804531

s

<

p99

1.711161

s

WASM call duration: 'agent_activity::get_random_agent_with_write_behaviour'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.003231

s

SD

0.519704

s

p50

0.00488

s

<

p95

2.251769

s

≤

p99

2.251769

s

WASM call duration: 'agent_activity_integrity::entry_defs'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.001189

s

SD

0.000572

s

p50

0.001118

s

<

p95

0.002285

s

<

p99

0.002892

s

Host function call duration: '__hc__agent_info_1'

Duration of this host function call invoked from within WASM.

mean

4.1e-05

s

SD

2.7e-05

s

p50

3.5e-05

s

<

p95

0.000101

s

≤

p99

0.000101

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.000553

s

SD

0.000238

s

p50

0.000564

s

<

p95

0.001021

s

<

p99

0.001155

s

Host function call duration: '__hc__create_link_1'

Duration of this host function call invoked from within WASM.

mean

0.000509

s

SD

0.000176

s

p50

0.000543

s

<

p95

0.00072

s

≤

p99

0.00072

s

Host function call duration: '__hc__get_agent_activity_1'

Duration of this host function call invoked from within WASM.

mean

0.089897

s

SD

0.344343

s

p50

0.087304

s

<

p95

0.803267

s

<

p99

1.709574

s

Host function call duration: '__hc__get_links_1'

Duration of this host function call invoked from within WASM.

mean

0.000379

s

SD

0.519633

s

p50

0.00042

s

<

p95

2.24708

s

≤

p99

2.24708

s

Host function call duration: '__hc__random_bytes_1'

Duration of this host function call invoked from within WASM.

mean

8e-06

s

SD

7e-06

s

p50

9e-06

s

<

p95

3.5e-05

s

≤

p99

3.5e-05

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.001843

s

SD

0.001055

s

p50

0.001855

s

<

p95

0.004325

s

<

p99

0.004767

s

Post-commit duration

Time spent executing post-commit workflows.

mean

0.000154

s

SD

7.4e-05

s

p50

0.000118

s

<

p95

0.000269

s

<

p99

0.00034

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

46.95

s

<

mean

441.59

s

SD

253.83

s

<

p95

837.66

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

30810

over 887.451074154s

mean rate

179725.52

/s

std

1.4291287e+06

/s

p5

468.01

/s

p95

527348.4

/s

peak

3.828616807e+07

/s

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

190.061865

s

SD

138.077756

s

p50

49.342974

s

<

p95

366.276612

s

<

p99

443.961042

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.023685

s

SD

0.152716

s

p50

0.110988

s

<

p95

0.476676

s

<

p99

0.596579

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.385268

s

SD

0.512144

s

p50

0.806884

s

<

p95

1.42888

s

<

p99

3.000946

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.158342

s

SD

0.078095

s

p50

0.057206

s

<

p95

0.240434

s

<

p99

0.35659

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

3.674889

s

SD

28.262652

s

p50

17.111053

s

<

p95

82.168756

s

<

p99

96.420264

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

20.963461

s

SD

37.977841

s

p50

3.06522

s

<

p95

120.991104

s

<

p99

170.540101

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

0.003146

s

SD

0.001446

s

p50

0.00263

s

<

p95

0.005176

s

<

p99

0.006218

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.002491

s

SD

0.003397

s

p50

0.001353

s

<

p95

0.005782

s

<

p99

0.021252

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000373

s

SD

0.000215

s

p50

0.000298

s

<

p95

0.000921

s

<

p99

0.000974

s

P2P request roundtrip: 'get'

Time spent sending a get request and awaiting its response.

mean

2.759156

s

SD

28.80136

s

p50

1.075024

s

<

p95

60.001448

s

<

p99

60.001506

s

P2P request roundtrip: 'get_links'

Time spent sending a get_links request and awaiting its response.

mean

0.560448

s

SD

0.393292

s

p50

0.401441

s

<

p95

1.510839

s

≤

p99

1.510839

s

P2P request roundtrip: 'get_agent_activity'

Time spent sending a get_agent_activity request and awaiting its response.

mean

0.080571

s

SD

28.733699

s

p50

0.349482

s

<

p95

60.001605

s

<

p99

60.002506

s

P2P request roundtrip: 'send_validation_receipts'

Time spent sending a send_validation_receipts request and awaiting its response.

mean

0.202402

s

SD

16.629596

s

p50

1.279614

s

<

p95

60.000591

s

<

p99

60.000809

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

53.94

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

74.49

KiB/s

<

mean

343.34

KiB/s

SD

554.61

KiB/s

<

p95

609.50

KiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

46.25

KiB/s

<

mean

283.23

KiB/s

SD

207.21

KiB/s

<

p95

659.87

KiB/s

Total bytes received

Total bytes received on primary network interface

count

7.23

GiB

mean

8.31

MiB/s

Total bytes sent

Total bytes sent on primary network interface

count

5.96

GiB

mean

6.86

MiB/s

CPU spike anomaly

CPU spike anomaly was detected

Detected

Warning CPU p99 reached 98.7%

Memory leak anomaly

Memory leak anomaly was detected

NotDetected

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

Detected

Warning Moderate swap usage (13.9% swap used)

System overload anomaly

System overload anomaly was detected

Detected

Warning 58% of hosts overloaded (load5/ncpus > 1.0)

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

21.6

%

<

mean

66.34

%

SD

25.33

%

<

p95

98

%

System

12.40

%

CPU percentiles

CPU usage percentiles

p50

73.98

%

p95

98.00

%

p99

98.70

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

19

hosts

mean time

528.61

s

Memory used percentage

Percentage of memory used

p5

6.75

%

<

mean

18.21

%

SD

9.1

%

<

p95

34.44

%

Memory available percentage

Percentage of available memory

p5

65.56

%

<

mean

81.79

%

SD

9.1

%

<

p95

93.25

%

Max host memory used

Maximum memory usage percentage across all hosts

max

45.39

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

13.87

%

Memory growth rate

Rate of memory growth over time

growth

0.32

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.28

MiB/s

Disk write throughput

Disk write throughput in MiB/s

119.51

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /

0/2

hosts

Mount Point /efi-boot

0/2

hosts

Mount Point /etc/hostname

0/12

hosts

Mount Point /etc/hosts

0/12

hosts

Mount Point /etc/nomad.d/client-meta.json

0/12

hosts

Mount Point /etc/resolv.conf

0/12

hosts

Mount Point /nix/store

0/2

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

4.22

5 min

4.34

15 min

4.29

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

57.69

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

0

%

<

mean

28.0255

%

SD

26.2255

%

<

p95

79.72

%

60 second average

27.9393

%

300 second average

29.7074

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

3.7135

%

SD

8.6352

%

<

p95

27.54

%

60 second average

3.7720

%

300 second average

3.6466

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

1.9578

%

SD

5.9386

%

<

p95

16.54

%

60 second average

2.0271

%

300 second average

1.8291

%

Holochain process CPU usage

CPU usage by Holochain process

p5

20.21

%

<

mean

59.48

%

SD

24.11

%

<

p95

96.55

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

253.33

KiB

<

mean

343.17

KiB

SD

61.41

KiB

<

p95

446.78

KiB

Holochain process threads

Number of threads in Holochain process

p5

29

threads

<

mean

34.34

threads

SD

6.54

threads

<

p95

50

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

71

file descriptors

<

mean

78.69

file descriptors

SD

10.19

file descriptors

<

p95

95

file descriptors

Write/`get_agent_activity` with volatile conductors

↑ Back to index

A scenario where write peers write entries, while get_agent_activity_volatile peers each query a single write agents activity with get_agent_activity but shutdown and restart their conductors at semi-random intervals.

Before a target write peer and the requesting get_agent_activity_volatile peer are in sync, this will measure the get_agent_activity call performance over a network. Once a write peer reaches sync with a get_agent_activity peer, the write peer will publish their actions and entries, and so the get_agent_activity calls will likely have most of the data they need locally. At that point this measures the database query performance and code paths through host functions.

Started

Thu, 17 Sep 2026 10:04:40 UTC

Duration

1800s

Peer count

30

Peer count at end

27

Behaviours

get_agent_activity_volatile (1 agent)
write (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

write_get_agent_activity_volatile_35182846570_1

Highest observed action_seq

The rate at which zero-arc readers observe new chain heads on the writers' chains via get_agent_activity. This reflects the DHT's ability to propagate agent activity ops and make them available to querying peers.

Mean

mean

53.92

/s

Max

Highest per-partition mean rate

max

360.47

/s

Min

Lowest per-partition mean rate

min

9.58

/s

get_agent_activity_full zome call timing

The time taken to call the zome function that retrieves information on a write peer's source chain.

Mean

mean

0.291935

s

SD

0.686939

s

Max

Highest per-partition mean latency

max mean

0.741345

s

Min

Lowest per-partition mean latency

min mean

0.018193

s

Running volatile conductors count

The number of conductors running by get_agent_activity_volatile peers.

p5

9

conductors

<

mean

12.82

conductors

SD

2.25

conductors

<

p95

16

conductors

Volatile conductor total running duration

The total running duration of a get_agent_activity_volatile conductor.

Mean

mean

718.070972

s

SD

386.743672

s

Max

Highest per-partition mean latency

max mean

1125.5625

s

Min

Lowest per-partition mean latency

min mean

539.794872

s

Volatile conductor running duration

The duration that a get_agent_activity_volatile conductor was running before being stopped.

Mean

mean

44.416007

s

SD

22.018028

s

Max

Highest per-partition mean latency

max mean

142.914491

s

Min

Lowest per-partition mean latency

min mean

24.8079

s

Volatile conductor stopped duration

The duration that a get_agent_activity_volatile conductor was stopped before being started again.

Mean

mean

26.229225

s

SD

22.79234

s

Max

Highest per-partition mean latency

max mean

41.320496

s

Min

Lowest per-partition mean latency

min mean

6.192766

s

Reached target arc

Did get_agent_activity_volatile peer reach their target arc in the moment before they were shutdown.

get_agent_activity_volatile_agent: uhCAk-8BekA2VJw6V6hCGBqPz7ek3CVK7M_uqTONk54nRZMDEHlPR

p5

0

<

mean

0.38

SD

0.49

<

p95

1

get_agent_activity_volatile_agent: uhCAk0vl_NLTdRvV3ozlq9fM-T_pvfEj89nE_4ncpOSzk3XlIPz8s

p5

0

<

mean

0.31

SD

0.46

<

p95

1

get_agent_activity_volatile_agent: uhCAk9CzVS9bwJDRrTMCyHn23EAAS5VjwDpxighDG7Lh3AxOzDu6k

p5

0

<

mean

0.23

SD

0.42

<

p95

1

get_agent_activity_volatile_agent: uhCAk9zrpyfmxUGKg2NVY0npNN8F1Dk49TbouvWWVW7xcL_MKB_PN

p5

0

<

mean

0.24

SD

0.43

<

p95

1

get_agent_activity_volatile_agent: uhCAkAh5uA2keAqgcA1U382CFsyefiyr1x8XMSl1q7EuJti-XhKoa

p5

0

<

mean

0.34

SD

0.47

<

p95

1

get_agent_activity_volatile_agent: uhCAkGd5b9fOt4PiINzwodz0r5M3n9J87hUoMRsX4dHKrhFBeRJ4W

p5

0

≤

mean

0

SD

0

≤

p95

0

get_agent_activity_volatile_agent: uhCAkHkjCu66ogK7O5oAi7PBdhTTBCajF3w1qoQiRjdY5cBJUMkko

p5

0

<

mean

0.24

SD

0.43

<

p95

1

get_agent_activity_volatile_agent: uhCAkKpqk21zYXxJSo-45lDq1o4pqpRHLP43LCeuzgLuM67DDjvi0

p5

0

<

mean

0.22

SD

0.41

<

p95

1

get_agent_activity_volatile_agent: uhCAkNkc1XoCieDPpVRgtMhI8UikS1FfHu6n2j-dfpXgh2jKDPPFd

p5

0

<

mean

0.27

SD

0.45

<

p95

1

get_agent_activity_volatile_agent: uhCAkOUmv0suO2Qez84xyGDg07W4lU-j8qcWnrEz9Ac88bvepl0UB

p5

0

<

mean

0.23

SD

0.42

<

p95

1

get_agent_activity_volatile_agent: uhCAkQ1H2QcW-_WOzj-AQgMXnTJohcbmh93pStAED1jVcYML9rAe7

p5

0

<

mean

0.27

SD

0.44

<

p95

1

get_agent_activity_volatile_agent: uhCAkS-g4P_cl5uZ1QZ-8AfprwTwvIssLOCWHjoeEBfpwPov-MC7h

p5

0

<

mean

0.19

SD

0.39

<

p95

1

get_agent_activity_volatile_agent: uhCAkd6I3M61m979rTy0xmGddDuh9PWWB_pRa_jkRWbxeUVwhst3_

p5

0

≤

mean

0

SD

0

≤

p95

0

get_agent_activity_volatile_agent: uhCAkeAxYkxyoAZValhd-c6ef-p1kCVuCkS6b9BGBG_koNgNstE9E

p5

0

<

mean

0.26

SD

0.44

<

p95

1

get_agent_activity_volatile_agent: uhCAkgS35xMN6yuPcQEFldFOH9cuO3Pn9wkCqMqDFDYyMvNB1i4nW

p5

0

<

mean

0.24

SD

0.43

<

p95

1

get_agent_activity_volatile_agent: uhCAkhGW40_GZQSTWE1NDwMxAeLkGkunOIqz55v7feT5LlNGFSE2e

p5

0

<

mean

0.25

SD

0.43

<

p95

1

get_agent_activity_volatile_agent: uhCAkk3JXxZVy4sQum-Wi-u40D_ISZ6BFhcEdVpJiwpr4kpXWRegD

p5

0

<

mean

0.25

SD

0.43

<

p95

1

get_agent_activity_volatile_agent: uhCAkk75ADAekwZssqFljVGR842lhu3rRWdnQ8nC_LYQ-xAeLne2a

p5

0

<

mean

0.28

SD

0.45

<

p95

1

get_agent_activity_volatile_agent: uhCAkkmPYoyKrxtxOi81qGr8mbWZbZCrb3jnmy03D8ILlifrsgsod

p5

0

<

mean

0.21

SD

0.4

<

p95

1

get_agent_activity_volatile_agent: uhCAkz6Bqf1ueoOhnjqt9H2Oou5A6-pFCrvQEz98ZscMJG5FeZ5q1

p5

0

<

mean

0.13

SD

0.33

<

p95

1

Error count

The number of errors accumulated.

179

Holochain MetricsClick to toggle visibility.

Cascade duration

Time taken to execute a cascade (get) query inside Holochain.

mean

0.120972

s

SD

1.969631

s

p50

0.438976

s

<

p95

1.53459

s

<

p99

3.183119

s

Cascade fetch errors

Count of network fetch errors during cascade calls.

total

3

over 1699.682164715s

mean rate

14.61

/s

std

43.3

/s

p5

0

/s

p95

158.44

/s

peak

181.28

/s

Zome call duration: 'agent_activity::announce_write_behaviour'

Duration of this zome call as measured by Holochain internally.

mean

0.005972

s

SD

0.001712

s

p50

0.005921

s

<

p95

0.010001

s

≤

p99

0.010001

s

Zome call duration: 'agent_activity::create_sample_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.004665

s

SD

0.001067

s

p50

0.004476

s

<

p95

0.006783

s

<

p99

0.009258

s

Zome call duration: 'agent_activity::get_agent_activity_full'

Duration of this zome call as measured by Holochain internally.

mean

0.147899

s

SD

0.685055

s

p50

0.472097

s

<

p95

1.622873

s

<

p99

3.235692

s

Zome call duration: 'agent_activity::get_random_agent_with_write_behaviour'

Duration of this zome call as measured by Holochain internally.

mean

0.004249

s

SD

0.214933

s

p50

0.005982

s

<

p95

0.557321

s

<

p99

0.626266

s

WASM call duration: 'agent_activity::announce_write_behaviour'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.005074

s

SD

0.001496

s

p50

0.004881

s

<

p95

0.008569

s

≤

p99

0.008569

s

WASM call duration: 'agent_activity::create_sample_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.003918

s

SD

0.000947

s

p50

0.00374

s

<

p95

0.005717

s

<

p99

0.007964

s

WASM call duration: 'agent_activity::get_agent_activity_full'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.145611

s

SD

2.004871

s

p50

0.453392

s

<

p95

1.589945

s

<

p99

3.202337

s

WASM call duration: 'agent_activity::get_random_agent_with_write_behaviour'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.003571

s

SD

0.212377

s

p50

0.00478

s

<

p95

0.556794

s

<

p99

0.625585

s

WASM call duration: 'agent_activity_integrity::entry_defs'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.001313

s

SD

0.000347

s

p50

0.001266

s

<

p95

0.001942

s

<

p99

0.002907

s

Host function call duration: '__hc__agent_info_1'

Duration of this host function call invoked from within WASM.

mean

3.3e-05

s

SD

8e-06

s

p50

3.6e-05

s

<

p95

4.4e-05

s

≤

p99

4.4e-05

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.000438

s

SD

0.000117

s

p50

0.000408

s

<

p95

0.000701

s

<

p99

0.000798

s

Host function call duration: '__hc__create_link_1'

Duration of this host function call invoked from within WASM.

mean

0.000482

s

SD

0.000149

s

p50

0.00046

s

<

p95

0.000735

s

≤

p99

0.000735

s

Host function call duration: '__hc__get_agent_activity_1'

Duration of this host function call invoked from within WASM.

mean

0.142072

s

SD

2.004612

s

p50

0.448688

s

<

p95

1.582906

s

<

p99

3.1961

s

Host function call duration: '__hc__get_links_1'

Duration of this host function call invoked from within WASM.

mean

0.00066

s

SD

0.212424

s

p50

0.000449

s

<

p95

0.555118

s

<

p99

0.623956

s

Host function call duration: '__hc__random_bytes_1'

Duration of this host function call invoked from within WASM.

mean

1e-05

s

SD

5e-06

s

p50

9e-06

s

<

p95

2.2e-05

s

≤

p99

2.2e-05

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.002067

s

SD

0.000672

s

p50

0.002251

s

<

p95

0.004264

s

≤

p99

0.004264

s

Post-commit duration

Time spent executing post-commit workflows.

mean

0.000146

s

SD

4.7e-05

s

p50

0.000131

s

<

p95

0.000236

s

<

p99

0.000311

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

3.31

s

<

mean

377.97

s

SD

535.41

s

<

p95

1557.74

s

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

206.139541

s

SD

216.732546

s

p50

85.608433

s

<

p95

731.108861

s

<

p99

999.456064

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.046655

s

SD

0.169262

s

p50

0.016083

s

<

p95

0.434355

s

<

p99

0.753581

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.050209

s

SD

0.274632

s

p50

0.02172

s

<

p95

0.39423

s

<

p99

1.201075

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.202453

s

SD

0.40079

s

p50

0.10468

s

<

p95

1.311011

s

<

p99

1.774645

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

11.778188

s

SD

42.945515

s

p50

27.915312

s

<

p95

120.431092

s

<

p99

142.225359

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

108.075383

s

SD

85.891574

s

p50

116.41058

s

<

p95

271.968962

s

<

p99

314.113711

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

0.006363

s

SD

0.003009

s

p50

0.001767

s

<

p95

0.009719

s

<

p99

0.013152

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.008191

s

SD

0.041945

s

p50

0.001424

s

<

p95

0.035997

s

<

p99

0.151973

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000389

s

SD

0.000224

s

p50

0.00041

s

<

p95

0.000846

s

<

p99

0.001424

s

P2P request roundtrip: 'get'

Time spent sending a get request and awaiting its response.

mean

8.076799

s

SD

27.228046

s

p50

1.376291

s

<

p95

60.001415

s

<

p99

60.001624

s

P2P request roundtrip: 'get_links'

Time spent sending a get_links request and awaiting its response.

mean

0.475079

s

SD

0.107206

s

p50

0.451584

s

<

p95

0.622128

s

≤

p99

0.622128

s

P2P request roundtrip: 'get_agent_activity'

Time spent sending a get_agent_activity request and awaiting its response.

mean

0.162359

s

SD

8.21142

s

p50

0.455012

s

<

p95

2.026532

s

<

p99

60.001249

s

P2P request roundtrip: 'send_validation_receipts'

Time spent sending a send_validation_receipts request and awaiting its response.

mean

1.002581

s

SD

17.091058

s

p50

1.14925

s

<

p95

60.000468

s

<

p99

60.001555

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

28.38

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

1.28

KiB/s

<

mean

191.24

KiB/s

SD

478.64

KiB/s

<

p95

727.34

KiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

1.74

KiB/s

<

mean

158.06

KiB/s

SD

288.75

KiB/s

<

p95

705.21

KiB/s

Total bytes received

Total bytes received on primary network interface

count

8.28

GiB

mean

4.74

MiB/s

Total bytes sent

Total bytes sent on primary network interface

count

6.84

GiB

mean

3.91

MiB/s

CPU spike anomaly

CPU spike anomaly was detected

Detected

Warning CPU p99 reached 99.0%

Memory leak anomaly

Memory leak anomaly was detected

NotDetected

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

Detected

Warning Moderate swap usage (13.9% swap used)

System overload anomaly

System overload anomaly was detected

Detected

Warning 22% of hosts overloaded (load5/ncpus > 1.0)

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

1.51

%

<

mean

37.81

%

SD

33.45

%

<

p95

98.3

%

System

9.44

%

CPU percentiles

CPU usage percentiles

p50

29.39

%

p95

98.30

%

p99

98.95

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

12

hosts

mean time

748.91

s

Memory used percentage

Percentage of memory used

p5

5.94

%

<

mean

16.76

%

SD

7.63

%

<

p95

30.88

%

Memory available percentage

Percentage of available memory

p5

69.12

%

<

mean

83.24

%

SD

7.63

%

<

p95

94.06

%

Max host memory used

Maximum memory usage percentage across all hosts

max

36.65

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

13.87

%

Memory growth rate

Rate of memory growth over time

growth

0.14

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.22

MiB/s

Disk write throughput

Disk write throughput in MiB/s

42.87

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /

0/2

hosts

Mount Point /efi-boot

0/2

hosts

Mount Point /etc/hostname

0/13

hosts

Mount Point /etc/hosts

0/13

hosts

Mount Point /etc/nomad.d/client-meta.json

0/12

hosts

Mount Point /etc/resolv.conf

0/13

hosts

Mount Point /nix/store

0/2

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

2.29

5 min

2.49

15 min

2.96

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

22.22

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

0

%

<

mean

14.5066

%

SD

25.6745

%

<

p95

84.35

%

60 second average

14.4653

%

300 second average

15.5501

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

2.7249

%

SD

8.868

%

<

p95

29.87

%

60 second average

2.8271

%

300 second average

3.0790

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

1.3194

%

SD

5.4275

%

<

p95

8.6

%

60 second average

1.4108

%

300 second average

1.6320

%

Holochain process CPU usage

CPU usage by Holochain process

p5

0

%

<

mean

28.46

%

SD

34.88

%

<

p95

97.43

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

57.69

KiB

<

mean

218.42

KiB

SD

123.74

KiB

<

p95

427.26

KiB

Holochain process threads

Number of threads in Holochain process

p5

1

threads

<

mean

26.9

threads

SD

10.23

threads

<

p95

36

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

0

file descriptors

<

mean

68.31

file descriptors

SD

24.46

file descriptors

<

p95

86

file descriptors

Write/Query

↑ Back to index

Creates an entry, then queries the source chain and performs a simple operation on the entries, then repeats.

Started

Thu, 17 Sep 2026 09:08:46 UTC

Duration

300s

Peer count

1

Peer count at end

1

Behaviours

default (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

write_query_35182846570_1

Write timing

The time it takes to complete the zome function call that writes the entry.

mean

0.030594

s

SD

0.014799

s

p50

0.029586

s

<

p95

0.053103

s

<

p99

0.059885

s

Write rate

The number of writes completed per second.

mean

96.69

/s

Query timing

The time it takes to complete the chain_query_count_len zome call that queries the source chain.

mean

0.072105

s

SD

0.039027

s

p50

0.070873

s

<

p95

0.132922

s

<

p99

0.149173

s

Query rate

The number of chain queries completed per second.

mean

96.66

/s

Error count

The number of errors accumulated during the run.

0

Holochain MetricsClick to toggle visibility.

WASM usage: 'crud::chain_query_count_len'

Metered usage count of the WASM ribosome for this zome function.

total

81931577879

over 270.004518151s

mean rate

3.0344511383e+08

/s

std

3.013692335e+07

/s

p5

2.4929280271e+08

/s

p95

3.2530019654e+08

/s

peak

3.2574316619e+08

/s

WASM usage: 'crud::create_sample_entry'

Metered usage count of the WASM ribosome for this zome function.

total

95582130

over 270.004518151s

mean rate

354002.96

/s

std

224812.33

/s

p5

196595.73

/s

p95

793535.32

/s

peak

1.24034238e+06

/s

WASM usage: 'crud::init'

Metered usage count of the WASM ribosome for this zome function.

total

0

over 270.004518151s

Not enough time data to show a trend.

WASM usage: 'crud_integrity::entry_defs'

Metered usage count of the WASM ribosome for this zome function.

total

23541896

over 270.004518151s

mean rate

87191

/s

std

55358.88

/s

p5

48421.56

/s

p95

195447.89

/s

peak

305496.55

/s

Zome call duration: 'crud::chain_query_count_len'

Duration of this zome call as measured by Holochain internally.

mean

0.029548

s

SD

0.009459

s

p50

0.027538

s

<

p95

0.038358

s

<

p99

0.039133

s

Zome call duration: 'crud::create_sample_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.003255

s

SD

3e-05

s

p50

0.003256

s

<

p95

0.003261

s

<

p99

0.003411

s

WASM call duration: 'crud::chain_query_count_len'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.028921

s

SD

0.009429

s

p50

0.026914

s

<

p95

0.037706

s

<

p99

0.038479

s

WASM call duration: 'crud::create_sample_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.002693

s

SD

2.9e-05

s

p50

0.002694

s

<

p95

0.002696

s

<

p99

0.002847

s

WASM call duration: 'crud::init'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.003888

s

SD

0

s

p50

0.003888

s

≤

p95

0.003888

s

≤

p99

0.003888

s

WASM call duration: 'crud_integrity::entry_defs'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.000937

s

SD

1.2e-05

s

p50

0.000933

s

<

p95

0.000951

s

<

p99

0.000952

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.000261

s

SD

0.000138

s

p50

0.000536

s

≤

p95

0.000536

s

≤

p99

0.000536

s

Host function call duration: '__hc__query_1'

Duration of this host function call invoked from within WASM.

mean

0.01222

s

SD

0.004133

s

p50

0.01142

s

<

p95

0.015936

s

<

p99

0.016246

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.001444

s

SD

4.7e-05

s

p50

0.001414

s

<

p95

0.001526

s

<

p99

0.001532

s

Post-commit duration

Time spent executing post-commit workflows.

mean

0.00011

s

SD

8e-06

s

p50

0.000111

s

<

p95

0.000114

s

<

p99

0.000115

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

16.78

s

<

mean

141.78

s

SD

80.78

s

<

p95

266.78

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

0

over 270.004531372s

Not enough time data to show a trend.

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.000433

s

SD

0

s

p50

0.000433

s

≤

p95

0.000433

s

≤

p99

0.000433

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.001817

s

SD

0.000158

s

p50

0.001802

s

<

p95

0.001891

s

<

p99

0.0019

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.024277

s

SD

0.007737

s

p50

0.02292

s

<

p95

0.030992

s

<

p99

0.031553

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

0.003076

s

SD

0

s

p50

0.003076

s

≤

p95

0.003076

s

≤

p99

0.003076

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

0.002609

s

SD

0

s

p50

0.002609

s

≤

p95

0.002609

s

≤

p99

0.002609

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

0.002273

s

SD

0.00077

s

p50

0.002148

s

<

p95

0.002932

s

<

p99

0.002985

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.001056

s

SD

0.000163

s

p50

0.001317

s

≤

p95

0.001317

s

≤

p99

0.001317

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000231

s

SD

7e-06

s

p50

0.000231

s

<

p95

0.000237

s

<

p99

0.000268

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

55.43

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

18.03

KiB/s

<

mean

123.85

KiB/s

SD

379.96

KiB/s

<

p95

1.10

MiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

13.60

KiB/s

<

mean

16.32

KiB/s

SD

4.91

KiB/s

<

p95

26.94

KiB/s

Total bytes received

Total bytes received on primary network interface

count

33.86

MiB

mean

123.85

KiB/s

Total bytes sent

Total bytes sent on primary network interface

count

4.46

MiB

mean

16.32

KiB/s

CPU spike anomaly

CPU spike anomaly was detected

NotDetected

Memory leak anomaly

Memory leak anomaly was detected

NotDetected

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

NotDetected

System overload anomaly

System overload anomaly was detected

NotDetected

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

66.99

%

<

mean

70.98

%

SD

2.23

%

<

p95

73.04

%

System

15.55

%

CPU percentiles

CPU usage percentiles

p50

71.63

%

p95

73.04

%

p99

77.20

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

0

hosts

mean time

0.00

s

Memory used percentage

Percentage of memory used

p5

23.45

%

<

mean

24.19

%

SD

0.73

%

<

p95

24.94

%

Memory available percentage

Percentage of available memory

p5

75.06

%

<

mean

75.81

%

SD

0.73

%

<

p95

76.55

%

Max host memory used

Maximum memory usage percentage across all hosts

max

24.19

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

0.00

%

Memory growth rate

Rate of memory growth over time

growth

0.28

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.00

MiB/s

Disk write throughput

Disk write throughput in MiB/s

2.72

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /etc/nomad.d/client-meta.json

0/1

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

2.05

5 min

1.23

15 min

0.59

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

0.00

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

17.25

%

<

mean

24.599

%

SD

3.8669

%

<

p95

30.09

%

60 second average

21.9386

%

300 second average

14.2941

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0034

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0014

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

0.3138

%

SD

0.6951

%

<

p95

1.41

%

60 second average

0.2438

%

300 second average

0.3090

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

0.1017

%

SD

0.2012

%

<

p95

0.45

%

60 second average

0.1045

%

300 second average

0.2059

%

Holochain process CPU usage

CPU usage by Holochain process

p5

53.2

%

<

mean

60.27

%

SD

11.68

%

<

p95

64.48

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

246.58

KiB

<

mean

256.13

KiB

SD

6.96

KiB

<

p95

270.11

KiB

Holochain process threads

Number of threads in Holochain process

p5

21

threads

<

mean

23.1

threads

SD

1.37

threads

<

p95

24

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

60

file descriptors

<

mean

64.41

file descriptors

SD

3.12

file descriptors

<

p95

67

file descriptors

Write/Read

↑ Back to index

Creates an entry and reads it back, then repeats.

Started

Thu, 17 Sep 2026 09:10:19 UTC

Duration

300s

Peer count

1

Peer count at end

1

Behaviours

default (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

write_read_35182846570_1

Write timing

The time taken to write the entry.

mean

0.043896

s

SD

0.026612

s

p50

0.03914

s

<

p95

0.088464

s

<

p99

0.125301

s

Write rate

The number of writes completed per second.

mean

103.75

/s

Read timing

The time taken to read the entry.

mean

0.046879

s

SD

0.030627

s

p50

0.040416

s

<

p95

0.098822

s

<

p99

0.138132

s

Read rate

The number of reads completed per second.

mean

103.75

/s

Error count

The number of errors accumulated over the run.

0

Holochain MetricsClick to toggle visibility.

Cascade duration

Time taken to execute a cascade (get) query inside Holochain.

mean

0.000611

s

SD

9.6e-05

s

p50

0.000579

s

<

p95

0.000722

s

<

p99

0.000738

s

WASM usage: 'crud::create_sample_entry'

Metered usage count of the WASM ribosome for this zome function.

total

106984785

over 280.006223676s

mean rate

382083.27

/s

std

245344.5

/s

p5

185872.85

/s

p95

897193.67

/s

peak

1.30112006e+06

/s

WASM usage: 'crud::get_sample_entry'

Metered usage count of the WASM ribosome for this zome function.

total

99812251

over 280.006223676s

mean rate

356467.46

/s

std

228816.04

/s

p5

170124.75

/s

p95

840616.13

/s

peak

1.21140882e+06

/s

WASM usage: 'crud::init'

Metered usage count of the WASM ribosome for this zome function.

total

0

over 280.006223676s

Not enough time data to show a trend.

WASM usage: 'crud_integrity::entry_defs'

Metered usage count of the WASM ribosome for this zome function.

total

13175186

over 280.006223676s

mean rate

47053.59

/s

std

30214.2

/s

p5

22890.26

/s

p95

110489.48

/s

peak

160233.05

/s

Zome call duration: 'crud::create_sample_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.003098

s

SD

0.000144

s

p50

0.00306

s

<

p95

0.003272

s

<

p99

0.003293

s

Zome call duration: 'crud::get_sample_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.00218

s

SD

0.000163

s

p50

0.002122

s

<

p95

0.002385

s

<

p99

0.002413

s

WASM call duration: 'crud::create_sample_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.002519

s

SD

0.000123

s

p50

0.002487

s

<

p95

0.00267

s

<

p99

0.002688

s

WASM call duration: 'crud::get_sample_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.001608

s

SD

0.000141

s

p50

0.001559

s

<

p95

0.001785

s

<

p99

0.00181

s

WASM call duration: 'crud::init'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.001152

s

SD

0

s

p50

0.001152

s

≤

p95

0.001152

s

≤

p99

0.001152

s

WASM call duration: 'crud_integrity::entry_defs'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.000798

s

SD

3.2e-05

s

p50

0.000785

s

<

p95

0.000841

s

<

p99

0.000847

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.000299

s

SD

1.9e-05

s

p50

0.000277

s

<

p95

0.000321

s

<

p99

0.000328

s

Host function call duration: '__hc__get_1'

Duration of this host function call invoked from within WASM.

mean

0.000645

s

SD

9.6e-05

s

p50

0.000613

s

<

p95

0.000756

s

<

p99

0.000772

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.001276

s

SD

6.1e-05

s

p50

0.001257

s

<

p95

0.001353

s

<

p99

0.001363

s

Post-commit duration

Time spent executing post-commit workflows.

mean

8.8e-05

s

SD

7e-06

s

p50

8.5e-05

s

<

p95

9.6e-05

s

≤

p99

9.6e-05

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

17.55

s

<

mean

147.55

s

SD

83.67

s

<

p95

277.55

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

0

over 280.00622043s

Not enough time data to show a trend.

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.004734

s

SD

0

s

p50

0.004734

s

≤

p95

0.004734

s

≤

p99

0.004734

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.001256

s

SD

0.000167

s

p50

0.001225

s

<

p95

0.001413

s

<

p99

0.001425

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.030601

s

SD

0.010911

s

p50

0.027701

s

<

p95

0.041881

s

<

p99

0.043005

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

0.00549

s

SD

0

s

p50

0.00549

s

≤

p95

0.00549

s

≤

p99

0.00549

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

0.001043

s

SD

0

s

p50

0.001043

s

≤

p95

0.001043

s

≤

p99

0.001043

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

0.002624

s

SD

0.000917

s

p50

0.002408

s

<

p95

0.003545

s

<

p99

0.003622

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.000971

s

SD

0.00055

s

p50

0.002017

s

≤

p95

0.002017

s

≤

p99

0.002017

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000266

s

SD

2.4e-05

s

p50

0.000263

s

<

p95

0.000291

s

<

p99

0.000294

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

61.96

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

18.14

KiB/s

<

mean

1.03

MiB/s

SD

2.13

MiB/s

<

p95

7.96

MiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

14.34

KiB/s

<

mean

37.43

KiB/s

SD

41.19

KiB/s

<

p95

157.66

KiB/s

Total bytes received

Total bytes received on primary network interface

count

287.07

MiB

mean

1.03

MiB/s

Total bytes sent

Total bytes sent on primary network interface

count

10.23

MiB

mean

37.43

KiB/s

CPU spike anomaly

CPU spike anomaly was detected

Detected

Warning CPU p99 reached 95.5%

Memory leak anomaly

Memory leak anomaly was detected

Detected

Warning Memory growing at 2.51 MiB/s

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

NotDetected

System overload anomaly

System overload anomaly was detected

NotDetected

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

68.89

%

<

mean

80.36

%

SD

8.38

%

<

p95

93.65

%

System

18.40

%

CPU percentiles

CPU usage percentiles

p50

79.28

%

p95

93.65

%

p99

95.49

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

1

hosts

mean time

139.29

s

Memory used percentage

Percentage of memory used

p5

7.97

%

<

mean

9

%

SD

0.78

%

<

p95

10.19

%

Memory available percentage

Percentage of available memory

p5

89.81

%

<

mean

91

%

SD

0.78

%

<

p95

92.03

%

Max host memory used

Maximum memory usage percentage across all hosts

max

9.00

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

0.00

%

Memory growth rate

Rate of memory growth over time

growth

2.51

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.01

MiB/s

Disk write throughput

Disk write throughput in MiB/s

6.51

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /etc/nomad.d/client-meta.json

0/1

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

4.16

5 min

2.41

15 min

1.09

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

0.00

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

19.49

%

<

mean

31.1638

%

SD

12.6485

%

<

p95

49.45

%

60 second average

24.2421

%

300 second average

11.4721

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

Holochain process CPU usage

CPU usage by Holochain process

p5

16.48

%

<

mean

31.8

%

SD

10.13

%

<

p95

50.16

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

26.01

KiB

<

mean

282.26

KiB

SD

133.65

KiB

<

p95

475.35

KiB

Holochain process threads

Number of threads in Holochain process

p5

8

threads

<

mean

26

threads

SD

7.27

threads

<

p95

33

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

8

file descriptors

<

mean

59.16

file descriptors

SD

18.6

file descriptors

<

p95

74

file descriptors

Write validated

↑ Back to index

Creates an entry, updates it and links the create to the update, then repeats. Each of the three actions is validated by the included hApp.

Started

Thu, 17 Sep 2026 09:11:51 UTC

Duration

300s

Peer count

1

Peer count at end

1

Behaviours

default (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

write_validated_35182846570_1

Write timing

The time it takes to complete the zome function call that writes the entry, updates it, and links to the update.

mean

0.083031

s

SD

0.041741

s

p50

0.079834

s

<

p95

0.149486

s

<

p99

0.181861

s

Write rate

The number of write zome function calls that complete per second.

mean

46.19

/s

Read timing

The time it takes to complete the zome function call that reads the data.

mean

0.12277

s

SD

0.059139

s

p50

0.114079

s

<

p95

0.2171

s

<

p99

0.299971

s

Read rate

The number of read zome function calls that complete per second.

mean

46.22

/s

Error count

The number of errors accumulated over the run.

0

Holochain MetricsClick to toggle visibility.

Cascade duration

Time taken to execute a cascade (get) query inside Holochain.

mean

0.005184

s

SD

0.002188

s

p50

0.004641

s

<

p95

0.008751

s

<

p99

0.009019

s

WASM usage: 'validated::create_sample_entry'

Metered usage count of the WASM ribosome for this zome function.

total

42890660

over 270.004872034s

mean rate

158851.39

/s

std

75253.91

/s

p5

89279.2

/s

p95

319360.67

/s

peak

394890.12

/s

WASM usage: 'validated::update_sample_entry'

Metered usage count of the WASM ribosome for this zome function.

total

50421696

over 270.004872034s

mean rate

186743.61

/s

std

88379.42

/s

p5

105039.55

/s

p95

371696.85

/s

peak

464599.61

/s

WASM usage: 'validated_integrity::entry_defs'

Metered usage count of the WASM ribosome for this zome function.

total

82826068

over 280.004904804s

mean rate

295802.35

/s

std

150801.36

/s

p5

157751.32

/s

p95

684058.16

/s

peak

699670.77

/s

WASM usage: 'validated_integrity::validate'

Metered usage count of the WASM ribosome for this zome function.

total

754562024

over 270.004872034s

mean rate

2.79462312e+06

/s

std

1.32467585e+06

/s

p5

1.55919221e+06

/s

p95

5.61881343e+06

/s

peak

6.94229445e+06

/s

Zome call duration: 'validated::create_sample_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.003812

s

SD

0.000271

s

p50

0.003766

s

<

p95

0.004049

s

<

p99

0.00405

s

Zome call duration: 'validated::update_sample_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.010753

s

SD

0.002589

s

p50

0.010168

s

<

p95

0.013115

s

<

p99

0.013233

s

WASM call duration: 'validated::create_sample_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.003128

s

SD

0.000234

s

p50

0.003084

s

<

p95

0.003338

s

<

p99

0.003339

s

WASM call duration: 'validated::update_sample_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.010072

s

SD

0.002551

s

p50

0.009491

s

<

p95

0.012411

s

<

p99

0.012531

s

WASM call duration: 'validated_integrity::entry_defs'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.000904

s

SD

5.3e-05

s

p50

0.000884

s

<

p95

0.000951

s

<

p99

0.000953

s

WASM call duration: 'validated_integrity::validate'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.003406

s

SD

0.000417

s

p50

0.0033

s

<

p95

0.003785

s

<

p99

0.003797

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.000386

s

SD

3.9e-05

s

p50

0.000383

s

<

p95

0.00042

s

≤

p99

0.00042

s

Host function call duration: '__hc__create_link_1'

Duration of this host function call invoked from within WASM.

mean

0.000354

s

SD

3.2e-05

s

p50

0.000348

s

<

p95

0.000387

s

≤

p99

0.000387

s

Host function call duration: '__hc__must_get_valid_record_1'

Duration of this host function call invoked from within WASM.

mean

0.00445

s

SD

0.0015

s

p50

0.004082

s

<

p95

0.006014

s

<

p99

0.006132

s

Host function call duration: '__hc__update_1'

Duration of this host function call invoked from within WASM.

mean

0.00708

s

SD

0.002362

s

p50

0.006536

s

<

p95

0.009293

s

<

p99

0.009424

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.001495

s

SD

0.000118

s

p50

0.00151

s

<

p95

0.001733

s

<

p99

0.001748

s

Post-commit duration

Time spent executing post-commit workflows.

mean

0.000115

s

SD

6e-06

s

p50

0.000116

s

<

p95

0.000121

s

≤

p99

0.000121

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

17.25

s

<

mean

147.26

s

SD

83.67

s

<

p95

277.26

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

0

over 280.004895376s

Not enough time data to show a trend.

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.001328

s

SD

0

s

p50

0.001328

s

≤

p95

0.001328

s

≤

p99

0.001328

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.001545

s

SD

0.000162

s

p50

0.00155

s

<

p95

0.001626

s

<

p99

0.002156

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.050641

s

SD

0.019198

s

p50

0.044619

s

<

p95

0.067636

s

<

p99

0.068695

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

0.001691

s

SD

0

s

p50

0.001691

s

≤

p95

0.001691

s

≤

p99

0.001691

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

0.001445

s

SD

0

s

p50

0.001445

s

≤

p95

0.001445

s

≤

p99

0.001445

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

0.003805

s

SD

0.00151

s

p50

0.003302

s

<

p95

0.005098

s

<

p99

0.005177

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.001554

s

SD

0.000122

s

p50

0.001478

s

<

p95

0.001666

s

<

p99

0.001671

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000302

s

SD

2.7e-05

s

p50

0.000297

s

<

p95

0.000326

s

<

p99

0.000345

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

65.41

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

18.37

KiB/s

<

mean

1.33

MiB/s

SD

2.08

MiB/s

<

p95

7.69

MiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

14.34

KiB/s

<

mean

44.26

KiB/s

SD

40.59

KiB/s

<

p95

150.67

KiB/s

Total bytes received

Total bytes received on primary network interface

count

385.47

MiB

mean

1.33

MiB/s

Total bytes sent

Total bytes sent on primary network interface

count

12.54

MiB

mean

44.26

KiB/s

CPU spike anomaly

CPU spike anomaly was detected

Detected

Warning CPU p99 reached 97.0%

Memory leak anomaly

Memory leak anomaly was detected

Detected

Warning Memory growing at 1.23 MiB/s

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

NotDetected

System overload anomaly

System overload anomaly was detected

NotDetected

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

74.7

%

<

mean

83.68

%

SD

7.02

%

<

p95

95.49

%

System

18.26

%

CPU percentiles

CPU usage percentiles

p50

81.22

%

p95

95.49

%

p99

97.04

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

1

hosts

mean time

186.21

s

Memory used percentage

Percentage of memory used

p5

8.81

%

<

mean

9.66

%

SD

1.14

%

<

p95

11.9

%

Memory available percentage

Percentage of available memory

p5

88.1

%

<

mean

90.34

%

SD

1.14

%

<

p95

91.19

%

Max host memory used

Maximum memory usage percentage across all hosts

max

9.66

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

0.00

%

Memory growth rate

Rate of memory growth over time

growth

1.23

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.05

MiB/s

Disk write throughput

Disk write throughput in MiB/s

6.78

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /etc/nomad.d/client-meta.json

0/1

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

5.31

5 min

3.28

15 min

1.53

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

0.00

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

22.28

%

<

mean

37.244

%

SD

13.073

%

<

p95

57.94

%

60 second average

34.4757

%

300 second average

18.5920

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

0.072

%

SD

0.2629

%

<

p95

0.51

%

60 second average

0.0550

%

300 second average

0.0150

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

0.0197

%

SD

0.0736

%

<

p95

0.14

%

60 second average

0.0143

%

300 second average

0.0030

%

Holochain process CPU usage

CPU usage by Holochain process

p5

14.14

%

<

mean

25.97

%

SD

9.71

%

<

p95

37.45

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

20.06

KiB

<

mean

326.77

KiB

SD

298.43

KiB

<

p95

857.01

KiB

Holochain process threads

Number of threads in Holochain process

p5

7

threads

<

mean

37.82

threads

SD

53.62

threads

<

p95

147

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

8

file descriptors

<

mean

84.21

file descriptors

SD

114.54

file descriptors

<

p95

321

file descriptors

Zero-Arc Create and Read

↑ Back to index

A zero-arc/full-arc mixed scenario with two types of zero-arc nodes -- ones that create data and ones that read data -- as well as full arc nodes to "relay" the data. The scenario has three roles:

zero_write: A zero-arc conductor that just creates entries with a timestamp field. Those entries are linked to a known base hash so that zero_read nodes can retrieve them.
zero_read: A zero-arc conductor that reads the entries created by the zero-arc node(s) and records the time lag between when the entry had been created and when it was first discovered.
full: A full-arc conductor that is just here to serve entries to zero arc nodes.

Started

Thu, 17 Sep 2026 11:41:31 UTC

Duration

900s

Peer count

30

Peer count at end

28

Behaviours

full (1 agent)
zero_read (1 agent)
zero_write (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

zero_arc_create_and_read_35182846570_1

Create rate

The number of timed entries created by the zero-arc write node(s) per second.

Mean

mean

68.47

/s

Max

Highest per-partition mean rate

max

82

/s

Min

Lowest per-partition mean rate

min

51.67

/s

Fetch lag timing

For each entry, the time lag between when it was created and when a zero-arc read node fetched it from the network via the get_timed_entries_network zome function.

Mean

mean

189.318007

s

SD

6.27922

s

Max

Highest per-partition mean latency

max mean

686.126638

s

Min

Lowest per-partition mean latency

min mean

1.857496

s

Fetch lag rate

The number of entries per second fetched by zero-arc read nodes.

Mean

mean

0

/s

Max

Highest per-partition mean rate

max

0

/s

Min

Lowest per-partition mean rate

min

0

/s

Delivery ratio

The proportion of created entries successfully fetched by all zero-arc read nodes.

0.0034

Entry created count

Total number of entries created by zero-arc write nodes, by agent.

Total

63320

Agents affected

10 / 10

Mean per agent

Total count divided by number of partitions, rounded to nearest whole number

6332

Max

Highest count in any single partition

7562

Min

Lowest count in any single partition

4682

Received count

Total number of entries fetched by zero-arc read nodes, by agent.

Total

2367

Agents affected

8 / 11

Mean per agent

Total count divided by number of partitions, rounded to nearest whole number

215

Max

Highest count in any single partition

788

Min

Lowest count in any single partition

0

Open connections

The number of currently open connections to other conductors.

full-arc

p5

0

<

mean

15.91

SD

12.01

<

p95

26

zero-arc

p5

4

<

mean

7.9

SD

3.64

<

p95

14

Error count

The number of errors accumulated across all nodes.

121

Holochain MetricsClick to toggle visibility.

Cascade duration

Time taken to execute a cascade (get) query inside Holochain.

mean

0.027043

s

SD

2.331474

s

p50

0.290262

s

<

p95

3.576356

s

<

p99

14.852874

s

Cascade fetch errors

Count of network fetch errors during cascade calls.

total

0

over 0s

Not enough time data to show a trend.

WASM usage: 'timed::created_timed_entry'

Metered usage count of the WASM ribosome for this zome function.

total

416317070

over 876.829045446s

mean rate

9.052399928e+07

/s

std

9.969400517e+07

/s

p5

505669.3

/s

p95

2.8675344274e+08

/s

peak

3.1345497158e+08

/s

WASM usage: 'timed::get_timed_entries_network'

Metered usage count of the WASM ribosome for this zome function.

total

16871270656

over 876.758202892s

mean rate

9.52519326901e+09

/s

std

2.74279205975e+10

/s

p5

1.081602219e+07

/s

p95

6.2202719588e+10

/s

peak

1.91260636611e+11

/s

WASM usage: 'timed_integrity::entry_defs'

Metered usage count of the WASM ribosome for this zome function.

total

197643312

over 889.166035081s

mean rate

2.6628998912e+08

/s

std

4.9361907398e+08

/s

p5

0

/s

p95

1.60281618085e+09

/s

peak

2.23459761114e+09

/s

Zome call duration: 'timed::created_timed_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.004766

s

SD

0.000869

s

p50

0.004634

s

<

p95

0.006039

s

<

p99

0.006228

s

Zome call duration: 'timed::get_timed_entries_network'

Duration of this zome call as measured by Holochain internally.

mean

0.05734

s

SD

20.048463

s

p50

0.589387

s

<

p95

5.646158

s

<

p99

144.198365

s

WASM call duration: 'timed::created_timed_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.004234

s

SD

0.00081

s

p50

0.004146

s

<

p95

0.005365

s

<

p99

0.005519

s

WASM call duration: 'timed::get_timed_entries_network'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.056546

s

SD

20.048483

s

p50

0.588155

s

<

p95

5.645142

s

<

p99

144.197397

s

WASM call duration: 'timed_integrity::entry_defs'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.00098

s

SD

0.000947

s

p50

0.001076

s

<

p95

0.003326

s

<

p99

0.003395

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.000266

s

SD

5e-05

s

p50

0.000273

s

<

p95

0.000335

s

<

p99

0.000348

s

Host function call duration: '__hc__create_link_1'

Duration of this host function call invoked from within WASM.

mean

0.00026

s

SD

5.2e-05

s

p50

0.000266

s

<

p95

0.000325

s

<

p99

0.000327

s

Host function call duration: '__hc__get_1'

Duration of this host function call invoked from within WASM.

mean

0.023257

s

SD

2.955349

s

p50

0.003054

s

<

p95

4.747412

s

<

p99

18.794005

s

Host function call duration: '__hc__get_links_1'

Duration of this host function call invoked from within WASM.

mean

0.037875

s

SD

1.454851

s

p50

0.579591

s

<

p95

4.296353

s

<

p99

6.111147

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.001483

s

SD

0.00127

s

p50

0.001598

s

<

p95

0.004822

s

<

p99

0.004938

s

Post-commit duration

Time spent executing post-commit workflows.

mean

9.2e-05

s

SD

5.9e-05

s

p50

0.000134

s

<

p95

0.000238

s

<

p99

0.000249

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

47.11

s

<

mean

441.72

s

SD

253.96

s

<

p95

837.99

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

0

over 889.166032054s

Not enough time data to show a trend.

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

27.492816

s

SD

65.463445

s

p50

0

s

<

p95

0.866313

s

<

p99

552.797626

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.110751

s

SD

0.146239

s

p50

0.001535

s

<

p95

0.233543

s

<

p99

0.447573

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.035955

s

SD

0.402432

s

p50

0.001351

s

<

p95

1.171335

s

<

p99

1.253918

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.091588

s

SD

0.040304

s

p50

0.008873

s

<

p95

0.116896

s

<

p99

0.131456

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

12.877336

s

SD

8.580605

s

p50

0.002661

s

<

p95

22.046092

s

<

p99

35.437176

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

1.338226

s

SD

1.292062

s

p50

0.002432

s

<

p95

1.757668

s

<

p99

7.166434

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

0.001487

s

SD

0.000976

s

p50

0.000722

s

<

p95

0.002577

s

<

p99

0.002717

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.0018

s

SD

0.002143

s

p50

0.001145

s

<

p95

0.002686

s

<

p99

0.016473

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000211

s

SD

0.000182

s

p50

0.000272

s

<

p95

0.000595

s

<

p99

0.00102

s

P2P request roundtrip: 'get'

Time spent sending a get request and awaiting its response.

mean

1.700618

s

SD

28.574729

s

p50

0.442043

s

<

p95

60.00157

s

<

p99

60.005209

s

P2P request roundtrip: 'get_links'

Time spent sending a get_links request and awaiting its response.

mean

0.022613

s

SD

29.77235

s

p50

0.319245

s

<

p95

60.00167

s

<

p99

60.001871

s

P2P request roundtrip: 'send_validation_receipts'

Time spent sending a send_validation_receipts request and awaiting its response.

mean

1.649355

s

SD

2.607735

s

p50

1.349395

s

<

p95

7.026691

s

≤

p99

7.026691

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

33.16

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

996

B/s

<

mean

240.76

KiB/s

SD

556.45

KiB/s

<

p95

887.82

KiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

665

B/s

<

mean

193.58

KiB/s

SD

308.46

KiB/s

<

p95

564.80

KiB/s

Total bytes received

Total bytes received on primary network interface

count

4.88

GiB

mean

5.62

MiB/s

Total bytes sent

Total bytes sent on primary network interface

count

3.92

GiB

mean

4.52

MiB/s

CPU spike anomaly

CPU spike anomaly was detected

Detected

Warning CPU p99 reached 96.1%

Memory leak anomaly

Memory leak anomaly was detected

NotDetected

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

Detected

Warning Moderate swap usage (13.9% swap used)

System overload anomaly

System overload anomaly was detected

Detected

Warning 28% of hosts overloaded (load5/ncpus > 1.0)

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

1.18

%

<

mean

44.52

%

SD

34.6

%

<

p95

95.29

%

System

11.36

%

CPU percentiles

CPU usage percentiles

p50

49.89

%

p95

95.29

%

p99

96.08

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

7

hosts

mean time

719.83

s

Memory used percentage

Percentage of memory used

p5

6.08

%

<

mean

17.21

%

SD

8.06

%

<

p95

33.5

%

Memory available percentage

Percentage of available memory

p5

66.53

%

<

mean

82.79

%

SD

8.06

%

<

p95

93.93

%

Max host memory used

Maximum memory usage percentage across all hosts

max

35.29

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

13.87

%

Memory growth rate

Rate of memory growth over time

growth

0.21

MiB/s

Disk read throughput

Disk read throughput in MiB/s

1.01

MiB/s

Disk write throughput

Disk write throughput in MiB/s

76.10

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /

0/2

hosts

Mount Point /efi-boot

0/2

hosts

Mount Point /etc/hostname

0/12

hosts

Mount Point /etc/hosts

0/12

hosts

Mount Point /etc/nomad.d/client-meta.json

0/11

hosts

Mount Point /etc/resolv.conf

0/12

hosts

Mount Point /nix/store

0/2

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

2.29

5 min

2.34

15 min

2.46

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

28.00

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

0

%

<

mean

12.615

%

SD

17.3054

%

<

p95

49.48

%

60 second average

12.4798

%

300 second average

12.2186

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

1.4169

%

SD

5.5862

%

<

p95

4.92

%

60 second average

1.5217

%

300 second average

1.7702

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

0.7598

%

SD

4.1658

%

<

p95

3.16

%

60 second average

0.8737

%

300 second average

1.1844

%

Holochain process CPU usage

CPU usage by Holochain process

p5

0.28

%

<

mean

36.61

%

SD

33.39

%

<

p95

93.38

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

220.12

KiB

<

mean

293.51

KiB

SD

73.86

KiB

<

p95

442.33

KiB

Holochain process threads

Number of threads in Holochain process

p5

23

threads

<

mean

30.44

threads

SD

6.3

threads

<

p95

40

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

55

file descriptors

<

mean

69.79

file descriptors

SD

12.38

file descriptors

<

p95

85

file descriptors

Zero-Arc Create Data

↑ Back to index

A zero-arc/full-arc mixed scenario where zero-arc nodes create data and full-arc nodes read the data. The scenario has two roles:

zero: A zero-arc conductor that just creates entries with a timestamp field. Those entries are linked to a known base hash so that full-arc nodes can retrieve them.
full: A full-arc conductor that reads the entries created by the zero-arc node(s) and records the time lag between when the entry had been created and when it was first discovered.

Started

Thu, 17 Sep 2026 10:54:44 UTC

Duration

900s

Peer count

30

Peer count at end

27

Behaviours

full (1 agent)
zero (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

zero_arc_create_data--high_zero_35182846570_1

Create rate

The number of timed entries created by the zero-arc node(s) per second.

Mean

mean

67.21

/s

Max

Highest per-partition mean rate

max

83.54

/s

Min

Lowest per-partition mean rate

min

49.4

/s

Sync lag timing

For each entry, the time lag between when it was created and when the full-arc node could read it via the get_timed_local_entries zome function.

Mean

mean

130.586849

s

SD

0.182836

s

Max

Highest per-partition mean latency

max mean

261.071387

s

Min

Lowest per-partition mean latency

min mean

0.10231

s

Sync lag rate

The number of entries per second received by full nodes.

Mean

mean

0

/s

Max

Highest per-partition mean rate

max

0

/s

Min

Lowest per-partition mean rate

min

0

/s

Delivery ratio

The proportion of created entries successfully received by all full-arc nodes.

0.0016

Entry created count

Total number of entries created by zero-arc nodes, by agent.

Total

153073

Agents affected

25 / 25

Mean per agent

Total count divided by number of partitions, rounded to nearest whole number

6123

Max

Highest count in any single partition

7682

Min

Lowest count in any single partition

4458

Received count

Total number of entries received by full-arc nodes, by agent.

Total

486

Agents affected

2 / 2

Mean per agent

Total count divided by number of partitions, rounded to nearest whole number

243

Max

Highest count in any single partition

485

Min

Lowest count in any single partition

1

Open connections

The number of currently open connections to other conductors.

full-arc

p5

3

<

mean

19.63

SD

8.93

<

p95

26

zero-arc

p5

2

<

mean

7.74

SD

5.14

<

p95

16

Error count

The number of errors accumulated across all nodes.

0

Holochain MetricsClick to toggle visibility.

Cascade duration

Time taken to execute a cascade (get) query inside Holochain.

mean

0.000533

s

SD

0.000104

s

p50

0.000444

s

<

p95

0.000557

s

<

p99

0.000562

s

Cascade fetch errors

Count of network fetch errors during cascade calls.

total

5

over 790.01557376s

mean rate

0.41

/s

std

0.4

/s

p5

0

/s

p95

1.02

/s

peak

1.02

/s

WASM usage: 'timed::created_timed_entry'

Metered usage count of the WASM ribosome for this zome function.

total

438461595

over 868.251303915s

mean rate

1.10470333719e+09

/s

std

2.58147724892e+09

/s

p5

3.0252919e+06

/s

p95

8.81620559113e+09

/s

peak

1.25209588827e+10

/s

WASM usage: 'timed::get_timed_entries_local'

Metered usage count of the WASM ribosome for this zome function.

total

21330556892

over 885.879046886s

mean rate

1.29169623783e+09

/s

std

1.18485111061e+09

/s

p5

5.821020449e+07

/s

p95

3.2973756064e+09

/s

peak

3.49578150418e+09

/s

WASM usage: 'timed_integrity::entry_defs'

Metered usage count of the WASM ribosome for this zome function.

total

56289552

over 888.251359598s

mean rate

1.8996575501e+08

/s

std

4.0680588428e+08

/s

p5

326209.71

/s

p95

1.33679207869e+09

/s

peak

2.25855336019e+09

/s

Zome call duration: 'timed::created_timed_entry'

Duration of this zome call as measured by Holochain internally.

mean

0.004993

s

SD

0.001157

s

p50

0.005211

s

<

p95

0.006751

s

<

p99

0.009348

s

Zome call duration: 'timed::get_timed_entries_local'

Duration of this zome call as measured by Holochain internally.

mean

0.012235

s

SD

0.017135

s

p50

0.00444

s

<

p95

0.053074

s

<

p99

0.05804

s

WASM call duration: 'timed::created_timed_entry'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.004427

s

SD

0.001055

s

p50

0.004671

s

<

p95

0.005853

s

<

p99

0.008433

s

WASM call duration: 'timed::get_timed_entries_local'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.011687

s

SD

0.017073

s

p50

0.00375

s

<

p95

0.052388

s

<

p99

0.057353

s

WASM call duration: 'timed_integrity::entry_defs'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.00094

s

SD

0.000237

s

p50

0.000992

s

<

p95

0.001364

s

<

p99

0.001754

s

Host function call duration: '__hc__create_1'

Duration of this host function call invoked from within WASM.

mean

0.000274

s

SD

6.4e-05

s

p50

0.000274

s

<

p95

0.00039

s

<

p99

0.000509

s

Host function call duration: '__hc__create_link_1'

Duration of this host function call invoked from within WASM.

mean

0.000271

s

SD

6.6e-05

s

p50

0.000275

s

<

p95

0.000396

s

<

p99

0.000518

s

Host function call duration: '__hc__get_1'

Duration of this host function call invoked from within WASM.

mean

0.000529

s

SD

0.00011

s

p50

0.000654

s

<

p95

0.000778

s

<

p99

0.000785

s

Host function call duration: '__hc__get_links_1'

Duration of this host function call invoked from within WASM.

mean

0.000842

s

SD

0.001152

s

p50

0.000307

s

<

p95

0.003582

s

<

p99

0.003902

s

Host function call duration: '__hc__zome_info_1'

Duration of this host function call invoked from within WASM.

mean

0.001417

s

SD

0.000347

s

p50

0.001493

s

<

p95

0.002078

s

<

p99

0.002651

s

Post-commit duration

Time spent executing post-commit workflows.

mean

0.000103

s

SD

5.5e-05

s

p50

0.000104

s

<

p95

0.00026

s

<

p99

0.000268

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

46.94

s

<

mean

442.42

s

SD

254.31

s

<

p95

837.82

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

0

over 888.251418897s

Not enough time data to show a trend.

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

5.45661

s

SD

41.357859

s

p50

0

s

<

p95

0.021586

s

<

p99

256.636769

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.075262

s

SD

0.074954

s

p50

0.001272

s

<

p95

0.163117

s

<

p99

0.393822

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.008409

s

SD

0.271522

s

p50

0.000988

s

<

p95

1.040934

s

<

p99

1.119007

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.09226

s

SD

0.038422

s

p50

0.079721

s

<

p95

0.131807

s

<

p99

0.154791

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

14.486735

s

SD

15.367804

s

p50

0.002537

s

<

p95

24.362816

s

<

p99

87.044587

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

2.263683

s

SD

5.97748

s

p50

0.001203

s

<

p95

0.021961

s

<

p99

41.301619

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

0.00393

s

SD

0.002005

s

p50

0.004739

s

<

p95

0.006931

s

<

p99

0.008257

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.003174

s

SD

0.005956

s

p50

0.001375

s

<

p95

0.007004

s

<

p99

0.03848

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.000219

s

SD

6.3e-05

s

p50

0.000218

s

<

p95

0.000358

s

<

p99

0.000543

s

P2P request roundtrip: 'get'

Time spent sending a get request and awaiting its response.

mean

16.427362

s

SD

29.927863

s

p50

0.039503

s

<

p95

60.001511

s

≤

p99

60.001511

s

P2P request roundtrip: 'send_validation_receipts'

Time spent sending a send_validation_receipts request and awaiting its response.

mean

0.049013

s

SD

0.003768

s

p50

0.041494

s

<

p95

0.049177

s

<

p99

0.049401

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

51.02

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

10.56

KiB/s

<

mean

146.87

KiB/s

SD

595.07

KiB/s

<

p95

737.78

KiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

22.78

KiB/s

<

mean

77.16

KiB/s

SD

56.88

KiB/s

<

p95

195.80

KiB/s

Total bytes received

Total bytes received on primary network interface

count

3.10

GiB

mean

3.57

MiB/s

Total bytes sent

Total bytes sent on primary network interface

count

1.63

GiB

mean

1.88

MiB/s

CPU spike anomaly

CPU spike anomaly was detected

Detected

Warning CPU p99 reached 95.6%

Memory leak anomaly

Memory leak anomaly was detected

NotDetected

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

Detected

Warning Moderate swap usage (13.9% swap used)

System overload anomaly

System overload anomaly was detected

Detected

Warning 38% of hosts overloaded (load5/ncpus > 1.0)

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

27.79

%

<

mean

64.03

%

SD

22.54

%

<

p95

94.91

%

System

13.02

%

CPU percentiles

CPU usage percentiles

p50

60.48

%

p95

94.91

%

p99

95.62

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

10

hosts

mean time

587.28

s

Memory used percentage

Percentage of memory used

p5

6.59

%

<

mean

17.39

%

SD

8.03

%

<

p95

33.04

%

Memory available percentage

Percentage of available memory

p5

66.96

%

<

mean

82.61

%

SD

8.03

%

<

p95

93.41

%

Max host memory used

Maximum memory usage percentage across all hosts

max

36.50

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

13.87

%

Memory growth rate

Rate of memory growth over time

growth

0.24

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.01

MiB/s

Disk write throughput

Disk write throughput in MiB/s

97.58

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /

0/2

hosts

Mount Point /efi-boot

0/2

hosts

Mount Point /etc/hostname

0/12

hosts

Mount Point /etc/hosts

0/12

hosts

Mount Point /etc/nomad.d/client-meta.json

0/12

hosts

Mount Point /etc/resolv.conf

0/12

hosts

Mount Point /nix/store

0/2

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

2.89

5 min

2.87

15 min

2.70

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

38.46

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

0

%

<

mean

13.4923

%

SD

12.9223

%

<

p95

36.07

%

60 second average

13.4993

%

300 second average

14.0193

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

2.315

%

SD

6.7339

%

<

p95

21.47

%

60 second average

2.4264

%

300 second average

2.5888

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

1.8093

%

SD

5.9572

%

<

p95

18.74

%

60 second average

1.9037

%

300 second average

1.9929

%

Holochain process CPU usage

CPU usage by Holochain process

p5

27.08

%

<

mean

57.28

%

SD

23.53

%

<

p95

93.68

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

232.27

KiB

<

mean

309.94

KiB

SD

58.76

KiB

<

p95

421.55

KiB

Holochain process threads

Number of threads in Holochain process

p5

31

threads

<

mean

33.38

threads

SD

4.53

threads

<

p95

38

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

67

file descriptors

<

mean

75.74

file descriptors

SD

10.71

file descriptors

<

p95

83

file descriptors

Zome Call, Single Value

↑ Back to index

Calls a zome function that returns a fixed value. This tests the maximum performance of zome calls when the zome function is not calling into Holochain other otherwise doing any work.

Started

Thu, 17 Sep 2026 09:13:36 UTC

Duration

300s

Peer count

1

Peer count at end

1

Behaviours

default (1 agent)

Holochain version

0.7.0

Wind Tunnel version

0.7.0

Run ID

zome_call_single_value_35182846570_1

Call timing

The time taken for the call to complete.

mean

0.003541

s

SD

0.002285

s

p50

0.002837

s

<

p95

0.00705

s

<

p99

0.012052

s

Call rate

The number of zome calls completed per second.

mean

2786.93

/s

Error count

The number of errors accumulated during the run.

0

Holochain MetricsClick to toggle visibility.

WASM usage: 'return_single_value::get_value'

Metered usage count of the WASM ribosome for this zome function.

total

307889978

over 280.001987383s

mean rate

1.0995992e+06

/s

std

414771.73

/s

p5

528132.37

/s

p95

1.76670748e+06

/s

peak

1.77190276e+06

/s

Zome call duration: 'return_single_value::get_value'

Duration of this zome call as measured by Holochain internally.

mean

0.001435

s

SD

0.000128

s

p50

0.001582

s

<

p95

0.001676

s

<

p99

0.001679

s

WASM call duration: 'return_single_value::get_value'

Duration of inner WASM calls for this function, excluding Holochain overhead.

mean

0.000899

s

SD

8.8e-05

s

p50

0.000998

s

<

p95

0.001068

s

<

p99

0.00107

s

Post-commit duration

Time spent executing post-commit workflows.

mean

8.2e-05

s

SD

0

s

p50

8.2e-05

s

≤

p95

8.2e-05

s

≤

p99

8.2e-05

s

Conductor uptime

Conductor uptime gauge. A drop in the trend indicates a restart during the run.

p5

17.14

s

<

mean

147.15

s

SD

83.67

s

<

p95

277.15

s

Integrated ops

Count of DHT ops integrated since the conductor started. Resets on restart.

total

0

over 280.001991759s

Not enough time data to show a trend.

Integration delay

Delay between an op being stored and being integrated. High values indicate the pipeline is falling behind.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Validation attempts per op

Number of validation attempts required per op. Values consistently above 1 indicate retries.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

App validation workflow duration

Time spent running the app validation workflow.

mean

0.001143

s

SD

0

s

p50

0.001143

s

≤

p95

0.001143

s

≤

p99

0.001143

s

Countersigning workflow duration

Time spent running the countersigning workflow.

mean

0

s

SD

0

s

p50

0

s

≤

p95

0

s

≤

p99

0

s

Integrate DHT ops workflow duration

Time spent running the integration workflow.

mean

0.001623

s

SD

0

s

p50

0.001623

s

≤

p95

0.001623

s

≤

p99

0.001623

s

Publish DHT ops workflow duration

Time spent running the publish workflow.

mean

0.005372

s

SD

0.001152

s

p50

0.005718

s

<

p95

0.00762

s

≤

p99

0.00762

s

System validation workflow duration

Time spent running the sys validation workflow.

mean

0.001736

s

SD

0

s

p50

0.001736

s

≤

p95

0.001736

s

≤

p99

0.001736

s

Validation receipt workflow duration

Time spent running the validation receipt workflow.

mean

0.002689

s

SD

0

s

p50

0.002689

s

≤

p95

0.002689

s

≤

p99

0.002689

s

DHT DB connection use time

Time spent holding DHT database connections.

mean

8.1e-05

s

SD

1.4e-05

s

p50

9.8e-05

s

<

p95

0.000111

s

<

p99

0.000113

s

Write transaction duration

Duration of exclusive write transactions across all databases.

mean

0.001277

s

SD

0.000437

s

p50

0.001878

s

≤

p95

0.001878

s

≤

p99

0.001878

s

Lair keystore: signing

Duration of Ed25519 signing requests to the Lair keystore.

mean

0.00031

s

SD

0

s

p50

0.00031

s

≤

p95

0.00031

s

≤

p99

0.00031

s

Host MetricsClick to toggle visibility.

User CPU usage

CPU usage by user space

54.16

%

Network receive rate (primary)

Rate of bytes received on primary network interface

p5

18.49

KiB/s

<

mean

1.40

MiB/s

SD

2.14

MiB/s

<

p95

7.69

MiB/s

Network send rate (primary)

Rate of bytes sent on primary network interface

p5

14.66

KiB/s

<

mean

44.85

KiB/s

SD

41.37

KiB/s

<

p95

150.83

KiB/s

Total bytes received

Total bytes received on primary network interface

count

391.56

MiB

mean

1.40

MiB/s

Total bytes sent

Total bytes sent on primary network interface

count

12.26

MiB

mean

44.85

KiB/s

CPU spike anomaly

CPU spike anomaly was detected

Detected

Warning CPU p99 reached 97.0%

Memory leak anomaly

Memory leak anomaly was detected

NotDetected

Disk full anomaly

Disk full anomaly was detected

NotDetected

Swap thrashing anomaly

Swap thrashing anomaly was detected

NotDetected

System overload anomaly

System overload anomaly was detected

NotDetected

CPU usage

Total CPU usage and kernel CPU usage

Total

p5

34

%

<

mean

71.89

%

SD

21.35

%

<

p95

95.49

%

System

17.74

%

CPU percentiles

CPU usage percentiles

p50

75.80

%

p95

95.49

%

p99

97.04

%

CPU usage above 80%

Number of hosts above 80% CPU and mean time spent above threshold for those hosts

count

1

hosts

mean time

139.29

s

Memory used percentage

Percentage of memory used

p5

7.06

%

<

mean

9.31

%

SD

1.59

%

<

p95

11.9

%

Memory available percentage

Percentage of available memory

p5

88.1

%

<

mean

90.69

%

SD

1.59

%

<

p95

92.94

%

Max host memory used

Maximum memory usage percentage across all hosts

max

9.31

%

Max host swap used percentage

Maximum swap space usage percentage across all hosts

max

0.00

%

Memory growth rate

Rate of memory growth over time

growth

-3.60

MiB/s

Disk read throughput

Disk read throughput in MiB/s

0.06

MiB/s

Disk write throughput

Disk write throughput in MiB/s

6.42

MiB/s

Disk space utilization risk

Number of hosts nearing disk space capacity by mount point

Mount Point /etc/nomad.d/client-meta.json

0/1

hosts

System load average

System load averages over 1, 5, and 15 minutes. This is an unnormalised value not divided by number of CPUs, so it is only meaningful if all machines have the same core count.

1 min

5.40

5 min

3.94

15 min

1.96

CPU overloaded hosts

Percentage of hosts that experienced CPU overload

0.00

%

CPU pressure

CPU pressure over 10, 60, and 300 second averages

10 second average

p5

2.58

%

<

mean

31.7369

%

SD

18.9033

%

<

p95

57.94

%

60 second average

33.9083

%

300 second average

23.4179

%

Memory pressure some

Memory pressure (some tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

Memory pressure full

Memory pressure (all tasks stalled) over 10, 60, and 300 second averages

10 second average

p5

0

%

≤

mean

0

%

SD

0

%

≤

p95

0

%

60 second average

0.0000

%

300 second average

0.0000

%

I/O pressure some

I/O pressure (some tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

0.0745

%

SD

0.267

%

<

p95

0.51

%

60 second average

0.0807

%

300 second average

0.0334

%

I/O pressure full

I/O pressure (all tasks stalled) over 10, 60 and 300 second averages

10 second average

p5

0

%

<

mean

0.0203

%

SD

0.0748

%

<

p95

0.14

%

60 second average

0.0193

%

300 second average

0.0034

%

Holochain process CPU usage

CPU usage by Holochain process

p5

14.13

%

<

mean

22.41

%

SD

5.9

%

<

p95

32.46

%

Holochain process memory (PSS)

Proportional Set Size memory of Holochain process

p5

139.24

KiB

<

mean

303.61

KiB

SD

329.07

KiB

<

p95

1.04

MiB

Holochain process threads

Number of threads in Holochain process

p5

25

threads

<

mean

45.1

threads

SD

62.76

threads

<

p95

189

threads

Holochain process file descriptors

Number of file descriptors used by Holochain process

p5

59

file descriptors

<

mean

101.59

file descriptors

SD

133.64

file descriptors

<

p95

413

file descriptors

App Install

App Install

DHT Sync Lag

First Call

Full-Arc Create (Validated) / Zero-Arc Read

Local Signals

Mixed Arc get_agent_activity

Mixed Arc must_get_agent_activity

Remote Call Rate

Remote Signals

Single-write, many-read

Validation Receipts

Write/get_agent_activity

Write/get_agent_activity with volatile conductors

Write/Query

Write/Read

Write validated

Zero-Arc Create and Read

Zero-Arc Create Data

Zome Call, Single Value

Mixed Arc `get_agent_activity`

Mixed Arc `must_get_agent_activity`

Write/`get_agent_activity`

Write/`get_agent_activity` with volatile conductors