App Install
Measure app installation speed. This scenario has two roles:
minimal: Installs a simple app which implements initialisation callbacks but otherwise doesn't contain a lot of code.large: Installs a larger app that contains some dependencies and some generated data as padding to make the bundle larger.
These are intended to catch a compilation slowdown or issues with copying around large WASMs (e.g. accidental cloning or tracing). In each case, the behaviour will uninstall the app it installed so that it can re-install it on the next iteration.
-
minimal(1 agent)
App Install
Measure app installation speed. This scenario has two roles:
minimal: Installs a simple app which implements initialisation callbacks but otherwise doesn't contain a lot of code.large: Installs a larger app that contains some dependencies and some generated data as padding to make the bundle larger.
These are intended to catch a compilation slowdown or issues with copying around large WASMs (e.g. accidental cloning or tracing). In each case, the behaviour will uninstall the app it installed so that it can re-install it on the next iteration.
-
large(1 agent)
DHT Sync Lag
Measure lag time between an agent publishing data and other peers being able to see it. This scenario has two roles:
write: A simple job that just creates entries with a timestamp field. Those entries are linked to a known base hash. For each write, the metricws.custom.dht_sync_sent_countis incremented.record_lag: A job that repeatedly queries for links from the known base hash. It keeps track of records that it has seen and when a new record is found, and calculates the time difference between the timestamp of the new record and the current time. That time difference is then recorded as a custom metric calledwt.custom.dht_sync_lag.
After each behaviour loop the metric ws.custom.dht_sync_recv_count is incremented.
-
record_lag(1 agent) -
write(1 agent)
agent:uhCAkQKH_sPxt19x5eyLwMGJq3zmifjWUQ9NlLgaRvHx1ugjo_vDG
agent:uhCAkRwC40egSFRBI_RVisZp1y_MD29GK_EV5mKUwYtm90FqUr91t
agent:uhCAkkqrWI_gP24sayJSqalI-xTzbH4eyc1vJp4RVW7vozYyGboDN
agent:uhCAkpHWQkDzffDeheMjcRC9p6ESC1k8WdGZdKrNVfTnl8KnCPXgz
agent:uhCAkvSaazvuBj5QAg83TddnESvh4tUANugx2_EIRT2e05gnM3WW4
agent:uhCAkJb4NKAAJY8m9UOREXA6lo2VgPtBeKh7fktEtAsrt7-fvDB8N
agent:uhCAkYpVp1RkHGvAnx3xmcv7R_EBz2fcDOM5teTIXBtHmpMx071kC
agent:uhCAk_euKRpIcot4mpUKFVax6j9XUbbQtbSApPqRdAtqe-lxILBQ3
agent:uhCAkiMgrONTVI6l8BarnYUYto85nPZ_yuVzv3H9-cqfKmHEwJBjS
agent:uhCAkJb4NKAAJY8m9UOREXA6lo2VgPtBeKh7fktEtAsrt7-fvDB8N
agent:uhCAkYpVp1RkHGvAnx3xmcv7R_EBz2fcDOM5teTIXBtHmpMx071kC
agent:uhCAk_euKRpIcot4mpUKFVax6j9XUbbQtbSApPqRdAtqe-lxILBQ3
agent:uhCAkiMgrONTVI6l8BarnYUYto85nPZ_yuVzv3H9-cqfKmHEwJBjS
First Call
This scenario installs a simple app which implements the init callback, and measures the time elapsed between start of install and calling init. It will uninstall the app it installed so that it can re-install it on the next iteration. This is required to re-run the initialisation callback.
- Default (1 agent)
init.No scenario template for full_arc_create_validated_zero_arc_read!
Local Signals
This scenario is for testing the speed and reliability of local signals. It uses a zome with a function that will emit 10,000 signals. The scenario measures how long it takes to send the signals and how many have been received by the time the zome call ends.
- Default (1 agent)
No scenario template for mixed_arc_get_agent_activity!
Mixed Arc must_get_agent_activity
A scenario where write agents create entries in batches of 10, while must_get_agent_activity agents each pick a random write agent and repeatedly attempt to create an entry that references the chain top of their latest batch. This reference means that the entry's validation function needs to make a must_get_agent_activity call.
The purpose of this scenario is to measure the time it takes for published agent activity data to be gossiped among authorities and become available to peers that query it via must_get_agent_activity.
This test is similar to Write Validated must_get_agent_activity, but some write agents are zero-arc and all must_get_agent_activity agents are zero-arc. This forces the must_get_agent_activity host call to go to the network.
-
full_write(1 agent) -
zero_must_get_agent_activity(1 agent) -
zero_write(1 agent)
action_seqaction_seq of a write agent successfully retrieved by a must_get_agent_activity agent. This reflects the DHT's ability to propagate agent activity ops and make them available to querying peers.write agent's creation of a batch and a must_get_agent_activity agent's successful discovery of the batch and creation/self-validation of a new entry that depends on it.must_get_agent_activity_agent:uhCAk44G1goteuB4CRgnQZ-kHTOL24a1z_vzSH5-TrJyM_3S5bJo6
must_get_agent_activity_agent:uhCAkcjRiKN91bi6oEII04YTFN9oQjqAuPMhHXMMxUrLNuO1i2NFb
must_get_agent_activity_agent:uhCAkdziXbFjSyrG3zUs9E4KnmDE9RtgyMIA6pip0ozz0Dol78Vjy
must_get_agent_activity_agent:uhCAkg2C0i7zOZNVzrSE7xafGTFfYDtSD7q8Bh1H2s24vrmKH0fmG
must_get_agent_activity_agent:uhCAkmQ5NwCMKNjPw7PleWKvaYvHvfl6ZlhTuj-RiybWJdrhrB4jN
must_get_agent_activity agent was able to discover batches and create/self-validate new entries that depend on them.must_get_agent_activity_agent:uhCAk44G1goteuB4CRgnQZ-kHTOL24a1z_vzSH5-TrJyM_3S5bJo6
must_get_agent_activity_agent:uhCAkcjRiKN91bi6oEII04YTFN9oQjqAuPMhHXMMxUrLNuO1i2NFb
must_get_agent_activity_agent:uhCAkdziXbFjSyrG3zUs9E4KnmDE9RtgyMIA6pip0ozz0Dol78Vjy
must_get_agent_activity_agent:uhCAkg2C0i7zOZNVzrSE7xafGTFfYDtSD7q8Bh1H2s24vrmKH0fmG
must_get_agent_activity_agent:uhCAkmQ5NwCMKNjPw7PleWKvaYvHvfl6ZlhTuj-RiybWJdrhrB4jN
create_validated_sample_entry zome call timingwrite agent's source chain.agent:uhCAk44G1goteuB4CRgnQZ-kHTOL24a1z_vzSH5-TrJyM_3S5bJo6
agent:uhCAkcjRiKN91bi6oEII04YTFN9oQjqAuPMhHXMMxUrLNuO1i2NFb
agent:uhCAkdziXbFjSyrG3zUs9E4KnmDE9RtgyMIA6pip0ozz0Dol78Vjy
agent:uhCAkg2C0i7zOZNVzrSE7xafGTFfYDtSD7q8Bh1H2s24vrmKH0fmG
agent:uhCAkmQ5NwCMKNjPw7PleWKvaYvHvfl6ZlhTuj-RiybWJdrhrB4jN
must_get_agent_activity calls.agent:uhCAk44G1goteuB4CRgnQZ-kHTOL24a1z_vzSH5-TrJyM_3S5bJo6
agent:uhCAkcjRiKN91bi6oEII04YTFN9oQjqAuPMhHXMMxUrLNuO1i2NFb
agent:uhCAkg2C0i7zOZNVzrSE7xafGTFfYDtSD7q8Bh1H2s24vrmKH0fmG
behaviour:full_write
behaviour:zero_must_get_agent_activity
behaviour:zero_write
Remote Call Rate
Test the throughput of remote_call operations. Each agent in this scenario waits for a certain number of peers to be available or for up to two minutes, whichever happens first, before starting its behaviour.
- Default (1 agent)
agent:uhCAkKDcEbRwiNZVaEDLtVWPDgWv-EaTrD8wKrMv7CXexiCyBTe-z
agent:uhCAkpSohH8f2Q_8Nz2s5xCMSqdWWa7WZ-NWMNKQtCKIxmYeB1UGB
agent:uhCAkKDcEbRwiNZVaEDLtVWPDgWv-EaTrD8wKrMv7CXexiCyBTe-z
agent:uhCAkpSohH8f2Q_8Nz2s5xCMSqdWWa7WZ-NWMNKQtCKIxmYeB1UGB
Remote Signals
This scenario tests the throughput of remote_signals operations.
- Default (1 agent)
Single-write, many-read
Creates an entry in the agent setup, then read one's own record back. This tests the maximum read performance for reading back local data when the chain is short.
- Default (1 agent)
read call timingread call rate (10s)Write/get_agent_activity
A scenario where write peers write entries, while get_agent_activity peers each query a single write agent's activity with get_agent_activity.
Before a target write peer and the requesting get_agent_activity peer are in sync, this will measure the get_agent_activity call performance over a network. Once a write peer reaches sync with a get_agent_activity peer, the write peer will publish their actions and entries, and so the get_agent_activity calls will likely have most of the data they need locally. At that point this measures the database query performance and code paths through host functions.
-
get_agent_activity(1 agent) -
write(1 agent)
action_seqwrite peer's source chain.get_agent_activity_full zome call timingwrite peer's source chain.agent:uhCAkY2DrBmamAnSxvz5JzpkmnB7xPZfh59nx_eOcEQLIorcyumMv
Write/Query
Creates an entry, then queries the source chain and performs a simple operation on the entries, then repeats.
- Default (1 agent)
Write/Read
Creates an entry and reads it back, then repeats.
- Default (1 agent)
Write validated
Creates an entry, updates it and links the create to the update, then repeats. Each of the three actions is validated by the included hApp.
- Default (1 agent)
Write Validated must_get_agent_activity
A scenario where write agents create entries in batches of 10, while must_get_agent_activity agents each pick a random write agent and repeatedly attempt to create an entry that references the chain top of their latest batch. This reference means that the entry's validation function needs to make a must_get_agent_activity call.
The purpose of this scenario is to measure the time it takes for published agent activity data to be gossiped among authorities and become available to peers that query it via must_get_agent_activity.
This test is similar to Mixed-Arc must_get_agent_activity, but all agents are full-arc.
-
must_get_agent_activity(1 agent) -
write(1 agent)
action_seqaction_seq of a write agent successfully retrieved by a must_get_agent_activity agent. This reflects the DHT's ability to propagate agent activity ops and make them available to querying peers.write agent's creation of a batch and a must_get_agent_activity agent's successful discovery of the batch and creation/self-validation of a new entry that depends on it.must_get_agent_activity_agent:uhCAkITNqSoIo4A94bApeANhgaYHCJPBrzIH_1rGbHdQTy-YbPcin
must_get_agent_activity agent was able to discover batches and create/self-validate new entries that depend on them.must_get_agent_activity_agent:uhCAkITNqSoIo4A94bApeANhgaYHCJPBrzIH_1rGbHdQTy-YbPcin
create_validated_sample_entry zome call timingwrite agent's source chain.agent:uhCAkITNqSoIo4A94bApeANhgaYHCJPBrzIH_1rGbHdQTy-YbPcin
must_get_agent_activity calls.agent:uhCAkITNqSoIo4A94bApeANhgaYHCJPBrzIH_1rGbHdQTy-YbPcin
Zero-Arc Create and Read
A zero-arc/full-arc mixed scenario with two types of zero-arc nodes -- ones that create data and ones that read data -- as well as full arc nodes to "relay" the data. The scenario has three roles:
zero_write: A zero-arc conductor that just creates entries with a timestamp field. Those entries are linked to a known base hash so thatzero_readnodes can retrieve them.zero_read: A zero-arc conductor that reads the entries created by the zero-arc node(s) and records the time lag between when the entry had been created and when it was first discovered.full: A full-arc conductor that is just here to serve entries to zero arc nodes.
-
full(1 agent) -
zero_read(1 agent) -
zero_write(1 agent)
agent:uhCAkIRazScmKmvBThFw-Fb2KhvUsmFxHFJDATarSkEqdCSJxWA-_
agent:uhCAkccMVnV5uJhn5rqbou_Nfk2_TYuzXVwiVo7Eq-l-OaEkcRQvC
agent:uhCAkszvGqlEJXW7JLgQQfGQWlMuAzRkbBR6TNOZ1Xugep1Pc-ehw
get_timed_local_entries zome function.agent:uhCAkiyw5K8Gucg1TsEI3ohB2LTSN7O-vUAAbVnXF8R8U2UdWey3K
agent:uhCAkn7GlJhdg-IbwwqBuQs9T-cGtbpLKnofhhtrCQ8TEl2sZOYf_
agent:uhCAkiyw5K8Gucg1TsEI3ohB2LTSN7O-vUAAbVnXF8R8U2UdWey3K
agent:uhCAkn7GlJhdg-IbwwqBuQs9T-cGtbpLKnofhhtrCQ8TEl2sZOYf_
Zero-Arc Create Data
zero: A zero-arc conductor that just creates entries with a timestamp field. Those entries are linked to a known base hash so that full-arc nodes can retrieve them.full: A full-arc conductor that reads the entries created by the zero-arc node(s) and records the time lag between when the entry had been created and when it was first discovered.
-
full(1 agent) -
zero(1 agent)
agent:uhCAkI7in3j3uX8ClgN9t0rdK3h6GTUbEaWol_eMzun4OHA8VY9qV
agent:uhCAkQQicbEoQdRCoHiX8zD60d7gUlXOTM3ztUhd3lhT2loNw5SZz
agent:uhCAkW0g7i-_gCnrayTC9qFxNtvzpl2TLChW9PYrqfF3et1bXXnBd
agent:uhCAkekl3DT8a9F1NPj5rWgU2UXt7jdvCOCTQdHoVcCOf3gajTmLR
agent:uhCAkqaZ-RJv3rzinwMdsCM9kc9sqWUMSbi_N3L2yWClOpMYNyc0S
agent:uhCAkw1xMpUsrcxBABkCCEf2Oc5xS_NdXSNZJd9APdo0l4V5xshTB
get_timed_local_entries zome function.agent:uhCAkCOQc2k_29fCxfA20dBnULBT2gP9gO_o-OxqU0RCGdvnLn4M4
agent:uhCAkDfowAxZTfWjlL7ogkexwmLVYCEjxWhXU94yTF3bgL7xbpZvj
agent:uhCAkXfqgz6RGcpVpHV2JnNK3PkNIGFeYXmw3eNGV9alpmrBoYEsT
agent:uhCAkwVx84_xplKxB5JiXrnbtDn5kxQ84RT-UXHrjRpowwVot3Kc5
agent:uhCAkCOQc2k_29fCxfA20dBnULBT2gP9gO_o-OxqU0RCGdvnLn4M4
agent:uhCAkDfowAxZTfWjlL7ogkexwmLVYCEjxWhXU94yTF3bgL7xbpZvj
agent:uhCAkXfqgz6RGcpVpHV2JnNK3PkNIGFeYXmw3eNGV9alpmrBoYEsT
agent:uhCAkwVx84_xplKxB5JiXrnbtDn5kxQ84RT-UXHrjRpowwVot3Kc5
Zero-arc Create Data (Validated)
A zero-arc/full-arc mixed scenario where zero-arc nodes create data that gets validated and full-arc nodes read the data. The scenario has two roles:
zero: A zero-arc conductor that just creates entries with a timestamp field. Those entries are linked to a known base hash so that full-arc nodes can retrieve them.full: A full-arc conductor that reads the entries created by the zero-arc node(s) and records the time lag between when the entry had been created and when it was first discovered.
-
full(1 agent) -
zero(1 agent)
agent:uhCAkCCago1xHGsfwlPi2C00DOvzSHoqwITTyIjBYbDV7Eg0D81c8
agent:uhCAkHzvxPY9eu_UMwZi_j1adCr6GnLememsloJ-bZLRQ8EKw2f7e
agent:uhCAkNxTIKtlYiQSGQYFfXwpzrcmtS2gsIJAvIA7Rg5iU4UHRkx7w
agent:uhCAkRsyQG3WC_Q2w7Np2oyRX0npPdNkn7LqIdfBPdCwBK2ViCfCO
agent:uhCAkTJfa_pAjnnGX-MeiAT-xPQ9Dio-jNEUbcMkwhCCpBBEagk1A
agent:uhCAkzRY8qgxM1wc3KAzf3EFDOEBCCrANPx3GD402ofpNCWr0qwo5
get_timed_local_entries zome function.agent:uhCAkGiIvXybdBdE_4O0ZkbYC8NIX80RhK-cP8jZ2-dGF34P0X1TN
agent:uhCAkdVtwz7sd4X8V9VVciDte2RR9y4NDYHxy-AChy4NOke-7-Hus
agent:uhCAkgT89k6ebi_1LcZym4VWcLAorbOzbd_jnMkxOo7BQMKHHxjhR
agent:uhCAkqya7k-kfopGL49EouPmyZU22qpGBYGdLh1Hb9m9St3KCHW5n
agent:uhCAkGiIvXybdBdE_4O0ZkbYC8NIX80RhK-cP8jZ2-dGF34P0X1TN
agent:uhCAkdVtwz7sd4X8V9VVciDte2RR9y4NDYHxy-AChy4NOke-7-Hus
agent:uhCAkgT89k6ebi_1LcZym4VWcLAorbOzbd_jnMkxOo7BQMKHHxjhR
agent:uhCAkqya7k-kfopGL49EouPmyZU22qpGBYGdLh1Hb9m9St3KCHW5n
Zome Call, Single Value
Calls a zome function that returns a fixed value. This tests the maximum performance of zome calls when the zome function is not calling into Holochain other otherwise doing any work.
- Default (1 agent)