Lesson 3: Daml Application Performance

Captions as YouTube publishes them, usually machine-generated, so names such as Daml and Canton can be misheard.

1. 0:00

   multi-party applications cut across multiple sync domains that may span several organizational boundaries how can you design these applications to ensure that they meet the expected performance requirements of the end user this lesson will first look at some of the key factors that impact damil application performance then cover a few techniques to help you build performant applications running on a distributed Ledger the first question is what is a

2. 0:30

   performance application a centralized application using traditional databases can technically process millions of messages per second at sub-second latencies the term transaction in this context means the number of read or write operations on a database however it can often take hours sometimes even days when a business transaction needs to cut across organizational boundaries due to integration and Reconciliation challenges

3. 1:00

   an example of such a business transaction is a delivery versus payment workflow between two or more organizations on the other hand for multi-party applications running on distributed Ledger Technologies or dlts conventional metrics do not offer a compelling picture dlts have considerably higher latencies than traditional databases the key difference is how to define performance a centralized application's performance is measured in terms of database reads

4. 1:32

   or writes messages or actions that are part of a larger business transaction within an organization the performance in a dlt's context is defined in terms of an end-to-end inter-organizational business transaction throughput which takes just a few seconds instead of hours or days the criteria for benchmarking DLT performance should be in terms of end-to-end business transaction throughput and business transaction latency not just transaction throughput

5. 2:02

   and latency since an end-to-end business transaction spans the entire Canton Network you must analyze the resources used across all the components of a Canon Network to design performant demo applications to understand the factors that influence the performance of a transaction let us take a simple scenario as an example a bank creates a proposal for Alice to create a bank account Alice can accept the proposal in which

6. 2:33

   case a new contract is created if she declines The Proposal then no action is taken here is the code that models the scenario the bank account template has a create proposal Choice which creates the account proposal the accept proposal choice in the account proposal template creates a bank account with the owner as the customer given this code here is the script that plays out the scenario Alice and Bank are two parties the bank submits the

7. 3:04

   action to create an account proposal for Alice and then Alice exercises the accept proposal choice running this script generates the transaction tree shown here a transaction has two types of elements that impact its performance first is the number of actions on The Ledger such as create exercise fetch or look up second is the payload that the transaction carries with it in this example tx0 has one create action and two data elements

8. 3:35

   in its payload bank and customer tx1 has two actions exercise and create it has three data elements in its payload bank account owner and balance tx1's transaction size is bigger than tx0 let us now look at the resources a transaction needs there are two types of resources the first type is the hardware and software resources that are available across the Canton Network the second

9. 4:08

   type of resource is the contracts that are created exercised upon and archived within a business transaction to understand how a transaction uses hardware and software resources you can trace its flow from the point it is submitted on one participant node to the point it is committed at all the relevant participant nodes in the Canton Network a transaction starts with the submitting node interpreting encrypting and submitting the sub-transactions to The Domain

10. 4:39

    the sequencer in the domain sequences these messages timestamps and multicasts them to the recipient nodes these nodes receive their respective sub-transactions validate them and send their confirmation back to the domain where the mediator Aggregates these confirmations and informs the sequencer the sequencer then multicasts confirmations to all the participating nodes which then commit the transaction in their local PCS in this way the

11. 5:10

    Canton protocol ensures atomicity of transactions and preserves privacy of data that flows across the network now let us look at the hardware and software usage across these steps in this table The Ledger transaction steps are listed in the leftmost column and the hardware and software resources of various Canton Network components are listed in the top three rows there are four main groups of resources first the submitting nodes CPU memory and its

12. 5:42

    private contract store or PCS second the sequencer CPU memory and the sequencer backend third the receiving participant nodes CPU memory and PCs and finally the network bandwidth for all these components starting with interpretation done at the submitting participant node the factors that drive up resource consumption are the complexity of transactions that drives up the CPU usage

13. 6:13

    the number and size of variables that drive up the memory usage the number of contracts referenced in the code that need to be fetched from The Ledger that drives up the read write operations on the PCS remember that a fetch action is also an update on The Ledger and indirectly adds to the transaction size and finally the number of views that need to be interpreted which consume CPU Cycles after interpretation comes the step of

14. 6:43

    blinding or encryption which is also a CPU and memory intensive operation the driving factor is the number of views because each view needs to be encrypted separately leading to CPU and memory consumption once encrypted the views are submitted to the sync domain this step requires serializing at the submitting node and deserializing at the sequencer leading to CPU and memory consumption at both ends the driving factor for consumption

15. 7:15

    of these resources as well as Network bandwidth is the transaction size as shown in the example before the transaction size is the size of its transaction tree and the size of its payload when the sequencer processes the messages the transaction size and number of views once again drive up the consumption of the sequencers back in storage and network bandwidth similarly the resource consumption in the validation and Confirmation steps is driven by factors such as transaction

16. 7:47

    size computation complexity number of contract fetches and the number of parties involved looking at all these driving factors together you can see that the CPU and memory on each of these components can be scaled up to improve performance the number of views and the number of validators depends on the nature of the business transaction and the number of parties involved which in turn depends on the business process that leaves the number of fetch

17. 8:18

    operations and the transaction size both these factors can be optimized in a demo application by using good programming practices to improve performance to understand how to reduce the number of fetches you need to see where you execute a fetch operation all the functions listed here fetch fetch by key and look a bikey are fetch operations a fetch is also carried out when you exercise a choice because it is exercised against a contract ID which

18. 8:50

    must be fetched a fetch is a ledger update operation which leads to a read as well as write operation on The Ledger for the transaction size the higher the number of functions use such as fetch create archive or exercise a choice the larger the transaction tree and the transaction size with all these characteristics and constraints what coding techniques can you use to improve the performance of your DML application the first technique is to reduce the number of times you read from or write

19. 9:24

    to The Ledger as much as possible one way is to skip writing any intermediate states of a contract to The Ledger for example if a choice requires a series of calculations that updates a field in the contract then write the contract to The Ledger only when the final value is calculated rather than writing the contract after every intermediate calculation second analyze the volatility of the fields in a template especially those that have a large payload a template

20. 9:55

    with a large payload can consume a lot of resources for operations such as database read and write encryption and decryption serialization and deserialization it also consumes Network bandwidth look for fields that almost never change versus those that have a high churn split the template into two one containing the relatively stable fields and the other containing the ones with high churn this way you will avoid unnecessarily carrying the stable fields in every

21. 10:27

    transaction thereby reducing the transaction size the third technique is batching if you have the same business operation being performed multiple times leading to multiple Ledger transactions then you should combine them together to be done in one Ledger transaction for example here Alice submits five business transactions in five separate do blocks which results in five Ledger updates this means that the steps from

22. 10:56

    interpretation to blinding submitting sequencing validating confirming and committing will be performed five times since these steps require accessing the DB and sending data over the network having them combined into one transaction will reduce the DB read write operations as well as Network traffic this is called batching however the cons of this approach is that the transaction latency of a batch transaction will go up marginally since

23. 11:28

    all transactions need to complete their processing cycle to be committed at one time also since all these transactions are grouped in one do block if any one of them fails all of them will fail however you can start with the assumption that you design your application so that transaction failures are an exception and not the norm so far this lesson covered techniques to overcome the performance constraints imposed by the hardware and software in a Canton Network the second type of resource that can

24. 12:02

    cause significant performance bottlenecks is a contract demo applications work with smart contracts that are immutable which makes them a consumable resource when you exercise a consuming choice on contracts they are consumed the advantage of managing contracts as a consumable resource is that it avoids the problem of double spend however the disadvantage of a contract being a resource is that while a ledger transaction is atomic it does have some latency

25. 12:35

    in distributed ledgers a transaction latency's order of magnitude is a few seconds in those few seconds when a transaction is sequenced the contracts to be consumed in it are in a pending status during this time there may be other transactions wanting to consume the same contract which will likely fail since the contract is not available this problem caused by multiple transactions trying to consume the sync contract is called contention

26. 13:06

    so how can you avoid contention to some extent contention is unavoidable in distributed Ledger systems however there are some best practices that can help minimize contention one approach is to use the concept of utxo or unspent transaction output in which the total balance of a digital asset is not stored in a single contract with large value but in a pool of individual tokens with small amounts that can be used independently of each other

27. 13:38

    think of it as keeping 100 coins instead of one dollar bill so that when different transactions want to consume a few cents each they block only those few cents and not the whole dollar this approach also makes these transactions independent of each other that means that you can even parallelize them in the client application giving a significant boost to Performance a second technique is to have the parties submit a request for their transactions to an operator party that

28. 14:09

    bundles all the requests checks for all the contracts to be consumed and then settles them in a batch let us take an example imagine you have an election contract where 100 parties vote between yellow and green each of the parties has a choice on the election which stores their vote on election if all 100 parties cast their vote concurrently only one of the 100 votes will go through instead of multiple parties trying to use a contract directly design your

29. 14:41

    application in such a way that they can submit a request to another party who has the authority to group and submit them in one batch thereby avoiding contention finally the batching technique discussed earlier to optimize hardware and software resource usage can also help in managing contracts as a resource batching puts the transactions that were otherwise causing contention to be

30. 15:06

    executed in a sequence thereby avoiding contention let us summarize the performance of multi-party applications should be measured in end-to-end business transaction throughput and latency there are two types of resources that can impact the performance of a damel application hardware and software resources across the Canton Network and the contracts being used in the business transactions on the hardware and software side the common factors driving resource

31. 15:36

    consumption are calculation complexity number and size of variables in those calculations number of views number of validators number of database fetches and the transaction size among these factors the number of fetches in a transaction and the transaction size can be optimized through damil programming techniques the lesson discussed three techniques to handle these two factors first skimp rating any intermediary states of the contract to The Ledger

32. 16:07

    second if a template has a large payload try to split it into two separating the non-volatile fields from the ones that have a high churn third batch the same business transaction being performed multiple times into one and submit it as one transaction the second type of resource you can manage to boost performance is a contract since contracts are consumable resources they lead to the problem of contention which impacts performance

33. 16:38

    while some contention is unavoidable you can adopt the utso approach to avoid contention another technique is to have the parties make a request to an operator party to use a contract instead of directly managing the contract the operator can then bundle the requests and submit them in a sequence to avoid contention the third technique is batching which also helps alleviate contention to a great extent