Lesson 6: Refactoring for Scalability

Captions as YouTube publishes them, usually machine-generated, so names such as Daml and Canton can be misheard.

1. 0:00

   before we dip into refactoring for scalability let's briefly review some Core Concepts we have decomposed the problem of analyzing workflows into understanding the events people decisions and consequences involved and how they are observed and authorized and we've looked at how this creates a core Loop for people to make business decisions we have analyzed an example workflow and developed a simple action-oriented design

2. 0:30

   and we have examined how contention in simple designs can limit the scalability of a distributed workflow and the principles of how to address this contention to regain a scalable workflow in this lesson we will show how to use these principles to refactor simple D designs to obtain scalability and performance for this lesson we will be considering a basic bank account model the most basic concept of a bank bank account includes two decisions the holder can deposit money and they can

3. 1:05

   withdraw money following the D design process we used with the swap market results in a single bank account template we begin with two choices deposit and withdrawal once we have a simple design such as this it is critical to consider the scaling requirements if the bank account will only be used a few times per minute then we are done this type of simple design we have developed so far will work for the vast

4. 1:34

   majority of workflows in a d application let's assume we can expect at most 5,000 transactions per working day that is a bit under 6 seconds per transaction so we are fine given our rule of THM of one transaction per second this is where it is critical to properly understand the true scaling and performance requirements of your workflow what if all almost half of those transactions arrive in the first couple of minutes of the day and almost

5. 2:05

   half just before close of business for example in the last 3 minutes that's going to mean a minimum of 20 transactions per second probably more if you want the transactions processed within those 5 minutes with reasonable latency this may be a silly scenario for a brick and mortar store example but if you think about the stock market most of the volume is traded ing Market open and close keep in mind that our model uses a single account

6. 2:36

   contract every deposit and every withdrawal will require archiving the bank account contract and creating the new updated contract in the simple design we have started with this means that each Bank transaction must wait for the previous transaction to finish so it can use the new updated contract and this will not be a problem if if the deposit and withdrawal workflows do not require significant throughput for your application from the last lesson we suggested this rule of thumb your design

7. 3:10

   should not require better than 1 second roundtrip latency to The Ledger at 20 transactions per second we would require 50 millisecond roundtrip latency to complete one transaction at a time the bank account is acting like a single cash register in a store and a single point of contention every transaction has to cue to access this is significantly faster than the rule of thumb 1 second latency mentioned earlier the bank account will

8. 3:42

   be a bottleneck and it will be impossible to meet this throughput requirement without fixing that so we will need to apply at least one of the approaches we discussed in the previous lesson recall we mentioned three techniques for scaling a down application the first was by adding more Hardware a lot of D application scalability comes from vertical and horizontal scaling of the participant and synchronizer domain nodes the other

9. 4:11

   two approaches include allowing multiple instances of the workflow to run simultaneously that means allowing multiple cache registers to operate simultaneously allowing batch workflow instances to share the same transaction which allows multiple it items to be sold in a single transaction adding more Hardware is done in one of two ways vertical scaling is literally just using bigger computers more memory more

10. 4:43

    CPUs while at the end of the project this may well help boost the performance of an already scalable application or after release to support a successful and growing application keep in mind that if your application design does not utilize the resource resources it already has adding more will not help much at the start of the project the goal is to build the sort of scalable design that can later benefit from this sort of boost when it comes to utilizing more Computing

11. 5:13

    resources not all designs are equal horizontal scalability involves adding more computers as nodes approach capacity adding more nodes to share the burden internally the D platform is designed to scale well this way but again your application design matters no amount of extra Hardware changes the fact that a d application is a distributed application and there is a minimum number of network connections required to commit a single

12. 5:45

    transaction this is why we have the 1C roundtrip rule of thumb so your application design matters if we want to unlock the full potential of the Dale platform the design approach to support SK scalability focuses on allowing the application to fully exploit any resources available to it and fall into one of two categories allow multiple instances of the workflow to run simultaneously and allow batches of operations to share the same Ledger

13. 6:17

    transaction if we return to our store cash register analogy this equates to allowing multiple cach registers to operate simultaneously and allowing multiple items to be sold in a single transaction let's return to the bank account example when we look at a typical bank account we see a couple of things the bank account has a bunch of Providence and business metadata account holder account ID and know your customer status these either do not change or

14. 6:51

    only rarely data is rarely updated is ideally suited to the sort of simple default workflow design we looked at before the bank account also has a balance field that changes extremely often with almost every business operation in a very real sense the contention preventing us from meeting our scaling requirement is not on the bank account contract but on this balance field the beginning of a project with a new design and no Downstream code

15. 7:25

    depending on this centralized bank account contract yet is the best time to start addressing this contention a good way to start addressing this is by splitting the bank account contract to reflect these different update patterns the standard approach to doing this has acquired the name unspent transaction output or utxo to do this we split these two sets of fields the account data from the balance data critically we do not

16. 7:55

    require that there be only one single centralized contract tracking the account balance instead we allow multiple account funds contracts and we Define the balance to be the sum of all balances for all account funds contracts associated with a bank account the deposit and withdrawal choices are still there on the bank account contract but we change them to reflect the new design deposits still does any business validation required to accept the

17. 8:26

    deposit but instead of updating a single contract it just creates a new account funds contract with the balance deposited by doing this deposit is now completely independent of any other operation as there is no contention on creating a new funds contract withdrawal also does the same business validations it was doing but now it takes a set of account fund contracts sufficient to cover the withdrawal it then archives them

18. 8:56

    creating a new account fund contract for any excess funds provided by doing this the withdrawal contract is now independent of any other operation that does not also try to Archive the same account funds ensuring this doesn't happen will require some engineering but is ultimately a manageable implementation detail there is no such thing as a free lunch so doing this does have some Downstream impacts we no longer have a single unitary balance so some

19. 9:29

    accounting or reporting workflows may now require more work you now need logic to select a suitable set of funds contracts to satisfy each withdrawal the solution is more complex than the simple design but if you have identified this as a place you need to scale this is necessary complexity but there are also benefits This lends itself nicely to supporting reserving funds escrow or partial locking of funds for other work workflows this also can allow a workflow

20. 10:02

    to prove the existence of funds without having to divulge the full details of an account or its full balance ultimately this is a very flexible design that can be adapted to a lot of different business workflows with minimal impact on the existing scaling and performance behavior of the system the scalability of this solution is only bounded by the maximum throughput of your Canton

21. 10:27

    deployment which will comfortably meet the 20 transactions per second we were aiming for the alternative approach is batching we start by showing the deposit and withdrawal choices as contracts on The Ledger now instead of exercising a deposit choice we create a deposit request contract and similarly we replace the withdrawal Choice with a withdrawal request contract in our cash register analogy creating one of these contracts corresponds to the the idea of

22. 10:59

    submitting our sale to the cashier batching transactions we then have to replace the old deposit and withdrawal choices with a single perform transactions choice that takes a set of requests and applies them to the account in a single transaction the bank then needs to run an automated process that collects batches of deposit and withdraw requests and submit them to the performed transactions choice on the appropriate bank account if we can run 20 transactions per batch which is

23. 11:32

    certainly possible then we can meet our scaling requirements batching has significantly more impact on our design than utxo in particular the deposit and withdrawal choices are no longer Atomic they are now subw workflows requiring a transaction to make the request a transaction to process the batch and a transaction to continue the request now that the deposit or withdrawal is complete this is going to complicate any joint workflow for example guaranteeing

24. 12:05

    a matching deposit and withdrawal is now going to include some form of synchronization process to ensure consistency Dale makes this much easier than conventional systems but you will need to include this process explicitly in your Dam model on the other hand batching does have two substantial benefits up to a point batching is tolerant of single points of contention it is therefore the first choice of optimization if you realize late in the

25. 12:37

    project that you have a performance bottleneck in your design you can tune the batch size to maximize your resource utilization if you aren't using batching you can almost always get more performance out of the same Hardware by introducing batching so either the utxo method or batching can get us to 20 trans actions per second we will require both if we are asked to scale to 2,000 the issue is twofold for any given Canton deployment

26. 13:10

    there will be both a maximum transaction size and duration before the system will start to reject them to protect its own stability also for a given node there is a maximum transaction throughput that is only partially affected by transaction size and duration this means that for a given deployment the first limitation is a maximum batch size we can accommodate before the resulting transaction is just too large to commit to The Ledger and the second limitation is a maximum

27. 13:42

    number of independent transactions we can handle regardless of how small and lightweight we make them so we will combine both approaches courtesy of our use of utxo we know these transactions are independent of each other so we don't need to change that part of our model then we add the deposit and withdrawal requests from the batching design only we Implement them in terms of the regular utxo deposit and withdrawal

28. 14:12

    choices this has various names here we will call it an account operation bundle finally we add a second way to process transactions via a helper contract that can accept lists of account operation bundles along with required supporting contracts as it is a single Choice exercising it will combine the entire batch of bundles into a single transaction this then allows a single transaction to process a batch of dozens or hundreds of account operations using

29. 14:45

    the unmodified utxo choices doing this solves the problems we had with batching as we still have access to the same horizontal scalability and atomicity guarantees we had with the the utxo model when we need it and we also gain access to improved vertical utilization through access to batching this is pretty much just the utxo model but faster the only downside is the added complexity required to handle the bundling of requests and the

30. 15:18

    absolute requirement that they are in fact independent of each other this combination of the utxo method and bundling of requests will satisfy by all but the most extreme throughput requirements so how would we handle 20,000 transactions per second at this point the sheer computational resources needed to evaluate 20,000 danal choices perform the cryptographic calculations the extreme number of IO and network operations per second exhaust the

31. 15:51

    capacity of any computer we can deploy to run them we will have no option but to use a full comp combination of vertical and horizontal scaling along with a design focused on making this possible to reach this sort of scale you will need to Shard your application over multiple participant nodes or depending on your application Shard across multiple domains this will require careful redesign of the Dale model to allow this part of the application to be split into

32. 16:23

    multiple independent shards each of the shards can be run in isolation on Independent participant nodes this will require engineering and business analysis to work closely together to ensure the requirements allow this sort of split and to accommodate the impact this sort of arrangement will have on other workflows in the application with this type of scale early design and architecture decisions are critical we strongly recommend you contact digital asset for

33. 16:54

    advice as this will require careful upfront planning not surprisingly aiming for this sort of scale will have a major impact on the application architecture the Dale models and a substantial amount of engineering and Analysis required to accommodate it on the other hand the scaling potential of sharding is only bounded by your ability to deploy more Computing resources and your ability to keep the application shards truly independent of course Computing

34. 17:26

    resources are not infinite but with that caveat there is literally no upper bound to the scalability you can get out of a Down application using these techniques over this course we have taken a journey that has started with the tools to analyze and conceptualize your business through the lens of the decisions that drive value this analysis was then mapped onto First Class D Concepts that directly support the authorization and visibility constraints

35. 17:56

    that arise where your workflows cross trust boundaries we then showed how an action-oriented analysis can be developed into a detailed design that seamlessly transitions from analysis through design and into implementation the resulting designs provide a flexible and effective foundation for the Dale models to support your application for the handful of performance critical designs we introduce the concepts and strategies required to refactor your designs to

36. 18:27

    unlock the scalability of the dkin platform and the criticality of considering these matters early in your project there is much more to developing a comprehensive application which includes front-end and back-end integration Automation and operations upgrades and application life cycle support we encourage you to Avail yourself of our other training offerings in these areas as you explore these other subjects we hope you will keep in

37. 18:57

    mind at the heart of every D application is a D model of your workflow Focus the value it empowers you to make