Lesson 4: Lifecycle of a Daml App

Captions as YouTube publishes them, usually machine-generated, so names such as Daml and Canton can be misheard.

1. 0:04

   the sdlc of a DL app is similar to that of a web Tube app or an Enterprise service it consists of the same design and development testing deployment to operation maintenance and upgrades phases that most system development projects consist of however the architecture of a DL app is different from traditional Enterprise architecture in that the Cross organization works flows and state are specified explicitly using demo code in this lesson you will

2. 0:36

   learn what considerations to pay attention to in the different sdlc phases to account for this difference General best practices and considerations comment to all software development projects are not called out in this lesson even though they apply to DL apps the same way they apply to any other project we focus on the Delta between your past experience and what's required to successfully Implement a daml project daml provides a schema language for workflows and their state the

3. 1:09

   implementation of this schema must form a part of the design phase daml provides a unique combination of benefits which is not found in other Technologies synchronization of the application workflow State across organizational boundaries Atomic transactions across multiple applications in in addition to these two core benefits daml provides need to know privacy and independent control over their app data for app users the last

4. 1:40

   two benefits make daml uniquely suitable for Enterprise applications traditionally State synchronization across organizational boundaries is implemented using some form of messaging which is often unreliable and requires significant manual reconciliation D's virtual shared Ledger with built-in support for Atomic transactions across organizations and applications greatly simplify this kind of state synchronization this virtual shared Ledger resembles a traditional database

5. 2:12

   but it does have different functional and nonfunctional characteristics because it spans multiple organizations in terms of performance a d solution will have a higher latency and utilize more resources than a traditional database speaking more broadly using distributed Ledger technology to run Cross organization workflows has different functional and performance characteristics compared to an app

6. 2:39

   backed by a database server that runs within a single trust boundary the functional characteristics include non-repudiation data sovereignty and privacy performance characteristics include extra latency and data storage as well as mutation overhead compared to database transactions so how do you account for DL and can performance characteristics in the design of a daml app since there's a cost to using The Ledger versus a

7. 3:09

   traditional database this Cost needs to be justified not everything should be implemented on Ledger how do you know what should go on Ledger versus off ledger to decide whether some information should be stored on Ledger it's helpful to use the distinction between data and state information that represents an essential fact in the application workflow is likely to be State rather than data when this information needs to be agreed upon by

8. 3:38

   multiple parties across organizational boundaries it is beneficial to store it on Ledger The Ledger is uniquely capable of synchronizing such information across trust boundaries using a consensus mechanism information that does not represent an essential workflow fact is likely to be data just like state data may need to be shared however there are many ways to efficiently share information using web 2 Technologies given the additional cost of storing information on Ledger

9. 4:11

   compared to traditional databases there should be a higher bar for putting data on Ledger versus using a ledger as a state machine similar consideration applies to workflows a workflow that does not cross organizational boundaries does not benefit from being on Ledger it may be implemented more efficiently off Ledger one of the unique benefits and the reasons to use daml and Canton is their capability to preserve privacy to realize this essential benefit at the

10. 4:43

    design stage you need to consider who needs what data to advance their workflow how to ship that data and how to authorize access to it during the development phase you want to decompose features across the different application components and you use backwards compatible changes and build feature Flags into the application changes to avoid having to do big bang changes at deployment time backwards compatibility is also very helpful when development is distributed across teams and

11. 5:15

    organizations in the lesson on daml app architecture we learned that an app may include app user developed backends and front ends this architecture brings an additional challenge of development coordination across multiple organizations daml is a great tool to facilitate multi-organization development as it unambiguously defines the Cross organization workflows and handles the synchronization of their state out of the box it's useful to view

12. 5:45

    DL as providing an interface description language for Cross organization workflows and lead the development by creating the daml code early in the project this said cross organizational coordination is always a challenge it needs to be weighed carefully against other factors that influence the app architecture in The Same Spirit we also recommend organizing your daml code into multiple Dar files using the following guidelines modularize the workflows

13. 6:16

    according to stakeholder groups and business processes in many apps different groups of stakeholders engage in different business processes and thus interact with different parts of the smart contract model modularizing smart contracts along the lines of stakeholder groups and business processes makes upgrades easier and helps provide some privacy of smart contract code since St files only need to be distributed to the participant nodes that host the parties

14. 6:45

    involved in the workflow separate public and private apis the DL templates and their choices serve as an API specification for the workflows that are coordinated on Ledger evolving the these apis requires coordination among the development teams of the components engaged in these workflows to ensure the components continue working after the upgrade the more development teams there are the higher the coordination overhead thus you want to make sure that you

15. 7:17

    clearly call out which of your apps workflows are public integration workflows and are meant for other development teams to build on D provides support for rolling out backwards compatible changes like adding a choice or a new field to a daml template however some changes in business requirements might require you to change the structure and flow of some of your workflows by minimizing the set of

16. 7:42

    public workflows and keeping internal workflows in separate Dar files you get more freedom over changing such internal workflows to address evolving business requirements in The Same Spirit we recommend separating interface definitions in their own package separate the test code unit tests for the Danel code that implements smart contracts workflows are usually written using Danel script this DL script code is compiled into Dar files however these

17. 8:13

    Dar files play a different role from the DAR files that Implement smart contracts in their workflows Dar files containing DL script are only used for testing and should never be deployed to a participant node daml script code should never be mixed with daml templates code testing of daml apps is fundamentally no different from testing any other system for example as with any other system testing should be automated as much as possible and done at the lowest

18. 8:45

    practical level to ensure fast test execution times and minimize the test code churn this said there are some specific recommendations that came from experience of building robust damel apps starting from from the bottom of the testing pyramid DL code needs unit tests that should be white box tests written in DL script for the front end we recommend unit testing with a mocked backend and ledger to verify the display logic backend testing requires unit

19. 9:16

    tests for internal components and integration tests for internal apis that is the apis that are only used by clients under the app provider's control to test public apis as part of backend testing you want to use blackbox behavioral tests interacting at the system boundaries we recommend sharing a long running instance of Canton across test runs to avoid repeatedly paying Canton startup cost as with testing any long running system isolation of test

20. 9:48

    data between test runs is very important to isolate the tests we recommend using per test participant users and parties one way to do this is to out allocate DL parties and participant users with testr run ID appended as a suffix to parties and usernames in your test harness moving on to the top of the testing pyramid endtoend tests are a special case of integration tests that test the business workflows between multiple end users and systems across

21. 10:20

    multiple participant nodes backends and front ends where front ends are involved we recommend using test tools like selenium or playright to orchestrate multiple browser sessions from your test Code test isolation can be achieved by bootstrapping the whole system from scratch for every test run or using a long running instance of the system and a strategy to associate business workflow instances to specific test runs the lad is often useful to enable fast

22. 10:52

    test execution times and quick iteration cycles for your development teams writing robust tasks for distributed systems is challenging due to data propagation delays and concurrent execution hence special attention needs to be paid to understanding and eliminating flaky tests tests that inconsistently and incorrectly fail otherwise they will wreak havoc in development productivity when developers encounter a flake and

23. 11:21

    just rerun their tests more flicky tests are introduced creating a vicious cycle to avoid this make sure sure you dedicate some development time to eliminating flaky tests and improving test performance another challenge is the workflows that use calendar or time functions in their logic for example the life cycling of a bond that involves processing a coupon payment from the issuer to the investors on coupon payment date when testing demo code you

24. 11:52

    can use the past time function to advance time for endtoend tests that have time dependencies we recomend recommend testing using a configuration where the wait times for workflows to advance are set to milliseconds to avoid long test execution times on CI we also recommend that you pause and resume automation from the test harness to avoid race conditions performance testing is Paramount to successful daml projects and should be performed continuously

25. 12:23

    from day one you should create separate performance tests for each relevant workflow as as you develop additional workflows add them to the existing performance tests or create new performance tests for these workflows test at your target scale using synthetic data where necessary make sure that synthetic data accurately mirrors production data characteristics such as size and shape take measurements of performance characteristics and reset them between test runs to identify

26. 12:54

    regressions it is strongly suggested that you do soap testing throughout the project have a long running continuous integration deployment with the core workflow of the system running at the peak load for each core workflow this is essential for detecting and addressing performance bottlenecks in the core workflows early you also want to set up alerting to learn when your long running

27. 13:17

    system breaks this alerting requires tuning or adjustments over time to ensure the right observability of the system we recommend doing these adjustments as part of the development as as that well-tuned alerting can then be reused in the operations phase to get notified when the system is unhealthy the deployment topology of a DL app is determined by the app architecture discussed in a previous lesson however in all cases we assume

28. 13:46

    Canton infrastructure including app providers and app users participant nodes is already set up at their respective administrative domains in all scenarios the shared Dar files need to be deployed on on all Canton participant nodes damel code serves as the API definition for the state and workflows synchronized across the participant nodes of the app users this is analogous to Proto files for a grpc server that need to be shared with the developers of grpc client apps we recommend storing

29. 14:20

    the DL code in a repo separate from the backend and frontend code and share with the app user organizations a tarball of this repo or read only access to the whole repo This Way app user organizations can review the source code and build it themselves to ensure that they are confident with the behavior of the DAR file they need to install on their participant nodes the app provider also always needs to deploy their backend and front end and configure the

30. 14:50

    front end to integrate with their IM IM each app user always needs to deploy their front end and configure it to integrate with their own I am independently of whether the front end is developed by the app provider or the app user and if the app users have their own backend they need to deploy that too since DL apps run on a distributed multi-party platform a level of cross organizational coordination is always required for deployment the less components that need to be deployed by the app users the less coordination is

31. 15:25

    required and the easier it is to deploy the app this said every possible architecture has its trade-offs which were discussed in detail in a previous lesson on deap architecture standard operational considerations apply to daps the two that are worth recalling are logging and monitoring to have good logs in production make sure the dev team regularly investigates the logs during development and testing for example by

32. 15:54

    capturing the logs and CI runs and fostering a culture of debug CI failures using these locks by default for monitoring we recommend that you capture the metrics for all your components and dashboard the golden signals which include latency traffic errors and saturation for all of them example dashboards for Canton components can be found in the D documentation you should set up alerting on these metrics so that you learn when your application becomes

33. 16:26

    unhealthy this alerting can be developed as part of the long running test instance for off lger components bug fixing and roll out of new features to support evolving business requirements does not present any new challenges compared to web 2 apps or Enterprise Services the recommendation is to follow standard design and development practices rolling out changes to DL code requires additional considerations for the following reasons damel code

34. 16:58

    represents present shared rules and upgrades to these rules need to be deployed across multiple administrative domains which requires cross organizational coordination the daml code serves as an API definition of the Cross organization workflows changes to this API must be reflected in all components that use it such as backends and front ends ideally these changes are done in a backwards compatible fashion to minimize code changes required ired independent components which is often

35. 17:31

    possible to achieve we will discuss the tools provided by DL to manage workflow changes and how to apply these tools when building daml applications in the following lesson in this lesson you've learned the considerations a Technical Solutions architect should have for successful execution of each phase of the life cycle of a DL app including design development testing deployment operation an maintenance specifically you've learned how the use of daml in a project

36. 18:04

    affects the design and development phases of the projects life cycle the recommended approach for testing the components of a DL app and for setting up the CI environment how the app architecture choices affect the deployment of a DL app and the operational and upgrading considerations for a DL app