Lesson 1: Bootstrapping Daml Functional Delivery

Captions as YouTube publishes them, usually machine-generated, so names such as Daml and Canton can be misheard.

1. 0:00

   welcome to delivering functional requirements the third course in the D Canton technical solution architect certification we have seen the value DL and Canton provide to a solution architecture in the topology course we explored the components that constitute a DL application and in this course we take the next step and interrogate what is involved in implementing the business functionality required to deliver a successful project this lesson will provide

2. 0:33

   guidance on how to bootstrap an iterative development and delivery structure and how to structure the design of an individual functional requirement we have found it is important to ensure the initial bootstrapping of the project provides a solid foundation for ongoing development while the risks associated with overs scoping a bootstrap are generally understood we have also seen frustration

3. 1:03

   and delays resulting from unders scoping this initial work in the name of getting started quickly to this end two critical elements of this Foundation include getting as quickly as possible to an automated endtoend demonstration of the core value driver of your app preparing the devops infrastructure to protect the velocity of team in particular automated testing unit integration and performance tests having the core workflow in place

4. 1:34

   early provides the maximum opportunity to evolve and explore how it can be best delivered and ensures problems in its design are discovered early devops infrastructure protects these advantages and maximizes planning flexibility in future iterations the benefits compound over time so aim to have these in place from the outset you can think of this combination of a core workflow and the necessary infrastructure and Scaffolding as a

5. 2:04

   minimal viable basis for ongoing development it will typically consist of a core workflow a small set of supporting workflows required by the core workflow initial devop infrastructure to support ongoing development and testing at a high velocity when deciding on the workflows to include keep in mind that generally damel workflows fall in into one of three categories onboarding resource management and resource

6. 2:37

   usage these form a dependency hierarchy resource usage workflows involve manipulating and utilizing value represented as contracts on The Ledger in a typical Securities application asset transfer would be a resource usage workflow not surprisingly then the Core Business workflow is generally a resource usage workflow the resource contracts in use in these workflows result from resource management workflows responsible for

7. 3:08

   enforcing the appropriate Authority and authorization when creating archiving and administration of resource contracts if asset transfer is a classic resource usage workflow asset issuance or account creation are classic Resource Management workflows these involve choices exercised on one or more contracts representing roles service agreements application user onboarding or similar Concepts involved in authority and authorization ultimately these

8. 3:41

   management contracts will originate from at least one onboarding workflow responsible for provisioning and establishing the initial Ledger state for a daml application user these Go by different names in different applications but creating application installation or service agreement contracts fall into this category based on these categories we can expand the minimal viable basis further along with initial testing and

9. 4:11

   devops infrastructure the initial endtoend workflows will typically include a working implementation of the core workflow one or two bootstrap management workflows sufficient to create the required resources and one onboarding workflow to provision the boot strap management workflows for a viable basis these supporting workflows only need to provide enough to allow an automated endtoend test to provision app users and establish the initial conditions for the core

10. 4:44

    workflow so focus on providing a reasonable happy path through the core workflow supported by minimal bootstrap management implementations and the simplest onboarding workflow you can manage all of these can be elaborated and improved later ideally the resulting endtoend demonstration touches as many components in the endtoend component architecture as possible this maximizes the integration footprint and ensures your minimum viable basis for ongoing development all

11. 5:18

    of the key design elements required by Future iterations an upfront investment in the enabling your developers to work effectively on the distributed DL application will pay off throughout the entirety of the project supporting High productivity and ensuring quality targets are met requires developer and test support infrastructure this is why we talk about this being a basis for ongoing development rather than some form of minimal viable version a daml application is a

12. 5:51

    distributed application with multiple nodes therefore in particular in addition to the usual continuous inter integration it is recommended that at the very least you set up some basic containerization and some form of artifact management both to support testing and to simplify the local Dev experience the final critical piece of the minimal viable basis is your test infrastructure one of the most impactful practices to reduce risk and sustain

13. 6:23

    developer velocity is continuous integration and automated testing a minimum viable basis for ongoing development should include unit or component level testing DL components should be tested using DL script other components should be tested using the native test infrastructure provided by whichever general purpose language they are implemented in endtoend and functional tests provide confidence in

14. 6:53

    delivery in addition as a distributed application having these tests automated and continuous integration is invaluable for protecting the velocity of your team compared to more centralized architectures distributed systems see more value from end to end tests so do not discard them in preference to unit testing prioritize addressing any flaky Behavior or non-determinism as this can undermine the velocity improvements provided by these tests it is important when

15. 7:26

    considering functional requirements to also ensure a assoc Associated non-functional requirements are satisfied as there can be tradeoffs for example in distributed applications it is not always obvious where contention will arise any early investment in improving your visibility of such issues as part of the development cycle pays dividends we therefore recommend ensuring that you have some form of automated load and scalability tests from the project

16. 7:57

    outset previous courses have covered the design and implementation of DL models and their role at the heart of multi-party business workflows the templates and choices that Define these workflows form the foundation of every DL application however as we have seen there is more to an endtoend workflow design than developing a suitable daml model in the rest of this lesson we will

17. 8:29

    consider what else is involved the design elements requiring consideration when addressing a functional requirement touch on every aspect of the application architecture this is a reasonable explanation for why focusing on a full end to end demonstration is so valuable when defining your basis for ongoing development as is usual when architecting a design these elements will be visited and Revisited in whatever order makes sense for the specific

18. 9:00

    requirement and you can expect some of them to have already been resolved by previous requirements in previous iterations as our example and because it is the first workflow that every deml application requires we will explore a provisioning workflow to focus on the design elements we will assume that the app user already has a working participant node connected to the synchronizer we will consider out of scope the physical deployment of any required front-end or backend services

19. 9:34

    but not their specification the purpose of a provisioning workflow during onboarding is to establish the initial Ledger state required to onboard a new application user the minimum on lger state required to represent this is a single active contract with a single choice that initiates a resource management workflow both the initial provision State and the initial on boarding workflow are things you may well revisit later for now the goal is to allow us to

20. 10:05

    get to the core workflow so we will keep this simple as our goal is a single contract signed by two parties the simplest thing that could possibly work here is an instance of the propose and accept pattern you have seen in other training in this case as we have named contract that represents successfully installing the app app install The Logical name for a request to provision the contract is app install request beyond the basic pattern there

21. 10:37

    is not much more to the model the only other consideration here is identifying any data required from the application user to complete the workflow then making sure that data is included in the request contract with that the workflow model is done for now expect to revisit it as you understand the business request re Ms better for now this will suffice which leads us to answer how to start the workflow from The Ledger perspective the

22. 11:08

    workflow starts when an app user creates an app install request contract from a whole of business perspective there is a process that results in your customer sending the command that creates that contract to The Ledger API from a daml application endtoend perspective this process potentially depends on and generates records both on and off Ledger that support the onboarding workflow the simplest answer to

23. 11:39

    initiating the daml application onboarding workflow will often be an operation script submitting the command to create the app install request contract directly to the Ledger API of the app users participant node a script is fine if the provisioning workflow is initiated by only a few users at a certain scale it may make sense to provide a UI to simplify The Human Side of the initiation process many other workflows will

24. 12:10

    involve a UI of some sort so we will use a UI for this example so this workflow will be initiated by an administrator and user in the app user organization via a webfront end operated by that organization when there is a UI involved the front end will need to be able to to authenticate the end user to these services so it will be the application users am that is responsible for identifying their administrator

25. 12:39

    authenticating them and verifying that they have the admin rights necessary to initiate the bootstrap process any backend Services involved in the workflow will also need to be authenticated against their respective participant node so consider how the app users backend will authenticate against the app app users participant node having the IT team of the app user allocate one or more service accounts in their IAM for this purpose will generally

26. 13:10

    suffice regardless how you resolve to authenticate backend Services as the foundation of trust privacy and non-repudiation we strongly recommend that authentication for a service stay within that Services trust boundary as we have previously mentioned an application front ENT should not typically access The Ledger directly instead its access to The Ledger should be mediated by a backend service so the next step is to identify what queries the front end needs to perform and

27. 13:43

    ensure these are provided by a backend service a good example of this is how the front end can determine if the application is already provisioned to determine whether provisioning is required the end user's front end can query the back end to check if the app install contract already exists if the app install contract does not exist then provisioning needs to be initiated by submitting a command to the app user participant node to create the required

28. 14:14

    app install request contract for now we will assume a user backend API service that supports basic HTTP interactions get will return the contract ID of either the app install contract or the app install request contract should either contract exist post will allow the front end to initiate provisioning this will accept an app install request record note that with both the get and the post Services their first step has to be to

29. 14:47

    authenticate the end user use whatever technology you prefer for now we will assume verifying a jot token contains the relevant privileges after authentic ating the end user as a valid administrator and validating the provided record the app user's backend should submit the relevant create command to their local participant node daml has no independent thread of execution was the request is on The Ledger the only way for the request to

30. 15:20

    be processed is for an off lger process to respond to the request by submitting another DL command to The Ledger this is why most workflows including this one will require some form of automation support the minimal automation is a persistent process run by the application provider in their backend that periodically queries pqs for active app install request contracts and in response exercises the accept install Choice via a command to the application

31. 15:52

    providers participant node this sort of busy Loop will get you started but it is important to mate promptly to production capable automations like those discussed in the DL application backend in the previous course signing an app install contract is a formal attestation that the application provider has approved the app user provisioning request so if onboarding an app user to your system requires other checks such as kyc or an internal customer

32. 16:23

    management system this automation step is an appropriate place to cons consult these systems before exercising the approved choice you should plan to perform these checks by querying the relevant systems using the data provided by the app user on the app install request also if required for auditing or other purposes data from these checks can be passed into the choice the application will also need to avoid duplicate requests we discuss uniqueness and

33. 16:56

    command a duplication in the next lesson and this course the damel workflow ends when the accept request choice is exercised the appinstall contract is Created from The Ledger perspective this is the end of the interorganizational workflow that daml and Canton facilitated from the perspective of internal business functional requirements it is likely at least one end user will need to be notified of the final result of provisioning it is also possible that other components and

34. 17:29

    services need to respond to a successful install within either or both organizations Enterprise infrastructure at its most basic this can be achieved by observing the creation of the app install contract the Canton synchronization protocol will ensure this is visible on the app users participant node as it hosts a stakeholder in the app install contract ultimately you will need to consider how is this visibility communic ated to the

35. 17:59

    frontend code and if the app user may want to expose this internally on an ESB or other internal Services what additional information might be required on The Ledger to allow the generation of a meaningful internal message for our purposes the app install endpoint we have already defined will suffice as a minimal basis for ongoing development with the obvious exceptions of the workflow model and the automation the involvement of daml in the rest of

36. 18:30

    the application is generally restricted to contract IDs and possibly Json encoded contracts beyond that a daml application web front end is designed and operates much like any other web front end the implementation of a pqs backed web service is not really any different from any other database backed web service you will find this a recurring theme When developing your DL application daml and Canton provide specific unique

37. 19:02

    capabilities but the non- daml components of your application utilize the same Technologies you are already familiar with and are used to developing optimizing and supporting as mentioned your first objective is likely a collection of three categories of workflow a basic onboarding workflow loot strap resource cre creation workflows and the core value resource utilization workflow future iterations proceed in the same fashion each requirement being analyzed in terms of initiation

38. 19:42

    authentication workflow backend apis automation external Integrations and notification eress finding good decompositions of requirements across the components is a craft akin to data schema design and requires practice this is also a strong motivation for proceeding incrementally it enables quick validation and fast learning of what works and what doesn't also both the velocity

39. 20:11

    improvements obtained from testing and devops infrastructure and the risks that accumulate in their absence acrew over time the cost of retrofitting pre-existing work into this infrastructure also increases as the complexity of the work increases over time so we recommend that this testing and Dev support infrastructure be considered a part of this initial minimal basis for ongoing development all development of your app is going to follow this

40. 20:43

    pattern iteration by iteration enhancing existing workflows by improving uis or adding new system Integrations extending existing workflows adding support for alternative and error cases ref factoring workflows to improve or remedy operational resilience and nfrs adding new resource types and or workflows to manage them adding new workflows to support alternate ways of utilizing those resources the sort of enhancement we

41. 21:16

    might consider to the provisioning workflow we just looked at includes as an app provider I may want to be able to list all provisioned applications a small incremental enhancement can Prov provide it add a new internal API endpoint to the provider's backend for example get admin installs authorized for app providers and users with admin rights backed by a simple query against the provider's

42. 21:45

    pqs then updating the front end to access the new end point and display the result completes the new feature notably this feature can be delivered without changing the DL model modifying automations or upgrading any data on The Ledger to conclude ultimately most of the considerations involved in implementing a DL application are the same considerations that apply to any modern Enterprise solution finding good

43. 22:15

    decompositions of business workflows across the components is a craft akin to data schema design and requires practice targeting a minimal viable basis for development will allow you to explore and gain experience with different decompositions starting with the core business value and devop infrastructure supports iteration to improve and expand the scope of your solution reducing risk and increasing the likelihood of a successful project