DerivHack2019 2/3: Section B - CDM Messages and Automation

Captions as YouTube publishes them, usually machine-generated, so names such as Daml and Canton can be misheard.

1. 0:00

   so for those of you arrived late there's the there's the github repo it's under digital - asset slash Theriot hack 2019 and over the next 30 minutes we're going to show you one of the worked examples we have there this hello world example there's a full readme tutorial here a lot of which I'm gonna talk over in in this session so if you want to follow along just have a look at that or clone

2. 0:34

   the repo but before we dive into that I want to take a couple of minutes just to talk about the high-level architecture or the high-level design of the programs I'm gonna show you so we're going to show you two different processes both written in Python the first one is going to take a CD and JSON file which we've taken from the read gnosis website in fact is the same one that Thomas just showed you a bit earlier and we're gonna

3. 1:14

   upload that into a ledger using an HTTP based API so not shown on this diagram is a little HTTP server that sits in front of the ledger and then I'm going to show you a second process which doesn't use an HTTP based API but rather an internal Python API and what that does is it sits and listens to events on the ledger for instance contract creations and then reacts as a result of those contract creations is a

4. 1:55

   written in Python but in fact as you can probably guess the the first one that I'm going to talk about could be any language that talks HTTP arrest and finally what you see here this blue cloud is our cloud offering which allows you to host the ledger and run automation processes it's called double and that will be the topic of next week seminar so I'm not going to go deep into that in this session and the final sort of warning

5. 2:33

   for the guys for more experience than have already done some demo programming you might have looked at our official documents website so doc Stano comm and the official API that we provide there and you should just be aware that we've specifically architected these examples to be able to sort of drag and drop them into the cloud deployment so they don't use the same API is that are advertised on our official website so just be conscious of that when you're looking at

6. 3:07

   these and for the hackathon as well so let me just reassure my screen and we could start looking at those programs okay so first I'm just going to check out the the kit hub repo and we're going to go into this hello CDM example and I'm just gonna check out a version that I know works for this presentation so in here you're gonna see the usual

7. 4:09

   thermal directory structure we've got a demo folder that has our demo model which Thomas already introduced we also have a Python directory that has the two processes I mentioned and there's also a UI directory which will be a topic of the last half an hour of this session so I won't talk about that right now and so what we're gonna do is we're gonna run this this first main program and like I said what it's going to do is gonna load up a CDM from Jason it's gonna use an

8. 4:48

   HTTP API to load that into our ledger and then it's going to read it back and exercise some choices on it so in order to be able to do that we need to actually bring up a local instance of a ledger which we do with the demo sandbox command but in fact sorry before we even do that we need to compile the demo code so let's do that done we'll build and you're gonna see a couple of warning messages there that's just because we're

9. 5:23

   including the whole CDN schema on there but not not using everything we import but once that's done once that's done then we should be able to start sandbox ledger and I want to bring your attention to a couple of the options I'm using here so one is ledger ID and that basically allows us to refer to this ledger in a specific way and it's quite important during authentication which I'll talk about in a moment and the

10. 6:04

    other one is static time which we also need in order to be compatible with the cloud-based deployment Sushi's ledger ID here and you see that we have loaded our CDM templates into the local ledger next what we're going to do is start up an HTTP process it's just basically an HTTP server that sits in front of the ledger and allows us to talk to it from the external world in here the important

11. 6:43

    switch that I want you to service this max inbound message size which you normally won't use but because the CDM schema is so large we need to use that to allow us to to pass these large messages in otherwise the contract creations will fail later on so let's go ahead and start okay so I'm actually going to show you now the the first scripts I'm just gonna run it and then we'll explain what what it's doing and really quickly before that we have to

12. 7:21

    load up some dependencies and we explicitly state this message integration library dependency we shall explain in a second why we need that so let's let's actually run our example we see a whole bunch of output there we see it sending the contract to the ledger reading it back and exercising a choice so let's have a look at the actual scripts and to see what it's doing I'm going to open that a separate window

13. 8:08

    here so what the script does is it just executes five methods sequentially and I'm gonna talk you through those five methods now the first one is is very simple it simply loads up a Jason CDM file in this case the cash transfer instance into a Python dictionary and prints it out so that's the first set of stuff that we saw here like I said before this is an example we pulled from the e recognizers CDM website there's some more there if you want to try them out and the next

14. 8:59

    function that we have here takes that Python dictionary and it's going to do some conversions on it although both both of the the output dictionaries a jason there's some sort of differences in how the CDM encoding goes and how the Tamil encoding for the HTTP API is and the easiest way to explain that is just to look at an example so the what we showed here was an original CDM file and I'm just going to look at the the last

15. 9:39

    two fields we have here so we have time stamp which is made up of a date time and a qualification field and if we look at what the translator did it's actually expanding this timestamp into two fields including a time zone and the qualification name has been mangled so that it has the enumeration name in it and there are lots of small things like this in indigestion schema that prevent us from just loading up directly but we've provided you in this repo

16. 10:20

    standalone Python library that's that's what I loaded up with and before and so that you don't have to do that yourselves so if we look at how that's called we just need to import this message integration library you'll need to load up some meter data from a JSON file to specify a decoding strategy and then you've just called the code on that and you also specify the type of CDM object

17. 10:56

    you're decoding in this case an event and if you want to do the opposite it's very simple you just to encode here instead of T code to bring you back in the opposite direction which we're not going to do here so then let's jump to the next function which takes that converted jason and actually uploads it into the ledger it's quite a simple call it just consists of a single call here

18. 11:31

    and this is actually the same for the next three methods and so it's just an HTTP POST request using the requests Python library where the first argument is the URI for the creation command that's this here and before providing the actual payload there's a few things we need to add so this comes back to what I mentioned when we started up the sandbox ledger the first one is a security token that we pass here as a token header and I'm just going to show you what that actually looks like it's

19. 12:21

    just a digest string you can see here and I'm actually gonna explain in this particular session how to do that but I'm gonna refer you to the tutorial I mentioned there's some links there explaining how you can generate these tokens the important thing that you need to know is that that is a hash of something that includes the ledger ID which we specified excuse me here so in this case hello CDN and so you

20. 12:54

    cannot reuse those security tokens across different Ledger's and if you change your secret but again please refer to the tutorial to understand fully how to use that the other thing which we need besides the payloads or before producing the payload is meets a field in here that says ledger effective time and we were essentially overwriting the current time which the ledger uses to check that the creation requests are

21. 13:26

    not stale and this is something we do for the double compatibility for being able to upload to the cloud and that's also something that I think might change but it will be discussed also you know next session when we talk more about uploading stuff into our cloud environment so finally the the arguments in here is the actual template we're generating and you can see that we're passing the demo dictionary which we translated in the previous step and what

22. 14:02

    this returns is the HTTP requests as a result so we can see that right at the bottom here we say sending transfer contractor ledger and the response is HTTP 200 response and then what we do after that is we read back the same contract from the ledger that's in this reads thermal Jayson flore ledger and as I just said it's very similar this is actually even simpler than the previous call again supposed we were providing the endpoint we provide our token again

23. 14:38

    and we provide a filter for the type of templates that we want to get back in this case we're passing a cash transfer excuse me is it transfer as the contract name which in this particular example doesn't really matter because we're gonna have one single type of contract but you can imagine that in your CDI model you might have you know transfers executions allocations or whatever and you only want specific subcontract back

24. 15:07

    so that's what this argument is for and then we move on to the the final method here which is exercise choice and that does essentially what Thomas already showed you except it does is does it externally from an external process and I'm just gonna remind you really briefly what what this does so if we have a look at the the actual JSON file this cash transfer you'll see somewhere further down we have I believe an identifier

25. 15:50

    let's see if I can find it or was it further up I think it's this one here the assign identifier and so what we're gonna do with this choice is in the ledger we're gonna overwrite the value with the with the hello message and we're going to increment the version number and again it's the same pattern as before we have the endpoint headers this meter field with the ledger time but in this case we have to specify the

26. 16:22

    choice name which is say hello and the choice arguments which in this case is going to be world so we're gonna say hello world and that's what we've just executed here so if you were to inspect the ledger at the moment you'd see a cash transfer contract in there that says hello world in the ID and version number two okay so now we're going to turn to the second process so let me just bring up the code

27. 17:04

    for that so that's the bot and to remind you this does not use the HTTP json api and it talks dialect directly with the ledger through an internal API and this can be hosted on the double cloud so the way of working with this is slightly different instead of making restful requests we have some callbacks and in this specific case we have an in it's callback which is triggered when we are connected to the ledger and ready to consume

28. 17:48

    contracts and in our example the only thing we're going to do is we're just going to say ready and listening for new transfer contracts and then we have another callback which is ledger creatives and this is triggered every time a new contract is created in this case we specify that we only want to get transfer contracts and what we're gonna do when a new transfer contracts is

29. 18:14

    created is we're going to pull up the list of all active contracts all active transfer contracts excuse me and we are also going to exercise the say hello choice on them but this time instead of changing them to say hello world we're going to change in to say hello CDM that's done over a year and so obviously there's some boilerplate to get this set up and the last line that we call is this run forever line which basically makes the program block and

30. 18:54

    listen for those events that we specified before so let's let's run this in the new window and actually see what happens it takes a little while to to load up about 15-20 seconds because it loads the whole CDM schema but we should soon see a message that says I'm ready and waiting for contracts there we go so now let's create a new contract with with the first script and see what happens so

31. 19:38

    I'm just going to run the same script I did before and it looks like it worked but when we look here we can see that there's some sort of a an exception that's being thrown and if we really screen our eyes we can see some where it says here dependency error couldn't find contracts so and so and this is actually expected so if we think about what we're doing is we're running two processes concurrently and they're both trying to

32. 20:07

    exercise the new contracts that are appearing there and as Thomas just explained to you one way of getting around this is using contract keys that solves this problem but in this specific case we're gonna do something much simpler we're just gonna not exercise a choice from the first script so let's look at the first script again and I'm going to comment out this bit that says exercise contract in fact we're just

33. 20:38

    gonna quit the program early there so let's try that one more time and yeah you see we're transferring the contracts sorry creating the contract reading it back but we're not actually exercising it and so if we now we look again at the output from the part we can say we can see sorry that it's greeted us greeting exercise choice on contract six hello CDN and the reason that you see

34. 21:08

    all those other ones like I mention before is because we're reading the entire contract list not just the the newest event that's that's appeared so that's pretty much everything that I was going to show in in this part of the tutorial so just to recap I've shown you two processes one which loads up CDM contracts from the CDM jason and translates them into a form that can be understood by the HTTP server that's spun up next to the ledger and the

35. 21:42

    second process is an automation process which doesn't use the HTTP json api he uses an internal python api we call dazzle for which there's also more documentation again on the in the repo in the tutorial and so I would suggest scythe if there any bits that went too quickly or you didn't understand that you refer back to the documentation this tutorial that we've put together or if

36. 22:08

    you have any other questions you can raise them as issues on the repo and we'll have a look at them so I don't know now if we got some more questions in the meantime yeah there was this two which are which we've answered but I want to point them out separately so the first one is how do parties share data with each other and come to an agreement is it to the J's CDM JSON file so the sharing of data on a demo ledger

37. 22:39

    is primarily done by Adi The Observer keyword so whenever you create the contract you can put any party you would want to see this contract as an observer on that contract and that basically this closes the contracts or the other part you can see it and then in terms of interaction and how to come to an agreement that would then be done by our choices so that's the second question is an observed we're allowed to execute transactions choices on a shared

38. 23:10

    contract so yes you can then declare a toy controller for a choice and for example to use the observer that you've put in on the contract and then this party is allowed to to exercise that choice and to also mention how you would then sort of come to an agreement so what lavinda has shown last time is basically that a typical pattern how you get multiple parties as signatories on the contract would be for example to

39. 23:41

    first propose a contract from one party where you have the first party as a signatory and gives the other party the choice to accept it and in that choice you would create the new contract that has both signatories so that's basically the pattern how you collect signatures in demo there's one more question what builders for events have been written and are available to use in the code so the Builder function that that thomas showed earlier was just an example of

40. 24:11

    how you would create a CDM event in demo i would expect that the hackathon that most of the CDM that you're dealing with is actually sort of provided to you and you load it onto the ledger and for those cases where you would have to produce then CDM off the ledger you would have to write your own builder but you can take the example that thomas is shown as a sort of a starting point and then just adopt it to whatever type

41. 24:42

    you'll have to create i'm going to add to that so these builders typically are very simple functions that just take a couple of arguments and then default most of the CDM fields accept the arguments that you want to set i can see here someone's asking do you have a doctor eyes ledger image available we might have one somewhere we don't have one as part of this repo but we'll make a note of that we might be able to provide something for the hackathon but

42. 25:14

    we can't promise anything right now so so there is a document for the download SDK which i've posted a link but we don't have one for the for the repository that we just show you another question came in if we need ten parties to sign a contract can one propose an other nine accept the offer asynchronously and they would get added cumulative Lyttelton are available so you couldn't do that on a on a single

43. 25:47

    contract but there is a there is a pattern to connect these signal D signatures in fact you can do it on single contract there's a pattern related to that that I think we have an example of that in the documentation if I'm not mistaken it's called something like multi-party agreement and we can probably look it up and provide the link run data so I guessed short answer is yes it's possible so you can collect acceptances a synchronously so the pond

44. 26:26

    party proposes all other party classic trolleys accept and then the proposal party needs to collect those acceptances to build up multi multi party acceptance country yeah that's correct and I see just the multi party agreement in the docks I think is sort of the synchronous way where you have a single contract which then and people one off the other side so that's another way of collecting signatures there's another question

45. 26:59

    regarding CDM being a to bill is there using thermal and how relevant it is well that's a bit difficult to sales to see diem is not will place the using demo but we have yeah we have built a tunnel library to support least a schema and the question how relevant is to different use cases i I can't really answer I can give you my opinion I mean there's there is a lot of Industry demand coming from the financial sector

46. 27:32

    you know see I'm being somewhat of a spiritual successor to its successor to to FP ml which has been widely successful there's a lot of interest in in adopting standard