Daml-ui-template

Hi,

Using the UI-Template as an example, I build the React static files and can serve them with Nginx. I think I now need to have a reverse-proxy to the sandbox (with another nginx?), but I cannot find an example to follow.

Every papers I read talk about the development server with proxy: http//localhost:7575… and setupProxy.js…but how to do proxying for the production build? I follow some nginx reverse proxy examples on the web but none works.

Can someone share me a nginx.conf for this example? Is it… location /v1 {proxy_pass http://localhost:7575} or location /v1/stream { …} and follows by something about websocket??

I have no programming experience. Am I in the right direction or what should I do for the production build?

Thank you.

Hey @EllisAllison,

I’d caveat my answer with none of our examples are intended to be used in production.

For the reverse proxy itself you would could do as the Nginx guide says:

location / {
    proxy_pass http://localhost:7575;
}

But a lot of this depends also on where/how you’re hosting your UI as well. Are both the UI and JSON API being hosted from the same machine?

Hi @EllisAllison,

We have an internal application called DAVL that is setup in that way, so hopefully its nginx configuration can serve as a reference point. These lines in particular seem to be what you’re looking for:

    location /v1/stream {
      proxy_pass http://${LEDGER_IP_PORT};
      proxy_http_version 1.1;
      proxy_set_header Upgrade \$http_upgrade;
      proxy_set_header Connection "Upgrade";
    }
    location /v1 {
      proxy_pass http://${LEDGER_IP_PORT};
    }

This is a Bash script that generates the actual nginx.conf file, so we expect substitutions for the LEDGER_IP_PORT env var. In a local environment (“production”-like), the resulting file looks like:

worker_processes auto;
pid /run/nginx.pid;

events {
  worker_connections 768;
}

http {
  sendfile on;
  tcp_nopush on;
  tcp_nodelay on;
  keepalive_timeout 65;
  types_hash_max_size 2048;
  include /etc/nginx/mime.types;
  default_type application/octet-stream;

  ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
  ssl_prefer_server_ciphers on;

  access_log /var/log/nginx/access.log;
  error_log /var/log/nginx/error.log;

  gzip on;

# <workaround>
# The UI currently does not support signing up, so we add a running Navigator
# to our setup. It will be served on 8080, so we also need to expose that port.
  server {
    listen 8080;
    server_name navigator;
    location / {
      proxy_pass http://navigator:4000;
    }
  }
# </workaround>

  # This serves https://davl.da-ext.net, with the load balancer doing TLS
  # termination.
  server {
    listen 80;
    server_name davl.da-ext.net;

    location /v1/stream {
      proxy_pass http://json-api:3000;
      proxy_http_version 1.1;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection "Upgrade";
    }

    location /v1 {
      proxy_pass http://json-api:3000;
    }

    root /app/ui;
    index index.html;
    location / {
      try_files $uri $uri/ =404;
    }

    # Disable all forms of caching on all requests.
    # A real production setup would require more granularity, but caching is
    # hard and we currently don't have big enough volumes.
    add_header Last-Modified $date_gmt;
    add_header Expires $date_gmt;
    add_header Cache-Control "no-store, no-cache, must-revalidate, proxy-revalidate, max-age=0";
    add_header Pragme "no-cache";
  }

  # Load balancer redirects plain HTTP to this port.
  server {
    listen 8081;
    server_name _;
    return 307 https://davl.da-ext.net$request_uri;
  }
}

Note that this is a toy application, restricted at the network level to a trusted internal network, so I am not advocating this as a production-grade configuration. In particular, very little attention has been paid to security here. If you intend to put this in front of the internet, please consult a security expert; the internet is a scary place.

But hopefully it should show you how to redirect just the calls to the API (they all start with /v1) to another machine.

Thank you for your advice. I raised the question as I am new to programming and I simply want to understand how things work and fit together. Some other local IT teams have wasted us USD400K+ and almost a year with some other technologies on a similar project. I was not the CEO for that project but I personally sacked them all. I want to know clearly how and why before we spend a dollar more this time. Please allow me to take you some time to explain my situations and intentions.

As a semi-retired investment banker from Hong Kong, I am now working as an investment advisor advising some industries in Mainland China on their corporate finance, in particular supply chain finance. For years I have been thinking about how the industry players can trade with each other in a self-regulated environment so that we, acting as investment houses can provide financing to them. DLT + smart contracts appear to be the perfect solution.

I was introduced to Daml during a FinTech course and I find out that the HK Exchange is going to launch a settlement program in Daml in 2022. Being an executive of an investment bank, I trade in the HKEx everyday for half of my life and so I decided to take a deeper look into Daml.

I love DAML at first sight from the IOU example. So far I find DAML not too difficult to learn to build something really workable. I have tried to learn Haskell to understand DAML better and so far I can still manage. I build a simple trading and finance model for the FMCG industry as my first attempt during the covid19 quarantine period. I want to show the potential industry players, commercial banks as well as government and investment funds my POC before serious investments are deployed to study it further and build it. Everyone I talk to love my ideas and they have all heard something about blockchain. But nobody knows how and cannot be sure if DLT can really work for them. I want to present them my little works with a sandbox feel but for certain reasons, I cannot use DABL (another long story). Perhaps a local ECS service like Aliyun or AWS in China may do the job to give them a simple UI to try the demo.

As an amateur programmer, React is the major problem I am facing. The Daml-Ui-template is really good as it is not difficult for me to change the contents to work with the sandbox which is running the contracts I created. However, I cannot figure out from the React codes how api calls are made to the sandbox, except that the development server can magically do it for me. Having to learn and master React to understand this seems an overkill to me and I am just looking for a simple example to follow. In particular, how api works. This UI is just a transitional thing for demo purposes only. At the end of the days every user will have their own tailored made UI with many other local functions built-in.

I teach myself about docker and nginx and now I can set up the react development in docker to work with my Daml contracts running in the sandbox docker too. I have also tried the daml-on-sawtooth example. To complete the picture on understanding the whole set up, I tried to build the React and I think I need a proxy for it to “talk” to the sandbox. But how?

My present intention is simple. An UI to demonstrate how we can use DLT + Daml to solve the problems and how my ideas may work. It is not really a production build. I can either:

1. put the React development and sandbox (+ triggers) in containers and set them up in ECS. The containers work locally but nothing show up in ECS (due to CORS problems or what?) BTW, Those are big docker containers consuming quite some 8G memory (but I don’t have to care too much on this at the moment); or

2. build the react UI and serve it with Nginx in a container to work with the sandbox (or more aggressively, daml-on-sawtooth?) in containers. But I do not know how to set them up.

Excuse me for the long story and I would be much appreciate if you could give me some ideas how I could move forward quickly.

Thanks again for your time and valuable advices.

Many thanks Gary. I do really want to give you a big hug from China.

It is mid-night here but I will give it a try right now and revert to you again on my results.

Don’t worry it is just a demo, although I do want to put in up on ECS. I believe whoever could bring down Aliyun due to my fault will be rewarded by Alibaba. I will also talk to the government officials I know here to give the genius a medal. LOL.

Hi @EllisAllison
We are a consulting firm focused on DAML smart contracts and have developed several solutions for banks, and healthcare firms deployed either on a local server, cloud server or on damlhub. You can check out one example for Equipment Financing use case that we created based on the ui-template and that we made open source
Feel free to take a look and reach out if you have any questions . (https://www.github.com/RethinkLedgers/ef-smartcontracts)

EllisAllison:

However, I cannot figure out from the React codes how api calls are made to the sandbox, except that the development server can magically do it for me.

First off, I’ll point out that using React is by no means necessary. We provide three libraries to interact with a Daml ledger from JavaScript: @daml/types, @daml/ledger and @daml/react.

@daml/types is the substrate on which the codegen relies; @daml/ledger is a set of functions to facilitate working with the ledger and builds on top of both @daml/types and the types generated by the codegen. @daml/react builds on all the other to integrate with React. It is perfectly possible to use @daml/ledger directly if you do not want to use React.

As for how things work, exactly, let’s take a look at the Getting Started Guide, i.e. the code you get when you run daml new my-proj --template=create-daml-app.

First, we wrap the entire application in a DamlLedger pseudo-HTML element. This creates a Ledger object as defined in @daml/ledger and keeps it around in some sort of React ambient/global state/ I’m not particularly fond of this approach myself (I prefer my variables to be explicit), but that’s the React way.

Within that (dynamic) context, you can use the use* functions from @daml/react to access the underlying, hidden Ledger object. For example, calling useFetch will essentially just call fetch on that hidden Ledger object, plus some client-side wrapping of the results into a type that “plays well” with React.

The useEffect dance, without going into all the details, essentially allows us to say “make this API call, and then later on when you get the result, update the UI to reflect this data”.

Hope that helped make it seem less magical; happy to answer further questions.

EllisAllison:

To complete the picture on understanding the whole set up, I tried to build the React and I think I need a proxy for it to “talk” to the sandbox. But how?

Hopefully my nginx.conf above answers that, but happy to give more specific pointers if not.

The main issue here is that, for various (good, security-related) reasons, browsers don’t want to talk to multiple servers. So we make a single server (nginx) handle all of the requests, some directly itself, and some by just redirecting the requests to another server. Note that in this kind of setup it is necessary for the nginx server to have access to the JSON API, but it is not necessary for the JSON API to be connected to the internet.

With the same caveats about security etc., the DAVL example also has its full Terraform configuration availble, if you know how to read those.

EllisAllison:

BTW, Those are big docker containers consuming quite some 8G memory

For simple demo purposes you should be able to use much smaller containers. I can run the entire DAVL system locally on a VM with 4GB of RAM in total. That includes a container for PostgreSQL, a container for nginx, a container for the daml sandbox, a container for the JSON API, and a container for Navigator (definitely shouldn’t run in production, but can be useful locally to inspect the state of the ledger).

Hi @EllisAllison and thanks for that excellent post.

EllisAllison:

As an amateur programmer, React is the major problem I am facing. The Daml-Ui-template is really good as it is not difficult for me to change the contents to work with

Without a doubt, as a non-professional like you, the React/UI is my Albatross also, however as @Gary_Verhaegen said, you can work around this using @daml/ledger

Welcome!

@Gary_Verhaegen I studied DAVL and tried to understand it as much as I could. I think it is too advance for my level. I am running the sandbox on my mac and I suppose the LEDGER_IP_PORT is http://localhost:7575? Anyway, I built the React daml-ui-template and tried to serve it with Nginx docker and I failed after many attempts. Here are my nginx.conf and my Dockerfile (Sorry, I don’t know how to paste codes here):

My nginx.conf

  server {
      listen 80 default_server;

      location / {
          root   /usr/share/nginx/html;
          index  index.html index.htm;
      }
      
      location /v1 {
        proxy_pass http://localhost:7575;
      }

      location /v1/stream {
        proxy_pass http://localhost:7575;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_set_header Host $host;
      }

      error_page   500 502 503 504  /50x.html;
      location = /50x.html {
          root   /usr/share/nginx/html;
      }
  }

My Dockerfile:

FROM nginx:alpine

COPY ./build /usr/share/nginx/html/

COPY nginx.conf /etc/nginx/conf.d/default.conf


EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]

Hi @quidagis Thank you for your encouragement.

As an old banker trying to use DAML to solve some financial problems, I would say DAML is not too difficult a smart contract language to learn for a beginner like me. But building a UI is really taking me too much time to learn. God knows how I managed to learn some HTML/CSS as well as some docker and nginx just trying to figure out how to build an UI in the last three months. The daml-ui-template is a good template for my need but I really cannot figure out how this magic box works.

You and @Gary_Verhaegen have pointed out that I can use @daml/types, @daml/ledger and @daml/react to interact with the ledger (which I think is bundled in the sandbox?) but shamefully, I don’t know all these methods. Sometimes I wish if the documentation could demonstrate how to interact with the ledger with a simple HTML form for non-programmers like me to understand what is happening. I know IT professionals need more information to build their hard works but I think DAML is also a very good tool for finance and business executives. It would be a pity if they are frightened off by the underlying technologies while most business and finance solutions should be naturally initiated by these non-IT executives. So I will continue the DAML exploration. Being a non-English speaker and without IT background, I think I could still manage at my old age to join you all to make it better.

Thank you again. Best to you all and your families!

EllisAllison:

I know IT professionals need more information to build their hard works but I think DAML is also a very good tool for finance and business executives. It would be a pity if they are frightened off by the underlying technologies while most business and finance solutions should be naturally initiated by these non-IT executives.

My thoughts exactly, and I have mentioned this to @nemanja who does great work in trying to help new Damlers learn better/faster.

Still, nothing great, is ever easy. Keep on coding, and asking questions here, you are in competent company, myself excluded

Thnx for the kind words @quidagis

@EllisAllison Did you get a chance to look at our interactive tutorials showing step-by-step how to

1. build an example social networking application with a React-based UI,
2. add a new UI feature to it, and
3. deploy it to Daml Hub

They might help with making UI progress and getting your application deployed without needing to configure a lot of things.

EllisAllison:

Sometimes I wish if the documentation could demonstrate how to interact with the ledger with a simple HTML form for non-programmers like me to understand what is happening.

Unfortunately we do not currently support interactions with HTML forms directly. The issue here is that HTML <form> send data in a format called, appropriately, “form data” (formally application/x-www-form-urlencoded), which is not a format we currently support. At this point in time, your only option to interact with a Daml ledger from a browser is to go through the JSON API, which requires JSON data. And the only way to generate JSON requests from a browser is with JavaScript.

We could remove the React/Semantic-UI bits of the create-daml-app template, but it would still need to use JavaScript to manage interactions with the JSON API.

EllisAllison:

pointed out that I can use @daml/types, @daml/ledger and @daml/react to interact with the ledger (which I think is bundled in the sandbox?)

(Yes, sandbox is an in-memory, ephemeral ledger that exposes the same API as other Daml ledgers, so you can develop your code against it and then later on deploy the same code against a production, persistent one.)

My point was that, if you are comfortable with JavaScript (and HTML, and CSS, and the DOM API) but not with React, you could choose to not use React and therefore not use @daml/react while still benefitting from the codegen and still using @daml/types and @daml/ledger. However, if you are not familiar with JS, CSS, HTML & DOM (+ JS build tooling, I suppose) I’m afraid I don’t have a good answer for you at the moment. Web development has become awfully complicated these days.

The best I can give you at the moment (and I’m well aware it’s not much, so apologies for that) is an encouragement to read through our getting started guide if you haven’t already. It is using slightly different tools than daml-ui-template (Semantic UI instead of Material UI, npm instead of yarn) but the general principles are the same, and there is a lot more documentation. The create-daml-app documentation is also kept up-to-date, and testing that the template works as expected is part of our release process, so you are much less likely to encounter unexpected issues than with daml-template-ui, which does not seem to be actively maintained. (It’s currently 5 Daml releases late.)

EllisAllison:

I am running the sandbox on my mac and I suppose the LEDGER_IP_PORT is http://localhost:7575?

Unfortunately the answer here is not straightforward. localhost is a fluent concept, as it literally means “the current machine”. So you can only be on localhost as seen from the same machine, where “machine” may not quite mean what you’d think it means.

Guessing a little bit, I am assuming that you have opened a terminal and typed daml sandbox (or daml start), so you have a sandbox running on your macOS “host” machine. At startup, the sandbox should output which port it’s listening to; something like:

INFO: Listening on localhost:6865 over plain text.

This is not what we’re interested in here. What you need to give to nginx is the hostname and port of the JSON API. If you’ve run daml start, it should automatically start (though that may depend on your daml.yaml); if so, it should print something like:

12:49:43.492 [http-json-ledger-api-akka.actor.default-dispatcher-7] INFO  com.daml.http.Main$ - Started server: ServerBinding(/127.0.0.1:7575)

This is the one we want. However, 127.0.0.1 is just as relative as localhost so that may not help us much. In particular, if you run nginx within Docker, things get a bit hairy. Docker only runs on Linux, which means that on macOS Docker has to run in a virtual machine. There are essentially two easy ways to get Docker up and running:

• Docker for Mac, which is imo a terrible idea but the one promoted by Docker, so you’re probably running that.
• docker-machine, which is a portable (non-macOS-specific) way to handle running Docker servers “outside” your local machine.

In both cases Docker will run in a Linux virtual machine; the main difference is that Docker for Mac tries to hide that fact and creates a very leaky abstraction that results in all sorts of hard-to-debug corner cases. Either way, the result will be that you have three networks at play here, and thus three possible meanings for localhost (== 127.0.0.1):

1. The host, macOS machine.
2. The Linux VM.
3. The Docker container.

Visibility across these networks is not complete. Typically, 1 can see 2 but 2 cannot see 1, and 2 can see 3 but 3 cannot see 2 (or 1). The issue you’ll have here is that your nginx instance runs in 3 but your JSON API runs in 1, and therefore redirecting traffic from nginx (3) to JSON API (1) is not possible.

There are two workable options here. The first is to run the sandbox and JSON API as Docker containers too. That way, they’d be in the same network as nginx and therefore visible to it. To achieve that, you could use the (development-only, definitely not supported for production) daml-sdk image we provide. This would look something like:

FROM digitalasset/daml-sdk:1.12.0

USER root
RUN mkdir /app
COPY .daml/dist/ /app/
RUN chown -R daml:daml /app
USER daml

Build this with docker build -t my-daml . (from the root of your project), and then run it twice with commands along the lines of:

docker run --name sandbox my-daml daml sandbox --address 0.0.0.0 --ledgerid ledger /app/my.dar

to start a sandbox with your DAR file (i.e. your compiled daml code) loaded, and

docker run --name json --link sandbox my-daml daml json-api --ledger-host sandbox --ledger-port 6865 --http-port 7575 --address 0.0.0.0

to run the JSON API. And then you can run your nginx machine with the --link json option such that, from within the nginx container, the hostname of the JSON API is json and your configuration becomes:

...
      location /v1 {
        proxy_pass http://json:7575;
      }

      location /v1/stream {
        proxy_pass http://json:7575;
...

Hope that helps.

@nemanja I had attempted all the tutorials and watched all the DAML Youtube videos. I have also tried deploying to the DAML Hub/DABL. (I personally has some reservations on using DAML Hub for demo purposes).

Anyway, thank you for your advice.

@Gary_Verhaegen Thank you very much indeed for your detailed explanations to a beginner like me.

In fact, I can run the React development in a container with another container running the sandbox in the same docker network. I then greedily tried to run them on ECS but as expected, I cannot access it with my local browser. I searched for answers and it seems there are CORS problems and ppl would only put the production build on a remote server with Nginx. That was how I got here to you with all these naive questions.

I will try your recommendations and again, thank you for your time and guidance.

EllisAllison:

I had attempted all the tutorials and watched all the DAML Youtube videos. I have also tried deploying to the DAML Hub/DABL. (I personally has some reservations on using DAML Hub for demo purposes).

Would love to hear more about what’s missing in the Your first feature tutorial and how we can improve it (that tutorial is supposed to help in explaining how to develop a new feature in Daml and connect it to a UI). Happy to get your feedback either here or in a private message (whatever works better for you really).

@nemanja The tutorial is great. However, I was not interested on it at the beginning as it is more like a Chat app while I was looking for business applications. In fact, it was the previous IOU example which made me love DAML at first sight. Anyway, I did the tutorial. I also like the daml-ui-template after watching a Youtube video on it and I could easily follow that video to modify a UI I want without having to know too much about React. The guidance to deploy to DAML Hub is very clear, including how to run triggers and initiated scripts. Those are all really great works.

My personal reservations on DABL/ DAML Hub are about users login. It is perfectly OK in real life as each participant can only have one account and one role in the real world. But for a MVP demo I think it is another story.

The power of DLT and DAML is users collaborations. i.e. a group of participants can interact with each other in a trusted and decentralised environment. So to enable a potential user to understand how a MVP demo might solve his needs, this potential user has to play different roles to see the whole picture. This is where I got some feedbacks from my friends and potential users:

1. Not the whole world can access Google or has Gmail. I connect to DABL from from China using VPN. Further, my friends in the banks told me that they cannot use VPNs.

2. To enable a user to play different roles to understand the MVP, this user got to login in with different account names/long party-codes and long token strings. Of course, I could create several dummy accounts for them but that was a crazy job. And they have to copy and paste all the long strings in order to login to play with the MVP. You can imagine how frustrating it is for them after several login and logout. They got lost completely and their attentions were drawn to these login things and not how the app solves their problems.

3. The MVP use party-codes and names. I understand the reasons for this but you can imagine how confusing it would be if you have to trade with four to five different parties with long codes. We can tell Alice, Bob and Cindy easily but not their party-codes. Some have suggested to build a kind of dictionary to display user names in stead of party codes, but that would defeat the purposes of quick demo deployment. Besides, for a non-programmer like me it will be an overkill.

I was looking for a simple quick solution to show something like the sandbox to all potential parties. Because of the above, I decided to put up the MVP demo on ECS and that was how I got all these questions about the UI.

Thank you again for your kind advices.

Thanks a million for the very thorough feedback @EllisAllison, much appreciated! Another option could be to use the Daml Navigator, a simple UI that can show the workflow (it all depends on how nice the UI has to be and what you wanna show with it)

Hi @ellisAllison

You mentioned above you are looking for a more streamlined user management for different parties in an easy-to-use sample app. This is one of the reasons we further enhanced the daml-ui-template and created an “equipment finance” use case and implemented some enhancements in the daml-ui-template, including:

• a more simple authentication models (eg. calculated JWT token passwords for login for any of the parties, which means you can log with no complicated JWT password required)
• and more user friendly party names (using PartyIdHint and parties.json) within the application

You can find more information about this in our read.me file on https://github.com/RethinkLedgers/ef-smartcontract/blob/d80c2e325bebcc12a07f3648d92524ac6e1adcc9/README.md

@Gary_Verhaegen I am afraid I have to bother you again with more guidance.

I followed your advice and set up the sandbox and json containers as:

docker run --name sandbox my-daml daml sandbox --address 0.0.0.0 --ledgerid my-app /app/my-app-0.0.1.dar

docker run --name json --link sandbox my-daml daml json-api --ledger-host sandbox --ledger-port 6865 --http-port 7575 --address 0.0.0.0

I then run my nginx container as:

docker run --name nginx --link json -p 80:80 my-nginx

Here is my nginx.conf :

server {
listen 80;
location / {
     root   /usr/share/nginx/html;
     index index.html index.htm;
     try_files $uri $uri/ /index.html;
}
    location /v1 {
        proxy_pass http://json:7575;
      }
    location /v/stream {
    proxy_pass http://json:7575; 
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Host $host;
    proxy_cache_bypass $http_upgrade;
}
}

and the Dockerfiles:

FROM nginx:stable-alpine

COPY build /usr/share/nginx/html

COPY nginx.conf /etc/nginx/conf.d/default.conf

EXPOSE 80

CMD ["nginx", "-g", "daemon off;"]

After many attempts to modify my nginx.conf, I still have the following two errors on my Chrome:

WebSocket connection to 'wss://api/data/localhost/v1/stream/query' failed: 

useStreamQuery: WebSocket connection failed.

What have I missed?

@bartcant Thank you. This is extremely helpful to me and I am studying this equipment finance application.
It is indeed a great learning material. I do love to see how daml could solve business problems.

Thank you very much for your advice.

EllisAllison:

location /v/stream {
proxy_pass http://json:7575;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}

It will take me some time to reproduce so I can make sure this is the issue, but at first glance it looks like there are two problems with this block: the location should be /v1/stream, not /v/stream, and it should have a line that sets the Connection header:

      proxy_set_header Connection "Upgrade";

From MDN:

Note : Connection: upgrade must be set whenever Upgrade is sent.

@Gary_Verhaegen I have revised the nginx.conf as follows but the same errors occurred on my Chrome:

server {
listen 80;

location / {
     root   /usr/share/nginx/html;
     index index.html index.htm;
     try_files $uri $uri/ /index.html;
}
    location /v1 {
        proxy_pass http://json:7575;
      }
    location /v1/stream {
    proxy_pass http://json:7575; 
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Host $host;
    proxy_set_header Connection "Upgrade";
}
}

The React build for the example really drives me nut. Looking at the Chrome errors, the request URL is:

ws://localhost:7575/v1/stream/query

and the exercise URL is:

https://api/data/localhost/v1/create

Do these have anything to do with the nginx location blocks? I don’t know as I have no idea on the React codes.

Please advise me further on this and it seems I am close to the end of this learning exercise. Many Many Thanks.

nginx.conf seems OK to me, but I’m a bit perplexed by the URLs you’re showing. Do you have any idea where those api/data/localhost are coming from? Also do you only have issues with streaming, or are exercises also failing?

Could you perhaps share your entire setup, including how to build the various Docker images involved?

@Gary_Verhaegen Let me repeat what I have done.

First, I git cloned the daml-ui-template and renamed it as “my-app”. Then I followed the Readme and “make build” everything. Thereafter, I “daml start” the sandbox as well as “yarn start” the UI. The my-app is running on http://localhost:3000. So far, everything is fine and I can play around with it to create and exercise and the UI will update automatically.

Since the React was also built. I tried it with “serve -s build” and open my Chrome on http://localhost:5000. The UI login screen did show up and after login (as Alice or Bob), I can see the UI without any data. This was expected and I opened the Chrome developer tool to find the following two errors:

WebSocket connection to 'wss://api:5000/data/localhost:5000/v1/stream/query' failed:

useStreamQuery: WebSocket connection failed.

I was not surprised to read them as I am sure the UI was not connected to the sandbox or api server. The funny thing happened when I tried with http://172.20.10.5:5000 as prompted ( 172.20.10.5 is my mac IP). I got a completely blank screen. The Chrome developer tool showed the following errors:

DOMException: Failed to construct 'WebSocket': The URL 'wss://api.20.10.5:5000:5000/data/172/v1/stream/query' is invalid.

Uncaught DOMException: Failed to construct 'WebSocket': The URL 'wss://api.20.10.5:5000:5000/data/172/v1/stream/query' is invalid.

I did not know what was going on as I have no React knowledge to find out what went wrong.

Anyway, I followed your previous advice and build up a “my-daml” docker image:

FROM digitalasset/daml-sdk:1.7.0
USER root
RUN mkdir /app
COPY .daml/dist/ /app/
RUN chown -R daml:daml /app
USER daml

I run the image for two containers:

docker run --name sandbox my-daml daml sandbox --address 0.0.0.0 --ledgerid my-app /app/my-app-0.0.1.dar

docker run --name json --link sandbox my-daml daml json-api --ledger-host sandbox --ledger-port 6865 --http-port 7575 --address 0.0.0.0

I set up my dockerfile to build the image “my-nginx”:

FROM nginx:stable-alpine
COPY build /usr/share/nginx/html
COPY nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]

with nginx.conf as:

server {
listen 80;

location / {
     root   /usr/share/nginx/html;
     index index.html index.htm;
     try_files $uri $uri/ /index.html;
}
    location /v1 {
        proxy_pass http://json:7575;
      }
    location /v1/stream {
    proxy_pass http://json:7575; 
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Host $host;
    proxy_set_header Connection "Upgrade";
}
}

I run this my-nginx image as:

docker run --name nginx --link json -p 80:80 my-nginx

I open the Chrome browser http://localhost and again see the UI without any data. The Chrome developer tool shows the two errors as:

WebSocket connection to 'wss://api/data/localhost/v1/stream/query' failed: 

useStreamQuery: WebSocket connection failed

I tried to create a new asset and got the error:

Uncaught (in promise) TypeError: Network request failed
    at XMLHttpRequest.E.s.onerror 

Another strange thing happened when I go to http://172.20.10.5 and the errors become:

WebSocket connection to 'wss://api.20.10.5/data/172/v1/stream/query' failed: 

useStreamQuery: WebSocket connection failed.

Please take note that the IP address was somehow changed.

The above are what I have tried in the last several weeks but still failed. I did try the create-daml-app template and I think its React build cannot be served either. Please advice.

Sorry to take so much of your time but I do really want to see how things are set up for production.

EllisAllison:

First, I git cloned the daml-ui-template and renamed it as “my-app”. Then I followed the Readme and “make build” everything. Thereafter, I “daml start” the sandbox as well as “yarn start” the UI. The my-app is running on http://localhost:3000. So far, everything is fine and I can play around with it to create and exercise and the UI will update automatically.

I can reproduce this.

EllisAllison:

Since the React was also built. I tried it with “serve -s build” and open my Chrome on http://localhost:5000. The UI login screen did show up and after login (as Alice or Bob), I can see the UI without any data. This was expected and I opened the Chrome developer tool to find the following two errors:

No idea what “serve” is or why you’d expect it to work here. I don’t have a serve binary on my machine. I’ll ignore this for now and skip to the Docker bits.

…

Ok, this took a bit a digging, but I found out the issue(s). The first one is with the template itself, which seems to assume it’s getting deployed to Daml Hub and nothing else. So no connection gets established because the JS code mangles the URL.

To get it to work, you’ll need to change the ui/src/config.tsx file to match your setup. Specifically:

diff --git a/ui/src/config.ts b/ui/src/config.ts
index c2421f5..5ab2971 100755
--- a/ui/src/config.ts
+++ b/ui/src/config.ts
@@ -5,12 +5,12 @@ export const isLocalDev = process.env.NODE_ENV === 'development';
 let host = window.location.host.split('.')
 
 const applicationId = 'daml-ui-template'
-export const ledgerId = isLocalDev ? applicationId : host[0];
+export const ledgerId = isLocalDev ? applicationId : 'my-app';
 
-let apiUrl = host.slice(1)
-apiUrl.unshift('api')
+//let apiUrl = host.slice(1)
+//apiUrl.unshift('api')
 
-export const httpBaseUrl = isLocalDev ? undefined : ('https://' + apiUrl.join('.') + (window.location.port ? ':' + window.location.port : '') + '/data/' + ledgerId + '/');
+export const httpBaseUrl = isLocalDev ? undefined : ('http://' + host.join('.') + (window.location.port ? ':' + window.location.port : '') /*+ '/data/' + ledgerId*/ + '/');
 
 // Unfortunately, the development server of `create-react-app` does not proxy
 // websockets properly. Thus, we need to bypass it and talk to the JSON API

Going over those changes:

• On Daml Hub, the application ID corresponds to the (first segment of the) hostname; this is not true for you, so let’s just set the app name to the correct value ('my-app').
• We don’t want to try and guess anything, really, from the hsotname, so we don’t need the apiUrl variable.
• We’re not running on https here so we need to set the url to http, otherwise it will infer a (secure) wss connection, which the browser will not be able to establish. Switching to http ensures we get a ws connection instead, which will work.
• We remove the /data/ledgerid bits from the URL.

Note that I’m expanding those changes so you can understand the issues and what’s happening here. A simple way to make it work is to just set httpBaseUrl to undefined, regardless of whether we’re in development or not.

You’ll need to rerun make build and rebuild your my-nginx Docker image after that change.

At this point the network connections will work, but there is still one problem: the JSON API is going to refuse any connection from nginx, because it is running in “production” mode and our current setup is insecure. The JSON API really wants you to use HTTPS in production between the browser and nginx (which is a good thing, but not quite an option here). You’ll need to add a flag to the json api startup command to tell it to accept HTTP connections: --allow-insecure-tokens.

With those two changes (config.tsx and --allow-insecure-tokens), you should be able to get everything working.

Just to clarify these lines, the ones that start with a single - need to be removed, and the ones that start with a single + need to be added. The - and + themselves should not be in the final file, they’re just indicators of what to add/remove.

Really each of these are just showing small changes to the original line (ex. host[0] to 'my-app'; or let to //let) but this is the way it’s represented in the diff format that Gary generated.

Gary_Verhaegen:

diff --git a/ui/src/config.ts b/ui/src/config.ts
index c2421f5..5ab2971 100755
--- a/ui/src/config.ts
+++ b/ui/src/config.ts
@@ -5,12 +5,12 @@ export const isLocalDev = process.env.NODE_ENV === 'development';
 let host = window.location.host.split('.')
 
 const applicationId = 'daml-ui-template'
-export const ledgerId = isLocalDev ? applicationId : host[0];
+export const ledgerId = isLocalDev ? applicationId : 'my-app';
 
-let apiUrl = host.slice(1)
-apiUrl.unshift('api')
+//let apiUrl = host.slice(1)
+//apiUrl.unshift('api')
 
-export const httpBaseUrl = isLocalDev ? undefined : ('https://' + apiUrl.join('.') + (window.location.port ? ':' + window.location.port : '') + '/data/' + ledgerId + '/');
+export const httpBaseUrl = isLocalDev ? undefined : ('http://' + host.join('.') + (window.location.port ? ':' + window.location.port : '') /*+ '/data/' + ledgerId*/ + '/');
 
 // Unfortunately, the development server of `create-react-app` does not proxy
 // websockets properly. Thus, we need to bypass it and talk to the JSON API

Right. For the avoidance of doubt, here is a working version of config.ts in full:

import * as jwt from "jsonwebtoken";

export const isLocalDev = process.env.NODE_ENV === 'development';

let host = window.location.host.split('.')

const applicationId = 'daml-ui-template'
export const ledgerId = isLocalDev ? applicationId : 'my-app';

//let apiUrl = host.slice(1)
//apiUrl.unshift('api')

export const httpBaseUrl = undefined;

// Unfortunately, the development server of `create-react-app` does not proxy
// websockets properly. Thus, we need to bypass it and talk to the JSON API
// directly in development mode.
export const wsBaseUrl = isLocalDev ? 'ws://localhost:7575/' : undefined;

export const createToken = (party : string) => jwt.sign({ "https://daml.com/ledger-api": { ledgerId, applicationId, admin: true, actAs: [party], readAs: [party] } }, "secret")

let loginUrl = host.slice(1)
loginUrl.unshift('login')

export const dablLoginUrl = loginUrl.join('.') + (window.location.port ? ':' + window.location.port : '') + '/auth/login?ledgerId=' + ledgerId;

export const damlPartyKey = applicationId + ".daml.party";
export const damlTokenKey = applicationId + ".daml.token";

Getting an app to work both locally and on projectdable with user friendly names at the same time was an initial pain point for us as well …

@gyorgybalazsi solved this by modifying the ui-template and implemented an approach where we can comment/uncomment a portion of our code in src/useKnownParties.tsx to

1 . get the known parties that are setup under projectdable
or
2. use a set of parties that are setup during a start-up script for the local version of the app

// Usage in components:
// import {useKnownParties} from '../UseKnownParties'
// Within the component:
// const {displayName, partyIdentifier} = useKnownParties () 

// YOU NEED THESE IMPORTS FOR RUNNING LOCALLY - START
import React, { useEffect } from 'react';
import { useLedger, useParty } from '@daml/react';
import Ledger, { PartyInfo } from '@daml/ledger';
// YOU NEED THESE IMPORTS FOR RUNNING LOCALLY - END

// YOU NEED THIS IMPORT FOR RUNNING ON DAML HB - START
/* import { PartyInfo } from '@daml/ledger';  */
// YOU NEED THIS IMPORT FOR RUNNING ON DAML HB - END

export function useKnownParties () {
    // YOU NEED THIS PART FOR RUNNING LOCALLY - START
     const [knownParties, setKnownParties] = React.useState<PartyInfo[]>([]);
    const ledger: Ledger = useLedger();
    useEffect(() => {
    const getKnownParties = async () => {
        let lst = await ledger.listKnownParties();
        setKnownParties(lst);
    } ;
    getKnownParties()
    }, [ledger]); 
    // YOU NEED THIS PART FOR RUNNING LOCALLY - END

    // YOU NEED THIS PART FOR RUNNING ON DAML HUB - START
    /*  const knownParties : PartyInfo[] = require('./parties.json')  */
    // YOU NEED THIS PART FOR RUNNING ON DAML HUB - END

    return {
        displayName : (id: string ): string => {
            return knownParties.filter((x : PartyInfo) => x.identifier === id)[0]?.displayName || id 
            },
        partyIdentifier : (displayName: string): string => {
            return knownParties.filter((x : PartyInfo) => x.displayName === displayName)[0]?.identifier || displayName 
            },
        knownPartyDisplayNames: knownParties.map((x : PartyInfo)  => x.displayName || x.identifier)
    }
}

full source code at
https://github.com/RethinkLedgers/ef-smartcontract

@Gary_Verhaegen After revising the config.ts and the json api, I can now get everything working. This is extremely useful for me to see how the UI can connect to the daml ledger and I can now proceed to explore further.

BTW, I find it very intuitive to see how you use the sdk docker image. I will also try to run triggers and scripts with the image.

I have to thank both of you @Gary_Verhaegen @anthony again for the detailed explanations and advices. A billion thanks and all the best to you all there.

It’s really great to see that it all worked out @EllisAllison!

You guys rock @Gary_Verhaegen @anthony! Always there to support forum members

EllisAllison:

I have to thank both of you @Gary_Verhaegen @anthony again for the detailed explanations and advices. A billion thanks and all the best to you all there.

You’re very welcome but I think Gary deserves most of the thanks