Contract Key comparison

HI
I want to understand the best way to match two different contract key.
I have a function that takes in a contract key as Value and tries to compare with the contract key in the created event.
Contract Key however, can be composed of signatories that is represented as a “set” in the daml java binding and being a set there is no fixed ordering.
This is causing below comparison to fail as the order of the parties in the signatories are different.

 fetchContractByContractKey(ValueOuterClass.Value contractKey, ArrayList<String> partyIds) {
        for (String partyId : partyIds) {
            for(EventOuterClass.CreatedEvent createdEvent : this.contractsCacheByParty.get(partyId).values()) {
                if(contractKey.equals(createdEvent.getContractKey())) {
                   True
                }
            }
        }
        throw new StatusRuntimeException(Status.NOT_FOUND);
   }

ValueOuterClass.Value is not normalized, it is only a serialization format; you should not use it as a data format in your program. For example, instances should not be compared for equality at all.

Instead, decode the contract key, and then compare for equality. There are a number of ways to do this.

1. If you use the fromCreatedEvent method generated by Java codegen, this produces a Contract with a key field that has been decoded.
2. Java codegen also produces numerous fromValue functions that can be used to parse a contract key, based on your knowledge of the key type.
3. You can invoke javaapi.data.Value.fromProto, which will do a partial normalization. Warning: this does not perform a full normalization! Template IDs and field names in this format may cause “equal” javaapi.data.Values to be unequal. It is safer to use codegen output when possible.

In Daml 2.3.0, there will be some new features to make writing functions like yours with Java codegen easier, depending on what contractsCacheByParty here is supposed to represent. You can experiment with these if you like by setting your sdk-version: 2.3.0-snapshot.20220509.9874.0.0798fe15.

You can get by without the new features, though, but the proper approach really depends on what contractsCacheByParty is here or what fetchContractByContractKey is supposed to return.

Stephen:

Thanks Stephen, We basically are trying to build a generic caching service to enable fetching of the contracts by key. Approach we are following is to maintain a db of (contract id, Created Event) and adjust for archived/;ledger updates using transaction service.
To enable “fetchByContractKey” query, we iterate over the Created Events to match the contract with that key.
Since this is a generic service, we dont have access to the codegen java bindings. (think of it as grpc version of your http json service)

Any pointers how we can achieve this without accessing the codegen java bindings.

brijg11:

Any pointers how we can achieve this without accessing the codegen java bindings.

Hmm, I suppose my suggestion 3 is your best bet, but you will need to write a recursive normalizer that does things like delete optional IDs and record field labels.

By the way, even in this case, you always ought to consider template ID when looking up by key. If you take a look at JSON API, exercise-by-key gRPC, or any similar, they all take a template ID as well. That’s because two unrelated templates can have the same key type. Another reason that applies to your situation is that Foo and (Party, String) end up with the same normalized runtime representation:

data Foo = Foo with
  bar: Party
  baz: String

I suspect you will find it most convenient to also use template ID as a grouping key for contractsCacheByParty, but maybe you have some fuzzier matching requirements.

Yes, we are checking for template Id also when fetching by key. Normalizing the contract key doesnt look straightforward, wondering if there are any utility or library to achieve this.

It’s not a short function, but it is conceptually straightforward. This recursive function is the essence of it.

1. Drop field names in records,
2. drop type constructors in record/variant/enum cases,
3. recur on elements in record, variant, map, textmap, list, optional cases,
4. pass through the value untouched in all other cases.

You can’t use the linked function directly yourself because it is written against com.daml.lf.value.Value, which is different from the javaapi.data.Value. And lacking match/case in Java means you’ll have more boilerplate. But the essence of normalization, deleting the type info and field names and recurring in the container cases, is exactly the same for you.

I’ve been looking at the implementation of the http-json api of fetching a contract by key (which is similar to the behavior we want) and it seems that a keyhash is created from a templateid and contract key in the database cache [here] and all querying is done against that. Is there some normalization logic that occurs to the parameter (key: Value) here before the hashing occurs that I am missing?

If yes, under what sorts of circumstances would javaapi.data.Value.fromProto not behave in a deterministic manner when called on the object from EventOuterClass.CreatedEvent.getContractKey()? We’ll need to write extensive test cases where normalization on the Contract Key is needed beyond the javaapi.data.Value.fromProto, but I can’t seems to find any obvious instances - would be much appreciated if you could point us in the right direction.

Extending to David’s question above, I see DamlRecord seem to implement “equals” method, is it reliable to depend on this method to test for equality. It seem to correctly work even when order of the signatories are reversed in the “id” field of the record’s gen_map.

digital-asset/daml/blob/d2e2c21684e2d9eade40f74990208125ad4b4ccb/language-support/java/bindings/src/main/java/com/daml/ledger/javaapi/data/DamlRecord.java#L110

      
        

            
public Map<@NonNull String, @NonNull Value> getFieldsMap() {
            
  return fieldsMap;
            
}
            

            
@Override
            
public String toString() {
            
  return "DamlRecord{" + "recordId=" + recordId + ", fields=" + fields + '}';
            
}
            

            
@Override
            
public boolean equals(Object o) {
            
  if (this == o) return true;
            
  if (o == null || getClass() != o.getClass()) return false;
            
  DamlRecord record = (DamlRecord) o;
            
  return Objects.equals(recordId, record.recordId) && Objects.equals(fields, record.fields);
            
}
            

            
@Override
            
public int hashCode() {
            

            
  return Objects.hash(recordId, fields);
        
      
    

DavidH:

Is there some normalization logic that occurs to the parameter (key: Value) here before the hashing occurs that I am missing?

In a sense, yes. What’s important is that the Value from both sides of the comparison has consistent presence or absence of the extra data. In the json-api, we do this by

1. setting verbose = true when querying from the ledger API, and
2. always including this data when parsing JSON representations of values, in ApiCodecCompressed.

Because both sides are consistent about this, the equality tests and hashing line up. I would prefer this fact about the presence of the extra data to be apparent at the type level, but at the moment, for expediency, it is not.

I can’t say how easy or hard this invariant is to preserve for your application.

DavidH:

where normalization on the Contract Key is needed beyond the javaapi.data.Value.fromProto, but I can’t seems to find any obvious instances - would be much appreciated if you could point us in the right direction.

The simplest test is when the key type is a 2-tuple. Tuples have the optional field names I’ve discussed above.

brijg11:

It seem to correctly work even when order of the signatories are reversed in the “id” field of the record’s gen_map.

Yes, but this is due to DamlGenMap#equals; DamlRecord is not doing the heavy lifting here. Using javaapi.data.Value instantly solves the exact issue that you had at top of thread, but is not sufficient to solve all potential accidental inequalities.