Canton 2.9.6

Canton 2.9.6 has been released on January 24, 2025. You can download the Daml Open Source edition from the Daml Connect Github Release Section. The Enterprise edition is available on Artifactory. Please also consult the full documentation of this release.

Summary

This is a maintenance release that fixes one high and four medium severity issues. Please update during the next maintenance window.

What’s New

Memory check during node startup

A memory check has been introduced when starting the node. This check compares the memory allocated to the container with the -Xmx JVM option. The goal is to ensure that the container has sufficient memory to run the application. To configure the memory check behavior, add one of the following to your configuration:

canton.parameters.startup-memory-check-config = warn  // Default behavior: Logs a warning.
canton.parameters.startup-memory-check-config = crash // Terminates the node if the check fails.
canton.parameters.startup-memory-check-config = ignore // Skips the memory check entirely.

Minor Improvements

• Two new metrics have been added that count the number of created and archived contracts observed by a participant. Contracts created as part of the standard Canton ping workflow are excluded from the tally.

participant_daml_parallel_indexer_creates
participant_daml_parallel_indexer_archivals

• A participant will now crash in exceptional cases during transaction validation instead of remaining in a failed state.
• Disabled the onboarding timeout for participants to support onboarding to domains with very large topology states without annoying warnings and timeouts.
• Removed warnings about failing periodic acknowledgements during initial domain onboarding of participants.
• Removed warnings about unhealthy sequencers during startup.

Bugfixes

(24-028, Medium): ACS export and party replication is broken after hard domain migration

Issue Description

The macros for the various steps for migrating a party look up domain parameters in the topology store, but don't filter out irrelevant domains. This results in the macro throwing an error because it finds multiple domain parameters after a hard domain migration, even though one of them comes from an inactive domain.

Affected Deployments

Participant

Affected Versions

2.8.0-2.8.11, 2.9.0-2.9.5

Impact

You cannot migrate a party to or from a participant that went through a hard domain migration.

Symptom

Calling repair.party_migration.step1_store_acs fails with the error "Found more than one (2) domain parameters set for the given domain and time!".

Workaround

The non-self-service work-around is to not call the party migration macros but replicate what these macros do.

Likeliness

The issue consistently occurs when calling the party migration macros after a hard domain migration.

Recommendation

Upgrade the involved participant nodes to the next patch release: 2.8.12 or 2.9.6.

(24-029, Medium): Domain topology manager gets stuck on too large batches

Issue Description

An off by one check fails in the topology dispatcher of the domain manager as batches are not limited to N but to N+1, while we check for N.

Affected Deployments

Domain and Domain topology manager nodes.

Affected Versions

All versions before 2.8, 2.8.0-2.8.11, 2.9.0-2.9.5

Impact

Topology transactions stop being propagated through the system.

Symptom

Participants cannot onboard to domains, parties do not appear on the domain, uploaded dars cannot be used.

Workaround

Restart domain topology manager.

Likeliness

Can happen under high topology management load which is rather unusual (adding thousands of parties at full speed).

Recommendation

Update during the next maintenance window.

(25-001, Medium): Newly onboarded participants may compute a wrong topology state during bootstrapping

Issue Description

When a participant is onboarded to a domain, the domain manager will send the topology state to the participant. The topology state is split into batches of 100. If the state contains an add and a subsequent remove of a topology transaction, and these two topology transactions are in the same batch (so less than 100 transactions apart), but the namespace certificate or identifier delegation is in a previous batch, then the participant will miss the removal of the topology transaction. In the common cases, the namespace delegation is always followed by a subsequent add, but it can happen.

Affected Deployments

Participant

Affected Versions

All versions before 2.8, 2.8.0-2.8.11, 2.9.0-2.9.5

Impact

Depends on the type of topology transaction, but the result is a fork in the topology state, which in a rare but theoretically possible case (observer nodes and participant using previously removed parties) might create a ledger fork, leading to participants disconnecting from the domain.

Symptom

If the missed transaction was a mediator domain state, then the participant will fail to submit transactions whenever it randomly selects the non-existent mediator.

Workaround

No workaround available. Manually repairing the topology state is likely possible, but not recommended.

Likeliness

Happens deterministically if the conditions are met, but the conditions are rare and require a specific sequence of events with removal of topology state.

Recommendation

Upgrade before removing topology state (disabling parties, rolling keys) or onboarding a new participant to a domain with a larger number of topology transactions that includes removals.

(25-002, Medium): Intermediate certificate renewal will delete topology state

Issue Description

A Canton node uses topology keys to sign topology transactions. The ultimate trust is tied to the root node key, which by default is held by the node, but can be moved offline. In such a case, the node may use an intermediate certificate to manage the topology state. In order to renew such intermediate certificates, the topology state needs to be re-issued in 2.x, which can be done using the convenience function node.topology.all.renew(oldKey, newKey). The convenience function contains an error that will instead of renewing the topology state, delete topology transactions of the type party to participant, mediator domain state and participant domain state (the ones that contain the replaceExisting flag).

Affected Deployments

Domain, Domain manager, Participant nodes.

Affected Versions

All versions before 2.8, 2.8.0-2.8.11, 2.9.0-2.9.5

Impact

Some of the topology state will be removed after running this operation.

Symptom

Parties, participants and mediators will be missing after running the operation.

Workaround

Manually re-add the missing parties, participants and mediators.

Likeliness

Deterministic if the convenience function is used.

Recommendation

Upgrade before renewing intermediate certificates.

(25-003, High): Identifier delegation cannot be renewed

Issue Description

A Canton node uses topology keys to sign topology transactions. The ultimate trust is tied to the root node key, which by default is held by the node, but can be moved offline. In such a case, the node may use an intermediate certificate to manage the topology state. If such an intermediate certificate is used to sign an identifier delegation (used as an intermediate certificate for a specific uid), then the identifier delegation cannot be renewed, as the renewal operation will remove the old and the new certificate from the in-memory state. Unfortunately, after a restart, the certificate could be loaded again which can cause a ledger fork.

Affected Deployments

Domain, Domain manager, Participant nodes.

Affected Versions

All versions before 2.8, 2.8.0-2.8.11, 2.9.0-2.9.5

Impact

The topology state signed with a particular key authorized by an identifier delegation will be removed from the state, and the key cannot be used to sign new transactions. After a restart of a node, the key would be loaded again, leading to a possible ledger fork.

Symptom

Topology state missing after an intermediate certificate renewal, with a possible subsequent ledger fork after a restart.

Workaround

Theoretically issue a new identifier delegation for a new key and re-create the topology state. In practice, upgrade all nodes before renewing intermediate certificates.

Likeliness

Deterministic if several intermediate certificates are used and one of them is rolled in the chain.

Recommendation

Update all nodes to a version with a fix before renewing intermediate certificates.

Compatibility

The following Canton protocol versions are supported:

Dependency	Version
Canton protocol versions	5

Canton has been tested against the following versions of its dependencies:

Dependency	Version
Java Runtime	OpenJDK 64-Bit Server VM Zulu11.72+19-CA (build 11.0.23+9-LTS, mixed mode)
Postgres	Recommended: PostgreSQL 12.22 (Debian 12.22-1.pgdg120+1) – Also tested: PostgreSQL 11.16 (Debian 11.16-1.pgdg90+1), PostgreSQL 13.18 (Debian 13.18-1.pgdg120+1), PostgreSQL 14.15 (Debian 14.15-1.pgdg120+1), PostgreSQL 15.10 (Debian 15.10-1.pgdg120+1)
Oracle	19.20.0