Canton 2.8.11

Canton 2.8.11 has been released on November 26, 2024. You can download the Daml Open Source edition from the Daml Connect Github Release Section. The Enterprise edition is available on Artifactory. Please also consult the full documentation of this release.

Summary

This is a maintenance release that provides performance improvements and fixes minor bugs.

What’s New

Minor Improvements

• Two new metrics have been added that count the number of created and archived contracts observed by a participant. Contracts created as part of the standard Canton ping workflow are excluded from the tally.

participant_daml_parallel_indexer_creates
participant_daml_parallel_indexer_archivals

• Two more metrics have been added to the db storage metrics: exectime and load to capture the execution time and load of the database storage pool.
• We added batch insertion to the single dimension event log to reduce the database load and improve performance.
• We reduced latency on the sequencer for processing and sequencing events from other nodes.

Node's Exit on Fatal Failures

Since v2.8.4 when a node encounters a fatal failure that Canton cannot handle gracefully yet, the node will exit/stop the process and relies on an external process or service monitor to restart the node's process.

Now a node also exits on failed transition from a passive replica to an active replica, which may result in an invalid state of the node.

The crashing on fatal failures can be reverted by setting: canton.parameters.exit-on-fatal-failures = false in the configuration.

Bugfixes

(24-027, Low): Bootstrap of the domain fails if the mediator or sequencer share the same key as the domain manager

Issue Description

Domain bootstrapping fails with a KeyAlreadyExists error when the signing key is shared between the mediator/sequencer and the domain manager.

Impact

You cannot bootstrap a domain when the signing key is shared between the domain manager and mediator or sequencer nodes.

Symptom

After calling bootstrap_domain we get a KeyAlreadyExists error.

Workaround

Use different signing keys for the mediator, sequencer and the domain manager.

Likeliness

This issue consistently occurs whenever we attempt to bootstrap a domain where the domain manager's signing key is shared with the mediator or the sequencer.

Recommendation

Upgrade to 2.8.11 when affected by this limitation.

(24-025, Low): Commands for single key rotation for sequencer and mediator node fail

Issue Description

The current commands for single key rotation with sequencer and mediator nodes (rotate_node_key and rotate_kms_node_key) fail because they do not have the necessary domain manager reference needed to find the old key and export the new key.

Affected Deployments

Sequencer and mediator nodes

Affected Versions

All 2.3-2.7, 2.8.0-2.8.10, 2.9.0-2.9.4

Impact

Key rotation for individual keys with sequencer or mediator nodes cannot be performed using the provided commands.

Symptom

Current single key rotation for sequencer and mediator, with commands rotate_node_key and rotate_kms_node_key, fails with an IllegalStateException: key xyz does not exist.

Workaround

Use the domain manager to rotate a mediator or sequencer key, or use the rotate_node_keys command with a domain manager reference to rotate all keys.

Likeliness

This issue consistently occurs when trying to rotate keys individually with sequencer or mediator nodes in a distributed environment.

Recommendation

Upgrade to 2.8.11 when affected, and run the rotate_node_key and rotate_kms_node_key commands with a reference to the domain topology manager to successfully perform the rotation.

(24-021, Medium): Participant replica fails to become active

Issue Description

A participant replica fails to become active under certain database network conditions. The previously active replica fails to fully transition to passive due to blocked database connection health checks, which leaves the other replica to fail to transition to active. Eventually the database health checks get unblocked and the replica transitions to passive, but the other replica does not recover from the previous active transition failure, which leaves both replicas passive.

Affected Deployments

Participant

Affected Versions

All 2.3-2.7 2.8.0-2.8.10 2.9.0-2.9.4

Impact

Both participant replicas remain passive and do not serve transactions.

Symptom

The transition to active failed on a participant due to maximum retries exhausted:

2024-09-02T07:08:56,178Z participant2 [c.d.c.r.DbStorageMulti:participant=participant1] [canton-env-ec-36] ERROR dd:[ ] c.d.c.r.DbStorageMulti:participant=participant1 tid:effa59a8f7ddec2e132079f2a4bd9885 - Failed to transition replica state
com.daml.timer.RetryStrategy$TooManyAttemptsException: Gave up trying after Some(3000) attempts and 300.701142545 seconds.

Workaround

Restart both replicas of the participant

Likeliness

Possible under specific database connection issues

Recommendation

Upgrade to the next patch release during regular maintenance window.

Compatibility

The following Canton protocol versions are supported:

Dependency	Version
Canton protocol versions	3, 4, 5

Canton has been tested against the following versions of its dependencies:

Dependency	Version
Java Runtime	OpenJDK 64-Bit Server VM Zulu11.70+15-CA (build 11.0.22+7-LTS, mixed mode)
Postgres	Recommended: PostgreSQL 12.22 (Debian 12.22-1.pgdg120+1) – Also tested: PostgreSQL 11.16 (Debian 11.16-1.pgdg90+1), PostgreSQL 13.18 (Debian 13.18-1.pgdg120+1), PostgreSQL 14.15 (Debian 14.15-1.pgdg120+1), PostgreSQL 15.10 (Debian 15.10-1.pgdg120+1)
Oracle	19.20.0