Security Advisory for Validator and Super Validator Operators
- Dear Community,
We are in a period of heightened security relevant activity in the industry in general and against that backdrop also within Canton Network.
For those that missed the patches related to the eMudhra and node.js RCE vulnerabilities that were announced in the last 7 days, the Canton and Splice teams have released up to date Docker images in which the affected downstream dependencies are bumped to patched versions.
In addition, we would like to remind you of security best practices for running validator nodes. The predominant attack vector we’ve observed in recent times is that attackers target validator operators’ internal networks or infrastructure to gain access to validator deployments.
For nodes that hold their namespace root keys locally and/or hold significant amounts of Canton Coin in wallets by node-internal keys, this can allow attackers to take control of the node and/or the Canton Coin in those wallets. Please stay vigilant as these types of attacks are being attempted in practice.
1. Always monitor your logs and CC balances for suspicious activity.
2. Have your incident response plans ready. When in doubt:- Turn off your validator
- Reach out to security@.... The foundation can help in various ways including by coordinating with the CEXs blocks those funds from leaving through them.
- Reach out on #validator-operators if you need broader community help.
- If you suspect your node’s root namespace key was compromised:
- Start monitoring the network for transactions signed with the compromised key.
- Un-host all parties (internal and external) from your validator (send topology transactions through a non-compromised node)
- Move the parties owned by the compromised root key to a non-compromised validator. If the root key was compromised, you will have to race against the attacker.
3. Consider all of the following hardening measures for your node deployment:- Keep your root namespace keys offline and separate:https://docs.canton.network/global-synchronizer/production-operations/key-management#offline-root-namespace-key
- Store your keys in a KMS/HSM to prevent any extraction:https://docs.canton.network/global-synchronizer/production-operations/validator-security#migrating-an-existing-validator-to-use-an-external-kms
- Store large amounts of CC and other assets only in Parties with external keys. Splice contains sweep automation that allows you to redirect funds to a wallet with external keys automatically.
- Use Covalidation to host parties across multiple nodes across multiple organizations.
- Review production access and use JIT production access controls.
- Review where credentials are being stored (wikis, docker images, local laptops, etc) and consider moving to secure vault.
- Patch all components deployed to production, including infrastructure (Splice 0.9.4 and Wallet Gateway 1.13.0)
The Canton and Splice teams are working on publishing runbooks to upgrade existing node deployments to convert to KMS-based key management.Pedro Neves
Community & Governance Manager