Discussions/App Development/After upgrading mainnet to Splice 0.6.7, validator-app fails during initialization while vetting packages. The participant rejects Replace VettedPackages serial Some(30) with TOPOLOGY_TOO_MANY_PENDING_TOPOLOGY_TRANSACTIONS, while the currently observed stForum ↗

After upgrading mainnet to Splice 0.6.7, validator-app fails during initialization while vetting packages. The participant rejects Replace VettedPackages serial Some(30) with TOPOLOGY_TOO_MANY_PENDING_TOPOLOGY_TRANSACTIONS, while the currently observed st

App Development2 posts114 viewsLast activity 20d ago
WU
wu_zhongxinOP
Jul 2026

Environment: mainnet

Splice version: 0.6.7

Canton version: 3.5.1

Participant: Solv-Mainnet-01

Validator backend fails during app init / vet packages.

validator-app error:

validator_backend app initialization: Vet packages failed

ABORTED: TOPOLOGY_TOO_MANY_PENDING_TOPOLOGY_TRANSACTIONS

Too many pending topology transactions on this node.

participant log:

Attempting to build, sign, and Replace VettedPackages

serial Some(30)

packages: … 185 more

TopologyManagerWriteService/Authorize failed with:

TOPOLOGY_TOO_MANY_PENDING_TOPOLOGY_TRANSACTIONS

Previous symptom:

check_establish_topology_mapping

FAILED_PRECONDITION

Condition is not yet observed

Proposed: serial = 30, packages = 180

Found: serial = 29, packages = 160

We already scaled validator-app to 0 and waited, then scaled back to 1, but the same error returned.

Need official guidance for repairing/clearing pending topology transactions/outbox for the participant.

After scaling validator-app to 0, participant no longer logs new Replace VettedPackages / TOPOLOGY_TOO_MANY_PENDING_TOPOLOGY_TRANSACTIONS within a 20m window.

Only sequencer reconnect logs remain.

However, scaling validator-app back to 1 reproduces the same vet packages failure.

How to inspect and clear/repair pending topology transactions on a Canton participant

after TOPOLOGY_TOO_MANY_PENDING_TOPOLOGY_TRANSACTIONS during VettedPackages Replace.

RA
radevG
20d ago

Could you also provide the exact steps that reproduce the issue, starting from the last known working state?
In particular:

  • The exact upgrade path and deployment steps
  • Whether the validator was running below 0.6.5 at the June 27 topology freeze
  • Any relevant participant or validator configuration changes
  • The complete first Authorize/ReplaceVettedPackages failure, including its error metadata
  • The synchronizer connection and topology-outbox status
  • Whether the issue can be reproduced in a clean environment or LocalNet

If possible, please share a minimal repository, deployment configuration, or sanitized Helm values/docker-compose setup that reproduces the failure. Without the reproduction steps and relevant configuration, it is difficult to determine whether this is a missed-LSU recovery issue, a synchronizer-outbox problem, or validator initialization state.

Since the validator appears to have missed the topology freeze, please confirm whether you have requested and followed the missed-LSU recovery instructions mentioned in the official MainNet announcement.
There is also a generic manual LSU roll-forward procedure, but it requires synchronizer-specific parameters communicated by the SVs and should not be executed speculatively.
I would avoid manually deleting topology state or constructing serial 30 until the correct recovery path has been confirmed. But if possible send a repo, I can review as well

← Back to Discussions