Sequencer endpoints timing out during DevNet validator onboarding (GKE)
Hi,
I am onboarding a Canton Network validator on GKE Autopilot (australia-southeast1) running Splice v0.5.18. Our validator successfully connects to all 14 DevNet scan nodes but cannot reach any sequencer endpoints — all connections time out.
What is working:
-
All 14 DevNet scan SVs return version 0.5.18
-
Participant node connects and authenticates successfully
-
Validator passes all init steps up to domain registration
Where it fails:
The validator gets stuck at RegisterSynchronizer with this error:
SYNC_SERVICE_BAD_CONNECTIVITY: The provided sequencer connections are inconsistent:
connection timed out after 30000ms: sequencer-0.sv-1.dev.global.canton.network.cumberland.io:443
connection timed out after 30000ms: sequencer-0.sv-1.dev.global.canton.network.tradeweb.com:443
PKIX path building failed: sequencer-0.sv-2.dev.global.canton.network.cumberland.io:443
We tested sequencer connectivity directly from inside the cluster:
bash
for host in \
sequencer-0.sv-1.dev.global.canton.network.cumberland.io \
sequencer-0.sv-2.dev.global.canton.network.cumberland.io \
sequencer-0.sv-1.dev.global.canton.network.tradeweb.com \
sequencer-0.sv-1.dev.global.canton.network.digitalasset.com \
sequencer-0.sv.dev.global.canton.network.digitalasset.com \
sequencer-0.sv-1.dev.global.canton.network.sync.global; do
printf "%s: " "$host"
curl -fsS --ipv4 --connect-timeout 10 --max-time 15 "https://$host:443" 2>/dev/null | head -1 \
|| echo "unreachable/timeout"
done
All six sequencer endpoints return unreachable/timeout with no response at all — not even a connection refused.
Our egress IP 35.244.95.187 has been whitelisted by the Canton Foundation for scan API access. Is sequencer access controlled by a separate whitelist? If so, what is the process to get 35.244.95.187 whitelisted for sequencer access on DevNet?
You’re querying the wrong migration id. sequencer-0 is for migration id 0. DevNet is on migration id 1. You can find this at SV Network Status - Canton Foundation which is also what the docs suggest here Kubernetes-Based Deployment of a Validator node — Splice documentation
Thank you @cocreature — that was exactly the issue. Setting domain-migration-id = 1 in the validator config caused the scan nodes to return sequencer-1.* endpoints instead of sequencer-0.*, and the validator successfully connected to all 14 sequencers and registered with the Global Synchronizer.