ConnectionX:participant=participant/synchronizerAlias=Synchronizer ‘global’/pool=main/connection=SEQ::Orb-1-LP-1::122015aa65241a458b7ea97edbde81ac778f07ad36c8b22238c30f1b29aed98f0e8b-0",“thread_name”:“participant-Synchronizer__global_-main-wallclock-0”,“level”:“INFO”,“trace-id”:“0ba995dbf5aa06e2f8e7e2df49d464d6”,“span-name”:“env_start_reconnect”,“span-id”:“b8872f7aaa2977fd”}
Aborting due to java.lang.OutOfMemoryError: Java heap space

A fatal error has been detected by the Java Runtime Environment:

Internal Error (debug.cpp:271), pid=7, tid=267

fatal error: OutOfMemory encountered: Java heap space

JRE version: OpenJDK Runtime Environment Temurin-21.0.10+7 (21.0.10+7) (build 21.0.10+7-LTS)

Java VM: OpenJDK 64-Bit Server VM Temurin-21.0.10+7 (21.0.10+7-LTS, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, linux-amd64)

Core dump will be written. Default location: Core dumps may be processed with “/usr/share/apport/apport -p%p -s%s -c%c -d%d -P%P -u%u -g%g -F%F – %E” (or dumping to /app/core.7)

An error report file with more information is saved as:

/app/hs_err_pid7.log

[300.278s][warning][os] Loading hsdis library failed

Package decentralized-canton-sync/docker/canton-participant · GitHub “/usr/bin/tini – /a…” 8 minutes ago Up 3 minutes (healthy) 5001-5002/tcp, 7575/tcp, 10013/tcp splice-validator-participant-1

gm gm, seems that participant process here is running out of Java heap space and triggering a JVM fatal error. JVM wrote a crash dump to /app/hs_err_pid7.log and the container was then restarted by Docker (which is why you see it back as healthy and Docker restarted it but it will OOM again under the same conditions and no explicit heap flags are set.

Maybe in Docker Compose override or environment config for the splice-validator-participant service add JVM options via the JAVA_OPTS or _JAVA_OPTIONS environment variable.

The -XX:+HeapDumpOnOutOfMemoryError and -XX:HeapDumpPath flags are valuable to keep as they’ll write a heap dump on the next OOM so you can analyze it with tools like Eclipse MAT if the issue persists.