Canton participant node warn message: DB_STORAGE_DEGRADATION

Hi team,

I’m running some performance tests in canton-rc6 but my seems like my participant node failed with below errors:

WARN  c.d.c.r.DbStorageSingle:participant=PARTICIPANT_NODE_1 tid:f7c22eee8942c617e5252918feb74e7b - DB_STORAGE_DEGRADATION(13,f7c22eee): A database task was rejected from the database task queue.
The full error message from the task queue is:
java.util.concurrent.RejectedExecutionException: Task slick.basic.BasicBackend$DatabaseDef$$anon$3@41e8dcee rejected from slick.util.AsyncExecutorWithMetrics$$anon$1@76876d89[Running, pool size = 50, active threads = 50, queued tasks = 1000, completed tasks = 94539] err-context:{messageFromSlick=java.util.concurrent.RejectedExecutionException: Task slick.basic.BasicBackend$DatabaseDef$$anon$3@41e8dcee rejected from slick.util.AsyncExecutorWithMetrics$$anon$1@76876d89[Running, pool size = 50, active threads = 50, queued tasks = 1000, completed tasks = 94539], location=RetryUtil.scala:99}
WARN  c.d.c.r.DbStorageSingle:participant=PARTICIPANT_NODE_1 tid:fb18bf1e557b6b5c8fadb3c1f25f7ee3 - 

Could you tell me what it means ? How to avoid this message?

Cheers,

Jean-Paul

Hi Jean-Paul,

It looks like the database task queue is full. We have some documentation about this here: Frequently Asked Questions — Canton 1.0.0-SNAPSHOT documentation.

The summary is that you don’t need to avoid this message – it doesn’t necessarily mean that anything is wrong. However, if this is happening frequently, you might want to consider increasing the number of parallel database connections. You can do this by changing the config options for the database, for example (for postgres): Static Configuration — Canton 0.27.1 documentation.

Noted with thanks. By any chance, would you know what could be the maximum connections supported? Because on my postgres configuration file, I used 100.

The limit for the number of database connections is configurable on the Postgres (/database) side. As far as I am aware, there is no limit from the Canton side.