Alarmd get stucks in dead-lock and stops processing events
Description
I've seen a few cases now where alarmd gets stuck with a stack similar to:
The query is stuck waiting on Postgres.
Further inspection shows that the query is actually stuck waiting for a lock, which held by another transaction that is not yet committed.
Looking back at the thread dump, we can find another thread with an open transaction, which is open, and will remain open until the other thread unblocks - hence the deadlock.
I've seen a few cases now where alarmd gets stuck with a stack similar to:
The query is stuck waiting on Postgres.
Further inspection shows that the query is actually stuck waiting for a lock, which held by another transaction that is not yet committed.
Looking back at the thread dump, we can find another thread with an open transaction, which is open, and will remain open until the other thread unblocks - hence the deadlock.