Several days in a row, now, I've seen OpenNMS seem to just... lose all its connections to Postgres. pg_top shows no connections at all. A lot of exceptions in log files about database issues, and the UI just gives lots of 404 errors.
Restarting OpenNMS seems to get everything going again.
Am I maybe missing some bit of Postgres tuning?
I've attached the jstack output from before me restarting OpenNMS this last time; I can try to dig up log entries that may be relevant as well.
Upgraded Mina to 2.0.16 in foundation-2017 and cherry-picked to release-20.0.2.
Mike Kelly August 10, 2017 at 10:13 AM
It's also possible this could be considered a regression from ?
Mike Kelly August 10, 2017 at 10:12 AM
After discussion in Mattermost, I tried upgrading to mina-core-2.0.16.jar, and that seems to have solved this issue. I see no detectable "jumps" in the open FDs graph anymore.
Diffing `lsof` output between when I restarted OpenNMS after that JAR file replacement and now shows no leakage of "eventpoll" or "pipe" filehandles.
Several days in a row, now, I've seen OpenNMS seem to just... lose all its connections to Postgres.
pg_topshows no connections at all. A lot of exceptions in log files about database issues, and the UI just gives lots of 404 errors.Restarting OpenNMS seems to get everything going again.
Am I maybe missing some bit of Postgres tuning?
I've attached the
jstackoutput from before me restarting OpenNMS this last time; I can try to dig up log entries that may be relevant as well.