Improve diagnostics and resiliency of JMS RPC (w/ embedded ActiveMQ)
Description
Acceptance / Success Criteria
None
Lucidchart Diagrams
Activity
Jesse White August 9, 2019 at 9:32 PM
I've added some shell commands for managing the embedded broker: opennms-activemq:stats and opennms-activemq:purge-queue
Can be used as follows:
admin@opennms> opennms-activemq:stats
Broker statistics:
Connections: 8
Memory percent usage: 436%
Memory usage: 87.2 MiB
Memory limit: 20.0 MiB
Destination statistics (top 5):
OpenNMS.Sink.Syslog (Queue)
Message count: 352
Enqueue count: 352
Dequeue count: 0
Cursor full: true
OpenNMS.MINION.RPC.Echo (Queue)
Message count: 0
Enqueue count: 8
Dequeue count: 9
Cursor full: true
OpenNMS.MINION.RPC.Collect (Queue)
Message count: 0
Enqueue count: 8
Dequeue count: 8
Cursor full: true
OpenNMS.MINION.RPC.Poller (Queue)
Message count: 0
Enqueue count: 8
Dequeue count: 8
Cursor full: true
OpenNMS.Sink.Heartbeat (Queue)
Message count: 0
Enqueue count: 4
Dequeue count: 4
Cursor full: true
admin@opennms> opennms-activemq:purge-queue OpenNMS.Sink.Syslog
Purging: OpenNMS.Sink.Syslog
admin@opennms> opennms-activemq:stats
Broker statistics:
Connections: 8
Memory percent usage: 2%
Memory usage: 489.3 KiB
Memory limit: 20.0 MiB
...
Fixed
Details
Assignee
Jesse WhiteJesse WhiteReporter
Jesse WhiteJesse WhiteSprint
NoneFix versions
Affects versions
Priority
Major
Details
Details
Assignee
Jesse White
Jesse WhiteReporter
Jesse White
Jesse WhiteSprint
None
Fix versions
Affects versions
Priority
PagerDuty
PagerDuty Incident
PagerDuty
PagerDuty Incident
PagerDuty

PagerDuty Incident
Created August 9, 2019 at 9:29 PM
Updated August 14, 2019 at 2:57 PM
Resolved August 14, 2019 at 2:57 PM
Goals:
1) Make it possible / easy to diagnose issues related to ActiveMQ flow control using the Karaf shell
2) Ensure that RPC producer threads are not blocked forever when flow control kicks in
3) Improve documentation around JMS RPC