Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Notification acknowlegde workflow does not work with BSMs

Description

I found out that the auto ackknowledge function in notifd does not work like expected with BSMs.
I configured a notification based on a BSM service problem event and a notification path that has a delay of 3 minutes configured.
Even if the alarm was cleared by the BSM service problem resolved event after 1 minute, 2 minutes later (when the defined 3 minutes were over), the notification was sent.

I'm not absolutely sure why this happens, but my assumption is, that the notifd config for auto-ack notification does not work with the match I've been using.

Here the configs:

Notifd

<auto-acknowledge resolution-prefix="RESOLVED: " uei="uei.opennms.org/bsm/serviceProblemResolved" acknowledge="uei.opennms.org/bsm/serviceProblem"> <match>businessServiceId</match> </auto-acknowledge>

Desitnation path

<path name="opsgenie-3m" initial-delay="3m"> <target> <name>opsgenie</name> <autoNotify>on</autoNotify> <command>opsgenie</command> </target> </path>

Notification:

<notification name="BSM-DNS-OpsGenie" status="on" writeable="yes"> <uei>uei.opennms.org/bsm/serviceProblem</uei> <rule>(ipaddr != '0.0.0.0')</rule> <destinationPath>opsgenie-3m</destinationPath> <text-message>&lt;b&gt;BSM Root Cause Analysis:&lt;/b&gt; %parm[rootCause]%&lt;p&gt;&lt;b&gt;Current Alarms:&lt;/b&gt; &lt;a href='https://cool_url'&gt;Grafana IT Alarm Board&lt;/a&gt;&lt;p&gt;&lt;b&gt;Documentation:&lt;/b&gt; %parm[documentation]%</text-message> <subject>%parm[businessServiceName]% has problems</subject> <numeric-message>WARN</numeric-message> <varbind> <vbname>businessServiceName</vbname> <vbvalue>~^DNS</vbvalue> </varbind> </notification>

I've attached two screenshot of the notification in the UI and events that happened.

Acceptance / Success Criteria

None

Attachments

2
  • 30 Jan 2020, 02:50 PM
  • 30 Jan 2020, 02:49 PM

Confluence content

mentioned on

Lucidchart Diagrams

Activity

Show:

Marcel Fuhrmann November 16, 2020 at 10:12 AM


I've tried that without success with ONMS 27.0.0.

 

When using this

<match>parm[businessServiceId]</match>

this error messages appears after reloading the notifd:

2020-11-16 10:49:13,679 WARN [Notifd:BroadcastEventProcessor-Thread] o.o.n.c.NotificationManager: Unknown match statement businessServiceId for UEI uei.opennms.org/bsm/serviceProblem. 2020-11-16 10:49:13,679 ERROR [Notifd:BroadcastEventProcessor-Thread] o.o.n.n.BroadcastEventProcessor: Failed to auto acknowledge notice. org.postgresql.util.PSQLException: No value specified for parameter 2. at org.postgresql.core.v3.SimpleParameterList.checkAllParametersSet(SimpleParameterList.java:257) ~[postgresql-42.2.5.jar:42.2.5] at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:292) ~[postgresql-42.2.5.jar:42.2.5] at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441) ~[postgresql-42.2.5.jar:42.2.5] at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365) ~[postgresql-42.2.5.jar:42.2.5] at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:143) ~[postgresql-42.2.5.jar:42.2.5] at org.postgresql.jdbc.PgPreparedStatement.executeQuery(PgPreparedStatement.java:106) ~[postgresql-42.2.5.jar:42.2.5] at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeQuery(ProxyPreparedStatement.java:52) ~[HikariCP-2.5.1.jar:?] at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeQuery(HikariProxyPreparedStatement.java) ~[HikariCP-2.5.1.jar:?] at org.opennms.netmgt.config.NotificationManager.acknowledgeNotice(NotificationManager.java:594) ~[opennms-config-27.0.0.jar:?] at org.opennms.netmgt.notifd.BroadcastEventProcessor.automaticAcknowledge(BroadcastEventProcessor.java:344) [opennms-services-27.0.0.jar:?] at org.opennms.netmgt.notifd.BroadcastEventProcessor.onEvent(BroadcastEventProcessor.java:258) [opennms-services-27.0.0.jar:?] at org.opennms.netmgt.eventd.EventIpcManagerDefaultImpl$EventListenerExecutor$2.run(EventIpcManagerDefaultImpl.java:189) [org.opennms.features.events.daemon-27.0.0.jar:?] at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) [?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] at org.opennms.core.concurrent.LogPreservingThreadFactory$2.run(LogPreservingThreadFactory.java:106) [opennms-util-27.0.0.jar:?] at java.lang.Thread.run(Thread.java:834) [?:?]

 

 

I've also tried this one since it is used often in event configs:

<match>%parm[businessServiceId]%</match>

But also here:

2020-11-16 11:08:09,517 WARN [Notifd:BroadcastEventProcessor-Thread] o.o.n.c.NotificationManager: Unknown match statement %parm[businessServiceId]% for UEI uei.opennms.org/bsm/serviceProblem. 2020-11-16 11:08:09,517 ERROR [Notifd:BroadcastEventProcessor-Thread] o.o.n.n.BroadcastEventProcessor: Failed to auto acknowledge notice. org.postgresql.util.PSQLException: No value specified for parameter 2. at org.postgresql.core.v3.SimpleParameterList.checkAllParametersSet(SimpleParameterList.java:257) ~[postgresql-42.2.5.jar:42.2.5] at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:292) ~[postgresql-42.2.5.jar:42.2.5] at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441) ~[postgresql-42.2.5.jar:42.2.5] at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365) ~[postgresql-42.2.5.jar:42.2.5] at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:143) ~[postgresql-42.2.5.jar:42.2.5] at org.postgresql.jdbc.PgPreparedStatement.executeQuery(PgPreparedStatement.java:106) ~[postgresql-42.2.5.jar:42.2.5] at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeQuery(ProxyPreparedStatement.java:52) ~[HikariCP-2.5.1.jar:?] at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeQuery(HikariProxyPreparedStatement.java) ~[HikariCP-2.5.1.jar:?] at org.opennms.netmgt.config.NotificationManager.acknowledgeNotice(NotificationManager.java:594) ~[opennms-config-27.0.0.jar:?] at org.opennms.netmgt.notifd.BroadcastEventProcessor.automaticAcknowledge(BroadcastEventProcessor.java:344) [opennms-services-27.0.0.jar:?] at org.opennms.netmgt.notifd.BroadcastEventProcessor.onEvent(BroadcastEventProcessor.java:258) [opennms-services-27.0.0.jar:?] at org.opennms.netmgt.eventd.EventIpcManagerDefaultImpl$EventListenerExecutor$2.run(EventIpcManagerDefaultImpl.java:189) [org.opennms.features.events.daemon-27.0.0.jar:?] at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) [?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] at org.opennms.core.concurrent.LogPreservingThreadFactory$2.run(LogPreservingThreadFactory.java:106) [opennms-util-27.0.0.jar:?]

at java.lang.Thread.run(Thread.java:834) [?:?]

Alejandro Galue October 12, 2020 at 8:26 PM

AFAIK, I believe the reason why it doesn't work is that the "match" tag expects either a node ID, interface IP Address, or service name, meaning it won't work with anything else. Hence <match>businessServiceId</match> is invalid. That said, businessServiceId is a valid event parameter, meaning you should do the following instead:

<match>parm[businessServiceId]</match>

Details

Assignee

Reporter

Labels

Doc Backlog Status

Doc Backlog Grooming Date

Components

Affects versions

Priority

Parent

PagerDuty

Created January 30, 2020 at 2:51 PM
Updated June 21, 2021 at 6:21 PM

Flag notifications