NSClient thresholding failed with undeclared throwable
Description
if you configure thresholds for NSClient datacollection you get in collectd "An undeclared throwable" and "Null collection timestamp when thresholding service NSClientpp".
No thresholds events are generated.
Datacollection is working fine - see in collectd.log: 2014-04-01 11:52:02,398 INFO [Collectd-Thread-2-of-50] RrdUtils: updateRRD: updating RRD file /var/lib/opennms/rrd/snmp/1/cpuProcTime.jrb with values '1396345922:6.2524' 2014-04-01 11:52:02,402 INFO [Collectd-Thread-2-of-50] RrdUtils: updateRRD: updating RRD file /var/lib/opennms/rrd/snmp/1/cpuPrivTime.jrb with values '1396345922:4.68738' 2014-04-01 11:52:02,403 INFO [Collectd-Thread-2-of-50] RrdUtils: updateRRD: updating RRD file /var/lib/opennms/rrd/snmp/1/cpuIntrTime.jrb with values '1396345922:0' 2014-04-01 11:52:02,405 INFO [Collectd-Thread-2-of-50] RrdUtils: updateRRD: updating RRD file /var/lib/opennms/rrd/snmp/1/cpuUserTime.jrb with values '1396345922:0'
if you configure thresholds for NSClient datacollection you get in collectd "An undeclared throwable" and "Null collection timestamp when thresholding service NSClientpp".
No thresholds events are generated.
Datacollection is working fine - see in collectd.log:
2014-04-01 11:52:02,398 INFO [Collectd-Thread-2-of-50] RrdUtils: updateRRD: updating RRD file /var/lib/opennms/rrd/snmp/1/cpuProcTime.jrb with values '1396345922:6.2524'
2014-04-01 11:52:02,402 INFO [Collectd-Thread-2-of-50] RrdUtils: updateRRD: updating RRD file /var/lib/opennms/rrd/snmp/1/cpuPrivTime.jrb with values '1396345922:4.68738'
2014-04-01 11:52:02,403 INFO [Collectd-Thread-2-of-50] RrdUtils: updateRRD: updating RRD file /var/lib/opennms/rrd/snmp/1/cpuIntrTime.jrb with values '1396345922:0'
2014-04-01 11:52:02,405 INFO [Collectd-Thread-2-of-50] RrdUtils: updateRRD: updating RRD file /var/lib/opennms/rrd/snmp/1/cpuUserTime.jrb with values '1396345922:0'
collectd-configuration.xml:
<package name="NSClient">
<filter>IPADDR IPLIKE ...</filter>
<service name="NSClientpp" interval="300000" user-defined="false" status="on">
<parameter key="nsclient-collection" value="default"/>
<parameter key="thresholding-enabled" value="true"/>
<parameter key="port" value="12489"/>
<parameter key="retry" value="3"/>
<parameter key="timeout" value="3000"/>
</service>
</package>
...
<collector service="NSClientpp" class-name="org.opennms.protocols.nsclient.collector.NSClientCollector"/>
nsclient-datacollection-config.xml (sorry - german system):
<wpms>
<!-- german -->
<wpm name="ProcessorProc" keyvalue="\Prozessor(_Total)\Prozessorzeit (%)" recheckInterval="3600000">
<attrib name="\Prozessor(_Total)\Prozessorzeit (%)" alias="cpuProcTime" type="Gauge"/>
</wpm>
<wpm name="ProcessorInt" keyvalue="\Prozessor(_Total)\Prozessorzeit (%)" recheckInterval="3600000">
<attrib name="\Prozessor(_Total)\Interruptzeit (%)" alias="cpuIntrTime" type="Gauge"/>
</wpm>
<wpm name="ProcessorPriv" keyvalue="\Prozessor(_Total)\Prozessorzeit (%)" recheckInterval="3600000">
<attrib name="\Prozessor(_Total)\Privilegierte Zeit (%)" alias="cpuPrivTime" type="Gauge"/>
</wpm>
<wpm name="ProcessorUser" keyvalue="\Prozessor(_Total)\Prozessorzeit (%)" recheckInterval="3600000">
<attrib name="\Prozessor(_Total)\Benutzerzeit (%)" alias="cpuUserTime" type="Gauge"/>
</wpm>
</wpms>
poller-configuration.xml:
<service name="NSClientpp" interval="300000" user-defined="false" status="on">
<parameter key="retry" value="2"/>
<parameter key="port" value="12489"/>
<parameter key="timeout" value="3000"/>
</service>
...
<monitor service="NSClientpp" class-name="org.opennms.netmgt.poller.monitors.NsclientMonitor"/>
threshd-configuration.xml:
<package name="NSClient">
<filter>IPADDR != '0.0.0.0'</filter>
<include-range begin="1.1.1.1" end="254.254.254.254"/>
<include-range begin="::1" end="ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff" />
<service name="NSClientpp" interval="300000" user-defined="false" status="on">
<parameter key="thresholding-group" value="NSClientpp"/>
</service>
</package>
thresholds.xml:
<group name="NSClientpp" rrdRepository="/var/lib/opennms/rrd/snmp/">
<threshold type="high" ds-type="node" value="2.0" rearm="0.0"
trigger="1" filterOperator="or" ds-name="cpuProcTime"/>
</group>
collectd.log:
2014-04-01 11:51:54,334 INFO [Collectd-Thread-2-of-50] CollectableService: run: starting new collection for 10.10.10.8/NSClientpp/NSClient
2014-04-01 11:51:54,338 DEBUG [Collectd-Thread-2-of-50] NSClientCollector: Checking availability of group ProcessorProc
2014-04-01 11:51:55,334 DEBUG [Collectd-Thread-2-of-50] NSClientCollector: Group ProcessorProc is available
2014-04-01 11:51:56,333 DEBUG [Collectd-Thread-2-of-50] DefaultCollectionAgent: getStorageDir: isStoreByForeignSource = false, foreignSource = null, foreignId = null, dir = 1
2014-04-01 11:51:56,333 DEBUG [Collectd-Thread-2-of-50] NSClientCollector$NSClientCollectionResource: Adding attribute org.opennms.protocols.nsclient.collector.NSClientCollector$NSClientCollectionAttribute: NSClientCollectionAttribute cpuProcTime=6.2524 to group AttrGroupType[name=ProcessorProc, ifType=all] for null@1
2014-04-01 11:51:56,333 DEBUG [Collectd-Thread-2-of-50] NSClientCollector: Checking availability of group ProcessorInt
2014-04-01 11:51:57,346 DEBUG [Collectd-Thread-2-of-50] NSClientCollector: Group ProcessorInt is available
2014-04-01 11:51:58,345 DEBUG [Collectd-Thread-2-of-50] DefaultCollectionAgent: getStorageDir: isStoreByForeignSource = false, foreignSource = null, foreignId = null, dir = 1
2014-04-01 11:51:58,345 DEBUG [Collectd-Thread-2-of-50] NSClientCollector$NSClientCollectionResource: Adding attribute org.opennms.protocols.nsclient.collector.NSClientCollector$NSClientCollectionAttribute: NSClientCollectionAttribute cpuIntrTime=0 to group AttrGroupType[name=ProcessorInt, ifType=all] for null@1
2014-04-01 11:51:58,345 DEBUG [Collectd-Thread-2-of-50] NSClientCollector: Checking availability of group ProcessorPriv
2014-04-01 11:51:59,360 DEBUG [Collectd-Thread-2-of-50] NSClientCollector: Group ProcessorPriv is available
2014-04-01 11:52:00,376 DEBUG [Collectd-Thread-2-of-50] DefaultCollectionAgent: getStorageDir: isStoreByForeignSource = false, foreignSource = null, foreignId = null, dir = 1
2014-04-01 11:52:00,376 DEBUG [Collectd-Thread-2-of-50] NSClientCollector$NSClientCollectionResource: Adding attribute org.opennms.protocols.nsclient.collector.NSClientCollector$NSClientCollectionAttribute: NSClientCollectionAttribute cpuPrivTime=4.68738 to group AttrGroupType[name=ProcessorPriv, ifType=all] for null@1
2014-04-01 11:52:00,376 DEBUG [Collectd-Thread-2-of-50] NSClientCollector: Checking availability of group ProcessorUser
2014-04-01 11:52:01,393 DEBUG [Collectd-Thread-2-of-50] NSClientCollector: Group ProcessorUser is available
2014-04-01 11:52:02,392 DEBUG [Collectd-Thread-2-of-50] DefaultCollectionAgent: getStorageDir: isStoreByForeignSource = false, foreignSource = null, foreignId = null, dir = 1
2014-04-01 11:52:02,392 DEBUG [Collectd-Thread-2-of-50] NSClientCollector$NSClientCollectionResource: Adding attribute org.opennms.protocols.nsclient.collector.NSClientCollector$NSClientCollectionAttribute: NSClientCollectionAttribute cpuUserTime=0 to group AttrGroupType[name=ProcessorUser, ifType=all] for null@1
2014-04-01 11:52:02,393 INFO [Collectd-Thread-2-of-50] OneToOnePersister: Persisting data for resource org.opennms.protocols.nsclient.collector.NSClientCollector$NSClientCollectionResource@2cd010f8
2014-04-01 11:52:02,394 DEBUG [Collectd-Thread-2-of-50] DefaultCollectionAgent: getStorageDir: isStoreByForeignSource = false, foreignSource = null, foreignId = null, dir = 1
2014-04-01 11:52:02,394 DEBUG [Collectd-Thread-2-of-50] AttributeGroup: Visiting Group AttrGroupType[name=ProcessorProc, ifType=all] for null@1
2014-04-01 11:52:02,394 DEBUG [Collectd-Thread-2-of-50] DefaultCollectionAgent: getStorageDir: isStoreByForeignSource = false, foreignSource = null, foreignId = null, dir = 1
2014-04-01 11:52:02,394 DEBUG [Collectd-Thread-2-of-50] AttributeGroup: AttrGroupType[name=ProcessorProc, ifType=all] for null@1.shouldPersist = true
2014-04-01 11:52:02,394 DEBUG [Collectd-Thread-2-of-50] NSClientCollector$NSClientCollectionAttribute: Visiting attribute NSClientCollectionAttribute cpuProcTime=6.2524
2014-04-01 11:52:02,396 DEBUG [Collectd-Thread-2-of-50] OneToOnePersister: Persisting NSClientCollectionAttribute cpuProcTime=6.2524
2014-04-01 11:52:02,396 DEBUG [Collectd-Thread-2-of-50] OneToOnePersister: Storing attribute NSClientCollectionAttribute cpuProcTime=6.2524
2014-04-01 11:52:02,396 DEBUG [Collectd-Thread-2-of-50] DefaultCollectionAgent: getStorageDir: isStoreByForeignSource = false, foreignSource = null, foreignId = null, dir = 1
2014-04-01 11:52:02,397 DEBUG [Collectd-Thread-2-of-50] JRobinRrdStrategy: createDefinition: filename /var/lib/opennms/rrd/snmp/1/cpuProcTime.jrb already exists returning null as definition
2014-04-01 11:52:02,397 DEBUG [Collectd-Thread-2-of-50] JRobinRrdStrategy: createRRD: skipping RRD file
2014-04-01 11:52:02,398 INFO [Collectd-Thread-2-of-50] RrdUtils: updateRRD: updating RRD file /var/lib/opennms/rrd/snmp/1/cpuProcTime.jrb with values '1396345922:6.2524'
2014-04-01 11:52:02,398 DEBUG [Collectd-Thread-2-of-50] RrdUtils: updateRRD: RRD update command completed.
2014-04-01 11:52:02,400 DEBUG [Collectd-Thread-2-of-50] DefaultCollectionAgent: getStorageDir: isStoreByForeignSource = false, foreignSource = null, foreignId = null, dir = 1
2014-04-01 11:52:02,400 DEBUG [Collectd-Thread-2-of-50] AttributeGroup: Visiting Group AttrGroupType[name=ProcessorPriv, ifType=all] for null@1
2014-04-01 11:52:02,400 DEBUG [Collectd-Thread-2-of-50] DefaultCollectionAgent: getStorageDir: isStoreByForeignSource = false, foreignSource = null, foreignId = null, dir = 1
2014-04-01 11:52:02,400 DEBUG [Collectd-Thread-2-of-50] AttributeGroup: AttrGroupType[name=ProcessorPriv, ifType=all] for null@1.shouldPersist = true
2014-04-01 11:52:02,400 DEBUG [Collectd-Thread-2-of-50] NSClientCollector$NSClientCollectionAttribute: Visiting attribute NSClientCollectionAttribute cpuPrivTime=4.68738
2014-04-01 11:52:02,401 DEBUG [Collectd-Thread-2-of-50] OneToOnePersister: Persisting NSClientCollectionAttribute cpuPrivTime=4.68738
2014-04-01 11:52:02,401 DEBUG [Collectd-Thread-2-of-50] OneToOnePersister: Storing attribute NSClientCollectionAttribute cpuPrivTime=4.68738
2014-04-01 11:52:02,401 DEBUG [Collectd-Thread-2-of-50] DefaultCollectionAgent: getStorageDir: isStoreByForeignSource = false, foreignSource = null, foreignId = null, dir = 1
2014-04-01 11:52:02,401 DEBUG [Collectd-Thread-2-of-50] JRobinRrdStrategy: createDefinition: filename /var/lib/opennms/rrd/snmp/1/cpuPrivTime.jrb already exists returning null as definition
2014-04-01 11:52:02,401 DEBUG [Collectd-Thread-2-of-50] JRobinRrdStrategy: createRRD: skipping RRD file
2014-04-01 11:52:02,402 INFO [Collectd-Thread-2-of-50] RrdUtils: updateRRD: updating RRD file /var/lib/opennms/rrd/snmp/1/cpuPrivTime.jrb with values '1396345922:4.68738'
2014-04-01 11:52:02,402 DEBUG [Collectd-Thread-2-of-50] RrdUtils: updateRRD: RRD update command completed.
2014-04-01 11:52:02,402 DEBUG [Collectd-Thread-2-of-50] DefaultCollectionAgent: getStorageDir: isStoreByForeignSource = false, foreignSource = null, foreignId = null, dir = 1
2014-04-01 11:52:02,402 DEBUG [Collectd-Thread-2-of-50] AttributeGroup: Visiting Group AttrGroupType[name=ProcessorInt, ifType=all] for null@1
2014-04-01 11:52:02,402 DEBUG [Collectd-Thread-2-of-50] DefaultCollectionAgent: getStorageDir: isStoreByForeignSource = false, foreignSource = null, foreignId = null, dir = 1
2014-04-01 11:52:02,403 DEBUG [Collectd-Thread-2-of-50] AttributeGroup: AttrGroupType[name=ProcessorInt, ifType=all] for null@1.shouldPersist = true
2014-04-01 11:52:02,403 DEBUG [Collectd-Thread-2-of-50] NSClientCollector$NSClientCollectionAttribute: Visiting attribute NSClientCollectionAttribute cpuIntrTime=0
2014-04-01 11:52:02,403 DEBUG [Collectd-Thread-2-of-50] OneToOnePersister: Persisting NSClientCollectionAttribute cpuIntrTime=0
2014-04-01 11:52:02,403 DEBUG [Collectd-Thread-2-of-50] OneToOnePersister: Storing attribute NSClientCollectionAttribute cpuIntrTime=0
2014-04-01 11:52:02,403 DEBUG [Collectd-Thread-2-of-50] DefaultCollectionAgent: getStorageDir: isStoreByForeignSource = false, foreignSource = null, foreignId = null, dir = 1
2014-04-01 11:52:02,403 DEBUG [Collectd-Thread-2-of-50] JRobinRrdStrategy: createDefinition: filename /var/lib/opennms/rrd/snmp/1/cpuIntrTime.jrb already exists returning null as definition
2014-04-01 11:52:02,403 DEBUG [Collectd-Thread-2-of-50] JRobinRrdStrategy: createRRD: skipping RRD file
2014-04-01 11:52:02,403 INFO [Collectd-Thread-2-of-50] RrdUtils: updateRRD: updating RRD file /var/lib/opennms/rrd/snmp/1/cpuIntrTime.jrb with values '1396345922:0'
2014-04-01 11:52:02,404 DEBUG [Collectd-Thread-2-of-50] RrdUtils: updateRRD: RRD update command completed.
2014-04-01 11:52:02,404 DEBUG [Collectd-Thread-2-of-50] DefaultCollectionAgent: getStorageDir: isStoreByForeignSource = false, foreignSource = null, foreignId = null, dir = 1
2014-04-01 11:52:02,404 DEBUG [Collectd-Thread-2-of-50] AttributeGroup: Visiting Group AttrGroupType[name=ProcessorUser, ifType=all] for null@1
2014-04-01 11:52:02,404 DEBUG [Collectd-Thread-2-of-50] DefaultCollectionAgent: getStorageDir: isStoreByForeignSource = false, foreignSource = null, foreignId = null, dir = 1
2014-04-01 11:52:02,404 DEBUG [Collectd-Thread-2-of-50] AttributeGroup: AttrGroupType[name=ProcessorUser, ifType=all] for null@1.shouldPersist = true
2014-04-01 11:52:02,404 DEBUG [Collectd-Thread-2-of-50] NSClientCollector$NSClientCollectionAttribute: Visiting attribute NSClientCollectionAttribute cpuUserTime=0
2014-04-01 11:52:02,404 DEBUG [Collectd-Thread-2-of-50] OneToOnePersister: Persisting NSClientCollectionAttribute cpuUserTime=0
2014-04-01 11:52:02,404 DEBUG [Collectd-Thread-2-of-50] OneToOnePersister: Storing attribute NSClientCollectionAttribute cpuUserTime=0
2014-04-01 11:52:02,405 DEBUG [Collectd-Thread-2-of-50] DefaultCollectionAgent: getStorageDir: isStoreByForeignSource = false, foreignSource = null, foreignId = null, dir = 1
2014-04-01 11:52:02,405 DEBUG [Collectd-Thread-2-of-50] JRobinRrdStrategy: createDefinition: filename /var/lib/opennms/rrd/snmp/1/cpuUserTime.jrb already exists returning null as definition
2014-04-01 11:52:02,405 DEBUG [Collectd-Thread-2-of-50] JRobinRrdStrategy: createRRD: skipping RRD file
2014-04-01 11:52:02,405 INFO [Collectd-Thread-2-of-50] RrdUtils: updateRRD: updating RRD file /var/lib/opennms/rrd/snmp/1/cpuUserTime.jrb with values '1396345922:0'
2014-04-01 11:52:02,405 DEBUG [Collectd-Thread-2-of-50] RrdUtils: updateRRD: RRD update command completed.
2014-04-01 11:52:02,406 DEBUG [Collectd-Thread-2-of-50] DefaultCollectionAgent: getStorageDir: isStoreByForeignSource = false, foreignSource = null, foreignId = null, dir = 1
2014-04-01 11:52:02,406 DEBUG [Collectd-Thread-2-of-50] AttributeGroup: Visiting Group AttrGroupType[name=ProcessorProc, ifType=all] for null@1
2014-04-01 11:52:02,406 DEBUG [Collectd-Thread-2-of-50] NSClientCollector$NSClientCollectionAttribute: Visiting attribute NSClientCollectionAttribute cpuProcTime=6.2524
2014-04-01 11:52:02,408 DEBUG [Collectd-Thread-2-of-50] ThresholdingSet: hasThresholds: node@cpuProcTime? true
2014-04-01 11:52:02,408 DEBUG [Collectd-Thread-2-of-50] ThresholdingVisitor: visitAttribute: storing value 6.2524 for attribute named cpuProcTime
2014-04-01 11:52:02,408 DEBUG [Collectd-Thread-2-of-50] DefaultCollectionAgent: getStorageDir: isStoreByForeignSource = false, foreignSource = null, foreignId = null, dir = 1
2014-04-01 11:52:02,408 DEBUG [Collectd-Thread-2-of-50] AttributeGroup: Visiting Group AttrGroupType[name=ProcessorPriv, ifType=all] for null@1
2014-04-01 11:52:02,409 DEBUG [Collectd-Thread-2-of-50] NSClientCollector$NSClientCollectionAttribute: Visiting attribute NSClientCollectionAttribute cpuPrivTime=4.68738
2014-04-01 11:52:02,409 DEBUG [Collectd-Thread-2-of-50] DefaultCollectionAgent: getStorageDir: isStoreByForeignSource = false, foreignSource = null, foreignId = null, dir = 1
2014-04-01 11:52:02,409 DEBUG [Collectd-Thread-2-of-50] AttributeGroup: Visiting Group AttrGroupType[name=ProcessorInt, ifType=all] for null@1
2014-04-01 11:52:02,409 DEBUG [Collectd-Thread-2-of-50] NSClientCollector$NSClientCollectionAttribute: Visiting attribute NSClientCollectionAttribute cpuIntrTime=0
2014-04-01 11:52:02,409 DEBUG [Collectd-Thread-2-of-50] DefaultCollectionAgent: getStorageDir: isStoreByForeignSource = false, foreignSource = null, foreignId = null, dir = 1
2014-04-01 11:52:02,409 DEBUG [Collectd-Thread-2-of-50] AttributeGroup: Visiting Group AttrGroupType[name=ProcessorUser, ifType=all] for null@1
2014-04-01 11:52:02,409 DEBUG [Collectd-Thread-2-of-50] NSClientCollector$NSClientCollectionAttribute: Visiting attribute NSClientCollectionAttribute cpuUserTime=0
2014-04-01 11:52:02,411 WARN [Collectd-Thread-2-of-50] CollectableService: run: failed collection for 10.10.10.8/NSClientpp/NSClient
2014-04-01 11:52:02,411 ERROR [Collectd-Thread-2-of-50] CollectableService: An undeclared throwable was caught during data collection for interface 10.10.10.8/NSClientpp
org.opennms.netmgt.collectd.CollectionException: An undeclared throwable was caught during data collection for interface 10.10.10.8/NSClientpp
at org.opennms.netmgt.collectd.CollectableService.doCollection(CollectableService.java:396)
at org.opennms.netmgt.collectd.CollectableService.run(CollectableService.java:298)
at org.opennms.netmgt.scheduler.LegacyScheduler$1.run(LegacyScheduler.java:201)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at org.opennms.core.concurrent.LogPreservingThreadFactory$3.run(LogPreservingThreadFactory.java:107)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.IllegalArgumentException: CollectionResourceWrapper: Null collection timestamp when thresholding service NSClientpp on node 1 (10.10.10.8)
at org.opennms.netmgt.threshd.CollectionResourceWrapper.<init>(CollectionResourceWrapper.java:129)
at org.opennms.netmgt.threshd.CollectorThresholdingSet.applyThresholds(CollectorThresholdingSet.java:97)
at org.opennms.netmgt.threshd.ThresholdingVisitor.completeResource(ThresholdingVisitor.java:189)
at org.opennms.netmgt.collectd.AbstractCollectionResource.visit(AbstractCollectionResource.java:119)
at org.opennms.protocols.nsclient.collector.NSClientCollector$NSClientCollectionSet.visit(NSClientCollector.java:248)
at org.opennms.netmgt.collectd.CollectableService.doCollection(CollectableService.java:383)
... 6 more
2014-04-01 11:52:02,412 DEBUG [Collectd-Thread-2-of-50] CollectableService: run: change in collection status, generating event.
2014-04-01 11:52:02,420 DEBUG [Collectd-Thread-2-of-50] CollectableService: sendEvent: Sent event uei.opennms.org/nodes/dataCollectionFailed for 1/10.10.10.8/NSClientpp