Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

GNMI OpenConfig Connector - TRANSIENT_FAILURE Issues - Unable to establish connection

Description

Setup:

  • Running Horizon 32.0.5 on Container


Noticing the below OpenConfig Server at `redacted-ipaddress is not in ready state, current state TRANSIENT_FAILURE exception in the karaf.log file and nothing gets initiated for datacollection, when we enabled OpenConfig GNMI

./karaf.log-2023-12-15T12:10:42,109 | INFO | Main | FileUpdateWatcher | 281 - org.opennms.core.lib - 32.0.5 | started watcher thread for file : openconfig-gnmi-telemetry.groovy ./karaf.log-2023-12-15T12:10:42,179 | INFO | Main | FileUpdateWatcher | 281 - org.opennms.core.lib - 32.0.5 | started watcher thread for file : sflow-host.groovy ./karaf.log-2023-12-15T12:10:42,389 | DEBUG | Main | OpenConfigConnector | 421 - org.opennms.features.telemetry.protocols.openconfig.connector - 32.0.5 | Starting new OpenConfig client for: `redacted-ipaddress` ./karaf.log-2023-12-15T12:10:42,815 | WARN | Main | OpenConfigClientImpl | 389 - org.opennms.features.openconfig.telemetry-client - 32.0.5 | OpenConfig Server at `redacted-ipaddress` is not in ready state, current state IDLE, retrying.. ./karaf.log:2023-12-15T12:10:43,815 | WARN | Main | OpenConfigClientImpl | 389 - org.opennms.features.openconfig.telemetry-client - 32.0.5 | OpenConfig Server at `redacted-ipaddress` is not in ready state, current state TRANSIENT_FAILURE, retrying.. ./karaf.log:2023-12-15T12:10:44,816 | WARN | Main | OpenConfigClientImpl | 389 - org.opennms.features.openconfig.telemetry-client - 32.0.5 | OpenConfig Server at `redacted-ipaddress` is not in ready state, current state TRANSIENT_FAILURE, retrying.. ./karaf.log:2023-12-15T12:10:45,817 | WARN | Main | OpenConfigClientImpl | 389 - org.opennms.features.openconfig.telemetry-client - 32.0.5 | OpenConfig Server at `redacted-ipaddress` is not in ready state, current state TRANSIENT_FAILURE, retrying.. ./karaf.log:2023-12-15T12:10:46,818 | WARN | Main | OpenConfigClientImpl | 389 - org.opennms.features.openconfig.telemetry-client - 32.0.5 | OpenConfig Server at `redacted-ipaddress is not in ready state, current state TRANSIENT_FAILURE, retrying.. ./karaf.log-2023-12-15T12:10:47,819 | INFO | Main | OpenConfigClientImpl | 389 - org.opennms.features.openconfig.telemetry-client - 32.0.5 | Closing the OpenConfig Client at /`redacted-ipaddress` ./karaf.log-2023-12-15T12:10:47,832 | WARN | pool-78-thread-1 | OpenConfigClientImpl | 389 - org.opennms.features.openconfig.telemetry-client - 32.0.5 | OpenConfig Server at `redacted-ipaddress` is not in ready state, current state IDLE, retrying.. ./karaf.log:2023-12-15T12:10:48,835 | WARN | pool-78-thread-1 | OpenConfigClientImpl | 389 - org.opennms.features.openconfig.telemetry-client - 32.0.5 | OpenConfig Server at `redacted-ipaddress` is not in ready state, current state TRANSIENT_FAILURE, retrying.. ./karaf.log:2023-12-15T12:10:49,835 | WARN | pool-78-thread-1 | OpenConfigClientImpl | 389 - org.opennms.features.openconfig.telemetry-client - 32.0.5 | OpenConfig Server at `redacted-ipaddress` is not in ready state, current state TRANSIENT_FAILURE, retrying.. ./karaf.log:2023-12-15T12:10:50,836 | WARN | pool-78-thread-1 | OpenConfigClientImpl | 389 - org.opennms.features.openconfig.telemetry-client - 32.0.5 | OpenConfig Server at `redacted-ipaddress` is not in ready state, current state TRANSIENT_FAILURE, retrying.. ./karaf.log:2023-12-15T12:10:51,837 | WARN | pool-78-thread-1 | OpenConfigClientImpl | 389 - org.opennms.features.openconfig.telemetry-client - 32.0.5 | OpenConfig Server at `redacted-ipaddress` is not in ready state, current state TRANSIENT_FAILURE, retrying.. ./karaf.log-2023-12-15T12:11:07,350 | DEBUG | pool-68-thread-1 | IfTttDaemon | 376 - org.opennms.features.ifttt - 32.0.5 | Disabled - skipping alarm polling. ./karaf.log-2023-12-15T12:11:07,351 | DEBUG | pool-68-thread-1 | IfTttDaemon | 376 - org.opennms.features.ifttt - 32.0.5 | Run complete. Next poll in 30s.

 

telemetryd-configuration.xml file

<!-- OpenConfig --> <connector name="OpenConfig-Connector" class-name="org.opennms.netmgt.telemetry.protocols.openconfig.connector.OpenConfigConnector" service-name="OpenConfig" queue="OpenConfig" enabled="true"> <package name="OpenConfig-Default"> <filter>IPADDR =='redacted-ipaddress'</filter> <parameter key="port" value="${requisition:oc.port|32767}"/> <parameter key="paths" value="openconfig-interfaces:/interfaces/interface[name=ae800]/subinterfaces/subinterface/ipv4/state/counters"/> <parameter key="frequency" value="300000000000" /> <parameter key="mode" value="gnmi"/> <parameter key="username" value="test.account"/> <parameter key="password" value="redacted-password"/> <parameter key="retries" value="2"/> <parameter key="interval" value="300"/> <parameter key="tls.enabled" value="true"/> <parameter key="tls.trust.cert.path" value="/opt/opennms/etc/certificate_chain.pem"/> </package> </connector> <queue name="OpenConfig"> <adapter name="OpenConfig-Adapter" class-name="org.opennms.netmgt.telemetry.protocols.openconfig.adapter.OpenConfigAdapter" enabled="true"> <parameter key="script" value="/opt/opennms/etc/telemetryd-adapters/openconfig-gnmi-telemetry.groovy"/> <parameter key="mode" value="gnmi"/> <package name="OpenConfig-Default"> <rrd step="300"> <rra>RRA:AVERAGE:0.5:1:2016</rra> <rra>RRA:AVERAGE:0.5:12:1488</rra> <rra>RRA:AVERAGE:0.5:288:366</rra> <rra>RRA:MAX:0.5:288:366</rra> <rra>RRA:MIN:0.5:288:366</rra> </rrd> </package> </adapter> </queue>

 

I’ve validated the GNMI Connection to the network device, from Underlying host as well as from within the Horizon Container running using gNMIc binary and it was working correctly and the same set of basic auth and certs are being used, which leads me to believe that there could be some issue with OpenNMS GNMI Client.

gnmic -a test-node:32767 -u test -p "***" --tls-ca /opt/opennms/etc/certificate_chain.pem subscribe --path "openconfig-interfaces:/interfaces/interface[name=ae800]/state/oper-status"

Acceptance / Success Criteria

None

Attachments

1
  • 15 Dec 2023, 12:59 PM

Activity

Show:

Chandra Gorantla January 11, 2024 at 10:39 PM

Fixed

Details

Assignee

Reporter

Labels

HB Grooming Date

HB Backlog Status

Components

Sprint

Fix versions

Affects versions

Priority

PagerDuty

Created December 15, 2023 at 12:35 PM
Updated January 11, 2024 at 10:40 PM
Resolved January 11, 2024 at 10:40 PM

Flag notifications