OpenNMS alarms pushed to Elasticsearch may have incorrect mapping

Description

Hi team. We’re using OpenNMS 30.0.1 with Elasticsearch 7.17.3. We are using Elasticsearch REST integration for events and alarms history, and have noticed that, sometimes, the index created in Elasticsearch might have an incorrect mapping of the date-type fields.

For what we have seen, since timestamps are pushed to Elasticsearch in an epoch format, the resulting index mapping can sometimes be wrongly created with a long field type instead of a date one. This results in a mapping conflict from Elastic point of view, as some fields like @first_event_time or @last_event_time will be treated like long numbers instead of dates. Even worse, this can turn Kibana’s discover function useless, as its histogram won’t be able to be rendered using a non-date field.

We have been able to work around the problem by manually set an index mapping enforcing the date fields. Would there be a way to have OpenNMS enforce this mapping against Elasticsearch? Or perhaps it is already doing it but there’s something else going on with our environment?

I’m attaching the Elastic index mappings before and after enforcing the date type field.

Acceptance / Success Criteria

None

Attachments

2

Activity

Show:

Alejandro García August 23, 2023 at 2:02 PM

Hi ! It’s not that we cannot set up an Horizon 32 lab environment, it’s that we have only been able to see this issue in production, where we’re receiving a lot of different SNMP traps and syslogs. Thus, it’s truly difficult for us to reproduce this same scenarion on lab environment.

We will be deploying OpenNMS 32 in production in the next couple of months, so we can wait and see if the problem still happens at that time, or if we were able to reproduce it before in testing environments.

Jeff Gehlbach August 1, 2023 at 4:16 PM

Hi , I’m the product manager for Horizon and Meridian. I’m glad that you’re part of our Horizon user base and hope you’re getting good value from the product, I cannot divert a developer from other work to validate this issue. Doing so would be unfair to our support customers.

If you need a hand setting up a Horizon 32 lab environment, please reach out to the user community on Discourse or Mattermost.

Anonymous July 21, 2023 at 2:11 PM

Hi Veena, it’s not so easy for us to upgrade to 32.0.0. Do you know if this issue is fixed in that version?

Veena Kannan July 11, 2023 at 8:04 PM

Would you please move to Horizon 32.0.0 release and update this issue with the results?

Details

Assignee

Reporter

HB Grooming Date

HB Backlog Status

Components

Affects versions

Priority

PagerDuty

Created July 3, 2023 at 7:25 PM
Updated August 23, 2023 at 2:02 PM