Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Clean up package structure

Description

This task is not directly related to performance but a clean structure makes digging through the code easier.

We are coming from a copy of the Opennms Newts code.  Since then a lot has changed and the packages don't really reflect our code structure.

This subtask is about reorganizing the classes in a clean package structure (and to rename the classes itself if their purpose is not clear).

Acceptance / Success Criteria

None

Lucidchart Diagrams

Activity

Patrick Schweizer June 5, 2020 at 6:22 PM

I moved the different bottlenecks to subtasks and made this ticket the umbrella for all optimizations.

Patrick Schweizer June 2, 2020 at 7:54 PM
Edited

Talking to ,

the goal is to make sure the integration layer is as performant as possible. The current structure is optimized for Newts / Cassandra, not for a tag based storage. We want to be able to wrote 40k samples / second.

In order to stress test and profile the system we can use the following commands:
One off:
admin@opennms> collect --persist --node 2 org.opennms.netmgt.collectd.SnmpCollector 127.0.0.1
Stress:
admin@opennms> stress-metrics --interfaces 5 --strings 2 --interval 30 --nodes 10000 --threads 10 --groups 5 --attributes 5

Patrick Schweizer June 2, 2020 at 2:40 PM

I looked into the different caches we have in the TimeseriesIntegrationLayer:

  • SearchableResourceMetadataCache: used byTimeseriesResourceStorageDao. Stores

  • TimeseriesSearcher.metricsUnderResource: (new) caches all Metrics that can be found under a resource (by a wildcard search). Caches results coming from TimeseriesStorage implementation.

  • TimeSeriesMetaDataDao.cache: caches all attributes associated with a resourceId. Caches results from the database.

Patrick Schweizer June 1, 2020 at 8:30 PM

TimeseriesMetaDataDao.storeMetadata cached only on reads, not on writes. I fixed that. We should see now a lot less writes I suppose...

Jesse White May 22, 2020 at 7:30 PM

With respect to caching, it appears that there is also room for improvement in the TimeSeriesMetaDataDao. When stressing the system with metrics - alot of time is spent inserting into the timeseries_meta table:

Preview unavailable

 

Fixed

Details

Assignee

Reporter

Sprint

Fix versions

Priority

Parent

PagerDuty

Created June 13, 2020 at 10:19 PM
Updated July 8, 2020 at 7:41 PM
Resolved July 8, 2020 at 7:41 PM

Flag notifications