Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Randomize ES1 cluster names so tests don't join each others' clusters

Description

It looks like the Elasticsearch 1 integration tests are flapping because occasionally the Bamboo machines will launch an Elasticsearch 1 service at the same time and they will cluster with each other via broadcast discovery. This can lead to extra events appearing in the ES index. Tests will then fail with a message like:

16-Mar-2017 06:22:26 2017-03-16 06:22:26,292 WARN [pool-192-thread-1] org.opennms.smoketest.minion.AbstractSyslogTest - ES search hits was not equal to 10100: 13100 16-Mar-2017 06:22:26 java.lang.AssertionError: ES search hits was not equal to 10100: 13100 16-Mar-2017 06:22:26 at org.junit.Assert.fail(Assert.java:88) 16-Mar-2017 06:22:26 at org.junit.Assert.assertTrue(Assert.java:41) 16-Mar-2017 06:22:26 at org.opennms.smoketest.minion.AbstractSyslogTest.lambda$pollForElasticsearchEventsUsingJest$0(AbstractSyslogTest.java:291) 16-Mar-2017 06:22:26 at com.jayway.awaitility.core.CallableCondition$ConditionEvaluationWrapper.eval(CallableCondition.java:100) 16-Mar-2017 06:22:26 at com.jayway.awaitility.core.ConditionAwaiter$ConditionPoller.run(ConditionAwaiter.java:179) 16-Mar-2017 06:22:26 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 16-Mar-2017 06:22:26 at java.util.concurrent.FutureTask.run(FutureTask.java:266) 16-Mar-2017 06:22:26 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 16-Mar-2017 06:22:26 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 16-Mar-2017 06:22:26 at java.lang.Thread.run(Thread.java:745)

Acceptance / Success Criteria

None

Lucidchart Diagrams

Activity

Show:

Seth Leger March 20, 2017 at 10:22 AM

commit 2720121719ea576abf42a71ca8d0a1d354d76013

Jesse White March 17, 2017 at 4:38 PM

Seth Leger March 16, 2017 at 3:48 PM

To verify that this fix is working on the Elasticsearch 1 test, you should be able to look at the cluster name and see that it has been changed to a randomized value:

http://[your_ip_address]:9200/_cluster/health?pretty=true

{ "cluster_name" : "opennms_test_1489693324250_42735162772506_2097514481", "status" : "yellow", "timed_out" : false, "number_of_nodes" : 1, "number_of_data_nodes" : 1, "active_primary_shards" : 5, "active_shards" : 5, "relocating_shards" : 0, "initializing_shards" : 0, "unassigned_shards" : 5 }
Fixed

Details

Assignee

Reporter

Components

Sprint

Fix versions

Affects versions

Priority

PagerDuty

Created March 16, 2017 at 12:30 PM
Updated March 20, 2017 at 10:22 AM
Resolved March 17, 2017 at 4:38 PM

Flag notifications