503 unavailable after upgrade to 24

Description

Hello,

Yesterday, I upgraded OpenNMS to version 24.1.0 using YUM repos of OpenNMS.

Now, Web UI still shows me that Service is unavailable :

HTTP ERROR 503

Problem accessing /opennms/. Reason:

Service Unavailable

 


Powered by Jetty:// 9.4.18.v20190429

 

Here is the extract of the jetty server log :

2019-07-16 08:42:42,154 WARN [qtp2094581240-10237827] o.e.j.s.HttpChannel: /opennms/login.jsp
java.lang.NullPointerException: null
at org.opennms.container.web.bridge.proxy.ProxyFilter.doFilter(ProxyFilter.java:96) ~[?:?]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) ~[jetty-servlet-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.extremecomponents.table.filter.AbstractExportFilter.doFilter(AbstractExportFilter.java:49) ~[?:?]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) ~[jetty-servlet-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.opennms.web.account.UserSettingsFilter.doFilter(UserSettingsFilter.java:67) ~[?:?]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) ~[jetty-servlet-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.opennms.web.servlet.StoreRequestPropertiesFilter.doFilter(StoreRequestPropertiesFilter.java:79) ~[opennms-web-api-24.1.0.jar:?]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) ~[jetty-servlet-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330) ~[spring-security-web-3.2.7.RELEASE.jar:3.2.7.RELEASE]
at org.opennms.web.springframework.security.AuthFilterEnabler.doFilter(AuthFilterEnabler.java:115) ~[org.opennms.features.springframework-security-24.1.0.jar:?]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) ~[spring-security-web-3.2.7.RELEASE.jar:3.2.7.RELEASE]
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118) ~[spring-security-web-3.2.7.RELEASE.jar:3.2.7.RELEASE]
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84) ~[spring-security-web-3.2.7.RELEASE.jar:3.2.7.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) ~[spring-security-web-3.2.7.RELEASE.jar:3.2.7.RELEASE]
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113) ~[spring-security-web-3.2.7.RELEASE.jar:3.2.7.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) ~[spring-security-web-3.2.7.RELEASE.jar:3.2.7.RELEASE]
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103) ~[spring-security-web-3.2.7.RELEASE.jar:3.2.7.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) ~[spring-security-web-3.2.7.RELEASE.jar:3.2.7.RELEASE]
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113) ~[spring-security-web-3.2.7.RELEASE.jar:3.2.7.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) ~[spring-security-web-3.2.7.RELEASE.jar:3.2.7.RELEASE]
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:154) ~[spring-security-web-3.2.7.RELEASE.jar:3.2.7.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) ~[spring-security-web-3.2.7.RELEASE.jar:3.2.7.RELEASE]
at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45) ~[spring-security-web-3.2.7.RELEASE.jar:3.2.7.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) ~[spring-security-web-3.2.7.RELEASE.jar:3.2.7.RELEASE]
at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:150) ~[spring-security-web-3.2.7.RELEASE.jar:3.2.7.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) ~[spring-security-web-3.2.7.RELEASE.jar:3.2.7.RELEASE]
at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:199) ~[spring-security-web-3.2.7.RELEASE.jar:3.2.7.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) ~[spring-security-web-3.2.7.RELEASE.jar:3.2.7.RELEASE]
at org.springframework.security.web.authentication.preauth.AbstractPreAuthenticatedProcessingFilter.doFilter(AbstractPreAuthenticatedProcessingFilter.java:107) ~[spring-security-web-3.2.7.RELEASE.jar:3.2.7.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) ~[spring-security-web-3.2.7.RELEASE.jar:3.2.7.RELEASE]
at org.springframework.security.web.authentication.preauth.AbstractPreAuthenticatedProcessingFilter.doFilter(AbstractPreAuthenticatedProcessingFilter.java:107) ~[spring-security-web-3.2.7.RELEASE.jar:3.2.7.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) ~[spring-security-web-3.2.7.RELEASE.jar:3.2.7.RELEASE]
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:110) ~[spring-security-web-3.2.7.RELEASE.jar:3.2.7.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) ~[spring-security-web-3.2.7.RELEASE.jar:3.2.7.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:101) ~[org.apache.servicemix.bundles.spring-web-4.2.9.RELEASE_1.jar:?]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) ~[spring-security-web-3.2.7.RELEASE.jar:3.2.7.RELEASE]
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87) ~[spring-security-web-3.2.7.RELEASE.jar:3.2.7.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) ~[spring-security-web-3.2.7.RELEASE.jar:3.2.7.RELEASE]
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192) ~[spring-security-web-3.2.7.RELEASE.jar:3.2.7.RELEASE]
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160) ~[spring-security-web-3.2.7.RELEASE.jar:3.2.7.RELEASE]
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) ~[org.apache.servicemix.bundles.spring-web-4.2.9.RELEASE_1.jar:?]
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262) ~[org.apache.servicemix.bundles.spring-web-4.2.9.RELEASE_1.jar:?]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) ~[jetty-servlet-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.springframework.orm.hibernate3.support.OpenSessionInViewFilter.doFilterInternal(OpenSessionInViewFilter.java:231) ~[org.apache.servicemix.bundles.spring-orm-4.2.9.RELEASE_1.jar:?]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[org.apache.servicemix.bundles.spring-web-4.2.9.RELEASE_1.jar:?]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) ~[jetty-servlet-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:101) ~[org.apache.servicemix.bundles.spring-web-4.2.9.RELEASE_1.jar:?]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602) ~[jetty-servlet-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540) ~[jetty-servlet-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) ~[jetty-server-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) ~[jetty-security-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) ~[jetty-server-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) ~[jetty-server-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1700) ~[jetty-server-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) ~[jetty-server-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) ~[jetty-server-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) ~[jetty-server-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) ~[jetty-servlet-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1667) ~[jetty-server-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) ~[jetty-server-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) ~[jetty-server-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) ~[jetty-server-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220) ~[jetty-server-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:152) ~[jetty-server-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) ~[jetty-server-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.opennms.netmgt.jetty.MDCHandler.handle(MDCHandler.java:46) ~[opennms-jetty-24.1.0.jar:?]
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) ~[jetty-server-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.eclipse.jetty.server.Server.handle(Server.java:505) ~[jetty-server-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370) [jetty-server-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267) [jetty-server-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) [jetty-io-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) [jetty-io-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) [jetty-io-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) [jetty-util-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) [jetty-util-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) [jetty-util-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) [jetty-util-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) [jetty-util-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:698) [jetty-util-9.4.18.v20190429.jar:9.4.18.v20190429]
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:804) [jetty-util-9.4.18.v20190429.jar:9.4.18.v20190429]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_144]

 

What kind of information do you need to help me in order to resolve this problem ?

Environment

CentOS 7 up to date

Acceptance / Success Criteria

None

Lucidchart Diagrams

Activity

Ronny Trommer July 17, 2019 at 4:45 PM

Happy you have it running again.

JRE July 17, 2019 at 3:12 PM

I applied the solution in link 1 and then Web interface came up.

 

Thanks !

 

Ronny Trommer July 16, 2019 at 12:38 PM

This issue seems related to your configuration during the upgrade. Can we first try to verify and solve this problem in your configuration with our community support platform on https://opennms.discourse.group/c/community-support? I've found a few hints from our Knowledgebase which seem related to your issue:

As soon we identify a bug or a problem which we need to address in our codebase we can reopen an issue with a more specific problem description.

Thank you in advance

Won't Fix

Details

Assignee

Reporter

Labels

Affects versions

Priority

PagerDuty

Created June 13, 2019 at 7:59 AM
Updated July 17, 2019 at 4:45 PM
Resolved July 16, 2019 at 12:38 PM

Flag notifications