swiftmq crash

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

swiftmq crash

gianfranco.soft
Hello,
we are currently experiencing a crash of the swiftmq which appears to be random.

We are trying to understand the reasons of the failure from the log files, but they don't seem to report anything special.

Please find attached the log files relative to a crash happened on 2016-03-17 at 16:42:23.818.

The application was then restarted on 2016-03-18 at 08:14:08.258.

info.log

Can you please help?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: swiftmq crash

IIT Software
Administrator
What does "crash" mean? A crash of the JVM? The info.log is ok. If there is anything then it's in the error.log or System.err (stderr), if you have that...
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: swiftmq crash

gianfranco.soft
A crash in the sense that the process is not running anymore.

error.log does not have an entry at the time of the crash.

System.err is not present. How can this be enabled?

In case of a JVM crash, I would expect a .pid file being produced. Where should it be?

Anything else I should look for?

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: swiftmq crash

IIT Software
Administrator
Sorry, I missed that:

2016-03-17 16:42:18.729/SwiftletManager/INFORMATION/stopAllSwiftlets
2016-03-17 16:42:18.754/192.168.0.90:33958/BlockingHandler/INFORMATION/Exception, EXITING: java.io.IOException: End-of-Stream reached
2016-03-17 16:42:18.754/192.168.0.90:33860/BlockingHandler/INFORMATION/Exception, EXITING: java.net.SocketException: Socket closed
2016-03-17 16:42:18.755/192.168.0.90:33130/BlockingHandler/INFORMATION/Exception, EXITING: java.net.SocketException: Socket closed
2016-03-17 16:42:18.755/192.168.0.90:33294/BlockingHandler/INFORMATION/Exception, EXITING: java.net.SocketException: Socket closed
2016-03-17 16:42:18.756/192.168.0.90:33970/BlockingHandler/INFORMATION/Exception, EXITING: java.net.SocketException: Socket closed
2016-03-17 16:42:18.756/192.168.0.90:33868/BlockingHandler/INFORMATION/Exception, EXITING: java.net.SocketException: Socket closed
2016-03-17 16:42:18.756/192.168.0.137:40199/BlockingHandler/INFORMATION/Exception, EXITING: java.net.SocketException: Socket closed
2016-03-17 16:42:18.757/sys$jms/INFORMATION/JMSConnection v400/192.168.0.90:33958/connection closed
2016-03-17 16:42:18.757/192.168.0.90:33509/BlockingHandler/INFORMATION/Exception, EXITING: java.io.IOException: End-of-Stream reached
2016-03-17 16:42:18.758/sys$jms/INFORMATION/JMSConnection v400/192.168.0.137:40199/connection closed
2016-03-17 16:42:18.758/sys$jms/INFORMATION/JMSConnection v400/192.168.0.90:33130/connection closed
2016-03-17 16:42:18.758/sys$jms/INFORMATION/JMSConnection v400/192.168.0.90:33294/connection closed
2016-03-17 16:42:18.758/sys$jms/INFORMATION/JMSConnection v400/192.168.0.90:33860/connection closed
2016-03-17 16:42:18.759/192.168.0.137:41695/BlockingHandler/INFORMATION/Exception, EXITING: java.io.IOException: End-of-Stream reached
2016-03-17 16:42:18.760/sys$jms/INFORMATION/JMSConnection v400/192.168.0.90:33509/connection closed
2016-03-17 16:42:18.760/sys$jms/INFORMATION/JMSConnection v400/192.168.0.90:33970/connection closed
2016-03-17 16:42:18.760/sys$jms/INFORMATION/JMSConnection v400/192.168.0.90:33663/connection closed
2016-03-17 16:42:18.761/sys$jms/INFORMATION/JMSConnection v400/192.168.0.90:33868/connection closed
2016-03-17 16:42:18.761/192.168.0.90:33663/BlockingHandler/INFORMATION/Exception, EXITING: java.net.SocketException: Socket closed
2016-03-17 16:42:18.762/192.168.0.137:42515/BlockingHandler/INFORMATION/Exception, EXITING: java.net.SocketException: Socket closed
2016-03-17 16:42:18.762/192.168.0.137:40206/BlockingHandler/INFORMATION/Exception, EXITING: java.net.SocketException: Socket closed
2016-03-17 16:42:18.762/192.168.0.137:40203/BlockingHandler/INFORMATION/Exception, EXITING: java.net.SocketException: Socket closed
2016-03-17 16:42:18.763/192.168.0.137:41327/BlockingHandler/INFORMATION/Exception, EXITING: java.net.SocketException: Socket closed
2016-03-17 16:42:18.781/sys$jms/INFORMATION/JMSConnection v400/192.168.0.137:40203/connection closed
2016-03-17 16:42:18.781/sys$jms/INFORMATION/JMSConnection v400/192.168.0.137:40206/connection closed
2016-03-17 16:42:23.818/SwiftletManager/INFORMATION/stopKernelSwiftlets

The router was shutdown orderly at that time. So someone did it, no crash.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: swiftmq crash

IIT Software
Administrator
In reply to this post by gianfranco.soft
Btw, you are running old client versions:

2016-03-17 16:42:18.760/sys$jms/INFORMATION/JMSConnection v400/192.168.0.90:33509/connection closed

You may want to upgrade your clients. Latest protocol version is v750.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: swiftmq crash

gianfranco.soft
In reply to this post by IIT Software
Hello,

some more info just to help me on my investigation.

We normally leave swiftmq running.

If we want to shut it down, we normally do:

kill -9 <PID>

How that "EXITING" process could have been triggered? Which command should have been launched?
I don't think a kill would have caused that kind of log.

Many thanks
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: swiftmq crash

IIT Software
Administrator
kill -9 is wrong because that's a hard kill and the shutdown hook of the router isn't executed. The proper way is kill <pid>.

The router was orderly shut down as stated in the info.log. 2 ways: kill <pid> or the cli command "halt" was executed.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: swiftmq crash

gianfranco.soft
Thanks,

another one:

is there any chance that the EXITING process could have been triggered by some internal (swiftmq) function call?

Thanks
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: swiftmq crash

IIT Software
Administrator
No.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: swiftmq crash

gianfranco.soft
OK. Thanks
Loading...