Correlation ID

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Correlation ID

TheQL
Hi there,

it is hard to impossible to log any messages going through SwiftMQ.

We have for instance a message flow where we pick up messages via a JMS Bridge, then have a locally hot deployed JMS app alter the message before it finally is consumed. The consumer logs the JMS ID of the message as INTRAVM-7/-xxxx/xxx/x, which makes sense, as the JMS app is of course running as INTRAVM-7.

Before that the JMS ID was -xxxxxxx/xxx/routername/jmsbridge/xxxxxx/xx, which also seems to make sense, as it was produced by the JMS bridge swiftlet.

There is no kind of correlation ID to help trace a message through the router and even if there were, nothing is logged to disk, unless maybe if you enable the trace swiftlet which I usually consider a bad idea in a production environment.

This puts us in a bad situation when our customer asks us to confirm that we retrieved and processed a message, we cannot do or refute that.

So if there isn't any way I am not seeing at the moment it would be great to first of all have some kind of correlation ID and being able to log it with some other useful information, like Queuename or name/id of the consumer/producer without being forced to cripple the router by enabling the trace swiftlet. Thanks for helping me out or considering this!
Reply | Threaded
Open this post in threaded view
|

Re: Correlation ID

IIT Software
Administrator
So you want the JMS Bridge to generate a unique id and set it as message property or the JMSCorrelationID?

The quickest way would be to add a line of code in your deployed app to set the JMSCorrelationID of the new (altered) message to the original message id generated from the JMS bridge.
Reply | Threaded
Open this post in threaded view
|

Re: Correlation ID

TheQL
You are right about that specific use case, adding that line of code would preserve the original message ID, but it would have to be written to a new JMS property and can't be reused as JMS message ID or could it? Anyway, still there was no logging available to later on follow the message through the router.

I know that usually a message is not passed around in the router but produced and consumed and then is gone, so this kind of logging would not serve too many people. But as we are migrating a lot of our message flow from SMTP to JMS we are sort of used to having a unique message ID to trace through our systems. Additionally that ID is also known on customer side, so if we are asked about a message with that specific ID we are able to prove what happened to it or if we received it at all. To duplicate this kind of transparency it would of course be necessary to already have and preserve a correlation ID on customer side which is then kept by the JMS bridge. I am not sure if anything like this is possible.

Nevertheless it might be a nice feature from time to time to log the jms ids of messages in their queues, even better so if it was a correlation ID to uniquely identify a message. Otherwise maybe the JMS bridge could log consumed and produced messages and their IDs, but I understand that wouldn't really enable us to prove anything to our customers if they ask us about a specific message.
Reply | Threaded
Open this post in threaded view
|

Re: Correlation ID

IIT Software
Administrator
The only information we can log is timestamp, JMS message id, destination name, action. Would that be useful for you?

We don't have the correlations between messages. That's an application thing and that's where JMSCorrelationID was introduced for.

Reply | Threaded
Open this post in threaded view
|

Re: Correlation ID

IIT Software
Administrator
In reply to this post by TheQL
Actually what I'm think of is a new "message" trace space in the Trace Swiftlet. Tracing would be done in the Queue Manager Swiftlet so you would see these actions: insert into a queue, read from a queue (lock), remove from a queue.
Reply | Threaded
Open this post in threaded view
|

Re: Correlation ID

TheQL
IIT Software wrote
We don't have the correlations between messages. That's an application thing and that's where JMSCorrelationID was introduced for.
Sure, I understand. So as long as the producer does not generate such an ID initially, there is no way of tracing the message from end to end.


IIT Software wrote
Actually what I'm think of is a new "message" trace space in the Trace Swiftlet. Tracing would be done in the Queue Manager Swiftlet so you would see these actions: insert into a queue, read from a queue (lock), remove from a queue.
That sounds quite good. If we were able to have a correlation ID from start, could you enable the trace swiftlet to log the JMS property? How could I select which properties to log if I don't want them all.

There existing queue trace swiftlet, has a very cluttered log. I would at least need a sensible filter mechanism to prevent the logs from growing too fast and producing too much I/O, but the idea itself sounds great!