Routing messages through a federated router network

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Routing messages through a federated router network

sanjan
Hi,

I have a router network setup as in the attached image.
The queues are located in the "queueing routers", and the "connection routers"  just serve as an interface for jms producer to connect,
the messages will be routed to the destination queue via "connection router".
the queues in "queueing routers" are configured as persistent.

I would like to know, when the messages being routed to the destination queue, are the temporarily stored in the "connection router" ?

Reply | Threaded
Open this post in threaded view
|

Re: Routing messages through a federated router network

IIT Software
Administrator
Yes, at the "Connection" routers they are stored into the routing queues rt$<destinationrouter> and transferred from there.
Reply | Threaded
Open this post in threaded view
|

Re: Routing messages through a federated router network

sanjan
Hey, Thanks for the reply, so are the messages written to the disk at the "connection router" as well?
(since by default the messages are persistent)
Reply | Threaded
Open this post in threaded view
|

Re: Routing messages through a federated router network

IIT Software
Administrator
Only if they are persistent.

Consider to save these "Connection" routers and let your producers connect to "Queue" routers directly.
Reply | Threaded
Open this post in threaded view
|

Re: Routing messages through a federated router network

sanjan
Thanks again.

about making connections to queue router directly,
the producer software can make connection to maximum of 2 routers.

We have 10000 queues that need to be load balanced and error tolerant as well. (eg: queue1 exists in both "queue router" 1 and 3)

since a router configured with 2GB max heap, can only contain about 1000 queues
we need about 20 routers  ( (1000 queues X10 routers ) x 2)

Do you have a better suggestion for me?
how can I achieve error tolerance and load balancing with a different setup?
Reply | Threaded
Open this post in threaded view
|

Re: Routing messages through a federated router network

IIT Software
Administrator
I don't have a better suggestion. You approach seems to be the way to go in this scenario. You may consider to disable the XA protocol on routing connections to reduce overhead.
Reply | Threaded
Open this post in threaded view
|

Re: Routing messages through a federated router network

sanjan
Hi,

I have a question.

lets say in the scenario above,

producer1 is connected to connectionrouter1 and sending messages to queue1 in queuerouter1.
now if queuerouter1 goes down for some reason, messages will be stored in connectionrouter1's rt$queuerouter1 queue.

And the producer1 will not know that the queue1 is no longer available.

How to make producer1 aware that the queue1 is offline now?

Regards,
Sanjan
Reply | Threaded
Open this post in threaded view
|

Re: Routing messages through a federated router network

IIT Software
Administrator
This decoupling is the core concept of JMS. The producer does not need to know whether a remote destination or a consumer is connected. The JMS server takes responsibility for that. In case of a federated router network in SwiftMQ it is ensured that the message is transferred to the remote destination once a route becomes available. In the meantime the message stays at the routing queue (store and forward).
Reply | Threaded
Open this post in threaded view
|

Re: Routing messages through a federated router network

sanjan
Ok, I get it . however I do not want messages being stored while routers are offline. our messaging system is time sensitive. So I was thinking if I can use clustered queues.

in the clustered queues  documentations under: message distribution, it seems that a message will be dispatched to a router containing a physical queue only if it can see a consumer is connected.

so in my case, when a router goes down, obviously it won't have any consumer connected. and messages will be routed to the router that is online and has a consumer connected.

Am I right?
Reply | Threaded
Open this post in threaded view
|

Re: Routing messages through a federated router network

IIT Software
Administrator
No. The Queue Manager of all routers in a network report their clustered queue metrics to topic swiftmq.cluster. This is done in an interval (default 10 s). This has nothing to do with reachable or not. If you send a message to a clustered queue at some router which has a consumer on some other router, the message is sent to the routing queue. If there is no active connection, the message stays at the routing queue until the connection is reactivated.

Note also that a consumer can be connected at a remote router some hops away. This mechanism takes place at each hop. Store and forward.