SWAP use in swiftmq 8.0

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

SWAP use in swiftmq 8.0

alexzap
We are having the following problem.
A lot of tmp queues are created and a file, named as the tmp queue, is cretated under this path:
swiftmq/store/instance1/swap

Several files are crated ( more than 5Gb).
The tmp queue is configurated as the default.

This situation is only becouse there are a lot of messages coming in and the number of these messages is more than the dafult  or can there be others causes?
This is an example of a tmp quese which have a file on the swp directory.

How can we avoid this problem ?


Thank you very much.
Reply | Threaded
Open this post in threaded view
|

Re: SWAP use in swiftmq 8.0

IIT Software
Administrator
Please have a look here and here.

Swap files contain non-persistent messages which cannot be held in the queue cache. It is an overflow storage. You can only avoid it by a) faster consuming or b) producing less messages. As long as your consuming rate of messages is lower than the producing rate, you'll create a backlog of messages which is first stored in the queue's memory cache and, if that is full, stored in swap files. These swap files are managed by the router and automatically deleted when they are empty.
Reply | Threaded
Open this post in threaded view
|

Re: SWAP use in swiftmq 8.0

alexzap
Thank you very much for your answer.

This issue is due to the originator which sends, in specific period of time, a lot of messages.
Is it possible to configure the queue so that when the number of bytes or messages in a tmp_queue is up a threashold the messages are discarded?

Thanks


Reply | Threaded
Open this post in threaded view
|

Re: SWAP use in swiftmq 8.0

IIT Software
Administrator
Are you using publish/subscribe, that is the sender sends to a topic and the tmp-queues are subscriber queues?

Or do the sender use point-to-point do it knows the tmp-queue and sends directly to it?
Reply | Threaded
Open this post in threaded view
|

Re: SWAP use in swiftmq 8.0

alexzap
We are using publish/subscribe.
Thanks
Reply | Threaded
Open this post in threaded view
|

Re: SWAP use in swiftmq 8.0

IIT Software
Administrator
In reply to this post by IIT Software
In that case define a slow subscriber condition.
Reply | Threaded
Open this post in threaded view
|

Re: SWAP use in swiftmq 8.0

alexzap
Thanks for your answer.

We have been already using the following cfg:

<swiftlet name="sys$topicmanager" flowcontrol-enabled="false">
    <slow-subscriber-conditions>
      <topic name="feeder_book" max-messages="2500" persistence-mode="all"/>

Two questions:
1- The persistence-mode is set to "all". Does it mean "persistent"?
Is this the reason why swiftmq writes the tmp files?
2- It seems that the max-messages parameter is ignored. If you look at the picture in the first post you will see 500 which is the default.

Thanks a lot.
Reply | Threaded
Open this post in threaded view
|

Re: SWAP use in swiftmq 8.0

IIT Software
Administrator
all means persistent an non-persistent messages. The 500 messages is the dimension of the queue's cache.

With 2500 messages you have at least one swap file per subscriber. Check the names of the swap files to see how many subscribers are swapping messages.
Reply | Threaded
Open this post in threaded view
|

Re: SWAP use in swiftmq 8.0

alexzap
We have a lot of tmp files. In the worst period, they reach 10 Gb of disk space.
This afternoon I'll check, but it seems that all subscribers suffers of the same problems swapping messages.

The router becames slow, goes in STANDALONE mode.
The only way to get rid of this situation is stop/start the router deleting swap files.
Reply | Threaded
Open this post in threaded view
|

Re: SWAP use in swiftmq 8.0

IIT Software
Administrator
Then you have many subscribers. Why not just limit the slow subscriber condition to 500 messages or less which is the size of your queue cache. In that case you won't have swap files at all!