SwiftMQ 9.5.0 degradation due to filestore size

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

SwiftMQ 9.5.0 degradation due to filestore size

mpoornima
Hi,

Today we have seen a case where the latency of message push / pull from any queue was taking insanely large amount of time (6-30 seconds for 1 message push whereas it generally takes 3-5ms). This was the case not only for old queues but also for newly created queues. The only suspicion we had was on the size of the filestore page.db. It was around 60G due to accumulation of messages through time. We drained all of these messages and performed a shrink which seemed to have no effect on the size. At the end we had to take a backup of the store, stop the router, truncate the file page.db and restart the router to fix the problem.  

We are still not sure what the issue was but if it were due to the size of the store, then, are there any restrictions on the size of this file? Also, what is the maximum size that we can reach beyond which performance of the router degrades? What do you recommend we do if we were to have queues with very high number of messages for a long period of time?    

Thanks
Poornima  
Reply | Threaded
Open this post in threaded view
|

Re: SwiftMQ 9.5.0 degradation due to filestore size

IIT Software
Administrator
This post was updated on .
As long as the file store can serve page requests out of the store cache, it is fast. 60 GB is really huge so nearly every page must be fetched from disk. The max theoretical size of the page.db is Integer.MAX_VALUE * 2048.

If the page.db doesn't shrink after you have consumed all message then you have created a queue or durable subscriber after the page.db had this size. Each queue/durable sub occupies at least 1 page in the store and it gets the next free one. A shrink is always performed to the highest page no in use.

Truncating the page.db by yourself may result in an inconsistent store. If you want a fresh store, delete transaction.log AND page.db.

Look here for further information.
Reply | Threaded
Open this post in threaded view
|

Re: SwiftMQ 9.5.0 degradation due to filestore size

mpoornima
After truncation and restart, the page.db size is now 67G. There is no degradation of performance this time though. Shrink is not shrinking the store. Do you think we will face the same performance issue soon? If so, how can I prevent this in the future?
Reply | Threaded
Open this post in threaded view
|

Re: SwiftMQ 9.5.0 degradation due to filestore size

IIT Software
Administrator
If shrink is not shrinking the store, just analyze it.

http://www.swiftmq.com/products/router/swiftlets/sys_store/analyze/index.html

67G is far beyond what we have tested. We have seen performance degradation far earlier when the cache was full.

I assume that you mean with "truncating" you've used file.setLength(n)? Are you doing the same when you consider a file of you Oracle DBMS too large? Just truncate it?