page.db shrinking after clearing up huge persistent traffic

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

page.db shrinking after clearing up huge persistent traffic

sanjan
Hi, May I know what needs to be done in order for page.db to return to its initial size after processing huge traffic?
Example:
I store 500,000 messages in a queue and the page.db grows to around 3GB in size.
then a consumer connects and consumes the queue until its empty.
But I don't see the size of the page.db reduced.
not even after I execute a shrink command on store swiftlet via CLI.

Reply | Threaded
Open this post in threaded view
|

Re: page.db shrinking after clearing up huge persistent traffic

sanjan
Well, it reduced size after sometime. not sure what actually made the page.db size to compress.
By the way iI have one more question.. what is the maximum recommended page.db size?
Reply | Threaded
Open this post in threaded view
|

Re: page.db shrinking after clearing up huge persistent traffic

TheQL
In reply to this post by sanjan
As far as I understand the page file can only shrink AFTER the last used page. So if there is a message being stored at a high page number, shrink won't be effective.
Reply | Threaded
Open this post in threaded view
|

Re: page.db shrinking after clearing up huge persistent traffic

sanjan
Thanks. but um... when all messages in all queues are consumed. there should be no more messages left in the page.db (in any page) right? or may be I'm missing something....
Reply | Threaded
Open this post in threaded view
|

Re: page.db shrinking after clearing up huge persistent traffic

IIT Software
Administrator
Yes, you are missing something. The page.db contains all kind of pages, message and index pages. If you e.g. create a queue or durable subscriber while you have some million messages in the persistent store, these index pages get high page numbers. So even if you consume all messages, these pages will stay there. There is no defragmentation, only shrink.
Reply | Threaded
Open this post in threaded view
|

Re: page.db shrinking after clearing up huge persistent traffic

sanjan
Thanks for the clarification. may I know what is your recommended maximum page.db size.

I am doing some resource planning. and I am thinking about using clustered queues.
I need to calculate how many routers should I use in order to maintain a fairly sized page.db during a consumer outage.

I have a queue. that would process 1.5 million messages per hour. and size per message is 6KB.
so literally if my consumer goes down. the page.db could grow to approx 9GB during 1 hour of downtime.
now if I make this a clustered queue and distribute across 3 routers, page.db on each router still could grow to 3GB (which I still think is too big) .

So I'd like to know what is your max recommended value?
Reply | Threaded
Open this post in threaded view
|

Re: page.db shrinking after clearing up huge persistent traffic

IIT Software
Administrator
3 GB should be ok but you need to configure some hard limit (e.g. maximum messages on a queue) to avoid an OOM.