Project Server 2007 Queue – How many threads is enough?

Working on a case recently I could see from the application logs that a server was having a hard time.  Timeouts, out of memory exceptions, lots of red – so something was up.  The customer’s comment was that this was period end processing so lots of timesheets and updates going through.  The queue was meant to avoid this problem, so I looked at the queue settings and both project and timesheet queues were set to use 10 threads, rather than the default 4.  This could well have been leading to their problems!

The queue is designed to limit the rate that work gets processed so that the peaks that can overload the server get spread out.  In project management terms think of it as leveling for the server – stopping the server from trying to do many things at once.  Another analogy is my weekend to-do list  If I had a list of 10 things I would spend at least Saturday morning deciding what to do, and which my least favorite task would be.  Then I might end up swapping tasks and my productivity would be poor.  My wife has learned that I am “single-threaded” and best throughput is achieved by giving me one job at a time.

How many threads is the right number?  This very much depends on your server configuration and also the workload mix in terms of both volumes of transactions and the size of each transaction.  You can obviously publish many more 10 line projects than 1000 line projects in the same time.  One rule of thumb some of our field guys use as a starting point is to set the number of threads to the number of available processors (or cores).  So if you have a single dual processor machine as your application server then 2 threads might be a good starting point; if you have a quad dual-core then you might be able to use 8 threads.  This will also depend on farm topology and other applications running on the same servers.  You wouldn’t want to use these figures and also have the server acting as a Web Front-End or running search or other processor intensive activities.

Monitoring performance counters and the application and ULS logs will enable you to fine tune the queue to work with your normal server loads – but please don’t just increase the number of queue threads expecting to make things work faster.  Time is nature’s way of keeping everything from happening at once – for project server we achieve the same thing with the queue!

Technorati Tags: Project Server 2007