Queue Etiquette: 4 items or less? Join the queue!

The queue is one of the new features that is taking some getting used to by our customers.  It is probably time for a fresh “queue problems” posting – that will be my next post, but the following came out of a discussion with Patrick Conlan yesterday.  The main job of the queue is to even out the flow of data to the database ensuring consistency of data and avoiding overload on the servers.

The queue process has some great logic built in to control the flow of data and ensure reliable data consistency – such as not pushing data into the reporting database while a cube build is under way, and not allowing a check-in if a save is incomplete.  There is also some optimization to avoid doing the same job more than once.  You may have seen “skipped for optimization” against some jobs.  These are jobs that do not have their own payload (unlike Save from Professional where the project changes are part of the message in the queue)  but are really just instructions to do something.  An example would be a publish – which has no payload of data but is just an instruction to take what is in the draft db for a particular project and push it into the published db (in very simple terms).  Obviously if there are several publish requests for the same project and no intermediate saves then you only need to do this job once – so you can skip all other occurrences. 

But beyond this (again in simple terms) it is really first come first served.

You need to consider this if you are developing an application that may push data into the Project Server queues.  If you submit thousands of jobs to the queue that may take a while to process then anyone doing normal routine stuff will be in the queue behind you and therefore delayed.  A better approach (assuming your stuff isn’t time critical) is to push a few jobs in to the queue at a time and check for their progress.  Once these have finished perhaps pause for a bit to allow other jobs some time then push in some more.  In this way the normal project saves, check-ins etc will not be right at the back of the queue but will get a chance to join a short queue – much like the express checkouts at the supermarkets.  A good rule of thumb might be pushing in as many jobs as you have threads running – so 4 items or less in a default system.

Technorati Tags: Project Server 2007