DRMacIver's Notebook

Good queues and bad queues

Published: 2025-04-06

This is a follow on from Sources, Sinks, and Flows and won’t make much sense if you haven’t read that.

An important category in this model is the queue. A queue is basically a source/sink pair in which stuff does nothing except enter and then leave at some point later.

Some queues are bad. Some queues are good.

Let me give you some examples:

A literal queue of people waiting to be seen at the shops is a queue.
The kitchen cleaning example has a lot of queues in it. Dirty dishes first move to the kitchen counter, where they are buffered until they are ready to be washed.
The place where I write down ideas for what to write is a queue.
That growing pile of papers that has been sitting ever grosing somewhere in your office waiting to be dealt with is also a queue.
The storage space in your attic that you keep putting stuff into because you don’t want to get rid of it but also don’t actually know what to do with it.

I’m using “queue” somewhat informally here, as not all of these have FIFO (first-in/first-out) behaviour. Please ignore that.

Some of these are very good because they improve the smooth flow of stuff through the system. You can get one task done before having to start on the next (imagine if you had to hand wash every dish as soon as you brought in into the kitchen rather than cleaning the dining room table first), and you can parallelise it (one person brings stuff into the kitchen, another washes).

The place where I write down ideas is also a pretty good queue, but it has a significant problem viewed as a queue: If the only way for items to come off the queue is for me to write something, items are being added to the queue much more often than they are being removed from it, and this causes the queue to grow unboundedly. My current solution to this is that when the list fills up I start a new list and copy over anything from the old list that still feels appealing.What will happen when all of the ideas seem appealing? Uhhh. Hope that doesn’t happen and if it does I guess I’ll just have to write more?? Problem for future David.

The pile of papers on your desk? That’s a bad queue. That queue is not working.

The two key features needed for a queue to be a good queue are:

Over some suitable period of time, as much has to come out of the queue as go into it. If you put stuff into a queue to “deal with it later”, you have to actually deal with it later, and the rate at which you deal with it must exceed the rate at which you create stuff to deal with.As the sage tells us, basic queueing theory means that queues either grow unboundedly over time or reliably return to empty.
The queue must either not fill up during the gap between stuff entering it and stuff leaving it, or you must have some graceful process for handling when it does.Which can, and sometimes should, be “you start throwing stuff away”. This is particularly a good idea for queues of ideas of stuff to do.

The biggest failure mode of queues is that often we introduce queues into our flows because we don’t want to deal with the thing. When this happens, the queue makes our life worse rather than better, because instead of having a small amount of stuff we don’t want to deal with we have a large amount of stuff we don’t want to deal with, and the result is our environment acquires a fnord.

In contrast, for stuff you’re perfectly happy to and have the capacity to deal with, a queue is often extremely useful for smoothing out irregularities - when you’ve got large bursts of production and a more gradual process of consumption such as in the dishes example - or to more efficiently connect up sources and sinks so that stuff doesn’t get lost and you don’t have to frantically multitask between different types of activity.