DRMacIver's Notebook

Mechanisms for talk scheduling and voting

Mechanisms for talk scheduling and voting

Published
2018-09-05

I’ve been thinking about mechanism design for conference scheduling again. I’ve previously argued that conference scheduling should be treated as an optimisation problem, but I no longer believe that’s true.

In particular I think the following hold:

Lets see some examples in support of this.

Suppose you’re running a Python conference, and 60% of the people attending are web developers and 40% are data scientists. You put together a set of talk proposals, people vote on them, and you take all of the top voted talks. What you end up with is of course a conference consisting entirely of web development talks.

(Note: Despite the running Python example, this post is not actually about The PyCon UK Schedule, which I’ve barely looked at.)

For some contexts maybe that’s OK, but given that a lot of the value in conferences is the hallway track, it’s nice to be able to put together heterogenous conferences. You could fix this by artificially selecting for certain subjects, but proportional representation seems like a much better approach because it doesn’t require you to know all the ways in which your audience is heterogenous in advance. So, in the above example, we would have roughly 60% web dev talks and 40% data science talks, but also if it turned out that about 10% of the audience were really excited about Flask, we could have about 10% Flask talks.

If the conference is single-track we’re more or less done: Pick your favourite (non party-list based, so probably some variant of STV), proportional voting system, use that to select your talks, and call it a day.

I’d like to pause here by saying that I’m increasingly a fan of single track conferences, so I think “do a single track conference and call it a day” might actually be the correct solution.

But lets suppose you’re less on board with that and want a multi-track conference.

For simplicity, lets imagine that our Python conference now has two rooms, with talks running in the same time slots in each room, and attendees now have to choose which of the two to attend. Lets say it’s a single day conference and there are five time slots, so ten talks.

According to our above PR argument, we should run six web dev talks, but does it really make sense for us to do so? There are only five time slots, so (by the pigeonhole principle if you want to get fancy about it) you’re inevitably going to put two web dev talks back to back. That might be OK - maybe you’re scheduling a Django and a Flask talk against each other - but maybe there’s a strict preference where there are five obviously best web dev talks and the sixth is pretty good (preferable by web devs to any data science talk) but not good enough (will not get any attendees when scheduled against any of the top five talks). What’s the point in selecting that talk given that?

In the other direction, lets say we have 20% of the audience who are really interested in random forests, and so we select two random forests talks, which we then proceed to schedule in the same time slot. Now despite 20% representation at the talk level, they only have 10% representation at the time slot level!

(I want to draw an analogy to gerrymandering here but I don’t think it quite works)

So, tracking creates an upper bound on how much proportional representation is worth doing, and also scheduling within those tracks affects the amount of proportionality you actually get.

So what to do about it?

Well, I’m not entirely sure. I started designing a whole complex system in support of this that this note was originally supposed to be about, but I decided I didn’t like it very much.

The basic ideas were:

  1. Give each participant a “voting currency” - everyone starts with an equal amount, and talk slots effectively get auctioned off, with the proceeds distributed among everyone equally (possibly among everyone who still has any interest in attending remaining talks).
  2. Participants will only vote for talks in slots that are strictly better for them than the talks already scheduled in that slot.
  3. Define a threshold of “Minimum number of people required to be worth running a talk”. Whenever a talk no longer would meet that requirement (because every slot it could be scheduled in has talks people prefer more), it is immediately excluded and the process restarts from the beginning. This is akin to how exclusions work in The Wright System of STV, and is designed to avoid “spoiler” talks, where people who preferred them effectively get screened off from voting in the process until the talk is excluded.

The details kinda became a weird hybrid of STV and the Vickrey-Clarke-Groves mechanism and the more I looked at it the less convinced I became that it was the right way to do things or that I actually understood how the VCG mechanism plays out in practice.

I do think the above examples are important to consider though.