DRMacIver's Notebook
A Boltzmann Agent with Very Bad Judgement
A Boltzmann Agent with Very Bad Judgement
As per previous post, it can make sense when looking at a set of consistent propositions to consider agents as Boltzmann samplers over the set of valid consistent beliefs, with their reliability measured by the expected number of true beliefs.
A thing I hadn’t previously realised is that this can cause an agent that is on average reliable to be reliably wrong for some propositions.
Consider a chain of propositions of the form \(P_1 \implies \ldots \implies P_n\). There are exactly \(n + 1\) possible consistent beliefs for this sampler (each defined by the first \(P_i\) that the agent believes), so the Boltzmann generating function is \(B(x) = 1 + \ldots + x^n\). Suppose \(n = 10\). Some simple maths (by which I mean I used sympy) shows that this agent ends up believing \(P_1\) with probability at least half only when \(x \approx 2\), which leds to the expected number of propositions believed being \(\approx 9\). So in order to achieve \(50%\) reliability on the base proposition we have to achieve \(90%\) overall reliability!
This isn’t very surprising in some sense, but probably puts a bound on how good we can expect judgement aggregation to be in this case.