DRMacIver's Notebook

We are surrounded by ghosts

Published: 2020-02-16

(This is part of an attempt to get back to using my notebook to write more half formed thoughts)

I recently read “Epistemic injustice in mathematics” by Rittberg, Tanswell, and Van Bendegem. In it they talk about the idea of ghost theorems:

There are mathematical results which are taken as accepted in a mathematical community, relied upon in talks, discussion, and proving further results, but which cannot be traced to a concrete proof in the literature. These results are part of the expert knowledge one is expected to have in certain communities and we will present examples below. … The kind of result we wish to discuss, i.e. the kind of result whose proof cannot be traced in the literature, thus seems to be a special kind of folk theorem for which we propose the term “ghost theorem”; these theorems are immaterial in the sense that they are not proven in the literature yet they “haunt” parts of daily mathematical life.

They highlight ghost theorems as a particular source of epistemic injustice against people trying to participate in mathematics as an epistemic community. They focus on an example of someone trying to get some results published, and having them be rejected because they were ghost theorems - even though their submission had never been published, it was “obvious”. I think it would still have been an injustice of sorts without that gatekeeping, because presumably these results are useful, and there is no way to discover them without privileged access to the community. As someone who does a lot of solo reading in a variety of fields, a kind of feral fan for a variety of academic disciplines, this is an injustice that particularly matters to me: There are probably many things that are completely obvious if you’ve been trained in, say, phenomenology, that I as an outsider will never discover no matter how much of the literature on the subject I read.

I’d like to call the more general phenomenon that this is a specific instance of this “ghost knowledge”: It is knowledge that is present somewhere in the epistemic community, and is perhaps readily accessible to some central member of that community, but it is not really written down anywhere and it’s not clear how to access it. Roughly what makes something ghost knowledge is two things:

It is readily discoverable if you have trusted access to expert members of the community.
It is almost completely inaccessible if you are not.

In this sense, most knowledge is ghost, particularly if you take an expansive view of what counts as an epistemic community.

A recent (more recent than the publishing of this paper) related example was “Eigenvectors from Eigenvalues: a survey of a basic identity in linear algebra” by Denton, Parke, Tao, and Zhang. A couple of physicists found an interesting (to them, and apparently to others who care about such things, I confess I find it too technical to be interesting - I understand the mathematics well enough but don’t work in this field so have a differently tuned sense of interest). The story of this as I understand it is roughly:

A bunch of physicists discovered an interesting and, apparently, novel identity.
They emailed Terence Tao and he almost immediately was able to provide multiple proofs of it.
They published a paper together.
It got a lot of press because Terence Tao is something of a big deal and discovering a new result in such a mature field is surprising.
Turns out it was actually a result that was not so much “well known” as repeatedly rediscovered in myriad different places, but had never made a splash before.
They wrote a new survey paper about this.

If you look at only parts (1-3) this is a perfect example of ghost knowledge. But the result already existed in the literature.

To quote a great philosopher:

“But the plans were on display…” “On display? I eventually had to go down to the cellar to find them.” “That’s the display department.” “With a flashlight.” “Ah, well, the lights had probably gone.” “So had the stairs.” “But look, you found the notice, didn’t you?” “Yes,” said Arthur, “yes I did. It was on display in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying ’Beware of the Leopard.”

It is very often the case that even when knowledge is written down somewhere, it’s nearly impossible to find it. This has happened to me in mathematics before: A friend and I once discovered a result that we thought was very interesting (under certain conditions, the best possible continuous approximation to a discontinuous fucntion exists), only to eventually discover that the result was known, it had just been published in an obscure mathematical journal, in the 1970s, in Poland, so virtually nobody knew about it. Fortunately we had never put in much serious effort into trying to publish it before discovering that (honestly we probably should have, but neither of us were professional mathematicians by that point), but it was still annoying.

The core problem that causes all of this is that there’s a leaky pipeline of knowledge from epistemic communities to the outside world. In order for you to discover a piece of knowledge:

It has to be interesting enough for someone to think it is worth writing down.
It has to be interesting enough that it gets accepted (though if not, it may end up on a random blog post if you’re lucky).
It has to be interesting or well organised enough that it gets surfaced in a way you can find.
It has to be accessible enough for you to be able to find it (e.g. it can’t use super technical terms that you’ll have no way to ever discover without access to an expert).

This pipeline is leaky enough that it would be very surprising if most knowledge produced by an epistemic community were accessible to you.

This may not seem like a big deal when you think of communities like mathematics, where most consumers of its contents are also members of the community, but it’s a big deal when you consider two things:

Everything is like this, including subjects that everyone would benefit from. I read a lot of therapy books for example and I’ll bet that there’s at least two orders of magnitude more ghost knowledge about therapy than I have access to.
Every community is an epistemic community. Many lessons are e.g. learned over and over again inside companies, and are never written down anywhere, or when they are are so defanged that nobody can benefit from them. The pipeline is different, but the problem is the same.

Given this, we are surrounded by ghosts: The information that is written down and that we can access is so thin on the ground compared to what’s actually out there, it’s astonishing that we ever get anything done.