DRMacIver's Notebook

What every programming tutorial gets wrong

What every programming tutorial gets wrong

Published
2022-05-29

Another piece of draft bankruptcy. This is my attempt to explain a common failure mode of programming tutorials. I got quite far, but eventually abandoned it.

Scene setting

A while ago I was helping teach a coding workshop to a bunch of people who had never coded before, and we were working through the written materials which were very “Do this, then do that”. Copy this code and you should see this happen, etc.

At some point a student asked me “OK, but what is actually going on here? What does any of this mean? I can follow the steps, but I have no idea what is going on.”

This was an incredibly reasonable question and I’m glad she asked it of me, and that I could answer it, because I can’t imagine that she would have come away with any knowledge of programming without it.

I can’t remember the specific thing we were talking about, but I think the example was something like:

>>> user = {}
>>> user["login"] = "Bob"
>>> user["login"]
"Bob"
>>> user["some other key"]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 'some other key'

This is entirely clear if you’re a programmer (especially if you’re a Python programmer, but it’s sufficiently universal that probably most programmers will have a pretty good idea of what’s going on), but if you’ve never programmed before it’s just so much noise.

In response to this question I would have said something as follows:

The program you’re using here is called the REPL(Read-Evaluate-Print-Loop, but don’t worry about that. It’s pronounced repple, like ripple with an eh). You’re entering commands into it by writing statements in a programming language called Python.

The interpreter keeps track of a number of “objects”, which are a chunk of data along with a particular set of commands that can you can use to change that data, ask questions about it, etc. There are also commands that let you interact with the computer more broadly (e.g. reading and writing files) but we won’t worry about them just yet in this tutorial.

Here what we’re doing is creating an object, giving it the name user that we can refer to it by, and issuing some commands and seeing what happens.

Specifically, we are:

  1. Creating an object of a specific type called a dictionary and referring to it by the name user. The curly brackets {} create a new dictionary, and “user =” is where we declare that we will refer to it as user.

  2. A dictionary is an object whose job is to keep track of a table of information. It maps keys to values, which are just arbitrary things you can put into it, so that if you tell it to associate a value with particular key, and then later ask for the value associated with that key, you get the same thing back.

  3. Telling the dictionary we have just created (referred to by its name, user) that it should associate the value “Bob” with the key “login”. “Bob” and “login” are objects of a type called string, which represents a piece of text.

  4. Looking up the key “login” and getting the expected value “Bob” back.

  5. Looking up the key “some other key” and getting an error that tells us that the dictionary has not yet been told to associate any values with that key.

This isn’t a perfect answer, because I just made it up on the spot and haven’t tested it against students to see what questions they ask, but it’s probably pretty good. It certainly gives them more of a hook to play around with dictionaries in the interpreter and see what happens, and it gives them better questions to ask than “Help I don’t understand anything about what’s going on”.

Theory, examples, and ability

I’m going to be using “theory”, “examples”, and “ability”, in slightly specific ways (which are possibly slightly idiosyncratic to me, although I think compatible with normal use) in this issue, so I’ll try to explain what I mean by them.

An example is just any concrete version of the thing being taught. For example, “do these steps in the REPL” above is an example. Showing someone a program is an example. Showing someone a mathematical proof is an example.

“Theory” is the knowledge that you need to answer questions like “What’s going on here?” and explain the behaviour of a particular example. For example, in the above, you need some sort of theory to answer the question “Why did typing user["login"] show "Bob" but typing user["some other key"] showed an error message?”

“Ability” is the set of things the student can now do themselves. The goal of teaching is to help the student expand their abilities. People don’t generally expand their abilities directly as the result of teaching - you can only learn to do things by doing things - but teaching points them in the right direction by giving them better ideas of things to try. They still learn by trial and error, but teaching can give them better things to try and help them understand their errors.

The goal of the above tutorial was to help them develop the ability to write simple Python programs involving dictionaries, and to some degree it succeeded. After going through this tutorial the student knows how to launch the Python REPL, and has at least one example of something reasonable to experiment with. Given a little bit of time to sit and play with the REPL they might come up with a pretty reasonable theory for explaining its behaviour.

Why do they need to develop that theory? Because theory gives them the ability to generalise. If you have a theory that can explain why a given action succeeded or failed, you now believe that any action that matches that same explanation will also work or fail. This lets you try out other things and see whether they confirm or deny your theory, which is typically a much more productive model of trial and error than one can manage without a theory of what’s going on.

Examples are vitally important

I should be clear: It is very good that the tutorial we were doing was dumping people in the REPL and getting them to play around with it. This is not my complaint, it’s the correct thing to do. My complaint is that the tutorial did not then contain anything like my answer above.

One of my pet peeves that has been coming out in places on the notebook blog recently is that I hate people who introduce theory without supporting examples in which to understand it. See e.g. People are bad at defining things.

In particular it’s often worth having examples first (this isn’t always possible), because then you can refer to concrete features of them while introducing the theory. You saw a bit of that in how I explained what was going on in the programming tutorial: I started with an example, and then I highlighted specific features of it to ground the theory (We’re creating a thing called an object, it’s a specific type of object called a dictionary, then we’re doing this to it, etc.)

The examples are particularly important. In order to understand a theory, you need to be able to see how it applies to real concrete things. Abstracting from concrete instances is literally the point of theory, so without seeing how it relates to the things it abstracts theory is just so many words.

On top of that, examples are often key to developing abilities. People do not, generally speaking, reason from first principles from a theory. If you read the literature on naturalised decision making, what people tend to do is recognition-primed decision making. They look at what’s in front of them, they reason by analogy, thinking “Oh, this looks like that thing I saw before, let me see what happens if I do the same thing that I did there…”. Having a rich and varied library of examples is vitally important for this.

(I’d recommend Cedric Chin’s A Framework for Putting Mental Models into Practice, but it’s also worth reading Gary Klein’s “Sources of Power” and maybe some of his other work.)

Theory helps you remember examples

Let’s look at our example again:

>>> user = {}
>>> user["login"] = "Bob"
>>> user["login"]
"Bob"
>>> user["some other key"]
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
KeyError: 'some other key'

Do you think you could recreate this example from memory? I certainly could, and I think most programmers could with only a little bit of thought. It possibly wouldn’t be character by character perfect, but you could certainly get the right general idea.

Now consider the following highly readable example. Do you think you could recreate this from memory?

<<< d"T{ o Bm
<<< d"T{sn)gK[ena o n:g}n
<<< d"T{sn)gK[ena
n:g}n
<<< d"T{sn"gyT g](T{ tTuna
>{ilT}ilt kyg"] {TlTe] li)) )i"]'1
   ,[)T nE"]h[e<n= )[eT b= [e Eyghd)T<
FTuc{{g{1 r"gyT g](T{ tTur

If you haven’t figured out the trick, this is the same example as above, I’ve just jumbled all the letters through a mapping that replaces them (e.g. > gets replaced with <).

Do you think you could recreate this example from memory? I sure couldn’t.

In some sense this contains exactly the same information as the original, but because you’re unable to easily turn it into a sensible representation, it’s much harder to remember and thus much harder to understand. As well as the Python specific bits you also rely on e.g. brackets matching, quotes matching, familiar understanding of the meaning of the = symbol, the ability to clearly distinguish which bits are identifiers because they’re contained in quotes, etc.

When we know what’s going on, we can describe the example as the following sequence of steps:

  1. Create a dictionary called user.

  2. Insert the value “Bob” into “user” with the key “login”

  3. Look up the key “login” in “user” and get the result

  4. Look up the key “some other key” in “user” and see an error from looking up a missing key

This is a high level representation of what’s going on. It describes the steps we are taking, not the specific details that we know. This is easy to remember, because it’s a set of comprehensible steps.

Now, we’re not expecting students to actually memorise specific examples, but ability to memorise is a pretty good proxy for how well the example fits in their head - if you can’t easily remember what should be a fairly simple example, that’s a sign that there’s too much going for you to really figure it out and you’re trying to learn too much at once.

This sort of translation into a high level representation is what theory does for you, and lets you organise your mental library of examples.

Theory helps you generalise

As well as helping you remember examples you’ve seen before, theory lets you adapt those previous examples to your needs.

This high level representation does a number of important things for us:

  1. It allows us to understand the meaning of the thing we are writing in terms of what we are trying to do rather than the literal characters on the screen.

  2. This makes it easier to remember because we can remember “Create a dictionary called user” separately from the literal syntax for creating a dictionary. We still have to remember both, but the latter is a fact we can more easily look up and will use often enough to remember.

  3. It lets us make predictions for what we would see if we changed the example.

Doing (the right) things is the most important

This particularly shows up in the trial and error process of learning. While you are still floundering around trying to figure things out, good specific examples are especially important because you will lean hard on emulating examples, so having good examples to emulate is crucial.

Postscript

This is where I stalled. I’m not quite sure why. I think partly the post was getting too long, and I didn’t know how to continue, or I ran out of energy.

It’s a known problem that I’m very bad at revisiting things once I’ve stopped writing them, and as a result I tend to fail to finish longer pieces. I would like to work on that but have not yet managed.