# How to get good at programming

*Epistemic status: very confident*

See also: A closely related [post](https://gwern.net/unseeing) by Gwern, another related [post](https://www.lesswrong.com/posts/nEBbw2Bc2CnN2RMxy/gears-level-models-are-capital-investments) by John, and some interesting [slides](https://www.cs.dartmouth.edu/~sergey/hc/rss-hacker-research.pdf#page=19) from a hacker's talk. None of the concepts here are new, but I've tried to lay them out in a more helpful frame.

* * *

When good programmers debug hard problems fast, it's usually because they understand the system well enough to [*track the important internal state*](https://www.lesswrong.com/posts/bhLxWTkRc8GXunFcB/what-are-you-tracking-in-your-head) in their head, letting them drastically *reduce the solution space they're searching over.*

This post contains my advice from ~5yrs of linux & programming experience on one of the primary ways to getting better at programming: *white-boxing*.

Definition and clarification
----------------------------

**Definition**: White-boxing, the process of taking a system you reason about purely in terms of input/output abstractions ("Autograd takes code and outputs gradients") into a system who's gears you understand ("Autograd takes code, records operations to construct a computational graph, then computes gradients via the chain rule")

There are three important things to understand about white-boxing:

First, White-boxing goes through various shades of gray. When you hit diminishing returns you want to switch to understanding another system. (Though if you find a topic fascinating then go ahead and do a deep dive!)

Second, It is higher value to white-box *leaky abstractions*. Autograd for ML is a great example of a leaky abstraction, if you mix up `permute` and `view` your gradients can be subtly wrong. See Karpathy's [great post](https://karpathy.medium.com/yes-you-should-understand-backprop-e2f06eab496b) for more on this. On the other hand, the CPU is a very good abstraction, unless you're doing something unfathomably cursed, you should never run into CPU bugs.

Third, and perhaps most important for building skill,[^289bs9hi65b] you must **notice** when you're going into brute-force search mode, and then **take action** by investing time in understanding the underlying system, until both the problem and solution make sense.

Absorbing the pattern
---------------------

Read Gwern's [list](https://gwern.net/unseeing#atoms) and then attempt to come up with *three new examples* of the pattern, ala [framing exercises](https://www.lesswrong.com/s/Fu7Euu3F96rKhFRWH). I used to think I had absorbed the concept, but I was still black-boxing things without realizing it. I encourage the reader do another exercise: *Come up with three examples of systems (preferably computer systems) that you've recently been partially black-boxing, and problems you ran into because of this.* Alternatively, come up with examples of you *doing black-box search, and how inefficient this was.* Try and install the [trigger-action-plan](https://www.lesswrong.com/posts/v4nNuJBZWPkMkgQRb/making-intentions-concrete-trigger-action-planning) for "notice black-box search, understand things instead"

It may seem I'm making a big deal of this, but it is *critical* to [notice](https://agentyduck.blogspot.com/p/noticing.html) when you don't understand something, and then *take action* by understanding it, making a note for later, or something else. Not doing this has caused me to unintentionally plateau for *years* at certain things (like CSS).

Conclusion
----------

Go out there and understand systems! Watch talks, read articles, reimplement existing software. We built computers, a human wrote every line of code that's being executed. **You** ***can*** **understand it**.

**Notice** when you're doing brute-force search due to a lack of understanding, and **take action** to build that understanding. The investment will pay off, often immediately, as a black-box search for solutions can be extremely inefficient.[^ai0jcih2uug]

Finally, [here's](https://github.com/codecrafters-io/build-your-own-x) a Github megalist of **resources** around "[building your own x](https://github.com/codecrafters-io/build-your-own-x)" \- one of the best ways to understand a system is to build it yourself, so go out there do that! open the black box!

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/aee313a815d05cbb3debae2b225b1be358073414436e445b.webp)

A black box being a leaky abstraction. Go out there and open it!

[^289bs9hi65b]: Me failing to follow this advice resulted in my CSS skills not improving for several years, as I would always go into the "try random stuff until it works" mode. 

[^ai0jcih2uug]: I am repeating this because it's that important.