TL;DR: Go deep, but make a map.

I would think that complicated topics like compiler construction or LISP implementation actually benefit from deep knowledge and understanding. There are a few topics where I feel like I have achieved some kind of mastery, and it comes with

a) the knowledge that I actually know nothing about the subject itself

b) a kind of uncanny "wisdom" about where to look for problems, further reference

c) the ability to explain the topic in terms of very simple concepts and examples, while still being able to answer most questions.

A concrete example I like to give about this is: I've been programming with all my heart for 18 years, 8 hours a day. Yet I still feel as dumb in front of a for loop for fear of making an off by one as in the first days. What I am able to do now however is know where the dragon lays, double check it, write some tests around it, and being able to pinpoint the problematic for loop in the complete program in a "usually" pretty short time. Or being able to look at the compiler part responsible when it's actually the compiler making the off by one. Or checking how threadsafe the whole thing is. If that doesn't work I could still rewrite the whole thing in a recursive fashion. Or asynchronously with a state machine. And sometimes I'm just a moron and I made a stupid mistake and took 5 hours to find it, and feel as much as a newbie as I did when I was 13.

For example, once you know "everything" about compiler construction: LALR parsing, AST construction, register allocation, SSA trees, static analysis, written a few compilers for your own usage, extended existing ones, wrote a few backends for LLVM, created a language of your own with some following on the internet, you realize you know nothing. You don't know how register allocation works on an IA64, or maybe you have no grasp anymore about how register banks work on modern CPU cores, or maybe you have no idea about how a JIT on ARM would work, and that's just register allocation. You know that you've given it a really serious try, but there is just no way that you actually know anything about compilers.

However, you are able to explain what a compiler does to a layman in simple terms, transforming a formula with multiplications into a formula without multiplications. You are able to explain what a compiler does to a beginning programmer by actually implementing one right in front of her, in 80 lines of python. And you are actually able to answer her questions, which can be extremely difficult questions, while being able to gauge the complexity of the answer and tailor fit it. That's why I think that deep knowledge is very valuable.

What I find very important to write down (and maybe communicate to others, I don't know) is the steps you are taking while learning a topic, the resources you are reading, the conversations you have. I learnt a few artistic fields in the last years, where practice as a physical and mental activity is there to engrain an intuitive understanding of the topic in your soul. In drawing and in music, it's more through practice than through thinking that I acquire new concepts. I play and focus and draw and do some exercises or copy some of the masters, and one day I wake up and I somehow know it's there. And the way and practice and exercises that led to that insight are very easy to forget. It's very easy to forget that it's probably from copying this one painting over and over that I suddenly understood how to render wood. The only way to communicate that kind of knowledge to other people is to have a whip and tell them "copy that painting 5 times over, then you'll understand" because it is so incredibly hard to fit that knowledge into words.

The process is probably slightly the same for more "left brain" activities, but it's easier to be rational about the way to go about it. Still, I now try to write down which book, which exercise and which program I did, in the hope of one day tracing down how some of my "expert" traits came to be.