I am self taught with no degree :( and think that the key is in being curious and exploring. A constant pressure/feeling of greater ignorance also helps drive that. I learned to program cause I wanted to make games so from there I picked up C, a tiny bit of numerical methods, pointers and finite state machines. Self teaching C++ was horrible so I quit computers for a couple of years. However, I came back to it but wanted a language as different as possible from C++. I tried python but my short term memory is terrible, I could not get my head around dynamic types. I tried scheme but was too dumb to get the syntax. I had an interest in pure maths so was drawn to haskell and ML. Simply being interested in haskell you get thrown in the deep end and get introduced to a bunch of theoretical stuff which I perused for a couple of years, including from the list:

Data Structures,
Programming Languages (http://www.cis.upenn.edu/~bcpierce/tapl/),
Code-as-Data,
Patterns (mostly in the guise of how pattern Y is an inferior version of feature X oh & monads are super patterns),
Functional Programming,
Object Composition,
Recursion,
Lambda Calculus,
Type Systems, Category Theory

Each of the above expands and leads to its own world. As a self learner you just have to keep exploring. For example with category theory, you pick up type theory and learn how they and sets all relate. From sites like this one you learn about the importance of unit testing and version control. From lambda calculus you can rewind to Frege, unrewind to zermelo and learn about first order logic and trace a line from Haskell to Weierstrass program - taken to its absurd conclusion with Bourbaki - to rigorize Calculus (along the way you may learn of infinitesimals and the hyperreals formulation of Calculus by robinson which I found more intuitive).

After a while I realized that I was really into Artificial intelligence, Graph theory and Subjective Probability theory. The latter two, I think will be the Calculus of the future[1]. This led me through Machine Learning, more much more numerical methods and brings me to today. I don't know much about search or compilers or a great understanding of system internals but I can pick it up if it interests me or I need it.

The downside to self learning is that with no teacher everything is harder. With no teacher you can't double check your model so really understanding takes longer. There is no need to test yourself so you are in danger of jumping around without having properly learned anything.

My solution is to read voraciously to index and just work on what I want so that if I need a concept, knowing of its existence allows me to know that I should learn more on that topic. at the least I can make that connection. Revisiting is key, if you don't understand something now move on but be sure to come back later. Relate/Analogize/search for treatments that most suit you. Good with programming and learning differentiation? Then derivatives as higher order functions paired with Euler's notation makes it way easier and makes expanding to higher dimensions more straightforward. Learning hard things gets easier if you keep at it and you continually expand your base. Its all very slow going though and knowing when to Explore and Exploit your current knowledge is tricky. Till recently I leaned too much towards explore but if you want to get stuff done you need to exploit with what you got. I doubt this approach would have been viable 15 years ago. Or before Google and Wikipedia citations. Finally, I hate academic papers behind paywalls.

[1]As the corner stone of techniques. I see the concept of entropy appearing everywhere and find the idea of quantum mechanics as a complex probability theory just incredible.