1 - Getting the Celestron Ultima 2000 "goto" telescope debugged and working. There were all kinds of problems: the software was a home-brew multi-threaded real-time operating system that ran on tiny little inexpensive processors, and had a lot of bugs. The closed loop control algorithms were broken. When we started, the telescope would go into "paint can shaker" mode when we told it to move to a star. The mechanical design had a lot of problems and went through several redesigns. There were problems with the encoders (sinusoidal errors too large to ignore, which ended up requiring software calibration and compensation), and the main axes were not sufficiently close to perpendicular, also requiring per-system calibration and software compensation. There were electrical problems requiring strategically placed capacitors and things. For crying out loud, we even had a Y2K problem! The scope had difficulty tracking objects over the zenith. The gearboxes had a significant amount of backlash, requiring software compensation. And on and on. It was an extremely difficult system to get working, but in the end it was the most satisfying thing I'd done in my career to that point. (And, it was nice to do something as a software guy for my father, Tom Johnson, the founder of Celestron.)

2 - The Integrated Medical Systems LS-1, a portable "ICU in a box" done under contract for the US Army. The box integrated multiple medical device subsystems (ventilator, ECG, infusion pumps, smart battery charging, invasive and non-invasive blood pressure, SpO2, etc. etc.) They communicated internally over 100-base-T cat-5, and also had to interface to hospital IT via wifi. And, there was a remote interface unit that had to be able to remotely monitor and control the entire system via internet, from the other side of the planet if necessary. We ended up doing a lot of work to ensure that the system performed exactly as expected and required by clinicians, even in these sort of remote-operation scenarios. (The thing the Army wanted was to have video and audio from the bedside to remote clinicians, and have the remote people act like team members that were coordinating their efforts with the bedside clinical team.)

3 - (Totally for the hell of it, in hobby mode) I was reading a bit about Shor's algorithm to use quantum computing to factor products of pairs of prime numbers. One key part of the algorithm depends on FFT's, to help with detection of lengths of cycles. If you have a vector of length N, consisting of K equally spaced non-zero elements, and K evenly divides N, then the FFT of that vector will have N/K evenly spaces non-zero elements, and crucially they will start in the zero'th element of the output vector. (There is no such requirement or assumption on the input side.) Happily enough, this is "almost" true if K does not happen to evenly divide N. But this is a bit difficult to prove. Some reasonably complete and rigorous presentations of quantum computing basically say, "The proof of this is outside of the scope of the present work." So, for fun I took it as a challenge to create an elementary proof of this lemma. It turned out to be quite hard, but REALLY fun and satisfying.

4 - Never been really happy with the standard presentations of red-black tree algorithms. It just felt like there was some underlying simplicity in there that was struggling to get out. So, I created a new formulation of red-black trees and corresponding algorithms. Did a web site (gregfjohnson.com/redblackbuilder.html) that illustrates these algorithms, and supports forward and backward execution and single-stepping through the algorithms. This web site was pretty darn hard to get correct.