I'm wrestling with a similar question, but from a different angle. My dream (long term) is to teach at the university level, but places won't hire you as a professor unless you have at least a Master's -- relevant work experience be damned.

That said, I also don't understand the culture at FAANGs. I'm at a senior engineer / architect level at a more "regular" company in an AI / ML role.

Regarding your reasons for pursuing a degree:

1) yes and no. In my experience, there is a sharp distinction between "book learnin'" and actual, productizable engineering. Academic datasets are idealistic. Researchers create novel architectures, and write their own optimizers.

In the "real world", data is messy. Incredibly messy. Most of my time spent on industrial ML projects is spent wrangling data -- cleaning it, auditing it, acquiring more of it, synthesizing it, etc. The "interesting" bits of more academic ML aren't used nearly as often as I first thought. Far more useful is learning how to do transfer learning (which academic institutions seem to rarely teach much about), and getting a solid grasp of the various kinds of standard models that are out there, so that you can plug them together like LEGO. This second part is probably the most useful thing you would get from an academic approach -- a wide array of fundamental algorithms and traditional ways of doing things ("Markov chains, yay!" -- something I should learn about one of these days).

And while it is helpful to have a large toolbox of various widgets to help you tackle different problems... honestly the world is moving so fast, I'm not entirely sure that academics would give you (or me) the leg up that we want.

Personally I've found a lot of success in personal projects and volunteering for whatever AI / ML projects I can. It's not perfect, and I still kindly feel like I'm missing some boat somewhere...

... but I think a lot of that is just because this field is moving so incredibly fast, it's difficult to stay on top of anything.

Normally I work in the field of computer vision, but lately I've been diving into natural language and LLMs -- especially embedding models and rerankers. Finally fine-tuned my own embedding models, and got (somewhat) useful results out of them. Really chuffed about that. It's such a long road, and I still have so far to go, but I'm learning and growing.

Is embarrassing to see my first attempts at building my own RAG from even just a few weeks ago. I identified a lot with what you said where you noted how much you still don't know about information retrieval systems -- I feel exactly the same way. But I think there's a lot you can learn by doing, rather than going to school.

It may sound like I'm trying to talk you (and me) out of going to university, but I'm really not. Even with all of the above, I still think there's benefit to me going back to school -- I'm personally eyeing the OMSCS program at Georgia Tech.

I don't know how Masters programs are received at your work, but at mine, the bachelors holders are right alongside the masters and those who were on doctoral tracks. Sometimes those people are better than me, and sometimes not. It's very collaborative, and it's not a clear "win" for people with extra degrees -- at least at my place of employment. I don't know how it is at FAANGs, so hourly someone else can speak better to that than I can.

Sorry this is so rambly, but I hope there were useful nuggets in here.

Whichever direction you go, I hope you find success! I'm interested in following along with you as you gather info, and would love to hear more about what you learn and decide!