This is the core challenge with measuring skills acquisition. To avoid talking my book about measuring developer skill, mind if I instead talk about my first love, which is learning foreign languages?

A lot of people have vague goals like "I want to learn French" or "I want to be fluent in Japanese." There is no defensible definition of the word "fluent." Instead, you should have specific goals which test ability to complete tasks that are representative of the larger set of tasks you need to be good at to achieve metagoals which are important to you.

This is why I care relatively little about "fluency in Japanese" and quite a bit about "what percentage of commercially significant terms in my apartment lease did I understand without having to ask a Japanese speaker to explain them to me?" That task is roughly representative of many tasks required to achieve my metagoal, which is "being a functioning adult / educated professional in Japanese society."

Now how do I measure progress? Well, I have some notion of groupings of tasks by difficulty level. The "apartment lease" task is in the same grouping and difficulty level as the "employment contract" task was or the "extract the relevant rule for recognizing SaaS revenue from the National Tax Agency's docs" was. Given roughly comparable levels of difficulty, if I start doing better on a task where previously I did poorly, then I'm progressing.

Why don't I just take Japanese tests yearly? Because my metagoal is not becoming the best Japanese test-taker there is. They are good from the perspective of many decisionmakers, since they allow decisionmakers to compare me against other people in a reproducible and cheap-at-the-margin fashion, but that doesn't get anything that I value. I don't care how I compare to Frank or Taro -- being better than Frank will not save me social embarrassment if I have to ask an accountant "Here is my... um, I don't know what the word is, but it's the piece of paper that records the historical prices I purchased by assets at and then their declining present value representing their worth diminishing over time as calculated by the straight line method. There's an accounting word I'm searching for here and I bet it is followed by the word 'schedule.' DEPRECIATION. Yep, that's the one, thanks."

Anyhow: all tasks are an opportunity to gauge my present skill level. I could, theoretically, even keep a journal of them, and I actually did that back in the day. (It's in the other room at the moment. Entries included things like "9/1/2005: Watched a nightly news broadcast about an earthquake. Understood it was about an earthquake and that there was one fatality. Didn't understand remaining 80%.") This gives me both a list of representative things to try leveling up on and some intrinsic motivation because when adding new entries to it I was occasionally pleasantly surprised how far I had come, where that very rarely occurred to me in the moment. (I often feel "Man I'm totally lost at this!" but exposure to the historical record shows that I'm totally lost on tasks which are MUCH HARDER than the tasks I struggled with years ago and that those tasks are now so below the trivial floor that I've forgotten they were once representative examples of things I wanted to strive to be able to do.)

Treating every little thing as an example to both practice and evaluate progress lets me work on focusing things which generally matter to me, gives me a reasonably quick evaluation cycle, and gives me frequent opportunities to achieve mini-victories even in the course of a long, long, looooong learning process.

Evaluating your skill with engineering is, roughly, similar. What relevant things can you do today which you weren't able to do a while ago? That's the progress you've made. Where possible, structure your life to expose you to new challenges which allow you to gauge your increase in skill, as long as marginal skill in that area is actually valuable to you in achieving your metagoals.