When considering just the energy balance, calories aren't an approximation. You see, energy is what we call a state function, it depends only on the state of the system and not how we got there. The catabolism of lipids, sugars and proteins into carbon dioxide, water and urea is an extremely complicated process, which I'm not sure we've even completely mapped yet. Last time I counted, the oxidation of glucose in our body takes about 33 steps. However, because we know the difference in energy (actually enthalpy but let's not split hairs) between the input and the output, we know the exact total amount of energy released in all those steps. And it has to be exactly the same as the energy released when simply burning the food in a furnace, because even though the process is wholly different, the begin and end products are the same.

This is barely nutrition, this is just basic thermodynamics. Again, because energy is a state function, no assumptions need to be made about the efficiency of any of the intermediate processes or about how the muscles/mitochondria/liver/kidneys/... work, energy conservation keeps us from having to mess with that.

The reason this analysis is so useful is because we're an endothermic species. The core of all of our bodies are at pretty much the same temperature. And as anyone who's had to pay a heating bill knows, heat isn't cheap. In fact, the bulk of our food energy is expended on keeping us warm. This is why there are no such things as slow or fast metabolism. If your metabolism were too slow to burn your resting energy expenditure in 24 hours, you'd die of undercooling. If your metabolism were significantly faster than normal, you'd die of overheating (or at least you'd feel significantly warm to the touch, be constantly sweating/flushed). In fact, a person's resting metabolic rate can be accurately predicted from their body composition [0], and doesn't differ between age or gender, as would be expected. This puts a pretty high lower limit on one's energy expenditure, absolutely nobody can maintain their weight on, say, 600 calories per day. You simply can't keep a 70 kg sack of mostly water at the right temperature with that energy budget, not with how badly isolated we are.

Even if it does seem oversimplified, it's no spherical cow. The energy balance is exact, because of basic physics and biochemistry: we have only one way of generating energy (oxidizing reduced carbon), only one way of storing energy for the long term (long chains of reduced carbon), no energy is lost (because heat generation is a 100% efficient process) and our digestion is very efficient (not feces, urine nor sweat contain significant amounts of carbohydrates, lipids, protein or any other high-energy compound). It's also useful because we have an unflexible lower limit to the amount of energy we expend in a day because of the way our bodies work. The wiggle-room on the previous assumptions only serves to make calories-in smaller (e.g. malabsorption) or calories-out larger (e.g. exercise), and thus would only make it easier to lose weight.

I won't deny the microbiome is important. There is a probably a correlation between gut biome and obesity. But that can only because it influences things like satiety or mood, because none of those bacteria seem to change the eventual energy equation, nor do they change how much heat you need to produce to keep warm or energy expended when lifting a weight. They're relevant because willpower is a finite resource, but they don't change the fact that any diet will in some sense need to consist of eating less calories than you burn in a day

0: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3139779/