My wife had a terrible autoimmune disease (dermatomyositis) and gained a lot of weight because she was pumped with huge amounts of steroids.

So I looked into some of the biochemistry, social, psychological, and neurological factors around weight loss, and there's a few crucial problems:

1. Most of the information and advice is unscientific, pro-industry, or outright fraud that peddles feel-good short-term weight loss for long-term financial gain;

2. The common-sense approach, "eat less", does not work – evidence suggest it fails to take into account how the brain and body work (and, in particular, how we make decisions);

3. There are at least three equilibriums in balance, of which we tend to deal only with the first:

i.) The short-term hunger cycle (days) – i.e. we eat when we're hungry;

ii.) The medium-term (months) "set-point", deep in the decision-making part of the brain, that crops up (there's a good Ted talk about this), that controls metabolic function - how much we eat, and of what we eat how much is stored as fat;

iii.) The long-term/result-based (months-years) hydrocarbon cycle - hormones emitted because of fat cells that are not flush with energy (i.e. living fat cells that can contain more hydrogen) trigger brain chemistry changes that appear to trigger reduced inhibitions during decision making. Binging.

4. Stigma and social support – understanding of weight-loss is widely misunderstood, oversimplified, taken for granted, or part of a counter-productive commercial motivation (i.e. in Western society we're surrounded by people who think you can pay for weight loss — "I went to Jenny Craig and lost 10lbs"). People trying to lose weight are often shunned from the very social settings needed for weight loss to occur, and commercially driven advice dominates our perceptions, leaving it tainted or geared towards short-term cycles of self-doubt, insecurity, impotence, and frustration, that in almost all cases (>95%, IIRC) leads to perpetual, cyclical weight gain.

It seems that for any weight loss to be permanent, it must deal with all three cycles. Media articles like the linked are helpful for spreading awareness of the complexity of the issue, but they rarely go beyond a widely-accepted single-focus theory ("eating > exercise"), which I recall being common knowledge in the 1980's (when we used to say "diet is 90% of the battle").

All that said, there appear to be two ways that have a substantial probability of permanent weight-loss:

A.) become a calorie-counting fanatic; or
B.) gastric bypass (staples, not the tube).

The number of people who have the mental fortitude and social setting to sustain A. is less than 0.5%, I seem to recall. Leaving bypass surgery as the only option, but it remains expensive and uncommon.

One article I read indicated that if everyone who needed bypass had it, many food companies in the USA would be driven out of business. I thought that was a fascinating insight into the meta-forces that could be at work.

The above is just a set of highlights from my recollection of critically reading hundreds of articles on the topic. What was particularly striking during my research was how weak (if not outright misleading) the vast majority of the articles were, with a few – sadly, exceedingly – rare exceptions.

In future we might come to understand the biology better, but that's a small part of the battle. I suspect much research needs to happen on the medium-term "set-point", but it would appear that we need not just research but a better formulation of how research is fundamentally driven, commercial food and weight-loss incentives are aligned with health and actual results, cultural attitudes that help sustain sensible choices rather than stigmatize and ostracize, and better ways for information about complex topics such as this to be disseminated in useful ways.