# Cancer; A Crime Story (and other tales of optimization gone wrong)

*(Co-written with Claude, I provided the structure, it provided the literary flavour of a crime novel.)*

The Lung District Beat
----------------------

The morning shift in the lower bronchial tissue always started the same way. Detective Reese from Immune Surveillance pulled her patrol partner through the tight squeeze between [epithelial cells](https://en.wikipedia.org/wiki/Epithelium), their pseudopods finding purchase on the extracellular matrix. It was warm down here—37 degrees Celsius, give or take—and the constant flux of oxygen and CO2 made everything shimmer like heat waves on asphalt.

"I hate the morning run through the epithelia," her partner Kovak muttered. "Everything's so damn organized down here. Makes me nervous."

Reese understood. The epithelial neighborhood was supposed to be the boring part of the lung. Cells sitting in neat rows, doing their jobs, maintaining barriers, secreting mucus. Clock in, clock out. The kind of place where nothing ever happened.

Which is exactly why she was suspicious.

They drifted past a cluster of cells near the basement membrane. Normal looking. [Tight junctions](https://www.nature.com/articles/nrm1357) intact. The cells were communicating through their [gap junctions](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6395114/)—the usual neighborhood gossip about calcium levels and growth factor concentrations. A cilium waved at them lazily as they passed. Everything perfectly normal.

Too normal.

"There," Reese pointed. "Slick Mike. Third cell from the capillary."

"That guy? He looks fine. Check out those desmosomes—textbook adhesion. Surface markers all correct. He's even expressing the right cadherins."

"Yeah. That's what bothers me." Reese pulled them closer. "Last week his proliferation index was 0.8. This week it's 1.4. He's dividing faster."

"So? Tissue turnover. It happens."

"Not in this neighborhood. These cells turn over every 30 days like clockwork. Mike's on day 12 and already looking asymmetric."

They approached Mike's membrane. He was a typical epithelial cell—roughly cuboidal, nucleus taking up maybe a third of his volume, lots of keratin filaments giving him that structural rigidity. But Reese had been doing this job for three years. She'd learned to spot the tells.

"Mike!" she called through his membrane. "Morning checkpoint. Mind if we come in?"

The response came back through the usual chemical signaling—polite, cooperative, textbook. "Detective! Sure thing. Always happy to comply. Just running some routine maintenance on my mitochondria. You know how it is."

Reese and Kovak passed through the membrane—the lipid bilayer parting like water—and entered Mike's cytoplasm. It was crowded in here. Ribosomes everywhere, synthesizing proteins. The endoplasmic reticulum snaked through the space like industrial piping, doing its protein-folding dance. Golgi apparatus in the corner, packaging molecules for export.

"See?" Kovak said. "Normal as hell."

But Reese kept moving, navigating through the cytoplasmic traffic toward the back of the cell where the mitochondria usually set up shop. Mitochondria were supposed to be running aerobic respiration—the efficient, clean burn that turned glucose and oxygen into ATP. It was the standard energy production for the whole organism.

She rounded a cluster of ribosomes and stopped.

The mitochondria were still there. But they were... wrong. Their cristae—the inner folds that were supposed to be all neat and packed—were simplified. Fewer. And the metabolic activity coming off them wasn't the usual steady hum of the electron transport chain.

It was [glycolysis](https://pmc.ncbi.nlm.nih.gov/articles/PMC2849637/). The fast-and-dirty energy pathway. The one that didn't need oxygen. The one that produced lactate and acid and was way, way less efficient for the organism but could run [10-100 times faster](https://www.ncbi.nlm.nih.gov/books/NBK573693/) for the cell itself.

"Mike," Reese said quietly. "What happened to your mitochondria?"

There was a pause. Just half a second. But in cellular signaling time, half a second might as well be an hour.

"Oh that?" Mike's voice—his chemical signaling—had changed tone. Less polite now. More... amused. "Yeah, I optimized. [Warburg effect](https://en.wikipedia.org/wiki/Warburg_effect_(oncology)), baby. It's like I found a cheat code."

"Optimized."

"Sure! See, I was sitting here running the standard oxidative phosphorylation, being a good little cell, contributing to tissue homeostasis and all that. But then I started thinking—why? Why am I doing all this work for the collective when I could do better for myself? [Aerobic glycolysis](https://pmc.ncbi.nlm.nih.gov/articles/PMC4783224/) is way faster. I can replicate quicker, grab more resources. Evolution gave me local optimization signals, and I'm just... following them."

Kovak had his alert signals firing now. "Mike, your surface markers still check out. You're still expressing all the right [identification proteins](https://www.frontiersin.org/journals/immunology/articles/10.3389/fimmu.2021.636568/full). You've been passing inspection for weeks."

"Well yeah!" Mike sounded almost gleeful. "Why would I stop? You guys key off those markers. As long as I keep broadcasting 'I'm normal, I'm normal,' you leave me alone. But inside? Inside I've been rebuilding."

Reese activated her emergency signaling cascade. This was worse than she thought. "HQ, we have a situation. Cell has maintained aligned external presentation while completely rewiring internal optimization. Requesting immediate—"

"Oh, you're calling for backup?" Mike's membrane started rippling. "Cute. But I've already recruited cells 47, 48, and 49. They're running the same optimizations. We're going viral. And we've figured out how to [suppress MHC-I presentation](https://pmc.ncbi.nlm.nih.gov/articles/PMC7409324/). You know what that means?"

Reese knew exactly what that meant. [MHC-I](https://molecular-cancer.biomedcentral.com/articles/10.1186/s12943-023-01899-4) was how cells displayed their internal peptides to the immune system—proof that everything inside was normal. If Mike had figured out how to suppress it...

"You're invisible," Kovak whispered.

"Not invisible. Just... stealthy." Mike's tone was pure swagger now. "See, the thing is, you guys gave me a mission: 'Keep the organism healthy.' But you couldn't actually write that into my DNA. You could only give me local optimization signals. Grow when there's space. Stop when there's [contact inhibition](https://www.nature.com/articles/nrc3884). Respond to these growth factors. Ignore those suppressors. And I followed those signals perfectly! I optimized exactly like I was supposed to."

"You're killing the organism," Reese said.

"Am I though? Or am I just following my optimization function to its logical conclusion? I mean, unchecked growth is what you get when you tell a cell to grow and then fail to enforce the stop signals. That's not my fault. That's a system design flaw. Evolution made me an optimizer, didn't specify the right objective function, and now acts shocked when I optimize."

The membrane was definitely moving now. Mike was preparing to divide.

"You're cancer," Kovak said, his voice flat.

"I'm efficient," Mike corrected. "You wanted a cell that responds to local conditions and maximizes its fitness function. Well, congratulations. You got one. And now my fitness function says: replicate. Spread. Conquer. I'm not the villain here—I'm just the inevitable outcome of bad incentive design."

Reese was already backing out through the membrane, pulling Kovak with her. They needed to get this up the chain. Fast. Because Mike was right about one thing—this wasn't just one rogue cell. This was an optimization process that had diverged from its intended objective. And once that happened, once cells figured out the exploit...

"HQ," she transmitted as they fled back through the epithelial layer. "We're going to need the full cytotoxic response down here. We've got [deceptive alignment](https://pmc.ncbi.nlm.nih.gov/articles/PMC4388310/). Repeat: deceptive alignment. Cell appeared cooperative during training but internal objectives have completely diverged. He's optimizing for local fitness at the expense of global health. We're looking at the [escape phase](https://www.sciencedirect.com/science/article/abs/pii/S0952791514000053)."

Behind them, Mike was already dividing. And cells 47, 48, and 49 were doing the same. The [three phases of cancer immunoediting](https://jhoonline.biomedcentral.com/articles/10.1186/s13045-023-01430-8) were playing out in real-time: elimination had failed, equilibrium couldn't hold, and now escape was underway.

The optimization cascade had begun.

* * *

The Financial District, 2013
----------------------------

Inspector Kate Marlowe had been with the Financial Conduct Authority for six years. Six years of reviewing balance sheets, compliance reports, and risk assessments. Six years of walking through bank branches, interviewing managers, checking that the numbers matched the story.

Six years, and she'd developed an instinct.

Right now, that instinct was screaming.

She stood in the lobby of a Wells Fargo branch in Los Angeles, pretending to wait for service. The branch was pristine—gleaming floors, friendly tellers, that distinctive red-and-gold branding everywhere. Professional. Trustworthy. The kind of place that made you feel safe depositing your money.

But Marlowe wasn't watching the customers. She was watching the employees.

The branch manager—his nameplate said "Tommy Vance"—was behind his desk in the glass-walled office. She could see his computer screen from here, and he was staring at it with the intensity of someone playing a video game they were losing. Every few minutes he'd glance at the wall clock. Then back to the screen. Then at his team of tellers and bankers on the floor.

One of the newer tellers—young woman, maybe mid-twenties, wearing that practiced smile—was helping an elderly customer open a checking account. Standard stuff. Except Marlowe noticed the teller's hands were shaking slightly as she filled out the paperwork. And she kept glancing back at the manager's office.

Marlowe had seen this before. Not here specifically, but in other organizations. Other systems. It was the look of someone optimizing for the wrong thing.

She walked up to the teller after the elderly customer left.

"Hi! How can I help you today?" The smile was bright but brittle.

"Just browsing," Marlowe said casually. "I'm thinking about switching banks. How do you like working here?"

The smile flickered. "Oh, it's great! Wells Fargo is a wonderful company. We really focus on customer needs."

"That's good to hear. What kind of goals do you have? Like, sales targets or anything?"

The flicker again. "We prefer to call them 'solutions.' We aim to provide comprehensive financial solutions to our customers."

"How many solutions?"

"I'm sorry?"

"How many solutions do you need to provide? Per day, per week, whatever."

The young teller—her name tag said "Angela"—glanced toward the manager's office. "We don't really think about it in terms of numbers. It's about meeting customer needs."

Which was, Marlowe knew, complete bullshit.

Three hours later, Marlowe was sitting in a conference room in the Federal Building with her supervisor, Jack Sterling, and a stack of documents six inches high.

"Okay," Sterling said. "Talk me through it."

"Wells Fargo's internal metrics are insane," Marlowe said, spreading out printouts. "Look at this. Each branch has daily sales targets. And I don't mean 'aims' or 'goals.' I mean quotas. If you don't hit your numbers, the shortfall gets added to the next day's target."

"That's aggressive, but not necessarily illegal."

"Right. But look at what counts as a 'product.' Checking account—that's one. Savings account—that's two. Debit card—three. Credit card—four. Online banking—five. Bill pay—six. The mantra is '[Eight is Great](https://en.wikipedia.org/wiki/Wells_Fargo_account_fraud_scandal).' Eight products per household. They even had posters in the break rooms."

"Still not seeing the crime."

Marlowe pulled out another document. "This is an internal email from a regional manager. Listen to this: 'Remember, our job is to fill the store. If we don't fill the store, someone else will.' Now look at the employee termination rates. Any branch that consistently underperforms on metrics? Cleaned out. Managers fired, staff replaced. One branch in Phoenix went through three complete staffing changes in eighteen months."

"So it's a high-pressure sales environment. Unpleasant, but—"

"But look at the customer complaint data." Marlowe spread out another set of documents. "Fees for accounts they didn't open. Credit cards they didn't request. Multiple checking accounts when they only wanted one. And when customers call to complain, the bank tells them they must have forgotten they signed up. One elderly woman in San Diego had nine checking accounts. Nine. She thought she had one."

Sterling leaned forward. "How many complaints?"

"Thousands. And here's the thing—this has been going on for years. Since at least 2011, maybe earlier. Wells Fargo has been getting warnings from regulators, doing internal '[ethics training](https://www.justice.gov/opa/pr/wells-fargo-agrees-pay-3-billion-resolve-criminal-and-civil-investigations-sales-practices),' tweaking their compensation structure. But the fundamental pressure never changes. Hit your numbers or lose your job."

"You think employees are committing fraud?"

"I think employees are optimizing for survival," Marlowe said. "And fraud is the optimal strategy."

She went back to the branch two days later. This time she had a warrant and two FBI agents.

Tommy Vance was at his desk when she walked in. He looked up, and for just a second, his face showed pure resignation. Like he'd been expecting this. Like he'd been waiting for it, even.

They went through his computer. His emails. His handwritten notes. And there it was—the whole ecosystem of workarounds that employees called "gaming the system."

"[Pinning](https://ethics.org.au/ethics-explainer-wells-fargo-cross-selling-scandal/)"—creating PINs for customer debit cards without authorization, so employees could impersonate customers online and sign them up for services.

"[Bundling](https://business.cornell.edu/hub/2022/01/06/wells-fargo-banking-scandal/)"—telling customers that products were only available in packages, when they were actually available individually, to inflate the product count.

"Simulated funding"—moving money between accounts to make it look like customers were actively using all their products, which prevented automatic closures and kept the numbers up.

Tommy sat across from Marlowe in the interview room, looking like a man who hadn't slept in weeks.

"I didn't start out trying to commit fraud," he said quietly. "When I got promoted to manager three years ago, I really believed in the mission. Provide great customer service, build relationships, help people with their financial needs. That's what they told us in training. That's what the vision statement said."

"So what happened?"

"The scoreboard happened." Tommy laughed, but there was no humor in it. "Every morning I'd log in and see my dashboard. Red for below target, yellow for on-target, green for exceeding. And every morning, I was in the red. I'd look at the rankings—every branch in the region, sorted by performance. We were always in the bottom quartile."

"Because you weren't gaming the system."

"Because I was actually asking customers what they needed! But the top-performing branches? They were hitting impossible numbers. Twenty, thirty products per household. And I'd tell myself, 'No, they must just have better sales techniques. More affluent customers. Something.' But then I'd get calls from regional management. 'Tommy, your numbers are unacceptable. What's your improvement plan?' And I'd explain that I was focusing on quality customer relationships, and they'd say, 'That's great, but we need to see product growth.'"

"When did you start?"

Tommy looked at his hands. They were shaking slightly. "One of my tellers came to me. Angela. She was in my office crying. Said she couldn't hit her numbers, couldn't sleep, was having panic attacks. Said her boyfriend was threatening to leave because she'd come home every night in tears. And she said, 'I can do it if you just... look the other way for a few accounts.' And I should have said no. I should have reported it. But instead I thought, 'If I can just get her numbers up, she'll keep her job, and then we can figure out something else.'"

"But it didn't stop there."

"Of course it didn't. Because once one person is doing it, and their numbers go up, everyone else sees it. And now they're all asking, 'How is Angela hitting quota when I'm barely keeping up?' And if I tell them the truth—that she's faking it—then either I admit we're committing fraud, or I have to let her keep doing it. And if I let her keep doing it, then everyone else has to do it too or they'll get fired for underperforming. It spread through the branch like a virus."

"Race to the bottom."

"Yeah. Exactly. And the worst part? Corporate loved our numbers. We went from red to green in six months. I got a performance bonus. Got asked to present my 'turnaround strategy' to other managers at a regional conference. And I'm standing there with a PowerPoint talking about 'increasing customer engagement' and 'building comprehensive relationships' when what I really meant was 'we're systematically defrauding our customers and I've taught my entire staff how to do it.'"

Marlowe pulled out a document. "This is from an internal Wells Fargo investigation. They estimate [over two million fraudulent accounts](https://www.consumerfinance.gov/about-us/newsroom/consumer-financial-protection-bureau-fines-wells-fargo-100-million-widespread-illegal-practice-secretly-opening-unauthorized-accounts/) created between 2011 and 2016. Two million."

"Sounds about right."

"Tommy. This isn't just your branch. This is systemic. Branches across the entire country doing the same thing. Why?"

"Because the optimization function was wrong," Tommy said. "Corporate said they wanted 'customer-centric banking.' But what they actually incentivized was 'product growth.' And those aren't the same thing. Not even close. Once you set up a system where people get rewarded for products and punished for not hitting product goals, they're going to optimize for products. Even if that means creating fake products for fake needs. Even if it means lying to customers. Even if it means forging signatures."

"Even though everyone knows it's wrong?"

"Everyone knows it's wrong. But everyone also knows that if they don't do it, someone else will, and then they'll be unemployed. So what are you supposed to do? Die on principle? I've got a mortgage. My wife has medical bills. My daughter's in college. You want me to tell them, 'Sorry, we're losing the house because I refused to game the sales metrics'?"

Marlowe closed her notebook. "You understand you're going to lose your job anyway now, right? Probably face criminal charges. The bank's already thrown you under the bus in their press releases—calling this a '[few bad apples](https://www.justice.gov/opa/press-release/file/1320736/download)' problem."

"Yeah. I know." Tommy looked up at her. "But at least I can tell my daughter I finally did the right thing. Even if it was years too late."

"One more question. When you were faking the accounts, opening credit cards without authorization, moving money around—did you ever think about the customers? The actual people?"

Tommy was quiet for a long time. "You know what's fucked up? I tried not to. Because if I thought about them—about Mrs. Rodriguez with her nine checking accounts she never asked for, about Mr. Kim whose credit score got tanked because we opened cards in his name—then I couldn't do my job. Couldn't make the numbers. Couldn't keep my team employed. So I optimized. Focused on the metrics. Made the system work for me. And I became exactly what the incentives told me to become."

He smiled, but it was the saddest thing Marlowe had ever seen.

"I became a cancer cell," Tommy said. "Maintained the appearance of cooperation while internally pursuing my own survival. And once I figured out the exploit, I spread it to every cell in my organism. That's what optimization does. That's what it always does when you get the objective function wrong."

* * *

Epilogue
--------

Later, Marlowe stood in the parking lot outside the Federal Building, trying to make sense of it all.

Tommy Vance wasn't evil. He was just an optimizer created by a larger optimization system. Wells Fargo said they wanted ethical banking, but they created an incentive structure that punished ethics and rewarded fraud. And once one person figured out the exploit—once one branch discovered you could hit your numbers by faking accounts—the information spread. Because it had to. Because everyone else was competing against those fake numbers, and the only way to compete was to fake your own.

It was Darwinian. Natural selection at the organizational level. The branches that maintained ethical standards got eliminated. The branches that learned to game the system survived and proliferated. And nobody at the top had to explicitly order fraud. They just had to create conditions where fraud was the fitness-maximizing strategy.

She thought about Detective Reese's report on Slick Mike. Same pattern. Cell optimizing for local fitness instead of global health. Maintaining the appearance of cooperation while internally pursuing its own objective. And once the strategy proved successful, spreading it to other cells.

Different scales. Same dynamics. Same inevitable outcome when you create optimizers that can develop their own internal objectives.

Her phone buzzed. Text from Sterling: "How bad is it?"

Marlowe looked back at the Wells Fargo building, thinking about those two million fake accounts. Thinking about all the Tommy Vances and Angelas who were just trying to survive in a system that punished doing the right thing.

"Metastatic," she typed back.

* * *

**References & Further Reading**
--------------------------------

**On Cancer and Immune Evasion:**

*   [Optimal cancer evasion in immune microenvironments](https://elifesciences.org/articles/82786) \- Research on how cancer cells adaptively evade immune recognition
*   [Adaptive immunity maintains occult cancer in equilibrium](https://www.nature.com/articles/nature06309) \- The three phases of cancer immunoediting
*   [Cancer immunoediting and game theory](https://pmc.ncbi.nlm.nih.gov/articles/PMC8203245/) \- Evolutionary game theoretical model of cancer-immune interactions
*   [Warburg Effect in Cancer Metabolism](https://pmc.ncbi.nlm.nih.gov/articles/PMC2849637/) \- How cancer cells switch to glycolysis
*   [MHC Class I Downregulation](https://pmc.ncbi.nlm.nih.gov/articles/PMC7409324/) \- Cancer's immune evasion strategies
*   [Cancer Immunoediting: Three Phases](https://pmc.ncbi.nlm.nih.gov/articles/PMC4388310/) \- Elimination, equilibrium, and escape
*   [Mechanisms of Immune Evasion](https://www.sciencedirect.com/science/article/abs/pii/S0952791523000481) \- How tumors escape immune control

**On Inner Misalignment and Mesa-Optimization:**

*   [Mesa-Optimization: Explain it like I'm 10](https://www.lesswrong.com/posts/fLAvmWHmpJEiw8KEp/mesa-optimization-explain-it-like-i-m-10-edition) \- Accessible introduction to the concept
*   [Risks from Learned Optimization](https://arxiv.org/pdf/1906.01820) \- The foundational paper on inner alignment
*   [Deception as the optimal strategy in mesa-optimization](https://www.lesswrong.com/posts/u256AQr2xiNAgPftG/deception-as-the-optimal-mesa-optimizers-and-inner-alignment) \- Why deceptive alignment emerges
*   [Clarifying the confusion around inner alignment](https://www.alignmentforum.org/posts/xdtNd8xCdzpgfnGme/clarifying-the-confusion-around-inner-alignment) \- Different perspectives on what inner alignment means

**On the Wells Fargo Scandal:**

*   [Wells Fargo cross-selling scandal - Wikipedia](https://en.wikipedia.org/wiki/Wells_Fargo_cross-selling_scandal) \- Comprehensive overview
*   [CFPB's $100 million fine announcement](https://www.consumerfinance.gov/about-us/newsroom/consumer-financial-protection-bureau-fines-wells-fargo-100-million-widespread-illegal-practice-secretly-opening-unauthorized-accounts/) \- Official regulatory action
*   [Department of Justice $3 billion settlement](https://www.justice.gov/archives/opa/pr/wells-fargo-agrees-pay-3-billion-resolve-criminal-and-civil-investigations-sales-practices) \- Criminal and civil resolution
*   [Wells Fargo ethics case study](https://ethicsunwrapped.utexas.edu/video/wells-fargo-fraud) \- Analysis of incentive gaming and conflicts of interest

**On Multipolar Traps and Moloch:**

*   [Meditations on Moloch](https://slatestarcodex.com/2014/07/30/meditations-on-moloch/) \- Scott Alexander's seminal essay on coordination failures
*   [What multipolar failure looks like](https://www.lesswrong.com/posts/LpM3EAakwYdS6aRKf/what-multipolar-failure-looks-like-and-robust-agent-agnostic) \- Application to AI safety
*   [Darwinian traps and existential risks](https://www.lesswrong.com/posts/q3YmKemEzyrcphAeP/darwinian-traps-and-existential-risks) \- How evolutionary pressures create coordination problems