abstract: There is increasing circumstantial evidence that the cuprate superconductors, and correlated-electron materials generally, defy simple materials categorization because of their proximity to one or more continuous zero-temperature phase transitions. This implies that the fifteen-year confusion about the cuprates is not fundamental at all but simply overinterpreted quantum criticality—an effect that seems mysterious by virtue of its hypersensitivity to perturbations, *i.e.* to sample imperfections in experiment and small modifications of approximation schemes in theoretical modeling, but is really just an unremarkable phase transition of some kind masquerading as something important, a sheep in wolf's clothing. This conclusion is extremely difficult for most physicists even to think about because it requires admitting that an identifiable physical phenomenon might cause the scientific method to fail in some cases. For this reason I have decided to explain the problem in a way that is nonthreatening, easy to read, and fun—as a satire modeled after a similar piece of Lewis Carroll's I once read. My story is humorous fiction. Any similarity of the characters to living persons is accidental. My apologies to Henry W. Longfellow. \[Published as Annals of Improbable Research **10**, No. 6 (May/June 2004), p. 8.\]
author: R. B. Laughlin
date: January 1, 2004
title: Hiawatha's Valence Bonding

# Introduction

> Since all men have imperfections
> Hanging bones inside their closets
> That they trust no one will notice
> Absent tips on where to find them,
> It will shock no one to learn that
> Even mighty Hiawatha
> Famous Chief of myth and legend
> Did some things he was not proud of
> While a brother in a frat house
> With a surly reputation
> At an unknown little college
> That his father helped to finance
> So that he would get admitted
> By the shores of Gitche-Gumee.

> Far from loving fields and flowers
> And the odor of the forest
> As one reads in all the textbooks
> Hiawatha hated woodlands
> And the animals one finds there,
> Whom he felt were always pooping,
> And the plants the critters fed on
> Down in dank and swampy bottoms,
> Nearly perfect grounds for breeding
> Mighty hordes of great mosquitoes
> Who were always lean and hungry
> And equipped with maps and radar
> Could detect where you were hiding
> To inflict their bites and torments,
> With their sneaky friends the black flies,
> And their angry friends the green flies,
> And the rocks ensnared by tree roots
> That existed just to trip you
> And would look improved as concrete
> In foundation for a condo.

> Thus the kindly, thoughtful image
> Of a noble man of Nature
> Was a total fabrication
> Of a team of gifted spin docs
> Hired later for this purpose.
> He was really just a tech nerd
> Who cared only for equations
> And explaining all behavior
> From the basic laws of physics
> Armed with only mathematics.

> Thus, instead of lakes and forests,
> Hiawatha worshipped Newton,
> Whose account of Kepler's orbits
> Built on rules that Galileo
> Had inferred from observation
> Plus the innocent assumption
> Of a law of gravitation
> Was a cosmic inspiration;
> And the brilliant Sadie Carnot,
> Whose insightful laws of heat flow
> Were deduced from working engines
> Absent microscopic theories;
> And the tragic Ludwig Boltzmann
> Who ascribed these laws to counting
> But fell victim to depression
> When he found no one believed him
> And so killed himself by jumping
> From an Adriatic tower.
> Hiawatha saw that Maxwell's
> Guessing missing laws of motion
> Needed for predicting light waves,
> Was the most transcendent genius,
> As was Albert Einstein's insight
> That the speed of light being constant
> Must mean time was not consistent
> And that mass could be converted
> Into heat and vice versa.
> Just as clear was that the Planck law
> Must imply DeBroglie's wavelength
> Was in force in any matter
> So that sharp atomic spectra
> And distinct atomic sizes
> And the laws of bond formation
> Came from quantum interference.

# Hiawatha's Mistake

> Thus it was that Hiawatha
> Came to be infatuated
> With the laws of quantum matter,
> Which means liquid noble gases,
> Neutrons in a burnt-out star core,
> Or just rocks so cryogenic
> They cannot get any colder,
> Even with improved equipment,
> Like the state of too much sliding
> On the ice of Gitche-Gumee
> After dark in dead of winter
> In an inexpensive loincloth.
> Pain and danger notwithstanding
> Quantum matter's simple structure
> Makes the eager physics tyro
> Quite unable to resist it.
> Hiawatha learned how atoms
> Self-assemble into crystals,
> How electrons move right through them,
> Waving past the rigid ions
> Thereby making them metallic
> In the absence of a bandgap
> Which arises from diffraction
> And prevents the charge from moving
> Thereby causing insulation,
> But by means of wires and doping
> With atomic imperfections
> When the bandgap is a small one
> Can be used to make transistors.
> In addition to the basics
> He learned how electric forces
> Like those seen in clinging woolens
> Cause some things to be magnetic
> Up until the lowly phonon,
> Quantum particle of sound wave,
> Storing heat the way that light does
> Mediates a strong attraction
> That can pair up two electrons
> Causing them to move together
> Overcoming all resistance
> And producing other magic
> Such as quantum oscillations.

> At this quite untimely moment
> Of his fragile student history
> When his mind was most suggestive
> Our poor hapless Hiawatha
> Had the terrible misfortune
> To fall in with wicked people
> Who were little more than con men
> And advanced in their profession
> Making theories of such matter
> That were not at all deductive
> But instead used mathematics
> As a way to sow confusion
> So that no one would discover
> That their stuff was pure opinion
> Spiced with politics and chutzpah
> So it looked somewhat like science
> Even though it really wasn't.

> How they did this was ingenious
> For it's not a simple matter
> To produce concrete equations
> That are absolutely hokum
> And escape without detection
> When they represent relations
> Of some quantities one measures
> Written down as abstract symbols
> That could easily be tested.
> What they did was deftly prey on
> Prejudicial ways of thinking
> That their colleagues thought were reasoned
> But were simply misconceptions,
> Generated during training
> They had all received as students,
> That the properties one wanted
> Were completely universal
> So details did not matter.
> But the data did not say this
> And, moreover, had they done so
> There would have been no good reason
> To think any more about it.
> So, while everyone was watching,
> They swapped in some new equations
> That they said would solve the problem
> On account of being much simpler
> But in fact described a system
> Very different from the first one
> And, moreover, was unstable,
> Balanced at competing phases,
> So that nobody could solve it
> Thus betraying the deception.

> Adding to the dazzling brilliance
> Of this coldly thought-out swindle
> They declared it *fundamental*
> So that all the strange creations
> Made by people trying to solve it
> And quite clearly not succeeding
> Proved it was a fount of deepness
> One should struggle to unravel
> Even if it took a lifetime.
> As a nifty added bonus
> Any hint you dropped in public
> That it might have no solution
> Simply meant you weren't a genius,
> Told the world that you were stupid,
> That you were a hopeless failure
> Who should not command a pencil.
> No one wanted to admit this
> So they'd cover up their failure
> And pretend that they had solved it
> Even though they clearly hadn't.
> This succeeded, for the most part,
> But in one respect it didn't,
> For their desperate need to publish
> And thereby maintain their funding
> Caused a massive flood of papers,
> Each quite different from the others,
> To descend upon the journals
> And to overwhelm and clog them.
> This would have been very funny
> Had it not been so pathetic.

> Hiawatha bought the story
> Took the bait, hook, line, and sinker
> And, like many other students
> Who'd been victimized before him,
> Got convinced that his strong math skills,
> Far exceeding those of others,
> Would reveal nature's mysteries
> When he solved the Hubbard model
> And its child the t-J model
> And the lattice Kondo model
> And the quantum spin glass model,
> All of which possessed the feature
> That no human being could solve them.

# Hiawatha Meets the Cuprates

> Nature has a sense of humor,
> As one learns by working with it,
> But it is an opportunist,
> So that life's most bitter lessons
> Often wind up learned the hard way
> When it moves to take advantage
> Of a single bad decision
> And compound it with some mischief
> Custom made for the occasion.

> Just when he'd resolved to strike out
> On his suicidal mission
> There occurred a bold announcement
> In a well-known German journal
> That a tiny lab near Zürich
> Had discovered a material
> With the structure of perovskite
> Made of oxygen and copper
> And some other stuff like strontium
> That when cooled to thirty kelvin
> Lost all traces of resistance.
> This event was simply shocking
> For existing quantum theory
> Said it had to get much colder
> For this special thing to happen,
> As did all the careful surveys
> Of the properties of metals,
> Which were very comprehensive
> And agreed well with the theory.
> Since the chemists were ambitious
> To somehow transcend this limit,
> Which they thought too academic,
> And someday kill all resistance
> Using no refrigeration,
> There ensued a feeding frenzy
> Worthy of a horror movie,
> Like what happens when a trawler
> Dumps its hold of tuna entrails
> Off a reef in north Australia.

> One example of this madness
> Was the *Physics Woodstock* conference
> That took place in mid-Manhattan
> Shortly after the announcement
> Where attendees got together,
> Comandeered a giant ballroom,
> And gave talks not on the program
> In a special all-night session
> Dedicated to the cuprates
> Which was packed to overflowing.
> There was talk of maglev transport,
> New kinds of computer circuit,
> Mighty, compact little motors
> And efficient power cables,
> All of which would soon be coming
> Thanks to this momentous breakthrough.
> But it turns out we don't have them
> For they weren't a big improvement
> Over things we had already
> And were hopelessly expensive.

> Then there were the frantic searches
> To find compounds that were better,
> Which one knew could be accomplished
> If one spent enough time looking,
> Since this stuff had lots of phases
> Subtly different from each other,
> And there had to be a best one.
> There was very rapid progress
> Culminating in a patent
> For a more complex material
> In the same broad class of structure
> Which performed at ninety kelvin,
> So much higher than the theory
> Would allow to ever happen
> Even with extreme assumptions
> That one knew it was in trouble.

> Almost overnight one found that
> Every spectrum known to science
> Had been taken on a cuprate.
> Their alleged profound importance
> Was, of course, a major factor,
> But what mattered most was tactics.
> Without need to tell one's funders,
> Since it could be done so quickly,
> One could telephone a chemist,
> Cut a deal to get some samples,
> Put them in one's apparatus—
> Presto! Out would come a paper
> That would instantly get published
> Even if it was a stinker.
> This produced a pile of data,
> Growing without bound, like cancer,
> That completely overwhelmed you
> By being mostly unimportant,
> Like the growing list of options
> Coming from your cable service.

> Often spectra weren't consistent,
> But, instead of getting angry
> As one would have in the old days,
> One would handle it maturely
> And just chalk it up to errors
> That occur when one is hasty
> Or has had bad luck with samples.
> But this tolerance, it turns out,
> Was a bargain with the devil
> For it later was discovered
> That enormous variation
> Was endemic to the cuprates,
> And that things not reproducing
> Due to complex phase inclusions,
> Foreign atoms in the sample,
> Careless oxygen annealing,
> Surface preparation methods,
> And a thousand other factors
> Was essential to their nature.

> Sadly, by the time this surfaced
> Shameful habits of denying
> That the differences existed
> Had become enshrined in writing,
> And so wedded to the culture,
> That they could not be corrected.
> It was now accepted practice
> In a public presentation
> Of experimental findings
> Not to mention other data
> Even if your own group took them.
> Grounds for this were rarely stated,
> Other than the innuendo
> That one's sorry competition
> Were a hopeless bunch of bozos
> Who did not know how to measure
> And therefore could not be trusted.
> It was likewise viewed as kosher
> To make up a little theory
> Or adopt somebody else's
> That gave all your findings meaning—
> Although not those of your colleagues,
> Which were, sadly, so imperfect
> They were simply inconsistent.
> But one never heard recanting,
> Since it would have meant admission
> That one's judgement had been faulty.

> Thus the cuprates' weird caprices
> Long escaping understanding
> Transformed into pseudotheories
> That, like gods on Mount Olympus,
> Were political creations
> That could not be killed with reason
> And, empowered as immortals,
> Took control of their creators,
> Warred among themselves for power,
> Schemed to have a lot of children,
> And, in general, made a circus
> Of the scientific method.

> Hiawatha, being a student,
> And, quite frankly, rather callow
> Did not have the slightest inkling
> That such nonsense ever happened.
> He believed the claims of science
> To be rather more objective
> Than competing kinds of knowledge
> On account of its precision
> And the fact that you could test it.
> Rather than the yawning snake pit
> Seething with disinformation
> That was really there before him,
> Certain death for young beginners,
> He saw just a chance for glory
> Something of immense importance,
> Judging from the acrimony
> Coursing through the talks and papers,
> And a vast supply of data
> On which one could build a theory
> And thereby become a hero,
> Much the way the dumber brother
> Of the famous brave Odysseus
> That no one has ever heard of,
> Sure he could outfox the sirens,
> Ordered that the men unbind him
> And, of course, succumbing quickly
> Dove right in and bashed his brains out.

# Hiawatha Escapes Reality

> Hiawatha's misconceptions
> Of the nature of the problem
> He was setting out to conquer
> Were not shared by everybody.
> Just as buzzards, with keen noses,
> Circling high above their breakfast
> Wait until it cannot hurt them,
> To swoop down and get to business,
> And ichneuman wasps impregnate
> Larval caterpillar victims
> With some eggs that grow to eat them,
> Thus not let them reach adulthood
> When they might be hard to handle,
> Hiawatha's crafty mentors
> Sensed that science had stopped working
> In the sub-field of the cuprates,
> As it had before in others
> Where their scams had been successful.
> Smelling death was close upon it,
> They resolved the time was ready.

> What ensued was simply awesome,
> Destined to go down in legend.
> They proposed a cuprate theory
> So magnificent in concept,
> So much bolder than the others
> That it blasted them to pieces
> Like some big atomic warhead,
> So outshined them in its glory
> Like a nova in the heavens
> That it blinded any person
> Who would dare to gaze upon it.
> Cuprates did these things, it stated,
> Just because a quirk of nature
> Made them like the *Hubbard model*,
> Which, as had been long established,
> Did some things quite fundamental,
> Not yet known to modern science,
> Which explained the crazy data,
> So to understand the cuprates
> One would have to solve this model.
> How colossal! How stupendous!
> It was absolutely foolproof!
> No one could disprove this theory
> With existing mathematics
> Or experimental data
> For exactly the same reasons
> Nor could they admit they couldn't,
> So they'd spend their whole lives trying,
> Blame themselves for being so stupid,
> And pay homage in each paper
> With the requisite citation!

> They left clues in great abundance
> That they'd made a vast deception
> Far surpassing P. T. Barnum's
> Most creative whims and musings
> Trusting that no one would catch them
> On account of being so guileless,
> Which they knew was part of science,
> Rather like the clever killer,
> Sure he can outsmart Columbo,
> Leaving marks upon the crime scene
> Then in later verbal sparring
> Hints at them in brazen taunting.
> One was that its short description,
> Resonating bonds of valence,
> Was the name that Linus Pauling
> Used for common bonds of benzene,
> Something so profoundly different
> From the physics of the cuprates
> That its use on this occasion
> Seemed to show a lousy word sense.
> But, in fact, it was inspired,
> For the permanent confusion
> Left by its uncertain meaning
> Like the data it reflected,
> Was defense against attackers,
> Made it very hard to target,
> Left its enemies bewildered.
> And the thoughtful usurpation
> Of a well-established brand name
> Had the lovely added feature
> Of dispatching pesky Pauling,
> Who had always been a nuisance,
> Down to Davy Jones's locker
> In the minds of younger people.
>
> There was also the assertion
> Running rampant through the theory
> That the essence of the cuprates
> Was coulombic insulation,
> Which, on close inspection, turned out
> No one could define precisely,
> With a few concrete equations,
> But was nonetheless a concept
> People thought they comprehended,
> Like the fancy secret contents
> Of competing brands of toothpaste
> That, of course, are total fictions
> Made up during lunch by ad guys.
> But the best clue by some margin
> Was the *deus ex machina*
> Known as Gutzwiller Projection,
> Which began life as a method
> For controlling the equations
> But was morphed on this occasion
> To a monsterous distortion
> Of the basic mathematics
> On the grounds it was insightful.
> But, in fact, it came from nowhere,
> And was just a simple dictat
> That an off-the-shelf conductor
> Could not be a quantum magnet
> While one forced it to become one
> Thus creating awful conflict
> When, in fact, there simply was none.

> Hiawatha, being clever,
> Quickly saw that he could do this,
> Saw that such manipulations
> Were, in fact, extremely easy,
> That a high school kid could do them,
> Once he got the key idea
> That one should evade the problem
> Of deducing the behavior
> From the actual equations
> By declaring that some answer
> Was correct because one said so
> And proceeding to defend it
> With a lot of complex symbols
> Simply cooked up to confuse things.
> Thus emboldened to abandon
> His perverse outdated fear of
> Uncontrolled approximations
> Hiawatha bit the bullet
> And jumped into cuprate theory
> With the fury of a madman,
> Doing reckless calculations
> Based on nothing but some gas fumes
> That produced some fine predictions,
> As one was inclined to call them,
> Matching some existing data
> But, of course, not matching others,
> Since they were not all consistent.
> He would then just pick and choose them
> As one would an orange or lemon
> In the local supermarket
> And declare the rest defective.
> Then he wrote up his conclusions
> In a little physics paper
> Loaded up with fearsome symbols
> Proving that he had credentials
> To make all these speculations,
> Sent it in for publication
> And then found an awful problem
> He had not anticipated.
> For the paper to be published
> It must get past refereeing
> Which, in theory, was for stopping
> False results from being reported
> But, in practice, was to censor
> Anyone whose work you hated,
> Somewhat of a sticky wicket
> For someone who's main objective
> Was to publish speculation.
> Hiawatha soon discovered
> Though the process of rejection
> That his papers could not make it
> If they championed new ideas
> Or in any way conflicted
> With the viewpoints of the experts
> Which, of course, were simply made up.

> Thus the mighty Hiawatha
> Found his plans to be a scholar
> Had an unexpected down side
> That would later prove quite fatal
> In that he was forced to pander
> In his writing for the public
> To a set of flakey concepts
> That he'd found extremely useful
> But had not had time to question,
> In exchange for recognition
> Needed for career advancement.
> For a while it did not matter
> But the problem slowly festered
> And one day poor Hiawatha,
> Waking to a huge disaster,
> Found himself up to his eyeballs
> In a soup of black corruption.

# Hiawatha and the Experiments

> Hiawatha's revelation
> Took a while to find its footing
> For, as happens in such cases,
> Many awful misconceptions
> Were embedded in his thinking
> Where they had been put on purpose
> And could only be uncovered,
> If at all, through painful hours
> Scrutinizing tiny details,
> Contemplating reams of data,
> Finding out who's stuff was careful,
> Tracking down suspicious rumors,
> Reading through a mass of papers,
> Slowly tossing out the bad ones,
> Racking up the airline mileage
> Going to humongous meetings,
> Thereby building up a fact base
> Cleansed of all manipulations.
> Over time, as things got clearer,
> Hiawatha grew unhappy
> Trying to reconcile his viewpoint
> With the facts that he had winnowed,
> Always finding that he couldn't.

> Hiawatha studied transport
> Both electrical and thermal
> That, one argued, showed the absence
> Of the Landau fermi surface
> Symptomatic of a metal
> Thereby proving one was dealing
> With a strange new state of matter.
> But he found in every instance
> That a sample made its coldest,
> So one knew what one was doing,
> Either showed disorder problems
> Generated by the chemists
> Or agreed with classic theory.
> Thus, like all those dot-com profits
> That they claimed would make you wealthy,
> But, in fact, were nonexistent,
> Arguments for novel physics
> Built upon the facts of transport
> Did not hold up on inspection.

> Hiawatha studied optics
> By and large his favorite spectrum
> For he knew that light reflection
> Measured dielectric functions
> In a way that used no theory,
> And it showed how loose electrons
> Moved about and caused the bonding.
> But, alas, the data varied
> From one sample to another
> Even after years of efforts
> To ensure that they were stable!
> This left lack of clear consensus
> Even over things that mattered.
> Understanding why this happened
> Was not really rocket science,
> For the Kramers-Krönig process
> Amplified the defect signals
> That were there in great abundance,
> Even though they all denied it,
> And depended on the process
> By which one prepared the sample,
> Something different for each grower
> And a closely-guarded secret.
> Also, things would change with doping,
> Something very hard to measure
> And which often wasn't constant
> As one moved across the sample
> Due to troubles in the furnace
> Which they claimed they'd licked but hadn't.
> Thus the stories of new physics
> Built upon results of optics,
> Like the troubled U. S. census
> Or the the streets of downtown Boston
> After weeks of too much snowing,
> Were polluted by disorder,
> And, moreover, were deceptive
> In that aspects of the spectra
> That were reasonably stable
> Like the strange non-Drude lineshapes
> Happened at such tiny wavelengths
> One could plausibly ascribe them
> To a nearby phase transition
> Rather than the state in question.
> Thus the stories were fantastic,
> And, like those that Richard Nixon
> Told while he was in the White House,
> Or that pop star Michael Jackson
> Claimed occurred in Los Olivos
> For the pleasure of the children,
> In the end would not hold water.

> Hiawatha studied neutrons
> Which he found he liked immensely
> Since they flowed from a reactor
> With big purple signs upon it
> Warning you of radiation
> That would kill you if allowed to,
> Since the neutrons went right through you
> But would sometimes choose to stop there
> And decay like little time bombs,
> Thus inducing stomach cancer.
> But they went through cuprates also,
> And that made them very useful,
> Since a few of them would scatter,
> And detecting those that did so
> Gave you lots of information
> From down deep inside the sample,
> Such as how the atoms ordered,
> How they moved when something hit them
> And if they were little magnets.
> But the bad news was the signal
> Was quite small and hard to measure,
> So one needed a detector
> Bigger than a Dempsey Dumpster
> And a truly mammoth sample,
> Leading to big compromises
> In the sample growing process
> They preferred deemphasizing
> But one knew was wreaking havoc
> On the meaning of the data.
> They would also never tell you
> What the measurement itself was,
> Since the neutron kinematics
> Made it sensitive to factors
> Like the speed spread of the neutrons
> And the tip of the detector
> And the path on which one moved things
> To survey deflection angles
> That were messy and annoying,
> So they'd first massage the data
> Using big computer programs
> To remove these nasty factors
> And report the program output,
> Representing you should trust it
> Just because they were the experts.
> But, of course, there were those upgrades
> And the quiet little tweaking
> That one always did at run time
> That one never heard reported.
> Once he caught these key omissions
> Hiawatha got suspicious,
> And quite quickly found the practice
> Of reporting neutron spectra
> In some secret custom units
> Given names like "counts" to fool you,
> Like those helpful content labels
> Found on packs of sandwich slices
> Listing salt and beef by-products,
> Thus preventing one from telling
> There was very poor agreement.
> All this made a clearer picture
> But it also meant the data
> Like the air-brushed prints in *Playboy*
> Were, in fact, manipulated,
> And that many strange behaviors
> Like the famous funny phonon
> Dogma said was nonexistent
> Got removed as standard practice
> On the grounds they should not be there.
> Thus his plan to use those spectra
> To pin down the magnetism
> Present sometimes in the cuprates
> On account of all the errors
> Ended up a dismal failure.

> Hiawatha studied currents
> Made when cold electrons tunnel
> Right across an insulator
> Where they should have been forbidden,
> Something very close to magic
> Rather like the twinkly transport
> People undergo on Star Trek,
> And it's also quite revealing
> Of important quantum pairing
> That goes on inside the cuprate.
> In the old days one would simply
> Oxidize a thin-film sample,
> Coat the oxide with another,
> Solder on two tiny contacts,
> Dunk the whole thing into vapors
> Made so cold that they were liquid,
> Then just measure plain resistance
> Of the two protruding wires,
> Which would vary with the voltage
> Thus producing useful data.
> Hiawtha read these papers
> With a mounting sense of horror,
> For the wild disagreement
> Even in the basic features
> From one sample to another
> Was so large it left one breathless.
> And, of course, the accusations
> That the other guys were morons
> Who just could not make good junctions
> Rose to unmatched heights of grandeur
> Even though the real villain,
> Obvious from spectral sharpness,
> Was the sample variation.
> Hiawatha's indignation
> Escalated when he found that
> Over time this fact got buried
> Since each group soon found a method
> Of preparing stable samples
> Different from that used by others
> And producing different spectra
> That they marketed as products,
> Thus evading any need to
> Answer penetrating questions.
> An important fact, however,
> That emerged from all these studies,
> Was that steady lossless currents
> Could indeed be made to flow from
> Films of lead into the cuprates
> If one made a pitted surface,
> Proving that the state of matter
> Operating in the cuprates
> Was not new and was not different.

> Hiawatha studied spectra
> Made when light shined on a sample
> Causes it to lose electrons
> Which fly out in all directions
> And one can detect by counting,
> Thus obtaining information
> Of their status in the sample
> Just before the light removed them.
> Hiawatha saw at once that
> Peaks for plain undressed electrons
> That were not supposed to be there
> In this great new state of matter
> Always were and had a sharpness
> At the resolution limit
> Of the latest new detector
> For the special ones at threshold,
> Where one knew what one was doing.
> In addition they were beaming
> In a lovely fourfold pattern
> With the symmetry of d-wave,
> Something that had been suggested
> They might do if they were simple,
> Just like those in other metals.
> Thus the arguments for strangeness
> Based on counting these electrons
> Lost their force as things got better,
> And in time were proved a failure.

> Hiawatha studied muons,
> Which he thought were even neater
> Than the more prosaic neutrons,
> Since they came from atom smashers
> That could also quickly kill you
> If you chose to be so careless,
> But they'd stop inside much better
> And once there, decay to gammas
> That were easily detected
> Since they'd even go through concrete,
> And, moreover, they'd be beaming
> In the muon's spin direction
> Just before it went to heaven.
> Thus, implanted in a cuprate
> They'd arrest at some location
> Known to no one but their Maker
> And precess like little searchlights,
> If there was some magnetism,
> Thus allowing you to see it
> Way deep down inside the sample.
> Thus with knowledge of their trapping
> And a batch of big detectors
> One could then back out the distance
> Of magnetic penetration.
> Hiawatha found this distance
> Shortened with increasing doping
> Just as theory said should happen,
> If one forced the hubbard model
> Not to be a quantum magnet
> By just saying that it wasn't,
> Which might well have been important
> Had it not been for the problem
> That this depth would not continue
> To decline with increased doping
> But instead would turn and lengthen.
> This effect was quite perplexing,
> Since no theory of the cuprates
> Even twisted hubbard models,
> Could account for such behavior,
> For it violated sum rules,
> Hence one just did not discuss it.
> But the meaning was transparent
> If one faced the facts with courage,
> For the samples were degrading
> In extremes of overdoping
> In some ways that weren't predicted
> And, moreover, weren't detected
> By techniques except for this one.
> This, in turn, implied these problems
> Might occur at other dopings
> And likewise escape detection
> Or, what's worse, be used to argue
> That new physics was occurring
> When, in fact, it was just garbage.
> Thus the trail blazed by muons
> Led out in the woods to nowhere.

> Hiawatha studied spin flips
> That the nuclei of atoms
> Undergo in great big magnets
> Near a radio transmitter
> Causing them to be antennas,
> Which absorb with complex lineshapes
> One can read if one's a genius
> But not, sadly, if one isn't,
> Since they, by and large, consist of
> Just a simple blobby bell curve
> With a width and displaced center,
> To which one must give some meaning—
> Not a simple undertaking.
> Thus the all-important Knight shift
> And spin-lattice relaxation,
> Noms de plume for width and center,
> Vastly different for the copper
> And the oxygen of cuprates,
> Were the source of endless theories,
> Often very thought-provoking,
> Stunning in sophistication,
> But, like all those glossy pamphlets
> Found in waiting rooms of dentists
> Urging you to practice flossing,
> Soon began to make you tired,
> Since the data mainly showed you
> That the stuff was not a metal
> In the sense of gold or iron
> Which, in fact, one knew already
> And was not a revelation.

> Hiawatha studied structure
> Of the surfaces of cuprates
> Freshly cleaved inside a vacuum
> So that air would not get on them
> And then probed with tiny needles
> One could move with great precision,
> By adjusting some piezos
> On which everything was standing.
> What he found was quite disturbing,
> For while atoms at the surface
> All had unperturbed positions,
> Showing that the cleave succeeded,
> There were also complex patterns
> On the scale of twenty atoms
> That appeared to be diffraction.
> This behavior might have come from
> Atoms underneath the surface
> That were missing or defective
> Or some novel magnetism
> Of a kind unknown to science,
> But the thing that so upset him
> Was that quantum interference
> Of the kind that he was seeing
> Could not happen if the lifetimes
> Were as short as he had thought them,
> And which had been used to argue
> For a brand new state of matter.
> Thus he soberly concluded
> That this matter wasn't different
> And the whole confounded story
> Was a misinterpretation
> Of a plain materials problem.

> Thus the Mighty Hiawatha
> Through the patient application
> Of the practices of science
> Tested over generations
> Slowly sloughed off misconceptions
> And, in face of mounting failure,
> Sadly came to the conclusion
> He'd been taken to the cleaners.

# Hiawatha Befriends the Robots

> Given all the clever swindles
> Lurking there to take our money,
> That, of course, are part of living,
> Like a virus for pneumonia
> Or a hungry venus fly trap,
> We must all be very thankful
> That the celebrated Law of Murphy
> Strikes at random without warning
> Causing even brilliant concepts,
> That appear completely foolproof
> Like distributing tobacco
> Or the business plan of Enron,
> To sometimes become derailed
> Due to something unexpected
> One was sure could never happen,
> Like a lawsuit from consumers,
> That requires intervention
> Of the most creative nature
> To prevent strategic meltdown.

> As it turns out, the idea
> That the conflict in the models
> One was using for the cuprates
> Due to nearby phase transitions
> Would both hamper their solution
> And engender rampant fibbing,
> Thus enshrining mass confusion
> One would then call proof of meaning
> With no need to fear exposure
> Had the unexpected weakness
> That someone might *solve* the model
> Using tons and tons of money
> And some capable computers
> To a crude degree sufficient
> To unmask the real problem
> Thus revealing the deception.

> Sure enough, that's just what happened.
> When the cuprates were discovered
> And the whole endeavor started
> One had not the slightest worry
> That these guys would ever solve it,
> Since the accuracy needed
> Was not clear in the beginning,
> So they uniformly low-balled
> With the too-familiar outcome
> That results were inconsistent.
> So they quarrelled over method
> And who had convergence problems
> And whose code was most clairvoyant
> Even though a child could see that
> They were different apparati,
> So the test that they were working
> Was agreement with each other.
> But, unlike the other issues
> That had come and gone before it,
> Cuprates lingered on as timely
> Long enough to cause a shake-out,
> For the money kept increasing
> Even as machines got cheaper
> And their power kept on growing—
> Due, of course, to needs of gaming,
> Rather than the ones of Lanczos
> Or the quantum monte carlo
> That one used for basic physics.
> So the robots kept on plugging
> As their owners upped the ante
> Very slowly, as did Wagner
> When composing *Ring* and *Tristan*
> And their stuff began converging!
> There, of course, was no agreement
> Over matters of the phases
> Such as whether it conducted
> When one cooled it down to zero,
> Since a crystal of electrons
> Was one state in competition.
> But at length scales one could access
> There was clearly dissipation
> Of a most peculiar nature
> In the dielectric function
> And the quantum magnetism,
> Just exactly as predicted
> By an ancient bunch of papers
> Over quantum phase transitions,
> Which these guys had never studied
> Since it was too esoteric
> And had not been seen in nature
> And was hated by their funders.
> But the thing that really clinched it
> Was the endless disagreement,
> That got worse as things proceeded
> And was very clearly cronic,
> Over type and shape of edges
> That would best produce convergence,
> Since one found that subtle changes
> In the way one built the model
> Would turn on and off the striping
> And therefore the insulation,
> So that whether it was present
> In the limit of large sample
> Simply could not be determined
> With the codes that they had written.

> This, of course, was a disaster
> For the plan to keep things murky
> And required drastic action
> To somehow repair the damage
> All this progress had created,
> And prevent these guys from seeing
> What was right beneath their noses.

> And one was not disappointed.
> Once again a flash of brilliance
> Like a great big city-buster
> Brighter than the sun at midday,
> Blazed across the dome of heaven
> Toward its final destination
> In the Guinness Book of Records.
> They declared the problem *over*!
> The computer guys had solved it!
> For their codes had proved the cuprates
> Were indeed the Hubbard model,
> And that's why the stuff conducted.
> Thus there was no urgent reason
> To pursue the matter further!
> One could zero out their budgets
> With no loss to human knowledge
> And, in fact, perhaps improve it
> Since this money was incentive
> To continue calculations
> That were clearly unimportant
> And report them in the journals
> Thus just adding to the clutter.

> Hiawatha, now much wiser
> Through his labors as a scholar
> And, quite frankly, some maturing
> Watched these things unfold before him,
> As he had on past occasions,
> But this time with eyes wide open
> And was filled with understanding.
> It was not a happy moment,
> For it meant that his own judgement
> As to what was good and worthy
> Had been faulty from the outset,
> Something for which he must answer.
> But instead of indignation
> And a passion to get even
> That he might have felt when younger
> Hiawatha, deep in thinking,
> Found himself consumed with sadness.
> He was not the only victim,
> For the guys who manned those robots,
> And were heroes of the cuprates—
> For through focussed dedication
> They had stumbled on the answer
> That the models were unstable
> And did *not* describe the cuprates,
> Since a modest perturbation
> Would profoundly change their nature—
> Were about to have their triumph
> Snatched from them by clever scoundrels
> Who, pretending to befriend them,
> Would then redefine their output
> To mean something that it didn't,
> Thus protecting their investment,
> But, of course, destroying others.

# Hiawatha's Lamentation

> Hiawatha's knowing sadness,
> Like the darkening at twilight
> Or a gathering storm in winter,
> Slowly gained in strength and deepened
> As he spent time in reflection,
> Working through the implications
> Of the things that he had witnessed
> For the cause of noble science
> That thus far had so beguiled him.
> It would simply not be manly
> To pretend he wasn't guilty
> Of ignoring frequent warnings
> That the needed path to nature
> Was obscured or nonexistent.
> It was clear that he'd been foolish
> To have bought this awful fiction
> And that blame must fall quite squarely
> On himself and not on others.
> But this candid *mea culpa*,
> Made in silence where it mattered,
> While it comforted his conscience,
> Did not quite assuage the wounding,
> For it begged the nagging question
> Of how they could have succeeded
> In hoodwinking all the people
> For so long without some doubting.
> It was simply not an option
> To presume these guys were stupid,
> Since the instruments they dealt with,
> Often built by hand from nothing,
> Needed great sophistication
> To deploy and mine for data.
> There was clearly something larger
> And extremely fundamental
> Working in the group dynamic
> That involved access to funding
> And the policy of journals
> And the need to service markets
> And the mythos of the subject
> One must use to make a living
> That these crooks had first deciphered,
> Then reduced with understanding,
> Then usurped to do their bidding.

> Hiawatha, turning inward,
> Thought for weeks about this problem
> During which he was obnoxious
> Due to his preoccupation.
> But at last he got an answer
> That made sense and was quite simple,
> Thus withstanding Occam's razor,
> So he thought that he believed it.
> When he'd set out on his mission
> He had understood the challenge
> Of the mastery of nature
> But not basic economics
> And the fact that art and science
> Both require sacrifices
> Of a clear financial nature
> That one sometimes just can't handle
> Nor, in fairness, should one do so
> Since a good guy pays the mortgage
> And supports the kids in college
> And the other things a body
> Has to do to keep the lights on.
> But, in fact, the compromises
> That one makes as part of living
> Such as saying what one has to
> For maintaining healthy cash flow
> Often toss big monkey wrenches
> In the fine machine of science
> And can stop it altogether
> In conflicted situations.
> Then the body, badly weakened,
> Barely able to keep breathing,
> Gets exploited by diseases,
> Such as villains lacking scruples
> Who descend on it like termites
> To a house that's been neglected,
> Wreaking terrible destruction
> On the lives of those affected.

> The conclusion of this story
> Is well known from all the textbooks.
> Hiawatha never wavered
> In his deep respect for physics,
> But he came by this adventure
> To the deeper understanding
> That to get things done that mattered
> Often was a social question,
> Not just logical abstraction,
> And, as well, a part of nature,
> Just the thing he thought he'd hated
> And had thrilled at desecrating
> As a tender freshman student
> In the little private college
> By the shores of Gitchee-Gumee.
> It was true that all the creatures
> Living in those swamps and woodlands
> Generated lots of pooping,
> But then so did real people,
> And the people poop was stronger,
> So that one could not ignore it.
> But one really would not want to,
> For the lesson of the cuprates
> Was that lack of understanding
> Of these basic group dynamics,
> Was a recipe for failure
> Since they were the central issue
> For most things that were essential.

> Thus the mighty Hiawatha
> Turned his mind to other problems
> Such as how to use resources
> That were his by luck and birthright
> Through the power of his father
> Which he'd been inclined to squander,
> But now realized he shouldn't.
> Thus he studied like a madman
> To acquire the skills of statecraft,
> Such as how to plan a project,
> How to give effective orders,
> How to make sure they were followed,
> How to get things done with meetings,
> And to leave the money grubbing
> Up to folks his father hired
> Such as all those gifted spin docs
> Who created key revisions
> Necessary for his image
> To be something people honored.
> Thus the pain of too much sliding
> On the ice in dead of winter
> In an inexpensive loincloth
> And his other misadventures
> Got removed, as did the cuprates,
> From his long official story.
> But the memory persisted
> And it helped to make him wiser
> For, of course, as he got older
> He had many bad encounters
> Not so different from the cuprates.

> But whenever he was troubled
> With a problem that would vex him
> He would cheer himself by thinking
> Of the special room in Hades
> Into which these happy people
> On account of their transgressions
> Would be ushered when they bagged it
> And be stuck in there forever,
> Forced to listen to each other
> Giving lectures on the cuprates.
> It would always leave him smiling.


abstract: We present a number of notable results from the VLT-FLAMES Tarantula Survey (VFTS), an ESO Large Program during which we obtained multi-epoch medium-resolution optical spectroscopy of a very large sample of over 800 massive stars in the 30 Doradus region of the Large Magellanic Cloud (LMC). This unprecedented data-set has enabled us to address some key questions regarding atmospheres and winds, as well as the evolution of (very) massive stars. Here we focus on O-type runaways, the width of the main sequence, and the mass-loss rates for (very) massive stars. We also provide indications for the presence of a *top-heavy* initial mass function (IMF) in 30 Dor.
author: Jorick S. Vink$^1$, C.J. Evans$^2$, J. Bestenlehner$^{1,3}$, C. McEvoy$^4$, O. Ramírez-Agudelo$^{2}$, H. Sana$^5$, F. Schneider$^{6}$; VFTS
title: The VLT-FLAMES Tarantula Survey

# Introduction

Massive star evolution is important for many fields of Astrophysics including supernovae (SNe; Levesque, these proceedings). Yet, it remains largely unconstrained (Langer 2012; Meynet these proceedings). Progress can be made using high-quality observations from nearby sources, as well as from large data-sets such as VFTS (Evans et al. 2011) discussed here. In parallel, VFTS data are analysed using state-of-the-art model atmospheres such as CMFGEN (Hiller & Miller 1998) and FASTWIND (Puls et al. 2005), as well as automatic fitting tools (Sabı́n-Sanjulián et al. 2014; Bestenlehner et al. 2014; Ramírez-Agudelo et al. 2017).

In addition to this observational progress, our VFTS collaboration strives to make theoretical progress on stellar winds and evolution, and we are in the unique position to confront our new models against VFTS data. In the following, we highlight a number of recent results that we argue make a real difference to our knowledge of massive stars.

## Motivation for the Tarantula region

The Tarantula region (30 Doradus) is the largest active star-forming region in our Local Universe for which individual spectra of the massive-star population can be obtained. Because it is the largest region, it provides a unique opportunity to study the most massive stars, including very massive stars (VMS) with masses up to 200-300 $M_{\odot}$ (Crowther et al. 2010; Bestenlehner et al. 2014; Martins 2015; Vink et al. 2015). This allows us to properly investigate whether the upper-IMF may be top-heavy (Schneider et al. 2017). Answering this question is important as these VMS that are thought to dominate the ionizing radiation and wind feedback from massive stars (Doran et al. 2013).

Another reason to study 30 Doradus is that testing massive star evolution *requires* large data-sets. For instance, the issue of the location of the terminal-age main sequence (TAMS) can only be addressed when the sample-size is sufficiently large to populate both the main-sequence with O-type stars (Sabı́n-Sanjulián et al. 2017; Ramírez-Agudelo et al. 2017) and B supergiants (McEvoy et al. 2015).

# Results on binarity, rotation rates, and runaways

The aims of VFTS were to determine the stellar parameters, such as $T_{\rm eff}$, $\log g$ & $\log L$ to place our objects on the HR-diagram; the mass and $\dot{M}$ to determine the evolution & fate of massive stars; and the helium (He) and nitrogen (N) abundances to test (rotational) mixing processes (Grin et al. 2016; Rivero-Gonzalez et al. 2012). All these parameters require sophisticated atmosphere modeling, but VFTS also offered some model-independent parameters including the rotational velocities $v$ $\sin$ $i$ and radial velocities (RVs) thanks to the multi-epoch nature of the survey. The latter allowed us to obtain information on the $\sim$ 50% frequency in 30 Dor (Sana et al. 2013) and the opportunity to study the dominant mechanism for runaways (Fig. ; Sana et al. in prep.).

Figure  might allow us to disentangle the dynamical runaway scenario (Gies & Bolton 1986) from the binary-SN kick scenario (Stone 1991), as the first scenario might produce relatively fast runaways, whilst one would expect the binary SN kick scenario to produce rapid rotators. Obviously, definitive conclusions can only be obtained when more sophisticated models become available.

# The width of the main-sequence and constraints on core overshooting

Figure  shows a zoomed-in version of the Hertzsprung-Russell diagram for both single and binary B supergiants from McEvoy et al. (2015). The position of the dark line indicates the position of the TAMS, with its location is determined by the value of the core overshooting parameter ($\alpha_{\rm ov}$) which is basically a "free" parameter (e.g. Vink et al. 2010; Brott et al. 2011) until astro-seismology on a large number of OB supergiants becomes available. The Brott et al. models employ a value of $\alpha_{\rm ov} = 0.335$, whilst the Geneva models (Georgy these proceedings) employ a smaller value. The VFTS results shown in Fig.  appear to suggest a *larger* value of $\alpha_{\rm ov}$ than 0.335.

Larger $\alpha_{\rm ov}$ makes bi-stability braking (BSB; Vink et al. 2010; Keszthelyi et al. 2017) feasible, which we test by showing $v$ $\sin$ $i$ of both VFTS and previous FLAMES-I results (Hunter et al. 2008) versus $T_{\rm eff}$ in Fig. . Note the presence of another "Region-of-Avoidance"[^1], where rapidly-rotating "cool" (cooler than the bi-stability location of 20 000 K; Petrov et al. 2016) B supergiants are simply not observed. The reason for this avoidance below 20 000 K could either involve BSB, or it might be that the slowly rotating cool B supergiants are He-burning objects (due to post red-supergiant evolution or binarity).

# The mass-loss rates

The mass-loss rates for O-type dwarfs were discussed by Sabı́n-Sanjulián et al. (2014; 2017), whilst those for the O giants and supergiants are plotted in the form of the wind-momentum-luminosity relation (WLR; Kudritzki & Puls 2000; Puls et al. 2008) in Fig. . Interestingly, the empirical WLR lies above the theoretical WLR (of Vink et al. 2001). Usually a discrepancy between theoretical and empirical values would be interpreted such that the theoretical rates would be too low, but here it is different, as it is widely accepted that empirical modeling is more dependent on wind clumping and porosity than theory (see Muijres et al. 2011 for theoretical expectations).

Indeed, it is more likely that the empirical WLR is too high, as a result of wind clumping, which has not been included in the analysis. This would imply that the empirical WLR would need to be lowered by a factor $\sqrt{D}$, where $D$ is the clumping factor, which is as yet uncertain. However, given the model-independent (from clumping & porosity) transition mass-loss rate (Vink & Gräfener 2012; next Sect.) a value of $D \simeq 10$ (with a mass-loss rate and WLR reduction of $\sim$`<!-- -->`{=html}3) would bring the empirical WLR and theory in reasonable agreement. None of this means that the theoretical rates for lower mass-and-luminosity O stars need necessarily to be correct. Therefore, spectral analysis of large data-sets of O-stars including clumping & porosity (Surlan et al. 2013; Sundqvist et al. 2014) are needed to provide definitive answers.

# Very Massive Stars

The most massive stars in VFTS were analysed by Bestenlehner et al. (2014), plotted in the HRD of Fig. . Over-plotted are VMS evolutionary tracks and the location of the ZAMS. The HRD shows the presence of 12 VMS (with $M$ $>$ 100 $M_{\odot}$; Vink et al. 2015), which enables us to derive the upper-IMF of 30 Dor for the first time. Figure compares the preferred value for the mass function to that of Salpeter. It is found that the slope is different to that of Salpeter (at $\sim$`<!-- -->`{=html}85% confidence), and also that a Salpeter IMF cannot reproduce the larger number of massive stars above 30 $M_{\odot}$ at $>$`<!-- -->`{=html}99% confidence (Schneider et al. 2017). As this result is obtained using the largest spectroscopic data-set ever obtained, and analysed with the most sophisticated analysis tools, we consider this the most robust test to date. A top-heavy IMF would have major implications for the interpretation of spectral modelling of high-redshift galaxies, as well as the ionizing radiation and kinetic wind energy input into galaxies. Answers will strongly depend on the mass-loss rates of these VMS, as discussed next.

Figure  shows VFTS results of the mass-loss rates of the most massive stars in 30 Dor (Bestenlehner et al. 2014). Whilst at relatively low values of the Eddington value $\Gamma$, the slope of the empirical data is consistent with that for O stars, those above the crossover point are not. Here the mass-loss rate kinks upwards, with a steeper slope. The winds have become optically thick, and show WR-like spectra. Also, above this critical $\Gamma$ point, the wind efficiency crosses unity, enabling a calibration of the absolute mass-loss rates for the first time (Vink & Gräfener 2012). Moreover, Bestenlehner et al. (2014) found profound changes in the surface He abundances exactly coinciding with the luminosity threshold where mass loss is enhanced. This suggests that Of/WN and WNh stars are objects whose H-rich layers have been stripped by enhanced mass-loss during their main-sequence life. Note that this mass-loss enhancement for VMS has not been included in most stellar evolution calculations, and this implies there will be many exciting surprises for extra-galactic applications of massive stars in the near future!

# Final Words

The VFTS has conclusively shown that binaries are common in 30 Dor. With a corrected close-binary fraction of $\sim$`<!-- -->`{=html}50% (Sana et al. 2013), we do not yet know whether this hints at a lower binary frequency at low metallicity, or it it is still consistent with the larger Galactic frequency of $\sim$`<!-- -->`{=html}70% when evolutionary considerations are taken into account. Either way, we now know we require both single & binary evolutionary models to make progress. Another interesting finding is that there is a high-velocity tail present in single O-type supergiants (Ramírez-Agudelo et al. 2013), which is not present in the spectroscopic binaries (Ramírez-Agudelo et al. 2015). This suggests that binary interactions need to be accounted for to understand the underlying rotational distribution.

The VFTS results also indicate that the main-sequence needs widening. This hints at a larger value for the core overshooting parameter than usually adopted. Finally, VMS up to at least 200$M_{\odot}$ are common in 30 Dor, but VMS mass-loss rates have been *under*estimated.

[^1]: The perceived lack of rapid rotators on the hot side of the diagram is not real, there are many rapidly rotating O-type stars. These O-stars are just not included here.


abstract: This paper explains the math behind a generative adversarial network (GAN)  model and why it is hard to be trained. Wasserstein GAN is intended to improve GANs' training by adopting a smooth metric for measuring the distance between two probability distributions.
author: Lilian Weng
OpenAI
`email@example.com`
title: From GAN to WGAN

# Introduction

Generative adversarial network (GAN)  has shown great results in many generative tasks to replicate the real-world rich content such as images, human language, and music. It is inspired by game theory: two models, a generator and a critic, are competing with each other while making each other stronger at the same time. However, it is rather challenging to train a GAN model, as people are facing issues like training instability or failure to converge.

Here I would like to explain the math behind the generative adversarial network framework, why it is hard to be trained, and finally introduce a modified version of GAN intended to solve the training difficulties.

# Kullback–Leibler and Jensen–Shannon Divergence

Before we start examining GANs closely, let us first review two metrics for quantifying the similarity between two probability distributions.

\(1\) **KL (Kullback–Leibler) Divergence** measures how one probability distribution $p$ diverges from a second expected probability distribution $q$.

$$D_{KL}(p \| q) = \int_x p(x) \log \frac{p(x)}{q(x)} dx$$

$D_{KL}$ achieves the minimum zero when $p(x) == q(x)$ everywhere.

It is noticeable according to the formula that KL divergence is asymmetric. In cases where $p(x)$ is close to zero, but $q(x)$ is significantly non-zero, the $q$'s effect is disregarded. It could cause buggy results when we just want to measure the similarity between two equally important distributions.

\(2\) **Jensen–Shannon Divergence** is another measure of similarity between two probability distributions, bounded by $[0, 1]$. JS divergence is symmetric and more smooth. Check this [post](https://www.quora.com/Why-isnt-the-Jensen-Shannon-divergence-used-more-often-than-the-Kullback-Leibler-since-JS-is-symmetric-thus-possibly-a-better-indicator-of-distance) if you are interested in reading more about the comparison between KL divergence and JS divergence.

$$D_{JS}(p \| q) = \frac{1}{2} D_{KL}(p \| \frac{p + q}{2}) + \frac{1}{2} D_{KL}(q \| \frac{p + q}{2})$$

Some  believe that one reason behind GANs' big success is switching the loss function from asymmetric KL divergence in traditional maximum-likelihood approach to symmetric JS divergence. We will discuss more on this point in the next section.

# Generative Adversarial Network

GAN consists of two models:

-   A discriminator $D$ estimates the probability of a given sample coming from the real dataset. It works as a critic and is optimized to tell the fake samples from the real ones.

-   A generator $G$ outputs synthetic samples given a noise variable input $z$ ($z$ brings in potential output diversity). It is trained to capture the real data distribution so that its generative samples can be as real as possible, or in other words, can trick the discriminator to offer a high probability.

These two models compete against each other during the training process: the generator $G$ is trying hard to trick the discriminator, while the critic model $D$ is trying hard not to be cheated. This interesting zero-sum game between two models motivates both to improve their functionalities.

Given,

| **Symbol** | **Meaning** | **Notes** |
|:--:|:---|:---|
| $p_{z}$ | Data distribution over noise input $z$ | Usually, just uniform. |
| $p_{g}$ | The generator's distribution over data $x$ |  |
| $p_{r}$ | Data distribution over real sample $x$ |  |

On one hand, we want to make sure the discriminator $D$'s decisions over real data are accurate by maximizing $\mathbb{E}_{x \sim p_{r}(x)} [\log D(x)]$. Meanwhile, given a fake sample $G(z), z \sim p_z(z)$, the discriminator is expected to output a probability, $D(G(z))$, close to zero by maximizing $\mathbb{E}_{z \sim p_{z}(z)} [\log (1 - D(G(z)))]$.

On the other hand, the generator is trained to increase the chances of $D$ producing a high probability for a fake example, thus to minimize $\mathbb{E}_{z \sim p_{z}(z)} [\log (1 - D(G(z)))]$.

When combining both aspects together, $D$ and $G$ are playing a *minimax game* in which we should optimize the following loss function:

$$\begin{aligned}
\min_G \max_D L(D, G)
& = \mathbb{E}_{x \sim p_{r}(x)} [\log D(x)] + \mathbb{E}_{z \sim p_z(z)} [\log(1 - D(G(z)))] \\
& = \mathbb{E}_{x \sim p_{r}(x)} [\log D(x)] + \mathbb{E}_{x \sim p_g(x)} [\log(1 - D(x)]
\end{aligned}$$

where $\mathbb{E}_{x \sim p_{r}(x)} [\log D(x)]$ has no impact on $G$ during gradient descent updates.

## What is the Optimal Value for D?

Now we have a well-defined loss function. Let's first examine what is the best value for $D$.

$$L(G, D) = \int_x \bigg( p_{r}(x) \log(D(x)) + p_g (x) \log(1 - D(x)) \bigg) dx$$

Since we are interested in what is the best value of $D(x)$ to maximize $L(G, D)$, let us label

$$\tilde{x} = D(x),
A=p_{r}(x),
B=p_g(x)$$

And then what is inside the integral (we can safely ignore the integral because $x$ is sampled over all the possible values) is:

$$\begin{aligned}
f(\tilde{x})
& = A log\tilde{x} + B log(1-\tilde{x}) \\
\frac{d f(\tilde{x})}{d \tilde{x}}
& = A \frac{1}{ln10} \frac{1}{\tilde{x}} - B \frac{1}{ln10} \frac{1}{1 - \tilde{x}} \\
& = \frac{1}{ln10} (\frac{A}{\tilde{x}} - \frac{B}{1-\tilde{x}}) \\
& = \frac{1}{ln10} \frac{A - (A + B)\tilde{x}}{\tilde{x} (1 - \tilde{x})} \\
\end{aligned}$$

Thus, set $\frac{d f(\tilde{x})}{d \tilde{x}} = 0$, we get the best value of the discriminator: $D^*(x) = \tilde{x}^* = \frac{A}{A + B} = \frac{p_{r}(x)}{p_{r}(x) + p_g(x)} \in [0, 1]$. Once the generator is trained to its optimal, $p_g$ gets very close to $p_{r}$. When $p_g = p_{r}$, $D^*(x)$ becomes $1/2$.

## What is the Global Optimal?

When both $G$ and $D$ are at their optimal values, we have $p_g = p_{r}$ and $D^*(x) = 1/2$ and the loss function becomes:

$$\begin{aligned}
L(G, D^*)
&= \int_x \bigg( p_{r}(x) \log(D^*(x)) + p_g (x) \log(1 - D^*(x)) \bigg) dx \\
&= \log \frac{1}{2} \int_x p_{r}(x) dx + \log \frac{1}{2} \int_x p_g(x) dx \\
&= -2\log2
\end{aligned}$$

## What does the Loss Function Represent?

According to the formula listed in Sec. , JS divergence between $p_{r}$ and $p_g$ can be computed as:

$$\begin{aligned}
D_{JS}(p_{r} \| p_g)
=& \frac{1}{2} D_{KL}(p_{r} || \frac{p_{r} + p_g}{2}) + \frac{1}{2} D_{KL}(p_{g} || \frac{p_{r} + p_g}{2}) \\
=& \frac{1}{2} \bigg( \log2 + \int_x p_{r}(x) \log \frac{p_{r}(x)}{p_{r} + p_g(x)} dx \bigg) + \\& \frac{1}{2} \bigg( \log2 + \int_x p_g(x) \log \frac{p_g(x)}{p_{r} + p_g(x)} dx \bigg) \\
=& \frac{1}{2} \bigg( \log4 + L(G, D^*) \bigg)
\end{aligned}$$

Thus,

$$L(G, D^*) = 2D_{JS}(p_{r} \| p_g) - 2\log2$$

Essentially the loss function of GAN quantifies the similarity between the generative data distribution $p_g$ and the real sample distribution $p_{r}$ by JS divergence when the discriminator is optimal. The best $G^*$ that replicates the real data distribution leads to the minimum $L(G^*, D^*) = -2\log2$ which is aligned with equations above.

**Other Variations of GAN**: There are many variations of GANs in different contexts or designed for different tasks. For example, for semi-supervised learning, one idea is to update the discriminator to output real class labels, $1, \dots, K-1$, as well as one fake class label $K$. The generator model aims to trick the discriminator to output a classification label smaller than $K$.

# Problems in GANs

Although GAN has shown great success in the realistic image generation, the training is not easy; The process is known to be slow and unstable.

## Hard to Achieve Nash Equilibrium

discussed the problem with GAN's gradient-descent-based training procedure. Two models are trained simultaneously to find a Nash equilibrium to a two-player non-cooperative game. However, each model updates its cost independently with no respect to another player in the game. Updating the gradient of both models concurrently cannot guarantee a convergence.

Let's check out a simple example to better understand why it is difficult to find a Nash equilibrium in an non-cooperative game. Suppose one player takes control of $x$ to minimize $f_1(x) = xy$, while at the same time the other player constantly updates $y$ to minimize $f_2(y) = -xy$.

Because $\frac{\partial f_1}{\partial x} = y$ and $\frac{\partial f_2}{\partial y} = -x$, we update $x$ with $x-\eta \cdot y$ and $y$ with $y+ \eta \cdot x$ simultaneously in one iteration, where $\eta$ is the learning rate. Once $x$ and $y$ have different signs, every following gradient update causes huge oscillation and the instability gets worse in time, as shown in Fig. 3.

## Low Dimensional Supports

| **Term** | **Explanation** |
|:--:|:---|
| Manifold | A topological space that locally resembles Euclidean space near each point. Precisely, when this Euclidean space is of dimension $n$, the manifold is referred as $n$-manifold. |
| Support | A real-valued function $f$ is the subset of the domain containing those elements which are not mapped to zero. |

discussed the problem of the supports of $p_r$ and $p_g$ lying on low dimensional manifolds and how it contributes to the instability of GAN training thoroughly.

The dimensions of many real-world datasets, as represented by $p_r$, only appear to be *artificially high*. They have been found to concentrate in a lower dimensional manifold. This is actually the fundamental assumption for *Manifold Learning*. Thinking of the real world images, once the theme or the contained object is fixed, the images have a lot of restrictions to follow, i.e., a dog should have two ears and a tail, and a skyscraper should have a straight and tall body, etc. These restrictions keep images away from the possibility of having a high-dimensional free form.

$p_g$ lies in a low dimensional manifolds, too. Whenever the generator is asked to a much larger image like 64x64 given a small dimension, such as 100, noise variable input $z$, the distribution of colors over these 4096 pixels has been defined by the small 100-dimension random number vector and can hardly fill up the whole high dimensional space.

Because both $p_g$ and $p_r$ rest in low dimensional manifolds, they are almost certainly gonna be disjoint (See Fig. ). When they have disjoint supports, we are always capable of finding a perfect discriminator that separates real and fake samples 100% correctly.

## Vanishing Gradient

When the discriminator is perfect, we are guaranteed with $D(x) = 1, \forall x \in p_r$ and $D(x) = 0, \forall x \in p_g$. Therefore the loss function $L$ falls to zero and we end up with no gradient to update the loss during learning iterations. Fig. 5 demonstrates an experiment when the discriminator gets better, the gradient vanishes fast.

As a result, training a GAN faces an dilemma:

-   If the discriminator behaves badly, the generator does not have accurate feedback and the loss function cannot represent the reality.

-   If the discriminator does a great job, the gradient of the loss function drops down to close to zero and the learning becomes super slow or even jammed.

This dilemma clearly is capable to make the GAN training very tough.

## Mode Collapse

During the training, the generator may collapse to a setting where it always produces same outputs. This is a common failure case for GANs, commonly referred to as *Mode Collapse*. Even though the generator might be able to trick the corresponding discriminator, it fails to learn to represent the complex real-world data distribution and gets stuck in a small space with extremely low variety.

## Lack of a Proper Evaluation Metric

Generative adversarial networks are not born with a good objection function that can inform us the training progress. Without a good evaluation metric, it is like working in the dark. No good sign to tell when to stop; No good indicator to compare the performance of multiple models.

# Improved GAN Training

The following suggestions are proposed to help stabilize and improve the training of GANs.

First five methods are practical techniques to achieve faster convergence of GAN training . The last two are proposed in  to solve the problem of disjoint distributions.

\(1\) **Feature Matching**

Feature matching suggests to optimize the discriminator to inspect whether the generator's output matches expected statistics of the real samples. In such a scenario, the new loss function is defined as $\| \mathbb{E}_{x \sim p_r} f(x) - \mathbb{E}_{z \sim p_z(z)}f(G(z)) \|_2^2$, where $f(x)$ can be any computation of statistics of features, such as mean or median.

\(2\) **Minibatch Discrimination**

With minibatch discrimination, the discriminator is able to digest the relationship between training data points in one batch, instead of processing each point independently.

In one minibatch, we approximate the closeness between every pair of samples, $c(x_i, x_j)$, and get the overall summary of one data point by summing up how close it is to other samples in the same batch, $o(x_i) = \sum_{j} c(x_i, x_j)$. Then $o(x_i)$ is explicitly added to the input of the model.

\(3\) **Historical Averaging**

For both models, add $\| \Theta - \frac{1}{t} \sum_{i=1}^t \Theta_i \|^2$ into the loss function, where $\Theta$ is the model parameter and $\Theta_i$ is how the parameter is configured at the past training time $i$. This addition piece penalizes the training speed when $\Theta$ is changing too dramatically in time.

\(4\) **One-sided Label Smoothing**

When feeding the discriminator, instead of providing 1 and 0 labels, use soften values such as 0.9 and 0.1. It is shown to reduce the networks' vulnerability.

\(5\) **Virtual Batch Normalization (VBN)**

Each data sample is normalized based on a fixed batch (*"reference batch"*) of data rather than within its minibatch. The reference batch is chosen once at the beginning and stays the same through the training.

\(6\) **Adding Noises**

Based on the discussion in Sec. , we now know $p_r$ and $p_g$ are disjoint in a high dimensional space and it causes the problem of vanishing gradient. To artificially "spread out" the distribution and to create higher chances for two probability distributions to have overlaps, one solution is to add continuous noises onto the inputs of the discriminator $D$.

\(7\) **Use Better Metric of Distribution Similarity**

The loss function of the vanilla GAN measures the JS divergence between the distributions of $p_r$ and $p_g$. This metric fails to provide a meaningful value when two distributions are disjoint.

Wasserstein metric is proposed to replace JS divergence because it has a much smoother value space. See more in the next section.

# Wasserstein GAN (WGAN)

## What is Wasserstein Distance?

*Wasserstein Distance* is a measure of the distance between two probability distributions. It is also called *Earth Mover's distance*, short for EM distance, because informally it can be interpreted as the minimum energy cost of moving and transforming a pile of dirt in the shape of one probability distribution to the shape of the other distribution. The cost is quantified by: the amount of dirt moved x the moving distance.

Let us first look at a simple case where the probability domain is discrete. For example, suppose we have two distributions $P$ and $Q$, each has four piles of dirt and both have ten shovelfuls of dirt in total. The numbers of shovelfuls in each dirt pile are assigned as follows:

$$P_1 = 3, P_2 = 2, P_3 = 1, P_4 = 4\\
Q_1 = 1, Q_2 = 2, Q_3 = 4, Q_4 = 3$$

In order to change $P$ to look like $Q$, as illustrated in Fig. , we:

-   First move 2 shovelfuls from $P_1$ to $P_2$ =\> $(P_1, Q_1)$ match up.

-   Then move 2 shovelfuls from $P_2$ to $P_3$ =\> $(P_2, Q_2)$ match up.

-   Finally move 1 shovelfuls from $Q_3$ to $Q_4$ =\> $(P_3, Q_3)$ and $(P_4, Q_4)$ match up.

If we label the cost to pay to make $P_i$ and $Q_i$ match as $\delta_i$, we would have $\delta_{i+1} = \delta_i + P_i - Q_i$ and in the example:

$$\begin{aligned}
\delta_0 &= 0\\
\delta_1 &= 0 + 3 - 1 = 2\\
\delta_2 &= 2 + 2 - 2 = 2\\
\delta_3 &= 2 + 1 - 4 = -1\\
\delta_4 &= -1 + 4 - 3 = 0
\end{aligned}$$

Finally the Earth Mover's distance is $W = \sum \vert \delta_i \vert = 5$.

When dealing with the continuous probability domain, the distance formula becomes:

$$W(p_r, p_g) = \inf_{\gamma \sim \Pi(p_r, p_g)} \mathbb{E}_{(x, y) \sim \gamma}[\| x-y \|]$$

In the formula above, $\Pi(p_r, p_g)$ is the set of all possible joint probability distributions between $p_r$ and $p_g$. One joint distribution $\gamma \in \Pi(p_r, p_g)$ describes one dirt transport plan, same as the discrete example above, but in the continuous probability space. Precisely $\gamma(x, y)$ states the percentage of dirt should be transported from point $x$ to $y$ so as to make $x$ follows the same probability distribution of $y$. That's why the marginal distribution over $x$ adds up to $p_g$, $\sum_{x} \gamma(x, y) = p_g(y)$ (Once we finish moving the planned amount of dirt from every possible $x$ to the target $y$, we end up with exactly what $y$ has according to $p_g$.) and vice versa $\sum_{y} \gamma(x, y) = p_r(x)$.

When treating $x$ as the starting point and $y$ as the destination, the total amount of dirt moved is $\gamma(x, y)$ and the traveling distance is $\| x-y \|$ and thus the cost is $\gamma(x, y) \cdot \| x-y \|$. The expected cost averaged across all the $(x,y)$ pairs can be easily computed as:

$$\sum_{x, y} \gamma(x, y) \| x-y \|
= \mathbb{E}_{x, y \sim \gamma} \| x-y \|$$

Finally, we take the minimum one among the costs of all dirt moving solutions as the EM distance. In the definition of Wasserstein distance, the $\inf$ (infimum, also known as \*greatest lower bound\*) indicates that we are only interested in the smallest cost.

## Why Wasserstein is better than JS or KL Divergence?

Even when two distributions are located in lower dimensional manifolds without overlaps, Wasserstein distance can still provide a meaningful and smooth representation of the distance in-between.

The WGAN paper exemplified the idea with a simple example.

Suppose we have two probability distributions, $P$ and $Q$:

$$\forall (x, y) \in P, x = 0 \text{ and } y \sim U(0, 1)\\
\forall (x, y) \in Q, x = \theta, 0 \leq \theta \leq 1 \text{ and } y \sim U(0, 1)\\
$$

When $\theta \neq 0$:

$$\begin{aligned}
D_{KL}(P \| Q) &= \sum_{x=0, y \sim U(0, 1)} 1 \cdot \log\frac{1}{0} = +\infty \\
D_{KL}(Q \| P) &= \sum_{x=\theta, y \sim U(0, 1)} 1 \cdot \log\frac{1}{0} = +\infty \\
D_{JS}(P, Q) &= \frac{1}{2}(\sum_{x=0, y \sim U(0, 1)} 1 \cdot \log\frac{1}{1/2} + \sum_{x=0, y \sim U(0, 1)} 1 \cdot \log\frac{1}{1/2}) = \log 2\\
W(P, Q) &= |\theta|
\end{aligned}$$

But when $\theta = 0$, two distributions are fully overlapped:

$$\begin{aligned}
D_{KL}(P \| Q) &= D_{KL}(Q \| P) = D_{JS}(P, Q) = 0\\
W(P, Q) &= 0 = \lvert \theta \rvert
\end{aligned}$$

$D_{KL}$ gives us infinity when two distributions are disjoint. The value of $D_{JS}$ has sudden jump, not differentiable at $\theta = 0$. Only Wasserstein metric provides a smooth measure, which is super helpful for a stable learning process using gradient descents.

## Use Wasserstein Distance as GAN Loss Function

It is intractable to exhaust all the possible joint distributions in $\Pi(p_r, p_g)$ to compute $\inf_{\gamma \sim \Pi(p_r, p_g)}$. Thus the authors proposed a smart transformation of the formula based on the Kantorovich-Rubinstein duality to:

$$W(p_r, p_g) = \frac{1}{K} \sup_{\| f \|_L \leq K} \mathbb{E}_{x \sim p_r}[f(x)] - \mathbb{E}_{x \sim p_g}[f(x)]$$

where $\sup$ (supremum) is the opposite of $inf$ (infimum); we want to measure the least upper bound or, in even simpler words, the maximum value.

### Lipschitz Continuity

The function $f$ in the new form of Wasserstein metric is demanded to satisfy $\| f \|_L \leq K$, meaning it should be *K-Lipschitz continuous*.

A real-valued function $f: \mathbb{R} \rightarrow \mathbb{R}$ is called $K$-Lipschitz continuous if there exists a real constant $K \geq 0$ such that, for all $x_1, x_2 \in \mathbb{R}$,

$$\lvert f(x_1) - f(x_2) \rvert \leq K \lvert x_1 - x_2 \rvert$$

Here $K$ is known as a Lipschitz constant for function $f(.)$. Functions that are everywhere continuously differentiable is Lipschitz continuous, because the derivative, estimated as $\frac{\lvert f(x_1) - f(x_2) \rvert}{\lvert x_1 - x_2 \rvert}$, has bounds. However, a Lipschitz continuous function may not be everywhere differentiable, such as $f(x) = \lvert x \rvert$.

Explaining how the transformation happens on the Wasserstein distance formula is worthy of a long post by itself, so I skip the details here. If you are interested in how to compute Wasserstein metric using linear programming, or how to transfer Wasserstein metric into its dual form according to the Kantorovich-Rubinstein Duality, read this awesome [post](https://vincentherrmann.github.io/blog/wasserstein/).

### Wasserstein Loss Function

Suppose this function $f$ comes from a family of K-Lipschitz continuous functions, $\{ f_w \}_{w \in W}$, parameterized by $w$. In the modified Wasserstein-GAN, the "discriminator" model is used to learn $w$ to find a good $f_w$ and the loss function is configured as measuring the Wasserstein distance between $p_r$ and $p_g$.

$$L(p_r, p_g) = W(p_r, p_g) = \max_{w \in W} \mathbb{E}_{x \sim p_r}[f_w(x)] - \mathbb{E}_{z \sim p_r(z)}[f_w(g_\theta(z))]$$

Thus the "discriminator" is not a direct critic of telling the fake samples apart from the real ones anymore. Instead, it is trained to learn a $K$-Lipschitz continuous function to help compute Wasserstein distance. As the loss function decreases in the training, the Wasserstein distance gets smaller and the generator model's output grows closer to the real data distribution.

One big problem is to maintain the $K$-Lipschitz continuity of $f_w$ during the training in order to make everything work out. The paper presents a simple but very practical trick: After every gradient update, clamp the weights $w$ to a small window, such as $[-0.01, 0.01]$, resulting in a compact parameter space $W$ and thus $f_w$ obtains its lower and upper bounds to preserve the Lipschitz continuity.

Compared to the original GAN algorithm, the WGAN undertakes the following changes:

-   After every gradient update on the critic function, clamp the weights to a small fixed range, $[-c, c]$.

-   Use a new loss function derived from the Wasserstein distance, no logarithm anymore. The "discriminator" model does not play as a direct critic but a helper for estimating the Wasserstein metric between real and generated data distribution.

-   Empirically the authors recommended RMSProp optimizer on the critic, rather than a momentum based optimizer such as Adam which could cause instability in the model training. I haven't seen clear theoretical explanation on this point through.

Sadly, Wasserstein GAN is not perfect. Even the authors of the original WGAN paper mentioned that *"Weight clipping is a clearly terrible way to enforce a Lipschitz constraint"*. WGAN still suffers from unstable training, slow convergence after weight clipping (when clipping window is too large), and vanishing gradients (when clipping window is too small).

Some improvement, precisely replacing weight clipping with *gradient penalty*, has been discussed in .


title: The Karlskrona manifesto for sustainability design

**Introduction** *Version 1.0, May 2015*
As software practitioners and researchers, we are part of the group of people who design the software systems that run our world. Our work has made us increasingly aware of the impact of these systems and the responsibility that comes with our role, at a time when information and communication technologies are shaping the future. We struggle to reconcile our concern for planet Earth and its societies with the work that we do. Through this work we have come to understand that we need to redefine the narrative on sustainability and the role it plays in our profession.
What is sustainability, really? We often define it too narrowly. Sustainability is at its heart a systemic concept and has to be understood on a set of dimensions, including social, individual, environmental, economic, and technical.
Sustainability is fundamental to our society. The current state of our world is unsustainable in more ways that we often recognize. Technology is part of the dilemma and part of possible responses. We often talk about the immediate impact of technology, but rarely acknowledge its indirect and systemic effects. These effects play out across all dimensions of sustainability over the short, medium and long term.
Software in particular plays a central role in sustainability. It can push us towards growing consumption of resources, growing inequality in society, and lack of individual self-worth. But it can also create communities and enable thriving of individual freedom, democratic processes, and resource conservation. As designers of software technology, we are responsible for the long-term consequences of our designs. Design is the process of understanding the world and articulating an alternative conception on how it should be shaped, according to the designer's intentions. Through design, we cause change and shape our environment. If we don't take sustainability into account when designing, no matter in which domain and for what purpose, we miss the opportunity to cause positive change.
**We recognize that** there is a rapidly increasing awareness of the fundamental need and desire for a more sustainable world, and a lot of genuine desire and goodwill - but this alone can be ineffective unless we come to understand that:

As a result, even though the importance of sustainability is increasingly recognized, many software systems are unsustainable, and the broader impacts of most software systems on sustainability are unknown.
**Thus, we propose the following initial set of principles and commitments:**
**Sustainability is systemic.** Sustainability is never an isolated property. Systems thinking has to be the starting point for the transdisciplinary common ground of sustainability.
**Sustainability has multiple dimensions.** We have to include those dimensions into our analysis if we are to understand the nature of sustainability in any given situation.
**Sustainability transcends multiple disciplines.** Working in sustainability means working with people from across many disciplines, addressing the challenges from multiple perspectives.
**Sustainability is a concern independent of the purpose of the system.** Sustainability has to be considered even if the primary focus of the system under design is not sustainability.
**Sustainability applies to both a system and its wider contexts** There are at least two spheres to consider in system design: the sustainability of the system itself and how it affects sustainability of the wider system of which it will be part.
**Sustainability requires action on multiple levels.** Some interventions have more leverage on a system than others. Whenever we take action towards sustainability, we should consider opportunity costs: action at other levels may offer more effective forms of intervention.
**System visibility is a necessary precondition and enabler for sustainability design.** The status of the system and its context should be visible at different levels of abstraction and perspectives to enable participation and informed responsible choice.
**Sustainability requires long-term thinking.** We should assess benefits and impacts on multiple timescales, and include longer-term indicators in assessment and decisions.
**It is possible to meet the needs of future generations without sacrificing the prosperity of the current generation.** Innovation in sustainability can play out as decoupling present and future needs. By moving away from the language of conflict and the trade-off mindset, we can identify and enact choices that benefit both present and future.
Sustainability design in the context of software systems is the process of designing systems with sustainability as a primary concern, based on a commitment to these principles.

**So what now? How do we start?**
Each of the following stakeholders can do something right now to get started.
**Software practitioners**: Try to identify effects of your project on technical, economic, environmental sustainability. Start asking questions about how to incorporate the principles into daily practice. Think about the social and individual dimensions. Talk about it with your colleagues.
**Researchers**: Identify one research question in your field that can help us to better understand sustainability design. Discuss it with your peers and think about how sustainability impacts your research area.
**Professional associations**: Revise code of ethics and practice to incorporate principles and explicitly acknowledge the need to consider sustainability as part of professional practice.
**Educators**: Integrate sustainability design in curricula for software engineering and other disciplines and articulate competencies required for successful sustainability design.
**Customers**: Put the concern on the table. Demand it in the next project.
**Users**: Demand that the products you use demonstrate that their designers have considered all dimensions of sustainability.

**Signed,**

*Christoph Becker*, University of Toronto & Vienna University of Technology

*Ruzanna Chitchyan*, University of Leicester

*Leticia Duboc*, State University of Rio de Janeiro

*Steve Easterbrook*, University of Toronto

*Martin Mahaux*, University of Namur

*Birgit Penzenstadler*, California State University Long Beach

*Guillermo Rodriguez-Navas*, Malardalen University

*Camille Salinesi*, Universite Paris 1

*Norbert Seyff*, University of Zurich

*Colin C. Venters*, University of Huddersfield

*Coral Calero*, University of Castilla-La Mancha

*Sedef Akinli Kocak*, Ryerson University

*Stefanie Betz*, Karlsruhe Institute of Technology