trentmkelly/LessWrong-43k huggingface dataset:
Announcement: AI alignment prize round 2 winners and next round
by cousin_it — 2018-04-16T03:08:20.412Z

We (Zvi Mowshowitz and Vladimir Slepnev) are happy to announce the results of the second round of the

AI Alignment Prize

, funded by Paul Christiano. From January 15 to April 1 we received 37 entries. Once again, we received an abundance of worthy entries. In this post we name five winners who receive $15,000 in total, an increase from the planned $10,000.

We are also announcing the next round of the prize, which will run until June 30th, largely under the same rules as before.

The winners

First prize of $5,000 goes to Tom Everitt (Australian National University and DeepMind) and Marcus Hutter (Australian National University) for the paper

The Alignment Problem for History-Based Bayesian Reinforcement Learners

. We're happy to see such a detailed and rigorous write up of possible sources of misalignment, tying together a lot of previous work on the subject.

Second prize of $4,000 goes to Scott Garrabrant (MIRI) for these LW posts:

Robustness to Scale

Sources of Intuitions and Data on AGI

Don't Condition on no Catastrophes

Knowledge is Freedom

Each of these represents a small but noticeable step forward, adding up to a sizeable overall contribution. Scott also won first prize in the previous round.

Third prize of $3,000 goes to Stuart Armstrong (FHI) for his post

Resolving human values, completely and adequately

and other LW posts during this round. Human values can be under-defined in many possible ways, and Stuart has been very productive at teasing them out and suggesting workarounds.

Fourth prize of $2,000 goes to Vanessa Kosoy (MIRI) for the post

Quantilal control for finite MDPs

. The idea of quantilization might help mitigate the drawbacks of extreme optimization, and it's good to see a rigorous treatment of it. Vanessa is also a second time winner.

Fifth prize of $1,000 goes to Alex Zhu (unaffiliated) for these LW posts:

Reframing misaligned AGI's: well-intentioned non-neurotypical assistants

Metaphilosophical competence can't be disentangled from alignment

Corrigible but misaligned: a superintelligent messiah

My take on agent foundations: formalizing metaphilosophical competence

Alex's posts have good framings of several problems related to AI alignment, and led to a surprising amount of good discussion.

We will contact each winner by email to arrange transfer of money.

We would also like to thank everyone else who sent in their work! The on...
Welcome to Portland SSC Meetup
by Noah Walton — 2018-04-16T03:03:24.864Z

We meet the second weekend of every month, alternating Saturdays and Sundays. We tend to discuss, and sometimes play games. If you've read some SSC, you're welcome to come. If you have any questions, or want to know where to go for the meetup, email me (nwalton125@gmail.com).
Remembering the passing of Kathy Forth.
by Elo — 2018-04-16T01:53:56.536Z

It is with sadness and regret of the loss of life that I inform you that Kathy Forth has passed away.

She will be remembered at the solstices and forever in our hearts as someone trying to make the world a better place in the ways they believe are most important.

This post is to serve as a eulogy. I know she didn't always agree and she didn't always make friends. Despite that she was a part of this community. I would appreciate anyone interested in saying kind words to do so below. Anyone unwilling to say kindness can keep it to themselves.

http://effective-altruism.com/ea/1n7/remembering_the_passing_of_kathy_forth/

https://www.facebook.com/forthk/posts/2077196849230469

https://www.facebook.com/Kathleen-Rebecca-Forth-Memorial-Page-184870828823873/?modal=admin_todo_tour

Requiem Mass service to be held on Saturday April 21, 2018 at 10:00 am CST at the following location:

The Church Of St. Francis
Liberal Catholic Church
12 School St.
Villa Park, Il. 60181
(630) 592-8482
Welcome to Chicago Rationality
by mingyuan — 2018-04-15T15:42:39.764Z

From January of 2017 through January of 2018, Chicago Rationality met weekly at either the University of Chicago or Harold Washington Library. The format of almost all events was to discuss assigned readings. We also held two large SSC meetups, which were more social, and a few applied rationality training sessions.

Anyone is welcome to organize a rationalist-adjacent event of any type (not just a reading group) under the banner of Chicago Rationality.
The Case Against Education
by Zvi — 2018-04-15T12:30:01.133Z

Previously: Something Was Wrong, Book Review: The Elephant in the Brain

Previously (Compass Rose): The Order of the Soul

Epistemic Status: No, seriously. Also literally.

They sentenced me to twenty years of boredom

for trying to change the system from within

I’m coming now I’m coming to reward them

First we take Manhattan, then we take Berlin

— Leonard Cohen, First We Take Manhattan

This was originally going to be my review of Bryan Caplan’s excellent new book, The Case Against Education. I was going to go over lots of interesting points where our ways of thinking differ. Instead, the introduction got a little sidetracked, so that worthy post will have to wait a bit.

First, we have the case against education.

As in: I See No Education Here.

I

What is school?

Eliezer Yudkowsky knows, but is soft peddling (from Inadequate Equilibria):

To paraphrase a commenter on Slate Star Codex: suppose that there’s a magical tower that only people with IQs of at least 100 and some amount of conscientiousness can enter, and this magical tower slices four years off your lifespan. The natural next thing that happens is that employers start to prefer prospective employees who have proved they can enter the tower, and employers offer these employees higher salaries, or even make entering the tower a condition of being employed at all.5

…

Anyway: the natural next thing that happens is that employers start to demand that prospective employees show a certificate saying that they’ve been inside the tower. This makes everyone want to go to the tower, which enables somebody to set up a fence around the tower and charge hundreds of thousands of dollars to let people in.6

Rick (of Rick and Morty) knows:

Nassim Talib knows (quote is from Skin in the Game):

The curse of modernity is that we are increasingly populated by a class of people who are better at explaining than understanding, or better at explaining than doing. So learning isn’t quite what we teach inmates inside the high-security prisons called schools.

Talib considers this fact – that school is a prison – so obvious he tosses it out as an off-hand remark with no explanation. He’s right. If you’re looking at a classroom, you too know. Something Was Wrong. This isn’t a playground designed to teach useful knowledge and inspire creativity. It is a prison where we learn to guess t...
You Are Being Underpaid
by RobertM — 2018-04-15T06:28:00.374Z

If you are a software engineer living in the United States, you are probably underpaid. (If you are a software engineering living outside the United States, this is probably still true, but I have no idea what market conditions are like out there, and my advice would sum up to "move to the United States", which I understand may not be very helpful.)

Let me qualify this by saying that if you are working at a place like Google, Facebook, or Netflix, you're probably doing fine - there might be straightforward ways for you to earn more money, but the scale will be smaller and it'll mostly consist of lateral job hopping. Keep in mind that "big tech company" is not what I'm pointing at here - if you work for Microsoft or Amazon there's a very real possibility you fall into the reference class of people I'm trying to reach.

Let me tell you a quick story. A couple years ago, I was talking to a friend of mine, who works as a software engineer for [redacted]. When he told me how much he was being paid ($140k base, total comp around $200k), I was pretty incredulous (this was before I'd read up on typical BigCo. pay scales, see Dan Luu's excellent blog post for more

info

). Sure, [redacted] was one of the biggest and flashiest tech companies in LA, and my friend filled a fairly specialized niche in a fairly specific industry, but the numbers still seemed insane. During that conversation, he told me, "Dude, I have nearly a decade of experience on you, but you could easily be making something close to this in a year or two." I didn't believe him, of course - that would require doubling my salary. At the time I had ~two years of professional experience and was earning about $80k, which was a recent step-up from a very low career-start of $45k.

A little over half a year ago, I decided for various reasons that I needed to relocate within LA. I'm the kind of person who strongly values having a short commute to work, so I hit up all of my recruiters, gave them a prescribed geographical reason and a salary range that started 30% higher than what I was making then, so $110-120k, and let them loose. The first two months were good - I had enough interviews to keep me busy (considering I was still working full-time), and quickly got into the groove of interviewing again, with an eye to quickly learning the thin...
Dire Bullshit
by Alicorn — 2018-04-15T04:59:54.318Z

When I bought my kid a door jumper (basically a bowl with leg holes dangling from a spring that could be attached to a doorframe) I read all the safety warnings.

Here they are.

In case you don't want to read all of them, there's a bunch of obviously legitimate and potentially legitimate safety warnings, and there's:

"NEVER use optional playmat with any product other than the door jumper."

To be clear, the described item is a rectangle. Of vinyl-or-something-backed fabric.

And this throws all the other safety warnings into question. They're taking their opportunity to warn me about danger to my child, and they're using it to sneak in imperatives about combining a harmless rectangle with other objects. This doesn't happen when you buy a yard of muslin from the store because you don't expect a yard of muslin to come with any safety warnings at all. But when you look to a source for advice, and have reason to take it seriously, it may accumulate cruft:

Recipes on food packaging that call specifically for their brand, of that ingredient and any others the company sells. (If you've been carefully purchasing NESTLE® TOLL HOUSE® chocolate chips because you believe they're essential to the recipe, good news, they are not.)
Institutions like banks and insurance companies, which might mail you important information you'd want to have, and might just be stamping "URGENT" on the envelopes because this improves some metric by getting you to open their random crap.
It's happened to me in a medical setting, ROT-13 if you want the extra example; cbfgcneghz, n ahefr gbyq zr gung rirel gvzr V hfrq gur onguebbz V unq gb fcenl jngre ba zlfrys nf n fbeg bs znxrfuvsg ovqrg. "Sbe ubj ybat?" V nfxrq, nffhzvat guvf unq gb qb jvgu urvtugrarq puvyqovegu-eryngrq vasrpgvba evfx. "Nyy gur gvzr," fur fnvq, "Sbe gur erfg bs zl yvsr?" V fnvq, naq fur abqqrq, naq V arire yvfgrarq gb nalguvat fur fnvq rire ntnva.

This is epistemic poison, but it's all over the place. Attention is memetic energy; memes will evolve to grab it as necessary. Refer your friends (through privileged one to one channels they don't tune out yet, of course) to gain in-game prizes! The next train is in two minutes on platform one, and remember, if you see something, say something. And we learn...
Raven Paradox Revisited
by Chris_Leong — 2018-04-15T00:08:01.907Z

The Raven Paradox Settled to My Satisfaction is a pretty good post, but there's a few things that we can note to make this problem even clearer:

We can simplify this problem so that there are only two colors (black and white) and two kinds of objects (ravens and laptops). The post sort of did this, but not very consistently. I just thought this was worth mentioning as non-black and non-raven are slightly more abstract and so slightly harder to reason with.
Actually, we can make this even more legible. Make the objects medicine and food; and the properties fresh and expired. It's then immediately clear that if we want to check that [All the medicine is fresh], we can either check each item of medicine to see if it is fresh or we can check all the expired objects and see that none of them is medicine. We're used to this kind of practical reasoning, so it's much easier for us than dealing with objects and colours.
If we have simplified the problem so there is only one shade of black, then A: [All ravens are black] is equivalent to B: [The first raven is black] plus C: [All ravens are the same color] apart from the degenerate case. This makes it rather clear why A is more likely to be true when there are less ravens. In particular, if there's only one raven, then we only have to worry about B since C is trivially true.
We tend to confuse the following [An observation of a white laptop is independent of the color distribution of ravens] with [An observation of a white laptop is independent of how many ravens of a particular colour we will observe]. The first is true apart from the restrictions imposed by the number of black ravens having to be integral, but the second is only true if we knew it was a laptop before this observation. If it could have been a raven then it can influence the number of ravens of a particular color by influencing the number of ravens in total.
We tend to expect some correlation between the color of animals. For example, we are quite ready to guess that all ravens are black after only seeing quite a small random sample. On the other hand, suppose that all the non-black things keep being non-living creatures, whilst many of the black things are non-raven living creatures. We might guess that being black provides an evolutionary advantage in this world and so guess that all ravens will be black without ever having seen

...
A Different Prisoner's Dilemma
by Serpent-Stare — 2018-04-14T15:54:13.928Z

I originally composed most of this post out of the things I said while counselling some dear friends of mine who were driving themselves into a panic spiral thinking about the approximately 1.7 people who were dying every second, and feeling responsible even though there was nothing they could do, at that time, to save them. I posted it to my personal blog, but decided to share it here as well, hoping that it contributes to the dicussion, and may help some other sensitive people to preserve themselves against despair, for the good of the future.

There is a certain kind of person who notices that the world is full of suffering, and after they have noticed this, they feel obligated to remind themselves regularly, even to great personal detriment and into depression. These people cannot bring themselves to turn away from suffering even to maintain their own health... not until they are so overwhelmed by compassion for other peoples' pain that they are at risk of breaking down with stress and illness. Why? ... Likely guilt, because they would perceive it as selfish to deliberately ignore someone else's suffering in order to feel better themselves, even if there is nothing they can do about it right now. Likely also fear... that if they did turn away, they would be making themselves into monsters, joining the complicit majority of people who do not act to prevent suffering, who do not seem to care.

I am one such a person, and the kinds of people I make friends with are often prone to this phenomenon. Jennifer Freed calls it "The Empath's Dilemma," but I am not under the impression that it's a term in common use.

A friend of mine came to me tonight stuck in a mental spiral of concern and guilt I recognized as the state of someone being overwhelmed by the Empath's Dilemma, and I swooped in with my own concern, to reassure and comfort them, to shake off the undeserved guilt and help them toward a mental condition from which they would, hopefully, be able to get a decent night's rest.

And then they asked me, "How did you get through this sort of a dark night of the soul [...] whenever that night was for you?"

This is my answer:

I constructed a question. A scenario that might be put to people by which to judge their preferences in a pinch, like the old standard one about pulling a switch to route a train onto a track where i...
How Going Meta Can Level Up Your Career
by Matt Goldenberg — 2018-04-14T02:13:02.380Z

Note: This is a post that I've been sitting on for a few years. It was one of the last posts I was working on for my career coaching business before I ended that business to start something higher impact.

Due to that, the tone isn't exactly crafted for LW, however I still think that it's great instrumental advice for leveling up your career, and a recent thread with Romeo Stevens convinced me that LWers might get something from it.

I think I've developed a good heuristic for how awesome a historical figure was - the more nicknames they had, the more awesome they were. And by that heuristic, John Boyd was pretty damn awesome. Known as 40-Second-Boyd, he could defeat any pilot who came to the military training academy he worked at within 40 seconds. Later, he came to be known as the Mad Major, Genghis John, The Ghetto Colonel, and more. And every one of those nicknames has a story behind it. He's a fascinating guy, and there are an incredible amount of career lessons you can learn from him.(I highly recommend you pick up this biography of him.) The one that jumped out most starkly is the pattern that Boyd exhibited throughout his career that allowed him to consistently progress.

The Four Phases of Boyd's Career

Boyd The Fighter Pilot

In phase one of Boyd's career, he was a fighter pilot for the US military. Boyd focused on learning every tactic to perfection; he learned the limits of the aircraft and used those limits to perform techniques never before considered. He became a master at the craft of dogfighting. Boyd became so good at pushing the aircraft to it's limits that he developed a strategy that allowed him to beat any challenger that came to his training academy in 40 seconds or less. This was how he came to be known as 40-Second-Boyd.

Boyd The Tactician

In the second phase of Boyd's career, he took everything he learned about the techniques he mastered, and asked himself an abstract question: why do I use certain skills at certain times? And what he found was that he could create a set of rules for the tactics he was using, a repeatable process that anyone could use to choose the correct tactic for the situation at hand. He became so proficient at dogfighting strategy that pilots from other military branches would attend the academy just to learn from Boyd - something nearly unheard of in the tightly silo...
Timothy Chu Origins Chapter 1
by alkjash — 2018-04-13T18:40:00.586Z

They remember his ideas, and how they saved the world.

But I remember the man, and how he saved me.

A great deal of ink (though in my estimation as yet too little) has by now been spilled about the life and times of Timothy Chu; among the more readable volumes on the legend are The Age of Tim and Inadequate in the Presence of Timothy. Reading the exploits of Timothy in his prime, it might be hard to believe that he too once knew loss, suffering, and human weakness.

But few of his adoring fans and biographers have the privilege of knowing Tim as a friend, as I do. With this unique privilege, I aim to tell the hidden story about the young man who became the legend, a young man who – while already extraordinary – lived a comparatively modest and down-to-earth life.

Chapter 1: From Steppes to Steps

It is tradition to seek wisdom at the Shaolin Temple as the final step towards manhood. At the age of fourteen, Timothy Chu is already a man, but as a man he knows the importance of tradition.

Tim climbs the steps to the temple four at a time.

Three monks, clad in plain grey-blue habits, sweep the steps up Shaoshi mountain, covertly guarding the only path to the temple.

Tim approaches the first monk, a hunchbacked and scarred man who presses his hands together in greetings.

“Amituofo, traveller. What business have you with Shaolin?”

Tim’s voice is usually unbelievably deep, but he softens it out of respect, “I have come to learn the Way from the Master of the temple.”

“I am Qiang, thirteenth seat of the fifty-third generation. To pass me you must demonstrate strength of will and body.”

Monk Qiang extends a wiry hand in the style of a Western handshake. Tim grips it automatically. Qiang tests Tim’s grip, hunching forward and pressing the force of his entire body into the handshake. The stone steps under Tim’s feet shudder and crack under the enormous force, but Tim himself seems immovable, keeping his pleasant smile and easy posture.

After progressively increasing the pressure for nearly a minute, Qiang gives way before the entire mountainside collapses under his legendary strength. He bows welcome to Tim and shifts aside to let him pass, letting go of his hand.

Tim catches the monk’s hand before it escapes.

“It’s my turn, now.”

Qiang feels an incredible pressure travel through his body from the center of his palm. The force ripples through his body, knocking him to hi...
5 general voting pathologies: lesser names of Moloch
by Jameson Quinn — 2018-04-13T18:38:41.279Z

Earlier, I wrote a primer on voting theory. Among the things I discussed were 5 types of pathologies suffered by different single-winner voting methods. I presented these as 5 sequential hurdles for voting method design. That is, since they are in what I view as decreasing importance and increasing difficulty, you should check your voting method against each hurdle in order, and stop as soon as it fails to pass.

Then I read Eliezer's book on Inadequate Equilibria, and Scott's "Meditations on Moloch". They argue that the point of civilization is to provide mechanisms to get out of pernicious equilibria, and the kakistotropic tendencies of civilization they characterize as "Moloch" are basically cases where pernicious incentives reinforce each other. I realized that the simple two-player games such as Prisoners' Dilemma that serve as intuition pumps for game theory lack some of the characteristics of my 5 voting pathologies. So I want to go back and explain those pathologies more carefully, to help build up intuition about how multi-player, single-outcome games differ from two-player ones.

A key point here is that I'm talking about single-winner voting methods; that is, "games" where the number of possible outcomes is far less than the number of players. In this case, it's not a matter of seeking an individual advantage for yourself; the only way for you to win is for your entire faction to win equally. This means that I will not be talking about the oldest and deepest name of Moloch, which is Malthus. All the Molochs in this essay can and should be killed or (mostly) tamed.

Also note that this essay is not the one I'd write if I were only trying to recruit the rationalist community to become electoral reform activists. As an activist, I think that the most important and short-term-viable electoral reforms are in the multi-winner space: solving the problem of coordinating public goods not directly through mechanism design, but indirectly through a combination of mechanism design and representation. Some of my reasons for thinking that are contingent and have no place here. The one that's not: I think that the problem of "ain't nobody got time for all that politics" is worse than the principal-agent problem of a well-designed representative mechanism. Regardless, I think that this comm...