trentmkelly/LessWrong-43k huggingface dataset:
Everything I ever needed to know, I learned from World of Warcraft: Incentives and rewards
by Said Achmiz — 2018-05-07T06:44:47.775Z
Link: https://blog.obormot.net/Everything-I-ever-needed-to-know-I-learned-from-World-of-Warcraft-Incentives-and-rewards

This is the second in a series of posts about lessons from my experiences in World of Warcraft. I’ve been talking about this stuff for a long time—in forum comments, in IRC conversations, etc.—and this series is my attempt to make it all a bit more legible. I’ve added footnotes to explain some of the jargon, but if anything remains incomprehensible, let me know in the comments.

Previous post in series: Goodhart’s law.

“How do we split the loot?”

That was one of the biggest challenges of raiding in World of Warcraft.

We’ve gotten 40 people together; we’ve kept them focused on the task, for several hours straight; we’ve coordinated their efforts; we’ve figured out the optimal strategy and tactics for taking down the raid boss; we’ve executed flawlessly (or close enough, anyway). Now, the dragon (or demon, or sentient colossus of magically animated lava) lies dead at our feet, and we’re staring at the fabulous treasure that was his, and is now ours; and the question is: who gets it?

The problem of the indivisible

“Why not divide it 40 ways? That’s only fair!”

If only it were that easy! But the loot can’t be split 40 ways, because it’s not just a giant pile of gold coins; it’s (for instance): a magic warhammer, a magic staff, and a magic robe. Three[1] items; each quite valuable and desirable; each of which can be given to one person, and cannot be sold or traded thereafter. We have to decide, here and now, which three out of the 40 members of the raid will receive one of these rewards. The other 37 get nothing.

What to do?

It would be difficult to overstate how much thought went into answering this question, among WoW players; how much effort was spent on debating it; how much acrimony it spawned; and how critical was a good answer to it, in determining success in the most challenging endeavors in the game (high-end raiding). Disagreements in matters of loot distribution broke friendships, and ill-advised loot policies cracked guilds in half.

Nor should this be at all surprising, for World of Warcraft was human civilization in microcosm. The task of organizing and coordinating a raid was project management; figuring out how to take down a raid boss was strategic planning, and practical epistemology, and mathematics; making money at the Auction House (in order to purchase performance-enhancing potions) was economics. But the distribution of loot—that was politics.

Pie

...
Fuzzy Boundaries, Real Concepts
by orthonormal — 2018-05-07T03:39:33.033Z

Summary: Certain basic concepts are still very useful, even if they have fuzzy or contested boundaries, or break down in edge cases. This is basically just working out a few examples of The Cluster Structure of Thingspace. Two important examples are honesty and consent.

Adam: I can't believe that scientists 'decided' Pluto wasn't a planet. The absolute gall!

Betsy: Oh hey Adam, the demotion of Pluto is a pretty neat story! The astronomers originally didn't have to think about the definition of a planet, because they were so clearly different from other bodies they could see in the Solar System, like asteroids and comets and moons; but then they discovered Eris and many more objects like it, and they had to figure out an actual definition...

Adam: But Pluto is obviously a planet! I learned it as part of my acronym- My Very Excellent Mother Just Served Us Nine Pizzas! A few new asteroids can't change that!

Betsy: Okay, but Eris is more massive than Pluto. Sure, it's farther out, but so are some gas giant exoplanets we've discovered around other stars. The astronomers needed to agree on a definition that wasn't incredibly convoluted, so they decided that planets needed to be massive enough that gravity makes them approximately round, and massive enough to fling smaller asteroids out of their orbit.

Adam: You make it sound like astronomers can just decide what counts as a planet or not. That's ridiculous!

Chris: You both are so naive. There's no such thing as a planet, there's only collections of atoms.

Betsy: Um, Chris, that's really not helpful here. See, there are several things that the eight planets have in common with each other that nothing else orbiting the Sun is even close to; for instance, on the "clearing its own orbit" front, each of the eight planets accounts for more than 99.98% of the total mass in its orbit, while no other object accounts for more than one-third. (Pluto accounts for only 7 percent of the total mass in its orbit.)

Chris: But what if we discovered another object the size of Pluto that only accounted for 99.9% of the mass in its orbit? 99%? 90%? It's completely arbitrary where you draw the line!

Adam: (mumbling) Pizzas! You can't just serve us nine nothings!

Betsy: Yes, Chris, there may or may not be a gray area when it comes to exoplanets, since these thing...
LW Open Source – Getting Started [warning: not up-to-date]
by Raemon — 2018-05-06T23:28:25.642Z

update: since posting this a few years ago, our architecture and practices have changed slightly. I've fixed some things but don't haven't checked all the pieces here to see if they're still accurate.

LessWrong 2.0 is open source, but so far we haven’t done a great job at making contributing a good experience. Here’s our first attempt at fixing this, which covers the basics of "how to contribute" and "high level overview of the site architecture."

tl;dr: Get a local copy of our git repo installed. This post may be out of date but but maybe still helpful for getting oriented.

Getting Started

Assumed Background Knowledge

Currently, this guide assumes that you know your way around a terminal and github, and have worked with javascript, html, sass/css, and ReactJS.

If you know at least some of those things and are excited to contribute, you can probably make decent headway (and if you run into specific obstacles, comment on them on this post or ping us on intercom, and we’ll see if we can update things to be more clear)

Setting up Locally

Go through the instruction on our Github repo.

You can poke around the codebase. Eventually, to make serious contributions you’ll want to read about our site architecture.

What Contributions Are Helpful?

The most reliably helpful thing would be to tackle outstanding

issues that have been tagged on github

.

In particular, you can filter them by the tag “

good first issue

.” (Some of these might require some explanation, but I expect I can explain them fairly easily to a new contributor)

Creating Issues

You can create a new issue. If so, please leave it untagged for the time being (so that admins can quickly look for untagged issues, and sort through them)

Bugs – If you run into a bug, the most helpful thing to do is to search for related keywords in the issue tracker. If you can’t find anything relevant, create a new issue. Try to provide as specific information as possible (your browser, exact links to the post or page where you had the issue, information on how to replicate the bug if you can)

Feature Requests – Feature requests will often need to undergo some discussion to get refined into something executable, and may sometimes need to be split into sub-features.

Features tend to cluster into either “things that are pretty straightforward and we almost certainly want to do” and “things that we have weird conceptual philosophical opinions about ...
LW Update 5/6/2018 – Meta and Moderation
by Raemon — 2018-05-06T20:33:10.554Z

User Facing Changes

New posts can only be submitted to your personal blog. Moderators will move it to the frontpage if it seems appropriate.
"Personal Blogposts" tab is now labelled "All Posts", and now contains meta posts.

We had deliberately been pushing on "disincentivize meta conversation", and still are, but since the Meta section basically has only been used for updates from the admins, it seemed like A) we could lean a bit in the other direction, B) I think having the third tier of posts on the frontpage be "all posts" is a lot more clear about what the frontpage hierarchy is trying to do.

Meta posts now appear properly in your drafts on your user profile.
Your user profile will now only show the 4 most recent drafts

I rarely found I needed to see anything but my most recent drafts, and meanwhile it was a bit annoying to scroll past it whenever I went to my profile page.

Moderator Changes

Moderators now have a sidebar that shows a list of all new unreviewed personal blogposts, with easy buttons for:

approve (and leave as a personal blogpost)
move to frontpage
delete

In the past month or so, we've a) continued seeing a rise in post volume, while b) a couple of our moderators have been more busy than usual. I've been increasingly worried about the signal/noise ratio on the site.

So this update has been focused mostly on some tools that make moderation much easier, which many users won't directly experience but which I hope will lead to a better site experience.

Longterm, my goal is to put all moderator To Do items in the sidebar so they are easily seen, and then make it easy and encouraged to maintain inbox zero on the sidebar. I'd hoping this will help both reactive "notice and delete spam" type actions, and "notice and promote good things" type actions.

Minor Naming Convention Update

I'll be calling these posts "LW Update [Date] – {Description}" from now on, since "update" wasn't very each to search for, and hopefully "LW Update" will be. (We'll eventually also make a better search tool, but even when it's done it seems like it should be easy to search naively)

Link to Git history. Commit a7c9879b5f5fa94f0a843993b10c85898913a43e
Welcome to SSC-LW Toronto [Edit With Your Details]
by JessRiedel — 2018-05-06T20:17:22.917Z

(The following are our suggestions for what kind of information is best to include in the welcome post of your group, feel free to replace them with whatever you think is best)

What kind of events does your group usually run? What does it usually do?

How frequently does your group organize events or meet?

Who would be a good fit for you group?

Should they have any particular skills or have done some specific background reading?
Advocating for factual advocacy 
by Jsevillamol — 2018-05-06T08:47:46.599Z

In this post I present a Hansonian view of morality, and tease out its consequences for advocacy. Beware, all of this is the result of armchair speculation.

The simplified model goes something like this:

Human morality is mostly a justification mechanism, trying to give coherence to the actions we were going to do anyway.
Concretely, when spousing a belief or its opposite is cheap (it does not fulfill a social function nor will it contradict our future actions) we will prefer to stick to the position that better fits our current plan.
In general our plan follows a minimal effort law, so we will stick to the side of the belief which is more convenient.
The other side of the coin is that when we receive new information about the world which changes what we will do in the future it has profound effects in our morality.

Example: Consider this study which reaches the conclusion that a higher minimum salary reduced the average payroll of a low income person by 125$. This is not a moral claim, yet for some it will feel like it is. Often we find that factual claims are processed by our brains as moral claims.

This explains why humans have so much difficulty with the "is" vs "ought" question - for the explicit models in the human brain, what is and what ought to be are both grounded on factual information.

This model makes a bold prediction: philosophical arguments such as "The Drowning Child" do not derive their strength from their moral validity, but instead from the factual revelation that it is possible to save a child's life for a rather modest sum of money.

This has direct relevance to advocacy, since it suggests that in order to change somebody's moral opinion on eg animalism we should not focus on trying to make moral arguments, but instead on giving people new data they previously didn't have and that increases the perceived convenience of taking a particular action eg objective figures on how much money does it cost to save an animal's life.

You can also go one meta level up and change how convenient it factually is to help with your cause of choice, by for example developing clean, affordable meat.

This contrast starkly with people trying to use emotional appeal or philosophically grounded arguments, which we would predict to have small long term effects.

Questions: What other predictions does this model make? Where do...
Predicting Future Morality
by ESRogs — 2018-05-06T07:17:16.548Z

Robin Hanson suggests that recent changes in moral attitudes (in the last few hundred years) are better explained by changing circumstances than by progress in moral reasoning.

This seems plausible to me. It also seems likely that there would be a bit of a lag between the change in circumstance and the common acceptance of the new morality. (The sexual revolution following the introduction of the pill seems like a good example.)

Suppose this is broadly right -- that moral attitudes follow circumstances. Is there anything we can predict about where moral attitudes will be in the next few decades (or economic doublings), based on either recent technological or economic changes, or on those we can see on the horizon?
Looking for AI Safety Experts to Provide High Level Guidance for RAISE
by Ofer — 2018-05-06T02:06:51.626Z

The

Road to AI Safety Excellence

(RAISE) initiative aims to allow aspiring AI safety researchers and interested students to get familiar with the research landscape effectively; thereby hopefully increasing the number of researchers that contribute to the field. To that end, we (the RAISE team) are trying to build a high-quality online course. You can see our pilot lesson

here

(under “Corrigibility 1”).

Most of the course segments will be based on distilled summaries of one or more papers. We already distilled ~9 papers on corrigibility for the first course segments, and used the distilled summaries to write video script drafts.

Our long-term goal is to cover as much of the AI safety research landscape as possible, in the most useful way possible. Therefore, we need guidance from experts who have extensive familiarity with the literature in one of the broad subfields of AI safety (i.e. the machine learning perspective or the Agent Foundations research agenda; or broad parts thereof). We realize that the time of such experts is a critically scarce resource. Therefore, we will ask them only for high-level guidance including:

1) Their idea of a good structure for a part of the course: a list of sections, and the subsections that might constitute each one.

2) Pointers to papers to base each subsection on.

If an expert expects contributing further to RAISE to be an effective use of their time, they could also choose to go over our lesson scripts and provide feedback before the

videos

are being recorded.

Should this role be an effective use of your time, please contact us at raise@aisafety.camp
Open question: are minimal circuits daemon-free?
by paulfchristiano — 2018-05-05T22:40:20.509Z

Note: weird stuff, very informal.

Suppose I search for an algorithm that has made good predictions in the past, and use that algorithm to make predictions in the future.

I may get a "daemon," a consequentialist who happens to be motivated to make good predictions (perhaps because it has realized that only good predictors survive). Under different conditions, the daemon may no longer be motivated to predict well, and may instead make "predictions" that help it achieve its goals at my expense.

I don't know whether this is a real problem or not. But from a theoretical perspective, not knowing is already concerning--I'm trying to find a strong argument that we've solved alignment, not just something that seems to work in practice.

I am pretty convinced that daemons are a real problem for Solomonoff induction. Intuitively, the problem is caused by "too much compute." I suspect that daemons are also a problem for some more realistic learning procedures (like human evolution), though in a different shape. I think that this problem can probably be patched, but that's one of the major open questions for the feasibility of prosaic AGI alignment.

I suspect that daemons aren't a problem if we exclusively select for computational efficiency. That is, I suspect that the fastest way to solve any particular problem doesn't involve daemons.

I don't think this question has much intrinsic importance, because almost all realistic learning procedures involve a strong simplicity prior (e.g. weight sharing in neural networks).

But I do think this question has deep similarities to more important problems, and that answering this question will involve developing useful conceptual machinery. Because we have an unusually strong intuitive handle on the problem, I think it's a good thing to think about.

Problem statement and intuition

Can the smallest boolean circuit that solves a problem be a daemon? For example, can the smallest circuit that predicts my behavior (at some level of accuracy) be a daemon?

Intuitively, if we have a daemon that is instrumentally or incidentally motivated to solve my problem, then there is some smaller circuit that solves the problem equally well but skips the instrumental reasoning. If my daemon is doing some complex reasoning to answer the question "Should I predict well?" we could just...
5-minute timer tool
by Gust — 2018-05-05T15:58:09.836Z
Link: https://gusbicalho.github.io/fiveminutetimer/

This is a 5-minute timer with a textarea that becomes readonly when the timer runs off. This might be useful for people doing resolve cycles, or people telling other people about resolve cycles!

(You can just link them to the page and say “Go to this site and solve your problem in five minutes!“)

[Programming related remarks below]

I also took this as an opportunity to experiment with React state management, as the main App class started to get pretty big. Ended up implementing something akin to Redux, but in a way I think could be made type-safe if built with Flow or TypeScript. I would like to hear about any library that implements something like what I tried here (I’d bet it already exists).

If you want to take a look, the source is MIT-licensed at: https://github.com/gusbicalho/fiveminutetimer.

Comments about the source code can be posted here, or as an issue in the Github repo, or in the version of this post in my personal website (which I'm shamelessly using this as an excuse to promote).
A possible solution to the Fermi Paradox
by Rafael Harth — 2018-05-05T14:56:03.143Z

[tl;dr: I argue that, if Many Worlds is true, the Survival Bias might explain the lack of observed life in the universe, under the assumption that there are no humans in most worlds where life on a reachable planet has reached technological maturity.]

[Content Warning: possibly unsettling]

Suppose you are offered a deal: you'll be put to sleep, cloned 99 times, and each of the 100 versions of you will be sent to an identical-looking room where they'll be woken up. Suppose that you all share the same consciousness: the lights are on for the same entity in all 100 copies (but they have no way of communicating with each other). A minute after waking up, one randomly chosen clone will receive five million dollars, while the other 99 will die a painless death, so quick that they will neither see it coming nor feel any pain.

It's not relevant for the plausibility of this argument whether you would take this deal (though it might have other implications). For now, suppose you take it. You're put to sleep and next thing you know, you wake up in a room. After you wait for a minute, the experimenter enters the room, hands you your five million dollars, and politely thanks you for your participation.

Should you be surprised? I'd say no. Nothing surprising has happened; in fact, there was only one way this could have gone all along. On the other hand, if the 99 unlucky copies were not killed but put in prison, I would argue that surprise is warranted. The survival bias is real, but it only applies when there is an experiment on a set of people, and a subset of those won't be able to tell afterwards.

Now consider how many times we narrowly avoided nuclear war. The base theory for why this happened is that we got lucky. But if nuclear war always results in your death[1], and if Many-Worlds is true, then us being still alive isn't surprising at all; rather it's the only observation possible.

Okay, so let's examine the Fermi Problem. Let p be the odds that primitive life on some planet results in a species inventing space travel, and let n be the number of other planets in reach with primitive life on them. A classical explanation for our observations either requires that species who reach earth generally choose to leave us undisturbed, or that (1−p)n be sufficiently large (that's the probability that no alien species in reach mak... ,,,,,,,,,,,,,,
The Sheepskin Effect
by Zvi — 2018-05-05T12:10:00.437Z

Previously: The Case Against Education, The Case Against Education: Foundations, The Case Against Education: Splitting the Education Premium Pie and Considering IQ

Epistemic Status: The spirit of Local Validity as a Key to Sanity and Civilization

The sheepskin effect is that completing the last year of high school, college or graduate school is much more profitable than completing any of the previous years, rivaling those other years combined. Employers seem to be paying for the degree (aka the sheepskin, which it’s printed on) rather than the human capital being built over time.

In the education chapter of Book Review: The Elephant in the Brain, I noted Robin relied on the sheepskin effect as strong evidence (along with other arguments, including impacts on national vs. personal income) that school was mostly signaling. Bryan Caplan does the same. He cites the data, seeing (on top of a 10% bonus in pay per year of school) 32% bonus pay for finishing high school, 10% for junior college, 30% for a bachelor’s degree and 18% for a masters. To those who claim this is mostly ability bias, he replies:

Ability bias explanations for sheepskin effects aren’t just hard to square with statistical evidence; they’re hard to square with the glaring fact that education spikes in degree years. If the labor market ignores credentials, why do so many college grads opt for zero graduate education? Are we supposed to believe one-third of the population has exactly the right ability to finish high school, but not advance to college? One-seventh has exactly the right ability to finish college, but not advance to graduate school?

…

To debunk sheepskin effects, correcting for these neglected abilities would have to drastically cut the payoff for degrees but not the payoff for years of schooling. What abilities would even conceivably qualify?

This seems like a straw man; no one thinks the labor market ignores credentials, so it’s easy to see why students act the way they do. Not only is not finishing high school severely punished as such (as the numbers show), not trying is at least sort of illegal. In addition, there’s a huge barrier to getting into college or graduate school, and large costs involved in starting, often involving relocation. Also, much of college is about being ready for the rest of college, and the early part of graduate school...