trentmkelly/LessWrong-43k huggingface dataset:
How do humans assign utilities to world states?
by Dorikka — 2015-05-31T20:40:33.151Z

It seems like a good portion of the whole "maximizing utility" strategy which might be used by a sovereign relies on actually being able to consolidate human preferences into utilities. I think there are a few stages here, each of which may present obstacles. I'm not sure what the current state of the art is with regard to overcoming these, and am curious regarding such.

First, here are a few assumptions that I'm using just to make the problem a bit more navigable (dealing with one or two hard problems instead of a bunch at once) - will need to go back and do away with each of these (and each combination thereof) and see what additional problems result.

The sovereign has infinite computing power (and to shorten the list of assumptions, can do 2-6 below)

We're maximizing across the preferences of a single human (Alice for convenience). To the extent that Alice cares about others, we're accounting for their preferences, too. But we're not dealing with aggregating preferences across different sentient beings, yet. I think this is a separate hard problem.

Alice has infinite computing power.

We're assuming that Alice's preferences do not change and cannot change, ever, no matter what happens. So as Alice experiences different things in her life, she has the exact same preferences. No matter what she learns or concludes about the world, she has the exact same preferences. To be explicit, this includes preferences regarding the relative weightings of present and future worldstates. (And in CEV terms, no spread, no distance.)

We're assuming that Alice (and the sovereign) can deductively conclude the future from the present, given a particular course of action by the sovereign. Picture a single history of the universe from the beginning of the universe to now, and a bunch of worldlines running into the future depending on what action the sovereign takes. To clarify, if you ask Alice about any single little detail across any of the future worldlines, she can tell you that detail.

Alice can read minds and the preferences of other humans and sentient beings (implied by 5, but trying to be explicit.)

So Alice can conclude anything and everything, pretty much (and so can our sovereign.) The sovereign is faced with the problem of figuring out what action to take to maximize across Alice's preferences. However, Alice is basically a sack of meat that has certain emotions...
Meetup : San Francisco Meetup: Board Games + Books giveaway
by rocurley — 2015-05-31T08:17:49.527Z

Discussion article for the meetup : San Francisco Meetup: Board Games + Books giveaway

WHEN:
01 June 2015 06:15:00PM (-0700)

WHERE:
1390 Market St., San Francisco, CA

We'll be meeting to hang out and play board games!
This is at our apartment because we have a table, call me at 301-458-0764 or tailgate to get in. As always, showing up late is fine.

Also, I've got a big pile of books, mostly scifi/fantasy, along with some EE, physics and math textbooks that I'm trying to give away. If you want some, please stop by!

Discussion article for the meetup : San Francisco Meetup: Board Games + Books giveaway
Group rationality diary, May 24th - June 13th
by Prismattic — 2015-05-31T03:41:41.803Z

This is the public group rationality diary for May 24th - June 13th, 2015. It's a place to record and chat about it if you have done, or are actively doing, things like:

Established a useful new habit

Obtained new evidence that made you change your mind about some belief

Decided to behave in a different way in some set of situations

Optimized some part of a common routine or cached behavior

Consciously changed your emotions or affect with respect to something

Consciously pursued new valuable information about something that could make a big difference in your life

Learned something new about your beliefs, behavior, or life that surprised you

Tried doing any of the above and failed

Or anything else interesting which you want to share, so that other people can think about it, and perhaps be inspired to take action themselves. Try to include enough details so that everyone can use each other's experiences to learn about what tends to work out, and what doesn't tend to work out.

Archive of previous rationality diaries

Note to future posters: no one is in charge of posting these threads. If it's time for a new thread, and you want a new thread, just create it. It should run for about two weeks, finish on a Saturday, and have the 'group_rationality_diary' tag.
Learning takes a long time
by JonahS — 2015-05-31T03:21:53.717Z

I recently realized that I had greatly underestimated the inferential distance between most of my readers and myself. Thinking it over, I realize that the bulk of the difference comes from a difference in perspectives on how long it takes to learn substantive things.

People often tell me that they're bad at math. I sometimes respond by saying that they didn't spend enough time on it to know one way or the other. I averaged ~25+ hours a week thinking about math when I was 16 and 17, for a total of ~2,500+ hours. I needed to immerse myself in the math to become very good at it, in the same way that I would need to live in French speaking country to get very good at French. If my mathematical activity had been restricted exclusively to coursework, I never would have become a good mathematician.

Math grad students who want to learn algebraic geometry often spend spend two years going through Hartshorne's dense and obscure textbook. it's not uncommon for students to learn interesting applications only after having gone through it. I find this practice grotesque, and I don't endorse it. I bring it up only to explain where I'm coming from. With the Hartshorne ritual as a standard practice, it's felt to me like a very solid achievement to present substantive material that readers can understand after only ~10 hours of reading and reflecting deeply.

It was so salient to me that one can't hope to become intellectually sophisticated without engaging in such activity on a regular basis that it didn't occur to me that it might not be obvious everyone. I missed the fact that most of my readers aren't in the habit of spending ~10 hours carefully reading a dense article and grappling with the ideas therein, so that even though I felt like I was making things accessible, I was still in the wrong ballpark altogether.

Thinking it over, I'm bemused by the irony of the situation. Even as I was exasperated by some readers' apparent disinclination to read articles very carefully and think about them deeply, I was blind to the fact that I was failing because I hadn't put thousands of hours into learning how to communicate to a general audience. Seeing how large my blindspot was made me realize "Oh... just as I had no idea how much time I need to put into developing my communication abilities to reach my readers, some of my rea...
Stupid Questions June 2015
by Gondolinian — 2015-05-31T02:14:09.409Z

This thread is for asking any questions that might seem obvious, tangential, silly or what-have-you. Don't be shy, everyone has holes in their knowledge, though the fewer and the smaller we can make them, the better.

Please be respectful of other people's admitting ignorance and don't mock them for it, as they're doing a noble thing.

To any future monthly posters of SQ threads, please remember to add the "stupid_questions" tag.
Rationality Quotes Thread June 2015
by Gondolinian — 2015-05-31T02:12:36.576Z

Another month, another rationality quotes thread. The rules are:

Please post all quotes separately, so that they can be upvoted or downvoted separately. (If they are strongly related, reply to your own comments. If strongly ordered, then go ahead and post them together.)

Do not quote yourself.

Do not quote from Less Wrong itself, HPMoR, Eliezer Yudkowsky, or Robin Hanson. If you'd like to revive an old quote from one of those sources, please do so here.

No more than 5 quotes per person per monthly thread, please.

Provide sufficient information (URL, title, date, page number, etc.) to enable a reader to find the place where you read the quote, or its original source if available. Do not quote with only a name.
Asymptotic Logical Uncertainty: The Benford Test
by Scott Garrabrant — 2015-05-31T00:30:37.000Z

This is the second post in a series on Asymptotic Logical Uncertainty.

All Turing Machines we discuss will have no input, and will output an infinite (or finite) sequence of bits. We let M(n) denote the nth bit output by the Turing machine M.

Let A(n)=n↑nn be the nth Ackermann number as defined in Conway and Guy's "The Book of Numbers."
Let ϕ1,ϕ2,… be a simple enumeration of sentences in first order logic over ZFC. Let L be a deterministic Turing machine such that L(n)=1 if and only if ϕn is provable with proof length at most A(n). We will be discussing Turing machines which are designed to try to guess the output of L in a very limited time. It will be clear later why we need L to be computable, and therefore must define L using bounded proofs.

Given a probabilistic Turing machine M, and a Time complexity function T, we define an infinite collection of dependent random variables MT(n) for n≥1.
MT(n) is defined to be M(n) if M outputs the first n bits in time at most T(n). If M does not output n bits in time T(n), then MT(n) is either 0 or 1 uniformly at random. These random variables are dependent, because we only run M once to determine MT(n) for all n.

Let M be a probabilistic Turing machine designed to try to guess the output of L in time at most T(n)=2n. Let S={s1,s2,…} be the subset of natural numbers defining the sentences ϕsn="The first digit of A(n) in base 10 is 1." Let B(n) be defined to be 1 if n is a power of 10, and 1log10 otherwise. We say that M satisfies the Benford test if

limn→∞|P(MT(sn)=1)−B(n)|=0.

The Benford test is a desired property, because we believe that ϕsn is always true when n is a power of 10, and otherwise is pseudorandomly true with probability 1log10. Formally, we are making the assumption that for any probabilistic Turing machine M, the probability that MT(si)=L(si) for all 1≤i≤n is in O(∏ni=1|L(si)+B(i)−1|).

The number 1log10 comes from Benford's Law. Intuitively, we expect that when n is not a power of 10, ϕsn is true 1log10 of the time, and A(n) is too large for there to be any procedure which predicts ϕsn better than randomly.

The Benford test is extremely easy to pass if you are allowed to hard code B(i) into your algorithm. What is more difficult is finding an algorithm which passes the test in a natural way.

The weak Benford test will refer to the version of the test where the algorithm is allowed to know that it will only be

...
Not because you "should"
by So8res — 2015-05-30T07:00:00.000Z
Link: https://mindingourway.com/not-because-you-should/

A few months ago, a friend of mine was describing her motivational issues to me. As an example, she explained she was having trouble making herself clean her room, despite her dissatisfaction with the constant messiness.

I asked: "Have you considered just not forcing yourself?"

She blinked, and cocked her head at me, and said "but then my room wouldn't get cleaned."

I called bullshit. Because look: either (a) you stop forcing yourself to clean the room, and you realize you don't actually care about having a clean room, and then your room stays messy and that's fine because you don't care; or (b) you stop forcing yourself to clean the room, and then you get a bit worried, because some part of you actually wants the room cleaned, so you listen to that part of yourself, and you work with it, and you find a time to clean the room because you want to.

Either way, you win. No need to use internal force.

This is a technique I've recommended before for motivational issues, and I recommend it again when dealing with shoulds. If you struggle with feelings of guilt, obligation, or inadequacy, then I strongly suggest the following remedy:

Just stop doing things because you "should".

As in, never let a "should" feel like a reason to do something. Only do things because they seem like the best thing to do after you've thought about it; never do things just because you "should."

A commenter to my last post said:

There's some meaning lost when you go from "I should X" to "If I X, I will achieve Y", which is "And I want to achieve Y enough to X, that's the best of the options."

I think this is mostly correct. Only mostly, because as far as I can tell most people don't tend to use "I should X" to mean "X is my best option." More frequently, I see people use it to mean "I would conclude that X is my best option if I knew more facts," or "I would conclude that X is my best option if I thought longer", or "I would conclude that X is my best option if I really cared about what I say I care about."

Regardless, all these various interpretations of "I should X" share one property: It's extremely difficult to make these claims about X while you're still deliberating.

If you ever happen to figure out which option is best, then don't slap the label "should" on it and go back to thinking about your options! If you know what the best option is, then stop deliberating and do it.

After the fact, lookin...
Group Bragging Thread (May 2015)
by Viliam — 2015-05-29T22:36:27.000Z

This is similar to the usual bragging threads, with one major exception: this thread is for groups, not individuals.

Please comment on this thread explaining awesome things that you have done with your fellow rationalists as a group. The lower bound on group size is three people.

Otherwise the rules are analogical: be as blatantly proud of your group as you feel, consider your group the coolest freaking group ever because of that awesome thing you're dying to tell everyone about. This is the place to do just that. - No work in progress, no proposals, only the awesome things you have already done.

And because this is the first such thread, feel free to write about anything that happened since the extinction of dinosars until the end of May 2015.

(Yes, organizing Less Wrong meetups is a valid example of an activity that belongs here, if at least three people participated. Please try to include more details than merely "we organized a LW meetup in <city_name>".)
Meetup : West LA: Wait a minute... just what is bias?
by abramdemski — 2015-05-29T22:18:36.265Z

Discussion article for the meetup : West LA: Wait a minute... just what is bias?

WHEN:
03 June 2015 07:00:00PM (-0700)

WHERE:
11066 santa monica blvd, la, ca

How to Find Us: Go into this Del Taco. We will be in the back room if possible.

Parking is free in the lot out front or on the street nearby.

Discussion: The definition of bias in statistics is significantly different from the definition used on LessWrong. Why is this? What are the differences? It turns out that the scientific literature on biases uses multiple notions of bias. This muddies the research. Applying different statistical tools, psychologists can use the same dataset to "prove" that people are both overconfident and underconfident. This statistical phenomenon is closely related to why the tails come apart.

Recommended Reading:

Why the Tails Come Apart

Simultaneous Overconfidence and Underconfidence (pdf)

What Exactly Is Bias

...What's a Bias, Again?

No prior exposure to Less Wrong is required; this will be generally accessible.

Discussion article for the meetup : West LA: Wait a minute... just what is bias?
Learning to get things right first time
by owencb — 2015-05-29T22:06:56.171Z

These are quick notes on an idea for an indirect strategy to increase the likelihood of society acquiring robustly safe and beneficial AI.

Motivation:

Most challenges we can approach with trial-and-error, so many of our habits and social structures are set up to encourage this. There are some challenges where we may not get this opportunity, and it could be very helpful to know what methods help you to tackle a complex challenge that you need to get right first time.

Giving an artificial intelligence good values may be a particularly important challenge, and one where we need to be correct first time. (Distinct from creating systems that act intelligently at all, which can be done by trial and error.)

Building stronger societal knowledge about how to approach such problems may make us more robustly prepared for such challenges. Having more programmers in the AI field familiar with the techniques is likely to be particularly important.

Idea: Develop methods for training people to write code without bugs.

Trying to teach the skill of getting things right first time.

Writing or editing code that has to be bug-free without any testing is a fairly easy challenge to set up, and has several of the right kind of properties. There are some parallels between value specification and programming.

Set-up puts people in scenarios where they only get one chance -- no opportunity to test part/all of the code, just analyse closely before submitting.

Interested in personal habits as well as social norms or procedures that help this.

Daniel Dewey points to standards for code on the space shuttle as a good example of getting high reliability code edits.

How to implement:

Ideal: Offer this training to staff at software companies, for profit.

Although it’s teaching a skill under artificial hardship, it seems plausible that it could teach enough good habits and lines of thinking to noticeably increase productivity, so people would be willing to pay for this.

Because such training could create social value in the short run, this might give a good opportunity to launch as a business that is simultaneously doing valuable direct work.

Similarly, there might be a market for a consultancy that helped organisations to get general tasks right the first time, if we knew how to teach that skill.

More funding-intensive, less la

...
What If It Rains?
by Zvi — 2015-05-29T21:29:04.000Z

Follow Up To: Full Service

Gas stations in New Jersey do not have awnings to protect customers from the rain. Let us presume for the moment that due to some combination of inertia, regulations and initial costs, it is not practical for them to create such awnings in the short term. Now suppose that New Jersey legalizes self-service. 90% of the time it is not raining, and the customers gain $0.25 of net utility from the cheap gas that comes from self-service. However, 10% of the time it is raining or snowing or something else unpleasant, and in those cases customers would on average be $5 better off if someone pumped their gas for them so they didn’t have to get out of the car. Some even get to avoid ruining their $60 haircuts.

A few stations switch to self-service. Ninety percent of the time, the sun is shining, and most customers choose to save a marginal amount of money and go to the self-service stations. With business drying up, more and more stations switch over, until most stations are self-service.

Then the rain comes, and everyone gets wet and miserable. The customers complain about how good things were in the old days with full service and say there ought to be a law!

One chain tries the obvious, which is to stay full-service, take a small loss when the sun is shining, and then charge an extra $4 per customer when it rains to make up for it. The customers go ballistic. Profiteer! Exploiter! Price gouger! Boycotts are announced and state senators are lobbied. Governor Christie warns that this is illegal activity under his interpretation of New Jersey law, the same way gas stations were not allowed to raise their prices after hurricane Sandy resulting in no one being able to buy gas without huge lines. The public cheers and his presidential campaign gets a boost, and the last of the full service branches go down. Self-service wins, everyone is miserable, and legislation gets reintroduced to return New Jersey to full service. Christie boasts that the repeal, passed over his veto, was a huge mistake, and adds mandatory self-service to his national platform, but loses anyway because he’s Christie.

What went wrong? Fundamentally, the problem is that consumers are unwilling to let prices change to meet supply and demand. Allocating scarce resources to those who value them most, and rewarding those who provide those resources so that people are encouraged to kee...
"Immortal But Damned to Hell on Earth"
by Bound_up — 2015-05-29T19:55:33.681Z

http://www.theatlantic.com/technology/archive/2015/05/immortal-but-damned-to-hell-on-earth/394160/

With such long periods of time in play (if we succeed), the improbable hellish scenarios which might befall us become increasingly probable.

With the probability of death never quite reaching 0, despite advanced science, death might yet be inevitable.

But the same applies also to a hellish life in the meanwhile. And the longer the life, the more likely the survivors will envy the dead. Is there any safety in this universe? What's the best we can do?
Misunderstandings of not understanding
by KatjaGrace — 2015-05-29T18:19:12.000Z

There are probably lots of attitudes to a thing we might call ‘understanding’ it. Here are examples of two:

“I understand combustion”

“I understand her feelings”

In the first statement, the person has a good explicit model of combustion, and the range of ways it can happen, and what causes it, and that sort of thing. In the second, her feelings feel natural and comprehensible to the speaker. T might have an intuitive sense for how her feelings will evolve, but they probably don’t have an explicit model of the psychology, biology or game theoretic reasons behind her feelings.

These different instances of ‘understanding’ probably vary on a few different dimensions that could be separated, but this is going to be a short blog post.

One thing that separates these kinds of understanding is their objects. People rarely say “I understand combustion” to mean that it feels intuitively natural to them, in the absence of any explicit model. And they rarely say “I understand her feelings” to mean that can answer questions about her alien neural chemistry.

There are some things where both kinds of understanding sometimes apply though. For instance “I don’t understand art” can mean “when I look at art, it doesn’t feel especially natural or good to me” or it can mean “I do not know what caused humans to like art so much”. The seconds speaker may love art.

I think failure to distinguish these kinds of understanding causes misunderstanding. For instance, suppose you say you don’t understand why people go out to bars. This could mean you don’t enjoy doing so personally, or that you don’t see why bars in particular evolved to be a big thing while public swimming pools full of tiny balls did not. If the two aren’t really distinguished in people’s minds, then they will suppose you are saying both, even once your elaboration strongly implies the latter.

Perhaps this is a reasonable shorthand, because in practice the only people who wonder why humans gather at bars in particular are those who aren’t enjoying bars enough. But I think it confuses things (for instance, impressions of how fun I am).

A particularly costly kind of misunderstanding is when people themselves assume they are saying both, because they think there is just on...