trentmkelly/LessWrong-43k huggingface dataset:
Consider Reconsidering Pascal's Mugging
by Rafael Harth — 2018-01-03T00:03:32.358Z

[Epistemic Status: confident considering the outside view]

The Pascal’s Mugging dilemma is this: a random person walks up to you in the street and says that if you don’t give them a dollar, they’ll destroy the earth tomorrow. Do you pay? Since the probability of them doing so cannot plausibly be low enough to make giving the dollar negative expected utility, decision theory (whether causal, evidential, or functional) says you do.

The dominant view is

Paying is obviously wrong. Therefore, this is a yet unsolved problem. There is probably a theoretical insight by which Decision Theory should be amended to avoid this behavior.

But this has never made sense to me. My view has always been

Decision Theory is almost certainly going to produce the correct output on this problem because it is conceptually simple. Therefore, either there are reasons why decision theory doesn’t actually tell you to pay, or paying is correct.

It’s strange to me that the intuition “you shouldn’t pay” is apparently valued much more highly than the intuition “FDT is correct,” to the point that the idea that it could possibly be correct to pay isn’t even on the table. So far, I have never read a satisfying way to deal with this problem, which strengthens my suspicion that there is none. Moreover, the ideas which I have seen mostly seem very misguided to me, particularly anything that involves rounding small numbers to zero or treating utility as non-linear. Therefore-

Objection!

... yes?

If your decision theory pays, then you can be exploited heavily, by being mugged repeatedly.

No I can’t. The probability that the mugger is telling the truth doesn’t plausibly increase if they ask multiple times, and the cost of being mugged repeatedly is high. A single dollar already has a chance to save the earth in other ways, and that chance increases at a roughly linear pace with repeated asks, both in the literal case as a human and in the metaphorical case as an aspiring AI. Finally, there is the fact that appearing muggable is negative utility.

Yeah, but that’s just a hack, and if your argument relies on this, then that’s terrible.

Why?

I don’t personally feel uncomfortable paying, or having an AI that would pay in such a scenario. As I said, the probability doesn’t increase with repeated asks, so that’s very similar to just asking for more in the first place.

So what if they do ask for more in the first place? ...
A Simple Two-Axis Model of Subjective States, with Possible Applications to Utilitarian Problems
by moridinamael — 2018-01-02T18:07:20.456Z

At the beginning of last year, I noticed that neuroscience is very confused about what suffering is. This bothered me quite a lot, because I couldn't see how Utilitarianism or even Consequentialism could ever be coherent without a coherent understanding of suffering.

The following is my simple model that has helped me think through some issues and clarified my intuitions a bit. I am both posting this to share what I feel is a useful tool, and inviting criticism.

"Utilitarianism" (which term I will use to refer to all applicable formulations of Utilitarianism) assumes a space of subjective states that looks like a number line.

Suffering is bad. Joy is good. They are negatives of each other, and straightforwardly additive.

This has always struck me as nonsense, not at all reflective of my inner experience, and thus a terrible basis for extrapolating ethics over populations.

It feels more natural to treat Joy and Suffering as two axes. There may or may not be possible mappings between them depending on context. It does not seem natural to assume they are fully orthogonal, but there is also no reason to assume any particular mapping. There is particularly no reason to assume the mapping is anything like "+1 suffering is identical to -1 joy". There is extra-super-no-reason to assume that there's some natural definition of a vector magnitude of joy-suffering that is equivalent to "utility".

If you're having a nice conversation with an old friend you haven't seen in a while and you're feeling good, it's likely that your subjective sense of suffering is quite low and your subjective sense of joy is quite high.

If you have a bad headache and you're lying in bed trying to rest, your subjective sense of suffering is pretty high, and your level of joy is quite low.

What interests me is the "mixed" scenario, where your friend is only in town briefly, and you happen to have a headache. You're happy to be able to visit with them, so there's a definite quality of joy to your subjectivity. You wish you didn't have a headache, but the headache doesn't somehow negate the joy in being able to talk to them. It's possible that forcing yourself to talk to them, rather than lie in bed, is making the headache worse, thus increasing the "suffering" component of your experience, but this is in a very important sense worth it because of the joy premium.

It's also entirely possible that the conv...
The Loudest Alarm Is Probably False
by orthonormal — 2018-01-02T16:38:05.748Z

Epistemic Status: Simple point, supported by anecdotes and a straightforward model, not yet validated in any rigorous sense I know of, but IMO worth a quick reflection to see if it might be helpful to you.

A curious thing I've noticed: among the friends whose inner monologues I get to hear, the most self-sacrificing ones are frequently worried they are being too selfish, the loudest ones are constantly afraid they are not being heard, the most introverted ones are regularly terrified that they're claiming more than their share of the conversation, the most assertive ones are always suspicious they are being taken advantage of, and so on. It's not just that people are sometimes miscalibrated about themselves- it's as if the loudest alarm in their heads, the one which is apt to go off at any time, is pushing them in the exactly wrong direction from where they would flourish.

Why should this be? (I mean, presuming that this pattern is more than just noise and availability heuristic, which it could be, but let's follow it for a moment.)

It's exactly what we should expect to happen if (1) the human psyche has different "alarms" for different social fears, (2) these alarms are supposed to calibrate themselves to actual social observations but occasionally don't do so correctly, and (3) it's much easier to change one's habits than to change an alarm.

In this model, while growing up one's inner life has a lot of alarms going off at various intensities, and one scrambles to find actions that will calm the loudest ones. For many alarms, one learns habits that basically work, and it's only in exceptional situations that they will go off loudly in adulthood.

But if any of these alarms don't calibrate itself correctly to the signal, then they eventually become by far the loudest remaining ones, going off all the time, and one adjusts one's behavior as far as possible in the other direction in order to get some respite.

And so we get the paradox, of people who seem to be incredibly diligently following the exact wrong advice for themselves, analogous to this delightful quote (hat tip Siderea) about system dynamics in consulting:

People know intuitively where leverage points are. Time after time I’ve done an analysis of a company, and I’ve figured out a leverage point — in inventory policy, maybe, or in th

...
My Predictions for 2018 (& a Template for Yours)
by Zachary Jacobi — 2018-01-02T01:19:12.373Z

For the past year, I've been eagerly awaiting the start of 2018. This wasn't out of any dissatisfaction with 2017, or because of any specific events that I expected to happen today. No, I was excited for 2018 because it meant I got to grade my 2017 predictions and make predictions for 2018.

I've found this whole process great fun and would like to try and spread the habit. This post will include my graded 2017 predictions, templates so that you can make your own predictions, my calibration calculations, and my new predictions for 2018.

If you want to be able to look forward to the start of 2019 like I looked forward to today, or if you want to be able to make a cool graph (like this one!)

I recommend trying out my 2018 prediction templates before reading (to avoid being anchored by anything I write). The templates focus on international events and come in two versions:

Long (which assumes you read global news a lot)
Short (which is less demanding)

With both these sheets, the idea is to pick a limited number of probabilities (I recommend 51%, 60%, 70%, 80%, and 90%) and assign one to each item that you have an opinion on. At the end of the year, you count the number of correct items in each probability bin and use that to see how close you were to ideal. This gives you an answer to the important question: "when I say something is 80% likely to happen, how likely, really, is it to happen?"

You can also make your own (or use the set of questions Slate Star Codex normally uses). If you do make your own, please link your post (and maybe also your template?) in the comments or post it to the front page. It's my hope that this post can serve as a convenient place for the LW community to look at the predictions of everyone who wants to participate in this experiment!

2017 Predictions

The following are my predictions for 2017, which were posted here.

Canada

Trudeau ends the year with a lower approval rating than he started – 60%
No bill introduced that changes the electoral system away from first past the post in 2019 – 50%
No referendum scheduled on changing the electoral system away from first past the post before 2019 – 70%
A bill legalizing marijuana is passed by the House of Commons – 90%
The senate doesn’t block attempts to legalize marijuana – 80%
At least one court finds the assisted dying bill isn’t in line with Carter v Canada – 60%
Ontari

...
Object level weight loss tips
by Elo — 2018-01-01T22:15:01.427Z

Original post: bearlamp.com.au/object-level-weight-loss-tips/

.bearlamp.com.au/object-level-weight-loss-tips/

Finding specific instructions on weight loss is difficult. They are all buried in the internet. Covered in clickbait and a waste of time to find. And plenty of them don't work. So here's my list. As object level as I can.

Weight loss is hard. Life is busy. But it's do-able

1. Make a list of healthy foods that you eat.

To me this looks like this:

lettuce
mushrooms
cottage cheese
white raddish
vegetables
leeks
pickles
onions
peas
carrots
...

This list specifically excludes asparagus and a few other green vegetables that I don't really eat despite them being supposedly good for me.

Now when you go shopping, only buy things on that list. Use an app to keep track of the list.

2. fix the "ooh it's cheap, it's on special I should buy it" habit.

I care more about health than I do about cheap products. Yet somehow I can still get tricked by a half price chocolate, or various other foods discounted. Fix the part of your brain that makes this mistake. Accept the better argument. Optimise for health over expense.

3. Remove unhealthy things from your house

If you eat it and it's unhealthy and as of this instant you don't want to - it's okay to throw it away. It's also okay to give it away. But ideally throwing it away is better.

4. Eat healthy as a baseline.

If you are at home, just eating the normal meals, they need to be the right things. If they are not right then you are probably on a general weight gain trajectory. Fix the basics first. If necessary learn to cook some healthy meals. You probably already know enough about what to cook.

5. Stop wasting food.

You know what would be a waste. It would be a waste to throw out that food because it's food. But you know what else would be a waste. It would be a waste to eat it when you don't need to. It would be wasteful to your waistline. When it's bad for you and going to make you more unhealthy. That would be a waste too. And that's the more embarrassing waste.

6. When you go out to food with friends.

Don't buy the "cheapest thing on the menu". Don't buy the "most delicious thing on the menu". Buy the "healthiest thing on the menu". If that's not healthy enough, don't buy anythi...
Suggest short Sequence readings for my college stat class
by palladias — 2018-01-01T19:46:57.655Z

I'm teaching one section of an intro-to-stat-for-non-math-folks class this spring as an adjunct. I'm planning to supplement the dry, fire-hose-y textbook with some outside readings (data journalism written by me, where students can reconstruct the analysis; excerpts from The Lady Tasting Tea; and some short posts from the Sequences).

On that last category, I'd love suggestions. I have a few in mind already ("Look into the Dark", the 'Lightness' section of "The 12 Virtues of Rationality", "Making Beliefs Pay Rent").

What would you suggest I pay special attention to? Also, anything you'd suggest as particularly good for pre-reading before the first class?
From Rationality to Power in 3 Steps
by Bound_up — 2018-01-01T15:56:54.173Z

I'm not at all confident I'm carving reality at its joints here, but this has been my path, so there's got to be something of reality mixed up in it somewhere.

Rationality, general learning ability
Social mechanics, tactics
Political dynamics, strategy

First, I learned to think clearly. While that gave me the potential to be good at anything, since it is a general learning multiplier, I was naturally, not immediately good at everything. On the contrary, learning rationality overwrote a lot of of my old instincts. While some of those instincts were completely valueless, others were a mix of good and bad. Overwriting them granted me many advantages, but also undid some good, particularly in social areas.

Second, I turned my fany new general learning ability on social mechanics, in time returning to my old social ability, and then surpassing it. Additionally, I'm doing so on purpose and by design, rather than accidentally. It's been said that a gentleman is one who never offends on accident, and now, I am one who (less often, anyway) doesn't commit faux pas by accident. Where once I was exasperatingly clueless, now I am a mixture of charm and purposeful offense. It's great!

Third, and this is really only just beginning, because I'm really only about halfway done with the second step, by my estimation, I'm beginning to perceive how all the individual social rules and possible maneuvers combine. I begin to see how whole groups are influenced, what governs them, how to squeeze power out of them. This has helped me appreciate that there are efforts to make people into useful contributors to groups, like cogs in a machine, in order to empower those groups, in order to create a store of political capital to be taken advantage of by those who know how to make withdrawals.

And that led me to realize that the teaching of rationality directly undoes these efforts. It is much harder to manipulate rationalists on the sly. Direct threats and rewards and so on are still available, of course, but you can't, to use one trick as an example, change the definition of a word and think that a proper rationalist will act any differently as a result. A rationlist would see through the illusory web of words to the essence beyond, to the concepts at play and how they relate to each other, with no import placed on which word-labels are used to...
Zvi Will Be In San Francisco January 15-21
by Zvi — 2018-01-01T13:50:00.333Z

Happy New Year, everyone!

I will be taking a week’s vacation (without the family) in San Francisco this month, flying in on January 15 and leaving on January 21. Some of that time will be used to get fresh air, relax and refresh, but the main goal is to connect with as many friends as possible, new and old, and catch up on all that is happening. I come seeking knowledge, friendship and advice, perhaps opportunity, and offering all of them in return.

Great things are afoot. Other great things should be.

I don’t know exactly where I will be staying yet, but I anticipate spending most nights in Berkeley.

If you would like to get together during my visit, let me know! Email is best (it’s the***@gmail.com, where *** is my first name) is always best if you want to contact me privately. If you’d like to do a public thing, the comments here are open.
AI Goal Alignment Entry: How to Teach a Computer to Love
by ExCeph — 2018-01-01T03:37:23.626Z

How to Teach a Computer to Love

Foreword

After weeks and weeks of generating and organizing ideas, I’ve concluded that I do not have a complete answer to the problem of AI goal alignment. However, I believe I have some important concepts that provide essential starting points for addressing it. Some of these concepts are ones I’ve mentioned before on my blog, but this is the first time I’ve used them in the context of artificial general intelligence. I am still uncertain which concepts are going to be challenged by this community and which concepts will be accepted as pre-established, but I’ll respond to questions as they come. I hope these ideas prove helpful to others who are working on this problem.

Introduction

It is difficult to overestimate the importance to humanity of paying close attention to the creation of artificial general intelligence (AGI, or generalized AI). With the ability to upgrade its software and hardware to make itself more able to solve problems and overcome obstacles, an AGI is extremely likely to become the dominant power in our world (as elaborated in Nick Bostrom’s Superintelligence). Due to this probable dominance, and its self-alteration ability, an AGI could very well become similar to a member of the Fair Folk (the faeries of lore from the British Isles): immensely powerful and potentially quite alien in its motivations. In order to somehow bind the will of such a being so that it does not decide to enslave or wipe out humanity, we need to understand several important and fundamental concepts.

To get a trivial argument out of the way, we can't use hardwired rules to ensure that an AGI protects humanity and its goals in any meaningful sense. Rules are based on semantics, and semantics are simplified representations of some aspect of reality. In any semantic system, no matter how well it is clarified, there is still ambiguity, and an agent unfamiliar with or uninterested in the purpose of the system can interpret the ambiguity in ways that defy what the principals intended, especially if it is creative enough to solve problems humans cannot. (Isaac Asimov deliberately demonstrates this well in his works. "Protect humans"? What is "protect"? What is "human"? How does it prioritize if it cannot protect everyone?) In addition, the AGI is virtually guaranteed to be able to exploit the inherent ambigui...
12/31/17 Update: Frontpage Redesign
by Ben Pace — 2018-01-01T03:21:11.408Z

Hey folks, Oli just pushed an update to the frontpage, which includes RSS feeds. I'll list the main changes here:

There are no longer two sections for Featured content and Frontpage posts
There is just one section, called 'Recent Posts', with some preset filters for you to choose from. These are: Curated, Frontpages, Community, Meta, and Daily.

Curated is just posts that mods have curated (what we had been calling 'featured')
Frontpage is all frontpage posts
Community is all the personal blogs as well as frontpage posts.
i.e. Curated is a subset of Frontpage is a subset of Community

If you're logged out, it will automatically show you just the Curated Content, and if you're logged in, it will remember whatever you had on last time you visited the page
When you're viewing a given filter, you'll see a little icon next to the filter that has the RSS button. If you click on this, you'll get a dialog box that let's you set the karma threshold for posts you want to see

i.e. you get an RSS link that only sends you posts that hit over x karma, where you set 0≤x≤100

There are also little icons on posts that distinguish Curated posts, Personal Blogposts, and Meta posts.

Also, posts on the frontpage are just a link to the post. I found users had issues of intending to click on the post and accidentally going to the author page, or voting on it. Now if you click on it anywhere, it just takes you to the post page.

(That's a lie, there's still a comment button on the far right that takes you straight to the comment section, but that's it.)

That's all folks! Best of skill for the new year, and I'll see some of you at the 80k pillow fight tonight.
January 2018 Media Thread
by ArisKatsaris — 2018-01-01T02:11:43.687Z

This is the monthly thread for posting media of various types that you've found that you enjoy. Post what you're reading, listening to, watching, and your opinion of it. Post recommendations to blogs. Post whatever media you feel like discussing! To see previous recommendations, check out the older threads.

Rules:

Please avoid downvoting recommendations just because you don't personally like the recommended material; remember that liking is a two-place word. If you can point out a specific flaw in a person's recommendation, consider posting a comment to that effect.

If you want to post something that (you know) has been recommended before, but have another recommendation to add, please link to the original, so that the reader has both recommendations.

Please post only under one of the already created subthreads, and never directly under the parent media thread.

Use the "Other Media" thread if you believe the piece of media you want to discuss doesn't fit under any of the established categories.

Use the "Meta" thread if you want to discuss about the monthly media thread itself (e.g. to propose adding/removing/splitting/merging subthreads, or to discuss the type of content properly belonging to each subthread) or for any other question or issue you may have about the thread or the rules.