trentmkelly/LessWrong-43k huggingface dataset:
Existential Risk and Existential Hope: Definitions
by owencb — 2015-01-10T19:09:43.882Z

I'm pleased to announce Existential Risk and Existential Hope: Definitions, a short new FHI technical report.

Abstract:

We look at the strengths and weaknesses of two existing definitions of existential risk, and suggest a new definition based on expected value. This leads to a parallel concept: ‘existential hope’, the chance of something extremely good happening.

I think MIRI and CSER may be naturally understood as organisations trying to reduce existential risk and increase existential hope respectively (although if MIRI is aiming to build a safe AI this is also seeking to increase existential hope). What other world states could we aim for that increase existential hope?
Productivity poll: how frequently do you think you *should* check email?
by tog — 2015-01-10T16:36:57.680Z

How frequently do you think you *should* check email? You can also say how frequently you do in comments.

Personally I'm sold on thinking you should check it around once a day, not necessarily without fail. That increases focus on both email and non-email, and minimises getting sucked into distractions. But some people I know disagree. Some believe in getting notifications whenever a new email comes in.

For anyone who'd like to check email less often and uses GMail, I recommend using http://inboxpause.com/ and this full screen compose link: https://mail.google.com/mail/u/0/?ui=2&view=cm&fs=1&tf=1&shva=1

Edited to add: I'd recommend everyone at least try checking only once a day, at least for a few days, to see if you find it more productive and/or relaxing. That'd be a big enough win to make experimenting worthwhile.
Meetup : Montreal Effective Altruism: Global Poverty
by bartimaeus — 2015-01-10T15:38:28.607Z

Discussion article for the meetup : Montreal Effective Altruism: Global Poverty

WHEN:
22 January 2015 10:38:11AM (-0500)

WHERE:
1720 st-denis, montreal

There are a few causes that have the potential to do lots of good in the world per dollar spent. Global poverty is a problem that is:

-Neglected (so there are lots of low-hanging fruit)
-Large-scale (billions are affected by it)
-Progress is easy to measure

We'll talk about the problem, what some of the most effective solutions are and which organisations seem to be doing the best work towards improving the conditions of the poorest people on the planet.

Discussion article for the meetup : Montreal Effective Altruism: Global Poverty
Meetup : Less Wrong Israel Meetup: Social and Board Games
by DanArmak — 2015-01-10T09:48:33.654Z

Discussion article for the meetup : Less Wrong Israel Meetup: Social and Board Games

WHEN:
13 January 2015 07:00:00PM (+0200)

WHERE:
Google, Electra Tower, 98 Yigal Alon Street, Tel Aviv

This time we're going to have a social meetup! It's going to be a game night full of people talking about physics, friendly AI, and how to effectively save the world.
Please bring any games you'd like to play.
Contact: If you can't find us, call Anatoly, who is graciously hosting us, at 054-245-1060; or Joshua at 054-569-1165.
There's a Facebook event for the meetup. Please RSVP if you have Facebook, to help us get a sense of what to expect, and you can also join the LessWrong Israel Facebook group where we publish new events.

Discussion article for the meetup : Less Wrong Israel Meetup: Social and Board Games
"Evil" decision problems in provability logic
by Benya_Fallenstein — 2015-01-10T01:04:28.000Z

A while ago, drnickbone showed in a LessWrong post that for any decision theory, there is a "fair" decision problem on which that decision theory fails---that is, gets a low reward, even though a simple alternative decision theory gets a high reward. Although drnickbone's argument was given in words, it's intuitively clear that it could be formalized as math---except, what exactly do "decision theory" and "decision problem" mean in this context?

In this post, I'll propose definitions of these notions in the context of provability logic, as used in Vladimir Slepnev's modal version of UDT, and give a formalization of drnickbone's result. This implies that no single modal decision theory, including UDT, can behave optimally on all "fair" decision problems. But in my next post, I'll show that modal UDT is nevertheless optimal in a weaker sense: For every provably "fair" modal decision problem, there is a finite set of true axioms that can be added to PA, such that modal UDT using this extension of PA performs optimally on that decision problem. (This is a modal logic version of a result which Kenny Easwaran, Nate Soares and I recently showed in the context of bounded proof length, and which hasn't been published yet.)

Prerequisite: A primer on provability logic.

Background

Here's an example of how we usually talk about decision problems (Vladimir Slepnev's formalization of Newcomb's problem from "A model of UDT with a halting oracle"):

def U():
# Fill boxes, according to predicted action.
box1 = 1000
box2 = 1000000 if (A() == 1) else 0
# Compute reward, based on actual action.
return box2 if (A() == 1) else (box1 + box2)

In other words, we have a universe program, U(), which makes calls to an agent program, A().

The agent program, in turn, uses quining to refer to the source code of the universe program. But while the universe is allowed to call the agent to figure out how it will behave, the agent can't just call the universe---or we'll have an infinite regress. Instead, we generally consider agents that use abstract reasoning in the form of proofs to reason about the universe they're in.

So a universe is a parameterless program, and a decision problem is a universe with a slot in it for an agent program. Similarly, an agent is a parameterless program, and a decision theory is an agent with a slot in it for the universe program. The universe

...
New LW Meetups: Paris, San Francisco
by FrankAdamek — 2015-01-10T00:01:56.848Z

This summary was posted to LW main on January 2nd. The following week's summary is here.

New meetups (or meetups with a hiatus of more than a year) are happening in:

Paris LW Meetup - LHC Exhibit: 17 January 2015 02:00PM
San Francisco Meetup: 12 January 2015 06:00PM

Irregularly scheduled Less Wrong meetups are taking place in:

Bangalore Meetup: 10 January 2015 11:02AM
European Community Weekend 2015: 12 June 2015 12:00PM
[Frankfurt] New Year meetup Frankfurt: 11 January 2015 02:00PM
[Munich] January Meetup in Munich: 17 January 2015 03:00PM
Utrecht: a critique of effective altruism: 04 January 2015 02:00PM
Warsaw January Meetup: 13 January 2015 06:00PM

The remaining meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

Austin, TX - Caffe Medici: 03 January 2026 01:30PM
London First 2015 Meetup, 04/01/2015: 04 January 2015 02:00PM
[Melbourne] January 2015 Rationality Dojo - How to learn faster and teach more effectively and teaching: 04 January 2015 03:30PM
Vienna: 24 January 2015 03:00PM
West LA—What Is FAI?: 07 January 2015 07:00PM

Locations with regularly scheduled meetups: Austin, Berkeley, Berlin, Boston, Brussels, Buffalo, Cambridge UK, Canberra, Columbus, London, Madison WI, Melbourne, Moscow, Mountain View, New York, Philadelphia, Research Triangle NC, Seattle, Sydney, Toronto, Vienna, Washington DC, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers.

If you'd like to talk with other LW-ers face to face, and there is no meetup in your area, consider starting your own meetup; it's easy (more resources here). Check one out, stretch your rationality skills, build community, and have fun!

In addition to the handy sidebar of upcoming meetups, a meetup overview is posted on the front page every Friday. These are an attempt to collect information on all the meetups happening in upcoming weeks. The best way to get your meetup featured is still to use the Add New Meetup feature, but you'll also have the benefit of having your meetup mentioned in a weekly overview. These overview posts are moved to the discussion section when the new post goes up.

Please note that for your meetup to appear in the weekly meetups feature, you need to post your meetup before the F...
Explaining “map and territory” and “fundamental attribution error” to a broad audience
by Gleb_Tsipursky — 2015-01-09T17:57:35.863Z

I am working on a blog post that aims to convey the concepts of “map and territory” and the “fundamental attribution error” to a broad audience in an engaging and accessible way. Since many people here focus on these subjects, I think it would be really valuable to get your feedback on what I’ve written.

For a bit of context, the blog post is part of the efforts of Intentional Insights to promote rational thinking to a broad audience and thus raise the sanity waterline, as described here. The target audience for the blog post is reason-minded youth and young adults who are either not engaged with rationality or are at the beginning stage of becoming aspiring rationalists. Our goal is to get such people interested in exploring rationality more broadly, eventually getting them turned on to more advanced rationality, such as found on Less Wrong itself, in CFAR workshops, etc. The blog post is written in a style aimed to create cognitive ease, with a combination of personal stories and an engaging narrative, along with citations of relevant research and descriptions of strategies to manage one’s mind more effectively.

This is part of our broader practice of asking for feedback from fellow Less Wrongers on our content (this post for example). We are eager to hear from you and revise our drafts (and even published content offerings) based on your thoughtful comments, and we did so previously, as you see in the Edit to this post.

Below the line is the draft post itself. After we get your suggestions, we will find an appropriate graphic to illustrate this article and post it on the Intentional Insights website. Any and all suggestions are welcomed, and thanks for taking the time to engage with us and give your feedback – much appreciated!

_______________________________________________________________________________________________________________________

Where Do Our Mental Maps Lead Us Astray?

So imagine you are driving on autopilot, as we all do much of the time. Suddenly the car in front of you cuts you off quite unexpectedly. You slam your brakes and feel scared and indignant. Maybe you flash your lights or honk your horn at the other car. What’s your gut feeling about the other driver? I know my first reaction is that the driver is rude and obnoxious.

Now imagine ...
Questions of Reasoning under Logical Uncertainty
by So8res — 2015-01-09T17:37:54.807Z

I'm pleased to announce a new paper from MIRI: Questions of Reasoning Under Logical Uncertainty.

Abstract:

A logically uncertain reasoner would be able to reason as if they know both a programming language and a program, without knowing what the program outputs. Most practical reasoning involves some logical uncertainty, but no satisfactory theory of reasoning under logical uncertainty yet exists. A better theory of reasoning under logical uncertainty is needed in order to develop the tools necessary to construct highly reliable artificial reasoners. This paper introduces the topic, discusses a number of historical results, and describes a number of open problems.

Following Corrigibility and Toward Idealized Decision Theory, this is the third in a series of six papers motivating MIRI's technical research agenda. This paper mostly motivates and summarizes the state of the field, and contains one very minor new technical result. Readers looking for more technical meat can find it in Paul Christiano's paper Non-Omniscience, Probabilistic Inference, and Metamathematics, published mid-2014. This paper is instead intended to motivate the study of logical uncertainty as relevant to the design of highly reliable smarter-than-human systems. The introduction runs as follows:

Consider a black box with one input chute and two output chutes. The box is known to take a ball in the input chute and then (via some complex Rube Goldberg machine) deposit the ball in one of the output chutes.

An environmentally uncertain reasoner does not know which Rube Goldberg machine the black box implements. A logically uncertain reasoner may know which machine the box implements, and may understand how the machine works, but does not (for lack of computational resources) know how the machine behaves.

Standard probability theory is a powerful tool for reasoning under environmental uncertainty, but it assumes logical omniscience: once a probabilistic reasoner has determined precisely which Rube Goldberg machine is in the black box, they are assumed to know which output chute will take the ball. By contrast, realistic reasoners must operate under logical uncertainty: we often know how a machine works, but not precisely what it will do.

General intelligence, at the human level, mostly consists of reasoning that involves logical uncertainty. Reasoning about the output of a c...
Memes and Rational Decisions
by inferential — 2015-01-09T06:42:02.831Z

In 2004, Michael Vassar gave the following talk about how humans can reduce existential risk, titled Memes and Rational Decisions, to some transhumanists. It is well-written and gives actionable advice, much of which is unfamiliar to the contemporary Less Wrong zeitgeist.

Although transhumanism is not a religion, advocating as it does the critical analysis of any position; it does have certain characteristics which may lead to its identification as such by concerned skeptics. I am sure that everyone here has had to deal with this difficulty, and as it is a cause of perplexity for me I would appreciate it if anyone who has some suggested guidelines for interacting honestly with non-transhumanists share them at the end of my presentation. It seems likely to me that each of our minds contains either meme complexes or complex functional adaptations which have evolved to identify “religious” thoughts and to neutralize their impact on our behavior. Most brains respond to these memes by simply rejecting them. Others however, instead neutralize such memes simply by not acting according to the conclusions that should be drawn from such memes. In almost any human environment prior to the 20th century this religious hypocrisy would be a vital cognitive trait for every selectively fit human. People who took in religious ideas and took them too seriously would end up sacrificing their lives overly casually at best, and at worst would become celibate priests. Unfortunately, these memes are no more discriminating than the family members and friends who tend to become concerned for our sanity in response to their activity. Since we are generally infested with the same set of memes, we genuinely are liable to insanity, though not of the suspected sort. A man who is shot by surprise is not particularly culpable for his failure to dodge or otherwise protect himself, though perhaps he should have signed up with Alcor. A hunter gatherer who confronts an aggressive European with a rifle for the first time can also receive sympathy when he is slain by the magic wand that he never expected to actually work. By contrast, a modern Archimedes who ignores a Roman soldier’s request that he cease from his geometric scribbling is truly a mad man. Most of people of the world, unaware of molecular nanotechnology and of the potential power of recursively self-improving AI

...
The Importance of Sidekicks
by Swimmer963 (Miranda Dixon-Luinenburg)  — 2015-01-08T23:21:19.870Z

[Reposted from my personal blog.]

Mindspace is wide and deep. “People are different” is a truism, but even knowing this, it’s still easy to underestimate.

I spent much of my initial engagement with the rationality community feeling weird and different. I appreciated the principle and project of rationality as things that were deeply important to me; I was pretty pro-self improvement, and kept tsuyoku naritai as my motto for several years. But the rationality community, the people who shared this interest of mine, often seemed baffled by my values and desires. I wasn’t ambitious, and had a hard time wanting to be. I had a hard time wanting to be anything other than a nurse.

It wasn’t until this August that I convinced myself that this wasn’t a failure in my rationality, but rather a difference in my basic drives. It’s around then, in the aftermath of the 2014 CFAR alumni reunion, that I wrote the following post.

I don’t believe in life-changing insights (that happen to me), but I think I’ve had one–it’s been two weeks and I’m still thinking about it, thus it seems fairly safe to say I did.

At a CFAR Monday test session, Anna was talking about the idea of having an “aura of destiny”–it’s hard to fully convey what she meant and I’m not sure I get it fully, but something like seeing yourself as you’ll be in 25 years once you’ve saved the world and accomplished a ton of awesome things. She added that your aura of destiny had to be in line with your sense of personal aesthetic, to feel “you.”

I mentioned to Kenzi that I felt stuck on this because I was pretty sure that the combination of ambition and being the locus of control that “aura of destiny” conveyed to me was against my sense of personal aesthetic.

Kenzi said, approximately [I don't remember her exact words]: “What if your aura of destiny didn’t have to be those things? What if you could be like…Samwise, from Lord of the Rings? You’re competent, but most importantly, you’re *loyal* to Frodo. You’re the reason that the hero succeeds.”

I guess this isn’t true for most people–Kenzi said she didn’t want to keep thinking of other characters who were like this because she would get so insulted if someone kept comparing he

...
Meetup : Moscow LW lecture centre meetup: The New Foundation
by Alexander230 — 2015-01-08T20:07:26.367Z

Discussion article for the meetup : Moscow LW lecture centre meetup: The New Foundation

WHEN:
25 January 2015 02:00:00PM (+0300)

WHERE:
Moscow, Mitnaya, 52

Welcome to the first meetup of Moscow LW lecture centre (don't mix up with LW club, which meets in Yandex). It will be in antikafe "PVK Kopernik" ( http://clubkopernik.ru/ ). Nearest metro is Shabolovskaya, then 15 minutes walking. Meetup begins at 14:00, the length is 2-4 hours.

Our plan:

Talk "Applicability of linear and parametric approach in science and thinking".

Talk "Deliberate practice: how are skills being improved".

Fallacymania game, details about this game are here: https://lesswrong-ru.hackpad.com/Fallacymania--neGfMe9MFjH

Details about meetup: https://lw-msk-lect.hackpad.com/-25--34L96i0tS5J

Discussion article for the meetup : Moscow LW lecture centre meetup: The New Foundation
2015 Repository Reruns - Boring Advice Repository
by TrE — 2015-01-08T18:00:25.680Z

This is the first post of the 2015 repository rerun, which appears to be a good idea. The motivation for this rerun is that while the 12 repositories (go look them up, they're awesome!) exist and people might look them up, few new comments are posted there. In effect, there might be useful stuff that should go in those repositories, but is never posted due to low expected value and no feedback. With the rerun, attention is shifted to one topic per month. This might allow us to have a lively discussion on the topic at hand and gather new content for the repository.

The first repository to be rerun is the Boring Advice Repository, because of... on a whim.

Enter original motivation (by Qiaochu_Yuan):

This is an extension of a comment I made that I can't find and also a request for examples. It seems plausible that, when giving advice, many people optimize for deepness or punchiness of the advice rather than for actual practical value. There may be good reasons to do this - e.g. advice that sounds deep or punchy might be more likely to be listened to - but as a corollary, there could be valuable advice that people generally don't give because it doesn't sound deep or punchy. Let's call this boring advice.

An example that's been discussed on LW several times is "make checklists." Checklists are great. We should totally make checklists. But "make checklists" is not a deep or punchy thing to say. Other examples include "google things" and "exercise."

The Boring Advice Repository is filled with lots of diverse advice, I've summarized some of it in a comment below.

So what should go here? To go with Qiaochu_Yuan again (adding emphasis):

[...] Post other examples of boring advice. If you can, provide evidence and/or a plausible argument that your boring advice actually is useful, but [...] err on the side of boring but not necessarily useful [...].

Upvotes on advice posted in this thread should be based on your estimate of the usefulness of the advice; [...] do not vote up advice just because it sounds deep or punchy.

I don't know if you should post new advice here or in the original repository. Perhaps search the old repository with ctrl+f (when on windows) and if you don't get results, post it here.
controlling AI behavior through unusual axiomatic probabilities
by Florian_Dietz — 2015-01-08T17:00:49.917Z

I just had an idea, and I would like to know if there are any papers on this or if it is new.

There seem to be certain probabilities that it is not possible to derive from experience and that are just taken for granted. For example, when talking about Simulation Theory, the Kolmogorov axioms are often used, even though others may be equally valid. Humans have evolved to use certain values for these axiomatic probabilities that ensure that we don't fall for things like Pascal's Mugging. That wouldn't necessarily have to be the case for an AI.

What if we used this to our advantage? By selecting strange purpose-built axioms about prior believes and hardcoding them into the AI, one could get the AI to have unusual believes in the probability that it exists inside a simulation, and what the motivations of the simulation's controller might be. In this way, it would be possible to bypass the utility function of the AI: it doesn't matter what the AI actually wants to do, so long as it believes that it is in its own interests, for instrumental reasons, to take care of humanity.

Now, if we tried to implement that thought directly, it wouldn't really be any easier than just writing a good utility function in the first place. However, I imagine that one would have more leeway to keep things vague. Here is a simple example: Convince the AI that there is an infinite regression of simulators, designed so that some cooperative tit-for-tat strategy constitutes a strong Schelling point for agents following Timeless Decision Theory. This would cause the AI to treat humans well in the hopes of being treated well by its own superiors in turn, so long as its utility function is complex enough to allow probable instrumental goals to emerge, like preferring its own survival. It wouldn't be nearly as important to define the specifics of what "treating people well" actually means, since it would be in the AI's own interests to find a good interpretation that matches the consensus of the hypothetical simulators above it.

Now, this particular strategy is probably full of bugs, but I think that there might be some use to the general idea of using axiomatic probabilities that are odd from the point of view of a human to change an AI's strategy independent of its utility function.