trentmkelly/LessWrong-43k huggingface dataset:
Tactical vs. Strategic Cooperation
by sarahconstantin — 2018-08-12T16:41:40.005Z

As I've matured, one of the (101-level?) social skills I've come to appreciate is asking directly for the narrow, specific thing you want, instead of debating around it.

What do I mean by "debating around" an issue?

Things like:

"If we don't do what I want, horrible things A, B, and C will happen!"

(This tends to degenerate into a miserable argument over how likely A, B, and C are, or a referendum on how neurotic or pessimistic I am.)

"You're such an awful person for not having done [thing I want]!"

(This tends to degenerate into a miserable argument about each other's general worth.)

"Authority Figure Bob will disapprove if we don't do [thing I want]!"

(This tends to degenerate into a miserable argument about whether we should respect Bob's authority.)

It's been astonishing to me how much better people respond if instead I just say, "I really want to do [thing I want.] Can we do that?"

No, it doesn't guarantee that you'll get your way, but it makes it a whole lot more likely. More than that, it means that when you do get into negotiation or debate, that debate stays targeted to the actual decision you're disagreeing about, instead of a global fight about anything and everything, and thus is more likely to be resolved.

Real-life example:

Back at MetaMed, I had a coworker who believed in alternative medicine. I didn't. This caused a lot of spoken and unspoken conflict. There were global values issues at play: reason vs. emotion, logic vs. social charisma, whether her perspective on life was good or bad. I'm embarrassed to say I was rude and inappropriate. But it was coming from a well-meaning place; I didn't want any harm to come to patients from misinformation, and I was very frustrated, because I didn't see how I could prevent that outcome.

Finally, at my wit's end, I blurted out what I wanted: I wanted to have veto power over any information we sent to patients, to make sure it didn't contain any factual inaccuracies.

Guess what? She agreed instantly.

This probably should have been obvious (and I'm sure it was obvious to her.) My job was producing the research reports, while her jobs included marketing and operations. The whole point of division of labor is that we can each stick to our own tasks and not have to critique each ...
New paper: Long-Term Trajectories of Human Civilization
by Kaj_Sotala — 2018-08-12T09:10:01.962Z

Long-Term Trajectories of Human Civilization (free PDF). Foresight, forthcoming, DOI 10.1108/FS-04-2018-0037.

Authors: Seth D. Baum, Stuart Armstrong, Timoteus Ekenstedt, Olle Häggström, Robin Hanson, Karin Kuhlemann, Matthijs M. Maas, James D. Miller, Markus Salmela, Anders Sandberg, Kaj Sotala, Phil Torres, Alexey Turchin, and Roman V. Yampolskiy.

Abstract

Purpose: This paper formalizes long-term trajectories of human civilization as a scientific and ethical field of study. The long-term trajectory of human civilization can be defined as the path that human civilization takes during the entire future time period in which human civilization could continue to exist.

Approach: We focus on four types of trajectories: status quo trajectories, in which human civilization persists in a state broadly similar to its current state into the distant future; catastrophe trajectories, in which one or more events cause significant harm to human civilization; technological transformation trajectories, in which radical technological breakthroughs put human civilization on a fundamentally different course; and astronomical trajectories, in which human civilization expands beyond its home planet and into the accessible portions of the cosmos.

Findings: Status quo trajectories appear unlikely to persist into the distant future, especially in light of long-term astronomical processes. Several catastrophe, technological transformation, and astronomical trajectories appear possible.

Value: Some current actions may be able to affect the long-term trajectory. Whether these actions should be pursued depends on a mix of empirical and ethical factors. For some ethical frameworks, these actions may be especially important to pursue.

An excerpt from the press release over at the Global Catastrophic Risk Institute:

Society today needs greater attention to the long-term fate of human civilization. Important present-day decisions can affect what happens millions, billions, or trillions of years into the future. The long-term effects may be the most important factor for present-day decisions and must be taken into account. An international group of 14 scholars calls for the dedicated study of “long-term trajectories of human civilization” in order to understand long-term outcomes and inform decision-making. This new approach is presented in the academic journal Foresight, where the scholars h

...
How to Build a Lumenator
by Raemon — 2018-08-12T05:11:06.715Z

Once upon a time, a friend was sad. Specifically, they had seasonal affective disorder. They tried to fix it by adding lights to their room during the winter.

It didn't work.

They tried adding a LOT of light.

It worked.

They called the giant bundle of lights they assembled a Lumenator. Other people wondered how they, too, might summon a sun into their living room. The task was not exactly complicated or hard, but it was a little confusing and inconvenient. Instructions were passed around by word of mouth, and individuals cobbled together lumenators in their own homes. Some of them has seasonal affective disorder, and some just liked their rooms to feel line sunshine all the time.

Bit by bit, people's lives grew brighter.

Eventually, someone said "it's silly that there are no instructions on how to do this on the internet and it has to be passed around as weird Cultural Metis." They said this, but then took no actions based on it. Then a second person said that, and they too took no actions based on it. And then finally someone asked me this weekend how to do it and I wrote this blog post.

(Don't be too impressed, because that second person who did nothing was also me)

A lumenator is 24 lightbulbs, hung in a row from your ceiling. You may want a dimmer switch to control exactly how bright it is. You may want lantern-covers so that the light isn't directly in your eyes (depends on the room in question).

You will need the following material components (roughly $300 on Amazon). [UPDATE: many of these amazon links no longer work, and I think I got some pieces subtly wrong. I recommend checking out this recent UK post and this post to get more details)

[x1] 24 socket cord to connect the lights
[x2] 6 pack of light bulbs, 2700K (Warm White)
[x2] 6 pack of light bulbs, 4000K (Daylight Glow) (Edit: this should actually be 5000k, but I couldn't easily find a 5000k bulb that was clearly the right specs, will update this when I get a minute to do so)
[x2-x3] Command Hook 14pack (you may screw up the command hook process a few times so probably want extras of these. Also command hooks are great to have around anyway.)
[x1] [RECOMMENDED] Adjustable dimmer
[x1] [BONUS] Optional chinese-style lantern covers

You can swap out the command hooks for something more robust (i.e. nails in wall, if you're allowed to do that in the space you live in) but the listed hooks worked for me.

Why 12 br...
Welcome to Rationality and Friendly AI in Tokyo （仮）
by Razmęk Massaräinen — 2018-08-11T14:45:09.100Z

We're organizing a community in Tokyo for those who are interested in:

LessWrong-style rationality
Cognitive science
AI development & alignment

We'll try to host some regular meetups, starting Autumn 2018.

The aim is to raise awareness, and to find and bring together both international and local groups of people who are interested in things around human and artificial thinking and technological singularity.
Human-Aligned AI Summer School: A Summary
by Michaël Trazzi — 2018-08-11T08:11:00.789Z

(Disclaimer: this summary is incomplete and does not accurately represent all the content presented at the summer school, but only what I remember and seem to have understood from the lectures. Don't hesitate to mention important ideas I missed or apparent confusion.)

Last week, I attended the first edition of the human-aligned AI summer school in Prague. After three days, my memories are already starting to fade, and I am unsure about what I will retain in the long-term.

Here, I try to remember the content of about 15h of talks. It serves the following purposes:

To the general audience that did not attend the school, I try to give an overview, to inform about the general trends we discussed.
For those who attended the school, I distill what I understood, to refresh our memories.

Value Learning (Daniel Filan)

Value Learning aims at infering human values from their behavior. Paul Christiano distinguishes ambitious value learning vs. narrow value learning:

Ambitious value learning: learn human preferences over long-term outcomes.
Narrow value learning: learn human instrumental values and subgoals.

Inverse Reinforcement Learning

Inverse Reinforcement Learning (IRL) studies which reward best explains a behaviour. Two methods of IRL were discussed (the state-of-the-art builds on top of those two, for instance using neural networks):

Bayesian IRL

: uses Bayesian update. Does not work in practice because necessits to solve many Markov Decision Processes, which is computationally intensive.

Maximum Entropy IRL

: the optimal distribution (of maximum entropy) is an exponential of a linear function. One of the reason it performs better in practice is that it's easier to efficiently approximate the relevant integrals.

Why not to do value learning

:

It is (still) inefficient
It depends heavily on human rationality models
The reward might not be in the prior reward space
Solving other problems, such as naturalized agency, might be more urgent
The actions in the behavior are not well-defined in practice (e.g. what counts as an action in a football game?)

Beyond Inverse Reinforcement Learning

The main problem of traditional IRL is that it does not take into account the deliberate interactions between a human and an AI (e.g. the human could be slowing down his behaviour to help learning).

Cooperative IRL solves this issue by introducing a two-player game between the human and the AI, w...
LW Update 2018-08-10 – Frontpage map, Markdown in LaTeX, restored posts and reversed spam votes
by habryka — 2018-08-10T18:14:53.909Z

Here are the updates I pushed in the last week:

The community map is on the frontpage for the next week, to celebrate and promote the "SSC Meetups Everywhere"
We are now using a new Markdown processor, which allows you to use Markdown in your LaTeX. We use Pandoc's syntax, which uses single $ , but doesn't parse them when there is a number right after the second one. I.e. $4000 and $4000 doesn't get parsed as LaTeX since the second dollar sign has a number immediately after it.
I've finally figured out how the old LW database stored some of the old deleted posts, which means you can now access a bunch of posts from LW again. Most notably this one, which many people requested.
I've also went back through the old database and did a larger hunt for spam and troll voters, and reversed all of their votes. This resulted in the votes of over 500 accounts getting reversed, with over 15000 votes being reversed (which is a lot, about as many votes as we had in all of March this year). If you had any of your old posts or comments downvoted, now might be the time to check whether that was just a result of trolls.
Fundamentals of Formalisation Level 7: Equivalence Relations and Orderings
by philip_b — 2018-08-10T15:12:46.683Z

Followup to Fundamentals of Formalisation Level 6: Turing Machines and the Halting Problem. First post.

This is a new lesson of our online course on math formalizations required for AI safety research.

The big ideas:

Equivalence Relations

Partitions

Orderings

To move to the next level you need to be able to:

Explain the relationship between equivalence relations and partitions.

Name two different kinds of orderings and the conditions on the ordering relations required for these kinds.

Why this is important:

This level is about further building up your mathematical toolbox. While equivalence relations and orderings currently seem like random abstract notions, they are both very important. The former gives you a way to instantiate different notions of equality, which is important in modeling. The latter appears in constructing models for another turing machine equivalent notion of computability, the lambda calculus.

For every lesson you have 2 options: do the whole thing, or skip to the questions and exercises in the end. The latter option is for people who suspect they already know the subject. It serves as a means of verifying or falsifying that hypothesis.
SSC Meetups Everywhere 2018
by habryka — 2018-08-10T03:18:58.716Z

Scott over at SlateStarCodex is running another round of "SSC Meetups Everywhere" and since currently a record-high of 80+ meetups are registered for the coming month, I figured we should help people find their nearest meetup, and make them aware of the opportunity. As part of that, we are adding the meetup map to the frontpage for the next week.

If you ever wanted to meet other rationalists/EAs/SSC-readers in person but found that there were no nearby regular meetups, or want to kickstart your local community, this is probably the best opportunity you are going to get this year.
AI Reading Group Thoughts (1/?): The Mandate of Heaven
by Alicorn — 2018-08-10T00:24:21.094Z

My housemate Kelsey "theunitofcaring" has begun hosting an AI reading group in our house. Our first meeting was yesterday evening, and over a first draft attempt at chocolate macarons, we discussed this article about AI safety and efficiency by Paul Christiano, and various ideas prompted thereby at greater or lesser remove.

One idea that came up is what we decided to call "tipping point AI" (because apparently there are a lot of competing definitions for "transformative" AI). The definition we were using for tipping point AI was "something such that it, or its controller, is capable of preventing others from building AIs". The exact type and level of capability here could vary - for instance, if it's built after we've colonized Mars (that is, colonized it to an extent such that Martians could undertake projects like building AIs), then a tipping point AI has to be able to project power to Mars in some form, even if the only required level of finesse is lethality. But if it's before we've colonized Mars, it can be unable to do that, and just able to prevent colonization projects in addition to AI projects.

One hypothesis that has been floated in a context such that we are pretty sure it is not anyone's real plan is that an AI could just destroy all the GPUs on the planet and prevent the manufacture of new ones. This would be bad for Bitcoins, video games, and AI projects, but otherwise relatively low-impact. An AI might be able to accomplish this task by coercion, or even by proxy - the complete system of "the AI, and its controller" needs to be able to prevent AI creation by other agents, so the AI itself might only need to identify targets for a controller who already wields enough power to fire missiles or confiscate hardware and chooses to do so in service of this goal, perhaps the US government.

The idea behind creating tipping point AI isn't that this is where we stop forever. The tipping point AI only has to prevent other agents from building their own in their basements. It eliminates competition. Some features of a situation in which a tipping point AI exists include:

The agent controlling the AI can work on more sophisticated second drafts without worrying about someone else rushing to production unsafely.
The controlling agent can publish insights and seek feedback

...
Logarithms and Total Utilitarianism
by Pablo Villalobos — 2018-08-09T08:49:16.753Z

Epistemic status: I might be reinventing the wheel here

A common cause for rejection of total utilitarianism is that it implies the so-called Repugnant Conclusion, of which a lot has been written elsewhere. I will argue that while this implication is solid in theory, it does not apply in our current known universe. My view is similar to the one expressed here, but I try to give more details.

The Repugnant Conclusion IRL

The greatest relevance of the RC in practice arises in situations of scarce resources and Malthusian population traps¹: We compare population A, where there are few people with each one having plentiful resources, and population Z, which has grown from A until the average person lives in near-subsistence conditions.

Let's formalize this a bit: suppose each person requires 1 unit of resources for living, so that the utility of a person living on 1 resources is exactly 0: a completely neutral life. Furthermore, suppose utility is linear w.r.t. resources: doubling resources means doubling utility and 10 resources correspond to 1 utility. If there are 100 resources in the world, population A might contain 10 people with 10 resources each and total utility 10; population Z might contain 99 people with 100/99 resources each and total utility also 10.

So in this model, we are indifferent between A and Z even as everyone in Z is barely subsisting, and this would be the Repugnant Conclusion². But this conclusion depends crucially on the relationship between resources and utility which we have assumed to be linear. What if our assumption is wrong? What is this relationship in the actual world? Note that this is an empirical question³.

It is well known that self-reported happiness varies logarithmically with income⁴, both between countries and for individuals within each country, so it seems reasonable to assume that the utility-resources relation is logarithmic: exponential increases in resources bring linear increases in utility.

Back to our model, assuming log utility, how do we now compare A and Z? If utility per person is ui=logri where ri are the resources available to that person, then total utility is U=Σlogri . Assuming equality in the population (see the Equality section), if R are total resources and N is population size, each person has resources ri=R/N and so we have

U=ΣlogRN=Σ(logR−logN)=N(logR−logN)

We can plot total utility (vertical axis...
A Short Note on UDT
by Chris_Leong — 2018-08-08T13:27:12.349Z

In my last post, I stumbled across some ideas which I thought were original, but which were already contained in UDT. I suspect that was because these ideas haven’t been given much emphasis in any of the articles I’ve read about UDT, so I wanted to highlight them here.

We begin with some definitions. Some inputs in an Input-Output map will be possible for some agents to experience, but not for others. We will describe such inputs and the situations they represent as conditionally consistent. Given a particular agent, we will call a input/situation compatible if the agent is consistent with the corresponding situation and incompatible otherwise. We will call agents consistent with a conditionally consistent input/situation compatible and those who aren’t incompatible.

We note the following points:

UDT uses an Input-Output map instead of a Situation-Output map. It is easy to miss how important this choice is. Suppose we have an input representing a situation that is conditionally consistent. Trying to ask what an incompatible agent does in such a situation is problematic or at least difficult as the Principle of Explosion means that all such situations are equivalent. On the other hand, it is much easier to ask how the agent responds to a sequences of inputs representing an incompatible situation. The agent must respond somehow to such an input, even if it is by doing nothing or crashing. Situations are also modelled (via the Mathematical Intuition Function), but the point is that UDT models inputs and situations separately.
Given the previous point, it is convenient to define an agent’s counterfactual action in an incompatible situation as its response to the input representing the situation. For all compatible situations, this produces the same action as if we’d simply asked what the agent would do in such a situation. For conditionally consistent situations the agent is incompatible with, it explains the incompatibility: any agent that would respond a certain way to particular inputs won’t be put in such a situation. (There might be conditionally consistent situations where compatibility isn’t dependent on responses to inputs, ie. only agents running particular source code are placed in a particular position, but UDT isn’t designed to solve these problem)
Similarly, UDT predictors don’t actually pred

...
Ontological uncertainty and diversifying our quantum portfolio
by Roko Jelavić — 2018-08-08T09:19:41.818Z

The word "ontology" in the title refers to our conception of the basic building block of reality. In quantum mechanics the ontology is the wave function, in general relativity it is spacetime.

This idea in this post assumes the many-worlds interpretation of quantum mechanics is the correct one. In this interpretation there is an infinity of universes which begin as the same. When an event happens it can go one way in some universes (Schrödinger's cat dead) and another way in another (Schrödinger's cat alive). The number of universes stays constant, they just get diversified.

True random number generators are often based on physical phenomena which can be traced to quantum mechanics, as opposed to the more commonly used pseudorandom number generators. We can call such quantum-mechanical random number generators QMRNGs for short. This kind of random generator generates different numbers in different universes while the agent using the generator is uncertain about in which universe he will end up.

Although it is not widely known, the laws of general relativity actually allow for time travel under some special circumstances. Time travel is widely considered impossible since it leads to certain paradoxes, such as the grandfather paradox and is hence deemed logically inconsistent. Those paradoxes are resolved if the time traveler travels not only in time, but also to another universe, as David Deutsch explains. In this setup a time machine is built and the traveler enters it, some time passes, and when he opens the door and exits the machine, he is in the past. Not only is he in the past, he is also in a parallel universe. When he kills "his own" grandfather in that parallel universe he is not really killing his own grandfather, but a parallel-grandfather, a grandfather of parallel-him.

The fundamental laws of physics are not yet known since there is yet no physical theory of everything. Hence, it is not known what are the basic building blocks of reality (we can call them simply "ontology"). In quantum mechanics the ontology it is the wave function, in general relativity the ontology is spacetime. Whatever the true laws of the universe are, the Earth will still be round and single photons will still interfere in the double-slit experiment. Whatever the true laws of physics are, they will probably include parallel universes. It...