trentmkelly/LessWrong-43k huggingface dataset:
Acausal trade: full decision algorithms
by Stuart_Armstrong — 2017-05-15T10:12:44.000Z

The description I gave here about how to balance and negotiate between different utility functions, is a bit incomplete, as this post will show. Here I'll give more details on possible decision algorithms the agents could run.

Here, two agents A1 and A2, who maximise u1 and u2, exist with probabilities q1 and q2, not necessarily independent. The joint probability that both agents exist is r12.

Waiting souls

In this perspective, we imagine that each utility function is represented by a disembodied entity, that negotiate the terms of acausal trade before the universe begins and before any agents exist or not. This is the Rawlsian veil of ignorance, that I said we'd ignore in the original post, with justifications presented here.

Now, however, we need to consider it, as a gateway to the case we want. How would the agents balance each other's utilities?

One possibility is that the agents assign equal weight to both utilities. In that case they will be both maximising u1+u2. But this poses a continuity problem as the probability of any agent declines towards 0. So the best option seems to be to have the agents agree to maximise q1u1+q2u2.

Then, in the situation presented in the previous post, both agents would increase the other utility until the marginal cost of doing so increased to either q2/q1 (for agent A1) or q1/q2 (for agent A2).

There is no "double decrease" in this situation.

Existing souls

Here we restrict ourselves to agents that actually exist. So, if for instance r12=0 (the two agents cannot both exist) then agent A1, should they exist, will have no compunction about not maximising u2 in any way.

One way of modelling this is to go back to the "waiting souls" formalism, but replace ui with u′i=uiIi where Ii is the indicator variable on whether agent Ai existed at any point in the universe. Thus all utilities depend on the existence of the agent that prefers them, in order to be maximised by anyone.

There is not longer a continuity issue with u′1+u′2 when the probabilities qi tend to zero, since low qi mean that changes to u′i become smaller and smaller in expectation.

So, when maximising u′1+u′2, the agent A1 will consider that increases to u′2 have an effect that is r12/q1 times as large as increases to u2 (while increases to u1 and u′1 are identical from its perspective, since the agent exists). Thus it will increase u2 until the marginal cost of further in

...
Open thread, May 15 - May 21, 2017
by Elo — 2017-05-15T07:06:41.703Z

If it's worth saying, but not worth its own post, then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should start on Monday, and end on Sunday.

4. Unflag the two options "Notify me of new top level comments on this article" and "Make this post available under..." before submitting.
Meetup : Baltimore / UMBC - Unsong Readalong Edition!
by l1n — 2017-05-14T16:51:45.145Z

Discussion article for the meetup : Baltimore / UMBC - Unsong Readalong Edition!

WHEN:
14 May 2017 08:00:00AM (-0400)

WHERE:
Performing Arts and Humanities Building, 1000 Hilltop Cir, Baltimore, MD 21250, United States

Casual discussion group! But this week we will be starting with a dramatic reading of the final chapter of Unsong.

Suggested Reading: Unsong. Meetup is on the 4th floor by the English department. Follow the voices. For help getting into the building, please text 619-357-6633.
Format: We meet and start hanging out at 8:00, but don’t officially start doing the meetup topic until 8:15-8:30 to accommodate stragglers. Parking is available anywhere on campus because its a weekend. I suggest Lot 8.

Discussion article for the meetup : Baltimore / UMBC - Unsong Readalong Edition!
Making decisions in a real computer - an argument for meta-decision theory research
by  — 2017-05-13T23:18:42.407Z

Decision theory is being used as the basis for AI safety work. This currently involves maximising expected utility of specific actions. Maximising expected utility is woefully inefficient for performing very rapid paced unimportant decisions, which occur frequently in computing. But these fast paced decisions will still need to be made in a way that is purpose oriented in an AI.
This article presents an argument that we should explore meta-decision theories to allow the efficient solution of these problems. Meta-decision theories are also more human-like and could have a different central problems to first order decision theories.

Decision theory tends to be the domain of economists, mathematicians, ethicists and other philosophers. However now it is being used in arguments about artificial intelligence and how to make it safe. If we are to try to implement an AI that follows any specific decision theory, it is brought into the real world domain of the computer. Now it has to deal with slightly less abstract world of bits and bytes. We have to make decision theories that take notice of the costs and consequences of making a decision.
AI is often seen through a dual lens, as both an agent making reasoned decisions about goals and as a computer able to process data and act at astounding speeds. These views have not entirely been integrated and this needs to be done for safe AI work.
AIs will still have to send data over the network and it will display images on a screen. These boring things are still up for optimisation, they still need to be decided upon. We will want these actions to be correctable if they are going badly wrong.

Decisions at 10Gbps

The example that will be followed through is networking. Consider a simplified network interface where the possible actions are outputting a 0 or a 1 across a wire.

We have an AI in control of this network interface, part of it’s job to send a file as quickly as possible to the other side. It has a 10 Gbps ethernet connection. It gets the optimal utility if it can send a 10 Gb file in 1 second. It also has to pick and chose which files to send, some are more important that others. Imagine doing an expected utility calculations for each bit. If you can have a machine with 300 GIPs (top of the range CPUs are in that order), and under ideal conditions could only use 30 instructions to do an expected utility calcu...
Why do we think most AIs unintentionally created by humans would create a worse world, when the human mind was designed by random mutations and natural selection, and created a better world?
by contravariant — 2017-05-13T08:23:21.415Z

As far as AI designers go, evolution has to be one of the worst. It randomly changes the genetic code, and then selects on the criterion of ingroup reproductive fitness - in other words, how well a being can reproduce and stay alive - it says nothing about the goals of that being while it's alive.

To survive, and increase one's power are instrumentally convergent goals of any intelligent agent, which means that evolution does not select for any specific type of mind, ethics, or final values.

And yet, it created humans and not paperclip maximizers. True, humans rebelled against and overpowered evolution, but in the end we ended up creating amazing things and not a universe tiled with paperclips(or DNA, for that matter).

Considering how neural network training and genetic algorithms are considered some of the most dangerous ways of creating an AI,

the fact that natural evolution managed to create us with all our goals of curiosity and empathy and love and science,

would be a very unlikely coincidence given that we assume that most AIs we could create are worthless in terms of their goals and what they will do with the universe. Did it happen by chance? The p-value is pretty small on this one.

Careless evolution managed to create humans on her first attempt at intelligence, but humans, given foresight and intelligence, have an extreme challenge making sure an AI is friendly? How can we explain this contradiction?
SlateStarCodex Meetups Everywhere: Analysis
by mingyuan — 2017-05-13T00:29:57.695Z

The first round of SlateStarCodex meetups took place from April 4th through May 20th, 2017 in 65 cities, in 16 countries around the world. Of the 69 cities originally listed as having 10 or more people interested, 9 did not hold meetups, and 5 cities that were not on the original list did hold meetups.

We collected information from 43 of these events. Since we are missing data for 1/3 of the cities, there is probably some selection bias in the statistics; I would speculate that we are less likely to have data from less successful meetups.

Of the 43 cities, 25 have at least tentative plans for future meetups. Information about these events will be posted at the SSC Meetups GitHub.

Turnout

Attendance ranged from 3 to approximately 50 people, with a mean of 16.7. Turnout averaged about 50% of those who expressed interest on the survey (range: 12% to 100%), twice what Scott expected. This average does not appear to have been skewed by high turnout at a few events – mean: 48%, median: 45%, mode: 53%.

On average, gender ratio seemed to be roughly representative of SSC readership overall, ranging from 78% to 100% male (for the 5 meetups that provided gender data). The majority of attendees were approximately 20-35 years old, consistent with the survey mean age of 30.6.

Existing vs new meetups

Approximately one fifth of the SSC meetups were hosted by existing rationality or LessWrong groups. Some of these got up to 20 new attendees from the SSC announcement, while others saw no new faces at all. The two established meetups that included data about follow-up meetings reported that retention rates for new members were very low, at best 17% for the next meeting.

Here, it seems important to make a distinction between the needs of SSC meetups specifically and rationality meetups more generally. On the 2017 survey, 50% of readers explicitly did not identify with LW and 54% explicitly did not identify with EA. In addition, one organizer expressed the concern that, “Going forward, I think there is a concern of “rationalists” with a shared background outnumbering the non-lesswrong group, and dominating the SSC conversation, making new SSC fans less likely to engage.”

This raises the question of whether SSC groups should try to exist separately from local EA/LW/rationalist/skeptic groups – this is of particular concern in locations w...
Meetup : [Metro Detroit / Ann Arbor], Michigan: LW and SSC meetup where I bring food
by excinera — 2017-05-12T20:51:37.288Z

Discussion article for the meetup : [Metro Detroit / Ann Arbor], Michigan: LW and SSC meetup where I bring food

WHEN:
13 May 2017 04:00:00PM (-0400)

WHERE:
255 E. Liberty St. Suite 225 Ann Arbor, MI 48103

Howdy, fellers! I wasn't able to set anything up last weekend due to being extremely unwell for a couple weeks, but now I am whole again, and I have arranged for a meeting space to be available at All Hands Active tomorrow, from 4:00 PM (16:00) until... whenever. Probably about 10:00 or midnight, from past experience.

Sorry for the short notice, but I was only just able to set this up yesterday, and blah blah blah, reasons. I don't have time to update the website right now so everything there is out of date at the moment but if you have any questions (or need a ride, or whatever), text or call me at (734) 707-1316.

The address is

All Hands Active
255 E. Liberty St.
Suite 225
Ann Arbor, MI 48103

Discussion article for the meetup : [Metro Detroit / Ann Arbor], Michigan: LW and SSC meetup where I bring food
Reasoning upon its own dark fiction
by Jacob Falkovich — 2017-05-12T04:40:19.000Z

This post is a loving tribute to Unsong, an amazing novel of kabbalistic magic, theodicy, and Biblical whale puns by Scott Alexander. The apocalyptic conclusion of Unsong is set for the evening of May 14th in both our universe and its own. This is not a coincidence – our own world may end up destroyed as well as Scott rearranges the higher planes of creation. If you have only two days left until the end of the world, you should spend them reading Unsong.

Speaking of Biblical things that aren’t coincidences, my life’s path has been predicted with remarkable precision by the Biblical story of Jacob.

The name “Jacob” comes directly from the Hebrew word for “heel”, and indeed, though I applied to many schools in the US, I became a Tar Heel. Both the patriarch and I lived in the city of Be’er Sheba before embarking on our travels. In the Torah, Jacob climbs a hill and realizes “How awesome is this place! This is none other than the house of God, and this is the gate of heaven.“ After leaving Be’er Sheba I arrived at Chapel Hill, an awesome place whose official motto is “The Southern Part of Heaven“.

The next scene in Jacob’s life takes place at an old well, and the same is true of me.

The Old Well, University of North Carolina in Chapel Hill

I already wrote about how both us Jacobs, while working in a foreign land, dealt with a choice between two romantic partners. People in modern times think that they invented non-traditional relationship structures and come up with complex terminology to describe them. For example, “hierarchical polyamory”, or “primary consensual nonmonogamy”. Those two mean the same thing: being committed to loving a single person above all others while maintaining open sexual relationships with other people. With two concubines and a second wife in addition to Rachel, Jacob invented “hierarchical polyamory” more than 3,500 years ago.

In addition to his birth name, Jacob also earned the name “Israel” later in life. I was born in Russia, and earned the right to call myself an Israeli when my family immigrated to Rehovot. Our relatives who chose the US over Israel initially arrived in Rochester, NY. Of course, it turned out that Rochester a twin city of Rehovot. In the Torah, “Rehovot” means “a broad ...
Gears in understanding
by Valentine — 2017-05-12T00:36:17.086Z

Some (literal, physical) roadmaps are more useful than others. Sometimes this is because of how well the map corresponds to the territory, but sometimes it's because of features of the map that are irrespective of the territory. E.g., maybe the lines are fat and smudged such that you can't tell how far a road is from a river, or maybe it's unclear which road a name is trying to indicate.

In the same way, I want to point at a property of models that isn't about what they're modeling. It interacts with the clarity of what they're modeling, but only in the same way that smudged lines in a roadmap interact with the clarity of the roadmap.

This property is how deterministically interconnected the variables of the model are. There are a few tests I know of to see to what extent a model has this property, though I don't know if this list is exhaustive and would be a little surprised if it were:

Does the model pay rent? If it does, and if it were falsified, how much (and how precisely) could you infer other things from the falsification?
How incoherent is it to imagine that the model is accurate but that a given variable could be different?
If you knew the model were accurate but you were to forget the value of one variable, could you rederive it?

I think this is a really important idea that ties together a lot of different topics that appear here on Less Wrong. It also acts as a prerequisite frame for a bunch of ideas and tools that I'll want to talk about later.

I'll start by giving a bunch of examples. At the end I'll summarize and gesture toward where this is going as I see it.

Example: Gears in a box

Let's look at this collection of gears in an opaque box:

(Drawing courtesy of my colleague, Duncan Sabien.)

If we turn the lefthand gear counterclockwise, it's within our model of the gears on the inside that the righthand gear could turn either way. The model we're able to build for this system of gears does poorly on all three tests I named earlier:

The model barely pays rent. If you speculate that the righthand gear turns one way and you discover it turns the other way, you can't really infer very much. All you can meaningly infer is that if the system of gears is pretty simple (e.g., nothing that makes the righthand gear alternate as the lefthand gear rotates counterclockwise), then the direction that the righthand gear turns determines whether the total number of gears is

...
Meetup : Chicago Rationality Reading Group
by mingyuan — 2017-05-11T23:49:31.350Z

Discussion article for the meetup : Chicago Rationality Reading Group

WHEN:
14 May 2017 01:00:00PM (-0500)

WHERE:
Harper Memorial Library Room 148, 1116 E 59th St, Chicago, IL 60637

The Chicago Rationality group meets every Sunday from 1-3 PM in Room 148 of Harper Memorial Library. Though we meet on the University of Chicago campus, anyone is welcome to attend.

This week we'll be returning to the reading group format! The readings are as follows:

http://slatestarcodex.com/2014/05/16/you-kant-dismiss-universalizability/

http://lesswrong.com/lw/2pv/intellectual_hipsters_and_metacontrarianism/

http://slatestarcodex.com/2013/04/08/whose-utilitarianism/

If you're interested in rationality-related events in the Chicago area, request to be added to our Google Group and I'll approve you!

Discussion article for the meetup : Chicago Rationality Reading Group
Meetup : Washington, D.C.: Survivorship Bias
by RobinZ — 2017-05-11T23:46:22.804Z

Discussion article for the meetup : Washington, D.C.: Survivorship Bias

WHEN:
14 May 2017 03:30:00PM (-0400)

WHERE:
Donald W. Reynolds Center for American Art and Portraiture

This week, we will be meeting in the courtyard to discuss survivorship bias - invalid inferences drawn only from the examples which made it through some winnowing process, and none of the ones that didn't - and some examples thereof.

Upcoming meetups:

May 21: Autism

May 28: Fun & Games

Discussion article for the meetup : Washington, D.C.: Survivorship Bias
Acausal trade: Introduction
by Stuart_Armstrong — 2017-05-11T12:03:35.000Z

I've never really understood acausal trade. So in a short series of posts, I'll attempt to analyse the concept sufficiently that I can grasp it - and hopefully so others can grasp it as well.

Other posts in the series: Double decrease, Breaking acausal trade, Trade in different types of utility functions, Being special, Multiple acausal trade networks.

The simplest model

There are N different rooms. Since labels are arbitrary, assume you are in room 1, without loss of generality. The agents in room i exist with probability pi, and have a utility ui, which they are motivated to maximise. Each agent only acts in their room. They may choose to diminish ui to increase one or more other uj with i≠j.

The agents will never meet, never interact in any way, won't even be sure of each other's existence, may not known N, and may have uncertainty over the values of the other uj's.

Infinities, utility weights, negotiations, trade before existence

There are a number of things I won't be considering here. First of all, infinities. In reality, acausal trade would happen in the real universe, which is likely infinite. It's not clear at all how to rank infinitely many causally disconnected world-pieces. So I'll avoid that entirely, assuming N is finite (though possibly large).

There's also the thorny issue of how to weigh and compare different utility functions, and/or the process of negotiation about how to divide the gains from trade.

I'll ignore all these issues, and see the ui as functions from states of the world to real numbers: individual representatives of utility functions, not equivalence classes of equivalence functions. And the bargaining will be a straight one for one increase and decrease: a fair deal is one where ui and uj get the same benefit -- as measured by ui and uj.

I'll also ignore the possibility of trade before existence, or Rawlsian veils of ignorance. If you are a ui maximiser, but you could have been a uj maximiser if things had been different, then you have no responsibility to increase uj. Similarly, if there are uj maximisers out there, then you have no responsibility to maximiser uj without getting any ui increases out of that.

Changing that last assumption could radically alter the nature of acausal trade (eg potentially reducing to simply maximising a universal prior utility function), so it's important to emphasise that that is being ignored.
That is not dead which can eternal lie: the aestivation hypothesis for resolving Fermi's paradox
by Stuart_Armstrong — 2017-05-11T09:16:54.280Z

That is not dead which can eternal lie: the aestivation hypothesis for resolving Fermi's paradox

Anders Sandberg, Stuart Armstrong, Milan M. Cirkovic

If a civilization wants to maximize computation it appears rational to aestivate until the far future in order to exploit the low temperature environment: this can produce a 1030 multiplier of achievable computation. We hence suggest the "aestivation hypothesis": the reason we are not observing manifestations of alien civilizations is that they are currently (mostly) inactive, patiently waiting for future cosmic eras. This paper analyzes the assumptions going into the hypothesis and how physical law and observational evidence constrain the motivations of aliens compatible with the hypothesis.

As far as I can tell, the paper's physics is correct (most of the energy comes not from burning stars but from the universe's mass).

However, the conclusions are likely wrong, because it's rational for "sleeping" civilizations to still want to round up stars that might be ejected from galaxies, collect cosmic dust, and so on.

The paper is still worth publishing, though, because there may other, more plausible ideas in the vicinity of this one. And it describes how future civilization may choose to use their energy.
Hidden universal expansion: stopping runaways
by Stuart_Armstrong — 2017-05-11T09:01:54.360Z

We have a new paper out, presenting the 'aestivation hypothesis'. It's another attempt to reconcile the fact that cosmic expansion seems very easy, yet we see no trace of any alien group doing it.

The idea is that civilizations expand rapidly, but then 'go to sleep', while they wait for the temperature to drop and it becomes possible to do computations with maximal efficiency.

There are some few problems with the theory, though - mainly, why would the civilizations conceal themselves? Even if they were sleeping, they should have some automated processes rounding up intergalactic gases, preventing stars from drifting out of galaxies, and so on.

But though it's hard to justify a civilization permanently hiding, there are reasons why a civilization might hide temporarily.

Consider the following diagram:

Here, a civilization is expanding from the red point, and will eventually reach Earth (drawn not entirely to scale). It's expanding at a decent fraction of light-speed. The red sphere is their physical expansion front, while the yellow sphere is the light expansion front. When that yellow reaches Earth, we will generally be able to notice their expansion, and have some time to react to it - unless they conceal themselves as they expand.

Why would they want to do that? It's not as if we could counter their expansion, or have any chance of resisting. But there is one thing that we might be able to do: flee. Imagine that we got a hundred years warning; we might be able to rush AI, Dyson the sun, build escape ships and launch them at a significant fraction of light speed, etc. They might never be able to catch us, and, as we or our AIs fled, we could develop technologies to reduce or damage our pursuers.

Therefore, it makes sense for the expanding civilization to conceal itself until it has any other civilizations completely surrounded. That means that Dyson swarms and other major feats of stellar engineering might be delayed by many years or decades by the red civilization. So that the 'noticeability front' - the distance at which other civilizations can see clear evidence of red's expansion - lags a bit behind their actual expansion front.