trentmkelly/LessWrong-43k huggingface dataset:
The 'why does it even tell me this' moment
by  — 2016-05-01T08:15:19.724Z

Edited based on the outline kindly provided by Gram_Stone, whom I thank.

There is a skill of reading and thinking which I haven't learned so far: of looking for implications as one goes through the book, simply putting it back on shelf until one's mind has run out of the inferences, perhaps writing them down. I think it would be easier to do with books that [have pictures]

- invite an attitude (like cooking shows or Darwin's travel accounts or Feynman's biography: it doesn't have to be "personal"),

- are/have been regularly needed (ideally belong to you so you can make notes on the margins),

- are either outdated (so you "take it with a grain of salt" and have the option of looking for a current opinion) or very new,

- are not highly specialized,

- are well-structured, preferably into one- to a-few-pages-long chapters,

- allow reading those chapters out of order*,

- (make you) recognize that you do not need this knowledge for its own sake,

- can be shared, or at least shown to other people, and talked about, etc. (Although I keep imagining picture albums when I read the list, so maybe I missed something.)

These features are what attracts me to an amateur-level Russian plant identification text of the 1948.** It was clearly written, and didn't contain many species of plants that the author considered to be easily grouped with others for practical purposes. It annoyed me when I expected the book to hold certain information that it didn't (a starting point - I have to notice something to want to think). This is merely speculation, but I suspect that the author omitted many of the species that they did because the book was intended to convey agricultural knowledge of great economic importance to the Soviet population of the time (although some included details were clearly of less import, botanists know that random bits trivia might help recognizing the plant in the field, which established a feeling of kinship - the realisation that the author's goal was to teach how to use the book, and how to get by without it on hand). I found the book far more entertaining to read when I realized that I would have to evaluate it in this context, even though one might think that this would actually make it more difficult to read. I was surprised that something as simple as glancing at a note on beetroot production rates could make me do more cognitive work than any cheap trick...
[LINK] Updating Drake's Equation with values from modern astronomy
by DanArmak — 2016-04-30T22:08:07.858Z

A paper published in Astrobiology: A New Empirical Constraint on the Prevalence of Technological Species in the Universe (PDF), A. Frank and W.T. Sullivan.

From the abstract:

Recent advances in exoplanet studies provide strong constraints on all astrophysical terms in the Drake equation. [...] We find that as long as the probability that a habitable zone planet develops a technological species is larger than ~ 10-24, humanity is not the only time technological intelligence has evolved.

They say we now know with reasonable certainty the total number of stars ever to exist (in the observable universe), and the average number of planets in the habitable zone. But we still don't know the probabilities of life, intelligence, and technology arising. They call this cumulative unknown factor fbt.

Their result: for technological civilization to arise no more than once, with probability 0.01, in the lifetime of the observable universe, fbt should be no greater than ~ 2.5 x 10-24.

Discussion

It's convenient that they calculate the chance technological civilization ever arose, rather than the chance one exists now. This is just the number we need to estimate the likelihood of a Great Filter.

They state their result as "[if we set fbt ≤ 2.5 x 10-24, then] at in a statistical sense were we to rerun the history of the Universe 100 times, only once would a lone technological species occur". But I don't know what rerunning the Universe means. I also can't formulate this as saying "if we hadn't already observed the Universe to be apparently empty of life, we would expect it to contain or to have once contained life with a probability of 1024", because that would ignore the chance that another civilization (if it counterfactually existed) would have affected or prevented the rise of life on Earth. Can someone help reformulate this?

I don't know if their modern values for star and planet formation have been used in previous discussions of the Fermi paradox or the Great Filter. (The papers they cite for their values date from 2012, 2013 and 2015.) I also don't know if these values should be trusted, or what concrete values had been used previously. People on top of the Great Filter discussion probably already updated when the astronomical data came in.
Stability of optimal predictor schemes under a broader class of reductions
by Vanessa Kosoy — 2016-04-30T14:17:35.000Z

We define a new class of reductions which preserve optimal predictor schemes, generalizing the previous notion of pseudo-invertible reductions. These are reductions that preserve the estimation problem on average but allow for large variance.

Definition 1

Fix an error space Δ of rank 2. Consider (f,μ), (g,ν) distributional estimation problems, ^ζ=(ζ,rζ,aζ) a {0,1}∗-valued (poly,log)-bischeme. ^ζ is called a Δ-pseudo-invertible weak reduction of (f,μ) to (g,ν) when there is a polynomial p:N→N s.t. the following conditions hold:

(i) Eμk[(EUrζ(k,j)[g(^ζkj(x))]−f(x))2]∈Δ

(ii) Prμk×Urζ(k,j)[νp(k)(^ζkj(x))=0]∈Δ

(iii) There is M>0 and ^R=(R,rR,aR) a Q∩[0,M]-valued (poly,log)-bischeme s.t.

Eνp(k)×UrR(k,j)[(^Rkj(y)−Prμk×Urζ(k,j)[^ζkj(x)=y]νp(k)(y))2]∈Δ

(iv) There is a {0,1}∗-valued (poly,log)-scheme ^ξ=(ξ,rξ,aξ) s.t.

Eμk×Urζ(k,j)[∑x′∈{0,1}∗|PrUrξ(k,j)[^ξkj(^ζkj(x,z),w)=x′]−Prμk×Urζ(k,j)[x′′=x′∣^ζkj(x′′,z′)=^ζkj(x,z)]|]∈Δ

Such ^ξ is called a Δ-pseudo-inverse of ^ζ.

^ζ=(ζ,rζ,aζ) a {0,1}∗-valued (poly,log)-scheme is called a Δ-pseudo-invertible weak reduction of (f,μ) to (g,ν) when it becomes such when adding trivial dependence on j.

Definition 2

An error space Δ of rank 2 is called stable when for any non-constant polynomial p:N→N and δ∈Δ, the function δ′(k,j):=δ(p(k),j) is in Δ.

Theorem

Fix Δ a stable error space of rank 2. Suppose there is a polynomial h:N2→N s.t. h−1∈Δ. Consider (f,μ), (g,ν) distributional estimation problems, ^ζ a Δ-pseudo-invertible weak reduction of (f,μ) to (g,ν) and ^Pg a Δ(poly,log)-optimal predictor scheme for (g,ν). Define ^Pf by

^Pkjf(x,y1y2…yh(k,j)z1z2…zh(k,j)):=1h(k,j)h(k,j)∑i=1^Pp(k),jg(^ζkj(x,yi),zi)

Here, we assume the lengths of the yi and zi are compatible with ^ζ and ^Pg respectively. Then, ^Pf is a Δ(poly,log)-optimal predictor scheme for (f,μ).

Proposition 1

Consider (f,μ), (g,ν) distributional estimation problems. Suppose ^ζ=(ζ,rζ,aζ) is a weak Δ-pseudo-invertible reduction of (f,μ) to (g,ν) and ^ξ=(ξ,rξ,aξ) is it's Δ-pseudo-inverse. Then there is δ∈Δ s.t. for any bounded function h:{0,1}∗2→R

|Eμk×Urζ(k,j)×Urξ(k,j)[h(^ξkj(^ζkj(x,z),w),^ζkj(x,z))]−Eμk×Urζ(k,j)[h(x,^ζkj(x,z))]|≤(sup|h|)δ(k,j)

Proposition 1 proved exactly as Proposition 2 for unidistributional estimation problems.

Proposition 2

Consider X a finite set, μ a probability measure on X, h:X→R and s,t∈R. Then

Eμ[(h(x)−s)2−(h(x)−t)2]=(Eμ[h(x)]−s)2−(Eμ[h(x)]−t)2

Proof o

...
Weekly LW Meetups
by FrankAdamek — 2016-04-29T15:50:27.706Z

This summary was posted to LW Main on April 29th. The following week's summary is here.

New meetups (or meetups with a hiatus of more than a year) are happening in:

Nairobi mini-Meetup #1: Double Crux: 30 April 2016 03:34PM

Irregularly scheduled Less Wrong meetups are taking place in:

Baltimore / UMBC Weekly Meetup: How To Actually Change Your Mind (part 3): 01 May 2016 03:00PM
European Community Weekend: 02 September 2016 03:35PM
San Francisco Meetup: Projects: 01 May 2016 06:15PM

The remaining meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

Raleigh, NC (RTLW) Discussion Meetup: 07 May 2022 07:30PM
Sydney Rationality Dojo - May: 01 May 2016 04:00PM
Sydney Rationality Dojo - July: 03 July 2016 04:00PM
Washington, D.C.: Utopias: 01 May 2016 03:30PM

Locations with regularly scheduled meetups: Austin, Berkeley, Berlin, Boston, Brussels, Buffalo, Canberra, Columbus, Denver, Kraków, London, Madison WI, Melbourne, Moscow, Mountain View, New Hampshire, New York, Philadelphia, Research Triangle NC, Seattle, Sydney, Tel Aviv, Toronto, Vienna, Washington DC, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers and a Slack channel for daily discussion and online meetups on Sunday night US time.

If you'd like to talk with other LW-ers face to face, and there is no meetup in your area, consider starting your own meetup; it's easy (more resources here). Check one out, stretch your rationality skills, build community, and have fun!

In addition to the handy sidebar of upcoming meetups, a meetup overview is posted on the front page every Friday. These are an attempt to collect information on all the meetups happening in upcoming weeks. The best way to get your meetup featured is still to use the Add New Meetup feature, but you'll also have the benefit of having your meetup mentioned in a weekly overview. These overview posts are moved to the discussion section when the new post goes up.

Please note that for your meetup to appear in the weekly meetups feature, you need to post your meetup before the Friday before your meetup!

If you check Less Wrong irregularly, consider subscribing to one or more city-specific mailing list in order to be notified when an irregular meetup is happening: Atlant...
Meetup : San Francisco Meetup: Projects
by rocurley — 2016-04-29T02:34:53.137Z

Discussion article for the meetup : San Francisco Meetup: Projects

WHEN:
01 May 2016 06:15:00PM (-0700)

WHERE:
1597 Howard St. San Francisco, CA

We'll be meeting to work on projects. Bring something to work on, come up with something there, or help other people out!
As always, I can be reached at 301-458-0764 if you need to be let in.

Discussion article for the meetup : San Francisco Meetup: Projects
Double Corrigibility: better Corrigibility
by Stuart_Armstrong — 2016-04-28T14:46:43.528Z

A putative new idea for AI control; index here.

Corrigibility was an attempt to allow an AI to safely change its values, without seeking to provoke or avoid the change. The idea is that, when the AI's utility changes from u to v at time t, it maximises a meta-utility U such that

U≤t = u

U>t = v + E(u|u→u) - E(v|u→v).

Here u→u designates the event that u remains unchanged, while u→v designates the change.

As has been shown, adding those expectation terms means the AI will not seek to provoke or resist such a utility change - it is indifferent to the change. Those expectation terms can be designated compensatory rewards.

What it is not indifferent to, though, is changes to E. If the AI knows that change u→v is coming, then it has incentives to change E to E' with a high E'(u|u→u) and a low E'(v|u→v). Its ability to rewire is limited by the fact that E' will serve as its future expectation estimator, so it won't mangle it completely, but it certainly can twist it a bit.

Notice that the AI is now no longer indifferent to the change of utility. It can benefit from rewiring E this way only if a change is coming up. Notice that E' probably can't rewire itself any more, so the rewired AI will be indifferent, but not before rewiring.

Double corrigibility

To solve this problem, we need to turn to corrigibility a second time. Let ρ be the AI's current expectation/probability estimator. Let ρ' be one that it can conceivably change into. Let C be a random variable designating the sum total of all compensatory rewards. Now, to guarantee that C is finite would require bounds on all the utility functions and some discount rate. In practice, if often will be finite. That's because E(C)=E(u0|u0→u0)-E(un |un-1→un, H), where u0 is the AI's first utility, un its last, and H intermediate history. So some milder restrictions on the ui should suffice to bound C.

Then, whenever the AI takes an action that changes ρ to ρ', it gets the special meta-compensatory reward

Eρ(C|ρ→ρ)-Eρ(C|ρ→ρ').

This prevents it from deriving any advantage to C via changes to ρ. Or, more precisely, it prevents it from deriving any expected advantage, that it can figure out using its current ρ.

Note that it doesn't prevent improvements to ρ in general, it just removes ...
Games for factoring out variables
by Stuart_Armstrong — 2016-04-28T13:38:10.000Z

All the methods proposed for factoring out B (for having the AI maximise a certain value while 'ignoring' it's impact via B) can be put on the same general footing. For some set A, define a function Q on A×A with Q(a,a′)≥0 and ∑a,a′Q(a,a′)=1.

Then for a utility u, the general requirements is for the AI to maximise the quantity:

EQ(u)=∑a,a′,bE(u|B=b,A=a)P(B=b|A=a′)Q(a,a′),

subject to some constraints on Q.

Let's play a game

Define the two-player game G by allowing each player to have moves in A. The expected utility, for player 1, of the moves (a,a′) is defined to be ∑bE(u|B=b,A=a)P(B=b|A=a′). To completely define the game, we need the expected utility for the second player. We'll just set that to ensure the game is symmetric the expected utility of player two for (a,a′) is the same as the expected utility for player one for (a′,a).

Then Q is effectively a probability distribution over the choice of possible moves for G. Standard symmetric games include the stag hunt, the prisoner's dilemma, and the coordination game. For our purposes, since the actions for have meaningful labels, we'll be considering skew coordination game given as follows:

+-----+-----+-----+
| | l | r |
+-----+-----+-----+
| l |(0,0)|(1,1)|
+-----+-----+-----+
| r |(1,1)|(0,0)|
+-----+-----+-----+

These 'games' will be mainly used to show that the various methods reach different solutions in different situations, hence that they are genuinely different methods.

Standard maximalisation

If we constrain Q(a,a′)=0 if a≠a′, then that equation becomes:

∑a,a′,bE(u|B=b,A=a)P(B=b|A=a′)Q(a,a′)=∑a,bE(u|B=b,A=a)P(B=b|A=a)Q(a,a)=∑aE(u|A=a)Q(a,a).

This makes Q(a,a) into the action distribution over A=a the AI must choose from to maximise u in the classical sense. In terms of G, this means the AI will make the superrational choice between the two players, if they lack any way of distinguishing themselves. Thus it will pick s,s for the stag hunt, c,c in the prisoner's dilemma, and l,l or r,r (or some mixture between the two) in the skew coordination game.

Double variable maximalisation

First consider that there are no additional constraints on Q. The AI can pick the values to be whatever it wants. It therefore acts like a dictator in G, choosing both player's outcomes. Under this, it will pick S,S for the stag hunt, D,C in the prisoner's dilemma, L,R or R,L (or some mixture between the two) i

...
Meetup : Baltimore / UMBC Weekly Meetup: How To Actually Change Your Mind (part 3)
by iarwain1 — 2016-04-28T12:10:35.773Z

Discussion article for the meetup : Baltimore / UMBC Weekly Meetup: How To Actually Change Your Mind (part 3)

WHEN:
01 May 2016 03:00:00PM (-0400)

WHERE:
Performing Arts and Humanities Bldg Room 456, 1000 Hilltop Cir, Baltimore, MD 21250

Meetup is in Room 456, the philosophy department conference room. As usual, parking restrictions don't apply on weekends so park wherever you want.

We are currently going through How to Actually Change Your Mind. This week we'll be discussing the last three sequences (Sequence I: Seeing with Fresh Eyes, Sequence J: Death Spirals and the Cult Attractor, and Sequence K: Letting Go).

Discussion article for the meetup : Baltimore / UMBC Weekly Meetup: How To Actually Change Your Mind (part 3)
Meetup : Washington, D.C.: Utopias
by RobinZ — 2016-04-28T01:28:52.633Z

Discussion article for the meetup : Washington, D.C.: Utopias

WHEN:
01 May 2016 03:30:00PM (-0400)

WHERE:
Reynolds Center

copy-pasted from a draft by the member who proposed the topic:

We’ll be gathering in the courtyard to discuss utopias. The idea behind the topic is, in light of changes such as automation, climate change, and the recent problems with the financial system, whether it’s possible to come up with a far-future society that is sustainable, self-consistent, and where people, if at all possible, can be happy as well. What problems do you think plague society today, what would be a solution, and how do we get there? It’s the ultimate in world-building!

As always, side conversations are allowed and encouraged.

Upcoming meetups:

May 8: Survivorship Bias

May 15: Fun & Games

May 22: Typical Mind Fallacy

Discussion article for the meetup : Washington, D.C.: Utopias
JFK was not assassinated: prior probability zero events
by Stuart_Armstrong — 2016-04-27T12:50:01.000Z

A lot of my work involves tweaking the utility or probability of an agent to make it believe - or act as if it believed - impossible or almost impossible events. But we have to be careful about this; an agent that believes the impossible may not be so different from one that doesn't.

Consider for instance an agent that assigns a prior probability of zero to JFK ever having been assassinated. No matter what evidence you present to it, it will go on disbelieving the "non-zero gunmen theory".

Initially, the agent will behave very unusually. If it was in charge of JFK's security in Dallas before the shooting, it would have sent all secret service agents home, because no assassination could happen. Immediately after the assassination, it would have disbelieved everything. The films would have been faked or misinterpreted; the witnesses, deluded; the dead body of the president, that of twin or an actor. It would have had huge problems with the aftermath, trying to reject all the evidence of death, seeing a vast conspiracy to hide the truth of JFK's non-death, including the many other conspiracy theories that must be false flags, because they all agree with the wrong statement that the president was actually assassinated.

But as time went on, the agent's behaviour would start to become more and more normal. It would realise the conspiracy was incredibly thorough in its faking of the evidence. All avenues it pursued to expose them would come to naught. It would stop expecting people to come forward and confess the joke, it would stop expecting to find radical new evidence overturning the accepted narrative. After a while, it would start to expect the next new piece of evidence to be in favour of the assassination idea - because if a conspiracy has been faking things this well so far, then they should continue to do so in the future. Though it cannot change its view of the assassination, its expectation for observations converge towards the norm.

If it does a really thorough investigation, it might stop believing in a conspiracy at all. At some point, the probability of a miracle will start to become more likely than a perfect but undetectable conspiracy. It is very unlikely that Lee Harvey Oswald shot at JFK, missed, and the president's head exploded simultaneously for unrelated natural causes. But after a while, such a miraculous explanation will start to become more l

...
Is the average ethical review board ethical from an utilitarian standpoint?
by ChristianKl — 2016-04-27T12:11:21.958Z

Many people argue that Facebook's study of how the emotions of it's users changed depending on the emotional content of messages in their facebook feed wouldn't have been approved by the average ethical review board because facebook didn't seek informed consent for the experiment.

Is the harm that the average ethical review board prevents less than the harm that they cause by preventing research from happening? Are principles such as requiring informed consent from all research participants justifiable from an utilitarian perspective?
JFK was not assassinated: prior probability zero events
by Stuart_Armstrong — 2016-04-27T11:47:40.765Z

A lot of my work involves tweaking the utility or probability of an agent to make it believe - or act as if it believed - impossible or almost impossible events. But we have to be careful about this; an agent that believes the impossible may not be so different from one that doesn't.

Consider for instance an agent that assigns a prior probability of zero to JFK ever having been assassinated. No matter what evidence you present to it, it will go on disbelieving the "non-zero gunmen theory".

Initially, the agent will behave very unusually. If it was in charge of JFK's security in Dallas before the shooting, it would have sent all secret service agents home, because no assassination could happen. Immediately after the assassination, it would have disbelieved everything. The films would have been faked or misinterpreted; the witnesses, deluded; the dead body of the president, that of twin or an actor. It would have had huge problems with the aftermath, trying to reject all the evidence of death, seeing a vast conspiracy to hide the truth of JFK's non-death, including the many other conspiracy theories that must be false flags, because they all agree with the wrong statement that the president was actually assassinated.

But as time went on, the agent's behaviour would start to become more and more normal. It would realise the conspiracy was incredibly thorough in its faking of the evidence. All avenues it pursued to expose them would come to naught. It would stop expecting people to come forward and confess the joke, it would stop expecting to find radical new evidence overturning the accepted narrative. After a while, it would start to expect the next new piece of evidence to be in favour of the assassination idea - because if a conspiracy has been faking things this well so far, then they should continue to do so in the future. Though it cannot change its view of the assassination, its expectation for observations converge towards the norm.

If it does a really thorough investigation, it might stop believing in a conspiracy at all. At some point, the probability of a miracle will start to become more likely than a perfect but undetectable conspiracy. It is very unlikely that Lee Harvey Oswald shot at JFK, missed, and the president's head exploded simultaneously for unrelated natural causes. But after a while, such a miraculous explanation will start to become more l...