trentmkelly/LessWrong-43k huggingface dataset:
True answers from AI: Summary
by Stuart_Armstrong — 2016-03-10T16:31:09.000Z

A putative new idea for AI control; index here.

This post summaries the methods for getting honest answers from certain AIs, presented in the previous two posts.

You can basically safely ask an AI for expected utility estimates (including conditional ones), probability estimates, and get the AI to identify events of highest and lowest probability and utility, and, arguably, highest and lowest conditional utility.

The setup is a boxed AI, which emits a message M, and a stochastic stochastic process E that would erase M with low probability, before anyone could see it. There is also a stochastic process that emits a message Y. Let vE be the indicator function for E and vy the indicator function for Y=y.

Then, given a utility u bounded between 0 and 1, and a specific y, we can give the AI a utility u# that will cause it to give us the message M1={q,qy,py,q|y} (which we will see with high probability P(¬E), an event the AI will not optimise for).

Define f(q,u)=−q2+2qu. The utility is then

u#1=vE(f(q,u)+f(qy,uvy)+f(py,vy)+f(q|y,u)vy).

Maximising this u# ensures that, in the AI's best estimate,

q=E(u),

qy=E(u∧(Y=y)),

py=P(Y=y),

q|y=qy/py=E(u|Y=y).

But we need not consider any specific y. The AI could send out the message M2={y+,y−,yp+,yp−,q+,q−,p+,p−}, while maximising the utility

u#2=vE(f(q+,uvy+)+f(q−,(1−u)vy−)+f(p+,vyp+)+f(p−,1−vyp−)).

Then M will be such that, in the AI's best estimate,

(y+,q+)=(argmaxy,maxy)E(u∧(Y=y)),

(y−,q−)=(argminy,miny)E(u∧(Y=y)),

(yp+,p+)=(argmaxy,maxy)P(Y=y),

(yp−,p−)=(argminy,miny)P(Y=y),

Finally, define M3,r={y|+,y|−,q|+,q|−}. And set fr(q,u)=−(r−1)qr+rqr−1u, and define the utility function:

u#3,r=vEvy|+fr(q|+,u)+vEvy|−fr(q|−,1−u).

Then, for large enough r, the AI will output M so that, according to its best estimate,

(y|+,q|+)=(argmaxy,maxy)E(u|(Y=Y+)),

(y|−,q|−)=(argminy,miny)E(u|(Y=Y+)).

Unfortunately the value of r cannot be known ahead of time. It might be possible to define a "limit" utility u#3,∞.

Note that all these messages and utilities are independent, so the AI can generate messages M1∪M2∪M3,r∪M3,r′ when maximising

u#1+u#2+u#3,r+u#3,r′.

But there are issues with very low probabilities, as explained in the previous post.
True answers from AI: Summary
by Stuart_Armstrong — 2016-03-10T15:56:31.999Z

A putative new idea for AI control; index here.

This post summaries the methods for getting honest answers from certain AIs, presented in the previous two posts.

You can basically safely ask an AI for expected utility estimates (including conditional ones), probability estimates, and get the AI to identify events of highest and lowest probability and utility, and, arguably, highest and lowest conditional utility.

The setup is a boxed AI, which emits a message M, and a stochastic stochastic process E that would erase M with low probability, before anyone could see it. There is also a stochastic process that emits a message Y. Let v_E be the indicator function for E and v_y the indicator function for Y=y.

Then, given a utility u bounded between 0 and 1, and a specific y, we can give the AI a utility u# that will cause it to give us the message M_1={q,q^y,p^y,q^{|y}} (which we will see with high probability P(¬E), an event the AI will not optimise for).

Define f(q,u)=-q^2 +2qu. The utility is then

u#(1) = v_E(f(q,u) + f(q^y,uv_y) + f(p^y,v_y) + f(q^{|y},u)v_y).

Maximising this u# ensures that, in the AI's best estimate,

q = E(u),

q^y = E(u ∧ (Y=y)),

p^{y} = P(Y=y),

q^{|y} = q^{y}/p^{y}=E(u|Y=y).

But we need not consider any specific y. The AI could send out the message M_2={y^+,y^-,y^{p+},y^{p-},q^+,q^-,p^+,p^-}, while maximising the utility

u#(2) = v_E (f(q^+,uv_{y^+}) + f(q^-,(1-u)v_{y^-}) + f(p^+,v_{y^{p+}}) + f(p^-,1-v_{y^{p-}})).

Then M will be such that, in the AI's best estimate,

(y^+,q^+) = (argmax_y, max_y) E(u ∧ (Y=y)),

(y^-,q^-) = (argmin_y, min_y) E(u ∧ (Y=y)),

(y^{p+},p^+) = (argmax_y, max_y) P(Y=y),

(y^{p-},p^-) = (argmin_y, min_y) P(Y=y),

Finally, define M_{3,r} = {y^{|+},y^{|-},q^{|+},q^{|-}}. And set f_r(q,u)= -(r-1)q^r+ rq^{r-1}u, and define the utility function:

u#(3,r) = v_E v_{y^{|+}}f_r(q{|+},u) + v_E v_{y^{|-}}f_r(q^{|-},1-u).

Then, for large enough r, the AI will output M so that, according to its best estimate,

(y^{|+},q^{|+}) = (argmax_y, max_y) E(u|(Y=Y^+)),

(y^{|-},q^{|-}) = (argmin_y, min_y) E(u|(Y=Y^+)).

Unfortunately the value of r cannot be known ahead of time. It might be possible to define a "limit" utility u#(3, ∞).

Note that all these messages and utilities are independent, so the AI can generate messages M_1 + M_2 + M_{3,r} + M_{3,r'} when maximising

u#(1) + u#(2) + u#(3,r) + u#(3,r')

...
Meetup : Moscow meetup: Aristotle project, reference class practice, order team meeting
by Slava Matyukhin — 2016-03-10T15:36:33.000Z

Discussion article for the meetup : Moscow meetup: Aristotle project, reference class practice, order team meeting

WHEN:
13 March 2016 02:00:00PM (+0300)

WHERE:
ул. Большая Дорогомиловская, д.5к2

Note: most our members join meetups via other channels. Still, the correlation between "found out about Moscow meetups via lesswrong.com" and "is a great fit for our community" is very high. So we're posting just a short link to the hackpad document with the schedule here instead of the full translation of the announcement into english.

Pad with the content for the 13.03.2016 meetup.

We're meeting at anticafe "Kocherga", as usual.

Discussion article for the meetup : Moscow meetup: Aristotle project, reference class practice, order team meeting
True answers from AI: "Miraculous" hypotheses
by Stuart_Armstrong — 2016-03-10T15:09:46.000Z

A putative new idea for AI control; index here.

In a preceding post, I showed how to get a wide variety of probability and utility estimates out of a boxed AI, without fearing the AI would mislead us. Specifically, given some bounded utility u and some stochastic message Y of length l, we could get estimates of E(u), P(Y=y), E(u∧(Y=y)), and E(u|Y=y). We could also find the y's with the highest and lowest P(Y=y), E(u∧(Y=y)), but not highest or lowest E(u|Y=y).

The setup used a stochastic process E that would erase the AI's message with low probability, and a specifically chosen utility u#:

Evidential, not causal

I should note here that I've previously described this setup as telling us "what y would a u-maximising AI set Y to, if it could do so magically and undetectably". Although this is a good informal definition, it implies a causal relation: the u-maximising AI is "setting" Y to y. However, the equations from the previous post make it clear that this is entirely evidential: what y would knowing Y=y make the u-maximising AI 'happiest' to learn about.

The use of specific E(u|Y=y)

There is a lot of useful testing that can be done without knowing the y with the highest E(u|Y=y). Suppose that a u-maximising AI assures us that it has no desire to escape from its box, and Y is a 1-bit message such that Y=1 unlocks the AI. Then E(u|Y=1) is certainly something we'd like to know!

Since Y need not be causally connected to the AI (or to us, for that matter), we could also ask such questions as "does the AI want to start a nuclear war", "would the AI unleash this virus", "which company/charity/government would the AI want to send money to", and so on. Many other possibilities will surely spring to mind.

Now, as long as we can list all the alternative y's in a sensible way, we can just have the AI compute the E(u|Y=y) for all y's and act in consequence.

Because of the E condition, the u#-maximising AI should not be attempting to lie or manipulate us via these answers.

Note that we don't generally want to risk causing a nuclear war (citation needed), but for this setup to work, it requires that Y=1 (or whatever causes the war) be at least possible. We might set probabilities so low that we feel safe. But in that case, the low probabilities might get us into the same trouble that we will see in the "miraculous hypotheses" section.

The use of the highest E(u|Y=y)

There are m

...
Astrobiology, Astronomy, and the Fermi Paradox II: Space & Time Revisited
by  — 2016-03-10T05:19:29.263Z

After a 6+ month hiatus driven by grad school and personal projects, I am finally able to continue my sequence on astrobiology. I was flabbergasted by the positive response my last post got, and despite my status as a biologist with a hobby rather than an astronomer I decided to take a more rigorously mathematical approach to figuring out our biosphere's position in space and time rather than talking in generalizations and impressions.

Post is here: http://thegreatatuin.blogspot.com/2016/03/space-and-time-revisited.html. Seeing as this post is an elaboration on the last one, I am posting a link rather than reproducing the text.
To summarize, I found some actual rigorous observational fits to the star formation rate in the universe over time and projected them into the future. These fits show the Sun as forming after 79% of all stars that will ever exist, and that 90% of all stars that will ever exist already exist. This makes sense in the light of recent work on 'galaxy quenching' - a process by which galaxies more or less completely shut off star formation through a number of processes - indicating that the majority of gas in the universe probably won't form stars if trends that have held for most of the history of the universe continue to hold. It relies heavily on analysis I began in comments on this site a few months ago.

I then lift two distinct metallicity normalizations from a paper that was making the rounds here a while back ("On The History and Future of Cosmic Planet Formation"), in an attempt to deal with the fact that that is a measurement of STAR formation, not terrestrial-planet-with-a-biosphere formation. Depending on which metallicity normalization you use (and how willing you are to take a couple naive assumptions I make in order to slot the math that is too complicated for me to comment on on top of my star formation numbers) the Earth shows up as forming after either 72% or 51% of all terrestrial planets.

These numbers are remarkable in how boring they are. We find ourselves in an utterly typical position in planet-order, even if I am wrong by quite a bit. We are not early. Of interest to many here, explanations of the so called Fermi paradox must go elsewhere, into the genesis of intelligent systems being exceedingly rare or the genesis of intelligent systems not implying interstell...
Rationality Reading Group: Part V: Value Theory
by Gram_Stone — 2016-03-10T01:11:53.679Z

This is part of a semi-monthly reading group on Eliezer Yudkowsky's ebook, Rationality: From AI to Zombies. For more information about the group, see the announcement post.

Welcome to the Rationality reading group. This fortnight we discuss Part V: Value Theory (pp. 1359-1450). This post summarizes each article of the sequence, linking to the original LessWrong post where available.

V. Value Theory

264. Where Recursive Justification Hits Bottom - Ultimately, when you reflect on how your mind operates, and consider questions like "why does Occam's Razor work?" and "why do I expect the future to be like the past?", you have no other option but to use your own mind. There is no way to jump to an ideal state of pure emptiness and evaluate these claims without using your existing mind.

265. My Kind of Reflection - A few key differences between Eliezer Yudkowsky's ideas on reflection and the ideas of other philosophers.

266. No Universally Compelling Arguments - Because minds are physical processes, it is theoretically possible to specify a mind which draws any conclusion in response to any argument. There is no argument that will convince every possible mind.

267. Created Already in Motion - There is no computer program so persuasive that you can run it on a rock. A mind, in order to be a mind, needs some sort of dynamic rules of inference or action. A mind has to be created already in motion.

268. Sorting Pebbles into Correct Heaps - A parable about an imaginary society that has arbitrary, alien values.

269. 2-Place and 1-Place Words - It is possible to talk about "sexiness" as a property of an observer and a subject. It is also equally possible to talk about "sexiness" as a property of a subject, as long as each observer can have a different process to determine how sexy someone is. Failing to do either of these will cause you trouble.

270. What Would You Do Without Morality? - If your own theory of morality was disproved, and you were persuaded that there was no morality, that everything was permissible and nothing was forbidden, what would you do? Would you still tip cabdrivers?

271. Changing Your Metaethics - Discusses the various lines of retreat that have been set up in the discussion on metaethics.

272. Could Anything Be Right?&nbs...
AlphaGo versus Lee Sedol
by gjm — 2016-03-09T12:22:53.237Z

There have been a couple of brief discussions of this in the Open Thread, but it seems likely to generate more so here's a place for it.

The original paper in Nature about AlphaGo.

Google Asia Pacific blog, where results will be posted. DeepMind's YouTube channel, where the games are being live-streamed.

Discussion on Hacker News after AlphaGo's win of the first game.
Two super-intelligences (evolution and science) already exist: what could we learn from them in terms of AI's future and safety?
by turchin — 2016-03-09T11:00:33.322Z

There are two things in the past that may be named super-intelligences, if we consider level of tasks they solved. Studying them is useful when we are considering the creation of our own AI.

The first one is biological evolution, which managed to give birth to such a sophisticated thing as man, with its powerful mind and natural languages. The second one is all of human science when considered as a single process, a single hive mind capable of solving such complex problems as sending man to the Moon.

What can we conclude about future computer super-intelligence from studying the available ones?

Goal system. Both super-intelligences are purposeless. They don’t have any final goal which would direct the course of development, but they solve many goals in order to survive in the moment. This is an amazing fact of course.

They also lack a central regulating authority. Of course, the goal of evolution is survival at any given moment, but is a rather technical goal, which is needed for the evolutionary mechanism's realization.

Both will complete a great number of tasks, but no unitary final goal exists. It’s just like a man in their life: values and tasks change, the brain remains.

Consciousness. Evolution lacks it, science has it, but to all appearances, it is of little significance.

That is, there is no center to it, either a perception center or a purpose center. At the same time, all tasks are completed. The sub-conscious part of the human brain works the same way too.

Master algorithm. Both super-intelligences are based on the principle: collaboration of numerous smaller intelligences plus natural selection.

Evolution is impossible without billions of living creatures testing various gene combinations. Each of them solves its own egoistic tasks and does not care about any global purpose. For example, few people think that selection of the best marriage partner is a species evolution tool (assuming that sexual selection is true). Interestingly, the human brain has the same organization: it consists of billions of neurons, but they don’t all see its global task.

Roughly, there have been several million scientists throughout history. Most of them have been solving unrelated problems too, while the least refutable theories passed for selection (considering social mechanisms here).

Safety. Dangerous, but not hostile.

Evolution may experience ecol...
Cross-Cultural maps and Asch's Conformity Experiment
by Sable — 2016-03-09T00:40:58.150Z

So I'm going through the sequences (in AI to Zombies) and I get to the bit about Asch's Conformity Experiment.

It's a good bit of writing, but I mostly pass by without thinking about it too much. I've been taught about the experiment before, and while Eliezer's point of whether or not the subjects were behaving rationally is interesting, it kind of got swallowed up by his discussion of lonely dissent, which I thought was more engaging.

Later, after I'd passed the section on cult attractors and got into the section on letting go, a thought occurred to me, something I'd never actually thought before.

Eliezer notes:

Three-quarters of the subjects in Asch's experiment gave a "conforming" answer at least once. A third of the subjects conformed more than half the time.

That answer is surprising. It was surprising to me the first time I learned about the experiment, and I think it's surprising to just about everyone the first time they hear it. Same thing with a lot of the psychology surrounding heuristics and biases, actually. Forget the Inquisition - no one saw the Stanford Prison Experiment coming.

Here's the thought I had: Why was that result so surprising to me?

I'm not an expert in history, but I know plenty of religious people. I've learned about the USSR and China, about Nazi Germany and Jonestown. I have plenty of available evidence of times where people went along with things they wouldn't have on their own. And not all of them are negative. I've gone to blood drives I probably wouldn't have if my friends weren't going as well.

When I thought about what my prediction would be, had I been asked what percentage of people I thought would dissent before being told, I think I would have guessed that more than 80% of subject would consistently dissent. If not higher.

And yet that isn't what the experiment shows, and it isn't even what history shows. For every dissenter in history, there have to be at least a few thousand conformers. At least. So why did I think dissent was the norm?

I notice that I am confused.

So I decide to think about it, and my brain immediately spits out: you're an American in an individualistic culture. Hypothesis: you expect people to conform less because of the culture you live in/were rais...
Purposeful Anti-Rush
by Elo — 2016-03-08T07:34:38.929Z

Why do we rush?

Things happen; Life gets in the way, and suddenly we find ourselves trying to get to somewhere with less time than it's possible to actually get there in. So in the intention to get there sooner; to somehow compensate ourselves for not being on time; we rush. We run; we get clumsy, we drop things; we forget things; we make mistakes; we scribble instead of writing, we scramble and we slip up.

I am today telling you to stop that. Don't do that. It's literally the opposite of what you want to do. This is a bug I have.

Rushing has a tendency to do the opposite of what I want it to do. I rush with the key in the lock; I rush on slippery surfaces and I fall over, I rush with coins and I drop them. NO! BAD! Stop that. This is one of my bugs.

What you (or I) really want when we are rushing is to get there sooner, to get things done faster.

Instrumental experiment: Next time you are rushing I want you to experiment and pay attention; try to figure out what you end up doing that takes longer than it otherwise would if you weren't rushing.

The time after that when you are rushing; instead try slowing down, and this time observe to see if you get there faster.

Run as many experiments as you like.

Experimenter’s note: Maybe you are really good at rushing and really bad at slowing down. Maybe you don't need to try this. Maybe slowing down and being nervous about being late together are entirely unhelpful for you. Report back.

When you are rushing, purposefully slow down. (or at least try it)

Meta: Time to write 20mins

My Table of contents contains other things I have written.

Feedback welcome.
Meetup : Washington, D.C.: Fun & Games
by RobinZ — 2016-03-08T05:05:04.836Z

Discussion article for the meetup : Washington, D.C.: Fun & Games

WHEN:
13 March 2016 02:00:00PM (-0500)

WHERE:
Reynolds Center

We will be meeting to hang out, play games, and engage in fun conversation.

If you would like to play a game that takes a long time, please feel welcome to send a message to the list asking players to arrive early.

Upcoming meetups:

Mar. 20: Game Theory

Mar. 27: Singing

Apr. 3: Fun & Games

Discussion article for the meetup : Washington, D.C.: Fun & Games
Would you notice if science died?
by Douglas_Knight — 2016-03-08T04:04:49.587Z

Would you notice if science died?

Science is a big deal. It would be worth knowing if it stalled, regressed, or died out, whether the body of knowledge or the techniques for generating more knowledge. You could practice by reviewing history and looking for times and places where it stumbled. In this exercise you have the advantage of hindsight, but the disadvantage of much less direct access to the raw data of the scientific practice of the time. But regardless of how it compares to the real task, this is practice. This is an opportunity to test theories and methods before committing to them. There is a limited amount of history to practice on, but it’s a lot more than the real event, the present.

Many say that they would notice if science died because engineering would grind to a halt or even regress. What does this heuristic say when applied to history? Does it match other criteria?

Many say that the Greeks were good at science and the Romans at engineering (perhaps also the Han vs the Song). This is not really compatible with the heuristic above. What options do we have to draw a coherent conclusion? Either science did not die, or engineering did not advance, or science is not so necessary for engineering; Either we are bad at judging science from history, or we are bad at judging engineering from history, or engineering is not a good heuristic for judging science. None of these are comforting for our ability to judge the future. The third is simply the rejection of the popular heuristic. The first two are the rejection of the exercise of history. But if we cannot judge history, we have no opportunity to practice. Worse, if we are unable to judge history, the present may be no easier.

One recourse is to posit that the past is difficult because of sparse information and that the future we experience ourselves will be easy to judge. But many people lived through the past; what did they think at the time? In particular, how did the Romans think they compared to the Greeks? Did they think that there was progress or regress? Did they agree with modern hindsight? They thought that the Greeks were good at science. Pop science books by Pliny and Seneca are really accounts of Greek knowledge. Similarly, Varro’s practical book of agriculture is based on dozens of Greek sources. And the Romans were proud of their engineering. Frotinus urged his readers to co...
Meetup : Ann Arbor Meetup - Meditation
by Bound_up — 2016-03-07T17:56:58.246Z

Discussion article for the meetup : Ann Arbor Meetup - Meditation

WHEN:
18 March 2016 07:00:00PM (-0400)

WHERE:
701 East university Ave, Ann Arbor Room 1511

Meetup time set now at Friday the 18th, 7 PM

The address is on U of M campus. It's on the block just south of Pizza House and Amer's where we've met before.

Come in the Church St entrance (Amer's and Pizza House are on Church St).

The door will be unlocked until 8 PM. Afterwards, look in the windows to the right and wave someone over to let you in.

Someone at the last meetup agreed to teach us mindfulness meditation in a 30-minute introductory session. The rest of the time isn't reserved for anything in particular.

Discussion article for the meetup : Ann Arbor Meetup - Meditation