trentmkelly/LessWrong-43k huggingface dataset:
What Evidence Is AlphaGo Zero Re AGI Complexity? 
by RobinHanson — 2017-10-22T02:28:45.764Z

Eliezer Yudkowsky write a post on Facebook on on Oct 17, where I replied at the time. Yesterday he reposted that here (link), minus my responses. So I’ve composed the following response to put here:

I have agreed that an AI-based economy could grow faster than does our economy today. The issue is how fast the abilities of one AI system might plausibly grow, relative to the abilities of the entire rest of the world at that time, across a range of tasks roughly as broad as the world economy. Could one small system really “foom” to beat the whole rest of the world?

As many have noted, while AI has often made impressive and rapid progress in specific narrow domains, it is much less clear how fast we are progressing toward human level AGI systems with scopes of expertise as broad as those of the world economy. Averaged over all domains, progress has been slow. And at past rates of progress, I have estimated that it might take centuries.

Over the history of computer science, we have developed many general tools with simple architectures and built from other general tools, tools that allow super human performance on many specific tasks scattered across a wide range of problem domains. For example, we have superhuman ways to sort lists, and linear regression allows superhuman prediction from simple general tools like matrix inversion.

Yet the existence of a limited number of such tools has so far been far from sufficient to enable anything remotely close to human level AGI. Alpha Go Zero is (or is built from) a new tool in this family, and its developers deserve our praise and gratitude. And we can expect more such tools to be found in the future. But I am skeptical that it is the last such tool we will need, or even remotely close to the last such tool.

For specific simple tools with simple architectures, architecture can matter a lot. But our robust experience with software has been that even when we have access to many simple and powerful tools, we solve most problems via complex combinations of simple tools. Combinations so complex, in fact, that our main issue is usually managing the complexity, rather than including the right few tools. In those complex systems, architecture matters a lot less than does lots of complex detail. That is what I meant by suggesting that architecture isn’t the key to AGI.

You might claim that once we have enough good simple tools,

...
Comparing LICDT and LIEDT
by abramdemski — 2017-10-21T23:41:46.000Z

Attempted versions of CDT and EDT can be constructed using logical inductors, called LICDT and LIEDT. It is shown, however, that LICDT fails XOR Blackmail, and LIEDT fails Newcomb. One interpretation of this is that LICDT and LIEDT do not implement CDT and EDT very well. I argue that they are indeed forms of CDT and EDT, but stray from expectations because they also implement the ratifiability condition I discussed previously. Continuing the line of thinking from that post, I discuss conditions in which LICDT=LIEDT, and try to draw out broader implications for decision theory.

Thanks to Scott and Sam for discussions shaping this post. Also thanks to many participants at AISFP for discussions shaping my current view of counterfactuals.

I'm not sure who gets credit for LICDT and LIEDT, but they've been discussed around MIRI since shortly after logical induction itself. LIEDT was sort of the obvious first thing to try; LICDT is a slight variation. (Scott thinks Jessica may have come up with LICDT.) They might be thought of as the punching bag for better logical induction DTs to be contrasted with (although, Tsvi wrote up a likely-better baseline proposal).

Both LICDT and LIEDT use a logical inductor which has been run for n steps, Pn. I'll abbreviate an agent as An (parameterizing by the same n as for the inductor), with An=a to say that the agent takes action a from some action set A. We can define self-referential sentences Sn↔Pn(Sn)<ϵ. Both LICDT and LIEDT explore when such sentences are true. We can take whichever action the agent least expects itself to do conditioned on its exploring. This forces the agent to take every action with frequency at least ϵ/|A| in the limit, and also makes the exploration pseudorandom in the sense that the logical inductor cannot predict it much better than to assign probability ϵ to exploration (and therefore, neither can any poly-time computable predictor).

When it isn't taking an action due to the exploration clause, LIEDT chooses actions based on the expected utility conditioning on each action. Utility is defined as a logically uncertain variable (LUV), in the terminology of the logical induction paper. Let Un be the LUV for the utility of An, and En(Un|An=a) be the conditional expectation of Un in Pn given that the agent takes action a. The conditional expectation is always well-defined thanks to the exploration, which

...
Introducing Goalclaw, personal goal tracker
by Nic_Smith — 2017-10-21T20:10:13.410Z

Quite a while ago, I wrote that there should be more software tools to assist with instrumental rationality. My recent attempt to create such a tool, GOALCLAW, is now available. GOALCLAW is a general goal tracking webapp which currently provides an average of how the tags entered for events day-to-day affect your goals, with plans to make more tag-based metrics and projections available in the near future.

GOALCLAW is new:

A few editing features are missing and should be added in the next few months

The built-in analysis needs to be expanded from averages

I'm very interested in feedback on how to make this a more useful goal-tracker

The general idea is to make patterns in what's going on around you and what you're doing a bit more obvious, so you can then investigate, verify/experiment, and act to achieve your goals

You can download information entered for importing into spreadsheets, stats program, etc.
Trope Dodging
by Hazard — 2017-10-21T18:43:34.729Z

Epistemic status: 2 1/2 hours of hammering out an idea that’s been in my head for a while. Not too novel, yet a few useful points at the end. Rehashes some ideas from A Human's Guide to Words.

I think that it is useful to model people as having filters that exist somewhere in their decision making process, filters which ask the question, “Does what I’m about to do match a behaviour trope that I’ve blacklisted?”

If the answer is yes, then the potential action gets vetoed and the brain goes back to the drawing board. If the answer is no, then the potential action gets bumped further up the decision making chain.

The effect of some filters are more visible than others. What many people would call one’s “social filter” is something that normally only vetoes doing or saying something after the potential action has already been brought to your conscious attention. You think about the accurate remark you could make about your colleague's outfit, but then don’t say it because your filter deemed it rude.

Ugh fields look a lot like filters that get stronger and stronger with time and operate in the no-man's-land between your opaque subconscious processes and your visible conscious ones.

If you want to be more specific about what it means to “match a trope”, we can replace trope with similarity cluster, and say that a potential action matches the trope if it is within a certain distance of the epicenter of the similarity cluster that a trope references.

For my personal use, I've called this sort of filtering Trope Dodging, but if another name seems better, I'm happy to concede.

Here’s what this could look like in day to day life:

Sasha thinks that whining and complaining are just terrible ideas. She grew up with siblings that always complained and who never tried to make their lives better. This left a strong impression on Sasha, and now she has a filter that keeps her from doing things that are too similar to the trope of “being whiny”. She doesn’t even like to discuss with her classmates about the difficulty of the various university course they are enrolled in, because her filter registers that as too close to “being whiny”.

You, being a lovable well read rationalist, can probably guess how these sorts of filters are less than optimal. Even if if you have determined that it really is in your best interest to not do anything that would be a

...
Halloween costume: Paperclipperer
by Elo — 2017-10-21T06:33:01.203Z

Original post: http://bearlamp.com.au/halloween-costume-paperclipperer/

Guidelines for becoming a paperclipperer for halloween.

Supplies

Paperclips (some as a prop, make your life easier by buying some, but show effort by making your own)

pliers (extra pairs for extra effect)

metal wire (can get colourful for novelty) (Florist wire)

crazy hat (for character)

Paperclip props. Think glasses frame, phone case, gloves, cufflinks, shoes, belt, jewellery...

if party going - Consider a gift that is suspiciously paperclip like. example - paperclip coasters, paperclip vase, paperclip party-snack-bowl

Epic commitment - make fortune cookies with paperclips in them. The possibilities are endless.

Epic: paperclip tattoo on the heart. Slightly less epic, draw paperclips on yourself.

Character

While at the party, use the pliers and wire to make paperclips. When people are not watching, try to attach them to objects around the house (example, on light fittings, on the toilet paper roll, under the soap. When people are watching you - try to give them to people to wear. Also wear them on the edges of your clothing.

When people ask about it, offer to teach them to make paperclips. Exclaim that it's really fun! Be confused, bewildered or distant when you insist you can't explain why.

Remember that paperclipping is a compulsion and has no reason. However that it's very important. "you can stop any time" but after a few minutes you get fidgety and pull out a new pair of pliers and some wire to make some more paperclips.

Try to leave paperclips where they can be found the next day or the next week. cutlery drawers, in the fridge, on the windowsills. And generally around the place. The more home made paperclips the better.

Try to get faster at making paperclips, try to encourage competitions in making paperclips.

Hints for conversation:

Are spiral galaxies actually just really big paperclips?

Have you heard the good word of our lord and saviour paperclips?

Would you like some paperclips in your tea?

How many paperclips would you sell your internal organs for?

Do you also dream about paperclips (best to have a dream prepared to share)

Conflict

The better you are at the character, the more likely someone might try to spoil your character by getting in your way, stealing your props, taking your paperclips. The more you are okay with it, the better. ideas like, "t

...
Halloween costume: Paperclipperer
by Elo — 2017-10-21T06:32:36.868Z

Original post: http://bearlamp.com.au/halloween-costume-paperclipperer/

Guidelines for becoming a paperclipperer for halloween.

Supplies

Paperclips (some as a prop, make your life easier by buying some, but show effort by making your own)

pliers (extra pairs for extra effect)

metal wire (can get colourful for novelty) (Florist wire)

crazy hat (for character)

Paperclip props. Think glasses frame, phone case, gloves, cufflinks, shoes, belt, jewellery...

if party going - Consider a gift that is suspiciously paperclip like. example - paperclip coasters, paperclip vase, paperclip party-snack-bowl

Epic commitment - make fortune cookies with paperclips in them. The possibilities are endless.

Epic: paperclip tattoo on the heart. Slightly less epic, draw paperclips on yourself.

Character

While at the party, use the pliers and wire to make paperclips. When people are not watching, try to attach them to objects around the house (example, on light fittings, on the toilet paper roll, under the soap. When people are watching you - try to give them to people to wear. Also wear them on the edges of your clothing.

When people ask about it, offer to teach them to make paperclips. Exclaim that it's really fun! Be confused, bewildered or distant when you insist you can't explain why.

Remember that paperclipping is a compulsion and has no reason. However that it's very important. "you can stop any time" but after a few minutes you get fidgety and pull out a new pair of pliers and some wire to make some more paperclips.

Try to leave paperclips where they can be found the next day or the next week. cutlery drawers, in the fridge, on the windowsills. And generally around the place. The more home made paperclips the better.

Try to get faster at making paperclips, try to encourage competitions in making paperclips.

Hints for conversation:

Are spiral galaxies actually just really big paperclips?

Have you heard the good word of our lord and saviour paperclips?

Would you like some paperclips in your tea?

How many paperclips would you sell your internal organs for?

Do you also dream about paperclips (best to have a dream prepared to share)

Conflict

The better you are at the character, the more likely someone might try to spoil your character by getting in your way, stealing your props, taking your pa...
Instrumental Rationality: Postmortem
by  — 2017-10-21T06:23:31.707Z

[In April, I set off to write a series of essays about instrumental rationality. Now that the project’s reached a pretty good stopping point, I’m looking back to see how my expectations and goals played out.]

Initial Goals:

[What I originally wanted the book to be like. My estimates vs reality]

Originally, I wanted to write something that would tie together all the current research on topics like motivation, planning, and habits. I felt like lots of LessWrong posts touched upon certain areas, like hyperbolic discounting, but there wasn’t a central place where it all came together.

I wanted a new central beacon to point people to when The Sequences didn’t quite fit.

I’d envisioned a sequence of essays which would give an overview of the latest developments in the field followed by concrete techniques, ala CFAR.

Here are some of the topics I’d originally wanted to cover:

Willpower

Attention

Habits

Behavioral Economics

Motivation

When I began this project, I had Planning 101 under my belt, which had taken me about 20 hours to complete. As far as base rates went, it seemed reasonable to think that the other topics would take a similar amount of time.

Looking back, it feels a little silly to think that Past Owen thought he could take on five more of those 20+ hour chunk projects. That would have easily been 100+ hours, in addition to editing, compiling, and a bunch of additional grunt work which snuck up on me.

(What was I thinking?)

In the end, I only managed to write one more primer—Habits 101—which ended up taking about twice as long, 40 hours, for just the actual writing portion. Reading up on the articles took up additional time, probably another 10 hours or more.

Though I didn’t complete my initial vision, I actually did fairly well on my own estimates:

I’d given myself an internal completion date of the beginning of September, and I actually finished around that time. (Hooray!)

I’d also estimated that I’d write about 10,000 words of new content for the book. And I went about 50% more than that, writing about 15,000 words of actual new content. (Hooray!)

(Although the counterfactual for the new content prediction isn’t as good as it seems because I probably would have written some of those blog posts regardless of whether or not I also held the intention to make a sequence.)

The Finished Result:

[Evaluating the end product, some things which didn't mak

...
Instrumental Rationality 7: Closing Disclaimer
by  — 2017-10-21T06:03:19.714Z

[Instrumental Rationality Sequence 7/7]

[A disclaimer that instrumental rationality as presented in this sequence is incomplete. Your feelings are also important! Pay attention to them.]

After reading through this sequence, you might be feeling very excited to go out and try some of this stuff out. You might think about imposing this instrumental rationality stuff upon many areas of your life.

And for that, I have a major cautionary warning.

Rationality can be dangerous because it's an ontology. And while it's not the One Ontology to Rule Them All, it can often feel like that when you're within the rationality framing.

Here’s an analogy:

Owen is a person who wants to get work done, but often finds himself playing video games. He also feels bad about doing so because it doesn’t fit in with his self-image. Maybe there’s also something here about society has shaped his values, but the actual root cause isn’t that important. The main point is that some part of him likes playing the games.

So he’s following his intuitive feelings, but also there’s guilt somewhere in the system.

Now let’s say he bumps into instrumental rationality—planning, habits, motivation—the whole package.

What I've presented in this sequence is a way of looking at things, kind of like a set of special glasses.

Once Owen puts on these glasses, he starts to see new opportunities to use his shiny new techniques, like TAPs, to try and remove his video-game-playing habit. But note that the very idea of techniques, of transforming concepts into concrete actionables, is only something he sees when he puts on his Rationality Glasses.

My worry is when people use rationality as the lens through which they view the world for so long that they forget that there’s something important that’s hidden underneath the Rationality Glasses layer. Owen might just end up thinking that the Rationality Glasses show how the world is, rather than merely a useful way of looking at the world.

And instrumental rationality, or at least the way that I’ve presented it here, will have its own ideological biases. This isn’t necessarily bad; it’s a necessary consequence of any way of looking at the world. There’ll always be implicit values for any system you choose to use.

For rationality, these values are about highly striving for things like Optimization and Self-Improvement. I worry that this implicit

...
AlphaGo Zero and the Foom Debate
by Eliezer Yudkowsky — 2017-10-21T02:18:50.130Z

AlphaGo Zero uses 4 TPUs, is built entirely out of neural nets with no handcrafted features, doesn't pretrain against expert games or anything else human, reaches a superhuman level after 3 days of self-play, and is the strongest version of AlphaGo yet.

The architecture has been simplified. Previous AlphaGo had a policy net that predicted good plays, and a value net that evaluated positions, both feeding into lookahead using MCTS (random probability-weighted plays out to the end of a game). AlphaGo Zero has one neural net that selects moves and this net is trained by Paul Christiano-style capability amplification, playing out games against itself to learn new probabilities for winning moves.

As others have also remarked, this seems to me to be an element of evidence that favors the Yudkowskian position over the Hansonian position in my and Robin Hanson's AI-foom debate.

As I recall and as I understood:

Hanson doubted that what he calls "architecture" is much of a big deal, compared to (Hanson said) elements like cumulative domain knowledge, or special-purpose components built by specialized companies in what he expects to be an ecology of companies serving an AI economy.
When I remarked upon how it sure looked to me like humans had an architectural improvement over chimpanzees that counted for a lot, Hanson replied that this seemed to him like a one-time gain from allowing the cultural accumulation of knowledge.

I emphasize how all the mighty human edifice of Go knowledge, the joseki and tactics developed over centuries of play, the experts teaching children from an early age, was entirely discarded by AlphaGo Zero with a subsequent performance improvement. These mighty edifices of human knowledge, as I understand the Hansonian thesis, are supposed to be the bulwark against rapid gains in AI capability across multiple domains at once. I was like "Human intelligence is crap and our accumulated skills are crap" and this appears to have been bourne out.

Similarly, single research labs like Deepmind are not supposed to pull far ahead of the general ecology, because adapting AI to any particular domain is supposed to require lots of components developed all over the place by a market ecology that makes those components available to other companies. AlphaGo Zero is much simpler than that. To the extent that nobody else can run out and build AlphaGo Zero, it's either beca

...
Rationality and overriding human preferences: a combined model
by Stuart_Armstrong — 2017-10-20T22:47:04.000Z

A putative new idea for AI control; index here.

Previously, I presented a model in which a “rationality module” kept track of two things: how well a human was maximising their actual reward, and whether their preferences had been overridden by AI action.

The second didn’t integrate well into the first, and was tracked by a clunky extra Boolean. Since the two didn’t fit together, I was going to separate the two concepts, especially since the Boolean felt a bit too... Boolean, not allowing for grading. But then I realised that they actually fit together completely naturally, without the need for arbitrary Booleans or other tricks.

Feast or heroin famine

Consider the situation detailed in the following figure. An AI has the opportunity to surreptitiously inject someone with heroin (I) or not do so (¬I). If it doesn’t, the human will choose to enjoy a massive feast (F); if it does, the human will instead choose more heroin (H).

So the human policy is given by π(I)=H,π(¬I)=F. The human rationality and reward are given by a pair (m,R), where R is the human reward and m measures their rationality - how closely their actions conform with their reward.

The module m can be seen as a map from rewards to policies (or, since policies are maps from histories to actions, m can be seen as mapping histories and rewards to actions). The pair (m,R) are said to be compatible if m(R)=π, the human policy.

There are three natural Rs to consider here: Rp, a generic pleasure. Next, Re, the ‘enjoyment’ reward, where enjoyment is pleasure endorsed as ‘genuine’ by common judgement. Assume that Rp(H)=1, Rp(F)=1/3, Re(F)=1/2, and Re(H)=0. Finally, there is the twisted reward Rt, which is Rp conditional on I and Re

There are two natural ms: mr, the fully rational module. And mf, the module that is fully rational conditional on I, but always maps to H if I is chosen: m(R)(I)=H, for all R.

The pair mr(Re) is not compatible with π: it predicts that the human would take action F following I (feast following injection). The reward Rp is compatible with neither m: it predicts H following ¬I (heroin following no injection).

The other three pairs are compatible: mr(Rt), mf(Rt), and mf(Re) all give the correct policy π.

Overriding rewards and regret

This leads to a definition of when the AI is overriding human rewards. Given a pair (m,R), with m(R)=π, and AI’s action A overrides the human reward if π|A i

...