trentmkelly/LessWrong-43k huggingface dataset:
Blog Post Day (Unofficial)
by Daniel Kokotajlo — 2020-02-18T19:05:47.140Z

TL;DR: You are invited to join us online on Saturday the 29th, to write that blog post you've been thinking about writing but never got around to. Comment if you accept this invitation, so I can gauge interest.

The Problem:

Like me, you are too scared and/or lazy to write up this idea you've had. What if it's not good? I started a draft but... Etc.

The Solution:

1. Higher motivation via Time Crunch and Peer Encouragement

We'll set an official goal of having the post put up by midnight. Also, we'll meet up in a special-purpose discord channel to chat, encourage each other, swap half-finished drafts, etc. If like me you are intending to write the thing one day eventually, well, here's a reason to make that day this day.

2. Lower standards via Time Crunch and Safety in Numbers

Since we have to be done by midnight, we'll all be under time pressure and any errors or imperfections in the posts will be forgivable. Besides, they can always be fixed later via edits. Meanwhile, since a bunch of us will be posting on the same day, writing a sloppy post just means it won't be read much, since everyone will be talking about the handful of posts that turn out to be really good. If you are like me, these thoughts are comforting and encouraging.

Evidence this Works:

MIRI Summer Fellows Program had a Blog Post Day towards the end, and it was enormously successful. It worked for me, for example: It squeezed two good posts out of me. (OK, so one of them I finished up early the next morning, so I guess it technically doesn't count. But in spirit it does: It wouldn't have happened at all without Blog Post Day.) More importantly, MSFP keeps doing this every year, even though opportunity cost for them is much higher (probably) than the opportunity cost for you or me. I don't know what else you had planned for Saturday the 29th... (Actually, if you do have something else planned, but otherwise want to participate in Blog Post Day, let me know. Maybe we can pick a different day.)

Side Benefits:

It'll be fun!
Big Yellow Tractor (Filk)
by Gordon Seidoh Worley — 2020-02-18T18:43:09.133Z

To the tune of "Big Yellow Taxi", with apologies to Joni Mitchell

we'll pave paradise and put up a parking lot

if that's what it takes to make all the wild suffering stop

the pain it only grows and grows

until we boil it up in our pot

we'll pave paradise and put up a parking lot

we'll take all the creatures and put 'em in a simulation

where there's no pain or death or starvation

oh! the pain it only grows and grows

until we boil it up in our pot

we'll pave paradise and put up a parking lot

hey all you people put away decision theory

i don't care about max utils if we can't end suffering, please!

the pain it only grows and grows

until we boil it up in our pot

we'll pave paradise and put up a parking lot

late last night I heard 'bout a brand new plan

a big yellow tractor's gonna clear away all the land

oh! the pain it only grows and grows

until we boil it up in our pot

we'll pave paradise and put up a parking lot
Training Regime Day 4:  Murphyjitsu
by Mark Xu — 2020-02-18T17:33:12.523Z

Introduction

Recall that applied rationality is being able to properly decide which advisors to use during decision making.

Imagine that I am throwing a ball up into the air and catching it. Occasionally, I might drop the ball because I am not very good at catching. Now, imagine that I throw the ball up and at the top of the arc, the ball just freezes. Are you surprised?

There is an advisor that is always trying to guess what move the world is going to make next. This part of you is called the inner simulator, or inner sim for short. This advisor has many powers but is particularly underused by most people. This technique aims to extract a lot of useful information from the inner sim.

Inner Simulator

Here are some prompts that will allow you to get in touch with your inner sim:

Imagine your closest companion. Imagine going up to them and flicking them on the nose. How do they react?

Imagine that the weather suddenly shifts. If it’s currently sunny, there is now a torrential downpour. If it’s currently raining, the rain abruptly ceases. How surprised are you?

Imagine throwing a rubber ball at the wall. How does it bounce? You can probably see the trajectory pretty well, but if I asked you to calculate it explicitly, would you be able to do it?

All of these events are pretty complicated, and yet it seems like most humans are easily able to make reasonably accurate predictions. Even more impressively, these predictions can be made extremely quickly. The inner sim is a very fast and powerful advisor.

Of course, speed and power have costs. The inner sim requires a lot of data. You (and your evolutionary lineage) live on Earth, where there is gravity and physics is generally Newtonian. Correspondingly, your inner sim is extremely well trained on Newtonian physics with Earth gravity. There are many reasons that humans find things like orbital mechanics and relativity so hard; one of them is that your inner sim is not very good at dealing with non-Newtonian situations that aren’t on Earth.

Similarly, you (and your evolutionary lineage) spend a lot of time interacting with other humans. Correspondingly, your inner sim is fairly good at navigating extremely complex social situations and modeling other people with high accuracy. However, if you've spent most of your life among people of a certain culture, your inner sim will make a lot of errors if you ever use it on peo

...
In a rational world is there a place for ideology?
by wachichornia — 2020-02-18T17:09:03.448Z

I'm reading Superforecasters and one of the things that differentiates good from bad forecasters is ideology: those who try to fit the world in their left/right wing view are less effective at forecasting.

Does Ideology still has a place in a rational world? Why (not)?

My limited perspective tells me in theory ideology would give you ideas to try and would bias potential solutions.
(In)action rollouts
by Stuart_Armstrong — 2020-02-18T14:48:19.160Z

Overall summary post here.

I've previously looked at subagents in the context of stepwise inaction baselines. But there have been improvements to the basic stepwise inaction design, to include inaction rollouts. I'll be looking at those in this post.

The baseline

The stepwise inaction baseline compares st, the current state, with s′t, what the current state would have been had the agent previously taken the noop action ∅t−1, instead of at−1, its actual action.

Fix a policy π0. Let s(t)t+τ be the state the environment would be in if the agent had followed π0 from state st, for τ turns. Let s′(t)t+τ be the same, except that it started from state s′t instead of st.

The inaction rollout has π0 being the noop policy, but that is not necessary. The basic idea is to capture delayed impacts of ∅t−1 by comparing not just st and s′t, but the s(t)t+τ and s′(t)t+τ as well.

Given some value function vk, define Vk so that Vk(s)=maxπvk(s,π). Or, equivalently, if π∗k is the policy that maximises Vk, Vk(s)=vk(s,π∗k). Then for a discount factor γ define the rollout value of a state ~st as

RVk(~st)=(1−γ)∞∑j=0γjVk(~s(t)t+k).

This is just the discounted future values of Vk, given ~st and the policy π0.

The impact measure is then defined, as in this post, as

DA(st;s′t)=∑k∈Kwkf(RVk(st)−RVk(s′t)),

with RVk replacing Vk.

To get away with a subagent

I will make one extra assumption here: that A grants SA knowledge of the policy that A is following[1].

As in this post, assume that the subagent is constructed in a single action aSA. And, after building the subagent, the agent A will only do ∅, so that there are no subsequent penalties (hence s′t+j=st+j, so s′(t+j)t+j+τ=s(t+j)t+j+τ and the RVk are strictly identical). If the first action of π0 is ∅t, then A will instead do ∅′t on that turn.

So we only need to worry about the penalty between st (following after aSA) and s′t (following after ∅t−1).

But now note that RVk itself is just a value measure; we could write it V′k. Then A just has to program SA so that, if A implements π0 followed by π∗k after τ turns, then SA will act to ensure that V′k(st) (a term that is a sum of various value functions) is equal to V′k(s′t) (from t onward, a constant).

Stricter impact measures/counterfactuals

There is something slightly dubious about the previous section. It assumes that A could counterfactually implement "π0 followed by π∗k after τ turns" (which necessa

...
Counterfactuals versus the laws of physics
by Stuart_Armstrong — 2020-02-18T13:21:02.232Z

Overall summary post here.

I've claimed that, given a subagent:

[...] restrictions on increased power for the agent ("make sure you never have the power to increase the rewards") become restrictions on the actual policy followed for the subagent ("make sure you never increase these rewards").

But how does this work? The problem is that "never have the power" is a counterfactual statement: it doesn't matter what you do, only what you could potentially do. But "never increase" is the opposite: it matters not what you could potentially do, only what you do.

How does the subagent change this? Well, the counterfactual version is at least informally clear: the agent A could, by changing its policy, increase certain rewards.

But now suppose that A creates a subagent SA that will itself not increase the rewards, and, moreover, will interfere with A if A attempts to do so. Now A cannot increase those rewards even if it changed its policy. The laws of the environment - which, from A's perspective, now include SA - prevent A from ever being able to to increase the rewards, and thus fulfil A's penalty function.

This is very much the A "chaining itself to the mast".

The counterfactual we'd want (but don't have)

What we want is for the counterfactual of the impact penalty to also include the subagent: the agent and subagent together definitely have the "power" to change the rewards by changing their own policies. But, from A's perspective, SA's policy is a brute fact about the world, not amenable to change.

We'd like for the counterfactual to also include SA's policy, but for that, we'd need to identify and define subagents - a very tricky problem.
How to actually switch to an artificial body – Gradual remapping
by George3d6 — 2020-02-18T13:19:07.076Z
Link: https://blog.cerebralab.com/How_to_actually_switch_to_an_artificial_body_%E2%80%93_Gradual_remapping

How to actually switch to an artificial body – Gradual remapping

Note: Please feel free to message me about inconsistencies, references which you think should be added (or claims you think the current references don't back, or where there's evidence to the contrary). But please note, this is *obviously* not meant to be definitive essay on the concept. In other words "epistemic status: low-to-mid confidence, but, I doubt you can go much beyond mid confidence on these type of subjects"

One of the oldest transhumanist tropes is the idea of transferring your brain into a computer, thus achieving immortality and beyond-human abilities… etc, etc.

I am one of the people that believe this is actually possible, as in, not in the “Oh, in the year 3000 people will be able to do this” sort of way but in the “Barring tremendous progress in accurate delivery of Okazaki factors, thymus transplants and various MTOR inhibitors and drugs that are marketed as something other than and MTOR inhibitor but are actually probably just a confounded MTOR inhibitor, mind-uploading will become my passion project once I hit my 30s”.

I Why popular portrayals of the idea are impractical

Considering this, it sort of dis-tresses me how popular culture portrays this. When I say popular culture here I really mean “every single thing I’ve ever read or seen ob the subject”… though chances are I might not be looking in the correct place ?

To give a few examples:

a) Wikipedia’s page on the subject: https://en.wikipedia.org/wiki/Mind_uploading

b) Wikipedia’s page on transhumanism in general: https://en.wikipedia.org/wiki/Transhumanism

c) One of the most popular doomed-venture/scams around this are, 2045 initiative:

http://2045.com/

d) One of the more realistic doomed-venture/scams around this area, Netcome: https://nectome.com/the-case-for-glutaraldehyde-structural-encoding-and-preservation-of-long-term-memories/

e) That one episode of black mirror that reminded everyone The Go-Go’s were a thing and might have just been an inside-joke about making a bunch of neckbeards watch a half-hour lesbian soap opera: https://www.youtube.com/watch?v=8dnn31TBoM4

Basically, the ideas presented are as follows:

Using {science fiction / magic} we’ll scan the brain (destroying it in the process) with

...
Wireheading and discontinuity
by Michele Campolo — 2020-02-18T10:49:42.030Z

Outline: After a short discussion on the relationship between wireheading and reward hacking, I show why checking the continuity of a sensor function could be useful to detect wireheading in the context of continuous RL. Then, I give an example that adopts the presented formalism. I conclude with some observations.

Wireheading and reward hacking

In Concrete Problems in AI Safety, the term wireheading is used in contexts where the agent achieves high reward by directly acting on its perception system or memory or reward channel, instead of doing what its designer wants it to do. It is considered a specific case of the reward hacking problem, which more generally includes instances of Goodhart’s Law, environments with partially observable goals, etc. (see CPiAIS for details).

What's the point of this classification? In other words, is it useful to specifically focus on wireheading, instead of considering all forms of reward hacking at once?

If solving wireheading is as hard as solving the reward hacking problem, then it's probably better to focus on the latter, because a solution to that problem could be used in a wider range of situations. But it could also be that the reward hacking problem is best solved by finding different solutions to specific cases (such as wireheading) that are easier to solve than the more general problem.

For example, one could consider the formalism in RL with a Corrupted Reward Channel as an adequate formulation of the reward hacking problem, because that formalization models all situations in which the agent receives a (corrupted) reward that is different from the true reward. In that formalism, it is shown by a No Free Lunch Theorem that the general problem is basically impossible to solve, while it is possible to obtain some positive results if further assumptions are made.

Discontinuity of the sensor function

I've come up with a simple idea that could allow us to detect actions that interfere with the perception system of an agent—a form of wireheading.

Consider a learning agent that gets its percepts from the environment thanks to a device that provides information in real time (e.g. a self-driving car).

This situation can be modelled as a RL task with continuous time and continuous state space, where each state x∈X⊆Rn is a data point provided by the sensor. At each time instant, the agent exe...
Set Ups and Summaries
by Elizabeth — 2020-02-18T05:00:01.056Z

Part of the research process I’m developing involves reading and thinking through the first and last chapter of a book, and first and last few paragraphs of a chapter, to get an expectation of what’s to come (combined with some other stuff I call this pre-reading). I’m currently pondering how much you can get out of this, and specifically if it’s fair to reject a work because it failed pre-reading.

Pre-reading is in part derived from advice in How to Read a Book to find a books “Unity”, the idea being that you’ll better incorporate information into your understanding if you know how it connects to the author’s larger point. I objected to HtRaB’s advice on this topic in my review, because it seemed to be trying to enforce an orderliness that reality does not support. Looking for a unified narrative encourages the author to throw out anything that doesn’t fit their narrative, and the reader to ignore it even when it’s included. Even in situations where there is a fairly clear narrative you might not know it yet, and it’s important to be able to share raw data without prematurely deciding what it means.

Then I pre-read chapter 12 of Children in Colonial America (an anthology: chapters have a common theme but each is meant to stand alone), and my immediate reaction was “nothing with this ending is going to be any good.” The chapter is discussing specific individuals down to the last paragraph, with no attempt to summarize what’s come before. The start is better but not by much- the last sentence of the first paragraph should clearly be the first, and the contextualization the rest of the paragraph provides should have come after, not before, when I know why I care what percentage of 1770s Boston’s population was made up of children.

The paragraphs in question:

Boston, the American Revolution’s “cradle of liberty,” was a town full of children. As in British North America as a whole, over half the population was under the age of adulthood. Children participated in political actions as early as Boston’s first public protest against the Stamp Act, on August 14, 1765: an organizer described “two or three hundred little boys with a Flagg marching in a Procession on which was King, Pitt & Liberty for ever.”1 The first Bostonian to die in political violence was a young boy. Apprentices both brought on and suffered in the Massacre of 1770, and pushed their way into the

...
[Productivity] How not to use "Important // Not Urgent"
by aaq — 2020-02-17T23:42:31.178Z

Epistemic status: Personal experience.

This is an Eisenhower matrix, named after President Dwight Eisenhower.

Eisenhower matrices have a very useful, very specific role in my life these days.

I usually bust one out on paper when I'm feeling overwhelmed by my own anxiety, and use it to focus my attention on what's in the first quadrant. I tell myself that I'll return to it by the time I finish the first quadrant, but usually by the time I've actually filled that quadrant out, my panic has subsided completely and I feel no issues in discarding the 2x2.

For a while, I also used the Eisenhower matrix as the basis for my Todoist Premium workflow. I actually still think that's a pretty good system for someone just getting started with using Todoist, and wanting a little bit more structure with how to use it outside of the box.

But I noticed that often, I would add things that I wanted to do way out in the future to the "Important // Not Urgent" filter. This is a bad idea, because economics tells us that ideas of things you want to do in the far future should probably be discounted quite heavily, relative to things you want to do in maybe the next month or so. You might well find that 5 years from now, you don't give a shit about learning to play Vivaldi or looking into moving to a different place; those are all things you can leave as amorphous ideas to play around with in your mind.

So my advice: Don't use the Urgent // Not Important quadrant for anything that isn't due more than two weeks out. That's a pretty aggressive cutoff point, yes, but we're talking planning your personal life here, not business; and we're talking a tool meant to focus your executive function in good directions. You don't want to overstretch it.

One objection that I could forsee someone saying is, "Well, what if I have a recurring task that I want to do every day, but it doesn't pay off immediately for me?", to which I say, the todo list is usually a fundamentally bad format for that, because you're probably talking about something that resembles practice (exercise, anything artistic, even studying).

Practice routines are time-delimited activites, not task-delimited ones. You usually want to do them for similar intervals of time, and they are usually done inside of similar environments. Use a calendar instea...
Training Regime Day 3: Tips and Tricks
by Mark Xu — 2020-02-17T18:53:24.808Z

Introduction

When you're trying to learn martial arts, usually you go to a physical location and get instructed by a real live person in real time. However, there are also books on martial arts. It is unknown to me whether or not anyone can actually learn anything useful about martial arts just by reading a book, even if they practice everything they have read.

When you're learning something in meat space, the information is transferred from one mind into words and then into another mind. The other mind can confirm or deny understanding, paraphrase the content, ask for examples, etc. The first mind can then translate different parts of their knowledge into words, translate the knowledge in a different way, confirm or deny the correctness of the paraphrasing, etc. This feedback loop works to minimize the distortion of information as it becomes compressed into this insufficiency we know as language.

In contrast, when you're learning something from written works, knowledge is translated into fixed and static text, which is then absorbed by another mind. The other mind attempts to reverse-translate what they have read into some form of knowledge. If the static text contains instructions, the other mind will attempt to divine the intent behind the written instructions and perform the tasks asked of them. It is unknown to me how much distortion occurs in the absence of a tight feedback loop that limits towards proper understanding. It is my hope that the distortion is minimal enough that reading my words and trying to do my exercises causes slight positive benefits instead of active harm.

Even so, there are a number of tips and tricks that can be used when approaching this material that will make it more probable that you will benefit from it.

Tips

Boggle

When I'm learning something in a traditional classroom setting, sometimes I find myself in mechanical student mode, where I just write everything down that the teacher says without even thinking about it at all. For example, the teacher will say something like "the Earth goes around the Sun" and then I'll copy down "the Earth goes around the Sun." I don't think much of this phrase because I have a vague idea of what "Earth", "Sun", and "around" mean and it seems not entirely incongruous to put them together.

It seems to me that this mode of thought contains a profound lack of something. We might call this something c

...