trentmkelly/LessWrong-43k huggingface dataset:
Restrictions that are hard to hack
by Stuart_Armstrong — 2015-03-09T14:05:15.000Z

A putative new idea for AI control; index here.

Very much in the spirit of "if you want something, you have to define it, then code it, rather than assuming you can get if for free through some other approach."

Difficult children

Suppose you have a child, that you sent to play in their room. You want them to play quietly and silently, so you want them:

"I'll be checking up on you!"

The child, however, has modelled you well, and knows that you will look in briefly at midnight and then go away. The child has two main options:

#. Play quietly the whole time.
#. Be as noisy as they want, until around 23:59, then be totally quiet for two minutes, then go back to being noisy.
We could call the first option obeying the spirit of the law, and the second obeying the letter.

AI's, restrictions, and information

We could model children as ever-destructive chaotic AIs (why yes, I am a parent - how did you guess?), and the warning as a restriction that human "controllers" try and put on the behaviour of the AI. Unfortunately, the AI will generally see the restriction and adapt to it, undermining its effectiveness. A lot of suggestions for AI control revolved around putting out suggestions of this type, so it's worth asking if there's a way to make them more rigorous. Is there a way to code a restriction such that the AI will obey it's spirit?

The thing that eventually leapt out when comparing the two behaviours is that behaviour 2 is far more informative about what the restriction was, than behaviour 1 was. From 2 we can deduce that something unusual was happening around midnight, and that one of the two modes of behaviour was likely to be penalised if it was done at another time. Moreover, if the restriction were removed, then behaviour 1 would continue to be sensible, while behaviour 2 would be stupid and pointless.

Let's try and formalise these intuitions.

Motivations

Restricting the AI's behaviour seems an unpromising approach, as any smart AI could behave in any suitable way to trick us. Much more fruitful would be to restrict the AI's motivations. I would imagine a scenario like this:

An AI A(u) with motivation u is tasked with the creation of subagent B(v) with motivation v. The subagent B(v) will be fed evidence E (chosen by A(u)), and will then go on and do stuff, during which time it will receive further evidence F (which A(u) can predict, and maybe constrain). There i

...
Restrictions that are hard to hack
by Stuart_Armstrong — 2015-03-09T13:52:19.313Z

A putative new idea for AI control; index here.

Very much in the spirit of "if you want something, you have to define it, then code it, rather than assuming you can get if for free through some other approach."

Difficult children

Suppose you have a child, that you sent to play in their room. You want them to play quietly and silently, so you want them:

"I'll be checking up on you!"

The child, however, has modelled you well, and knows that you will look in briefly at midnight and then go away. The child has two main options:

Play quietly the whole time.

Be as noisy as they want, until around 23:59, then be totally quiet for two minutes, then go back to being noisy.

We could call the first option obeying the spirit of the law, and the second obeying the letter.

AI's, restrictions, and information

We could model children as ever-destructive chaotic AIs (why yes, I am a parent - how did you guess?), and the warning as a restriction that human "controllers" try and put on the behaviour of the AI. Unfortunately, the AI will generally see the restriction and adapt to it, undermining its effectiveness. A lot of suggestions for AI control revolved around putting out suggestions of this type, so it's worth asking if there's a way to make them more rigorous. Is there a way to code a restriction such that the AI will obey it's spirit?

The thing that eventually leapt out when comparing the two behaviours is that behaviour 2 is far more informative about what the restriction was, than behaviour 1 was. From 2 we can deduce that something unusual was happening around midnight, and that one of the two modes of behaviour was likely to be penalised if it was done at another time. Moreover, if the restriction were removed, then behaviour 1 would continue to be sensible, while behaviour 2 would be stupid and pointless.

Let's try and formalise these intuitions.

Motivations

Restricting the AI's behaviour seems an unpromising approach, as any smart AI could behave in any suitable way to trick us. Much more fruitful would be to restrict the AI's motivations. I would imagine a scenario like this:

An AI A(u) with motivation u is tasked with the creation of subagent B(v) with motivation v. The subagent B(v) will be fed evidence E (chosen by A(u)), and will then go on and do stuff, during which time it will receive further evidence F (which A(u) can predict, and maybe cons...
Counterfactual trade
by owencb — 2015-03-09T13:23:54.252Z

Counterfactual trade is a form of acausal trade, between counterfactual agents. Compared to a lot of acausal trade this makes it more practical to engage in with limited computational and predictive powers. In Section 1 I’ll argue that some human behaviour is at least interpretable as counterfactual trade, and explain how it could give rise to phenomena such as different moral circles. In Section 2 I’ll engage in wild speculation about whether you could bootstrap something in the vicinity of moral realism from this.

Epistemic status: these are rough notes on an idea that seems kind of promising but that I haven’t thoroughly explored. I don’t think my comparative advantage is in exploring it further, but I do think some people here may have interesting things to say about it, which is why I’m quickly writing this up. I expect at least part of it has issues, and it may be that it’s handicapped by my lack of deep familiarity with the philosophical literature, but perhaps there’s something useful in here too. The whole thing is predicated on the idea of acausal trade basically working.

0. Set-up
Acausal trade is trade between two agents that are not causally connected. In order for this to work they have to be able to predict the other’s existence and how they might act. This seems really hard in general, which inhibits the amount of this trade that happens.

If we had easier ways to make these predictions we’d expect to see more acausal trade. In fact I think counterfactuals give us such a method.

Suppose agents A and B are in scenario X, and A can see a salient counterfactual scenario Y containing agents A’ and B’ (where A is very similar to A’ and B is very similar to B’). Suppose also that from the perspective of B’ in scenario Y, X is a salient counterfactual scenario. Then A and B’ can engage in acausal trade (so long as A cares about A’ and B’ cares about B). Let’s call such trade counterfactual trade.

Agents might engage in counterfactual trade either because they do care about the agents in the counterfactuals (at least seems plausible for some beliefs about a large multiverse), or because it’s instrumentally useful as a tractable decision rule which works as a better approximation to what they’d ideally like to do than similarly tractab...
Open thread, Mar. 9 - Mar. 15, 2015
by MrMind — 2015-03-09T07:48:20.660Z

If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should be posted in Discussion, and not Main.

4. Open Threads should start on Monday, and end on Sunday.
Meetup : HPMOR Wrap Party - Los Angeles
by ladyastralis — 2015-03-09T04:08:44.495Z

Discussion article for the meetup : HPMOR Wrap Party - Los Angeles

WHEN:
14 March 2015 06:00:00PM (-0700)

WHERE:
2800 E Observatory Rd, Los Angeles, CA 90027

Meeting Place: Griffith Observatory front lawn

2800 E Observatory Rd, Los Angeles, CA 90027

Date/Time: March 14, 2015: 6:00pm

Cost: Free access to the complex, planetarium shows are $7

Contact email: ladyastralis at gmail

Confirmed attendees: About 14 (as of 3/8)

Please bring: A (picnic) blanket, some snacks/food, some way to read HPMOR that has its own light source (I called the observatory -- they turn off the lights pretty early), and a thermos of hot cocoa. Don't forget a coat!

Notes: The final planetarium show is at 8:45pm. A fitting tribute.
The complex closes at 10:00pm.
I will be wearing my Ravenclaw scarf.

Looking forward to finally meeting other HPMOR fans!
Amanda

Discussion article for the meetup : HPMOR Wrap Party - Los Angeles
Meetup : San Francisco Meetup: Board Games
by rocurley — 2015-03-08T23:24:09.657Z

Discussion article for the meetup : San Francisco Meetup: Board Games

WHEN:
09 March 2015 06:15:00PM (-0700)

WHERE:
1390 Market St., San Francisco, CA

(Note that this is back in my apartment, not app academy: we have tables and boardgames here).

We'll be meeting up to hang out and play games. If anyone has a co-op game, we could play biased boardgames, but we don't have any.

Discussion article for the meetup : San Francisco Meetup: Board Games
Meetup : Canberra HPMOR Wrap Party!
by DanielFilan — 2015-03-08T22:56:53.578Z

Discussion article for the meetup : Canberra HPMOR Wrap Party!

WHEN:
15 March 2015 06:00:00PM (+1100)

WHERE:
70/10 Thynne St, Bruce, ACT

Harry Potter and the Methods of Rationality is finally coming to a conclusion! To celebrate, we invite all Canberran Less Wrongers to our wrap-up party. (:
Snacks will be provided.

Make sure you read all the new chapters so that we can have some interesting discussions.

Note that this is on Sunday instead of Saturday (π day), and there will be no LW meeting on Saturday.

Discussion article for the meetup : Canberra HPMOR Wrap Party!
The economy of weirdness
by KatjaGrace — 2015-03-08T20:29:38.000Z

It is often said that you should spend your weirdness budget wisely. You should wear a gender-appropriate suit, and follow culture-appropriate sports, and use good grammar, and be non-specifically spiritual, and support moderate policies, and not have any tattoos around either of your eyes. And then on the odd occasion, when it happens to come up, you should gather up your entire weirdness budget and make a short, impassioned speech in favor of invertebrate equality. Or whatever you think is the very most effective use of weirdness. In short: you only get so much weirdness, so don’t use it up dressing like a clown or popularizing alternative sleep schedules.

While I agree the oddball activist will often get less airtime than her unassuming analog, and that weirdness is often a cost, the issue seems more complex. Let us better explore weirdness budgeting.

Model #1: Weirdness is badness

A first simple model is that people don’t like weird things, so if you have any, they will like you less in expectation. Weirdness is a kind of badness. On this model, I suppose the reason you would want to be weird at all is that you just are weird, and it is hard or unpleasant to keep it under control.

Some characteristics are certainly like this. For instance, being shockingly unable to open corkscrews, or tending to fart really loudly. These are just bad characteristics though, and don’t seem like they need to be budgeted differently from other bad but not weird characteristics, like being lazy and stupid. I don’t think this is what people have in mind when they say to spend your weirdness budget wisely.

Model #2: Weirdness is rarity is bad

Here is a closely related model. Weird traits are not inherently bad, but they are inherently unusual, and being unusual is inherently bad. On this model, the reason you want to have a weird trait could be that you like the trait, and so you want to make it less unusual.

If many people feel that way, then on this model, weird traits are tragedies of the commons. e.g. If everyone could be naked in the street, the world would be a better place for everyone. But sadly, because nobody does it, anyone who starts is socially punished. So it is only the very altruistic person who will pull off their pants and be ostracized for the common good.

Model #3: Weirdness among the cool kids is bad

This is like the last model, but explain...
Harry Potter and the Methods of Rationality discussion thread, March 2015, chapter 117
by Gondolinian — 2015-03-08T19:00:44.722Z

This is a new thread to discuss Eliezer Yudkowsky’s Harry Potter and the Methods of Rationality and anything related to it. This thread is intended for discussing chapter 117.

Plans for next chapter release:

Ch. 118 will post on March 9th, 2015, at 12pm Pacific (7pm UTC).

There is a site dedicated to the story at hpmor.com, which is now the place to go to find the authors notes and all sorts of other goodies. AdeleneDawner has kept an archive of Author’s Notes. (This goes up to the notes for chapter 76, and is now not updating. The authors notes from chapter 77 onwards are on hpmor.com.)

Spoiler Warning: this thread is full of spoilers. With few exceptions, spoilers for MOR and canon are fair game to post, without warning or rot13. More specifically:

You do not need to rot13 anything about HP:MoR or the original Harry Potter series unless you are posting insider information from Eliezer Yudkowsky which is not supposed to be publicly available (which includes public statements by Eliezer that have been retracted).

If there is evidence for X in MOR and/or canon then it’s fine to post about X without rot13, even if you also have heard privately from Eliezer that X is true. But you should not post that “Eliezer said X is true” unless you use rot13.
Plane crashes
by michael_b — 2015-03-08T16:50:53.108Z

So. Inevitably after a plane crash a discussion comes up where someone may say that they're worried about flying now, and someone else pulls out the statistic that driving to the airport is more dangerous than flying. I think this reasoning is basically correct on the long-term, but not appropriate in the short-term.

Suppose it's the day after flight MH370 mysteriously disappeared. Information is extremely sketchy. You're about to get on a similar plane, operated by the same airliner, taking off from the same airport flying the same route. Should you get on the plane? That is, are you wrong to worry more than usual when we have no idea what happened to MH370? I would say no. The complete disappearance of flight MH370 without warning and without a trace the day before says **update your priors** at least for the short-term.
Before the seed. Intro
by  — 2015-03-08T15:14:15.478Z

Til;dr a speculation of how kin selection could have driven evolution of inter-generational dialogue in ferns.

Here I offer a very broad review of knowledge about ferns & 'fern allies' we have today. The topic is possible organism-level adaptations for kin selection and population structuring.

Why ferns?

Imagine a flowering plant (as a child would draw, to generalize.)

It has roots, a stem, leaves, flowers and fruit. In its flowers, anthers and pistils negotiate pollination like the adults they are. In its fruit, baby plants dream, covered by integuments. The plant is one complex, breathing thing.

On no level it is perfect or 'harmonious'. There are resource allocation problems, meaning all seeds do not have equal chances to mature. There is competition between fathers. There are herbivores and malformations and volcanic eruptions, and the plant has pretty little defences against them, but -

- when you look at the seed -

- not entirely powerless. The tender sperm meets the egg inside a complex organ, protected from drying out and capable of recognizing specific pollen. The mother plant provides nutrition and encapsules the embryo in layers optimized for both survival and germination. Once established, it becomes an environment for its own progeny: the sporophyte (the adult specimen producing spores) hosts plenty of gametophytes (i.e. the sperm and eggs developing from the spores). The sporophyte is a mighty diploid (or triploid, or more...), while the gametophyte is haploid (generally), meaning it has but one set of chromosomes - half as much inherent genetic diversity to meet all the various demands posed by the environment.

The layout is not so bad, after all.

...and now imagine a fern.

Their spores fall down wherever chance takes them, and they are already haploid. They give rise to small gametophytes, whose sperm must swim for torturous centimetres to reach the ovule of another, or risk adding to genetic load of itself if (and that is a big if) inbreeding is actually possible.

With seeds, you have a fortress with nurseries hidden behind the most capable biochemical locks that have evolved in plants. Without seeds, you have farflung holdings struggling for survival, which your own fronds might deprive of sunlight. And oh, there's the problem of promoting own genomes without going extinct because of homozygosity.

Can adult sporophytes of seedless land pl

...
Bragging Thread March 2015
by Morendil — 2015-03-08T13:50:06.746Z

Your job, should you choose to accept it, is to comment on this thread explaining the most awesome thing you've done this month. You may be as blatantly proud of yourself as you feel. You may unabashedly consider yourself the coolest freaking person ever because of that awesome thing you're dying to tell everyone about. This is the place to do just that.

Remember, however, that this isn't any kind of progress thread. Nor is it any kind of proposal thread. This thread is solely for people to talk about the awesome things they have done. Not "will do". Not "are working on". Have already done. This is to cultivate an environment of object level productivity rather than meta-productivity methods.

So, what's the coolest thing you've done this month?

(Previous Bragging Thread)
Impartial ethics and personal decisions
by Emile — 2015-03-08T12:14:04.205Z

Some moral questions I’ve seen discussed here:

A trolley is about to run over five people, and the only way to prevent that is to push a fat bystander in front of the trolley to stop it. Should I?

Is it better to allow 3^^^3 people to get a dust speck in their eye, or one man to be tortured for 50 years?

Who should I save, if I have to pick between one very talented artist, and five random nobodies?

Do I identify as an utilitarian? a consequentialist? a deontologist? a virtue ethicist?

Yet I spend time and money on my children and parents, that may be “better” spent elsewhere under many moral systems. And if I cared as much about my parents and children as I do about random strangers, many people would see me as somewhat of a monster.

In other words, “commonsense moral judgements” finds it normal to care differently about different groups; in roughly decreasing order:

immediate family

friends, pets, distant family

neighbors, acquaintances, coworkers

fellow citizens

foreigners

sometimes, animals

(possibly, plants...)

… and sometimes, we’re even perceived as having a *duty* to care more about one group than another (if someone saved three strangers instead of two of his children, how would he be seen?).

In consequentialist / utilitarian discussions, a regular discussion is “who counts as agents worthy of moral concern” (humans? sentient beings? intelligent beings? those who feel pain? how about unborn beings?), which covers the later part of the spectrum. However I have seen little discussion of the earlier part of the spectrum (friends and family vs. strangers), and it seems to be the one on which our intuitions agree the most reliably - which is why I think it deserves more of our attention (and having clear ideas about it might help about the rest).

Let’s consider two rough categories of decisions:

impersonal decisions: what should government policy be? By what standard should we judge moral systems? On which cause is charity money best spent? Who should I hire?

personal decisions: where should I go on holidays this summer? Should I lend money to an unreliable friend? Should I take a part-time job so I can take care of my children and/or parents better? How much of my money should I devote to charity? In which country should I live?

Impartial utilitarianism and consequentialism (like the questi...
Can we talk about mental illness?
by riparianx — 2015-03-08T08:24:37.166Z

For a site extremely focused on fixing bad thinking patterns, I've noticed a bizarre lack of discussion here. Considering the high correlation between intelligence and mental illness, you'd think it would be a bigger topic.

I personally suffer from Generalized Anxiety Disorder and a very tame panic disorder. Most of this is focused on financial and academic things, but I will also get panicky about social interaction, responsibilities, and things that happened in the past that seriously shouldn't bother me. I have an almost amusing response to anxiety that is basically my brain panicking and telling me to go hide under my desk.

I know lukeprog and Alicorn managed to fight off a good deal of their issues in this area and wrote up how, but I don't think enough has been done. They mostly dealt with depression. What about rational schizophrenics and phobics and bipolar people? It's difficult to find anxiety advice that goes beyond "do yoga while watching the sunrise!" Pop psych isn't very helpful. I think LessWrong could be. What's mental illness but a wrongness in the head?

Mental illness seems to be worse to intelligent people than your typical biases, honestly. Hiding under my desk is even less useful than, say, appealing to authority during an argument. At least the latter has the potential to be useful. I know it's limiting me, and starting cycles of avoidance, and so much more. And my mental illness isn't even that bad! Trying to be rational and successful when schizophrenic sounds like a Sisyphusian nightmare.

I'm not fighting my difficulties nearly well enough to feel qualified to author my own posts. Hearing from people who are managing is more likely to help. If nothing else, maybe a Rational Support Group would be a lot of fun.