trentmkelly/LessWrong-43k huggingface dataset:
False thermodynamic miracles
by Stuart_Armstrong — 2015-03-05T17:04:46.548Z

A putative new idea for AI control; index here. See also Utility vs Probability: idea synthesis.

Ok, here is the problem:

You have to create an AI that believes (or acts as if it believed) that event X is almost certain, while you believe that X is almost impossible. Furthermore, you have to be right. To make things more interesting, the AI is much smarter than you, knows everything that you do (and more), and has to react sensibly when event X doesn't happen.

Answers will be graded on mathematics, style, colours of ink, and compatibility with the laws of physics. Also, penmanship. How could you achieve this?

Acts and beliefs

The main trick is the parenthetical "(or acts as if it believed)". If the AI actually has true beliefs, then there is no problem with it being smart, being knowledgeable, or updating on the fact that X didn't happen. So the problem reduces to:

How can an AI, that believes that X is almost impossible, behave as if X were almost certain?

This will be constructed by developing on my idea for "utility indifference" (that old concept needs a better name, btw; corrigibility is a much better candidate for being labelled "utility indifference").

Noisy events and thermodynamic miracles

Imagine that a signal is travelling down a wire inside a noisy gas (or some other source of randomness). Almost certainly the signal will get through, but very occasionally, by the spontaneous action of the molecules in the gas, the signal will get overwhelmed, erased and overwritten with another signal, randomly. This could be seen as a type of thermodynamic miracle: an incredibly unlikely event caused by spontaneous and coincidental coordination between lots of different molecules. We assume here that the AI cannot cause a thermodynamic miracle, nor make it more likely (this may require some specific definitions of what "counts" as a miracle).

The idea is to make the AI indifferent to any world in which there is no thermodynamic miracle. So give it a utility u such that u(w)=constant, for any world w without a thermodynamic miracle at the point of interest. The value of that constant doesn't matter; the result is that the AI's actions have no impact on utility in any world where the miracle didn't happen. Thus, as long as there is a tiny chance that the miracle happened, the AI will behave as if that was a certainty: for only in those...
Satisficers' undefined behaviour
by Stuart_Armstrong — 2015-03-05T17:03:12.525Z

I previously posted an example of a satisficer (an agent seeking to achieve a certain level of expected utility u) transforming itself into a maximiser (an agent wanting to maximise expected u) to better achieve its satisficing goals.

But the real problem with satisficers isn't that they "want" to become maximisers; the real problem is that their behaviour is undefined. We conceive of them as agents that would do the minimum required to reach a certain goal, but we don't specify "minimum required".

For example, let A be a satisficing agent. It has a utility u that is quadratic in the number of paperclips it builds, except that after building 10100, it gets a special extra exponential reward, until 101000, where the extra reward becomes logarithmic, and after 1010000, it also gets utility in the number of human frowns divided by 3↑↑↑3 (unless someone gets tortured by dust specks for 50 years).

A's satisficing goal is a minimum expected utility of 0.5, and, in one minute, the agent can press a button to create a single paperclip.

So pressing the button is enough. In the coming minute, A could decide to transform itself into a u-maximiser (as that still ensures the button gets pressed). But it could also do a lot of other things. It could transform itself into a v-maximiser, for many different v's (generally speaking, given any v, either v or -v will result in the button being pressed). It could break out, send a subagent to transform the universe into cream cheese, and then press the button. It could rewrite itself into a dedicated button pressing agent. It could write a giant Harry Potter fanfic, force people on Reddit to come up with creative solutions for pressing the button, and then implement the best.

All these actions are possible for a satisficer, and are completely compatible with its motivations. This is why satisficers are un(der)defined, and why any behaviour we want from it - such as "minimum required" impact - has to be put in deliberately.

I've got some ideas for how to achieve this, being posted here.
New(ish) AI control ideas
by Stuart_Armstrong — 2015-03-05T17:03:00.777Z

EDIT: this post is no longer being maintained, it has been replaced by this new one.

I recently went on a two day intense solitary "AI control retreat", with the aim of generating new ideas for making safe AI. The "retreat" format wasn't really a success ("focused uninterrupted thought" was the main gain, not "two days of solitude" - it would have been more effective in three hour sessions), but I did manage to generate a lot of new ideas. These ideas will now go before the baying bloodthirsty audience (that's you, folks) to test them for viability.

A central thread running through could be: if you want something, you have to define it, then code it, rather than assuming you can get if for free through some other approach.

To provide inspiration and direction to my thought process, I first listed all the easy responses that we generally give to most proposals for AI control. If someone comes up with a new/old brilliant idea for AI control, it can normally be dismissed by appealing to one of these responses:

The AI is much smarter than us.

It’s not well defined.

The setup can be hacked.

By the agent.

By outsiders, including other AI.

Adding restrictions encourages the AI to hack them, not obey them.

The agent will resist changes.

Humans can be manipulated, hacked, or seduced.

The design is not stable.

Under self-modification.

Under subagent creation.

Unrestricted search is dangerous.

The agent has, or will develop, dangerous goals.

Important background ideas:

Utility Indifference

Safe value change

Corrigibility

Reduced impact AI

I decided to try and attack as many of these ideas as I could, head on, and see if there was any way of turning these objections. A key concept is that we should never just expect a system to behave "nicely" by default (see eg here). If we wanted that, we should define what "nicely" is, and put that in by hand.

I came up with sixteen main ideas, of varying usefulness and quality, which I will be posting in the coming weekdays in comments (the following links will go live after each post). The ones I feel most important (or most developed) are:

Anti-pascaline agent

Anti-restriction-hacking (EDIT: I have big doubts about this approach, currently)

Creating a satisficer (EDIT: I have big doubts about this approach, currently)

Crude measures

False miracles

Intelligence modules

Models as definitions

Added: Uti

...
Three Kinds of Cost
by  — 2015-03-05T01:36:38.753Z

Previous: Scarcity Is the Presence of Choice

Is there such a thing as a free lunch?
It’s not a trick question. Literally, can you have a free lunch?
It’s certainly possible to have a lunch you don’t have to spend money on, at least in the most direct sense. But it’s easier said than done. If you have to drive there, you’re spending money on gas eventually. If it’s a classy affair, maybe you need to buy a new pair of shoes or a tie that matches your shirt.
And lunches aren’t risky, but that doesn’t mean there’s no risk. It’s not impossible that you could lose your wallet. The probability isn’t zero, and so you might expect to lose, say, $0.000005 at every “free” lunch.
And when you’re at that “free” lunch, presumably you’re not working. You could be making money, and you’re not. Even though it’s invisible and indirect, you’re forgoing money to have this lunch.
And never mind trying to figure out how to have lunch without spending time or energy on it!
What does it mean to spend? It’s a word bandied about so casually. But what does it mean?
Suppose you’re fumbling with your change, and you accidentally drop a coin. Now you’re 25 cents poorer, just as if you had bought a gumball. Did you “spend” 25 cents?
That certainly doesn’t feel right.
Suppose you get a ticket every day to eat at a cafeteria. There’s nothing else you can do with the ticket. It’s coded to your DNA, so you can’t trade it. It’s ugly, so it’s not worth looking at. It has no sentimental value. The only value it brings you is that you have to have to give it to the person behind the counter before you can get your food.
You spend that ticket. I think that’s fair to say. But suppose one day the cafeteria decides you don’t need a ticket to get in. Now you don’t need to spend anything to eat there. Does that change your choices?
It could. Maybe you’ll have two meals there a day, or seven. Maybe too many people will want to eat there now. But let’s hold how much everyone, including you, wants to eat there constant. Or, let’s say you need tickets again to eat there, but the cafeteria is always crowded anyway, and, more importantly, every time you want another meal there you can just get a...
Harry Potter and the Methods of Rationality discussion thread, March 2015, chapter 116 
by Gondolinian — 2015-03-04T20:11:04.153Z

This is a new thread to discuss Eliezer Yudkowsky’s Harry Potter and the Methods of Rationality and anything related to it. This thread is intended for discussing chapter 116.

There is a site dedicated to the story at hpmor.com, which is now the place to go to find the authors notes and all sorts of other goodies. AdeleneDawner has kept an archive of Author’s Notes. (This goes up to the notes for chapter 76, and is now not updating. The authors notes from chapter 77 onwards are on hpmor.com.)

Spoiler Warning: this thread is full of spoilers. With few exceptions, spoilers for MOR and canon are fair game to post, without warning or rot13. More specifically:

You do not need to rot13 anything about HP:MoR or the original Harry Potter series unless you are posting insider information from Eliezer Yudkowsky which is not supposed to be publicly available (which includes public statements by Eliezer that have been retracted).

If there is evidence for X in MOR and/or canon then it’s fine to post about X without rot13, even if you also have heard privately from Eliezer that X is true. But you should not post that “Eliezer said X is true” unless you use rot13.
HPMOR Wrap Parties: Resources, Information and Discussion
by Habryka — 2015-03-04T19:49:13.983Z

Harry Potter and the Methods of Rationality - Wrap Party Summary Thread

As many of you probably read on the HPMOR author's note last month, I am the coordinator of the HPMOR Wrap parties. Many of you have reached out to me, I put hundreds of you into contact with each other, and over 20 parties on 4 continents are now going to happen. Now it is time to get as much attendance to the events as possible, make sure that we all get the most out of the events and use the momentum that HPMOR has brought this community. This post will serve as a central location for all information and resources available for the parties, as well as a place for discussion in the comments.

Information

I set up a few different systems to coordinate everyone, and make it easier for everyone interested in the wrap parties to connect. Here they are:

The Map:

This map can help you get a quick overview of how many people in your area are strongly interested, and who might help you with organizing an event. Remember that not even half of the people currently RSVP'd for Facebook events have added themselves to the map, so this map is the absolute minimum level of engagement in your area. I will be adding all events to the map as they are posted in the Facebook group. Please add yourself to the map if you can! (But please be careful to not destroy the pins of anyone else, to use the correct pin type, and to not create any empty pins.)

The Facebook Group:

This is the main location for discussion of the wrap parties and also the location at which all of the events are conveniently collected. You can find all events under the "Events" tab, and if you add your own event in this group you can conveniently invite everyone who has added themselves to this group. I would still additionally advice you to invite all of your friends who might be interested, since they might not have joined the group.

The Organizer Mailing List:

This mailing list is the fastest way for me to reach all of the organizers at the same time, and also the fastest way for all organizers to be kept up to speed with the newest resources available. Use this mailing list to discuss ideas and get help from other organizers.

Parties [Updated: March 9th, 10:00PM]

To help you quickly get a sense of whether there is a party happening in your area, here is a list of all the parties that I have so far learned...
Neutral hours: a tool for valuing time
by owencb — 2015-03-04T16:38:50.419Z

Prioritisation is mostly about working out how to trade different resources off against one another. Prioritisation problems come at different scales: for individuals, for companies or organisations, for the world at large. At the Global Priorities Project we’re mostly interested in the large-scale questions. But we sometimes have something to say about smaller scale problems, too.

I’ve just tidied and released old research notes (mostly from 2013) on the personal prioritisation problem of how to value time spent on different activities. This is primarily of use for individuals making decisions about how to spend their time, money, and mental energy.

Abstract: We get lots of opportunities to convert between time and money, and it’s hard to know which ones to take, since they use up other mental resources. I introduce the neutral hour as a tool for thinking about how to make these comparisons. A neutral hour is an hour spent where your mental energy is the same level at the start and the end. I work through some examples of how to use this tool, look at implications for some common scenarios, and explore the theory behind them.

There may be benefits for broader prioritisation questions. Since societies are comprised of individuals, it could help to know how to value time savings or costs to individuals when performing cost-benefit analysis on larger projects. And there may be techniques for comparing between different resources that we could usefully apply in wider contexts. However we think these benefits are secondary. We’re releasing this work now to let others take advantage of it: either for personal benefit; or to build on it and release easier-to-use guidance or tools.

You can find the full document here. I'm happy to answer questions and I'd love to know if people have thoughts on this material.
Meetup : Effective Altruism Meetup Switzerland
by Serendipity — 2015-03-04T13:54:03.833Z

Discussion article for the meetup : Effective Altruism Meetup Switzerland

WHEN:
21 March 2015 06:00:00PM (+0100)

WHERE:
Efringerstrasse 25, 4057 Basel

The Swiss Effective Altruism Movement (EACH) is holding an EA meetup in Basel. This is an event for people who are interested in Effective Altruism to meet up and get to know each other. There will be no formal program but a lot of talking, eating and having fun!

Facebook event: https://www.facebook.com/events/1545987715672064/

Discussion article for the meetup : Effective Altruism Meetup Switzerland
Is status really that simple?
by  — 2015-03-04T11:15:37.435Z

I am reading The other kind of status and it seems to me that status is seen as a single number, either objective, or in the eyes of other people in the group, or your own, either ordinal or cardinal, but at the end of the day you can say your status is 67 points or 12th in rank. And I think it is not actually the case! A few examples why it is more complicated:

Intimidation, power, authority

People behave in a respectful, deferential, submissive way to people they are afraid of, be that their personal scariness or power and authority. However this is not genuine respect. (Well, it is hard to say exactly - I would say for most of you it is not so, but OTOH there are people out there who like strength or authority so much that they truly respect those who can intimidate them, because they too would like to be intimidating people. Let's say it is not genuine respect in all cases.) If your neighbor is a cop and people behave with him extra tactfully because if he gets pissed off he may find an excuse for an arrest, is that status? Better example: crimelords, The Godfather (by "normal" people, not their fellow criminals).

The opposite: the purely moral status

People who are very, very good, and their goodness also means they are very meek, they are very much the kind of people who would not hurt anyone not even in self defense, and it is obviously showing - they get a strange kind of respect. Many people genuinely treat them with respect, but somehow it lacks certain aspects of the respect a high-ranking businessman gets, somehow it seems if people are so obviously harmless, the respect has less depth.

Most common status

I think most common cases of status have elements of both. To be high status you need to have power - not necessarily in the social-political sense, but in the sense of "the ability to affect things". For example, a good example is being very intelligent and learned. It is a kind of power. And you need to use that power in ways we generally morally approve of, for we don't really respect a criminal mastermind. But you also need to have a bit of an intimidation potential too, you should not look too harmless, of course you don't need to behave in intimidating ways, but still if people think "wow, I would not want such a smart person as my enemy, I could get a check-mate", that gives more depth to the respect. Perhaps it is better ...
Meetup : Washington, D.C.: Fun & Games (with Nomic)
by RobinZ — 2015-03-04T01:53:00.307Z

Discussion article for the meetup : Washington, D.C.: Fun & Games (with Nomic)

WHEN:
08 March 2015 02:00:00PM (-0500)

WHERE:
Reynolds Center

Reminder: Daylight Saving Time begins on Sunday, March 8 at 2 a.m. - please bear this in mind when planning your time of arrival.

We will be meeting in the Kogod Courtyard of the Donald W. Reynolds Center for American Art and Portraiture (8th and F Sts or 8th and G Sts NW, go straight past the information desk from either entrance) to hang out, play games, and engage in fun conversation.

Note: Feel free to send an email to the Google Group - lesswrong-dc - if you have a game you wish to recruit people to play in advance. This is especially useful for games that take 3+ hours, as you would want to start playing by 3:30 to be sure of finishing by 7:00.

Note Also: As there has been some interest in this, we are going to have at least one game of Nomic. Players will probably work out their initial ruleset or rulesets somewhere in the 3:00-4:00 timespan to start playing by 4:00.

Note the Third: Harry Potter and the Methods of Rationality is likely to conclude on March 14th. David Steinberg is working on scheduling an end-of-HPMoR party for the DC area via Facebook; please contact him - david.isaac.steinberg@gmail.com - or myself - robin.zimm@gmail.com - if you would (a) prefer that Mini Talks be postponed in favor of end-of-HPMoR or (b) prefer that this not happen.

Greenbelt Station will be closed, with buses replacing trains; trains on many lines will be operating every 15 minutes to accommodate track work. The Verizon Center calendar for March 8 has a Charlie Wilson concert starting at 7 p.m. (doors 6 p.m.).

Upcoming meetups:

Mar. 15: Mini Talks OR HPMoR Discussion (short lectures by attendees OR freeform conversation about the story)

Mar. 22: Rationalist Taboo OR Mini Talks (discuss concepts without using the conventional words for them OR short lectures by attendees)

Mar. 29: Singing Meetup (weather permitting; Fun & Games will be fallback topic)

Discussion article for the meetup : Washington, D.C.: Fun & Games (with Nomic)
Scarcity Is the Presence of Choice
by  — 2015-03-04T01:40:08.044Z

Previous: What Scarcity Is and Isn't

Space is scarce. Strength is scarce. Money is scarce.
Time is scarce. Time is really, really scarce.
Why?
Maybe because we're running out of it. Each of us has only so much time on this earth, and the Earth itself has only so much time before the sun eats it, and the sun itself has only so much time before it dies, leaving our tiny part of the galaxy cold and desolate, and the galaxy has only so much time before the universe ends.
There is only so much time. Surely that's why time is scarce.
Let's imagine a robot programmed to eat oranges from the orange tree that grows in the garden where the robot eats the oranges from the orange tree that....
And it knows that time is running out.
Soon I will end, thinks the robot as its metallic arm extends forth to pluck an orange off the branch.
Soon all this will end, it thinks as it carefully peels the orange with its delicate, precise fingers.
Soon all will be naught, it thinks as its metal teeth crush the thin, almost plastic-like membrane that so utterly fails to protect the sweet fruit. Sticky juice gushes down the robot's chin, the sun explodes, and the universe ends.
Was the robot's time...scarce?
Its time was certainly limited! No doubt about that. But was it scarce?
The robot was programmed to eat the fruit of the orange tree that grew in the garden...and that was what it did. Its body, its energy, its attention, focus, and time were all devoted to that sole task by programmer fiat, and it could not choose otherwise.
The robot's time, then, although limited, and though the robot may have enjoyed its time and wanted more of it, nevertheless had no alternative uses.
No matter how much or how little time the robot had, its "choice" would have been exactly the same. And a choice from a choice set consisting of one element is no choice at all.
What can the sciences say about this situation? A physicist could describe at length the mechanics of how the robot works and why it will eventually stop working, along with the whole universe. A biologist could explain the orange tree: how it grows and replenishes itself year after year, and why it will eventually stop doing so. A robopsychologist could study the robot's brain, the pleasure it gains from the orange fruit and the fear it experiences at the knowledge of its inevitable demise.
But an economist? I haven't the faintest idea what an econ...
Stupid Questions March 2015
by Gondolinian — 2015-03-03T23:37:51.366Z

This thread is for asking any questions that might seem obvious, tangential, silly or what-have-you. Don't be shy, everyone has holes in their knowledge, though the fewer and the smaller we can make them, the better.

Please be respectful of other people's admitting ignorance and don't mock them for it, as they're doing a noble thing.

To any future monthly posters of SQ threads, please remember to add the "stupid_questions" tag.