trentmkelly/LessWrong-43k huggingface dataset:
Minimax as an approach to reduced-impact AI
by jessicata — 2015-04-02T22:00:04.000Z

Summary: if we're not sure what the right utility function is, we might use the minimax decision rule
to create a low-impact AI that is not bad for any possible utility function. There are also some flawed ways to combine minimax with value learning and corrigibility that might be worth improving on. This is a writeup of one idea from my visit at MIRI last week, during which I worked mostly with Benja and also with Nate to some extent.

The minimax decision rule

The minimax decision rule maximizes the minimum expected utility under some set of uncertain parameters.
The uncertainty that a minimaxer has over these parameters can be considered a form of Knightian uncertainty.

For reduced-impact AI it will be useful to consider the utility function itself as an unknown parameter.
Suppose we have some set V of utility function representatives.
A utility function representative is a function mapping each outcome to a real number, defining a VNM preference relation; notably,
a single VNM preference relation can correspond with multiple utility function representatives, which are translations
and scalings of each other. Now we could initially define the minimax rule as:

argmaxπ∈Πminv∈VE[v(X)|O;π]
where π is the agent's policy (contained in the set Π), O consists of
some universe-locating observations programmed into the AI,
and X is the outcome.
O is meant to locate our universe in a way that prevents the minimaxer
from making decisions from "behind the veil" (which might cause it to,
say, optimize v1 at the expense of v2 in our universe
and vice versa in another universe).
Minimaxers are not VNM in general due to failing the axiom of independence.

Why might we consider programming a reduced-impact AI to use minimax? If we are uncertain
about the correct utility function, but know that it is in some set,
then we could define V to contain a representative for each utility function.
To reduce impact, we would like each representative to assign 0 utility to the
expected status quo given O (i.e. ∀v∈V:E[v(X)|O;π=shutdown]=0). This way, the AI will only take actions if no utility function
in our set loses out relative to the status quo (conditioned on O). This is similar to
Stuart Armstrong's satisficer design.
With such a design, the AI might be unable to take any actions other than shutting down
(for example, if for some v both v and −v appear in V),
but at least it w

...
Stupid Questions April 2015
by Gondolinian — 2015-04-02T21:29:36.177Z

This thread is for asking any questions that might seem obvious, tangential, silly or what-have-you. Don't be shy, everyone has holes in their knowledge, though the fewer and the smaller we can make them, the better.

Please be respectful of other people's admitting ignorance and don't mock them for it, as they're doing a noble thing.

To any future monthly posters of SQ threads, please remember to add the "stupid_questions" tag.
[LINK] Scott Adam's "Rationality Engine". Part III: Assisted Dying
by Shmi — 2015-04-02T16:55:29.684Z

Scott Adams, the author of the Dilbert comic and several books, my favorite being How to Fail at Almost Everything and Still Win Big, named his debating format The Rationality Engine. He calls it this way because he claims that it is "the system for turning irrational opinions into rational outcomes". He applies it to several polarizing issues, those this site tends to label "Politics" and "Mind Killer" and shy away from.

His first application, investigating the gender pay gap, seems to have worked pretty well, resulting in several unexpected conclusions. His second, Who is More Anti-Science? I found to be slightly less impressive, but still producing a rather balanced output.

Now he is applying it to the debate about Assisted Dying. Scott's goal is to have a law passed in California that is similar to the ones already in effect in Oregon and several other places.

Scott will debate Jimmy Akin, a prominent contributor to Catholic Answers.

I am quite attracted to Scott's attempts at hands-on instrumental rationality, and on a rather grand scale to boot. They are very much in the spirit of his latest book.

Currently he is accepting suggestions for questions and links for all sides of the issue. Feel free to contribute.

EDIT: I think adding cryonics to the discussion would only complicate the issue and not be helpful, but that's just a guess.
[LINK] Interview with "Ex Machina" director Alex Garland
by philh — 2015-04-02T13:46:56.324Z

http://www.engadget.com/2015/04/01/ex-machina-alex-garland-interview/

The title says he "embraces the rise of superintelligent AI", but that isn't really supported by the text.

What struck me about this was that he seems to just take good habits of thought for granted.

I instinctively disagreed with this, but I didn't have the sort of armory to disagree with it on his terms, so I started reading as much as I could.

What other sorts of AI books have you read?

I pretty much would read everything I could. I tried to read people like Penrose, who were arguing against what I instinctively believed.

That's a good way to solidify your argument.

I don't want to dignify it from my point of view, because I can't stress enough I'm a real layman. So I can understand the principles of an argument, but when it comes to the actuality [...] I really don't understand it.

Being clear about these things is important. Otherwise, we're very quick to conflate stuff, and suddenly you'll be talking about the sentience of Siri. And Siri doesn't have any fucking sentience. AI is probably too broad of a term to be useful at the moment.
Request for help: Android app to shut down a smartphone late at night
by The_Jaded_One — 2015-04-02T11:38:47.994Z

I have been playing around with life hacking ideas inspired by hyperbolic discounting.

One idea that seems to have worked reasonably well is was the idea that I could get to bed on time better if my computer simply switched itself off at a certain time, with absolutely no way (that I am capable of executing!) to make it work until the morning. I found and paid for a piece of software that does this - isurveillance shutdown timer. Unfortnately, this seems to have just shifted my late night computer use to my android mobile device, though gaming sessions that last till 5am are a thing of the past.

So, I'd like an android app that shuts down your (rooted) android phone if it is ever detected on within a prespecified time window on a particular day - e.g. between 11pm and 6am, with no way for the user to circumvent the shutdown. If the user restarts the phone, it should shut down again immediately when it finds out that the time is not within the specified window.

I have looked for something like this on Google Play, however most offerings will shut down the phone *once*, but it will stay on if you switch it on again.

LW being a community of tech-savvy people, I was wondering whether anyone was interested in building such an app? It probably isn't hard to do if you are already an android developer, and I think it would really improve my life, and possibly the lives of other people. You could even make it a paid app - I'd pay. In fact I will commit to paying $50 for the app if someone develops this app and it works as described. If the community finds it useful, I'd expect there'd be some karma in it too. Alternatively if anyone can *find* such an app, I'd be extremely grateful.

A more advanced version of this would be to lock the phone into "emergency calls only" mode within a specific time window. I don't know how hard that would be to pull off.

This idea might even be good enough to turn into a business - millions of people around the world have the same problem. The requirement to root the device obviously puts something of a dampener on the viability of a business, there may be legal issues with rooting devices as part of a business.
Why bitcoin?
by Grant — 2015-04-02T01:24:09.789Z

I'm a software dev who is considering becoming a bitcoiner, mostly to explore its possibilities. I think a currency free from the baggage of the modern financial systems will allow great things to be done. I see lots of other people are thinking the same way (there are numerous BC prediction markets, for example).

However, I don't want to invest time and money in a seriously flawed or doomed system.

BC appears to have at least one potentially-fatal flaw: the 51% attack. I'm unsure why it was assumed this would not be a problem? Profits from mining would seem to increase when on reaches over 50% of the world's mining power. This would seem to encourage powerful mining pools. While current norms and other incentives may discourage black-hat miners, I don't think it is reasonable to rely on these incentives.

Edit: In other words, is there an economy of scale in being the dominate miner?

Edit 2: While it looks like there was a successful BC double-spend, it was the result of a white-hat exploiting a bug, not a 51% attack. However, a few altcoins (e.g. reddcoin) have been the target of 51% attacks, so my research on their repercussions will start there.

In addition, BC would appear to have a number of other flaws:

The necessity for each wallet to contain the entire block chain. Edit: Apparently I was reading some dated information. This is wrong.

Governments have never seemed keen to give up their monopoly on the money supply.

The computing power wasted by mining.

It complects the generation of a public ledger with a specific currency.

Side note: After reading about BC and 51% attacks, I am beginning to think "the network effect is the mind killer" might be a more general expression of "politics is the mind killer". There is a lot of noise out there.

Help and insight is appreciated.
Thinking well
by Vaniver — 2015-04-01T22:03:41.634Z

Many people want to know how to live well. Part of living well is thinking well, because if one thinks the wrong thoughts it is hard to do the right things to get the best ends.

We think a lot about how to think well, and one of the first things we thought about was how to not think well. Bad ways of thinking repeat in ways we can see coming, because we have looked at how people think and know more now about that than we used to.

But even if we know how other people think bad thoughts, that is not enough. We need to both accept that we can have bad ways of thinking and figure out how to have good ways of thinking instead.

The first is very hard on the heart, but is why we call this place "Less Wrong." If we had called it something like more right, it could have been about how we're more right than other people instead of more right than our past selves.

The second is very hard on the head. It is not just enough to study the bad ways of thinking and turn them around. There are many ways to be wrong, but only a few ways to be right. If you turn left all the way around, it will point right, but we want it to point up.

The heart of our approach has a few parts:

We are okay with not knowing. Only once we know we don't know can we look.

We are okay with having been wrong. If we have wrong thoughts, the only way to have right thoughts is to let the wrong ones go.

We are quick to change our minds. We look at what is when we get the chance.

We are okay with the truth. Instead of trying to force it to be what we thought it was, we let it be what it is.

We talk with each other about the truth of everything. If one of us is wrong, we want the others to help them become less wrong.

We look at the world. We look at both the time before now and the time after now, because many ideas are only true if they agree with the time after now, and we can make changes to check those ideas.

We like when ideas are as simple as possible.

We make plans around being wrong. We look into the dark and ask what the world would look like if we were wrong, instead of just what the world would look like if we were right.

We understand that as we become less wrong, we see more things wrong. We try to fix all the wrong things, because as soon as we accept that something will always be wrong we can not move past that thing.

We try to be as cl

...
April 2015 Media Thread
by ArisKatsaris — 2015-04-01T18:34:56.555Z

This is the monthly thread for posting media of various types that you've found that you enjoy. Post what you're reading, listening to, watching, and your opinion of it. Post recommendations to blogs. Post whatever media you feel like discussing! To see previous recommendations, check out the older threads.

Rules:

Please avoid downvoting recommendations just because you don't personally like the recommended material; remember that liking is a two-place word. If you can point out a specific flaw in a person's recommendation, consider posting a comment to that effect.

If you want to post something that (you know) has been recommended before, but have another recommendation to add, please link to the original, so that the reader has both recommendations.

Please post only under one of the already created subthreads, and never directly under the parent media thread.

Use the "Other Media" thread if you believe the piece of media you want to discuss doesn't fit under any of the established categories.

Use the "Meta" thread if you want to discuss about the monthly media thread itself (e.g. to propose adding/removing/splitting/merging subthreads, or to discuss the type of content properly belonging to each subthread) or for any other question or issue you may have about the thread or the rules.
Rationality Quotes Thread April 2015
by Vaniver — 2015-04-01T13:35:48.660Z

Another month, another rationality quotes thread. The rules are:

Please post all quotes separately, so that they can be upvoted or downvoted separately. (If they are strongly related, reply to your own comments. If strongly ordered, then go ahead and post them together.)

Do not quote yourself.

Do not quote from Less Wrong itself, HPMoR, Eliezer Yudkowsky, or Robin Hanson. If you'd like to revive an old quote from one of those sources, please do so here.

No more than 5 quotes per person per monthly thread, please.

Provide sufficient information (URL, title, date, page number, etc.) to enable a reader to find the place where you read the quote, or its original source if available. Do not quote with only a name.
Feedback on promoting rational thinking about one's career choice to a broad audience
by Gleb_Tsipursky — 2015-03-31T22:44:53.767Z

I'd appreciate feedback on optimizing a blog post that promotes rational thinking about one's career choice to a broad audience in a way that's engaging, accessible, and fun to read. I'm aiming to use story-telling as the driver of the narrative, and sprinkling in elements of rational thinking, such as agency and mere-exposure effect, in a strategic way. The target audience is college-age youth and young adults, as you'll see from the narrative. Any suggestions for what works well, and what can be improved would be welcomed! The blog draft itself is below the line.

P.S. For context, the blog is part of a broader project, Intentional Insights, aimed at promoting rationality to a broad audience, as I described in this LW discussion post. To do so, we couch rationality in the language of self-improvement and present it in a narrative style.

____________________________________________________________________________________________________________

Title:

"Stop and Think Before It's Too Late!"

Body:

Back when I was in high school and through the first couple of years in college, I had a clear career goal.

I planned to become a medical doctor.

Why? Looking back at it, my career goal was a result of the encouragement and expectations from my family and friends.

My family immigrated from the Soviet Union when I was 10, and we spent the next few years living in poverty. I remember my parents’ early jobs in America, my dad driving a bread delivery truck and my mom cleaning other people’s houses. We couldn’t afford nice things. I felt so ashamed in front of other kids for not being able to get that latest cool backpack or wear cool clothes – always on the margins, never fitting in. My parents encouraged me to become a medical doctor. They gave up successful professional careers when they moved to the US, and they worked long and hard to regain financial stability. It’s no won

...
On the Boxing of AIs
by tailcalled — 2015-03-31T21:58:08.749Z

I've previously written about methods of boxing AIs. Essentially, while I do see the point that boxing an AI would be nontrivial, most people seem to have gone too far, and claim that it is impossible. I disagree that it's impossible and aim to explain some methods

So, let's start with why people would want to box AIs. As you probably know, letting an AI roam freely results in the destruction of everything humans care about, unless that AI has been programmed very carefully. That amount of destruction would be somewhat unfortunate, so if we have to start an AI, we want to know that it works.

Unfortunately, figuring out if an AI works is tricky. This made some people invent the concept of AI boxing, where you don't give an AI direct access to anything except a human, so you can test the AI thoroughly and exploit it without letting it destroy the world. This turns out not to work, because humans are stupid and let out the AI.

My first solution to that is to not tell the AI about its human overlords. Discussion about that solution made me realize that the are other solutions that work in other cases. Essentially, the problem with the original boxing method is that it tries to do everything, while in reality, you need to do some trade-offs between the realism of the AI's situations and the amount of knowledge you get from the AI.

With my original method, I tried to crank up the second of those variables as far as possible while maintaining perfect security. This makes it harder to test morality-related things, but you get to see lots of details in how the AI behaves.

I promised to post another method for boxing the AI today, but I actually have two new methods. Here they are:

Separate the Gatekeeper and the Observer

How do you prevent the AI from convincing the Gatekeeper from letting it out? By not letting it talk to the Gatekeeper. If the person speaking to the AI (let's call them the Observer) doesn't know the Gatekeeper (the Gatekeeper is the person who controls whether to release the AI), they can't let the AI out.

You could do all sorts of things, such as using virtual reality and a simulated mini-universe to see if, say, the AI would make a good society. You still have to make sure that it has gotten the plan to make a good society from its morality rather than from wanting to escape and destroy everything.

Afterwards, the Observer can make a low-detail rep...
Bitcoin value and small probability / high impact arguments
by vbuterin — 2015-03-31T16:48:23.910Z

I had a rather fun debate with people from the always cynical r/buttcoin reddit community, discussing my estimation of the expected value of Bitcoin in the future, which was predicated on what I estimated as a 5% probability that it will displace part of gold due to its superior properties in the store-of-value realm:

http://np.reddit.com/r/Buttcoin/comments/30tqud/ceo_of_ethereum_spreading_quality_fud_can_we_get/

Note that I am certainly not a Bitcoin maximalist or ideologue; I've become quite a bit more skeptical lately and am actually much closer to the "currency meh, blockchain cool" perspective that is becoming pretty mainstream in the parts of the broader IT and finance community that I've spent the most time interacting with. Here I ended up articulating and defending a position I actually disagree with in any reasonable sense of the term "disagree"; the debate is entirely on whether the probability of the position being correct is 5% (my view) or 0.0000000005% (what seems like their view).

I'd like to ask this community, to what extent are my position and my arguments correct? I'm interested first in a few object-level issues:

1. Gold is a Veblen good both in its store-of-value and its aesthetic use cases. If gold was not rare, people would not care about it for jewellery purposes, as plenty of other forms of jewellery exist with a better aesthetics-to-cost ratio. Gold's premium over other pretty things is purely a result of status/prestige considerations. Hence, the "floor" to which gold could fall due to a simple equilibrium flip is very low (perhaps $50 from industrial use). So Bitcoin's lack of a "fundamental use value" floor is not a serious disadvantage of Bitcoin against gold.

2. A $100 price floor is 90% as bad as a $0 floor. To see why, note that you can make the custom asset of { 9 parts BTC, 1 part oil } which has a pretty identical ratio of current price to price-floor-from-fundamental-use-value.

3. The probabilities of the various events required for BTC to receive this status are roughly within an order of magnitude of the 5% mark.

4. There is a long-term risk to black swan supply increases in gold due to any of { space mining, nanotech-enabled ultracheap earth mining, nuclear transmutation }; this does not exist for BTC

And also particularly in one very important meta-level issue: is my expected-value estimation (10% of gold market cap = $700b...
[link] Thoughts on defining human preferences
by Kaj_Sotala — 2015-03-31T10:08:56.972Z

https://docs.google.com/document/d/1jDGpIT3gKZQZByO6A036dojRKMv62KEDEfEz87VuDoY/

Abstract: Discussion of how we might want to define human preferences, particularly in the context of building an AI intended to learn and implement those preferences. Starts with actual arguments about the applicability of the VNM utility theorem, then towards the end gets into hypotheses that are less well defended but possibly more important. At the very end, suggests that current hypothesizing about AI safety might be overemphasizing “discovering our preferences” over “creating our preferences”.
Open thread, Apr. 01 - Apr. 05, 2015
by MrMind — 2015-03-31T10:06:16.996Z

If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should be posted in Discussion, and not Main.

4. Open Threads should start on Monday, and end on Sunday.
Meetup : MIRI paper reading party
by agilecaveman — 2015-03-31T08:53:14.226Z

Discussion article for the meetup : MIRI paper reading party

WHEN:
05 April 2015 12:00:00PM (-0700)

WHERE:
109 15th Ave, Seattle WA, 98122

https://www.facebook.com/events/860474784017422/

We will be following: https://intelligence.org/2014/12/23/new-technical-research-agenda-overview/ In order.

The papers are independent though, so it's stop by even if you missed the first one.

This time we'll get to: 2. https://intelligence.org/files/TowardIdealizedDecisionTheory.pdf

Generally the format is to alternate reading a chunk and discussing it.

Discussion article for the meetup : MIRI paper reading party