trentmkelly/LessWrong-43k huggingface dataset:
Meetup : Sequences Reading Group
by DreamFlasher — 2017-02-01T10:01:46.240Z

Discussion article for the meetup : Sequences Reading Group

WHEN:
23 February 2017 07:15:00PM (+0100)

WHERE:
Berlin

We meet about monthly to read the sequences posts. We focus on extracting insights from the LessWrong sequences and applying them to improve our rationality.

We read together and then discuss, no preparation necessary. Each session is independent, as the blog posts are independent, any time is a great time to join.

This group is mainly coordinated via meetup.com: https://www.meetup.com/de-DE/LessWrong-Rationality-Meetup-Berlin/

Discussion article for the meetup : Sequences Reading Group
February 2017 Media Thread
by ArisKatsaris — 2017-02-01T08:31:02.548Z

This is the monthly thread for posting media of various types that you've found that you enjoy. Post what you're reading, listening to, watching, and your opinion of it. Post recommendations to blogs. Post whatever media you feel like discussing! To see previous recommendations, check out the older threads.

Rules:

Please avoid downvoting recommendations just because you don't personally like the recommended material; remember that liking is a two-place word. If you can point out a specific flaw in a person's recommendation, consider posting a comment to that effect.

If you want to post something that (you know) has been recommended before, but have another recommendation to add, please link to the original, so that the reader has both recommendations.

Please post only under one of the already created subthreads, and never directly under the parent media thread.

Use the "Other Media" thread if you believe the piece of media you want to discuss doesn't fit under any of the established categories.

Use the "Meta" thread if you want to discuss about the monthly media thread itself (e.g. to propose adding/removing/splitting/merging subthreads, or to discuss the type of content properly belonging to each subthread) or for any other question or issue you may have about the thread or the rules.
Censoring out-of-domain representations
by orthonormal — 2017-02-01T04:09:51.000Z

(An idea from a recent MIRI research workshop; similar to some ideas of Eric Drexler and others. Would need further development before it's clear whether this would do anything interesting, let alone be a reliable part of a taskifying approach.)

If you take an AI capable of pretty general reasoning, and you ask it to do a task in a restricted domain, you can't necessarily count on it to find a solution that stays conceptually within that domain. This covers a couple of different failure modes, including wireheading (the AI modifies its own sensory inputs rather than the external object you wanted it to modify) and manipulation of humans (the AI influences human psychology rather than the thing you wanted it to directly affect).

Directly forbidding actions outside a domain seems tricky if we can't define the domain in closed form (this is especially the case if we have an AI affecting humans, and we don't want it to develop models of human psychology sufficient for manipulating them). One thing we could try instead is ensuring that the AI doesn't "know too much", in its internal representations, outside its domain.

"Know too much", here, can be defined in the sense of Censoring Representations with an Adversary. We add to our network some additional networks seeking to extract out-of-domain information from internal representations, and then by reversing the sign of the backprop from that network to the representation, we cause it to erase such information.

Furthermore, so that we don't destroy any hope of decent task performance, we can erase out-of-domain information only to the degree that it exceeds the info from some known-safe "black box" model of the environment outside the domain.

Examples of tasks that we might want to use this on:

Play well on a suite of interactive fiction games, but without understanding human psychology any better than some baseline natural language model (e.g. test whether it can predict text conversations between humans).

Optimize the newsfeed algorithm for a social network, as measured by analytics data, again without learning human psychology beyond some simple baseline model.

Do engineering or chemistry within a physically enclosed box, without learning anything about the world outside that box beyond a simple model of it (e.g. ask it questions about external objects, including the AI's own hardware, which you've put outsid

...
How often do you check this forum?
by JenniferRM — 2017-01-30T16:56:54.302Z

I'm interested from hearing from everyone who reads this.

Who is checking LW's Discussion area and how often?

1. When you check, how much voting or commenting do you do compared to reading?

2. Do bother clicking through to links?

3. Do you check using a desktop or a smart phone? Do you just visit the website in browser or use an RSS something-or-other?

4. Also, do you know of other places that have more schellingness for the topics you think this place is centered on? (Or used to be centered on?) (Or should be centered on?)

I would ask this in the current open thread except that structurally it seems like it needs to be more prominent than that in order to do its job.

If you have very very little time to respond or even think about the questions, I'd appreciate it if you just respond with "Ping" rather than click away.
Meetup : Washington, D.C.: Typical Mind Fallacy
by RobinZ — 2017-01-30T16:21:51.040Z

Discussion article for the meetup : Washington, D.C.: Typical Mind Fallacy

WHEN:
05 February 2017 03:30:00PM (-0500)

WHERE:
Donald W. Reynolds Center for American Art and Portraiture

We will be meeting in the courtyard to talk about the typical mind fallacy: the assumption that the way your own mind operates is shared by someone else, even when this may not be true.

Upcoming meetups:

Feb. 12: Fun & Games

Feb. 19: Nonsense

Feb. 26: Create & Complete

Discussion article for the meetup : Washington, D.C.: Typical Mind Fallacy
Open thread, Jan. 30 - Feb. 05, 2017
by MrMind — 2017-01-30T08:31:22.712Z

If it's worth saying, but not worth its own post, then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should start on Monday, and end on Sunday.

4. Unflag the two options "Notify me of new top level comments on this article" and "Make this post available under..." before submitting.
Error and Terror: Are We Worrying about the Wrong Risks?
by philosophytorres — 2017-01-30T01:07:33.211Z

I would be happy to get feedback on this article, originally posted by the IEET:

When people worry about the dark side of emerging technologies, most think of terrorists or lone psychopaths with a death wish for humanity. Some future Ted Kaczynski might acquire a masters degree in microbiology, purchase some laboratory equipment intended for biohackers, and synthesize a pathogen that spreads quickly, is incurable, and kills 90 percent of those it infects.

Alternatively, Benjamin Wittes and Gabriella Blum imagine a scenario in which a business competitor releases “a drone attack spider, purchased from a bankrupt military contractor, to take you out. … Upon spotting you with its sensors, before you have time to weigh your options, the spider—if it is, indeed, an attack spider—shoots an infinitesimally thin needle … containing a lethal dose of a synthetically produced poison.” Once this occurs, the spider exits the house and promptly self-destructs, leaving no trace behind it.

This is a rather terrifying picture of the future that, however fantastical it may sound, is not implausible given current techno-developmental trends. The fact is that emerging technologies like synthetic biology and nanotechnology are becoming exponentially more powerful as well as more accessible to small groups and even single individuals. At the extreme, we could be headed toward a world in which a large portion of society, or perhaps everyone, has access to a “doomsday button” that could annihilate our species if pressed.

This is an unsettling thought given that there are hundreds of thousands of terrorists—according to one estimate—and roughly 4 percent of the population are sociopaths—meaning that there are approximately 296 million sociopaths in our midst today. The danger posed by such agents could become existential in the foreseeable future.

But what if deranged nutcases with nefarious intentions aren’t the most significant threat to humanity? An issue that rarely comes up in such conversations is the potentially greater danger posed by well-intentioned people with access to advanced technologies. In his erudite and alarming book Our Final Hour, Sir Martin Rees distinguishes between two types of agent-related risks: terror and error. The difference between these

...
My current take on the Paul-MIRI disagreement on alignability of messy AI
by jessicata — 2017-01-29T20:52:12.000Z

Paul Christiano and "MIRI" have disagreed on an important research question for a long
time: should we focus research on aligning "messy" AGI (e.g. one found through
gradient descent or brute force search) with human values, or on developing
"principled" AGI (based on theories similar to Bayesian probability theory)? I'm going to present my current model of this disagreement and additional thoughts about it.

I put "MIRI" in quotes because MIRI is an organization composed of people who
have differing views. I'm going to use the term "MIRI view" to refer to some
combination of the views of Eliezer, Benya, and Nate. I think these three
researchers have quite similar views, such that it is appropriate in some
contexts to attribute a view to all of them collectively; and that these
researchers' views constitute what most people think of as the "MIRI view".

(KANSI AI complicates this disagreement
somewhat; the story here is that we can use "messy" components in a KANSI AI but
these components have to have their capabilities restricted significantly. Such
restriction isn't necessary if we think messy AGI can be aligned in general.)

Intuitions and research approaches

I'm generally going to take the perspective of looking for the intuitions
motivating a particular research approach or produced by a particular research
approach, rather than looking at the research approaches themselves. I expect
it is easier to reach agreement about the how compelling a particular intuition
is (at least when other intuitions are temporarily ignored), than to reach
agreement on particular research approaches.

In general, it's quite possible for a research approach to be inefficient while
still being based on, or giving rise to, useful intuitions. So a criticism of a
particular research approach is not necessarily a criticism of the intuitions
behind it.

Terminology

A learning problem is a task for which the AI is supposed to output some
information, and if we wanted, we could give the information a score
measuring how good it is the task, using less than ~2 weeks of labor. In
other words, there's an inexpensive "ground truth" we have access to. This
looks a little weird but I think this is a natural category, and some of the
intuitions relate to learning and non-learning problems. Paul has written
about learning and non-learning problems
here.

An AI system is aligned if it is

...
On motivations for MIRI's highly reliable agent design research
by jessicata — 2017-01-29T19:34:37.000Z

(this post came out of a conversation between me and Owen Cotton-Barratt, plus a follow-up conversation with Nate)

I want to clarify my understanding of some of the motivations of MIRI's highly reliable agent design (HRAD) research (e.g. logical uncertainty, decision theory, multi level models).

Top-level vs. subsystem reasoning

I'll distinguish between an AI system's top-level reasoning and subsystem reasoning. Top-level reasoning is the reasoning the system is doing in a way its designers understand (e.g. using well-understood algorithms); subsystem reasoning is reasoning produced by the top-level reasoning that its designers (by default) don't understand at an algorithmic level.

Here are a few examples:

AlphaGo

Top-level reasoning: MCTS, self-play, gradient descent, ...

Subsystem reasoning: whatever reasoning the policy network is doing, which might involve some sort of "prediction of consequences of moves"

Deep Q learning

Top-level reasoning: the Q-learning algorithm, gradient descent, random exploration, ...

Subsystem reasoning: whatever reasoning the Q network is doing, which might involve some sort of "prediction of future score"

Solomonoff induction

Top-level reasoning: selecting (Cartesian) hypotheses by seeing which make the best predictions

Subsystem reasoning: the reasoning of the consequentialist reasoners who come to dominate Solomonoff induction, who will use something like naturalized induction and updateless decision theory

Genetic selection

It is possible to imagine a system that learns to play video games by finding (encodings of) policies that get high scores on training games, and combining encodings of policies that do well to produce new policies.

Top-level reasoning: genetic selection

Subsystem reasoning: whatever reasoning the policies are doing (which is something like "predicting the consequences of different actions")

In the Solomonoff induction case and this case, if the algorithm is run with enough computation, the subsystem reasoning is likely to overwhelm the top-level reasoning (i.e. the system running Solomonoff induction or genetic selection will eventually come to be dominated by opaque consequentialist reasoners).

Good consequentialist reasoning

Humans are capable of good consequentialist reasoning (at least in comparison to current AI systems). Humans can:

make medium-term predictions about complex systems containing other humans

mak

...
Significantly Invested
by Jacob Falkovich — 2017-01-29T05:40:16.000Z

I have been making my way through the EconTalk archives, which I feel is more educational hour-for-hour than a university degree in economics. Today I listened to finance professor Campbell Harvey talking about statistical significance in investment management. He shares a story that ties together a lot of what I wrote about in the last few “math classes“:

Harvey: A number of years ago I was shown some research, at a high-level meeting, at one of the top 3 investment banks in the world. And this person was presenting the research, and basically he had found a variable that looked highly significant in beating the market, with a regression analysis, as you said. And it turned out that this variable was the 17th monthly lag in U.S. Industrial Production.

Russ: Yeah. I’ve always known that’s an important factor. [sarcastic] But that’s the beauty of his approach: nobody knows it, but by his fabulous deep look at the data, he uncovered this secret relationship that no one else knows.

Harvey: So, 17th lag? That seems a little unusual. So, usually we think of maybe the 2nd because one month the data isn’t available because of the publication delay. Maybe the 3rd. But the 17th–where’s that coming from? And then he basically said, ‘Well, that’s the only one that worked.’

Harvey shouldn’t be surprised: he teaches at a business school. Business schools have to cover a lot of ground: leadership (organizing a party), marketing (telling people about the party), supply chain (buying the beer), accounting (Venmoing beer money) and networking (party). The MBAs who go on to become VPs at top investment banks have time for just one mandatory statistics class, which covers p < 0.05 and not much else.

Of course, p-values doesn’t sound impressive enough for finance professionals. Instead, an investment strategy is evaluated using the Sharpe Ratio: the ratio of excess return to volatility.

Taking a measurement (excess returns) and dividing it by the standard deviation (volatility) is simply calculating the test statistic. The test statistic is a normalized measure of how far the result is from the null – the null for investment is the return on a “risk free” asset like US treasuries. The test statistic generates the p-value, they are one and the same. Each (arbitrary and useless) p-value...
AI Safety reading group
by SoerenE — 2017-01-28T12:07:17.681Z

I am hosting a weekly AI Safety reading group, and perhaps someone here would be interested in joining.
Here is what the reading group has covered so far:
http://airca.dk/reading_group.htm
Next week, on Wednesday the 1st of February 19:45 UTC, we will discuss "How Feasible is the Rapid Development of Artificial Superintelligence?" by Kaj Sotala. I publish some slides before each meeting, and present the article, so you can also join if you have have not read the article.

To join, add me on Skype ("soeren.elverlin"). General coordination happens on a Facebook group, at

https://www.facebook.com/groups/AISafetyAarhus/

You can see the time in your local timezone here:

https://www.timeanddate.com/worldclock/fixedtime.html?msg=AI+Safety+Reading+Group&iso=20170201T1945&p1=1440&ah=1
Facets of Problems v0.1
by  — 2017-01-28T12:00:42.672Z

Problems can be decomposed into parts that are shared among different problems, these parts I'll call facets (for want of a better word, if there is some art I am ignorant of let me know). Each facet fundamentally effects how you approach a problem, by changing the class of problem being solved. Facets can be seen as parts of paradigms extended into everyday life.

For example, when trying to path find, given only the map, you may use something like the A* algorithm. But if you have a map and an oracle to tell you that the optimal path runs through certain points, you can use that information decompose the problem into solving the shortest path between those points. Having that oracle is a facet to a problem. Another facet might be that you know of an automated doorway on a shortcut that is open and closed at different times. You no longer have a fixed map so the A* algorithm is not appropriate. You'll have to represent that probabilistically or try and figure out the pattern of the opening, so you can predict exactly when it is open.

There are a number of distinct ways that facets can impact your problem solving . These can be because :

a facet suggests new sources of information to solve a problem (Epistemic)

a facet constrains the problem in a hard to discover way that make it easier to solve (Constraints)

a facet makes things harder to solve but more likely a good solution will be found if the facet is true (Inconveniences)

a facet means you have to manipulate your other facets (Meta)

A problem can have many facets and they interact in a non-trivial fashion. Having the wrong facets can be very bad they form the systems inductive bias.

I think facets can impact different things:

How we approach the world ourselves (Are we making use of all the facets that we can? How do the facets we are exploiting interfere? Do we have damaging facets?).

How we design systems that interact with the world. Enumerating the facets of a problem is the first step to trying to solve it.

Epistemic status: Someone has probably thought of this stuff before. Hoping to find it. If they haven't and people find it useful I'll do a second version.

Base Assumptions: That throwing lots of resources at a single approach with no regard to facets is impractical (No AIXI type solution)

Epistemic

These facets can allow you to move more quickly to solving your problem by giving you i...
Emergency learning
by Stuart_Armstrong — 2017-01-28T10:05:59.737Z

Crossposted at the Intelligent Agent Foundation Forum.

Suppose that we knew that superintelligent AI was to be developed within six months, what would I do?

Well, drinking coffee by the barrel at Miri's emergency research retreat I'd...... still probably spend a month looking at things from the meta level, and clarifying old ideas. But, assuming that didn't reveal any new approaches, I'd try and get something like this working.

Standard setup

Take a reinforcement learner AI, that we want to safely move a strawberry onto a plate. A human sits nearby and provides a reward based on inspecting the AI's behaviour.

As it stands, this setup is completely vulnerable to reward hacking. The reward is not provided for safe moving of the strawberry; instead the reward is provided by having the human judge that the task has been accomplished and then pressing a button. Taking control of the human or control of the button is likely to be possible for a superintelligent AI; and, as it stands, that would be mandated by this reward function.

Learning from positive and various negative examples

Could we have the AI instead learn what the reinforcement signal "should be"? It seems that it might at least be possible, if we can make the AI learn from both positive and negative examples. I'd make five categories of examples from which the AI could learn. It may be too dangerous to have the superintelligent AI used directly in constructing these examples; in that case, the rewards would be given to a simpler, dumber version of the AI, and the examples passed on to the superintelligent AI for offline training.

Simple positive and negative examples. These are the basic examples from above: the AI completes the task or fails to, and gets the consequent reward. The AI stays within its room and the human is sober, rested, uninfluenced, and so on.

Simple more dubious examples. These are examples where the AI gets a reward, but the learning process judges that these rewards were not fairly allocated. Maybe the human was drunk, or not paying attention. Maybe they slipped, and pressed the button by accident. Or they were in a hurry to end their shift, and so pressed the button. All these examples are given a score from -1 to 1. Those with a score of 1 are taken as legitimate examples of rewards, those with a score of 0 are taken as random reward giving, while those with a score of

...