trentmkelly/LessWrong-43k huggingface dataset:
Model-building and scapegoating
by Benquo — 2018-07-27T16:02:46.333Z
Link: http://benjaminrosshoffman.com/model-building-and-scapegoating/

When talking about undesirable traits, we may want to use simple labels. On one hand, simple labels have the virtue of efficiently pointing to an important cluster of behavioral predictions. On the other, they tend to focus attention on the question of whether the person so described is good or bad, instead of on building shared models about the causal structure underlying the perceived problem.

Slate Star Codex recently posted a dialogue exploring this through the example of the term "lazy." (Ozy's response is also worth reading.) I think that Scott's analysis itself unfortunately focuses attention on the question of whether assigning simple labels to adverse traits is good or bad (or alternately, true or false) instead of on building shared models about the causal structure underlying the perceived problem.

When I call someone lazy, I am doing two things. The first is communicating factual information about that person, which can help others avoid incurring costs by trusting the lazy person with some important tasks. This is shared model-building, and it's going to be more salient if you're focused on allocating resources to mitigate harm and produce things of value. In other words, if you're engaged in a community of shared production.

The second is creating a shared willingness to direct blame at that person. Once there's common knowledge that someone's considered blameworthy, they become the default target for exclusion if the group experiences a threat. This can be as simple as killing them and taking their stuff, so there's more per survivor to go around, but this can also take the form of deflecting the hostility of outsiders to the supposed one bad apple. This dynamic is called scapegoating, and it's going to be more salient when zero-sum dynamics are more salient.

Even though I may intend to do only one of these, it's actually quite hard to act along one of these dimensions without side effects along the other. For instance, Protocols of the Elders of Zion was famously intended to cast blame on Jews, but Japanese readers with no cultural history of anti-Semitism concluded that if Jews were so powerful, they should make sure to stay on their good side, and consequently the Japanese government made diplomatic overtures to Jewish leaders in the 1930s despite their alignment with the Axis powers....
Robustness to fundamental uncertainty in AGI alignment
by Gordon Seidoh Worley — 2018-07-27T00:41:26.058Z
Link: https://arxiv.org/abs/1807.09836

Abstract: The AGI alignment problem has a bimodal distribution of outcomes with most outcomes clustering around the poles of total success and existential, catastrophic failure. Consequently, attempts to solve AGI alignment should, all else equal, prefer false negatives (ignoring research programs that would have been successful) to false positives (pursuing research programs that will unexpectedly fail). Thus, we propose adopting a policy of responding to points of metaphysical and practical uncertainty associated with the alignment problem by limiting and choosing necessary assumptions to reduce the risk false positives. Herein we explore in detail some of the relevant points of uncertainty that AGI alignment research hinges on and consider how to reduce false positives in response to them.

If you've been following along, I've been working to a particular end the past couple months, and that end is this paper. It's currently under review for journal publication, but you can read the preprint now! This marks the first in what I expect to be several papers exploring and explaining my belief that we can better figure out how to solve alignment via phenomenology and philosophical investigation because there are key questions at the heart of alignment that are poorly examined and not well grounded. This paper is intentionally conservative in its methods since it's the first (you'll notice, aside from a few citations, I stay within the analytic philosophical tradition), and I believe this is more compelling to my target audience of AI researchers, but later papers may make more direct use of phenomenological methods.

It's also the soft launch of the Phenomenological AI Safety Research Institute so that there's a place to work on these ideas. We have no money, but if you're interested in this line of research I'd be happy to talk to you about potential collaborations or research projects we need help with.
The Experimental Apparatus
by Elo — 2018-07-26T22:16:35.782Z

Previous: The problem of other minds (Greaterwrong)

Greaterwrong // bearlamp (my blog)

Science really likes to isolate variables. From first grade, dropping a ball to measure gravity experiment. We talk about the wind resistance on the ball, and how it might affect a measurement. So we repeat the experiment for reliability. Assuming if we have slightly different wind, and we average out the noise we can probably work out the answer on average by getting repeatability.

The “astute” of us will think about who drops the ball, how the ball drops and how we can drop the ball in an impartial way so as to not affect the experiment ourselves when we run it.

This process presumes there is some kind of setup that can isolate us from the equation when we drop the ball. And we can probably drop the ball in such a way that it isn’t too important for the results of the experiment.

The “way too astute for their own good” will start to think about local density effects of having humans around and moving and how humans will still influence the experiment with our teeny gravitational pulls even if it’s irrelevant for the most part.

It’s easier for inert experiments to isolate a variable - It’s just a ball and just gravity. We take this principle from the ball and gravity and we apply it to Psychology. If we can just stand back and watch people. We can do psychology research without disturbing them.

Well actually we know that we have trouble with repeatability of experiments. It’s remarkably hard to do an experiment the same every time. And we struggle with it and we wrestle with it. And we isolate enough variables that eventually we believe we can predict “how often people cheat given certain conditions” (Dan Ariely studies human biases and how certain conditions influence people to cheat, he’s great fun to read but that’s not so important right now).

Somehow there’s this trouble with isolating variables. All of science struggles with it. All of model making struggles with it. Einstein struggled with it. Any time we try to learn we struggle with it. We wonder - Was that the gravity on the ball that I’m trying to isolate and measure, or was that just the wind on the ball...

Next: The Feedback Problem
Culture, interpretive labor, and tidying one's room
by Benquo — 2018-07-26T20:59:52.227Z
Link: http://benjaminrosshoffman.com/culture-interpretive-labor-and-tidying-ones-room/

While tidying my room, I felt the onset of the usual cognitive fatigue. But this time, I didn't just want to bounce off the task - I was curious. When I inspected the fatigue, to see what it was made of, it felt similar to when I'm trying to thread a rhetorical needle - for instance, between striking too neutral a tone for anyone to understand the relevance of what I'm saying, and too bold of a tone for my arguments to be taken literally. In short, I was shouldering a heavy burden of interpretive labor.

Why would tidying my room involve interpretive labor?

It turns out, every item in my room is a sort of crystallized intention, generally past-me. (We've all heard the stories of researchers with messy rooms who somehow knew where everything was, and lost track of everything when someone else committed the violent act of reorganizing the room, thus deindexing it from its owner's mind.) As I decided what to do with an item, I wanted to make sure I didn't lose that information. So, I tried to Aumann with my past self - the true way, the way that filters back into deep models, so that I could pass my past self's ideological turing test. And that's cognitively expensive.

It's generally too aggressive to tidy someone's room without their permission, unless they're in physical danger because of it. But to be unwilling to tidy my own room without getting very clear explicit permission from my past self for every action - or at least checking in - is pathologically nonaggressive.

Once I realized this, it became easier to tidy my room, but the problem is not limited to that. Part of the reason why my cabin retreat was so helpful to me was that it limited my ability to accept social invitations or other bids for my attention. In those cases as well, I don't feel compelled to agree, but I do feel compelled to do enough interpretive labor to understand why this other person thinks I should do a thing.

I know of a few approaches to this problem, none of which seem fully adequate.

One approach is to blithely bulldoze the accumulated intentions of their environment. When people do this with respect to their own prior intentions, they end up too impulsive and disorganized to deal with the complexity of the modern world, and often severely indebted to extractive schemes like credit cards. When they do this with respect to ...
Prediction Markets: When Do They Work?
by Zvi — 2018-07-26T12:30:00.565Z

Epistemic Status: Resident Expert

I’m a little late on this, which was an old promise to Robin Hanson (not that he asked for it). I was motivated to deal with this again by the launch of Augur (REP), the crypto prediction market token. And by the crypto prediction market token, I mean the empty shell of a potential future prediction market token; what they have now is pretty terrible but in crypto world that is occasionally good for a $300 million market cap. This is, for now, one of those occasions.

The biggest market there, by far, is on whether Ether will trade above $500 at the end of the year. This is an interesting market because Augur bets are made in Ether. So even though the market (as of last time I checked) says it’s 74% percent to be trading above $500 and it’s currently $480 (it’s currently Thursday on July 26, and I’m not going to go back and keep updating these numbers). When I first saw this the market was at 63%, which seemed to me like a complete steal. Now it’s at 74%, which seems more reasonable, which means the first ‘official DWATV trading tip’ will have to wait. A shame!

A better way to ask this question, given how close the price is to $500 now, is what the ratio of ‘given Ether is above $500 what does it cost’ to ‘given Ether is below $500 what does it cost’ should be. A three to one ratio seems plausible?

The weakness (or twist) on markets this implies applies to prediction markets generally. If you bet on an event that is correlated with the currency you’re betting in, the fair price can be very different from the true probability. It doesn’t have to be price based – think about betting on an election between a hard money candidate and one who will print money, or a prediction on a nuclear war.

If I bet on a nuclear war, and win, how exactly am I getting paid?

Robin Hanson, Eliezer Yudkowsky and Scott Sumner are big advocates of prediction markets. In theory, so am I. Prediction markets are a wonderful thing. By giving people a monetary incentive to solve problems and share information, we can learn probabilities (what will GDP be next year?) and conditional probabilities (what will GDP be next year if we pass this tax cut bill?) and use the answers to make the best decision. This method of making decisions is call...
Would you benefit from audio versions of posts?
by Raemon — 2018-07-26T04:53:20.733Z

I know two people who have a hard time reading, but totally enjoy audio versions of blogposts and were only able to consume the sequences via audiobook.

Curious how common this is, and how much low hanging fruit there is vis-a-vis either making audio versions of posts easily accessible, or finding a high quality text-to-speech thing to provide it by default.
Don't Get Distracted by the Boilerplate
by johnswentworth — 2018-07-26T02:15:46.951Z

Author’s Note: Please don’t get scared off by the first sentence. I promise it's not as bad as it sounds.

There’s a theorem from the early days of group theory which says that any continuous, monotonic function which does not depend on the order of its inputs can be transformed to addition. A good example is multiplication of positive numbers: f(x, y, z) = x*y*z. It’s continuous, it’s monotonic (increasing any of x, y, or z increases f), and we can change around the order of inputs without changing the result. In this case, f is transformed to addition using a logarithm: log(f(x, y, z)) = log(x) + log(y) + log(z).

Now, at first glance, we might say this is a very specialized theorem. “Continuous” and “monotonic” are very strong conditions; they’re not both going to apply very often. But if we actually look through the proof, it becomes clear that these assumptions aren’t as important as they look. Weakening them does change the theorem, but the core idea remains. For instance, if we remove monotonicity, then our function can still be written in terms of vector addition.

Many theorems/proofs contain pieces which are really just there for modelling purposes. The central idea of the proof can apply in many different settings, but we need to pick one of those settings in order to formalize it. This creates some mathematical boilerplate. Typically, we pick a setting which keeps the theorem simple - but that may involve stronger boilerplate assumptions than are strictly necessary for the main idea.

In such cases, we can usually relax the boilerplate assumptions and end up with slightly weaker forms of the theorem, which nonetheless maintain the central concepts.

Unfortunately, the boilerplate occasionally distracts people who aren’t familiar with the full idea underlying the proof. For some reason, I see this problem most with theorems in economics, game theory and decision theory - the sort of theorems which say “either X, or somebody is giving away free money”. People will come along and say “but wait, the theorem assumes Y, which is completely unrealistic!” But really, Y is often just boilerplate, and the core ideas still apply even if Y is relaxed to something more realistic. In fact, in many cases, the confusion is over the wording of the boilerplate! Just because we use the word “bet”, doesn’t mean people need to be at a casino for the theorem to apply.

A few...
Fading Novelty
by  — 2018-07-25T21:36:06.303Z

A core aspect of human experience is our pursuit of novelty. There is something tantalizing about seeking out new pleasures, sensations, and experiences that feels hard-coded into how we operate. “Variety is the spice of life” and all that.

Conversely, things which were once new eventually lose their shine over time, and our search for novelty continues. Things which once enamored us are left by the wayside as our attention is captured by shinier, newer things. “The novelty has faded” and all that.

A few examples to drive the point home:

Songs which sounded so entrancing upon the first few listenings become dull after being put on repeat.
Foods which were so delicious during the first few tastings become bland after being eaten day after day.
Clothes which looked so beautiful when initially worn fade into yet another outfit after being worn over and over.

Repetition dulls us.

In psychology, this phenomenon whereby repeated exposure to a stimulus leads to a decreased response is typically referred to as satiation or habituation. (This is rather unfortunate, as I’ve become accustomed to using “habituate” to refer to the act of making something a habit, which makes searching for papers on Google Scholar confusing.) This general pattern of a reduced response is quite ubiquitous across nature. For example, animals which leap into a prepared state upon hearing a loud noise soon grow to ignore it if the noise isn’t paired with actual danger.

In short, it’s a basic form of learning.

From a survival standpoint, a bias towards newness is reasonable. Things in our environment which did not change, e.g. trees, shrubs, or familiar tribe members, likely presented less of a threat than new additions, e.g. fresh tracks, gathering storm clouds, or a stranger in our midst. Had we not had such a filter for newness, our thoughts might have looked like:

“Oh wow, that’s new! Look at that majestic tree! It looks just as good as it did yesterday! Oh wow, that’s new! Look at that lush grass! It’s so fluffy! Oh wow, that’s new! Look at that tiger. It’s so— ”

Constantly taking note of everything in your environment is costly; focusing on just what’s new is an effective optimization that often doesn’t require much of a trade-off.

Yet, despite it’s useful roots, I think that fading novelty is also responsible for some of the difficulties we experience with learning and self-improvement.

On the sel...
Narrow AI Nanny:
 Reaching Strategic Advantage via Narrow AI to Prevent Creation of the Dangerous Superintelligence
by avturchin — 2018-07-25T17:12:32.442Z

Abstract: As there are no currently obvious ways to create safe self-improving superintelligence, but its emergence is looming, we probably need temporary ways to prevent its creation. The only way to prevent it is to create a special type of AI that is able to control and monitor the entire world. The idea has been suggested by Goertzel in the form of an AI Nanny, but his Nanny is still superintelligent, and is not easy to control. We explore here ways to create the safest and simplest form of AI which may work as an AI Nanny, that is, a global surveillance state powered by a Narrow AI, or AI Police. A similar but more limited system has already been implemented in China for the prevention of ordinary crime. AI police will be able to predict the actions of and stop potential terrorists and bad actors in advance. Implementation of such AI police will probably consist of two steps: first, a strategic decisive advantage via Narrow AI created by an intelligence services of a nuclear superpower, and then ubiquitous control over potentially dangerous agents which could create unauthorized artificial general intelligence which could evolve into Superintelligence.

Keywords: AI – existential risks – surveillance – world government – NSA

Highlights:

· Narrow AI may be used to achieve a decisive strategic advantage (DSA) and acquire global power.

· The most probable route to DSA via Narrow AI is the creation of Narrow AI by the secret service of a nuclear superpower.

· The most probable places for its creation are the US National Security Agency or the Chinese Government.

· Narrow AI may be used to create a Global AI Police for global surveillance, able to prevent the creation of dangerous AIs and most other existential risks.

· This solution is dangerous but realistic.

Pemalink: https://philpapers.org/rec/TURNAN-3

Content

1. Introduction

2. The main contradiction of the AI safety problem: AI must simultaneously exist and not exist

3. Decisive strategic advantage via Narrow AI

3.1. Non-self-improving AI can obtain a decisive advantage

3.2. Narrow AI is used to create non-AI world-dominating technology

3.3. Types of Narrow AI which may be used for obtaining a DSA

3.4. The knowability of a decisive advantage

4. AI-empowered reconnaissance organization of a nuclear superpower is the most probable place of origin of a Narrow AI DSA

4.1. Advantages of a secret Narrow AI program i...
Opinion Article Against Measuring Impact
by ole.koksvik — 2018-07-25T07:37:01.020Z
Link: https://www.theguardian.com/global-development/2018/jul/16/buzzwords-crazes-broken-aid-system-poverty

A strange opinion article in The Guardian today: it is not entirely clear whether the authors object to a concern with effectiveness, or just think that "assessing the short-term impacts of micro-projects" is somehow misguided (and if so, why that is).
The Evil Genie Puzzle
by Chris_Leong — 2018-07-25T06:12:53.598Z

Foolish you. You decided to rub a lamp in the hope that a genie would appear and you found one, only it's an Evil Genie. He tells you only have two options: either wishing for your perfect life or being pelted with rotten eggs. You start thinking about how wonderful your new life will be, but of course, there's a catch.

In the past, wishing for a perfect life would have come with side-effects attached, but due to past abuses, the Council of Genies has created new, updated rules which ban any unwanted side-effects for the person who makes the wish or any of their loved one's. However, there's no rule against messing with other people. He tells you that he has already predicted your choice and if he predicted that you will wish for a perfect life, he has already created a million clones of you in identical rooms on Mars who will believe that they've just rubbed the lamp. However, since they've only hallucinated this, he doesn't have any obligations towards them and he will use his powers to torture them if they will choose the perfect life. On the other hand, if he predicts that you will choose to be pelted with rotten eggs, then he won't create any clones at all. Genies are well known to be perfect predictors.

Assuming you are a perfectly selfish agent who doesn't care about their clones, what decision ought you take? If you choose to be pelted, there's a 100% chance that if you had wished for a perfect life, you would have received it. On the other hand, if you wish for a perfect life, there's an overwhelming chance that you will wish that you had wished to be pelted. No matter what decision you make, it seems that you will immediately regret having made it (at least before it is revealed whether you are the original in the case where you choose the perfect life).

Extra Info: Some people might argue that perfect clones necessarily are you, but they don't actually have to be perfect clones. The genie could just clone someone else and give them the memory of finding the lamp and this ought to create the requisite doubt as to whether you are a real human on Earth or a clone on Mars.

Prior version of this punished the clones for existing or if it is predicted that they chose the perfect life, but the clones are now punished if they choose the perfect life.