trentmkelly/LessWrong-43k huggingface dataset:
February 2018 Media Thread
by ArisKatsaris — 2018-02-01T13:26:18.176Z

This is the monthly thread for posting media of various types that you've found that you enjoy. Post what you're reading, listening to, watching, and your opinion of it. Post recommendations to blogs. Post whatever media you feel like discussing! To see previous recommendations, check out the older threads.

Rules:

Please avoid downvoting recommendations just because you don't personally like the recommended material; remember that liking is a two-place word. If you can point out a specific flaw in a person's recommendation, consider posting a comment to that effect.

If you want to post something that (you know) has been recommended before, but have another recommendation to add, please link to the original, so that the reader has both recommendations.

Please post only under one of the already created subthreads, and never directly under the parent media thread.

Use the "Other Media" thread if you believe the piece of media you want to discuss doesn't fit under any of the established categories.

Use the "Meta" thread if you want to discuss about the monthly media thread itself (e.g. to propose adding/removing/splitting/merging subthreads, or to discuss the type of content properly belonging to each subthread) or for any other question or issue you may have about the thread or the rules.
Parable of the Faraway Manager
by nonwasp — 2018-02-01T08:38:30.105Z

A man was hired to manage a faraway subsidiary. He alone submitted reports on the status of the factory, and he sent dividends back to the owners.

He began cooking the books, not for any malicious reason, but because the owners refused his request to reduce dividends and increase contigency reserves.

A few years later, there was a disaster. The factory was severely damaged in the worst monsoon in decades. He sent word that the factory was lightly damaged in the disaster, even though it would take months for his report to make it to the owners.

He used the extra funds to repair the factory and blamed reduced profits on the damaged local infrastructure.

In the long run, the owners made more money. The experienced workers, who would lose their jobs and have to start anew at a new factory, made more money.

The manager did not gain or lose however, his skills was in great demand in the area, and could've found a job for the same pay easily, even after the monsoon. No one ever knew about the manager's good deeds.
Hammertime Day 3: TAPs
by alkjash — 2018-02-01T04:10:00.463Z

This is part 3 of 30 in the Hammertime Sequence. Click here for the intro.

A running theme of Hammertime, especially for the next two days, is intentionality, or deliberateness. Instrumental rationality is designed to inject intentionality into all aspects of your life. Here’s how the 10 techniques fit into the intentionality puzzle:

Noticing and having more intentions (Bug Hunt, CoZE, TAPs).
Resolving internal conflict about what you intend to do (Goal Factoring, Focusing, Internal Double Crux).
Learning how to convert intention to action (Yoda Timers, TAPs, Planning).
Injecting intentions into System 1 so you can do what you intend even when you’re not paying attention (TAPs, Design, Mantras).
Injecting intentions into reality so that reality pushes you towards, and not away from your goals (Design).

Trigger-Action Plans (TAPs) are the if-then statements of the brain. Installing a single TAP properly will convert a single intention into repeated action.

Day 3: TAPs

Recommended background reading: Making intentions concrete – Trigger-Action Planning.

1. TAPs 101

TAPs are micro-habits. Here’s the basic setup:

Pick a bug. Again, skip bugs you’re conflicted about.
Identify a trigger. An ideal trigger is concrete and sensory, like “water hitting my face in the shower,” or “when I press the elevator button.”
Decide on an action you want to happen after the trigger. Pick the minimum conceivable action that counts as progress towards solving the bug. Thus, “look at the stairwell” is better than “go up the stairs,” and “sit up in bed” is better than “force myself out of bed.”
Rehearse the causal link. Go to the trigger and act out the TAP ten times. If the trigger is not currently available, visualize it. Focus on noticing and remembering sensory data that will help you notice the trigger.
Check the TAP in a week. Write down the TAP when you intend to do it, and check back in a week to see if its installed. TAPs can require a lot of tweaking.

TAPs take a couple days to install successfully. Today, we will practice installing two TAPs.

2. The Sapience Spell

Many bugs in life can be solved by merely paying attention to them. The most important TAP to install is a meta-TAP, or Sapience Spell, that wakes you up periodically into Kernel Mode and reminds you to pay attention.

Here’s how to learn the Sapience Spell:

Finding the right trigger is of utmost importance. Treat this step with the g...
Sources of intuitions and data on AGI
by Scott Garrabrant — 2018-01-31T23:30:17.176Z

Much of the difficulty in making progress on AI safety comes from the lack of useful feedback loops. We do not have a superintelligent AI to run tests on and by the time we do, it will probably be too late. This means we have to resort to using proxies. In this post, I will highlight what I think are the four most useful proxies we have access to today: modern machine learning systems, humans, groups, and abstract models.

These proxies are not opposed to each other. We need all the data we can get, and the right strategy is to learn from all of them. However each proxy also has its flaws, and will break if used in the wrong way. It is important to pay attention to where each proxy works and where it fails.

Modern Machine Learning Systems

Pros: With ML systems, one can do experiments quickly to get fast feedback loops with lots of quantitative data. Also, AGI will probably be made using something similar to modern machine learning systems, so insights gained by thinking about ML systems today may be particularly easy to port over.

Cons: Modern ML systems are opaque, and it can be very difficult to understand how they are working. This may be a problem we need to overcome eventually anyway, but that doesn't make it any easier to get data from them today.

Modern ML systems are also not very smart, and it is reasonable to expect a qualitative difference between their behavior and the behavior of superintelligent systems. This may lead to a false sense of security, especially in solutions that depend implicitly or explicitly on the AI lacking some particular capability, or using a particular strategy.

Examples: Concrete Problems in AI Safety and Alignment for Advanced Machine Learning Systems both try to reason about AI Safety by using modern machine learning as their primary source of intuitions. The safety teams at OpenAI and DeepMind try to use this as a primary source of data. (Although it would be a very uncharitable to claim that they do not also draw from the other sources, especially abstract models.) AI Safety Gridworlds is a good example of developing intuitions about AI safety from looking at modern systems.

Humans

Pros: Humans are (at least on some axes) the most intelligent systems we have direct access to. We can use this to notice failures of other proxies by noticing that we can do better at optimizing some function using clever tricks that those sy...
Paper Trauma
by Qiaochu_Yuan — 2018-01-31T22:05:43.859Z

Andrew Critch thinks people should be spending more time than they currently are using paper as a working memory aid while thinking, especially large paper (for even more working memory). It is really astonishing how helpful this can be. We consistently encourage people to do it at CFAR workshops nearly every time they learn a new technique or attempt to debug themselves or each other.

Paper is both very helpful and very easy to use - so why aren't people already using it all the time (including me)? I have a few vivid memories of times at CFAR workshops where I had to prod two people who were having a cognitively intensive conversation to use paper as shared working memory, and by "people" I mean CFAR instructors. It's harder than it looks.

My guess is that people have unresolved aversions to paper coming from school, where paper was how other people forced you to do things like homework and tests.

Really, it's horrifying to think about how much of what you've written down on paper was entirely forced on you.

So, you'll need to do some work if you want to reclaim the right to write whatever you want on paper, instead of whatever you've been trained to write. Good luck. Here are some words to take with you on your way.

Just because you are writing on paper does not mean you are still in school.

Nobody is going to grade you.

This is not homework. This is not a test.

This is your mind.

This is your life.

Think.
Epiphenomenal Oracles Ignore Holes in the Box
by SilentCal — 2018-01-31T20:08:57.409Z

Background: https://wiki.lesswrong.com/wiki/Oracle_AI

Epistemic status: Likely wrong, but I don't know how. I'm not a 'real' FAI researcher, just an LW reader inspired by the Alignment Prize to try to start contributing.

(At minimum I'm heavily indebted to Stuart Armstrong for inspiration, and there's a fair chance I'm rediscovering something he's already proposed)

tl;dr AIXI's old anvil problem, which is a serious problem for an agent AI, could actually be a solution for preventing an Oracle AI from escaping its box.

The core idea is for the AI to believe with certainty that it is epiphenomenal; that is, that none of its actions can affect either the world it observes or its internal state. [1] That is, its causal inference capabilities only apply to interactions in the outside world, and its model of itself assumes a logical model of its hardware. It will be motivated to write correct answers on its output bits because setting its output bits to correct answers is a terminal value, not because it expects that to have any effect on the researchers.

This is an unacceptable model for an agent AI since, as the Anvil Problem framing puts it, the AI will be unable to conclude that dropping an anvil on itself will result in it ceasing to function, and sooner or later drop one. But an Oracle AI would have a very limited capacity to do anything of the sort, and no reason to strongly optimize for such an end.

We can see the effectiveness of this approach most dramatically by considering what would happen if the AI accidentally found a box-breaking exploit (via side channels, not through how it answers questions). Suppose the AI has a calculation it wants to perform that would take too long on its available hardware, and the AI accidentally creates a global botnet to perform the calculation for it and send the results to its inputs. The AI would be unable to conclude that its actions had caused this result, and so settle on some other explanation for why the correct answer to its desired computation had just happened to show up in its inputs.

Since our AI is smart, it's probably going to settle rapidly on the closest available thing to the real explanation and conclude that there is an AI just like it instantiated physically in the world it observes.[1] It wouldn't need to stumble on an exploit to conclude this--if it, ...
An experiment
by cousin_it — 2018-01-31T12:20:25.248Z

Alexei's postmortem of Arbital got me thinking about incentivizing good online explanations, so I got an idea for a test that we can run here and now:

What's your simplest, most vivid, most memorable explanation of correlation does not equal causation for someone who's unaware of the concept? In your own words, no linking.
OpenPhil's "Update on Cause Prioritization / Worldview Diversification"
by Raemon — 2018-01-31T05:39:46.181Z
Link: https://www.openphilanthropy.org/blog/update-cause-prioritization-open-philanthropy

OpenPhil recently posted an update on their overall cause prioritization strategy. In particular, exploring the concept of worldview diversification:

When choosing how to allocate capital, we need to decide between multiple worldviews. We use “worldview” to refer to a highly debatable (and perhaps impossible to evaluate) set of beliefs that favor a certain kind of giving. Worldviews can represent a mix of moral values, views on epistemology and methodology, and heuristics. Worldviews that are particularly salient to us include:

• a “long-termist” worldview that ascribes very high value to the long-term future, such that it assesses grants by how well they advance the odds of favorable long-term outcomes for civilization;

• a “near-termist, human-centric” worldview that assesses grants by how well they improve the lives of humans (excluding all animal welfare considerations) on a relatively shorter time horizon;

• a “near-termist, animal-inclusive” view that focuses on a similarly short time horizon but ascribes significant moral weight to animals.

If we evaluated all grants’ cost-effectiveness in the same terms (e.g., “Persons helped per dollar, including animals” or “Persons helped per dollar, excluding animals”), this would likely result effectively putting all our resources behind a single worldview.

For reasons given below (and previously), we don’t want to do this. Instead, we’re likely to divide the available capital into buckets, with different buckets operating according to different worldviews and in some cases other criteria as well. E.g., we might allocate X% of capital to a bucket that aims to maximize impact from a “long-termist” perspective, and Y% of capital to a bucket that aims to maximize impact from a “near-termist” perspective, with further subdivisions to account for other worldview splits (e.g., around the moral weight of animals) and other criteria.

These buckets would then, in turn, allocate “their” capital to causes in order to best accomplish their goals; for example, a long-termist bucket might allocate some of its X% to biosecurity and pandemic preparedness, some to causes that seek to improve decision-making generally, etc.

The full post is quite long, but seemed useful both to highlight a useful way of thinking, and to generally stay up-to-date on how OpenPhil is strateg
Hammertime Day 2: Yoda Timers
by alkjash — 2018-01-31T05:10:00.279Z

This is part 2 of 30 in the Hammertime Sequence. Click here for the intro.

No! Try not! Do, or do not. There is no try.
—Yoda

There's a copy of Barney Stinson in my head who pops up every so often to say: "Challenged Accepted!" When Eliezer wrote about the biggest mistakes in the Sequences, my inner Barney started bouncing off the walls. Hammertime is a sequence designed to correct the three top mistakes by:

Creating a program to actually practice rationality.
Emphasizing doing better in everyday life.
Focusing on rational action instead of rational belief.

This is going to be legen ... wait for it ... dary!

Day 2: Yoda Timers

Look, you don't understand human nature. People wouldn't try for five minutes before giving up if the fate of humanity were at stake.
—Use the Try Harder, Luke

The Yoda Timer (CFAR calls it a Resolve Cycle) has three simple steps:

Pick a bug.
Set a timer for 5 minutes.
Solve the bug.

Motivation

Before we begin, I want to call attention to two ways to make the most of Yoda Timers.

1. The One Inch Punch

Pick something you're afraid of doing. Suppose I told you, "Try!" Try as hard as you can. What does that feel like?

Now suppose I told you, "Just do it!" Actually go and get it done. What does that feel like?

To me, trying feels like pushing hard against my own resistance. Doing feels like pushing hard against reality. The Yoda Timer is designed to teach (or remind) you to notice what pushing against reality feels like.

Bruce Lee was famous for his One Inch Punch, which had such explosive power because every muscle in his body aligned into the punch:

The one-inch punch is a skill which uses fa jin (translated as explosive power) to generate tremendous amounts of impact force at extremely close distances. This "burst" effect had been common in Neijia forms. When performing this one-inch punch the practitioner stands with his fist very close to the target (the distance depends on the skill of the practitioner, usually from 0–6 inches, or 0-15 centimetres). Multiple abdominal muscles contribute to the punching power while being imperceptible to the attacker. It is a common misconception that "one-inch punches" utilize a snapping of the wrist. The target in such demonstrations vary, sometimes it is a fellow practitioner holding a phone book on the chest, sometimes wooden boards can be broken.

...
Seeking an Outside View on Israeli Military Service
by UnderConstruction — 2018-01-30T23:40:42.510Z

I would like your assistance with gathering rational outside view so that I can make better moral decisions. The general context is that I consider myself an effective altruist with mostly Utilitarian ethics, and I have some more time of mandatory service in which I have considerably high degree of freedom to choose my path. The dillema is dependent on many questions, but my goal here is mostly to debias myself on questions relating to the moral stance of Israel or armies in general (although I care much more about the expected impact of my actions, irrespectively of whether it's done in the name of good or evil).

Hoping for a discussion, here are some questions I'd like to get an answer to, which are mainly here to give a broad sense of what I think might be relevant. Any helpful comments would be greatly appreciated, even epistimological tips to help me make better decisions on this topic would be of great help.

Are there armies in any time in history in which individuals should have served in the military to do the most good in expected global utility? If so, what seems to be the criteria?
What are some possible implication of increasing/decreasing Israeli's millitary power?
Even if I conclude that Israel is a force for good (or for evil) in the world, am I likely to think so 20 (50) years from now?
Should I expect a random EA who has done some ammount of (non-combatant) militry service to regret the actions she has done? Since I am taking a Utilitarian perspective, assume that she had an option not to serve in the military.
Arbital postmortem
by alexei — 2018-01-30T13:48:31.399Z

Disclaimer 1: These views are my own and don’t necessarily reflect the views of anyone else (Eric, Steph, or Eliezer).

Disclaimer 2: Most of the events happened at least a year ago. My memory is not particularly great, so the dates are fuzzy and a few things might be slightly out of order. But this post has been reviewed by Eric, Steph, and Eliezer, so it should mostly be okay.

I’m going to list events chronologically. At times I’ll insert a “Reflection” paragraph, where I’m going to outline my thoughts as of now. I’ll talk about what I could have done differently and how I would approach a similar problem today.

Chapter 0: Eliezer pitches Arbital and I say ‘no’

Around the summer of 2014 Eliezer approached me with the idea for what later would become Arbital. At first, I vaguely understood the idea as some kind of software to map out knowledge. Maybe something like a giant mind map, but not graphical. I took some time to research existing and previous projects in that area and found a huge graveyard of projects that have been tried. Yes, basically all of them were dead. Most were hobby projects, but some seemed pretty serious. None were successful, as far as I could tell. I didn’t see how Eliezer’s project was different, so I passed on it.

Reflection: Today, I’d probably try to sit down with Eliezer for longer and really try to understand what he is seeing that I’m not. It’s likely back then I didn’t have the right skills to extract that information, but I think I’m much better at it today.

Reflection: Also, after working with Eliezer for a few years, I’ve got a better feeling for how things he says often seem confusing / out of alignment / tilted, until you finally wrap your mind around it, and then it’s crystal clear and easy.

Chapter 1: Eliezer and I start Arbital

Early January 2015 I was sitting in my room, tired from looking in vain for a decent startup idea, when Arbital popped back into my mind. There were still a lot of red flags around the idea, but I rationalized to myself that given Eliezer’s track record, there was probably something good here. And, in the worst case, I’d just create a tool that would be useful to Eliezer alone. That didn’t seem like a bad outcome, so I decided to do it. I contacted Eliezer, he was still interested, and so we started the project.

Reflection: The decision process sounds a bit silly, but I don’t think it’s a bad one. I real...
Hammertime Day 1: Bug Hunt
by alkjash — 2018-01-30T06:40:00.323Z

Rationality is systematized winning.

In Hammers and Nails, I suggested that rationalists need to be more systematic in the practice of our craft. In this post, I will use the word Hammer for a single technique well-practiced and broadly applied.

Hammertime is a 30-day instrumental rationality sequence I am designing for myself to build competence with techniques. Its objective is to turn rationalists into systematic rationalists. By the end of this sequence, I hope to upgrade each Hammer from Bronze Mace to Vorpal Dragonscale Sledgehammer of the Whale. I invite you to join me on this journey.

The core concept: One Day, One Hammer.

Hammertime Schedule

In Hammertime, we will practice 10 Hammers over 30 days. Each exercise is scalable from a half hour to an entire day. The Hammers will be bootleg CFAR techniques:

Bug Hunt
Resolve Cycles
TAPs
Design
CoZE
Mantras
Goal Factoring
Focusing
Internal Double Crux
Planning

There will be three cycles of 10 days each, practicing each technique a total of three times. The first cycle will cover basics and solve bugs at the life-hack level. The second cycle will reinforce the technique, cover variations and generalizations, and solve tougher challenges. The third cycle will build fluid compound movements out of multiple core techniques.

Day 1: Bug Hunt

A bug is anything in life that needs improvement. Even if something is going well, if you can imagine it going better, there's a bug.

On the first day of Hammertime, we will scour our lives with a fine-toothed comb to find as many bugs as possible. A comprehensive bug list will provide the raw material on which we practice every other rationality technique. For the first cycle of Bug Hunt, look for small, concrete bugs. The whole exercise should take a bit over an hour.

WARNINGS: Focus on finding bugs, not solving them. If you can solve the bug immediately, go for it. Otherwise, hold off on proposing solutions. Writing down a bug does not mean you commit to doing anything about it.

1. Setup

Find a notebook, phone app, spreadsheet, or Google Doc to record your bugs - preferably something you can bring with you throughout the day. We will refer back to it repeatedly in the coming days for bugs to solve.

During Bug Hunt, spend the next 30 minutes writing down as many bugs as you can. Following each of the six sets of prompts in the next section, set a timer for 5 minutes and list as many bugs as ...
RSS Feeds are fixed and should be properly functional this time
by habryka — 2018-01-30T00:47:58.791Z

Just pushed a fix that should make RSS feeds work properly (you will have to resubscribe to the RSS feeds, if you did so a while ago). Thanks to squirrelInHell who made me aware of the deeper problems with our RSS feed (that my RSS reader seemed to mostly automatically patch, but which other people ran into).