trentmkelly/LessWrong-43k huggingface dataset:
Unyielding Yoda Timers: Taking the Hammertime Final Exam
by TurnTrout — 2018-04-03T02:38:48.327Z

The Hammertime Final Exam

Difficulty: Vorpal Dragonscale Sledgehammer of the Whale

Write all three. For each essay, give yourself five minutes to brainstorm and five minutes to write.1

Preflection

Design an instrumental rationality technique.

I have one "spirit" which I regard as being the best version of myself and which I can trust to reliably make good choices. Unfortunately, when I'm making decisions, another spirit is often more or less in charge. One approach to mitigating this is making a TAP wherein you simulate my post facto opinion of the (usually bad) choice you're about to make. This interpolates between your current spirit and the normatively-correct one. Pretending to be someone else / another version of yourself is, in my experience, surprisingly effective.

Confusion Identification

Introduce a rationality framework.

You're doing your homework, and you're stuck. You don't know what to do next.

One approach is to set a Yoda Timer for 5 minutes; write down a broad-strokes explanation of the problem, where you are presently, and what a solution would look like. Then go to a friend (or, if you're alone, an imaginary friend) and explain it. They don't have to understand, but you have to understand what you're saying while you're saying it. You don't have to have a gears-level understanding of the solution (yet), but you should have a System 1 understanding of your outline.

Be on the lookout for any mental bumps, any areas you gloss over with a trace of discomfort. These dark crevices beckon to you; into them you must descend if you are to emerge solution in hand.

Emotion Propagation

Describe a cognitive defect.

At my undergraduate institution, I was incredibly social - I eagerly crossed cliques to make new friends wherever I went. This demeanor carried over to my first trimester of graduate school; then, I became depressed.

After taking a few months leave, I managed to fully recover. I returned to my graduate institution and found my desire to socialize was neutered. This problem plagued me for nearly a year; it is only within the last month that I pinpointed the cause. My attitude towards interpersonal interaction never left the depressed state.

Just as the human mind doesn't enforce belief consistency, it won't necessarily propagate attitudinal changes to relevant areas of your life. Do ... ,,
Musings on Exploration
by Diffractor — 2018-04-03T02:15:17.000Z

[epistemic status: Polemical, representative of my opinions and not those of any others, plausibly flawed in some places, generally endorsed.]

Looking at the two main settings of decision theory so far, Proof-based decision theory and Logical Inductor decision theory, they both have exploration steps.

Proof-based decision theory has an implicit exploration step, where in models of PA where there's a proof that the agent doesn't take a particular action, it is possible to prove arbitrarily good outcomes, inducing the agent to take the action.

Logical Inductor decision theory has an explicit exploration step, where if the agent is sufficiently certain that it won't take a particular action, it takes that action.

Both of these are motivated by the issue where, if it is certain that an agent won't take a particular action, arbitrarily bad guesses of what happens if the agent takes an action can persist, and won't be removed, because the agent doesn't take that action. Both forms of exploration step can be thought of as maintaining a model/tiny probability where the action is taken, so EDT-style conditioning works. However, conditionals are not counterfactuals (This particular link is extra-important to read)

Further, this isn't an issue specifically with 0-probability events, in general, the estimates of what happens conditional on a low-probability event are worse than estimates of what happens conditional on a high-probability event, as shown in theorem 3.3 of the Optimal Poly-Time Estimator paper. In short,
if you've got an optimal estimator PL for the indicator function that dictates whether property L holds, and you've got an optimal estimator PL⋅f for the function that's 0 when L(x) is false, and f(x) when L(x) is true, then PL⋅fPL is an optimal estimator for f(x) conditional on L being true. However, the error in the original optimal estimators is blown up by a factor of 1D(n), where D(n) is the portion of the probability distribution where property L holds. As the probability of something shrinks, the error in using standard conditional probabilities increases because a tiny error in P(ϕ∧ψ) gets amplified when you divide by P(ψ), which is very small.

#Troll Bridge

Troll Bridge is the decision theory problem where there's a troll that blows up a bridge that you want to cross precisely when your exploration step is active. The utility of staying on your side of t

...
Suffering and Intractable Pain
by Gordon Seidoh Worley — 2018-04-03T01:05:30.556Z
Link: https://mapandterritory.org/suffering-and-intractable-pain-e7115a5acae3

I’ve been writing a lot about AI alignment lately, so let’s take a break from that to talk about a lighter subject: suffering.

Suppose you contract the rare and as yet untreatable disease boneitis. The doctors estimate you have one year to live. In response you might spend the last year of your life living it up, pour all your efforts into finding a cure, or become so depressed that you end your own life before boneitis can. Whatever course of action you might take, we can measure it along a dimension ordering your potential responses from least to most trying to affect change in the world, ranging from totally accepting and abiding the disease to totally rejecting it and working to prevent it from killing you. Let’s call this the accept-reject measure. Where do you think, given things I have written, I would personally advise you to fall along this dimension?

I ask because several people have expressed to me a belief that I would take an accept-only strategy in this and similar scenarios, and this is a dangerous misunderstanding of my thinking to the extent that others may model their actions on my writing. Certainly I think most people would benefit by their own standards to abide more and try to change the world less, but how I see this working is perhaps even more important than that I see it, because naively applied a move in this direction encourages quietism, deathism, and general tolerance of suffering. And the key to how I see more abiding helping is through more abiding only intractable pain so we can transform it from suffering into a neutral or even positive experience.

The Phenomenological Origin of Suffering

We start by asking, “what is suffering?”. Or, more tractably, “why do I suffer?”. We approach this phenomenologically via a reduction of suffering, and to do that we need first to identify the intentional relationship under consideration, but we run into a problem right away because in English we say “I suffer”, making the object of experience unclear. Thus before our reduction even begins we find ourselves challenged to understand what it looks like to suffer.

To get our ground, let’s explore the etymology of “suffer”. It has its roots in Proto-Indo-European “bher”, the action of carrying or bearing as in “I carry/bear a heavy load”. ...
Why Karma 2.0? (A Kabbalistic Explanation)
by Ben Pace — 2018-04-02T20:43:02.032Z

It was Easter Sunday, the obvious time to improve the karma system.

Why? Well, firstly, why do we even have a karma system? You might think that the obvious explanation is Buddhist karma, but the Bible can in fact explain this fact all by itself.

In the West, the word ‘karma’ derives from the biblical word ‘Carmel’. Carmel means ‘garden’, and Mt. Carmel is the place where the prophet Elijah brought back the people to their allegiance to God via empiricism - testing whether he or the Priests of Baal would be able to cause God to set their bulls afire. As such, Karma is a natural gardening tool for a community that is built around empiricism.

Secondly, why did karma need to connect to various levels of size? Well, Easter is the holiday of rebirth.

In the East, rebirth is closely connected to karma - the quantity of karma that one has determines what life form you will become in your next life. In Buddhism, there are six realms of rebirth. In decreasing order of size, there are the heavenly, demigod, human, animals, ghosts, and residents of hell. Similarly on LessWrong, we presently have six different classes of karma, and all users fit into them. So the karma quantity should determine which of the six sizes you fit into.

That is why on Easter Sunday it was necessary that we changed the gardening tools on this site-of-empiricism to alter the users into six sizes.
Internal Diet Crux
by Jacob Falkovich — 2018-04-02T05:05:52.951Z

Crossposted from Putanumonit.com.

Dieter: Dude, we’re fat. We should do something about it. Anything at all, really. How about we stop eating refined carbs from animal sources between 2-7:15 pm each day?

Dad bod: How about we don’t do that? We’re spending so much energy thinking about diets you’d think that activity by itself should make us lose weight. And yet, like everything else we’ve tried, it doesn’t. Maybe we should face reality and accept that the pounds are there to stay.

Dieter: You mean give up like a lazy fat loser?

Dad bod: More like optimize allocation of self-improvement effort to more fruitful pursuits.

Dieter: I was going to start calling you names, but that’s not going to help us have a conversation. How about we introduce ourselves instead? I’m the part of Jacob that hates being overweight and thinks we need to take dieting more seriously.

Dad bod: And I’m the part that thinks dieting involves a lot of suffering with no payoff, and that it’s perfectly OK for Jacob to keep living inside a portly dad bod.

Dieter: “Portly”, huh? Curious that you didn’t go for any of the less dignified synonyms, like “pudgy” or “chubby”. Let’s start with the facts: Jacob is 185 lbs at 5’9″, which comes out to a BMI of 27.3. That’s smack in the middle of the “Overweight” BMI range.

Internalized Scott Alexander: You know BMI isn’t very scientific, you should measure your body fat composition and lean muscle mass…

Dieter: Shut up, Scott. Jacob is of average height, has an average body type, and his bones aren’t made of adamantium. We’re right in the middle of the range that BMI is calibrated on. We’ve been avoiding doing body-fat measurements because we’re afraid of seeing the results. Would anyone here not press a button that would make Jacob weigh 165 pounds tomorrow?

Dad bod: I would press that button, but it probably doesn’t exist. Jacob was 165 lbs at age 18 at the end of combat boot camp, and since then has been steadily gaining 2 lbs a year. There has been almost no variation around this slow trend-line. Diets, exercise plans, changing countries, dating super fit girls and curvy ones – none of that has made any measurable impact at all. Since hitting age 30 we seem to have stabilized around 185 lbs. So why do we keep wanting to “do something about Jacob’s weight”?

Dieter: Have we ever really tried a diet plan? Communism has been tried and found wanting, but dieting h...
Can corrigibility be learned safely?
by Wei Dai — 2018-04-01T23:07:46.625Z

EDIT: Please note that the way I use the word "corrigibility" in this post isn't quite how Paul uses it. See this thread for clarification.

This is mostly a reply to Paul Christiano's Universality and security amplification and assumes familiarity with that post as well as Paul's AI alignment approach in general. See also my previous comment for my understanding of what corrigibility means here and the motivation for wanting to do AI alignment through corrigibility learning instead of value learning.

Consider the translation example again as an analogy about corrigibility. Paul's alignment approach depends on humans having a notion of "corrigibility" (roughly "being helpful to the user and keeping the user in control") which is preserved by the amplification scheme. Like the information that a human uses to do translation, the details of this notion may also be stored as connection weights in the deep layers of a large neural network, so that the only way to access them is to provide inputs to the human of a form that the network was trained on. (In the case of translation, this would be sentences and associated context, while in the case of corrigibility this would be questions/tasks of a human understandable nature and context about the user's background and current situation.) This seems plausible because in order for a human's notion of corrigibility to make a difference, the human has to apply it while thinking about the meaning of a request or question and "translating" it into a series of smaller tasks.

In the language translation example, if the task of translating a sentence is broken down into smaller pieces, the system could no longer access the full knowledge the Overseer has about translation. By analogy, if the task of breaking down tasks in a corrigible way is itself broken down into smaller pieces (either for security or because the input task and associated context is so complex that a human couldn't comprehend it in the time allotted), then the system might no longer be able to access the full knowledge the Overseer has about "corrigibility".

In addition to "corrigibility" (trying to be helpful), breaking down a task also involves "understanding" (figuring out what the intended meaning of the request is) and "competence" (how to do...
Global insect declines: Why aren't we all dead yet?
by eukaryote — 2018-04-01T20:38:58.679Z

One study on a German nature reserve found insect biomass (e.g., kilograms of insects you’d catch in a net) has declined 75% over the last 27 years. Here’s a good summary that answered some questions I had about the study itself.

Another review study found that, globally, invertebrate (mostly insect) abundance has declined 35% over the last 40 years.

Insects are important, as I’ve been told repeatedly (and written about myself). So this news begs a very important and urgent question:

Why aren’t we all dead yet?

This is an honest question, and I want an answer. Insects are among the most numerous animals on earth and central to our ecosystems, food chains, etcetera. 35%+ lower populations are the kind of thing where, if you’d asked me to guess the result in advance, I would have expected marked effects on ecosystems. By 75% declines – if the German study reflects the rest of the world to any degree – I would have predicted literal global catastrophe.

Yet these declines have been going on for apparently decades apparently consistently, and the biosphere, while not exactly doing great, hasn’t literally exploded.

So what’s the deal? Any ideas?

Speculation/answers welcome. Try to convey how confident you are and what your sources are, if you refer to any.

(If your answer is “the biosphere has exploded already”, can you explain how, and why that hasn’t changed trends in things like global crop production or human population growth? I believe, and think most others will agree, that various parts of ecosystems worldwide are obviously being degraded, but not to the degree that I would expect by drastic global declines in insect numbers (especially compared to other well-understood factors like carbon dioxide emissions or deforestation.) If you have reason to think otherwise, let me know.)

Crossposted from my personal blog. Also see Brian Tomasik's comment on the original post, which I haven't assessed in detail but which seems important.
Announcing Rational Newsletter
by Alexey Lapitsky — 2018-04-01T14:37:50.461Z

Hi,

I noticed in the past there were a few efforts to start a periodic feed with highlights from the rationalist community.

Here's my attempt: Rational Newsletter

It's a weekly recap of the best articles from the rationalist community of LessWrong, Slate Star Codex, Overcoming Bias, Intelligence.org and similar sources.

All posts are curated by hand, and going forward I'm planning to keep ~5 posts per category, short and sweet. I wanted to ask for your feedback:

Do you like the format?
Do you like the classification and the quantity of the articles?

Any feedback and criticism is super welcome!
April Fools: Announcing: Karma 2.0
by habryka — 2018-04-01T10:33:39.961Z

Ever since we started work on the new LessWrong, improving the karma system has been top priority for us.

We started giving users a higher vote-weight when they themselves had accumulated enough karma, and have plans to implement various more complicated schemes, such as Eigenkarma. But we recently realized that more important than how karma is calculated, is how karma actually influences the user experience.

The key purpose of a karma system is to allocate the attention-bandwidth on the site towards users who make the most valuable contributions. Historically we have done this via sorting, but sorting is only a small fraction of how we can possibly allocate attention. And as soon as we realized this, the next step was obvious.

Adjust the size of your content, based on your karma

This means, as you get more karma on the site, your comments literally get larger. At the beginning, you will be a mere dust-speck among giants in the comments, but after you engage with the site for a bit, your words can tower over those of your contemporaries.

This changes everything. Vertical space on the page is now properly allocated according to our best guess about what content you will want to read. People's influence and history on the site is clearly communicated to anyone skimming a comment thread. Competing with your friends for karma now literally translates to determining who has the larger "D". The positive effects are too numerous to all list exhaustively here.

We believe this truly revolutionizes the way karma works on LessWrong, which is why we are proud to call this new system "Karma 2.0". We also believe there are many more promising improvements in this direction. We are soon planning to experiment with coloring your comments green or red based on the ratio of upvotes to downvotes your content received, and adjusting the line-height of your posts based on our estimate of how vacuous your claims are (to properly signal to the user the correct ratio of content to "hot air"). Stay tuned for the upcoming "Karma 2.1" and "Karma 2.2" which will implement these features.

However, if for some inscrutable reason, you want to disable Karma 2.0, you can do so by editing your profile (click on your username and then click "Edit Account") and checking the "Deactivate Karma 2.0" checkbox.

Signed Oliver Habryka, Ben...
Welcome to LessWrong Prague [Edit With Your Details]
by Jan_Kulveit — 2018-04-01T10:23:36.557Z

(The following are our suggestions for what kind of information is best to include in the welcome post of your group, feel free to replace them with whatever you think is best)

What kind of events does your group usually run? What does it usually do?

How frequently does your group organize events or meet?

Who would be a good fit for you group?

Should they have any particular skills or have done some specific background reading?
One-Year Anniversary Retrospective - Los Angeles
by RobertM — 2018-04-01T06:34:27.315Z

In the spirit of previous retrospectives, here's one from Los Angeles.

Last Wednesday, we had our 52nd consecutive meetup. This is noteworthy, I think, for a couple reasons:

We do weekly rather than the more common bi-weekly or monthly meetups
We suffered from a long period of near-zero attendance after most of our core members left to fight the good fight with MIRI in late-2015 to early-2016

Here are the things that may have contributed to the success:

I "rebooted" the meetup when Scott announced SSC Meetups Everywhere. This was an excellent Schelling Point which brought back a lot of older members and also many newcomers. We had ~22-24 people show up over the course of the night, which was a little overwhelming even given the spaciousness of our location. There were probably 5-6 conversations going at any given point. The breakdown was ~10 regulars and 12+ newcomers. 2 of those newcomers became consistent regulars and a few others showed up at least one more time. (It's worth noting that the new regulars have nearly universally been people who live close to the meetup - Los Angeles is a big city and transportation sucks!)
Topics! Having topics seems to be very important, even if the focus given to them at the actual meetup is minimal (we do always discuss the topic, but there's no rigid structure/schedule, and for the most part conversation flows naturally).
Social bonding. Post-reboot, the focus of the group changed from "we're here to discuss this topic and interesting insights related to it, or to learn something new" to "we're here to hang out with people who, in general, think this is an interesting topic, but we have lots of other stuff to talk about too". Many members of the group also started spending social time with each other outside of the meetup, including throwing semi-regular parties and traveling together.
Reliability. Knowing that a sufficient number of people will be at the meetup is a strong motivator for both regulars and newcomers to attend. Obviously this is hard to bootstrap, but the reboot and Topics! both factored into this. Our smallest group size (near a major holiday) was 4 people, and our largest (excluding the first one) was ~14. 8-10 is the more usual number. As the organizer, I have made a point of attending every meetup. It's fairly fortunate that all of th

...
My take on agent foundations: formalizing metaphilosophical competence
by zhukeepa — 2018-04-01T06:33:10.372Z

I have some rough intuitions about the purpose of MIRI's agent foundations agenda, and I'd like to share them here. (Note: I have not discussed these with MIRI, and these should not be taken to be representative of MIRI's views.)

I think there's a common misconception that the goal of agent foundations is to try building an AGI architected with a decision theory module, a logical induction module, etc. In my mind, this is completely not the point, and my intuitions say that approach is doomed to fail.

I interpret agent foundations as being more about providing formal specifications of metaphilosophical competence, to:

directly extend our understanding of metaphilosophy, by adding conceptual clarity to important notions we only have fuzzy understandings of. (Will this agent fall into epistemic pits? Are its actions low-impact? Will it avoid catastrophes?) As an analogy, formally defining mathematical proofs constituted significant progress in our understanding of mathematical logic and mathematical philosophy.
allow us to formally verify whether a computational process will satisfy desirable metaphilosophical properties, like those mentioned in the above parenthetical. (It seems perfectly fine for these processes to be built out of illegible components, like deep neural nets—while that makes them harder to inspect, it doesn't preclude us from making useful formal statements about them. For example, in ALBA, it would help us make formal guarantees that distilled agents remain aligned.)

I want to explore logical induction as a case study. I think the important part about logical induction is the logical induction criterion, not the algorithm implementing it. I've heard the implementation criticized for being computationally intractable, but I see its primary purpose as showing the logical induction criterion to be satisfiable at all. This elevates the logical induction criterion over all the other loose collections of desiderata that may or may not be satisfiable, and may or may not capture what we mean by logical uncertainty. If we were to build an actual aligned AGI, I would expect its reasoning process to satisfy the logical induction criterion, but not look very much like the algorithm presented in the logical induction paper.

I also think the logical induction criterion provides an exact formalization—a necessary AND sufficient ...