trentmkelly/LessWrong-43k huggingface dataset:
Instant stone (just add water!)
by jasoncrawford — 2019-11-13T22:33:39.903Z
Link: https://rootsofprogress.org/instant-stone-just-add-water

Originally posted on The Roots of Progress, January 6, 2018

From the time that humans began to leave their nomadic ways and live in settled societies about ten thousand years ago, we have needed to build structures: to shelter ourselves, to store our goods, to honor the gods.

The easiest way to build is with dirt. Mud, clay, any kind of earth. Pile it up and you have walls. A few walls and a thatched roof, and you have a hut.

Earthen hut with thatched roof in Sudan - Petr Adam Dohnálek / Wikimedia

But earthen construction has many shortcomings. Dirt isn’t very strong, so you can’t build very high or add multiple stories. It tends to wash away in the rain, so it really only works in hot, dry climates. And it can be burrowed through by intruders—animal or human.

We need something tougher. A material that is hard and strong enough to weather any storm, to build high walls and ceilings, to protect us from the elements and from attackers.

Stone would be ideal. It is tough enough for the job, and rocks are plentiful in nature. But like everything else in nature, we find them in an inconvenient form. Rocks don’t come in the shape of houses, let alone temples. We could maybe pile or stack them up, if only we had something to hold them together.

If only we could—bear with me now as I indulge in the wildest fantasy—pour liquid stone into molds, to create rocks in any shape we want! Or—as long as I’m dreaming—what if we had a glue that was as strong as stone, to stick smaller rocks together into walls, floors and ceilings?

This miracle, of course, exists. Indeed, it may be the oldest craft known to mankind. You already know it—and you probably think of it as one of the dullest, most boring substances imaginable.

I am here to convince you that it is pure magic and that we should look on it with awe.

It’s called cement.

Let’s begin at the beginning. Limestone is a soft, light-colored rock with a grainy texture, which fizzes in the presence of acid. Chalk is a form of limestone. What distinguishes limestone and makes it useful is a high calcium content (“calcium” and “chalk” are cognates). Specifically, it is calcium carbonate (CaCO3), the same substance that makes up seashells. In fact, limestone, a sedimentary rock, is often formed from crushed seashells, compressed over eons.

Limestone from a quarry in southern Germany - Hannes Grobe / Wikimedia

Limestone can be used for many purpo...
Robin Hanson on the futurist focus on AI
by abergal — 2019-11-13T21:50:01.893Z

Robin Hanson

Robert Long and I recently talked to Robin Hanson—GMU economist, prolific blogger, and longtime thinker on the future of AI—about the amount of futurist effort going into thinking about AI risk.

It was noteworthy to me that Robin thinks human-level AI is a century, perhaps multiple centuries away— much longer than the 50-year number given by AI researchers. I think these longer timelines are the source of a lot of his disagreement with the AI risk community about how much of futurist thought should be put into AI.

Robin is particularly interested in the notion of ‘lumpiness’– how much AI is likely to be furthered by a few big improvements as opposed to a slow and steady trickle of progress. If, as Robin believes, most academic progress and AI in particular are not likely to be ‘lumpy’, he thinks we shouldn’t think things will happen without a lot of warning.

The full recording and transcript of our conversation can be found here.
[Link] (EA Podcast) Social Status: The Key to the Matrix Part I
by Daniel_Gambacorta — 2019-11-13T21:09:24.657Z

A new episode of Global Optimum has been released! Global Optimum is a podcast aimed at making altruists more effective. This episode is about the psychology of social status. The desire for status is a fundamental human motive. Understanding status can help us understand many otherwise puzzling features of our world. In this episode, I apply our understanding of status psychology to analyze various dynamics and trends within effective altruism.

This episode features:

-How do people behave differently when they are high vs low status?

-How did human social status evolve?

-Should you try to dampen your desire for status?

-Are EAs too credential-focused?

-Is publishing in academic journals overrated?

-Can you get more done by working alone than by starting an organization?

-What causes groups to splinter?

-How has effective altruism “professionalized?” What are the upsides and downsides of this trend?

Full transcript

The podcast is available on all podcast apps.

Listen here
Reconsolidation Through Experience
by Matt Goldenberg — 2019-11-13T20:04:39.345Z

At this level, you're simply sitting with the schema. Oftentimes, this alone can produce a dramatic shift through memory reconsolidation simply as a matter of re-evaluating the old schema when you get a look at it. I also include non-memory reconsolidation techniques in this category that work through extinction of the old association.

Experiencing Evidence

Memory Collection

Memory collection involves finding and allowing yourself to experience every time you can remember when you felt or acted through a particular belief schema. For each piece of evidence, you allow yourself to think and feel through the schema in that particular memory, then move on to the next one. By simply allowing ourselves to see all the evidence for a schema with our current perspective, we can often come to a more nuanced understanding of the belief. I first learned about this technique through PJ Eby,who calls it the Feeling Elimination Technique. The book, thinking things done, that it was included in, is no longer available as far as I'm aware.

Imaginal Exposure

Imaginal exposure is a process of taking a particular salient memory/fear/trauma etc, and simply running through the experience over and over. This can often trigger memory reconsolidation, and, even if not, can change your schema through simple exhaustion induced extinction. EMDR is a version of imaginal exposure that adds the modality of moving your eyes back and forth as you run through the memory. My current best view of the evidence is that EMDR has no long term benefit over imaginal exposure, but the data is confusing.

Experiencing Beliefs

The Acceptance Statement

An acceptance statement is a tool from Coherence Therapy, that involves taking a belief, and the choices you've made about a belief and saying them out loud. For instance I could state out loud "I should not express myself. If I do, I'll be ostracized. I've decided it's more important to not be ostracized than to be fully seen." Saying this out loud is often enough to trigger the schema, and hearing out out loud is often enough to automatically trigger the questioning of the schema.

You can enhance this process by (in your imagination or in real life), saying the statement outloud to someone who has been affected by the choices you've made. I could imagine saying to my partner: "I should not express myself. If I do,...
The Hierarchy of Memory Reconsolidation Techniques
by Matt Goldenberg — 2019-11-13T20:02:43.449Z

As discussed in the previous post, the aim is to start with the techniques that least challenge existing schema, and then work up the hierarchy to techniques that more strongly challenge the existing schema.

Experience involves simply sitting with the schema as it is, and seeing if change comes from prolonged viewing of the schema.

Question involves posing questions about the schema, to draw attention to ways that it may be wrong.

Challenge involves directly providing counters to the schema, and seeing how it reacts.

Dissonance involves holding both the counter to the schema and the schema itself at once.

Change involves attempting to directly change how you represent the schema to take your counter into account.
Practical Guidelines for Memory Reconsolidation
by Matt Goldenberg — 2019-11-13T19:54:10.097Z

This post details a set of guidelines for working with the memory reconsolidation tools in the rest of the sequence. Use it to get the most out of your memory reconsolidation procedure.

Start with the More Cognitively Fused Schema

For every belief schema you're working with, there's (at least) two belief schema's at play. There's the side that believes a particular thing, and then there's a side that wants you to question the belief in that thing. As a general rule, you should always start with the side that's more cognitively fused.

As an example, I was working with someone who was having issues going to bed on time, and wanted to change that. Before we started looking at the schema of "I should avoid ruminating by staying up late," We first examined the schema of "I should get more sleep."

By starting with the schema that you're more cognitively fused with, you avoid confirmation bias and end up with more accurate beliefs at the end.

The Resistance is the Way

If at any point, you encounter resistance to working on a particular technique with a particular schema, what you've found is a "Meta-schema" that believes changing this belief would be harmful. Rather than push through this resistance, loop back to the beginning of the Debugging process, and work with this new schema.

As an example, I found myself trying to change the schema that "I should avoid failure". I kept getting resistance, looped back, and found the schema "Most people should like me.", only once I worked on reconsolidating that schema was I able to return to the original schema.

Reverse Your Fusion

For any given technique, there's two ways you can approach it. You can work with the schema from "Inside" experiencing it as who you are, or you can work with it from the "Outside", putting some distance between yourself and the schema. As a general rule, I recommend reversing whatever your default is. If you frequently cognitively fuse with a schema, I recommend creating some distance/dissociation from it. If you frequently distance/dissociate from a schema, when doing the technique try as much as you can to fuse with it. This shift in perspective often goes a long way in providing new perspectives on the schema and allowing the re-consolidation to take place.

Go With the Most Salient Access

...
A Practical Theory of Memory Reconsolidation
by Matt Goldenberg — 2019-11-13T19:52:20.364Z

Memory Reconsolidation is one candidate for a scientific of theory of "How to Actually Change Your Mind." In this post, I'll give a few fake frameworks about how memory reconsolidation works, in order to provide intuition pumps for the rest of the sequence.

Schemas as Belief Clusters

For the purposes of this sequence, we'll describe a schema as a particular cluster of felt Aliefs that point towards a goal or set of goals.

I often find it useful to think of beliefs as arranged in a hierarchy of goals, as theorized in Perceptual Control Theory, Connection Theory, and Predictive Processing Theory.

Schemas, or Parts, then represent clusters of these beliefs trying to achieve some specific need.

In this framework, schema's don't represent any sort of natural boundary, and we can expand them and contract them at will based on what we're working on. Schemas themselves can overlap without natural boundaries, and can even contain their own internal conflicts.

3 Steps to Reconsolidate Schemas

In Unlocking the Emotional Brain, the authors state three steps needed for memory re-consolidation:

1. Activate the Old Schema

2. Challenge the Old Schema

3. Learn a Replacement Schema

All of the techniques covered in this sequence implicitly or explicitly cover the three steps. However, step #3 is also augmented later on in the Debugging Process through the "Integration" step.

The Tradeoff Between Activation and Challenge

There's an inherent tradeoff between activating the old schema and challenging it. The stronger and more perceptible the challenge is, the more likely it is to be strong enough to trigger the reconsolidation. However, this strength comes with two downsides:

1. Challenging yourself in a very strong way doesn't feel good. It feels internally violent.

2. The stronger the challenge, the more likely you are to deactivate the schema you're challenging, creating resistance and making reconsolidation impossible.

For this reason, we start with the techniques that provide the weakest challenges, and gradually work our way up to stronger and stronger challenge, waiting till we feel a shift towards a more nuanced and accurate schema.

4 Schema Access Points

In Kaj's Post on Unlocking the Emotional Brain, he points to a paper by Lane et al that give 3 ways to access a schema:

Because I think the names for these 3 methods to acces...
[AN #73]: Detecting catastrophic failures by learning how agents tend to break
by Rohin Shah — 2019-11-13T18:10:01.544Z

Find all Alignment Newsletter resources

here

. In particular, you can

sign up

, or look through this

spreadsheet

of all summaries that have ever been in the newsletter. I'm always happy to hear feedback; you can send it to me by replying to this email.

Audio version

here

(may not be up yet).

Highlights

Rigorous Agent Evaluation: An Adversarial Approach to Uncover Catastrophic Failures

(Jonathan Uesato, Ananya Kumar, Csaba Szepesvari et al) (summarized by Nicholas): An important problem in safety-critical domains is accurately estimating slim probabilities of catastrophic failures: one in a million is very different from one in a billion. A standard Monte Carlo approach requires millions or billions of trials to find a single failure, which is prohibitively expensive. This paper proposes using agents from earlier in the training process to provide signals for a learned failure probability predictor. For example, with a Humanoid robot, failure is defined as the robot falling down. A neural net is trained on earlier agents to predict the probability that the agent will fall down from a given state. To evaluate the final agent, states are importance-sampled based on how likely the neural network believes they are to cause failure. This relies on the assumption that the failure modes of the final agent are similar to some failure mode of earlier agents. Overall, the approach reduces the number of samples required to accurately estimate the failure probability by multiple orders of magnitude.

Nicholas's opinion: I am quite excited about the focus on preventing low likelihood catastrophic events, particularly from the standpoint of existential risk reduction. The key assumption in this paper, that earlier in training the agent will fail in related ways but more frequently, seems plausible to me and in line with most of my experience training neural networks, and the experiments demonstrate a very large increase in efficiency.

I’d be interested to see theoretical analysis of what situations would make this assumption more or less likely in the context of more powerful future agents. For example, one situation where the failure modes might be distinct later in training is if an agent learns how to turn on a car, which then makes states where the agent has access to a car have significantly higher likelihood of catastrophic failures than they did before.

Tech

...
Insights from the randomness/ignorance model are genuine
by Rafael Harth — 2019-11-13T16:18:55.544Z

(Based on the randomness/ignorance model proposed in 1 → 2 → 3.)

The bold claim of this sequence thus far is that the randomness/ignorance model solves a significant part of the anthropics puzzle. (Not everything since it's still incomplete.) In this post I argue that this "solution" is genuine, i.e. it does more than just redefine terms. In particular, I argue that my definition of probability for randomness is the only reasonable choice.

The only axiom I need for this claim is that probability must be consistent with betting odds in all cases: if H comes true in two of three situations where O is observed, and this is known, then P(H|O) needs to be 23, and no other answer is acceptable. This idea isn't new; the problem with it is that it doesn't actually produce a definition of probability, because we might not know how often H comes true if B is observed. It cannot define probability in the original Presumptuous Philosopher problem, for example.

But in the context of the randomness/ignorance model, the approach becomes applicable. Stating my definition for when uncertainty is random in one sentence, we get

Your uncertainty about H, given observation O, is random iff you know the relative frequency with which H happens, evaluated across all observations O′ that, for you, are indistinguishable to O with regard to H.

Where "relative frequency" is the frequency of H compared to ¬H, i.e. you know that H happens in n out of m cases. A good look at this definition shows that it is precisely the condition needed to apply the betting odds criterion. So the model simply divides everything into those cases where you can apply betting odds and those where you can't.

If the Sleeping Beauty experiment is repeated sufficiently often using a fair coin, then roughly half of all experiments will run in the 1-interview version, and the other half will run the 2-interview version. In that case, Sleeping Beauty's uncertainty is random and the reasoning from 3 goes through to output 23 for it being Monday. The experiment being repeated sufficiently often might be considered a reasonably mild restriction; in particular, it is a given if the universe is large enough that everything which appears once appears many times. Given that Sleeping Beauty is still controversial, the model must thus be either nontrivial or wrong, hence "genuine".

Here is an alternative justification for my defi...
Mosquito Net Fishing
by jefftk — 2019-11-13T13:30:01.731Z

I recently

saw
a study claiming:

Distributed mosquito nets are intended to be used for malaria
protection, yet increasing evidence suggests that fishing is a primary
use for these nets, providing fresh concerns for already stressed
coastal ecosystems.

—The
perverse fisheries consequences of mosquito net malaria prophylaxis in
East Africa (Jones and Unsworth, 2019)

Mosquito nets are harmful fishing tools because (a) they're
insecticide-treated and (b) with such small holes you catch a lot of
immature fish before they've had a chance to reproduce. But how
harmful this practice is overall depends on how widespread it is.

The paper only tries to answer the question "is using a mosquito net
for fishing a good idea," and to show that it's a "primary use" of
distributed bednets they cite The
use of mosquito nets in fisheries: A global perspective (Short
et al. 2018). This paper, however, doesn't describe the kind of research
that could back up their claim.

They ran on online survey which they left open to anyone to respond
for two months in 2015. They got 94 responses from people saying
they'd seen mosquito net fishing and 36 from people saying they
hadn't. This sort of survey can be useful for getting initial
qualitative information about whether it's a thing, how it looks, and
where to find it. On the other hand, it's not a good tool for
estimating frequency: you don't know what sort of penetration the
survey got or how much people only responded to the survey when they
had something to share.

A study that tried to answer the question of whether distribution of
mosquito nets leads to mosquito net fishing would make a sample of
waterside communities that had recent net distributions and compare
them to a similar sample that had not. But this is not that study,
and it's not capable of supporting a "primary use" claim.

Now, mosquito net fishing could still be bad even if only a very small
fraction of nets were used that way, due to the indiscriminate way it
collects fish regardless of size. It doesn't mean "people aren't
sleeping under nets" but could mean "nets might be making communities
worse off." But there's not much reason to think this, and this study
doesn't give much new information. GiveWell's 2015
blog post in response to an alarmist
New York Times article is still very reasonable:

Randomized control trials cons

...
[Link] Is the Orthogonality Thesis Defensible? (Qualia Computing)
by ioannes — 2019-11-13T03:59:00.955Z

Full title: Is the Orthogonality Thesis Defensible if We Assume Both Valence Realism and Open Individualism?

https://qualiacomputing.com/2019/11/09/is-the-orthogonality-thesis-defensible-if-we-assume-both-valence-realism-and-open-individualism/

An excerpt:

The cleanest typology for metaphysics I can offer is: some theories focus on computations as the thing that’s ‘real’, the thing that ethically matters – we should pay attention to what the *bits* are doing. Others focus on physical states – we should pay attention to what the *atoms* are doing. I’m on team atoms, as I note here: Against Functionalism.

My suggested takeaway: an open individualist who assumes computationalism is true (team bits) will have a hard time coordinating with an open individualist who assumes physicalism is true (team atoms) — they’re essentially running incompatible versions of OI and will compete for resources.

As a first approximation, instead of three theories of personal identity – Closed Individualism, Empty Individualism, Open Individualism – we’d have six. CI-bits, CI-atoms, EI-bits, EI-atoms, OI-bits, OI-atoms.

Whether the future is positive will be substantially determined by how widely and deeply we can build positive-sum moral trades between these six frames.
What I’ll be doing at MIRI
by evhub — 2019-11-12T23:19:15.796Z

Note: This is a personal post describing my own plans, not a post with actual research content.

Having finished my internship working with Paul Christiano and others at OpenAI, I’ll be moving to doing research at MIRI. I’ve decided to do research at MIRI because I believe MIRI will be the easiest, most convenient place for me to continue doing research in the near future. That being said, there are a couple of particular aspects of what I’ll be doing at MIRI that I think are worth being explicit about.

First, and most importantly, this decision does not represent any substantive change in my beliefs regarding AI safety. In particular, my research continues to be focused around solving inner alignment for amplification. My post on relaxed adversarial training continues to represent a fairly up-to-date form of what I think needs to be done along these lines.

Second, my research will remain public by default. I have discussed with MIRI their decision to make their research non-disclosed-by-default and we agreed that my research agenda is a reasonable exception. I strongly believe in the importance of collaborating with both the AI safety and machine learning communities and thus believe in the need for sharing research. Of course, I also fully believe in the importance of carefully reviewing possible harmful effects from publishing before disclosing results—and will continue to do so with all of my research—though I will attempt to publish anything I don’t believe to pose a meaningful risk.

Third—and this should go without saying—I fully anticipate continuing to collaborate with other researchers at other institutions such as OpenAI, Ought, CHAI, DeepMind, FHI, etc. The task of making AGI safe is a huge endeavor that I fully believe will require the joint work of an entire field. If you are interested in working with me on anything (regarding inner alignment or anything else) please don’t hesitate to send me an email at evanjhub@gmail.com.