trentmkelly/LessWrong-43k huggingface dataset:
Social awkwardness as an application of analysis paralysis
by Paperhopes — 2019-10-01T06:16:21.379Z

That day I was walking in the dull corridors of the university. I stumbled upon this research poster on Alpha Centauri, the kind which they only display in dull corridors. Shortly after, my double-PhD physics teacher (the kind they put in dull corridors) walked behind me saying "hi". Yes, a physics teacher saying "hi", play your suspension of disbelief card. I was taken by surprise and wondered about the reasons he had to greet me (usually he didn't), but I managed to remember I had to answer. I answered "hjwjv".

Later on, I tried wanting to say "hey" and a "hey" sound emanated from my vocal cord: the volition-and-body-movements-are-connected belief still paid rent.

As a thesis-angsted graduate student, I thought...Analysis Paralysis...

There seems to be profound reasoning behind oneself overboard paralytic social thinking: a consciousness of the incompressible whole that is such a concept as a person and, by extension, the induced exponential data processing from trying to scale this whole, the type of thinking that forbids any action. Analysis Paralysis.

Consider the brain: billions of neurones firing in what any derivative would consider an instantaneous lapse of time, capable of reason, technique, reflective thinking, of abstraction, of abstraction square. The current mathematical instruments are unable to model this complexity and the current humans will probably be unable to cognitively model those models between two lines of dialogue.

To “hjwjv” when you want to “hey” is to acknowledge in the other’s self this same bunch of dreads, dreams, sorrows, ambitions that define your own world; an empathetic awe realization of the hidden information singularizing into a slangish west Germanic greeting device. Unable to take into account the size of prior knowledge collapsing in a “hey”, the hardware crashes. Humbly, the shy acknowledges that the complexity of a mere conversation is beyond the scope of seizable magnitudes.

Now one might take this as proof of moral superiority. Only the social incompetent considers people as ends, not mere tools to his goals. Only he can conceptualize the deep true nature of a human-human relation. If the charismatic extravert seems to know, the shy him knows he does not. There is nothing to change about it.

But it doesn't work...
Announcing the Farlamp project
by rmoehn — 2019-10-01T02:16:16.205Z

Announcing the Farlamp project

Project definition:

I'm studying the impact of overseer failure on RL-based IDA,
because I want to know under what conditions the amplification
increases or decreases the failure rate,
in order to help my reader understand whether we need to
combine capability amplification with explicit reliability
amplification in all cases.

In this project I will:

Take the implementation of iterated distillation and amplification from
Christiano et al.'s ‘Supervising strong learners by amplifying weak
experts’ and adapt it to reinforcement
learning. (It is using supervised learning now.)

Introduce overseer failures and see how they influence the overall failure
rate.

Write a paper about the results.

Overseer failures in SupAmp and
ReAmp contains a
more extensive introduction, as well as an explanation of the relevant terms,
concepts etc.

The project repo contains all the public
artifacts I have produced so far.

At the moment I'm expanding my ML skills using the book Hands-On Machine
Learning with Scikit-Learn &
TensorFlow.
After this I will start working on the IDA code.

Paul Christiano has been funding this project,
giving me a chance to try my hand at research (again). The next evaluation will
be in December/January. Depending on the progress, the funding and therewith the
project can be discontinued.

I'm also Looking for remote writing
partners.
Looking for remote writing partners (for AI alignment research)
by rmoehn — 2019-10-01T02:16:10.198Z

I'm looking for other junior researchers to form a distributed writing group for
mutual support. Please get in touch if you're interested.

Who?

One to three other people and I.

I do AI alignment research independently. I'm working on the Farlamp project
(see the GitHub repo or the project
announcement on
LessWrong).
And I live in Japan.

What?

Read and discuss one another's work.

Duration: 15 min x number of participants

Where?

Each at their desk, all connected by video call.

When?

Weekly.

Why?

Because in a group we can:

Bring up questions, responses, suggestions for each other's research.

Review each other's outlines and drafts.

Keep each other disciplined.

Support each other in other ways.

How?

If you want to join, then PM or email me (<given name of President
Nixon>.moehn@posteo.de).

We start and end our meetings on time.

We talk about the state of our work, free-form or following prompts. This
format will change based on what we find works well, how far a person is in
their research etc.

Whence?

Idea, content and rationale derived from Booth et al.: The Craft of
Research, p. 32. Here is the whole section about it:

Creating a Writing Group

A downside of academic research is its isolation. Except for group projects,
you'll read and write mostly alone. But it doesn't have to be that way. Look for
someone other than your instructor or adviser who will talk with you about your
progress, review your drafts, even pester you about how much you've written.
That might be a generous friend, but even better is another writer so that you
can comment on each other's ideas and drafts.

Best of all is a group of four or five people working on their own projects who
meet regularly to read and discuss one another's work. Early on, each meeting
should start with a summary of each person's project in this three-part
sentence: I'm working on X because I want to find out Y, so that I (and you)
can better understand Z (more about this in 3.4). As your projects advance,
develop an opening "elevator story," a short summary of your project that you
could give someone on the way to a meeting. It should include your research
question, your best guess at an answer, and the kind of evidence you expect to
use to support it. The group can then follow up with questions, responses, and
suggestions.

Don't limit your talk to just your story, however. Talk about your-readers: Why
should

...
Please Take the 2019 EA Survey!
by Peter Wildeford — 2019-09-30T21:21:17.988Z
Link: https://www.surveymonkey.co.uk/r/EAS2019LW

The 2019 Effective Altruism Survey is now live at the following link: https://www.surveymonkey.co.uk/r/EAS2019LW

The average completion time for this year’s survey is 20 minutes.

The survey will close on the 15th of October at midnight BST.

The EA Survey provides valuable information about the demographics of the EA community, how people get involved, how they donate, what causes they prioritize, their experiences of EA, and more.

This year, we worked more closely than ever with the Centre for Effective Altruism, 80,000 Hours, and other EA organizations to identify questions that would be as valuable as possible to their work. Overall, we received more than 100 requests for questions to be included from various members of the EA community, which meant making many tough decisions about what questions we could include, while keeping the survey to an appropriate length.

This year, the Centre for Effective Altruism has generously donated a prize of $1000 USD that will be awarded to a randomly selected respondent to the EA Survey, for them to donate to any of the organizations listed on EA Funds. Please note that to be eligible, you need to provide a valid e-mail address so that we can contact you.

We would like to express our gratitude to the EA Meta Fund for supporting our work.
Elimination of Bias in Introspection: Methodological Advances, Refinements, and Recommendations
by Gordon Seidoh Worley — 2019-09-30T20:23:13.139Z
Link: https://philpapers.org/rec/TRNEOB

Radek Trnka & Vit Smelik

New Ideas in Psychology 56 (2020)

Abstract

Building on past constructive criticism, the present study provides further methodological development focused on the elimination of bias that may occur during first-person observation. First, various sources of errors that may accompany introspection are distinguished based on previous critical literature. Four main errors are classified, namely attentional, attributional, conceptual, and expressional error. Furthermore, methodological recommendations for the possible elimination of these errors have been determined based on the analysis and focused excerpting of introspective scientific literature. The following groups of methodological recommendations were determined: 1) a better focusing of the subject’s attention to their mental processes, 2) providing suitable stimuli, and 3) the sharing of introspective experience between subjects. Furthermore, the potential of adjustments in introspective research designs for eliminating attentional, attributional, conceptual, and expressional error is discussed.
List of resolved confusions about IDA
by Wei Dai — 2019-09-30T20:03:10.506Z

AI Alignment is a confusing topic in general, but even compared to other alignment topics, IDA seems especially confusing. Some of it is surely just due to the nature of communicating subtle and unfinished research ideas, but other confusions can be cleared up with more specific language or additional explanations. To help people avoid some of the confusions I or others fell into in the past while trying to understand IDA (and to remind myself about them in the future), I came up with this list of past confusions that I think have mostly been resolved at this point. (However there's some chance that I'm still confused about some of these issues and just don't realize it. I've included references to the original discussions where I think the confusions were cleared up so you can judge for yourself.)

I will try to maintain this list as a public reference so please provide your own resolved confusions in the comments.

alignment = intent alignment

At some point Paul started using "alignment" refer to the top-level problem that he is trying to solve, and this problem is narrower (i.e., leaves more safety problems to be solved elsewhere) than the problem that other people were using "alignment" to describe. He eventually settled upon "intent alignment" as the formal term to describe his narrower problem, but occasionally still uses just "aligned" or "alignment" as shorthand for it. Source

short-term preferences ≠ narrow preferences

At some point Paul used "short-term preferences" and "narrow preferences" interchangeably, but no longer does (or at least no longer endorses doing so). Source

preferences = "actual" preferences (e.g., preferences-on-reflection)

When Paul talks about preferences he usually means "actual" preferences (for example the preferences someone would arrive at after having a long time to think about it while having access to helpful AI assistants, if that's a good way to find someone's "actual" preferences). He does not mean their current revealed preferences or the preferences they would state or endorse now if you were to ask them. Source

corrigibility ≠ based on short-term preferences

I had misunderstood Paul to be using "corrigibility to X" as synonymous with "based on X's short-term preferences". Actually "based on X's short-term preferences" is a way to achieve corrigibility to X, because X's short-term preferences likely includes "be corri...
Meeting the Dragon in Your Garage.
by valentinslepukhin — 2019-09-30T19:55:44.461Z

Suppose you observe the following dialogue:

A: If we do X we observe Y.

B: We tried X and we did not observe Y.

A: Apparently, you did not do X correctly.

B: Apparently, there is no Y. The reports of Y are just mistakes in the experiment.

There are many people joining both positions. Zero hypotheses, or hypothesis of no Y, of course, is simpler than hypothesis of Y that can be observed via X. No theory predicts Y, neither the existence of Y invalidates current theories.

Now, you are another explorer, and you already have all the necessary equipment to do X (and you can ask people supporting A how to do X correctly). Would you try to do it to observe Y?

Consider the following cases:

1) Y is a new astronomical object. X is the equipment (a telescope) to observe it, and where to look. Assume photography is not invented yet.

2) Y is a new generation of elementary particles. X is the design of the experiment.

3) Y is God. X is a specific way to pray if you want to understand if God exists or not (i.e., not a prayer of healing, intercessory prayer etc).

If your answer is different for these three examples, what is the difference?
Connectome-specific harmonic waves and meditation
by Gordon Seidoh Worley — 2019-09-30T18:08:45.403Z

TL;DR: meditation is a process for altering the harmonics of brainwaves to produce good brain states.

Pulling together a few threads:

The theory of connectome-specific harmonic waves suggests one thing that happens in the brain is that oscillations in neuron activity (brainwaves) interact to form harmonics (correlated brainwaves that form patterns) that may explain certain aspects of brain and thereby mental activity.
We can of course look at what happens with these harmonics in people on LSD.

Probably related: people "hallucinating" all the time

Meditation and psychedelics do similar things to the brain.
Psychedelics can cause people to perform a process we might call "annealing" by analogy to the metallurgical process ("normalizing" would also work as a metaphor, as would "quenching + tempering" in the case of bad trips/trauma with after care). I've also been using the annealing metaphor talk about what happens during meditation (no links, all in person), as have others.

Related metaphor: meditation as reinforcement learning (cf. simulating annealing)

"Good" is grounded as "minimization of prediction error". Explanation:

Neurons form hierarchical control systems.

cf. a grounding of phenomenological idealism using control systems, and its implications
cf. the hierarchy is recursive and reasserts itself at higher levels of organization

Those control systems aim to minimize prediction error via negative feedback (homeostatic) loops.
The positive signal ("good") of the control system occurs when prediction error is minimized.

By extension "bad" is when prediction error is maximized and "neutral" is when the threshold for signaling good or bad is not crossed (having a neutral signal reduces jitter).

One way to reduce prediction error is with symmetry. This means that we usually think symmetry is good.
Meditation, and anything that sets up harmonic neuronal oscillation, makes brain activity more symmetric, hence better or good.

I think we can infer a lot from these observations, but I'll leave those for another post.
[AN #66]: Decomposing robustness into capability robustness and alignment robustness
by Rohin Shah — 2019-09-30T18:00:02.887Z

Find all Alignment Newsletter resources

here

. In particular, you can

sign up

, or look through this

spreadsheet

of all summaries that have ever been in the newsletter. I'm always happy to hear feedback; you can send it to me by replying to this email.

Starting this week, we have a few new summarizers; you can always find the whole team

here

. I (Rohin) will continue to edit all of the summaries and opinions, and add some summaries and opinions of my own.

Audio version

here

(may not be up yet).

Highlights

2-D Robustness

(Vladimir Mikulik) (summarized by Matthew): Typically when we think about machine learning robustness we imagine a scalar quantity representing how well a system performs when it is taken off its training distribution. When considering

mesa optimization

(

AN #58

), it is natural to instead decompose robustness into two variables: robust capabilities and robust alignment. When given an environment that does not perfectly resemble its training environment, a mesa optimizer could be dangerous by competently pursuing a mesa objective that is different from the loss function used during training. This combination of robust capabilities without robust alignment is an example of a malign failure, the most worrisome outcome of creating a mesa optimizer.

Matthew's opinion: Decomposing robustness in this way helps me distinguish misaligned mesa optimization from the more general problem of machine learning robustness. I think it's important for researchers to understand this distinction because it is critical for understanding why a failure to solve the robustness problem could plausibly result in a catastrophe rather than merely a benign capabilities failure.

Rohin's opinion: I strongly agree with this distinction, and in fact when I think about the problem of mesa optimization, I prefer to only think about models whose capabilities are robust but whose objective or goal is not, rather than considering the internals of the model and whether or not it is performing search, which seems like a much hairier question.

Technical AI alignment

Iterated amplification

Finding Generalizable Evidence by Learning to Convince Q&A Models

(Ethan Perez et al) (summarized by Asya): This paper tries to improve performance on multiple-choice questions about text passages using a technique similar to

AI safety via debate

(

AN #5

). The set-up consists of a judge...
Thermal Mass Thermos
by jefftk — 2019-09-30T13:10:01.637Z

A thermos does a great job with liquids: the insulation means you can
pour in hot soup in the morning, and enjoy hot soup at lunch. But it
doesn't do nearly as good a job with solids: fill one with rice and it
quickly falls into the "danger zone".

One recommendation you'll see is that you should fill the thermos with
hot water, let it sit, and then swap the water for the food. This
does help, but because the thermos doesn't have that much thermal mass
it only helps some. What if we put a rock in the thermos to hold
heat, and preheat it along with the rest of the thermos? How much
does this help? Does this help enough?

The general rule is that hot food shouldn't be below 140F for more
than two hours, because substantial bacteria can grow, and the closer
it is to 100F the worse it is. My goal here is to be able to send hot
dry food with Lily's lunch, and I pack it four hours before she gets
to eat it, so the question is: what's the temperature two hours in,
which is two hours before lunch time? If that's above 140F we're ok,
if not we should figure something else out.

I ran an experiment on one cup of cooked rice, the kind
of dry food you're not normally supposed to put in a thermos. I
heated it in the microwave until it was steaming hot (it measured
~188F with an infrared thermometer):

I tested this in three configurations:

T1: a thermos that has been pre-heated with boiling water.

T2: same as T1, but with the addition of a 4.3oz (122g) rock
with volume 1.5oz (45ml).

T3: same as T2, but the thermos has been preheated with two
rounds of boiling water.

After two hours I measured the temperature of the rice:

T1: 108F

T2: 136F (143F in the middle)

T3: 142F (148F in the middle)

The rock is clearly helping a lot, though it looks I should do the
full T3 treatment with two rounds of boiling water in this particular
thermos.

This works, but it is kind of hacky: rocks are odd shaped, hard to
clean, rattle around, hide food, etc. How else could we solve this?
Ideas:

Use water instead of a rock, because water has excellent heat
capacity by volume and very good heat capacity by weight. Have two
sections: an outer one you pour boiling water into, and an inner one
for the dry food. Keep heat from leaving the overall enclosure, but
allow heat to move freely between the two sections.

Something microwave-safe which u

...
What's your favorite notetaking system?
by Elizabeth — 2019-09-30T06:18:18.024Z

Abram recently wrote up the Zettelkasten system for notetaking while doing research. Do you have an opinion on an alternative system, and if so, what is it?

Things you might optionally include but should not let the lack of preclude answering:

How the system works
Plusses and minuses of the system
Who or what problems you think it works especially well and poorly for
Comparisons to other notetaking systems.
Your final judgement of the system.
Noticing Frame Differences
by Raemon — 2019-09-30T01:24:20.435Z

Previously: Keeping Beliefs Cruxy

When disagreements persist despite lengthy good-faith communication, it may not just be about factual disagreements – it could be due to people operating in entirely different frames — different ways of seeing, thinking and/or communicating.

If you can’t notice when this is happening, or you don’t have the skills to navigate it, you may waste a lot of time.

Examples of Broad Frames

Gears-oriented Frames

Bob and Alice’s conversation is about cause and effect. Neither of them are planning to take direct actions based on their conversation, they’re each just interested in understanding a particular domain better.

Bob has a model of the domain that includes gears A, B, C and D. Alice has a model that includes gears C, D and F. They’re able to exchange information, and their information is compatible,and they each end up with a shared model of how something works.

There are other ways this could have gone. Ben Pace covered some of them in a

sketch of good communication

:

Maybe they discover their models don’t fit, and one of them is wrong
Maybe combining their models results in a surprising, counterintuitive outcome that takes them awhile to accept.
Maybe they fail to integrate their models, because they were working at different levels of abstraction and didn’t realize it.

Sometimes they might fall into subtler traps.

Maybe the thing Alice is calling “Gear C” is actually different from Bob’s “Gear C”. It turns out that they were using the same words to mean different things, and even though they’d

both read blogposts warning them about that

they didn’t notice.

So Bob tries to slot Alice’s gear F into his gear C and it doesn’t fit. If he doesn’t already have reason to trust Alice’s epistemics, he may conclude Alice is crazy (instead of them referring to subtly different concepts).

This may cause confusion and distrust.

But, the point of this blogpost is that Alice and Bob have it easy.

They’re actually trying to have the same conversation. They’re both trying to exchange explicit models of cause-and-effect, and come away with a clearer understanding of the world through a reductionist lens.

There are many other frames for a conversation though.

Feelings-Oriented Frames

Clark and Dwight are exploring how they feel and relate to each other.

The focus of the conversation might be navigating their particular relationship, or helping Clark understand why ...
The Thinking Ladder - Wait But Why
by Liron — 2019-09-29T18:51:00.409Z
Link: https://waitbutwhy.com/2019/09/thinking-ladder.html

The Thinking Ladder is the latest post in a huge ongoing Wait But Why series called The Story of Us.

Our beliefs make up our perception of reality, they drive our behavior, and they shape our life stories. History happened the way it did because of what people believed in the past, and what we believe today will write the story of our future.

So it seems like an important question to ask: Why do we actually come to believe the things we end up believing?

It's a great introduction to many core rationality and LessWrong concepts which is fun and accessible to a general audience.

I recommend the whole series and really all of Wait But Why.