trentmkelly/LessWrong-43k huggingface dataset:
D&D.SCP: Anomalous Acquisitions Evaluation & Ruleset
by aphyer — 2022-02-22T18:19:22.408Z

This is a follow-up to last week's D&D.Sci scenario: if you intend to play that, and haven't done so yet, you should do so now before spoiling yourself.

A web interactive to test your solution is available here. This also presents the thrilling* conclusion to the story, with a wide variety** of skillfully-written* endings depending on your actions***!

*Maybe. **Five. ***And on your luck.

Full generation code is also available here if you are interested, or you can read on and find out.

RULESET

An SCP object has a Value and a Danger.

When an object is successfully stolen, Marshall Carter & Dark security teams attempt to reduce its costs, while MC&D sales teams attempt to maximize revenue gained.

Revenue gained starts at a baseline of $2MM * (Value^2). This is multiplied by (3+1d6)/6 depending on performance of sales team.

Costs start at a baseline of $2MM * (Danger^2). This is multiplied by (9-1d6)/6 depending on performance of security team.

Profit is equal to Revenue minus Costs.

Overall, an object whose Value equals its Danger will average a slight profit. Objects with Value > Danger will tend to generate profits, while those with Danger > Value will tend to generate losses.

The squaring of Value and Danger means that e.g. Value 6 Danger 5 is much more profitable than Value 1 Danger 0 (with a baseline of $72 MM revenue and $50 MM costs being much better than a baseline of $2 MM revenue and $0 costs).

You have very good insight into the Danger of an object via the SCP Foundation's classifications:

Any object with Danger 5+ is classified as Keter.
Any object with Danger 3-4 is classified as Euclid.
Any object with Danger 0-2 is classified as Safe.

There was low-hanging-fruit available in preferentially targeting Safe objects (though this was not always optimal).

The more difficult problem was figuring out how to use tags to identify objects with high Value.

However, there were also two Anomalies hidden in the dataset - SCPs that broke the usual rules. If you fed the data into a machine learning program without sanity-checking it first (cough), these may have shot you in the foot.

ANOMALIES

Two SCPs in the dataset were anomalous - they did not operate under the same rules as the rest of the data, and instead existed as horrible tricks to sabotage people who did not sanity-check their da...
What's the status of TDCS for improving intelligence?
by FinalFormal2 — 2022-02-22T17:27:00.687Z

I recently heard about transcranial direct current stimulation as a technology for improving memory and attention, and I thought it sounded like a great area to attempt some munchkin activity.

However, it seems like the science isn't very clear.

Does anyone have a more informed perspective on the efficacy behind it?
Learning By Writing
by HoldenKarnofsky — 2022-02-22T15:50:19.452Z

I have very detailed opinions on lots of topics. I sometimes get asked how I do this, which might just be people making fun of me, but I choose to interpret it as a real question, and I’m going to sketch an answer here.

You can think of this as a sort of sequel to Minimal-Trust Investigations. That piece talked about how investigating things in depth can be valuable; this piece will try to give a sense of how to get an in-depth investigation off the ground, going from “I’ve never heard of this topic before” to “Let me tell you all my thoughts on that.”

The rough basic idea is that I organize my learning around writing rather than reading. This doesn’t mean I don’t read - just that the reading is always in service of the writing.

Here’s an outline:

Step 1

Pick a topic

Step 2

Read and/or discuss with others (a bit)

Step 3

Explain and defend my current, incredibly premature hypothesis, in writing (or conversation)

Step 4

Find and list weaknesses in my case

Step 5

Pick a subquestion and do more reading/discussing

Step 6

Revise my claim / switch sides

Step 7

Repeat steps 3-6 a bunch

Step 8

Get feedback on a draft from others, and use this to keep repeating steps 3-6

The “traditionally” hard parts of this process are steps 4 and 6: spotting weaknesses in arguments, trying to resist the temptation to “stick to my guns” when my original hypothesis isn’t looking so good, etc.

But step 3 is a different kind of challenge: trying to “always have a hypothesis” and re-articulating it whenever it changes. By doing this, I try to continually focus my reading on the goal of forming a bottom-line view, rather than just “gathering information.” I think this makes my investigations more focused and directed, and the results easier to retain. I consider this approach to be probably the single biggest difference-maker between "reading a ton about lots of things, but retaining little" and "efficiently developing a set of views on key topics and retaining the reasoning behind them."

Below I'll give more detail on each step, then some brief notes (to be expanded on later) on why this process is challenging.

My process for learning by writing

Step 1: pick a topic. First, I decide what I ...
Harms and possibilities of schooling
by TsviBT — 2022-02-22T07:48:09.542Z

To explore better possibilities for nurturing new minds, and to care about the problem in the first place, it helps to remember what's wrong with what we do to new minds. John Taylor Gatto speaks about this from experience and insight:
Seven Lessons Taught in School, 1991

(If you're going to read the following, at least read the seven lessons (part I) of that essay.)

Here's another list of harms caused by schooling.

1. You aren't a mind, and don't bother trying to behave like one.

Children naturally attend to things until they're done with them:

From Maria Montessori, My System of Education, 1915:
IPFS pdf link

A little girl, about three years of age, was deeply absorbed in the work of placing wooden blocks and cylinders in a frame for that purpose. The expression of her face was that of such intense attention, that it was almost a revelation to me. Never before had I seen a child look with such "fixedness" upon an object, and my conviction about the instability of attention which goes incessantly from one thing to another, a fact which is so characteristic in little children, made the phenomenon the more remarkable to me.

I watched the child without interrupting her, and counted how many times she would do her work over and over. It seemed that she was never going to stop. As I saw that it would take a very long time, I took the little armchair on which she was sitting and placed child and chair on the big table. Hastily she put the frame across the chair, gathered blocks and cylinders in her lap, and continued her work undisturbed. I invited the other children to sing, but the little girl went on with her work and continued even after the singing had ceased. I counted forty-four different exercises which she made, and when she finally stopped, and did so absolutely independently from an exterior cause that could disturb her, she looked around with an expression of great satisfaction, as if she were awakening from a deep and restful sleep.

The impression I received from the observation was that of a discovery. The same phenomenon became very common among those children, and it was noticed in every school in every country where my system was introduced; therefore it can be considered as a constant reaction which takes place in connection with certain exterior conditions that can be well established. Each time a similar "polarization" of the attention occur

...
Butterfly Ideas
by Elizabeth — 2022-02-22T07:40:08.072Z

Or “How I got my hyperanalytical friends to chill out and vibe on ideas for 5 minutes before testing them to destruction”

Sometimes talking with my friends is like intellectual combat, which is great. I am glad I have such strong cognitive warriors on my side. But not all ideas are ready for intellectual combat. If I don’t get my friend on board with this, some of them will crush an idea before it gets a chance to develop, which feels awful and can kill off promising avenues of investigation. It’s like showing a beautiful, fragile butterfly to your friend to demonstrate the power of flight, only to have them grab it and crush it in their hands, then point to the mangled corpse as proof butterflies not only don’t fly, but can’t fly, look how busted their wings are.

You know who you are

When I’m stuck in a conversation like that, it has been really helpful to explicitly label things as butterfly ideas. This has two purposes. First, it’s a shorthand for labeling what I want (nurturance and encouragement). Second, it explicitly labels the idea as not ready for prime time in ways that make it less threatening to my friends. They can support the exploration of my idea without worrying that support of exploration conveys agreement, or agreement conveys a commitment to act.

This is important because very few ideas start out ready for the rigors of combat. If they’re not given a sheltered period, they will die before they become useful. This cuts us off from a lot of goodness in the world. Examples:

A start-up I used to work for had a keyword that meant “I have a vague worried feeling I want to discuss without justifying”. This let people bring up concerns before they had an ironclad case for them and made statements that could otherwise have felt like intense criticism feel more like information sharing (they’re not asserting this will definitely fail, they’re asserting they have a feeling that might lead to some questions). This in turn meant that problems got brought up and addressed earlier, including problems in the classes “this is definitely gonna fail and we need to make major changes” and “this excellent idea but Bob is missing the information that would help him understand why”.

This keyword was “FUD (fear, uncertainty, doubt)”. It is used in exactly the opposite way in cryptocurrency circles, where it means “you are trying to increase our anxiety wi

...
ELK Proposal: Thinking Via A Human Imitator
by TurnTrout — 2022-02-22T01:52:41.794Z

I want to use the AI’s intelligence to figure out how to translate into the human ontology. The hope is to route a smart entity's performance through a dumb entity's understanding and thereby get the smart entity to solve interpretability-by-the-dumb-entity. While my proposed current architecture overcomes a class of counterexamples in a manner which I find elegant, it is still broken by several plausible counterexamples.

A previous version of this post linked to a Google doc.

Previously, on ELK

This section summarizes ELK. Skip if you’re already familiar.

In the mythos of

Eliciting Latent Knowledge

, there exists a forbidden vault locking away an enormous diamond:

In order to fend off thieves, this vault brims with controllable traps and tricks. We've trained a really smart AI to take in a video feed, and consider an action sequence (e.g. "open the trapdoor"), and then predict what the vault camera will show if those actions are taken. At the end of the day, we just want to know whether the diamond is safe. Looking at the camera is not always enough:

But the AI is really smart. It knows what happened to the diamond. Can we elicit its latent knowledge?

The original ELK document tries to figure out how to train a "reporter" to figure out what's going on in a really smart AI predictor. The intended reporter would directly translate from the AI's ontology to the human’s ontology, and then answer whether the diamond is there or not. In the original ELK document, the reporter is wired as follows:

The training questions/answers are generated by a human considering a range of simple, human-comprehensible situations (i.e. (video, action sequence) tuples), querying their model of what happened, and answering e.g. whether the diamond is safe or not:

We never want there to be a situation where the AI predictor (the "figure out what's going on" module) unambiguously knows that the diamond was taken, but the reporter says "the diamond is fine." More precisely, we don't want such a configuration to plausibly minimize training loss.

The direct translator minimizes training loss because it answers all questions correctly on the training distribution. So far, so good. However, the human simulator reporter also minimizes training loss, by extracting the predicted video completion from the AI predictor, and then conditioning the human label...
Reward Good Bets That Had Bad Outcomes
by Neel Nanda — 2022-02-22T00:25:56.818Z

Introduction

I am a very anxious person. One of the most damaging ways this manifests is that I am pretty risk-averse and afraid of failure. In situations of uncertainty, I often want to freeze up, and know I’ll feel safer doing nothing.

This is a really bad problem! If I let this dictate my life, I lose a ton of value. In particular, there are a lot of areas in my life that are hits-based, where the best way to be successful is to persevere through many failures and

seek the upside risk

of things occasionally going super well. I want to be someone who can be a great researcher, find really awesome friends, and

generally be ambitious

about all areas of my life going well. And to achieve this, it is important that I be the kind of person who can take actions with high expected value, and persevere through failures without feeling paralysed. The key thing going wrong here is that I beat myself up over bad outcomes even when I had no way of knowing it wouldn’t work out, given the information at the time. And anxiety gives me a negative prior and causes suboptimal outcomes to stick in my mind. Failures feel painful in a way that missed opportunities do not.

My solution to this is to think in bets, not outcomes. To clearly notice all of the good bets that I take, the actions that I endorse given what I knew at the time, and to be happy about each of those. And to think of my life in terms of this, rather than the concrete outcomes of the bet, and whether that was a success or failure. At the end of the day, the only thing I can control is the bets that I make, and the policies I follow, and there will always be uncertainty on the outcomes. And if I make a good bet with a bad outcome, I should be happy about this, not sad! I refuse to let my negative emotions be tied to things fundamentally outside my control.

My Underlying Model

I first formed this view when I did a trading internship a few years ago. In settings like financial trading or poker, the fundamental skill is about engaging well with uncertainty, and getting past anxieties is a key part of that! And noticing all of the expected value I was missing when I froze up did a lot for helping me notice this failure mode and learning how to solve it. And though the lessons generalise, real life is often a much harder learning environment than these settings - I make fewer bets and so get fewer data points, and it’s...
The Art and Science of Intuition Pumping
by adamShimi — 2022-02-22T00:18:21.535Z

Epistemic status:

exploratory

What is “intuition pumping”? I had seen the term used on LW, probably used it myself, without wondering that much what “intuition pumps” even are. Pure pedagogical illustrations? Mere rhetorical devices? Subtle epistemic tools? My intuition about intuitions favored the latter, but I also knew well how intuition can mislead.

To make sense of all of this, I went back to the source, the inventor and master wielder of intuition pumps: Daniel Dennett. Not only did he coin the term and use it abundantly, but his “

Intuition Pumps and Other Tools for Thinking

” teaches how to use intuition pumps his way.

Spoiler alert: they can and should be used as thinking tools to find out the essential parts of the problem. But that requires a meta-level analysis that Dennett calls “turning the knobs”— checking the robustness of the intuition to various changes in the story.

Intuition pumps’ origin story

Everyone I’ve read on intuition pumps, Dennett included, points to his response to Searle’s

Chinese Room

as the birth of the term.

(

The milk of human intentionality

, Dennett, 1980)

Searle's form of argument is a familiar one to philosophers: he has constructed what one might call an intuition pump, a device for provoking a family of intuitions by producing variations on a basic thought experiment. An intuition pump is not, typically, an engine of discovery, but a persuader or pedagogical tool — a way of getting people to see things your way once you've seen the truth, as Searle thinks he has. I would be the last to disparage the use of intuition pumps — I love to use them myself — but they can be abused. In this instance I think Searle relies almost entirely on ill-gotten gains: favorable intuitions generated by misleadingly presented thought experiments.

What’s interesting here is that Dennett seems to have expanded his views on the usefulness of intuition pumps: here he explicitly criticizes the use of intuition pumps for discovery and clarification, but in Intuition Pumps and Other Tools for Thinking he clearly count them as thinking tools that have positive and research-relevant uses. He also gives a bit more of a definition there.

(

Intuition Pumps and Other Tools for Thinking

, Dennett, 2013)

Other thought experiments are less rigorous but often just as effective: little stories designed to provoke a heartfelt, table-thumping intu

...
Do any AI alignment orgs hire remotely?
by RobertM — 2022-02-21T22:33:04.765Z

tl;dr: qualified software engineer considering what their next job might be; now thinking about direct work as a serious option.

Previous plan was something like:

SELECT * FROM big_tech_co
WHERE location = 'remote'
ORDER BY team_fit, salary

For a variety of reasons, I'm not a huge fan of this plan anymore.

New plan:

Check the job pages of all the AI alignment orgs I know
Check 80000 Hours jobs board in case I missed something
???
Post question on LW

I didn't find anything looking at the job pages of the AI alignment orgs that I'm familiar with, and 80000 Hours didn't bring up anything that fit the bill either, so here we are.

Me:

mission-aligned, long-time member of the community (since ~2013) and meetup organizer (since 2017)
fairly strong software engineer, mostly backend-focused but have recently picked up enough React to be able to meaningfully contribute to an existing project on the front-end (no design skills to speak of, yet). Can also rapidly onboard myself to an unfamiliar codebase. Legible artifacts demonstrating these claims beyond a resume include:

a non-trivial contribution to major open-source library
going from zero to one with a SaaS product on the side while working full-time

happy to travel something like 10% of the time, especially to the Bay, to integrate professionally & socially
happy to do something like a part-time work trial (outside of my core working hours, but including weekends), and willing to take a few days off to do this in-person if the fit seems good
in general, happy to do some non-standard things that might not be expected of me at a typical tech job, be agent-y, etc., particularly to compensate for the downsides to an organization of my working remotely

You:

an organization that works either directly on AI alignment, or a "meta" org that e.g. better enables others to work on AI alignment
willing to hire someone on these (or similar) terms!
most everything else I'm looking for seems pretty strongly correlated with that, and I'm aware that there aren't exactly a wealth of options to filter on

Does anyone know of any orgs that I might have missed?

Lightcone & Redwood don't seem to be hiring remotely
Alignment AI doesn't specify whether they're hiring remotely (but since they're in the UK, you'd imagine they'd say if they were)
MIRI & ARC aren't hiring engineers

Most of the other orgs I'm familiar with seem to be doing differently-targe...
Convoy Crackdown
by Zvi — 2022-02-21T22:10:02.289Z

The convoy has been dispersed. Yet Tyler Cowen was clearly correct that this will be a very important event. The story has barely begun.

[Writer’s Note: This post violates some of the usual lines that I set for myself regarding politics, because I do not know how to communicate the situation without doing so, nor does it seem like a situation one can safely ignore. I did my best to keep it to a minimum. As before, I will be staying out of the comments, and ask that related discussions be confined to related posts and conducted to minimize the political component, but I acknowledge that this may not be fully possible in this case and will use discretion.]

Previously in Convoy and Convoy Continued, the puzzle was what was happening and what might happen next in response. Authorities were uncertain what to do. The situation was somewhat out of control. Violence was a distinct possibility, or at least the fear of such violence was informing decisions and rhetoric. There were big disputes over the essential nature of the protest, the protesters, what they wanted and why they were there. As far as I could tell the protests were unpopular but that too was disputed. Every edit seemed like it reflected more information from more sources, but more often that not that only meant increased uncertainty.

Most of that is now resolved. The convoy has been broken up by police without incident and with zero violence (or at least, zero violence that didn’t involve police acting like police usually act), only minor threats to permanently confiscate people’s dogs, in exactly the most basic and obvious way. A few people are still hanging around, but they no longer threaten public order. The three leaders have been arrested. When one engages in civil disobedience and makes oneself intentionally obstructing traffic for weeks, one eventually gets arrested and faces the consequences of being briefly in jail and having a record, and perhaps more if one committed more serious crimes and the state can convince a jury of that. That’s how it’s supposed to work. There were less than 200 arrests and they towed less than 50 trucks.

There’s just one little problem.

Before breaking up the protest, as I noted last time, Prime Minister Trudeau1 invoked the Emergencies Act.

The Emergencies Act may have been a little bit about the need to commandeer towing capacity, but we can now be very certai...