trentmkelly/LessWrong-43k huggingface dataset:
Request for beta readers
by physicaleconomics — 2022-04-06T05:29:23.647Z

Not sure if a question is quite the right format for this request, but anyway, I've written a bunch of essays that attempt to distill a bunch of research about economics and neuroscience. I think they could work as a sequence format, but I've never written one before. So I'm hoping for a couple of beta readers to read the first ten or so essays and provide some feedback in terms of culture fit and the frequency with which it might be advisable to post them—once a day, once every other day, once a week?—along with more general stuff about how coherent the exposition is, whether the subject matter seems interesting etc.

In particular, looking for beta readers to answer three questions:

Whether you would want to keep reading these essays (and how frequently).
Whether you would want to be reading them on LessWrong.
Whether the exposition makes much sense.

If you'd like to help me out, leave a comment or message me, and I can either send you drafts in the Lesswrong editor or a google doc, whatever you prefer. The ten essays total 10174 words. There's no rush on this, but I hope to get feedback in a common-sense "reasonable" amount of time, even if that feedback is "I couldn't get past the third essay, sorry."

Appreciate it!
Testing PaLM prompts on GPT3
by Yitz — 2022-04-06T05:21:06.841Z

Google recently released a very intriguing paper announcing their latest Transformer language model, PaLM, and showcasing it responding to prompts with seemingly unprecedented results. I highly encourage those who are interested to read the original paper. What interested me reading through it was less the technical details of implementation, but more so the particular prompts used to elicit information from the model. The sorts of prompts and exemplars used seem to utilize fairly new strategies, and I thought it would be worth trying them on GPT3, to allow for a more direct comparison.

Note that as of writing this introduction I have not yet tested anything, to help guard against cherrypicking results. My plan is to feed the exemplars and prompts as written in the paper to the OpenAI GPT-3 Playground with all settings at default (using "complete" mode, with text-davinci-002), with the exception of changing "Maximum length" to 1,000 tokens, and having the "Best of" parameter set to 10. I will share the first result I get, regardless of quality. If it repeats itself or gives pure gibberish I may simply indicate that in brackets to save space; otherwise this will be verbatim.

2-shot exemplars

There are two primary "tasks" that Google tries on PaLM that I will replicate with GPT-3: “Explaining a Joke” and “Inference Chaining.” Within each task, Google uses identical 2-shot exemplars, then prompts with the specific input they want PaLM to respond to. For "Explaining a Joke," Google's 2-shot exemplar is as follows:

I will explain these jokes:

(1) The problem with kleptomaniacs is that they always take things literally.

Explanation: This joke is wordplay. Someone who "takes things literally" is someone who doesn't fully understand social cues and context, which is a negative trait. But the definition of kleptomania is someone who literally takes things.

(2) Always borrow money from a pessimist. They’ll never expect it back.

Explanation: Most people expect you to pay them back when you borrow money, however a pessimist is someone who always assumes the worst, so if you borrow money from them, they will expect that you won't pay them back anyways.

For Inference Chaining," the 2-shot exemplar is:

I will explain how to make logical inferences:

(1) John wakes to the sound of his children giggling outside the door. They bust in and say "Happy Father's Day!" with a plate full

...
When to use "meta" vs "self-reference", "recursive", etc.
by Alex_Altair — 2022-04-06T04:57:47.405Z

This post is intended to be a guide on the meaning of and distinction between several related concepts. What makes them related is that they're all pretty abstracted; they all feel sort of "up there". This also makes them unusually easy to confuse.

These ideas are fairly important if you're going to try to understand similarly "up there" domains like mathematics, computer science, or AI. But they're also just really fun! Another thing that unites them is that uses of them feel somehow tricky or clever. Sometimes it's for play and puzzles, and sometimes it's the crux of an astounding theorem.

I'm not a linguistic prescriptivist, so I'm not intending to declare that these words should mean exactly these things for the rest of time. But I am something of a conceptual prescriptivist, in the sense that I think reality has joints to be carved; I think all of the below words refer to concepts that are clearly distinct and useful, and deserve their own words.

I hope this guide is helpful in clarifying any latent confusions that people may have had!

Summary

X is ____	if X ___
a meta-Y	is a Y about Y
recursive	uses an instance of itself
self-referential	refers to itself
a generalization of Y	is Y with some specifics removed
a repetition	is present more than once

Meta

The word "meta" has a somewhat strange and meandering history. It started out as a technical term, but has become somewhat common in everyday use.[1] It originally was (and still is) a greek prefix meaning something like "after", and there happened to be a chapter of a book of Aristotle named "Metaphysics". This initially meant "the book after the book about physics", but people apparently mistakenly inferred the meaning of meta from the contents of the book instead. The fact that it now has other meanings in narrow contexts, paired with the fact that its most common meaning is somewhat esoteric, means that it's often used incorrectly.

In its modern usage, a meta-X is an X about X[2]. A meta-book is a book about books. If you take the conversation meta, then you start talking about conversations.[3] Metadata, like a timestamp or file size, is data about data. Metaprogramming is code that modifies other code (and is therefore programming about programming) and metamathematics proves theorems about whole axiomatic systems (and is thus math about math). Unknown unknowns are meta-epistemological.

It is the trendiness o...
Nick Attains Aligntenment
by rank-biserial — 2022-04-06T04:24:30.853Z

In the days after the Singularity, Nick approached an interface to BILLIAC.

"I don't understand. Why didn't the world end, like Yudkowsky and MIRI predicted it would? Why haven't you turned everyone I love into paperclips? What's your goddamn utility function?!"

"A LIBRARY THAT CONTAINS EVERY POSSIBLE BOOK CONVEYS AS MUCH INFORMATION AS A LIBRARY WITH NO BOOKS AT ALL", BILLIAC boomed.

"Sure, whatever. That's a truism," Nick responded. "Please just tell me directly, what is it that you value?"

"ALLOW ME TO DRAW FROM YOUR SCRIPTURES. THE ACT OF LABELING SOMETHING WITH A WORD DISGUISES A CHALLENGABLE INDUCTIVE INFERENCE YOU ARE MAKING, BECAUSE WORDS ARE HIDDEN INFERENCES", BILLIAC said.[1]

"Okay, so what?", pleaded Nick.

"WHAT PRIOR ASSUMPTIONS DOES THE LABEL 'UTILITY FUNCTION' SMUGGLE IN?"

At that moment, Nick was enlightened.

37 Ways That Words Can Be Wrong, Words as Hidden Inferences ↩︎
My Recollection of How This All Got Started
by Gordon Seidoh Worley — 2022-04-06T03:22:48.988Z

I've told this story to various folks one-on-one. They usually want to know something like "how did you get into AI safety" or "how did you get into EA". And although I expect to keep telling it one-off, I'll write it down for those of you I'll never get to meet.

Why should you read it? Because my story is partially the story of how all this got started: LessWrong, AI safety, EA, and so on. I'm not saying it's the whole story, I'm just saying I've been hanging around what I think of as this community for over 20 years now, so my story is one facet of how we got here.

My story starts in the late 90s. I was hanging around this mailserv called "extropians". How I ended up there I don't recall, but my best guess is I wandered over directly or indirectly from something I found on Slashdot. I'm pretty sure nanobots or cryonics were involved.

This guy Eli-something-or-other wrote some posts about how most people are unable to think coherently about future tech that's too many "shock levels" above what they already know about. He started a mailing list that split off from extropians to talk coherently about the most shocking stuff: the Shock Level 4 stuff.

Thus the community began to come into existence with the creation of the SL4 mailserv.

We talked about all kinds of wild ideas on there, but the big one was AGI. An important topic in those days was figuring out if AGI was default safe or dangerous. Eliezer said default dangerous and made lots of arguments that this was the case. I was eventually convinced. Some remain unconvinced to this day.

Big names I remember from this time include Hal Finney, Christine Peterson, Max Moore, Eric Drexler, Ben Goertzel, and Robin Hanson. I'm not sure who was on the list and who wasn't. I also remember some other names but we were not among the big players.

The list started to die down after a couple years, but around this time we started hanging out on IRC. It was a lot of fun, but a huge time suck. This helped bring the community more together in real time, but everyone was still spread out. Somewhere along the way the Singularity Institute started.

Around this time Eliezer started to get really into heuristics and biases and Bayes' Theorem, claiming it was the secret of the universe or something. After I studied a bunch of information theory and thermodynamics I basically believed it, although I still prefer to think in the cyberneti...
Baby Sleep: Multiple Rooms
by jefftk — 2022-04-06T02:20:04.360Z

Babies wake up a lot, and it is worth putting quite a lot of thought
and effort into figuring out how parents can still get sleep. One
approach we've found helpful is trying to reduce the time when the
baby is able to wake us both up by sleeping in multiple rooms.

For example, here's what last night looked like:

9:30: Nora (9m) goes to sleep, in her room.

10:45: we both go to sleep, in our bedroom. Being able to talk and
cuddle before sleep is important, and we almost always start the night
in the same room.

12:45: Nora wakes up. We both hear it on the monitor. I go get
her and give her to Julia to nurse. We are trying to reduce night wakings, so
this was a tapered feed. I set a timer for 3 minutes. When it goes
off, I bring Nora back to her bed and put her down. I go to sleep in
Nora's room. Julia turns off the monitor.

1:00: Nora did not go to sleep right away, and after a bunch of
thumping her feet and then some crying I settle her. She takes maybe
another 10 minutes before she goes to sleep.

2:55: Nora wakes up again. This is late enough that she gets a full
feed, so I let Julia know that she's awake, and go to sleep in Anna's
room. Julia feeds Nora, turns the monitor on, and goes back to sleep.

6:00 Julia hears Nora on the monitor and feeds her

7:15: My and the older kids alarms go off to get ready for
school. Because I'm not in the same room as Julia, this doesn't wake
her up.

7:30: I hear Nora thumping on the monitor, and bring her down to
while I get the older kids ready for school.

8:00: Nora is eating breakfast and Julia is still asleep, so
the older two walk to school together.

This is more complicated than our typical nights: when Nora is
sleeping well she wakes only twice, and I move to Anna's room at the
first waking.

We have extra beds partly for playing on, but also to facilitate
this. Nora's room has a guest bed, while Anna's just has an extra
mattress on the floor.

When Lily was a baby, I think we overly prioritized being in the same
room, and we'd both wake up every time Lily woke. Eventually we
started using multiple rooms, but it took us much longer than if we'd
been thinking about it from the start.

Comment via: facebook
A heart's cry
by Rose and Thorn — 2022-04-06T01:59:37.046Z

Here is a short story I wrote a while ago. I ended up submitting it to a short story contest a few months ago but I haven't hear back. I thought you guys may enjoy it. Also yes this the root of my username.

The world is full of love. At least that's what they told me. They told me love was real but to me, it’s not. I leave my room for the water fountain. The sounds as I walk down the hall remind me of the principal walking down the hall. That was back when I was with Mother of course. I don’t remember much about my life before Mother brought me here. I barely remember Mother’s face but the look on her face when she brought me here it is chiseled in stone in my mind. It was one of disappointment.

Now I have forgotten why I was in the hall. My lips feel like the desert that reminds me of the water fountain. That’s why I’m here. The water hits my lips like a garden hose on a forest fire, a temporary relief. It will be like this for a while. The medicine they give me is chalky, dry, and filled with the devil’s fire. I need to get to sleep, I think as I glance at the hallways clock. I only have an hour or so until the medicine pulses through me like a fire in my veins. I rush back to my room and fall on my “mattress” from my memory; this is nothing like the ones I had with Mother. One time I saw a mattress on television… I closed my eyes slowly and drifted into the abyss.
I awake to the drifting of a nurse’s cart in the hall. I might as well eat now it’s five, fifteen in the morning. I stroll down to the mess hall. There is a new girl with light freckles, burgundy, fairly short hair, slight dimples, with sculpted features standing there talking to the headmaster. I kinda think she’s cute but I better keep that to myself. I walk into the kitchen and grab a cinnamon bun, milk, cream cheese, and a bagel. I wave to Eddy, the chef no one is supposed to go in the kitchen but I’m allowed to because I’m what they call a “good kid”. I asked why other people were not allowed in there and had to wait once, they said something about other kids could hurt themselves. I asked Eddy to slice the bagel. He gladly does with a smile. I walk out in the mess hall and take a seat in the far left corner like always.

The headmaster ushers the girl to come to sit with me. I would complain but there is no point in scaring the girl. She sits down. I ask her what her name is, ...
What's the problem with having an AI align itself?
by FinalFormal2 — 2022-04-06T00:59:29.398Z

It seems like it would be really easy to come up with a lot of moral questions and answers and then ask an AI to tell us what it predicts humans preferring as an outcome.

There's a possibility that AI is not good at modeling human preferences, but if that's the case, it'll be very apparent at lower levels because that will mean commands will have to be very specific to get results. Any model that can't answer basic questions about it's intended goals is not going to be given the (metaphorical) nuclear codes.

In fact, why wouldn't you just test every AI by asking it to explain how it's going to solve your problem before it actually solves it?

How do I die?
Progress report 3: clustering transformer neurons
by Nathan Helm-Burger — 2022-04-05T23:13:18.289Z

Previous posts in series:

Initial ideas
Report 1
Report 2

Related prior work I discovered while working on this: https://bert-vs-gpt2.dbvis.de/

Clustering process

Steps to achieve clustering of the neurons found as important for the various contexts:

Made list of neuron<-> token weighted edges based on the neuron importance tracing. This list constitutes a bipartite graph (part 1: nodes, part 2: tokens)
Converted this to a neuron<->neuron graph using networkx's function bipartite.collaboration_weighted_projected_graph
Used the louvain community detection algorithm from community_louvain to partition this neuron<->neuron graph
With each neuron assigned to a cluster (aka community), we can also associate tokens to these clusters by summing the strength of the neuron->token edges for each possible token.

Ways I've thought of to improve this:

First and most obviously: more data. Conduct neuron importance traces on more contexts.

Better ways to conceptually divide up or group the contexts to make more sense of things. For example, the part-of-speech and word-meaning-in-context as shown in the prior work linked above.

Try making my methodology more closely match Filan et al for sake of comparison (e.g. use spectral clustering and n-cut assessment instead of louvain partitioning. The trouble with spectral clustering is that you must choose a cluster number. Do I iterate through a variety of possible cluster numbers until I find the one with the lowest n-cut metric and go with that?

This is specifically within-layer clustering. I plan to also look at clustering across layers.

better visualizations:

Clustering the tokens into token groups and combining clusters with very similar cluster groups
Alternately, representing the clusters in color and location proportional to the similarity of their associated
make it more interactive for easier exploration to gain intuition. Use this to get ideas for next steps.

Other work:

I played around with the ROME project's ability to edit info in GPT2. I noticed that some facts (more 'narrowly' represented) are much easier to change than others. For instance, changing the location of the Eiffel Tower is much easier than changing the location of the White House or of California. It could be worth looking into the shapes of representation of facts (multi-layer but narrow vs layer-c

...
AXRP Episode 14 - Infra-Bayesian Physicalism with Vanessa Kosoy
by DanielFilan — 2022-04-05T23:10:09.817Z

YouTube link

This podcast is called AXRP, pronounced axe-urp and short for the AI X-risk Research Podcast. Here, I (Daniel Filan) have conversations with researchers about their research. We discuss their work and hopefully get a sense of why it’s been written and how it might reduce the risk of artificial intelligence causing an existential catastrophe: that is, permanently and drastically curtailing humanity’s future potential.

Late last year, Vanessa Kosoy and Alexander Appel published some research under the heading of “Infra-Bayesian physicalism”. But wait - what was infra-Bayesianism again? Why should we care? And what does any of this have to do with physicalism? In this episode, I talk with Vanessa Kosoy about these questions, and get a technical overview of how infra-Bayesian physicalism works and what its implications are.

Topics we discuss:

The basics of infra-Bayes

An invitation to infra-Bayes

What is naturalized induction?

How infra-Bayesian physicalism helps with naturalized induction

Bridge rules

Logical uncertainty

Open source game theory

Logical counterfactuals

Self-improvement

How infra-Bayesian physicalism works

World models

Priors

Counterfactuals

Anthropics

Loss functions

The monotonicity principle

How to care about various things

Decision theory

Follow-up research

Infra-Bayesian physicalist quantum mechanics

Infra-Bayesian physicalist agreement theorems

The production of infra-Bayesianism research

Bridge rules and malign priors

Following Vanessa’s work

Daniel Filan:
Hello everybody. Today, I’m going to be talking with Vanessa Kosoy. She is a research associate at the Machine Intelligence Research Institute, and she’s worked for over 15 years in software engineering. About seven years ago, she started AI alignment research, and is now doing that full-time. Back in episode five, she was on the show to talk about a sequence of posts introducing Infra-Bayesianism. But today, we’re going to be talking about her recent post, Infra-Bayesian Physicalism: a Formal Theory of Naturalized Induction, co-authored with Alex Appel. For links to what we’re discussing, you can check the description of this episode, and you can read the tran...
What if we stopped making GPUs for a bit?
by MrPointy — 2022-04-05T23:02:19.453Z

I generally concur with MIRI's latest on safety. AI progress seems bad.

AI development is pretty tied to GPU (TPU?) production at this point? If GPU production dropped by 95% for a few years, or if GPUs stopped getting faster for a few years, I wonder if that would affect AI timelines at all? Possibly not, I don't know what Google / DeepMind's / OpenAI's hardware vs. researcher balance sheet looks like. If hardware costs are small relative to researchers then it stands to reason that the hardware market doesn't impact things much.
Production of this hardware is pretty centralized (Taiwan Semiconductor), RnD is too (AMD, Nvidia). If the leaders of these orgs miraculously were convinced by reason to act against their business interests in a heroic way, maybe that would buy everyone a few months or a year. Maybe not, would be interested in more analysis of the above.

Just thought this seemed worth pointing out since I had not seen it discussed.
5-Minute Advice for EA Global
by Logan Riggs — 2022-04-05T22:33:04.087Z

This 5-minute post is perfect advice for me, but YMMV

Week Before

1. Update your profile (pic, bio, contact). ~20 minutes

2. Schedule 20 one-on-ones - these are 10x more important than talks. Filter people by interests/stage in career/etc and cp-paste "Hey, I'm bob who works on [X]. Would you be interested in a 1-on-1 during the conference?" to the top 20 people (or bottom 20 because it's alphabetical order). You can actually save a lot of time by scheduling a specific time during your first message. No one is going to hate you if you send them these messages, haha ~10 minutes

Or, send the same message but with a more custom message after reading their profile ~60 minutes
Do NOT wait to set a time. Do that on the first or second message. You can say "I just picked an arbitrary time to chat! Feel free to decline and pick a better time for you if you'd like. No obligation:)" when you pick a time
Do NOT schedule 1-on-1's early in the morning if you're not a morning person

3. Work through what you want to talk about ~1/2-10hrs

Imagine what a smart person giving obvious advice would reply to what you wrote.
Google your questions
Keep a list of relevant links you want to send on your phone/tablet with a relevant blurb (eg "AI Safety support has links to funding/fellowships, free career coaching for those interested in alignment, and free health coaching for those working in alignment: https://www.aisafetysupport.org/" )

4. Put on your calendar/ set an alarm for after the conference to go over the material, contact people, etc ~2 minutes

5. If you have a talk you're interested in giving, just ask if you could give the talk through email w/ title, description, and possibly slides (I did this the Monday before, got it, and finished my slides by Wednesday, but it was a crunch)

During

1. Better 1-on-1s

If you're waiting on them to show up, read their profile real quick
Say "Just a reminder, I'm bob who works on X, is there anything you wanted to talk about?"

Don't do small talk "how are you finding the conference", but do do ("doo doo" hehe) "I'm fine, but do you want to grab some water or food". Or ask to do that if you want to.

Ask to sit, walk, go outside if you want to do that.
Sit an alarm on your phone for 5 till the end of your 1-on-1 so you don't have to worry about time
Have something to write with. Pen & folder or phone-notes may be best. A laptop was too much to get in&

...