trentmkelly/LessWrong-43k huggingface dataset:
Oh No My AI (Filk)
by Gordon Seidoh Worley — 2021-06-11T15:05:48.733Z

To the tune of "Because I Got High" with apologies to Afroman.

I was gonna save the world
when I built AI
I was gonna make everyone's life great
with my new AI
But I built it unaligned, I don't know why (hey hey)
oh no my AI
oh no my AI
oh no my AI

I made a really nice box
for my AI
I gave it a really good lock
to hold my AI
But it talked its way outside, I'm gonna cry (yeah yeah)
oh no my AI
oh no my AI
oh no my AI

I made my AI smart
smarter than I
So we could do really cool stuff
all thanks to AI
But it outsmarted me, and now we'll die (oh no)
oh no my AI
oh no my AI
oh no my AI

I wish I'd made it better
a Friendly AI
I shoulda listened to Yudkowsky
when I built AI
But instead we're paperclips, and I know why (say why)
because my AI
is unaligned
oh no my AI

This song is very much amenable to additional verses, so I hope folks have fun writing additional ones in the comments!
Would you like me to debug your math?
by Gurkenglas — 2021-06-11T10:54:58.018Z

At AI safety camp I noticed that people are bad at math and when I look over their shoulder as they code I notice nested for loops that could have been one matrix multiplication.

Therefore I'll try offering this service to the public: You screenshare as you do your daily tinkering, I watch for algorithmic or theoretical squiggles that cost you compute or accuracy or maintainability.

Maybe some people are less effective pair programming than alone. You choose whether we pretend I'm not there, whether you think out loud, or whether we talk theory instead.

We might learn engineering from each other. I have shown multiple people IDE features such as debuggers, and I still don't know a way as elegant as I feel it should be to, say, spin up a minimal transformer implementation for tinkering in a few minutes.

If your workflow is embarassing, that's fine: You are particularly likely to benefit, and I expect this is actually quite common. Edit: For every person so far, I would prefer one like them scheduling a session to not.

I did Java years ago, Haskell less years ago and Python the last few months. Of course, pseudocode is the real language. Here's examples of me rewriting other people's Haskell code: https://codereview.stackexchange.com/users/84131/gurkenglas

Edit: I got 5 meetings from this post out of ~280 views. I'd like that percentage to be higher! My current criteria for accepting people are: If you've found this post, I'm accepting you.

Schedule here: https://calendly.com/gurkenglas/consultation
What are the best ways to improve resiliency?
by Nicole Ross — 2021-06-11T03:22:26.355Z

What are things you can do to make you more robust to "life shit" that happens? What have you tried and found to be pretty helpful?
To kick things off, here are a few things I've tried, which have helped to varying degrees:

working fewer hours
exercising
focusing on improving my sleep (with mixed success)
journaling
therapy, especially ACT, CBT, and EMDR
How do you keep track of your own learning?
by DirectedEvolution — 2021-06-10T20:25:08.888Z

I don't know if the educational system's way of trying to measure student learning is any good. But it faces a tough challenge! It can't look directly into your mind and see the knowledge stored there. It can't surveille your activities and see how much you've been studying. Instead, it tries to measure learning with grades, and then tries to see if grades correlate with achievements later in life.

But individual students do, to some extent, have access to this privileged information about how they're studying and what they know. They might deceive themselves in any number of ways. But I would expect that most students have ways of assessing their own progress in learning, and beliefs about how accurate and useful their self-assessments are.

For example, students know the extracurricular articles they've read, and the conversations and projects they've participated in. Students can introspect about their own mind, and reflect on the study activities they've undertaken, like flashcard review, taking notes, or reading. Students have a level of self-awareness about their confidence level and raw ability in solving novel problems.

Do you have any non-obvious ways of measuring your own progress in learning? How much faith do you have in your own self-assessment? When you're reading and studying, how much do you think of the activity as "building a new skill," and how much of it is a transient test or a challenge, like a game?
Covid 6/10: Somebody Else’s Problem
by Zvi — 2021-06-10T13:10:00.795Z

Logistics note: This week’s post includes some non-time-sensitive items from last week, including much of the section on the lab leak hypothesis, as my available time last week was limited. Also, I’ll be in NYC from 6/11 until 6/16, so if you’d like to chat, let me know – I promise asking will cost you at most zero points.

Covid-19 is not quite done with the United States of America. The Indian variant (“Delta”) delivering a ‘one last scare’ moment is plausible. But unless there’s a new variant that can escape from the mRNA vaccines and we are unable to respond rapidly enough, a possibility I now put at most at 10%, Covid-19 is mostly done with the United States of America.

There are essentially three things left.

Live life. There’s the question of how we can safely transition back to a normal life here in the good old USA, and what that new normal life will look like. This includes how worried we should be about potential future strains or issues of seasonality.
Rest of the world. Covid-19 is probably mostly done with the USA. Other countries without our vaccine access are not so fortunate. It is quite plausible that the majority of deaths from Covid-19 are in the future rather than the past.
Postmortem. There’s the question of how we can learn from what happened. We need to adjust our models of the world, and what methods we can use to make sense of it or change it for the better. We also specifically need to do what we can to deal properly with the next pandemic or other crisis. Thus questions like the origins of the virus and what to do about what seems like the absurdly terrible idea that is Gain of Function research.

It is easy to view almost all of this as Somebody Else’s Problem. One could say, I’m vaccinated and can go live life, give or take a few medium-term mask requirements. If someone else in America isn’t vaccinated, one can decide they made a choice. We can leave others to prepare for the next pandemic and go back to not caring much about what is happening overseas.

Or, if to one’s credit one wants to be or remain in the habit of caring about somebody else’s problems, one can balance such cares between all the various problems we have. There are a lot of problems out there. I do think the Covid-19 endgame, and how many people will end up vaccinated versus infected, is still for now sufficiently up in the air and still looms large en...
ML is now automating parts of chip R&D. 
How big a deal is this?
by Daniel Kokotajlo — 2021-06-10T09:51:37.475Z

Article; paper

...the authors’ floorplan solutions have been incorporated into the chip designs for Google’s next-generation artificial-intelligence processors. This means that the solutions are good enough for millions of copies to be printed on expensive, cutting-edge silicon wafers. We can therefore expect the semiconductor industry to redouble its interest in replicating the authors’ work, and to pursue a host of similar applications throughout the chip-design process.

My current guess is that this is not a big deal. Surely these AI-optimizations will result in something like 10% improvement in AI-training-FLOPS-per-dollar, not 100%+, so they won't really change timelines or anything else strategically important. And it won't even be 10% improvement every year from now on, but more like 10% this year, a further 5% next year, a further 2.5% the year after that, etc. as the low-hanging fruit from floorplan optimization is picked. OTOH, this plausibly will reduce the time it takes to design new chips by a lot... but I'd be surprised if that was the main bottleneck anyway. I would have thought ramping up production was the main bottleneck.

I know very little about the chip industry though. Anyone care to correct me?
“Necessary Claims”: A technique to structure complex decisions
by fgrosseholz@yahoo.com — 2021-06-10T05:16:34.409Z

Life often turns out to be too complicated to fit into a human brain. If you’ve ever tried to choose from fifty flavors of ice cream, or to decide whether to buy a house, you know what I mean.

Humans tend to make complex decisions based on simplified models called heuristics. Heuristics sometimes turn out to lack important factors, be biased, or hard to explain. Often, that’s fine - the stakes are low, you don’t have time and no one needs to understand your reasoning (e.g., choosing from fifty flavors of ice cream). “Necessary Claims'' is for the cases where a high risk of being incomplete, biased or hard to explain isn’t fine: Stakes are high, and you may need to involve others in your decision (e.g., buying a house).

Breaking down a decision into necessary claims helps to not overlook important factors, and it gives you a story to tell. In the beginning, your story might sound like “we will probably decide this way, if x turns out to be true and y turns out to be false”. And later on, it will change to “we have decided this way, because we are highly confident that x is true and somewhat confident that y is false”.

Note that this technique formalizes a process that may seem obvious to some people. To me, it was hugely helpful to think about it explicitly and use it systematically, and I hope it will be to others, too. It also builds on several techniques taught by

CFAR

, although I am not officially affiliated with CFAR, this post reflects my personal experience and opinions and all mistakes are my own. I'm grateful to Elizabeth Garrett and Kyle Scott for feedback on an initial draft.

In a nutshell

To make a decision, you list all the necessary claims that would need to be true for you to decide one way. You also list your confidence that each claim is true. You then do analyses to support (or refute) those claims, until your aggregate confidence is high enough to decide.

Description of the technique, using an example

Say you and your partner are considering buying a house in the city you live in. Houses are expensive, so this is high-stakes, and you pool your money, so you want to decide together. You are aware that this decision will take up a few of your Saturdays, and decide to invest the time. First, you agree on the decision you want to make: “Should we buy a house here and now?”

If you have trouble defining the decision at this step, you might co...
Reply to Nate Soares on Dolphins
by Zack_M_Davis — 2021-06-10T04:53:15.561Z

A similar definition of intelligence was expressed by Aquinas as "the ability to combine and separate"—the ability to see the difference between things that seem similar and to see the similarities between things which seem different.

—A. R. Jensen

In a June 2021 Twitter thread, Nate Soares, executive director of the Machine Intelligence Research Institute, asserts, "The definitional gynmastics [sic] required to believe that dolphins aren't fish are staggering." (Archived.)

Soares elaborates:

Suppose for argument that we adopt the (dubious but sadly common) assumption that words like "fish" should have a genealogical definition. Then, just as whales are mammals, mammals are fish—as you can see by tracing the lineages.

Which is to say, if we look at the least common ancestor of all things that are clearly fish, and define a "fish" to be one of its descendants, then dolphins—and humans, and frogs, and birds—are fish.

Now suppose instead we take this as the reductio ad absurdum that it is, and accept that words like "fish" should be functionally rooted, according to macroscopic human-relevant features.

Then the natural denotation of "fish" is, I claim, the collection of all the swimmy creatures, which clearly includes dolphins.

Indeed, this is quite likely what "fish" used to mean—"Jonah was swallowed by a fish", etc. etc.

Yet somehow, once we figured out about genealogy, the pedants were like "well actually this fish's uncle was a fuzzy pigdear, so it's not actually a fish, you uneducated idiot, you absolute moron" and then we all forgot what "fish" meant out of sheer shame or something???

(I feel a sense of betrayal about this. Usually the pedants are my people! How did it go so wrong?)

So, look: this isn't about who the fish's uncle is. When a kid points at a whale and says "look, a fish", and you're like "haha no, its tail flaps horizontally and its gradma had hair", who's in the wrong here?

But Soares is failing to address the strongest case in favor of phylogenetic definitions, even for vernacular words rather than specialist jargon. It's true that in most everyday situations, people don't directly care about which animals are evolutionarily related to each other. But the function of word definitions is not to capture everything the word means. If words were identical with their definitions, and you defined humans as "mortal featherless bipeds", then you would...
The Apprentice Experiment
by johnswentworth — 2021-06-10T03:29:27.257Z

About two months ago, someone asked me what I would do with more funding. Other than the obvious (i.e. generally improve my own quality-of-life in minor ways), my main answer was: take on an apprentice. I have some models about how best to train people for this sort of work, and an apprentice would allow me to test those models while also supporting my own research. I started laying groundwork for that plan - in particular,

Specializing in Problems We Don’t Understand

laid out my main background model.

Then, about a month ago,

Aysajan

put up a short post titled “

Can I be Your Apprentice?

” - essentially an open call to people on LW doing cool work. We talked, it seemed like a good fit, so the apprentice experiment kicked off ~3 weeks ago.

This post will provide more detail on models, motivation, the plan, etc, including a section for Aysajan to introduce himself.

Background Models

First background model:

Specializing in Problems We Don’t Understand

. Problems-we-don’t-understand are similar to each other in a way which problems-we-do-understand are not. In the context of scientific research, preparadigmatic research in different fields is similar in a way which research within a paradigm is not. There are general skills and knowledge useful for finding/creating structure de novo, as opposed to working within some already-mapped structure.

Furthermore, while problems-we-don’t-understand may require some specialized knowledge, specialized knowledge of the field is never the rate-limiting step; if it were, then the problem would already be tractable to people steeped in the existing specialized knowledge of the field. If a problem is tractable within the current paradigm, then it isn’t preparadigmatic. Broad, generalizable skills/knowledge are much more important for problems-we-don’t-understand than for problems-we-do-understand.

The linked post goes into more detail on how one can train and specialize in problems-we-don’t-understand.

Second background model:

Selection Has A Quality Ceiling

. If we want people with a lot of skill in a lot of areas, trying to hire such people directly is Hard, in a big-O sense. As the number of traits we’re filtering for increases, the number of people we have to test in order to find one with all the requisite traits increases exponentially. The big-O requirements training skills are much better: as long as learning one skill doesn’t ma...
Changing my life in 2021, halfway through
by Borasko — 2021-06-10T01:36:50.997Z

This year I decided to really try and fix some major parts of my life I have been neglecting. I started in January and I think I have been making good progress. I will give links to the resources I think helped improve my life so you can look over them and implement them too if you are interested.

I will be making a far more fleshed out post for my end of year review, but I’m making this both as a reflection for myself and as a mini time-capsule to look back at.

Why I did this:

It was mid 2020 when I realized I wanted to be an AI researcher and work on some of the most fun / interesting problems of humans. The only problem was I just graduated with a business administration degree and knew almost nothing past high school algebra in terms of math. I screwed around the rest of the year doing some random walks through math and coding, learning things like the power rule for calculus without knowing anything else about calculus or what a derivative was. At the end of the year and after consuming a lot more of Less Wrong content (which I found as a corollary of my newfound interest in AI). I realized I should stop random walking not only math, but I needed to stop random walking my entire life and should make everything functionally better.

0-100(ish(hopefully)) Machine Learning Study guide:

I didn’t want to spent 4 more years to get another degree, academia has a lot of pro’s and con’s but ultimately it came down that I think I could learn the same material on my own faster, pay less, but with the trade-off of having a harder time proving myself as a competent programmer. So I decided to self learn with the goal of learning so much and becoming such a good programmer I could manage to slam my foot into the door and get a job with no C.S (computer science) degree.

At the beginning of this year I was opine-ing in a babble thread how I was upset that most learning sources sucked. They seemed to be far too densely packed to be useful. I will be fully honest and say I took a two in one, “how to code in python and learn machine learning” class on Udemy at the same time, needless to say I was way in over my head and didn’t finish it as I knew little programming and less math. If only there was a curriculum list that could go through many classes and actually teach rigorous, practical and theoretical programming.

I looked around a little bit, but since anybod...
Finite Factored Sets: Orthogonality and Time
by Scott Garrabrant — 2021-06-10T01:22:34.040Z

The main way we'll be using factored sets is as a foundation for talking about concepts like orthogonality and time. Finite factored sets will play a role that's analogous to that of directed acyclic graphs in Pearlian causal inference.

To utilize factored sets in this way, we will first want to introduce the concept of generating a partition with factors.

3.1. Generating a Partition with Factors

Definition 16 (generating a partition). Given a finite factored set F=(S,B), a partition X∈Part(S), and a C⊆B, we say C generates X (in F), written C⊢FX, if χFC(x,S)=x for all x∈X.

The following proposition gives many equivalent definitions of ⊢F.

Proposition 10. Let F=(S,B) be a finite factored set, let X∈Part(S) be a partition of S, and let C be a subset of B. The following are equivalent:

C⊢FX.
χFC(x,S)=x for all x∈X.
χFC(x,S)⊆x for all x∈X.
χFC(x,y)⊆x for all x,y∈X.
χFC(s,t)∈[s]X for all s,t∈S.
χFC(s,t)∼Xs for all s,t∈S.
X≤S⋁S(C).

Proof. The equivalence of conditions 1 and 2 is by definition.

The equivalence of conditions 2 and 3 follows directly from the fact that χFC(s,s)=s for all s∈x, so χFC(x,S)⊇χFC(x,x)⊇x.

To see that conditions 3 and 4 are equivalent, observe that since S=⋃y∈Xy, χFC(x,S)=⋃y∈XχFC(x,y). Thus, if χFC(x,S)⊆x, χFC(x,y)⊆x for all y∈X, and conversely if χFC(x,y)⊆x for all y∈X, then χFC(x,S)⊆x.

To see that condition 3 is equivalent to condition 5, observe that if condition 5 holds, then for all x∈X, we have χFC(s,t)∈[s]X=x for all s∈x and t∈S. Thus χFC(x,S)⊆x. Conversely, if condition 3 holds, χFC(s,t)∈χFC([s]X,S)⊆[s]X for all s,t∈S.

Condition 6 is clearly a trivial restatement of condition 5.

To see that conditions 6 and 7 are equivalent, observe that if condition 6 holds, and s,t∈S satisfy s∼⋁S(C)t, then χFC(s,t)=t, so t=χFC(s,t)∼Xs. Thus X≤S⋁S(C). Conversely, if condition 7 holds, then since χFC(s,t)∼⋁S(C)s for all s,t∈S, we have χFC(s,t)∼Xs. □

Here are some basic properties of ⊢F.

Proposition 11. Let F=(S,B) be a finite factored set, let C and D be subsets of B, and let X,Y∈Part(S) be pa...
Knowledge is not just mutual information
by Alex Flint — 2021-06-10T01:01:32.300Z

Financial status: This is independent research. I welcome financial support to make further posts like this possible.

Epistemic status: This is in-progress thinking.

This post is part of a sequence on the accumulation of knowledge. Our goal is to articulate what it means for knowledge to accumulate within a physical system.

The challenge is this: given a closed physical system, if I point to a region and tell you that knowledge is accumulating in this region, how would you test my claim? What are the physical characteristics of the accumulation of knowledge? We do not take some agent as the fundamental starting point but instead take a mechanistic physical system as the starting point, and look for a definition of knowledge in terms of physical patterns.

The previous post looked at measuring the resemblance between some region and its environment as a possible definition of knowledge and found that it was not able to account for the range of possible representations of knowledge. This post will explore mutual information between a region within a system and the remainder of the system as a definition of the accumulation of knowledge.

Formally, the mutual information between two objects is the gap between the entropy of the two objects considered as a whole, and the sum of the entropy of the two objects considered separately. If knowing the configuration of one object tells us nothing about the configuration of the other object, then the entropy of the whole will be exactly equal to the sum of the entropy of the parts, meaning there is no gap, in which case the mutual information between the two objects is zero. To the extent that knowing the configuration of one object tells us something about the configuration of the other, the mutual information between them is greater than zero. Specifically, if we would have had to ask some number N of yes-or-no questions to identify the configuration of the environment without any knowledge of the configuration of the region of interest, and if knowing the configuration of the region of interest reduces the number of yes-or-no questions we need to ask by M, then we say that there are "M bits" of mutual information between the region of interest and its environment.

Mutual information is usually defined in terms of two variables whose exact values are unknown but over which we have probability distributions. In this post, th...