trentmkelly/LessWrong-43k huggingface dataset:
Failed Adventures in Delay
by jefftk — 2022-09-20T02:20:03.387Z

One thing that would be fun to be able to include with

Kingfisher is a fiddle
delay. So I tried making one! It seems like it should be pretty
simple: take audio in, wait some number of milliseconds, then send it
back out. With some playing around at home, it seemed like a delay
that matched the current tempo and repeated things three times could
be good. I coded something up, and it seemed to work ok.

I finished it just before we went down to play CDNY, and I brought it
along. Unfortunately, this is a dance with a relatively short amount
of time for setup: the hall opens at 6:45 (though we were able to get
in a bit before that), the beginner's workshop starts at 7:00 (though
the beginners were, as usual, a bit late, and we had extra time), and
the dance starts at 7:30. This is very tight; if I'm not running
sound I try to have an hour for setup (6pm arrival before a 7pm
workshop) and it usually takes ~40min to get everything set up and
sound checked. Which all meant that the delay was not a priority, and
while we did line check it visually during the workshop (when you
can't be noisy) we didn't get farther than that.

Unfortunately, it didn't work out. We tried checking it during a
walkthrough, and then turned it off as soon as the set started. Which
was fine, and the set was good—I suspect none of the the dancers
even noticed! But here's what it sounded like, with the fiddle mic
panned hard left and the delay hard right, though it wasn't this loud.
I turn the delay on at 0:07:

(mp3)

There are several things going wrong in that clip, and I'm not sure
how much each is responsible. The big one is that the fiddle mic (AT
Pro-35 clip on) doesn't only pick up fiddle: it also picks up the
monitor and general hall noise. Normally this is a nuisance, because
it can lead to feedback, and it makes the overall sound mildly muddier
than it would normally be, but it's worth it because a mic gives you
the best fiddle tone to work with. With a delay, however, it's now
much more of a problem: no one wants to hear each of the caller's
words four times.

Another issue is that this particular hall is a rectangular box, full
of hard reflective parallel surfaces. This means you have a lot of
resonance. You don't want zero resonance (I've played in a hall like
that, also in NYC, and all the energy from the dancers is just
swallowed up by the per...
Gene drives: why the wait?
by Metacelsus — 2022-09-19T23:37:17.595Z
Link: https://denovo.substack.com/p/gene-drives-why-the-wait

(Crossposted from my Substack)

If you’ve been following biology news over the last few years, you might have heard of an interesting concept called a “gene drive”. The overall idea is to engineer a genetic allele that transmits itself to all offspring of a sexually reproducing organism, instead of being inherited by 50% as usual. This allele can also perform some other biological function (a relevant example is causing female sterility).

A gene drive spreads through a population. From Esvelt et al. 2014 (CC-BY)

In multiple trials, modern CRISPR-based gene drives have shown high efficacy in spreading through populations of caged Anopheles mosquitoes and completely suppressing their reproduction. Since Anopheles mosquitoes are the only ones that transmit malaria, causing their extinction would directly save hundreds of thousands of lives per year. Similar gene drives targeted to other types of mosquitoes (Aedes, Culex, etc.) could also eliminate diseases such as dengue fever, Zika virus, and West Nile virus.

However, in spite of promising laboratory trials, gene drives have not yet been deployed in the wild. But why not?

History of gene drives

Although the technology to build effective gene drives did not exist until recently, the idea has been around for a while. In fact, gene drives occur naturally. Some well-known examples are transposons in flies, homing endonucleases in algae, and segregation distorters in mice.

The idea of engineering a site-specific nuclease as a gene drive was developed as early as 2003, and in the decade that followed there were several efforts to develop these, with the labs of Austin Burt and Andrea Crisanti taking a lead role. These early systems showed some biased inheritance, but were not stable for more than a few generations.

The advent of CRISPR as a gene editing system opened up a new opportunity. A paper in 2014 by Kevin Esvelt and co-workers proposed Cas9 as a nuclease for a gene drive, with several properties making it ideal for the task.

It lacks repetitive sequences that caused problems with earlier gene drives using zinc-finger nucleases or TALENs.
It has a very high efficiency of cutting.
It is easy to target a new site by simply changing the guide RNA.
Several nearby sites could be targeted at once, using different guide RNAs.

From Esvelt et al. 2014 (CC-BY)

CRISPR-based gene drives quickly gained popularity in the field, and...
Prize idea: Transmit MIRI and Eliezer's worldviews
by elifland — 2022-09-19T21:21:13.156Z

Motivation

Claim (80% confidence): At least 50% of the disagreement between people who align more with MIRI/Eliezer and those who align more with opposing clusters of views (

Christiano

,

Garfinkel

, etc.) is caused not by rational disagreement, but by more subconscious emotional stuff like tastes, vibes, culture, politics, etc.

Elicit Prediction (forecast.elicit.org/binary/questions/28xD2oJyG)

In particular, I think Eliezer’s writing tends to appeal more to people who:

Enjoy reading fiction, and are okay with points primarily being made via parables

Similarly, don’t mind long pieces with no summaries and a lack of clear organizational structure of the claims being made and the evidence for them

Aren’t turned off by perceived arrogance (I’m not taking a stance in this post on whether the arrogance level is justified or not)

Past attempts to communicate their worldview such as the

MIRI conversations

have helped some, but I think mostly weren’t attacking the core issue for which my current guess is large differences in communication styles.

For people who aren’t high on the above axes, I think Eliezer’s writing often tends to be fairly hard to read and offputting, which is unfortunate. It leads to people not taking the points as seriously as they should (and perhaps has the opposite effect on those who the style resonates with). While I disagree with MIRI/Eliezer on a lot of topics, I agree with them to some extent on a lot and think it’s very valuable to understand their worldview and build a “MIRI-model”.

I’ve written briefly about my personal experience taking Eliezer and MIRI less seriously than I should have

here

. I still haven’t read most of the sequences and don’t intend to read HPMOR, but I now take MIRI and Eliezer’s worldviews much more seriously than I used to.

Proposal

An idea to mitigate this is to give out substantial prizes for write-ups which transmit important aspects of the MIRI/Eliezer worldview in ways which are easier for people with different tastes to digest.

A few possible candidates for this “transmission” include (not an exhaustive list!):

The Sequences

Eliezer’s and Nate’s portion of the

MIRI conversations

2022 MIRI Alignment Discussion

AGI Ruin: A List of Lethalities

may be a good candidate, though while extremely important I think this one is already decently structured and argued relative to some other pieces

I...
Rationality Dojo Berlin Handout
by UnplannedCauliflower — 2022-09-19T20:11:02.774Z

This year, like in 2018 and 2019, on Monday after the Less Wrong Community Weekend, we held an Open Rationality Dojo. This time I organised this event and throughout LWCW fielded many questions, usually “what does this Dojo look like?” or “what kind of topics do you cover?”, sometimes from people who said they couldn’t attend anyway, but were still interested in the idea, also partly because it seemed to be much in line with what Duncan’s keynote speech urged us to do. The following document was written to satisfy most of the questions asked, published to the LWCW participants and now LW at large, with the approval of other Dojo members

Why have a Rationality Dojo?

The Rationality Dojo in Berlin was created out of the need to practice rationality on a more regular basis and with the support and peer pressure of other people. Over the years the Dojo has morphed many times and reflects what works for us at the time of writing of this document; it is by no means the one true way to hold a Rationality Dojo

Who attends?

The participants of the Dojo are a somewhat closed group. Because we might cover quite intimate or sensitive topics, we don’t want strangers showing up unannounced and in order to foster trust assume the rule “what happens in the Dojo, stays in the Dojo”. Every now and then someone is brought to our attention, we discuss if we think this person would be a good addition to the group and then invite them or not. We urge the members to come to every Dojo, because that’s how everyone can get the most out of this meeting and we can stay in touch with others’ projects

Before the pandemic each week we would meet as a group of 8-12 people, which meant that we often had enough content to have to split into two rooms for some of the sessions. In my opinion a Dojo of more than 15-18 members showing up regularly every week would be too large to realistically be able to keep in touch with everyone

The pandemic almost killed our Dojo: initially we tried meeting online, but many people had had enough of zoom meetings as a part of their work and also the format of a zoom conference doesn’t foster a natural discussion; people were dropping out also when some meetings turned into social calls instead of augmenting rationality in our lives. In spring (of 2022) we had to move the location of the Dojo meetings and used that as a Schelling point of reviving it - reached out ...
A noob goes to the SERI MATS presentations
by Lowell Dennings — 2022-09-19T17:35:21.352Z

I have been cooped up in suburbia trying to take a break. But not having a routine and being around peers made the days feel monotonous. I was procrastinating writing my Prospect reflection and operations guide, and ended up reading alignment papers and being confused about how to figure out what was important.

Then, I got an message to go to the end of the SERI MATS program presentations. I knew about the evening party beforehand, but now I decided to go since I was more willing to go all the way to Berkeley if it was an all day event. Plus, it would be nice to see the SERI MATS people again and what they did over the summer.

The day before, I did a time-money tradeoff calculation, and I decided I was going to wake up early to take the bus and BART instead of Uber since I'm hesitant to spend money.

Morning came, and I wanted to stay in bed a little longer since my sleep was disturbed. I knew it made sense to pay for the Uber, but I got stressed thinking back and forth. A friend family didn't have anything to do and said he could drive me, and I relented.

Many lane changes and highway exits later, I walked into the venue. There were less people than I expected and I took a seat on the left wing. I felt a little out of place during the first few presentations.

They were pretty technical with a bunch of concepts I didn't learn yet, and it seemed like they were changing this little particular thing to make some result different. During the first few presentations, I was trying to figure out why this thing they did was important, but as we got to later presentations, it got harder and harder to even figure out what was going on, so I decided to write questions to help me understand, but struggling at that too.

I was telling someone how I was taking notes and that I was reading alignment papers at home, and he said he was going to try to solve the problems from scratch without reading a bunch of papers since they may not be relevant. We both agreed that part of the problem was defining and understanding the problem in the first place, and I thought that maybe reading papers to help gather context and exploring research processes would be good too.

My friend and I enjoyed some bitter bok choy during lunch. I was looking forward to his presentation, and I was pleasantly surprised when I understood everything. To be fair, it was about strategy and not very technical,...
Do bamboos set themselves on fire?
by Malmesbury — 2022-09-19T15:34:13.574Z

Cross-posted from Telescopic Turnip.

As we all know, the best place to have a kung-fu fight is a bamboo forest. There are just so many opportunities to grab pieces of bamboos and manufacture improvised weapons, use them to catapult yourself in the air and other basic techniques any debutant martial artist ought to know. A lesser-known fact is that bamboo-forest fights occur even when the cameras of Hong-Kong filmmakers are not present. They may even happen without the presence of humans at all. The forest itself is the kung-fu fight.

It's often argued that humans are the worst species on Earth, because of our limitless potential for violence and mutual destruction. If that's the case, bamboos are second. Bamboos are sick. The evolution of bamboos is the result of multiple layers of shear brutality, with two imbricated levels of war, culminating in an apocalypse of combustive annihilation. At least, according to some hypotheses.

Bamboo wars: bamboo-made famine

Recommended soundtrack for this part

If you enter a bamboo forest and wait for a long time, you may have the chance to witness their strange mating ritual. And by "a long time", I mean you'll probably have to wait for decades before you see anything happen at all. But eventually, all the bamboos in the forest will start to produce an absolutely massive amount of seeds, all at the same time.

Picture from here

And then they all die. The swarm of seeds will give birth to a new generation of bamboos, and it will take another few decades before you see any flower again. Based on historical texts relating the event, we can estimate that some species like Phyllostachis bambusoides flower only once every 120 years:

If you want to know all recorded history of bamboo flowering, this 1976 paper has six full pages of it.

That's really puzzling. Wouldn't a species with a shorter cycle, hence a higher growth rate, take over the forest exponentially fast? Why would bamboos wait for so long before blooming?

The extension of flowering cycles to ridiculous lengths probably happened in two stages. The first stage is about bamboos fighting their predators, the second is about bamboos fighting each other. Then, there is a speculative bonus third stage about bamboos destroying every form of life that has the misfortune to be nearby.

Imagine you're a bamboo. You just released a batch a freshly-baked seeds, only to discover that all the ...
PIBBSS (AI alignment) is hiring for a Project Manager
by Nora_Ammann — 2022-09-19T13:54:12.242Z

TLDR; PIBBSS is hiring for a full-time Project Manager who will be responsible for running the second iteration of the PIBBSS Summer Research Fellowship.

To apply, please complete this

application form

.

---

PIBBSS aims to facilitate knowledge transfer from fields studying intelligent behaviour in natural systems to AI safety and alignment.

The Project Manager will be supported by TJ and Nora (who ran the fellowship in 2022) to help transfer learnings from last year’s fellowship, and work alongside (and manage) 1-3 team members to help execute the program.

More information about the role here.

Application form

We accept and evaluate applications on a rolling basis. Note that we are looking to hire as soon as possible and no later than early October.

We’re happy to discuss this opportunity with any potential applicants. Feel free to contact me with any questions you might have at:

fellowship@pibbss.ai

.
Quintin's alignment papers roundup - week 2
by Quintin Pope — 2022-09-19T13:41:27.104Z

Introduction

Last week's paper roundup (more or less by accident) focused mostly on path dependence of deep learning and the order of feature learning. Going forwards, I've decided to have an explicit focus for each week's roundup. This week's focus is on the structure/redundancy of trained models, as well as linear interpolations through parameter space.

I've also decided to publish each roundup on Monday morning (Edit: or try to, at any rate).

Papers

Residual Networks Behave Like Ensembles of Relatively Shallow Networks

In this work we propose a novel interpretation of residual networks showing that they can be seen as a collection of many paths of differing length. Moreover, residual networks seem to enable very deep networks by leveraging only the short paths during training. To support this observation, we rewrite residual networks as an explicit collection of paths. Unlike traditional models, paths through residual networks vary in length. Further, a lesion study reveals that these paths show ensemble-like behavior in the sense that they do not strongly depend on each other. Finally, and most surprising, most paths are shorter than one might expect, and only the short paths are needed during training, as longer paths do not contribute any gradient. For example, most of the gradient in a residual network with 110 layers comes from paths that are only 10-34 layers deep. Our results reveal one of the key characteristics that seem to enable the training of very deep networks: Residual networks avoid the vanishing gradient problem by introducing short paths which can carry gradient throughout the extent of very deep networks.

My opinion:

This paper suggests that neural nets are redundant by default, which gives some intuition for why it's often possible to prune large fractions of a network's parameters without much impact on the test performance, as well as the mechanism by which residual connections allow for training deeper networks: residual connections allow shallow nets to communicate directly with the input / output space, so they allow for deep nets to be built from ensembling shallow nets.

I think it also points away from neural nets implementing a Kolmogorov or circuit simplicity prior.

On the Effect of Dropping Layers of Pre-trained Transformer Models

Transformer-based NLP models are trained using hundreds of millions or even billions of

...
Some notes on solving hard problems
by Joe Rocca — 2022-09-19T12:58:45.306Z

Caveats/Notes:

This is a cleaned-up version of some personal notes that I've had sitting in Google Docs for a while. There's a lot of hand-waving, probably some over-generalising, and many points could do with some/more examples. Please take each suggestion with a large grain of salt, and feel free to add your thoughts via the comments.
I'm not sure how far these thoughts will transfer outside of my particular problem domain, and problem-solving style. I tend to solve problems by building up visual intuitions/models/algebras, rather than leaning heavily on symbols/equations.
There are some overlaps between various suggestions below - they shouldn't be read as completely orthogonal points. Some ideas are special cases of other ideas, or are basically the same idea from a different perspective.[1]
By "hard" problems, I mean hard for me - as in, something that I wouldn't expect to be able to sit down and solve in one session, say, and which might end up taking days, weeks or longer. That said, some of the stuff here is probably relevant for problem solving more generally.

Develop multiple perspectives on the problem

You should work hard to "see" your problem and potential solutions from multiple perspectives. By "perspective" I mean a high-level mental model that you use to frame your thinking about the problem. A perspective might center around an analogy, or a particular data structure, or a line of reasoning rooted in some low-level principles.

Once you've got multiple perspectives, you can play the perspectives against one another to get a more fundamental understanding of the problem. One perspective will give you insights which unblock your progress on another perspective.

Each perspective is just a mental tool to help you think about the problem - it is specialised for understanding certain types of behavior/constraints that are difficult to understand with the other perspectives.

Good perspectives make important aspects and constraints of your problem "obvious". They make it easy to quickly reason about your problem, and neatly encapsulate constraints so that it requires little effort to keep them all in your head. If you find yourself having to "strain" your brain to pull implications out of a mental model, then it may be that you're using the wrong one for that particular task[2].

A quote from Claude Shannon that I found on this topic:

[...] try to restate

...
Safety timelines: How long will it take to solve alignment?
by Esben Kran — 2022-09-19T12:53:55.847Z
Link: https://forum.effectivealtruism.org/posts/9iGFjYnRquxiy29jm/safety-timelines-how-long-will-it-take-to-solve-alignment

TLDR; no one talks about how far we are towards safe AGI and the focus is on when AGI doom arrives. We want to readjust this focus and measure progress, guide and facilitate research, and evaluate projects in AI safety for impact. We also ask you to add your views to

this survey

.

Prelude

It was a hot London Summer day and fourteen people had gathered to discuss the future of AI safety. To ascertain the mission criticality of our endeavor, we drew everyone’s AGI timelines on a whiteboard. The product of our estimates was close to the Metaculus estimate and some expressed concern at the stress of impending doom.

We then proceeded to draw our expected timelines for when alignment would be solved and very few had ever analyzed this. The few that expressed their timelines ended at 15 years hence, in 2037. And I thought this was a remarkably hopeful message!

A case for hope

Let us be optimistic and expect the median arrival for the solution to alignment is 2037 estimated as a Gaussian with a standard deviation of 10 years[1]. If we estimate the probability that the solution will come before AGI based on sampling the probability mass of the

Metaculus’ forecast for AGI

and our “Safety Timeline” (or “alignment solution timeline”), we can calculate the probability that alignment will be solved before an AGI is released:

P(solution<AGI)=Mean(Sample(Normal(2037,10))<Sample(Metaculus(weak AGI)))=58.73

We can then insert our conditional prior for the probability that the solution will also lead to human flourishing P(human flourishing|solution) and the prior probability of doom from AGI to calibrate P(AGI-caused doom)[2]. In this case, we set P(human flourishing|solution) at a 90% effectiveness probability and the P(doom)=Mean(P(doom)) from 24 researchers in AI safety (see the list in the Appendix). From this, we can calculate the probability mass for the calibrated P(doom), CP(doom).

Mean(P(doom))=28.73

Now we introduce the previous graph with the updated CP(doom) for any year on the AGI timelines. We calculate the uncalibrated P(doom) by multiplying P(AGI|year) by the mean of P(doom) estimates while CP(doom) is P(doom)⋅(1−CDF(P(solution|year)⋅P(AGI safe|solution)), i.e. P(doom) is multiplied by the complement of the cumulative probability distribution for safe AGI ...