trentmkelly/LessWrong-43k huggingface dataset:
Innovation, Stagnation, and Paratrooper Operations
by Davis_Kingsley — 2022-05-06T20:59:30.527Z

I recently came across a very interesting book: "When Failure Thrives: Institutions and the Evolution of Postwar Airborne Forces", by Dr. Marc DeVore. The short version of this book's thesis is that airborne parachute assaults have generally not been historically all that effective in warfare (with perhaps limited windows of effectiveness at various times), but that institutional politics and biases have led to them often being maintained in military training and doctrine despite this ineffectiveness.
This strikes me as an especially interesting case study in "civilizational inadequacy" type models, what structures can support (or stifle!) innovation, evaluations of what levels of play the military is operating on, and so on. Let's jump into it!

Military Obsolescence

DeVore opens by discussing how organizational inertia leads to military establishments being attached to specific tactics or technologies well after the time where those methods have become obsolete, leading to bad performance against opponents who have better adapted to new developments.

One particularly striking case of this that I'm familiar with from previous study is that several European armies still fielded cuirassiers -- horsemen equipped with metal breastplates and swords -- at the advent of World War One, far after such were obsolete!

One might be surprised to learn that this photograph was taken in Paris in the year 1914!

DeVore later points out that in the military, obsolete tactics and technologies often exist much longer than in many other areas of human endeavor. Part of this is due to simple lack of test data to draw from, which allows biased conclusions to run rampant:

Why then do obsolescent tactics and technologies persist within military organization? The equivalent of such holdovers in the commercial sector—such as a large firm refusing to use container ships or the internet—is virtually unknown and would swiftly lead to bankruptcy. One reason for greater inertia in military organizations lies in the incomplete and intermittent nature of how military organizations are tested. Indeed, there is no certain method to ascertain how effective armed forces are short of forcing them to conduct a wide-range of military operations against a wide variety of live opponents. Moreover, even the so-called lessons of recent wars are notoriously difficult to interpret because wars are c

...
Upgrading Imagination: The Promise Of DALL-E 2 As A Tool For Thought
by Tharin — 2022-05-06T18:12:31.825Z
Link: https://echoesandchimes.com/2022/05/upgrading-imagination-the-promise-of-dall-e-2/

[Cross-posting this here, from my blog, Echoes and Chimes]

When I was a kid, I’d spend the hours after school on the circular trampoline we had assembled in our backyard. I’d go into a kind of trance. Though my body would be engaged in jumping up and down — or, when that grew dull, and when the blue plastic cover that obscured the springs wore thin and blew away, in walking around the trampoline’s perimeter in a game of balance — my mind flew far away. I constructed an elaborate fictional world in which to pass my afternoons.

Hopelessly derivative, it was inspired by whatever media I was consuming at the time - I recall the Ratchet and Clank series, and Avatar: The Last Airbender both being heavily influential. I envisioned a galaxy of planets, each with distinct people, cultures, and economic activities. The system was quasi-federal; the political relations complex. For example, I recall creating the planet “Laboris”, the lab planet, where new technology would be developed and tested, before being shipped to worlds the star-system over. There was a planet dedicated to retail and advertising - animated by the spirit of Moloch, I now realise. And so on. Populating these planets, in starring roles, were my friends, family members, and any other people in my life at the time, all reimagined in the art style of my mind’s eye.

As my real life changed, so did my fictional one. Alliances were forged and broken. Civilisations rose and fell. It was a serial drama, with some weeks covering mere days, and some days spanning decades. As serious challenges came up at school — a block of tests, for example — I would envision each obstacle taking a physical form, like an elaborate boss fight. Although the details escape me, I recall my Grade 7 exams mapping onto the mythology of 2007’s Heavenly Sword.

I found great joy in specificity — I’d try to imagine the content of the advertisements that ran on the intergalactic cable channels, the outfits worn by my fellow adventurers, that kind of thing. In pursuing these specifics, I’d frequently run into the limits of my own imaginative capacity. While I was strong on narrative (a consequence of all the books I’d swallowed), and while each character I created evoked strong vibes in my brain, I was quite unable to picture exactly what they looked like. It was like trying to pin fog to paper.

This confrontat...
Contra Dance Mask Policy
by jefftk — 2022-05-06T15:10:01.979Z

After getting a

question
in the BIDA Facebook group, I was curious what mask policies contra
dances are using. I looked at the dances marked as active on

trycontra.com and checked their
websites for mask requirements (

sheet).

Of the 56 dances that have resumed, 31 (55%) require masks. Of those
31:

2 (6%) require surgical or better.

4 (13%) require a surgical + cloth or better.

4 (13%) require high-filtration masks (N95, KN95, KF94, etc)

Now that high-filtration masks are widely available, it does seem like
a weird compromise to require masking but allow low-filtration options
like cloth or surgical, especially when I haven't seen anyone wearing
a P100. Specifically:

Most of society is no longer requiring masks: bars, nightclubs,
workplaces, transit, etc. In MA, one of the more cautious states on
this issue, the state requires
masks only in healthcare, paratransit, shelters, and jails.

This means that the reason to require masks at dances is to
allow people to attend for which it would otherwise be too
risky.

A group wearing surgical masks poses a risk to individuals
(wearing the mask of their choice) that is roughly (per microcovid) 1/4 as risky as if
the group were fully unmasked.

An individual wearing a P100 is at ($16)
(again, per microcovid) about 1/7 the
risk of one wearing a high-filtration mask.

This means that if a group switches from masks-required to
masks-optional and more cautious individuals switch to P100s,
risk to those individuals very likely goes down.

Which then has me wondering: why do we see people saying that dances
need to require masks, but not wearing P100s? Some guesses:

People have adjusted to high-filtration masks now being
available, but not to the availability of P100s.

The P100 masks are more expensive up front. On the other hand,
the part you wear lasts years, the replacement filters are ~$6/pair,
and a filter pair lasts much longer than a disposable high-filtration
mask, so the cost should be similar or lower over time. I also
suspect that in most communities people who prefer dancing without a
mask would be willing to cover the cost of P100s for people who
need them if we could sort out a good way to do this?

People may think P100s are less comfortable. My experience is
that they are a bit more comfortable: slightly more pressure on the
face but more spread

...
The case for becoming a black-box investigator of language models
by Buck — 2022-05-06T14:35:24.630Z

Interpretability research is sometimes described as neuroscience for ML models. Neuroscience is one approach to understanding how human brains work. But empirical psychology research is another approach. I think more people should engage in the analogous activity for language models: trying to figure out how they work just by looking at their behavior, rather than trying to understand their internals.

I think that getting really good at this might be a weird but good plan for learning some skills that might turn out to be really valuable for alignment research. (And it wouldn’t shock me if “AI psychologist” turns out to be an economically important occupation in the future, and if you got a notable advantage from having a big head start on it.) I think this is especially likely to be a good fit for analytically strong people who love thinking about language and are interested in AI but don’t love math or computer science.

I'd probably fund people to spend at least a few months on this; email me if you want to talk about it.

Some main activities I’d do if I was a black-box LM investigator are:

Spend a lot of time with them. Write a bunch of text with their aid. Try to learn what kinds of quirks they have; what kinds of things they know and don’t know.
Run specific experiments. Do they correctly complete “Grass is green, egg yolk is”? Do they know the population of San Francisco? (For many of these experiments, it seems worth running them on LMs of a bunch of different sizes.)
Try to figure out where they’re using proxies to make predictions and where they seem to be making sense of text more broadly, by taking some text and seeing how changing it changes their predictions.
Try to find adversarial examples where they see some relatively natural-seeming text and then do something really weird.

The skills you’d gain seem like they have a few different applications to alignment:

As a language model interpretability researcher, I’d find it very helpful to talk to someone who had spent a long time playing with the models I work with (currently I’m mostly working with gpt2-small, which is a 12 layer model). In particular, it’s much easier to investigate the model when you have good ideas for behaviors you want to explain, and know some things about the model’s algorithm for doing such behaviors; I can imagine an enthusiastic black-box investigator being quite helpful for our

...
Open Problems in Negative Side Effect Minimization
by Fabian Schimpf — 2022-05-06T09:37:58.871Z

Acknowledgments

We want to thank Stuart Armstrong, Remmelt Ellen, David Lindner, Michal Pokorny, Achyuta Rajaram, Adam Shimi, and Alex Turner for helpful discussions and valuable feedback on earlier drafts of this post.

Fabian Schimpf and Lukas Fluri are part of this year’s edition of the AI Safety Camp. Our gratitude goes to the camp organizers: Remmelt Ellen, Sai Joseph, Adam Shimi, and Kristi Uustalu.

TLDR;

Negative side effects are one class of threats that misaligned AGIs pose to humanity. Many different approaches have been proposed to mitigate or prevent AI systems from having negative side effects. In this post, we present three requirements that a side-effect minimization method (SEM) should fulfill to be applied in the real world and argue that current methods do not yet satisfy these requirements. We also propose future work that could help to solve these requirements.

Introduction

Avoiding negative side-effects of agents acting in environments has been a core problem in AI safety since the field started to be formalized. Therefore, as part of our AI safety camp project, we took a closer look at state-of-the-art approaches like AUP and Relative Reachability.

After months of discussions, we realized that we were confused about how these (and similar methods) could be used to solve problems we care about outside the scope of the typical grid-world environments.

We formalized these discussions into distinct desiderata that we believe are currently not sufficiently addressed and, in part, maybe even overlooked.

This post attempts to summarize these points and provide structured arguments to support our critique. Of course, we expect to be partially wrong about this, as we updated our beliefs even while writing up this post. We welcome any feedback or additional input to this post.

The sections after the summary table and anticipated questions contain our reasoning for the selected open problems and do not need to be read in order.

Background

The following paragraphs make heavy use of the following terms and side-effect minimization methods (SEMs). For a more detailed explanation we refer to the provided links

MDP: A

Markov Decision Process

is a 5-tuple ⟨S,A,T,R,γ⟩ In the setting of side-effect minimization, the goal generally is to maximize the cumulative reward without causing (negative) side-effects.

RR: In its sim...
Your Utility Function is Your Utility Function
by David Udell — 2022-05-06T07:15:24.439Z

Spoilers for mad investor chaos and the woman of asmodeus (planecrash Book 1).

The Watcher spoke on, then, about how most people have selfish and unselfish parts - not selfish and unselfish components in their utility function, but parts of themselves in some less Law-aspiring way than that. Something with a utility function, if it values an apple 1% more than an orange, if offered a million apple-or-orange choices, will choose a million apples and zero oranges. The division within most people into selfish and unselfish components is not like that, you cannot feed it all with unselfish choices whatever the ratio. Not unless you are a Keeper, maybe, who has made yourself sharper and more coherent; or maybe not even then, who knows? For (it was said in another place) it is hazardous to non-Keepers to know too much about exactly how Keepers think.

It is dangerous to believe, said the Watcher, that you get extra virtue points the more that you let your altruistic part hammer down the selfish part. If you were older, said the Watcher, if you were more able to dissect thoughts into their parts and catalogue their effects, you would have noticed at once how this whole parable of the drowning child, was set to crush down the selfish part of you, to make it look like you would be invalid and shameful and harmful-to-others if the selfish part of you won, because, you're meant to think, people don't need expensive clothing - although somebody who's spent a lot on expensive clothing clearly has some use for it or some part of themselves that desires it quite strongly.

--Eliezer Yudkowsky, planecrash

I've been thinking a lot lately about exactly how altruistic I am. The truth is that I'm not sure: I care a lot about not dying, and about my girlfriend and family and friends not dying, and about all of humanity not dying, and about all life on this planet not dying too. And I care about the glorious transhuman future and all that, and the 1050 (or whatever) possible good future lives hanging in the balance.

And I care about some of these things disproportionately to their apparent moral magnitude. But, what I care about is what I care about. Rationality is the art of getting more of what you want, whatever that is; of systematized winning, by your own lights. You will totally fail in that art if you bulldoze your values in a desperate eff...
Getting GPT-3 to predict Metaculus questions
by MathiasKB — 2022-05-06T06:01:04.485Z

Can GPT-3 predict real world events? To answer this question I had GPT-3 predict the likelihood for every binary question ever resolved on Metaculus.

Predicting whether an event is likely or unlikely to occur, often boils down to using common sense. It doesn't take a genius to figure out that "Will the sun explode tomorrow?" should get a low probability. Not all questions are that easy, but for many questions common sense can bring us surprisingly far.

Experimental setup

Through their API I downloaded every binary question posed on Metaculus.
I then filtered them down to only the non-ambiguously resolved questions, resulting in this list of 788 questions.

For these questions the community's Mean Squared Error was 0.19, a good deal better than random!

Prompt engineering

GPT's performance is notoriously dependent on the prompt it is given.

I primarily measured the quality of prompts, on the percentage of legible predictions made.
Predictions were made using the most powerful DaVinci engine.

The best performing prompt was optimized for brevity and did not include the question's full description.

A very knowledgable and epistemically modest analyst gives the following events a likelihood of occuring:

Event: Will the cost of sequencing a human genome fall below $500 by mid 2016?
Likelihood: 43%

Event: Will Russia invade Ukrainian territory in 2022?
Likelihood: 64%

Event: Will the US rejoin the Iran Nuclear Deal before 2023?
Likelihood: 55%

Event: <Question to be predicted>
Likelihood: <GPT-3 insertion>

I tried many variations, different introductions, different questions, different probabilities, including/excluding question descriptions, etc.

Of the 786 questions, the best performing prompt made legible predictions for 770. For the remaining 16 questions GPT mostly just wrote "\n".

If you want to try your own prompt or reproduce the results, the code to do so can be found in this Github repository.

Results

GPT-3's MSE was 0.33, which is about what you'd expect if you were to guess completely at random. This was surprising to me! GPT Why isn't GPT better?

Going into this, I was confident GPT would do better than random. After all many of the questions it was asked to predict, resolved before GPT-3 was even trained. There's probably some of the questions it knows the answer to and still somehow gets wrong!

It seems to me that GPT-3 is struggling to translate beliefs into ...
Apply to the second iteration of the ML for Alignment Bootcamp (MLAB 2) in Berkeley [Aug 15 - Fri Sept 2]
by Buck — 2022-05-06T04:23:45.164Z

Redwood Research is running another iteration of MLAB, our bootcamp aimed at helping people who are interested in AI alignment learn about machine learning, with a focus on ML skills and concepts that are relevant to doing the kinds of alignment research that we think seem most leveraged for reducing AI x-risk. We co-organized the last iteration of the bootcamp with Lightcone in January, and there were 28 participants. The program was rated highly (see below for more), and several participants are now working full-time on alignment. We expect to start on Aug 15 but might push it back or forward by a week depending on applicant availability.

Apply here

by May 27.

We’re expecting to have space for about 40 participants. We’ll pay for housing, travel, and food, as well as salaries for the TAs. We will help you get visas to travel to the US for the bootcamp.

We’re now accepting applications for participants and TAs. TAs are expected to either know this material already or have a month free before MLAB to study all the content.

Last time

the schedule was roughly the following:

Prep work: Pytorch array programming
Week 1: Pytorch, optimization

Implement a renderer in pytorch, as an exercise in mathematical array programming
Implement ResNet from scratch in pytorch, implementing all the layers from scratch and loading weights from a trained model.
Implement interpretability techniques on the ResNet.
Implement SGD and other local optimization algorithms, run remote hyperparameter searches on a simple architecture
Implement a simple clone of some of Pytorch, with particular focus on the implementation of backpropagation
(Optional) CUDA programming day–write various CUDA kernels, see how close to the performance of Pytorch’s kernels you can get

Week 2: Transformers

Implement BERT from scratch, load weights from the real pretrained BERT
Implement GPT-2, implement beam search
Fine tune BERT on classification, fine-tune GPT-2 on some specific corpus
Look at various interpretability techniques on GPT-2
Data-parallel training

Week 3

Pipeline parallelism
Tensor parallelism
Deep RL (DQN, policy gradient)
RL algorithms on language models
More transformer interpretability
(Optional)

ELK

day

Week 4: Optional final projects week, Q&As with various alignment researchers

This time, we’ll probably have more systematic transformer interpretability content, because we’ve spent a lot of time since MLA...
Moral Illusions
by Martin Sustrik — 2022-05-06T04:00:22.712Z

The interesting thing about optical illusions is that one may be aware of the illusion, yet, it does not go away.

It seems that the lines on the picture have different lengths even though one positively knows that they are exactly the same.

Now consider this statement:

Until January 2016, all nine situations which the International Criminal Court (ICC) had been investigating were in African countries. None were in European or American countries. ICC is therefore biased against Africa.

It doesn't take much thought to realize that a country with war criminals in jail is better off than a country with war criminals at large. So, if anything, the ICC is biased against Europe and America.

But knowing that doesn't make the moral illusion go away. Read the quote again and it still feels like Africa is being wronged. Repeat as much as you want: Yep, still there. Africa is being wronged.
Write posts business-like, not story-like
by Shmi — 2022-05-05T20:13:08.495Z

There are a number of posts here where people try to tell a story, to provide some sort of intuition pumping. Eliezer did it a lot and it worked for him. Assume you are not as good a writer, and stick to the business/academic writing format.

Start with a summary of your post (ideally, the post title should be an even shorter summary, e.g. "Alignment approach MOOSE fails on the dataset FORREST"), then a brief background and motivation, then your central point, then add a conclusion. Do not force the reader to go through a story ("you wake up in a windowless room, unsure of what is going on...") before they get to see your point, if any.

This is not a hard and fast rule, but a decent default.
Ethan Caballero on Private Scaling Progress
by Michaël Trazzi — 2022-05-05T18:32:18.673Z
Link: https://theinsideview.github.io/ethan

Some quotes from the latest episode of my podcast, The Inside View. You can access the audio, video and transcript here. The key insight is that we are only seeing the tip of the iceberg w.r.t. Large Language Models Scaling, and Alignment can be seen as an inverse scaling problem.

Alignment as an Inverse Scaling Problem

"All alignment is inverse scaling problems. It’s all downstream inverse scaling problems. All of alignment is stuff that doesn’t improve monotonically as compute, data and parameters increase [...] because sometimes there’s certain things where it improves for a while, but then at a certain point, it gets worse. So interpretability and controllability are the two kind of thought experiment things where you could imagine they get more interpretable and more controllable for a long time until they get superintelligent. At that point, they’re less interpretable and less controllable."

"Then the hard problem though is measurement and finding out what are the downstream evaluations because say you got some fancy deceptive AI that wants to do a treacherous turn or whatever. How do you even find the downstream evaluations to know whether it’s gonna try to deceive you? Because when I say, it’s all a downstream scaling problem, that assumes you have the downstream test, the downstream thing that you’re evaluating it on. But if it’s some weird deceptive thing, it’s hard to even find what’s the downstream thing to evaluate it on to know whether it’s trying deceive. "

On Private Research at Google, Deepmind

"I know a bunch of people at Google said, yeah, we have language models that are way bigger than GPT-3, but we just don’t put them in papers. "

"The DeepMind language models papers, they were a year old when they finally put them out on arXiv, Gopher and Chinchilla. They had the language model finished training a year before the paper came out. "

On Thinking about the Fastest Path

"You have to be thinking in terms of the fastest path, because there is extremely huge economic and military incentives that are selecting for the fastest path, whether you want it to be that way or not. So, you got to be thinking in terms of, what is the fastest path and then how do you minimize the alignment tax on that fastest path. Because the fastest path is the way it’s probably gonna happen no matter what."

"The person who wins AGI is whoever has the best funding model for sup

...