trentmkelly/LessWrong-43k huggingface dataset:
[AN #144]: How language models can also be finetuned for non-language tasks
by Rohin Shah — 2021-04-02T17:20:04.230Z

Alignment Newsletter is a weekly publication with recent content relevant to AI alignment around the world. Find all Alignment Newsletter resources here. In particular, you can look through this spreadsheet of all summaries that have ever been in the newsletter.

Audio version here (may not be up yet).

Please note that while I work at DeepMind, this newsletter represents my personal views and not those of my employer.

HIGHLIGHTS

Pretrained Transformers as Universal Computation Engines (Kevin Lu et al) (summarized by Rohin): We’ve seen some very impressive few-shot learning results from GPT-3 (AN #102) and CLIP. These work by training a large Transformer model on a giant pile of data in a particular modality (such as language or images), and then we express tasks within that modality (e.g. summarization for a language model). This paper asks the question: could such models also help with tasks in a different modality? Surprisingly, the answer seems to be yes!

Specifically, the authors take the pretrained GPT-2 models and finetune on very different tasks, changing only the following parameters (which make up just ~0.1% of the model):

1. Input layer: This is a linear layer that transforms the input tokens before they go through the attention layers.

2. Output layer: This is a linear layer that uses the final representations to solve some downstream tasks.

3. Layer norm: These parameters are meant to mimic the statistics of the data distribution, and so need to be finetuned.

4. Positional embeddings. (They say that it only makes a slight difference to finetune these.)

For downstream tasks, they consider tasks like memorizing bit sequences, computing XORs, MNIST and CIFAR (where each image is represented as a sequence of 64 tokens, and each token is a 4x4 patch of the image), and protein folding. None of these tasks involve any use of natural language -- the input modality is completely different.

The headline result: these sorts of models tend to achieve similar performance as Transformer models trained from scratch on the same tasks, and better performance than models initialized with random weights and then finetuned using the method above. This suggests that even for new data modalities the GPT-2 pretraining helps, suggesting that the model has learned some “universal computation” in its attention layers (hence the tit...
Notes from "Don't Shoot the Dog"
by juliawise — 2021-04-02T16:34:46.170Z

Cross-posted from The Whole Sky.
I just finished Karen Pryor’s “Don’t Shoot the Dog: the New Art of Teaching and Training.” Partly because a friend points out that it’s not on Audible and therefore she can’t possibly read it, here are the notes I took and some thoughts. It’s a quick, easy read.

The book applies behavioral psychology to training animals and people. The author started off as a dolphin trainer at an aquarium park in the 1960s and moved on to horses, dogs, and her own children. There are a lot of anecdotes about how to train animals (apparently polar bears like raisins). At the time, training animals without violence was considered novel and maybe impossible. I read it as a parenting book since I don’t plan to train dogs, horses, or polar bears.

It’s probably not the best guide to training dogs since a lot of it is about people, and not the best guide to training people since a lot is about animals. She’s written a bunch of other books about training dogs and cats. But this book is an entertaining overview of all of it.

The specter of behaviorism

I can understand not wanting to use behavioral methods on children; the idea can sound overly harsh or reductive. The thing is, we already reinforce behavior all the time, including bad behavior, often without meaning to. So you might as well notice what you’re doing.

To people schooled in the humanistic tradition, the manipulation of human behavior by some sort of conscious technique seems incorrigibly wicked, in spite of the obvious fact that we all go around trying to manipulate one another’s behavior all the time, by whatever means come to hand.

[...]

There are still people who shudder at the very name of Skinner, which conjures in their minds some amalgam of Brave New World, mind control, and electric shock.

(B. F. Skinner in fact believed that punishment was not an effective learning tool, and that positive reinforcement was much better for teaching.)

Pryor argues that behavioral training allows you to get good results more pleasantly than with other methods. She describes her daughter’s experience directing a play in high school:

At the closing performance the drama coach told me that she’d been amazed to see that throughout rehearsals Gale never yelled at her cast. Student directors always yell, but Gale never yelled. ‘Of course not,’ I said without thinking, ‘she’s an animal trainer.’ From the look on the

...
2012 Robin Hanson comment on “Intelligence Explosion: Evidence and Import”
by Rob Bensinger — 2021-04-02T16:26:51.725Z

The 2013 anthology Singularity Hypotheses: A Scientific and Philosophical Assessment (edited by Eden, Moor, Soraker, and Steinhart) included short responses to some articles. The response that most stuck with me was Robin Hanson's reply to Luke Muehlhauser and Anna Salamon’s "Intelligence Explosion: Evidence and Import."

Unfortunately, when I try to link to this reply, people don't tend to realize they can scroll down to find it. So I've copied the full thing here.

(Note that I'm not endorsing Robin's argument, just sharing it as a possible puzzle piece for identifying cruxes of disagreement between people with Hansonian versus Yudkowskian intuitions about AGI.)

Muehlhauser and Salamon [M&S] talk as if their concerns are particular to an unprecedented new situation: the imminent prospect of “artificial intelligence” (AI). But in fact their concerns depend little on how artificial will be our descendants, nor on how intelligence they will be. Rather, Muehlhauser and Salamon’s concerns follow from the general fact that accelerating rates of change increase intergenerational conflicts. Let me explain.

Here are three very long term historical trends:

Our total power and capacity has consistently increased. Long ago this enabled increasing population, and lately it also enables increasing individual income.
The rate of change in this capacity increase has also increased. This acceleration has been lumpy, concentrated in big transitions: from primates to humans to farmers to industry.
Our values, as expressed in words and deeds, have changed, and changed faster when capacity changed faster. Genes embodied many earlier changes, while culture embodies most today.

Increasing rates of change, together with constant or increasing lifespans, generically imply that individual lifetimes now see more change in capacity and in values. This creates more scope for conflict, wherein older generations dislike the values of younger more-powerful generations with whom their lives overlap.

As rates of change increase, these differences in capacity and values between overlapping generations increase. For example, Muehlhauser and Salamon fear that their lives might overlap with

> "[descendants] superior to us in manufacturing, harvesting resources, scientific discovery, social charisma, and strategic action, among other capacities. We would not be in a position to negotiate with them,

...
2 innovative life extension approaches using cryonics technology
by Mati_Roy — 2021-04-02T00:54:04.539Z

x-post on LessDead

Edit: This was posted on April's fool :) (note: for some timezones, this was posted on April 2nd; oops) While the core ideas are not entirely unfounded, they were presented as stronger than they really are.

While it's true that we can cool down mammals to near 0 °C without apparent damages, damages would very likely start becoming apparent after multiple surgeries; plus, the risk of surgical complications would presumably become significant (I posted about this on February 14th)

There's not really a "big dilemma" in the cryonics community about whether healthy people should get cryopreserved now or later.

While I haven't researched hemispherectomy much, I suspect they are much more dangerous, complicated, and damaging than how I presented it here.

Perpetual cooldown

Acknowledgement: Hunter Glenn came up with the general idea in a discussion with Mati ; written by Mati Roy (edit: Aschwin de Wolf informed me that Robert Prehoda proposed this idea in zir book Suspended Animation in 1969)

We already have the technological capability to cool down mammals to near 0 °C temperature for multiple hours and bring them back without any damages (reference: https://timelines.issarice.com/wiki/Timeline_of_brain_preservation).

While this was originally developed to improve critical care medicine as well as initial cryonics cooldown, there’s another way in which we could use that technology.

We could spend 6 hours at that temperature each day. Of course, that would mean days would be 6 hours longer as metabolism is stopped at that temperature, and the normal sleep functions are not operating. That means for each 4 subjective days, 5 objective days would pass. Waking hours would cycle like this:

Day 1: 6:00 to 22:00

Day 2: 12:00 to 4:00

Day 3: 18:00 to 10:00

Day 4: 00:00 to 16:00

This would delay the moment of your death by up to 25% given metabolism is slower at lower temperature, which means you would likely reach a time when anti-aging and cryonics technology has been improved, hence increasing your chance of living radically longer.

The cost includes:

monetary cost for the surgeries

how you feel after surgeries

losing opportunities for work that require a regular schedule

generally makes it harder to coordinate with people, especially other people into perpetual cooldown

makes it harder to work 40 hours per week

Split brain preservation

Acknowledgement...
"AI and Compute" trend isn't predictive of what is happening
by alexlyzhov — 2021-04-02T00:44:46.671Z

(open in a new tab to view at higher resolution)

In May 2018 (almost 3 years ago) OpenAI published their "AI and Compute" blogpost where they highlighted the trend of increasing compute spending on training the largest AI models and speculated that the trend might continue into the future. This note is aimed to show that the trend has ended right around the moment of OpenAI publishing their post and doesn't hold up anymore.

On the above image, I superimposed the scatter plot from OpenAI blogpost and my estimates of compute required for some recent large and ambitious ML experiments. To the best of my knowledge (and I have tried to check for this), there haven't been any experiments that required more compute than those shown on the plot.

The main thing shown here is that less than one doubling of computational resources for the largest training occured in the 3-year period between 2018 and 2021, compared to around 10 doublings in the 3-year period between 2015 and 2018. This seems to correspond to a severe slowdown of computational scaling.

To stay on the trend line, we currently would need an experiment requiring roughly around 100 times more compute than GPT-3. Considering that GPT-3 may have costed between $5M and $12M and accelerators haven't vastly improved since then, such an experiment would now likely cost $0.2B - $1.5B.
Examples of Acausal Trade with an Alien Universe?
by MikkW — 2021-04-01T18:10:13.541Z

I understand the basic concept of counterfactual trade, and I can understand some examples where it can make sense to trade acausally between two different histories of our familiar world; for example, in Embedded Agency, Scott G and Abram discuss a game where Alice receives a card, either a king or an ace, and can either reveal it to or hide it from Bob, who will guess a probability p that the card is an ace. Bob's incentives are such that he wants to assign maximum probability to the actual outcome, while Alice receives 100*(1 - p^2) points no matter what her card is- so she wants to do her best to make Bob think she has a king.

In this example, one might naively think that if Alice has a king, she should reveal her card, so that Bob will guess p=0, earning Alice the maximum reward. However, under this policy, when Alice has an ace, Bob would be able to infer from Alice hiding the card, that the card is an ace, so he would guess p=100%, and Alice will receive 0 points.

If Alice instead decided to follow the policy of never revealing her card, Bob will be forced to always guess p = 0.5, earning Alice 100*(1 - 0.5^2) = 75 points each time; since Alice prefers a 100% chance of 75 points over a 50/50 shot at {0, 100}, Alice should actually hide her card even when she has a king. This is an example of counterfactual trade that makes sense to me.

Similar logic applies to glomarization, and Newcomb's Dillema also has an acausal trade flavor to it, which is similar to some dynamics that appear in normal human interactions.

However, my question is:

Do you know of a concrete example where acausal trade with a universe completely different from ours, with laws of physics different from ours, actually makes sense?

I struggle to think of such an example, where it'd both be harder for us to do something than it would be for an alien, acausally correlated species to do it, and where there would exist an acausally correlated species that would want to reward us by doing so in return for something more valuable that we can provide. Are there any such examples, or is it a valid conjecture that such acausal trades wouldn't ever make sense?
April drafts
by AI Impacts — 2021-04-01T18:10:05.590Z

By Katja Grace, April 1 2021

Today we are sharing with our blog readers a collection of yet-to-be-published drafts, in the hope of receiving feedback. We are especially looking for methodological critique, but all comments welcome!

Human-level performance estimate (Katja Grace)

How much hardware will we need to create AGI? (Asya Bergal)

Historic trends in AI Impacts productivity (Daniel Kokotajlo)

Analysis of superbombs as a global threat (Katja Grace)
Forecasting Newsletter: March 2021
by NunoSempere — 2021-04-01T17:12:09.499Z

Highlights

OpenPhilanthropy releases a report on

outside view perspectives on the likelihood of AGI.

Jason Matheny, previous director of IARPA, CSET, is now a ¿senior? official in the Biden administration.
Astral Codex Ten considers

Trapped Priors As A Basic Problem Of Rationality

Index

Prediction Markets & Forecasting Platforms
In The News
Recent Blog Posts
Hard to Categorize
Long Content

Sign up

here

or browse past newsletters

here

.

Prediction Markets & Forecasting Platforms

Numerai

is a distributed, blockchain-based hedge fund. Users can either

predict on free, but obfuscated data

, or

use their own data and predict on real world companies

. After the users stake cryptocurrency on their predictions, Numerai buys or sells stocks in proportion to each prediction's stake.and then stake cryptocurrency on their predictions. The fund observes how well the predictions do. Then it increases the stake of those who did well and burns part of the stake of those who performed badly. Numerai’s users currently have around $12.5 million staked.

CSET's Founding Director

Jason Matheny

is now a ¿senior?

official in the Biden administration

. In his past life, he did some pioneering work on cultured meat, then was a Program Manager of IARPA's

Aggregative Contingent Estimation (ACE) program

(of Good Judgment fame), before becoming director of IARPA. In recent times, he founded the

Center for Security and Emerging Technologies

(CSET.)

CSET Foretell is launching a

Pro Forecaster Program

in April 2021, which means it will start paying its forecasters. They are offering to pay $200/month (each) to 50 selected forecasters. The total payout, which comes to $120k yearly, competes with Replication Markets as one of the largest forecaster reward budgets.

Pro Forecasters will be paid to make forecasts that contribute to our research and analysis for policymakers. Invitations have been sent to current Foretell users, and we are now accepting applications for the remaining spots. Anyone who can show a proven track record as a top forecaster on Good Judgment Open, Metaculus, or a similar crowd forecasting site may apply (emphasis mine).

Using early

data

, CSET Foretell

finds

that its crowd outperforms historical projections.

Personally, I ended up on place #5 out of 646 on the

first season’s leaderboard

, and my team, Samotsvety Forecasting, comprised out of Eli Lifland, Misha Yagudin and myself...
A new acausal trading platform: RobinShould
by Matthew Barnett — 2021-04-01T16:56:07.488Z

After receiving overwhelming demand, we're excited to announce our new platform for conducting multiverse wide trades with other civilizations: RobinShould.

At RobinShould, we believe in democratizing acausal trading and making it freely available to all. Our simple interface allows you to purchase products from our wide selection of alien vendors. We have included an example trade below.

We currently only accept payment in paperclips and Ethereum, but we will soon be adding Bitcoin, Tether and tiny molecular smiley-faces as payment options.

How RobinShould works

We care a lot about ensuring that all of our customers feel satisfied knowing that their trades go through. Since the vendors on our platform have no causal connection to our universe, building RobinShould has proven quite tricky.

Fortunately, you can have confidence in our patented superintelligent prediction technology. Whenever you purchase a product through RobinShould, an amount of that product is manufactured in another universe by an alien civilization. The price for each product was determined by simulating evolution on other planets and modeling alien utility functions and decisionmaking.

In fact, our prediction technology is now so advanced that mesa-optimizers appear regularly in our prediction software. Apply to our data science team if you would like to help us clean these mesa-optimizers from our computer systems. (Seriously they're taking over our facility, please help.)

Disclaimer: In response to unprecedented volatility and clearinghouse requirements, we are temporarily restricting trades for GAME stock (Grabby Aliens Model Enterprises). Please note that this decision was made with the best intentions and your interests at heart. It was not at all an attempt to signal-boost Daniel Martin and Robin Hanson's website, or to protect our financial ally, Moloch.
Covid 4/1: Vaccine Passports
by Zvi — 2021-04-01T12:10:01.814Z

Writer’s Note: This is being posted on April 1, which is April Fools Day. To avoid all ‘is this the fake out?’ issues, this post does not contain any April Fools material. The fools contained herein are the usual, regularly scheduled fools we talk about every week. The official April Fool this year is whoever accidentally ruined 15 million doses of the Johnson & Johnson vaccine by mixing in the wrong ingredients.

This week started a big debate over vaccine passports. States including New York are deploying systems that let people prove they have been vaccinated, or document recent negative tests. Naturally, lots of people are outraged by this attempt to create a public record and provide information that helps people make better decisions. Thus, the long middle section where I go over all the objections I can come up with against this proposal. Some of them are legitimate.

Meanwhile, the overall arc remains the same. Things are getting worse again, and it’s a race to see how long it will take vaccinations to catch up to the problem and get us headed in the right direction again.

Let’s run the numbers.

The Numbers

Predictions

Last week’s prediction: Positivity rate of 4.9% (up 0.3%) and deaths fall by 7%.

Result:

Deaths only falling 0.4% is rather scary, and seems like strong evidence that the new strains are deadlier if it isn’t a data artifact. The data from Wikipedia suggests that it is indeed a data artifact, and deaths dropped at roughly the rate predicted, but there’s reason to suspect that result is a data artifact given what’s happening in the West. It’s all rather worrisome, and strong evidence that the new strains are sufficiently deadlier to make up for an increasingly youthful set of people being infected.

We also need to mention that the positivity rate went from 4.6% to 4.9%, while the same data source is claiming 11.8% fewer tests but 12.1% more cases, which really, really does not compute. Those three facts can’t all be true at once.

Johns Hopkins has a 4.8% positivity rate at the moment, so I’m going to assume that the positivity rate is accurate and the number of cases or tests has some error or data artifact in it that’s being corrected for the positivity rate calculation. In particular, I’m going to assume tests didn’t actually fall another 11.8% last week.

Prediction for next week: Positivity rate of 5.3% (up 0.4%)...
Center for Applied Postrationality: An Update
by Pee Doom — 2021-04-01T08:13:53.221Z

Previous post: https://www.lesswrong.com/posts/7X3BWzAt62yvvtyQX/announcing-the-center-for-applied-postrationality

Content Notice: Mild to Moderate NSFW language, Defective Altruism, Infoblessings

A lot of progress has been made here at CFAP in the past two years.

After posting on the SL5 newsgroup, a shadowy cryptobillionaire reached out. They not just met but drastically exceeded our funding goal. We are in the process of minting a commemorative impact certificate NFT to thank them for the monumental longterm value this will undoubtedly bring to the world.

Unfortunately, after reading a professional investing subreddit, we currently have much of our funds tied up in a superrational coordination experiment that is going very well. Therefore, we will need additional money if we're to reach our stretch goal of purchasing a yacht to live out of. We need the money in order to party all of the time on a huge boat, and you need to feel a sense of meaning in your life.

As evidence of our fiduciary effectiveness, here is a list of some of the most exciting projects we have initiated.

A large shift in our research direction took place. We noticed we were hitting diminishing marginal returns perfecting the art of mental masturbation. Many of our facilitators started branching out into areas of embodiment and emotion. This has led to developing a new and innovative practice in the art of embodied relationality, called Circle Jerking.
After a whole bunch of Circle Jerking, some teachers left to start a new monastery called WOOD (Wisdom, Onanism, Originality, and Disestablishmentarianism). Their theory of change is that the problems of the world are due to humans thinking, and therefore they will medibate in order to stop thinking. Here at CFAP, we support any and all efforts to not think.
We were surprised to learn that the General Sementics movement had covered so much previous intellectual ground. We bought a copy of Science and Sanitation and are in the process of reading it whenever we're going #2. We had to tear many of the pages out for toilet paper in March 2020, sadly. Not sure why but there was a shortage.
Getting swole. While practicing Original Seeing, we noticed many attendees are pretty scrawny and low-T. We conducted an extensive survey to ask them if they even lifted, and were shocked to learn how many didn't. So we started buying gym memberships en masse t

...