trentmkelly/LessWrong-43k huggingface dataset:
[AN #160]: Building AIs that learn and think like people
by Rohin Shah — 2021-08-13T17:10:04.335Z

Alignment Newsletter is a weekly publication with recent content relevant to AI alignment around the world. Find all Alignment Newsletter resources here. In particular, you can look through this spreadsheet of all summaries that have ever been in the newsletter.

Audio version here (may not be up yet).

Please note that while I work at DeepMind, this newsletter represents my personal views and not those of my employer.

HIGHLIGHTS

Building Machines That Learn and Think Like People (Brenden M. Lake et al) (summarized by Rohin): The core claim of this 2016 paper is that we should focus on building AI systems that work as flexibly as humans do. For example, a human can learn how to play the Atari game Frostbite in just a couple of hours, way faster than typical deep RL algorithms -- and in addition, after this they will likely be able to transfer zero-shot to new reward functions, such as “lose as quickly as possible”, “maximize the number of fish”, “beat the level with as little time to spare as possible”, and so on. How can we build AI systems that mimic this feat? Deep RL certainly doesn’t get us there. Similarly, while neural networks can learn to classify digits and characters with thousands of examples, humans can learn new characters from a single example, which then allows them to perform many different tasks such as classification, generation, parsing it into different pen strokes, etc. Since the paper was written neural nets have made progress on few-shot classification, but are still quite far from the flexibility that humans display.

You might reasonably object that humans have rich priors built from years of lived experience, as well as innate knowledge baked in by evolution; in contrast, a neural network has to learn from scratch. The authors agree: in their view, the challenge is how to imbue rich priors into artificial agents, so that they too can exhibit these impressive behaviors that humans show. Their preferred approach is to take inspiration from human learning and intelligence as much as possible. In this paper, they identify three main ingredients to recreate that flexibility, and provide an overview of the existing literature:

1. Developmental software: This refers to the basic capabilities that children have, even before they learn language. These are called “intuitive theories” in cognitive science; think of “i...
OpenAI Codex: First Impressions
by specbug — 2021-08-13T16:52:33.874Z
Link: https://sixeleven.in/codex

OpenAI organised a challenge to solve coding problems with the aid of an AI assistant. This is a review of the challenge, and first impressions on working with an AI pair-programmer.

OpenAI Codex

OpenAI is an AI research and development company. You might have heard some buzz about one of its products: GPT-3. GPT-3 is a language model that can generate human-like text. It can be used for chatting, text auto-completion, text summarisation, grammar correction, translation, etc.

Checkout OpenAI API to access the playground.

Codex is a descendant of GPT-3, trained on natural language data and publicly available source-codes (e.g. from public GitHub repos). Codex translates a natural language prompt to code. It is the very model that powers GitHub Copilot — an AI pair-programmer (checkout the site for demos, it is fascinating).

Credits: OpenAI

OpenAI recently released an API to access Codex (in beta). The demos attached with the release were a cause for consternation. Codex is proficient in a dozen (programming) languages. It can be used for code generation, refactoring, autocompletion, transpilation (translating source-code b/w languages), code explanation, etc. To show off Codex, OpenAI recently organised a challenge.

The Challenge

The challenge was to solve a series of (five) programming puzzles in Python. The only twist — you can use Codex as a pair-programmer. It was a time-judged competition, with a temporal cap. Not surprisingly, Codex itself was a participant (not just as a helper)!

The problems were simple. ~830 "people" (Codex included) were able to solve all five of them. I had to solve the first two challenges manually (OpenAI server issues). "Had to" because it was a race against time (& top 500 win an OpenAI t-shirt). For the other three, however, I was able to call in the cavalry (it was pretty climactic).

The novel experience of watching an AI auto-generate code is amazing. Just type a docstring — describing the procedure — and watch the code develop. If you're an old-time programmer, you'll get the notion when you experience it.

Illustration

I've illustrated one problem statement where I used Codex to generate a solution.

PROBLEM
Parse the given Python source code and return the list of full-qualified paths for all imported symbols, sorted in ascending lexicographic order.

CONSTRAINTS
The input will not contain any wildcard imports (from ... import *). Ignor

...
How would the Scaling Hypothesis change things?
by Aryeh Englander — 2021-08-13T15:42:03.449Z

The Scaling Hypothesis roughly says that current Deep Learning techniques, given ever more computing power, data, and perhaps some relatively minor improvements, will scale all the way to human-level AI and beyond. Let's suppose for the sake of argument that the Scaling Hypothesis is correct. How would that change your forecasts or perspectives on anything related to the future of AI?

Would your forecasts for AI timelines shorten significantly?
Would your forecasts change for the probability of AI-caused global catastrophic / existential risks?
Would your focus of research or interests change at all?
Would it change your general perspective on the current and/or future of AI?
Would it change any forecasts or perspectives of yours in areas that aren't AI themselves but which might be affected by AI?
Would it perhaps even change your perspective on life?
Coase's "Nature of the Firm" on Polyamory
by 1a3orn — 2021-08-13T13:15:47.709Z

It occurred to me that Coase's views on The Nature of the Firm might help explain why polyamory in its modern form is not particularly common or popular.

That sentence might be enough for you to grok what I'm getting at, and honestly that's the form in which the thought first came to be, but nevertheless let me try to explain what I mean.

Coase's original essay -- and the whole body of thought proceeding from it -- seeks to answer why coorporations / firms emerge. That is, it seeks to ask where people are hired for indefinite periods of time, for less-precisely-defined work rather than contracted for definite amounts of time, for precisely-defined work. If you believe in a strong version of efficiency of markets, you might expect it to almost always be cheaper to contract than to hire, because the allocation of resources by a market should be more efficient than the allocation of resources within a non-market organization. Why wouldn't my software company just hire a contractor for everything they needed to be done, rather than relying on me, a non-expert in many things they would like me to do?

The answer, of course, is that there are transaction costs to using the market. There's a cost to searching for and finding a trustworthy contractor, which is avoided by keeping me around. There's the cost of a stronger asymmetry of information and asymmetry of benefit in the case of the contractor, which makes me a little more trustworthy because I'm going to be stuck with the code I write for a longer period of time. And so on and so forth.

Polyamory seems like an attempt to unbundle a group of frequently-bundled relationship goods in a way analogous to how contracting different workers can be an attempt to unbundle a group of frequently-bundled commercial goods. Vis, in polyamory you frequently unbundle from each other the following:

- Sexual satisfaction

- Intellectual companionship

- Long-term companionship and dependency

- Childbearing and rearing

Or even decompose these further: i.e., different flavors of sex and different flavors of companionship. But finding someone for each of these involves transaction costs. So you have the costs of searching for and finding trustworthy people in all these roles. And you have the stronger asymmetry of information and of benefit because of the more ephemeral nature of the relationships.

This is really just a rephrase of things I kno...
A review of "Agents and Devices"
by adamShimi — 2021-08-13T08:42:40.637Z

In the small group of people who care about deconfusing goal-directedness and like Daniel Dennett's intentional stance, there's one paper everyone mentions: Agents and Devices, by Orseau, McGill and Legg. It's also pretty much the only paper ever mentioned on that topic.

Given that this paper seems to formalize the intentional stance enough for running experiments on it, and how I already wrote that a formalized intentional stance sounds like the most promising deconfusion of goal-directedness, I thought it was important to review it to clarify what I think is missing.

Spoiler: I don't find their proposal satisfying as a formalization of the intentional stance, but it's a great first step in making it more concrete and objectionable.

Thanks to Victoria Krakovna and Ramana Kumar who always ask me how my current attempt at deconfusing goal-directedness is related to this paper.

Summary

What's this paper about?

Basically, the author create two classes of hypotheses:

devices which are just programs with the right interface,
and agents which are optimal (technically ϵ-greedy optimal) policies for some reward functions.

These classes are combined into mixtures through priors -- speed prior over the devices and simplicity prior (technically switching prior) over the rewards/goals respectively. Next the two mixtures are combined in equal weights to form one mixture, which is used as prior on applying Bayes rules to some observed trajectory.

So after getting a trajectory, the result is a posterior on our mixture of agents and devices, which gives us both which of agent and device is more probable, but also the more probably hypothesis among those classes.

They actually run some experiments in a simple maze, which give the expected results: following walls and running in circles are considered as devices, following a goal or switching goals are considered as agents, and random behavior is considered as neither.

What I like

As a first attempt to make the intentional stance formal enough to experiment on and criticize, the paper does his job perfectly.

Here are also some ideas I found particularly interesting:

(The switching prior) I hadn't heard of it, but it sounds like a pretty neat way to add changes of goals to the formalization.
(Using the speed prior to deal with interpreters) I need to read this part in more detail, but the choice of a speed prior over the simplicity prior

...
Is microCOVID "badness" superlinear?
by Optimization Process — 2021-08-13T06:37:49.478Z

That is: if I have a choice between 200 microCOVIDs now, or 100 now and 100 next month, does it matter which one I pick? (For bonus points: how big is the difference? And does the answer change if it's 20000 vs 10000/10000?)

(Alternatively: what keywords would I search to find studies on this? I'm sure data exists, but I'm coming up empty on Google.)

On the one hand, straightforwardly no: in my cohort of infinite parallel selves, every microCOVID we take on makes almost exactly one-in-a-million of us get sick (at least until a substantial fraction of us are sick, i.e. I've taken on tens of thousands of microCOVIDs), so every microCOVID is equally bad. (Something something axiom of independence, and we're all VNM-rational agents, right?)

On the other hand, maybe yes, because of complicated biology reasons, where if you inhale 1M viruses over the course of a year, at some point you'll probably get COVID, but if you snort them all at once then you're, I dunno, effectively giving the infection a head start of several doubling times, and you're gonna get COVID real bad.
Technical Predictions Related to AI Safety
by lsusr — 2021-08-13T00:29:43.817Z

If you want to create a policy of nuclear safety then you need to understand:

Manufacturing nuclear weapons requires access to uranium.

Uranium enrichment is a big, slow activity that's difficult to hide from enemy nation-states.

Nuclear weapons are so expensive only major governments can afford them.

Missiles are the best way to deploy nuclear weapons.

If you want to figure out a strategy for bioweapon safety then you need to understand that infectious biological agents are an asymmetric weapon. They are most likely to be intentionally deployed by non-state actors or as a last-ditch retaliation by states on the verge of collapse. A policy of bioweapon safety should defend against these attack vectors.

An analysis of AI safety must start by figuring out the capabilities and limitations of artificial intelligence.

Software: Past, Present and Future

There are three kinds of software.

Traditional, declarative, hard-coded or procedural software is hand-coded. Though traditional software has been around for many decades, I consider it to have taken off with the proliferation of personal computers in the 1980s.

Machine learning (ML) or deep learning minimize an error function by throwing mountains of data at a high-parameter model. ML took off recently with the application of Nvidia GPUs to the training of neural networks.

Bayesian reasoning machines and artificial general intelligence (AGI) are science fiction. They won't be forever.

Traditional Software

Traditional software does exactly what a human being tells it to. Traditional software is based around a single-CPU von Neumann architecture. Modern computers parallelize software across multiple CPUs via multthreading or multiprocessing. Multiprocessing works fine when you're running separate applications like LibreOffice Writer, LibreOffice Calc and LibreOffice Impress. Multiprocessing works badly when you're trying to throw more compute at a single monolithic program via hand-coded software. You can reduce serial dependencies by writing your software in a functional paradigm, but even the best hand-coded stateless software tends to eventually hit a serial bottleneck.

Related to the serial bottleneck, the other problem with traditional software is human programmers are limited by how much complexity we can juggle. If no human can hold more than seven items in his or her working memory then a software program...
Power-seeking for successive choices
by adamShimi — 2021-08-12T20:37:53.194Z

From single choice to successive choices

Alex Turner (and collaborators) have done a great work deconfusing instrumental convergence for a single choice. As a quick summary (read the paper and sequence if you want more details), they show that for any distribution over reward functions, if there are more "options" available after action 1 than after action 2, then most of the orbit of the distribution (the set of distributions induced by applying any permutation on the MDP, which thus permutes the initial distribution) has optimal policies that do action 1.

(A subtlety that matters is the meaning of optimal here: the result applies for all distributions only for average-step reward, which never cares about transient rewards and only value what happens in the loops. That being said, the distributions for which this doesn't generalize to Blackwell optimality are a a set of zero measure -- where Blackwell optimality is the "classic" RL notion of optimality with discount factor close to 1, which thus takes into account transient rewards)

So it's not so much telling us something about a specific distribution over reward functions (which may incentivize the action with less options) but about its orbit. It's giving teeth to the fragility of value with regard to instrumental convergence: we have to hit a target in the orbit that represents less than half of it.

Alex's recent combinatorial generalizations considerably strengthen this point: if action 1 allows n times more options than action 2, at least nn+1 of the orbit has optimal policies taking action 1.

For big enough n, the target to hit for no instrumental convergence shrinks to vanishingly small.

In this post, I adapt these result for the case of successive choices: what is the proportion of the orbit for which it is optimal to take the first k instrumental convergent choices? In addition to symmetry, additional information is required, in the form of the intersection of the sets of options for successive non-instrumentally convergent choices.

(Note: technically power-seeking and instrumental convergence are slightly different, but I'll use the more common -- in this context -- power-seeking by simplicity)

Thanks to Alex Turner for answering my many questions, pointing out a problem in a previous draft, and for all the work that gave me the tools for this little contribution.

Two examples

Can ...
Obesity Epidemic Explained in 0.9 Subway Cookies
by tinyanon — 2021-08-12T15:45:22.431Z

If we're going to be talking about the obesity epidemic, it might be useful to actually look at some numbers to see how much change it takes to explain the increase in average weight. Given people keep talking about the 1980 inflection point I'm going to use a typical person from that time period as my starting point and track him for five years with minor behavioral changes.

Starting as a 1980's 165lb 5'8 35 y/o man who goes out swing dancing twice a week

As you can see, it doesn't take much behavioral change to account for the change in average weight we've seen over the last 40 years. A little more sedentary or a little less satiating food per calorie which would lead to eating more. We don't need fancy contamination theories.

I've seen some really smart people overcomplicate this. Eliezer made a Facebook post asking for advice losing weight saying not to bother telling him to eat less because it doesn't work. A few years later he made a LessWrong post saying he lost weight by eating less. The simple things are the answer, the question is just how to eat less and move more without making yourself miserable.

Some things that will help if you are trying to lose weight and find that you're hungry and miserable all the time:

High fiber foods are often very satiating per calorie (veggies)
Low calorie density per volume/weight tend to be very satiating per calorie (veggies, lean meats)
Sugary and oily food tend to have high calorie density and shouldn't make up the bulk of your diet
Nonnutritive sweeteners can help you satisfy your sweet tooth without breaking the bank on calories
You don't have to lift/run, you can find the activity that you find fun
Time restricted feeding (often called intermittent fasting) tends to vastly increase satiety
You don't have to track calories forever, you can track things for a few months to build more of an intuition for calorie content and then stop
Pick habits you'll actually stick to. No point in driving yourself crazy for 6 days until you snap and eat a whole cake on day 7
Jaynes-Cox Probability: Are plausibilities objective?
by  — 2021-08-12T14:23:23.555Z

Is the objectivity of plausibility assignments assumed in the Jaynes-Cox formulation of probability theory?

This is what I mean by “the objectivity of plausibility assignments”:

A and B are propositions. (A|B) is the plausibility of A given that B is true and is represented with a real number as a result of our desiderata. Is the quantity (A|B) uniquely determined by A and B?

If this is the case, is this one of the assumptions that we make (implicitly or explicitly) or can this be derived from our desiderata?

If not, then in what sense is the plausibility of A given B objective?

Thank you
Covid 8/12: The Worst Is Over
by Zvi — 2021-08-12T13:30:00.842Z

Good news, everyone! Andrew Cuomo has resigned, and Andrew Cuomo is the worst.

I will damn well take it, because it’s not like he doesn’t also deserve to resign in disgrace for the stuff that officially got him, and again, also to say it for what is hopefully one last time, Andrew Cuomo Is The Worst. Hence, The Worst Is Over. Sing it high, sing it low. (HT: Meme source)

The title this week does not as reliably or fully refer to the Delta variant or the Covid-19 pandemic. Things are still steadily getting worse. But the turning point is plausibly in sight, as case growth slows, and I doubt we have more than one doubling left before things peak.

Main event this week was continued arguments over mandates, both for vaccines and for masks and other NPIs. I’m making one last attempt this week to explain my reasoning on vaccination mandates, as I continue to get people disagreeing for a variety of reasons, and disambiguating the disagreements seems worthwhile; I tested it out in the comments last week and it seemed productive.

Let’s run the numbers.

The Numbers

Predictions

Prediction from last week: 855,000 cases (+45%) and 4,100 deaths (+40%).

Result: 744k cases (+26%) and 3,725 deaths (+29%).

The case numbers are very encouraging. They’re still increasing, and there’s always the chance this was a data fluctuation that will be actively undone, but it probably wasn’t and represents either the control system kicking in or us nearing a natural peak. I still expect a similar increase next week, but I’d estimate a 35% chance that within two weeks we get about as high case counts as we’re going to see in this wave, and 55% it happens within three weeks.

I’m still predicting a +35% rise in deaths, as the cases from the last few weeks make their way through the system, and hope to be pleasantly surprised.

Prediction for next week: 900k cases (+21%) and 5,028 deaths (+35%).

Deaths and Cases

Grouping these together, because there’s a combined mystery to solve.

Here’s the case numbers.

Date	WEST	MIDWEST	SOUTH	NORTHEAST	TOTAL
Jun 17-Jun 23	23,854	12,801	26,456	6,464	69,575
Jun 24-Jun 30	23,246	14,521	31,773	6,388	75,928
Jul 1-Jul 7	27,413	17,460	40,031	7,065	91,969
Jul 8-Jul 14	45,338	27,544	68,129	11,368	152,379
Jul 15-Jul 21	65,913	39,634	116,933	19,076	241,556
Jul 22-Jul 28	94,429	60,502	205,992	31,073	391,996
Jul 29-Aug 4	131,197	86,394	323,063	48,773	589,427
Aug 5-Aug 11	157,553	110,978	409

...
New GPT-3 competitor
by Quintin Pope — 2021-08-12T07:05:49.074Z

AI21 has trained a new language model, Jurassic-1, whose largest version has 178 billion parameters (GPT-3 had 175 billion). This paper gives limited technical details.

There already were several models that used far more parameters than GPT-3, but they were either mixture of expert models or only word embeddings. They required much less compute to train/use, but were less powerful than a dense transformer like GPT-3 or the new Jurassic-1.

The interesting thing about Jurassic-1 is that it really doesn’t go much beyond GPT-3. It has a larger vocabulary and slightly optimized architecture. Jurassic-1 only has a bit more parameters than GPT-3, whereas prior trends indicated that any GPT-3 successor would use at least an order of magnitude more parameters. Since GPT-3, much work has gone towards improving transformer architecture (e.g., linear time self attention and neural architecture search), but little of that is visible in Jurassic-1. Maybe companies don’t think it’s economically viable to scale beyond GPT-3 or run many experiments with different architectures at that scale?

Also, Jurassic-1 is a unidirectional model, like GPT-3 (meaning it's forced to process text from left-to-right). This means GPT-3 can only process a given word using the context provided by the previous words. This causes unidirectional models problems for most tasks other than text generation. For example, other than GPT-3, all the top models in the SuperGLUE benchmark leaderboard are bidirectional models. It's interesting AI21 chose to compete with OpenAI using a model that provides the same class of service (text generation) as GPT-3, rather than specialize in, e.g., text classification, where a bidirectional model would be better.