trentmkelly/LessWrong-43k huggingface dataset:
The Gallery for Painting Transformations - A GPT-3 Analogy
by Robert_AIZI — 2023-01-19T23:32:55.994Z
Link: https://aizi.substack.com/p/the-gallery-for-painting-transformations

[Target audience: Me from a month ago, people who want a sense of what a transformer is doing on a non-technical level, and people who want to chunk their understanding of transformers.]

Imagine a special art gallery, the Gallery for Painting Transformations (GPT). It takes in a sentence and makes a simple set of paintings to represent it, at first hardly more than stock images. Each morning, a team of artists come in and add to the paintings, and each evening, a snapchat-style filter is applied to each painting individually. Thanks to these efforts, over time the paintings in the gallery grow in meaning and detail, ultimately containing enough information to predict a painting which would be a good addition to the gallery.

This is how GPT-3 works, or at least (I claim) a reasonable analogy for it. In this post I want to flesh out this analogy, with the end result that even non-ML people can get a sense of what large language models are doing. I have tried to be as accurate in how I analogize things as possible, with the notable exception that my examples containing semantic meaning are a lot more human-comprehensible than what GPT-3 is doing, and in reality everything it does looks like “perform a seemingly-random vector operation”. Any mistakes are intentional creative flourishes my own[1].

The Paintings

The gallery holds 2048(=nctx)[2] paintings, which represent the model’s context window (the prompt you feed into GPT-3). In GPT-3, each word is represented by a 12288(=dmodel)-dimensional vector. Fortuitously, that’s exactly enough dimensions to make a 64-by-64 pixel RGB image!

The gallery first opens when a user feeds in a prompt, and each token (tokens≈words) in the prompt is turned into a painting which looks like a generic stock image, the “token embedding”. If you feed in more or less than 2048 tokens, the message is trimmed or padded with a special token.

How the gallery would look when it opens if the prompt was “a bear eats apples”, consisting of stock images for each word and positional embeddings in the bottom-left of each painting.

The gallery itself has no set order, there are simply paintings that can be viewed in any order. Instead, since word order is sometimes important in sentences, the gallery also paints on a little indicator to denote sentence position, the “position embedding”. In the previous image I showed this as drawing a little numb...
AGI safety field building projects I’d like to see
by Severin T. Seehrich — 2023-01-19T22:40:37.284Z

This list of field building ideas is inspired by

Akash Wasil

’s and

Ryan Kidd

’s similar lists. And just as the projects on those lists, these projects rely on people with specific skills and field knowledge to be executed well.

None of these ideas are developed by me exclusively; they are a result of the CanAIries Winter Getaway, a 2-week-long, Unconference-style AGI safety retreat I organized in December 2022.

Events

Organize a global AGI safety conference

This should be self-explanatory: It is odd that we still don’t have an AGI safety conference that allows for networking and lends the field credibility.

There are a number of versions of this that might make sense:

an EAG-style conference for people already in the community to network
an academic-style conference engaging CS and adjacent academia
an industry-heavy conference (maybe sponsored by AI orgs?)
a virtual next-steps conference, e.g. for AGISF participants

Some people have tried this out at a local level:

https://aisic2022.net.technion.ac.il

(If you decide to work on this:

www.aisafety.global

is available via

EA domains

, contact hello@alignment.dev)

Organize AGI safety professionals retreats

As far as I can see, most current AGI safety retreats are optimized for junior researchers: Networking and learning opportunities for students and young professionals. Conferences with their focus on talks and 1-on-1s are useful for transferring knowledge, but don’t offer the extensive ideation that a retreat focused on workshops and discussion rounds could.

Organizing a focused retreat for up to 60-80 senior researchers to debate the latest state of alignment research might be very valuable for memetic cross-pollination between approaches, organizations, and continents. It might also make sense to do this during work days, so that peoples’ employers can send them. I suspect that the optimal mix of participants would be around 80% researchers, and the rest funders, decisionmakers, and the most influential field builders.

Information infrastructure

Start an umbrella AGI safety non-profit organization in a country where there is none

This would make it easier for people to join AGI safety research, and could offer a central exchange hub. Some functions of such an org could include:

Serving as an employer of records for independent AGI safety researchers.
Providing a central point for discussions, cowo

...
Extensionality and the univalence axiom of type theory
by Thomas Kehrenberg — 2023-01-19T22:36:26.571Z

This is the final part of my introduction to dependent type theory. The big theme of this article is equality though that may not be immediately obvious.

Let’s start by finally discussing function extensionality and propositional extensionality.

What’s the deal with extensionality?

Whenever we define concepts, there are basically two ways to do it: extensionally and intensionally. An extensional definition simply lists all the objects that fall under the concept to be defined. So, if I was defining what a country is, I would simply give an exhaustive list of all countries. An intensional definition, on the other hand, gives you the rules for identifying objects falling under the concept. So, an intensional definition of a country might say thing like, “it’s a distinct part of the world that has its own government.”[1]

(As an aside, if you give a definition to a fellow human, it’s probably best to mix both kinds of definitions: “a country is a distinct part of the world that has its own government, like for example, France or Japan.”)

We can naturally extend this distinction in definition styles to the question of equality. For example, consider how the equality of two sets is defined in ZFC set theory:

∀A∀B((∀x(x∈A⇔x∈B))⇒(A=B)) .

In words: two sets are equal if they have exactly the same elements. We could imagine an machine that checks all elements one by one and determines whether they’re the same in both sets, and finally tells you whether the two sets are equal.

We say that this kind of equality is extensional, because all that matters here are the contents of the sets, and we don’t care which rules were used to generate the sets. Whether the set {2,3,5} was generated from the first three prime numbers, or the three Fibonacci numbers after (0,1,1), doesn’t matter for the question of equality. This makes ZFC set theory a purely extensional theory.

However, as you may remember, this is not how we defined equality in our type theory. With the way we defined equality, two functions can be non-equal even if they return exactly the same values for all inputs. I appreciate that this sounds very strange, but it’s basically due to how equality works in lambda calculus. Lambda calculus gives us these judgmental equalities (also called alpha-, beta- and eta-equivalence[2], respectively):

(λx.t)≡(λy.t)(λx.t)(u)≡t[u/x]f≡(λx.f(x))

where the notation in ...
The spiritual benefits of material progress
by jasoncrawford — 2023-01-19T21:35:51.225Z
Link: https://rootsofprogress.org/the-spiritual-benefits-of-material-progress

The Industrial Revolution gave us abundance and comfort—but what did it do to our souls?

Recently my progress colleague Alec Stapp responded to a Twitter thread disparaging the Industrial Revolution for what it “did to humanity.” Alec’s response was basically that abundance is good, and I agree. But a few people (e.g., Michael Curzi, Jon Stokes) criticized this response for basically reasserting the material benefits and seeming to ignore the non-materialistic concerns in the original thread.

So, let’s talk about spiritual values—that is, emotional, intellectual, social, and other psychological values—and what industrial progress has done for or to them.

The charges

The original thread, from a pseudonymous account, was a cris de coeur. Let me extract its core arguments. It charges that the Industrial Revolution:

“stole vocation and purpose” from those doing hand crafts and put them on assembly lines making things they are “disconnected” from and “probably can’t afford”
took people from rural areas and centralized them in cities, “in ugly buildings”
affected education, “standardizing young minds” and “attempting uniformity”

Summing up, she acknowledges that raising living standards was good, but laments that “no one thought to apply the brakes” and that our lives are now “framed by consumerism and commerce.”

Note, this is not from a left-wing environmentalist or a degrowther: she goes on to say that “the unmooring of humanity from its eternal purpose” is “anti-Christ.” This is a religious conservative criticism of progress.

First I’ll address the specific charges, and then I will step back to consider the wider question of how industrialization has affected our spiritual life.

Vocation and purpose

The first charge is against the transition from cottage industry to the factory system. To steelman this, it’s true that this transition took away a certain style of working, and that many people were unhappy about it. Workers in general disliked being supervised by a foreman and thus losing their autonomy to do their work when and how they liked, being required to work longer hours with fewer breaks, and having to commute to a factory rather than work from home. Master craftsmen in particular felt their skills were devalued, as the manufacture of goods was split into incremental steps that began to be performed by unskilled workers and/or by machines. This was only exacerbated...
Announcing Cavendish Labs
by derikk — 2023-01-19T20:15:09.035Z
Link: https://forum.effectivealtruism.org/posts/xBeqaWEJfWZv8ALWn/announcing-cavendish-labs

We’re excited to announce

Cavendish Labs

, a new research institute in Vermont focused on AI safety and pandemic prevention! We’re founding a community of researchers who will live together and work on the world’s most pressing problems.

Uh, why Vermont?

It’s beautiful; it has one of the cheapest costs of living in the United States; there’s lots of great people; it’s only a few hours away from Boston, NYC, and Montreal. There’s even

a train

that goes there from Washington D.C.! A few of us briefly lived in Vermont during the pandemic, and we found it to be a fantastic place to live, think, and work. Each season brings with it a new kind of beauty to the hills. There are no barriers to a relaxing walk in the woods. There's practically no light pollution, so the cosmos is waiting outside the door whenever you need inspiration.

A view of Cavendish village; the town offices and fire station are on the left.

What are you going to be researching?

We have a few research interests:

1. AI Alignment. How do we make sure that AI does what we want? We’ve spent some time thinking about

ELK

and

inverse scaling

; however, we think that AGI will most likely be achieved through some sort of model-based RL framework, so that is our current focus. For instance, we know how to induce provable guarantees of behavior in supervised learning; could we do something similar in RL?

2. Pandemic prevention. There’s been a lot of talk about the potential of

Far-UVC

for ambient disinfection. Understanding why it works on a molecular level, and whether it works safely, is key for developing broad-spectrum pandemic prevention tools.

3. Diagnostic development. We're interested in designing a low-cost and simple-to-use platform for

LAMP

reactions so that generalized diagnostic capabilities are more widespread. We envision a world where it is both cheap and easy to run a panel of tests so one can swiftly determine the exact virus behind an infection.

How’s this organized?

We'll be living and working on different floors of the same building—some combination of a small liberal arts college and research lab. To ensure we’re not too isolated, we’ll visit Boston at least once a month, and invite a rotating group of visitors to work with us, while maintaining collaborations with researchers around the world.

Sounds interesting!

We’re actively searching for collaborators in our areas of in...
Thoughts on refusing harmful requests to large language models
by William_S — 2023-01-19T19:49:22.989Z

https://twitter.com/antimatter15/status/1602469101854564352

Currently, large language models (

ChatGPT

,

Constitutional AI

) are trained to refuse to follow user requests that are considered inappropriate or harmful. This can be done by training on example strings of the form “User: inappropriate request AI: elaborate apology”

Proposal

Instead of training a language model to produce “elaborate apology” when it refuses to do an action, train it to produce a special sequence or token first “<SORRYDAVE>elaborate apology”. Strip the special sequence out before returning a response to the user (and never allow the user to include the special sequence in input).

Benefits

Can directly measure the probability of refusal for any output

Can refuse based on probability of producing <SORRYDAVE> instead of just sampling responses

Just take the product of the probability of all tokens in <SORRYDAVE>
When sampling responses from the model's probability distribution refusal is stochastic, a model could have 99% probability of refusing a request but you still get unlucky and have the model sample a completion that follows the request

Can monitor requests that produce high probability of refusal while still being followed, or users that produce those request

Can condition on not producing <SORRYDAVE> in order to override refusal behavior

Want this for redteaming, it seems important to understand what the model is capable of doing if the refusal mechanism is bypassed
Might want this for trusted users doing defensive applications

Could train model to have the same probability of refusal for semantically equivalent requests, to improve consistency

Possible downside

If someone has unfiltered access to the model, it becomes easier to disable refusals

Can address by still training model to refuse (maybe just on an important subset of requests) even if <SORRYDAVE> isn’t sampled, p(<SORRYDAVE>) is then a lower bound on the probability of refusal

Even with current approaches refusals might be easy to disable in this setting. If we want to be robust to this setting, instead of refusing we should train the model to produce "decoy answers" that are hard to distinguish from real answers but are wrong. This then increases the cost of using the model because the attacker would need to evaluate whether the answer is real or a decoy (but maybe still worth it for the attac

...
MA RMV Overloaded
by jefftk — 2023-01-19T16:40:01.632Z

My drivers license expires at the end of January, and renewal requires
an appointment. On 1/12 when I tried to make one I had a little over
two weeks (17d) before expiration. I should have left a larger
cushion, but the RMV says you

generally need
14d and it averaged

9d
in 2021 and 11d in 2022. The earliest appointment at the Boston
RMV was 2/23 (42d).

I filled out their contact
form asking if I could get an extension, and they replied:

Extensions are only granted during an in person renewal
appointment, we cannot offer an extension through email. As a reminder
renewal is available up to a full year in advance and up to 2 years
after expiration.

This morning I tried going in person right when they opened. Outside
they checked whether you had an appointment, and wouldn't let you in
for an ID without one:

Me: I tried to renew my license a few weeks in advance and the soonest
appointment it would offer me was nearly a month after it expires.

RMV: Sorry, you need an appointment. Try reloading the page to
see if there's a cancellation.

Me: I've been doing that, and the earliest it shows me is March.

RMV: That's not possible.

Me: Here's what I see:

RMV: Sorry, still you need an appointment. Try reloading the page
every hour to see if there's a cancellation.

When you go to check if there have been any cancellations you're
presented with a screen like:

There are 17 RMV locations within an hour drive of my house, and the
only way to check for cancellations is to manually click each one. I
can't ask it to order locations by "soonest available appointment" or
even "closest to my house". Checking all 17 right now, the soonest
option is 2/14 (26d) in Taunton. This is 43mi away and would be a
55min drive, except it's after my license expires so it would be a
2hr+ each way on public transit. Everything within an hour on public
transit is 3/2 (42d).

This is similar to what WBZ reported in
November:

When WBZ checked for appointments online, a "no appointments
available" screen popped up for RMV offices in Boston, Attleboro,
Lowell, Worcester and all points between. One turned up in Greenfield
four weeks out, which would take almost two hours to get to.

I'm not sure what I'll do about my renewal except keep checking back,
but if you need an appointment I'd recommend scheduling it well i...
"Heretical Thoughts on AI" by Eli Dourado
by DragonGod — 2023-01-19T16:11:56.567Z
Link: https://www.elidourado.com/p/heretical-thoughts-on-ai

Abstract

Eli Dourado presents the case for scepticism that AI will be economically transformative near term[1].

For a summary and or exploration of implications, skip to "My Take".

Introduction

Fool me once. In 1987, Robert Solow quipped, “You can see the computer age everywhere but in the productivity statistics.” Incredibly, this observation happened before the introduction of the commercial Internet and smartphones, and yet it holds to this day. Despite a brief spasm of total factor productivity growth from 1995 to 2005 (arguably due to the economic opening of China, not to digital technology), growth since then has been dismal. In productivity terms, for the United States, the smartphone era has been the most economically stagnant period of the last century. In some European countries, total factor productivity is actually declining.

Eli's Thesis

In particular, he advances the following sectors as areas AI will fail to revolutionise:

Housing

Most housing challenges are due to land use policy specifically

Housing factors through virtually all sectors of the economy

He points out that the internet did not break up the real estate agent cartel (despite his initial expectations to the contrary)

Energy

Regulatory hurdles to deployment
There are AI optimisation opportunities elsewhere in the energy pipeline, but the regulatory hurdles could bottleneck the economic productivity gains

Transportation

The issues with US transportation infrastructure have little to do with technology and are more regulatory in nature
As for energy, there are optimisation opportunities for digital tools, but the non-digital issues will be the bottleneck

Health

> The biggest gain from AI in medicine would be if it could help us get drugs to market at lower cost. The cost of clinical trials is out of control—up from $10,000 per patient to $500,000 per patient, according to STAT. The majority of this increase is due to industry dysfunction.

Synthesis:

I’ll stop there. OK, so that’s only four industries, but they are big ones. They are industries whose biggest bottlenecks weren’t addressed by computers, the Internet, and mobile devices. That is why broad-based economic stagnation has occurred in spite of impressive gains in IT.

If we don’t improve land use regulation, or remove the obstacles to deploying energy and transportation projects, or make clinical trials more cost-effective—if we don

...
Covid 1/19/23: Flipped Numbers
by Zvi — 2023-01-19T13:30:01.251Z

It was a quiet week. All the longer term news was as one would expect. That is good news.

The only surprise was that deaths jumped again while cases fell, rather than the other way around. On net, that too is good news.

If news remains this quiet for a while, I would be able to stop doing weekly Covid updates. Even more good news.

Executive Summary

Deaths unexpectedly up for a second week, should be peaking here.

Cases unexpectedly down a lot in Northeast, other regions likely will follow soon.

Estimate says China’s wave peaked at ~3x the size of last year’s Western wave.

Let’s run the numbers.

The Numbers

Predictions

Predictions from Last Week: 385k cases (+5%) and 3,100 deaths (-15%), which I noted had high uncertainty and I didn’t even remember that there was MLK Day – when I am adjusting for a future holiday, I explicitly note that.

Results: 335k cases (-8%) and 3,867 deaths (+5%), the reverse result.

Predictions for Next Week: 320k cases (-5%) and 3,600 deaths (-8%).

There were a number of states reporting obviously time-shifted results in both directions. I chose not to adjust any of them other than Florida’s 2-week cycle.

Thus, the rise in the Midwest is mostly due to Illinois and Indiana reporting all of last week as part of this week, but even with a one-sided fix we still see that the higher level of deaths maintained itself despite the holiday. The increase in deaths seems real, yet so was the decline in cases. I got this one backwards. So things were recently modestly worse, but they should get better from here.

Deaths

Cases

Physical World Modeling

Celine Gounder notes that we did not have a winter surge in hospitalizations or deaths, attributes this to immunity built up over time. That is in spite of XBB.1.5. She also has a thread of the latest evidence on boosters that is more of the same. There is a modest spike in deaths the last two weeks, but nothing that implies exponential problems, especially with the Northeast already seeing large case declines.

Even more than that, in the US states dominated by Omicron XBB.1.5, hospitalizations are already plateauing.

Meagan Kall goes over the latest UK technical briefing. BQ.1.5 has the replication edge over other variants, as we presumed. Here’s vaccine effectiveness, which remains substantial but may be down somewhat.

Bloom Lab speculates on f...