trentmkelly/LessWrong-43k huggingface dataset:
AI #71: Farewell to Chevron
by Zvi — 2024-07-04T13:40:05.905Z

Chevron deference is no more. How will this impact AI regulation?

The obvious answer is it is now much harder for us to ‘muddle through via existing laws and regulations until we learn more,’ because the court narrowed our affordances to do that. And similarly, if and when Congress does pass bills regulating AI, they are going to need to ‘lock in’ more decisions and grant more explicit authority, to avoid court challenges. The argument against state regulations is similarly weaker now.

Similar logic also applies outside of AI. I am overall happy about overturning Chevron and I believe it was the right decision, but ‘Congress decides to step up and do its job now’ is not in the cards. We should be very careful what we have wished for, and perhaps a bit burdened by what has been.

The AI world continues to otherwise be quiet. I am sure you will find other news.

Table of Contents

Introduction.

Table of Contents.

Language Models Offer Mundane Utility. How will word get out?

Language Models Don’t Offer Mundane Utility. Ask not what you cannot do.

Man in the Arena. Why is Claude Sonnet 3.5 not at the top of the Arena ratings?

Fun With Image Generation. A map of your options.

Deepfaketown and Botpocalypse Soon. How often do you need to catch them?

They Took Our Jobs. The torture of office culture is now available for LLMs.

The Art of the Jailbreak. Rather than getting harder, it might be getting easier.

Get Involved. NYC space, Vienna happy hour, work with Bengio, evals, 80k hours.

Introducing. Mixture of experts becomes mixture of model sizes.

In Other AI News. Pixel screenshots as the true opt-in Microsoft Recall.

Quiet Speculations. People are hard to impress.

The Quest for Sane Regulation. SB 1047 bad faith attacks continue.

Chevron Overturned. A nation of laws. Whatever shall we do?

The Week in Audio. Carl Shulman on 80k hours and several others.

Oh Anthropic. You also get a nondisparagement agreement.

Open Weights Are Unsafe and Nothing Can Fix This. Says Lawrence Lessig.

Rhetorical Innovation. You are here.

Aligning a Smarter Than Human Intelligence is Difficult. Fix your own mistakes?

People Are Worried About AI Killing Everyone. The path of increased risks.

Other People Are Not As Worried About AI Killing Everyone. Feel no AGI.

The Lighter Side. Don’t. I said don’t.

Language Model

...
The Dumbification of our smart screens
by Itay Dreyfus — 2024-07-04T06:32:36.672Z
Link: https://productidentity.co/p/the-dumbification-of-our-smart-screens

Here we are at the peak of an eternal technology era.

AI agents are on their way. Driverless taxis are making a

debut

. 4G is an old fashion. We’re heading to a seemingly great utopia.

Perhaps the greatest invention of this era is the smartphone. The small pocket device has evolved into various shapes while computerizing many aspects of our lives. But once saviors, our smart screens have become a source of modern suffering. We have been hitting the wall too many times, and it seems we had enough.

We have arrived at a hard-screen diet era.

In

The ‘Enshittification’ of TikTok

, Cory Doctorow shares his thesis of how platforms become useless by manipulating their audiences, each one at a time. First, they give massive attention for zero cost to newcomers; then they shift focus to their business customers; eventually, they turn against everyone to satisfy shareholders.

Search Amazon to find sponsored ads instead of what you’re looking for. Open Instagram to see what people you don’t follow are doing. Tweet an external link to be ranked as spam. Visit Facebook for a digital trip in a gigantic Times Square.

It’s a chain of deception—or perhaps the greatest scheme of the digital age. You don’t get what you search for. You don’t get connected with your friends. Your online browsing becomes aimless. Everything has become eshittified.

A similar, yet not identical trick is tied to what we used to call a cellphone.

Smartphones should have made us smarter. Or at least that’s what we thought. In the beginning, it was exciting. Suddenly heavy lifting became effortless in the palm of our hands: taking photos, exchanging emails, playing games, or turning on a flashlight.

What more could we ask for?

The rational mind has followed the linear path of progressive overload in tech for quite some time. Like in the gym, the more you work out, the heavier weights you lift to grow your muscles. But does using more technology over time yield the same results? As it appears, it only weakens our mental muscle. Pulling heavyweights of information is so exhausting that it hurts.

We indeed became dependent on our screens as

our digital footprint increased

. But after years of drowning in the ocean of allegedly tech advancements, there’s an emerging trend that might cross the boundaries of itself to become a positive, healthier norm.

In

What’s the future of AI hardware?

I wrote:

Yet sometimes it feels like

...
Introduction to French AI Policy
by Lucie Philippon — 2024-07-04T03:39:45.273Z

This post was written as part of the AI Governance Fundamentals course by BlueDot. I thank Charles Beasley and the students from my cohort for their feedback and encouragements.

Disclaimer: The French policy landscape is in rapid flux, after president Macron called for a snap election on 1st and 7th July. The situation is still unfolding, and the state of French AI policy may be significantly altered.

At various AI governance events, I noticed that most people had a very unclear vision of what was happening in AI policy in France, why the French government seemed dismissive of potential AI risks, and what that would that mean for the next AI Safety Summit in France.

The post below is my attempt at giving a quick intro to the key stakeholders of AI policy in France, their positions and how they influence international AI policy efforts.

My knowledge comes from hanging around AI safety circles in France for a year and a half, and working since January with the French Government on AI Governance. Therefore, I’m confident in the facts, but less in the interpretations, as I’m no policy expert myself.

Generative Artificial Intelligence Committee

The first major development in AI policy in France was the creation of a committee advising the government on Generative AI questions. This committee was created in September 2023 by former Prime Minister Elisabeth Borne.[1]

The goals of the committee were:

Strengthening AI training programs to develop more AI talent in France
Investing in AI to promote French innovation on the international stage
Defining appropriate regulation for different sectors to protect against abuses.

This committee was composed of notable academics and companies in the French AI field. This is a list of their notable member:

Co-chairs:

Philippe Aghion, an influential French economist specializing in innovation.

He thinks AI will give a major productivity boost and that the EU should invest in major research projects on AI and disruptive technologies.

Anne Bouverot, chair of the board of directors of ENS, the most prestigious scientific college in France. She was later nominated as leading organizer of the next AI Safety Summit.

She is mainly concerned about the risks of bias and discrimination from AI systems, as well as risks of concentration of power.

Notable members:

Joëlle Barral, scientific director at Google
Nozha Boujemaa, co-chair of the OECD AI expert grou

...
How predictive processing solved my wrist pain
by max_shen — 2024-07-04T01:56:20.162Z

This is a post about a period of major suffering I went through, and how I acquired the knowledge and skill to eventually lift me out of it.

Disclaimer: this is not medical advice — consider it a personal narrative, perhaps a starting point for exploration.

A fire burning

[Note: this section is about my personal experience. You can skip to the technical arguments here]

I am about to give up. My eyes are bleary after four days of staring at numpy arrays. I want to smash my keyboard and throw it out into the snow, but I have promised myself that I will fix this bug by the end of the week.

Besides a dull headache, I feel the usual ache in my wrists. This time, though, my forearms are also hurting. A dull heat rises from my wrists to the soft muscle below my elbow. Another half hour of coding, the pain in my wrists becomes sharper. So I stop for the day, knowing that my wrists will be back to normal to fix this problem tomorrow. Or so I think.

As a CS grad student, I code a lot, so make sure to have good postural and ergonomic hygiene. I use a Microsoft Sculpt keyboard, hotkeys to minimize mouse use. My wrists are neutral and my screen is at head height. It more or less keeps various aches at bay.

The next morning, wrists sore, I begin to type, and within fifteen minutes, the dull heat in my forearms becomes hot. My fingers become stiff; my typing slows to a crawl. My hands feel like they have been submerged in snow – cold, inflexible, useless.

In the doctor’s office, I explain the situation:

Me (suppressing panic, explaining the situation): What’s going on?

Doctor: You have tendonitis or a repetitive strain injury — the tendons in your hand are inflamed from overuse.

Me: You’re saying my hand is damaged from me typing too much? But I’m actually typing less than I was last semester.

Doctor (shrugging): To be honest, we don’t really know what’s going on. But you should see a physiotherapist. It’s up to you, but here’s a referral slip.

The physiotherapist tells me my muscles are imbalanced — some auxiliary muscles need strengthening. She gives me wrist exercises.

I notice I am confused

. Last semester I was spending twice the time typing than I am now. Also, before this flare up, I could do pull-ups just fine. Muscle and tendon weakness can’t be the cause.

Four weeks later and it’s even worse. I’m using my forearms to open doorknobs, eating frozen...
80,000 hours should remove OpenAI from the Job Board (and similar EA orgs should do similarly)
by Raemon — 2024-07-03T20:34:50.741Z

I haven't shared this post with other relevant parties – my experience has been that private discussion of this sort of thing is more paralyzing than helpful. I might change my mind in the resulting discussion, but, I prefer that discussion to be public.

I think 80,000 hours should remove OpenAI from its job board, and similar EA job placement services should do the same.

(I personally believe 80k shouldn't advertise Anthropic jobs either, but I think the case for that is somewhat less clear)

I think OpenAI has demonstrated a level of manipulativeness, recklessness, and failure to prioritize meaningful existential safety work, that makes me think EA orgs should not be going out of their way to give them free resources. (It might make sense for some individuals to work there, but this shouldn't be a thing 80k or other orgs are systematically funneling talent into)

There plausibly should be some kind of path to get back into good standing with the AI Risk community, although it feels difficult to imagine how to navigate that, given how adversarial OpenAI's use of NDAs was, and how difficult that makes it to trust future commitments.

The things that seem most significant to me:

They promised the superalignment team 20% of their compute-at-the-time (which AFAICT wasn't even a large fraction of their compute over the coming years), but didn't provide anywhere close to that, and then disbanded the team when Leike left.
Their widespread use of non-disparagement agreements, with non-disclosure clauses, which generally makes it hard to form accurate impressions about what's going on at the organization.
Helen Toner's description of how Sam Altman wasn't forthright with the board. (i.e. "The board was not informed about ChatGPT in advance and learned about ChatGPT on Twitter. Altman failed to inform the board that he owned the OpenAI startup fund despite claiming to be an independent board member, giving false information about the company’s formal safety processes on multiple occasions. And relating to her research paper, that Altman in the paper’s wake started lying to other board members in order to push Toner off the board.")
Hearing from multiple ex-OpenAI employees that OpenAI safety culture did not seem on track to handle AGI. Some of these are public (Leike, Kokotajlo), others were in private.

This is before getting into more openended arguments ...
Notes on Tuning Metacognition
by JoNeedsSleep — 2024-07-03T19:54:59.732Z

Summary: Reflections and practice notes on a metacognitive technique aimed at refining the process of thinking, rather than the thoughts themselves.

Epistemic Status: Experimental and observational, based on personal practice and reflections over a brief period.

Introduction

While doing a simple math problem, I realized that my faculty of thinking was often confused and inefficient. We were drawing polygons which corresponded to multi-holed toruses (or formally genus g surfaces), and in trying to generalize the square → one-holed torus pattern to an octagon, I completely forgot to use the previous pattern of 'opposite sides gets glued together' in any reasonable sense.

genus g surfaces

Zooming out to my broader life, I had been practicing Vipassana meditation for some time, and I was starting to notice incrementally finer thoughts, emotions, and thought patterns in my daily life.

Given the above experience, I hypothesized that if I focused my attention on how to think better rather than the specific thoughts, I can probably learn to learn, reason, and discover which much more efficiency. Importantly, I had a strong faith that our ability to learn is not a constant determined at birth.

The Technique

I stumbled upon an enlightening post Tuning your Cognitive Strategies which taught, in my language, the following technique:

Awareness of mental processes rather than objects of thoughts, and
Reward Mechanism for the quality of the thought process.

The two-step structure is rather similar to what I practice in Vipassana meditation, which teaches:

Fine-grained awareness of every sensation in the body, since every thought and emotion corresponds to a sensation, and
Equanimity with those sensations.

This tuning strategy basically imposes a reward modeling subprocess onto your mode of thinking. Instead of focusing on specific patterns of thinking, for instance a conscious steer towards more rigorous reasoning when intuition tries to jump in, the technique teaches you to assign conscious reward to your way of thinking which then informs the deeper parts of your consciousness what processes should be encouraged next time.

Even for the short interval that I've practiced, I've found that this technique is extremely powerful in accelerating efficiency of thought. Since it interferes with the subconscious mind, I advise caution.

There's not a lot of case studies online for ...
When Are Results from Computational Complexity Not Too Coarse?
by Dalcy — 2024-07-03T19:06:44.953Z

Tl;dr, While an algorithm's computational complexity may be exponential in general (worst-case), it is often possible to stratify its input via some dimension k that makes it polynomial for a fixed k, and only exponential in k. Conceptually, this quantity captures the core aspect of a problem's structure that makes specific instances of it 'harder' than others, often with intuitive interpretations.

Example: Bayesian Inference and Treewidth

One can easily prove exact inference (decision problem of: "is P(X)>0?") is NP-hard by encoding SAT-3 as a Bayes Net. Showing that it's NP is easy too.

Given a 3-SAT instance ϕ over Q1,…Qn, one can cleverly encode it as a Bayes Net Bϕ such that PBϕ(X=x1)>0 if and only if ϕ is satisfiable (From Koller & Friedman 2009).

Therefore, inference is NP-complete, implying that algorithms are worst-case exponential. But this can't be the typical case! Let's examine the example of a Bayes Net whose structure is a chain A⟶B⟶C⟶D, and say you want to compute the marginals P(D).

The Naive Algorithm for marginalization would be to literally multiply all the conditional probability distribution (CPD) tables for each of the Bayes Net's nodes, and sum over all the variables other than X. If we assume each variable has at most v values, then the computational complexity is exponential in the number of variables n.

P(D)=∑A∑B∑CP(A,B,C,D), which is O(vn).

But because of the factorization P(A,B,C,D)=P(A)P(B|A)P(C|B)P(D|C) due to the chain structure, we can shift the order of the sum around like this:

P(D)=∑CP(D|C)∑BP(C|B)∑AP(A)P(B|A)

and now the sum can be done in O(nv2). Why?

Notice ∑AP(A)P(B|A) is P(B), and to compute P(B=b) we need to multiply v times and sum v−1 times, overall O(v).
This needs to be done for every b, so O(v2). Now we have cached P(B), and we move on to ∑BP(C|B)P(B), where the same analysis applies. This is basically dynamic programming.

So, at least for chains, inference can be done in linear time in input size.

The earlier NP-completeness result, remember, is a worst-case analysis that applies to all possible Bayes Nets, ignoring the possible structure in each instance that may make some easier to solve than the other.

Let's attemp...
Musings on LLM Scale (Jul 2024)
by Vladimir_Nesov — 2024-07-03T18:35:48.373Z

In a recent interview, Dario Amodei claimed that cost of training is (starting with models already available)

Right now, $100 million. There are models in training today that are more like a $1 billion. I think if we go to $10 or a $100 billion, and I think that will happen in 2025-2026, maybe 2027, ...

(Epistemic status: Fermi estimates, 8 is approximately 10 which is greater than 9.)

Assuming $40,000 per H100 and associated infrastructure in a datacenter, $1 billion gives 25K H100s, which matches the scale of for example Meta's new training clusters and requires about 40MW of power. At $2 per hour, training time cost of 25K H100s reaches $100 million in 80 days, which seems reasonable if on the short side for a production training run. The cost of time matches $1 billion at 2.3 years. An H100 (SXM) is rated for 2e15 FLOP/s in BF16 (my impression is this is usually stable out of the box). This becomes 4e15 FLOP/s in FP8, which seems practical if done carefully, no degradation in pre-training loss compared to FP32. The $100 million run then translates to 9e25 FLOPs at 30% utilization in BF16, or 2e26 FLOPs in FP8. (For some reason this SemiAnalysis estimate is 2x lower, peak 2e20 FLOP/s for 100,000 H100s at FP8, possibly the sparsity footnote in H100 specification for the 4000 teraFLOP/s figure is the culprit.)

This is maybe 10x original GPT-4, estimated at 2e25 FLOPs. The leading models (Claude 3.5 Sonnet, Gemini 1.5 Pro, GPT-4 Omni) cost $15-20 per million output tokens, compared to $75-120 for once-frontier models Claude 3 Opus, Gemini 1 Ultra, original GPT-4. Given a Chinchilla optimal model, if we reduce its active parameters 3x and increase training compute 3x, we get approximately the same performance, but it's now at least 3x cheaper for inference. This increases data 10x, which if everything else fails can be obtained by repeating the old data, giving 30x overtraining in compute compared to what is Chinchilla optimal for the smaller model. Llama-3-70b is overtrained 10x, Llama-3-8b 90x, though they don't use MoE and their performance is lower than for MoE models with the same active parameters and training cost.

Beyond $100 million

The current frontier models are overtrained on compute that could enable even smarter models. Compute is increasing, but it mostly goes to reduction of inference cost, and only a little bit to capabilities. Why aren't any...
Static Analysis As A Lifestyle
by adamShimi — 2024-07-03T18:29:37.384Z
Link: https://epistemologicalfascinations.substack.com/p/static-analysis-as-a-lifestyle

I’ve been watching French Top Chef (the best Top Chef, fight me) with my wife again, and I’m always impressed by how often the mentoring chefs, all with multiple michelin stars and years of experience, can just guess that a dish will work or that it will be missing something.

By far, whenever a chef points to an error (not a risk, an error), it’s then immediately validated experimentally: either the candidate corrected it and the jury comments positively on that aspect of the dish, or they refused to and failed because of that aspect of the dish.

Obviously, this incredible skill comes from years of cooking experience. But at its core, this is one of the fundamental idea of epistemology that experts and masters rediscover again and again in their field:

static analysis

.

The core intuition of static analysis is that when you write a computer program, you can check some things without even running it, just by looking at it and analyzing it.

What most programmers know best are

type systems

, which capture what can be done with different values in the program, and forbid incompatible operations (like adding a number and a string of characters together, or more advanced things like

using memory that might already be deallocated

).

But static analysis is far larger than that: it include verifying programs with proof assistants, model checking where you simulate many different possible situations without even running tests, abstract interpretation where you approximate the program so you can check key properties on them…

At its core, static analysis focuses on what can be checked rationally, intellectually, logically, without needing to dirty your hands in the real world. Which is precisely what the mentoring chefs are doing!

They’re leveraging their experience and knowledge to simulate the dish, and figure out if it runs into some known problems: lack of a given texture, preponderance of a taste, lack of complexity (for the advanced gastronomy recipes that Top Chef candidates need to invent)…

Another key intuition from static analysis which translates well to the Top Chef example is that it’s much easier to check for specific failure modes than to verify correctness. It’s easier to check that I’m not adding a number and a string than it is to check that I’m adding the right two number, say the price of the wedding venue and the price of the DJ.

It’s this aspect of static anal...