trentmkelly/LessWrong-43k huggingface dataset:
AI #61: Meta Trouble
by Zvi — 2024-05-02T18:40:03.242Z

Note by habryka: This post failed to import automatically from RSS for some reason, so it's a week late. Sorry for the hassle.

The week’s big news was supposed to be Meta’s release of two versions of Llama-3.

Everyone was impressed. These were definitely strong models.

Investors felt differently. After earnings yesterday showed strong revenues but that Meta was investing heavily in AI, they took Meta stock down 15%.

DeepMind and Anthropic also shipped, but in their cases it was multiple papers on AI alignment and threat mitigation. They get their own sections.

We also did identify someone who wants to do what people claim the worried want to do, who is indeed reasonably identified as a ‘doomer.’

Because the universe has a sense of humor, that person’s name is Tucker Carlson.

Also we have a robot dog with a flamethrower.

Table of Contents

Previous post: On Llama-3 and Dwarkesh Patel’s Podcast with Zuckerberg.

Introduction.

Table of Contents.

Language Models Offer Mundane Utility. Take the XML. Leave the hypnosis.

Language Models Don’t Offer Mundane Utility. I have to praise you. It’s my job.

Llama We Doing This Again. Investors are having none of it.

Fun With Image Generation. Everything is fun if you are William Shatner.

Deepfaketown and Botpocalypse Soon. How to protect your image model?

They Took Our Jobs. Well, they took some particular jobs.

Get Involved. OMB, DeepMind and CivAI are hiring.

Introducing. A robot dog with a flamethrower. You in?

In Other AI News. Mission first. Lots of other things after.

Quiet Speculations. Will it work? And if so, when?

Rhetorical Innovation. Sadly predictable.

Wouldn’t You Prefer a Nice Game of Chess. Game theory in action.

The Battle of the Board. Reproducing an exchange on it for posterity.

New Anthropic Papers. Sleeper agents, detected and undetected.

New DeepMind Papers. Problems with agents, problems with manipulation.

Aligning a Smarter Than Human Intelligence is Difficult. Listen to the prompt.

People Are Worried About AI Killing Everyone. Tucker Carlson. I know.

Other People Are Not As Worried About AI Killing Everyone. Roon.

The Lighter Side. Click here.

Language Models Offer Mundane Utility

I too love XML for this and realize I keep forgetting to use it. Even among humans, every time I see or use it I think ‘this is great, this is exce...
Why is AGI/ASI Inevitable?
by DeathlessAmaranth — 2024-05-02T18:27:17.486Z

Hello! My name is Amy.
This is my first LessWrong post. I'm about somewhat certain it will be deleted, but I'm giving it a shot anyway, because I've seen this argument thrown around a few places and I still don't understand. I've read a few chunks of the Sequences, and the fundamentals of rationality sequences.

What makes artificial general intelligence 'inevitable'? What makes artificial superintelligence 'inevitable'? Can't people decide simply not to build AGI/ASI?

I'm very, very new to this whole scene, and while I'm personally convinced AGI/ASI is coming, I haven't really been convinced it's inevitable, the way so many people online (mostly Twitter!) seem convinced.

While I'd appreciate to hear your thoughts, what I'd really love is to get some sources on this. What are the best sequences to read on this topic? Are there any studies or articles which make this argument?

Or is this all just some ridiculous claim those 'e/acc' people cling to?

Hope this doesn't get deleted! Thank you for your help!
AISN #34: New Military AI Systems
Plus, AI Labs Fail to Uphold Voluntary Commitments to UK AI Safety Institute, and New AI Policy Proposals in the US Senate
by Corin Katzke — 2024-05-02T16:12:47.783Z
Link: https://newsletter.safe.ai/p/ai-safety-newsletter-34-new-military

Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required.

Subscribe here to receive future versions.

Listen to the AI Safety Newsletter for free on Spotify.

AI Labs Fail to Uphold Safety Commitments to UK AI Safety Institute

In November, leading AI labs committed to sharing their models before deployment to be tested by the UK AI Safety Institute. But reporting from Politico shows that these commitments have fallen through.

OpenAI, Anthropic, and Meta have all failed to share their models with the UK AISI before deployment. Only Google DeepMind, headquartered in London, has given pre-deployment access to UK AISI.

Anthropic released the most powerful publicly available language model, Claude 3, without any window for pre-release testing by the UK AISI. When asked for comment, Anthropic co-founder Jack Clark said, “Pre-deployment testing is a nice idea but very difficult to implement.”

When asked about their concerns with pre-deployment testing, Meta’s spokesperson argued that Meta is an American company and should only have to comply with American regulations, even though the US has signed agreements with the UK to collaborate on AI testing. Other lab sources mentioned the possibility of leaking intellectual property secrets, and the risk that safety testing could slow down model releases.

This is a strong signal that AI companies should not be trusted to follow through on safety commitments if those commitments conflict with their business interests. Because of the ongoing race among AI labs, all AI developers face pressure to keep up with their competitors at the expense of safety, even if they’re concerned that AI development poses catastrophic risks to humanity.

Fortunately, there are several ongoing efforts to turn voluntary commitments into legal requirements. The UK government said it plans to develop “targeted, binding requirements” to ensure safety in frontier AI development. In California, a bill is being considered which would require companies to self-certify that they’ve evaluated and mitigated catastrophic risks before releasing AI systems, and allow them to be sued for violations of this law. Given the fragility of voluntary commitments to AI safety, future policy work should aim to make those commitments legally binding.

For more on ...
How to write Pseudocode and why you should
by Johannes C. Mayer — 2024-05-02T15:53:44.014Z

TLDR: Writing pseudocode is extremely useful when designing algorithms. Most people do it wrong. Mainly because they don't intentionally try to keep the code as abstract as possible. Possibly this happens because in normal programming a common workflow is to focus on getting the implementation of one function right before moving on.

I like it when I tell somebody about an idea, and they ask me "How would you write this in Python?" To be able to explain your idea to the Python interpreter, you need to have a really good understanding of it. Also trying to explain your idea to the Python interpreter often forces you to improve your understanding.

However, in my experience, it is much better to start by writing high-level pseudocode, to iteratively build your understanding. Descending into the pit of implementation details should happen in concert with how your understanding grows.

The main advantage of this is that this makes it much faster. It's hard to put numbers on it but I am sure it makes you overall at least 2x faster, compared to only working with runnable code. It's Probably more like 5-100x though.

Writing pseudocode properly is a skill you need to learn. It's very different from writing a program that you can execute.

The most amazing thing about this process is that at a certain point, long before you have a concrete implementation, you will feel like all conceptual obstacles have fallen, and only grunt programming remains.

You need to be somewhat careful because it is possible to feel like this, even when there is still a problem. A potentially useful heuristic here is to take a couple of steps more towards the concrete implementation and see if everything you're doing now is just really straightforward.

How to write Pseudocode

One of the core skills in writing pseudocode is to constantly ask yourself whether you can get away with not implementing something.

Start with Types

Usually, I start by just writing down the type signatures of the functions and data structures involved. I find it useful to make functions pure wherever possible, as this automatically makes the type signatures more informative.

Comments

The second step is usually to write some very high-level comments about what a function does in natural language.

Then I start to write lower and lower level comments describing what things this function does, e.g. what other functions are called...
AI #62: Too Soon to Tell
by Zvi — 2024-05-02T15:40:04.364Z

What is the mysterious impressive new ‘gpt2-chatbot’ from the Arena? Is it GPT-4.5? A refinement of GPT-4? A variation on GPT-2 somehow? A new architecture? Q-star? Someone else’s model? Could be anything. It is so weird that this is how someone chose to present that model.

There was also a lot of additional talk this week about California’s proposed SB 1047.

I wrote an additional post extensively breaking that bill down, explaining how it would work in practice, addressing misconceptions about it and suggesting fixes for its biggest problems along with other improvements. For those interested, I recommend reading at least the sections ‘What Do I Think The Law Would Actually Do?’ and ‘What are the Biggest Misconceptions?’

As usual, lots of other things happened as well.

Table of Contents

Introduction.

Table of Contents.

Language Models Offer Mundane Utility. Do your paperwork for you. Sweet.

Language Models Don’t Offer Mundane Utility. Because it is not yet good at it.

GPT-2 Soon to Tell. What is this mysterious new model?

Fun With Image Generation. Certified made by humans.

Deepfaketown and Botpocalypse Soon. A located picture is a real picture.

They Took Our Jobs. Because we wouldn’t let other humans take them first?

Get Involved. It’s protest time. Against AI that is.

In Other AI News. Incremental upgrades, benchmark concerns.

Quiet Speculations. Misconceptions cause warnings of AI winter.

The Quest for Sane Regulation. Big tech lobbies to avoid regulations, who knew?

The Week in Audio. Lots of Sam Altman, plus some others.

Rhetorical Innovation. The few people who weren’t focused on SB 1047.

Open Weights Are Unsafe And Nothing Can Fix This. Tech for this got cheaper.

Aligning a Smarter Than Human Intelligence is Difficult. Dot by dot thinking.

The Lighter Side. There must be some mistake.

Language Models Offer Mundane Utility

Write automatic police reports based on body camera footage. It seems it only uses the audio? Not using the video seems to be giving up a lot of information. Even so, law enforcement seems impressed, one notes an 82% reduction in time writing reports, even with proofreading requirements.

Axon says it did a double-blind study to compare its AI reports with ones from regular offers.

And it says that Draft One results were “equal to or better than” regular police rep

...
Whiteboard Program Traceing: Debug a Program Before you have the Code
by Johannes C. Mayer — 2024-05-02T15:30:10.500Z

TL;DR:
A Whiteboard Program Trace is where you write down a data structure on a whiteboard and step-by-step walk through how an algorithm A would manipulate this data, i.e. you execute A for one specific input in a visual way. Usually, I do this before I have understood all the details of A, in order to figure them out.

Imagine you are debugging some program and you step through every line of code and have a display of what are all the local and global variables in the given context.

What if instead of first writing the program and then debugging it, you start by debugging the program that you would like to have? All you need is a very high-level understanding of your program.

To do this write down on a whiteboard, write down some data structure, and then walk through, step by step, what modifications the algorithm performs on the data structure.

I do something like this in this video:

I write down the data structure and describe what properties the graph has.

I write down the result of an algorithm that embeds a graph into 2D Euclidean space (each vertex gets assigned a coordinate).

I describe the general problem of creating a plan from a to b where a and b are nodes. Then I create a specific instance of this problem by marking one node as the start node and one node as the target node in the graph.

I am running BFS on the graph. Note that I draw the squiggly edges in the order that BFS would unroll them.

I describe a procedure for calculating a displacement vector.

I describe how the displacement vector represents all possible plans.

I made this video to make sure that I understand the Vector planning algorithm, before writing the Vector Planning in a Lattice Graph post.

Note that I am somewhat confused at step 6. I go on and on about how this represents all possible paths, which I did not think about before recording the video. And I don't give an exact algorithm for unrolling the vector into the plan, in the video.
Which skincare products are evidence-based?
by Vanessa Kosoy — 2024-05-02T15:22:12.597Z

The beauty industry offers a large variety of skincare products (marketed mostly at women), differing both in alleged function and (substantially) in price. However, it's pretty hard to test for yourself how much any of these product help. The feedback loop for things like "getting less wrinkles" is very long.

So, which of these products are actually useful and which are mostly a waste of money? Are more expensive products actually better or just have better branding? How can I find out?

I would guess that sunscreen is definitely helpful, and using some moisturizers for face and body is probably helpful. But, what about night cream? Eye cream? So-called "anti-aging"? Exfoliants?
Q&A on Proposed SB 1047
by Zvi — 2024-05-02T15:10:02.916Z

Previously: On the Proposed California SB 1047.

Text of the bill is here. It focuses on safety requirements for highly capable AI models.

This is written as an FAQ, tackling all questions or points I saw raised.

Safe & Secure AI Innovation Act also has a description page.

Why Are We Here Again?

There have been many highly vocal and forceful objections to SB 1047 this week, in reaction to a (disputed and seemingly incorrect) claim that the bill has been ‘fast tracked.’

The bill continues to have substantial chance of becoming law according to Manifold, where the market has not moved on recent events. The bill has been referred to two policy committees one of which put out this 38 page analysis.

The purpose of this post is to gather and analyze all objections that came to my attention in any way, including all responses to my request for them on Twitter, and to suggest concrete changes that address some real concerns that were identified.

Some are helpful critiques pointing to potential problems, or good questions where we should ensure that my current understanding is correct. In several cases, I suggest concrete changes to the bill as a result. Two are important to fix weaknesses, one is a clear improvement, the others are free actions for clarity.

Some are based on what I strongly believe is a failure to understand how the law works, both in theory and in practice, or a failure to carefully read the bill, or both.

Some are pointing out a fundamental conflict. They want people to have the ability to freely train and release the weights of highly capable future models. Then they notice that it will become impossible to do this while adhering to ordinary safety requirements. They seem to therefore propose to not have safety requirements.

Some are alarmist rhetoric that has little tether to what is in the bill, or how any of this works. I am deeply disappointed in some of those using or sharing such rhetoric.

Throughout such objections, there is little or no acknowledgement of the risks that the bill attempts to mitigate, suggestions of alternative ways to do that, or reasons to believe that such risks are insubstantial even absent required mitigation. To be fair to such objectors, many of them have previously stated that they believe that future more capable AI poses little catastrophic risk.

I get making mistakes, indeed i...
What are the Activities that make up your Research Process?
by Johannes C. Mayer — 2024-05-02T15:01:01.437Z

There are a bunch of activities that I engage in when doing research. These include but are not limited to:

Figuring out the best thing to do.

Talking out loud to force my ideas into language.

For the last 3 months I have been working maybe 50 hours per week by meeting with people and doing stream-of-thought reasoning. That was very productive. Probably in large part because of this.

Even when working alone I try to use this. The main thing that holds me back from using it all the time when working alone is that it can be quite awkward.

Recording myself explaining something, usually on a whiteboard. This is useful to check:

Check if my understanding is good enough yet to write a post.

Helps remove the awkwardness when talking to yourself (because you are not).

Trying to explain an idea on the whiteboard.

I mainly use whiteboards when I am still at the stage of being confused.

Writing pseudocode.

Similar to forcing yourself to explain something in natural language.

Notice where you are confused by not being able to express something.

Writing a concrete implementation we can run.

I rarely do this because it is so slow, probably because I have not acquired sufficient software engineering skills yet.

I expect that writing programs can be very useful for getting observations that you could not easily generate in your head. E.g. Mandelbrot did make plot fractals.

Writing down things that we have figured out on a whiteboard or any other process in rough notes.

Writing a distillation of the thing I have figured out, such that I can understand these notes 1 year from now.

Reflecting on how it went.

Writing public posts, that convey concepts to other people.

My main questions are:

What research processes do you use?

When do you use them?

What do you get out if it goes well?

Also, feel free to mention great posts about this. I am most interested in processes that you personally use on a regular basis.
How do you Select the Right Research Acitivity in the Right Moment?
by Johannes C. Mayer — 2024-05-02T14:45:30.428Z

In doing research, I have a bunch of activities that I engage in, including but not limited to:

Figuring out the best thing to do.

Talking out loud to force my ideas into language.

Trying to explain an idea on the whiteboard.

Writing pseudocode.

Writing a concrete implementation we can run.

Writing down things that we have figured out on a whiteboard or any other process in rough notes.

Writing a distillation of the thing I have figured out, such that I can understand these notes 1 year from now.

Reflecting on how it went.

Writing public posts, that convey concepts to other people.

My models about when to use what process are mostly based on intuition right now.

I expect that if I had more explicit models this would allow me to more easily notice when I am using the wrong kind of process for whatever I am trying to achieve at that moment.

One of the core problems here seems to be, to get good at noticing when it would be good to distill something. You don't want to trash your documents by having lots of confused thoughts in them, while also capturing all the important insights that you had.

Have you encountered this problem, and if so how do you handle it?
Can stealth aircraft be detected optically?
by Yair Halberstadt — 2024-05-02T07:47:00.101Z

5th generation military aircraft are extremely optimised to reduce their radar cross section. It is this ability above all others that makes the f-35 and the f-22 so capable - modern anti aircraft weapons are very good, so the only safe way to fly over a well defended area is not to be seen.

But wouldn't it be fairly trivial to detect a stealth aircraft optically?

This is what an f-35 looks like from underneath at about 10 by 10 pixels:

You and I can easily tell what that is (take a step back, or squint). So can GPT4:

The image shows a silhouette of a fighter jet in the sky, likely flying at high speed. The clear blue sky provides a sharp contrast, making the aircraft's dark outline prominent. The jet's design appears sleek and streamlined, with distinct wings and a tail fin, typical of modern military aircraft.

A specialised image classifier should also be able to do so no problem at all, and much faster.

The patriot radar has a range of about 100km. Let's say we wanted to match that.

An f-35s wingspan is 11m. I want to detect the F-35 at 100kms out in all directions. Then we need one pixel per metre, over the surface of a semisphere with radius 100km.

That equates to a bit less than one hundred billion pixels. You can buy phones with 100 million pixel cameras for under $1000, and the majority of the cost is not the camera. This suggests you could use an array to cover the entire sky for under a million dollars, an order of magnitude less than the cost of a radar. The rest of the infrastructure you would need would also cost, including some beefy GPUs, but not enough to increase the cost more than an order of magnitude.

You'd need two of these to perform triangulation and calculate the distance and size of the object, as well as to exclude false positives (small bird close to the camera). You'd probably want 3 for greater reliability. But that should be relatively simple to do.

This only works in good weather, but denying the enemy use of their aircraft in good weather seems pretty useful, especially in the middle east where it's continuously sunny half the year.

Also, unlike a radar, this doesn't advertise it's location whenever it's used.
So why isn't this done?
My best guesses are that it would be fairly easy to camouflage the f-35, or that the performance characteristics of this solution wouldn't be good enough to guide missiles, only to detect the existence o...