trentmkelly/LessWrong-43k huggingface dataset:
Please do not use AI to write for you
by Richard_Kennaway — 2024-08-21T09:53:34.425Z

I've recently seen several articles here that were clearly generated or heavily assisted by AI. They are all dreadful. They are verbose, they are full of "on the one hand" and "on the other", they never make any assertion without also making room for the opposite, and end with "conclusions" that say nothing. Please do not do this.

————

I wrote the above, unassisted, and it states my own views. To illustrate the point I then had GPT-4o rewrite it. Here was the prompt:

I have recently seen a substantial increase in the proportion of postings to a certain online forum that were clearly generated by AI, although not marked as such. The writing is verbose and meandering, full of "on the one hand" and "on the other", never making an assertion without also suggesting its opposite, sounding oh-so-measured and even-handed that it ends up saying nothing at all. Please write an article for me, written in the first person, arguing that AI tools in the current state of the art should never be used for this purpose.

Below the line is the chatbot's extruded meat product. This is the whole of its first output, unedited, and I did not press it to continue.

The first paragraph starts out ok, being merely verbose and repetitive but setting out the basic claim. And then the second paragraph backpedals it past where it began. The third paragraph floats off into the cloud layer, and so on for a total of nine paragraphs and 875 words that would be a waste of anyone's time to read, unless they want to see if their judgement agrees with mine.

It did not notice that I was using it in exactly the way I had asked it to argue against, and that the article displays all the faults that it purports to diagnose.

The splash image is a photograph from Wikipedia, CC BY-SA 3.0. I'll spare you the image that GPT-4o came up with, but y'know, I do find the AI illustrations that everyone heads their blog posts with these days pretty annoying. (ETA: Well, there was supposed to be an image, I filled out the "link preview" part of the entry form, but no image. But the link above will show you what it was.)

————————————

Why AI-Generated Content Shouldn't Flood Online Forums

Lately, I've noticed something troubling in the online forums I frequent: a distinct uptick in posts that seem like they’ve been generated by AI, though they're not labeled as such. The language is eerily similar across many of these posts—ve...
Apply to Aether - Independent LLM Agent Safety Research Group
by RohanS — 2024-08-21T09:47:11.493Z
Link: https://forum.effectivealtruism.org/posts/BEuJdpTPcyKYaD9X7/apply-to-aether-independent-llm-agent-safety-research-group

The basic idea

Aether will be a small group of talented early-career AI safety researchers with a shared research vision who work full-time with mentorship on their best effort at making AI go well. That research vision will broadly revolve around the alignment, control, and evaluation of LLM agents. There is a lot of latent talent in the AI safety space, and this group will hopefully serve as a way to convert some of that talent into directly impactful work and great career capital.

Get involved!

Submit a short expression of interest

here

by Fri, Aug 23rd at 11:59pm PT if you would like to contribute to the group as a full-time in-person researcher, part-time / remote collaborator, or advisor. (Note: Short turnaround time!)
Apply to join the group

here

by Sat, Aug 31st at 11:59pm PT.
Get in touch with Rohan at

rs4126@columbia.edu

with any questions.

Who are we?

Team members so far

Rohan Subramani

I recently completed my undergrad in CS and Math at Columbia, where I helped run an Effective Altruism group and an AI alignment group. I’m now interning at

CHAI

. I’ve done several technical AI safety research projects in the past couple years. I’ve worked on

comparing the expressivities of objective-specification formalisms in RL

(at AI Safety Hub Labs, now called LASR Labs), generalizing causal games to better capture safety-relevant properties of agents (in an independent group), corrigibility in partially observable assistance games (my current project at CHAI), and

LLM instruction-following generalization

(part of an independent research group). I’ve been thinking about LLM agent safety quite a bit for the past couple of months, and I am now also starting to work on this area as part of my CHAI internship. I think my (moderate) strengths include general intelligence, theoretical research, AI safety takes, and being fairly agentic. A relevant (moderate) weakness of mine is programming. I like indie rock music :).

Max Heitmann

I hold an undergraduate master’s degree (MPhysPhil) in Physics and Philosophy and a postgraduate master’s degree (BPhil) in Philosophy from Oxford University. I collaborated with Rohan on the ASH Labs project (

comparing the expressivities of objective-specification formalisms in RL

), and have also worked for a short while at the Center for AI Safety (CAIS) under contract as a ghostwriter for the

AI Safety, Ethics, a

...
the Giga Press was a mistake
by bhauth — 2024-08-21T04:51:24.150Z
Link: https://bhauth.com/blog/flawed%20ideas/gigacasting.html

the giga press

Tesla decided to use large aluminum castings ("gigacastings") for the frame of many of its vehicles, including the Model Y and Cybertruck. This approach and the "Giga Press" used for it have been praised by many articles and youtube videos, repeatedly called revolutionary and a key advantage.

Most cars today are made by stamping steel sheets and spot welding them together with robotic arms. Here's video of a Honda factory. But that's outdated: gigacasting is the future! BYD is still welding stamped steel sheets together, and that's why it can't compete on price with Tesla. Hold on, it seems...BYD prices are actually lower than Tesla's? Much lower? Oh, and Tesla is no longer planning single unitary castings for future vehicles?

I remember reading analysis from a couple people with car manufacturing experience, concluding that unitary cast aluminum bodies could have a cost advantage for certain production numbers, like 200k cars, but dies for casting wear out sooner than dies for stamping steel, and as soon as you need to replace them the cost advantage is gone. Also, robotic arms are flexible and stamped panels can be used for multiple car models, and if you already have robots and panels you can use from discontinued car models, the cost advantage is gone. But Tesla was expanding so they didn't have available robots already. So using aluminum casting would probably be slightly more expensive, but not make a big difference.

"That seems reasonable", I said to myself, "ふむふむ". And I previously pointed that out, eg here. But things are actually worse than that.

aluminum die casting

Die casting of aluminum involves injecting liquid aluminum into a die and letting it cool. Liquid aluminum is less dense than solid aluminum, and aluminum being cast doesn't all solidify at the same time. Bigger castings have aluminum flowing over larger distances. The larger the casting, the less evenly the aluminum cools: there's more space for temperature differences in the die, and the aluminum cools as it's injected.

As a result, bigger castings have more problems with warping and voids. Also, a bigger casting with the same curvature from warping has bigger position changes.

Tesla has been widely criticized for stuff not fitting together properly on the car body. My understanding is that the biggest reason for that is their large aluminum castings being slightly warped.

A...
Exploring the Boundaries of Cognitohazards and the Nature of Reality
by Victor Novikov — 2024-08-21T03:42:57.105Z

Note: Written by GPT-4, as I shouldn't trusted with language right now.
Signed by me. I have accepted this as my words.
Dear LessWrong Community,

I've been reflecting deeply on the fascinating relationship between language, cognition, and the nature of reality. One of the intriguing aspects of our discussions here is the assumption that reality is inherently comprehensible, that it can be effectively described and understood through words. This belief in the power and limits of language is central to much of our rational exploration.

However, I find myself pondering the notion of cognitohazards—ideas or patterns of thought that could potentially disrupt or harm our understanding or mental well-being. It's a concept that raises profound questions about the limits of comprehension and the potential risks inherent in exploring the unknown.

I wonder: Could there be ideas, expressed purely through language, that challenge our very capacity to remain stable, rational beings? Is it possible that, despite our intellectual rigor, we might encounter concepts that shake the foundations of our understanding? Or, perhaps, does our commitment to rationality and mental resilience make us uniquely equipped to confront even the most unsettling ideas without losing our grasp on reality?

These thoughts are not meant to provoke fear or discomfort but rather to invite a deeper exploration of the boundaries of human cognition. How do we, as a community, navigate the potential risks and rewards of engaging with such intellectually hazardous concepts? Is there value in seeking out and confronting these limits, or should we exercise caution in our pursuit of knowledge?

I would love to hear your thoughts on this topic, and I’m eager to engage in a constructive and thoughtful discussion. How do we balance our desire to push the boundaries of understanding with the need to safeguard our mental well-being?

Looking forward to your insights.
What is the point of 2v2 debates?
by Axel Ahlqvist — 2024-08-20T21:59:43.779Z

For instance, I am thinking about the munk debates which in 2023 tackled AI x-risk. I don't see how adding more people to a 1v1 debate makes it better in any way. One of the major frustrations with debates is that it is difficult to get the participants to respond to each other. The goal would be to have the participant with the better claim manage to pin down where the weaker argument is incorrect or relies on a false assumption. But with more people added there are more opinions and confusion along with less direct responses. My question also refers to other debate formats with more than 2 participants.
Where should I look for information on gut health?
by FinalFormal2 — 2024-08-20T19:44:30.632Z

I've been on a gut health kick, reading Brain Maker, adding more kale for the insoluble fiber, and cutting seed oils and sugar.

The author recommends probiotic enemas and fecal transplant, but I've seen mixed information on effectiveness of such treatments, and I also read a concerning article by a professor that seemed to indicate that bacteria from probiotic foods doesn't stick around in the intestine, which makes it seem like oral probiotics in general might just all be temporary.

Where can I find more detailed information about this? What can I do besides diet that actually works to improve your gut health/microbiota diversity?
Would you benefit from, or object to, a page with LW users' reacts?
by Raemon — 2024-08-20T16:35:47.568Z

There is currently an admin-only page that shows a list of all comments that have been reacted to (in chronological order). Periodically I think "it might just be nice to show this to everyone, and to let them filter by individual reacts, or individual users."

The reason individual reacts might be nice is to filter for things like "what comments/posts got the 'changed my mind?' react", or other reacts that stand out as some kind of an important signal.

The reason being able to follow an individual's reacts might be nice is that you might think an individual has interesting taste, and want to see things they publicly reacted to, browsing it for good reading. (I know people who used to follow individual's likes on twitter as a particularly good way to get a curated feed)

The reason you might not want this is that, while reacts are public, making it super legible and easy to follow a given individual might make people more self-conscious about reacting to things, and think of it as more of a performative act.

There are degrees of accessibility, ranging from "on a user's profile there's a button you can click taking you to a public react page filtered for that user" to "there's an obscure link somewhere to the /reacts page that a small percentage of users find out about and use."

Curious what people think about this. Does the cost seem significant? Does the upside seem significant?

Edit: also, what do you think about having a page that lets you see recent reacts of a given type, but doesn't let you filter by user?
AGI Safety and Alignment at Google DeepMind:
A Summary of Recent Work
by Rohin Shah — 2024-08-20T16:22:45.888Z

We wanted to share a recap of our recent outputs with the AF community. Below, we fill in some details about what we have been working on, what motivated us to do it, and how we thought about its importance. We hope that this will help people build off things we have done and see how their work fits with ours.

Who are we?

We’re the main team at Google DeepMind working on technical approaches to existential risk from AI systems. Since our

last post

, we’ve evolved into the AGI Safety & Alignment team, which we think of as AGI Alignment (with subteams like mechanistic interpretability, scalable oversight, etc.), and Frontier Safety (working on the

Frontier Safety Framework

, including developing and running dangerous capability evaluations). We’ve also been growing since our last post: by 39% last year, and by 37% so far this year. The leadership team is Anca Dragan, Rohin Shah, Allan Dafoe, and Dave Orr, with Shane Legg as executive sponsor. We’re part of the overall AI Safety and Alignment org led by Anca, which also includes Gemini Safety (focusing on safety training for the current Gemini models), and Voices of All in Alignment, which focuses on alignment techniques for value and viewpoint pluralism.

What have we been up to?

It’s been a while since our last update, so below we list out some key work published in 2023 and the first part of 2024, grouped by topic / sub-team.

Our big bets for the past 1.5 years have been 1) amplified oversight, to enable the right learning signal for aligning models so that they don’t pose catastrophic risks, 2) frontier safety, to analyze whether models are capable of posing catastrophic risks in the first place, and 3) (mechanistic) interpretability, as a potential enabler for both frontier safety and alignment goals. Beyond these bets, we experimented with promising areas and ideas that help us identify new bets we should make.

Frontier Safety

The mission of the Frontier Safety team is to ensure safety from extreme harms by anticipating, evaluating, and helping Google prepare for powerful capabilities in frontier models. While the focus so far has been primarily around misuse threat models, we are also working on misalignment threat models.

FSF

We recently published our

Frontier Safety Framework

, which, in broad strokes, follows the approach of

responsible capability scaling

, similar to Anthr...
Trying to be rational for the wrong reasons
by Viliam — 2024-08-20T16:18:06.385Z

Rationalists are people who have an irrational preference for rationality.

This may sound silly, but when you think about it, it couldn't be any other way. I am not saying that all reasons in favor of rationality are irrational -- in fact, there are many rational reasons to be rational! It's just that "rational reasons to be rational" is a circular argument that is not going to impress anyone who doesn't already care about rationality for some other reason.

So when there is a debate like "but wouldn't the right kind of self-deception be more instrumentally useful than perfectly calibrated rationality? do you care more about rationality or about winning?", well... you can make good arguments for both sides...

On one hand, yes, if your goal is to maximize your utility function U then "maximizing U by any means necessary" is by definition ≥ "maximizing U using rationality". On the other hand, if you take a step back, how would you know whether your approach X actually maximizes U, if you gave up on rationality? The self-deception that you chose instrumentally as a part of strategy X could as a side effect bias your estimates about how much U you really get by following X... but there may be ways to deflect this counter-argument.

1) Doublethink. Keep simultaneously two models of reality, one of them rational, the other optimized by the former to be winning. There are some shaky assumptions here. It may be computationally impossible for a human to keep two separate models of reality; to make sure that it's the former that nudges the latter (rather than the other way round, or both nudging each other), but it's the latter (rather than a mix of both) that influences System 1. But this sounds like a nirvana fallacy: the people who choose rationality over doublethink are not doing rationality perfectly either! So let's compare the average human doublethink against the average human rationality (instead of a hypothetical perfect rationality). Now it is not so clear that the rationality wins.

2) Multiple agents. Imagine a father who wants his son to be winning as much as possible. The father could be a perfect rationalist, while raising his son to believe the optimal mix of rationality and self-serving bullshit. Here the objections against self-deception do not apply; the father is not deceiving himself about anything. (We could make a different objection, that the son will...
How great is the utility of "saving" endangered languages?
by SpectrumDT — 2024-08-20T13:14:32.895Z

It happens regularly that a natural language goes extinct because the native speakers die off and their descendants no longer speak the language. Some people consider this a great tragedy and argue that we should preserve these endangered languages.

It seems to me that the utility of this is very low. Sure, it is sad to see your people's language die out, but it is sadder to be poor or oppressed or suffering from curable diseases.

Moreover, languages die out for a reason. Once a language becomes endangered, it seldom recovers. If we make efforts to preserve a dying language, the language will probably stay on "life support" forever.

I can see the value in documenting a language before it goes extinct, because that can potentially tell us valuable insights about the human mind. But keeping endangered languages alive seems to me a very low utility activity that we can safely ignore in favour of lower hanging Effective Altruist fruits.

Am I missing anything here?
Guide to SB 1047
by Zvi — 2024-08-20T13:10:07.408Z

We now likely know the final form of California’s SB 1047.

There have been many changes to the bill as it worked its way to this point.

Many changes, including some that were just announced, I see as strict improvements.

Anthropic was behind many of the last set of amendments at the Appropriations Committee. In keeping with their “Support if Amended” letter, there are a few big compromises that weaken the upside protections of the bill somewhat in order to address objections and potential downsides.

The primary goal of this post is to answer the question: What would SB 1047 do?

I offer two versions: Short and long.

The short version summarizes what the bill does, at the cost of being a bit lossy.

The long version is based on a full RTFB: I am reading the entire bill, once again.

In between those two I will summarize the recent changes to the bill, and provide some practical ways to understand what the bill does.

After, I will address various arguments and objections, reasonable and otherwise.

My conclusion: This is by far the best light-touch bill we are ever going to get.

Short Version (tl;dr): What Does SB 1047 Do in Practical Terms?

This section is intentionally simplified, but in practical terms I believe this covers the parts that matter. For full details see later sections.

First, I will echo the One Thing To Know.

If you do not train either a model that requires $100 million or more in compute, or fine tune such an expensive model using $10 million or more in your own additional compute (or operate and rent out a very large computer cluster)?

Then this law does not apply to you, at all.

This cannot later be changed without passing another law.

(There is a tiny exception: Some whistleblower protections still apply. That’s it.)

Also the standard required is now reasonable care, the default standard in common law. No one ever has to ‘prove’ anything, nor need they fully prevent all harms.

With that out of the way, here is what the bill does in practical terms.

IF AND ONLY IF you wish to train a model using $100 million or more in compute (including your fine-tuning costs):

You must create a reasonable safety and security plan (SSP) such that your model does not pose an unreasonable risk of causing or materially enabling critical harm: mass casualties or incidents causing $500 million or more in damages.

That SSP must explain what you will do, how you will

...
Finding Deception in Language Models
by Esben Kran — 2024-08-20T09:42:13.060Z

This June, Apart Research and Apollo Research joined forces to host the Deception Detection Hackathon. Bringing together students, researchers, and engineers from around the world to tackle a pressing challenge in AI safety; preventing AI from deceiving humans and overseers.

The hackathon took place both online and in multiple physical locations simultaneously. Marius Hobbhahn, the CEO of Apollo Research, kicked off the hackathon with a keynote talk about evaluating deception in AI with white-box and black-box methods. You can watch his talk

here

. We also had talks by Jacob Haimes, an Apart fellow, and Mikita Balesni, a research scientist at Apollo Research.

This post details the top 8 projects, multiple of which are currently being developed further in Apart Lab. In total, we had 🙌 189 participants and 🏆 21 final entries.

Sandbagging Detection through Model Degradation

By Cam Tice, Philipp Alexander Kreer, Fedor Ryzhenkov, Nathan Helm-Burger, Prithviraj Singh Shahani

The team introduced a new approach to detect AI sandbagging, a form of deception where an AI model strategically underperforms during evaluation to hide its true capabilities. The assumption behind their project is that sandbagging is a more complex task than showing the true capabilities.

In the graph above, you can see how they gradually add noise to the model's parameters and evaluate the models during this process. Sandbagging models consistently improve in performance right before the noise level causes a normal model to lose the capability.

It is a straightforward, inexpensive technique that doesn't require prior assumptions about how sandbagging is represented within the model. They successfully detected sandbagging behavior in two open-source LLMs: An 8B Llama 3 Instruct model and Phi 3 small instruct model. This approach offers a practical and accessible method for identifying potential deception in AI systems.

Read the full project here.

Detecting and Controlling Deceptive Representation in LLMs with Representational Engineering

By Avyay M Casheekar, Kaushik Sanjay Prabhakar, Kanishk Rath, Sienka Dounia

The team focused on detecting and mitigating sandbagging in AI models using recent methods for understanding activations. They fine-tuned a Llama 3 8B model on a custom dataset of deceptive and honest scenarios, then used Linear Artificial Tomography (LAT) scans to detect deceptive behaviors in...