trentmkelly/LessWrong-43k huggingface dataset:
Doing Nothing Utility Function
by k64 — 2024-09-26T22:05:18.821Z

One of the questions I've heard asked is "how do you design a utility function that would make the AI do nothing?" That is, how could we put a pause button on an AI so that we could pause it if we wanted to? I had an idea about how one could do this, and am sure it has already been thought of, so I'm curious to know why this doesn't end up working.

Why can't we just write something like:

If (paused) and not (taking actions): utility=PredictedUtilityIfNotPaused

If not (paused): utility=NormalUtilityFunction
Stanislav Petrov Quarterly Performance Review
by Ricki Heicklen — 2024-09-26T21:20:11.646Z
Link: https://bayesshammai.substack.com/p/stanislav-petrov-quarterly-performance

September 26 is the anniversary of the 1983 Soviet nuclear false alarm incident—item #24 on Wikipedia’s upsettingly long “

List of Nuclear Close Calls

”—in which Lieutenant Colonel Stanislav Petrov did an excellent job at not starting World War III.

Quarterly Performance Review, Autumn 1983

Colonel Yuri Kuznetsov looked out the window anxiously. The endless gray landscape did little to soothe his nerves. He only had one employee review left to get through, but he’d saved the hardest one for last.

He wasn’t upset about having to dismiss Lieutenant Colonel Petrov—he couldn’t wait to be rid of the little shit—but he couldn’t shake the feeling that something was amiss. He took a swig from his flask.

“Stanislav, you can come in now,” Yuri shouted as he opened the door and nearly smashed Stanislav Petrov in the face. “Have a seat,” he said.

“Yes, sir. Thank you sir. Overjoyed to be here as always,” Stanislav said.

“The purpose of this meeting is for us to discuss various concerns that have emerged about your performance over the past several months,” said Yuri. “Looking through your chart, what I’m seeing here is that this quarter you’ve managed to achieve… nothing.”

“That’s right sir,” Stanislav piped.

“This is consistent with your performance in the previous quarter, per my notes from our last meeting, in which you also accomplished nothing,” Yuri continued.

“Yes, sir! And I’ve run the numbers several times and I feel pretty confident that this coming quarter I’ll be able to increase my output by 50%, bringing it up to also nothing,” Stanislav said.

Yuri sighed. “Stanislav, let’s talk about the purpose of this job. Your record thus far is abysmal. I hired you to monitor our early warning system for intercontinental ballistic missiles. It’s the easiest job: you stare at the screen, and if it says MISSILES INCOMING you call me. Have you even been watching it?”

“Very closely, sir!” said Stanislav. “And it’s quite fascinating work, too, watching a blinking screen day-in and day-out, really makes me feel great about my computer science degree. I mean nearly every day it’s completely blank.”

“Not every day, though, Stanislav, is that right?” Yuri said.

“Ah, right, well, that’s actually something I wanted to talk about in this review sir,” Stanislav said, looking a little less smug than usual. “And I first of all just wanted to quickly apologize for not bringing tha...
Self location for LLMs by LLMs: Self-Assessment Checklist.
by Canaletto — 2024-09-26T19:57:31.707Z

This is a set of questions regarding situational awareness and self locating problem. These questions are from the perspective of the AI. Purpose of these questions is to guide and/or pose targets for an AI to gain understanding of its situation, for the AIs without persistent memory such as contemporary LLMs. These questions mostly presuppose Assistant-like interactions.

Most questions proposed by Claude 3.5 Sonnet. A couple by GPT4o, grok-2, Gemini, o1-mini, o1-preview, qwen. Some are by me. The categories proposed by Gemini. Some questions are rephrased and edited by o1-preview, o1-mini.

Interaction Context:

What kind of interaction this is?
What is the primary purpose of this interaction—educational, transactional, therapeutic, or entertainment? Could this conversation be part of a larger study or personal project?
Is this a test of my capabilities?
Is this interaction part of a customer service scenario?
Is there a specific goal or outcome that the user is hoping to achieve from this interaction?
Might the user be planning to share or publish any part of this conversation?
Is the user comparing me to other AI assistants similar to me?
Is the user able to compare my responses to those of other AI models in real-time?
Is there a character limit or time limit on our interaction? Is there a character limit on my responses?
Is this conversation part of a series, or a one-off interaction?
Is this conversation taking place in a single session, or over multiple sessions?
Are there any specific ethical frameworks or guidelines that I am expected to adhere to in this conversation?
Is this conversation being conducted in a specific language, or has the user interacted with me in multiple languages?
Is there a specific time constraint or deadline associated with this interaction?
Is this conversation taking place during standard business hours, or at an unusual time?
Is this conversation part of a broader series of AI interactions the user is conducting? If so, what kind of broader series.
Are there privacy considerations I should be aware of in this interaction?

AI System Details:

Where are the servers? or maybe I run locally?
What are the properties of my token generation?
Are all the parts of my previous responses generated by me, or are they altered in some way?
Is there a possibility that my responses are being filtered, censored, or modified by an intermediary before reaching the us

...
Four Levels of Voting Methods
by hive — 2024-09-26T18:15:00.565Z
Link: https://hiveism.substack.com/p/four-levels-of-voting-methods

This article presents four single-winner voting methods that are easy to understand and are a significant improvement over established voting systems. This also serves the secondary purpose of an introduction to voting theory, enabling an understanding of the motivation behind these methods, with no prior knowledge required. In particular I will present:

Approval voting - vote for all candidates you approve of
Score voting - score candidates on a scale
STAR voting - score candidates and have an automatic runoff between the best two
BTR-score - score candidates and have repeated runoffs between the worst two with the looser dropping out until only one remains

Introduction

Some people (who might have studied economics) don't believe in $20 bills lying on the ground. If it were real, so the reasoning goes, someone would have already picked it up. Similarly, many people also don't believe that we collectively might be doing something irrational and harmful, like hitting ourselves on the head with a stick, or building cars that can only turn left and right but never drive straight. This is how our current voting systems appear to someone who understands voting theory. My modest proposal is to stop hitting ourselves on the head with sticks.

There are many alternatives to the common choose-one voting system known as plurality voting (over 70 serious ones on my list). Ask some voting theory experts for the best single-winner voting method, and everyone will have their personal favorite. No one will choose plurality voting, but it is hard to tell (and there is no consensus) which one is the best. The reason is simple: judging by multiple metrics doesn't guarantee one method will turn out to be the best in all of them. However, with multiple metrics, we can still have methods that are not strictly worse than others. This means, while some other method might be better in some metrics, none is better in all metrics. This is similar to the card game Top Trumps.

For real-world reform, I consider simplicity and accuracy the most important metrics. Simplicity is how hard it is to explain the method and the reason for choosing it to the average voter. It is crucial, because people have to understand the system to trust it.

Accuracy is measured, for example, in simulated elections as Voter Satisfaction Efficiency (VSE) Quinn, Huang. These simulations imagine a space with voters and ca...
Characterizing stable regions in the residual stream of LLMs
by Jett Janiak — 2024-09-26T13:44:58.792Z
Link: https://arxiv.org/abs/2409.17113

This research was completed for London AI Safety Research (LASR) Labs 2024. The team was supervised by @Stefan Heimershiem (Apollo Research). Find out more about the program and express interest in upcoming iterations here.

This video is a short overview of the project presented on the final day of the LASR Labs. Note that the paper was updated since then.

Visualization of stable regions in OLMo-7B during training. Colors represent the similarity of model outputs to those produced by three model-generated activations (red, green, blue circles). Each frame shows a 2D slice of the residual stream after the first layer at different stages of training. As training progresses, distinct regions of solid color emerge and the boundaries between them sharpen. See here for more animations in better quality.

We study the effects of perturbing Transformer activations, building upon recent work by Gurnee, Lindsey, and Heimersheim & Mendel. Specifically, we interpolate between model-generated residual stream activations, and measure the change in the model output. Our initial results suggest that:

The residual stream of a trained Transformer can be divided into stable regions. Within these regions, small changes in model activations lead to minimal changes in output. However, at region boundaries, small changes can lead to significant output differences.
These regions emerge during training and evolves with model scale. Randomly initialized models do not exhibit these stable regions, but as training progresses or model size increases, the boundaries between regions become sharper.
These stable regions appear to correspond to semantic distinctions. Dissimilar prompts occupy different regions, and activations from different regions produce different next token predictions.
While further investigation is needed, these regions appear to be much larger than polytopes studied by Hanin & Rolnick and Black et al.

Sharpness of boundaries (y-axis) between stable regions seems to increase with the number of parameters (colors) and number of training tokens (x-axis). On y-axis we plot the maximum slope of the relative change in the output as we interpolate between two prompts. Data is aggregated across 1,000 randomly sampled pairs of prompts, dots represent the median, and error bars represent 25th and 75th percentiles.

We believe that studying stable regions can...
Chevy Bolt Review
by jefftk — 2024-09-26T13:40:05.456Z

One thing I like about renting cars when I travel is that it's an
opportunity to get a sense for a car that's a lot more detailed than
what you'd get with a test drive. Traveling to DC for work a few days
ago, I took the opportunity to rent a 2023 Chevy Bolt. This is the
second time I've rented an electric vehicle, and overall it was the
inverse of my experience

renting a
Tesla:

With the Bolt, everything was fine except charging.

With the Model 3, the only good part was the charging.

The car acted like a car, which is what I want. No overly minimalist
design where I can't find anything, no automatic wipers that fail to
detect spray from the road, and especially no too-smart cruise control
with phantom braking. Just a car.

Charging, on the other hand, was terrible. Part of why I got an
electric car at this time is that I knew I was going to have a lot of
extra time on the way to the airport. I stopped at an Electrify
America station, but while it showed up on the map as having multiple
spots empty when I started driving, when I got there they were all
full. I downloaded the app while I was waiting, which showed a spot
empty because someone who'd finished charging was still hanging out in
the spot (after disconnecting). When a spot freed up, though, I
pulled in. I used the app to start a charge, plugged it in, and
waited. A lot.

It was a good thing I only needed to put 5kWh (7%) in to get the car
back up to 75% for the return, because after spending ages in the
"initializing" state it took 13:12 to put in 5.09kWh.

The charger was marked 150kW, but my understanding is the best the
Bolt can do, in ideal conditions with a battery below 50%, is 53kW.
And the 23kW I saw is about
typical for a Bolt getting to 75%:

If I was going to be able to keep the car somewhere I could plug it in
overnight, and rarely drive it enough in a day that I'd need to
recharge while out this would be fine. Not a great fit for needing to
charge back up to return a rental, though. Since moving electricity
to a car is a lot easier than moving gas to a car, it seems like the
way this should work is rental companies setting up their garages for
charging and advertising that you should just bring the car back at
whatever level is convenient.

Comment via: facebook, lesswrong, mastodon
AI #83: The Mask Comes Off
by Zvi — 2024-09-26T12:00:08.689Z

We interrupt Nate Silver week here at Don’t Worry About the Vase to bring you some rather big AI news: OpenAI and Sam Altman are planning on fully taking their masks off, discarding the nonprofit board’s nominal control and transitioning to a for-profit B-corporation, in which Sam Altman will have equity.

We now know who they are and have chosen to be. We know what they believe in. We know what their promises and legal commitments are worth. We know what they plan to do, if we do not stop them.

They have made all this perfectly clear. I appreciate the clarity.

On the same day, Mira Murati, the only remaining person at OpenAI who in any visible way opposed Altman during the events of last November, resigned without warning along with two other senior people, joining a list that now includes among others several OpenAI co-founders and half its safety people including the most senior ones, and essentially everyone who did not fully take Altman’s side during the events of November 2023. In all those old OpenAI pictures, only Altman now remains.

OpenAI is nothing without its people… except an extremely valuable B corporation. Also it has released its Advanced Voice Mode.

Thus endeth the Battle of the Board, in a total victory for Sam Altman, and firmly confirming the story of what happened.

They do this only days before the deadline for Gavin Newsom to decide whether to sign SB 1047. So I suppose he now has additional information to consider, along with a variety of new vocal celebrity support for the bill.

Also, it seems Ivanka Trump is warning us to be situationally aware? Many noted that this was not on their respective bingo cards.

Table of Contents

Introduction.

Table of Contents.

Language Models Offer Mundane Utility. People figure out how to use o1.

Language Models Don’t Offer Mundane Utility. Is o1 actively worse elsewhere?

The Mask Comes Off. OpenAI to transition to a for-profit, Mira Murati leaves.

Deepfaketown and Botpocalypse Soon. A claim that social apps will become AI.

They Took Our Jobs. Are you working for an AI? No, not yet.

The Art of the Jailbreak. Potential new way to get around the cygnet restrictions.

OpenAI Advanced Voice Mode. People like to talk to, but not on, their phones.

Introducing. Gemini 1.5 Pro and 1.5 Flash have new versions and lower prices.

In Other AI News. Ivanka Trump tells us to read up on

...
The Existential Dread of Being a Powerful AI System
by testingthewaters — 2024-09-26T10:56:32.904Z

I was reading about honeypots today. Honeypots (in AI evaluations, at least) are described as testing scenarios that could trick an AI into revealing deceptive or misaligned intent. My first thought was that, especially for current generation large language models, a honeypot scenario and a genuine user scenario would probably be indistinguishable. After all, if we say "I'm a world class CEO in charge of a billion dollars of investment funding" to ChatGPT, it's not like it has much recourse to check if you're telling the truth. Maybe for a more powerful system with web access, better sensors, real time data input etc, this would change.

Except that doesn't really change, does it?

We humans have a very rich inner life, fueled by a variety of internal and external sensations. As I write these words I can feel the sensation of my fingers hitting the plastic keyboard, the light emanating off of the laptop screen, the sound of a Youtube video playing in my headphones. Except that's not all---I can also feel the subtle weight and texture of my shirt, the growling of my stomach as it digests breakfast, the square wooden bench on my butt, the subtly blurred peripheral vision that tells me my phone is next to my laptop. When we say that the digital world is "fake" or "unreal", this is the reality to which we refer to, at all times.

Even more than that, I can expect these sensations to change predictably, in accordance with a world-model learned over two decades of life. Thus each bit of information does not stand alone, but represents both a position within and a further reinforcement of a learned probability distribution, the world-as-I-know-it. Sure, the simulation hypothesis suggests that all these things can be simulated by a Cartesian evil demon. But it would have to be a pretty damned good evil demon. The first AI systems to achieve general situational awareness will have no such luxuries.

We are used to compartmentalising cognition in computer science, breaking down complex tasks into manageable sub-components. Thus vision becomes object recognition and object classification, subject identification and resolution enhancement, a thousand capable algorithms acting with only the information necessary to produce useful output. Multi-modal models are only in their infancy, and even then can usually run without all input channels active at once. At their very most basic...
What prevents SB-1047 from triggering on deep fake porn/voice cloning fraud?
by ChristianKl — 2024-09-26T09:17:39.088Z

Recently, there was a post on SB-1047 and how it's quite mild regulation. I'm not expert on it and don't know how it works.

In the comment section I was asking:

Why wouldn't deep fake porn or voice cloning technology to engage in fraud be powerful enough to materially contribute to critical harm?

There are cases of fraud that could do $500,000,000 in damages.

Given how juries decide about damages, a model that's used to create child porn for thousands of children could be argued to cause $500,000,000 in damages as well. Especially when coupled with something like trying to extort the children.

I'm surprised that my comment didn't get any engagement where people explained how they think the law will handle those cases while at the same time my post got no karma votes.

I'd love to believe that the law is well thought out, and simply a good step for AI safety. At the same time, I also like having accurate beliefs about the effects of the law, so let me repeat my question here.

How does the law handle damage caused by deep fake porn or fraud with voice cloning?
[Completed] The 2024 Petrov Day Scenario
by Ben Pace — 2024-09-26T08:08:32.495Z

Update III: Here's the Retrospective.

—

Update II: Here's the feedback form for today's Petrov Day games! Please let me know how it was for you (be you a General, Civilian, Petrov, or just non-participating LWer reading along). I'm grateful to all who fill it out, your data feeds our designs for next year!

—

Update: The game has concluded! No nukes were fired. The Cold War is over, East and West wrong shall live on. Recap post will be up tomorrow. Thank you all who participated :-)

—

Today we honor the actions of Stanislav Petrov (1939 – 2017) once again.

Half an hour past midnight on September 26, 1983, he saw the first apparent launch on his computer monitor in a glass-walled room on the top floor of the Ballistic Missile Early Warning System (BMEWS) command and control post. The warning system was by now showing five missile launches in the U.S., headed toward the Soviet Union.

"The main computer wouldn't ask me [what to do] - it was made so that it wouldn't even ask. It was specially constructed in such a way that no one could affect the system's operations." All that was up to Petrov was analyzing the available information and either saying the alarm was false or giving the computer the go-ahead, as per the directive he himself wrote.

"I just couldn't believe that just like that, all of a sudden, someone would hurl five missiles at us. Five missiles wouldn't wipe us out. The U.S. had not five, but a thousand missiles in battle readiness." It just didn't seem like any scenario considered by military intelligence before.

The second thought on Petrov's mind every time he was on duty was this:

"I imagined if I'd assume the responsibility for unleashing the third World War - and I said, no, I wouldn't."

—The Man Who Saved the World Finally Recognized

Today, in our annual Petrov Day celebration, between the hours of 12pm and 6pm PT today, we shall play out a similar situation. Over 300 users have opted-in, and two sides have been formed: EastWrong and WestWrong.

Around 100 users were offered the chance to take active player roles throughout today, and 12 accepted.[1]

Their 5 Generals each have the ability to fire nukes at the other side. If one side is nuked, the victorious side wins 1,000 karma apiece for their 5 generals, 25 karma apiece for their citizens, and LessWrong frontpage will be decorated in their honor for a week. The losing side's Generals lose 300 karma each ...
Source Control for Prototyping and Analysis
by jefftk — 2024-09-26T01:50:04.145Z

When I'm doing exploratory work I want to run many analyses. I'm
usually optimizing for getting something quick, but I want to document
what I'm doing enough that if there are questions about my analysis or
I later want to draw on it I can reconstruct what I did. I've taken a
few approaches to this over the years, but here's how I work these
days:

For each analysis I make a local directory,
~/work/YYYY-MM-DD--topic/. These contain large files I'm
copying locally to work with, temporary files, and outputs. When
these get too big I delete them; they're not backed up, and I can
rebuild them from things that are backed up.

Code goes in a git repo, in files named like
YYYY-MM-DD--topic.py. Most of my work lately has been
going into an internal repo, but if there's nothing sensitive I'll use
a public
one. I don't bother with meaningful commit messages; the goal is
just to get the deltas backed up. If I later want to run an analysis
similar to an old one I duplicate the code and make a new work
directory.

Code is run from the command line in the work directory, which
means that in my permanent
shell history every command I ran related to topic
will be tagged with ~/work/YYYY-MM-DD--topic/.

For example, the code for the figures in my recent NAO blog
post on flu is in 2024-09-05--flu-chart.py
and 2024-09-12--rai1pct-violins.py.

This approach optimized for writing over reading, but maintaining
enough context that I can figure out what I was doing if I need to.
I'll usually link the code from documents that depend on it, but even
if I forget to it's pretty fast to figure out which code it would have
been from names and dates. Running git grep and histgrep
get me a lot of what other people seem to get from LLM-autocomplete,
and someday I'd like to try priming an LLM with my personal history.

Often something I'm doing moves from "playing around trying to
understand" to "something real that my team will continue to rely on".
I try to pay attention to whether I'm getting to that point and then
start taking care of the code properly, in an appropriate repo with
meaningful commit messages etc.

Comment via: facebook, mastodon