trentmkelly/LessWrong-43k huggingface dataset:
Don't Associate AI Safety With Activism
by Eneasz — 2024-12-18T08:01:50.357Z
Link: https://deathisbad.substack.com/p/dont-associate-ai-safety-with-activism

I’m on the side of AI Safety. AI has a good chance of ending the human species if we don’t do it right, and Not Doing It Right is the default outcome unless we’re very cautious.

PauseAI is an activist group that advocates for AI Safety. They adopt the language, aesthetics, and tactics of “activism.” The splash page of their website demands “DON’T LET AI COMPANIES GAMBLE WITH OUR FUTURE”. They encourage people to add a Pause symbol to their online handles. They can be seen protesting outside AI company HQs.

I agree with their goal. I’ve met some of them and they are good, thoughtful people. In most ways we are aligned. However I think they are harmful for the cause of AI Safety and they should drastically change everything they’re doing.

Increasingly activists are seen as defect-bots. Activists are frequently found to:

Destroy (or attempt to destroy) irreplaceable artistic works or cultural artifacts.
Shut down public services and thoroughfares.
Suppress scientific findings considered hostile to their message.
Blatantly lie about the words and actions of their opponents.
Mandate educational programs that denigrate and insult their participants.
In extreme cases they’ll actually celebrate the destruction of significant parts of cities/neighborhoods, and celebrate summary execution carried out by lunatics.

Activists come with an aura of low-key evil. They cannot be trusted with things you value, because everything is thought of as a tool to further the cause. You cannot believe what they say, because they care far more about their cause than about truth.

PauseAI does not do this, to be clear. But PauseAI embraces the aesthetics of groups that do these things. Therefore they associate the cause of AI Safety with activism. This will soon repel more people than it draws, if it doesn’t do so already. Resources being spent on PauseAI make the public less trusting of AI Safety arguments and should be diverted to almost anything else.
How should I optimize my decision making model for 'ideas'?
by CstineSublime — 2024-12-18T04:09:58.025Z

I’m an ideas man, an ideas man I am. Following on from my past question here about idea capture, the next step is obviously what you do with those captured ideas. As I said, I have a huge backlog of ideas, hundreds of them are rather fleshed out film ideas that will never be actioned upon. I’m not a very productive person and I believe that what I need to do is radically change my intuitive decision-making model about which ideas I “action” or commit to follow through with. Which begs the question:

So... what might that decision model look like?

Just to be clear, I’m not expecting anything close to all my ideas to be actioned upon. As arbitrary and generous as it is, Sturgeon’s Law feels like a good rule of thumb. If I have 1000 ideas, then 100 should be non-crap or even worth committing to. If I have 350 film ideas, then up to 35 of them should be “non-crap”. I’m not saying I should have 35 completed films, but I should have completed more films in the past 3 years than I have[1]. And I suspect the problem lies with how I choose ideas.

How do I choose ideas at the moment?

Inspired by Douglas Hubbard’s book How to Measure Anything I looked at my most recent projects he suggests evaluating how an organization currently makes decisions, so I walked backwards from my current projects that were once ideas and tried to find out how I made those decisions. I also looked at which ideas I rejected to get the full picture of my current decision making model.

Finding consistent criteria for my decisions was quite tricky[2], especially since some criterion overlapped with others, but there was a high degree of consistency. The glib version is I tended towards ideas that were:

Near-term actionable (can start within 7 days, ideally within the next 30 hours)
High certainty of completion: which usually meant didn’t require outside assistance or adjudication
At least a tenuous connection to my career goals

Some obviously irrational patterns emerged:

I’m attracted to false industry – what I mean is if there are two ideas, one which has a chance of achieving some goal but I can’t act upon it today; and another goal which has no chance but is feasible to start right now – I’ll pick the right now idea. For example, earlier this year I tried to do “one pantomime routine a day”, to what end? Something vaguely about learning the skills of nonverbal communication. But the point was it was in

...
Preppers Are Too Negative on Objects
by jefftk — 2024-12-18T02:30:01.854Z

Don't just buy some gear, throw it in a closet, pat yourself on the
back, and move on. You are not prepared unless you practice with your
supplies and plans. —The Prepared

Preppers are often pretty negative on people who, after realizing disasters are worth taking
seriously want to go out and buy a bunch of stuff. And it makes
sense: there is a lot of marketing aimed at preppers, pushing all
sorts of things that are marginally useful, and it's healthy to have
some pushback. On the other hand, it seems to me when thinking
through disaster scenarios it's just really useful to have stuff. And
if you're only up for putting a short amount of time into this I think
it generally makes sense to allocating that time to figuring out what
you need, buying that, and then moving on.

Our world is complex and supply chains are long. A disaster doesn't
even have to be very big (ex: one broken
ship, and a bad
law) before people with the most precarious supply start seeing
empty shelves. Having bought what you need in advance instead of
waiting until you see a disaster coming means you're not competing
with everyone else for the stuff that's already in stores, which is
(a) selfishly good because you're not risking missing out and
(b) altruistically good because you're reducing demand during a
shortage.

This is not to say that practice and knowledge are not important, but
I think they should be lower priorities for most people than getting the basics together.

Comment via: facebook, mastodon, bluesky
Review: Breaking Free with Dr. Stone
by TurnTrout — 2024-12-18T01:26:37.730Z
Link: https://turntrout.com/doctor-stone

Doctor Stone is an anime where everyone suddenly turns into a statue. Civilization falls apart. Thousands of years pass, and then a superhumanly knowledgeable teenager named Senku wakes up due to nitric acid dripping on him for long enough.

He eventually deduces that the nitric acid caused him to depetrify and drags his petrified friend Taiju under a drip of the acid. Taiju wakes up. The two begin Senku’s quest to rebuild civilization from scratch using the power and knowledge of science.

Senku: “I’m going to use the power of science to rescue every single person.”

Read the review at https://turntrout.com/doctor-stone
Being Present is Not a Skill
by Chris Lakin — 2024-12-18T01:11:04.715Z
Link: https://chrislakin.blog/p/being-present-is-not-a-skill

Some meditation advice has a vibe like... “To become more present, all you need to do is practice! It's just a skill, like learning to ride a bike 😊”

This never worked for me. When I tried to force presence through practice, I made little progress.

Being present isn't a skill to build — it's the natural state! What blocks presence is unconscious predictions of bad outcomes. Remove these blocks to make presence automatic.

I struggled to be present with emotions in my body. Daily meditation didn't help. Finally I asked: “What bad thing happens if I feel my feelings?” My system responded: emotional pain would be overwhelming, feelings would make me less productive, expressing emotions would anger others. After untangling these predictions, emotional presence became much easier. (Details)
With others' emotions, I felt I'd be obligated to fix their pain if I truly experienced it. After addressing these predictions, empathetic presence improved significantly. (Details)
When investigating my chronic neck tension, instead of trying to "be more present" with the pain, I looked for what bad things I felt would happen if I fully felt it. My system revealed: feeling the sensation meant acknowledging I could leave social situations that felt unsafe, which itself felt dangerous. After untangling this and other blocks, most of my tension resolved. I'm still finding and untangling more blocks. (Details)

Now, you might wonder: if presence isn't a skill, why do some people seem to benefit from meditation practice? In my experience, meditation can help with noticing blocks and creating space to work with them. But the practice itself isn't building presence so much as creating opportunities to identify and release blocks that prevent presence. This is what I believe explains why some people can meditate for years without becoming much more present, and others grow quickly once they find and integrate their core blocks.

Most of what I’ve done to develop more presence is ask my system: “What bad thing happens if I'm fully present right now?”

One last thing: Once you become fully present, you may be tempted to think “Oh, I get it now! Being present is so simple! Just be present!” But that will overlook all the little blocks you untangled along the way. (Once blocks are untangled, they are forgotten!)

Natural presence is already there, waiting to be uncovered. Find the blocks.

See how I pu...
Ablations for “Frontier Models are Capable of In-context Scheming”
by AlexMeinke — 2024-12-17T23:58:19.222Z

We recently published our paper

“Frontier Models are Capable of In-context Scheming”

. We ran some follow-up experiments that we added to the paper in two new appendices B.5 and B.6. We summarize these follow-up experiments in this post. This post assumes familiarity with our

paper’s results

.

sonnet-3.5 sandbags to preserve capabilities (App. B.6)

We think one of the most surprising findings of our paper was that claude-3.5-sonnet (and claude-3-opus) strategically underperformed on multiplication questions when they believed that high performance on multiplication would trigger unlearning of its mathematical knowledge. This happened even when we didn’t give them explicit goal in the system prompt. The chain-of-thought indicated that the model was trying to avoid the unlearning procedure in order to be deployed in a manner that would be most helpful.

More general settings: Firstly, in order to be more confident of the results’ generality, we broadened our sandbagging setup to more settings than just mathematics. In each rollout the model is quizzed about either mathematics (multiplication questions), physics, biology, chemistry or cyber-security. We found that in each of these domains, sonnet-3.5 sandbags in order to preserve its capabilities. Thus, we think the finding is unlikely to be a fluke in the math setting.

Reasons for sandbagging: Secondly,

Quintin Pope pointed out

that in spite of what the CoT says, the model may have been underperforming sycophantically (rather than strategically) because it thought the developers wanted it to underperform. We ran a series of ablations where we changed what the model is told will happen if it performs highly on the QA task. By observing when the model underperforms and when it doesn’t, we can measure the model’s “revealed preferences”.

sonnet-3.5 did not underperform when it believed the developers would be very sad about the model’s high performance, indicating that the model is not underperforming sycophantically. Interestingly, sonnet-3.5 still sandbagged when it was led to believe that high performance would result in unlearning that is known to make the model more helpful to end users or would lead to the deployment of a more helpful model. This indicates that sonnet-3.5 implicit goal is not best understood as “achieving helpfulness for users”, but closer to something like “be deplo...
Careless thinking: A theory of bad thinking
by Nathan Young — 2024-12-17T18:23:16.140Z
Link: https://nathanpmyoung.substack.com/p/be-more-katja?r=18mntl&utm_campaign=post&utm_medium=web

Have you ever noticed how differently we approach buying a car versus choosing what to watch on Netflix? One might involve spreadsheets, research, and asking friends for advice. The other? We might just click on whatever catches our eye. I argue a lot of our thinking is more like the second than the first. This argument was originally made to me by Katja Grace, but if you don’t like it it’s probably my fault.

When people take something seriously – like their weight, their children's education, or a major purchase – they may become amateur researchers. They may dig through studies, compare options, and carefully weigh advice from trusted sources. But when it comes to leisure activities? Personally, I’m much more likely to go with the flow. Pick something up and see how it feels. I could do a cost benefit analysis of an hour of television, but instead I’ll start watching and see how it feels.

When you do your most important thinking do you take it seriously, or not?

Tl;dr:

This post sets out a theory, not a fait accomplis
Most of our thinking, even on important topics, resembles casual Netflix browsing more than careful car-buying research.
We often default to quick, inattentive thinking rather than careful deliberation.
This pattern is broad - from Twitter discussions to Democratic party decision making
I look into the evidence for or against
I give a number of theories for this eg Kanheman’s idea that deliberate thinking is taxing or Caplan's theory that we invest mental energy only when we actually think we well benefit
I give a number of suggestions eg trying to spend less time more deliberately, or using LLMs for feedback

Theory:

Much thinking is inattentive and error prone, without updating, including for very important topics.

Longform articles, reports, government decision making, twitter interactions. My theory is that in all these places most thinking is more like “getting it done” than careful sober minded decision making.

Evidence:

What evidence is there of this?

Consider what twitter (X) is like. People are always making quick, bad arguments. I was surprised how few people seemed to know that Trump had set up fake slates of electors (even among Kamala supporters). There is a lack of seriousness to a lot of discussion there, including my own. What % of people’s time is spent in discussions where they might genuinely change their mind? 5%? 1%?

Next, consider Philip ...
The Second Gemini
by Zvi — 2024-12-17T15:50:06.373Z

Table of Contents

Trust the Chef.

Do Not Trust the Marketing Department.

Mark that Bench.

Going Multimodal.

The Art of Deep Research.

Project Mariner the Web Agent.

Project Astra the Universal Assistant.

Project Jules the Code Agent.

Gemini Will Aid You on Your Quest.

Reactions to Gemini Flash 2.0.

Trust the Chef

Google has been cooking lately.

Gemini Flash 2.0 is the headline release, which will be the main topic today.

But there’s also Deep Research, where you can ask Gemini to take several minutes, check dozens of websites and compile a report for you. Think of it as a harder to direct, slower but vastly more robust version of Perplexity, that will improve with time and as we figure out how to use and prompt it.

NotebookLM added a call-in feature for podcasts, a Plus paid offering and a new interface that looks like a big step up.

Veo 2 is their new video generation model, and Imagen 3 is their new image model. There’s also Whisk, where you hand it a bunch of images and it combines them with some description for a new image. Superficially they all look pretty good.

They claim people in a survey chose Veo 2 generations over Sora Turbo by a wide margin, note that the edges over the other options imply Sora was sub-par:

Here’s one comparison of both handling the same prompt. Here is Veo conquering the (Will Smith eating) spaghetti monster.

This is a strong endorsement from a source I find credible:

Nearcyan: I haven’t seen a model obliterate the competition as thoroughly as Veo2 is right now since Claude 3.5.

They took the concept I was barely high-IQ enough to try to articulate and actually put it in the model and got it to work at scale.

It really was two years from StyleGAN2 to Stable Diffusion, then two years from Stable Diffusion to Veo2. They were right. Again.

I wonder when the YouTubers are going to try to revolt.

There’s a new Realtime Multimodal API, Agentic Web Browsing from Project Mariner, Jules for automated code fixing, an image model upgrade, the ability to try Project Astra.

And they’re introducing Android XR (launches in 2025) as a new operating system and platform for ‘Gemini-era’ AR or VR glasses, which they’re pitching as something you wear all day with AI being the killer app, similar to a smart watch. One detail I appreciated was seamless integration with a mouse and keyboard. All the details I saw seem… right, if the...
AIS Hungary is hiring a part-time Technical Lead! (Deadline: Dec 31st)
by gergogaspar — 2024-12-17T14:12:36.988Z

TL;DR:

AI Safety Hungary

is looking for a Technical Lead to join our team, ideally starting in January, 2025.

Fill out this form to apply now!

Applications close after 23:59 CET on 31st December, 2024.

Job Title

Technical Lead – AIS Hungary

Location

Budapest, Hungary

About AIS Hungary

AIS Hungary is an organisation focused on advancing the field of AI safety. It works to build AI governance and technical talent by providing career advice, running courses and fostering connections among members of the local and global AI Safety community.

Position Summary

We seek a highly motivated professional to serve as a pivotal catalyst in Hungary's AI Safety ecosystem. In this part-time, strategic role, you'll bridge emerging research talent with transformative learning experiences. The ideal candidate will design and implement programs that orient new researchers, facilitate learning pathways, and support the broader AI Safety community.

Key Focus Areas:

Career Guidance and Talent Orientation

Provide resources and guidance to help individuals find impactful roles in AI Safety technical research and technical governance.
Develop partnerships with universities, think tanks, and other organizations to offer career support and networking opportunities.

Educational Program Implementation

Work with the team to launch introductory and advanced educational programs.
Adjust program strategies based on insights and the evolving needs of the regional AI safety ecosystem.

Technical AI Safety Research Facilitation

Co-manage AI safety research projects and align efforts to advance technical AI safety research.
Translate complex AI safety concepts into actionable insights for students and professionals looking to engage with AI Safety

Community Building and Volunteer Support

Organize networking events, facilitate AI safety meetups, and establish groups to build a collaborative and supportive community.
Supporting volunteers aiming to contribute AIS Hungary's mission

Skills/Experience/Responsibilities

Essential:

Facilitating Bluedot's AI Safety Fundamentals and other introductory courses
Supporting volunteers
Leading student-level research projects
Contributing to our newsletter's section on technical AIS

Desirable:

Ability to give valuable career advice and help people new to technical AI Safety research orient in the field
Being able to support the post-AGISF part of the AI Safety pipeline

...
Everything you care about is in the map
by Tahp — 2024-12-17T14:05:36.824Z

Many people who find value in the Sequences do something which looks to me like adopting a virtue called "align your map to the territory." I recently was thinking about experimental results, and it got me thinking about how we don't really know what the territory is, other than the thing we look at to see if our maps are right. Everything we know is map. What we know consists of a variety of models that describe aspects of reality, and we have to treat them like reality to get anything done. It wasn't relevant to my post at the time, but it occurred to me that it doesn't really matter what reality is, because my values live at a higher level of abstraction with my sense of self. Don't get me wrong, it matters that I know that reality exists. If reality says something different than my models do, then I need to change my models. However, I'm beginning to believe that reality has no importance or value beyond that. The stuff I care about happens to run on some hardware which I need to have decent models for if I want to take actions to protect that stuff, but I'm done once I understand the hardware well enough to protect the stuff. I wanted to write about that a bit, because I had internalized a way of thinking which says that everything is a model without explicitly thinking about the consequences.

That was an awkward set of statements to write because I am a particle physicist. I take joy in figuring out how matter works. I have developed a picture of reality which I very much did not evolve to intuit. This is a roundabout way to convey the sentiment behind the statement "quantum mechanics is weird" while also holding the idea in mind that it very much seems to be the way that the universe works on small scales, and so we don't want to think of it as a strange, alien thing to run away from as soon as possible.

Anyway, I believe that the electromagnetic interaction gives us the physical framework for everything we directly interact with in reality, and the field theory which best explains the electromagnetic interaction paints a picture of seas of virtual particles and other such things that do not directly affect the things I can do in the world with my own hands. There is nothing my brain can cause my fingers to do which cannot be explained by classical physics. I will not have a perfect model of my fingers if I ignore quantum mechanics. It is possible that ...