trentmkelly/LessWrong-43k huggingface dataset:
Reformative Hypocrisy, and Paying Close Enough Attention to Selectively Reward It.
by Andrew_Critch — 2024-09-11T04:41:24.872Z

People often attack frontier AI labs for "hypocrisy" when the labs admit publicly that AI is an extinction threat to humanity. Often these attacks ignore the difference between various kinds of hypocrisy, some of which are good, including what I'll call "reformative hypocrisy". Attacking good kinds of hypocrisy can be actively harmful for humanity's ability to survive, and as far as I can tell we (humans) usually shouldn't do that when our survival is on the line. Arguably, reformative hypocrisy shouldn't even be called hypocrisy, due to the negative connotations of "hypocrisy". That said, bad forms of hypocrisy can be disguised as the reformative kind for long periods, so it's important to pay enough attention to hypocrisy to actually figure out what kind it is.

Here's what I mean, by way of examples:

***

0. No Hypocrisy —
Lab: "Building AGI without regulation shouldn't be allowed. Since there's no AGI regulation, I'm not going to build AGI."
Meanwhile, the lab doesn't build AGI. This is a case of honest behavior, and what many would consider very high integrity. However, it's not obviously better, and arguably sometimes worse, than...

1. Reformative Hypocrisy:

Lab: "Absent adequate regulation for it, building AGI shouldn't be allowed at all, and right now there is no adequate regulation for it. Anyway, I'm building AGI, and calling for regulation, and making lots of money as I go, which helps me prove the point that AGI is powerful and needs to be regulated."
Meanwhile, the lab builds AGI and calls for regulation. So, this is a case of honest hypocrisy. I think this is straightforwardly better than...

2. Erosive Hypocrisy:
Lab: "Building AGI without regulation shouldn't be allowed, but it is, so I'm going to build it anyway and see how that goes; the regulatory approach to safety is hopeless."
Meanwhile, the lab builds AGI and doesn't otherwise put efforts into supporting regulation. This could also be a case of honest hypocrisy, but it erodes the norm that AGI should regulated rather than supporting it.
Some even worse forms of hypocrisy include...

3. Dishonest Hypocrisy, which comes in at least two importantly distinct flavors:

a) feigning abstinence:
Lab: "AGI shouldn't be allowed."
Meanwhile, the lab secretly builds AGI, contrary to what one might otherwise guess according to their stance that building ...
A necessary Membrane formalism feature
by ThomasCederborg — 2024-09-10T21:33:09.508Z

Summary

Consider a Membrane that is supposed to protect Steve. An AI wants to hurt Steve but does not want to pierce Steve's Membrane. The Membrane ensures that there is zero effect on predictions of things inside the membrane. The AI will never take any action that has any effect on what Steve does or experience. The Membrane also ensures that the AI will not have access to any information that is within Steve's Membrane. One does not have to be a clever AI to come up with a strategy that an AI could use to hurt Steve without piercing this type of Membrane. The AI could for example create and hurt minds that Steve cares about, but not tell him about it (in other words: ensure that there is zero effect on predictions of things inside the membrane). If Bob knew Steve before the AI was built. And Bob wants to hurt Steve. Then Bob presumably knows Steve well enough to know what minds to create (in other words: there is no need to have any access to any information that is within Steve's Membrane).

This illustrates a security hole in a specific type of Membrane formalism. This particular security hole can of course be patched. But if a Membrane is supposed to actually protect Steve from a clever AI, then there is a more serious issue that needs to be dealt with: a clever AI that wants to hurt Steve will be very good at finding other security holes. Plugging all human-findable security holes is therefore not enough to protect Steve. This post explores the question: ``what would it take to create a Membrane formalism that actually protects Steve in scenarios where Steve shares an environment with a clever AI (without giving Steve any special treatment)?''.

The post does not propose any such formalism. It instead describes one necessary feature. In other words: it describes a feature that a Membrane formalism must have, in order to reliably protect Steve in this context. Very briefly and informally: the idea is that safety requires the prevention of scenarios where a clever AI wants to hurt Steve. For Steve's Membrane to ensure this, it must be extended to encompass any adoption of Steve-referring preferences by a clever AI.

Thanks to Chris Lakin for regranting to this research.

Protecting an individual that shares an environment with a clever AI is difficult

One can construct Membrane formalisms for all sorts of reasons. The present post will cover one specific ca...
Formalizing the Informal (event invite)
by abramdemski — 2024-09-10T19:22:53.564Z

Formalizing the Informal

One way to view MIRI's Agent Foundations research is that it saw the biggest problem in AI safety as "human preferences are informal, but we need to somehow get formal guarantees about them" -- and so, in response, it set out to make a formal-informal bridge.

Recently, I’ve been thinking about how we might formally represent the difference between formal and informal. My prompt is something like: if we assume that classical probability theory applies to “fully formal” propositions, how can we generalize it to handle “informal” stuff?
I’m going to lead a discussion on this tomorrow, Wednesday Sept. 11, at 11am EDT (8am Pacific, 4pm UK).

Discord Event link (might not work for most people):

https://discord.com/events/1237103274591649933/1282859362125352960

Zoom link (should work for everyone):

https://us06web.zoom.us/j/6274543940?pwd=TGZpY3NSTUVYNHZySUdCQUQ5ZmxQQT09

You can support my work on Patreon.
AI #80: Never Have I Ever
by Zvi — 2024-09-10T17:50:08.074Z

(This was supposed to be on Thursday but I forgot to cross-post)

Will AI ever make art? Fully do your coding? Take all the jobs? Kill all the humans?

Most of the time, the question comes down to a general disagreement about AI capabilities. How high on a ‘technological richter scale’ will AI go? If you feel the AGI and think capabilities will greatly improve, then AI will also be able to do any particular other thing, and arguments that it cannot are almost always extremely poor. However, if frontier AI capabilities level off soon, then it is an open question how far we can get that to go in practice.

A lot of frustration comes from people implicitly making the claim that general AI capabilities will level off soon, usually without noticing they are doing that. At its most extreme, this is treating AI as if it will only ever be able to do exactly the things it can already do. Then, when it can do a new thing, you add exactly that new thing.

Realize this, and a lot of things make a lot more sense, and are a lot less infuriating.

There are also continuous obvious warning signs of what is to come, that everyone keeps ignoring, but I’m used to that. The boat count will increment until morale improves.

The most infuriating thing that is unrelated to that was DOJ going after Nvidia. It sure looked like the accusation was that Nvidia was too good at making GPUs. If you dig into the details, you do see accusations of what would be legitimately illegal anti-competitive behavior, in which case Nvidia should be made to stop doing that. But one cannot shake the feeling that the core accusation is still probably too much winning via making too good a product. The nerve of that Jensen.

Table of Contents

Introduction.

Table of Contents.

Language Models Offer Mundane Utility. Sorry, what was the question?

Language Models Don’t Offer Mundane Utility. A principal-agent problem?

Fun With Image Generation. AI supposedly making art, claims AI never will.

Copyright Confrontation. OpenAI asks for a mix of forgiveness and permission.

Deepfaketown and Botpocalypse Soon. How to fool the humans.

They Took Our Jobs. First it came for the unproductive, and the call centers.

Time of the Season. If no one else is working hard, why should Claude?

Get Involved. DeepMind frontier safety, Patel thumbnail competition.

Introducing. Beijing AI Safety and Govern

...
The Best Lay Argument is not a Simple English Yud Essay
by J Bostock — 2024-09-10T17:34:28.422Z

Epistemic status: these are my own opinions on AI risk communication, based primarily on my own instincts on the subject and discussions with people less involved with rationality than myself. Communication is highly subjective and I have not rigorously A/B tested messaging. I am even less confident in the quality of my responses than in the correctness of my critique.

If they turn out to be true, these thoughts can probably be applied to all sorts of communication beyond AI risk.

Lots of work has gone into trying to explain AI risk to laypersons. Overall, I think it's been great, but there's a particular trap that I've seen people fall into a few times. I'd summarize it as simplifying and shortening the text of an argument without enough thought for the information content. It comes in three forms. One is forgetting to adapt concepts for someone with a far inferential distance; another is forgetting to filter for the important information; the third is rewording an argument so much you fail to sound like a human being at all.

I'm going to critique three examples which I think typify these:

Failure to Adapt Concepts

I got this from the summaries of AI risk arguments written by Katja Grace and Nathan Young here. I'm making the assumption that these summaries are supposed to be accessible to laypersons, since most of them seem written that way. This one stands out as not having been optimized on the concept level. This argument was below-aveage effectiveness when tested.

I expect most people's reaction to point 2 would be "I understand all those words individually, but not together". It's a huge dump of conceptual information all at once which successfully points to the concept in the mind of someone who already understands it, but is unlikely to introduce that concept to someone's mind.

Here's an attempt to do better:

So far, humans have mostly developed technology by understanding the systems which the technology depends on.
AI systems developed today are instead created by machine learning. This means that the computer learns to produce certain desired outputs, but humans do not tell the system how it should produce the outputs. We often have no idea how or why an AI behaves in the way that it does.
Since we don't understand how or why an AI works a certain way, it could easily behave in unpredictable and unwanted ways.
If the AI is powerful, then the consequences of

...
Economics Roundup #3
by Zvi — 2024-09-10T13:50:06.955Z

I am posting this now largely because it is the right place to get in discussion of unrealized capital gains taxes and other campaign proposals, but also there is always plenty of other stuff going on. As always, remember that there are plenty of really stupid proposals always coming from all sides. I’m not spending as much time talking about why it’s awful to for example impose gigantic tariffs on everything, because if you are reading this I presume you already know.

The Biggest Economics Problem

The problem, perhaps, in a nutshell:

Tess: like 10% of people understand how markets work and about 10% deeply desire and believe in a future that’s drastically better than the present but you need both of these to do anything useful and they’re extremely anticorrelated so we’re probably all fucked.

In my world the two are correlated. If you care about improving the world, you invest in learning about markets. Alas, in most places, that is not true.

The problem, in a nutshell, attempt number two:

Robin Hanson: There are two key facts near this:

Government, law, and social norms in fact interfere greatly in many real markets.

Economists have many ways to understand “market failure” deviations from supply and demand, and the interventions that make sense for each such failure.

Economists’ big error is: claiming that fact #2 is the main explanation for fact #1. This strong impression is given by most introductory econ textbooks, and accompanying lectures, which are the main channels by which economists influence the world.

As a result, when considering actual interventions in markets, the first instinct of economists and their students is to search for nearby plausible market failures which might explain interventions there. Upon finding a match, they then typically quit and declare this as the best explanation of the actual interventions.

Yep. There are often market failures, and a lot of the time it will be very obvious why the government is intervening (e.g. ‘so people don’t steal other people’s stuff’) but if you see a government intervention that does not have an obvious explanation, your first thought should not be to assume the policy is there to sensibly correct a market failure.

No Good Very Bad Capital Gains Tax Proposals

Kamala Harris endorses Biden’s no-good-very-bad 44.6% capital gains tax rate proposal, including...
Amplify is hiring! Work with us to support field-building initiatives through digital marketing
by gergogaspar — 2024-09-10T08:56:42.822Z

See the accompanying launch post here to read more about the project's background.

TLDR:

Amplify is an early-stage project supporting local Effective Altruist and AI Safety local groups as well as other field-building organisations. We have recently received a grant from The Effective Altruism Infrastructure Fund to turn what started as a passion project into an established organisation. We are adding to our team of open-minded, highly driven, altruistic and collaborative team members to keep up with our clients and their growing needs. Deadline: 23rd September, 2024 Apply here!
With any questions, please email gergo@enais.co
Who are we looking for?

We plan to hire someone working part-time (20-30 hours per week) on this project for 6-10 months. We are looking for either a senior or junior marketing specialist. You can learn more about the roles below:

Senior Marketing Specialist

The Senior Marketing Specialist is responsible for the planning and implementation of client requests we receive from non-profits. Familiarity and experience with best practices for paid ads on Meta and LinkedIn and an eagerness to learn and implement the most cutting-edge techniques for social media campaigns are core to being successful in this position.
We are growing, and priorities can change rapidly. The ideal candidate will thrive in a fast-paced environment and be able to shift direction with ease while maintaining flexibility in working with other teams and projects. Additionally, they are hungry to learn and ready to take their experience to the next level.
This role will:
-Implement and manage paid social campaigns in Meta and LinkedIn
-Research and evaluate campaigns to identify and execute optimisations to improve the efficiency and performance of campaigns
-Work to support the strategy through campaign setup, configuration, and reporting of paid social campaigns for our clients
-Create campaign performance reports, including insights, to share with clients
-Research and source new clients and opportunities
-Work with the Director to proactively apply best practices and new recommendations to client campaigns
-Implement A/B testing to gather data that can be used for better informing campaign performance and optimization
-Support the quality control process ensuring the accuracy of the campaign efforts

Experience and Skill Sets:

Must have:
-3+ years of marketing experience in setting up an...
What bootstraps intelligence?
by invertedpassion — 2024-09-10T07:11:21.819Z

Cross post from: https://invertedpassion.com/what-bootstraps-intelligence/

A musing on how intelligence comes to be.

The bedrock of intelligence is abstractions – the thing we do when we throw away a lot of information and just emphasise on a subset of it (e.g. calling that thing an apple instead of describing all its atoms and their x, y, z positions).

But where does the drive to form abstractions comes from? What if it rose from our desire to communicate with others? Since communication bandwidth is always limited, we are driven to find most efficient way of getting an idea across which leads to abstractions. Imagine a world where energy and time is unlimited, we might be communicating all x,y,z positions of things instead of putting labels on them.

We form abstractions when we notice some aspects of situations that we have encountered so far that we expect to find again in future situations. So the label “apple” becomes an efficient placeholder for all apple-like objects that are sweet and can be eaten.

It’s interesting to note that

languages navigate a tradeoff between efficiency and fidelity

. We want to communicate what we mean but we cannot spend enormous amount of energy on it. But we can’t spend too less energy too since communication channels are noisy and the recipient may not understand what we mean.

But since infinitely many abstractions for the same information is possible, how do we select which one to use? For example, why did the Arabic numbering system win over other alternative abstractions of numbers? Well, our

abstractions are grounded by usefulness

. In math, base-10 number system beat base-9 number system because we have 10 fingers and hence easier to count in base-10. (Of course, historical accidents may lead to initially inefficient systems such as Gregorian calendar to get adapted).

Why do we need to communicate useful abstractions? Because we are driven to co-operate by virtue of our shared gene pool. We know sharing useful abstractions are going to help the copies of our genes spread.

In summary, intelligence may have bootstrapped via the drive to help copies of ourselves survive better by sharing useful tools/abstractions.

PS; I find it interesting that in almost all AGI architectures, there’s no notion of replication, genes, survival, efficiency, inter-agent communication, energy, etc. – which might ultimately prove to be a useful framewor...
Physical Therapy Sucks (but have you tried hiding it in some peanut butter?)
by Declan Molony — 2024-09-10T05:54:47.000Z

The worst part about physical therapy is not knowing if it’s working. The timescale for improvement is dreadfully long and the day-to-day changes are imperceptible.

65% of people who start physical therapy fail to adhere to their prescribed regimen. One of the primary factors for non-compliance is the lack of positive feedback. Physical therapy goes against human nature and our desire for immediate results.

After developing a shoulder issue, I experienced the same thing many people do: performing seemingly useless exercises and hoping my shoulder would improve rapidly. Without noticeable progress, I quickly became one of the 65%. The whole thing just wasn’t satisfying enough for me to want to do it.

What changed, and finally healed my shoulder, was hiding my physical therapy in some peanut butter.

Growing up, my family dog refused to take her pills. But that changed when we started hiding her pills in peanut butter. Remembering this experience gave me an idea for physical therapy.

Hiding something in peanut butter: surrounding and hiding a difficult task between more enjoyable tasks.

While physical therapy sucks, I do enjoy lifting weights at the gym. I had taken a years-long sabbatical from the gym because of my injury. But I returned when I realized I could work out my lower body without irritating my shoulder. In between squat sets, I began sneaking in some rotator cuff work.

At the end of my workout, I rewarded myself by doing a chest press movement with the lightest of dumbbells that wouldn’t aggravate my shoulder.

I was already in the habit of writing my reps & sets in an exercise journal (thanks

Tim Ferriss

), so I also started recording my shoulder pain level each week.

What gets measured gets managed. — Peter Drucker

My hope was that while I may not feel a perceptible difference in pain day-to-day, perhaps over time a descending trend in my pain level would emerge.

I started at a pain level of ~3.5/10. After four months I became pain free. As author James Clear likes to say in his book Atomic Habits:

What is immediately rewarded is repeated. What is immediately punished is avoided.

Perhaps the reason physical therapy compliance is so low is because too much focus is given to the mechanics of exercises and not enough emphasis is placed on how behavior change happens. Hiding a difficult task between enjoyable tasks (for me that's exercise; your p...
Simon DeDeo on Explore vs Exploit in Science
by Elizabeth — 2024-09-10T03:40:08.311Z

Good news everyone: I’m getting into podcasting!

As part of my investigation into how scientific paradigms are created, I hosted a talk by Simon DeDeo, Professor Of Many Topics at Carnegie Mellon and the Santa Fe Institute, recorded at the recent ILIAD conference in Berkeley. He discusses the idea of explore vs exploit in science. Spoiler: he comes down very hard on the exploit side, based primarily on a case study of Darwin.

For the video averse among you, here’s the transcript.

I have two more episodes in editing, but am on the hunt for more guests. I’m looking to talk to scientists or ex-scientists about how ideas are created and spread in their field. Bonus points if they can talk about Chaos, Complexity, or Catastrophe Theory in particular.

Thanks to my Lightspeed Grants and my Patreon patrons for supporting this work.