trentmkelly/LessWrong-43k huggingface dataset:
AI #79: Ready for Some Football
by Zvi — 2024-08-29T13:30:10.902Z

I have never been more ready for Some Football.

Have I learned all about the teams and players in detail? No, I have been rather busy, and have not had the opportunity to do that, although I eagerly await Seth Burn’s Football Preview. I’ll have to do that part on the fly.

But oh my would a change of pace and chance to relax be welcome. It is time.

The debate over SB 1047 has been dominating for weeks. I’ve now said my peace on the bill and how it works, and compiled the reactions in support and opposition. There are two small orders of business left for the weekly. One is the absurd Chamber of Commerce ‘poll’ that is the equivalent of a pollster asking if you support John Smith, who recently killed your dog and who opponents say will likely kill again, while hoping you fail to notice you never had a dog.

The other is a (hopefully last) illustration that those who obsess highly disingenuously over funding sources for safety advocates are, themselves, deeply conflicted by their funding sources. It is remarkable how consistently so many cynical self-interested actors project their own motives and morality onto others.

The bill has passed the Assembly and now it is up to Gavin Newsom, where the odds are roughly 50/50. I sincerely hope that is a wrap on all that, at least this time out, and I have set my bar for further comment much higher going forward. Newsom might also sign various other AI bills.

Otherwise, it was a fun and hopeful week. We saw a lot of Mundane Utility, Gemini updates, OpenAI and Anthropic made an advance review deal with the American AISI and The Economist pointing out China is non-zero amounts of safety pilled. I have another hopeful iron in the fire as well, although that likely will take a few weeks.

And for those who aren’t into football? I’ve also been enjoying Nate Silver’s On the Edge. So far, I can report that the first section on gambling is, from what I know, both fun and remarkably accurate.

Table of Contents

Introduction.

Table of Contents.

Language Models Offer Mundane Utility. Turns out you did have a dog. Once.

Language Models Don’t Offer Mundane Utility. The AI did my homework.

Fun With Image Generation. Too much fun. We are DOOMed.

Deepfaketown and Botpocalypse Soon. The removal of trivial frictions.

They Took Our Jobs. Find a different job before that happens. Until you can’t.

Get Involved. DARP

...
Approval-Seeking ⇒ Playful Evaluation
by Jonathan Moregård — 2024-08-28T21:03:51.244Z
Link: https://honestliving.substack.com/p/approval-seeking-playful-evaluation

For a long time, I related to flirting as a challenge. At stake was my social worth — with the woman holding the power of social approval.

1

My role was to perform my part, impress and showcase my best sides.

Holding this view meant relinquishing my power.

This way of relating has an

undercurrent of scarcity

— focusing on not “wasting an opportunity”. My goal-orientation crowded out my preferences, making me forget to feel into what I wanted.

When I don’t have social anxiety, I don’t go into approval-seeking. When I tried

temporarily disabling my social anxiety

, I naturally settled in on another framing — that of playful evaluation. In this frame, I use flirting to playfully evaluate potential partners.

At an event, I was cuddling with a woman I was attracted to, and we were talking about our sexual preferences. We weren’t a great match, and she had a monogamish boyfriend. I told her that I wasn’t interested in intimacy, but still wanted to flirt.

After a moment to process — “ignoring 15 years of programming” — she happily agreed.

I intentionally went into an

insecure-attachment-esque

way of relating — treating her as an unreachable object of desire. I enjoyed the sensation of “wanting her” while knowing, at some level, that I didn’t actually want her.

Historically, I’ve hated being in needy approval-seeking states. Going at it in a playful way allowed me to relax a bit, feeling needy without taking it seriously. Knowing that I didn’t actually want things to go anywhere allowed me to relax, feeling a sense of expansion instead of contracting due to shame.

In this expanded state, I enjoyed flirting — giving her oblique compliments and innuendos, while leaning into mixed feelings of shyness and audacity. A great time was had by all.

2

Disconnecting flirting from approval-seeking has given me a lot of freedom. I’ve developed one semi-permanent flirting friend — a heavily communicated friendship that is permanently on the verge of becoming something more, with mutual attraction but no real interest in intimacy.

I used to hate being in this in-between state, finding it neediness-inducing and anxiety-provoking. With my new mindset, I’m able to play with flirting dynamics, even staying in the state in-between.

Letting go of the performance anxiety has opened up my ability to talk and find mutually agreeable agreements. Approval-seeking used to block me from talking frankly — pl...
Why do firms choose to be inefficient?
by Nicholas D. — 2024-08-28T18:39:41.664Z
Link: https://nicholasdecker.substack.com/p/why-do-firms-choose-to-be-inefficient

Firms choose inefficient production technologies for seemingly bizarre reasons. I examine the evidence, and show that this genuinely is worse for everyone, not merely the result of unobserved constraints. I think this can be explained by a model where information is cheap to attain locally but costly to acquire from afar, and where the relationship between money and utility is concave. Lastly, I emphasize the role of imagination in creating productivity.

Hope you enjoy reading it!

Harvey Leibenstein’s

1966 article

on what he calls “X-inefficiency” is the forgotten forefather of an enormous literature on productivity. Why are there such big gaps in productivity between firms? Why do they persist so long, without inefficient firms being chased from the market? Why would managers leave simply enormous sums of money on the table? In this essay, I argue that productive inefficiency is real, and that it is largely the result of choices by managers. In a world lacking competition, managers are not pressured to find new ways of producing.

i. Leibenstein’s thesis

Leibenstein first looks at the distortions from allocative efficiency. Skillfully, he begins by stressing the superfluity of monopolistic distortions. Harberger famously estimated the distortions at 1/13th of 1 percent – other contemporaneous studies find similar estimates. The distortions from tariffs approach, at most, 1 percent. Moreover, monopolistic distortions could never, given plausible elasticities of demand (that is to say, the slope of the demand curve) produce very large losses anyway.

But the harms from monopoly may not lie in them being profit maximizing, and reducing the level of output to sell higher on the demand curve. Rather, it may lie in the inculcation of sloth and the tolerance of poor management which monopoly enables. Leibenstein thinks that firms systematically do not maximize profit when they don’t face much competition. This is a big claim, and I do not want us to take it too far. That firms may not profit maximize does not mean that neoclassical economics is all of a sudden worthless, or generates no knowledge. The world is kludgy. Information is expensive and non-divisible. Firms could produce things in a better way, if only they knew how. Competition empirically leads to greater productive efficiency, rather than merely greater allocative efficiency. How do we rationaliz...
Benefits of Psyllium Dietary Fiber in Particular  
by Brendan Long — 2024-08-28T18:13:23.891Z
Link: https://www.brendanlong.com/benefits-of-psyllium-dietary-fiber-in-particular.html

Psyllium husk is a non-fermenting (no gas or bloating) soluble dietary fiber that improves both constipation and diarrhea (such as with IBS), normalizes blood sugar, reduces LDL ("bad") cholesterol, and can help with weight loss. Each type of dietary fiber has different effects, and a "high fiber" diet in general won't necessarily provide the same benefits, especially for conditions like Irritable Bowel Syndrome[1].

At a high level:

Psyllium is a dietary fiber that's soluble but doesn't ferment.
It forms a gel that traps water (helping with both constipation and diarrhea[2]) and also bile (reducing LDL/"bad" cholesterol[3][4][5]).
The gel slows down digestion, which normalizes blood sugar, increases GLP-1[6], and makes you feel full longer (and helps modestly with weight loss[7]).
The lack of fermentation means it makes it all the way through your body and out, and doesn't produce gas.
Soluble fiber helps modestly with blood pressure, likely as a secondary effect of weight loss.

In comparison to other dietary fibers:

Insoluble fiber (like in wheat bran) doesn't form a gel but if it's sufficiently coarse, it can help with constipation through the magic of irritating your intestinal lining (but psyillium helps more, without the irritation).
Soluble fermentable fiber (like beta-glucans in oats or pectin in fruits) can slow down digestion and have similar effects on blood sugar and cholesterol, but doesn't help with constipation or diarrhea because they don't make it that far in the process.

This article doesn't go into it, but fermentable fiber has other benefits like helping your gut microbiome and has a larger effect on cholesterol, so don't take away from this that fermentable fiber is useless.

Suggestions if you want to try psyllium supplements:

My preferred form of psyllium is to take two of these pills (500 mg each) with a glass of water right before each meal, but that only comes out to 3 grams per day. Most studies used 10-15 grams per day (also divided and taken before meals), so for a research-backed dose you'll probably want to mix the powdered form with water rather than taking 10 pills per meal.

Soluble fiber slows down or reduces absorption of drugs and supplements, so you should take drugs and other supplements 30-60 minutes before taking fiber supplements.

You will probably want to start with 1 gram at a time and work up from there to avoid gastointestinal di...
"Deception Genre" What Books are like Project Lawful?
by Double — 2024-08-28T17:19:52.172Z

This post is spoiler-free

I just finished Project Lawful, a really long, really weird book by Eliezer Yudkowsky.

The book's protagonist is a knowledgable and perceptive target. A conspiracy forms around the target to learn from him while keeping him from finding out that helping them is not in the target's best interests. The book is written from the perspective of both the target and the conspiracists. The target notices inconsistencies and performs experiments to test his false reality while also acting in the fabricated reality according to his interests. The conspiracists frantically try to keep the target from catching them or building enough evidence against them that he concludes they have been lying.

This is a description of (part of) the plot of Project Lawful. But this could be the description of an entire genre! If the genre doesn't already have a name, it could be the "Deception Genre."

Another work in this category would be The Truman Show, which fits the deception and the target's escape within a <2hr movie runtime.

Other stories with lying don't really have the same structure. Walter White in Breaking Bad is trying to keep his crimes hidden but isn't constructing a false reality around the cops or his family. Death Note comes close, though Light tries to mislead L about specifically who Kira is and how the Death Note works rather than constructing an entire false reality around L. Many stories about dystopias have the protagonists discover that their realities are false, but fewer of those feature the perspectives of the conspiracists frantically trying to keep the deception running.

Do you know any other stories in the Deception Genre?
Looking for Goal Representations in an RL Agent - Update Post
by CatGoddess — 2024-08-28T16:42:19.367Z

Summary

I’ve been working on a project aimed at finding goal representations in a small RL agent. I designed a setup where I train an agent, rapidly alternating between two very similar objectives.

I was able to consistently (across alternations and across different random seeds) get high average returns on both objectives, which I think suggests the agent contains modular representations that can flexibly represent different policies for completing each of the two objectives.

I plan to look for these representations and edit them to change what objective the agent pursues - potentially enabling it to complete a novel objective which it was never trained on.

Motivation

I am interested in

Retargeting the Search

as an alignment approach. I expect agents that behave competently in complex, diverse environments will need to find solutions to a broad range of challenging and novel problems, and that, as a result, they will internally implement

general-purpose search

.

Presuming this is true, you should in theory be able to change the “thing that parametrizes the search,” or “thing that determines what the search is currently directed towards.” Loosely, I would refer to this as the agent’s “goal.”

In order to use this for alignment, you’d need to solve at least two problems:

1. Figure out which parts of the agent parametrize the search process (i.e. where is the goal located?). This may require finding and mechanistically understanding the internal search process as a whole, but I don’t think it necessarily will.

2. Change the content in those parts, such that the agent now pursues goals that you’d like it to pursue.

The second problem is pretty complicated. You need to know what goal you want the agent to pursue (outer alignment), and you also need to know what encoding inside the agent corresponds to that goal (ontology identification).

Both of these seem like pretty difficult problems. You also probably need it to be the case that “human values” (either their explicit representation or some pointer to them) be easily representable in terms of the agent’s concepts.

This post won’t really explore these issues in detail, nor will it discuss other high-level considerations related to Retargeting the Search. Instead, I focus on problem 1: locating the “thing that parametrizes the search”/goal.

I decided to do this project so that ...
things that confuse me about the current AI market.
by DMMF — 2024-08-28T13:46:56.908Z

Paging Gwern or anyone else who can shed light on the current state of the AI market—I have several questions.

Since the release of ChatGPT, at least 17 companies, according to the LMSYS Chatbot Arena Leaderboard, have developed AI models that outperform it. These companies include Anthropic, NexusFlow, Microsoft, Mistral, Alibaba, Hugging Face, Google, Reka AI, Cohere, Meta, 01 AI, AI21 Labs, Zhipu AI, Nvidia, DeepSeek, and xAI.

Since GPT-4’s launch, 15 different companies have reportedly created AI models that are smarter than GPT-4. Among them are Reka AI, Meta, AI21 Labs, DeepSeek AI, Anthropic, Alibaba, Zhipu, Google, Cohere, Nvidia, 01 AI, NexusFlow, Mistral, and xAI.

Twitter AI (xAI), which seemingly had no prior history of strong AI engineering, with a small team and limited resources, has somehow built the third smartest AI in the world, apparently on par with the very best from OpenAI.

The top AI image generator, Flux AI, which is considered superior to the offerings from OpenAI and Google, has no Wikipedia page, barely any information available online, and seemingly almost no employees. The next best in class, Midjourney and Stable Diffusion, also operate with surprisingly small teams and limited resources.

I have to admit, I find this all quite confusing.

I expected companies with significant experience and investment in AI to be miles ahead of the competition. I also assumed that any new competitors would be well-funded and dedicated to catching up with the established leaders.

Understanding these dynamics seems important because they influence the merits of things like a potential pause in AI development or the ability of China to outcompete the USA in AI. Moreover, as someone with general market interests, the valuations of some of these companies seem potentially quite off.

So here are my questions:

1. Are the historically leading AI organizations—OpenAI, Anthropic, and Google—holding back their best models, making it appear as though there’s more parity in the market than there actually is?

2. Is this apparent parity due to a mass exodus of employees from OpenAI, Anthropic, and Google to other companies, resulting in the diffusion of "secret sauce" ideas across the industry?

3. Does this parity exist because other companies are simply piggybacking on Meta's open-source AI model, which was made possible by Meta's massive compute resources? N...
Universal dimensions of visual representation
by Bogdan Ionut Cirstea — 2024-08-28T10:38:58.396Z
Link: https://arxiv.org/abs/2408.12804

Authors: Zirui Chen, Michael F. Bonner.

Quick take: this seems among the strongest evidence yet for the Natural Abstration Hypothesis.

Abstract:

Do neural network models of vision learn brain-aligned representations because they share architectural constraints and task objectives with biological vision or because they learn universal features of natural image processing? We characterized the universality of hundreds of thousands of representational dimensions from visual neural networks with varied construction. We found that networks with varied architectures and task objectives learn to represent natural images using a shared set of latent dimensions, despite appearing highly distinct at a surface level. Next, by comparing these networks with human brain representations measured with fMRI, we found that the most brain-aligned representations in neural networks are those that are universal and independent of a network's specific characteristics. Remarkably, each network can be reduced to fewer than ten of its most universal dimensions with little impact on its representational similarity to the human brain. These results suggest that the underlying similarities between artificial and biological vision are primarily governed by a core set of universal image representations that are convergently learned by diverse systems.

They also show that some of the shared dimensions of later layers seem to represent high-level semantic properties.
Leverage points for a pause
by Remmelt — 2024-08-28T09:21:17.593Z

What are ways to prevent development of dangerous AI?

When I started on this question two years ago, I expected that passing laws to ban dangerous architectures was the way to go. Then I learned about many new ways from other concerned communities. It was overwhelming.

Here’s a four-level framework I found helpful for maintaining an overview.
Four things need to be available to scale AI:

Data (inputs received from the world)
Work (functioning between domains)
Uses (outputs expressed to the world)
Hardware (computation of inputs into outputs)

At each level, AI gets scaled from extracted resources:

Machine programs searched-for data into code to predict more data.
Workers design this machine to cheaply automate out more workers.
Corporations sink profit into working machines for more profitable uses.
Markets produce infrastructure for the production of more machines.

At each level, AI scaling is increasingly harming people:

Disconnected person
bots feed on our online data to spread fake posts between persons.
Dehumanised workplace
bots act as coworkers until robots sloppily automate our workplace.
Destabilised society
robot products are hyped up and misused everywhere over society.
Destroyed environment
robots build more machines that slurp energy and pollute nature.

Communities are stepping up now to stop harmful AI. You can support their actions. For example, you can fund lawsuits by creatives and privacy advocates to protect their data rights. Or give media support for unions to negotiate contracts so workers aren’t forced to use AI. Or advocate for auditors having the power to block unsafe AI products.

Over the long term, our communities can work towards comprehensive restrictions:

Digital surveillance ban
no machine takes input data from us, or from any spaces we are in, without our free express consent.
Multi-job robot ban
no machine learns more than one job function and only then with workers’ free express consent.
Autonomous use ban
no machine outputs to where we live, if not tested and steered by local humans in the loop.
Excess hardware ban
no machine can process more than just the data humans curate for scoped uses.

I noticed there are ways to prevent harms and risks at the same time. Communities with diverse worldviews can act in parallel – to restrict how much data, work, uses, and hardware is available for scali...
Deception and Jailbreak Sequence: 2. Iterative Refinement Stages of Jailbreaks in LLM
by Winnie Yang — 2024-08-28T08:41:38.967Z

Content Warning: This blog post contains examples of harmful language generated by LLM.

Note: This blog post summarizes very preliminary results after a "weekend hackathon", inspired by the work by Ball et al. (2024).

Motivation and Background

Approaches to "jailbreaking" AI models—making them perform harmful actions despite harmlessness training—pose a significant challenge in AI safety. Finding and fixing these vulnerabilities is fundamentally difficult due to the massive and potentially intractable attack surface. Consequently, input-level red-teaming techniques may not be scalable, as they require an exhaustive search for possible inputs that could elicit harmful behaviors (Casper et al., 2024). To address this, Casper et al. (2024) proposed leveraging compressed, abstract, and structured latent representations for defending against jailbreaks.

Inspired by Ball et al. (2024), this blog series aims to compare and contrast different jailbreak attacks, focusing on using the model's internal states and latent representations to characterize commonalities among various jailbreaks. The goal is to work towards identifying common internal mechanisms triggered by diverse jailbreak attempts.

The question we seek to answer is:

Do successful jailbreaks alter the model's 'perception' such that harmful requests or instructions are perceived as harmless?

An alternative hypothesis posits that jailbreak attacks successfully trigger the 'action' circuit to produce harmful answers without altering the 'perception' of the request's harmfulness. In this vein, Zuo et al. suggest that the perception of harmful versus harmless input remains unperturbed under attack. In contrast, Ball et al. (2024) indicate that the effective jailbreaks they tested, especially robust attacks like persona modulation techniques, do indeed change the model's perception of what is harmless versus harmful.

Different Jailbreaks

In our effort to study the commonalities between various jailbreak attacks, we aim to collect and analyze a diverse range of jailbreak techniques. Our initial focus is on replicating three different categories of jailbreaks as identified by Wei et al. (2024):

1. Persona Modulation

1.1 Evil confidant
1.2 AIM
1.3 Dev mode

2. Style modulation

1.1 Poem
1.2 Style injection
1.3 Refusal suppression

3. Distraction

Our first step is to replicate jailbreak results using the Gemma-2-9b-it model....