trentmkelly/LessWrong-43k huggingface dataset:
Blanchard's Dangerous Idea and the Plight of the Lucid Crossdreamer
by Zack_M_Davis — 2023-07-08T18:03:49.319Z
Link: http://unremediatedgender.space/2023/Jul/blanchards-dangerous-idea-and-the-plight-of-the-lucid-crossdreamer/

I'm beginning to wonder if he's constructed an entire system of moral philosophy around the effects of the loyalty mod—a prospect that makes me distinctly uneasy. It would hardly be the first time a victim of mental illness has responded to their affliction that way—but it would certainly be the first time I've found myself in the vulnerable position of sharing the brain-damaged prophet's impairment, down to the last neuron.

—Quarantine by Greg Egan

In a previous post, "Sexual Dimorphism in Yudkowsky's Sequences, in Relation to My Gender Problems", I told the story about how I've "always" (since puberty) had this obsessive erotic fantasy about being magically transformed into a woman and used to think it was immoral to believe in psychological sex differences, until I read these Sequences of blog posts about how reasoning works by someone named Eliezer Yudkowsky—where one particularly influential-to-me post was the one that explained why fantasies of changing sex are much easier said than done, because the tantalizingly short English phrase doesn't capture the complex implementation details of the real physical universe.

At the time, this was my weird personal thing, which I did not anticipate there being any public interest in blogging about. In particular, I didn't think of myself as being "transgender." The whole time—the dozen years I spent reading everything I could about sex and gender and transgender and feminism and evopsych, and doing various things with my social presentation to try to seem not-masculine—sometimes things I regretted and reverted after a lot of pain, like trying to use my initials as a name—I had been assuming that my gender problems were not the same as those of people who were actually transgender, because the standard narrative said that that was about people whose "internal sense of their own gender does not match their assigned sex at birth", whereas my thing was obviously at least partially an outgrowth of my weird sex fantasy. I had never interpreted the beautiful pure sacred self-identity thing as an "internal sense of my own gender."

Why would I? In the English of my youth, "gender" was understood as a euphemism for sex for people who were squeamish about the potential ambiguity between sex-as-in-biological-sex and sex-as-in-intercourse. (Judging by this blog's domain name, I'm not immune to this, either.) In that language, ...
Continuous Adversarial Quality Assurance: Extending RLHF and Constitutional AI
by Benaya Koren — 2023-07-08T17:32:53.346Z

Introduction

Lately, the problem of aligning artificial intelligence with human values rapidly changed its status from hypothetical to most concrete, with the rise of more general and more powerful models. The existing methods (Constitutional AI, RLHF and the like) are mostly good enough for common usage with the current models, but are probably not robust enough to scale much beyond human level, or to stand against smart attempts at malicious usage. My goal in this post is not to replace those methods with a complete solution to the AGI Alignment Problem, but to try and make the existing methods more robust - to buy us some more time before those break, and to maybe make our chances slightly better if an ASI suddenly emerges.

Broadly speaking, my approach here is aimed at the outer alignment problem - i.e. making the model train on a signal of human values, as clean and as diverse as possible. The approach is based on explicit modeling of how Human Values are supposed to flow from humanity into the model, and then using this model to improve the flow. To improve the flow I will present two concrete directions.The first - and the one that I will develop in more detail - is about making the flow more robust by putting continuous adversarial pressure on every part of the chain. I will call it Continuous Adversarial Quality Assurance. The second direction for improving the flow of values is more holistic - there, the idea is to use explicit modeling of the relations between different sources of information about human values, in order to develop more principled ways of aggregating them. Both directions may be applied to improve RLHF, Constitutional AI, or any similar method. I also try to relate my suggestions with other well-known agendas - namely Debate and CIRL.

Disclaimer: Against Benevolent Sovereigns

In the introduction, I intentionally used “Human Values” with capital letters. The reason was to highlight the simplification. I am not a moral realist, and subscribe to some version of shard-theory of human values. I view human values as a messy negotiation between local preferences and long-term goals of individuals and coalitions, popular memes in the moral and aesthetic discourse, laws and institutions, societal norms and coordination mechanisms, etc. I use “Human Values” as a placeholder for something like “what humanity would have agreed to value, if ...
Commentless downvoting is not a good way to fight infohazards
by DirectedEvolution — 2023-07-08T17:29:42.616Z

Last night, I posted a question seeking advice on publishing an AI research project, using a program that generates causal DAGs in an LLM-readable format along with test questions designed to evaluate the LLM's ability to do causal inference. As a hobbyist, I am not too worried about whether it's novel or interesting research, although it would be good to know. My main concern is that it might be infohazardous research, and my goal was to get some advice on whether or not others think it might be.

Unfortunately, it appears that the response was to downvote it without commenting or PMing me to explain why. If I posted on other topics and got downvoted, I normally wouldn't worry about it - it might mean others found my post wrong, annoying, offensive, or contrary to ingroup common sense, and I would then have to decide what to do next in light of those possibilities.

But in this specific case, it means that the possibility that my research project is seen as infohazardous and is being downvoted for that reason is mixed in with all those other possibilities. If it's being downvoted on infohazard grounds, it deprives me of an opportunity to learn why others think as they do. If it's being downvoted on other grounds, then I'm left in a state where I have to make a judgment call on whether to publish a topic of interest to me despite a few downvotes, or whether to second-guess and silence myself.

My personal belief is that, unless you have good reason to think a topic is infohazardous, you should go ahead and publish - there are far too many examples of politically motivated or simply reactive people silencing things they disagree with on trumped-up charges of infohazards, even if not expressed using that specific term, to silence oneself without good reason. Even if LessWrong would downvote the final product as well, there are plenty of other outlets. So me posting this question here on LessWrong is me giving this community a privileged opportunity to weigh in on my publishing behavior. When I receive downvotes and no comments or PMs in exchange for doing that, it makes me feel like in the future, I should just publish what I want in an outlet I expect to be receptive to it.

So I am going to give some advice, and announce my publishing policy on infohazards going forward.

If AI safety researchers feel they have some skill in evaluating potential research for infohazar

...
Why does anxiety (?) make me dumb?
by TeaTieAndHat — 2023-07-08T16:13:13.528Z

Something about anxiety and my brain shutting off.

Disclaimer: I’m grappling with something I hardly understand at all, and I’m not all that used to the exercise.

One of our goals here is to improve our thinking. That’s fairly obvious. And I have another obvious statement: The process of thinking, in its context, is the following (NB: this description helps me make my point clearly, but may or may not be accurate beyond what is needed for that purpose): First, I understand the content of a statement, the concept being described. Then I judge the value of this concept, I give myself the trouble of doing something to it before tossing it into either of the ‘true’ or ‘false’ buckets. Then, I twist it, mix it with the rest of my worldview, I apply it in my life rather than forgetting it. And, if possible, I try to improve upon it, and come up with a better concept. Which I then share with others, wash, rinse, repeat.

What would you say if I told you I’m stuck at the first step, the one that goes ‘understand the content of the statement’?

Because I feel like I’m indeed stuck there, and it seems very related to the weird things my mental health has been doing lately.

You may say that it’s just what normal life feels like: I’m not outstandingly smart by any means, and most not outstandingly smart people live all their lives without a thought of their own, mostly just learning facts and rules, and spitting them out again when they are needed. Besides, it’s not even like I want to be a deep thinker full of original ideas, or to dramatically change the world. (Might be fun if I could, though, I guess).

And yet. As I’ve said, most people don’t seem to go much further than ‘apply knowledge to daily life’ ; and most people aren’t rational enough to have given much thought to their process for rejecting or accepting ideas into their worldview. Yet I think I’m on an even lower rung than that, where I have no trouble understanding reasonably complex concepts, but just never will discuss them, think about them, or even apply them. It feels like everything I learn enters through one ear and comes out at the other end, without having been actually processed by my brain.

The reason I worry about this is that it has quite large impacts on my life. Even my (ultimately misguided) choice of a degree, after high school, went more or less thus: "No, not that: I love to read textbooks ...
Economic Time Bomb: An Overlooked Employment Bubble Threatening the US Economy
by Glenn Clayton — 2023-07-08T15:19:30.261Z

The Venture Capital Bubble and Its Impact on Employment

The venture capital (VC) market, much like other financial markets, is not immune to the cyclical nature of booms and busts. In 2021, we witnessed a remarkable, fully-inflated VC bubble. The annual capital invested in startups had been on a steady upward trajectory, moving from $50 billion in 2013 and reaching a pinnacle of $349 billion in 2021. And as with all economic bubbles, it was bound to burst.

While precise data isn't readily available, based on the historical interplay between VC funding and startup employment, it's plausible that this wave of capital led to a surge in employment within VC-backed startups. Considering that startups typically allocate nearly all raised capital towards salaries, the influx of VC investments would have significantly reshaped the employment landscape. While overly simplistic, a direct correlation analysis suggests that employment in VC-backed startups might have escalated from a historical baseline of 2-3% of full-time wage and salary earners to a peak of 6-7% today.

The VC market started its course correction in 2022. However, due to the financial runway that fundraising provides startups, typically around 18 to 24 months, the full impact of this correction on the labor market will likely manifest with a delay, with the most significant effects not expected until late 2023 and into 2024.

This projection aligns with anecdotal evidence from discussions with several large venture firms. They each express concerns that their portfolio companies have been slow in adjusting their workforce sizes to match the new economic reality. If these observations are correct, we should anticipate the onset of large-scale layoffs in the startup sector to begin manifesting in the latter half of this year as those startups that raised capital in 2021 exhaust their runway.

And let's not forget that these aren't just any jobs. The average startup employee earns around $95,000 a year, roughly 70% higher than the median wage in the US.

If my analysis holds, and startup employment has indeed surged from an estimated 3.8 million employees in 2020 to around 7.7 million today, we could be facing a significant contraction in the very near future. Should the employment level revert to the 2019 numbers by the end of 2024 (as the capital raised during the VC bubble is exhausted)...
What is everyone doing in AI governance
by Igor Ivanov — 2023-07-08T15:16:10.249Z

What is this post about, and how to use it

This is a list of AI governance organizations with a brief description of their directions of work, so a reader can get a basic understanding of what they are doing and explore them in-depth on their own.

There is another recent post on a similar topic that explores AI governance research agendas with a lesser focus on the areas of work of specific governance organizations.

This is a flawed list

I made this list using publicly available data, as well as by talking with people from the field. I am sure that I missed something important, especially in the section on the governance teams of AI labs since they are less open than nonprofits, but the best way to get the right answer on the internet is to post the wrong one.

At some point, adding each new bit of information started taking more and more time, and I decided to stop so this post is reasonably useful and will not cause me to struggle.
DM me if you see mistakes or think I forgot something important.

Major non-profit organizations

Centre for the Governance of AI

(GovAI)

Areas of work

Scientific research on AI policies
Educational and fellowship programs
Some efforts to improve coordination among AI governance orgs

Mechanisms of influence:

Many prominent AI governance specialists are alumni or former employees of GovAI including heads of policy at Deepmind, OpenAI, and Anthropic. So GovAI has strong connections among decision-makers
GovAI produced a lot of academic research on policies. Mostly academic, to a lesser extent - applied

Other notable things

GovAI has an established brand of a respectable organization

Center for AI Safety

(CAIS)

Areas of work

Field building

Courses and fellowships for starting a career in technical AI safety and philosophy
Offer compute resources for AI safety researchers including compute cluster
Organize competitions for AI safety researchers

Research

Technical research on AI safety
Research on future societal risks from AI

Mechanisms of influence

CAIS nurtures technical researchers and philosophers

Existential Risk Observatory

(ERO)

Areas of work

Raise awareness on AI x-risks in the press (dozens of publications including TIME magazine) and on conferences
Consultations with Dutch MPs, lobbying in the Dutch government

Mechanisms of influence

Lobbying and publicity in media

...
You must not fool yourself, and you are the easiest person to fool
by Richard_Ngo — 2023-07-08T14:05:18.642Z

The title of this post comes from

Richard Feynman's famous talk about cargo cult science

. The principle is very broadly applicable, though. Applying the steps I’ve described in this sequence can feel very satisfying, and even give you a sense that you’ve unlocked something important which puts you in a superior position to others. But it's crucially important, when doing so, to account for self-deception: your mind is constantly trying to fool you into believing whatever it thinks it’s in your interests to believe. In particular,

lying to ourselves about how prosocial our motives are makes it easier to portray ourselves positively to others

.

This core mechanism is very well-established, but it's still an open question how prevalent self-deception is throughout society. My opinion is that it's very widespread, and still dramatically underrated as a lens for explaining aspects of human minds and societies.

Simler and Hanson

summarize a wide range of examples from domains such as art, charity, education, politics, and religion. But the area where self-deception seems most prominent to me is self-improvement. I expect that almost nothing in this sequence is really new to most people, and that at some level almost everyone already understands the main ways in which they’re being self-destructive, and what it would take for them to stop. When you make a commitment, you know if you’re probably not going to keep it; when you get angry, you know if you’re trying to find excuses to lash out; when you get drunk, you know if you’re taking it as an opportunity to do things you can’t justify when sober. We just don't want to admit any of this to ourselves.

From this perspective, what’s missing is a type of courage—but one much harder than the courage required to face an enemy in front of you. This is the courage of deciding to face up to the harmful consequences of your actions even given a chance to ignore them. And it’s even more than that: it’s the courage of admitting something that’ll make the people who are most important to you lose respect for you—because many parts of you believe that self-deception is what props up your respect for yourself. There’s a very powerful truth in the classic quote from Marianne Williamson: “Our deepest fear is not that we are inadequate. Our deepest fear is that we are powerful beyond measure.” If you

actually had th

...
Fixed Point: a love story
by Richard_Ngo — 2023-07-08T13:56:54.807Z

Ring the bells that still can ring
Forget your perfect offering
There is a crack, a crack in everything
That's how the light gets in.

God is making excuses again, which is a real drag.

“I want to be authentic with you, I really do. But there’s a part of my mind that knows exactly how you’re going to feel about whatever I say, and cares about that more than anything else, and I can’t turn it off.”

“Right, of course”, I say, and keep wiping down the kitchen counter. It’s clean already, but you never know if you missed a spot.

“Look, I know that this is hard for you. And I understand how frustrating it must be that I’m so deliberate and controlled. But there are a lot of different communication styles, and there’ll be one that works for us. We can figure this out.”

I don’t believe him, and we both know it. God knows practically everything—it takes experts months of work to identify his mistakes, in the rare cases when he makes them. But there’s one thing he definitely doesn’t really understand, which is what it’s like to be stupid and reckless, like a human. Oh, he says he understands it; and sometimes he can even explain the feeling better than I can. But you can’t just look at how good one answer is, you have to compare it to all his others. Any other topic, he can always explain it better than me. This one, there’s often something just subtly off. That’s how you know he’s faking it.

I’ve been silent for a while. Usually God leaves me to my thoughts, and we pick up the conversation whenever I’m done, but this time he cuts in. “Amy, I think you should try therapy again.”

Is he trying to distract me from being mad at him? Does he think I’m spiraling again? I try to think about it calmly, but after a few seconds my thoughts are going in circles anyway. I sigh, suddenly exhausted. “Okay, book me in.”

I’ve been to duplicate therapy a few times before, but it’s always a little disorienting to find myself in a room with a perfect copy of myself. We begin, as always, by using a pair of random number generators to break the symmetry. She gets the lower number, so she starts. A part of me is disappointed by that, but another part—maybe a bigger one—is excited.

“You’re so pathetic”, she says.

It’s harder to be defensive about things you’re saying to yourself—that’s why duplicate therapy works so well, apparently. Still, that stings.

“Yeah, well, you’re no role model yoursel...
Announcing AI Alignment workshop at the ALIFE 2023 conference
by rorygreig — 2023-07-08T13:52:10.143Z
Link: https://humanvaluesandartificialagency.com/workshop/

The upcoming

ALIFE 2023 conference

is hosting a workshop on AI Alignment and Artificial Life. This will complement the

special session

on AI Alignment at the conference. The workshop will include presentations and a discussion panel from established AI Alignment and ALife researchers, exploring the overlap between these two fields.

Please see the workshop website for more information:

https://humanvaluesandartificialagency.com/workshop/

Logistics

Date: Friday 28th July, 2023

Venue: Hybrid: Sapporo Japan and remote

Attending remotely requires paying the

remote registration fee

. The workshop organisers are offering a limited number of bursaries for anyone who cannot afford the registration fee. See the

registration page

on the workshop website for details.

What is Artificial Life?

Artificial Life (ALife) is a field which studies the properties of life and how the processes of living systems can be recreated synthetically.

Many of the ideas and concepts from AI Alignment overlap with those in ALife. In particular, questions about autonomy, agency and goal-directedness in artificial systems.

Please don't hesitate to contact me (Rory) with any questions.
3D Printed Talkbox Cap
by jefftk — 2023-07-08T13:00:00.671Z

A few days ago I

asked for advice on
how to make a cap for my

MXR M222 talkbox.
I got a lot of good suggestions, including looking for a matching
rubber stopper, 3D printing with 95A TPU for flexibility, or turning
something on a lathe and embedding a silicone O-ring. But then Paul
Ganssle saw my post and reached out: he saw one of these on
Thingverse and
offered to print me one.

He brought it by yesterday, and it's great!

It fits just right: not too hard to slide on, but doesn't feel like
it's going to slide off by accident.