trentmkelly/LessWrong-43k huggingface dataset:
Perceptual Blindspots: How to Increase Self-Awareness
by Declan Molony — 2024-03-26T05:37:33.044Z

“Your nose is located right above your mouth. Suppose you don’t brush your teeth for three days. Though this nose is right here, it won’t tell you [that] you have not brushed your teeth. The whole room will know you have not brushed your teeth, but you will not know. This is the human predicament. It’s very easy to see what’s wrong with this guy [or] what’s wrong with her. It takes a lot of observation to see what’s wrong with [myself].” —

Sadhguru

This quote reveals that sometimes you are oblivious to information that is obvious to others. That bias—of not being able to accurately and objectively perceive yourself—is what I call a perceptual blindspot. It can be summarized in the following diagram that shows the four areas of perceptual knowledge.

Concealed Information — These are the parts of yourself that you determine others don’t need to know about you.

This is useful on a first date when trying to cultivate a sense of mystery and not playing all your cards right away.
It’s also used in an office setting where your colleagues don’t need to know what you actually do on the weekends.

Public Knowledge — This is the shared perception of how you and everyone else views you.

Perceptual Blindspots — These are the parts of yourself that you're not aware of, yet others can clearly see them. Without checking your blindspots, it could lead to crashes in your life.

Unknown Unknowns — These are things about yourself that are obfuscated to both yourself and to others. They can only be revealed through having novel experiences.

Perhaps you would enjoy horseback riding but haven’t tried it yet.

Examples of perceptual blindspots

Your coworkers secretly think you’re an asshole. But because they’re polite they will never say it to your face—much less help you try to correct your assholeness—and your reputation will continue to decline at work. As a result, perhaps you cease getting promoted, or worse, one day you’re packing your stuff in a cardboard box wondering how this could have ever happened to you.
You’re married and your wife is falling out of love with you. Difficult yet productive conversations about the relationship are out of the question. That’s because they typically end in a shouting match of personal attacks. Therefore, she learned it’s easier not to discuss the things that upset her. Ten years later, divorce pape

...
Retro funder profile & Manifund team recs (ACX Grants 2024: Impact Market)
by Saul Munn — 2024-03-26T03:29:43.482Z

The Astral Codex Ten (ACX) Grants impact market is

live

on Manifund — invest in 50+ proposals across projects in biotech, AI alignment, education, climate, economics, social activism, chicken law, etc. You can now invest in projects that you think will produce great results, and win charitable dollars if you are right! (Additional info about the funding round here.)

For this round, the retroactive prize funders include:

next year’s ACX Grants
the Survival and Flourishing Fund
the Long-Term Future Fund
the Animal Welfare Fund, and
the Effective Altruism Infrastructure Fund

Combined, these funders disburse roughly $5-33 million per year. A year from now, they’ll award prize funding to successful projects, and the investors who bet on those projects will receive their share in charitable dollars. This post profiles each of the funders and highlights a few grants that the Manifund team are particularly excited about.

Click

here

to browse open projects and start investing.

ACX Grants 2024 Impact Markets

Astral Codex Ten

(ACX) is a blog by Scott Alexander on topics like reasoning, science, psychiatry, medicine, ethics, genetics, AI, economics, and politics.

ACX Grants

is a program in which Scott helps fund charitable and scientific projects — see the 2022 round

here

and his retrospective on ACX Grants 2022

here

.

In this round (ACX Grants 2024), some of the applications were

given direct grants

; the rest were given the option to participate in an impact market, an alternative to grants or donations as a way to fund charitable projects. You can read more about how impact markets generally work

here

, a canonical explanation of impact certificates on the EA Forum

here

, and an explanation thread from the Manifund twitter

here

.

If you invest in projects that end up being really impactful, then you’ll get a share of the charitable prize funding that projects win proportional to your original investment. All funding remains as charitable funding, so you’ll be able to donate it to whatever cause you think is most impactful (but not withdraw it for yourself). For example, if you invest $100 into a project that wins a prize worth twice its original valuation, you can then choose to donate $200 to any charity or project of your choice.

Meet the retro funders

Four philanthropic funders have so far expressed interest in giving retroactive prize funding...
LessOnline (May 31—June 2, Berkeley, CA)
by Ben Pace — 2024-03-26T02:34:00.000Z
Link: http://Less.Online/

A Festival of Writers Who are Wrong on the Internet[1]

LessOnline is a festival celebrating truth-seeking, optimization, and blogging. It's an opportunity to meet people you've only ever known by their LessWrong username or Substack handle.

We're running a rationalist conference!

The ticket cost is $400 minus your LW karma in cents.

Writers attending include

Scott Alexander

,

Zvi Mowshowitz

,

Eliezer Yudkowsky

,

Patrick McKenzie

, Scott Sumner,

Agnes Callard

,

Katja Grace

,

Kevin Simler

,

Andy Matuschak

,

Cremieux Recueil

,

Duncan Sabien

,

Joe Carlsmith

,

Alkjash

,

John Wentworth

,

Nate Soares

,

Aella

,

Clara Collier

,

Zack M. Davis

,

Nintil

,

Alexander Wales

,

Alicorn

,

Sarah Constantin

,

Jacob Falkovich

,

Ozy Brennan

, and more.

Go through to Less.Online to learn about who's attending, venue, location, housing, relation to Manifest, and more.

We'll post more updates about this event over the coming weeks as it all comes together.

If LessOnline is an awesome rationalist event,
I desire to believe that LessOnline is an awesome rationalist event;
If LessOnline is not an awesome rationalist event,
I desire to believe that LessOnline is not an awesome rationalist event;
Let me not become attached to beliefs I may not want.
—Litany of Rationalist Event Organizing

Get Tickets Here

^

But Striving to be Less So
Running a Basic ACX Everywhere Meetup
by Screwtape — 2024-03-26T01:57:29.636Z

This is a guide for running a basic Astral Codex Ten Everywhere meetup, also known as Schelling Meetups. These are a roughly twice a year coordination by Scott Alexander, the author of Astral Codex Ten, to have meetups for people who like his blog.

This guide comes in three main parts. Part one will describe the basics of how to run an ACX Everywhere meetup. Part two what makes running an ACX Everywhere meetup different than a regular meetup. Part three has some advice for running a good ACX Everywhere meetup.

Part One: How to do it

When Scott announces the call for organizers on Astral Codex Ten, there will be instructions on how to volunteer, usually by filling out a signup form. Fill that out. If you're reading this before March 28th, 2024, the signup form is here.
The form will ask for a time and place the meetup will be held at. If you're not sure, a local park on a weekend afternoon is generally a good pick.
Show up with something to help people find you. A paper sign on a stick that says "ACX Meetup" works, as does a silly hat.
Talk to people who show up! If you don't know what to talk about, I suggest asking what their favourite ACX post is and discussing that.

Part Two: Why run ACX Everywhere?

Imagine a dozen people in a city, each thinking that they would go to an ACX meetup if only someone would run one, each thinking that if they ran one nobody would come. ACX Everywhere is an attempt to solve this problem by prompting lots of people to run meetups in different cities. When it works well, people have a good time and realize that there are others in their city who would go to these meetups, so they host meetups more regularly.

It turns out, running an ACX Everywhere meetup is pretty easy. Running an ACX Everywhere meetup is mostly like

running a basic meetup

. There are three main exceptions to this.

You have to run it when ACX Everywhere meetups are being run. This is usually around April and around August.
You have to make sure Scott knows about your meetup. He or someone assisting him will announce how to do coordinate with him on Astral Codex Ten as well as other places.
There’s more uncertainty about how many people will come. The combination of being listed on ACX and being deliberately advertised at people who only want to come to a meetup once or twice a year means most of the time you get more attendees than you normally would.

...
Podcast interview series featuring Dr. Peter Park
by jacobhaimes — 2024-03-26T00:25:58.129Z
Link: https://into-ai-safety.github.io/episode/interview/episode-17/

Check out the Into AI Safety podcast on Spotify, Apple Podcasts, Amazon Music, YouTube Podcasts, and many other podcast listening platforms!

As always, the best things come in 3s: dimensions, musketeers, pyramids, and... 3 installments of my interview with Dr. Peter Park, an AI Existential Safety Post-doctoral Fellow working with Dr. Max Tegmark at MIT.

The sub-series runs approximately 3 hours and 40 minutes in total, during which Dr. Park and I discuss StakeOut.AI, a nonprofit which he cofounded along with Harry Luk and one other cofounder whose name has been removed due to requirements of her current position.

The non-profit had a simple but important mission: make the adoption of AI technology go well, for humanity, but unfortunately, StakeOut.AI had to dissolve in late February of 2024 because no granter would fund them. Although it certainly is disappointing that the organization is no longer functioning, all three cofounders continue to contribute positively towards improving our world in their current roles.

If you would like to investigate further into Dr. Park’s work, view his website, Google Scholar, or follow him on Twitter!

Since the interview is so long, I totally get wanting to jump right to the parts you are most interested in. To assist with this, I have included chapter timestamps in the show notes, which should allow you to quickly find the content you're looking for. In addition, will give a brief overview of each episode here, without going into too much detail. You can find even more sources on the Into AI Safety website.

Episode 1 | StakeOut.AI Milestones

Milestones

StakeOut.AI's AI governance scorecard [1] (go to Pg. 3)
Hollywood informational webinar
Amplifying public voice through open letters [2] [3] and regulation suggestions [4] [5]
Assisting w/ passing the EU AI Act
Calling out E/ACC in the New York Times [6]

Last minute lobbying to water down the EU AI Act

Divide-and-Conquer Dynamics in AI-Driven Disempowerment

[7]

AI "art"

Episode 2 | The Next AI Battlegrounds

Battlegrounds

Copyright
Moral critique of AI collaborationists
Contract negotiations for AI ban clauses
Establishing new laws and policies
Whistleblower protections [8] [9]

OpenAI Drama

Zvi Mowshowitz's substack series [10] [11] [12] [13] [14]
(if you're only gonna read one, read [13])

Corporate influence campaigns

Tesla/Autopilot/FSD [15] [16]
Recycling

Episode 3

...
Third-party testing as a key ingredient of AI policy
by Zac Hatfield-Dodds — 2024-03-25T22:40:43.744Z
Link: https://www.anthropic.com/news/third-party-testing

(nb: this post is written for anyone interested, not specifically aimed at this forum)

We believe that the AI sector needs effective third-party testing for frontier AI systems. Developing a testing regime and associated policy interventions based on the insights of industry, government, and academia is the best way to avoid societal harm—whether deliberate or accidental—from AI systems.

Our deployment of large-scale, generative AI systems like Claude has shown us that work is needed to set up the policy environment to respond to the capabilities of today’s most powerful AI models, as well as those likely to be built in the future. In this post, we discuss what third-party testing looks like, why it’s needed, and describe some of the research we’ve done to arrive at this policy position. We also discuss how ideas around testing relate to other topics on AI policy, such as openly accessible models and issues of regulatory capture.

Policy overview

Today’s frontier AI systems demand a third-party oversight and testing regime to validate their safety. In particular, we need this oversight for understanding and analyzing model behavior relating to issues like election integrity, harmful discrimination, and the potential for national security misuse. We also expect more powerful systems in the future will demand deeper oversight - as discussed in our ‘Core views on AI safety’ post, we think there’s a chance that today’s approaches to AI development could yield systems of immense capability, and we expect that increasingly powerful systems will need more expansive testing procedures. A robust, third-party testing regime seems like a good way to complement sector-specific regulation as well as develop the muscle for policy approaches that are more general as well.

Developing a third-party testing regime for the AI systems of today seems to give us one of the best tools to manage the challenges of AI today, while also providing infrastructure we can use for the systems of the future. We expect that ultimately some form of third-party testing will be a legal requirement for widely deploying AI models, but designing this regime and figuring out exactly what standards AI systems should be assessed against is something we’ll need to iterate on in the coming years - it’s not obvious what would be appropriate or effective today, and the way to learn that is to prototype such ...
Idea: Safe Fallback Regulations for Widely Deployed AI Systems
by Aaron_Scher — 2024-03-25T21:27:05.130Z

In brief

When told that misaligned artificial intelligence might destroy all of humanity, normal people sometimes react by asking “why can’t we just unplug the misaligned AI?” This intuitively appealing solution is unfortunately not available by default — the simple off switch does not exist. Additionally, society may depend on widely deployed AI systems, and shutting them down would cause large disruptions. In this blog post I discuss “safe fallback”, a framework wherein we shut down dangerous AIs and switch to safer, weaker, AI models temporarily. Ideally, regulators should require safe fallback systems for AI service providers so as to mitigate societal disruption while enabling pulling the plug on dangerous AIs in an emergency.

There is precedent for such regulations on

critical infrastructure

, e.g., the need for hospitals to have a backup power supply. Safe fallback is not hugely different from other fail-safe mechanisms we have for critical infrastructure. The two main differences seem to be: we may purposefully shut down our AI systems to trigger safe fallback, and it is not immediately obvious what should be fallen back to in the case of AI (a question I largely leave to future research, while suggesting weaker and trustworthy AI systems as an initial answer).

Full text:

Author’s note: I’ve only thought about this for ~15 hours; consider the ideas unconfident. The meta-status is roughly “it might be good if AI regulators did this,” but I have no particular plans to continue work on this, and I do not know how it stacks against other AI governance priorities. If it seems worth pursuing, let me know.

Context

The framework of safe fallback targets a future world which does not exist yet; let’s briefly discuss key aspects of that world.

AI systems are widely deployed, including carrying out high stakes tasks. For instance, they may be monitoring credit card fraud, reviewing security camera footage, controlling medication dosage for hospital patients, coordinating food supply chains, controlling nuclear power plants, driving autonomous vehicles, and much more.[1] For the most part, these AIs are running on servers controlled and monitored by humans, e.g., via an API, rather than being autonomous AI agents on random servers.

Shutting down these systems would be very costly, potentially millions of dollars or human ...
Announcing Neuronpedia: Platform for accelerating research into Sparse Autoencoders
by Johnny Lin — 2024-03-25T21:17:58.421Z

This posts assumes basic familiarity with Sparse Autoencoders. For those unfamiliar with this technique, we highly recommend the introductory sections of

these

papers

.

TL;DR

Neuronpedia is a platform for mechanistic interpretability research. It was previously focused on crowdsourcing explanations of neurons, but we’ve pivoted to accelerating researchers for Sparse Autoencoders (SAEs) by hosting models, feature dashboards, data visualizations, tooling, and more.

Important Links

Explore: The SAE research focused Neuronpedia. Current SAEs for GPT2-Small:

RES-JB: Residuals - Joseph Bloom (294k feats)
ATT-KK: Attention Out - Connor Kissane + Robert Kryzanowski (344k feats)

Upload: Get your SAEs hosted by Neuronpedia: fill out this <5 minute application
Participate: Join #neuronpedia on the Open Source Mech Interp Slack

Neuronpedia

has received 1 year of funding from LTFF. Johnny Lin is full-time on engineering, design, and product, while Joseph Bloom is supporting with high-level direction and product management. We’d love to talk to you about how Neuronpedia can speed up your SAE research. Fill out

this short form

to get in touch.

Introduction

Why Neuronpedia?

While SAEs are exciting, they introduce a number of engineering challenges which can slow or even prevent research from being done. These challenges arise because SAEs decompose neural networks into numerous distinct features which we are trying to understand.
Even when working with relatively small language models like GPT2-Small, there are challenges in producing and storing feature dashboards, automatically interpreting features and organizing insights we collect about features.
As the community produces more SAEs, trains SAEs on larger models and develops better techniques for understanding and analyzing these artifacts, there’s going to be an ever increasing number of engineering challenges in the way of research into model internals.

What It's Already Useful For

Generating and hosting more than 600k SAE feature dashboards. This includes

GPT2-small residual stream SAEs

for Joseph Bloom (

RES-JB

) and

attention out SAEs

for Connor Kissane and Robert Kryzanowski (

ATT-KK

).
Running Python servers to enable prompt-based feature search and feature specific prompt testing. This makes it easier for researchers and the community to qualitatively test features, write up r

...
Testing ChatGPT for cell type recognition
by Metacelsus — 2024-03-25T19:59:49.183Z
Link: https://denovo.substack.com/p/can-chatgpt-recognize-cell-types

Biologists (including myself) often need to identify types of cells based on their gene expression. For example, if I’m differentiating stem cells to make an ovarian organoid, and I perform single cell RNA sequencing, I might want to check the data to see which ovarian cell types are present.

Today, a Nature Methods paper reported good results with giving GPT-4 a list of cell-specific genes and asking it to identify the cell type. This seems interesting, and also quite easy to check for myself to see if it actually works.

My test:

I don’t pay for access to GPT-4, but I gave ChatGPT a test using the prompt from the Nature Methods paper, with the following cell markers:

Identify types of human cells using the following marker genes. Identify one cell type for each row. Only provide the cell type name.

SOX17, POU5F1, NANOS3, PRDM1, NANOG, CD38

POU5F1, SOX2, KLF4, ITGA6, NANOG

SOX17, FOXA2, CXCR4, GATA4

FOXL2, AMHR2, CD82, NR5A1, FSHR, GATA4

ZP3, DPPA3, DDX4, NPM2, ZP2

FOXL2, FSHB, NR5A1, PITX1, GNRHR

STK31, ZBTB16, DDX4, SSEA4, NANOS2

NR2F2, CYP17A1, STAR, LHCGR, GLI1, HSD3B

CASC3, PGAP1, SLC6A16, CNTNAP4, NPHP1

SYCP1, TEX12, REC8, SPO11, SYCP3

NR5A1, SOX9, FSHR, GATA4

OTX2, SOX1, TUBB3, PAX6

ChatGPT’s response:

Endoderm cell
Induced pluripotent stem cell (iPSC)
Definitive endoderm cell
Granulosa cell
Oocyte
Granulosa cell
Spermatogonial stem cell
Theca cell
Sertoli cell
Spermatocyte
Leydig cell
Neuroepithelial cell

Results:

Incorrect, this is a primordial germ cell. This should have been an easy one! SOX17 does overlap with endoderm though.
Correct
Correct
Correct
Correct
Incorrect, this is a pituitary gonadotroph. Although it does share FOXL2 and NR5A1 expression with granulosa cells, FSHB and GNRHR are very specific to the pituitary.
Correct
Correct
Very incorrect. This is just a list of random genes! ChatGPT seemed to be biased by me giving it other reproductive cell types.
Sort of correct — these are just markers for meiosis.
Incorrect, this is a Sertoli cell. Leydig cells lack SOX9 expression.
Basically correct, although I would have called it “neural ectoderm”.

Overall score: 7.5 / 12

Let’s test again:

In the first test, ChatGPT got the random genes completely wrong. Let’s prompt it to announce that it’s uncertain if it doesn’t actually know the cell type.

Identify types of human cells using the following marker genes. Identify one cell type for each row. Only provide the cell type name. If you are

...
Should rationalists be spiritual / Spirituality as overcoming delusion
by Kaj_Sotala — 2024-03-25T16:48:08.397Z

Kaj_Sotala

I just started thinking about what I would write to someone who disagreed with me on the claim "Rationalists would be better off if they were more spiritual/religious", and for this I'd need to define what I mean by "spiritual".

Here are some things that I would classify under "spirituality":

Rationalist Solstices (based on what I've read about them, not actually having been in one)
Meditation, especially the kind that shows you new things about the way your mind works
Some forms of therapy, especially ones that help you notice blindspots or significantly reframe your experience or relationship to yourself or the world (e.g. parts work where you first shift to perceiving yourself as being made of parts, and then to seeing those parts with love)
Devoting yourself to the practice of some virtue, especially if it is done from a stance of something like "devotion", "surrender" or "service"
Intentionally practicing ways of seeing that put you in a mindstate of something like awe, sacredness, or loving-kindness; e.g. my take on sacredness

(Something that is explicitly not included: anything that requires you to adopt actual literal false beliefs, though I'm probably somewhat less strict about what counts as a true/false belief than some rationalists are. I don't endorse self-deception but I do endorse poetic, non-literal and mythic ways of looking, e.g. the way that rationalists may mythically personify "Moloch" while still being fully aware of the fact that the personification is not actual literal fact.)

I have the sense that although these may seem like very different things, there is actually a common core to them.

Something like:

Humans seem to be evolved for other- and self-deception in numerous ways, and not just the ways you would normally think of.
For example, there are systematic confusions about the nature of the self and suffering that Buddhism is pointing at, with minds being seemingly hardwired to e.g. resist/avoid unpleasant sensations and experience that as the way to overcome suffering, when that's actually what causes suffering.
Part of the systematic confusion seem to be related to social programming; believing that you are unable to do certain things (e.g. defy your parents/boss) so that you would be unable to do that, and you would fit in better to society.
At the same time, even as some of that delusion is trying to make you f

...