trentmkelly/LessWrong-43k huggingface dataset:
A brief theory of why we think things are good or bad
by David Johnston — 2024-10-20T20:31:26.309Z

Moral reasoning depends on believing that some things are good and others are bad. Some of these things seem very believable - I am quite firmly convinced that intense pain is bad. Though they seem very compelling, it’s hard to point to strong evidence that proves the judgement. On the other hand it's easy to point to strong motivations for believing such things - I don’t want to be tortured - and we know that motivation can cause people to believe things. The same mechanism that causes us to believe things we want to be true could also be responsible for our basic moral judgements of goodness and badness.

We believe many things because we are somewhat rational; we consider hypotheses, compare them with observed evidence, and emphasise those that are more compatible. Goodness and badness defy this practice; normal reasoning does not produce hypotheses of the form "if X is morally good then I will observe Y". One could object that such hypotheses are reasonable where Y is a belief or an attitude. If torture is bad, then if I try it I will observe that a) I believe it is bad and b) I don't like it. But this is either circular (I believe torture is bad because I believe it is bad) or it says that my attitude towards torture is the cause of my belief in its badness, which is roughly what I am arguing.

It's true that if we accept some fundamental assumptions about what makes things good or bad we can then go about considering hypotheses and evidence as usual; if happiness is good then an intervention X is good if I see data Y indicating that it increases happiness. The subject of this post is why we believe the fundamental assumptions ("happiness is good"), not the theories built on top of them.

We also know people often engage in motivated reasoning. If X being true is consistent with my desires, then I might find myself believing that X is true and coming up with post-hoc justifications for this belief.

Combining these points: there are at least two mechanisms by which come to beliefs. One is, crudely, when evidence causes the belief, and the other is when desire causes the belief. I have proposed that belief in fundamental assumptions of goodness and badness are not of the "caused by evidence" type. I suggest, therefore, that they are of the "caused by desire" type.

The badness of pain and suffering is a particularly plausible case. We universally...
Thinking in 2D
by sarahconstantin — 2024-10-20T19:30:05.842Z

Note: this is a “little” post processing some half-baked thoughts; I’ve got a more standard research-based post about cancer therapy queued up as well. Soon!

After wrapping up the Roots of Progress conference, which was spectacular, I’ve been thinking about people and organizations in a new and seemingly more “mature” way.

Within the range of the kinds of work I care about (roughly, advancing science and technology & their applications to human benefit), there’s a “radical/moderate” spectrum and a “small/large” spectrum.

Radical vs Moderate

The “radical/moderate” spectrum is about how sharply the project diverges from the status quo. Radical things are calling for bigger changes and also have more risk of failure; moderate things are calling for more incremental changes and are safer bets.

In Science

My sweet spot on that spectrum, the place where I feel most comfortable working, is in a medium zone when it comes to scientific risk. My ideal “scientific thesis” is mature enough to be supported by published research from “credible” institutions/journals1 but is new enough that I can easily meet people in the field who haven’t heard of it, or controversial enough that I can easily meet experts who don’t buy it. It’s at the point where early-stage startups and FROs are viable, but usually way too soon for Big Pharma or the NIH. In my “sweet spot”, there’s usually one or two unproven ideas that entail substantial technical/scientific risk…not lots and lots.

E.g. with transcranial focused ultrasound there’s a lot of literature that shows that ultrasound does modulate the brain’s activity. There’s scientific risk in “what neuromodulation protocol will be effective against a disease or for enhancing cognitive/emotional function?” and there’s technical risk in “building a hardware device and software protocol that maximizes safety/effectiveness/convenience” but those are both sort of iterative search-and-improvement processes that seem tractable to me. “Where do we go from here” seems very concrete and operationalizable: what’s the R&D roadmap, what sorts of expertise do we need on board, etc. But, nobody’s done it yet and plenty of people are still skeptical that it can (or should) be done. That’s my “sweet spot.”

My “sweet spot” is mine, not the single objectively correct place.

When I meet someone who’s proposing something “big if true” and cool but way more ...
Podcast discussing Hanson's Cultural Drift Argument
by vaishnav92 — 2024-10-20T17:58:41.416Z
Link: https://moralmayhem.substack.com/p/the-hanson-hypothesis-global-unity

In the latest episode of Moral Mayhem, we discuss Robin Hanson's essays on cultural drift -- examining whether and how global monoculture and reduced existential pressures may lead to biologically maladaptive cultural values. I've seen at least one post on this topic on LW.

@Robin_Hanson will be joining us next week to continue this discussion. Feel free to comment with questions you'd be interested in having Hanson respond to.
Advice on Communicating Concisely
by EvolutionByDesign — 2024-10-20T16:45:41.053Z

Many people I encounter regularly tell me that I have difficulties communicating spoken ideas in a concise, timely, fashion. I've been reading Less Wrong since I was fifteen, and ask because its likely that people here have useful suggestions. Are there any books on communicating concisely that I ought to read?
Ambiguities or the issues we face with AI in medicine
by Thehumanproject.ai — 2024-10-20T16:45:31.341Z

Abstract

With AI gradually becoming more relevant to healthcare, we are running into a diverse set of issues related to ambiguous medical data, expert disagreement, and biased outcomes. For AI to make accurate medical predictions, significant improvements in data collection, standardization, and ethical oversight are necessary, which come with their own set of additional challenges. In this thought piece I will lay out the issue of what ambiguities are, how they get to be and why they are so problematic in the medical AI context.

Preface

This is one text of a collection of essays and thought pieces that lies at the intersection of AI and other topics.

I’d highly appreciate to receive ideas, feedback and opinions, as well as engage in meaningful discussions about the topics I cover in this collection of essays, which I seek to publish over the course of the next weeks and months.

I am aware that they might have some typos, punctuation or grammatical errors, as I am just an individual writing this is my free time off work.

I hope to engage with a community of people who share my passion for AI, and I’d highly appreciate getting some other perspectives on these topics as these texts are based on my understanding of AI, the world and some additional thoughts and notions. I might not explain things to their full extent, or in a way that makes my chain of thoughts a little bit hard to understand if you come from a different domain, so if there are sections within the texts that are not explained sufficiently feel free to reach out. For the sake of not burdening you with too much to read there will also be some over-simplifications in these texts.

Please don’t see this as something set in stone. I am open to hearing different perspectives and broadening my view on this matter, as I am sure that there are points of view out there that are so far out of my field of view that I am unable to consider them as of now.

Introduction - Ambiguities in the real world

We primarily define our world through various forms of communication—whether verbal, visual, or written. These forms of communication are how we express ourselves, our feelings and share our experiences with others. Without words, it would be incredibly difficult to convey concepts or share our thoughts. For instance, the term "ghosting" didn’t exist just a few years ago, bu...
The Personal Implications of AGI Realism
by xizneb — 2024-10-20T16:43:37.870Z

Superintelligence Is On The Horizon

It’s widely accepted that powerful general AI, and soon after, superintelligence, may eventually be created.[1] There’s no fundamental law keeping humanity at the top of the intelligence hierarchy. While there are physical limits to intelligence, we can only speculate about where they lie. It’s reasonable to assume that even if we hit an S-curve in progress, that plateau will be far beyond anything even 15 John von Neumann clones could imagine.

Gwern was one of the first to recognise the "scaling hypothesis"; others followed later. While debate continues over whether scaling alone will lead to AI systems capable of self-improvement, it seems likely that scaling, combined with algorithmic progress and hardware advancements, will continue to drive progress for the foreseeable future. Dwarkesh Patel estimates a "

70% chance scaling + algorithmic progress + hardware advances will get us to AGI by 2040

". These odds are too high to ignore. Even if there are delays, superintelligence is still coming.

Some argue it's likely to be built by the end of this decade; others think it might take longer. But almost no one doubts that AGI will emerge this century, barring a global catastrophe. Even skeptics like Yann LeCun predict AGI could be reached in “years, if not a decade.” As Stuart Russell noted, estimates have shifted from “30-50 years” to “3-5 years.”

Leopold Aschenbrenner calls this shift "AGI realism." In this post, we focus on one key implication of this view—leaving aside geopolitical concerns:

We are rapidly building machines smarter than the smartest humans. This is not another cool Silicon Valley boom; this isn’t some random community of coders writing an innocent open source software package; this isn’t fun and games. Superintelligence is going to be wild; it will be the most powerful weapon mankind has ever built. And for any of us involved, it’ll be the most important thing we ever do.

Of course, this could be wrong. AGI might not arrive until later this century, though this seems increasingly unlikely. Nevertheless, it’s a future we must still consider.

Even in a scenario where AGI arrives late in the century, many of us alive today will witness it. I was born in the early 2000s, and it’s more probable than not that AGI will be developed within my lifetime. While much attention is paid to the technical, geopol...
Safety tax functions
by owencb — 2024-10-20T14:08:38.099Z
Link: https://strangecities.substack.com/p/safety-tax-functions

In thinking about the future of AI safety, it sometimes seems helpful to think in terms of “finding a solution to the alignment problem”. At other times, it seems helpful to think in terms of “paying a safety tax”. On the face of it, these two concepts seem hard to reconcile: one of them is about the moment where we have a solution and can stop investing resources into safety; the other presumes something like a constant fraction of resources going into safety, forever.

This is a post for concept geeks. I’ll explain that by allowing safety tax to vary with capability level for a hypothetical dangerous technology, we can represent both of the above dynamics as special cases of a more general class of functions. While I don’t think I’ve got to the point of having these concepts fully crisp, they seem sharp enough that I’ve been finding them helpful.

In a follow-up post, I’ll look more specifically at my picture of the safety tax landscape for AI — arguing that it’s unusually dynamic (in the sense that specific advances in AI might change the available tradeoffs), and that the peak safety tax requirements probably come in the period when AI is mildly- to moderately-superintelligent.

Paradigm cases

Suppose we have a potentially dangerous technology. The danger accrues as the technology gets more powerful; it needs sufficient investment in safety to keep it safe. How much? It depends on the nature of the technology and the potential problem.

Once-and-done problems

Some technological problems have “solutions”, in a pretty clean sense. After you’ve identified a solution, you’re basically good to keep on applying that solution whenever the problem comes up, and the cost of applying it is trivial compared to the total cost of the technology. You can solve the problem of lossy signals with error-correcting codes. You can solve the problem of buildings burning down by constructing things out of fire-resistant materials, even for much larger buildings than you originally thought of. Let’s call these once-and-done problems.

Ongoing problems

In contrast, some problems scale as the fundamentals of the underlying technology change, and you need to keep up with that, rather than having a single clean solution. Think computer security, where further advances in the underlying infrastructure open up more vectors for attack, so defence is a moving target. Let’s call these ong...
Exploring the Platonic Representation Hypothesis Beyond In-Distribution Data
by rokosbasilisk — 2024-10-20T08:40:04.404Z

The Platonic Representation Hypothesis (PRH) suggests that models trained with different objectives and on various modalities can converge to a shared statistical understanding of reality. While this is an intriguing idea, initial experiments in the paper focused on image-based models (like ViT) trained on the same pretraining (ImageNet) dataset. This raises an important question:

Does PRH hold only when models are trained on data from the same distribution?

To explore this question, the experiment was extended to ImageNet-O—a dataset specifically designed with out-of-distribution (OOD) images compared to ImageNet. Using ImageNet-O, the correlation analysis of alignment scores across various metrics for image classification models was re-evaluated.

The outcome? PRH holds true in the OOD setting as well, which challenges the notion that a shared data distribution is a prerequisite for this convergence. This observation carries significant implications for AI alignment research, suggesting that a deeper underlying structure may govern how models develop representations of reality, even when the training data differs.

Below are the correlation plots comparing results from the original in-distribution experiment to those with OOD data:

But does this mean that the model's align even for purely randomly generated data?

The answer is NO.

This plot shows the correlation of the alignment scores for the models on purely randomly generated images.

One key observation here is:

On randomly generated data, PRH is not so true, I.e, the models did not share a statistical model of reality.
On OOD data of ImageNet-O dataset they have shared, and have high correlation. But on ImageNet-O the predictions are wrong but with higher confidence.I.e, all the models classified the images wrongly on this outlier data yet predictably. This is particularly interesting since we can infer that they fail predictably.

The notebook documenting these experiments is available here.

What's Next?

The next step in this research is to identify datasets or conditions where PRH might be falsifiable
Electoral Systems
by RedFishBlueFish — 2024-10-20T03:25:20.677Z

Crossposting from my substack.

I figured I would crosspost to LessWrong since electoral systems seem like the type of thing rationalists would be interested in.

This is the latest post in my

constitutional design series

. It is the quintessential topic of constitutional design, the one that everyone loves to talk about. I hope to show and have shown that constitutional design is far more than just electoral systems, but it’s obviously an important topic and worth discussing. First I’ll discuss electoral systems for electing a single person, and then I’ll discuss multi-member electoral systems for electing a legislature. But before that…

Social Choice Theory Background

Maximizing Societal Utility

If you want a prescription for who objectively “should” win an election given certain voter preferences, there’s basically two you could choose. The first is to define each voter as giving some utility to each candidate, and then choose the candidate that maximizes total societal utility.

If you’re a

preference utilitarian

and you view these utility values as direct measures of people’s preferences, this is of course the optimal choice.[1] But if you’re a

hedonic

utilitarian like I am, or if you view these utility functions as people’s political preferences which are separate from people’s real preferences in life, or if you don’t stand by a certain ethical theory and just want something vaguely democratic, I don’t think this method comes out looking amazing.

This method says that someone who hates one candidate and loves another should have more say in an election than someone whose preferences are milder. This might be desirable because not doing so can lead you to tyranny of the majority, where 51% of the population hogs all the resources and is happy while the other 49% is miserable. But on the other hand, I think people can get really passionate about politics in ways that don’t merit more say, like being terrified of immigrants that don’t actually hurt them. And besides, there are other methods in society for people with stronger preferences to have more of an effect (activism, interest groups, also they probably have higher turnout in elections).

So overall I’m kind of ambivalent on this prescription.

The Condorcet Criterion

The other prescription for who should win an election is to choose the Condorcet winner (called the “Condorcet Criterion” for a voting system)....
Overcoming Bias Anthology
by Arjun Panickssery — 2024-10-20T02:01:23.463Z
Link: https://overcoming-bias-anthology.com/

Part 1: Our Thinking

Near and Far

1 Abstract/Distant Future Bias

2 Abstractly Ideal, Concretely Selfish

3 We Add Near, Average Far

4 Why We Don't Know What We Want

5 We See the Sacred from Afar, to See It Together

6 The Future Seems Shiny

7 Doubting My Far Mind

Disagreement

8 Beware the Inside View

9 Are Meta Views Outside Views?

10 Disagreement Is Near-Far Bias

11 Others' Views Are Detail

12 Why Be Contrarian?

13 On Disagreement, Again

14 Rationality Requires Common Priors

15 Might Disagreement Fade Like Violence?

Biases

16 Reject Random Beliefs

17 Chase Your Reading

18 Against Free Thinkers

19 Eventual Futures

20 Seen vs. Unseen Biases

21 Law as No-Bias Theatre

22 Benefit of Doubt = Bias

Part 2: Our Motives

Signaling

23 Decision Theory Remains Neglected

24 What Function Music?

25 Politics isn't about Policy

26 Views Aren't About Sights

27 Why Do Bets Look Bad?

28 Homo Hypocritus

29 Resolving Your Hypocrisy

30 Errors, Lies, and Self-Deception

Norms

31 Enforce Common Norms On Elites

32 Identity Norms

33 Exclusion As A Substitute For Norms, Law, & Governance

34 How Idealists Aid Cheaters

35 Beware Mob War Strategy

36 Automatic Norms

37 10 Implications of Automatic Norms

38 Automatic Norm Lessons

39 Automatic Norms in Academia

Fiction

40 Plot Holes & Blame Holes

41 Fairy Tales Were Cynical

42 Why Fiction Lies

43 Biases Of Fiction

44 Why We Fight Over Fiction

45 Stories Are Like Religion

46 More Stories As Religion

The Dreamtime

47 This is the Dream Time

48 DreamTime

49 Dreamtime Social Games

50 We Moderns Are Status-Drunk

51 Earth: A Status Report

52 On Teen Angst

Part 3: Our Institutions

Prediction Markets

53 Prediction Markets "Fail" To Mooch

54 Seeking Robust Credible Expertise Buyers

55 Prediction Markets Need Trial and Error

56 New-Hire Prediction Markets

57 Shoulda-Listened Futures

58 Brand Truth Narrowly

Academia

59 Fixing Academia Via Prediction Markets

60 Intellectual Prestige Futures

61 Academic Stats Prediction Markets

62 How To Fund Prestige Science

Medicine

63 Medical Doubts OpEd

64 Medical Market Failures

Paternalism

65 Paternalism Is About Status

66 Rulesy Folks Push Paternalism

67 Universal Basic Dorms

Law

68 Elites Must Rule

69 Status App Concept

70 Our Prestige Obsession

71 Yay Stability Rents

72 Conditional Harberger Tax Games

73 Reliable Private-Enough Physical Identity

74 Freedom Isn't Free

75 Quality Regs Say 'High Is Good'

76 Socialism: A Gift You'd Exchange?

77 Vouch For Pandemic Passports

78 Can We Tame Political Mind...
If far-UV is so great, why isn't it everywhere?
by Austin Chen — 2024-10-19T18:56:58.910Z
Link: https://strainhardening.substack.com/p/if-far-uv-is-so-great-why-isnt-it

Crossposting this essay by my friend Gavriel Kleinwaks, who is approximately the world expert in Far UV deployment.

[Edit: Austin is very kind--I am not close to being the world expert in far UV deployment; there are people who run/used to run companies trying to do that, and researchers who work with them very closely, who know more about far-UV deployment, and I'm largely consolidating information from them. -Gavriel]

Disclaimer: In the interest of avoiding the problem where a post languishes in drafts forever, I am not attempting to be particularly rigorous, but I’m happy to answer questions to fill in details about how I’ve come to the conclusions I’m writing about. This post is informed by my work experience at 1Day Sooner, but only reflects my personal takes, not the official stance of my organization.

I work on indoor air quality to reduce disease burden and pandemic risk, and far-UV is an extremely promising intervention. One of the questions I get most often is, “If this is so promising, why isn’t it everywhere?” The answer requires a lot of breakdown and I get a lot of follow-up questions as soon as I start to answer, so this post is an attempt to run through the entire conversation at once. I’d never actually get around to posting this if I also included a deep dive on far-UV or tried to prove its value, so if you don’t know yet why it’s promising, you can check out

this report

.

Top line takeaway: Far-UV isn’t widely used because no individual organization has the incentive or bandwidth to install it, and the industry is stuck in a vicious cycle, making companies unreliable. Filtration is already widely used, but more user-friendly filters would greatly improve implementation.

Q: What are the major roadblocks to widespread use of high-quality filters?

A: The market is hard for consumers to navigate; it’s difficult for consumers to be well-informed enough to assess which filters are high-quality. For example, ionizers (ozone producing and, according to some researchers, basically ineffective

1

) and portable HEPA filters (no harmful byproducts, universally agreed to be useful) are both sold as “air purifiers.”

Q: What are the major roadblocks to widespread use of far-UV?

A:

Far-UV is not intuitive to most people (since other forms of UV are directly harmful to interact with), and people responsible for improving air quality in a given public space may n

...