trentmkelly/LessWrong-43k huggingface dataset:
MIRI’s 2024 End-of-Year Update
by Rob Bensinger — 2024-12-03T04:33:47.499Z

MIRI is a nonprofit research organization with a mission of addressing the most serious hazards posed by smarter-than-human artificial intelligence. In our general strategy update and communications strategy update earlier this year, we announced a new strategy that we’re executing on at MIRI, and several new teams we’re spinning up as a result. This post serves as a status update on where things stand at MIRI today.

We originally planned to run a formal fundraiser this year, our first one since 2019. However, we find ourselves with just over two years of reserves, which is more than we’ve typically had in our past fundraisers.

Instead, we’ve written this brief update post, discussing recent developments at MIRI, projects that are in the works, and our basic funding status. In short: We aren’t in urgent need of more funds, but we do have a fair amount of uncertainty about what the funding landscape looks like now that MIRI is pursuing a very different strategy than we have in the past. Donations and expressions of donor interest may be especially useful over the next few months, to help us get a sense of how easy it will be to grow and execute on our new plans.

You can donate via our Donate page, or get in touch with us here. For more details, read on.

About MIRI

For most of MIRI’s 24-year history, our focus has been on a set of technical challenges: “How could one build AI systems that are far smarter than humans, without causing an extinction-grade catastrophe?”

Over the last few years, we’ve come to the conclusion that this research, both at MIRI and in the larger field, has gone far too slowly to prevent disaster. Although we continue to support some AI alignment research efforts, we now believe that absent an international government effort to suspend frontier AI research, an extinction-level catastrophe is extremely likely.

As a consequence, our new focus is on informing policymakers and the general public about the dire situation, and attempting to mobilize a response.

Learn more:

For a basic overview of why AI progress poses such an extreme risk, see Will Superintelligent AI End the World? (short TED talk) and AGI Ruin (longer explainer).
For an overview of what’s changed at MIRI and what we’ve been working on, see our 2024 Mission and Strategy Update and our 2024 Communications Strategy.
For more resources, check out our Media and Research pages, o

...
Who are the worthwhile non-European pre-Industrial thinkers?
by Lorec — 2024-12-03T01:45:31.445Z

At some point I became decently widely read in "Western philosophy", of the tradition that goes from Athens through Italy and Germany and Britain to the U.S. [ forking off into Philosophy and Science only after the Industrial Revolution ]. But somehow, I never acquired any operational sense of any of the corresponding "shadow" networks of writings that were only discovered by 'Western' philosophers to be published in Greek, German, Latin, or English after industrialization. Centrally, I'm thinking of East Asian, Indian, and Islamic authors, although there could be more philosophically productive pre-industrial cultures of which I'm ignorant.

I know there are compendia out there, but starting as I am from a position of almost total helpless ignorance on the object level, I trust random sources almost not at all to be discriminating on the actual value of people's ideas.

My anemic existing context: I wouldn't be a LessWronger if I hadn't read Musashi, but as far as I know he just didn't write much and wasn't very much in dialog with the rest of "his" culture [like a Socrates who was never succeeded]. Years ago I tried to read Confucius and bounced, due to disagreeing with all of his opinions. I also tried Sun Tzu, but only bounced off him because I found tactics boring at the time, and might try him again. I've heard that both pre-Industrial India and 11C-13C Islam were expert at medicine and [more in Islam's case] calculation, but I haven't retained any specific names [other than al-Khwarizmi, who didn't sound any more interestingly-idealist to read than, say, Blaise Pascal, though I could have misapprehended].

I've asked around and so far been recommended Mozi, who looks promising, and Xunzi. If you care to humor a cultural [& linguistic] monoglot's embarrassingly Knightian uncertainties: wrt wherever your locality of expertise is, where should I start? What should I know first?
A Paradox of Simulated Suffering
by arusarda — 2024-12-02T23:44:08.824Z

Abstract for Paper on Simulated Suffering:

This paper interrogates a fundamental epistemological problem: Can an artificial system's simulation of suffering be morally distinguishable from genuine suffering? By exploring the philosophical boundary between algorithmic emotion modelling and actual phenomenological experience, we challenge traditional binary distinctions between authentic and simulated consciousness.

The proposed research examines whether advanced machine learning models that can precisely predict and replicate human emotional responses create a new ontological category of experience - one that exists between genuine consciousness and sophisticated mimicry. Key philosophical provocations include: If an AI can predict human emotional responses with 99.9% accuracy, does the remaining 0.1% margin represent genuine difference or merely an arbitrary computational limitation?

This investigation proposes that the very act of perfectly modelling suffering constitutes a form of suffering itself, thus rendering our current philosophical frameworks of consciousness inadequate.
Levels of Thought: from Points to Fields
by HNX — 2024-12-02T20:25:02.802Z

Been trying to put together a framework for analyzing the way people think, process, approach, and prioritize information; for some time now. The same few patterns seem to come up rather consistently. The following is an attempt to systematize them a little bit.

To be clear, a few of these appear to be rather involuntary, naturalistic, and compulsive: less of a "this is the way people are", more of a "way they routinely choose to be".

Nevertheless, the more I think about them, the more they make sense.

Pointers

By the far the most common, most reactive, most emotional, and the most vocal of all. In politics, the first to repost any and all "events" they happen to personally resonate with. Whether towards the positive or negative side of the spectrum, usually matters little.

"Biden's gaffs are out of control: what just happened."

"Trump's completely deranged: here's the proof."

Political conversations inherently lend themselves rather poorly to any genuine scrutiny, as duly noted already. They do reveal quite well, nevertheless, the ones who are the most content and the least concerned, at once, with snap judgements and singular point-based conclusions. Traders of mental real estate belonging to the audience of their respective channel and/or platform rarely seem to care about their role in this regard, just as well.

It doesn't quite stop at politics, nevertheless. People reposting interview segments of any pop-tech CEO thoughtfully waving their hands and saying a great deal of words, so few of which carry any genuine information. With a few promptly adopting whatever word salad their ears had the misfortune of exposing themselves to as a holy gospel, some others procede to focus on the most dramatic, emotionally charged "hot takes":

"Musk's takeover of Twitter is genius / in shambles."

"It's all over / a new era for OpenAI and Altman now."

Does it reveal a bit more about the state of affairs within the mind of an average "influencer" and/or "news" organization in the space? Certainly. Neither would be considering such rhetoric if it didn't "work", however. The demand is clearly there. Whether the ones in demand are consciously aware of its ability to hijack their own attention span towards the latest "OMG" moment is of secondary importance here.

People, on average, are clearly content, satisfied, complacent enough to outsource their considerations to ...
From Code to Managing: Why Being a 'Force Multiplier' Matters to Me More Than Being a Coding Wizard

by cloak — 2024-12-02T20:10:57.743Z
Link: https://www.reddit.com/r/ClaudeAI/comments/1h52lht/from_code_to_leadership_why_being_a_force/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

TLDR provided by claude:
I've been hit with the sticks of what-not-to-do throughout my career (thanks OG mentors), and honestly? Those failures taught me more than any fancy degree could. Customer obsession + data + critical thinking, that's where the magic happens... implementing stuff? That's just the easier slice of the pie.

But here's the thing that's tugging at my heart - I'm realizing my real joy isn't in being the engineering wizard (I'm good, but I'm no Anthropic-level architect). It's in being the person with the magnifying glass, hunting for clues and empowering others to do the same. It's this beautiful intersection of work and purpose that just... hits different.

So I'm making this small step, this gradual shift from IC to whatever-comes-next. Strong opinion, loosely held. Might fail again. Might hurt. But that's the secret sauce they don't tell you - hope is a choice, and success comes through muscle memory, built one failure at a time. ❤️
A case for donating to AI risk reduction (including if you work in AI)
by tlevin — 2024-12-02T19:05:06.658Z

I work on Open Philanthropy’s AI Governance and Policy team, but I’m writing this in my personal capacity – several senior employees at Open Phil have argued with me about this!

This is a brief-ish post addressed to people who are interested in making high-impact donations and are already concerned about

potential risks from advanced AI

. Ideally such a post would include a case that reducing those risks is an especially important (and sufficiently tractable and neglected) cause area, but I’m skipping that part for time and will just point you to

this 80,000 Hours problem profile

for now.

Contrary to a semi-popular belief that donations in global catastrophic risks merely “funge” with major donors, there are several ways for individual donors, including those giving small amounts, to reduce global catastrophic risks from AI. These include donating to:

Work that would be less impactful if they were funded by the major funders, or if it were majority-funded by those funders, or would generally benefit from greater funding diversity for reasons of organizational health and independence.
Work that major funders won’t be able to discover, evaluate, and/or fund quickly enough, e.g. time-sensitive events, individual projects, or career transitions.
Work that encounters legal restrictions on size of donation, like political campaigns, political action committees/donor networks.
Work in sub-areas that major funders have decided not to fund.

You can donate to that kind of work either directly (by giving to the organizations or individuals) or indirectly (by giving through funds like the

AI Risk Mitigation Fund

, the

LTFF

, Longview’s

Emerging Challenges Fund

, or

JueYan Zhang

’s AI Safety Tactical Opportunities Fund.

Advantages to giving directly:

You can give to political campaigns/PACs/donor networks as well as 501(c)(4) lobbying/advocacy organizations, which the funds might not be able to do, though I’m not sure about all of them. (For political candidates, this probably means not giving in December 2024 and saving for future opportunities.)
Some funds might pose reputational issues for some especially reputation-sensitive recipients.
You can move especially quickly for things in the “time-sensitive event/project/transition” category.
You don’t have to defer to someone else’s judgment (and can help ease the grant evaluation capacity bottleneck!).

Advan

...
Fertility Roundup #4
by Zvi — 2024-12-02T14:30:05.968Z

There is little sign that the momentum of the situation is changing. Instead, things continue to slowly get worse, as nations in holes continue to keep digging. The longer we wait, the more expensive the ultimate price will be. We will soon find out what the new administration does, which could go any number of ways.

Table of Contents

Not Enough Dakka.

Embryo Selection.

Costs.

Proving that Dakka Works.

IVF.

Genetics.

Cultural Trends.

Denial.

Urbanization.

The Marriage Penalty.

The Biological Clock.

Technology Advances.

Big Families.

Au Pairs.

Childcare Regulations.

The Numbers.

The Housing Theory of Everything.

Causes.

The Iron Law of Wages.

South Korea.

Georgia (the Country).

Japan.

China.

Italy.

Northwestern Spain.

Russia.

Taiwan.

The United Kingdom.

Ancient Greece.

Israel.

More Dakka.

Perception.

Your Own Quest.

Help Wanted.

Not Enough Dakka

South Korea since 2006 has spent just over 1% of GDP on baby making incentives.

It is not doing that much. But then what would you expect?

As Emmett Shear says, wake me when it’s a lot higher. Which they are indeed proposing. I do not understand why people like Tim Carney respond the opposite way.

Tim Carney: This ought to puncture the notion that affordability or inequality is the source of America’s Baby Bust.

Dan Peters: Watch, they’ll just say it’s not enough money.

There’s no winning with facts.

Yes. Yes I will. It’s not enough money.

Paid sick leave decreases, rather than increases, fertility. This is despite paid sick leave being obviously very helpful when being pregnant, and also when having a child, which makes you sick more often and gives such flexibility higher value in general. Women today want more children than they have. If paid sick leave is decreasing fertility, something is going very wrong.

Why would this have the opposite effect? Could this be a wealth effect, a classic case of giving people what you think they want, rather than what they actually want or need? The paper’s suggestion that this facilitates use of birth control? How does that even interact with ‘sick leave’ and why is this even an issue? Something else?

My best guess is that this essentially forces the women to take more sick leave, but in a way that they see as hurting their careers, or that increases how much they expect their careers to be hurt, or causes employers to try and stop women from having kids, ...
Conjecture: A Roadmap for Cognitive Software and A Humanist Future of AI
by Connor Leahy — 2024-12-02T13:28:57.977Z
Link: https://www.conjecture.dev/research/conjecture-a-roadmap-for-cognitive-software-and-a-humanist-future-of-ai

On Carcinogenic Complexity, Software Senescence and Cognitive Provenance: Our roadmap for 2025 and beyond

It is mandatory to start any essay on AI in the post-ChatGPT era with the disclaimer that AI brings huge potential, and great risks. Unfortunately, on the path we are currently on, we will not realize those benefits, but are far more likely to simply drown in terrible AI

slop

, undermine systemic cybersecurity and

blow ourselves up

.

We believe AI on its current path will continue to progress exponentially, to the point where it can automate, and summarily replace, all of humanity. We are unlikely to survive such a transition.

Powerful technology always comes with powerful risks, but this does not mean we have to wait idly for the other shoe to drop. Risks can be managed and prevented, while harnessing the benefits. We have done it with aviation, nuclear and other risky technologies, and we can do it again. But currently we are not on track to do so with AI.

What are we doing wrong with AI, and how can we do better? Given where we are currently with AI technology, what would it look like to actually build things safely, and usefully?

We think the answers are downstream of practical questions of how to build cognitive software well.

AI is often seen as a weird brain in a box you ask questions to and try desperately to cajole into doing what you ask of it. At

Conjecture

, we think about this differently. We want to take AI seriously as what it is, a software problem.

What would it mean to take AI seriously as software?

Part 1: Cognitive Software

The field of AI is weird. AIs are not like traditional software. They are more “grown” than they are “written”. It’s not like traditional software, where an engineer sits down and writes down line by line what an AI should do. Instead, you take a huge pile of data and “grow” a program on that data to solve your problem.

How these “grown” programs work internally is utterly obscure to our current methods of understanding, similar to how e.g. the human genome and its consequences for health are still extremely murky in practice. Even if we have the full genome of a patient sequenced, while we might be able to notice a few gene mutations that are well known to have certain effects, most of the genome and its functioning is completely unintelligible to us. We are in a similar situation w...
2024 Unofficial LessWrong Census/Survey
by Screwtape — 2024-12-02T05:30:53.019Z

The Less Wrong General Census is unofficially here! You can take it at this link.

Update: The census is closed, thank you all for taking it! I plan to have the data out sometime in January.

The oft-interrupted annual tradition of the Less Wrong Census is once more upon us!

If you are reading this post and identify as a LessWronger, then you are the target audience. If you are reading this post and don't identify as a LessWronger, you just read posts here or maybe go to house parties full of rationalists or possibly read rationalist fanfiction and like talking about it on the internet, or you're not a rationalist you're just, idk, adjacent, then you're also the target audience.

If you want to just spend five minutes answering the basic demographics questions before leaving the rest blank and hitting submit, that's totally fine. The survey is structured so the fastest and most generally applicable questions are (generally speaking) towards the start. At any point you can scroll to the bottom and hit Submit, though you won't be able to add more answers once you do. It is about 2/3rds the size of last year's survey if that helps.

The survey shall remain open from now until at least January 1st, 2025. I plan to close it sometime on Jan 2nd.

I don't work for LessWrong, but I do work on improving rationalist meetups around the world. Once the survey is closed, I plan to play around with the data and write up an analysis post like this one sometime in late January.

Remember, you can take the survey at this link.

Update: Survey is closed, and I plan to have the data out in mid to late January.

Ancient tradition is that if you take the survey you can comment here saying you took the survey, and people upvote you for karma.
Drexler's Nanotech Software
by PeterMcCluskey — 2024-12-02T04:55:20.432Z
Link: https://bayesianinvestor.com/blog/index.php/2024/12/01/drexlers-nanotech-software/

Two months ago I attended Eric Drexler's launch of
MSEP.one. It's open source software, written by
people with professional game design experience, intended to catalyze
better designs for atomically precise manufacturing (or generative
nanotechnology, as he now calls it).

Drexler wants to draw more attention to the benefits of nanotech, which
involve large enough exponents that our intuition boggles at handling
them. That includes permanent health (Drexler's new framing of life
extension and cures for aging).

He hopes that a decentralized network of users will create a rich
library of open-source components that might be used to build a nanotech
factory. With enough effort, it could then become possible to design a
complete enough factory that critics would have to shift from their
current practice of claiming nanotech is impossible, to arguing with
expert chemists over how well it would work.

Drexler hopes to gamify the software enough that people will use it for
fun. My cursory impression, based on playing with it for less than an
hour, is that it's not very close to being fun. I don't know if
that's even a reasonable goal. The software feels more professional and
easy to use than what I recall of similar software 20 years ago. Using
it still seems like work. I expect it will be hard to get many people to
use it without paying them.

Protein-based nanotech versus Diamondoid-style

MSEP.one currently supports development of diamondoid-style nanotech,
which produces pretty pictures, and seems somewhat likely to enable the
simplest and most understandable atomically precise factories. The
downside is that designs of this type are not very close to being
buildable with current tools. There isn't even a clear path to building
the appropriate tools.

There is at least as much interest in building atomically precise
systems out of proteins and/or DNA. The tools to build those designs
mostly work today. MSEP.one does not yet support such designs. The main
downsides are that they're harder to design and visualize. Proteins in
particular are big messy-looking blobs. It typically requires
sophisticated software to determine whether two of them will fit
together well. I haven't looked much at how hard it is to design a
protein to fit an arbitrary shape, but my impression is that it requires
a painful amount of trial and error.

Yet the presence of life indicates that protein-b...
Sorry for the downtime, looks like we got DDosd
by habryka — 2024-12-02T04:14:30.209Z

We were down between around 7PM and 8PM PT today. Sorry about that.

It's hard to tell whether we got DDosd or someone just wanted to crawl us extremely aggressively, but we've had at least a few hundred IP addresses and random user agents request a lot of quite absurd pages, in a way that was clearly designed to avoid bot-detection and block methods.

I wish we were more robust to this kind of thing, and I'll be monitoring things tonight to prevent it from happening again, but it would be a whole project to make us fully robust to attacks of this kind. I hope it was a one-off occurence, but also, I think we can figure out how to make it so we are robust to repeated DDos attacks, if that is the world we live in, though I do think it would mean strapping in for a few days of spotty reliability while we figure out how to do that.

Sorry again, and boo for the people doing this. It's one of the reasons why running a site like LessWrong is harder than it should be.
Is malice a real emotion?
by landscape_kiwi — 2024-12-01T23:47:55.823Z

I assume you are familiar with Hanlon's razor.

Never attribute to malice that which is adequately explained by stupidity.

Is there any "true" malice? Can't almost all "malice" be explained by greed, ignorance, or some other deadly sin?

Is malice a real emotion or an illusion people came up with to justify actions of their opposition? Malice is mentioned wherever there is conflict between humans (and that's a lot.)

As always, someone has already thought of this in some form. Here are some quick starting points:

The Wikipedia page for Hanlon's razor proceeds to list extended versions of Hanlon's razor by other people.
Malice in law is defined as "an actual intention to do the particular kind of harm that in fact was done; or recklessness as to whether such harm should occur" which doesn't really care about the exact motivations of the harm.

I might be (am) naive, but I don't believe that malice exists as a real emotion (at least the way it is mostly thought/imagined as.) Seeing the bad side of the world mostly through media/paper, has most likely mislead my understanding of malice.
Please don't confuse the emotion malice (wish to cause harm for the sake of causing harm) with the more general action malice (intention to cause harm with a motivation.)

So, is malice a real emotion? And assuming yes, what is the best way to interpret it?