trentmkelly/LessWrong-43k huggingface dataset:
An explanation of evil in an organized world
by KatjaGrace — 2024-05-02T05:20:06.240Z

A classic problem with Christianity is the so-called ‘problem of evil’—that friction between the hypothesis that the world’s creator is arbitrarily good and powerful, and a large fraction of actual observations of the world.

Coming up with solutions to the problem of evil is a compelling endeavor if you are really rooting for a particular bottom line re Christianity, or I guess if you enjoy making up faux-valid arguments for wrong conclusions. At any rate, I think about this more than you might guess.

And I think I’ve solved it!

Or at least, I thought of a new solution which seems better than the others I’ve heard. (Though I mostly haven’t heard them since high school.)

The world (much like anything) has different levels of organization. People are made of cells; cells are made of molecules; molecules are made of atoms; atoms are made of subatomic particles, for instance.

You can’t actually make a person (of the usual kind) without including atoms, and you can’t make a whole bunch of atoms in a particular structure without having made a person. These are logical facts, just like you can’t draw a triangle without drawing corners, and you can’t draw three corners connected by three lines without drawing a triangle. In particular, even God can’t. (This is already established I think—for instance, I think it is agreed that God cannot make a rock so big that God cannot lift it, and that this is not a threat to God’s omnipotence.)

So God can’t make the atoms be arranged one way and the humans be arranged another contradictory way. If God has opinions about what is good at different levels of organization, and they don’t coincide, then he has to make trade-offs. If he cares about some level aside from the human level, then at the human level, things are going to have to be a bit suboptimal sometimes. Or perhaps entirely unrelated to what would be optimal, all the time.

We usually assume God only cares about the human level. But if we take for granted that he made the world maximally good, then we might infer that he also cares about at least one other level.

And I think if we look at the world with this in mind, it’s pretty clear where that level is. If there’s one thing God really makes sure happens, it’s ‘the laws of physics’. Though presumably laws are just what you see when God cares. To be ‘fundamental’ is to matter so much that the universe runs on the clockwork of ...
Why I stopped working on AI safety
by jbkjr — 2024-05-02T05:08:56.813Z
Link: https://jbkjr.me/posts/2024/05/why_i_am_no_longer_ai_safety/

Here’s a description of a future which I understand Rationalists and Effective Altruists in general would endorse as an (if not the) ideal outcome of the labors of humanity: no suffering, minimal pain/displeasure, maximal ‘happiness’ (preferably for an astronomical number of intelligent, sentient minds/beings). (Because we obviously want the best future experiences possible, for ourselves and future beings.)

Here’s a thought experiment. If you (anyone - everyone, really) could definitely stop suffering now (if not this second then reasonably soon, say within ~5-10 years) by some means, is there any valid reason for not doing so and continuing to suffer? Is there any reason for continuing to do anything else other than stop suffering (besides providing for food and shelter to that end)?

Now, what if you were to learn there really is a way to accomplish this, with method(s) developed over the course of thousands of human years and lifetimes, the fruits of which have been verified in the experiences of thousands of humans, each of whom attained a total and forevermore cessation of their own suffering?

Knowing this, what possible reason could you give to justify continuing to suffer, for yourself, for your communities, for humanity?

Why/how this preempts the priority of AI work on the present EA agenda

I can only imagine one kind of possible world in which it makes more sense to work on AI safety now and then stop suffering thereafter. The sooner TAI is likely to arrive and the more likely it is that its arrival will be catastrophic without further intervention and (crucially) the more likely it is that the safety problem actually will be solved with further effort, the more reasonable it becomes to make AI safe first and then stop suffering.

To see this, consider a world in which TAI will arrive in 10 years, it will certainly result in human extinction unless and only unless we do X, and it is certainly possible (even easy) to accomplish X in the next 10 years. Presuming living without suffering is clearly preferable to not suffering by not living, it is not prima facie irrational to spend the next 10 years ensuring humanity’s continued survival and then stop suffering.

On the other hand, the more likely it is that either 1) we cannot or will not solve the safety problem in time or 2) the safety problem will be solved without further effort/...
[Linkpost] Silver Bulletin: For most people, politics is about fitting in
by Gunnar_Zarncke — 2024-05-01T18:12:43.238Z
Link: https://www.natesilver.net/p/for-most-people-politics-is-about

Nate Silver tries to answer the question: "How do people formulate their political beliefs?"

An important epistemological question that is, he says, under-discussed.

He lays out his theory:

I think political beliefs are primarily formulated by two major forces:

Politics as self-interest. Some issues have legible, material stakes. Rich people have an interest in lower taxes. Sexually active women (and men!) who don’t want to bear children have an interest in easier access to abortion. Members of historically disadvantaged groups have an interest in laws that protect their rights

Politics as personal identity — whose team are you on. But other issues have primarily symbolic stakes. These serve as vehicles for individual and group expression — not so much “identity politics” but politics as identity. People are trying to figure out where they fit in — who’s on their side and who isn’t. And this works in both directions: people can be attracted to a group or negatively polarized by it. People have

different reasons for arguing about politics

, and can derive value from a sense of social belonging and receiving reinforcement that their choices are honorable and righteous.

1

Notice what’s missing from my list? The notion of politics as a battle of ideas. This is not to suggest that people don’t hold reasonable moral intuitions about political affairs. But when it comes to mass popular opinion, the number of people who are interested in ideas for ideas sake is vanishingly small.
Launching applications for AI Safety Careers Course India 2024
by Axiom_Futures — 2024-05-01T17:55:38.013Z

Announcing open applications for the

AI Safety Careers Course India

2024!

Axiom Futures

has launched its flagship AI Safety Careers Course 2024 to equip emerging talent working on India with foundational knowledge in AI safety. Spread out across 8-10 weeks, the program will provide candidates with key skills and networking opportunities to take their first step toward an impactful career in the domain. Each week will correspond with a curriculum module that candidates will be expected to complete, and discuss with their cohort during the facilitated seminar. We expect a set of candidates to pursue applied projects of their choice.

The program is aimed at undergraduate and (Master’s/PhD) graduate students, and young professionals. If you are a high-school student with demonstrated interest in AI safety, we encourage you to apply. We expect applicants to come from a diverse set of backgrounds, including but not limited to STEM, economics, law, and public policy, etc.
The course is primarily aimed at Indian citizens, NRIs, OIC card holders, and Indians living, studying, and working abroad. Having said that we encourage candidates from all nationalities and regions to apply - especially other Global South countries.

Applications close 23:59 IST, 19 May 2024. Expected program dates are 10 June–31 August, 2024.

Apply now

! Find more information on our

webpage

. If you know someone talented who would benefit from this program, consider referring them. We have a few spots open in our referral program– please reach out to partner with us if you are interested.
Axiom Futures is incubated by Impact Academy. You can follow the

LinkedIn

page or

subscribe

to the newsletter to stay informed about similar opportunities.
Shane Legg's necessary properties for every AGI Safety plan
by jacquesthibs — 2024-05-01T17:15:41.233Z

I've been going through the FAR AI videos from the alignment workshop in December 2023. I'd like people to discuss their thoughts on Shane Legg's 'necessary properties' that every AGI safety plan needs to satisfy. The talk is only 5 minutes, give it a listen:

Otherwise, here are some of the details:

All AGI Safety plans must solve these problems (necessary properties to meet at the human level or beyond):

Good world model
Good reasoning
Specification of the values and ethics to follow

All of these require good capabilities, meaning capabilities and alignment are intertwined.

Shane thinks future foundation models will solve conditions 1 and 2 at the human level. That leaves condition 3, which he sees as solvable if you want fairly normal human values and ethics.

Shane basically thinks that if the above necessary properties are satisfied at a competent human level, then we can construct an agent that will consistently choose the most value-aligned actions. And you can do this via a cognitive loop that scaffolds the agent to do this.

Shane says at the end of this talk:

If you think this is a terrible idea, I want to hear from you. Come talk to me afterwards and tell me what's wrong with this idea.

Since many of us weren't at the workshop, I figured I'd share the talk here to discuss it on LW.
KAN: Kolmogorov-Arnold Networks
by Gunnar_Zarncke — 2024-05-01T16:50:58.124Z
Link: https://arxiv.org/abs/2404.19756

ADDED: This post is controversial. For details see the comments below or the post Please stop publishing ideas/insights/research about AI (which is also controversial).

Abstract:

Inspired by the Kolmogorov-Arnold representation theorem, we propose Kolmogorov-Arnold Networks (KANs) as promising alternatives to Multi-Layer Perceptrons (MLPs). While MLPs have fixed activation functions on nodes ("neurons"), KANs have learnable activation functions on edges ("weights"). KANs have no linear weights at all -- every weight parameter is replaced by a univariate function parametrized as a spline. We show that this seemingly simple change makes KANs outperform MLPs in terms of accuracy and interpretability. For accuracy, much smaller KANs can achieve comparable or better accuracy than much larger MLPs in data fitting and PDE solving. Theoretically and empirically, KANs possess faster neural scaling laws than MLPs. For interpretability, KANs can be intuitively visualized and can easily interact with human users. Through two examples in mathematics and physics, KANs are shown to be useful collaborators helping scientists (re)discover mathematical and physical laws. In summary, KANs are promising alternatives for MLPs, opening opportunities for further improving today's deep learning models which rely heavily on MLPs.
Manifund Q1 Retro: Learnings from impact certs
by Austin Chen — 2024-05-01T16:48:33.140Z

Manifund is a philanthropic startup that runs a website and programs to fund awesome projects. From January to now, we wrapped up 3 different programs for impact certificates (aka venture-style funding for charity projects): ACX Grants, Manifold Community Fund, and the Chinatalk essay competition.

Overall, we’ve learned a lot and are happy with the projects we’ve funded, but are less excited by impact certs than before — it’s been hard to get investor interest, and we still haven’t found a use case where certs led to better funding decisions. For the next quarter, we’re trying out a bunch of things (including Manifest, regranting, and prize challenges) while looking to get product-market fit.

What we’ve been working on in Q1

ACX Grants 2024

Manifund hosted the 2024 round of ACX Grants, which included both direct funding to some projects and an impact market for any project that opted-in and did not receive direct funding. ACX directly funded 33 projects for a total of $1.35mm; on the impact market, investors funded 12 projects to their minimum bars, for ~$50k total at a combined valuation of ~$200k.

Background

Astral Codex Ten

(ACX) is a blog by Scott Alexander on topics like reasoning, science, psychiatry, medicine, ethics, genetics, AI, economics, and politics.

ACX Grants

(ACXG) is a program in which Scott helps fund charitable and scientific projects — see the 2022 round

here

and his retrospective on ACX Grants 2022

here

.

In this round (ACX Grants 2024), some of the applications were

directly funded

by Scott; the rest were given the option to participate in an impact market, an alternative to grants or donations as a way to fund charitable projects. Manifund transferred funds from donors to grantees; hosted project proposals on our website; and ran the impact marketplace.

ACXG Direct Funding

Scott evaluated all of the applications along with his team of judges, then directed a total of $1.35M to 33 projects. You can see the results on

ACX

or on the

“Grants” tab

of Manifund’s ACXG 2024 page. Example grants:

Open source predictors for polygenic screening

, by Gene Smith

Building anti-mosquito drones

, by Alex Toussaint

Solving Far-UVC ozone generation

, by Jacob Swett

This was relatively straightforward to run, as making direct grants is similar to what we did in our regranting program from last year. The biggest operational burden was co...
ACX Covid Origins Post convinced readers
by ErnestScribbler — 2024-05-01T13:06:20.818Z

ACX recently posted about the Rootclaim Covid origins debate, coming out in favor of zoonosis. Did the post change the minds of those who read it, or not? Did it change their judgment in favor of zoonosis (as was probably the goal of the post), or conversely did it make them think Lab Leak was more likely (as the "Don't debate conspiracy theorists" theory claims)?

I analyzed the ACX survey to find out, by comparing responses before and after the post came out. The ACX survey asked readers whether they think the origin of Covid is more likely natural or Lab Leak. The ACX survey went out March 26th and was open until about April 10th. The Covid origins post came out March 28th, and the highlights on April 9th. So we can compare people who responded before the origins post came out to those who responded after[1]. We should be careful, though, since those who fill out the survey earlier could be different than those who filled out later, and this could create a correlation which isn't causal.

I used a Regression Discontinuity Design on the time of the response to see if there was a break in the trend of responses right at the time the Covid post went up. Figuratively, this compares respondents "right before" the post to "right after" so can help assuage the confound fears.

I find that the post made readers more likely to think that the origin was indeed zoonosis. And this is highly significant. Here are the results, in charts.

Analysis

Here is the number of responses over time, with the timings of the posts highlighted. We'll mostly just need the timing of the Covid origins post, which is around response 4,002.

I'm assuming that readers who responded to the survey after the post went up have read the post before responding. This is the post engagement data[1] which shows within a few days of posting, most views of the post already took place.

The ACX Survey asked respondents what they thought about Covid origins.

I substracted 3 from the questionnaire response, to analyze a centered scale, for convenience. Here are the sliding window averages of 1,000 responses.

There are some fluctuations, but quite clearly there is a break in the trend at the time of the post, with readers starting to give scores more towards zoonosis. Looks like the post lowered responses by about 0.5 points (this takes time to transition in the chart, because of the sliding wind...
LessWrong Community Weekend 2024, open for applications
by UnplannedCauliflower — 2024-05-01T10:18:21.992Z

Main event page

Friday 13th September- Monday 16th September 2024 is the 11th annual Less Wrong Community Weekend (LWCW) in Berlin. This is the world’s largest rationalist social gathering which brings together 250+ aspiring rationalists from across Europe and beyond for four days of intellectual exploration, socialising and fun.

We’re expanding to 250+ participants and taking over the whole hostel. This year there will only be us during the event: a huge variety of spaces to talk, relax and have fun with a higher sense of security and freedom.

The EAGx and LWCW happen this year during the same weekend, due to limited availability of conference centres. It is an unkind choice having to pick one community over the other. To freely decide which sessions to attend and where, we will offer a reduced ticket that includes 3x bed & breakfast at the hostel for you to enjoy the unique LWCW atmosphere and community, as well as join the talks at EAGx during the day.

We are delighted to have Anna Riedl for this year's key note. Anna is a cognitive scientist and conducts research on rationality under radical uncertainty, a phenomenon in the intersection of psychology, economics, neuroscience and artificial intelligence, directly relevant for improving human and institutional decision-making in real life.

That said the majority of the content will be participant driven in an unconference style: on Friday afternoon we put up six wall-sized daily planners and by Saturday morning the attendees fill them up with 100+ workshops, talks and activities of their own devising. Most are prepared upfront but some are just made up on the spot when inspiration hits.

Previous years’ schedules have included…

Double Cruxing
Hamming Circles
Gendlin Focusing
Applied Rationality workshops
Circling
Authentic Relating games
Improvisation theater
Introduction to stand up comedy
Writing rationalist fiction
Dance workshops
Acapella singing
Icebreaker games
Lightning talks
Celebrating failure groups
Giant outdoor chess Penultima
Dungeons & Dragons
Kung Fu basics
Board games

Breathwork workshops
Ecstatic dancing
Radical Honesty workshops
Playfighting for adults
Polyamory and relationships workshops
Sex Q&A roundtable
Quantified self workshops
Moral philosophy debates
AI safety Q&A
How to handle fear of AI Doom
Value drift in EA
The neurobiology of psychedelics
The science of longevity
Morn

...
Take SCIFs, it’s dangerous to go alone
by latterframe — 2024-05-01T08:02:38.067Z

Coauthored by Dmitrii Volkov1, Christian Schroeder de Witt2, Jeffrey Ladish1 (1Palisade Research, 2University of Oxford).

We explore how frontier AI labs could assimilate operational security (opsec) best practices from fields like nuclear energy and construction to mitigate near-term safety risks stemming from AI R&D process compromise. Such risks in the near-term include model weight leaks and backdoor insertion, and loss of control in the longer-term.

We discuss the Mistral and LLaMA model leaks as motivating examples and propose two classic opsec mitigations: performing AI audits in secure reading rooms (SCIFs) and using locked-down computers for frontier AI research.

Mistral model leak

In January 2024, a high-quality 70B LLM leaked from Mistral. Reporting suggests the model leaked through an external evaluation or product design process. That is, Mistral shared the full model with a few other companies and one of their employees leaked the model.

Mistral CEO suggesting adding attribution to the HuggingFace repo with the leaked LLM

Then there’s LLaMA which was supposed to be slowly released to researchers and partners, and leaked on 4chan a week after the announcement[1], sparking a wave of open LLM innovation.

Potential industry response

Industry might respond to incidents like this[2] by providing external auditors, evaluation organizations, or business partners with API access only, maybe further locking it down with query / download / entropy limits to prevent distillation.

This mitigation is effective in terms of preventing model leaks, but is too strong—blackbox AI access is insufficient for quality audits. Blackbox methods tend to be ad-hoc, heuristic and shallow, making them unreliable in finding adversarial inputs and biases and limited in eliciting capabilities. Interpretability work is almost impossible without gradient access.

Black-Box Access is Insufficient for Rigorous AI Audits, Table 1

So we are at an impasse—we want to give auditors weights access so they can do quality audits, but this risks the model getting leaked. Even if eventual leaks might not be preventable, at least we would wish to delay leakage for as long as possible and practice defense in depth. While we are currently working on advanced versions of rate limiting involving limiting entropy / differential privacy budget to allow secure remote mod...
AXRP Episode 30 - AI Security with Jeffrey Ladish
by DanielFilan — 2024-05-01T02:50:04.621Z

YouTube link

Top labs use various forms of “safety training” on models before their release to make sure they don’t do nasty stuff - but how robust is that? How can we ensure that the weights of powerful AIs don’t get leaked or stolen? And what can AI even do these days? In this episode, I speak with Jeffrey Ladish about security and AI.

Topics we discuss:

Fine-tuning away safety training

Dangers of open LLMs vs internet search

What we learn by undoing safety filters

What can you do with jailbroken AI?

Security of AI model weights

Securing against hackers vs AI exfiltration

The state of computer security

How AI labs could be more secure

What does Palisade do?

AI phishing

More on Palisade’s work

Red lines in AI development

Making AI legible

Following Jeffrey’s research

Daniel Filan:
Hello, everybody. In this episode, I’ll be speaking with Jeffrey Ladish. Jeffrey is the director of Palisade Research, which studies the offensive capabilities of present day AI systems to better understand the risk of losing control to AI systems indefinitely. Previously, he helped build out the information security team at Anthropic. For links to what we’re discussing, you can check the description of this episode and you can read the transcript at axrp.net. Well, Jeffrey, welcome to AXRP.

Jeffrey Ladish:
Thanks. Great to be here.

Fine-tuning away safety training

Daniel Filan:
So first I want to talk about two papers your Palisade Research put out. One’s called LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B, by Simon Lermen, Charlie Rogers-Smith, and yourself. Another one is BadLLaMa: Cheaply Removing Safety Fine-tuning From LLaMa 2-Chat 13B, by Pranav Gade and the above authors. So what are these papers about?

Jeffrey Ladish:
A little background is that this research happened during MATS summer 2023. And LLaMa 1 had come out. And when they released LLaMa 1, they just released a base model, so there wasn’t an instruction-tuned model.

Daniel Filan:
Sure. And what is LLaMa 1?

Jeffrey Ladish:
So LLaMa 1 is a large language model. Originally, it was released for researcher access. So researchers were able to request the weights and they could download the weights. But within a couple of weeks, someone had leaked a torrent file to the weights so that anyone in the world could download the weights. So that was LLaMa 1, released by M...