trentmkelly/LessWrong-43k huggingface dataset:
Let's Design A School, Part 2.4
School as Education - The Curriculum (Phase 3, Specific)
by Sable — 2024-06-05T21:40:49.002Z
Link: https://affablyevil.substack.com/p/lets-design-a-school-part-24

In previous posts, we laid out our model of school-as-education:

Phase 1 was literacy and numeracy,

Phase 2 was core civilizational requirements and survey courses,

Phase 3 was core adulting requirements and self-study,

and went into detail about the core civilizational requirements of phase 2.

This post, we’ll dive into the core adulting requirements students will need to pass to graduate from our school. Students will likely take these classes around current high school age - think 14-17.

Core Adulting Requirements

What does it mean to grow up? To become an adult?

Every society of which I am aware has an abundance of coming-of-age stories. Children and adolescents go on some kind of journey, learn responsibility or self-restraint or to appreciate their parents, and return having grown up from the experience.

But while stories inspire us, they rarely provide actually useful, concretely demonstrated, and clearly explained life skills.

That’s what the core adulting curriculum is for.

What does one need to know to be a functional adult in this day and age? What kind of life skills are most useful? How does one develop emotionally beyond adolescence?

These classes won’t substitute for life experience or simple time spent living, but I think they can give plenty of students a head start on their journey.

Basic Money

Motivation

People say that money can’t buy happiness.

Whether or not that’s the case, it is true that poverty can buy an awful lot of misery.

Learning how to handle money is one of the most important - and most neglected - topics everyone should learn. Not because money is evil (it isn’t), but because your ability to manage it has a powerful effect on your life. Whether you save for retirement, accrue consumer debt, or invest your money, the effect over time will exert a massive influence on your future.

In Order To Pass

To pass the course, students will need to take an in-person exam demonstrating understanding of:

How to use a bank, including savings and checking accounts, including interest

Example response: When you put money in a bank, you’re basically loaning it to the bank. You can always get your money back out, but the bank pays you a little for loaning them your money. This is the interest payment you get from a bank. A savings account is usually just an account you use to accumulate money, whereas a checking account is usually the place money will enter or lea

...
METR is hiring ML Research Engineers and Scientists
by Xodarap — 2024-06-05T21:27:39.276Z
Link: https://metr.org/blog/2024-05-16-ml-engineers-needed/

METR is developing evaluations for AI R&D capabilities, such that evaluators can determine if further AI development risks a “capabilities explosion”, which could be extraordinarily destabilizing if realized.

METR is hiring ML research engineers/scientists to drive these AI R&D evaluations forward.

Why focus on risks posed by AI R&D capabilities? It’s hard to bound the risk from systems that can substantially improve themselves. For instance, AI systems that can automate AI engineering and research might start an explosion in AI capabilities – where new dangerous capabilities emerge far more quickly than humanity could respond with protective measures. We think it’s critical to have robust tests that predict if or when this might occur.

What are METR’s plans? METR has recently started developing threshold evaluations that can be run to determine whether AI R&D capabilities warrant protective measures such as information security that is resilient to state-actor attacks. Over time, we’d like to build AI R&D evaluations that smoothly track progress, so evaluators aren’t caught by surprise. Having researchers and engineers with substantial ML R&D experience themselves is the main bottleneck to progress on these evaluations.

Why build AI R&D evaluations at METR? METR is a non-profit organization that collaborates with government agencies and AI companies to understand the risks posed by AI models. As a third party, METR can provide independent input to regulators. At the same time, METR offers flexibility and compensation competitive with Bay Area tech roles, excluding equity.
Book review: The Quincunx
by cousin_it — 2024-06-05T21:13:55.055Z

The Quincunx is a 1989 novel by Charles Palliser, set in early 1800s England. I want to recommend it to everyone because it's really good, and it might be relevant to the AI transition. Let me try to explain.

The surface level of the book is a kind of mishmash of Dickensian themes. The main character is caught in a complicated inheritance dispute involving multiple families, each having histories of murder, uncertain parentage, stolen and returned documents and so on. The plot contains numerous puzzles that are fun to solve, the amount of planning is really kind of amazing, there are tons of details and everyone lies or makes mistakes but it still connects logically.

But the really interesting level of the book is the social level. The main character doesn't just progress through a bunch of plot puzzles; he also starts out as a child of minor nobility and then moves through society downward. His journey is a kind of descent into hell, ending up in the lowest levels of poverty existing in the early 1800s. The book is very well researched in that regard, borrowing a lot from the fantastic "London Labor and the London Poor". There are parallel plotlines involving rich and poor people, and the book paints a vivid picture of how the rich prey upon the poor.

England at that time was conducting enclosures. Basically, rich people put up fences around common land to graze sheep on it. The poor were left with no land to grow food on, and had to go somewhere else. They ended up in cities, living in slums, trying to find scarce work and giving their last pennies to slumlords. In short, it was a story of mass impoverishment of the population, conducted by the state and upper levels of society, who all benefited from it.

In the book we get a tour of all of it. From the countryside being hollowed out, to the city with the desperate search for work, the run-down lodgings, the drinking, prostitution, crime (we spend a bit of time with the protagonist living in a gang), the sometimes horrifying occupations that people are pushed into (like scrounging for coins in sewer tunnels under the city while avoiding tides). The injuries, disabilities, early deaths. Where Dickens called out specific social ills in order to fix them, like the workhouses in Oliver Twist, Palliser says society as a whole is unjust. His account is so historically detailed that it somehow transcends time, makes ...
How should I think about my career?
by Chico — 2024-06-05T18:11:41.088Z

Hi, I am relatively new to LessWrong and Rationalism but I've been trying to learn more about it over the last year or so. I am a Computer Science student from the UK and am about to graduate from university. I have been trying to figure out my career path from here and in the process have been applying to jobs and post-grad courses in a pretty scattershot way. I think though that I should maybe consider taking a more methodical approach.

My philosophy in life is broadly that there is little point to it, but that I can justify my own existence if I can do some measure of good. If I can make a difference to enough people or to the world and leave it a better place than I found it then at least I wasn't entirely pointless or a complete waste of space, oxygen and other natural resources. So far, I have spent my life learning and becoming a functioning adult, but now it's time to start really earning my place here.

So the question is, how can I do the most amount of good using the time and skills that I have? Clearly just working in some random corporate job, or for a defence company could earn me a very good living, but won't do much good.

Luckily, I am a problem solver, and there are many, many problems to solve. The challenge, then, is which to solve and how?

My first question is how do I find the problem that I should be working on? I guess I'm asking how can I quantify the most important issues we face as a species and as a society and how can I figure out which which ones I can have the greatest impact on?

There are 3 options: Get a job, start a business/non-profit or do research. There are many ways of mixing and matching these but these are the 3 main career options. I think that which one I choose depends greatly on what issue I'm tackling but any ideas on how I should think about that rationally would also help.

Thank you in advance for any answers.
AISN #36: Voluntary Commitments are Insufficient Plus, a Senate AI Policy Roadmap, and Chapter 1: An Overview of Catastrophic Risks
by Corin Katzke — 2024-06-05T17:45:25.261Z
Link: https://newsletter.safe.ai/p/ai-safety-newsletter-35-voluntary

Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required.

Subscribe here to receive future versions.

Listen to the AI Safety Newsletter for free on Spotify.

Voluntary Commitments are Insufficient

AI companies agree to RSPs in Seoul. Following the second AI Global Summit held in Seoul, the UK and Republic of Korea governments announced that 16 major technology organizations, including Amazon, Google, Meta, Microsoft, OpenAI, and xAI have agreed to a new set of Frontier AI Safety Commitments.

Some commitments from the agreement include:

Assessing risks posed by AI models and systems throughout the AI lifecycle.
Setting thresholds for severe risks, defining when a model or system would pose intolerable risk if not adequately mitigated.
Keeping risks within defined thresholds, such as by modifying system behaviors and implementing robust security controls.
Potentially halting development or deployment if risks cannot be sufficiently mitigated.

These commitments amount to what Anthropic has termed Responsible Scaling Policies (RSPs). Getting frontier AI labs to develop and adhere to RSPs has been a key goal of some AI safety political advocacy — and, if labs follow through on their commitments, that goal will have been largely accomplished.

RSPs are useful as one part of a “defense in depth” strategy, but they are not sufficient, nor are they worth the majority of the AI safety movement’s political energy. There have been diminishing returns to RSP advocacy since the White House secured voluntary AI safety commitments last year.

Crucially, RSPs are voluntary and unenforceable, and companies can violate them without serious repercussions. Despite even the best intentions, AI companies are susceptible to pressures from profit motives that can erode safety practices. RSPs do not sufficiently guard against those pressures.

Binding legal requirements to prioritize AI safety are necessary. In a recent essay for the Economist, Helen Toner and Tasha McCauley draw on their experience as former OpenAI board members to argue that AI companies can’t be trusted to govern themselves. Instead—as is the case in other industries—government must establish effective safety regulation.

One promising area of regulation is compute security and governance. Compute is a scarce and neces...
GPT2, Five Years On
by Joel Burget — 2024-06-05T17:44:17.552Z
Link: https://importai.substack.com/p/import-ai-375-gpt-2-five-years-later

Jack Clark's retrospective on GPT2 is full of interesting policy thoughts, I recommend reading the whole thing. One excerpt:

I've come to believe that in policy "a little goes a long way" - it's far better to have a couple of ideas you think are robustly good in all futures and advocate for those than make a confident bet on ideas custom-designed for one specific future - especially if it's based on a very confident risk model that sits at some unknowable point in front of you.

Additionally, the more risk-oriented you make your policy proposal, the more you tend to assign a huge amount of power to some regulatory entity - and history shows that once we assign power to governments, they're loathe to subsequently give that power back to the people. Policy is a ratchet and things tend to accrete over time. That means whatever power we assign governments today represents the floor of their power in the future - so we should be extremely cautious in assigning them power because I guarantee we will not be able to take it back.

For this reason, I've found myself increasingly at odds with some of the ideas being thrown around in AI policy circles, like those relating to needing a license to develop AI systems; ones that seek to make it harder and more expensive for people to deploy large-scale open source AI models; shutting down AI development worldwide for some period of time; the creation of net-new government or state-level bureaucracies to create compliance barriers to deployment (I take as a cautionary lesson, the Nuclear Regulatory Commission and its apparent chilling effect on reactor construction in the USA); the use of the term 'safety' as a catch-all term to enable oversight regimes which are not - yet - backed up by quantitative risks and well developed threatmodels, and so on.

I'm not saying any of these ideas are without redeeming qualities, nor am I saying they don't nobly try to tackle some of the thornier problems of AI policy. I am saying that we should be afraid of the power structures encoded by these regulatory ideas and we should likely treat them as dangerous things in themselves. I worry that the AI policy community that aligns with longterm visions of AI safety and AGI believes that because it assigns an extremely high probability to a future AGI destroying humanity that this justifies any action in the present - after all, if you thought

...
Who wants to be invited to the LW Metamodern dialogue?
by hunterglenn — 2024-06-05T16:39:07.534Z

I want to start a metamodern dialogue. Ideally, it will lead to a Metamodern handshake (or many of them), ie, we figure out how to help local people and systems get even more of what they selfishly want by aiming also at helping avert the meta-crisis, participating in the spirit of the times.

The way I see it is win-win-win. If I get more of what I selfishly want, that works, and if the world gets better at the same time, even better, and if that completely fails, then this is a good way to prepare for death and doom, so it's a win even if it fails! Why isn't everyone doing this?

Here's a MM presentation/introduction I did. LMK if you want to be added to the MM dialogue here on LW:
Nonreactivity: a simple model of meditation
by cesiumquail — 2024-06-05T16:26:51.167Z

Here’s my model of how meditation works:

Nonreactivity reduces suffering.
You can gradually develop nonreactivity through practice.

This model is easy to understand, and it clarifies a lot of confusing philosophical stuff in other models.

For example, “non-self” is one of the Three Characteristics of experiential reality in Buddhism. What we call the “self” is just a pattern of sensations, including the sensations of intention, effort, attention, thinking, voluntary muscle movement, tension in the head and chest, etc. Those sensations are just like any other sensations, but we react to them more, and we call that increased reactivity "identification". In other words, by seeing non-self and ceasing to identify with sensations, we’re actually just decreasing reactivity.

Another example is “enlightenment”, or “levels of awakening”. In my model, you just gradually decrease reactivity, and your suffering is gradually reduced. There’s no need for any benchmarks of achievement. It’s just a continuous process of making your experiential life easier. Maybe at some point you reach a stable equilibrium of minimal unnecessary suffering, but you don’t really need a name for it.

My hope is that this model will let people get the benefits of meditation more easily, without having to navigate all the confusing and frustrating philosophical stuff.

I should say from the start that the goal is to find a balance of nonreactivity and engagement that works for you. Personally, I do a lot of fun and engaging stuff every day and I would encourage anyone interested in meditation to keep living an active and fulfilling life, including working on projects you find meaningful.

What I mean by “nonreactivity”

Nonreactivity is having less intense responses to things. It’s a property of your mind and nervous system.

Throughout life, we develop habitual reactivity to all sorts of things. For example, you might be reactive to interpersonal conflict, abandonment, loneliness, mortality, physical pain (meaning you have an unusually intense psychological response to pain), failure, status hierarchies, social interaction, money, rejection, intimacy, talent, physical attractiveness, intelligence, lust, gender, power, sex, emotional sincerity, the concept of hell, food, physical fitness, height, politics, your self-image, the feeling of missing out, family, etc.

But there are also more subtle forms of reactiv...
graphpatch: a Python Library for Activation Patching
by Occam's Laser — 2024-06-05T15:08:47.416Z

This post is an announcement for a software library. It is likely only relevant to those working, or looking to start working, in mechanistic interpretability.

What is graphpatch?

graphpatch is a Python library for activation patching on arbitrary PyTorch neural network models. It is designed to minimize the amount of boilerplate needed to run experiments making causal interventions on the intermediate activations in a model. It provides an intuitive API based on the structure of a torch.fx.Graph representation compiled automatically from the original model. For a somewhat silly example, I can make Llama play Taboo by zero-ablating its output for the token representing "Paris":

with patchable_llama.patch(
{"lm_head.output": ZeroPatch(slice=(slice(None), slice(None), 3681))}
):
print(
tokenizer.batch_decode(
patchable_llama.generate(
tokenizer(
"The Eiffel Tower, located in",
return_tensors="pt"
).input_ids,
max_length=20,
use_cache=False,
)
)
)

["
 The Eiffel Tower, located in the heart of the French capital, is the most visited"]

Why is graphpatch?

graphpatch is a tool I wished had existed when I started my descent into madness entry into mechanistic interpretability with an attempt to replicate ROME on Llama. I hope that by reducing inconveniences (trivial and otherwise) I can both ease entry into the field and lower cognitive overhead for existing researchers. In particular, I want to make it easier to start running experiments on "off-the-shelf" models without the need to handle annoying setup—such as rewriting the model's Python code to expose intermediate activations—before even getting started. Thus, while graphpatch should work equally well on custom-built research models, I focused on integration with the Huggingface ecosystem with:

Support for PreTrainedModels loaded on multiple devices or with CPU or disk offloading
Support for 8-bit quantization as provided by bitsandbytes
Support for convenience functions such as generate() in the above example

How do I graphpatch?

graphpatch is available on PyPI and can be installed with pip:

pip install graphpatch

You can read an overview on the GitHub page for the project. Full documentation is available on Read the Docs.

I have also provided a Docker image that might be useful for starting your own mechanistic interpretability experime...
Startup Stock Options: the Shortest Complete Guide for Employees
by Boris T — 2024-06-05T15:03:49.777Z
Link: https://borisagain.substack.com/p/startup-stock-options-guide

This is a short, but comprehensive, guide to stock options, with specific example outcomes for employees and a sprinkle of Bayesian modeling of your chance to cash out with at least $1M.

Unlike other guides, this is not a stream of financial terminology. Instead, it is specific and answers the real questions with concrete examples. "What happens if I have vested and unvested stock options and the company is acquired?" and the like.

Most of all I tried to provide tools for tackling the main question: "Is it worth it?"

Topics covered: basics, how startup exits work, possible outcomes for stock options holders, taxes, dilution, how long you will wait, and how lucky do you have to be to make money. And, of course, how to lose all of your money. There are so many ways.

This guide exists because I followed the advice of a guy I met at the Lisbon rationality meetup. The advice was to take all your interests seriously and write out ideas to completion. The topic was mildly interesting to me and I went a little bit too deep in the rabbit hole.

Hope you find this useful!
Aggregative Principles of Social Justice
by Cleo Nardo — 2024-06-05T13:44:47.499Z

1. Introduction

1.1. Three aggregative principles

This article examines aggregative principles of social justice. These principles state that a social planner should make decisions as if they will face the aggregated personal outcomes of every individual in the population. Different conceptions of aggregation generate different aggregative principles.

Aggregative principles avoid many theoretical pitfalls of utilitarian principles. Unlike utilitarianism, aggregative principles do not require specifying a social welfare function, which is notoriously intractable. Moreover, they seem less prone to counterintuitive conclusions such as the repugnant conclusion or the violation of moral side constraints.[1]

There are three well-known aggregative principles:

Live every life once (LELO)
Harsyani's Lottery (HL)
Rawls' Original Position (ROI)

By the end of this article, we will see that these three aggregative principles are instances of an vast family of similar principles.

1.2. Living Every Life Once

The idea, as articulated below by William MacAskill, is that a social planner should make decisions as if they will live out every individual's life (past, present, and future) in sequence. We will call this principle of social justice "Live Every Life Once" (LELO).[2]

Imagine living the life of every human being who has ever existed — in order of birth. Your first life begins about 300,000 years ago in Africa. After living that life and dying, you travel back in time to be reincarnated as the second-ever person, born slightly later than the first, then the third-ever person, and so on.

[...] If you knew you were going to live all these future lives, what would you hope we do in the present? How much carbon dioxide would you want us to emit into the atmosphere? How careful would you want us to be with new technologies that could destroy, or permanently derail, your future? How much attention would you want us to give to the impact of today’s actions on the long term?

William MacAskill (2022), "The Case for Longtermism"

MacAskill's hope is that the social planner, following LELO, would choose policies benefiting each individual because they anticipate living each individual's life, and they would avoid policies harming any individual for the same reason. For example, the social planner wouldn't choose to emit dangerous pollution that will harm the health of future generations, becaus...