trentmkelly/LessWrong-43k huggingface dataset:
Slack gives you space to notice/reflect on subtle things
by Raemon — 2022-04-24T02:30:34.737Z

Followup to "My slack budget: 3 surprise problems per week"

Previously, I thought the reasons to preserve slack in your life (or in your organization) were to:

Avoid using up all of your resources
Avoid hitting a crisis where you suddenly have multiple surprise problems, and you have no choice but to either do a shitty job handling all of them, or deliberately not handle some of them at all (and deal with the consequences)
Avoid feeling really unpleasant
Allow you to live up to principles / be more pro-social.

This year in December/January, the Lightcone Infrastructure team (where I work) took on a large number of difficult projects at once. I was thinking about how wise/unwise this was, and chatting with John Wentworth about it. I listed the problems-with-lack-of-slack.

He said (something like): "Oh, that's not the point of slack. Or, not the part I'm most interested in. The point of slack is to give you the space to notice subtle things and think about them."

A rough model is something like, here are three types of things you can do:

You can take actions on whatever stuff it is you like to do on-purpose. (Typically your day job, or fun projects, or whatever)
You can rest/recover/do-random-fun-things.
You can... be cognizant of stuff going that isn't immediately relevant to the first two things, and mull it over, and notice new, potentially fruitful trains of thought about them.

When you're got too many things to do and stuff is constantly exploding and demanding your attention, #3 is the first thing to go. You often need to be putting out fires (#1), and if you do that too much and did into energy reserves your body will eventually be like "No, screw that, time to burn out for a bit and spend a week tired and recovering." (#2)

But, there won't be a moment where you experience a clear failure-and-control-mechanism that pushes you to spend time on #3. You just... won't notice a thing that you might have noticed.

And this is particularly important when you're working on problems you don't understand how to solve (such as AI alignment, or how to improve institutions, or learn/teach rationality, or, just, any ol' problem in your life you're currently confused about, or maybe haven't even yet realized that you have)

In the explore/exploit dichotomy, when solving a problem-you-don't-understand, having a train of thoughts in "explore" mode is pretty valuable. Slack gives y...
Calling for Student Submissions: AI Safety Distillation Contest
by Aris — 2022-04-24T01:53:03.591Z

At EA UC Berkeley, we’re launching an ongoing series of contests called the Artificial Intelligence Misalignment Solutions (AIMS) series. This second contest, the

Distillation Contest

, is now open to any student enrolled in a university/college: here are our

interest

and

submission

forms! The contest has prizes as large as $2,500 and closes on May 20th. If you’re interested, please fill out the interest form so that I can send you updates on resources and changes in the contest! This blog post restates the information that is on our website, with a bit more explanation of the contest's purpose.

(This is cross-posted on the EA Forum and Lesswrong.)

Purpose

AIMS Series

I think that it is currently difficult for university students to find tangible ways to engage with AI Safety. Generally, by creating a series of AI Safety contests, I hope to:

Help build social capital for students who are interested in Alignment and potentially good at it.
Create ways for people to test their fit for Alignment work.
Create a “brand” around my contests over time so that CS students recognize its name and winners recommend the contests to their friends. Hopefully, this name recognition would also increase the ability to create partnerships with CS orgs as well.

For this specific contest, I’m inspired by the arguments that the field of AI Alignment

needs more distillers

to improve communication

within the field

, as well as to make their research accessible to a wider audience. The Distillation Contest aims to produce value by:

Recruiting CS students who have never heard of EA or Alignment before (I will be doing this outreach at UC Berkeley through advertising, but other organizers are welcome to advertise to their own groups for recruitment).
Increasing the engagement of students who are already interested in Alignment.
Potentially producing useful distillations of Alignment research and increasing accessibility to said research.

Contest description

The Distillation Contest asks that participants:

1) Pick an article/post/research paper on AI Alignment/Safety (ideally from our list below) that would benefit from being more clearly explained.
2) Indicate which ideas or sections of their chosen research should be distilled. Applicants can either distill a whole post/article, a specific part of the post/article, or multiple posts/articles.
3) Create a distillation: a clearer

...
Rationality Dojo
by lsusr — 2022-04-24T00:53:57.384Z

Suppose you asked me to start a rationality dojo. What would I do?

I would tell you "Do not start a rationality dojo. It is a bad idea." A science camp, a programming camp, a math camp, a rhetoric camp and a Bible camp are all much better ideas.

But what if you offered me a lot of money? Then I would do it even though it is a bad idea.

Part 1: Philosophy

Lesson 1: What is Justice?

I would start by giving every student a toga. I'd decorate the facility in the style of Ancient Greece. Two teachers would wear togas too. One teacher would play the role of Socrates. The other teacher would play the role of Aristotle. There would be a buffet with modern Greek food (because ancient Greek food is missing tasty ingredients like tomatoes).

The purpose of Socrates is to teach the things Socrates teaches in the Dialogues of Plato.

The purpose of Aristotle is to teach the students to question authority.

Aristotle was wrong about basically everything. At the end of Lesson 1, students will be informed that they never should have trusted Aristotle.

Lesson 2: What is the Way?

Same as previous, except we use four Chinese scholars instead: Laozi (Daoism), Sunzi (practical Daoism), Mozi (legalism) and Confucious (Confucianism). Decoration is Chinese. Food is Chinese.

Students will be encouraged to use the Greek philosophical ideas they learned in Lesson 1 against the Chinese philosophers of Lesson 2.

Lesson 3: Economics

This time it's economists: Queen Elizabeth (mercantilism), Adam Smith (libertarian), Karl Marx (communist), Joseph Stalin (Communist), John Keynes (Keyenesian economics), Leonidas (slavery)

Lesson 4: Government

George Washington, Alexander Hamilton, Mao Zedong, Che Guevara, Xerxes, Mohammad, Catherine the Great, Leonidas

Lesson 5: Civil Rights

Martin Luther King Jr., Martin Luther, Moses, etcetera.

Lesson 6+

…you get the idea.

Summary

You might notice that at no point do I teach "rationality". That's because rationality is all about distrusting authority. Authority cannot (directly) teach students to distrust it. Distrust must be taught indirectly.

Part 2: Science

Lesson 1: Prove, via experiment, the Earth is round.

Lesson 2: Measure the diameter of the Earth.

Lesson 3: Mathematical proofs.

Lesson 4+: …you get the idea.

Part 3: Rhetoric

Students will have mock debates against each other. Timers will be used.

Part 4: Technology

Students will be taught to use the Linu...
Re: So You Want to Be a Dharma Teacher
by lsusr — 2022-04-23T22:31:33.400Z
Link: http://hardcorezen.info/so-you-want-to-be-a-dharma-teacher/7843

When I first started getting into Zen there was a brief period when I thought I wanted to be a dharma teacher. I remember telling my first teacher, Tim, I wanted to be a dharma teacher and he just gave me a look. It’s hard to describe that look, but I remember how it felt. And how that look felt is why I dropped the whole idea of becoming a dharma teacher.

―Brad Warner

The same could be said for Rationality. You should not aspire to teach Rationality. There are two types of teaching:

Teaching young children. There's nothing wrong with aspiring to teach children. Teaching children is a noble, necessary profession.

Teaching adults. Aspiring to teach adults puts you on shaky ground. Aspiring to teach rationality to adults is downright dangerous.

Why? Because personal experience trumps pedagogical skill.

It is not the critic who counts; not the man who points out how the strong man stumbles or where the doer of deeds could have done them better. The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood; who strives valiantly; who errs, who comes short again and again, because there is no effort without error and shortcoming; but who does actually strive to do the deeds; who knows great enthusiasms, the great devotions; who spends himself in a worthy cause; who at the best knows in the end the triumph of high achievement, and who at the worst, if he fails, at least fails while daring greatly, so that his place shall never be with those cold and timid souls who neither know victory nor defeat. Shame on the man of cultivated taste who permits refinement to develop into fastidiousness that unfits him for doing the rough work of a workaday world. Among the free peoples who govern themselves there is but a small field of usefulness open for the men of cloistered life who shrink from contact with their fellows. Still less room is there for those who deride the slight of what is done by those who actually bear the brunt of the day; nor yet for others who always profess that they would like to take action, if only the conditions of life were not exactly what they actually are. The man who does nothing cuts the same sordid figure in the pages of history, whether he be a cynic, or fop, or voluptuary. There is little use for the being whose tepid soul knows nothing of great and generous emotion, of the high pride, the s

...
Ineffective Altruism
by lsusr — 2022-04-23T22:07:09.946Z

There are many things I can put my time toward.

I started a company that provides a low-cost tool that helps people who pull their hair out.

I volunteer as a coach for my neighborhood highschool Robotics Club.

I frequently blog about rationality.

I sometimes blog about machine learning.

These do not have the same impact. My technology work helps more people per hour invested than my volunteering at Robotics Club. Does that mean I should put all of my time into inventing technology and none of it into Robotics Club?

No. That would be premature optimization. The root of all evil is premature optimization.

Occasionally someone who has used my inventions will message me and tell me it changed their life. Occasionally. Occasionally a reader of my blog will tell me they found it useful. Occasionally. When I help out at Robotics Club, the teenagers are happy to see me. Every. Single. Time.

I recently flew down to San Francisco to teach a class and give a speech. The organizers offered to put me in a fancy hotel. Instead, I found the cheapest AirBnb in the area and walked for an hour to the location. Partially I did this to save money, but I also did it to stay in touch with reality.

Evil happens when you are separated from the pain you inflict upon other people.

The host of my AirBnb was an elderly Black man. He grew up poor. His mother would clean white folks' houses all day to earn money and then clean his home to save money. His parents worked so hard to survive they had little time to raise their children. Two of his sisters became pregnant at age thirteen.

When I arrived at my host's home, he gave me a white towel with lots of visible stains. But it was clean. I immediately used it to wipe my face.

My AirBnb host is really into Black Power, but he never pushed his political beliefs on me. He wanted to know what it was like to be rich. I wanted to know what it was like to be poor. Were I to go to a fancy hotel, the system would make sure I never had to interact with a man like him.

Not that AirBnb didn't try. We communicated via SMS instead of AirBnb's website because AirBnb's website has an algorithm that scans our messages for keywords and punishes hosts it thinks did a poor job—regardless of the star rating a customer like me provides.

This man who was born before the Civil Rights Act of 1964 already lives in a dystopia run by an AI.
Progress Report 5: tying it together
by Nathan Helm-Burger — 2022-04-23T21:07:03.142Z

Previous: Progress report 4

First, an example of what not to do if you want humanity to survive: make an even foom-ier and less interpretable version of neural nets. On the spectrum of good idea to bad idea, this one is way worse than neuromorphic computing. In fact, they even had a paragraph in their paper discussing their method in contrast to Hebbian learning, and showing how their method is more volatile and unpredictable. Great. https://arxiv.org/abs/2202.05780

On the flip side of the coin, here's some good stuff that I'm happy to se being worked on.

Some other work that has been done with nostalgebraist's logit-lens: Looking for Grammar in all the Right Places Alethea Power

Tom Frederik working on a projected called 'Unseal' https://github.com/TomFrederik/unseal (to unseal the mystery of transformers), expanding on the PySvelte library to include aspects of the logit-lens idea.

Inspired by this, I started my own fork from his library to work on a interactive visualization that creates an open-ended framework for tying together multiple ways of interpreting models. I'm hoping to use it to put together the stuff I've found or made so far, as well as whatever I find or make next. It just seems generally useful to have some sort of way to put a bunch of different 'views' of the model together side-by-side in order to scan across all of them to get a more inclusive view of things.

Once I get this tool a bit more fleshed out, I plan to start trying to plan the audit game with it, using toy models that have been edited.

Here's a presentation of my ideas I made for the conclusion of the AGI safety fundamentals course.

https://youtu.be/FarRgPwBpGU

Among the things I'm thinking of putting in are ideas related to these papers.

One: A paper mentioned by jsteinhardt here https://www.lesswrong.com/posts/qAhT2qvKXboXqLk4e/early-2022-paper-round-up :

Summarizing Differences between Text Distributions with Natural Language

(w/ Ruiqi Zhong, Charlie Snell, Dan Klein)

This paper discusses their language data summarization technique, which I think will be cool to use someday, but also along the way to building that they had to do some clustering which I think sounds like a useful thing to include in my visualization and contrast with the neuron-importance topic clustering. I hope to also revisit and improve on the neuron-imp...
The New Right appears to be on the rise for better or worse
by Chris_Leong — 2022-04-23T19:36:58.661Z

We don't talk much about politics here because politics is the mind-killer. But even if it weren't the case, we might still want to avoid it as knowledge of day-to-day politics tends to lose its value fast.

On the other hand, I think it's important to have at least some minimalistic knowledge of larger trends. I don't think we should let our aversion to politics put us in a position where we don't know what is going on in the world until it smacks us in the face[1]. You might not care about politics, but politics cares about you. It is useful to know which way the wind is blowing, whether you wish to support or oppose growing trends; to hide; or some combination of all three.

The article in Vanity Fair convinced me that the New Right[2] is a phenomenon worth having at least having a cursory knowledge of [3]. While before I had assumed that it was an irrelevant niche on the internet, it now seems to be gaining real-world influence and a certain level of "coolness", so ignoring it seems like a mistake.

(I have tried to avoid stating an opinion due to the general norm of "no politics" on Less Wrong).

^

For example, I felt like I was smacked in the face by Trumpism. When he first started trending I had no idea of what was going on or why people were considering voting for him.

^

I'm not counting Trumpism as New Right. Considering Trump has already been president, this post would be coming several years too late.

^

It may seem as though I'm announcing old news since Yarvin commented on Overcoming Bias before Less Wrong split off. However, a) it was before my time (and before the time of most people here) and b) as far as I can tell New Right thought seems to have made some shifts since then.
[ASoT] Consequentialist models as a superset of mesaoptimizers
by leogao — 2022-04-23T17:57:40.130Z

TL;DR: I split out mesaoptimizers (models which do explicit search internally) from the superset of consequentialist models (which accomplish goals in the world, and may or may not use search internally). This resolves a bit of confusion I had about mesaoptimizers and whether things like GPT simulating an agent counted as mesaoptimization or not.

Editor’s note: I’m experimenting with having a lower quality threshold for just posting things even while I’m still confused and unconfident about my conclusions, but with this disclaimer at the top. Thanks to Vivek Hebbar, Ethan Perez, Owain Evans, and Evan Hubinger for discussions.

UPDATE: The idea in this post is basically the same as the idea of "mesa-controllers" in this post.

What do I mean by consequentialist models?

By consequentialist models I mean models which optimize the world in the

Alex Flint sense of optimization

; i.e narrowing world states to some goal in a way robust to some perturbations. In other words, it’s able to achieve consequences. The model doesn't have to be fully consequentialist either, it just has to have some

kernel of consequentialist structure

by virtue of actually achieving its goals sometimes. Of course, the degree to which a model is consequentialist is more of a scalar quantity than a discrete yes/no thing; for my purposes it really doesn't matter where to draw a dotted line that dictates when a model "becomes consequentialist".

(For what it's worth I would totally have called "consequentialist models" just mesaoptimizers and what other people call mesaoptimizers as like "searching mesaoptimizers" or something, but that would only create even more confusion than already exists)

For instance, the policy network of alphago alone (i.e none of the MCTS) is consequentialist, because it consistently steers the world into states where it's winning, despite my attempts to make it not win. Basically any RL policy is a fairly consequentialist model by this definition, since they channel the set of all states to the set of states that achieve high reward. I think of mesaoptimizers as a subset of consequentialist models, in that they are consequentialist, but they implement this consequentialism using explicit search rather than some other random thing. Explicit search is a kind of optimization, but not all optimization has to be search; you could have symbol manipulation, clever he...
Report likelihood ratios
by Ege Erdil — 2022-04-23T17:10:22.891Z

A few weeks ago, Thomas Philippon released a paper in which he argued that total factor productivity growth was piecewise linear rather than exponential. The paper received attention both on social media and on LessWrong itself.

I'll use this paper to illustrate a problem that I think is widespread, so while part of what I will say is a criticism of this specific paper the point I'm trying to make is more general.

What is TFP?

First, some background: total factor productivity (TFP) is often (wrongly, in my view) used as a proxy of technology in growth economics. Technically, it's the residual estimate of a log-linear regression of GDP on some quantifiable variables, such as the labor and capital stock of a country. The idea is to assume that there is a relationship Y=AL1−αKα between the GDP Y, labor stock L and capital stock K of countries for some α, estimate α from a linear regression

log(Y)=c+αlog(K)+(1−α)log(L)+ε

and then use the measured α to back out estimates for A across countries and across time. A is then the "total factor productivity" - it's the part of GDP that's unexplained by how many people are working and how much capital equipment they have at their disposal.

The GDP of an individual country can therefore increase for three reasons: it has more total hours worked, more capital equipment or more total factor productivity. Notice that this statement is vacuously true and is a tautological consequence of the definition of total factor productivity.

It's a robust fact that TFP as defined above increases in most countries over time. What this means is that not only do countries expand their labor force and acquire more capital thanks to investment, they also become "more efficient" at using those inputs in some sense. For instance, if our definition of TFP is relative to labor and capital stocks, it might fail to capture that workers are also becoming more skilled. Economists usually call this "human capital" and we can try to include it in a regression, but since TFP is defined to be all the stuff we left out of the regression, if we only work with L and K then workers becoming more skilled will show up in our data as an increase in A.

The question that's the subject of Philippon's paper is about the growth process of A. Economists usually model TFP as an exponentially growing process, say, log(At+1)=log(At)+μ+εt where μ is an average growth rate and...
Skilling-up in ML Engineering for Alignment: request for comments
by CallumMcDougall — 2022-04-23T15:11:00.397Z

TLDR: Please fill out

this survey

if you are a potential or established ML Engineer, to let us know your thoughts and preferences on the best way to gain engineering skills.

—

We at

Cambridge Effective Altruism CIC

are interested in the

demand for engineering skills in alignment

, and what we can do to help increase the supply. Two ways of increasing supply of engineers in alignment are:

Bringing in existing, experienced engineers interested in working on alignment.
Helping undergraduates and others interested in alignment to gain engineering skills, and contribute in the future.

We will address recruitment of existing engineers in another post. This post focuses on point 2, skilling-up. We want to discover what, if anything, field builders can do to accelerate the acquisition of engineering skills amongst those interested in the alignment problem. We hope this will prime the field with altruistically motivated, highly skilled people in months or years to come.

We are keen to capture a broad view on people’s thoughts about what would be most helpful for them in gaining better engineering skills, and to understand how much interest there would be for efforts that aid that.

To do that, we’ve created a

survey

with our key questions. We would love to hear from you if

any

of the following apply:

You are interested in ML engineering for alignment, and think you could get a lot of value through some form of structured education (e.g. bootcamps or online courses),
You have previous experience of some form of structured education related to coding or ML (either as a participant or as an instructor / organiser), and are able to share details of what worked / didn’t work well in this programme,
You have experience working in ML engineering, and can share details of how you skilled up in this field, or what in your opinion makes a great engineer,
You are an employer for a role related to ML engineering, and you would be happy to share views on what kinds of skills and/or qualifications you look for in potential hires,
You would be interested in assisting the development of any programmes that were founded as a result of this work in some way (e.g. as an organiser or mentor).

We look forward to hearing from you!