trentmkelly/LessWrong-43k huggingface dataset:
Opportunistic Time-Management
by Richard Henage — 2024-03-13T21:38:01.209Z

Be willing to break from your routine if you're in the mood to do the normally-less-savory items now. For example, I normally eat breakfast, work for half an hour, and then go on a run (I look forward to going on a run, not to working, but for me working is more important), and I later work some more. But if I'm done with my half hour of work and I'm in the middle of something engaging and wouldn't mind working some more, it's better to keep working and do the run later, breaking my routine. Similarly, if it's late at night and I could sleep, but I also wouldn't mind getting on my laptop and doing some work, I should do the work now, and push my bedtime and wake-up time back an hour or two. Otherwise, there's a good chance I'll wake up in the morning and dislike the idea of working.
I think this concept is part of a more general rule. There are important tasks and unimportant tasks. There are unsavory tasks and savory tasks. Whenever an important, unsavory task temporarily seems somewhat savory, take the opportunity to do the task instead of whatever you're doing now. Whenever an unimportant, savory task you would normally do temporarily seems somewhat unsavory, take the opportunity to not do the task, and instead do a more important task.
The procrastination vs deadline cycle seems to be a special case of this idea. When the deadline looms, the normally unsavory task becomes somewhat savory (or maybe the task is still unsavory, but your level of motivation rises to compensate: same thing). Many people learn to do their tasks at this time. If the downsides (stress, sloppier work) are minimal, this is actually a viable strategy.
As you pay attention to yourself and learn which tasks are important/unimportant and savory/unsavory, and which ones have a chance to flip their savoriness, you can start to develop a mental lookout list: e.g., watch for times when I need a break from working, and washing the dishes suddenly seems okay.
It's almost like a buy low, sell high mentality on the stock market. You can accumulate value merely by making transactions (doing tasks) at the right time.
AI governance and strategy: a list of research agendas and work that could be done.
by  — 2024-03-13T21:23:46.849Z

AI governance and strategy: a list of research agendas and work that could be done

This document was written by Nathan Barnard and Erin Robertson.

We have compiled a list of research agendas in AI governance, and we’ve written some possible questions that people could work on. Each section contains an explanation for why this theme might be relevant for existential risk and longtermist focussed governance, followed by a short description of past work. We propose some questions for each theme, but we prioritise clarity over completeness.

The content is focussed on questions which seem most important to Nathan personally, with a focus on questions which seem most useful on the margin. We have drawn often on other people’s lists, in an attempt to represent a more consensus view. Neither Erin nor Nathan have ever held AI governance research or implementation positions, and Nathan has been an independent researcher for less than a year.

A theme throughout these questions is that Nathan thinks it would be useful to try to get more high quality empirical work. A good example is this paper on policy persistence and policy windows from Freitas-Groff that has credible causal estimates of how persistent policy is, which he thinks is a really important question for prioritisation of different interventions in AI governance.

This document is divided into topics, each includes:

A brief discussion on theory of change: why might this work be useful?
Examples of past work in this domain: who’s working on this?
2-4 questions which people can answer with some description.

Topics:

AI regulation and other standard tools

Compute governance

Corporate governance

International governance

Misuse

Evals

China

Information security

Strategy and forecasting

Post TAI/ASI/AGI governance

AI regulation and other standard tools

Theory of change

Historically, government regulation has been successful at reducing accident rates in other potentially dangerous industries - for instance air travel, nuclear power plants, finance and pharmaceuticals. It’s plausible that similar regulatory action could reduce risks from powerful AI systems.

Past work

A dominant paradigm right now is applying the standard tools of technology regulation - and other non-regulatory means of reducing harm from novel and established technologies - to AI. This paradigm seems particularly important right now because of the ...
Highlights from Lex Fridman’s interview of Yann LeCun
by Joel Burget — 2024-03-13T20:58:13.052Z

Introduction

Yann LeCun is perhaps the most prominent critic of the “LessWrong view” on AI safety, the only one of the three "godfathers of AI" to not acknowledge the risks of advanced AI. So, when he recently appeared on the Lex Fridman podcast, I listened with the intent to better understand his position. LeCun came across as articulate / thoughtful[1]. Though I don't agree with it all, I found a lot worth sharing.

Most of this post consists of quotes from the transcript, where I’ve bolded the most salient points. There are also a few notes from me as well as a short summary at the end.

Limitations of Autoregressive LLMs

Lex Fridman (00:01:52) You've said that autoregressive LLMs are not the way we’re going to make progress towards superhuman intelligence. These are the large language models like GPT-4, like Llama 2 and 3 soon and so on. How do they work and why are they not going to take us all the way?

Yann LeCun (00:02:47) For a number of reasons. The first is that there [are] a number of characteristics of intelligent behavior. For example, the capacity to understand the world, understand the physical world, the ability to remember and retrieve things, persistent memory, the ability to reason, and the ability to plan. Those are four essential characteristics of intelligent systems or entities, humans, animals. LLMs can do none of those or they can only do them in a very primitive way and they don’t really understand the physical world. They don’t really have persistent memory. They can’t really reason and they certainly can’t plan. And so if you expect the system to become intelligent without having the possibility of doing those things, you’re making a mistake. That is not to say that autoregressive LLMs are not useful. They’re certainly useful. That they’re not interesting, that we can’t build a whole ecosystem of applications around them… of course we can. But as a pass towards human-level intelligence, they’re missing essential components.

(00:04:08) And then there is another tidbit or fact that I think is very interesting. Those LLMs are trained on enormous amounts of texts, basically, the entirety of all publicly available texts on the internet, right? That’s typically on the order of 10^13 tokens. Each token is typically two bytes, so that’s 2*10^13 bytes as training data. It would take you or me 170,000 years to just read through this at ei

...
On the Latest TikTok Bill
by Zvi — 2024-03-13T18:50:05.398Z

TikTok Might Get Banned Soon

This attempt is getting reasonably far rather quickly, passing the House with broad support.

Alec Stapp: TikTok bill to remove influence of CCP:

– passed unanimously out of committee

– GOP leadership says they’ll bring it to the floor for a vote next week

– Biden says he’ll sign the bill if passed

Can’t believe it’s taken this long, but should be done soon.

It’s been obvious for years that we shouldn’t let China control a black-box algorithm that influences >100 million American users.

JSM: Can this stand up to court scrutiny though?

Alec Stapp: Yes.

It then passed the house 352-65, despite opposition from Donald Trump.

Manifold is as of now around 72% that a bill will pass, similar to Metaculus. Consensus is that it is unlikely that ByteDance will divest. They will fight in court, and if they lose they likely are not bluffing about letting TikTok be shut down or banned in America, Metaculus only has a 12% chance they will sell this year.

The bill now goes on to the Senate. I see about a 43% chance it passes there within the month, and a 71% chance it will happen this year. Those numbers seem reasonable to me.

The main purpose of this post is to go over arguments for and against the bill, and also what the bill actually would and would not do.

I have long been in favor on principle of banning or forcing divestiture of TikTok. Then I saw the Restrict Act, and that was clearly a no-good, very-bad bill.

My view of the current bill, after a close reading, is that it is vastly better, and about as good as we could reasonably expect. It seems positive and I hope it passes, whether or not ByteDance folds and agrees to divest. I expect it to pass constitutional muster, although one cannot be sure.

To make them easy to find:

Here is Noah Smith’s case for banning TikTok.

Here is Matthew Yglesias’s case for banning TikTok.

This is a profile of Make Gallagher, who is leading the charge to pass the bill.

I go over various arguments for and against the bill, and for and against forcing divestiture of or banning TikTok in general, as well, as well as other related developments. The good argument against the bill is the libertarian concern about expansion of government powers, and what else the government might do. I do not believe it should carry the day on this bill, but I definitely get why one might think so.

Execution

...
Recommended book for a balanced take and lessons learned from covid pandemic response
by Martin Hare Robertson — 2024-03-13T18:14:27.371Z

I was an avid reader of TheZvi during the pandemic and really appreciated the in depth analysis of studies and the impacts of covid policies. It seems clear in hindsight that there is much to legitimately criticise about overall pandemic policies and some of the particular details which arguably became far too sticky and in some cases made very little sense.

This whole area has become incredibly politically polarised with people avidly defending Covid lockdowns and masks, and other folks avidly criticising them.

Has anyone yet published a reasonably balanced summary of some of the things which went well, and about policies which didn’t actually make sense, either at the time, or in hindsight?

I’m guessing this while topic will be argued for another 10 years, but I’m wondering what some of the best analysis in 2024 is?
Laying the Foundations for Vision and Multimodal Mechanistic Interpretability & Open Problems
by Sonia Joseph — 2024-03-13T17:09:17.027Z

Behold the dogit lens. Patch-level logit attribution is an emergent segmentation map.

Join our Discord

here

.

This article was written by Sonia Joseph, in collaboration with Neel Nanda, and incubated in Blake Richards’s lab at Mila and in the MATS community. Thank you to the Prisma core contributors, including Praneet Suresh, Rob Graham, and Yash Vadi.

Full acknowledgements of contributors are at the end. I am grateful to my collaborators for their guidance and feedback.

Outline

Part One: Introduction and Motivation
Part Two: Tutorial Notebooks
Part Three: Brief ViT Overview
Part Four: Demo of Prisma’s Functionality

Key features, including logit attribution, attention head visualization, and activation patching.
Preliminary research results obtained using Prisma, including emergent segmentation maps and canonical attention heads.

Part Five: FAQ, including Key Differences between Vision and Language Mechanistic Interpretability
Part Six: Getting Started with Vision Mechanistic Interpretability
Part Seven: How to Get Involved
Part Eight: Open Problems in Vision Mechanistic Interpretability

Introducing the Prisma Library for Multimodal Mechanistic Interpretability

I am excited to share with the mechanistic interpretability and alignment communities a project I’ve been working on for the last few months. Prisma is a multimodal mechanistic interpretability library based on

TransformerLens

, currently supporting vanilla vision transformers (ViTs) and their vision-text counterparts CLIP.

With recent rapid releases of multimodal models, including Sora, Gemini, and Claude 3, it is crucial that interpretability and safety efforts remain in tandem. While language mechanistic interpretability already has strong conceptual foundations, many research papers, and a thriving community, research in non-language modalities lags behind. Given that multimodal capabilities will be part of AGI, field-building in mechanistic interpretability for non-language modalities is crucial for safety and alignment.

The goal of Prisma is to make research in mechanistic interpretability for multimodal models both easy and fun. We are also building a strong and collaborative open source research community around Prisma.

You can join our Discord here.

This post includes a brief overview of the library, fleshes out some concrete problems, and gives steps for people to get started.

Prisma Goa

...
I was raised by devout Mormons, AMA [&|] Soliciting Advice
by ErioirE — 2024-03-13T16:52:19.130Z

I was raised by devout Mormons in Mormon central (Northern Utah). It’s hard to accurately capture the scope of the conditioning via writing.
Standard tenets of Mormon doctrine include:

No tea/coffee/alcohol
No premarital sex
Keep the sabbath (Sunday) holy
Mandatory 10% tithe
Book of Mormon as “most correct of any book”
Follow (i.e. obey) Church leaders
The Church is the only one with the Truth, all others are wrong
Read scriptures every day, pray several times a day (guilt enforced)
You should gain a Spiritual Witness of the Truth by praying about it

In my case my parents are True Believers and devout even in comparison to the other Mormon families around. There’s no official list of “Things okay/not okay to do on Sunday”, and so different families have different opinions. For my family, that meant “No shopping/work/activities-of-any-kind-not- directly-related-to-worship” I wasn’t even allowed to do homework on Sundays, nor read non-religious books, etc.
Strangely enough, they did emphasize the importance of education and both my parents were college graduates. You would think that would increase the risk of learning dangerous critical thinking skills, but when you’ve brainwashed the victims to assume a priori that emotional/spiritual experiences >= empirical evidence they do all the compartmentalization themselves.
Nobody ever *said* the term “separate magisteria”, but that’s what it was held to be.

I had just enough critical thinking as a child that some things always bothered me, but I didn’t have the necessary knowledge to pin down why that was, and everyone around me regularly emphasized the Truth of our beliefs.
The most egregious of these 'bothers' being "Pray until you get a spiritual witness", which in retrospect was flagrant disregard of Conservation of Expected Evidence. (No answer? You haven't prayed enough. And even if it doesn't happen for years, it just 'wasn't the right time'. No amount of failed experiments will ever be allowed to be considered counter evidence. Very convenient.)
I wish I could say that once I became a teenager I shook off the conditioning and declared myself an atheist, but that’s not how it happened. I *wanted* it to be true, in part because I love my parents and trusted them more than was healthy. Compartmentalization is a helluva drug ala Robert Aumann.

It was only after moving out and spending a few months living on...
Relational Agency: Consistently Reaching Out

by Jonathan Moregård — 2024-03-13T14:34:12.780Z
Link: https://open.substack.com/pub/honestliving/p/unlocking-relational-agency?r=k6ix3&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true

When I moved to Gothenburg, I found myself barely knowing anyone. Being a social person, this was a rough state of affairs. I started going to events. It took some time for me to meet a person I liked hanging out with, but eventually, I met a person I could dive into deep conversations together with. After having spent most of the event breaks talking, I looked at him and said: “I like talking to you. I want to do it more. How do I get in touch with you?”

I then started reaching out regularly, turning a random encounter into a long-term friendship. This post will cover a simple technique I used to reach out consistently, along with the hurdles that stand in the way for many. I have more tricks up my sleeve, and may write follow-up posts covering them.

The Murder of Relational Agency

Many people struggle socially after leaving school. They spend their youth forming convenient friendships, connecting with people they happen to spend time with. Once they leave school, friendships stop happening automatically.

School kills relational agency, with many people reaching adulthood lacking basic relational skills — cultivating friendships outside the confines of an institution.

Meanwhile, friendships and real social connections are becoming increasingly valuable — the most important things in life cannot be bought. This is not a platitude about the power of love — relationships get you

access to

things like social contexts, jobs/collaborations and other powerful network goodies.

Relational agency is a proverbial thousand-dollar bill lying on the sidewalk.

Let me tell you how to pick it up.

Automated Outreach Reminders

A key aspect of building and maintaining relationships is to connect regularly. Automatic school-relationships form around shared classes, with people you happen to be paired up with. As an adult, you can’t rely on a system to meet others — you need to do it yourself. Your capacity to reach out to others will shape your relationships to a large degree — you’d better put energy into doing this well.

Many people leave this up to chance, reaching out once in a while when they happen to remember. Their relationships tend to fizzle out over time, with life getting in the way. After a long period of neglect, these people are left alone, unsure where all their friendships went.

This is a very preventable outcome. Reaching out consistently isn’t rocket science...
What could a policy banning AGI look like?
by TsviBT — 2024-03-13T14:19:07.783Z

[Caveat lector: I know roughly nothing about policy!]

Suppose that there were political support to really halt research that might lead to an unstoppable, unsteerable transfer of control over the lightcone from humans to AGIs. What government policy could exert that political value?

[That does sound relaxing.]

Banning AGI research specifically

This question is NOT ASKING ABOUT GENERALLY SLOWING DOWN AI-RELATED ACTIVITY. The question is specifically about what it could look like to ban (or rather, impose an indefinite moratorium on) research that is aimed at creating artifacts that are more capable in general than humanity.

So "restrict chip exports to China" or "require large vector processing clusters to submit to inspections" or "require evals for commercialized systems" don't answer the question.

The question is NOT LIMITED to policies that would be actually practically enforceable by their letter. Making AGI research illegal would slow it down, even if the ban is physically evadable; researchers generally want to think publishable thoughts, and generally want to plausibly be doing something good or neutral by their society's judgement. If the FBI felt they had a mandate to investigate AGI attempts, even if they would have to figure out some only-sorta-related crime to actually charge, maybe that would also chill AGI research. The question is about making the societal value of "let's not build this for now" be exerted in the most forceful and explicit form that's feasible.

Some sorts of things that would more address the question (in the following, replace "AGI" with "computer programs that learn, perform tasks, or answer questions in full generality", or something else that could go in a government policy):

Make it illegal to write AGIs.

Make it illegal to pay someone if the job description explicitly talks about making AGIs.

Make it illegal to conspire to write AGIs.

Why ask this?

I've asked this question of several (5-10) people, some of whom know something about policy and have thought about policies that would decrease AGI X-risk. All of them said they had not thought about this question. I think they mostly viewed it as not a very salient question because there isn't political support for such a ban. Maybe the possibility has been analyzed somewhere that I haven't seen; links?

But I'm still curious because:

I just am. Curious, I mean.

Maybe there will

...
Clickbait Soapboxing
by DaystarEld — 2024-03-13T14:09:29.890Z
Link: https://daystareld.com/clickbait-soapboxing/

Someone on Twitter said:

I am guilty of deliberately stating things in a bold & provocative form on here in order to stimulate discussion. Leaving hedges & caveats for the comments section. On net, I think this is better than alternatives, but I’m open to being convinced otherwise.

And I finally felt the urge to write up thoughts I’ve had about what I’ll call “clickbait soapboxing” for the past year or so. A disclaimer is that I feel like I could write a whole book on this sort of thing, and will inevitably have more complex thoughts about what I say here that comes off as simple.

Also, I’m not super confident I am right to feel so strongly about how bad it seems, and also also, I personally like many people (like the above poster) who regularly do this.

But I don’t feel at all confident that people doing it are tracking all the effects it has, and they certainly don’t seem to acknowledge it. So this seems maybe like it’s useful to say explicitly.

First off, some of these are clearly a “me” thing. For example, I have trouble trusting people to be as capable of “actual” vulnerability or sincerity when they don’t put effort into representing their thoughts accurately. It feels, at best, like a shield against criticism: “I was wrong on purpose!”

But I know others struggle with inhibition/social anxiety: “I’d rather speak boldly, knowing I’m wrong in some way, than not speak at all!” Which, yeah, makes sense! But are you planning to ever address the root cause? Is it healing, or cope/crutch? (Not judging, I really don’t know!)

In any case, there are still externalities. Illusion of transparency is real! Typical mind fallacy is real!

Should you care? shrug What makes us care about anything we say in the first place? Just don’t motte-bailey “communicating for self-expression” or "processing out loud" vs “sharing ideas and learning" or "talking about True Things."

As for me (and maybe others out there like me), the effects include things like thinking:

“Did this person actually change their mind? Do they actually believe the more nuanced thing? Or are they just backpedaling due to getting stronger pushback than expected?”

As well as:

“Are they actually interested in learning and sharing interesting ideas? Or are they optimizing for being interesting and getting followers?”

And

“If they misinform someone, would they care? Would they do it on purpose, if it got them ...