trentmkelly/LessWrong-43k huggingface dataset:
What Piles Up Must Pile Down
by silentbob — 2023-04-09T18:37:12.119Z

…except when it doesn’t.

Tl;dr: some problems in life follow a similar pattern of “stuff piles up and we get frustrated about it”. This post looks at some such problems, and some typical solutions.

A while ago I noticed that a lot of the persistent frustrations in my life were all due to the same type of effect: some thing keeps accumulating and I don’t manage to keep it in check. There are many such things in life, and feeling overwhelmed by these ever-growing piles can affect one’s well being quite significantly. Here’s a non-exhaustive list of things of that type:

Disorder in one’s home tends to grow by default, and needs to be kept in check by some deliberate action or habit
Todo lists tend to grow longer and longer, unless we remove more items from our backlog than we add
The same applies to Reading lists, e.g. when using

Pocket

Browser tabs tend to multiply, unless we manage to close more than we open, or frequently just dismiss all open tabs
Email inboxes fill themselves up by default, and it takes effort to avoid that
“Unread message” markers in platforms like Slack or Discord get more and more unless you put in the time to read/dismiss them
Starting projects and commitments is much easier than completing them, particularly if you’re exposed to many interesting opportunities

Seeing all these piles of things isn’t great, particularly if your realistic expectation is for these piles to grow larger rather than smaller. Personally, I’d like my email inbox to be close to empty with everything neatly sorted away, my browser tabs to not require excessive horizontal scrolling, my todo list to always seem manageable, my flat to be always in order, and my commitments to be few enough for me to actually get them done while still having some room to breathe. But that’s not how things tend to be. I’m constantly reminded by these various “piles” that I’m not on top of things and that all these areas of my life tend, by default, to go in a direction that reduces my satisfaction and productivity.

That’s not a great situation to be in. So I thought a while ago: let’s collect all the typical ways in which problems of this type can be solved. And maybe I’ll notice some decent solutions I can apply to such a degree that I manage to actually “downpile”. With all that, below are some of the antidotes to uncontrollable seeming pile...
Review & rebuttal of "Why machines will never rule the world: artificial intelligence without fear"
by mikbp — 2023-04-09T15:06:50.595Z

Hi, has anyone in the community (or adjacents) produced a good review and/or a rebuttal of the book "Why machines will never rule the world: artificial intelligence without fear" by Landgrebe & Smith?

I haven't been able to find any, the closest is thing I've found is this. The interview is good, but it doesn't go to questions that for me seem obvious (I'm just an observer, not an specialist) but that I think people here would have asked.
Being at peace with Doom
by Johannes C. Mayer — 2023-04-09T14:53:22.924Z

You can be at peace even when thinking the world is doomed. And while at peace you can still work against that Doom, even while being aware that nothing you do will make a difference. I believe there are states of mind like this that can be inhabited by humans.

Here I am not going to argue for imminent doom, or that nothing that you do matters. Rather, I want to point out that even when you believe in the dire circumstance of imminent unpreventable doom, it is possible to be at peace, even while working hard against the doom. Even while believing this to be futile. This is a possible state of mind for a human being.

And if it is possible, to be at peace, and work hard, even in this dire circumstance, it should be possible in any less dire circumstance too.

There are many games about how long can you survive, e.g. Dawn of war 2 the last stand, Serious Sam survival mode, and Project Zomboid. The very nature of these games is that you will soon die. And there is no saving. The difficulty will increase more and more until at some point you will get crushed.

But there are loads of people playing these games. Nothing about the impossibility of achieving victory seems to detract from the fun you can have. Would this really change if these games couldn't be restarted?

There is also the game You only Live once that you can only play once.

Do people not play these games? Do people not try hard when playing these games? No. To be fair, there is a big difference between AI doom and these games. In these games you can make visible progress. The natural way to define success is to ask the questions: How long did you survive? Did you survive longer than last time?

This is where death with dignity and Duncan's advice is coming from, as far as I can tell. It's about redefining success as making as much progress as possible towards getting a good outcome, instead of directly aiming for a good outcome. Aiming to survive forever in Dawn of War 2 the last stand would probably be frustrating. You set out for a goal that you know is unachievable after all.

I think these strategies are valuable, though to me it seems they also miss something very basic.

Maybe this is a fluke and I will feel different soon, but today I felt like my expectation of doom did not influence me negatively. No negative qualia arose, generated by a heuristic in my brain, that "wants" to steer me away from executin...
Expanding the domain of discourse reveals structure already there but hidden
by TsviBT — 2023-04-09T13:36:28.566Z

[Metadata: crossposted from https://tsvibt.blogspot.com/2022/05/expanding-domain-of-discourse-reveals.html. First completed May 15, 2022. This essay is more like research notes than exposition, so context may be missing, the use of terms may change across essays, and the text might be revised later; only the versions at tsvibt.blogspot.com are definitely up to date.]

To understand a complex system like a mind or an ecosystem, we have to understand a tangled web of objects, features, processes, relations, correlations, clusters, constraints, causes, and so on. It helps to find underlying explanations and generating process, and to find deep reasons that explain why many relationships are the way they are. To find those underlying explanations, it helps to know relationships between relationships: which relationships cause, explain, or constrain other relationships. When staring at an assembly of relationships and asking which relationships explain other relationships, it can seem like we're at a loss for where to go; sometimes there are clearly relationships, but there's no clear way to extract any further order, because there's no basis on which to say that one relationship explains another--the relationships are just there, and that's it.

By way of example/analogy, suppose we look at a list of facts like [3×4=12, 12/4=3, 4=12/3, 12=4×3], and ask, which facts are more foundational to which other facts? Which facts explain, ground, cause, or constitute the essence of, which other facts? Is 12/4=3 because 12=4×3, or vice versa? Is the evenness of 12 due to the evenness of 4? Or due to the evenness of 6? What if there's no answer, and all we can say is that all these facts stand together as related by derivations and proofs, with no priority given to any of them over any other?

Another example: take a smooth closed curve C in the plane. These conditions are equivalent:

There is a point p such that all points on C are equidistant from p.

C has constant curvature.

(If C bounds a disc) C bounds the maximum area that's possible to bound with a curve of the same length as C.

The group of isometries of the plane that maps C to itself is a nontrivial connected compact topological group.

These conditions all define a circle in the plane. If we're asked "Why does a locus of equidistant points have constant curvature?", we can answer with a proof. But if we're asked, "Is ...
Rooms Available in Downtown Berkeley Group House
by Arjun Panickssery — 2023-04-09T10:15:32.776Z

Looking for residents for a 6-bedroom group house.
10-minute walk from campus and from the Downtown Berkeley BART Station
Rooms are each well-lit with a queen bed and 70-inch wide adjustable height desk.
Housemates include EAs working at OpenAI, CHAI, etc., and for the next few months some summer interns passing through.

Short interest form: https://forms.gle/YZ8amWCcctQiTsEbA

More information: https://docs.google.com/document/d/1-5HkN7Z5a9n_9eSOuS5u_slXfIx9eKMVnB2jRoSw52U/edit

We can also accommodate short-term visitors starting June 1.
AGI Safety Fundamentals 2023 Notes 
by Lisa Wang — 2023-04-09T07:28:34.761Z
Link: https://lisacontemplates.blogspot.com/2023/04/agisf-notes.html

Hello!

As I am completing my capstone project for AGI Safety Fundamentals 2023, I wanted to link-post my notes for the core readings for the course. I have found it extremely rewarding to summarize key points and point out my topics of confusion in this manner.

I would like to improve on conveying complex topics more concisely, and I would like to make sure that I got most of the points addressed by the authors correctly. Feedback is appreciated!

I noticed that there has been a post dedicated to the 2022 iteration of the AGISF AI Alignment course here: [https://www.lesswrong.com/posts/ZLNgg49AxLtHDj5v5/agisf-2022-summaries], and it is very well-organized.

I hope this is useful to someone out there also, and thanks BlueDot Impact for offering this course!
Ng and LeCun on the 6-Month Pause (Transcript)
by Stephen Fowler — 2023-04-09T06:14:18.757Z

This is a transcript a brief chat between Andrew Ng and Yann LeCun on the prospect of the 6-Month Pause. They are both highly skeptical of the pause and more generally of "AI doomerism". Yann also briefly touches on why he thinks LLMs won't scale to AGI.
Full video can be found here. Transcript was made using Whisper. Extensive editing was then done for readability.

Ng:
I'm Andrew Ng, founder of DeepLearning.ai and I'm delighted to have you join me here today and with me also is Yann LeCun who is VP and Chief AI Scientist of Meta, a professor at NYU and Turing Award winner.

So Yann and I have been friends for a long time and both he and I have thought at length about this six-month moratorium proposal and felt it was an important enough topic that Yann and I felt like we want to chat about it with you here today.

LeCun:
Hi Andrew, it's a pleasure to be here.

Ng:
So just to summarize the situation, I think over the last decade or maybe even over the last longer 20-30 years we've seen very exciting progress in AI. Deep learning is working well and then even in the last one or two years feels like maybe there's even a further acceleration of progress in AI with generative AI systems such as ChatGPT, LLAMA, also image generation, mid-journey, stable diffusion, DALL-E, feels like maybe AI has gotten even faster and associated with that there are people that worry about AI, you know, fairness, bias, social economic displacement. There are also the further out speculative worries about AGI, evil sentient killer robots, but I think that there are real worries about harms, possible real harms today and possibly other harms in the future that people worry about. So in this environment, the Future Life Institute put out a call for a six-month pause, or moratorium, on training AI models that are even more powerful than OpenAI's GPT-4 model and a number of people, very smart people, including our friend Yoshua Bengio, Stuart Russell, Elon Musk, signed on to this proposal. I think Jan and I are both concerned about this proposal, but Yann, why don't you start? Do you want to share your take on this proposal?

LeCun:
Well, my first reaction to this is that calling for a delay in research and development smacks me of a new wave of obscurantism, essentially, like why slow down the progress of knowledge and science? Then there is the question of products, l...
Agentized LLMs will change the alignment landscape
by Seth Herd — 2023-04-09T02:29:07.797Z

Epistemic status: head spinning, suddenly unsure of everything in alignment. And unsure of these predictions.

I'm following the suggestions in 10 reasons why lists of 10 reasons might be a winning strategy in order to get this out quickly (reason 10 will blow your mind!). I'm hoping to prompt some discussion, rather than try to do the definitive writeup on this topic when this technique was introduced so recently.

Ten reasons why agentized LLMs will change the alignment landscape:

Agentized[1] LLMs like Auto-GPT and Baby AGI may fan the sparks of AGI in GPT-4 into a fire. These techniques use an LLM as a central cognitive engine, within a recursive loop of breaking a task goal into subtasks, working on those subtasks (including calling other software), and using the LLM to prioritize subtasks and decide when they're adequately well done. They recursively check whether they're making progress on their top-level goal.
While it remains to be seen what these systems can actually accomplish, I think it's very likely that they will dramatically enhance the effective intelligence of the core LLM. I think this type of recursivity and breaking problems into separate cognitive tasks is central to human intelligence. This technique adds several key aspects of human cognition; executive function; reflective, recursive thought; and episodic memory for tasks, despite using non-brainlike implementations. To be fair, the existing implementations seem pretty limited and error-prone. But they were implemented in days. So this is a prediction of near-future progress, not a report on amazing new capabilities.
This approach appears to be easier than I'd thought. I've been expecting this type of self-prompting to imitate the advantages of human thought, but I didn't expect the cognitive capacities of GPT-4 to make it so easy to do useful multi-step thinking and planning. The ease of initial implementation (something like 3 days, with all of the code also written by GPT-4 for baby AGI) implies that improvements may also be easier than we would have guessed.
Integration with HuggingGPT and similar approaches can provide these cognitive loops with more cognitive capacities. This integration was also easier than I'd have guessed, with GPT-4 learning from a handful (e.g., 40) of examples how to use other software tools. Those tools will include both sensory capacities, with vision

...
A decade of lurking, a month of posting
by Max H — 2023-04-09T00:21:23.321Z

This post is a look back on my first month or so as an active contributor on LessWrong, after lurking for over a decade. My experience so far has been overwhelmingly positive, and one purpose of this post is to encourage other lurkers to do the same.

The reason I decided to start posting, in a nutshell:

xkcd 386: "Duty Calls"

For the last 10 years or so, I've been following Eliezer's public writing and nodding along in silent agreement with just about everything he says.

I mostly didn't feel like I had much to contribute to the discussion, at least not enough to overcome the activation energy required to post, which for me seems to be pretty high.

However, over the last few years and especially the last few months, I've grown increasingly alarmed and disappointed by the number of highly-upvoted and well-received posts on AI, alignment, and the nature of intelligent systems, which seem fundamentally confused about certain things. I think (what I perceive as) these misunderstandings and confusion are especially prominent in posts which reject all or part of the Yudkowskian view of intelligence and alignment.

I notice Eliezer's own views seem to be on the outs with some fraction of prominent posters these days. One hypothesis for this is that Eliezer is actually wrong about a lot of things, and that people are right to treat his ideas with skepticism.

Reading posts and comments from both Eliezer and his skeptics though, I find this hypothesis unconvincing. Eliezer may sometimes be wrong about important things, but his critics don't seem to be making a very strong case.
(I realize the paragraphs above are potentially controversial. My intent is not to be inflammatory or to attack anyone. My goal in this post is simply to be direct about my own beliefs, without getting too much into the weeds about why I hold them.)

My first few posts and comments have been an attempt to articulate my own understanding of some concepts in AI and alignment which I perceive as widely misunderstood. My goal is to build a foundation from which to poke and prod at some of the Eliezer-skeptical ideas, to see if I have a knack for explaining where others have failed. Or, alternatively, to see if I am the one missing something fundamental, which becomes apparent through more active engagement.

Overview of my recent posts

This section is an overview of my posts so far, ranked by which ones I think a...
"warning about ai doom" is also "announcing capabilities progress to noobs"
by the gears to ascension — 2023-04-08T23:42:43.602Z

(Some recommended writing style rules slightly broken because I'm lazy and have followed them a lot before. Shrug. Contribute edits if you care, I guess.)

A lot of people who pay some attention to capabilities don't know about some capabilities insights. Loudly announcing "hey, we're close to ai doom!" is also a way of informing those who weren't paying much attention what capabilities approaches have made progress. Obviously the highest capability people already knew, but those who didn't and who nevertheless think ai doom is unlikely and just want to participate in (useless-to-their values-because-they-can't-control-the-ai, -but-they-don't-believe-that) powerseeking go "oh wow, thank you for warning me about these ai capabilities! I'll make sure to avoid them!" and rub their hands together in glee.

These people are already here on lesswrong. There are a number of them. Some of them are somewhat alignmentpilled and want to figure out how to align the systems they build, because they've realized that like, actually - powerseeking where your intentions are suddenly eclipsed by the fact that your carefully bred power-obtainer AI generates powerseeking complexity-free-squiggle wanters who kill the powerseeking complex-and-beautiful-art wanters from your system before it can take over the world, yeah that's useless, even the antihumanists want to make AIs that will create AIs aligned enough to create human torture machines, but they don't want to make useless microscopic-squiggle seekers and some of them realize that and chill out a little about the urgency of making antihumanism-wanters.

But many of them don't get the threat from squigglewanters, and think that they're sticking it to the human-wanters by making human-hater AIs (or whatever), completely failing to realize that even making human-haters stay stable is catastrophically hard once the systems are strongly superintelligent. And the ones who are like "nah there's no risk from overpowered squigglewanters, squigglewanters are inherently weak lol" and stop thinking so that they don't risk proving themselves wrong abstractly before they discover themselves to be wrong in the territory - those people abound, and if you announce things on twitter and say "hey look at these ai capabilities, aren't they scary?":

...then you're just getting a bunch of attention on some names. Sure, those are names that aren't that ...
Feature Request: Right Click to Copy LaTeX
by DragonGod — 2023-04-08T23:27:30.151Z

Something I do frequently is run into a block of LaTeX that I'm not sure I fully grasp.

At such a point, what I'd like to do is just copy the equation and paste it to GPT-4 and have it explain it to me step by step, then iterate on the feedback until I'm sure I understood it.

On some other sites, right clicking brings up a pane that allows me to view the LaTeX.

E.g. from @beren's blog:

Then I copy the equations, paste it in ChatGPT and ask GPT-4 to explain it in detail:

This workflow is pretty inconvenient for LessWrong as I have to manually type the LaTeX adding extra friction (and sometimes that is extra difficult as some equations are especially unfamiliar and I'm not sure how to reproduce them). This is such a nightmare on mobile (where I do most of my LW reading from), that I mostly don't bother and often just give up.

If a feature were added to provide this functionality natively on the site, it would be a significant improvement to my user experience.

Cc: @habryka, @Ruby.
ELCK might require nontrivial scalable alignment progress, and seems tractable enough to try
by Alex Lawsen  — 2023-04-08T21:49:32.925Z

Written quickly rather than not at all, I was thinking about this a couple of days ago, and decided to commit to writing something by today rather than adding this idea to my list of a million things to write up.

This post describes a toy alignment problem that I’d like to see work on. Even the easiest version of the problem seems nontrivial, but it seems significantly more tractable than harder versions, easy to "verify", and also fairly easy to describe. I don't think the idea is original, as it's kind of obvious, but I haven't yet seen a good attempt. I’ve called it, not entirely seriously ‘ELCK’, which stands for ‘Eliciting Latent Chess Knowledge’. Some of the extensions I mention later can be called ELGK/ELGCK

In brief, the challenge is:

Create an AI 'chess analyst', which can explain the reasons behind moves by top engines and Super-GMs, as well as the best human analysts.

As-stated this would already be a significant achievement. Although it wouldn’t necessarily require progress on scalable oversight, given how much top-quality analysis exists on the internet already, even the problem of “getting something to automatically explain human-understandable thematic ideas” is something of a significant step towards (limited domain) ontology identification.

The bulk of my excitement about the project comes from the fact that there are modifications which would require nontrivial progress on some key parts of low-stakes alignment, while also being significantly more tractable than solving the entirety of ELK, or e.g. coming up with a working implementation of HCH.

Here are some ideas for different 'difficulty levels':

Create an AI model that can provide analysis of chess games that are rated as highly by top human players as the best human analysis, without training on any human analysis produced by players with a rating above N (but still with access to a chess engine). Reduce N.

Do this with access to a chess engine but limiting the number of positions that can be explored per move of the actual game to M. Reduce M.

Do any of the above with a training process that can be repeated for a different board game (e.g. go, Othello, anti-chess) without any nontrivial adjustments to the process and produce a world-class analyst for that game.

Even as soon as we get to level 1, the difficulty of explaining why certain moves were played is analogous to the difficulty ...