trentmkelly/LessWrong-43k huggingface dataset:
AI improving AI [MLAISU W01!]
by Esben Kran — 2023-01-06T11:13:17.321Z
Link: https://newsletter.apartresearch.com/posts/hundreds-of-research-ideas-w01

Over 200 research ideas for mechanistic interpretability, ML improving ML and the dangers of aligned artificial intelligence. Welcome to 2023 and a happy New Year from us at the ML & AI Safety Updates!

Watch this week's MLAISU on

YouTube

or listen to it on

Spotify

.

Mechanistic interpretability

The interpretability researcher Neel Nanda

has published

a massive list of 200 open and concrete problems in mechanistic interpretability. They’re split into the following categories:

Analyzing toy models

: Diving into models that are much smaller but trained the same way as large models. These are way easier to analyze than large models and he has made 12 small models available.

Looking for circuits in the wild

: Inspired by the paper

“interpretability in the Wild”

, can we use mechanistic interpretability on real-life language models?

Interpreting algorithmic problems

: Algorithms are highly interpretable and learned as a clearly interpretable structure. We can for example observe that

grokking happens

when an algorithm is generalized within the network.

Exploring polysemanticity and superposition

: Superposition is when one feature is spread across multiple neurons in a network and gives problems in our interpretation of what neurons represent. Can we find better ways to understand or mitigate this effect?

Analyzing training dynamics

: Understanding how models change over training is very interesting for identifying how and when capabilities emerge.

These are great projects to go for and we’re collaborating with Neel Nanda to run a

mechanistic interpretability hackathon

the 20th of January! As Lawrence Chan mentions in

a new post

; we need to touch reality as soon as possible, and these hackathons are a great way to get fast and concrete research results. You can join us but you can also

run a local hackathon site

!

ML improving ML

Thomas Woodside summarizes

a collaborative project to map cases where ML systems are self-improving. There are already 11 different major research projects that have shown machine learning systems used to improve other systems and we assume that there is much more happening behind the scenes since these are only published papers.

Several of the projects use models to create data that another model is fine-tuned on while a few relate to speed-ups in running and developing machine learning syst...
AI Safety Camp, Virtual Edition 2023
by Linda Linsefors — 2023-01-06T11:09:07.302Z
Link: http://aisafety.camp/

AI Safety Camp connects you with an experienced research lead to collaborate on a research project – helping you try your fit for a potential career in AI Safety research.

The applications for AI Safety Camp’s Virtual Edition in 2023 are now open!

AI Safety Camp Virtual 8 will be a 3.5-month long online research program from 4 March to 18 June 2023, where participants form teams to work on pre-selected projects.

We value people with diverse backgrounds and skillsets, such as cybersecurity or cognitive science. Not all projects require participants to have prior experience in AI Safety, mathematics or machine learning. You will be able to read in detail about the research topics & each project’s skill requirements for our upcoming edition by following the Project Proposal links below.

Projects

Projects you can apply to…

Conceptualise AGI dynamics

Uncertainty -> Soft Optimization with Jeremy Gillen

Inquire into Uncontrollable Dynamics of AGI with Remmelt Ellen

Discussing and Crystallising a Research Agenda Based on Positive Attractors and Inherently Interpretable Architectures with Robert Kralisch

Investigate transformer models

Cyborgism with Nicholas Kees Dupuis

Understanding Search in Transformers with Michael Ivanitskiy

Interdisciplinary Investigation of DebateGPT with Paul Bricman

Finetune language transformers

Does Introspective Truthfulness Generalize in LMs? with Jacob Pfau

Inducing Human-Like Biases in Moral Reasoning LMs with Bogdan-Ionut Cirstea

Create annotation tools

Behavioral Annotation Framework for the Contextualized and Personalized Fine-Tuning of Foundation Models with Eleanor “Nell” Watson

Review and analyse literature

How Should Machines Learn from Default Options? with En Qi Teo

Literature Review of the Neurological Basis of Human Values and Preferences with Linda Linsefors

Machine learning for Scientific Discovery: the Present and Future of Science-Producing AI Models with Eleni Angelou

Propose public policy/communication

Policy Proposals for High-Risk AI Regulation with Koen Holtman

Developing Specific Failure Stories about Uncontrollable AI with Karl von Wendt

Apply to AISC Virtual 2023

Apply if you…

want to try out & consider ways you could help ensure that future AI performs safely and in line with what people value upon reflection;
are ready to dig into our research leads’ r

...
Kakistocuriosity
by LVSN — 2023-01-06T07:38:56.493Z

I'm coining the word 'kakistocurious'.
Its etymology derives from

the Greek prefix kakisto-, meaning 'worst', as in 'kakistocracy' (rule of the worst),
and the English suffix -curious, meaning 'wanting to know', as in 'bicurious'.

People who are kakistocurious think about stereotypically bad things and wonder if they are really as bad as people say those things are.

I made up this word because it was necessary for uniquely identifying my personality using only four words (the other three words are 'empathetic', 'extreme', and 'discriminatory').

Kakistocuriosity is not the same as contrarianism. Contrarianism is when you often defend unpopular positions, or prefer to. Kakistocuriosity can lead to incidental contrarianism, and contrarianism can even lead to kakistocuriosity in the case of someone who wants to defend unpopular positions but absolutely requires that the positions are accurate before defending them.

Kakistocuriosity, importantly, is not curiosity about things that are bad regardless of what anyone thinks; it is curiosity about things that seem bad and may or may not actually be bad. This distinction is not obvious from the etymology, unfortunately.

If you wanted, you could call people who routinely explore stereotypically bad things 'kakistonauts'.
AI Safety Camp: Machine Learning for Scientific Discovery 
by Eleni Angelou — 2023-01-06T03:21:37.823Z

Applications for AI Safety Camp 2023 have opened!

I'll be Research Lead for the project Machine learning for scientific discovery: the present and future of science-producing AI models.

Summary:

Machine learning models have recently found remarkable success in

quantitative reasoning

and in producing significant research results in specific scientific fields such as biology (e.g., with

AlphaFold

) and

chemistry

. This might be an indication that in the (near-term) future, AI models will be able to generate and/or test novel hypotheses or even produce research worthy of a Nobel prize. This project is about mapping out the current state-of-the-art of science-producing AI models. This will allow for a more comprehensive understanding of the “cognitive properties” or capabilities of the available models. The project focuses on models that generate impressive results in solving quantitative problems (e.g., Minerva) or have led to important breakthroughs and acceleration of scientific research, such as in the case of AlphaFold. The team will collect the relevant research papers, review them, and study the development of capabilities necessary for scientific reasoning in relation to AI risk and progress in alignment research.

If you're unsure whether you'd be a good fit, take the short quiz you can find on this page. Regardless, err towards applying. If you have any questions, feel free to message me.
Metaculus Year in Review: 2022
by ChristianWilliams — 2023-01-06T01:23:07.050Z
Link: https://metaculus.medium.com/metaculus-year-in-review-2022-c92ff9cd87a0

The following is Metaculus's year-in-review CEO letter written by Gaia Dempsey.

2022 was a year of progress, growth, and change at Metaculus.

We matured and grew as an organization.

Mission Update: In the spring, we

expanded the scope of our mission

, creating a solid foundation for our work. Specifically, our mission is to build epistemic infrastructure that enables the global community to model, understand, predict, and navigate the world’s most important and complex challenges. We also clearly defined the three primary ways we enact our mission, namely: providing forecasts as a public service, fostering a global forecasting community, and supporting forecasting research.
Pro Forecaster Program: In response to a sharp increase in global uncertainty with the outbreak of the war in Ukraine, we designed new operational procedures including an infohazard review process, and recruited over 30 candidates into an exceptionally talented

Pro Forecasting team

.
Core Funding: In the summer, we

secured a $5.5M core funding grant

from

Open Philanthropy

to expand our work.
Becoming a Public Benefit Corporation: Shortly after, we committed to providing three specific public benefits in our charter, reflecting our mission statement, as part of the process of

becoming a Public Benefit Corporation

.
Growing Our Team: With a mandate to expand our platform, programs, and operations, we set about upgrading our hiring processes thanks to great

resource recommendations

from Jim Savage (

a

), and Aaron Hamlin (

b

), and got laser focused on recruiting. Over the course of the year we doubled our overall headcount, building four strong new teams:

leadership

, AI forecasting, engineering, and design (some engineering start dates are in early 2023).

We launched over a dozen forecasting tournaments and generated over 1000 aggregate forecasts.

Expansion of Forecasting Programs: Over the last year, we expanded our existing partnerships and began collaborations with a number of fantastic new partners. In total, we launched

13 new forecasting tournaments

offering a total of $77,500 in prize money, with a number of programs delivering policy-relevant predictions on public health, biosecurity, and nuclear-risk. Outside of these core themes (each of which I’ll touch on later),

Forecasting Our World In Data

was a standout launch: a project that “probes the long-term future, delivering predictions on top

...
UDASSA
by Jacob Falkovich — 2023-01-06T01:07:17.563Z

Cross-posted, as always, from Putanumonit.

When constructing a theory of the universe and everything else, most people start with the universe.

For example: a theory that the universe was created by God, starting with important things like grass (Genesis 1:12), the moon (Genesis 1:16), and big crocodiles (Genesis 1:21), before finally getting to humans. These humans eventually also gain self awareness, but that doesn’t happen until Genesis 3.

Or: there are a bunch of possible

mathematical structures

, and some of them exist because they describe physical structures, and some of those physical structures are complex enough to become Self-Aware Substructures™. I think those “SAS” refer to you and me, though I’m not sure. Max gave me a copy of Our Mathematical Universe at

this nerdy comedy thing I did

but I never got around to finishing it.

What I do know: if a theory of the universe starts with the universe, it usually takes a while to get to conscious beings that can have experiences. Self awareness feels somewhat tacked on to whatever structure is supposed to explain grass and moons.

But what if we started from just the fact of experience itself? Taking a few steps beyond the most narrow solipsism here is a short list of observation you may arrive at, in order of decreasing certainty:

You have some sort of conscious experience.
You have different sorts of conscious experience.
Some experiences (“thoughts”) seem to follow others in a chain, as if your internal state is crunching through some process step by step. Let’s call this process “computation”.
But other experiences (“perceptions”) aren’t entirely predicted by your prior internal state, suggesting there’s some external world that creates them.
This external world seems orderly. That is: you can compress what’s happening into a short description of what’s where and how it behaves according to fixed rules, as opposed to everything happening at random and all over the place.
But insofar as this external world gives rise to experience, it doesn’t seem to do it in a computationally efficient manner. There’s way too much empty space where no experience is likely happening, a lot of hard-to-compute physics happening inside unconscious (?) stars as opposed to inside conscious brains, etc.

This all may suggest the following theory:

Computation happens, and may result in an “experience moment” like the one you’re having...
The Involuntary Pacifists
by Capybasilisk — 2023-01-06T00:28:49.109Z

This might be an idea better put to r/WritingPrompts (or ChatGPT), but a scenario has occurred to me which might count either as a near-utopia, or extreme dystopia, depending on your point of view.

Imagine a future Earth where a great power (this can be a superintelligent AI, benevolent alien race, or just straight-up God, depending on your preferences) has imposed a version of the libertarian harm principle on humanity. Most actions a human can engage in today would be available to you, you just wouldn't be able to commit an act of violence against another human. Sure, you might have an extremely strong desire to harm a human, but try as you might, your brain just can't send the signals to the rest of your body in order to pull that trigger/prepare that device/program that killbot.

We should also define "harm" very narrowly here. A person scamming an elderly lady out of her life savings is definitely harm, so is hurling racial epithets at someone, but I think we should here reserve the word for the most serious of crimes. These involve a violation of bodily integrity. Murder, assault, rape, and forcible confinement. What would this world where you can't do violence to a fellow human be like? Getting your head bashed in with a tire iron during a road rage incident would be something you read about in historical fiction. Same with war. You can gather all the men and material you possible can, but you simply wouldn't be able to use them. Seems pretty utopian to me, though I'm aware sometimes important details are hidden from cursory glances at a possible world. A possible problem is how do you punish that guy who scammed the grandma? Taking him into custody would necessarily involve acts of violence no longer available to humans.

We could even limit this to non-consensual violence. All you MMA types, having formally agreed to accept violence from an opponent, would be free to punch each other's faces in. You could even have voluntary wars, where all the people who really miss the sound and fury of battle can meet up in central Australia or somewhere and have at it, leaving the rest of humanity in peace.

It would be interesting to see how conflict among humans evolves under these conditions. Two primitive tribes are competing for a scarce water source. So one tribe comes to take it from the other by force, and then I guess they stand around with constipated express...
Get an Electric Toothbrush. 
by Cervera — 2023-01-05T21:08:21.726Z

Okay so, with the inspirations of the concise posts about lifestyle that user @jefftk does, I want to share one of my most recent discoveries with LW, one that with little economical input can offer you a better quality of life.

GET AN ELECTRIC TOOTHBRUSH!!

Really! A 20 bucks Oral-B department store is fine!

Do you remember the nice feeling when you go to your dentist for a cleanup and you leave with that smooth, polished feeling on your teeth that sometimes last you days? I do remember it, I got the exact same feeling from my first brush with an Electric toothbrush.

I got a Basic Oral-B Vitality[1] from my grocery store a week ago as a suggestion from my girlfriend, and is the best palpable investment of the year for me.

You don't need anything fancy. It doesn't have to be a big investment like the 100$ plus ones with pressure sensors, a basic, 2 minute timer electric toothbrush will get you 90% of the way there, a YouTube video tutorial about how to use them effectively, another 5%

Stuff that's good to know.

You need to touch the gums with the toothbrush
Almost no pressure is necessary, I hold it with two fingers to remind myself, the rapid oscillation of the head will do the rest
Just move it from teeth to teeth, no more "toothbrush" movement required.

After a week of use, I feel confident enough to share the news. it's the superior way of cleaning your teeth, although I miss the caveman feeling of nuking my teeth with a hard brush from time to time.

^

This is European branding, I'm almost certain you can find the same product in the US or other markets from Oral-B with different names, a quick search gives me This stuff but I'm sure you can find something similar on your local Walmart
Discursive Competence in ChatGPT, Part 1: Talking with Dragons
by Bill Benzon — 2023-01-05T21:01:48.287Z

Version 1, January 5, 2022

Title above, URLs, abstract, contents, and introduction below:

Academia.edu: https://www.academia.edu/94409729/Discursive_Competence_in_ChatGPT_Part_1_Talking_with_Dragons
SSRN: https://ssrn.com/abstract=4318832
Research Gate: https://www.researchgate.net/publication/
366897197_Discursive_Competence_in_ChatGPT_Part_1_Talking_with_Dragons_Discursive_Competence_in_ChatGPT_Part_1_Talking_with_Dragons

Abstract: Noam Chomsky’s idea of linguistic competence suggests a new approach to understanding how LLMs work. This approach requires careful analysis of text. Such analysis indicates that ChatGPT has explicit control over sophisticated discourse skills: 1) It possesses the capacity to specify high-level structures that regulate the organization of language strings into specific patterns: e.g. conversational turn-taking, story frames, film interpretation, and metalingual definition of abstract concepts. 2) It is capable of analogical reasoning in the interpretation of films and stories, such as Spielberg’s Jaws and A.I., and Tezuka’s Astro Boy stories. It must establish an analogy between some abstract interpretive theory (e.g. the ideas of Rene Girard) and people and events in a story. 3) It has some understanding of abstract concepts such as justice and charity. Such concepts can be defined over concepts that exhibit them (metalingual definition). ChatGPT recognizes suitable stories and can revise them. 4) ChatGPT can adjust its level of discourse to accommodate children of various ages. Finally, much of ChatGPT’s discourse seems formulaic in a way similar to what Parry/Lord found in oral epic.

Contents

Introduction: Walking among dragons 2
What is in the rest of this document? 6
Calibration: Understanding a Seinfeld Bit 9
Conversing with ChatGPT about Jaws, Mimetic Desire, and Sacrifice 12
ChatGPT on Spielberg’s A.I. Artificial Intelligence and AI Alignment 23
Extra! Extra! In a discussion about Astro Boy, ChatGPT defends the rights of robots and advanced AI 27
Pumpkins, the Falcon Heavy, and Groucho Marx: High level discourse structure in ChatGPT 30
High level discourse structure in ChatGPT...
Transformative AI issues (not just misalignment): an overview
by HoldenKarnofsky — 2023-01-05T20:20:06.424Z

If this ends up being the most important century due to advanced AI, what are the key factors in whether things go well or poorly?

More detail on why AI could make this the most important century (Details not included in email - click to view on the web)

A lot of my previous writings have focused specifically on the threat of “misaligned AI”: AI that could have dangerous aims of its own and defeat all of humanity. In this post, I’m going to zoom out and give a broader overview of multiple issues transformative AI could raise for society - with an emphasis on issues we might want to be thinking about now rather than waiting to address as they happen.

My discussion will be very unsatisfying. “What are the key factors in whether things go well or poorly with transformative AI?” is a massive topic, with lots of angles that have gotten almost no attention and (surely) lots of angles that I just haven’t thought of at all. My one-sentence summary of this whole situation is: we’re not ready for this.

But hopefully this wil

...
How to slow down scientific progress, according to Leo Szilard
by jasoncrawford — 2023-01-05T18:26:12.121Z
Link: https://rootsofprogress.org/szilard-on-slowing-science

Leo Szilard—the physicist who first conceived of the nuclear chain reaction and who urged the US to undertake the Manhattan Project—also wrote fiction. His book of short stories, The Voice of the Dolphins, contains a story “The Mark Gable Foundation,” dated 1948, from which I will present to you an excerpt, without comment:

“I’m thinking of setting up a trust fund. I want to do something that will really contribute to the happiness of mankind; but it’s very difficult to know what to do with money. When Mr. Rosenblatt told me that you’d be here tonight I asked the mayor to invite me. I certainly would value your advice.”

“Would you intend to do anything for the advancement of science?” I asked.

“No,” Mark Gable said. “I believe scientific progress is too fast as it is.”

“I share your feeling about this point,” I said with the fervor of conviction, “but then why not do something about the retardation of scientific progress?”

“That I would very much like to do,” Mark Gable said, “but how do I go about it?”

“Well,” I said, “I think that shouldn’t be very difficult. As a matter of fact, I think it would be quite easy. You could set up a foundation, with an annual endowment of thirty million dollars. Research workers in need of funds could apply for grants, if they could make out a convincing case. Have ten committees, each composed of twelve scientists, appointed to pass on these applications. Take the most active scientists out of the laboratory and make them members of these committees. And the very best men in the field should be appointed as chairmen at salaries of fifty thousand dollars each. Also have about twenty prizes of one hundred thousand dollars each for the best scientific papers of the year. This is just about all you would have to do. Your lawyers could easily prepare a charter for the foundation. As a matter of fact, any of the National Science Foundation bills which were introduced in the Seventy-ninth and Eightieth Congresses could perfectly well serve as a model.”

“I think you had better explain to Mr. Gable why this foundation would in fact retard the progress of science,” said a bespectacled young man sitting at the far end of the table, whose name I didn’t get at the time of introduction.

“It should be obvious,” I said. “First of all, the best scientists would be removed from their laboratories and kept busy on committees passing on applications fo

...