trentmkelly/LessWrong-43k huggingface dataset:
Join the interpretability research hackathon
by Esben Kran — 2022-10-28T16:26:05.270Z

TLDR; Participate online or in-person in London, Aarhus, and Tallinn on the weekend 11th to 13th November in a fun and intense

AI safety research hackathon

focused on interpretability research. We invite mid-career professionals to join but it is open for everyone (also no-coders) and we will create starter code templates to help you kickstart your team’s projects.

Join here

.

Below is an FAQ-style summary of what you can expect (navigate it with the table of contents on the left).

What is it?

The Interpretability Hackathon is a weekend-long event where you participate in teams (1-6) to create interesting and fun research. You submit a PDF report that summarizes and discusses your findings in the context of AI safety. These reports will be judged by our panel and you can win up to $1,000!

It runs from 11th Nov to 13th Nov (in two weeks) and you’re welcome to join for a part of it (see further down). We get an interesting talk by an expert in the field and hear more about the topic.

Everyone can participate and we encourage you to join especially if you’re considering AI safety from another career . We prepare templates for you to start out your projects and you’ll be surprised what you can accomplish in just a weekend – especially with your new-found friends!

Read more about how to join, what you can expect, the schedule, and what previous participants have said about being part of the hackathon below.

Where can I join?

You can join the event both in-person and online but everyone needs to make an account and join the jam on the

itch.io page

.

The in-person locations include the LEAH offices in London right by UCL, Imperial, King’s College, and London School of Economics (

link

); Aarhus University in Aarhus, Denmark (

link

), and Tallinn, Estonia (

link

). The virtual event space is on GatherTown (

link

).

Everyone should join the Discord to ask questions, see updates and announcements, find team members, and more.

Join here

.

What are some examples of interpretability projects I could make?

You can check out a bunch of interesting, smaller interpretability project ideas on

AI Safety Ideas

such as

reconstructing the input from neural activations

,

evaluating the alignment tax of interpretable models

, or

making models’ uncertainty interpretable

.

Other examples of practical projects can be to find new ways to visualize featu...
Syncretism
by Annapurna — 2022-10-28T16:08:17.639Z
Link: https://jorgevelez.substack.com/p/syncretism

The act or system of blending, combining or reconciling inharmonious elements.

Kalpa, Sutlej Valley, India

I learned the word syncretism when I was reading about Kalpa, a village in the Indian Himalaya that I visited on October 19, 2022. In ancient times, Kalpa was at the crossroads of Tibet and India, and received constant influence from both cultures. What I found truly fascinating about Kalpa is that part of its population practices syncretism. Instead of choosing between Hinduism or Buddhism, they decided to practice both, merging its rituals and customs. Furthermore, the religions not just only coexist harmoniously in Kalpa: They share temples and festivals.

Syncretism between Hinduism and Buddhism is not only practiced in Kalpa; it is common in the rest of the Kinnaur District of Himachal Pradesh as well as other areas of India. As far as I am aware, in modern times there aren’t many places in the world where two major religions are blended together like Hinduism and Buddhism are in Kalpa. This got me thinking: Why doesn’t this happen more often in modern times?

Rakchham, Baspa Valley, India

I believe more syncretism between faiths does not happen because modern society would dismiss it: It would likely mock it and call it pseudo-faith. I believe this is a shame: to me it makes sense that if religious syncretism was practiced and accepted more in society, the world would be more harmonious. The Kinnauris were too isolated to be told that merging religious practices was wrong, and by the time their lands were well connected to the modern world, syncretism was very ingrained in their culture.

Syncretism does not only apply to theology: it also applies to ideology. In the past, people would blend schools of thought, mainly in the name of compromise. As political polarization continues to thrive in the world, my experience spending time in a place where religious syncretism got me thinking: How can we apply political syncretism in our communities so that we can live in more harmoniously?

Key Monastery, Spiti Valley, India
Pondering computation in the real world
by Adam Shai — 2022-10-28T15:57:27.419Z

Epistemic Status: Exploratory. Asking questions and setting up some intuition pumps. Would very much like to make my thoughts on this topic more precise.

Some systems that compute

Adders

Here's an electrical circuit you can build at home.

From https://circuitdigest.com/tutorial/half-adder-circuit-and-its-construction

It's called a half adder circuit. By flipping the toggles on the left in the red box you set two binary inputs, e.g. you can represent 0+1 or 1+1. In response, the circuit will do a highly structured electromagnetic dance to make the green and red lights in the bottom right turn on or off, in order to represent the output of binary addition. We say: "this circuit computes addition."

Here is a marble adding machine.

By putting in marbles at certain locations at the top of the machine, you can input binary numbers. A series of see-saw toggles change their positions under the influence of gravity and the marbles. This mechanical dance results in marbles resting in output positions at the bottom of the machine, indicating the results of binary addition. We say: "this circuit computes addition."

Different physical systems can be associated with the same computation.

Integrators

This is Emmy.

Emmy is in high school, and has some math homework. She has to plot the solution, x(t), to the following differential equation.

150d2xdt2=−9.8x(0)=0,dxdt(0)=15

She takes a pencil and a piece of paper and writes the equation down, and then starts using the rules she learned for symbolic integration. Her first move is to write dxdt=−9.8150x+C, because she knows the rule that the integral of a constant K is Kx+C. She continues in this manner and gets her final result and plots it, by hand. She gets full marks on this question. We say: "Emmy has integrated the differential equation."

This is Isaac.

Isaac is in Emmy's class, and got the same homework problem. He gathers an apple, a stopwatch, and a ruler.

He takes the apple, and throws it straight up in the air at 15 meters per second and as best he can, simultaneously starts the stopwatch. The first thing he does is measure how high, in meters, the apple goes, and how long it takes to get to that maximum height. He throws the ball again, this time measuring the height and time slightly offset from the maximum. He does this 1,000 times. On every throw he writes down the time from the throw at which he m...
Ukraine and the Crimea Question
by ChristianKl — 2022-10-28T12:26:51.982Z

When the United States invaded Iraq, they were hoping to be greeted as liberators. In a similar vein, the current discourse suggests that if Ukraine uses its military to successfully liberate Crimea, Crimeans will greet Ukrainians as liberators. The West that currently supports Ukraine has to make a decision on whether to support Ukraine in retaking Crimea. Understanding what's likely going to happen in Crimea should Ukraine reconquer Crimea is vital to making good decisions.

Girard talks about how society uses scapegoating to assign the blame to one individual to end conflicts with other people. The Western media likes to scapegoat Putin because it is an easy way to relate to the conflict without having to understand anything about the underlying circumstances. In both 2014 and in 2022 Putin increased his

approval rating

by > +15% by ordering his military into Ukraine. If those conflicts just exist because Putin is crazy, it wouldn't push his approval that much. That's slightly more than the growth in

approval

that George Bush got when he started the Iraq war.

You could argue that George Bush only got this bump in approval because of the propaganda of the United States government and the willingness of the United States media to uncritically broadcast that propaganda. By the same token the Russian media also uncritically broadcasts their state propaganda that makes the population happier about going to war.

Unfortunately, it’s true that government propaganda makes citizens approve of war when they shouldn’t. War creates a lot of suffering and it would be great if nations would be more cautious about starting them. On the other hand, it shows that wars are often not just fought because the person at the top wants to fight the war. Many actors take part in a complex system that increases presidential approval at the start of a war.

How much fascism is there in Ukraine?

To understand the current conflicts, it’s vital to understand what the Russian discourse means when it talks about Nazis. Crimeans largely consume Russian media and their views are shaped by the Russian discourse. It’s both useful to understand what they think and what’s objectively happening.

Unfortunately, the current Western discourse isn’t very truth-seeking. Twitter censored a journalist who posted a

collage

out of photos obtained either from mainstream report...
New book on s-risks
by Tobias_Baumann — 2022-10-28T09:36:57.642Z

I have just published my new book on s-risks, titled Avoiding the Worst: How to Prevent a Moral Catastrophe. You can find it on

Amazon

,

read the

PDF version

,

or listen to the audio version.

The book is primarily aimed at longtermist effective altruists. I wrote it because I feel that s-risk prevention is a somewhat neglected priority area in the community, and because a single, comprehensive introduction to s-risks did not yet exist. My hope is that a coherent introduction will help to strengthen interest in the topic and spark further work.

Here’s a short description of the book:

From Nineteen Eighty-Four to Black Mirror, we are all familiar with the tropes of dystopian science fiction. But what if worst-case scenarios could actually become reality? And what if we could do something now to put the world on a better path?

In Avoiding the Worst, Tobias Baumann lays out the concept of risks of future suffering (s-risks). With a focus on s-risks that are both realistic and avoidable, he argues that we have strong reasons to consider their reduction a top priority. Finally, he turns to the question of what we can do to help steer the world away from s-risks and towards a brighter future.

For a rough overview, here’s the book's table of contents:

Part I: What are s-risks?

Chapter 1: Technology and astronomical stakes

Chapter 2: Types of s-risks

Part II: Should we focus on s-risks?

Chapter 3: Should we focus on the long-term future?

Chapter 4: Should we focus on reducing suffering?

Chapter 5: Should we focus on worst-case outcomes?

Chapter 6: Cognitive biases

Part III: How can we best reduce s-risks?

Chapter 7: Risk factors for s-risks

Chapter 8: Moral advocacy

Chapter 9: Better Politics

Chapter 10: Emerging technologies

Chapter 11: Long-term impact

And finally, some blurbs for the book:

“One of the most important, original, and disturbing books I have read. Tobias Baumann provides a comprehensive introduction to the field of s-risk reduction. Most importantly, he outlines sensible steps towards preventing future atrocities. Highly recommended.”

— David Pearce, author of The Hedonistic Imperative and Can Biotechnology Abolish Suffering?

“This book is a groundbreaking contribution on a topic that has been severely neglected to date. Tobias Baumann presents a powerful case for averting worst-case scenarios that could involve vast amounts of sufferin

...
Cryptic symbols
by Adam Scherlis — 2022-10-28T06:44:41.108Z
Link: https://adam.scherlis.com/2022/10/28/cryptic-symbols/

Here’s something I’ve been playing with:

Explanation coming soon, but you may enjoy trying to figure it out on your own.

[EDIT 10-29: I initially uploaded an incorrect image, included below for posterity:]
Prizes for ML Safety Benchmark Ideas
by joshc — 2022-10-28T02:51:46.369Z

“If you cannot measure it, you cannot improve it.” – Lord Kelvin (paraphrased)

Website:

benchmarking.mlsafety.org

– receiving submissions until August 2023.

ML Safety lacks good benchmarks, so the

Center for AI Safety

is offering $50,000 - $100,000 prizes for benchmark ideas (or full research papers). We will award at least $100,000 total and up to $500,000 depending on the quality of submissions.

What kinds of ideas are you looking for?

Ultimately, we will are looking for benchmark ideas that motivate or advance research that reduces

existential risks from AI

. To provide more guidance, we’ve outlined four research categories along with

example ideas

.

Alignment: building models that represent and safely optimize difficult-to-specify human values.
Monitoring: discovering unintended model functionality.
Robustness: designing systems to be reliable in the face of adversaries and highly unusual situations.
Safety Applications: using ML to address broader risks related to how ML systems are handled (e.g. for cybersecurity or forecasting).

See

Open Problems in AI X-Risk [PAIS #5]

for example research directions in these categories and their relation to existential risk.

What are the requirements for submissions?

Datasets or implementations are not necessary, though empirical testing can make it easier for the judges to evaluate your idea. All that is required is a brief write-up (

guidelines here

). How the write-up is formatted isn’t very important as long as it effectively pitches the benchmark and concretely explains how it would be implemented. If you don’t have prior experience designing benchmarks, we recommend reading

this document

for generic tips.

Who are the judges?

Dan Hendrycks

,

Paul Christiano

, and

Collin Burns

.

If you have questions, they might be answered on

the website

, or you can post them here. We would also greatly appreciate it if you helped to spread the word about this opportunity.

Thanks to Sidney Hough and Kevin Liu for helping to make this happen and to Collin Burns and Akash Wasil for feedback on the website. This project is supported by the Future Fund regranting program.
Anatomy of change
by Jose Miguel Cruz y Celis — 2022-10-28T01:21:33.053Z

Once you start generating awareness as a human of how bad things are, the risks we face and the things that should change, it's hard to understand how you can play a role.

I personally struggled for a while at some stage in my life with this and through research and experience I've come to understand more of the landscape of how change is enabled. I decided to create this diagram which I think can be useful to keep in mind as a mental model and to share it with people who might struggle to grasp how their current role in society can influence change.

The basic premise is that we all start as humans, as we gain awareness of the status quo and if everything goes well we generate will to effect change. The first level we hit is that of a citizen and a citizen really has a small individual lever but can en-masse generate "demand of change" which in turn shapes policies and the technologies we build.

I consider lifestyle choices, consumer choices, voting, raising your voice, supporting a cause, etc all forms of a similar mechanism of protest that a citizen can make use of. A citizen can then become an activist, engineer, entrepreneur, scientist, politician, business leader or philosopher.

Each of these professions has a particular role in society and a particular lever.

An activist generates awareness through creating noise around a certain issue or translating that issue for a more general public. I consider in the context of change an artist or a teacher to be an activist.

An entrepreneur creates a startup which in turn develops and distributes new technologies.

A scientist does research which leads to greater insight and in turn content to translate into awareness, better policies or technologies.

A politician can lobby, make proposals and do politics which can lead to better policies.

A business leader has less of a lever, because in general they're already part of the establishment and have an innovator's dilemma. They're slow but can their actions can marginally lead to better practices, products and services.

A philosopher questions the status quo and our current mindset and proposes new ways of thinking about it which can in turn lead to better frameworks and concepts.
Nash equilibria of symmetric zero-sum games
by Ege Erdil — 2022-10-27T23:50:23.583Z

This post is meant to be a reference for some basic properties of Nash equilibria in symmetric zero-sum games. I'll focus on two player games for simplicity, though the content of the post straightforwardly generalizes to the case of N>2 players. I won't assume familiarity with game theory in this post, though the post could still be somewhat technical and you should consider yourself appropriately warned of this.

Preliminaries

Introducing the terminology

For the purposes of this post, a N-player game is characterized by a reward function r:∏iAi→∏iR which takes an N-tuple of player "actions" (a1,a2,…,aN) and maps this to an N-tuple (r1,r2,…,rN) of rewards that are received by each of the N players. We will interpret "action" liberally to mean "deterministic strategy", so an element of Ai is a deterministic strategy that encodes moves that player i will make in every possible situation that could arise in the game. We will work explicitly with finite games: these are games for which the sets Ai are finite sets. Here the underlying game could be a game that involves randomness: in this case r would simply represent the expected reward of playing the game.

We will also require a probabilistic notion of strategy. A strategy si for player i is simply a probability distribution over the finite set of actions Ai, so it's a function si:Ai→[0,1] that must sum to 1. I will denote the set of all strategies of player i by Si, and I will also engage in some operator overloading to extend the definition of the reward function r to strategies in the obvious way by treating the decisions of distinct players as independent events and taking expectations:

r(s1,…,sN)=∑a1∈A1,…,aN∈aNr(a1,…,aN)∏isi(ai)

A game is said to be zero-sum if the reward function r has the property that ∑iri=0 regardless of the actions chosen by the players, and symmetric if the action sets Ai are all identical for all of the players and the reward function is covariant under (or commutes with) permutations of its inputs: if we apply some permutation to which players are playing which actions, the associated reward vector must also get permuted in the same way. Intuitively, this means in a symmetric game there's really "no distinction" between the players of the game.

When the game is a two-player game, the zero-sum condition gets rid of one of the degrees of freedom in the definition of reward, so put...
Good psychology books/books that contain good psychological models?
by shuffled-cantaloupe — 2022-10-27T23:04:03.701Z

In Poor Charlie's Almanack, Charlie Munger (https://en.wikipedia.org/wiki/Charlie_Munger), states that it useful to have 20 or so psychological models that one can run through--like
a flight attendant's checklist, in order to avoid misjudgement.
Charlie states that most psych textbooks are bad for a variety of reasons (the brightest people aren't writing them, good psych experiments might require being unethical, etc. etc.)
Nevertheless, he claims that there are some psych books/studies/etc. that are worth reading. However, aside from 1-2 examples, he doesn't tell the reader what they are.

What are good psych books/papers/etc. that go over the common psychological models one should know for usage in day-to-day life?
Podcast: The Left and Effective Altruism with Habiba Islam
by garrison — 2022-10-27T17:41:05.136Z

I recently rebooted my interview podcast, The Most Interesting People I Know (found wherever you find podcasts). I focus on EA and left-wing guests, and have been pretty involved in both communities for the last 5 years. Some example guests: Rutger Bregman, Leah Garcés, Lewis Bollard, Spencer Greenberg, Nathan Robinson, Malaika Jabali, Emily Bazelon, David Shor, and Eric Levitz.

I just released a long conversation with Habiba Islam, an 80K career advisor and lefty, about the relationship between EA and the left.

This is not an attempt to paper over differences between the two communities, or pretend that EA is more left-wing than it is. Instead, I tried to give an accurate description of both communities, where they are in hidden agreement, where they actually disagree, and what each can learn from the other.

Habiba is so sharp and thoughtful throughout the conversation. We're very lucky to have her!

I hope this could be a good reference text as well as an onboarding ramp for leftists who might be open to EA.

I think there's a real gap in the EA media-verse on the intersection of left-wing politics and EA, and we're almost certainly missing out on some great people and perspectives who would be into EA if they were presented with the right arguments and framing.

I have no delusions that all leftists would be into EA if they only understood it better, but I think there are tons of bad-faith criticisms and genuine misunderstandings that we could better address. I think we can have a healthier and more productive relationship with the left.

If you'd like to support the show, here are some things you can do:

Personally recommend the show/particular episodes to friends. Apparently, this is how podcasts best grow their audiences.
Share the podcast/episode on social media (I'm on Twitter @garrisonlovely)
Rate and review the show on Apple Podcasts.
Give me feedback (anonymous form here). You can also email me at tgarrisonlovely@gmail.com