trentmkelly/LessWrong-43k huggingface dataset:
How to embark on a journey of self-discovery (and potentially succeed)
by Ester Dobiášová — 2023-06-04T18:46:52.762Z
Link: https://ladyesik.wordpress.com/2023/06/02/how-to-embark-on-a-journey-of-self-discovery-and-potentially-succeed/

It started innocently – I wanted to write down my philosophy of life, to become more aware of the foundations of my decision making, thinking and maybe even feeling. This year, I’m going through a housing search, a job reassessment, as well as changes in relationships. I wanted to produce at least a few signposts for these life milestones as I decide which way to go; and to better navigate myself so I don’t beat around the bush.

How did it turn out? Discussions with effective altruists yielded a list of 120 questions (for other ways to evaluate annually, see here or here, for example), only one of which relates to life philosophy, accompanied by a subtle inquiry about my values, and capped by the question: „Am I living in alignment with my life philosophy and core values?“ In answering it, I zigzag between my cognitive biases and other biases, who am I trying to keep my face from? No one will see it, it will only torture me on AGI in the future.

I took the questions on a month-long tour of Europe, which has paid off in the past. I don’t want to share my answers with you, but tools and tips on how I try to answer the hard questions. It makes the whole process of self-discovery easier. I don’t know if my approach is a good one, but the process itself gives me stability and makes me happy. I hope you find something useful as well.

Values and Purpose

Do I have meaning and direction in my life? What is my philosophy of life? What do I want from life? What are my core values? Am I living in accordance with my life philosophy and core values?

I’ll start with the simplest: values. On the Clearer Thinking website, you’ll find tools for improving your decision-making or making positive changes in your life. Among other things, there is an intrinsic values test. Honestly, what I find most interesting about it is the reasoning behind why it is important to know your intrinsic values:

Avoid „value traps“: value traps involve pursuing something that appears valuable (e.g. because society says it is valuable), but which doesn’t actually bring you closer to realizing your true intrinsic values.
Reduce unhelpful social guilt: we often feel pressure to seek the values that we are taught when we are young, or that those around us have. If these differ from our own personal intrinsic values, we may feel guilt, shame and insecurity, since those things we really care about are dif

...
AI Safety Fundamentals: An Informal Cohort Starting Soon!
by Tiago de Vassal — 2023-06-04T17:15:47.646Z

Hi Everyone,

We'll be starting an informal cohort on AI Safety Fundamentals.

Come join-us on Discord. We'll do a poll next week to find a slot that works for most people.
(if you're interested in participating as an expert, do reach out as well)

We'll do the kick-off session on the 17th of June, and then do the course at the pace of one session per week for eight weeks (you'll be able to discuss the material in-between with other members). I'm currently reaching out to aligment researchers interested in popping in for insights and answering questions we might have.

We'll then do the AI Governance course (https://aisafetyfundamentals.com/ai-governance-curriculum), and then the AI Alignment 201 course (https://aisafetyfundamentals.com/alignment-201-curriculum) - you should be able to singlehandedly solve AGI by the end.

Please note that this is not an official cohort, there is one starting on governance this summer (you can join on AI Safety Fundamentals' website), and one on AI Alignement in September (registrations not opened yet).

Bye!

Tiago
How to Think About Activation Patching
by Neel Nanda — 2023-06-04T14:17:42.264Z
Link: https://www.neelnanda.io/mechanistic-interpretability/attribution-patching#how-to-think-about-activation-patching

This is an excerpt from my post on attribution patching, that I think is of more general interest, around how to think about the technique of activation patching in mechanistic interpretability, and what it can and cannot teach us. You don't need to know what attribution patching is to read this, check out this section for an overview of activation patching

What Is the Point of Patching?

The conceptual framework I use when thinking about patching is to think of a model as an enormous mass of different circuits. On any given input, many circuits will be used. Further, while some circuits are likely very common and important (eg words after full stops start with capitals, or "this text is in English"), likely many are very rare and niche and will not matter on the vast majority of inputs (eg "this is token X in the Etsy terms and conditions footer" - a real thing that GPT-2 Small has neurons for!). For example, on IOI, beyond the actual IOI circuit, there is likely circuits to:

Detect multi-token words (eg names) and assemble them into a single concept
Detect that this is English text
Detect that this is natural language text
Encode unigram frequencies (under which John is 2-3x as likely as Mary!)
Copying all names present in the context - if a name is present in the text it's way more likely to occur next than an unseen name!
That this is the second clause of a sentence
That Indirect Object Identification is the right grammatical structure to use
That the next word should be a name

As a mech interp researcher, this is really annoying! I can get traction on a circuit in isolation, and there's a range of tools with ablations, direct logit attribution, etc to unpick what's going on. And hopefully any given circuit will be clean and sparse, such that I can ignore most of a model's parameters and most activation dimensions, and focus on what's actually going on. But when any given input triggers many different circuits, it's really hard to know what's happening.

The core point of patching is to solve this. In IOI, most of the circuits will fire the same way regardless of which name is the indirect object/repeated subject. So by formulating a clean and corrupted input that are as close as possible except for the key detail of this name, we can control for as many of the shared circuits as possible. Then, by patching in activations from one run to another, we will not affect th...
A Disneyland Without Children
by L Rudolf L — 2023-06-04T13:06:46.323Z
Link: https://nosetgauge.substack.com/p/a-disneyland-without-children

The spaceship swung into orbit around the blue-grey planet with a final burn of its engines. Compared to the distance they had travelled, the world, now only some four hundred kilometres below and filling up one hemisphere of the sky, was practically within reach. But Alice was no less confused.

"Well?" she asked.

Charlie stared thoughtfully at the world slowly rotating underneath their feet, oceans glinting in the sunlight. "It looks lickable", he said.

"We have a task", Alice said, trying to sound gentle. Spaceflight was hard. Organic life was not designed for it. But their mission was critical, they needed to move fast, and Charlie, for all his quirks, would need to be focused.

"What's a few minutes when it will take years for anything we discover to be known back home?" Charlie asked.

"No licking", Alice said.

Charlie rolled his eyes, then refocused them on the surface of the planet below. They were just crossing the coast of one of the larger continents. Blue water was giving way to grey land.

"Look at the texture", Charlie said. They had seen it from far away with telescopes, but there was something different about seeing it with their bare eyes. Most of the land surface of the planet was like a rug of fine grey mesh. If there had been lights, Alice would have guessed the entire planet's land was one sprawling city, but as far as their instruments could tell, the world had no artificial lighting.

As far as they could tell, the world also had no radio. They had broadcast messages at every frequency they could, and in desperation even by using their engines to flash a message during their deceleration burn. No response had come.

Alice pulled up one of the telescope feeds on the computer to look closer at the surface. She saw grey rectangular slabs, typically several hundred metres on a side, with wide roads running between them. The pattern was not perfect - sometimes it was irregular, and sometimes there were smaller features too. Some of the smaller ones moved.

"Are they factories?" Charlie asked.

"I'd guess so", Alice said, watching on the telescope feed as a steady stream of rectangular moving objects, each about ten metres long, slid along a street. Another such stream was moving along an intersecting street, and it looked like they would crash at the intersection, but the timing and spacing was such that vehicles from one stream crossed the road just as there...
I bet everyone 1000€ that I can make them dramatically happier & cure their depression in 3 months!
by EternallyBlissful — 2023-06-04T12:30:10.843Z

Update 9/23: I wrote this blog post a while ago. I no longer endorse the style and some of the content of this post. It still agree with all the essence of it, so it's still here. You might also want to check out these slides (optimized to make sense on their own) to see my latest thinking on this topic.

Important Caveat: All of the following are true: I'm serious about this bet. I'm also trying to sell something with this bet. I think this post is very useful/has an important message even if you aren't interested at all in my service or in the bet per se.

The title says it all: I bet you 1000€ that I can dramatically improve your subjective well-being and psychological health as well as cure your depression if you have any, all in 3 months, with a significant part of the improvements persisting permanently.

How it works

You have to follow through with some version that works for you of a full-time brute-forcing happiness (BFH) program for at least 3 months. This program consists of a long list of meditation, introspection, self-therapy, lifestyle, and various other exercises and interventions, done more or less full-time. See details below.
I will be your "BFH buddy" for those 3 months. This means I will stay with you for most or all of the whole time - up to 24/7 if you want - and support you with this program in every possible way. I will be your personal:

accountability partner: I help you in whatever way works best for you to follow through with the program. This is my most important task.
coach & mentor: I am pretty well knowledgeable and experienced with meditation, most self-therapy techniques, and everything else emotional work. I can answer all your questions, share advice, and devise the details of the program together with you based on your specific needs.
emotional support partner: I'm there to talk to you about everything, give you a hug whenever you want one, and listen emphatically to everything you want to talk about. I’m not a trained therapist and will not try to be yours, but I do have some experience in this regard. I also know a fair bit about re-traumatization prevention and practicing mindfulness safely.
personal assistant: I can do various small tasks for you (say, 1-2 hours/day) such that you can fully focus uninterruptedly on your mental health & the BFH program. For example, I can do various household chores like shoppi

...
Do You Really Want Effective Altruism?
by williamsae — 2023-06-04T08:06:43.080Z

There is something fundamentally different between the way nature solves complex problems and the way human beings currently do. When nature wants to solve problems beyond a certain level of complexity it uses networks. From whatever evidence we can find that remains of the billions of years of the history of this earth, we don't see even one example of nature having solved complex problems like vision, or flight, or cognition, through single cells. Nature solves such complexity through multi-cellularity (through networks of cells). Nature also solves the problem of exponentially increasing outcomes through networks. As an example, from the two gamete cells that combine to form a fertilized ovum only able to travel centimeters within its mother's womb, grows a blue whale consisting of quadrillions of cells that can swim thousands of kilometers through the world's oceans. In achieving this, nature exponentially increased the outcomes achievable by the cells of the whale.

Human outcomes, whether sustainable economic growth or access to affordable education, are achieved through products and services delivered by individuals and/or organizations. Inserting a network based approach when solving human problems that might be faced anywhere along the entire product or service life cycle from research and design to manufacturing takes a different way of thinking about things. If humans were asked to manufacture something like a whale we might attempt to calculate how much raw materials it requires, how much plankton, how much water, how much oxygen, etc. But nature didn't' manufacture the whale in a single project. It manufactured a self-sustaining network of cells that self-assembled additional networks to become the whale.

Nature’s method for solving complex problems through the design and manufacture of organisms is somehow able to explore every region in the available design configuration space of those organisms. Nature conducts this exploration in order to find the optimal definition of any problem it is trying to solve, and in order to find the optimal solution for that problem. Humans on the other hand are more goal directed and look at problems on much shorter time scales. We aren’t able to see problems and we aren’t able to look for solutions except through the limitations of our cognitive biases. We also don’t have a universal way of looking at problems and...
[Link Post]  Bytes Are All You Need: Transformers Operating Directly On File Bytes 
by Capybasilisk — 2023-06-03T22:45:46.139Z

Modern deep learning approaches usually transform inputs into a modality-specific form. For example, the most common deep learning approach to image classification involves decoding image file bytes into an RGB tensor which is passed into a neural network. Instead, we investigate performing classification directly on file bytes, without the need for decoding files at inference time. Using file bytes as model inputs enables the development of models which can operate on multiple input modalities. Our model, \emph{ByteFormer}, achieves an ImageNet Top-1 classification accuracy of 77.33% when training and testing directly on TIFF file bytes using a transformer backbone with configuration similar to DeiT-Ti (72.2% accuracy when operating on RGB images). Without modifications or hyperparameter tuning, ByteFormer achieves 95.42% classification accuracy when operating on WAV files from the Speech Commands v2 dataset (compared to state-of-the-art accuracy of 98.7%). Additionally, we demonstrate that ByteFormer has applications in privacy-preserving inference. ByteFormer is capable of performing inference on particular obfuscated input representations with no loss of accuracy. We also demonstrate ByteFormer's ability to perform inference with a hypothetical privacy-preserving camera which avoids forming full images by consistently masking 90% of pixel channels, while still achieving 71.35% accuracy on ImageNet.

https://arxiv.org/abs/2306.00238
Optimization happens inside the mind, not in the world
by azsantosk — 2023-06-03T21:36:43.633Z

Optimization happens inside the mind (map), not in the world (territory). Reflecting about this made me noticeably less confused about how powerful AI agents capable of model-based planning and self-modification will act.

My model of future AI

An AI that is a powerful optimizer will probably do some kind of model-based planning. It will search on its world-model for more powerful actions or moves.

Because the real-world is very complicated, the real state and decision spaces will now be known. Instead, the AI will probably create a generative model G that contain states and actions at different levels of abstraction, and that can sample states and rewards s′,r←G(s,a).

It may have a strong intuition, or generative policy network p(s), that can be sampled as a generative model of its own actions.

But it will probably also have an amplification algorithm leading to a more powerful search policy π(s). Similar to AlphaGo, which runs a MCTS algorithm using its neural network policy and the transition probabilities for the game to generate a distribution of stronger moves, a powerful AI may also run some kind of search algorithm using its generative policy network p and its generative world-model G to come up with stronger actions, represented by its search policy π(s).

Powerful search with a perfect world-model

The search algorithm is what lets the AI think for longer, getting potentially more expected rewards the more compute power it uses. It is also a potential source of self-improvement for the intuition policy p, which can be self-trained to maximize similarity to π.

If we assume that the generative world-model G contains a perfect abstraction and simulation of the world, then this search process is going to optimize the world itself, and if G happens to return r based on how many more paperclips are in s′ compared to s, then the AI agent, with a powerful enough search algorithm and with enough computing power, will kill everyone and turn the entire light-cone into paperclips.

However, the abstractions may be inadequate, and under given abstractions the predicted rewards and world-model transitions from G may be quite distinct from the real reward and state transitions, and for a given G this is more likely to happen in practic...
What would a post that argues against the Orthogonality Thesis that LessWrong users approve of look like?
by Thoth Hermes — 2023-06-03T21:21:48.602Z

This question is motivated by the following reasons:

Not many pieces exist that argue against the Orthogonality Thesis (on LessWrong, or anywhere, to my knowledge). Of those that do, none have received positive feedback.

Commenters on those pieces have stated that it is not, in principle, impossible that they would up-vote such a piece, only that none thus far have met or exceeded their standards for what they would consider to be a successful attempt (even if they were not ultimately persuaded by the arguments).

What attributes would a "successful attempt" (even one that does not persuade you to disbelieve the Orthogonality Thesis) have?
A Double-Feature on The Extropians
by Maxwell Tabarrok — 2023-06-03T18:27:47.429Z

Link-post for two pieces I just wrote on the Extropians.
The Extropians were an online group of techno-optimist transhumanist libertarians active in the 90s who influence a lot of online intellectual culture today. Prominent members include Eliezer Yudkowsky, Nick Bostrom, Robin Hanson, Eric Drexler, Marvin Minsky and all three of the likely candidates for Satoshi Nakamoto (Hal Finney, Wei Dai, and Nick Szabo).
The first piece is a deep dive into the archived Extropian forum. It was super fun to write and I was constantly surprised about how much of the modern discourse on AI and existential risk had already been covered in 1996.

The second piece is a retrospective on predictions made by Extropians in 1995. Eric Drexler, Nick Szabo and 5 other Extropians give their best estimates for when we'll have indefinite biological lifespans and reproducing asteroid eaters.
What exactly does 'Slow Down' look like?
by Steve M — 2023-06-03T18:11:33.785Z

I see a lot of posts regarding regulation, usually these involve some sort of phrase like 'Slow Down' or 'Shut Down'. Pragmatically, what does that look like? Is there an actual draft of a proposed bill somewhere that anyone who had the ear of a Senator could point to and say "do that"?

If that does not exist, shouldn't getting a single draft of the bill which (even if imperfect) everyone agrees to support as a first step, be the top priority?
Announcing AISafety.info's Write-a-thon (June 16-18) and Second Distillation Fellowship (July 3-October 2)
by steven0461 — 2023-06-03T02:03:01.364Z

Stampy’s

AI Safety Info

is an interactive FAQ started by

Rob Miles

that aims to be the best one-stop source of information about AI existential safety, gathering summaries and links on each of hundreds of subtopics.

Now that the

first fellowship

is nearing an end, we’re preparing for a second distillation fellowship from July 3rd through October 2nd. Distillation fellows will be working with each other and with me to improve the site further by editing the existing answers and writing new ones. They’ll be paid $2500 per month for three months working full-time as remote contractors. (Partial fellowships will probably be possible as well.) We’ll have at least five spots for fellows, and maybe more.

We’re also announcing a

hackathon-style event

, which we’re organizing together with

Apart Research

, on the 16th, 17th, and 18th of June. People are invited to write and edit articles during whichever parts of those days best fit their schedules. (If you’re unable to attend on those days, you can still participate by choosing another contiguous three-day period between now and the event—for example, next weekend—to write all your entries in.)

Collaboration on the event will take place on

Discord

as well as on

gather.town

. I’ll be online for those three days to lead the event and answer any questions. We encourage people to work together on entries during that time, but we encourage one individual to be in charge of each entry.[1]

Submitting good articles and helping other people write their articles in the write-a-thon will be the main criteria we’ll use to select distillation fellows. There will also be prize money for the best entries: $1000, $600, $300, and $100 for the best four, and $200 for one randomly chosen (serious) entry.

We will have a more specific list of suggested questions to use for entries later, but for now, you can look at the “Not started” part of Stampy’s

list of questions

, and take a look at existing articles and the

style guidelines

for what an article should look like. (You can also use a new question of your own devising; to make sure you’re not wasting your effort on a question that won't fit on the site, you can

ask on Discord

or check with me.)

We wanted to put this announcement up without waiting longer, but we’re still finalizing some of the details, so chec...
Terry Tao is hosting an "AI to Assist Mathematical Reasoning" workshop
by junk heap homotopy — 2023-06-03T01:19:08.398Z
Link: https://terrytao.wordpress.com/2023/06/02/ai-to-assist-mathematical-reasoning-a-workshop/

The workshop is on 12-14 June, following a previous workshop on "Machine-assisted proofs" held last Feb.

Maybe an opportunity to nerd snipe him and his high-powered colleagues on alignment? This event might mark a turning point in his thinking about it as well since Yann LeCun is part of the organising committee—which also entails much care lest a half-baked attempt inoculates him from the entire field.

Anyone who knows him or is otherwise in UCLA who can weigh in on the feasibility of this?