trentmkelly/LessWrong-43k huggingface dataset:
understanding bureaucracy
by dhruvmethi — 2024-10-17T17:55:27.336Z

Successful organizations maximize the formula shown below:

Meaningful OutputResources Spent

Meaningful output: meaningful output is any output aligned with the organization’s purpose. For example, Feed My Starving Children’s purpose is to deliver food to impoverished areas. A meaningful output for them is a successful food delivery.
Resources spent: this is mainly just time and money.

As an organization, you have two levers to improve: increase meaningful output while using the same amount of resources, or maintain the same meaningful output while decreasing the amount of resources spent. When companies hire people, they’re hoping that their meaningful output will increase far more than the increase in cost, and when companies conduct layoffs, they’re hoping that their meaningful output will reduce much less than their resources.

Few things frustrate me more than bureaucratic organizations. They completely butcher the formula above. Within the mess of systems, committees, and processes, the production of meaningful output becomes impossible, and sustaining this complexity requires an immense amount of resources.

The worst part is that they’re extremely static and difficult to change. This is because:

Inertia - the size and complexity of bureaucracy create tremendous inertia behind the status quo. It takes an overwhelming amount of energy to both map it and change it in a positive way. Few people are willing to take the plunge, and fewer power brokers within the bureaucracy will support any change because it would threaten their position.
Illusory work - the effort required to survive within the bureaucracy leaves little room and energy to change it. Here’s an example from Google:

Google has 175,000+ capable and well-compensated employees who get very little done quarter over quarter, year over year. Like mice, they are trapped in a maze of approvals, launch processes, legal reviews, performance reviews, exec reviews, documents, meetings, bug reports, triage, OKRs, H1 plans followed by H2 plans, all-hands summits, and inevitable reorgs. The mice are regularly fed their “cheese” (promotions, bonuses, fancy food, fancier perks) and despite many wanting to experience personal satisfaction and impact from their work, the system trains them to quell these inappropriate desires and learn what it actually means to be “Googley” — just don’t rock the boat. — source

Organization...
AI #86: Just Think of the Potential
by Zvi — 2024-10-17T15:10:06.552Z

Dario Amodei is thinking about the potential. The result is a mostly good essay called Machines of Loving Grace, outlining what can be done with ‘powerful AI’ if we had years of what was otherwise relative normality to exploit it in several key domains, and we avoided negative outcomes and solved the control and alignment problems. As he notes, a lot of pretty great things would then be super doable.

Anthropic also offers us improvements to its Responsible Scaling Policy (RSP, or what SB 1047 called an SSP). Still much left to do, but a clear step forward there.

Daniel Kokotajlo and Dean Ball have teamed up on an op-ed for Time on the need for greater regulatory transparency. It’s very good.

Also, it’s worth checking out the Truth Terminal saga. It’s not as scary as it might look at first glance, but it is definitely super far out.

Table of Contents

Introduction.

Table of Contents.

Language Models Offer Mundane Utility. More subscriptions means more utility.

Language Models Don’t Offer Mundane Utility. Then again, neither do you.

Deepfaketown and Botpocalypse Soon. Quality remains the limiting factor.

They Took Our Jobs. But as Snow Crash foretold us, they’ll never take our pizza.

Get Involved. UK AISI hiring technical advisor, Tarbell Grants for AI reporting.

Introducing. Grok 2 gets a proper API.

In Other AI News. It’s time to go nuclear.

Truth Terminal High Weirdness. When the going gets weird, the weird turn pro.

Quiet Speculations. Are the labs holding back?

Copyright Confrontation. New York Times sends a cease and desist to Perplexity.

AI and the 2024 Presidential Election. Very briefly getting this out of the way.

The Quest for Sane Regulations. A proposal all reasonable people should agree on.

The Week in Audio. Matt Stone asks, is all Sam Altman does go on podcasts?

Just Think of the Potential. They could be machines of loving grace.

Reactions to Machines of Loving Grace. Much agreement, some notes of caution.

Assuming the Can Opener. I would very much like a can opener.

Rhetorical Innovation. People often try to convince you that reason is impossible.

Anthropic Updates its Responsible Scaling Policy. New and improved.

Aligning a Smarter Than Human Intelligence is Difficult. Are you smart enough?

The Lighter Side. The art of the possible.

Language Models Offer Mundane Utility

...
Concrete benefits of making predictions
by Jonny Spicer — 2024-10-17T14:23:17.613Z
Link: https://fatebook.io/blog/concrete-benefits-of-making-predictions

Your mind is a prediction machine, constantly trying to anticipate the world around you and altering its forecasts based on new information. It’s always doing this as a background process. But what would happen if you deliberately trained this skill? Could you get better at predicting your projects, your life, and the future?

Sadly, I don’t have a crystal ball, but I have found trying to see the future valuable. I can't be certain about what will happen, but by deliberately practising the skill of prediction, I can create feedback loops that allow me to better anticipate what the future will bring. This has indeed helped me become happier, better to interact with, and more impactful.

Concretely, making probabilistic predictions has helped me:

Improve my productivity
Make better decisions about my career and living situation
Set and make progress towards my goals
Prioritise work tasks
Communicate project timelines and uncertainties
Keep myself accountable
Manage anxiety
Understand my relationships better
Improve my metacognition, along with related skills

I’m building Fatebook to make it as quick and easy as possible for you to harness the power of prediction and reap some of the benefits above, while remaining completely free. The first step to getting value from predictions is to figure out what to make those predictions about, which I’ve had plenty of time to tinker with.

Which questions are worth asking?

Practice and creativity have allowed me to get more value out of the questions I ask, and the predictions I make. For example, it has helped me communicate with my team better around project timelines, hiring decisions and strategy prioritization. We share predictions with one another using Fatebook for Slack on questions like:

This question allows me to set expectations very accurately with my team. If writing the blog post is taking longer than expected, I can decrease my prediction that it’ll be done by Friday, which is a low-friction way of letting others know about my updated timelines.

Sharing predictions makes it much easier to reach consensus about what work should be top priority. If we’re aiming to grow and my team all predict that implementing the “sign up with email” feature is more likely to get us 500 new users, then it’s obvious we should do it first. Or, if the probability of either feature attracting new users is low, that’s...
Arithmetic is an underrated world-modeling technology
by dynomight — 2024-10-17T14:00:22.475Z
Link: https://dynomight.net/arithmetic/

Of all the cognitive tools our ancestors left us, what’s best? Society seems to think pretty highly of arithmetic. It’s one of the first things we learn as children. So I think it’s weird that only a tiny percentage of people seem to know how to actually use arithmetic. Or maybe even understand what arithmetic is for. Why?

I think the problem is the idea that arithmetic is about “calculating”. No! Arithmetic is a world-modeling technology. Arguably, it’s the best world-modeling technology: It’s simple, it’s intuitive, and it applies to everything. It allows you to trespass into scientific domains where you don’t belong. It even has an amazing error-catching mechanism built in.

One hundred years ago, maybe it was important to learn long division. But the point of long division was to enable you to do world-modeling. Computers don’t make arithmetic obsolete. If anything, they do the opposite. Without arithmetic, you simply can’t access a huge fraction of the most important facts about the world.

The magic lives in a thing called “units”.

Chimps

It’s amazing how much we don’t know about nutrition. For example, would you live longer if you ate less salt? How much longer? We can guess, but we don’t really know.

To really be sure, we’d need to take two groups of people, get them to eat different amounts of salt, and then see how long they live. This is expensive, ethically fraught, and runs into the problem that when you tell people to eat differently, they usually ignore you.

So I’ve often wondered: Why don’t we do these experiments on animals? Why not get two big groups of chimpanzees, and feed them different amounts of salt? Chimps aren’t people, but it would tell us something, right?

Why don’t we do this? Because arithmetic.

How much would such a study cost? To figure this out, you will need three numbers:

The average lifespan of a chimp.
The cost to maintain one chimp in a research facility for one day.
The number of chimps you’d need for a study.

Let’s do these. First, how long do chimps live? In captivity the average seems to be around 36.3 years. (Incidentally, female chimps seem to live 25% longer than males—imagine human women lived until 90 while men died at 71.)

Second, how much does it cost to maintain a chimp? Capaldo et al. looked at the average costs in various research facilities in the US in 2009. They estimate around $75/day (in 2024 dollars).

Finally, how man...
The Computational Complexity of Circuit Discovery for Inner Interpretability
by Bogdan Ionut Cirstea — 2024-10-17T13:18:46.378Z
Link: https://arxiv.org/abs/2410.08025 

Authors: Federico Adolfi, Martina G. Vilas, Todd Wareham.

Abstract:

Many proposed applications of neural networks in machine learning, cognitive/brain science, and society hinge on the feasibility of inner interpretability via circuit discovery. This calls for empirical and theoretical explorations of viable algorithmic options. Despite advances in the design and testing of heuristics, there are concerns about their scalability and faithfulness at a time when we lack understanding of the complexity properties of the problems they are deployed to solve. To address this, we study circuit discovery with classical and parameterized computational complexity theory: (1) we describe a conceptual scaffolding to reason about circuit finding queries in terms of affordances for description, explanation, prediction and control; (2) we formalize a comprehensive set of queries that capture mechanistic explanation, and propose a formal framework for their analysis; (3) we use it to settle the complexity of many query variants and relaxations of practical interest on multi-layer perceptrons (part of, e.g., transformers). Our findings reveal a challenging complexity landscape. Many queries are intractable (NP-hard, \sigma_2^p hard), remain fixed-parameter intractable (W[1]-hard) when constraining model/circuit features (e.g., depth), and are inapproximable under additive, multiplicative, and probabilistic approximation schemes. To navigate this landscape, we prove there exist transformations to tackle some of these hard problems (NP- vs. \sigma_2^p-complete) with better-understood heuristics, and prove the tractability (PTIME) or fixed-parameter tractability (FPT) of more modest queries which retain useful affordances. This framework allows us to understand the scope and limits of interpretability queries, explore viable options, and compare their resource demands among existing and future architectures.

Seems like bad news for enumerative interp agendas.
Is there a known method to find others who came across the same potential infohazard without spoiling it to the public?
by hive — 2024-10-17T10:47:05.099Z

Say (in the extreme case) you are Einstein in the moment where he realized that E=mc², and you think so far ahead that you can imagine the nuclear bomb. You don't know if it is possible to build, but you decide to keep your work secret. As the field of physics advances, other will come across the same insight. How can these isolated, silent individuals find the others without going public about it? Is there some kind of shelling point they can construct without knowing of each other?

Edit: Just to be clear and avoid confusing; I don't have any such potentially dangerous insight. I was thinking in a more general way about AI policy. Advocating for an agreement that no ones creates an AI with property X, would just be an invitation to try building an AI with property X. Just like you wouldn't read aloud a list with bad words in preschool. That got me thinking that there should be a way to check with others, before publishing.
It is time to start war gaming for AGI
by yanni kyriacos — 2024-10-17T05:14:17.932Z

In this episode of the Making Sense podcast with Sam Harris, Barton Gellman from The Brennan Center For Justice discusses how he "organized five nonpartisan tabletop exercises premised on an authoritarian candidate winning the presidency to test the resilience of democratic institutions".
"The 175 participants across five exercises were Republicans, Democrats, and independents; liberals, conservatives, and centrists. They included veterans of the first Trump administration and previous administrations of both parties. "
This seems like an extremely valuable exercise when trying to prepare for long-tail risks.
--------------

I often think about about this post. It asks the seriously neglected question: Why was the AI Alignment community so unprepared for this moment?
I think we're going to get competent Digital Agents soon (< 2 years). I think when they arrive, unless we work urgently, we will again feel like we were extremely unprepared.
I'd like to see either a new AI Safety organisation created to run these exercises with key decision makers (e.g. Government, Industry, maybe Academia), or have an existing org (CAIS?) take on the responsibility.
Every morning we should be repeating the mantra: there are no parents in the room. It is just us.
--------------
More here on the program:
"In May and June 2024, the Brennan Center organized five nonpartisan tabletop exercises premised on an authoritarian candidate winning the presidency to test the resilience of democratic institutions. The antidemocratic executive actions explored in the scenarios were based on former President Donald Trump’s public statements about his plans for a potential second term in office.

We do not predict whether Trump will win the November election, and we take no position on how Americans should cast their votes. What we have done is simulated how authoritarian elements of Trump’s agenda, if he is elected, might play out against lawful efforts to check abuses of power.

The 175 participants across five exercises were Republicans, Democrats, and independents; liberals, conservatives, and centrists. They included veterans of the first Trump administration and previous administrations of both parties.

Among them were former governors, former cabinet members, former state attorneys general, former members of the House and Senate, retired flag and general officers, labor leaders, faith l...
Reinforcement Learning: Essential Step Towards AGI or Irrelevant?
by Double — 2024-10-17T03:37:04.635Z

A friend of mine thinks that RL is a dead end: LLMs are much better at problem solving, exploration, and exploitation than any RL algorithm. And I agree that LLMs are better than RL on RL's tasks: companies even have LLMs controlling robots nowadays.
The part where we disagree is that I see RL as the step that goes beyond LLMs. LLMs can only consume so much data, and get so good at predicting the next word. At some point, they will predict exactly what an expert would say, and then they will be exactly expert level (except much faster and more scaleable).
If you combine a bunch of LLMs into some organizational structure, then you at best get a company of experts. [1]

But to create a system beyond the capabilities of LLMs, either created by humans or expert-level LLMs, the training will need to go beyond human-generated data. It will need to intelligently explore the world to test its hypotheses and improve its mind. Essentially, in order for LLMs to go beyond larping human experts, they need RL.

In Project Lawful, Eliezer's smart character made a remark that there are some hypotheses that are impossible (computationally intractable) to learn passively from data (as LLMs do), but are possible to learn actively (as RL can). I couldn't find that proof, and would appreciate if someone could find it.[2]

And if I'm wrong and there are intelligence amplification methods for LLMs that do not involve RL, I'd like to know I'm wrong.

^

The expert-level or the expert-organization AI may be enough to kickstart an intelligence explosion, but the point I'm trying to make stands.

^

The work of Dana Angluin might be a starting point for a search.
The Cognitive Bootcamp Agreement
by Raemon — 2024-10-16T23:24:05.509Z

The Cognitive Bootcamp is intended to be fairly intense intense. I ask attendees to read through the Workshop agreement and a) doublecheck that the workshop is right for them b) let me know how we might want to tailor it to meet their needs.

The target audience is people who:

have decision-making power on a large project, which tackles confusing problems;
are not bottlenecked on executive function;
and, who don’t have a sneaking suspicion they’re at risk of burnout.

The commitment I’m asking if you come is that you spend basically the whole time either:

aiming to think basically as hard as you can.
napping / taking a walk / etc (preferably without devices).
talking to me if the workshop feels off in some way.

Beforehand, you’ll send me your default plan for the next ~week, month or quarter (whatever the longest timescale you plan on). You’ll work on improving this plan.

Your primary goal, if you come, should be to learn at least one new skill, starting from "it’s too cumbersome to use productively", and aiming to reach "Juuuust fluent enough that you can start applying it to your day job, practicing so it becomes easy.”

I’ll present ~4 skills I think are valuable for solving confusing problems, and exercises that are helpful for grinding on them. You’re welcome to pick other skills, or other exercises, that you think will help you better. But, for each session, you need to clearly explain a) why I think the original exercise was important, b) why the thing you’ll do instead is better. (We’ll chat until we both understand each other and feel good about it)

Previously, I’ve found that giving people too much leeway in changing the curriculum results in the workshop losing focus, but too little leeway results in people not quite getting what they need. This is the middle ground that I've found works best.

The Goal: Solve confusing, intractable problems

Many of the most important problems in the world are confusing, intractable, and seem impossible. They are really important to solve anyway.

But many people who try, end up in failure modes such as…

Rabbit-holing, fixating on an approach which doesn’t work, or takes too long.
Goodharting, substituting an easier problem, maybe without even noticing.
Despair. It’s impossible. You give up.
Zombified Agency, mechanically execute virtuous-seeming plans, but something inside you is dead and hollow and the plan p

...
Bitter lessons about lucid dreaming
by avturchin — 2024-10-16T21:27:04.725Z

The amount of effort is not proportional to the result. One lucid dream (LD) can take hours or even dozens of hours of effort. On average, a practitioner experiences several dozen LDs in their lifetime before quitting. If they don't quit, they dedicate their entire life to it, day and night, trying endless techniques, practicing reality checks, etc.
All techniques are absolutely useless compared to the effectiveness of galantamine. The effectiveness of galantamine is directly proportional to its dosage. Thus, 16 mg will almost certainly send you into a powerful lucid dream or out-of-body experience. Therefore, all techniques like WILD are absolutely meaningless.
Non-lucid dreams are often more interesting than lucid ones. In a lucid dream, I find myself as my usual self, just in some virtual reality, which is only slightly more interesting than a computer game or watching a movie. But in a non-lucid dream, I magically transform into someone else or find myself in a different, impossible, mysterious world.
Lucid dreams are easiest to practice between the ages of 20-30, but during this same period of life, all other possible forms of entertainment are also most accessible. I don't actually know of cases where lucid dreams turned out to be more useful than regular entertainment. It's more like a lottery: you either get a lucid dream or you don't. There are very few cases where people actually practiced something or learned something interesting in them.
There are exceptions: some people, like M, always have lucid dreams. But this indicates that their brain is simply wired differently. The bitter lesson is that some people's brains are just "hardwired" so that they can always be aware of themselves in dreams, most often these are women. Whereas men are usually more interested in LDs.
In a sense, the main dream of lucid dreaming enthusiasts is some kind of erotic adventure. Eroticism in LDs rarely succeeds. But in the end, when the dream ends, all this virtual reality disappears, leaving the same feeling as after watching porn.Good onanism with great fantasy may be a better alternative with guaranteed satisfaction.
Almost all effects of LD may be achieved in active imagination sessions: daydreaming visualizations which are not guided, but in which you allow your subconscious to drive the process..
Illusion of permanence: Even if you've learned to induce lucid dreams reg

...