trentmkelly/LessWrong-43k huggingface dataset:
The man and the tool
by pedroalvarado — 2022-11-25T19:51:15.511Z

The man and the first tools

150,000 years ago there were Homo Sapiens very similar to us in Africa. However, they did not begin to dominate the world until about 70,000 years ago. Between 150,000 and 70,000 years ago, Homo Sapiens was just another animal that struggled to survive, a weak and clumsy animal, and not the animal that it is today, the dominant animal on planet Earth.

Between 70,000 and 30,000 years ago, Homo Sapiens started making cool things like boats, oil lamps, bows and arrows, and needles. These achievements are thought to have been the product of a revolution in the cognitive abilities of Sapiens. There are different theories of the cause of what is known as the cognitive revolution. Although it is interesting to investigate the causes of the cognitive revolution, it is also interesting to analyze its consequences.

The creation and use of tools by humans did not begin 70,000 years ago. The human being and his ancestors have been creating and using tools for millions of years. The first tools used by our human ancestors were stones, and this fact marked the beginning of what we know as the Stone Age. The Stone Age began approximately 3 million years ago and ended when metals were discovered and used. Stone tools were made by gradually grinding away stones to make hammers, spear and arrow points, knives, and scrapers.

The progress of the tools of the human being is evident, 3 million years ago we had stones and now we have rockets and spaceships. It seems that there is a correlation between the tools of man and the evolution of man. The evolution of the human produced tools and the production of tools made the human progress. The use of tools leverages us, amplifies our capabilities. The tools are what we currently call technology.

A scene from a movie that I find fascinating about the tools of man is from the movie 2001: A Space Odyssey. I am fascinated by the contrast it makes between bone, a primitive tool, and a spaceship, the pinnacle of human ingenuity. The change of scene between the bone and the spaceship symbolizes the millions of years of evolution of man and his tools.

Scene of 2001: A Space Odyssey

In the world there are many animals that are far superior to us physically speaking. For example, we are not very fast running, we do not have claws, we cannot fly, we do not have a good sense of smell, we do not have protective armor, we are ...
Mechanistic anomaly detection and ELK
by paulfchristiano — 2022-11-25T18:50:04.447Z

(Follow-up to Eliciting Latent Knowledge. Describing joint work with Mark Xu. This is an informal description of ARC’s current research approach; not a polished product intended to be understandable to many people.)

Suppose that I have a diamond in a vault, a collection of cameras, and an ML system that is excellent at predicting what those cameras will see over the next hour.

I’d like to distinguish cases where the model predicts that the diamond will “actually” remain in the vault, from cases where the model predicts that someone will tamper with the cameras so that the diamond merely appears to remain in the vault. (Or cases where someone puts a fake diamond in its place, or…)

(ELK images by María Gutiérrez-Rojas)

One approach to this problem is to identify (the diamond remains in the vault) as the “normal” reason for the diamond to appear on camera. Then on a new input where the diamond appears on camera, we can ask whether it is for the normal reason or for a different reason.

In this post I’ll describe an approach to ELK based on this idea and how a the same approach could also help address deceptive alignment. Then I’ll discuss the empirical and theoretical research problems I’m most excited about in this space.

ELK and explanation

Explanations for regularities

I’ll assume that we have a dataset of situations where the diamond appears to remain in the vault, and where that appearance is always because the diamond actually does remain in the vault. Moreover, I’ll assume that our model makes reasonable predictions on this dataset. In particular, it predicts that the diamond will often appear to remain in the vault.

“The diamond appears to remain in the vault” corresponds to an extremely specific pattern of predictions:

An image of a diamond is a complicated pattern of millions of pixels.
Different cameras show consistent views of the diamond from different angles, suggesting that there is a diamond “out there in the world” being detected by the cameras.
The position and physical characteristics of the diamond appear to be basically constant over time, suggesting that it’s “the same diamond.”

In one sense the reason our model makes these predictions is because it was trained to match reality, and in reality the camera’s observations have these regularities. (You might call this the “teleological explanation.”)

But we could also ignor...
The Least Controversial Application of Geometric Rationality
by Scott Garrabrant — 2022-11-25T16:50:56.497Z

I have been posting a lot on instrumental geometric rationality, with Nash bargaining, Kelly betting, and Thompson sampling. I feel some duty to also post about epistemic geometric rationality, especially since information theory is filled with geometric maximization. The problem is that epistemic geometric rationality is kind of obvious.

A Silly Toy Model

Let's say you have some prior beliefs P0 at time 0, and you have to choose some new beliefs P1 for time 1. P0,P1∈ΔW are both distributions over worlds. For now, let's say you don't make any observations. What should your new beliefs be?

The answer is obvious, you should set P1=P0. However, if we want to phrase this as a geometric maximization, we can say

P1=argmaxP∈ΔWGw∼P0P(w).

This is saying, imagine the true world is sampled according to P0, and geometrically maximize the probability you assign to the true world. I feel silly recommending this because it is much more complicated that P1=P0. However, it gives us a new lens that we can use to generalize and consider alternatives.

For example, we can consider the corresponding arithmetic maximization,

P1=argmaxP∈ΔWEw∼P0P(w).

What would happen if we were to do this? We would find the world with the highest probability, and put all our probability mass on that world. We would anticipate that world, and ignore all the others.

This is a stupid way to manage our anticipation. Nobody is going around saying we should arithmetically maximize the probability we assign to the true world. (However, people are going around saying we should arithmetically maximize average utility, or arithmetically maximize our wealth.)

Not only does arithmetic maximization put all our anticipatory eggs in one basket, it also opens us up to all sorts of internal politics. If we take a world and add some extra features to it to split it up into multiple different worlds, this changes the evaluation of which world is most efficient to believe in. This is illustrating two of the biggest virtues of geometric rationality: proportional representation, and gerrymander resistance.

A Slightly Less Silly Toy Model

Now, lets assume we make some observation X⊆W, so we know that the real world is in X. We will restrict our attention to probability distributions that assign probability 1 to X. However, if we try to set

P1=ar...
Planes are still decades away from displacing most bird jobs
by guzey — 2022-11-25T16:49:32.344Z

Originally published here: https://guzey.com/ai/planes-vs-birds/

Note: Parts of this essay were written by GPT-3, so it might contain untrue facts.

Introduction

Many of my friends are extremely excited by planes, rockets, and helicopters. They keep showing me videos of planes flying at enormous speed, rockets taking off from the ground while creating fiery infernos around them, and of helicopters hovering midair seemingly denying the laws of gravity.

I've been on a plane already, and it was nothing special. It was just a big metal tube with a bunch of people inside. It was loud and it smelled weird and I had to sit in a tiny seat for hours. So what is it that makes planes so special? Is it the fact that they're machine? Is it the fact that they're big? Is it the fact that they cost a lot of money?

Here's the thing: all human-built artificial flight (AF) machines are incredibly specialized and are far away from being able to perform most of the tasks birds -- the only general flight (GF) machines we are aware of -- can perform.

More than 200 years after hot air balloons became operational and more than 100 years after the first planes flew, it's clear that building a GF machine is much harder than anticipated and that we are nowhere close to reaching bird-level abilities.

1. Planes vs eagles

First, take a look at this video of an eagle catching a goat, throwing it off a cliff, and then feasting on it:

I haven't ever seen a plane capable of catching a live animal and deliberately throwing it off a cliff. Not in 1922, not in 2022. Not even a tech demo. Such a feat vastly exceeds the abilities of any planes we have built, however fast they can fly.

2. Planes vs cuckoos

Second, let's watch this video of a cuckoo chick ejecting the eggs of its competitors out of a nest:

You could say that this ability has nothing to do flight but, again, this misses the forest for the trees. Building a GF machine is not about Goodharting random "flight" benchmarks by flying high and fast, it's about real-world performance on tasks GF machines created by nature are capable of. And, however impressive planes are, as soon as we try to see how well they perform in the real-world, they can't even match a cuckoo chick.

3. Planes vs a hummingbirds

Third and final example. Take a look at the hummingbird's amazing ability to maintain stability in the harshest aerial conditions:

Take any plane we have bui...
Take part in our giant study of cognitive abilities and get a customized report of your strengths and weaknesses!
by spencerg — 2022-11-25T16:28:52.538Z
Link: https://www.guidedtrack.com/programs/5n1j78u/run?source=lesswrongandeaforum

If you've ever wondered what your cognitive strengths and weaknesses are, you may want to take part in our giant scientific study of cognitive abilities! We are testing 40 claims in the academic literature simultaneously, and give you a customized report at the end showing what you're good and less good at!
It requires a desktop or laptop computer (doesn't work on mobile) and at least 10 minutes of uninterrupted time - but the longer you go, the more detailed analysis you'll get about yourself at the end!
Click here to take part now and get your results:

https://www.guidedtrack.com/programs/5n1j78u/run?source=lesswrongandeaforum
Thanks for participating!
Guardian AI (Misaligned systems are all around us.)
by Jessica Rumbelow — 2022-11-25T15:55:43.939Z

Work done @ SERI-MATS, idea from a conversation with Ivan Vendrov at Future Forum earlier this year.

Misaligned systems are all around us. They are what make me watch another video of a man in filthy shorts building a hut using only tools made from rocks and his own armpit hair. And the reason I have never, ever watched a single episode of Flavourful Origins in isolation. Maybe they make you mindlessly seek cat gifs, or keep you scrolling twitter in a cosy fug of righteous indignation long after you should be asleep. They could also be the reason your uncle is a bit more xenophobic now than he used to be. A bit more dismissive of the ‘snowflakes’. A bit more superior, and less kind.

None of this is new, of course. Advertising and propaganda have been around for a long time. But it feels different now – or at least it does to me. I was born in 1990 – I haven’t got a chance. TikTok’s fabulous algorithm steamrollers my brain. I look up, glazed after an undetermined period of involuntary consumption, wondering what happened and why I feel so hollow. I had to delete the app.

Maybe this isn’t you. I’m prone to flow states of both the deeply productive and malign kinds: good at losing myself in a task, terrible at multitasking. But surely, no-one really looks back at a four hour TikTok binge and thinks “that was an excellent use of my time”.

(This isn’t to say that personalised recommender systems or targeted ads or compelling news feeds are bad per se – just that they’re probably not very well aligned with your longer term goals.)

So, probably none of the AI systems in your life are optimising hard for your flourishing right now. Money, yes; attention, yes; but long term, real happiness? Probably not.

(Though, I guess there are things like AI-powered fitness or habit forming apps, etc?)

Could we change that? I’ve been kicking an idea around for a while. It’s not very well formed yet, but it’s something like making wrappers around existing algorithms to shift the optimisation objective. Sick of twitter bickering? A guardian AI could adjust your news feed to give you content that’s more satisfying and informative, and less likely to drag you into pointless arguments. Want to refocus your YouTube recommendations to give you great maths lecture content, without being derailed by popular science videos? Or, stay down with the kids on TikTok without get...
Intuitions by ML researchers may get progressively worse concerning likely candidates for transformative AI
by Viktor Rehnberg — 2022-11-25T15:49:29.426Z

Epistemic status: Explorative. See results more as a sketch towards a possible issue than a proper derivation of reliable values. Research behind exact numbers is limited and calculations are without error propagation. I'd rather get the idea out there than wait until I find the time to do it properly.

This post will sketch at a concern I had recently but also point out one reasons for why it isn't as bad as it may seem.

Model

In short, the insight comes from the fact that amount spent training the largest ML systems scales faster than the spending the median ML researcher spends on training their systems. Thus giving most ML researchers a more and more skewed view of how the best ML systems behave.

From current compute trends we have either a 10 month doubling time or a 6 month doubling time. To translate this to increase in funding for these systems we will have to estimate how the cost of compute changes over time.

As usual we can consider Moore's law with a 24 month doubling time. We can also consider recent trends in price of GPUS per FLOP/s which suggest that for half precision, the amount of FLOP/$ has a doubling time of 18 months[1].

This would mean that funding for state of largest ML systems is doubling at least every 22 months as

210−218≈222.

Now the question that remains is how most ML researchers spending on training ML systems scale. This I don't have any good numbers. Though, I found some old data that suggests that US national funding towards Information and Intelligent Systems (the relevant subgroup) has doubled about every 22 months. Which would mean that the same scaling factor applies as for the largest models.

However, I doubt that the number of ML researchers have remained constant during this time. In fact the number of peer-reviewed AI publications on the topic of AI seem to have doubled every 25 months[2]. While it could be that the ML researchers are simply increasing the amount of published articles exponentially, I expect that it is rather because the amount of researchers are increasing. If the amount of ML researchers double every 30 months[3], then this would mean that the amount of spending per ML researcher instead doubles every 80-90 months or so.

This would mean that the largest ML models will double compared to the size of models used by ML researchers every 30 months. And experiences by ML researchers will grow exponentially furt...
Refining the Sharp Left Turn threat model, part 2: applying alignment techniques
by Vika — 2022-11-25T14:36:08.948Z
Link: https://vkrakovna.wordpress.com/2022/11/25/refining-the-sharp-left-turn-threat-model/

A

Sharp Left Turn

(SLT) is a possible rapid increase in AI system capabilities (such as planning and world modeling). This post will outline our current understanding of the most promising plan for getting through an SLT and how it could fail (conditional on an SLT occurring).

In a

previous post

, we broke down the SLT threat model into 3 claims:

Capabilities will generalize far (i.e. to many domains)
Alignment techniques that worked previously will fail during this transition
Humans can’t intervene to prevent or align this transition

We proposed some possible mechanisms for Claim 1, while this post will investigate possible arguments and mechanisms for Claim 2.

Plan: we use alignment techniques to find a goal-aligned model before SLT occurs, and the model preserves its goals during the SLT.

We can try to learn a

goal-aligned

model before SLT occurs: a model that has beneficial goals and is able to reason about its own goals. This requires the model to have two properties: goal-directedness towards beneficial goals, and situational awareness (which enables the model to reason about its goals). Here we use the term "goal-directedness" in a weak sense (that includes humans and allows incoherent preferences) rather than a strong sense (that implies expected utility maximization).

One can argue that the goal-aligned model has an incentive to preserve its goals, which would result in an aligned model after SLT. Since preserving alignment during SLT is largely outsourced to the model itself,

arguments

for alignment techniques failing during an SLT don't imply that the plan fails (e.g. it might be fine if interpretability or ELK techniques no longer work reliably during the transition if we can trust the model to manage the transition).

Step 1: Finding a goal-aligned model before SLT

We want to ensure that the model is goal-oriented with a beneficial goal and has situational awareness before SLT. It's important that the model acquires situational awareness at the right time: after it acquires beneficial goals. If situational awareness arises when the model has undesirable goals, this leads to deceptive alignment.

Thus, our model search process would follow a decision tree along these lines:

If situational awareness is detected without goal-directedness, restart the search.
If undesirable goal-directedness or early sig

...
Fair Collective Efficient Altruism
by Jobst Heitzig — 2022-11-25T09:38:18.586Z

In this post, I propose to explore a novel methodology by which a collective of decision makers (simply called "we" here) such as the current generation of humanity could make an altruistic collective decision in the following type of situation:

The decision is about about a number of paths of actions (called "options" here) that have potentially vast consequences in the future and potentially generating quite different populations, such as deciding to colonise space.
There might be moral uncertainty about the moral status of some of the beings that might be affected by the decision, such as other species.
These beings (called "potential stakeholders" here) might be quite different from humans so that trying to estimate their subjective wellbeing and compute WELLBYs seems too speculative to base the decision on. It might also not be justified to assume that these beings are rational, have complete preferences or are even expected utility maximizers whose preferences can be encoded in a von-Neuman–Morgenstern utility function.

Epistemic status: Highly speculative but based on years of theoretical research into collective decision making.

The proposed methodology is based on the following ideas and rationale:

Even though a quantitative estimation of subjective wellbeing might not be possible, the members of the collective (called the "deciders" here) might be able to estimate what an affected being "would have preferred us to do" via an "empathy exercise" similar to what János (John) Harsányi assumed is possible to perform interpersonal comparisons of preferences (but without assuming von-Neumann–Morgenstern utility functions).
Since such empathetic preference estimations (EPEs) are still bound to be uncertain and somewhat speculative:

The number of different EPEs needed should be kept as low as possible.
They should mostly be about what the being's favourite action would be since for this one only needs to imagine what the being would do in our situation. The necessary EPEs should not be about pairs of lotteries of options (as would be required to estimate interpersonally comparable cardinal preferences such as WELLBYs), but they might have to be about pairs of options or about a comparison between a single option and a certain lottery of options. If the latter is necessary, the number of different lotteries used in the EPEs should be as low as possible.
The EPEs shoul

...
If humanity one day discovers that it is a form of disease that threatens to destroy the universe, should it allow itself to be shut down?
by Shmi — 2022-11-25T08:27:14.740Z

This idea is a sci-fi trope by now, but it kind of goes to the core of corrigibility, as I understand it. It is getting less sci-fi and closer to reality lately, with a potential single-goal super-intelligent AGI created by humans transforming the universe into something uniform and boring. So, assuming AI research shows that with high certainty humanity is bound to destroy the universe, or at least everything we currently value in it, does it make sense to voluntarily... stop and make room for less destructive processes?
Could a single alien message destroy us?
by Writer — 2022-11-25T07:32:24.889Z
Link: https://youtu.be/st9EJg_t6yc

Merely listening to alien messages might pose an extinction risk, perhaps even more so than sending messages into outer space. Our new video explores the threat posed by passive SETI and potential mitigation strategies.

Below, you can find the script of the video. Matthew Barnett, the author of this related post, wrote the first draft. Most of the original draft survives, but I've made significant restructuring, edits, deletions, and additions.

One day, a few Earthly astronomers discover something truly remarkable. They’ve pointed their radio telescopes at a previously unexplored patch in the night sky, and recorded a binary message that is too inexplicable to have come from any natural source. Curious about what the distant aliens have sent us, the scientists begin trying to decipher the message. After an arduous process of code-breaking, the scientists find that the message encodes instructions on how to build a device. Unfortunately, the aliens left no description about what the device actually does.

Excited to share their discovery with the world, the astronomers agree to publish the alien instructions on the internet, and send a report to the United Nations. Immediately, the news captivates the entire world. For once, there is indisputable proof that we are not alone in the universe. And what’s more: the aliens have sent us a present, and no one knows what its purpose might be.

In a breathtaking frenzy that surpasses even the Space Race of the 1960s, engineers around the world rush to follow these instructions, to uncover the secrets behind the gift the aliens have left for us.

But soon after, a horrifying truth is revealed: the instructions do not describe a cure to all diseases, or a method of solving world hunger. Rather, the aliens have sent us explicit, and easy to follow instructions on how to build a very powerful bomb: an anti-matter explosive device with the yield of over one thousand hydrogen bombs. The most horrifying part is that the instructions require only common household materials, combined in just the right way.

The horror of this development begins to sink in around the world. Many propose that we should censor the information, in an attempt to prevent a catastrophe. But the reality is that the information is already loose. Sooner or later, someone will build the bombs, either from raw curiosity, or deliberate ill-intent. And then, ri...