trentmkelly/LessWrong-43k huggingface dataset:
Outrage and Statistics into Policy
by jefftk — 2023-04-10T14:00:01.280Z

Here's a pattern that doesn't work very well: a tragedy catches our
attention, we point to statistics to show it's an example of a
distressingly common problem, and we propose laws to address the
issue. Except the event is rarely representative of the larger
problem, and so these policy changes won't help much with the issues
reflected in the statistics. Instead, we should combine the anger and
passion the tragedy evokes with a deeper interpretation of the
statistics to identify what changes we most need.

For example, with the recent
Nashville school shooting a lot of people are giving statistics
like how there were 600+ mass
shootings and 51
school shootings in 2022, or how guns are now the largest
cause of death for children. But let's look into the kind of
events these statistics represent.

The Gun Violence Archive maintains a listing
of mass shootings: incidents in which at least four people are
shot. They link news stories for each, and while they're frustrating
and depressing they're very rarely "someone senselessly shoots up an
elementary school". Instead they're people fighting and one of them
pulls
out a gun, people arguing at a park and then escalating to shooting
on the highway, or a targeted attack at
a garage. In half of the incidents exactly four people are shot,
since most shootings are fewer people and the cutoff is four.

Education Week maintains a listing
of school shootings: incidents in which at least one person is
shot on school property or a school bus. Looking over them they're
again mostly not the kind that makes national news. Two
teenagers fighting in the schoolyard and one of them has
a gun, a drug deal between two armed people in a school parking
lot escalates
into a shooting, or an adult was shot in their car in the school
parking lot after
a basketball game.

If you look at the Nashville school shooting, or others that have made
especially
large impacts
nationally, they're attacks against younger kids. And hearing the
"number one cause of death among children" statistic you might think
this is common. But "children" here is being used literally to mean
"under 18": 47% of those dying were 17, and 76% were 16 or 17. Only
18% were under 15:

Source: CDC, code

None of this is to minimize the suffering and loss these shootings
represent: ten incidents in which four people die isn't better than
one in w...
Tim Berners-Lee found it hard to explain the web
by Tor Økland Barstad — 2023-04-10T13:33:49.845Z

According to Tim Berners-Lee, explaining his ideas about the World Wide Web was at times quite challenging:

«So going back to 1989, I wrote a memo suggesting the global hypertext system. Nobody really did anything with it.»

«Once you've had an idea like that it kind of gets under your skin. And even if people don't read your memo - well, actually he [my boss] did. It [his copy] was found after he died. He had written "Vague, but exciting" in pencil in the corner.»

«In general it was difficult. It was really difficult to explain what the web would be like. It's difficult to explain to people now that it was difficult then. But then - well, there was no web, so things like "click" didn't have the same meaning. I can show somebody a piece of hypertext, a page which has got links, and we click on the link and bing - there'll be another hypertext page. Not impressive. You know, we've seen that - we've got things on hypertext on CD-ROMs.»

Probably just bells and whistles

In terms of impact, it's unusual (but not unheard of) for ideas to rank more highly than the World Wide Web.

But, I suspect, it's not so unusual for ideas to be similarly difficult to grok (and sometimes much harder!).

And although it's not a perfect analogy, I think there is some relevance here to AI alignment, and ideas people propose.

We can't listen to everyone at length. There aren't enough hours in the day. So we need to form heuristics such as:

If people are unable to explain their ideas convincingly and succinctly, then this is often because they are internally confused. If they can't succinctly convey what the "core" of their idea is, we can disregard them.

This is a useful heuristic. But the more strongly we rely on it, the more we are at risk of false negatives.
Current UK government levers on AI development
by rosehadshar — 2023-04-10T13:16:22.443Z

This is a link post for

this

collection of current UK government levers on AI development.

At the end of 2022, I made a

collection of information on current UK government levers on AI development

, focused on levers which seem to me to have potentially significant implications for the governance of advanced AI.

The primary audience I’m intending for the collection is people who work in or are considering working in AI governance and policy, and I hope it will be useful as an input into:

Building more detailed models of how the UK government might affect AI development and deployment.
Getting an overview of the policy status quo in the UK.
Thinking about which policy areas are likely to matter more for managing transitions to advanced AI.
Thinking about how important influencing the UK government is relative to other actors.

In this post, I try to situate current UK government levers in the broader context, to give a sense of the limits of the collection.

Some initial caveats:

The collection is based exclusively on publicly available information, not on conversations with relevant government officials.
I’m not an expert in the UK government or in AI policy.
The factual information in the collection hasn’t been vetted by relevant experts. I expect there are things I’ve misunderstood, and important things that I’ve missed.
The collection is a snapshot in time. To the best of my knowledge, the information is up to date as of April 2023, but the collection will soon get out of date. I am not going to personally commit to updating the collection, but would be excited for others to do so. If you’re interested, comment on this post or on the collection, or send me a message.
I am not advocating that particular actors should try to pull any particular lever. I think it’s easy to do more harm than good, and encourage readers to orient to the collection as a way of thinking about how different trajectories might play out, rather than as a straightforward input into which policies to push. I think that figuring out net positive ways of influencing policy could be very valuable, but requires a bunch of work on top of the sorts of information in this collection.

This collection is just a small part of the puzzle. Two aspects of this which I’ll unpack in a bit more detail below:

The actions of the UK government might not matter.
Even conditional on UK government actions matt

...
Safe AGI Complexity: Guessing a Higher-Order Algebraic Number
by Sven Nilsen — 2023-04-10T12:57:14.606Z

Introduction: A rational number can represent an arbitrary sequence of bits where bit patterns either terminate or repeat infinitely. This provides a mathematical toy model for reasoning about AGI safety. A bit sequence can be translated into a sequence of actions. The goal is to prove safety of some bit sequence without bounds. While this toy model does not allow input to the AGI, it offers useful insights into the complexity of safe AGI in the case where the entire deterministic world state and human values have been internalized.
I have been thinking about numbers of the form:
(a / b)^(1 / c)
Where a, b, c are natural numbers. In number theory, this is called "Radical Extensions of Rational numbers" (I will use RER for short in this blog post).

RER is closed under multiplication, but not addition:
(a0/a1)^(1/a2) * (b0/b1)^(1/b2) = ( (a0^b2*b0^a2) / (a1^b2*b1^a2) )^(1 / (a2*b2) )

RER covers the real line a little better than rationals, by including some irrational numbers, but not completely due to the Abel-Ruffini theorem, which implies that there are more irrational numbers than can be constructed using a generalization of RER to solutions of polynomial equations (up to some limit below the more irrational solutions).

These "more irrational" numbers are called "higher-order algebraic numbers" or "transcendental algebraic numbers" in number theory.

Translated as an analogy to computer science:

A natural number might be thought of as a "source" of some program
A rational number might be thought of as a "computationally reducible" calculation using some source as input
A RER might be thought of as a simple encryption algorithm
A higher-order algebraic number might be thought of as a recursive encryption scheme that can make it arbitrary difficult to interpret some message

Translated as an analogy to AI safety:

A natural number might be thought of as a sequence of actions
A rational number might be thought of as operational safety of a sequence of actions
A RER might be thought of as simple yet non-trivial contextual safety of a sequence of actions
A higher-order algebraic number might be thought of as a human goal

Since human goals are difficult to encode, one might think about them using this analogy, as guessing a higher-order algebraic number.

The analogy also gives some insight into predictability. For example, consider translat...
Welcome to the decade of Em
by Ozyrus — 2023-04-10T07:45:35.684Z

When Robin Hanson was writing Age of Em in 2016, he was thinking that Ems will be possible with advancement of brain-scanning technology in a hundred years; what he didn't anticipate was the transformer models, and their capabilities, as well as introduction of AutoGPT paradigm (I proposed the name ICA Simulacra in this post, but it is obvious now that AutoGPT is the label everyone will be going with).
If capabilities will increase further as they did before, the "new paradigm" of worker+ChatGPT will not even live for several years; AutoGPT will replace that.
Why do you need a marketing department?
Just hire an org who can set up a fleet of AutoGPT "Ems" to do these tasks, hire one expert marketer to verify their outputs and make sure they are all in line, and save thousands of dollars.
Hell, when you see that this expert marketing guy is just signing off without any changes, setup another AutoGPT in his place.
And then some crypto guys, riding the wave of AI hype, will make a DAO consisting solely of AutoGPT Ems. And this or the other one down the line will succeed.
And some other guys, or the same ones, make an org with a sole objective of introducing better AutoGPT Ems.
If government will act quick and smart, they will manage to introduce UBI before wide-scale riots and society collapse.
And then we die either way.
Age of Em will be a decade of Em at best.
Open & Welcome Thread – April 2023
by habryka — 2023-04-10T06:36:03.545Z

If it’s worth saying, but not worth its own post, here's a place to put it.

If you are new to LessWrong, here's the place to introduce yourself. Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are invited. This is also the place to discuss feature requests and other ideas you have for the site, if you don't want to write a full top-level post.

If you're new to the community, you can start reading the Highlights from the Sequences, a collection of posts about the core ideas of LessWrong.

If you want to explore the community more, I recommend reading the Library, checking recent Curated posts, seeing if there are any meetups in your area, and checking out the Getting Started section of the LessWrong FAQ. If you want to orient to the content on the site, you can also check out the Concepts section.

The Open Thread tag is here. The Open Thread sequence is here.
Contra LeCun on "Autoregressive LLMs are doomed"
by rotatingpaguro — 2023-04-10T04:05:10.267Z

I answer LeCun's arguments against LLMs as exposed in this lesswrong comment. I haven't searched thoroughly or double-checked in detail LeCun's writings on the topic. My argument is suggestive-hand-waving-stage.

Introduction

Current large language models (LLMs) like GPT-x are autoregressive. "Autoregressive" means that the core of the system is a function f (implemented by a neural network) that takes as input the last n tokens (the tokens are the atomic "syllables" used by the machine) and produces as output a probability distribution over the next token in the text. This distribution can be used to guess a good continuation of the text.

The function f is fixed, it's always the exact same function at each step. Example: given the current token list

x=[John,<space>,ate,<space>,a,<space>,ban],

we evaluate f(x), which gives high probability to, say, the token "ana". We concatenate this token to the list, obtaining

x′=[John,<space>,ate,<space>,a,<space>,ban,ana].

Applying again the function, we evaluate f(x′), which this time turns out to assign higher probability to "<space>". However, the computation that first lead to "ana", then to "<space>", is the same. There is not some hidden "mind state" of the thing, with slots for concepts like "John", "the banana", et cetera; all there is, is the function mapping the last up-to-n past tokens to the probabilities for the next token. If there are "concepts", or "schemes", or "strategies", they somehow appear strictly within the computation of the function, and are cleared at each step.

LeCun is a very famous Machine Learning researcher. In these slides and this tweet, he explains why he thinks that (quoting) "Auto-Regressive LLMs are doomed."

The argument against autoregressive LLMs

I report verbatim Razied's summary of the argument, plus a pair of follow-up comments I picked:

I will try to explain Yann Lecun's argument against auto-regressive LLMs, which I agree with. The main crux of it is that being extremely superhuman at predicting the next token from the distribution of internet text does not imply the ability to generate sequences of arbitrary length from that distribution.

GPT4's ability to impressi

...
An AI Realist Manifesto: Neither Doomer nor Foomer, but a third more reasonable thing
by PashaKamyshev — 2023-04-10T00:11:53.802Z

Cross-posted from substack

AI has been a hot topic in recent Twitter discourse, with two opposing camps dominating the conversation: the Doomers and the AI builders. The Doomers, led by Eliezer Yudkowsky and other rationalists, advocate for caution and restraint in the development of AI, fearing that it could pose an existential threat to humanity. Prominent figures in this camp include Elon Musk, who has expressed concerns about the potential dangers of AI while also founding AI-focused companies like OpenAI and up-and-coming “BasedAI.” On the other side of the debate are the AI builders, including Yann LeCunn and Sam Altman, who are eager to push the boundaries of AI development and explore its full potential. While some members of this group have been dismissed as "idiot disaster monkeys" by Yudkowsky, I will refer to them as "Foomers" for the purposes of this blog post. The divide between these two camps is significant, as it represents a fundamental disagreement about the future of AI and its potential impact on society.

The debate around AI often centers on the concept of superintelligence, which refers to AI that surpasses human intelligence in every way. Doomers argue that superintelligence could pose an existential threat to humanity, as it would be capable of outsmarting humans and achieving its goals at any cost. This is particularly concerning given that the goals of such an AI would be difficult, if not impossible, to specify in advance. If the goals are misaligned with human values, the consequences could be catastrophic. The AI builders or "Foomers" tend to downplay these risks, arguing that superintelligence could be used for the benefit of humanity if developed and controlled properly. However, the Doomers counter that the risks are too great and that any attempt to control superintelligence is likely to fail. As such, the debate remains a contentious one, with both sides offering many arguments.

While Foomers may reject critique through thought experiments and argue for incremental improvement of AI through trial and error, there seems to be a lack of engagement from both sides in identifying the underlying assumptions and values that shape the debate. This can lead to the same discourse tiling Twitter with copies of itself without any meaningful progress. As a result, many people are left frustrated and exhausted by the deba...
Near-mode cryonics: A thought experiment
by Mati_Roy — 2023-04-09T22:21:19.704Z

(x-post: LessDead)

Set-up: In a world that’s otherwise the same, an anthropomorphic God comes to you with a gun and a check, and offers you the following deal, should you accept it. (God can perfectly see the future and is honest.)

Dilemma: God will give you a check of X USD if you accept to be immediately and instantly killed iff the current cryonics protocole is sufficient to preserve identity.

Alternative: God kills you iff the following is true: If you were signed up for cryonics and died this year, you would get reanimated at some point in the future.

Question: What’s the minimum X for which you would accept this deal? What probability of dying are you trading this for?

Comparison:

Say 100,000 USD and 5%. This means this deal would give you:

100% earning 100k USD

5% of dying (right now, instead of ~0%)

If you die and get cryopreserved:

100% losing 100k USD (i.e. a typical cost for state of the art cryonics)

5% of living (instead of 0%)

If you die and don’t get cyropreserved:

100% of saving 100k USD (by not paying for cryonics)

5% more chance of dying (100% instead of 95%)

In this sense, taking the deal is similar to not getting cryopreserved (and vice versa).

Disanalogies:

There are significant disanologies of course:

The type of life you would live

How long you life would be if you didn’t die

The social cost of signing up for cryonics

I still think this might be a useful thought experiment to bring this dilemma to a more near mode way of thinking. It also reverses the default option.
3 entirely different things we call “Time”
by Vasyl Dotsenko — 2023-04-09T20:08:02.759Z
Link: https://medium.com/p/a6988b94ca2d

Confusion about the reality of time comes from the fact that there are three entirely different things we call “time.” When we ask a physicist if time is real, we would get an answer about an entirely different phenomenon compared to the answer to the same question of a philosopher.

The aim of this article is to reduce confusion when tackling the question of reality of time. It turns out that each of the three things we call "Time" can be compared with other more familiar things, giving us better intuition about the reality of each of them.

Dimension of Time

Spacetime

is a four-dimensional manifold in which an observer perceives three dimensions as “space” and one as “time.” All four spacetime dimensions can be either fundamental themselves or emergent from some more fundamental phenomena. We know from experiments that the exact direction in the four-dimensional manifold an observer perceives as “time” depends on the observer’s velocity and gravity in the vicinity. So, we are confident that the Dimension of Time is as real as the other three Dimensions of Space.

Arrow of Time

All the dimensions of space are isotropic, meaning that the laws of physics are the same in all possible directions in space. Dimension of Time, on the other hand, is anisotropic, meaning that there are some laws of physics that are different, when you look from the past into the future, compared to when you look from the future into the past. For example, entropy of a closed system increases as you move from the past into the future. We suspect that the Arrow of Time is at least as real as the lack of any special direction in space.

Cursor of Time

There is a privileged moment in time we call “now,” and we perceive the privilege constantly moving from moment to moment from the past into the future.

It naїvely feels like “now” moves synchronously for all observers in the universe, i.e., all the observers would agree that today is Tuesday all across the entire universe. We know from experiments this is not true. We know that for causally disconnected events, it is possible to have observers who would disagree about the order of these events. In other words, the consistent for all observers notion of “now” is incompatible with general relativity.

Privileged moment called “now” doesn’t show up in any formulas or models in physics. Moreover, in order to move from the past to the future, “now” would n...
Foom seems unlikely in the current LLM training paradigm
by Ocracoke — 2023-04-09T19:41:30.001Z

Epistemic status: The idea here has likely been articulated before, I just haven't noticed it, so it might be worth pointing it out again.
Foom describes the idea of a rapid AI takeoff caused by an AI's ability to recursively improve itself. Most discussions about Foom assume that each next iteration of improved models can in principle be developed and deployed in a short amount of time. Current LLMs require huge amounts of data and compute to be trained. Even if GPT-4 or similar models were able to improve their own architecture, they would still need to be trained from scratch using that new architecture. This would take a long time and can't easily be done without people noticing. The most extreme Foom scenarios of models advancing many generations in < 24 hours seem therefore unlikely in the current LLM training paradigm.
There could be paths towards Foom with current LLMs that don't require new, improved models to be trained from scratch:

A model might figure out how to adjust its own weights in a targeted way. This would essentially mean that the model has solved interpretability. It seems unlikely to me that it is possible to get to this point without running a lot of compute-intensive experiments.
It's conceivable that the recursive self-improvement that leads to Foom doesn't happen on the level of the base LLM, but on a level above that, where multiple copies of a base model are called in a way that results in emergent behavior or agency, similar to what Auto-GPT is trying to do. I think this approach can potentially go a long way, but it might ultimately limited by how smart the base model is.

Insofar as it is required to train a new model with 100s of billions of parameters from scratch in order to make real progress towards AGI, there is an upper limit to how fast recursive self-improvement can progress.