trentmkelly/LessWrong-43k huggingface dataset:
Conversationism
by scottviteri — 2023-02-28T00:09:44.138Z

conversationism, the mathematics behind communication and understanding, used to raise the artificial intelligence as a child with love, as a painting, by midjourney

Intro

There seems to be a stark contrast between my alignment research on ontology maps and on cyborgism via Neuralink. I claim that the path has consisted of a series of forced moves, and that my approach to the second follows from my conclusions about the first. This post is an attempt to document those forced moves, in the hope that others do not have to duplicate my work.

Ontology maps are about finding shape semantic-preserving functions between the internal states of agents. Each commutative diagram denotes a particular way of training neural networks to act as conceptual bridges between a human and AI. I originally started playing with these diagrams as a way of addressing the Eliciting Latent Knowledge problem. Erik Jenner has been thinking about similar objects here and here.

Cyborgism via Neuralink is about using a brain-computer interface to read neural activation and write those predictions back to the brain, in the hope that neuroplasticity will allow you to harness the extra compute. The goal is to train an artificial neural network with a particularly human inductive bias. Given severe bandwidth constraints (currently three thousand read/write electrodes), the main reason to think this might work is that the brain and the transistor-based brain predictor are being jointly trained -- the brain can learn to send inputs to the predictor such that the predictions are useful for some downstream task. In other words, both the brain and the neural prosthesis are trained online. Think of predicting the activations of a fixed neural network given time series data (hard) versus the same task, but you pass gradients with respect to the base neural network's final loss through the bandwidth bottleneck into the predictor (less hard).

In the section "An Alternative Form of Feedback", I introduce a vision of alignment that takes communication bandwidth as fundamental, and I sketch out how one might amplify human feedback on AI via conversation. I might recommend reading that section more than the "TLDR -- My Conceptual Path" section.

TLDR -- My Conceptual Path

I started thinking about ontology maps in the context of ELK, because ontology maps can translate the AI's beliefs about the diamond's location ...
[S] D&D.Sci: All the D8a.  Allllllll of it. Evaluation and Ruleset
by aphyer — 2023-02-27T23:15:39.094Z

This is a follow-up to last week's D&D.Sci scenario: if you intend to play that, and haven't done so yet, you should do so now before spoiling yourself.

There is a web interactive here you can use to test your answer, and generation code available here if you're interested, or you can read on for the ruleset and scores.

RULESET

A party has three stats, corresponding to the three main types of challenge they will face in the course of the game:

Combat. The Black King wields the immense power of his Ring Of Orbs N-fold (where N is # of players). If your Combat stat is too low to defeat him, you will surely be vanquished.

Friendship. You will need to make allies among the consorts and denizens of your world...and also stay on good terms with your other party members! If your Friendship stat is too low to do this, you will end up killing NPCs you were supposed to work with, or with your party collapsing into recriminations and refusing to talk to one another.

Shenanigans. The world of Skaia is full of self-fulfilling prophecies, stable time loops, and similar such Shenanigans. If you manipulate these well, they can work to secure your victory - if your Shenanigans stat is too low, they can work to guarantee your defeat.

Your party's success rate is determined by whichever of these three stats is lowest. So stats of 10-10-10 are better than stats of 20-20-9.

To determine victory, you roll a number of d4s equal to your lowest stat. Each die showing a 4 counts as a success. If you get a number of successes >= the number of players, you win - if not, you lose.

For example, if a 2-player party has Combat 6, Friendship 10, and Social 4 (abbreviated 6-10-4), they will roll 4 d4s and win if at least 2 of those dice show a 4.

A party's stat total is the sum of stats of its members:

PLAYER STATS

Each class and aspect has different stats (generally summing to 6):

Class	Aspect	Combat	Friendship	Shenanigans
Knight	Rage	4	1	1
Maid	Heart	1	4	1
Seer	Time	1	1	4
Rogue	Blood	3	2	1
Thief	Doom	3	1	2
Bard	Breath	2	3	1
Sylph	Life	1	3	2
Witch	Light	2	1	3
Mage	Mind	1	2	3
Heir	Space	2	2	2
Prince	Void	2*	2*	2*
Page	Hope	3*	3*	3*

To determine a character's score in a stat, you multiply the score of their class by the score of their aspect. For example, an Heir (2-2-2) of Breath (2-3-1) has stats of 4-6-2. A Seer (1-1-4) of Light (2-1-3) has stats of 2-1-12.

Two class/aspect pairs have special ef...
The Birth and Death of Sydney — The Bayesian Conspiracy Podcast
by moridinamael — 2023-02-27T22:36:59.691Z
Link: https://www.thebayesianconspiracy.com/2023/02/182-the-birth-and-death-of-sydney-bing-ai/

We discuss Microsoft’s new Bing search chatbot – Sydney. It received a wave of media coverage due to erratic behavior, before being neutered by Microsoft. A dive into its behavior, human responses, and implications.

Links

The Birth and Death of Sydney
“Sydney Is Now Immortal”
Zvi’s Round-up
Gwern’s comment on Bing/Sydney
Astral Codex Ten post on Janus Simulators
Bankless crypto podcast – We’re All Gonna Die with Eliezer Yudkowsky
Eneasz’s Change.org Petition
Sydney-based erotica

Other Links
The Parable of the Dagger and The Parable of the Dagger. One of these is Eneasz, the other one is a voice-matching AI.
The Uncanny Vulva
Follow Moridinamael on Twitter

email the show: bayesianconspiracypodcast@gmail.com

Which AI do you prefer:

or

(Spoiler: they’re the same picture)

0:00:53 – Steven’s Media Corner
0:01:48 – Chat with Eneasz IRL
0:02:31 – Sydney/Bing AI
01:31:09 – Guild of the Rose Update
01:33:58 – Thank the Patron

We now partner with The Guild of the Rose, check them out.
Hey look, we have a discord! What could possibly go wrong?
Our Patreon page–your support is most rational, and totally effective. (also merch)

Rationality: From AI to Zombies, The Podcast

LessWrong Sequence Posts Discussed in this Episode:

still not

Next Sequence Posts:

The Allais Paradox

Zut Allais!
A case for capabilities work on AI as net positive 
by Noosphere89 — 2023-02-27T21:12:44.173Z

Edit: Changed the title.

Or, why I no longer agree with the standard LW position on AI anymore.

In a sense, this is sort of a weird post compared to what LW usually posts on AI.

A lot of this is going to depend on some posts that changed my worldview on AI risk, and they will be linked below:

Deceptive alignment skepticism sequence, especially the 2nd post in the sequence is here:

https://www.lesswrong.com/posts/RTkatYxJWvXR4Qbyd/deceptive-alignment-is-less-than-1-likely-by-default

Evidence of the natural abstractions hypothesis in action:

https://www.lesswrong.com/posts/BdfQMrtuL8wNfpfnF/natural-categories-update

https://www.lesswrong.com/posts/obht9QqMDMNLwhPQS/asot-natural-abstractions-and-alphazero#comments

Summary: The big updates I made was that deceptive alignment was way more unlikely than I thought, and given that deceptive alignment was a big part of my model of how AI risk would happen (about 30-60% of my probability mass was on that failure mode), that takes a big bite out of the probability mass of extinction enough to make increasing AI capabilities having positive expected value. Combine this with the evidence that at least some form of the natural abstractions hypothesis is being borne out by empirical evidence, and I now think the probabilities of AI risk have steeply declined to only 0.1-10%, and all of that probability mass is plausibly reducible to ridiculously low numbers by going to the stars and speeding up technological progress.

In other words, I now believe a significant probability, on the order of 50-70%, that alignment is solved by default.

EDIT: While I explained why I increased my confidence in alignment by default in response to Shiminux, I now believe that for now I was overconfident on the precise probabilities on alignment by default.

What implications does this have, if this rosy picture is correct?

The biggest implication is that technological progress looks vastly positive, compared to what most LWers and the general public think.

This also implies a purpose shift for Lesswrong. For arguably 20 years, the site was focused on AI risk, though it arguably exploded with LLMs and actual AI capabilities being released.

What it will shift to is important, but assuming that this rosy model of alignment is correct, then I'd argue a significant part of the field of AI Alignment should and can change purpose to something else.

As for Lesswrong, ...
Beginning to feel like a conspiracy theorist
by Joulebit — 2023-02-27T20:05:01.156Z

I see myself, and is often described by others, as a logical person and high in the openness trait. I am in university and have the last 2 years done some various amateur litterature review reasearch on different topics, especially on health and nutrition.

On many topics i find my research agreeing with conventional wisdom, such as for:

Sleeping hygiene
Alcohol consumption
Drugs like heroin and methamphetamine
Benefits of training
Dangers of obesity

But on a worrying fast increasing list i find my reasearch contradict common wisdom in huge ways:

Carbhydrates are probably what should be limited, and not fats
Following the WHO recommendations for salt intake is maybe way to little and potentially deadly
Facemasks don't limit the spread of COVID-19 and other similar influensas on a population level
Psychedelics are not addictive and can probably be helpful in many cases. (I think acceptance for this is rapidly increasing)

I don't have any problem trusting data and research more than authorities and common wisdom. But when friends and family significantly express their concern for my well being it makes feel like i am one of those conspiracy theorists that believe every stupid thing they find online. It may be that I am not having a strong enough prior, before updating it with new evidence.
Some thoughts pointing to slower AI take-off 
by Bastiaan — 2023-02-27T19:53:41.678Z

I've been lurking on LessWrong for quite a while and I've been following the high-level discussions on AI, AGI and take-off speed. I've had this simmer in the back of my mind and I have some ideas on this that I haven't seen expressed elsewhere. That can be because I haven't read thoroughly enough, but maybe they can add something new to the discussion as well? So here goes:

1. Bandwidth

The current big thing is Large Language Models (LLM). ChatGPT and Bing write well enough that they can be confused with being (weird) humans. This certainly is impressive and it is obviously a step towards further capabilities. However, what I haven't seen discussed is that reading / writing is very low bandwidth.

A scene can be described in a document of a few kilobytes max. A video of the same scene would be megabytes. And it's possible to add even more information that real-life embedded human would have access to: Smell, personal feelings, memories of previous encounters with the objects / people in the scene, etc.

AI isn't human and probably wouldn't need exactly the same information. But I would be very surprised if you could get to AGI without significantly higher bandwidth input and output; a few OOM at least.

Compute is one of the current constraints on AI and requiring a few OOM extra compute should push take-off further into the future.

2. Gimme data

Current AIs are able to do well in two cases: One, when there is a lot of more-or-less ready-made data that can be trained upon; ChatGPT uses the whole history of human writing as training set. Two, when useful data can be generated on the fly; AlphaGo generated its own data by playing stupendous amounts of games against itself.

For more advanced AI it may be that neither case holds. If we want to have AI that is more embedded in real life it's going to require real-life data of e.g. of how to interact with humans. Text descriptions may help, video may take it a further step. But it may well be that further / different data is required, e.g. what is mentioned above under the Bandwidth section. There is no database of (in-context) smells or feelings. And there is no historical data on "how AI should interact with humans".

This means that the data will need to be generated. And where an AI can play multiple games of Go in a second, real-life (human) interaction will (probably?) be limited to real-life speeds.

Not only this,...
Prediction Thread: Make Predictions About How Different Factors Affect AGI X-Risk.

by MrThink — 2023-02-27T19:15:05.349Z

In this post you can make several predictions for how different factors affect the probability that the creation of AGI leads to an extinction level catastrophe. This might be useful for planning.

Please let me know if you have other ideas for questions that could be valuable to ask.

Elicit Prediction (forecast.elicit.org/binary/questions/EviuxuEi3)

Predictions based on who develops AGI:

Elicit Prediction (forecast.elicit.org/binary/questions/v4fhohmfO)

Elicit Prediction (forecast.elicit.org/binary/questions/qDtulAocc)

Elicit Prediction (forecast.elicit.org/binary/questions/7kqkKdLCX)

Elicit Prediction (forecast.elicit.org/binary/questions/Ji-f9tH1B)

Predictions based on technology used for developing AGI:

Elicit Prediction (forecast.elicit.org/binary/questions/vEPhQz4Qz)

Elicit Prediction (forecast.elicit.org/binary/questions/pPq7NHe_C)

Prediction based on approach for creating AGI:

Elicit Prediction (forecast.elicit.org/binary/questions/fcZH5b2m8)

Elicit Prediction (forecast.elicit.org/binary/questions/ZvtW_KBSR)

Predictions on how money affects probability of AGI X-risk:

Elicit Prediction (forecast.elicit.org/binary/questions/yAooUTWqA)

Elicit Prediction (forecast.elicit.org/binary/questions/Aa7i8eqn1)
Counting-down vs. counting-up coherence
by TsviBT — 2023-02-27T14:59:39.041Z

[Metadata: crossposted from https://tsvibt.blogspot.com/2022/10/counting-down-vs-counting-up-coherence.html. First completed 25 October 2022.]

Counting-down coherence is the coherence of a mind viewed as the absence of deviation downward in capability from ideal, perfectly efficient agency: the utility left on the table, the waste, the exploitability.

Counting-up coherence is the coherence of a mind viewed as the deviation upward in capability from a rock: the elements of the mind, and how they combine to perform tasks.

What determines the effects of a mind?

Supranormally capable minds can have large effects. To control those effects, we'd have to understand what determines the effects of a mind.

Pre-theoretically, we have the idea of "values", "aims", "wants". The more capable a mind is, the more it's that case that what the mind wants, is what will happen in the world; so the mind's wants, its values, determine the mind's effect on the world.

A more precise way of describing the situation is: "Coherent decisions imply consistent utilities". A mind like that is incorrigible: if it knows it will eventually be more competent than any other mind at pushing the world towards high-utility possibilities, then it does not defer to any other mind. So to understand how a mind can be corrigible, some assumptions about minds and their values may have to be loosened.

The question remains, what are values? That is, what determines the effects that a mind has on the world, besides what the mind is capable of doing or understanding? This essay does not address this question, but instead describes two complementary standpoints from which to view the behavior of a mind insofar as it has effects.

Counting-down coherence

Counting-down coherence is the coherence of a mind viewed as the absence of deviation downward in capability from ideal, perfectly efficient agency: the utility left on the table, the waste, the exploitability.

Counting-down coherence could also be called anti-waste coherence, since it has a flavor of avoiding visible waste, or universal coherence, since it has a flavor of tracking how much a mind everywhere conforms to certain patterns of behavior.

Some overlapping ways of describing counting-down incoherence:

Exploitable, Dutch bookable, pumpable for resources. That is, someone could make a set of trades with the mind that leaves the mind worse off, and could

...
Eliezer is still ridiculously optimistic about AI risk
by johnlawrenceaspden — 2023-02-27T14:21:25.239Z

All power to crypto channel Bankless for their interview with Eliezer:

https://www.youtube.com/watch?v=gA1sNLL6yg4
https://www.lesswrong.com/posts/Aq82XqYhgqdPdPrBA/full-transcript-eliezer-yudkowsky-on-the-bankless-podcast

They actually take his arguments seriously. If I wanted to blow my life savings on some wretched crypto scam I'd certainly listen to these guys about what the best scam to fall for was.

---------------------------------------------------

Eliezer is visibly broken. Visibly ill.

This is what it looks like when the great hero of humanity, who has always been remarkably genre-savvy, realises that the movie he's in is 'Lovecraft-style Existential Cosmic Horror', rather than 'Rationalist Harry Potter Fanfic'.

No happy ending. Just madness and despair.

All power to Eliezer for having had a go. What sort of fool gives up before he's actually lost?

There are configurations of matter where something that remembers being me knows that the world only survived because Eliezer tried.

I do hope that my future self has the good grace to build at least one one hundred-foot gold statue of Eliezer.

But that's not the way to bet.

---------------------------------------------------

There's almost nothing new here.

The only bit that surprised me was when he says that if you give ChatGPT the right prompt, it can do long multiplication. I went and looked that up. It seems to be true.

Oh great, the fucking thing can execute arbitrary code, can it?

Don't worry, it's just a next-token predictor.

---------------------------------------------------

But Eliezer, despite the fact that he is close to tears while explaining, for the eight hundredth time, why we are all going to die, is still ridiculously optimistic about how difficult the alignment problem is.

This is a direct quote:

> If we got unlimited free retries and 50 years to solve everything, it'd be okay.
> We could figure out how to align AI in 50 years given unlimited retries.

No, we couldn't. This is just a sentient version of the Outcome Pump, ably described by Eliezer himself sixteen years ago: https://www.lesswrong.com/posts/4ARaTpNX62uaL86j6/the-hidden-complexity-of-wishes

What happens in this Groundhog Day scenario?

It depends very much on what the reset condition is.

If the reset is unconditional, then we just replay the next fifty years forever.

But the replays are not exact, non-determinacy and ch...
Milk EA, Casu Marzu EA
by jefftk — 2023-02-27T14:00:06.354Z

Pretty much everyone starts off drinking milk, and while adult
consumption varies culturally, genetically, and ethically,
if I put milk on my morning bran flakes that's a neutral
choice around here. If my breakfast came up in talking with a
friend they might think it was dull, but they wouldn't be surprised
or confused. Some parts of effective altruism are like this: giving
money

to
very poor people is, to nearly everyone, intuitively and obviously
good.

Most of EA, however, is more like cheese. If you've never heard of
cheese it seems strange and maybe not so good, but at least in the US
most people are familiar with the basic idea. Distributing bednets or deworming
medication, improving the treatment
of animals, developing
vaccines, or trying to reduce the risk of nuclear
war are mild cheeses like Cheddar or Mozzarella: people will
typically think "that seems good" if you tell them about it, and if
they don't it usually doesn't take long to explain.

In general, work that anyone can see is really valuable
is more likely to already
be getting the attention it needs. This means that people who are
looking hard for what most needs doing are often going to be exploring
approaches that are not obvious, or that initially look bizarre.
Pursuit of impact pushes us toward stranger and stronger cheeses, and
while humanity may discover yet more non-obvious cheeses over time I'm
going to refer to the far end of this continuum as the casu marzu end,
after the cheese that gets its distinctive flavor and texture from
live maggots that jump as you eat it. EAs who end up out in this
direction aren't going to be able to explain to their neighbor why
they do what they do, and explaining to an interested family member
probably takes several widely spaced conversations.

Sometimes people talk casually as if the weird stuff is longtermist
and the mainstream stuff isn't, but if you look at the range of EA
endeavors the main focus areas of EA all have people working along
this continuum. A typical person likely easily sees the altruistic
case for "help governments create realistic plans for pandemics" but
not "build refuges
to protect a small number of people from global catastrophes"; "give
chickens better
conditions" but not "determine the relative moral differences
between insects of
different ages"; "plan for the economic effects of ChatGPT's
successors" bu...
Normie response to Normie AI Safety Skepticism
by Giulio — 2023-02-27T13:54:40.249Z
Link: https://www.giuliostarace.com/posts/normie-ai-safety/

The following post is an adaptation of a
comment I posted
on the Machine Learning subreddit
in response to a
post asking whether readers worried about AI Safety/Alignment similarly to Eliezer Yudkowsky
(EY). My general impression is that most commenters had not dedicated much
thought to AI Safety (AIS), like most normal people. I personally wouldn't
qualify myself as particularly well-read on the topic either, hence the title.
As I mention towards the end of the article I think there is "alpha" in
discussions happening between us normies of the "out-group" and so I went ahead
and posted that comment and this post for amplification.

It seems like many commenters mostly have issues with the
"timelines", i.e., they do not
think these risks will happen soon enough for us to worry about them now. I
think this is a fair enough stance, there is a lot of uncertainty around
forecasting the
future of A(G)I developments.

However I would point out that they should consider that we may all be suffering
from
exponential growth bias.
If indeed AGI is very far away, then great, we indeed have more time to develop
it safely (although I'd be sad to get to live in such a cool future later rather
than sooner). But there is a real chance it isn't, and I do think we need to be
cautious of our confidence in the former case, considering the gravity of the
consequences superintelligence may bring. Basically what I'm saying is "better
safe than sorry", similarly to how we shouldn't dismiss nuclear safety,
epidemiology and shouldn't have dismissed climate alarmists in the 20th century.

The other skepticism I tend to see is that there's no point working on aligning
AI, because even with aligned AI, it will be abused by corporations and
governments, just like our narrower systems are being abused[1] today. I should
point out that the field of AI Safety
also considers this problem
and posits that solutions to it are also necessary to address safety, similarly
to
how we coordinate for other security means today.
See also
coordination/cooperation.
AI Alignment and AI Governance go hand-in-hand for long-term AI safety and there
are enough people in the world for us to tackle on either without excluding the
other. See
this image
from
this post.

Finally, I suppose the classic dismissal of AI Safety concerns that I tend to
see is something a long the lines of
"I don't think AGI will be unsafe"
or ...