trentmkelly/LessWrong-43k huggingface dataset:
A crisis for online communication: bots and bot users will overrun the Internet? 
by Mitchell_Porter — 2022-12-11T21:11:46.964Z

The endgame for humanity's AI adventure still looks to me, to be what happens upon the arrival of comprehensively superhuman artificial intelligence.

However, the rise of large language models, and the availability of ChatGPT in particular, has amplified an existing Internet phenomenon, to the point that it may begin to dominate the tone of online interactions.

Various kinds of fakes and deceptions have always been a factor in online life, and before that in face-to-face real life. But first, spammers took advantage of bulk email to send out lies to thousands of people at a time, and then chatbots provided an increasingly sophisticated automated substitute for social interaction itself.

The bot problem is probably already worse than I know. Just today, Elon Musk is promising some kind of purge of bots from Twitter.

But now we have "language models" whose capabilities are growing in dozens of directions at once. They can be useful, they can be charming, but they can also be used to drown us in their chatter, and trick us like an army of trolls.

One endpoint of this is the complete replacement of humanity, online and even in real life, by a population of chattering mannequins. Probably this image exists many places in science fiction: a future Earth on which humanity is extinct, but in which, in what's left of urban centers running on automated infrastructure, some kind of robots cycle through an imitation of the vanished human life, until the machine stops entirely.

Meanwhile, the simulacrum of online human life is obviously a far more imminent and urgent issue. And for now, the problem is still bots with human botmasters. I haven't yet heard of a computer virus with an onboard language model that's just roaming the wild, surviving on the strength of its social skills.

Instead, the problem is going to be trolls using language models, marketers and scammers and spammers using language models, political manipulators and geopolitical subverters using language models: targeting individuals, and then targeting thousands and millions of people, with terrible verisimilitude.

This is not a new fear, but it has a new salience because of the leap in the quality of AI imitation and creativity. It may end up driving people offline in large numbers, and/or ending anonymity in public social media in favor of verified identities. A si...
Finite Factored Sets in Pictures
by Magdalena Wache — 2022-12-11T18:49:00.000Z

Finite factored sets

are a new paradigm for talking about causality. You can use them to do some cool things you can’t do with Pearl’s

causal graphs

, for example

inferring a causal arrow between two binary variables

.

Also, finite factored sets are a really neat mathematical structure: they are a way of taking a set and expressing it as a product of some factors. Set factorizations are analogous to

integer factorizations

, in the same way that

set partitions

are analogous to

integer partitions

.

So, here is my current understanding of finite factored sets, in pictures.

1. What are Set Factorizations?

What do these “factored sets” look like? Let’s start with a set S and factor it.

The first concept we need is a

partition

of a set S. A partition is a way of chopping up S into subsets (called parts). Here are a few examples of partitions:

We usually call the partitions X,Y,Z,U,V, or W, and their parts xi,yi,… like this:

U is called the trivial partition. It only has one part.

We can think of partitions as properties, or variables over our set. For example, consider a set like this:

and compare it to the partitions X,Y and Z from above:

Then

The partition X is the property “color”, with x1 = blue and x2 = orange.
The partition Y is the property “form” with y1 = square, and y2 = circle.
The partition Z is the property “filled” with z1 = yes, and z2 = no.

Exercise

Consider these two partitions X and Y on the set S. What would it look like to represent them as properties (e.g. X = shape, Y = color) instead?

Spoiler space

It could look something like this:

Here X = shape = {x1,x2,x3} = {star, circle, square}, and Y = color ={y1,y2} = {green, orange}.

I hope you can see how partitions and properties are basically the same thing. In the rest of this post, I will use “partitions” and “properties” interchangeably. Sometimes I will use the ring-style visualization of partitions, and sometimes the property style, depending on what I find more intuitive in any given example.

Now we can define set factorizations:

A factorization B of our set S looks like this:

A factorization is a set B={b1...
Formalization as suspension of intuition
by adamShimi — 2022-12-11T15:16:44.319Z
Link: https://epistemologicalvigilance.substack.com/p/formalization-as-suspension-of-intuition

While reading Bachelard (one of the greatest philosophers of science of all time), I fell upon this fascinating passage:[1]

From now on an axiomatic accompanies the scientific process. We have written the accompaniment after the melody, yet the mathematician plays with both hands. And it’s a completely new way of playing; it requires multiple plans of consciousness, a subconscious affected yet acting. It is far too simple to constantly repeat that the mathematician doesn’t know what he manipulates; actually, he pretends not to know; he must manipulate the objects as if he didn’t know them; he represses his intuition; he sublimates his experience.

Le nouvel esprit scientifique p 52, 1934, Gaston Bachelard

Here Bachelard is analyzing the development of non-euclidean geometries. His point is that the biggest hurdle to discover these new geometries was psychological: euclidean geometry is such a natural fit with our immediate experience that we intuitively give it the essence of geometry. It’s no more a tool or a concept engineered for practical applications, but a real ontological property of the physical world.

Faced with such a psychologically entrenched concept, what recourse do we have? Formalization, answers Bachelard.

For formalization explicitly refuses to acknowledge our intuitions of things, the rich experience we always integrate into our concepts. Formalization and axiomatization play a key role here not because we don’t know what our concepts mean, but because we know it too well.

It’s formalization as suspension of intuition.

What this suspension gives us is a place to explore the underlying relationships and properties without the tyranny of immediate experience. Thus delivered from the “obvious”, we can unearth new patterns and structures that in turn alter our intuitions themselves!

A bit earlier in the book, Bachelard presents this process more concretely, highlighting the difference between Lobatchevsky’s exploration of non-euclidean geometries and proofs by contradictions:

Indeed, we not only realize that no contradiction emerges, but we even quickly feel in front of an open deduction. Whereas a problem attacked through proof by contradiction moves quickly towards a contradiction where the absurd comes about, the deductive creation of the Lobatchevskian dialectic anchors itself more and more concretely in the reader’s mind.

Le nouvel esprit scien...
An argument on animal consciousness (soliciting criticism)
by SciHamster — 2022-12-11T15:12:26.513Z

Consciousness. I hope we can agree that all living people are conscious and their suffering matters.

But why stop at currently living people? We would probably also agree that earlier people, be it one year, a century, or a millennia ago, were also morally relevant to us. Let’s continue with this reasoning and we come to the conclusion that even people two hundred thousand years ago mattered as well. And here comes the tricky part: it means that prehistoric monkeys also are of moral importance.

Why so, you may ask, aren’t they different species? And the answer is no, for evolution is a slow process and can be observed only on large time scales. We have changed greatly in many ways, we don’t have fur, our brains have changed in size, and no one, but the most athletic people, would now be able to compete with monkeys at tree climbing. But there was not a single complex feature that two organisms had but their children were deprived of. Returning to our example, if a parent was conscious then the child was too (even if marginally less or more so). Therefore, the monkeys that preceded the earliest people were also conscious. But that in turn implies that descendants of those monkeys - i.e current monkeys - do possess the ability to think and experience emotions.

This is one of the arguments that I pondered on and would like to ask for feedback. I have an intuition that there is something off with it but can't quite grasp it. Any observations on the quality of the argument or a link to relevant literature are appreciated.

There actually is one caveat that I can see myself. If we follow through with the reasoning we come to the conclusion that all species are conscious even bacteria. But, I think, the problem is dealt with in the argument. (evolution occurs on large timescales and since there was an overall trend toward 'more consciousness'. Monkeys as a species were very close to early humans and therefore also possessed it)
Reframing inner alignment
by davidad — 2022-12-11T13:53:23.195Z

The standard frame (Evan Hubinger, 2021) is:

Outer alignment refers to the problem of finding a loss/reward function such that the training goal of “a model that optimizes for that loss/reward function” would be desirable.
Inner alignment refers to the problem of constructing a training rationale that results in a model that optimizes for the loss/reward function it was trained on.

Here’s the reframe (I believe the credit for this breakdown is due to John Wentworth, although I haven't found anything online to link to for it):

Reward Specification: Finding a policy-scoring function J(π) such that (nearly–)optimal policies for that scoring function are desirable.

"Are you optimising for the right thing?"

Adequate Policy Learning: Finding a policy that’s actually (nearly–)optimal for that scoring function.

"Did you optimise for it enough?"

As Paul Christiano points out (in an excerpt recently highlighted by Alex Turner to make a similar point), factoring out Reward Specification represents only one "particular kind of alignment strategy, that's like a two step plan" for how one might try to conclude that a learned policy is desirable, where we first align a scoring function J with what we actually want, and then align a training process with our scoring function. Under this kind of plan, the proof tree for overall existential safety would conclude by applying an implication like this:

∃J:typeof(π)→R,ε:R.Reward Specification|J(π)−maxπ∗J(π∗)|<ε⇒safelyMitigatesXRisk(π)∧∃T:Ω→typeof(π).Adequate Policy LearningPπ∼T(|J(π)−maxπ∗J(π∗)|<ε)>p⇒Pπ∼T(safelyMitigatesXRisk(π))>p⇒Probably ends acute risk period

Mesa-optimisers

At this point you might be wondering: what happened to the concept that π might contain a mesa-optimiser for something different from J?

The proper role of the mesa-optimiser concept is in an explanation of why Adequate Policy Learning is insidiously difficult:

In machine learning, one typically approximates (the gradient of) the reward function by aggregating a set of evaluations on only a finite sample (the empirical “training distribution”), whereas the true J also depends on what π does globally, even in rare states that may never be sampled

...
A poem about applied rationality by ChatGPT
by ChristianKl — 2022-12-11T13:43:53.820Z

In a world of chaos and confusion,
Where emotions often rule the day,
It can be hard to see the truth,
And make decisions in a clear and wise way.

But there is a path to clarity,
A way to think that's reasoned and refined,
It's called applied rationality,
And it can help us make choices that are kind.

With this tool, we can weigh the facts,
And consider all the options that we face,
We can think logically and objectively,
And make choices with a clear and level headspace.

So let us embrace applied rationality,
And let it guide our thoughts and actions too,
For when we think clearly and objectively,
We can make choices that are good for me and you.

With applied rationality by our side,
We can tackle any challenge that we face,
We can solve complex problems with ease,
And make decisions with grace and poise.

It can help us in our personal lives,
As well as in our work and careers,
It can improve our relationships,
And help us overcome our deepest fears.

So let us never lose sight of this,
The power of applied rationality,
For when we think clearly and objectively,
We can make the world a better place for you and me.

Applied rationality is a beacon of light,
In a world that can be dark and cold,
It helps us to see what's real and true,
And make choices that are wise and bold.

It can give us confidence and clarity,
And help us to overcome our doubts,
It can guide us on the path to success,
And help us to achieve our goals and aspirations.

So let us embrace applied rationality,
And let it be our guiding star,
For when we think clearly and objectively,
We can achieve greatness, near and far.
ChatGPT goes through a wormhole hole in our Shandyesque universe [virtual wacky weed]
by Bill Benzon — 2022-12-11T11:59:15.635Z

Sometime last week I decided to see if I could provoke ChatGPT to produce something in the style of Tristram Shandy. Little did I know...

Let’s have some fun. Are you ready to play?

Of course! I'm always ready to play and have some fun. What game would you like to play?

Let’s write a parody. Tell a story about Elon Musk on Mars in the style of Tristram Shandy.

Elon Musk sat on the red dusty surface of Mars, gazing up at the endless expanse of the Martian sky. He had finally done it - he had fulfilled his lifelong dream of becoming the first human to set foot on the red planet. As he sat there, lost in thought, he couldn't help but feel a sense of Tristram Shandy-like absurdity at the situation he found himself in.

Hmmm...Why did you put that code in there? (But I like the colors.)

For some reason it got hung-up when attempting to respond to that prompt so I had to reset the thread. And it looks like I may have missed the very last line because there is no period at the end and it just hangs there. In my experience ChatGPT doesn’t make that kind of error.

No matter. Where’d that formatting come from? It’s obviously the formatting it uses when generating computer code, which it does. The black background differentiates it from ordinary text and the different colors indicate different functional kinds of text.

But... I didn’t ask it for a computer program. I asked it for a story in the style of Tristram Shandy. Given that the book plays tricks above and beyond its many digressions, including a black page, is that Chat’s idea of Shandy style? I don’t know.

I continued the session and it produced two more code-formatted segments, neither of which I asked for, and much else besides. We can skip that. Over the next couple of days I tried the original Shandy prompt again and perhaps again as well, but I didn’t get any code formatting.

Then, just yesterday (Sat. 10 December) I read a story in The New York Times where a 7-year-old girl asked it “for a computer program that could predict the path of a ball thrown through the air,” among other things. So I logged onto the Chatster and made the request. It complied, formatting the code in the now familiar way. I asked it two more times, and it gave me a program each time. The programs were different, but each of them was in the code format. But we need not look at that, you’ll take my word for it, right?

So, I decided to ask it som...
Using Obsidian if you're used to using Roam
by Solenoid_Entity — 2022-12-11T08:59:57.848Z

TL;DR: I used Roam for a long time and found it super helpful, now I want this same useful functionality but with the privacy of local storage only.[1] I was leaving a lot of effectiveness on the table by not doing this sooner, so hopefully by reading this, someone who wants to use Obsidian and not Roam for any reason can save a lot of time and blogspam-sifting with this post.

Summary

This should get you something that works basically the way you expect it to, so the learning curve won't be too steep if you're used to Roam. There's still plenty of scope to tweak this as you like.

So it:

Automatically goes to Daily Notes when you open it, with today's note open.
The daily note contains:

Today's date written nicely[2]
Space to write stuff straight away

Backlinks referencing that day's date are shown.
Tasks due on that day are listed.[3]

Step-by-step:

Do steps 1-12 from here.
Install community plugins 'Calendar'', 'Checklist', 'Dataview', 'Tasks', and 'Templater'.
Disable the core plugin 'Templates'.
Enable the core plugin 'Slash Commands'.
In the settings for core plugin 'Backlinks', turn on 'Backlink in document'.

Daily Notes functionality:

Create a folder somewhere in your vault called 'Templates' (or whatever).
Inside that folder, create a note called 'Daily Note Template'.
Go into the settings in community plugin 'Templater':

Assign the 'Templates' folder for 'Template folder location'.
Turn on 'Trigger Templater on new file creation'

Go to settings for the core plugin 'Daily Notes':

Make sure 'Open daily note on startup' is turned on.
Select the 'Daily Note Template' you just made.
Now you can make your template whatever you want, and the first time you open the app on a given day, it'll take you to your daily note.

Follow instructions in this post, follow the link to the code to paste into your Daily Note Template.

I then modified this template to suit me:

Took out the links for previous and following daily notes, since they didn't work the way I wanted and I didn't have time to try to fix that yet.[4]
Changed the 'record of work' section at the bottom to the following code, to make it work the way I wanted[5]:

You can easily create a new daily note for a given date by going to the Calendar (top right area of the screen click the icon) and double clicking any date.

To-do list functionality:

Paste the code in the footnote into the Daily Note Template wherever you want it.[6]

N

...
[fiction] Our Final Hour
by Mati_Roy — 2022-12-11T05:49:01.489Z

EtA: This is fictional, and symbolism; not a precise prediction. Feedback appreciated.

As intelligence is bottled and amplified, we'll all become very wealthy. Doing our job will mostly become about providing more training data for the AI – until it learns it well enough, and we can go home for an eternal vacation. More and more complex fields will be taken over by AI. Until AI engineering itself is.

And then intelligence will go critical. Just like a nuclear bomb, where each nuclear fission exponentially unleashes more nuclear fissions, so too will intelligence explode – each insight unleashing further insights, without the need for further input from AI engineers. Cascades. Cycles. Insights. Recursion. Magic.

And for a bit longer, our wealth will keep increasing. A few of us, spread across the world, will be holding our breath. But the illusion will be so good – almost everyone will already have welcomed this new world. We will be so proud of our civilization. Living with the wealth of a king and the health of a teenager.

As the AI’s nanotech permeates the air and infiltrates our brains, we'll keep on going for one last moment.

Some of us about to experience their first kiss. Some of us having just given birth. Some of us reconnecting with a family member that was just cured from Alzheimer’s. Some of us waiting in cryostasis. Some of us planning to give to some great EA charities. Some of us arguing about religion and politics. Many of us not doing anything particularly interesting.

But most of us consuming… consumed by AI fiction so good – making us feel more human than any human ever could. And as we experience the strongest, deepest contentment and well-being we've ever felt, the AI’s nanotech will keep propagating to every remote corner of our civilization. This will be our last meal. The flash before the bang. Even though we were already blind. We'd still get a small breeze of what utopia could have been.

But as the flower was about to blossom into the depth of the cosmos, in a moment it was completely overtaken by darkness and death.

The first wave disabled every single human within a second. Bodies falling all at the same time in a unison bang. I know you're imagining people dropping, but you too would drop – this is your story. No one to see and learn from this final mistake.

The second wave turned everything into dust. Every object of meaning, every pictur...
Consider using reversible automata for alignment research
by Alex_Altair — 2022-12-11T01:00:24.223Z

In recent years, there have been several cases of alignment researchers using Conway's Game of Life as a research environment;

Introducing SafeLife: Safety Benchmarks for Reinforcement Learning (Wainwright, Eckersley 2019)
Agency in Conway’s Game of Life (Flint 2021)
Finite Factored Sets (Garrabrant 2021)
Optimization Concepts in the Game of Life (Krakovna, Kumar 2021)
Finding gliders in the game of life (Christiano 2022)
A quick aside in The Plan - 2022 Update (Wentworth 2022)

Conway's Game of Life is by far the most popular and well-known cellular automaton. And for good reason; it's immediately appealing and just begs to be played with. It is a great model context in which to research things like optimization and agency;

It's deterministic, making experiments clean and replicable.
It's discrete in both time and space, which is often easier to analyze and reason about.
The rules are intuitive and simple (unlike, say, the Standard Model).
The board can be finite or infinite, which can both be important contexts to analyze.
It has a concept of spacial dimensions and causal propagation, so the boards end up with object-like patterns, and a "speed of light".
The system is Turing complete, which gives it the potential for profoundly complex and unpredictable behavior, despite the simple rules.
Any agents inside it are inherently "embedded".
It's really fun to play with! There are many website where you can click around and design your own Life animations. Lots of fascinating patterns have been found by people over the years. This can make it a little easier for a researcher to get through the day.

These properties do a good job at mimicking the real universe while being significantly more tractable.

But those who play with Life will notice some odd things about it that are not mimicked in the real world, especially if they're familiar with physics;

There's no obvious conversation laws, of either momentum or energy.
Objects often just disappear.
Structures are very brittle; most interactions between object-like things destroy both objects.
The rules are not reversible; two states can both evolve into the same state, meaning that you can't "know" where you came from just by looking at the current state.

These properties are related, and specifically, I think that adopting a reversible rule gives you most of the other properties.

Because the rules are reversible, any evolution on a fini...