trentmkelly/LessWrong-43k huggingface dataset:
Does ChatGPT’s performance warrant working on a tutor for children? [It’s time to take it to the lab.]
by Bill Benzon — 2022-12-19T15:12:05.804Z
Link: https://new-savanna.blogspot.com/2022/12/does-chatgpts-performance-warrant.html

Of course we already have sophisticated robot toys and companion robots for, e.g. older people, but I have no direct experience with any of these. Tutoring is quite different.

Here’s a dialog I had yesterday with ChatGPT:

Tell me about the digestive process. We put food and drink in our mouths, then what happens?

The digestive process begins when you put food and drink in your mouth. When you chew, you mechanically break down the food into smaller pieces, which makes it easier to digest. As you chew, your salivary glands produce saliva, which moistens the food and helps to break it down further.

After you swallow the food, it passes down the esophagus, a muscular tube that connects your mouth to your stomach. The muscles in the walls of the esophagus contract in a coordinated way, a process called peristalsis, to push the food down towards the stomach.

Once the food reaches the stomach, it is mixed with stomach acid and enzymes, which continue to break it down into smaller pieces. The stomach also mixes the food with digestive juices and churns it to form a thick liquid called chyme.

From the stomach, the chyme is slowly released into the small intestine, where it is mixed with bile, a greenish-yellow fluid produced by the liver and stored in the gallbladder. Bile helps to digest fats and absorb fat-soluble vitamins.

As the chyme moves through the small intestine, the walls of the intestine absorb the nutrients from the chyme. The small intestine is lined with tiny finger-like projections called villi, which help to increase the surface area for absorption.

Any undigested food and waste products pass into the large intestine, where most of the water and electrolytes are absorbed back into the body. The remaining material, which is mostly indigestible fiber and bacteria, is compacted and eliminated from the body through the rectum and anus as feces.

That's a general overview of the digestive process. It's a complex and highly coordinated process involving many different organs and systems in the body.

Young children don’t understand things as well as adults do. That’s too complicated for a three-year old. How would you explain the digestive process to a three-year old?

Sure, I can give you a simpler explanation of the digestive process for a three-year-old child.

When we eat, our body breaks down the food into tiny pieces that our body can use for energy and nourishment.

...
Conditions for Superrationality-motivated Cooperation in a one-shot Prisoner's Dilemma
by Jim Buhler — 2022-12-19T15:00:37.802Z

Summary

It has been argued that, if two very similar agents follow decision theories allowing for

superrationality

(e.g.,

EDT

and

FDT

), they would cooperate in a

prisoner’s dilemma

(PD) (see e.g.,

Oesterheld 2017

). But how similar do they need to be exactly? In what way? This post is an attempt at addressing these questions. This is, I believe, particularly relevant to the work of the

Center on Long-Term Risk

on acausal reasoning and the foundations of rational agency (see

section 7 of their research agenda

).

I’d be very interested in critics/comments/feedback. This is the main reason why I’m posting this here. :)

Normal PD

Consider this traditional PD between two agents:

Alice/Bob	C	D
C	3, 3	0, 5
D	5, 0	1, 1

We can compute the expected payoffs of Alice and Bob (UAliceand UBob) as a function of p (the probability that Alice plays C) and q (the probability that Bob plays C):

UAlice(p,q)=4q−pq−p+1

UBob(p,q)=4p−qp−q+1

Now, Alice wants to find p∗ (the optimal p, i.e., the p that will maximize her payoff). Symmetrically, Bob wants to find q∗. They do some quick math and find that p∗=q∗=0=0, i.e., they should both play D. This is the unique Nash equilibrium of this game.

Perfect-copy PD

Now, say Alice and Bob are perfect copies. How does it change the game presented above? We still have the same payoffs:

UAlice(p,q)=4q−pq−p+1

UBob(p,q)=4p−qp−q+1

However, this time, p=q. Whatever one does, that’s evidence that the other does the exact same. They are decision-entangled[1].

What does that mean for the payoff functions of Alice and Bob? Well, decision theorists disagree. Let’s see what the two most popular decision theories (CDT and EDT) say, according to my (naive?) understanding:

EDT: “Alice should substitute q for p and her formula. Symmetrically, Bob should do the exact opposite in his”.

UAlice(p,q)=4p−pp−p+1
UBob(p,q)=4q−qq−q+1

CDT: “Alice should hold q fixed. Same for Bob and p. They should behave as if they could change their action unilaterally through some kind of magic.” Therefore, CDT computes the dominant strategy from the original payoffs, ignoring the fact that p=q.

UAlice(p,q)=4q−pq−p+1
UBob(p,q)=4p−qp−q+1

For CDT, p∗=q∗=0 just like in the normal PD above. For EDT, however, we now get p∗=q∗=1 (Alice and ...
Next Level Seinfeld
by Zvi — 2022-12-19T13:30:00.538Z

Thanks to Jordan Schneider gifting a last minute ticket for an amazing seat, serendipity led me to go to Friday night’s performance by Jerry Seinfeld at the Beacon Theater.

If I had to describe the show in one word, I’d say it was unsurprising. Jerry was Jerry. He had thoughts. Mostly he had complaints. They were all Very Seinfeld.

If I had two words I might say mildly amusing. Which was good enough for a worthwhile evening. Live performances are something special. Every time I’ve gone out to a comedy show, even if a bunch of it was kind of lame, I have been happy I came. The correct bar for worth watching is actually lower in person than at home.

The thought I couldn’t shake as I went home was, what GPT level was that on?

This first came to me during the opening act. His opening act was if anything too on the nose. Very Jerry. Much Seinfeld.

It felt very much like he had GPT-X put together a package of Standard Mildly Amusing Jokes and Perspectives, with the prompt that the performer was black, old, male, never married and the opening act for Jerry Seinfeld. Given the material, delivery was solid.

The question was, what was the X in GPT-X?

I am confident our current incarnation, which is GPT-3.5 or so, can’t do it.

What about GPT-4? Could go either way, if you could try out bits and select winners.

What about GPT-5? Yes, absolutely, if it is what I’d expect, that should work.

So I’d give that act a GPT-level rating of 4.25.

Jerry himself was on a higher level. Not as high as his peak. I’d give him a 5.

What about the best stand-up shows and specials?

I’ve seen 6s. I worry I may have never seen a 7.
CEA Disambiguation
by jefftk — 2022-12-19T13:20:05.558Z

In 2011 effective altruists in Oxford had two main organizations:

Giving What We Can and

80,000 Hours. They wanted to
incorporate, and created the

Centre for
Effective Altruism as an umbrella organization to host them: [1]

Over time they started running other projects: conferences, supporting local
groups, the EA
forum, community
health, etc. There was effectively a "CEA" community-focused
organization within the "CEA" umbrella organization:

This was pretty confusing: when someone said "CEA" did they mean the
organization focused on the EA community ("CEA runs EA Global") or the
umbrella organization ("80k is part of CEA")? This got even more
confusing as there started to be more organizations and projects:

In September 2022 the umbrella organization renamed itself to the
Effective Ventures Foundation:

Unfortunately the announcement
wasn't very clear about what specifically was changing, and a lot of
people are still confused about when to say "CEA" and when to say
"EV". Hopefully this history and the diagrams clear things up a bit!

[EDIT: changed 'EVF' to 'EV'; Shakeel
says they prefer the latter.]

[1] This is also the origin of using effective
altruism to refer to the movement.
Why mechanistic interpretability does not and cannot contribute to long-term AGI safety (from messages with a friend)
by Remmelt — 2022-12-19T12:02:37.160Z

tl;dr: Reasons why the scope of application of mechanistic interpretability is too limited to prevent long-term lethal AGI misalignment. Hooks into reasoning, not covered below, why any physically possible methods to inspect internals (and externals) are insufficient for correcting out eventual carbon-life-toxic interactions of AGI with the environment.

Message exchange with a friend

How to read below:

In this private message exchange, a friend tried to clarify my conceptual and concrete technical reasons for why mechanistic interpretability falls short
(building on more general arguments from a researcher much smarter than me).
Most of my side of the exchange is below, in more or less chronological order. Regular text are verbatim copy-pastes, save for light spelling and format edits. Italicised text I added later to offer context.
I added minimal quotes of my conversation partner to show what I was replying to, with their kind permission.

On an important side-tangent – why I think Eliezer Yudkowsky does not try to advocate for people to try to prevent AGI from ever being built

[2:09 pm, 08/11/2022] Remmelt:
> A (kind of) random question that just came up in my brain:
why do you think Eliezer Yudkowsky doesn’t try to advocate for people to try to prevent AGI from ever being built?
> Is he, relative to you, just too pessimistic that this is possible, and does he put more hope into alignment being possible even though he’s also very pessimistic about that?
Yeah, good question!

My impression from reading recent writings by Eliezer is that he mentally models the social coordination problem (which is definitely hard) as practically impossible because you have all these independent actors making decisions about developing advanced AI systems and you cannot really force them in any grand orchestrated scheme to stop them from acting.

[2:10 pm, 08/11/2022] Remmelt:
> Is he, relative to you, just too pessimistic that this is possible, [...]
And based on that he concludes something like you said.

[2:10 pm, 08/11/2022] Remmelt:
But humans are not independent actors. We’re social actors, and social interactions imply a sort of interdependence in people’s actions.

[2:12 pm, 08/11/2022] Remmelt:
Also, we share the same needs to continue to exist as wetware containing soups of carbon-chained molecules.

[2:12 pm, 08/11/2022] Remmelt:
And we share billions of years

...
Hacker-AI and Cyberwar 2.0+ 
by Erland Wittkotter — 2022-12-19T11:46:24.380Z

I have been hacked, and some (selected) files have been deliberately deleted - this file/post was part of it. I found suspicious software on my computer; I made screenshots - these screenshots were deleted together with several files from my book “2028 - Hacker-AI and Cyberwar 2.0”. I could recover most from my backup.
So far, I have hesitated to post sensational warnings on using smartphones in malware-based cyberwars designed to decapitate governments. Still, I wrote this paper out of concern that this could happen – and cheap marketing talk does not and will not change that.
I don’t know who hacked me, but I assume they got this post. It is spicy (and scary). It is published here to protect my family and myself. I have beefed up my security. Certainly, these measures are no match for what I believe attackers could do. Therefore, I established measures to guarantee that my research will survive and be published before it is officially published as a book.
I changed my password credentials, but I am under no illusion that this is enough to protect this publication. I have asked friends to save and print this post on their local systems (and not to sign into this site while doing this). I will certainly repost it if an impersonator deletes (or modifies) it with my credentials. (Lesswrong doesn’t have 2FA – and I am not suggesting that because I assume it is insufficient anyway).

Cyberwar is currently fought in two main forms: (1) propaganda, disinformation, and weaponization of social media and (2) disruption of services like electricity, communication, and Internet (information) services. Other considered forms of cyberwar are aimed at what wars usually do: create indiscriminate destruction for innocent casualties, i.e., increasing the cost of war. In the last decades, (kinetic) weapons got surgical/limited in their damage-related applications. Overall, waging war is very expensive – in human lives, economically/socially, and environmentally. War also has costly, unpredictable side effects, which deters nations from using military means to pursue political goals. This fundamental calculus will change with Hacker-AI and Cyberwar 2.0+. Hacker AI can make cyberwar a low-risk, low-cost decision. Cyberwar can become a compelling choice due to significant first-strike advantages and fabricated misdirections/accusations against innocent parties.

1. Hacker-AI

Hacke...
Non-Technical Preparation for Hacker-AI and Cyberwar 2.0+
by Erland Wittkotter — 2022-12-19T11:42:52.289Z

I have been hacked, and some (selected) files have been deliberately deleted - this file/post was part of it. I found suspicious software on my computer; I made screenshots - these screenshots were deleted together with several files from my book “2028 - Hacker-AI and Cyberwar 2.0”. I could recover most from my backup.

So far, I have hesitated to post sensational warnings on using smartphones in malware-based cyberwars designed to decapitate governments. Still, I wrote this paper out of concern that this could happen – and cheap marketing talk does not and will not change that.

I don’t know who hacked me, but I assume they got this post. It is spicy (and scary). It is published here to protect my family and myself. I have beefed up my security. Certainly, these measures are no match for what I believe attackers could do. Therefore, I established measures to guarantee that my research will survive and be published before it is officially published as a book.

I changed my password credentials, but I am under no illusion that this is enough to protect this publication. I have asked friends to save and print this post on their local systems (and not to sign into this site while doing this). I will certainly repost it if an impersonator deletes (or modifies) it with my credentials. (Lesswrong doesn’t have 2FA – and I am not suggesting that because I assume it is insufficient anyway).

Hacker-AI is AI used in hacking to increase speed for creating new malware that is easier and more targeted in its use. It could become an advanced super-hacker tool (gaining sysadmin access in all IT devices), able to steal crypto-keys, any secret it needs or uses on devices every available feature. As discussed in “Hacker-AI and Digital Ghosts – Pre-ASI” and “Hacker-AI – Does it already exist?”, Hacker-AI helps malware to avoid detection (as a digital ghost) and make itself irremovable on visited devices.

In the paper/post “Hacker-AI and Cyberwar 2.0+”, I have hypothesized that Hacker-AI is an attractive cyber weapon for states waging cyberwar by facilitating the following cyberwar-related actions or capabilities:

Surveillance (audio, video, and usage) of smartphones or other IT devices - at scale (every system)
Selective access denial for some people caused by their device’s malware
Direct threatening of targeted people via AI bots on devices
Realtime Deep-Fakes and their

...
An Effective Grab Bag
by stavros — 2022-12-19T10:29:09.630Z

Preface

I've been exploring low cost ways to increase resilience in the AI safety community as part of my work on Alignment Continuity; grab bags are one such intervention.
Since the Russia-Ukraine war kicked off, I've had a few EAs ask me about this kind of thing and so I decided that instead of just making a bag I'd make a guide too.
Thanks to everyone that helped and offered feedback and suggestions.

Overview

Banana for scale, not emergencies.

Grab bags are a tool for increasing your resilience in emergencies.

Not to be confused with 'bug out bags' - they are not intended to be your sole resource for several days/weeks spent living in the woods - it helps to think of a grab bag as an emergency toolkit.

A grab bag is defined as much by where it is as by what's inside it - it's something you have with you, or nearby, most of the time; it's something you can grab with only a moment's notice.[1]

For example, if there's a fire you should be able to reach your bag without delaying your exit from the building.

A few good places to keep a grab bag:

Your car
Your home office
By your front door

Contents and Packing

The contents are organized into several bags[2] - a first aid kit, [yourstuffhere], hygiene kit, utility kit, food and water, and spare clothes (also yours).
The bags are packed in a specific order so everything is easily accessible and the first aid kit is the first thing you see when you open the bag.

Unpacking the bag, also gif version

On the outside of the bag I've added a small LED dongle for ease of access in dark environments and a USB flash drive.

Packing the bag, also gif version

Your Bag

The Bag

A rucksack or backpack is a good option, but I've chosen to build around a drybag - highly waterproof, rugged and much cheaper than a waterproof backpack.

All drybags are more-or-less created equal - this one comes with reflective patches, plenty of attachment points, and a shoulder strap.

Size

20 Litres is the sweet spot for a one person bag, if you opt for a 10L bag you're going to have to make difficult sacrifices with the contents.
If you find yourself needing 30-40L, for example if making a grab bag for 2+ people, then it might be wiser to use a backpack.

Your Bag

An off the shelf grab bag isn't especially effective. Everyone's needs and situation are different and so the contents of every bag should be tailored to the owner.
Below are some suggestions t...
Slick hyperfinite Ramsey theory proof
by Alok Singh — 2022-12-19T08:40:06.610Z

Blog power laws

My dissection post is 80% of this blog's
traffic. Before writing that, 80% of traffic came from Vim's conceal
feature.

The math posts are almost never read, which is depressing because they're the
ones with the least obvious insights. The few that do read them tend to really
like them.

This post isn't going to buck that trend, since it won't make sense unless you
already know nonstandard analysis, and this isn't going to be the post that
teaches you nonstandard analysis either.

Claim

Let g be a graph where every finite subgraph is n-colorable. Then the whole graph g is n-colorable.

Proof

If g is finite, then the whole thing is trivial because g is a finite subgraph of itself and is n-colorable by hypothesis, and we're done.

Now assume g is infinite. Construct g*, the nonstandard extension of g. It is a fact (which I don’t prove but appeal to Joel David Hamkins) that any infinite structure can be embedded inside a larger hyperfinite one. The trick of approximating an infinite thing by a larger hyperfinite one is called hyperfinite approximation. In this case, we construct H, a hyperfinite subgraph of g* that contains g

In symbols: $$g \subseteq H \subseteq g^*$$.

$$H$$ is $$n$$-colorable, because all finite subsets of $$g$$ are, so by the Transfer Principle all (hyper)finite subsets of $$g^*$$ are too, including $$H$$.

But $$g$$ is contained in $$H$$, so it can't require more colors than $$H$$ does. Therefore, $$g$$ is $$n$$-colorable $$\QED$$.

This is the Compactness Theorem in disguise because "all finite substructures" carry over to the whole infinite structure.
The True Spirit of Solstice?
by Raemon — 2022-12-19T08:00:30.273Z

A year ago at the Winter Solstice afterparty, I had a 5 hour discussion with various folk about what things are or aren't appropriate for Solstice. Some example debates:

"Is Solstice primarily a rationality holiday? An EA holiday? The broader secular community?"
"How essential is the journey from light, into darkness, into light?"
"Is it okay to have a Solstice where we don't sing Brighter Than Today?"
"How important are singalongs vs speeches?"
"How important is it for singalongs to sound polished, vs for them to feel like an organic part of the community? Is it appropriate to pay professional musicians?"
"How important is transhumanism or x-risk?"
"Is it good or bad to change lyrics over time?"
"How important is it to celebrate Solstice on literal astronomical Solstice? If you don't, why are we calling it Solstice? Is it important for the name to be clear?"
"Is it okay to have one solstice someday with a 'bad ending', where instead of climbing back out of the darkness hopefully, we just... sit with it, and accept that maybe it might be what the future holds?"

Many of these answers are subjective. But are they entirely subjective? I think this is a super interesting question.

I generally support individual solstice organizers doing whatever they think is right, and not worrying much about what other people think. But I think it's an interesting question whether or not a "True Spirit of Solstice" exists.

You can have whatever arbitrary aesthetics you want. But, some aesthetics can be more or less self-consistent than others. And some aesthetics can be more, or less, in alignment with reality than others.

It's important to be able to propagate facts into your aesthetics.

Ancient, Eternal, and yet Personal

Here are two reasons I think people want holidays:

To be connected to something greater than themselves. To know that they are part of a story that started before them and will continue after them, something vast and eternal and powerful and beautiful.
To celebrate something oddly specific about them. My tribe, my family. A costly signal that I am part of this ingroup instead of that other one over there.

These are in tension. Holidays vary in how much of each they offer.

Often, people want to know that they are not alone in their oddly-specific narrative. That their oddly-specific-ness is, in fact, powerful and vast or at least enduring. A centra...