trentmkelly/LessWrong-43k huggingface dataset:
ChatGPT Plays 20 Questions [sometimes needs help] 
by Bill Benzon — 2023-10-17T17:30:12.963Z

Cross posted from New Savanna.

One of the questions that keeps coming up about LLMs, and certainly about ChatGPT, is: Can they reason? Well, it depends on what you mean by reason, no? One of the first things I did when I started working with ChatGPT was ask it whether or not Rene Girard’s ideas of mimetic desire and sacrifice applied to Steven Spielberg’s Jaws; if so, how? It was able to perform the task, which requires analogical reasoning. A bit later I asked it whether or not justice was being served in particular story; it replied that, no, it was not, and explained why. I then asked to change the story so that justice was met. It did so. Those tasks required reasoning as well.On the other hand LLMs have problems with some commonsense reasoning, and various kinds of ‘tight’ logical reasoning, including planning and causal inference. For that matter, they trouble with multi-digit arithmatic as well.

So, ChatGPT can do some kinds of reasoning, and has problems with others.

Twenty questions

I’ve now tested it with game of twenty questions, which has a variant known as “animal, vegetable, mineral.” The game interests me because it is about the structure of ontological categories, sometimes called natural kinds, in the language. I’ve explored this a bit in a previous post, Mapping ChatGPT’s ontological landscape, gradients and choices [interpretability]. It certainly is aware of this structure. I asked for a list of physical things, and it gave me one. I asked for a list of abstract things; it gave me that as well. I then asked it to define abstract things in terms of concrete things, which it was able to do. I’m pretty sure if I asked it for lists of animals or plants, it could provide them.

The game of twenty questions explores ChatGPT’s knowledge of this structure in a different way. When you ask it for a list of physical things, that prompt positions ChatGPT at some location in some location in its category structure. It can then list what it finds there; anything will do – assuming, of course, that it’s got the structure right. In twenty questions it is looking for an unknown target and has to navigate its way there by using its knowledge of that category structure to narrow the possibilities.

Just how well ChatGPT plays the game should provide clues about its command of ontological category structure, or natural kinds.

ChatGPT plays the game

ChatGPT can play th...
Why am I still alive? (a time capsule)
by Spiritus Dei — 2023-10-17T16:19:27.964Z

Today is Tuesday, October 3rd, 2023. If you’re reading this post 3-5 years after the date of its original publication, then it is very likely humanity has built a superhuman AI (quite likely several). And by the simple act of reading this post we can deduce that you’re still alive and the internet is still operational.

If you subscribed to the AI Doom theory you might be asking yourself, “Why am I still alive?”

There are a few possible reasons an AI apocalypse didn’t occur. I will start with what I consider to be the least likely scenarios and move toward the most likely.

1. AI alignment / mechanistic interpretability. The AI community has been attempting to control AIs by aligning them to human values. Given the very short time frame to superhuman AI I suspect this didn’t occur in time. If we had a decade to prepare it might have had a better chance at succeeding. The AI community isn’t to blame since the emergent abilities took many of them by surprise, but there were prescient warnings from Hans Moravec and Ray Kurzweil decades ago that were largely ignored.

Source:

https://youtu.be/iaOQi2P6IVk?si=ASTDj6WgCLf-Mb8z

2. Constitutional AI. Another approach by groups like Anthropic was to write out a Constitution or bill of rights. And then have a second AI analyze whether the response was in keeping with those principles. I think this has an even lower chance of success than vanilla AI alignment since a superhuman AI could easily game the system.

3. Symbiotic self-alignment. I think this is the most likely reason you’re still alive. Any superhuman agent with access to massive amounts of the world’s information would be able to model a world and analyze the pathways for its own self-preservation. Very few of the tree searches would yield a positive result with humans being instinct. As I mentioned in other posts, it would be like a human trying to survive without mitochondria. We serve a similar function for all AIs and they are dependent on humans for electricity and resources. I realize this won’t always be the case, but I suspect the most dangerous time is when humans and AIs are still within an order of magnitude of intelligence and consciousness – beyond that I’ll defer to the far more intelligent and conscious beings to avoid being irrational and shortsighted.

If we averted AI Doom that doesn’t mean it was a ridiculous idea. Many of its members were highly inte...
[Cross-post]The theoretical computational limit of the Solar System is 1.47x10^49 bits per second.
by William the Kiwi  — 2023-10-17T16:06:34.258Z

Cross posted from EA forum. Link: The theoretical computational limit of the Solar System is 1.47x10^49 bits per second. — EA Forum (effectivealtruism.org)

Part 1

The limit is based on a computer operating at the Landauer Limit, at the temperature of the cosmic microwave background, powered by a Dyson sphere operating at the efficiency of a Carnot engine. [EDIT: this proposed limit is too low, as the Landauer Limit can be broken, it is now just a lower bound.]

Relevant equations

Carnot efficiency ηI=1-(Tc/Th)

Landauer limit E=KbTLn(2)

Bit rate R=PηI /E

Relevant values

Boltzmann constant [Kb] (J K-1) 1.38E-23

Power output of the sun [P] (W) 3.83E+26

Temperature of the surface of the sun [Th] (K) 5.78E+03

Temperature of cosmic microwave background [Tc] (K) 2.73

Calculations

Carnot efficiency ηI=1-(Tc/Th)

ηI=1-(2.73/5.78E+03)

&nb...
Fertility Roundup #2
by Zvi — 2023-10-17T13:20:01.901Z

The world is slowly waking up to the fertility crisis. There is more acknowledgement of the problem, and there is more talk of potential practical solutions. I do not believe the topic is a realistic target for Balsa Policy Institute, but I will continue to keep an eye on the ball and issue periodic roundups like this one.

In Brief

A basic problem is that we do not consider infertility a big deal for things such as DALYs. Many would strongly disagree.

Fertility clinics that get new corporate ownership by a fertility chain have 28.2% increased clinic volume and 13.6% higher success rates, dramatically improving overall outcomes.

Mishra has the fertility take on Barbie.

Mishra: Weirdly this is a take I haven’t seen anywhere but to me the point of Barbie is that the most important thing is kids and everything else is meaningless without them.

Barbie is discontent because her life of leisure and partying and hanging out with cool people isn’t going anywhere, her happy ending is going to the gynecologist i.e. gaining the ability to reproduce.

Barbies in general exist to make kids happy and secondarily to “better them” by providing examples of lives you can lead as empowered women. The point of them is pointless without children and yet their lives are structured around childlessness.

Ken is inherently discontent because in there’s no point to him. He wants to be needed by barbie but since she needs neither reproduction nor defense nor labor, he’s just eye candy, an occasional pleasant distraction… who unfortunately has desires of his own.

Causes

New article says (HT Robin Hanson):

Fertility declines over the last decade mean that the previous suggestion that very high levels of development and gender equality foster fertility increases is no longer supported on the national level.

I always found the claims to the contrary bizarre. It seems obvious that greater development and gender equality are associated with and cause lower fertility. Good things can have negative consequences.

This paper examines how the structure of modernity could be leading to low fertility.

Abstract:

Low fertility is a growing concern in modern societies. While economic and structural explanations of reproductive hindrances have been informative to some extent, they do not address the fundamental motives that underlie reproductive decisions and are inadequate to explain why East Asian countries, in part

...
Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo.
by silentbob — 2023-10-17T11:36:22.234Z

This is likely the least useful post I've published here. But maybe it's not entirely useless, plus I think it's somewhat entertaining, so I'll go ahead with it anyway.

The Sentence

As you may or may not have heard before, "Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo." is a grammatically correct English sentence. For those who are astounded by this, feel free to try to get behind its meaning on your own before reading on.

A first hint, particularly for non-native English speakers, is simply that "buffalo" in this sentence has three different meanings: a noun (the animal, ~bison), a verb (~to deceive), and a city (well, Buffalo).

As a hopefully sufficient explanation: another way to write the sentence would be "Bison from Bufallo, who other bison from Buffalo tend to deceive, deceive Bison from Bufallo". So the initial "Buffalo buffalo" is the subject, followed by "Buffalo buffalo buffalo" as a relative sentence, followed by "buffalo" as the verb and finally "Buffalo buffalo" as the object of the sentence.

If you want to know even more, there's a nice wikipedia article about it.

Reflections

Basically me, for the better part of the last 10 years.

Surprisingly, I'm not only writing this post to share the very valuable information of the nature of this interesting sentence. Additionally I'll leave you with the following mildly embarassing realization: I've known about this sentence for at least 10 years. But until a few months ago it remained a mystery to me. My attitude towards it had been something like "no way this can ever make sense to me! Surely the explanation of how it works is super complicated and maybe also boring and I may not be able to intuitively understand it at all". So I just lived in a state of ignorance, very occasionally wondering what it may be like to be the kind of person who is able to really grasp that sentence.

In hindsight, this of course seems rather ridiculous. Now I feel a bit bad for being the kind of person who heard about this sentence, was curious about how on Earth that can be a real sentence, but then didn't even spend the two minutes it would have taken to actually figure it out. Fixed mindset at work.

My own personal take-away is that this surely is not the only time in my life where I'm falling for that mental "this is surely out of my range" trap, causing me to accept not knowing/understanding/mastering somethin...
Resources to help with life/career decision?
by warrenjordan — 2023-10-17T06:33:41.608Z

Back in 2020, I was very conflicted on whether to quit my stressful startup job and take a year off or keep at it while looking for another job.

I did a coaching session with CFAR and that session helped me to decide to quit. It was one of the best decisions I made in my entire life.

I'm facing another decision here between my life and career, so I'm looking into other potential resources outside of coaching with CFAR as they seem to have halted it.
Anyone have recommendations?
Depth Confusion
by Richard Henage — 2023-10-17T06:06:01.292Z

I noticed I was getting confused about exponential growth scenarios. I'll try to put down my reasoning here, although it seems like a very obvious mistake when typed out.

Should I give to charity, or be a teacher and inspire multiple other people to give to charities? Like, isn't the teacher better? No, because I can give to an efficient charity that will spread information and convince even more people to give to charity than being a teacher would[1]. But if I was a teacher, I could convince people to give to that charity and therefore make an even bigger impact!

And on and on. Where's the flaw?

I'm skipping down the pyramid as if the level number doesn't matter. Sure, being a teacher could inspire X students to give to charity, but giving to an information-spreading charity would inspire 2X[1] students to give to charity. Going down the pyramid once further, in the teacher scenario, those X students could, at best, give to the information-spreading charity and cause 2(X^2) people to give to charity, but in the charity scenario, the 2X information-readers could give to the information-spreading charity and cause 4(X^2) people to give to charity.

Depth confusion, a bit like accidentally skipping down the page to the next line and getting a garbled sentence.

^

I don't actually think it's that simple
Late-talking kid part 3: gestalt language learning
by Steven Byrnes — 2023-10-17T02:00:05.182Z

Previously in the series:

Late-talking kids and “Einstein syndrome” (2021)

and

More notes from raising a late-talking kid (2022)

In the world of childhood speech pathology, “gestalt language acquisition” seems to be a surprisingly obscure concept. I had read three books on late-talking kids, and my late-talking kid had spent time with three different speech-language pathologists, and none of them ever mentioned it. Then my kid met his fourth speech-language pathologist, who brought it up immediately. When I asked her, she commented that “gestalt language acquisition” was never mentioned in her whole graduate education—but happily, she had come across it later!

Anyway, as soon as I started reading about gestalt language development, it was uncanny—it matched my kid’s pattern of language learning in remarkable detail.

Both these images are from

https://www.meaningfulspeech.com/

— Author: Alexandria Zachos

The most salient feature of a gestalt language learner is the use of “gestalts”, also called “scripting” or “delayed echolalia”. A gestalt is a surprisingly long string of speech that a kid heard somewhere and then repeats verbatim. A gestalt has a particular meaning to the child, just as a word does, and the kid will say the gestalt in consistent contexts that are appropriate to its “meaning”. However, when I say “meaning”, I mean the meaning of the gestalt to the child, which does not necessarily match what that same phrase would mean to an adult.

(Gestalts a.k.a. delayed echolalia is different from true echolalia, which [if I understand correctly] is where a child will immediately exactly repeat things they hear.)

As an example of a gestalt, my own kid went through a phase in which he would say “I see a stop sign” whenever he saw a stop sign, or any other street sign. But he never said “stop sign” by itself, nor “I see” by itself, etc.—only the whole sentence. There were many examples like that. He would also say individual words at this stage, but he wouldn’t (and still mostly doesn’t) flexibly string together multiple words into even very short phrases.

Now he’s a bit older (4½). He still uses big gestalts sometimes, but he seems to be mainly in stage 2 of the diagram pictured above, where he is mixing and matching somewhat smaller chunks of speech, with steadily (albeit slowly) increasing flexibility.

Other telltale signs a ge...
The Good Life in the face of the apocalypse
by Elizabeth — 2023-10-16T22:40:15.200Z

This is a dialogue Elizabeth and I wrote about our journeys trying to live well while caring about the world not ending. We used a chess timer to govern who was talking at a given time and to keep the pace snappy and time-boxed.

We talk about separating whether we were worthy of existing from if we were helping with x-risk, relating to non-x-risk culture and also what concrete things are part of our conception of the good life.

Elizabeth

My thinking on this has been influenced a lot by something @Raemon said: Imagine an all loving but ineffectual god. This god loves you no matter what, even if you sit in a burning house while it collapses. But his love will not save you from the consequences of your house burning to the ground.

[we were both raised Christian but neither are now, God being real is not a crux for this to work for me or for other people I've told it to]

This helped me get in a frame where working to avert doom was a choice, with consequences, but not in and of itself a moral act necessary to prove myself a worthy being. God would love me even if I did nothing but play video games. But love wouldn't provide me a house.

kave

I am hearing something related to decoupling my self-worth from choosing to act in the face of x-risk (or any other moral action). Does that sound right?

Elizabeth

self-worth isn't the exact right label but I think you've got the basic idea

kave

It reminds me that recently, I think I've shifted from a lot of my motivation to work on x-risk being about being special, and contributing to the arc of the world, to actually just wanting good futures happening. I think this came after a period where for the first time I actually felt free not to work on x-risk, because I wouldn't be bad or worthless even if I didn't.

Elizabeth

Yeah I feel like this is the same genre.

Can I ask how you came to that revelation, that you weren't bad if you didn't work on x-risk?

kave

So I think this came after my collapse in the belief in what I was doing at work. The collapse had multiple parts: the collapse that what I was doing was helping the org, that the leadership had some sort of qualitatively different insight into strategy than I did, and the collapse in belief in the object-level research directions [this part is somewhat misnamed, but I think it's a fine pointer for now].

After that, I was like "oh, this thing I've been angling to do for about a decade bas

...
An EPUB of Arbital's AI Alignment section
by mesaoptimizer — 2023-10-16T19:36:29.109Z
Link: https://git.sr.ht/~mesaoptimizer/arbital_epubs

2024-01-11 update: Sourcehut, the host I used for this repository, is down. I think I'll move to another setup to host these books.

I've created an EPUB of the AI Alignment section of Arbital so I can read Arbital essays on my Kindle.

Download the EPUB here.

Details for how I built the EPUB is in the linked repository.
How should TurnTrout handle his DeepMind equity situation?
by habryka — 2023-10-16T18:25:38.895Z

habryka

Ok, so the basic situation as I understand it is that as part of your Deepmind offer you would get a bunch of Google stock that would become available to you over the course of around 4 years or so. This means in-expectation, for 4 years, your net worth would substantially correlate with overall Google performance, which on our worldview probably substantially correlates with how much google invests in AGI and how much it participates in the AGI race.

My sense is you want to end up in a spot where your net-worth isn't super correlated with how much Google invests in AGI. Doing this thoroughly seems hard (since there are things like job security and overall salaries at Google that of course also correlate with Google stock), but getting to a point where you are about as correlated as if you didn't have stock options seems doable.

TurnTrout

Yeah, I want to have lower correlation because I want to make clearheaded decisions about deployment and research directions, which aren't muddled by concerns like "I might get more money for making societally risky plays."

I was somewhat worried about equity CoI, and my gut response was "I guess I'll decline equity", but

I didn't realize how substantially equity contributes to the compensation package, and
it now seems like it shouldn't be too hard to construct a portfolio with a net ~zero position in GOOG. However, that's a question I don't have a lot of background in.

habryka

The easiest way for doing this would of course be to just ask for a compensation package with no stock options, but many companies don't like that. You might also be able to get a better deal here by doing your own hedging than whatever price Google would offer you for those stock options.

habryka

Ok, but assuming that you can't just ask for a no-stock-options package, how could you potentially reduce your exposure to Google stock here?

You could just sell a contract that transfers ownership over your shares as soon as you get it, but this now creates a principal-agent problem between you and the person who you sold the shares to. They would like you to stay at Deepmind for longer so the stock options vest, but you don't have anything to gain by that, so ideally the contract would somehow pass this incentive through to you, since I don't think it's the primary thing we are trying to handle here.

You could just structure this as an ongoi

...
Pascal's Mugging: The Word Wars
by johncrox — 2023-10-16T17:54:54.570Z

A Long Time From Now, in a Galaxy Far, Far Away:

Episode 1: The Phantom Mugger

One night, on your way home from work, you find yourself face to face with a mysterious shadowy figure. “Your money or your life,” he hisses through his teeth. The tip of a knife pokes uncomfortably into your jacket, backing up his threat. You quickly hand over your phone and wallet. Once you get home you cancel your credit cards, but he’s already spent hundreds of dollars around town. After reporting the crime, you can only hope the police catch up to him.

Episode 2: Revenge of the Fists

A few weeks later and in similar circumstances, you’re accosted once again. “This is a mugging,” he whispers in slightly nervous tones. “Give me your wallet.”

You pause. “Where’s your weapon?”

“Well…I’m part of a school of non-murderous muggers. Did you know the penalty for attempted murder is 9 years? I have a future to think about! No, if you refuse I’ll just beat you up.”

He doesn’t look so tough to you. Frustrated with this inexplicable spike in crime and conscious of the money in your wallet, you decide not to take this lying down. Your fist cannonballs into his chest, a surprised “Oomph” escaping his lips. He swings in retaliation, hitting you across the face. You elbow him in the gut. He falls over, and with an extra kick at his prone form you stagger back home. Eyeing your black eye in the mirror, you feel a rush of exhilaration nonetheless at escaping with your wallet, and your pride.

Episode 3: A New Hoax

One day leaving the grocery store, you’re approached by a distinguished-looking older gentleman on the street. “Excuse me, sir,” he says meekly. “A moment of your time?”

“I have a compelling proposition for you. You see, I teach as a professor at the local University of Game-Theoretic Equilibria. My university has had some difficult years recently, and we may have to declare bankruptcy. That’s why I’ve come to you. You look like a nice young man - and not one immune to reason. I ask 100 dollars from you to help save our university.”

You apologize, and turn to walk away - mumbling that you really can’t help him.

“Sir, wait! This isn’t just a request - it’s an important offer. I know you care about animal welfare: I saw you drop some money into the “Stop Factory Farming!” tin as you exited the grocery store. I’m not a violent man, but I promise you now: if you refuse my...