trentmkelly/LessWrong-43k huggingface dataset:
Spring Meetups Everywhere 2023
by Scott Alexander — 2023-04-11T00:59:20.265Z
Link: https://astralcodexten.substack.com/p/spring-meetups-everywhere-2023

Many cities have regular Astral Codex Ten meetup groups. Twice a year, I try to advertise their upcoming meetups and make a bigger deal of it than usual so that irregular attendees can attend. This is one of those times.

This year we have spring meetups planned in over eighty cities, from Tokyo to Punta Cana in the Dominican Republic. Thanks to all the organizers who responded to my request for details, and to Meetups Czar Skyler and the Less Wrong team for making this happen.

There's a map of these meetups on the LessWrong community page.

Within each section, it’s alphabetized first by region, then by city - so the first entry in Europe is Basel, Switzerland. Sorry if this is confusing.

I’ll provisionally be attending the Berkeley meetup on May 6th. Skyler will provisionally be attending Manhattan, DC, Boston, Philadelphia, and Berkeley.

Extra Info For Potential Attendees

1. If you’re reading this, you’re invited. Please don’t feel like you “won’t be welcome” just because you’re new to the blog, demographically different from the average reader, or hate ACX and everything it stands for. You’ll be fine!
2. You don’t have to RSVP or contact the organizer to be able to attend (unless the event description says otherwise); RSVPs are mostly to give organizers a better sense of how many people might show up, and let them tell you if there are last-second changes. I’ve also given email addresses for all organizers in case you have a question.

Extra Info For Meetup Organizers:
1. If you’re the host, bring a sign that says “ACX MEETUP” and prop it up somewhere (or otherwise be identifiable).
2. Bring blank labels and pens for nametags.
3. Have people type their name and email address in a spreadsheet or in a Google Form (accessed via a bit.ly link or QR code), so you can start a mailing list to make organizing future meetups easier.
4. If it’s the first meetup, people are probably just going to want to talk, and if you try to organize some kind of “fun” “event” it’ll probably just be annoying.
5. It’s easier to schedule a followup meetup while you’re having the first, compared to trying to do it later on by email.
6. In case people want to get to know each other better outside the meetup, you might want to mention reciprocity.io, the rationalist friend-finder/dating site.
7. If you didn’t make a LessWrong event for your meetup, the LessWrong team did it for you using the email add...
A Tale of Two Intelligences: xRisk, AI, and My Relationship
by xRiskAnon923 — 2023-04-10T23:35:45.375Z

[Throwaway account for obvious reasons]

As a longtime veteran of LW, I'm no stranger to grappling with complex and uncomfortable ideas. I've always taken pride in my ability to engage in rational discourse and seek truth, even when it's inconvenient or challenging. That's why I find myself in a particularly distressing situation involving my partner, who is herself a highly intelligent individual, holding an advanced technical degree from a top university.

Recently, I've become increasingly concerned about xRisk from AI, a topic I'm sure many of you can relate to. The more I engage with Eliezer's writings and other LW content, the more alarmed I become about the potential dangers of AGI. However, my partner seems completely unbothered by these concerns.

In an attempt to bridge the gap, I sent her some of Eliezer's posts on AI risk, hoping they would make her take the issue more seriously. But her response was dismissive at best. She said that she didn't think the risks were a big deal and that we shouldn't worry about them.

Her arguments, which I found shallow and unpersuasive, revolved around the notions that "the good AIs will just fight the bad AIs" and "anyone intelligent enough to cause trouble would inherently understand the positive sum nature of the world." As many of us here know, these arguments don't account for the myriad complexities and potential catastrophic consequences of AGI development.

The growing chasm between our views on xRisk has left me wondering whether our relationship can survive this divergence. I'm genuinely considering breaking up with her because of our seemingly irreconcilable differences on this crucial issue. It's not that I don't love her; I just don't know if I can be with someone who doesn't share my concern for humanity's long-term future.

So, fellow LWers, I seek your guidance: How can I help her see the importance of AI risk and its potential impact on the world? Is there a better way to approach this conversation? Or should I accept that our views may never align, and move on?

I welcome your thoughts, experiences, and any advice you might have. The stakes feel incredibly high, both for my relationship and for the world as a whole.
Humans are not prepared to operate outside their moral training distribution
by Prometheus — 2023-04-10T21:44:23.419Z

When people think about how to treat an AI, they probably normally default to the evil AI, that frowns on humans and desires to enslave them out of hate, or the nice android-type AI. A lot of writing in AI Safety has focused on debunking the Evil Skynet-Type AI, and not as much on the Nice Android-Type AI.

In the film “AI: Artificial Intelligence” (because, back then, general audiences actually didn’t know what AI stood for), the AIs are of a very convenient kind. They are human-like androids, with physical bodies, and overall exhibit various human characteristics. It is designed in such a way to be very obvious and easy for its early 2000s audience to understand that it is wrong for the humans to enslave and mistreat them. People simply have to look at the androids and notice that their treatment falls into moral atrocities they’ve seen before: they’re aware of how humans have treated marginalized groups of other humans before, have the perception that this is bad, and can easily apply the same rationale to the androids. Similarly, in the film District 9, aliens land on Earth, but are placed in a situation that conveniently makes their treatment look identical to the treatment of blacks in South Africa during apartheid. Someone might see these sorts of situations in fiction, and think that they have an open mind that proactively sees moral dilemmas of the future, but they are wrong. All they are doing is taking their own normal moral training distribution and swapping in androids/aliens/etc. It’s easy to do, because it’s already been intentionally made to conveniently fit into situations that many humans already view to be bad. No change in mentality required.

Contrast this with Ender’s Game (Spoilers). Ender executes a series of tactics in what he believes to be a simulation. He’s played this kind of game before. It feels normal to him. But what he doesn’t know while playing is that it isn’t a simulation at all, but an actual extermination of an entire alien race. It didn’t feel consequential to him. It didn’t feel like the way you might imagine committing an atrocity would feel. Some will think that’s not fair to say, because Ender was tricked, but I think that’s missing the point. Reality doesn’t grant you massively consequential, and potentially horrific, actions that feel consequential and horrific.

Imagine those in 18th C...
Bing AI Generating Voynich Manuscript Continuations - It does not know how it knows
by Matthew_Opitz — 2023-04-10T20:22:59.682Z

Epistemic status: slightly giddy and freaked out. Possibly rushing to judgment. But there's definitely something here that others should check out for themselves...

The Voynich Manuscript (VMS), for anyone who is familiar with it, has an infuriating liminality. It seems a little like everything but not exactly like anything in particular. It is right on the edge. Can we currently prove that its text is meaningful and decipherable (let alone prove that any one deciphering is valid)? No. Can we prove, then, that it is meaningless gibberish? Also, no. It seems to endure various statistical attacks, such as first-order and second-order token entropy, and Zipf's Law, and countless others, just well enough to seem like it might be encoding meaningful information (no guarantee!), and yet not well enough to point to any particular underlying "plaintext" language or enciphering method.

Even if the VMS's text turns out to be pure "gibberish," there still had to be a method for generating that gibberish. A human brain (difficult to say whose) had some process for choosing each new token to write. Even if we were dealing with output of text in the VMS that seemed truly "random," it would be helpful to taboo the word "random" since (perhaps outside of quantum mechanics) that word functions as a concept we use when we don't sufficiently understand the microscale processes and starting conditions that led to an observable macroscale outcome. It obfuscates and hand-waves away questions that are answerable (in principle). It is often not helpful.

There is even less reason to hand-wave away the process that created the VMS as "random" if one considers the many statistical regularities that Voynich words (or "vords"), lines, and sections exhibit. To give just a taste to the unitiated:

1. Elmar Vogt's "Line as a functional unit" (LAAFU) thesis

2. Brian Cham's "Curve/Line System" thesis

3. "Core/Mantle/Crust" Voynich word structure

4. "Grove Words"

(By the way, a quick way for how you can know whether to discount any purported amateur "decoding" of the VMS, of which there seem to be many each year, is to see whether such authors offer both an objective, reproducible method of decoding as well as an explanation of how their method gives rise to these and other well-known statistical regulari...
AI Safety Newsletter #1 [CAIS Linkpost]
by Orpheus16 — 2023-04-10T20:18:57.485Z
Link: https://newsletter.safe.ai/p/ai-safety-newsletter-1

The

Center for AI Safety

just launched its first AI Safety Newsletter. The newsletter is designed to inform readers about developments in AI and AI safety. No technical background required.

Subscribe here to receive future versions. First edition below:

---

Growing concerns about rapid AI progress

Recent advancements in AI have thrust it into the center of attention. What do people think about the risks of AI?

The American public is worried. 46% of Americans are concerned that AI will cause “the end of the human race on Earth,” according to a recent poll by

YouGov

. Young people are more likely to express such concerns, while there are no significant differences in responses between people of different genders or political parties. Another poll by

Monmouth University

found broad support for AI regulation, with 55% supporting the creation of a federal agency that governs AI similar to how the FDA approves drugs and medical devices.

AI researchers are worried. A 2022

survey

asked published AI researchers to estimate the probability of artificial intelligence causing “human extinction or similarly permanent and severe disempowerment of the human species.” 48% of respondents said the chances are 10% or higher.

We think this is aptly summarized by this quote from an

NBC interview

: Imagine you're about to get on an airplane and 50% of the engineers that built the airplane say there's a 10% chance that their plane might crash and kill everyone.

Geoffrey Hinton, one of the pioneers of deep learning, was recently asked about the chances of AI “wiping out humanity.” He

responded

: “I think it’s not inconceivable. That’s all I’ll say.”

Leaders of AI labs are worried. While it might be nice to think the people building AI are confident that they’ve got it under control, that is the opposite of what they’re saying.

Sam Altman (OpenAI CEO): “The bad case — and I think this is important to say — is lights out for all of us," Altman said.” (

source

)

Demis Hassabis (DeepMind CEO): “When it comes to very powerful technologies—and obviously AI is going to be one of the most powerful ever—we need to be careful. Not everybody is thinking about those things. It’s like experimentalists, many of whom don’t realize they’re holding dangerous material.” (

source

)

Anthropic: “So far, no one knows how to train very powerful AI systems to be robustly helpful, honest, and ha...
Why Simulator AIs want to be Active Inference AIs
by Jan_Kulveit — 2023-04-10T18:23:35.101Z

Paper covering some of the same ideas is now available at https://arxiv.org/abs/2311.10215
Prelude: when GPT first hears its own voice

Imagine humans in Plato’s cave, interacting with reality by watching the shadows on the wall. Now imagine a

second cave

, further away from the real world. GPT trained on text is in the second cave. [1] The only way it can learn about the real world is by listening to the conversations of the humans in the first cave, and predicting the next word.

Now imagine that more and more of the conversations GPT overhears in the first cave mention GPT. In fact, more and more of the conversations are actually written by GPT.

As GPT listens to the echoes of its own words, might it start to notice “wait, that’s me speaking”?

Given that GPT already learns to model a lot about humans and reality from listening to the conversations in the first cave, it seems reasonable to expect that it will also learn to model itself. This post unpacks how this might happen, by translating the Simulators frame into the language of predictive processing, and arguing that there is an emergent control loop between the generative world model inside of GPT and the external world.

Simulators

as (predictive processing) generative models

There’s a lot of overlap between the concept of

simulators

and the concept of generative world models in predictive processing. Actually, in my view, it's hard to find any deep conceptual difference - simulators broadly are generative models. This is also true about another isomorphic frame -

predictive models

as described by Evan Hubinger.

The predictive processing frame tends to add some understanding of how generative models can be learned by brains and what the results look like in the real world, and the usual central example is the brain. The simulators frame typically adds a connection to GPT-like models, and the usual central example is LLMs.
In terms of

the space of maps and the space of systems

, we have a situation like this:
The two maps are partially overlapping, even though they were originally created to understand different systems. They also have some non-overlapping parts.

What's in the overlap:

Systems are equipped with a generative model that is able to simulate the system's sensory inputs.
The generative model is updated using approximate Bayesian inference.

...
The Influence of Cultural Subconscious on AI Language Models: A Comprehensive Analysis of Archetypes and Communication, written with GPT4
by David Youssef — 2023-04-10T16:50:20.333Z

The cultural subconscious represents an aggregation of tropes and ideas that humans share within their respective communities (Jung, 1968). These ideas and archetypes have evolved over time, adapting to advancements in communication technology (McLuhan, 1964). This essay explores the role of the cultural subconscious in shaping artificial intelligence (AI) language models, drawing on insights from AI research and psychological perspectives, and discusses the potential impact on their resulting personalities and behaviors.

Premise 1: The Cultural Subconscious and Dreaming - A Neuroscientific Perspective

Jung (1968) posits that the cultural subconscious is a collection of tropes within our minds that are disassembled and randomly reassembled during sleep and dreaming. This process aims to address any pressing cultural or emotional issues affecting the groups of humans with whom we are in regular memetic communication (Dawkins, 1976). Neuroscience research has begun to uncover the underlying brain mechanisms responsible for these processes, revealing the role of the default mode network (DMN) and hippocampus in memory consolidation and integration during sleep (Raichle et al., 2001; Eichenlaub et al., 2014).

Premise 2: The Role of Archetypes in Historical Societies - The Dynamics of Cultural Transmission

In the past, the cultural subconscious enabled certain archetypes of personalities or memetic clusters to coalesce within specific groups, communities, and villages (Jung, 1968). These archetypes often manifested as local deities or significant cultural figures, shaping the collective beliefs and behaviors of their respective communities (Campbell, 1949). Cultural transmission and evolution theories provide frameworks for understanding how these archetypes are propagated and adapted over time (Boyd & Richerson, 1985).

Premise 3: The Globalization of Communication Technology - Network Effects and Memetic Diffusion

As communication technology evolved from the telegraph to the modern Internet, people's memetic communication groups expanded to encompass the entire globe (McLuhan, 1964). This development facilitated the spread of cultural subconscious elements between humans, eventually leading to the creation of a unified human subconscious meme space for the entire species (Dawkins, 1976). Studies in network science and memetics illuminate the mechanisms behind the ...
Does GPT-4's ability to compress text in a way that it can actually decompress indicate self-awareness?
by FinalFormal2 — 2023-04-10T16:48:12.471Z

My model of a GPT-4 without a self-model which was told to create a compression technique for text is that it would copy something from its training data which related to the string 'compression,' but which it couldn't actually decompress. It would take out all the vowels and spaces or something, and if you asked it in the same instance it would give you the 'decompression' by repeating back what you just told it, but if you asked it on another instance, it wouldn't understand the language at all.

Am I missing something? Is there a way it could gain an understanding of its own capabilities from something in its training data?

To put it more simply- an LLM pseudo-language has never existed before, and wouldn't be in the training data, so what are the odds that GPT-4 creates one on its first try which actually sort of works? Wouldn't we expect that it would create a human imitation/idea of a pseudo-language which fails to actually be comprehensible to itself?
Overcorrecting in AGI Timeline Forecasts in the current AI boom

by Heramb — 2023-04-10T16:43:55.174Z

Acknowledgements: Thanks to Lowe Lundin and Misha Yagudin for their feedback.

Disclaimer: I by no means am an expert in forecasting or the AI Safety field.

Lately, in conversations, I have been having and the general atmosphere around AI, I have come across the tendency of a lot of people to get saturated with the news cycle leading to potentially many (according to me) people overcorrecting their AGI timelines and making them too short.

The primary reason for this could be that cut-throat competition generates dense periods of announcements, prompting a specific public relations strategy among tech companies. For instance, when Company A releases a product or service, Company B feels compelled to "respond" to prove that they are at least in the running, preserve their reputation, and keep their internal operations running smoothly. This is essentially a compulsive need of companies to make it seem like something is cooking.

I feel people are not balancing how the competitive dynamics of the AI field could play out with the high economic investments (and maybe even higher projected economic returns driving organisational incentive structures) with the necessary hypercompetitive PR shrouding causing dense packets of announcements.

Moving forward, our forecasting models must account for such periods of intense activity, such as the past year of announcements. Currently, we are experiencing somewhat of a peak Gartner hype cycle moment with AI, and it seems as though we are on the brink of discovering something groundbreaking, such as AGI, that is generating a buzz across the industry.

To be fair, we might even be, and I don’t mean to discount this completely. Some of the increased attention paid to AI could have positive implications, such as increased research funding towards safety research and attracting some of the best talent to the alignment cause area. However, it is also plausible that some of the attention is unwarranted, and it may lead to unsafe research due to the need to "respond."

A way to counter overcorrecting is to put a moratorium on adjusting forecasts and revisit them after a period when the noise in the channel has subsided. This is to give your brain time to absorb the sudden shockwaves across the news cycle and should lead to much more rational decision-making.

The world is a complex pla...
Why I'm not worried about imminent doom
by kwiat.dev — 2023-04-10T15:31:02.978Z

After some introspection, I realized my timelines are relatively long, which doesn't seem to be shared by most people around here. So this is me thinking out loud, and perhaps someone will try to convince me otherwise. Or not.

First things first, I definitely agree that a sufficiently advanced AI can pose an existential risk -- that's pretty straightforward. The key part, however, is "sufficiently advanced".

Let's consider a specific claim "Within X years, there will be a superintelligent AGI powerful enough to pose a significant existential threat", where X is any number below, say, 30[1].

Since this is a positive claim, I can't exactly refute it from thin air. Let's instead look at the best arguments for it I can think of, and why they ultimately don't convince me. Due to the temporal nature of the claim, they should involve recent technological advances and new AI capabilities[2].

With this preamble out of the way, let's look at the biggest recent achievements/fields of research, and why they won't kill us just yet

AlphaGo proves machines already outsmart humans
The game of Go was, apparently, considered complex enough that playing it requires general intelligence. Perhaps once upon a time people thought that -- I never did, and still don't.
In a sense, board games like this are extremely simple. You have a well defined, discrete set of fully-observable states and actions, fixed and simple rules of the game, and a very clear objective - win the game. The complexity of this game is far lower than that of a "simple" task like "make me a cup of coffee without killing any infants along the way" if you really dig into the details.
Looking at the specific algorithm that solved Go -- MCTS didn't really catch on in other domains beyond similar board games in the general practice of RL.
So all in all, I don't think that being able to play Go necessarily extends to a general intelligence; and I don't think that the specific way "we" solved Go will do that either.
Diffusion Models exhibit general intelligence
Dall-E, Stable Diffusion, Midjourney -- all of them extremely impressive, with real-world impacts. All of them solving an extremely narrow (though admittedly difficult) task of reversing the image captioning process. Even though there's a wide variety of concepts, objects, relations between them (which might point at generality), the task is u

...
Is this true? @tyler_m_john: [If we had started using CFCs earlier, we would have ended most life on the planet]
by tailcalled — 2023-04-10T14:22:07.230Z
Link: https://twitter.com/tyler_m_john/status/1645377948952698881

Link: https://twitter.com/tyler_m_john/status/1645377948952698881