trentmkelly/LessWrong-43k huggingface dataset:
How popular is ChatGPT? Part 2: slower growth than Pokémon GO
by Richard Korzekwa  — 2023-03-03T23:40:04.494Z

Rick Korzekwa, March 3, 2023

A major theme in reporting on ChatGPT is the rapid growth of its user base. A commonly stated claim is that it broke records, with over 1 million users in less than a week and 100 million users in less than two months. It seems not to have broken the record, though I do think ChatGPT’s growth is an outlier.

Checking the claims

ChatGPT growth

From what I can tell, the only source for the claim that ChatGPT had 1 million users in less than a week comes from this tweet by Sam Altman, the CEO of OpenAI:

I don’t see any reason to strongly doubt this is accurate, but keep in mind it is an imprecise statement from a single person with an incentive to promote a product, so it could be wrong or misleading.

The claim that it reached 100 million users within two months has been reported by many news outlets, which all seem to bottom out in data from Similarweb. I was not able to find a detailed report, but it looks like they have more data behind a paywall. I think it’s reasonable to accept this claim for now, but, again, it might be different in some way from what the media is reporting1.

Setting records and growth of other apps

Claims of record setting

I saw people sharing graphs that showed the number of users over time for various apps and services. Here is a rather hyperbolic example:

That’s an impressive curve and it reflects a notable event. But it’s missing some important data and context.

The claim that this set a record seems to originate from a comment by an analyst at investment bank UBS, who said “We cannot remember an app scaling at this pace”, which strikes me as a reasonable, hedged thing to say. The stronger claim that it set an outright record seems to be misreporting.

Data on other apps

I found data on monthly users for all of these apps except Spotify2. I also searched lists of very popular apps for good leads on something with faster user growth. You can see the full set of data, with sources, here.3 I give more details on the data and my methods in the appendix.

From what I can tell, that graph is reasonably accurate, but it’s missing Pokémon GO, which was substantially faster. It’s also missing the Android release of Instagram, which is arguably a new app release, and surpassed 1M within the first day. Here’s a table summarizing the numbers I was able to find, listed in chronological ...
Acausal normalcy
by Andrew_Critch — 2023-03-03T23:34:33.971Z

This post is also available on the EA Forum.

Summary: Having thought a bunch about acausal trade — and proven some theorems relevant to its feasibility — I believe there do not exist powerful information hazards about it that stand up to clear and circumspect reasoning about the topic. I say this to be comforting rather than dismissive; if it sounds dismissive, I apologize.

With that said, I have four aims in writing this post:

Dispelling myths. There are some ill-conceived myths about acausal trade that I aim to dispel with this post. Alternatively, I will argue for something I'll call acausal normalcy as a more dominant decision-relevant consideration than one-on-one acausal trades.
Highlighting normalcy. I'll provide some arguments that acausal normalcy is more similar to human normalcy than any particular acausal trade is to human trade, such that the topic of acausal normalcy is — conveniently — also less culturally destabilizing than (erroneous) preoccupations with 1:1 acausal trades.
Affirming AI safety as a straightforward priority. I'll argue that for most real-world-prevalent perspectives on AI alignment, safety, and existential safety, acausal considerations are not particularly dominant, except insofar as they push a bit further towards certain broadly agreeable human values applicable in the normal-everyday-human-world, such as nonviolence, cooperation, diversity, honesty, integrity, charity, and mercy. In particular, I do not think acausal normalcy provides a solution to existential safety, nor does it undermine the importance of existential safety in some surprising way.
Affirming normal human kindness. I also think reflecting on acausal normalcy can lead to increased appreciation for normal notions of human kindness, which could lead us all to treat each other a bit better. This is something I wholeheartedly endorse.

Caveat 1: I don't consider myself an expert on moral philosophy, and have not read many of the vast tomes of reflection upon it. Despite this, I think this post has something to contribute to moral philosophy, deriving from some math-facts that I've learned and thought about over the years, which are fairly unique to the 21st century.

Caveat 2: I’ve been told by a few people that thinking about acausal trade has been a mental health h...
Comments on OpenAI's "Planning for AGI and beyond"
by So8res — 2023-03-03T23:01:29.665Z

Sam Altman shared me on a draft of his OpenAI blog post

Planning for AGI and beyond

, and I left some comments, reproduced below without typos and with some added hyperlinks. Where the final version of the OpenAI post differs from the draft, I’ve noted that as well, making text Sam later cut red and text he added blue.

My overall sense is that Sam deleted text and occasionally rephrased sentences so as to admit more models (sometimes including mine), but didn't engage with the arguments enough to shift his own probability mass around on the important disagreements.

Our disagreements are pretty major, as far as I can tell. With my comments, I was hoping to spark more of a back-and-forth. Having failed at that, I'm guessing part of the problem is that I didn't phrase my disagreements bluntly or strongly enough, while also noting various points of agreement, which might have overall made it sound like I had only minor disagreements.

To help with that, I've added blunter versions below, in a bunch of cases where I don't think the public version of the post fully takes into account the point I was trying to make.

(Though I don’t want this to take away from the positive aspects of the post, since I think these are very important as well. I put in a bunch of positive comments on the original draft, in large part because I think it's worth acknowledging and reinforcing whatever process/author drafted the especially reasonable paragraphs.)

I don’t expect Sam to hear me make a blunt claim and instantly update to agreeing with me, but I put more probability on us converging over time, and clearly stating our disagreements for the benefit of readers, if he understands my claims, knows I still disagree, and feels license to push back on specific things I’ve re-asserted.

Formatting note: The general format is that I include the text of Sam's original draft (that I commented on), my comment, and the text of the final post. That said, I don't want to go blasting someone's old private drafts across the internet just because they shared that draft with me, so in some cases, the original text is redacted, at Sam's request.

Sam’s draft: Our mission is to ensure that AGI benefits all of humanity. The creation of AGI should be a tremendous shared triumph that everyone contributes to&n...
Why are counterfactuals elusive?
by Martín Soto — 2023-03-03T20:13:48.981Z

Produced as part of SERI MATS 3.0. Thanks to Vivek Hebbar and Paul Colognese for discussion.

TL;DR (spoiler):

Behind the problem of human counterfactuals creeps the problem of understanding abstraction / ontology identification.

A nice theory of counterfactuals would be useful for many things, including low-impact measures for corrigible AI:

a flooded workshop changes a lot of things that don't have to change as a consequence of the cauldron being filled at all, averaged over a lot of ways of filling the cauldron. [the natural operationalization of this averaging requires counterfactuals]

So whence the difficulty of obtaining one?

Well, we do have at least one well-defined class of counterfactuals: "just take a chunk of atoms, replace it by another, and continue running the laws of physics". This is a discontinuity in the laws of physics that would never take place in the real world, but we don't care about that: we can just continue running the mathematical laws of physics from that state, as if we were dealing with a Game of Life board.[1]

But this doesn't correspond to our intuitive notion of counterfactuals. When humans think about counterfactuals, we are basically changing the state of a latent variable inside our heads, and rerunning a computation. For example, maybe we change the state of the "yesterday's weather" variable from "sunny" to "rainy", and rerun the computation "how did the picnic go?".

The problem with this is our latent variables don't neatly correspond to parts of physical reality. Sometimes they don't even correspond to any parts of physical reality at all! And so, some (in fact, most) of the variable changes we offhandedly perform, don't univocally correspond to physical counterfactuals natively expressed in our laws of physics.

If you just replace a three-dimensional cube of atmosphere to include a rainy cloud, people will notice a cloud appeared out of nowhere. So as a necessary consequence, people will be freaked out by this artificial fact, which is not at all what you had in mind for your counterfactual. Sometimes you'll be able to just add the cloud when no one is looking. But most times, and especially when dealing with messier human concepts, the physical counterfactual will be under-determined, or even none of them will correspond to what you had in mind, using your neatly compartmentalized variables.[2]

This is not to say human ...
Situational awareness in Large Language Models
by Simon Möller — 2023-03-03T18:59:32.148Z

I’m grateful to Bogdan Cirstea, Konstantin Pilz and

Raphaël S

for providing feedback on this post.

This post tries to clarify the concept of situational awareness, in particular with respect to current large language models.

What is situational awareness

Not writing anything new here, just summarizing prior work.

(It’s worth noting that the usage of the term here is different from what’s usually meant by

situational awareness

in humans.)

Ajeya Cotra

introduced the term of situational awareness in the context of AI Safety and

Richard Ngo et al.

recently elaborated on it. Situational awareness describes the degree to which an AI system understands its environment and its own state and behavior, in particular when that understanding causes specific behavior (such as

deceptive alignment

). It’s a spectrum rather than a binary property of the model.

Considering existing models:

RL agents that play in toy worlds have arguably no sense that they have been “trained by humans” and that there exists another world outside the one they are in.
ChatGPT can talk about many aspects of the world, including human psychology and machine learning. It also readily says that it’s a Large Language Model trained by OpenAI and can

hallucinate source code

that resembles a language model interface.

Considering “biological agents”:

Small lab animals probably have a somewhat stable sense of self and might perceive humans as agents different from themselves. But they most certainly don’t understand that they are part of a science experiment.
Dogs build a stronger relationship with their trainers and they understand things like “if I steal the food while the human isn’t looking, I’m less likely to get yelled at”.
School children have even higher situational awareness.

The hypothesis is that models could gain a high degree of situational awareness exceeding that of e.g. adult humans since it would be useful for a range of real-world tasks.

For example, there's a lot of incentives to train models to help with ML development and so these models will know details of the ML training process. They might be able to perform even better if they also introspect themselves.
A model that's trained with RLHF could get a high reward if it's explicitly understood and modeled human preferences and thought processes. It might e.g. want to produce answers that seem particularly pla

...
AI Governance & Strategy: Priorities, talent gaps, & opportunities
by Orpheus16 — 2023-03-03T18:09:26.659Z

Over the last few weeks, I’ve had 1:1s with several AI governance professionals to develop better models of the current AI governance & strategy landscape. Some topics that regularly came up include:

What are the most important/neglected areas in AI governance & strategy?
What is the theory of change behind projects in these areas, and where might they break?
What are the current talent gaps & opportunities in these areas?

This post is my attempt to summarize some takeaways from those conversations. I list some “priority areas” in AI governance & strategy, summarize them briefly, and describe potential talent gaps in each area. I don't claim that my list is comprehensive, and I welcome people to add their own ideas in the comments.

If you think you may have some of the relevant talents/aptitudes and are interested in working in any of these areas, feel free to reach out to me, and I may connect you to relevant professionals. (Feel free to have a low bar for reaching out; I'll ask you for more information if needed.)

Please also be aware that there are downside risks in each of these areas. I suggest you get in touch with relevant professionals before “jumping in” to any of these areas.

Priority Areas

I refer to “priority areas” as topics that frequently came up when talking with AI governance professionals. Caveats: This is not a rigorous method, this list is not comprehensive, some topics were excluded intentionally, the list probably overweights topics that I evaluate as valuable (on my inside view), and priorities will inevitably change as the field continues to evolve.

For each priority area, I offer a brief summary, as well as a description of the kinds of career aptitudes that might make someone an especially good fit for working in the area.

Model evaluations

Summary: There are many ways models could be dangerous, but it’s difficult to detect these failure modes. Can we develop and implement “tests” that help us determine if a model is dangerous?

Some people are working on technical tests that can determine if a model has dangerous capabilities or appears to be misaligned. Others are thinking more broadly about what kinds of evals would be useful. Some people are focused on creating agreements that labs or governments could implement (e.g., if a Deception Eval is triggered, everyone agrees to stop scaling until Y evidenc...
Measuring Ads Opt-Out Compliance
by jefftk — 2023-03-03T16:00:03.586Z

There's an interesting preprint out by Liu, Iqbal, and Saxena,

Opted Out, Yet Tracked: Are
Regulations Enough to Protect Your Privacy?. They describe an
experiment they ran to figure out whether your choices on consent
dialog boxes actually do anything. The idea is that if they work you
expect to see consistently lower bids in the "opt out" than "opt in"
case. While I think the experiment wasn't quite right, with a few
tweaks it would be really informative.

The basic design makes sense:

Limit to publishers who run client-side auctions ("header
bidding") so you can see the bids.

Visit some sites on a range of topics so advertisers can start
estimating your interests and decide to market to you. (And also a
control group that skips this step.)

Visit a publisher and see how much advertisers bid if you
opt in vs opt out.

The main issue I see is that their implementation of (2) was probably
not good enough to get advertisers excited about bidding specifically
for their views:

3.3.1 Simulating Interest Personas. Since
advertisers bidding behavior is different for different user
interests, we simulate 16 user interest personas to capture a wide
spectrum of bidding behavior. User personas are based on 16 Alexa
top websites by categories lists. To simulate each persona, we
initialize a fresh browser profile in an OpenWPM instance, on a fresh
EC2 node with a unique IP, iteratively visit top-50 websites in each
category, and update browser profile after each visit. Our rationale
in simulating personas is to convince advertisers and trackers of each
persona's interests, so that the advertisers bid higher when they
target personalized ads to each persona. In addition to the
above-mentioned 16 personas, we also include a control persona, i.e.,
an empty browser profile. Control persona acts as a baseline and
allows us to measure differences in bidding behavior.

The sixteen personas they used were "Adult, Art, Business, Computers,
Games, Health, Home, Kids, News, Recreation, Reference, Regional,
Science, Shopping, Society, and Sports".

These are pretty bland, and I wouldn't expect advertisers knowing
you'd visited some sites in those categories to change their behavior
much. This study would have been much better if they'd used some
sites with very strong commercial intent and high profit margins, like
mattresses, car sales, ...
ChatGPT tells stories, and a note about reverse engineering: A Working Paper
by Bill Benzon — 2023-03-03T15:12:05.544Z

Cross-posted from New Savanna.

I've uploaded a new working paper. Title above, links, abstract, content, and opening sections below.

Academia.edu: https://www.academia.edu/97862447/ChatGPT_tells_stories_and_a_note_about_reverse_engineering_A_Working_Paper
SSRN: https://ssrn.com/abstract=4377178
Research Gate: https://www.researchgate.net/publication/368978013_ChatGPT_tells_stories_and_a_note_about_reverse_engineering_A_Working_Paper

Abstract: I examine a set of stories that are organized on three levels: 1) the entire story trajectory, 2) segments within the trajectory, and 3) sentences within individual segments. I conjecture that the probability distribution from which ChatGPT draws next tokens follows a hierarchy nested according to those three levels and that is encoded in the weights off ChatGPT’s parameters. I arrived at this conjecture to account for the results of experiments in which ChatGPT is given a prompt containing a story along with instructions to create a new story based on that story but changing a key character: the protagonist or the antagonist. That one change then ripples through the rest of the story. The pattern of differences between the old and the new story indicates how ChatGPT maintains story coherence. The nature and extent of the differences between the original story and the new one depends roughly on the degree of difference between the key character and the one substituted for it. I conclude with a methodological coda: ChatGPT’s behavior must be described and analyzed on three levels: 1) The experiments exhibit surface level behavior. 2) The conjecture is about a middle level that contains the nested hierarchy of probability distributions. 3) The transformer virtual machine is the bottom level.

Contents

How you might read this paper 2
Story coherence 2
My procedure 2
Conjecture: A nested hierarchy of probability distributions 3
Could we develop a quantitative model? 6
12 Experiments: The Chronicles of Princess Aurora and William the Lazy 7
Reverse Engineering ChatGPT 19

How you might read this paper

This paper is about the twelve experiments that make up the two-thirds of the text in this paper. One way to read the paper, perhaps even the preferred way, is to start by reading the plain text paragraph in the next section. That tells you what I did in those experiments. Move from that directly to the experiments. Read through them, thi...
Group Wiki Walk
by Screwtape — 2023-03-03T15:10:53.024Z

Summary: Indulge your curiosity together by wandering Wikipedia and reading interesting things.

Tags: Medium, Repeatable, Experimental

Purpose: Learn something new in a group. You don’t need to learn it deeply; in the same way that it can be fun to point at a cool flower and say “hey, come take a look at this!” it’s fun to point at a neat bit of trivia and say “hey, did you know that?”

Materials: You need at least one device that can browse the web. It’s very helpful to connect that device to a big screen everyone can see, like a projector or a big TV. In some variations, people will want their own internet capable devices like a smartphone.

Announcement Text: "Have you ever gotten lost in Wikipedia? One question leads to another, and there’s always another blue link you haven’t checked out yet. While this is fun, the first thing I usually want to do after spending a few hours learning new things is tell someone else. Why not skip that interim step of looking for someone to talk to about what I just learned and learn with a friend?

We’ll have a main screen we can all read, and we’ll take a walk around the internet together. Come looking to get curious about the world."

Description: Make sure everyone can see the main screen. Browse to Wikipedia. Then moderate the main screen, scrolling down at a regular pace and taking feedback for where to go next.

Suggested hand gestures are waving down when someone is ready to scroll downward, a flat palm when they want you to stop, and raising a hand when someone has a suggestion for a link to click. Depending on the size of your group this may likely be totally unnecessary and people can just call out what they want.

Variations: You can vary how directed a wiki walk is meant to be. On the one extreme, you might do what I call a Wiki Race and try to get from a specific start point (“Japan”) to a specific end point (“St. Bernard”) in as little time and as few link clicks as possible. On the other end of the spectrum, you can just start with the Wikipedia homepage and ask people what looks interesting. The middle ground I like is having a target we’re trying to walk towards, but not taking that target too seriously and eagerly diverting after distractions.

You can vary how grouped up everyone should be. On one extreme, everyone can have their own laptops or smartphones and wander in entirely separate directions...
Robin Hanson’s latest AI risk position statement
by Liron — 2023-03-03T14:25:04.545Z
Link: https://www.overcomingbias.com/p/ai-risk-again

“While I’ve written on this many times before, it seems time to restate my position.”

“While I agree that this is a logically possible scenario, not excluded by what we know, I am disappointed to see so many giving it such a high credence, given how crazy far it seems from our prior experience. Yes, there is a sense in which the human, farming, and industry revolutions were each likely the result of a single underlying innovation. But those were the three biggest innovations in all of human history. And large parts of the relevant prior world exploded together in those cases, not one tiny part suddenly exterminating all the rest.

In addition, the roughly decade duration predicted from prior trends for the length of the next transition period seems plenty of time for today’s standard big computer system testing practices to notice alignment issues. And note that the impressive recent AI chatbots are especially unlike the systems of concern here: self-improving very-broadly-able full-agents with hidden intentions. Making this an especially odd time to complain that new AI systems might have killed us all.”

Seems not much has changed in the Yudkowsky vs. Hanson position over the years, i.e. still assigning high vs. low existential risk.