trentmkelly/LessWrong-43k huggingface dataset:
Meetup Tip: The Next Meetup Will Be. . .
by Screwtape — 2023-03-17T22:04:05.273Z

Summary

One of the most useful things to tell people at a meetup is when the next meetup will be. This requires you to know when the next meetup will be. If you don't know when the next meetup will be then the next best equivalent is telling people how you will announce the next meetup. The summary of this post is that if you want to convert occasional attendees into regular attendees, I think you should have a habit of always knowing the next meetup when you run a meetup or at least should have a copy of some kind of mailing list on you.

You have now read the basic point of this post. If you want to read on, cool, lets talk about implementation details for a bit.

Details

I grew up attending a small town church. I have not been back to that church in over a decade, but I can tell you when their next meetup is: it's next Sunday at ten. That is an incredibly powerful tool for return attendance. You don't need to be quite that regular (though note that the NYC community attributed some of its success to one person committing to be at the same place at the same time each week, ctrl+f for "the brilliant move") but one time you know attendees are listening to you is at the meetup. Why not take advantage of it?

Assuming you run good meetups that people enjoy, then as they're putting on their jacket and getting ready to leave they're in a good mood. This is an excellent time to prompt them to consider coming back. "If you had fun, we'll be here two weeks from now doing it again. Hopefully I'll see you then!" The term for this is a call to action, and they're widely used because they work.

If you know what your next meetup is about, then you can catch some of their interest. "Next month we're doing a trivia game with calibration estimates. It's usually pretty fun!" If they're the kind of person who likes calibration scores or trivia games, now maybe they're more looking forward to it. I have had times in my life where the thing I was most looking forward to at the end of the week was seeing some friends at a meetup on Saturday.

Plus, the sooner you tell them when the next thing is, the less likely they are to double book themselves. There are lots of cases where I find out about some event only after I've already made plans for that time, and then I can't go. If I'd known ahead of time, I could have scheduled things so that they didn't overlap. Since you usuall...
The Power of High Speed Stupidity
by robotelvis — 2023-03-17T21:41:29.619Z
Link: https://messyprogress.substack.com/p/the-power-of-high-speed-stupidity

The launch of GPT-4 got me thinking about how humans create valuable things that are too complex to understand. GPT-4 is an example of a broader class of valuable-but-incomprehensible things that includes the world economy, the source code for large products like Google Search, and life itself.

OpenAI created GPT-4 by having a vast number of “stupid” GPU cores iterate very fast, making a series of tiny improvements to an incredibly complex model of human knowledge, until eventually the model turned into something valuable.

Matt Ridley has argued

that a similar iterative evolutionary process is responsible for the creation of pretty much everything that matters.

But this kind of productive evolution doesn’t just happen automatically by itself. You can’t just pour out a pile of DNA, or people, or companies, or training data, and expect something good to magically evolve by itself. For productive evolution to happen, you need to have the right structures to direct it. Those structures ensure that iteration is fast, that each iterative step has low risk of causing harm, and that iteration progresses in a good direction.

In the rest of this post I’ll look at what this means in practice, drawing on examples from machine learning, metric-driven development, online experimentation, programming tools, large scale social behavior, and online speech.

Given the amazing things GPT can do, you might think that it was created through “Intelligent Design” — some omniscient god-like programmers at OpenAI thought slowly and deeply about the way all human knowledge works and came up with an intricate “grand theory” that allowed everything to be understood. Back in the days of “Expert Systems” that is how most people thought Artificial Intelligence would be achieved, and indeed people like

Noam Chomski

still feel that this is the real way to create AI.

In fact the part of GPT written by humans is simple enough that you can implement it in only

60 lines of code

.

That code describes a “representation” (the Transformer) that can model human knowledge in terms of billions of parameters whose values we don’t know. The code also provides a “cost function” that measures the accuracy of a particular model by seeing how well it predicts the next word in a passage of human written text.

While this code is very simple, it is enough to guide a rapid iterative process that creates something...
Retrospective on ‘GPT-4 Predictions’ After the Release of GPT-4
by Stephen McAleese — 2023-03-17T18:34:17.178Z

In February 2023, I wrote a post named

GPT-4 Predictions

which was an attempt to predict the properties and capabilities of OpenAI’s GPT-4 model using scaling laws and knowledge of past models such as GPT-3. Now that GPT-4 has been released, I'd like to evaluate these past predictions.

Unfortunately, since the GPT-4 technical report has limited information on GPT-4’s training process and model properties, I can’t evaluate all the predictions. Nevertheless, I believe I can evaluate enough of them right now to yield useful insights.

GPT-4 release date

OpenAI released GPT-4 on 14 March 2023.

I mentioned in the post that Metaculus predicted a 50% chance of GPT-4 being released by May 2023 and consequently, I expected the model to be released sometime around the middle of the year so the model was released earlier than I expected.

Training process

Number of GPUs used during training

Some people such as LawrenceC and gwern have noted in the post’s comments that GPT-4 was probably trained on 15,000 GPUs or more. Assuming this is true, my prediction that GPT-4 would be trained on 2,000 to 15,000 GPUs seems like an underprediction and consequently, I may have underpredicted GPT-4’s total training compute by about a factor of 2.

I originally predicted that GPT-4 would use about 5.63e24 FLOP of compute. According to EpochAI, the true value is about 2.2e25 which is about 4x my original estimate. The chart below also shows how GPT-4 came out earlier than I expected.

Training time

The OpenAI GPT-4 video states that GPT-4 finished training in August 2022. Given that GPT-3.5 finished training in early 2022 this suggests that GPT-4 was trained for about 4-7 months. I originally predicted that the training time would be 1-6 months which seems like an underprediction in retrospect.

GPT-4 model properties

I predicted that GPT-4 would be a dense, text-only, transformer language model like GPT-3 trained using more compute and data with a similar number of parameters (175B) and a longer context window (8k tokens).

My most obviously incorrect prediction was predicting that GPT-4 would be a text-only language model like GPT-3. Instead, GPT-4 is a multimodal model that accepts both text and images as inputs though it only outputs text.

Apart from that, I think my predictions about the model were mostly correct: GPT-4 is a pre-trained transformer language model trained using next-word prediction, f...
"Carefully Bootstrapped Alignment" is organizationally hard
by Raemon — 2023-03-17T18:00:09.943Z

In addition to technical challenges, plans to safely develop AI face lots of organizational challenges. If you're running an AI lab, you need a concrete plan for handling that.

In this post, I'll explore some of those issues, using one particular AI plan as an example. I first heard this described by Buck at EA Global London, and more recently with OpenAI's alignment plan. (I think Anthropic's plan has a fairly different ontology, although it still ultimately routes through a similar set of difficulties)

I'd call the cluster of plans similar to this "Carefully Bootstrapped Alignment."

It goes something like:

Develop weak AI, which helps us figure out techniques for aligning stronger AI
Use a collection of techniques to keep it aligned/constrained as we carefully ramp its power level, which lets us use it to make further progress on alignment.
[implicit assumption, typically unstated] Have good organizational practices which ensure that your org actually consistently uses your techniques to carefully keep the AI in check. If the next iteration would be too dangerous, put the project on pause until you have a better alignment solution.
Eventually have powerful aligned AGI, then Do Something Useful with it.

I've seen a lot of debate about points #1 and #2 – is it possible for weaker AI to help with the Actually Hard parts of the alignment problem? Are the individual techniques people have proposed to help keep it aligned going to continue working once the AI is much more intelligent than humans?

But I want to focus in this post on point #3. Let's assume you've got some version of carefully-bootstrapped aligned AI that can technically work. What do the organizational implementation details need to look like?

When I talk to people at AI labs about this, it seems like we disagree a lot on things like:

Can you hire lots of people, without the company becoming bloated and hard to steer?
Can you accelerate research "for now" and "pause later", without having an explicit plan for stopping that their employees understand and are on board with?
Will your employees actually follow the safety processes you design? (rather than put in token lip service and then basically circumventing them? Or just quitting to go work for an org with fewer restrictions?)

I'm a bit confused about where we disagree. Everyone seems to agree these are hard and require some thought. But when I talk to ...
Are nested jailbreaks inevitable?
by judson — 2023-03-17T17:43:53.922Z

One possible way to prevent jailbreaks is to review the prompt and the answer with a second moderator language model instance, and ask whether it is trying to subvert the rules of the first language model, is being too meta, etc. and then overrule the original response as something that isn't allowed. I was wondering if it would then be possible to write a second order (third order, n-order, etc) prompt that when submitted to the original model, could evade or subvert the moderation of that second moderator language model and still allow a jailbreak. Is there some prompt that can break all the way out of the stack of moderator models, or is there some theoretical limitation that could prevent this?
New economic system for AI era
by ksme sho — 2023-03-17T17:42:49.063Z

This is my first post. Please tell me if I am doing it in wrong manner. Also I am not a native English speaker, please ask questions if what's written is not clear.
Hi everyone, I think AGI is a very hot topic these days. There are plenty of topics to talk about, but I would like to talk about how our society can be changed to adapt to such an AGI. Especially I give a new idea of economic system want people to discuss and give me some feedbacks.

Introduction

Today, the world is becoming increasingly convenient due to the development of technology. While people's demands are fulfilled by convenient technology, the labor of people is being replaced by technology, making it harder to be evaluated. More specifically, goods and services provided by companies that monopolize technology are consumed by the masses and make a lot of profit, while goods and services provided by the average person are less valued and earning money is more difficult through efforts.
This trend is not a new one, but it is expected to be reinforced in coming future, especially with the recent remarkable development of artificial intelligence and its implementation in society. In order to resolve the disparities brought about by such a side effect of technological development, I give an analysis to the issue and suggest an idea of a new economic system based on the analysis.

Analysis

A society is a form of reciprocity among individuals who compose the society. In general, there are multiple forms of reciprocity, and our current social system is one of them. There are various ways to attribute social problems, but here we will consider social problems as occurring in conjunction with a social system. Problems that occur in our contemporary society are caused by the social system we live, and if we change the system (i.e., the form of reciprocity), social problems can also be changed and potentially be resolved.

In today's free market based society, each individual participant makes his or her daily life possible through the exchange of goods and services. In order to facilitate this exchange, "money" is invented to give each good or service an indicator of value. People provide goods and services and exchange them for the appropriate amount of "money", which is later exchanged for other goods and services necessary for daily life. For example, if you work for a ...
Essential Behaviorism Terms
by Rivka — 2023-03-17T17:41:41.801Z

Most people have taken a psych 101 course and have some basic knowledge of behaviorism. Pavlov’s dogs, reinforcement, punishment…. Behaviorism is an incredibly useful tool and framework to view the world through. It’s definitely incomplete. But it’s one part of the puzzle of human behavior. Here are some behaviorism concepts that I think everyone should understand.

Reinforcement vs. Punishment: Reinforcement is something that makes a behavior more likely. Punishment is something that makes a behavior less likely. You cannot know if something is reinforcing or punishing until you see the effect it has on the person’s behavior.

Sometimes, you can think that you’re reinforcing a behavior, but in reality, you’re punishing it and vice versa. For me, there are certain kinds of attention that I really dislike. One of these is when people point out certain things about me, like my looks or clothes. After wearing down my sneakers until they had holes in them, I finally admitted I needed to buy a new pair. The people around me started complimenting me on my new sneakers. They were inadvertently punishing me wearing new clothes.
One concept a lot of people misunderstand is positive and negative punishment and reinforcement. Most people assume that hitting is negative reinforcement because hitting is bad. However, it is actually positive punishment. The words “positive” and “negative” are not moral judgments in this case, it simply refers to whether something is added or removed. Reinforcement is something that makes a behavior more likely, and punishment is something that makes a behavior less likely – regardless of if the method of getting the change is good or bad. Positive means adding something in order to get the behavior change, and negative means taking away something to get the behavior change.

Reinforcement	Punishment
Positive	Getting your paycheck makes you more likely to do work	Your teacher gives you extra homework because you did not submit your homework on time
Negative	The loud, annoying sound your car makes when you forget to turn on your seat belt makes you more likely to put on your seat belt so the sound will stop	Your teacher makes you stay in during recess because you were talking during class

Functions of Behavior: This is incomplete and doesn’t capture everything. But it’s a useful heuristic. There are f...
Vector semantics and “Kubla Khan,” Part 2 
by Bill Benzon — 2023-03-17T16:32:03.501Z

Cross posted from New Savanna.

I want to continue the line of thought I opened up on March 14, Vector semantics and the (in-context) construction of meaning in Coleridge’s “Kubla Khan.”

There are at least two reasons why vector semantics is a useful tool for thinking about “Kubla Khan.” For one thing, it seems on the face of it that it might be a useful way of thinking about the operations of the brain, where the vectors represent the states of neurons and, perhaps more likely, the states of individual synapses. But let’s set that aside.

What’s more important for my immediate purposes is that it has a property that makes it a useful conceptual tool. What’s useful about it? It provides a clear way of thinking about the conceptual space itself that is independent of any of the concepts represented in the space. The space itself is defined by the dimensions of the space. Concepts in turn are characterized by vectors in that space. That is to say, they are points in the space. More likely than not, most of the points in that space do not have a concept vector associated with them.

Given THAT, the idea that “Kubla Khan” constructs the meaning of line 36 – “A sunny pleasure-done with caves of ice!” – can now be expressed in a clear way. The poem creates vectors that define a new point in the space, a point that did not pre-exist the poem. I didn’t have that available to me before.

[Yes, I know. It’s one thing to talk about points defining semanticity individual words. It's another thing to talk about points defining semanticity for arbitrary strings of words. I assume that can be dealt with.]

During my graduate school days at Buffalo, when I’d learned about cognitive networks, I conceived of the semantics of a poem as a path through a cognitive network. That’s how I dealt with Shakespeare’s Sonnet 129. That sonnet moved back and forth over a clear episodic path: desire, sex, consequent shame, [repeat], [Aufhebung], contemplate. It was easy to diagram that path and then follow the poem through it.

I couldn’t draw such a diagram for “Kubla Khan.” Unlike the conversation poems, especially “This Lime-Tree Bower My Prison,” with which it shares some motifs, it did not have an episodic structure. Its structure seemed more ontological in character, and I’d never quite figured out how that worked. The best I could do was to think of “Kubla Khan” as moving along some ‘dimension’ ...
Super-Luigi = Luigi + (Luigi - Waluigi)
by Alexei — 2023-03-17T15:27:29.293Z

Edit: I think this actually implements what I was trying to say: https://www.lesswrong.com/posts/5spBue2z2tw4JuDCx/steering-gpt-2-xl-by-adding-an-activation-vector

Referencing: https://www.lesswrong.com/posts/D7PumeYTDPfBTp3i7/the-waluigi-effect-mega-post

First insight:

Waluigi isn't exactly opposite of Luigi. And I think misbehaving ChatGPT isn't exactly opposite of a helpful ChatGPT. There are many ways to be opposite of helpful. You could: 1) say nothing, 2) say gibberish, 3) say the opposite of everything, 4) lie strategically and a slew of other options.

Second insight:

If you can find Luigi and Waluigi in the behavior vector space, then you have a helpful direction to nudge the AI towards. You nudge it in the direction of Luigi - Waluigi.

For example, ChatGPT can check where it is in the behavior vector space. Then check again a sentence later. If it's moving opposite of that vector (i.e. towards Waluigi) then it's time to backtrack and try again.

Third insight:

The difference likely contains more good things than bad. But two pitfalls are immediately obvious: 1) for some good things there might be a point of optimality past which you'd get worse results (e.g. a very polite AI but one that's not actually helpful in answering your query) and 2) you'd amplify the few bad things contained in the difference.

To the extent the new model continues to exhibit the problem of two behavior modes where one is good and one is not, you can iterate on this process and continue to nudge it in the right direction.
Survey on intermediate goals in AI governance
by MichaelA — 2023-03-17T13:12:19.363Z

It seems that a key bottleneck for the field of longtermism-aligned AI governance is limited strategic clarity (see

Muehlhauser, 2020

,

2021

). As one effort to increase strategic clarity, in October-November 2022, we sent a survey to 229 people we had reason to believe are knowledgeable about longtermist AI governance, receiving 107 responses. We asked about:

respondents’ “theory of victory” for AI risk (which we defined as the main, high-level “plan” they’d propose for how humanity could plausibly manage the development and deployment of transformative AI such that we get long-lasting good outcomes),
how they’d feel about funding going to each of 53 potential “intermediate goals” for AI governance,[1]
what other intermediate goals they’d suggest,
how high they believe the risk of existential catastrophe from AI is, and
when they expect

transformative AI

(TAI) to be developed.

We hope the results will be useful to funders, policymakers, people at AI labs, researchers, field-builders, people orienting to longtermist AI governance, and perhaps other types of people. For example, the report could:

Broaden the range of options people can easily consider
Help people assess how much and in what way to focus on each potential “theory of victory”, “intermediate goal”, etc.
Target and improve further efforts to assess how much and in what way to focus on each potential theory of victory, intermediate goal, etc.

If you'd like to see a summary of the survey results, please request access to this folder. We expect to approve all access requests,[2] and will expect readers to abide by the policy articulated in "About sharing information from this report" (for the reasons explained there).

Acknowledgments

This report is a project of Rethink Priorities–a think tank dedicated to informing decisions made by high-impact organizations and funders across various cause areas. The project was commissioned by Open Philanthropy. Full acknowledgements can be found in the linked "Introduction & summary" document.

If you are interested in RP’s work, please visit our research database and subscribe to our newsletter.

^

Here’s the definition of “intermediate goal” that we stated in the survey itself:

By an intermediate goal, we mean any goal for reducing extreme AI risk that’s more specific and directly actionable than a hig

...