trentmkelly/LessWrong-43k huggingface dataset:
Excessive AI growth-rate yields little socio-economic benefit.
by Cleo Nardo — 2023-04-04T19:13:51.120Z

Basic argument

In short —

There exists an adaptive limit representing the maximum rate at the economy can adapt to new technology.
Exceeding this limit yields no socio-economic benefit, in the general case.
Exceeding this limit imposes significant risks and costs.
AI growth-rate currently exceeds the adaptive limit.
Therefore, we should slow rate of AI growth-rate.

Is this argument sound?

(1) There exists an adaptive limit.

The economy is a complex adaptive system — it tries to maintains a general equilibrium by responding to exogenous changes. These responses are internal homeostatic processes such as the price mechanism, labour movements, and the stock exchange.

However, these homeostatic processes are non-instantaneous. Prices are sticky, employees have contracts, and the NASDAQ closes at 8:00 PM. The internal homeostatic processes which equilbriate the economy operate on a timescale of years, not weeks.

Imagine if OpenAI had released ChatGPT-3.5 but hadn't released ChatGPT-4. The release of ChatGPT-3.5 would've had a profound impact on the economy. Here's the standard neo–Keynesian story —

Firstly, people start using ChatGPT-3.5 to automate part of their jobs, increasing their productivity. Some employees use ChatGPT-3.5 to complete more tasks within the same working hours, whereas other employees use ChatGPT-3.5 to complete the same tasks within fewer working hours. In both cases, value is created.

Because employees are completing more tasks, the companies start shipping more products and services. Supply goes up, and quality goes up too. So investors and shareholders receive more money. Because employees take more leisure time, their jobs looks more attractive to job-seekers. Hence the supply for labour increases, and the wages decrease, and the demand for such labour, at the new price, increases. Because the labour costs have dropped, competing companies will lower prices to gain more market share. And new industries, which would've been unprofitable in the previous economy, are now profitable and viable. This leads to a real-terms wage increase.

After a few years, the economy reaches (or at least approaches) a new general equilibrium in which...

People have more money to spend.
The products and services are more abundant, cheaper, and of a higher quality.
People have more leisure to enjoy themselves.

This is what we call an "AI Summer Harvest".

Notice that the benefit ...
Penalize Model Complexity Via Self-Distillation
by research_prime_space — 2023-04-04T18:52:41.063Z

When you self-distill a model (e.g. train a new model using predictions from your old model), the resulting model represents a less complex function. After many rounds of self-distillation, you essentially end up with a constant function. This paper makes the above more precise.

Anyway, if you apply multiple rounds of self-distillation to a model, it becomes less complex. So if the original model learned complex, power-seeking behaviors that doesn't help it do well on the training data, this behavior would likely go away after several rounds of self-distillation. Self-distillation allows you to essentially get the minimum complexity model that still does well on the test set. Thus, I think it's promising from an AI safety standpoint.
Giant (In)scrutable Matrices: (Maybe) the Best of All Possible Worlds
by 1a3orn — 2023-04-04T17:39:39.720Z

It has become common on LW to refer to "giant inscrutable matrices" as a problem with modern deep-learning systems.

To clarify: deep learning models are trained by creating giant blocks of random numbers -- blocks with dimensions like 4096 x 512 x 1024 -- and incrementally adjusting the values of these numbers with stochastic gradient descent (or some variant thereof). In raw form, these giant blocks of numbers are of course completely unintelligible. Many hold that the use of such giant SGD-trained blocks is why it is hard to understand or to control deep learning models, and therefore we should seek to make ML systems from other components.

There are several places where Yudkowsky or others state or strongly imply that because SGD-trained models with huge matrices are unintelligible, we should seek some more easily-interpretable paradigm.

I'm going to argue against that. I think that a better alternative is probably not possible; that the apparent inscrutability of these models actually has little-to-nothing to do with deep learning; and finally that this language -- particularly to the non-rationalist -- suggests unwarranted mystery.

0: It Is Probable That Generally Intelligent Systems Must be Connectionist

Imagine a universe in which it is impossible to build a generally intelligent system that is not massively connectionist. That is, imagine a world where the only way to get intelligence from atoms is to have a massive number of simple, uniform units connected to each other -- or something that is a functional equivalent of the same.

In such a world, all smart animals would have become smart by scaling up the number of such units that they have. The dominant evolutionary species might become intelligent by scaling up its head size, despite paying the evolutionary cost of making childbirth dangerous and painful by doing so. Flying species that could not afford this extra weight of scaling up skull volume might take another approach, shrinking their neurons to pack more of them into a given volume. Even animals far distant from the dominant species along the phylogenetic tree and in which the evolution of high levels of intelligence occurred entirely separately, would do so by scaling up their brains.

The dominant species, once it could make information-processing equipment, might try for many years to make some generally intelligent system without massive c...
Play My Futarchy/Prediction Market Mafia Game
by Arjun Panickssery — 2023-04-04T16:12:30.730Z
Link: https://arjunpanickssery.substack.com/p/play-my-futarchyprediction-market

In the social deduction game

Mafia

(broadly the same game as Werewolf, Town of Salem, etc) the minority mafiosi—who know each other's identities—play against the majority villagers in alternating day phases, where the players select someone to kill, and night phases, where the mafia picks someone to kill while select players perform secretly assigned roles like investigating others or protecting them from attack.

I'm hosting a game of mafia where instead of voting, there’s a live prediction market where people can buy and sell shares in each player. The highest-priced player at the end of each day phase will be executed and their role and faction made public, at which point their shares will pay out $100 if they’re revealed to be in the mafia.

I hosted a game like this before with 40 players over the pandemic but the execution was poor—only a quarter of them really understood how the market worked and what the prices meant, the bot running the market crashed at one point, and the game used normal voting (gaining more money allowed you to buy bonus items at daily auctions instead).

The game will take place over about a week. Each day/night cycle in the game will last about one calendar day, so you don’t need to pay constant attention to the chatter but can just check in, post your thoughts, and bet on the market when you’re free. I might stretch out the phases depending on what time zones need to be accommodated.

I’m looking for 24 players. You can also join the Discord server just to spectate:

https://discord.gg/D5DQVs3wP4

More rules explanation:

https://docs.google.com/document/d/1DfAzHfAprjK9sjSHMvRNjWz866NIX41YVp3ktBrqVIU/edit
Steelman / Ideological Turing Test of Yann LeCun's AI X-Risk argument?
by Aryeh Englander — 2023-04-04T15:53:19.099Z

Yann LeCun has been saying a lot of things on social media recently about this topic, only some of which I've read. He's also written and talked about it several times in the past. Most of what I've seen from him recently seems to not be addressing any of the actual arguments, but on the other hand I know he's discussed this in many forums over several years, and he's had the arguments spelled out to him so many times by so many people that it's hard for me to believe he really doesn't know what the substantive arguments are. Can someone who's read more of Yann's arguments on this please give their best understanding of what he's actually arguing, in a way that will be understandable to people who are familiar with the standard x-risk arguments?
Given the Restrict Act, Don’t Ban TikTok
by Zvi — 2023-04-04T14:40:03.162Z

While others have already written similar posts, given my previous position it seems necessary for me to do so as well – the Restrict Act is a no good very bad bill, and having seen the bill I realize that I was wrong to support banning TikTok.

For a while I have been in favor of banning TikTok, if it does not divest its Chinese ownership and modify the software to stop collecting outside user data. TikTok is Chinese spyware. Letting it be installed on most phones, given the data it is collecting, is not something we can or should abide.

Then someone actually proposed a bill, S 686 or the Restrict Act. I was reminded why it is almost never a good idea to ban things. Rather than a narrow bill to allow the banning of TikTok, we got a bill that vastly expands government powers, a Patriot Act for the Internet.

This is why we cannot have nice things. These people clearly cannot be trusted to regulate such matters.

So I’m admitting I was wrong. TikTok is still Chinese spyware, it is still not a good use of your time, you personally should not use it.

If this is how things are, however, then that’s where this ends. Don’t ban TikTok.

TikTok Is Chinese Spyware

TikTok is Chinese Spyware. It can’t read your texts and emails directly, but it can do a lot of other things short of that. If we have the ability to ban TikTok without vastly destroying our civil liberties in general, we should ban TikTok.

Noah Smith agrees.

Spying is the most commonly cited reason for banning TikTok, because it’s the easiest to prove. Tiktok has admitted tracking journalists’ physical movements and sending the data to its Chinese parent company. But physical location is probably only the tip of the iceberg of the data TikTok can collect, which includes faceprints, voiceprints, browsing history, text messages, and pretty much anything you do on your phone. And as Ben Thompson wrote back in 2020, that information basically becomes the property of the Chinese Communist Party.

[From Ben]:

All Chinese Internet companies are compelled by the country’s National Intelligence Law to turn over any and all data that the government demands, and that power is not limited by China’s borders. Moreover, this requisition of data is not subject to warrants or courts, as is the case with U.S. government requests for data from Facebook or any other entity…If anything it would be a something of a surprise w

...
Running many AI variants to find correct goal generalization
by avturchin — 2023-04-04T14:16:34.422Z

Problem: An AI may incorrectly generalize a goal we present it.

Proposal: Run many slightly different instances of AI and find the “Schelling point” around correct generalization.

The idea is that if we run many similar AIs or copies of the same AI in parallel, we will be able to detect goal’s mis-generalizations, as such wrong generalizations will be different in most AIs. The only “Schelling point” will be the correct generalization, at which many AIs will converge.

For example, we want AI to clean a room from bowling balls. We ask AI to “remove all spherical objects”. We run many slightly different copies of the AI and they produce the following suggestions:

1st AI: suggests to remove balls

2nd AI: suggests to remove balls

3d AI: suggests to remove balls and lamps

4th AI: suggests to remove balls and human heads.

1st and 2nd AI have come to the same conclusion, and it is the right one. 3rd and 4th AIs have different conclusions and these conclusions are wrong. 4th AI’s conclusion is dangerous.

Counterarguments:

Some misgeneralizations could also be Schelling points.
Powerful AIs can guess that other similar AIs exists and start some adverse cooperation.

The first problem could be (partly) solved by proper randomisation of initial parameters, weights or inputs. For example, a question could be presented several times in different forms for different instances of the AI. E;g.: “Remove all round objects”, “remove all circular objects”, “remove all sport utilities”.

There could be two “peaks” in generalization which means that our presented goal has intrinsic ambiguity. We could then ask AI why it has different interepations.

In real life, this approach is also used: we could say a driver “Turn left. Turn there on Main Street. Here!” In that case, we formulate our goal several times in different words hoping that it will eventually converge into correct model inside driver’s mind. The driver’s mind selects solution which satisfies all three commands, and disregard those solutions which satisfy only one command.

Another example: If we have a set of calculators which either produce correct answer or wrong one, we could find the broken calculators as they all will produce different answers, and all correct calculators will produce the same answer.
Invocations: The Other Capabilities Overhang?
by Robert_AIZI — 2023-04-04T13:38:14.315Z
Link: https://aizi.substack.com/p/invocations-the-other-capabilities

Abstract: An LLM’s invocation is the non-model code around it that determines when and how the model is called. I illustrate that LLMs are already used under widely varying invocations, and that a model’s capabilities depend in part on its invocation. I discuss several implications for AI safety work including (1) a reminder that the AI is more than just the LLM, (2) discussing the possibility and limitations of “safety by invocation”, (3) suggesting safety evaluations use the most powerful invocations, and (4) acknowledging the possibility of an “invocation overhang”, in which an improvement in invocation leads to sudden capability gains on current models and hardware.

Defining Invocations, and Examples

An LLM’s invocation is the framework of regular code around the model that determines when the model is called, which inputs are passed to the LLM, and what is done with the model’s output. For instance, the invocation in the

OpenAI playground

might be called “simple recurrence”:

A user provides an input string. The input to the LLM is this string, unchanged except for tokenization.
Run the LLM on this input, producing logits.
Predict the next token as some probabilistic function of the logits (ex: at temperature 0 the next token prediction is the argmax of the logits).
Append this token to the end of the user’s input string.
Repeat steps 2-4 with the new string until you get an [END_OF_STRING] token or reach the max token limit.
Display the result as plain text.

Note how many steps in “using the LLM” do not involve the actual model! Here are some ways this invocation can be varied:

Augmenting the prompt in simple recurrence, such as providing few-shot examples,

chain-of-thought

prompting, or text like “You are an AI assistant. User: [text]. You: “.
Monitoring outputs to adjust them. For instance, in the

New York Times “interview” with Bing

, there is a moment where “[Bing writes a list of destructive acts, including hacking into computers and spreading propaganda and misinformation. Then, the message vanishes, and the following message appears.]” This is clearly not simple recurrence, because simple recurrence never deletes tokens. Instead, a separate part of the invocation (perhaps even another instance of the same model!) must be monitoring the text and deleting parts of it under some condition.
Embedding tools or API calls, such as Bing searches or

plug-ins

. I don’t

...
Wanted: Mental Health Program Manager at Rethink Wellbeing 
by Inga G. — 2023-04-04T11:49:40.547Z

Are you an experienced coach, or therapist interested in taking on a new challenge? Then this is for you! You can apply here (<20min).

Background

Our emerging organization,

Rethink Wellbeing

aims to improve mental resilience and productivity at scale, starting in the Effective Altruism Community. We plan to do so by developing an integrative stepped-care system that, among others, includes peers to train psychological skills together, combined with accountability schemes and proven digital courses, and workbooks.

We’d love your support as a program manager and “super”-facilitator to

co-create, test and run our first batch of programs

, as well as the corresponding training of other facilitators. The job opening is for someone with 50–100% FTE, working as a contractor or employee, preferably full-time, starting as soon as possible. Funding is available until the end of Aug, and we are fundraising to extend this period.

What we offer you

A high-impact job opportunity
An opportunity to work with a great team and community (short-term or long-term)
Plenty of scope to shape the future of the programs and organization
Flexible remote work
Lots of learning and self-development
Competitive salary that depends on the level of experience, need, and responsibility

What the program manager ideally brings

preferably a degree in psychology/ psychotherapy or a comparable one,
track record of an extraordinarily high participant or client ratings,
experience with group facilitation, e.g., moderation, workshops, hosting events, or similar, ideally online,
experience with treatment methods that foster mental health, connection, or productivity,
experience with developing programs or trainings, ideally online,
bonus: experience with training of facilitators, therapists, or coaches,
bonus: experience in community building, working with Effective Altruism, Rationalist, or adjacent communities.

Programs we run

We run online groups of 4–6 participants, running 1.5 - 2 hours sessions per week for ~ 6 weeks. In these groups, participants boost their mental wellbeing or productivity together, applying proven psychological techniques such as:

Cognitive Behavioral Therapy,
Accountability, and Behavior Change,
Compassion, and mindfulness,
Inner-Parts Work, such as Internal Family Systems.

The facilitators of these groups do not teach the group but rather operate as mode...
Where Free Will and Determinism Meet
by David Bravo — 2023-04-04T10:59:36.474Z

Imagine an ordinary-looking text editor. It has the standard bold, italics, and underline styles, and the option to undo your latest editing command. The right arrow next to the undo icon, however, far from represents the typical redo feature. Whether you last undid a command or not, it can be clicked at any time, and its distinctive quality is that it can predict your next action.

In a sense, this Writing Assistant is an oracle that anticipates you with unerring accuracy. It is an exceptionally powerful autocompletion tool for forecasting the word or sentence that you were intending to type.

But then, isn’t this Writing Assistant always wrong? If it was smart enough, it would know that you never actually type the text it says you will type; what you actually do is to just click the right arrow for the assistant to autocomplete the text. This assistant is doomed to eternal falsity.

Or not.

What this Writing Assistant does is to model a world where you hadn’t called it up, and predict what you would have typed in that world. It is a Counterfactual Oracle: it returns predictions that would have been true under slightly different circumstances (circumstances specified by the programmer: for example, “as if this Writing Assistant wasn’t available” or “as if the writer chose not to use the Writing Assistant”).

The strength of this oracle is not to calculate every minute detail of every quark’s trajectory, because that would make its model perpetually wrong: in its counterfactual world, the quarks would have had to skip the laws of physics in order not to follow the deterministic path they did follow.

The strength of the oracle is to strike a balance of rightness: to not model the world at the exceedingly low level of quarks so that it would always be wrong because those quarks would have had to disobey the laws of physics; and to not model it at an exceedingly high level so that it would be wrong because it had omitted important details of the person’s character and dispositions.

This would be the level of the superintelligent Counterfactual Oracle: the level needed to predict a different, human intelligence.

Now consider a different oracle: an Ordering Assistant. This oracle shows up in the digital menu of your favourite Italian restaurant, and when asked it prognosticates — correctly — what dish you will pick. "Pineapple pizza", it responds with an ...
Strategies to Prevent AI Annihilation
by  — 2023-04-04T08:59:53.867Z

Tl;DR: A strong tactic against annihilation by an ASI, and for maximizing the chance of a post-scarcity utopia, is to take advantage of an ASI's likely bias towards minimizing risk

With practical, easily accessible language models releasing at an exponential rate, and realistic timelines to AGI having dramatically shortened, worrying about a possible AI terrible-future-death-of-all-humanity is in.

What has emerged is a landscape of speculation over the likelihood of this scenario, ranging from qualified experts to anyone with a Twitter account and sufficient cadre of tech-connected followers. What is concerning is that optimism or pessimism doesn't seem to correlate with esteem or intelligence; being more in-the-know regarding AI doesn't make one less fearful of potential doom. Even the most naïve optimists seem to believe the chance of Artificial General Intelligence quickly taking off into humanity-annihilating Artificial Super Intelligence is greater than zero.

A lot of plans to prevent evil ASI are based on the hope that strategies can be put in place to minimize the potential negative impact of an ASI: to make sure that the right™ people with the right™ alignment strategies are keeping a close watch over the data centers.

The problem with hope and plans predicated on anticipated future action is that their efficacy, as the number of bad alternative possibilities increases, gradually tends towards zero. What we need are tactics against annihilation that can predictably resist reduction in probable efficacy as new information emerges.

Compared to the energy expenditure of a space-faring fleet of von Neumann probes, it would not be very difficult for a resource-rich ASI to keep all humans all alive and happy. With this in mind, a strong ASI risk-mitigation plan is to make it favorable to an ASI's risk-mitigation and efficiency-maximization strategy to keep all humans alive and happy.

Stochastic Dead Man's Switch

This tactic would be to communicate to an ASI that sometime prior to its emergence, a random assortment of humans had dead-man's switches implanted inside them. These switches would trigger if they were to die in a manner than is improbable in normal circumstances, but highly probable in an evil ASI scenario. These switches would trigger an arsenal of nuclear bombs, EMP devices, or anything that creates as much havoc as possible.

For this to wo...
Best Ways to Try to Get Funding for Alignment Research?
by RGRGRG — 2023-04-04T06:35:05.356Z

Hey Everyone! I recently left my FAANG job to split my time between doing Alignment Research (70%) and investigating start-up ideas (30%).

If I decide to fully commit to Alignment Research, what is the best way to go about applying for and/or getting funding? (In a perfect world, this funding would cover compute and SF living expenses).

Thanks! RGRGRG