trentmkelly/LessWrong-43k huggingface dataset:
AI as a Civilizational Risk Part 6/6: What can be done
by PashaKamyshev — 2022-11-03T19:48:52.376Z

Fix or destroy social media

One of the critical positive developments is the potential of Elon Musk to buy Twitter. The acquisition happened as the essay was being finalized. Positive effects that can happen are cracking down on bots and re-vamping moderation to avoid bad AI-driven mischaracterization of public opinion. However, the main benefit would be a potential implementation of a non-optimization feed ranking

algorithm in the vein of TrustRank

. Proper feed ranking would promote socially cohesive ideas instead of wedge issues.

Aside from Elon's specific actions around Twitter, most social media needs to be destroyed or drastically reformed. We need to be careful around ranking algorithms. Any algorithm with an "optimization nature" rather than a "contractual nature" must be viewed with suspicion. At the very least, science needs to test the effect of using some websites for a prolonged time. If it causes mental health issues in people or small groups, this is a sign of imposing unacceptable externalities. Setting up such tests requires a good assessment of mental health problems and how to measure them correctly. Even lacking great assessments with the crude approximation that we have today, we can design social media that does not slowly cause people to go insane. In addition to personal defense, there needs to be "group defense" against hostile outside optimization. This reasoning has caused me to research this area, developing TrustRank, hoping that it becomes a core algorithm in the future of social media, similar to the way PageRank is a core algorithm of current search engines. Even correctly measuring social cohesion can give decision-makers some ideas about how to preserve it. Of course, this requires decision-makers that care about the nation's well-being, the absence of which is part of the problem. We would also need to implement solutions to the governments forcing social media to use AI to de-platform people with valuable insights. However, since these issues are well-known, they are more likely to happen by default through web3 systems.

However, completely replacing social media is not enough to stave off civilizational risk. We still have the background problems of social cohesions, which are not new but reappear every so often in civilization. Elites need to i...
A Mystery About High Dimensional Concept Encoding
by Fabien Roger — 2022-11-03T17:05:56.034Z

Epistemic status: I ran some experiments with surprising results, and I don’t have a clear intuition of what it means for the way concepts are encoded in model activations. Help is welcome!

[EDIT] I made a mistake while analyzing the data, and it changes some of the early interpretations of the results. It doesn't change the fact that using a classifier is a poor way of finding the directions which matter most, but it lifts most of the mystery.

[EDIT 2] I found a satisfying explanation of the phenomenon in Haghighatkhah, 2022. Helpful pictures are included in the paper.

The mystery

If you want to find the “direction” corresponding to a concept in a Large Language Model, “natural” methods fail to find a crucial direction: you need to remove >5 dimensions for the model’s ability to use the concept to be affected. This seems to show that the concept is encoded in “a large number of dimensions”. But adversarial methods are able to find one direction which is quite significant (large effect when “removed”). How is that? What does that mean about how information is encoded by LLM?

More details

Let’s say you want to remove the concept of gender from a GPT-2’s activations between layers 23 and 24 (out of 48). (Both are >100 dimensional vectors). A natural thing to do is to have a bunch of sentences about guys and girls, train a linear classifier to predict if the sentence is about guys or girls, and use the direction the linear classifier gives you as the “direction corresponding to the concept of gender”. Then, you can project GPT-2’s activation (at inference time) on the plane orthogonal to the direction to have the model not take gender into account anymore.

This naive attempt fails: you can train a classifier to classify sentences as “guy-sentence” or “girl-sentence” based on the projected activations, and it will get 90% accuracy (down from 95%), and if you project GPT-2’s activation in the middle of the network, it will make only a tiny difference in the output of the network. This is a method called INLP (Ravfogel, 2020) (the original technique removes multiple directions to obtain a measurable effect).

But if you jointly train the projection and the linear classifier in an adversarial setup, you will get very different results: accuracy drops to ~50%, and GPT-2’s behavior changes much more, and sometimes outputs “she” when “he” is more appropria...
Why do we post our AI safety plans on the Internet?
by Peter S. Park — 2022-11-03T16:02:21.428Z

Cross-posted from the EA Forum.

TL;DR: It is plausible that AGI safety research should be assumed compromised once it is posted on the Internet, even in a purportedly private Google Doc. This is because the corporation creating the AGI will likely be training it on as much data as possible. And whenever the AGI knows in advance of our plan “If we see Sign X of misalignment from the AI, we should shut it down and retrain,” it can use this plan against us: for example, by hiding from us Sign X that it would have shown under normal circumstances. If true, this concern implies that the impact of EAs’ past and current efforts on AI safety may have been fundamentally limited by our insufficient security mindset. Thus, we EAs should vigorously investigate whether this concern is well-founded, and make any necessary reforms in our AI safety research norms as soon as possible.

-----

AI capabilities are continuing to scale up. This trend seems likely to continue given empirical scaling laws, the decreasing cost of compute, and AI corporations’ unabating desire to build increasingly powerful AI. If past trends continue, future AI systems will likely be scaled up enough to eventually be able to create sophisticated, long-term plans for whatever mysterious goals they have evolved during training.

A system that can create long-term plans at a comparable or higher capability than humans is easy to make dangerous by accident. This problem is exacerbated by the fact that so far, nobody has found a way to robustly align an AI system towards a benign goal: or indeed, any specific goal.

Many of us EAs prudently consider the societal risk posed by future unaligned AI to be one of the most pressing x-risks—if not the most pressing x-risk—in the world right now. Hundreds of us EAs devote our careers to AI safety, and the community as a whole donates about $50 million to AI safety every year. Our prudence, persistence, and self-sacrifice on this pressing issue deserve to be commended.

However, I have recently thought of a concern that, if true, implies that the impact of EAs’ past and current efforts on AI safety may have been fundamentally limited by our suboptimal modus operandi for these efforts. The concern is the following:

The corporation that creates the first AGI will (despite our best efforts) train it on as much data as possible, likely the whole Internet and po...
Multiple Deploy-Key Repos
by jefftk — 2022-11-03T15:10:03.820Z

Let's say you want to give a machine access to one of your repos but
not all of them. This is a reasonable thing to want, and so GitHub
offers

deploy
keys. But what if you want to give it access to two repos?

The natural thing to try would be to add the same public key to your
second repo, but GitHub rejects this:

Error: Key is already in use

I think this error is because they don't want you to get into a
situation where multiple machines are using the same key. If that
happened and you needed to revoke one machine's access, you'd be
stuck. In this case, however, we only have one machine and we're
trying to use the same key for two repos. I don't see any issues with
that setup, and while maybe I'm not being imaginative enough I think
GitHub should probably be checking for duplicate deploy keys on a
per-repo basis instead of globally?

Still, what can we do with GitHub as it is? Generate more keys and
use aliases!

I'm going to walk through this assuming you're starting from scratch
trying to check out both github.com/you/repo1 and
github.com/you/repo2. If you already have
repo1 working and don't mind having the two repos
configured differently, just follow the repo2 steps.

First, generate a ssh key for each repo:

$ ssh-keygen -t ed25519 -C "machineName-repo1"
Enter file in which to save the key: /home/you/.ssh/id_ed25519.repo1
...

$ ssh-keygen -t ed25519 -C "machineName-repo2"
Enter file in which to save the key: /home/you/.ssh/id_ed25519.repo2
...

Then visit github.com/you/repo1/settings/keys and
github.com/you/repo1/settings/keys and paste the contents
of ~/.ssh/id_ed25519.repo1 and
~/.ssh/id_ed25519.repo2 respectively.

At this point you've created the keys and told GitHub to respect them,
but you haven't told ssh on your machine when to use
which key. You do that in ~/.ssh/config:

Host github-repo1
HostName github.com
User git
IdentityFile ~/.ssh/id_ed25519.repo1
IdentitiesOnly yes

Host github-repo2
HostName github.com
User git
IdentityFile ~/.ssh/id_ed25519.repo1
IdentitiesOnly yes

Anyway, now you can check out your repos:

$ git clone github-repo1:you/repo1.git
$ git clone github-repo2:you/repo2.git

The reason this works is that git, like anything else that uses ssh, doesn't
actually interpret the host name or set up the connection. It ...
Covid 11/3/22: Asking Forgiveness
by Zvi — 2022-11-03T13:50:00.448Z

It was a quiet week on the Covid front. That is a very good thing.

If anything, the theme of this week was looking backwards to how things were handled. The CDC is (at least half-heartedly) looking to fix things. There was an offer of ‘pandemic amnesty’ by someone who thinks they don’t need amnesty. There was more consideration of counterfactual actions on the libertarian roads not taken.

There was, however, a rise in cases and deaths. This raises the chances of a substantial winter wave.

Executive Summary

Cases and deaths rise modestly, raising chance of winter wave.

Forgiveness is often offered where it needs to also be requested.

Things are the largely the good kind of quiet.

Let’s run the numbers.

The Numbers

Predictions

Predictions from Last Week: 218k cases (+0%) and 2,150 deaths (-5%)

Results: 240k cases (+10%) and 2,618 deaths (+8%).

Predictions for Next Week:

Last week I missed that North Dakota reported -151 deaths. I have now corrected that to zero. If I had caught it I would have then revised this week’s prediction up somewhat, still would have missed low. We are far enough away from the major declines in cases that deaths no longer declining is plausible.

We continue to see deaths and cases move together even when there is no obvious change in reporting that would seem to be causing this.

Despite that, I do think the rise in cases is likely real. Given the timing and how things are going with variants, that means it is likely to continue for a bit. I would interpret one more week of rise as reasonably strong evidence of a future substantial winter wave, which was always a possibility. The regional patterns all match what one would expect to see here. I wouldn’t expect it to get out of hand, certainly not anything like we saw last year.

Deaths

Cases

Booster Boosting

No noteworthy new attempts.

Variants

No news, which is by default good news. New variants are still taking over.

Physical World Modeling

A bunch of people are tweeting the same chart with very different ideas about what it means.

The usual Long Covid brigade has one interpretation, because it is their interpretation for everything.

Those who read economic data for a living have a different one.

The second group seems clearly right here, or at least citing the far bigger effect. This is not ‘million...
K-types vs T-types — what priors do you have?
by Cleo Nardo — 2022-11-03T11:29:00.809Z

Summary: There is a spectrum between two types of people, K-types and T-types. K-types want theories with low kolmogorov-complexity and T-types want theories with low time-complexity. This classification correlates with other classifications and with certain personality traits.

Epistemic status: I'm somewhat confident that this classification is real and that it will help you understand why people believe the things they do. If there are major flaws in my understanding then hopefully someone will point that out.

Edits: Various clarifying remarks.

K-types vs T-types

What makes a good theory?

There's broad consensus that good theories should fit our observations. Unfortunately there's less consensus about to compare between the different theories that fit our observations — if we have two theories which both predict our observations to the exact same extent then how do we decide which to endorse?

We can't shrug our shoulders and say "let's treat them all equally" because then we won't be able to predict anything at all about future observations. This is a consequence of the No Free Lunch Theorem: there are exactly as many theories which fit the seen observations and predict the future will look like X as there are which fit the seen observations and predict the future will look like not-X. So we can't predict anything unless we can say "these theories fitting the observations are better than these other theories which fit the observations".

There are two types of people, which I'm calling "K-types" and "T-types", who differ in which theories they pick among those that fit the observations.

K-types and T-types have different priors.

K-types prefer theories which are short over theories which are long. They want theories you can describe in very few words. But they don't care how many inferential steps it takes to derive our observations within the theory.

In contrast, T-types prefer theories which are quick over theories which are slow. They care how many inferential steps it takes to derive our observations within the theory, and are willing to accept longer theories if it rapidly speeds up derivation.

Algorithmic characterisation

In computer science terminology, we can think of a theory as a computer program which outputs predictions. K-types penalise the kolmogorov complexity of the program (also called the description complexity), whereas T-types penalise the time-comp...
Information Markets 2: Optimally Shaped Reward Bets
by eva_ — 2022-11-03T11:08:49.126Z

Sequel to Information Markets, which contains a long text outline of what I consider to be the correct alternative to Prediction Markets, which I don't like for a long list of reasons.

This post is intended to fill in the gap in the original regarding what an ideally shaped bet to prove a belief to someone wanting to buy true information efficiently actually looks like.

Aligned Incentives

Suppose the seller has Utilityfunction Us, and the community Uc. Without the sellers information, the market will believe distribution X, and so make decision D. With the sellers information added, the market will believe distribution X′, and so make decision D′.

For the seller to have aligned incentives, we need:

E[Us|X′+D′]>E[Us|X′+D] (if the info is true, you should share it)

E[US|X+D]>E[US|X+D′] (if the info is false, you shouldn't share it).

The difference between these two define a Bias for or against telling the truth, and which may need to be overcome to convince them to share honestly. For now, we assume an unbiased source has no difference in Utility depending on the markets decision, and so that complexity can be skipped over. They can still hava a difference in utility in terms of the effort of the cost of making the claim, or the reward the market offers for telling it information, and we can give them shaped incentives with an Optimally Shaped Bet.

Reshaping Interests

To motivate a seller to communicate information, we want to offer them R(X′;X) in profit for telling us X, if it's actually true. If it's False, we want a loss sufficient to pay someone for correcting it back:

E[Bet(X;X)|X]=0 adding no information gives no payout

E[Bet(X′;X)|X]=−E[Bet(X;X′)|X] a wrong bet must lose enough to correct it back

E[Bet(X′;X)|X′]=R(X′;X)>0 a correct bet gives you expected profit

For our function to be optimal, we need that the seller who believes X′ is best off buying a bet that communicates their sincere beliefs, meaning that:

X′=argmaxXCE[Bet(XC;X)|X′]

We also want to minimise risk, if it is true information, because excessive risk becomes a transactional cost, subject to above constraints:

Bet=argminBVar[B(X′;X)|X′]

Whats the chance this is sufficient to exactly pin down our mystery functions Bet, at least up to a constant factor of how much subsidy is offered?

Bet(X′;...
The Rational Utilitarian Love Movement (A Historical Retrospective)
by Caleb Biddulph — 2022-11-03T07:11:28.679Z
Link: https://ratutilove.substack.com/p/the-rational-utilitarian-love-movement

I found this post the other day and quite enjoyed it. Even if you disagree with all of its conclusions, it's at least a fun story full of hilarious references to esoteric bits of EA/rationalist culture. I recommend taking a read!

Vague spoilers below

The post is an alternate-history parody of EA, in which all of the bad parts of the movement are amplified just enough to make it truly terrible. It would never work as e.g. a submission to the EA Criticism Contest, but it hits on things that I think would be hard to get across in an intellectual essay (I guess it's just that darn pathos).
The Mirror Chamber: A short story exploring the anthropic measure function and why it can matter
by mako yass — 2022-11-03T06:47:56.376Z

This story warrants and motivates the study of the hard problem of consciousness, presenting a plausible future scenario in which a notion of quantifiable anthropic measure as a property of an arrangement of matter demands operational definition on pain of death.

I know of no other piece of writing that motivates the hard problem of consciousness this forcefully, or at all, really.

The first section explains the quite plausible future history that might lead to the Mirror Chamber occurring in reality. If you are in a hurry, it's okay to skip straight to the section The Mirror Chamber

Our World

It's expensive to test new artificial brains. It's expensive because you can't tell whether a brain is working properly without putting some real human mind in it. For a long time, you just had to find someone who'd been granted a replication license, who was willing to put their life (or what is essentially the life of a close sibling) at risk just to help a manufacturing org to make a little incremental step forward.

You can't just acquire a replication license and give it to the manufacturing org's Ceo. Replication licenses cannot be bought. If they could, they would be very expensive in the wake of Replica Insurgencies, the ongoing Replica Purges, and all of the controversy over Replica Enfranchisement.

Nor can you take one of the "free" mind-encodings that abound over the shaded nets and use it in secret. We do have our Jarrets, who will claim to have consented to be instantiated in any situation their downloaders might desire, however perverted. For better or worse we have decided that Jarretism is a deeply wretched form of insanity, closely linked to open Carcony. It is illegal to intentionally transmit or instate any of the mind-encodings traded on the shaded nets.

I should explain Carcony. Carcony is what we call the crime underlying the insurgencies. A sickness in the body of mankind in which an errant contingent have begun to replicate like a plague, build armies, factories, gross, hypernormative nations where the openness and compromise of genuine humanity are not present.

Our society's memory of Carcony is still bleeding. The laws governing these sorts of activity are rightly very restrictive, but the work still needs to be done. Faster, more efficient neobrains need to be tested. Sound heads won out, a dialogue was had, a solution was devised.

A new professional c...
Open Letter Against Reckless Nuclear Escalation and Use
by Max Tegmark — 2022-11-03T05:34:44.529Z

There's been ample interest in the probability that the Ukraine conflict escalates into a nuclear way, ranging from predictions on Metaculus, Manifold and Polymarket to estimates by David Orr, myself and others. Aside from estimating this probability, what can we do to reduce it?

Current discourse is often framed as if there were only two options:

Be forceful
Deescalate by appeasement, effectively capitulating to nuclear blackmail

This is misleading, because there's more than one way to stand strong (including going on the offense militarily): (a) pursuing or (b) eschewing escalation that has negligible military value. Suppose a bully punches you. If you choose to stand strong, I'd encourage you not to combine it with spitting in his face, poking fun of his appearance and goading him to pull out that gun you see he has holstered. Unfortunately, the Ukraine conflict has seen many instances of analogous escalation without significant military benefit, and arguably not only from one side. Stigmatizing such reckless behavior can in my opinion reduce the risk of nuclear war without any appeasement or concessions. I'm therefore inviting you to join me and many others as a signatory on an open letter that aims both to de-normalize nuclear threats and to re-mainstream non-appeasing de-escalation strategies that reduce the risk of nuclear war without giving into blackmail.

Would you like to join as a signatory? This would be wonderful, because it would help reduce the probability of the greatest catastrophe in human history. To read and potentially sign the open letter, please click here.
Lazy Python Argument Parsing
by jefftk — 2022-11-03T02:20:05.466Z

When I'm writing real Python programs I use

argparse.
When I'm writing quick scripts, I destructure

sys.argv[1:]. For example:

width, depth, height = sys.argv[1:]
...

Or, if I need to import the file:

def start(width, depth, height):
...

if __name__ == '__main__':
start(*sys.argv[1:])

This is a very simple way of handling positional arguments. It
ignores sys.argv[0] which is likely the name of the
script, and then assigns the remaining arguments to the variables. If
I provide the wrong number of arguments it's a bit shouty, but it's
clear enough for a quick script and does remind me of the intended
arguments:

Traceback (most recent call last):
File "example.py", line 2, in
width, depth, height = sys.argv[1:]
ValueError: too many values to unpack (expected 3)

Comment via: facebook