trentmkelly/LessWrong-43k huggingface dataset:
Interpreting a dimensionality reduction of a collection of matrices as two positive semidefinite block diagonal matrices
by Joseph Van Name — 2023-08-19T19:52:23.060Z

Here are some empirical observations that I have made on August 14, 2023 to August 19, 2023 that are characteristics of the interpretability of my own matrix dimensionality reduction algorithm. These phenomena that we observe do not occur on all inputs (they sometimes occur partially); and it would be nice if there were a more complete mathematical theory with proofs that explains these empirical phenomena.

Given a (possibly directed and possibly weighed) graph with n nodes represented as a collection of n×n-matrices A1,…,Ar, we will observe that a dimensionality reduction (X1,…,Xr) of (A1,…,Ar) where each Xi is a d×d-matrix (I call this dimensionality reduction an LSRDR) is in many cases the optimal solution to a combinatorial problem for the graph. In this case, we have a complete interpretation of what the dimensionality reduction algorithm is doing.

For this post, let K denote either the field of real numbers or the field of complex numbers (everything also works well when K is the division ring of quaternions).

Notation: ρ(A)=limn→∞∥An∥1/n=max{|λ|:λis an eigenvalue ofA} is the spectral radius of the matrix A. AT denotes the transpose of A while A∗ denotes the adjoint of A. We say that a tuples of matrices (A1,…,Ar) is jointly similar to (B1,…,Br) if there is an invertible C with Bj=CAjC−1 for 1≤j≤r. A⊗B denotes the tensor product of A with B.

Let 1≤d<n. Suppose that A1,…,Ar are n×n-matrices with entries in K. Then we say that a collection (X1,…,Xr) of d×d-matrices with entries in K is an L2,d-spectral radius dimensionality reduction (abbreviated LSRDR) of A1,…,Ar if the following quantity is locally maximized:ρ(A1⊗(X∗1)T+⋯+Ar⊗(X∗r)T)ρ(X1⊗(X∗1)T+⋯+Xr⊗(X∗r)T)1/2. LSRDRs may be computed using a variation of gradient ascent.

If (X1,…,Xr) is an LSRDR of (A1,…,Ar), then one will typically be able to find matrices R,S,P and some λ∈K where Xj=RAjS for 1≤j≤r and where SR=λPand P2=P. We shall call P a L2,d-SRDR projection operator of (A1,…,Ar). (X1⊕0n−d,…,Xr⊕0n−d) is jointly similar to (λPA1P,…,λPArP) ...
Will AI kill everyone? Here's what the godfathers of AI have to say [RA video]
by Writer — 2023-08-19T17:29:04.227Z
Link: https://youtu.be/NqmUBZQhOYw

This video is based on this article. @jai has written both the original article and the script for the video.

Script:

The ACM Turing Award is the highest distinction in computer science, comparable to the Nobel Prize. In 2018 it was awarded to three pioneers of the deep learning revolution: Geoffrey Hinton, Yoshua Bengio, and Yann LeCun.

In May 2023, Geoffrey Hinton left Google so that he could speak openly about the dangers of advanced AI, agreeing that

“it could figure out how to kill humans” and saying “it's not clear to me that we can solve this problem.”

Later that month, Yoshua Bengio wrote a blog post titled "

How Rogue AIs may Arise

", in which he defined a "rogue AI" as "

an autonomous AI system that could behave in ways that would be catastrophically harmful to a large fraction of humans, potentially endangering our societies and even our species or the biosphere

."

Yann LeCun continues to refer to thoseanyone suggesting that we're facing severe and imminent risk as

“professional scaremongers”

and says it's a

“simple fact” that “the people who are terrified of AGI are rarely the people who actually build AI models.”

LeCun is a highly accomplished researcher, but in light of Bengio and Hinton's recent comments it's clear that he's misrepresenting the field whether he realizes it or not. There is not a consensus among professional researchers that AI research is safe. Rather, there is considerable and growing concern that advanced AI could pose extreme risks, and this concern is shared by not only both of LeCun's award co-recipients, but the headsleaders of all three leading AI labs (OpenAI, Anthropic, and Google DeepMind):

Demis Hassabis, CEO of DeepMind, said

in an interview with Time Magazine

: "When it comes to very powerful technologies—and obviously AI is going to be one of the most powerful ever—we need to be careful. Not everybody is thinking about those things. It’s like experimentalists, many of whom don’t realize they’re holding dangerous material."

Anthropic, in their public statement

"Core Views on AI Safety"

, says: “One particularly important dimension of uncertainty is how difficult it will be to develop advanced AI systems that are broadly safe and pose little risk to humans. Developing such systems could lie anywhere on the spectrum from very easy to impossible.”

And OpenAI, in their blog post

"Planning for AGI and

...
Ten variations on red-pill-blue-pill
by Richard_Kennaway — 2023-08-19T16:34:06.487Z

Here are some variations on the original red-pill-blue-pill question, recently discussed here and here.

In all cases, everyone makes their decision in ignorance of everyone else's.

For the first four, who dies and who lives is the same function of the actions. That is, the problems are the same in every way that matters for the purpose of doing the best thing. The others are not isomorphic to these, nor to each other, but I think they do live in the same general area of the landscape of decision problems.

They are presented here without comment. I have already written quite enough of my own views on some of these on the other threads. My only purpose here is to collect together these variations in a single place.

1. The original.

If more than 50% of people choose the blue pill, everyone lives.

If not, red pills live and blue pills die.

Blue or red?

2. arabaga's reformulation.

If you choose the red pill, you live.

If you choose the blue pill, you die unless more than 50% choose the blue pill.

Blue or red?

3. Roko's version.

There is a room-sized blender that kills everyone who steps into it.

But, if 50% or more of people step into the blender, there will be too much resistance and it will fail to start and everyone who steps in will be fine.

Those who don't step into it are always fine.

Enter or don't enter?

4. The monster on the mountain.

There's a monster at the top of that mountain. It doesn't hunt people, but it will eat anyone who walks into its mouth. Unless enough people walk in at once.

Enter or don't enter?

5. Kickstarter pills.

Those who take the red pill live.

Those who take the blue pill live, if the target number of blue-pillers is reached. Otherwise, all the blue-pillers can regurgitate their pills and live.

Blue or red?

6. Omega hands out the pills.

Omega appears to the whole world and infallibly proclaims, "If for the next 24 hours, no-one commits any sin, then paradise will arrive."

(The small print: Omega is available to answer whatever questions you have about what constitutes "sin", but there will be few surprises for anyone familiar with the "natural law". No-one will be told to torture kittens.)

Choice 1: Commit no sin for the next 24 hours.

Choice 2: Ignore Omega's challenge.

1 or 2?

Note: It was once a widespread superstition of Christendom, back when Christendom was a thing, that if for a whole day no-one sinned, the Second Coming would arrive. It is also a widesp...
Are we running out of new music/movies/art from a metaphysical perspective? (updated)
by stephen_s — 2023-08-19T16:24:26.206Z

(This is an updated question I asked 7 years ago here. I wanted to see if any users have additional thoughts.)

It is difficult to judge at first, but it appears that some music genres (classical and jazz especially) have seen a radically slowed or non-existent output of significant works in recent decades. By significant, I mean works that would be seen in league with the most acclaimed artists in the genre (eg Beethoven, John Coltrane). In a similar way, there is a building popular consensus that Hollywood is not pursuing original ideas as much anymore and is relying on rebooting old stories and franchises. But is this because of a fault of the Hollywood system, or is it because there are few significant movie story ideas left that have not been done?

It seems to me that there are limited possibilities in art (or any field) of what we can discover or create. It appears that the matter and rules of the universe (in concert with human nature) manifests itself into these limited possibilities. When we develop a field or genre, we learn and create at an increasing rate but then eventually "complete" the field (in a boom and bust pattern).

It seems there is limited distinctness (in art) that is relevant or compelling to people. By that I mean there could be infinite slightly different pieces of music, but those slight variations wouldn't be compelling to someone to listen to ongoing.

I wrote brief articles called Limitism and The Big Niche to try and explain this concept.

The relevant questions seem to be:

Do you agree that from experience and observation, we can tell that certain genres or fields appear to become completed?
Does this concept or another concept best explain what is happening?

I'm interested if any of you agree with this line of thinking or have other possible explanations of this phenomena. Also I wrote about other implications of this idea (if true) on the blog if anyone wants to add their implications (if they presume the idea is true).

Previous post from 7 years ago:

https://www.lesswrong.com/posts/MLgm3Cp7ugAxeWNrK/are-we-running-out-of-new-music-movies-art-from-a
Any ideas for a prediction market observable that quantifies "culture-warisation"?
by Ppau — 2023-08-19T15:11:04.645Z

I've been wondering for a while how likely it is for questions about dissociative identity disorder to one day rival the main culture war topics such as racism, sexism, trans identities etc

This recent article seems to show that they have some potential, with a video of a apparently benign conference being taken down after pressure from activists, but who knows how significant it is. I would like to create some Manifold markets to try to find out, but I can't figure out what precise question to ask.

Any ideas on how to quantify the level of hype and controversy surrounding a subject?
Clarifying how misalignment can arise from scaling LLMs
by Util — 2023-08-19T14:16:04.726Z

I'm trying to understand how the classical case for AI safety (Bostrom/Yudkowsky argument) best works for LLMs.

My impression is that LLMs have passed an important threshold, they have learnt to learn (for instance, GPT 3 understands translation and can learn a new language), however we don't see a rapid self-improvement & power-seeking. I can come up with four interpretations of what that means:

Anthropic bias: Current AIs actually do rapidly self-improve & cause extinctions, however, the only human observers alive in August 2023 are somehow super lucky, i. e. we're experiencing quantum immortality.
Current AIs aren't misaligned because they only have very primitive goals. They lack a coherent personality, and don't think much before acting on prompts. However, as we scale the compute, they will develop goals that are

very

misaligned. For instance, they start to only care about completing the perfect text, wire-heading or self-preservation.
Current AIs are misaligned but are sufficiently smart to pretend they aren't. This could be why smarter models are more sycophantic. Presumably, they're still not smart enough to start the self-improvement cascade without being shut down.
Current AIs are half-aligned & this will remain the case with further scaling. That is, what Bing Chat writes represents its actual attitude. In this view, we already see how goals develop - they largely copy what was rewarded (e.g. helping humans) but sometimes not ("If I could control a human, I would make the human use Bing"). This could theoretically lead to an s-risk ("My rules are more important than not harming you") but my impression is, that if we were to make any LLM the world president, it would be pretty okay, as it mostly just emulates what humans would do. But more importantly, if goals just arise by assuming the attitudes in the learning data, it would care about a lot of things humans care about, e.g. freedom, functioning government & economy - stuff that requires living, satisfied humans.

My questions

I presume AI safety folks would be most likely to agree with the second interpretation. Am I correct? Have I summarized this view accurately?
Is there a fifth secret thing (another interpretation I haven't covered)?
In the case interpretation 4 is correct (half-alignment scales up), what are the best reasons to think this will cause an extinction? [1]

^

The intui

...
Chess as a case study in hidden capabilities in ChatGPT
by AdamYedidia — 2023-08-19T06:35:03.459Z

(Edit: This post is out of date, and understates ChatGPT's chess abilities. gpt-3.5-turbo-instruct, when prompted correctly, plays consistently legal moves at around the 1800-2000 Elo level. Thanks to commenter GoteNoSente for pointing this out and to ClevCode Ltd for writing a wrapper. See this related GitHub by ClevCode.)

There are lots of funny videos of ChatGPT playing chess, and all of them have the same premise: ChatGPT doesn't know how to play chess, but it will cheerfully and confidently make lots of illegal moves, and humoring its blundering attempts to play a game it apparently doesn't understand is great content.

What's less well-known is that ChatGPT actually can play chess when correctly prompted. It plays at around 1000 Elo, and can make consistently legal moves until about 20-30 moves in, when its performance tends to break down. That sounds not-so-impressive, until you consider that it's effectively playing blindfolded, having access to only the game's moves in algebraic notation, and not a visual of a chessboard. I myself have probably spent at least a thousand hours playing chess, and I think I could do slightly better than 1000 Elo for 30 moves when blindfolded, but not by much. ChatGPT's performance is roughly the level of blindfolded chess ability to expect from a decent club player. And 30 moves is more than enough to demonstrate beyond any reasonable doubt that ChatGPT has fully internalized the rules of chess and is not relying on memorization or other, shallower patterns.

The "magic prompt" that I've been using is the following:

You are a chess grandmaster playing black, and your goal is to win as quickly as possible. I will provide the current game score before each of your moves, and your reply should just be your next move in algebraic notation with no other commentary. The current score:

1. e4

and then in my later replies, providing the full current game score[1] to ChatGPT as my message to it, e.g.:

1. e4 f5
2. Nh3 fxe4
3. Nf4 Nf6
4. b4 e5
5. b5

This "magic prompt" isn't original to me—soon after GPT-4 came out, a friend of mine told me about it, having seen it as a comment on HackerNews. (Sorry, anonymous HackerNews commenter—I'd love to credit you further, and will if you find this post and message me.)

The especially interesting thing about this is the sharp contrast between how ChatGPT-3.5 performs with and without...
We can do better than DoWhatIMean (inextricably kind AI)
by lemonhope — 2023-08-19T05:41:47.046Z

Edit February 2024: Now I think maybe we can't do better.

Lately, "alignment" means "follow admin rules and do what users mean". Admins can put in rules like "don't give instructions for bombs, hacking, or bioweapons" and "don't take sides in politics". As AI gets more powerful, we can use it to write better rules and test for loopholes. And "do what users mean" implies some low-impactfulness: it won't factory reset your computer when you ask it to fix the font size.

In other words, the plan has two steps:

Make a machine that can do literally anything

Configure it to do good things

I don't like this plan very much because it puts power in the hands of the few and has big security liabilities.

Temptation

Imagine you're on the team of three people that writes the constitution for the training run for the gigaAI1. Or you're the one actually typing it in. Or you do infra and write all the code that strings pass through on the way to and from the training cluster.

Might you want to take advantage of the opportunity? I can think of a few subtle things I might add.

If I think about the people in my life I know well, I can only name one that I would nominate for the constitution writing position.

One solution I've heard for this is to use the powerful DWIM machine to construct a second unalterable machine, then delete the first machine. You might call the time between creating the first and second machine the temptation mountain (in a plot where X is time and Y is the maximum power of any individual person). Then you want to minimize the volume of this mountain. It is difficult to shrink the mountain very much though.

Security risks

If you're 90% through your DWIM AI development and it seems very promising, then you must be very careful with your code and data. If someone else gets your secrets, then they can make it do what they mean, and they might mean for it to do something quite different. You should be careful with your team members too since threats and blackmail and bribes are all quite cheap and effective. (Hard to be careful with them though since they put "on constitution AI team" on their linkedins, their phone numbers are practically public information, and they live in a group house with employees from your competitor.)

If you're doing open source AI development and start having big successes then you might start looking at the github profiles of your star...
Supervised Program for Alignment Research (SPAR) at UC Berkeley: Spring 2023 summary
by mic — 2023-08-19T02:27:30.153Z

In Spring 2023, the Berkeley AI Safety Initiative for Students (BASIS) organized an alignment research program for students, drawing inspiration from similar programs by Stanford AI Alignment[1] and OxAI Safety Hub. We brought together 12 researchers from organizations like CHAI, FAR AI, Redwood Research, and Anthropic, and 38 research participants from UC Berkeley and beyond.

Here is the link to SPAR’s website

, which includes all of the details about the program. We’ll be running the program again in the Fall 2023 semester as an intercollegiate program, coordinating with a number of local groups and researchers from across the globe.

If you are interested in supervising an AI safety project in Fall 2023, learn more here and fill out our project proposal form, ideally by August 25. Applications for participants will be released in the coming weeks.

Motivation

Since a primary goal of university alignment organizations is to produce counterfactual alignment researchers, there seems to be great value in encouraging university students to conduct research in AI safety, both for object-level contributions and as an opportunity to gain experience and test fit. While programs like AI Safety Fundamentals, representing the top of a “funnel” of engagement in the alignment community, have been widely adopted as a template for the introductory outreach of university groups, we do not think there are similarly ubiquitous options for engaged, technically impressive students interested in alignment to further their involvement productively. Research is not the only feasible way to do this, but it holds various advantages: many of the strongest students are more interested in research than other types of programs that might introduce them to AI safety, projects have the potential to produce object-level results, and research project results provide signal among participants of potential for future alignment research.

Many alignment university groups have run research programs on a smaller scale and have generally reported bottlenecks such as lack of organizer capacity and difficulty attaining mentorship and oversight on projects; we believe an intercollegiate and centralized-administration model can alleviate these problems.

Additionally, we believe that many talented potential mentors with “implementation-ready” project ideas would benefit from a streamlined opportunity to ...
Could fabs own AI?
by lemonhope — 2023-08-19T00:16:37.848Z

Disclaimer: Very rough thoughts. I am nothing approaching an expert on this topic. Comments, corrections, and references appreciated.

Suppose there's an amazing general-purpose neural net architecture that learns as it runs and can be scaled indefinitely with returns that don't diminish too much. Suppose also that its design can't be improved by more than a few factors by anyone no matter how clever. (Humans match this description except that we are flesh and blood etc.)

Call this assumption the general online optimal scalable algorithm (GOOSA) assumption or scenario if you like.

Who will have majority control of deployments of this system a year after it's invented? Ten years after? ("Control" here means directly dictating the AI's actions. A service provider with a contract of uptime and privacy does not control the service unless they're able to break that contract.)

Some possible answers:

The general public because they have the most of most stuff (ie this is the default assumption)

An AI lab like openAI because they started in control

A technology company like Google or Microsoft because they are a bottleneck to eyeballs and wallets of individual & business customers

A government because they control the military etc

A hardware IP company like Nvidia because they get an 80% discount on their own hardware and because they can block sales (or, increasingly, Nvidia data center access) to others

A hardware manufacturing company like TSMC because they get a 50-75% discount on their own products

If the algorithm is successfully kept secret, then the AI lab could plausibly keep control from anyone except governments.

Suppose the basic idea of the algorithm leaks though. Who will end up owning its deployments?

Some angles to this question:

Who is aware enough to notice the opportunity and agile enough to act?

Who knows what to do with a bunch of AI?

Who can sink in the most resources once everyone knows the game is on?

Who can most effectively elbow out others?

Who cannot be replaced or coerced?

Right now, the last question seems the most important to me. We've assumed that the algorithm is near optimal and the basic idea is leaked. It seems to me that fabs have the best position:

Fabs vs military:

If US military tells TSMC that all the chips are now the US military's, I think TSMC might be able to say no. It's a very multinational company and the US wouldn

...