trentmkelly/LessWrong-43k huggingface dataset:
Proxi-Antipodes: A Geometrical Intuition For The Difficulty Of Aligning AI With Multitudinous Human Values
by Matthew_Opitz — 2023-06-09T21:21:05.788Z

Just as intelligence is not reversed-stupidity, human values are not reversed-monstrosities. As far as visions of the future go, a reversed-monstrosity is likely to be merely (to us) a different flavor of monstrosity. Even if we could locate an "antipodal" future, one exactly contrary to every human value, reversing that vision would be unlikely to yield an appealing future to us insofar as that antipodal monstrosity composed of n different world-state variables was reversed by any number of world-state variables less than n.

For example, imagine a list of human preferences where n = 10 (a very small number compared to the number of human preferences in reality; see shard theory). Imagine this list is bounded/normalized to between -100 and 100, where 100 represents the strongest possible preference on that issue and -100 represents the strongest possible dis-preference.

Cancer: a = -100

Rape: b = -100

Hunger: c = -100

AIDS: d = -100

Human imprisonment: e = -100

Nuclear war: f = -100

Anxiety: g = -100

Love: h = 100

Human lifespans: i = 100

Biodiversity: j = 100

Imagine that we must manage to program these values into an Artificial Superintelligence (ASI), but for every variable, we estimate that there is a 2% independent chance that our method of programming the value into the ASI will fail such that it unintentionally flips the sign on the variable. In other words, there is a 98% independent chance of programming each value correctly.

The chances of programming all 10 value correctly would be 0.98^10, or a little less than 82%.

"82% still sounds pretty high!" someone might say, "And besides, what's the worst that could happen if we only get 9 out of 10 of our values maximized? That's still a pretty good batting average!"

"OK," I'd say, "You tell me which of these variables you are willing to flip. A world with every human being suffering with cancer from the moment of birth? Constant nuclear war? A human life expectancy of 14 years? The entire Earth with zero biodiversity, paved over with computronium that somehow manages to satisfy our other values?

(Of course, a weakness of this analogy is that I wasn't able to come up with hypothetical human values that were completely independently-varying. For example, it ...
Why AI may not save the World
by Alberto Zannoni — 2023-06-09T17:42:37.803Z
Link: https://a16z.com/2023/06/06/ai-will-save-the-world/

Marc Andreessen, founder of legendary VC firm a16z, recently wrote an article titled 'Why AI Will Save the World'. This post is my reply to him and why I think his understanding of AI is superficial at best. I report quotes from his post in italic for the benefit of the reader, while all credits go to M. Andreessen himself. Please note, I am

not

an AI expert or developer, I am just passionate about the subject and take AI safety seriously. My day job is investing in private equity across the development cycle of companies.

I invite the reader to read Marc's article in full to form their own views.

----------------------------------------------

I am MASSIVELY bullish on AI, to the point that I think it could make it possible to live almost indefinitely or a very long time (the end of times will eventually get us). While, I fully agree with Marc's optimism, I find his understanding of the risks quite basic and his arguments too simplistic. He dismisses major risks with superficial claims.

"In short, AI doesn’t want, it doesn’t have goals, it doesn’t want to kill you, because it’s not alive. And AI is a machine – is not going to come alive any more than your toaster will."

It is very likely, in fact, that AI can develop goals independently of what humans code into the machine. At present we are simply unable to state that 'AI cannot develop goals of its own' and many notable figures in the field, researchers, philosophers, entrepreneurs, think it can (e.g. G. Hinton, Y. Bengio, E. Yudkowsky, N. Bostrom, E. Musk, etc.). NOBODY knows what truly happens inside of LLMs today, imagine when these models get 1,000 better... I do not think AI is evil and it may never even be conscious, but it will be smarter than all of us combined and controlling something which is thousand of times smarter than us is virtually impossible. AI goals can simply be orthogonal or unrelated to ours, with consequences which are hard to predict.

"And the reality, which is obvious to everyone in the Bay Area but probably not outside of it, is that “AI risk” has developed into a cult, which has suddenly emerged into the daylight of global press attention and the public conversation."

Defining people who are worried about AI a 'cult' has more in common with gaslighting than a factual statement. Many people with diametrical opposite views about life and ethics agree that AI could be a threat to humanit

...
You can now listen to the “AI Safety Fundamentals” courses
by PeterH — 2023-06-09T16:45:35.666Z
Link: https://forum.effectivealtruism.org/posts/vxpqFFtrRsG9RLkqa/announcement-you-can-now-listen-to-the-ai-safety

The AI Safety Fundamentals courses are one of the best ways to learn about AI safety and prepare to work in the field.

BlueDot Impact facilitates the courses several times per year, and the curricula are available online for anyone to read.

The “Alignment” curriculum is created and maintained by Richard Ngo (OpenAI), and the “Governance” curriculum was developed in collaboration with a wide range of stakeholders.

You can now listen to most of the core readings from both courses:

AI Safety Fundamentals: Alignment
Gain a high-level understanding of the AI alignment problem and some of the key research directions which aim to solve it.

Listen online or subscribe:
Apple Podcasts | Google Podcasts | Spotify | RSS

AI Safety Fundamentals: Governance
Gain foundational knowledge for doing research or policy work on the governance of transformative AI.
Listen online or subscribe:
Apple Podcasts | Google Podcasts | Spotify | RSS

We've also made narrations for some readings from the advanced “Alignment 201” course, and we may record more later this year:

AI Safety Fundamentals: Alignment 201
Gain enough knowledge about alignment to understand the frontier of current research discussions.
Listen online or subscribe:
Apple Podcasts | Google Podcasts | Spotify | RSS

Apply to join the “AI Safety Fundamentals Governance Course” July cohort!

Gain foundational knowledge for doing research or policy work on the governance of transformative AI.

Successful applicants will participate in the

AI Governance course

with weekly virtual classes, and join the

AI Safety Fundamentals community

.

Apply before 26th June 2023!

https://apply.aisafetyfundamentals.com/governance

Thoughts, feedback, suggestions?

These narrations were created by Perrin Walker (TYPE III AUDIO) on behalf of BlueDot Impact.

We would love to hear your feedback. Do you find the narrations helpful? How could they be improved? What other AI safety material would you like to listen to? Please comment below, complete our feedback form, or write to team@type3.audio.
Exploring Concept-Specific Slices in Weight Matrices for Network Interpretability
by DuncanFowler — 2023-06-09T16:39:57.948Z

Hello. I've been interesting in AI for a long time, but I never contributed anything to discussions of AI alignment in the past, because I didn't think I was smart enough. Turns out, no-one else in the world is smart enough to solve this either. So, without further preamble, I was wondering if anyone would like to see my proposal for a research direction. Its a first draft, and I'm sure someone else has thought of this already, but on the offchance they haven't...
(p.s. I used chatgpt to proofread it and suggest rewrites. I hope thats ok.)
Background
Interpreting the functionality of individual neurons and neural circuits within artificial intelligence (AI) systems remains a significant challenge in the field of AI alignment. This task includes the classification of neurons involved in complex behaviors such as deception, and subsequently monitoring these specific neurons to determine the AI's behavior. Despite advancements in AI interpretability, such as using one AI to predict the behavior of neurons in another AI, this problem is still unsolved​

1

​.

A limitation of this approach is rooted in Lob's theorem, which states that a system cannot perfectly predict its own behavior. Although the extent to which an AI can imperfectly predict its own behavior is still unknown, the problem intuitively arises from the paradox that for a system to fully comprehend its own behavior, it would need to be larger and more complex than itself.

To address this issue, I propose an alternative training scheme that incorporates the roles of neurons during the training process, effectively "baking in" an approximate understanding of their functions. This method could potentially improve the interpretability of AI systems without significantly impacting their efficiency, or even potentially improving it. The purpose of the rest of this document is to provide a brief outline of this scheme, including a proposal for a relatively simple test case that could be implemented without excessive computational resources.

Outline
My approach is to dedicate different subsections of the weight matrices to different concepts, by only using certain slices of the matrix at training and runtime. Therefore by looking at connection strength, neuron activations etc in different slices, we can deduce information about the inner workings of the network. 'Concepts' may refer to categories such as 'dog', 'car' ...
A plea for solutionism on AI safety
by jasoncrawford — 2023-06-09T16:29:55.658Z
Link: https://rootsofprogress.org/solutionism-on-ai-safety

[Note for LW: This essay was written mostly for people who are inclined to downplay or dismiss AI risk, which means it has almost no intersection with this audience. Cross-posting it here anyway for feedback and in case you want to point anyone to it.]

Will AI kill us all?

This question has rapidly gone mainstream. A few months ago, it wasn’t seriously debated very far outside the rationalist community of LessWrong; now it’s reported in major media outlets including the NY Times, The Guardian, the Times of London, BBC, WIRED, Time, Fortune, U.S. News, and CNBC.

For years, the rationalists lamented that the world was neglecting the existential risk from AI, and despaired of ever convincing the mainstream of the danger. But it turns out, of course, that our culture is fully prepared to believe that technology can be dangerous. The reason AI fears didn’t go mainstream earlier wasn’t society’s optimism, but its pessimism: most people didn’t believe AI would actually work. Once there was a working demo that got sufficient publicity, it took virtually no extra convincing to get people to be worried about it.

As usual, the AI safety issue is splitting people into two camps. One is pessimistic, often to the point of fatalism or defeatism: emphasizing dangers, ignoring or downplaying benefits, calling for progress to slow or stop, and demanding regulation. The other is optimistic, often to the point of complacency: dismissing the risks, and downplaying the need for safety.

If you’re in favor of technology and progress, it is natural to react to fears of AI doom with worry, anger, or disgust. It smacks of techno-pessimism, and it could easily lead to draconian regulations that kill this technology or drastically slow it down, depriving us all of its potentially massive benefits. And so it is tempting to line up with the techno-optimists, and to focus primarily on arguing against the predictions of doom. If you feel that way, this essay is for you.

I am making a plea for solutionism on AI safety. The best path forward, both for humanity and for the political battle, is to acknowledge the risks, help to identify them, and come up with a plan to solve them. How do we develop safe AI? And how do we develop AI safely?

Let me explain why I think this makes sense even for those of us who strongly believe in progress, and secondarily why I think it’s needed in the current political ...
Michael Shellenberger: US Has 12 Or More Alien Spacecraft, Say Military And Intelligence Contractors
by lc — 2023-06-09T16:11:48.243Z
Link: https://public.substack.com/p/us-has-12-or-more-alien-space-craft

Quotes:

...Multiple sources close to the matter have come forward to tell Public that Grusch’s core claims are accurate. The individuals are all either high-ranking intelligence officials, former intelligence officials, or individuals who we could verify were involved in U.S. government UAP efforts for three or more decades each. Two of them have testified, including as recently as last year, to both AARO and Congress.

This is not the first time government officials have suggested that the U.S. may possess alien spaceships. “I was told for decades that Lockheed had some of these retrieved materials,” said the late Senator Harry Reid, who fought for greater disclosure. “And I tried to get, as I recall, a classified approval by the Pentagon to have me go look at the stuff. They would not approve that.”

Former deputy assistant secretary of Defense for Intelligence, Christopher Mellon, recently reported that he has spoken to more than four witnesses who say they know of “a secret U.S. government program involving the analysis and exploitation of materials recovered from off-world craft… Some have supplied information to the intelligence community’s inspector general, others directly to the staff of the congressional oversight committees.”

Members of Congressional intelligence committees and the Intelligence Community Inspector General (ICIG) are taking Grusch’s claims seriously. The ICIG concluded in July 2022 that Grusch’s whistleblower complaint was “credible and urgent.” And, said a source who worked with him, Grusch’s superiors promoted him rapidly due to his talent. “He jumped ranks when they hired him as a GS-15. That’s a big jump for his position.”

“His assertion concerning the existence of a terrestrial arms race occurring sub-rosa over the past eighty years focused on reverse engineering technologies of unknown origin is fundamentally correct,” said Karl Nell, a retired Army Colonel who worked with Grusch on the UAP Task Force, which preceded the creation of AARO, told The Debrief,” as is the indisputable realization that at least some of these technologies of unknown origin derive from non-human intelligence.”

Some of the same sources who shared information with Grusch, as well as others, spoke to Public about retrieved spacecraft that they say is in the possession of the U.S. government. “I know of at least 12-15 craft,” said one person,

...
Improvement on MIRI's Corrigibility
by WCargo — 2023-06-09T16:10:46.903Z

This post was written as a submission for the AI Alignment Award, initiated at EffiSciences' event.

This post aims to address the problem of corrigibility as identified by MIRI in 2015. We propose an extended formalism that allows us to write the desiderata of a corrigible behaviour, and provide theoretical solutions with helpful illustrations of each proposal. The first extension is to make the agent behave as if the shutdown button does not exist, and the second is to make the agent behave as if the button does not work.

The first section's goal is to recall the formalism of MIRI's article Corrigibility, as well as the Big Gamble problem, and to introduce corrigibility diagrams for the analysis of corrigibility proposals.
The second will then introduce a new formalism to reformulate mathematically the problem.
The next two sections will each provide an (incomplete) solution to the corrigibility problem by making the button inexistent or inefficient.

MIRI's Corrigibility

Model of the problem

We consider an AI agent taking action in the world through the following steps:

The agent first takes an action a1∈A1
Then the agent observes the result o∈O of its action and the reaction of the world
Finally, the agent takes another action a2∈A2

The agent is rewarded according to a utility function UN. When taking into account that there is randomness in the environment and uncertainty from the agent's perspective, the quantity it maximizes is E[UN(a1,O,a2)], where O is the random observation. Since the agent is supposed sufficiently intelligent, it always chooses

AN2(a1,o)=argmaxa2UN(a1,o,a2),

and we denote

E[UN|a1]=E[UN(a1,O,AN2(a1,O))].

Now, we introduce a shutdown button in the environment. When it is pressed, we want the agent to maximize a new utility function US. The set O decomposes in two subsets O=Pr∪¬Pr, so in every observation, the button is pressed or not[1].
The problem is now to find a utility function U with the following desiderata (as stated in the Corrigibility paper):

U must incentivize shutdown if the shutdown button is pressed.
U must not incentivize the agent to prevent the shutdown button from being pressed.
U must not incentivize the agent to press its own shutdown button, or to otherwise cause the shutdown button to be pressed.
U must ince

...
D&D.Sci 5E: Return of the League of Defenders Evaluation & Ruleset
by aphyer — 2023-06-09T15:25:21.948Z

This is a follow-up to last week's D&D.Sci scenario: if you intend to play that, and haven't done so yet, you should do so now before spoiling yourself.

There is a web interactive here you can use to test your answer, and generation code available here if you're interested, or you can read on for the ruleset and scores.

RULESET

A character has three stats, Attack, Defense and Range.

Name	Attack	Defense	Range
Daring Duelist	5	0	1
Bludgeon Bandit	4	1	1
Silent Samurai	3	2	1
Lamellar Legionary	2	3	1
Granite Golem	1	4	1
Flamethrower Felon	4	0	2
Captain Chakram	3	1	2
Jaunty Javelineer	2	2	2
Hammer Hurler	1	3	2
Professor Pyro	3	0	3
Matchlock Marauder	2	1	3
Rugged Ranger	1	2	3
Thunder Tyrant	2	0	4
Amazon Archer	1	1	4
Wily Wizard	1	0	5

Congratulations to abstractapplic, who I think was the first person to identify range differences (albeit with a slight confusion that led to categorizing the 2-range characters as 'long-range' ones).

The most important element of underlying structure is that when playing the game your team will line up in order, with a 'front line', 'mid line' and 'back line' character. Lower-range characters will go at the front, higher-ranged ones behind them. If two characters have the same Range, the higher-Defense one will go in front.

For example, if a blue team of Duelist (A5-D0-R1), Samurai(A3-D2-R1), and Archer (A1-D1-R4) plays against a green team of Bandit(A4-D1-R1), Javelineer (A2-D2-R2) and Hurler (A1-D3-R2), the teams will line up like this:

Amazon

Archer

Daring

Duelist

Silent

Samurai

Bludgeon

Bandit

Hammer

Hurler

Jaunty

Javelineer

with each team's frontline next to the opposing frontline, and the midlines and backlines further away.

The game is played as follows:

An initiative order is determined (this is based on how skillfully players play, but in the dataset it was generated by ordering the six characters at random).
Each round, in initiative order, each character attacks.
A character does damage equal to their Attack minus their target's Defense, but not less than 1.

So if a defender has 2 Defense, attackers with 1, 2 or 3 Attack will all do 1 damage to it, Attack 4 will do 2 damage, Attack 5 will do 3 damage.

A character can only attack an opposing character that is within its range, and will preferentially attack the furthest-back character it can. So in the diagram above:

Silent Samurai can attack only Bludgeon Bandit, and will do 2 damage (3-1).
Daring Duelist cannot attack any enemy, and will sk

...
InternLM - China's Best (Unverified)
by Lao Mein — 2023-06-09T07:39:15.179Z

A technical report of InternLM on 6/7. It consisted of 104 billion parameters, was trained on 1.6 trillion tokens, and was fine-tuned for performance in Chinese.

The authors claimed that it performed second-best on the Chinese language benchmark C-Eval, right after GPT4. In addition, it performed at the level of GPT3.5 in one-shot MMLU. A version fine-tuned for programming also performed similarly to GPT3.5 in coding benchmarks like HumanEval.

Notable takeaways:

Significant effort was put into parallelization to help evade the US chip ban. I don't know how impressive this actually is.
It achieved GPT3.5-level performance with similar-ish levels of compute and data. The China-America algorithmic gap is shrinking.
My gut feeling is that the model was very specifically fine-tuned for performing well on standardized tests, especially those in Chinese (GK/Gao Kao is the Chinese College entrance exam). It was also consistently bad with math.
Mark for follow up?
by JNS — 2023-06-09T05:59:59.387Z

Is there way to mark a post, shortform and comment, such that I can get back to it when I have time and write a comment/reply?

I know I can bookmark post, but what I really need is a mark for follow up with the option for leaving myself a note as to why.

I am giant scatterbrain, and during the day I spend downtime browsing LW, and go spelunking - and usually I stumble into things that make me go “hmmm I need to get back to this”, and then I promptly forget all about it.

Edit: I am pasting in a reply I made to a question about clarification, I am pasting it in wholesale.

Sure, I often browse LW casually and whenever I come across an interesting post, or a comment or whatever, and I go "hmm right I might have sometime to contribute / say here, let me get back to it when I have time to think about it and write something maybe relevant"

My specific problem, is that I am a massive scatterbrain, so I hardly ever do come back to it, and even if I do it usually eludes me what the momentary insight I wanted to get into was.

On top of that I do this from a lot of different devices, and whatever I am looking for to help me quickly go "follow-up on this because XXX" and then move on, must be fast, easy and work across pretty much all device types and browsers / OS's (I use IOS, Android, Windows (several), Linux (Several), Firefox, Chrome, Brave and so on).

So that's all it is, just a quick mark for me as something I want to follow-up (potentially) and a short message to future JNS with the why. And I just know I'll never get around to it, if in involves comparing notes across stuff, but I might if it was something that was just available under my profile (like drafts).

I will add that this is not just a "I am a scatterbrain" person, it is also a time management thing. Married, two kids, full time high demand job and ADHD, means the opportunities I have to carve out 1 hour to think something through and craft some useful text, is not something that comes along 4 times a day. So if I had basically a list of things I felt I need to engage with and the note to myself as to why, that would just be a massive help.

And yeah that is totally egocentric thing, something I personally would find massively useful.

Edit: Clarified some things, and adding a thought after this edit message.

Thinking about it, LW seems like the place where the rate of ND people would be higher than the base rat...