trentmkelly/LessWrong-43k huggingface dataset:
What is good Cyber Security Advice?
by Gunnar_Zarncke — 2022-10-24T23:27:58.428Z

This is a request for cybersecurity advice. I motivate why Lesswrong would benefit from a repository of such advice and why that is relevant for alignment projects (I'm working on one) and propose a structure for answers.

In Six Dimensions of Operational Adequacy in AGI Projects, Yudkowsky asks for

Strong opsec: Operational security adequate to prevent the proliferation of code (or other information sufficient to recreate code within e.g. 1 year) due to e.g. Russian intelligence agencies grabbing the code.

And defines the following levels:

Token: Random people are not allowed to wander through the building.

Improving: Your little brother cannot steal the IP. Stuff is encrypted. Siloed project members sign NDAs.

Adequate: Major governments cannot silently and unnoticeably steal the IP without a nonroutine effort. All project members undergo government-security-clearance-style screening. AGI code is not running on AWS, but in an airgapped server room. There are cleared security guards in the server room.

Excellent: Military-grade or national-security-grade security. (It's hard to see how attempts to get this could avoid being counterproductive, considering the difficulty of obtaining trustworthy command and common good commitment with respect to any entity that can deploy such force, and the effect that trying would have on general mindsets.)

Security is also one of the Asilomar AI Principles:

6) Safety: AI systems should be safe and secure throughout their operational lifetime, and verifiably so where applicable and feasible.

I am a member of the AI Alignment project aintelope and work to properly secure the project as we grow. Right now, we have security at the token level (as defined above): A private Github project, a private Slack group, and some token vetting of people with access.

How can we do better? What resources are there? First, I looked at Lesswrong. There are the tags Computer Security & Cryptography and Security Mindset, which have some relevant posts, including Security Mindset and Ordinary Paranoia, and Secure homes for digital people. There is also Work on Security Instead of Friendliness? which is about the relation between AI Alignment and Security. But there is little material that helps me secure our project. I think this community, with many people working on alignm...
Connections between Mind-Body Problem & Civilizations
by oblivion — 2022-10-24T21:55:51.888Z

Recently I got into the “mind-body problem,” a philosophical term for the question of how our mental consciousness/awareness is related to our physical body/world. Take my thoughts with a grain of salt.

What I noticed is that a civilization’s institutions influence its understanding of consciousness. Note that I’m crudely defining ‘civilization’ as a group with a common culture & history, like the West, the Islamic world, or the Sinosphere.

For example, René Descartes’ mind-body dualism theory, which was widely influential during the 17th century, argued that “the nature of the mind is completely different from that of the body, and therefore it is possible for one to exist without the other” (IEP). Descartes was a Christian, and his theory would appear to support Christianity’s belief in a soul that continues on after death; a sort of ‘mind matter’ that can exist separately from the body. And on the cosmic scale, Christian dualism posits that God and Creation are distinct but interrelated, akin to Cartesian mind-body dualism.

Moving forward a few centuries. Materialism, which supports that consciousness is simply a result of complex electrochemical interactions, is arguably much more popular nowadays. Current acceptance of materialist theories appears to reflect free-market capitalism’s ideals of material gain and mechanistic efficiency. Are we simply cogs in the machine of production?

Certainly all I’m noting here are correlations, as one could argue that the advent of science has shaped philosophy, and that is what accounts for the shift in mind-body theories towards materialism. Following the Enlightenment and Scientific Revolutions, it’s only natural that society took on a more mechanistic understanding of the world. After all, science has provided logical explanations for the movement of the planets, heredity, disease, and much more.
A Good Future (rough draft)
by Michael Soareverix — 2022-10-24T20:45:45.029Z

It is the year 2500.

Humanity has overcome its challenges. AI alignment has been solved and a benevolent god watches over us, allowing us to create and share what we desire. But it no longer interferes too much; at least, not too much in this realm.

There are a lot of augmented humans. They have essentially uploaded their minds via nanotechnology. Every night for 5 years, tiny nanomachines would replace their neurons with synthetic ones until they were fully synthetic. With these synthetic neurons in place, they could think up to 10,000x faster and could add different dimensions to their consciousness, like the ability to see infrared or to have an intuitive understanding of quantum mechanics or to controls hundreds of different bodies at once.

I am one of those augmented humans. My consciousness is digital, composed of photonic signals instead of neurotransmitters. Right now, I am independent and not connected to the sea of computing infrastructure that surrounds me. I’m human, except that I can vary the speed of how fast I think. Right now, actually, I am thinking just 1.5 times faster than a biological human. I could be thinking more than a thousand times faster (light is so much faster than sound, and our biological nervous system transmits at the speed of sound) but I like the way the world moves at this speed.

I am not on Earth. I am about 100 light-years away from Earth in a gigantic solar system that belongs to me. Within this space, I control every atom. Well, not all of them. Those atoms that belong to others are still protected by god. I cannot inflict pain on others without their permission and even then, there are strict limits on the magnitude of suffering that is allowed to exist.

But functionally, since I don't want to cause anyone pain, I can do everything I want to do.

Nanomachines and swarms of self-replicating probes are currently disassembling the planets in orbit of this star. We’re building a Dyson swarm / Matrioshka brain. Basically, a swarm of orbiting computers, powered by the sun. Within this massive computing infrastructure, I have built a universe.

I've been building this universe for ~490 years. I originally intended to be a writer, then a VR developer, then an AI reseacher. Now... I am a worldbuilder in the most literal sense. So are millions of others, and I hope someday I'll experience some of their simulations as well. But for now,...
A Barebones Guide to Mechanistic Interpretability Prerequisites
by Neel Nanda — 2022-10-24T20:45:27.938Z
Link: https://neelnanda.io/mechanistic-interpretability/prereqs

Co-authored by Neel Nanda and Jess Smith

Check out Concrete Steps for Getting Started in Mechanistic Interpretability for a better starting point

Why does this exist?

People often get intimidated when trying to get into AI or AI Alignment research. People often think that the gulf between where they are and where they need to be is huge. This presents practical concerns for people trying to change fields: we all have limited time and energy. And for the most part, people wildly overestimate the actual core skills required.

This guide is our take on the essential skills required to understand, write code and ideally contribute useful research to mechanistic interpretability. We hope that it’s useful and unintimidating. :)

Core Skills:

Maths:

Linear Algebra:

3Blue1Brown

or

Linear Algebra Done Right

Core goals - to deeply & intuitively understand these concepts:

Basis
Change of basis
That a vector space is a geometric object that doesn’t necessarily have a canonical basis
That a matrix is a linear map between two vector spaces (or from a vector space to itself)

Bonus things that it’s useful to understand:

What’s singular value decomposition? Why is it useful?
What are orthogonal/orthonormal matrices, and how is changing to an orthonormal basis importantly different from just any change of basis?
What are eigenvalues and eigenvectors, and what do these tell you about a linear map?

Probability basics

Basics of distributions: expected value, standard deviation, normal distributions
Log likelihood
Maximum value estimators
Random variables
Central limit theorem

Calculus basics

Gradients
The chain rule
The intuition for what backprop is - in particular, grokking the idea that backprop is just the chain rule on multivariate functions

Coding:

Python Basics

The “how to learn coding” market is pretty saturated - there’s a lot of good stuff out there! And not really a clear best one.
Zac Hatfield-Dodds recommends Al Sweigart's Automate the Boring Stuff and then Beyond the Basic Stuff (both readable for free on inventwithpython.com, or purchasable in books); he's also written some books of exercises. If you prefer a more traditional textbook, Think Python 2e is excellent and also available freely online.

NumPy Basics

Try to do the first ~third of these: ​​

https://github.com/rougier/numpy-100

. Bonus points for doing them in pytorch on tensors :)

ML:

Rough grounding in ML.&nbs

...
POWERplay: An open-source toolchain to study AI power-seeking
by Edouard Harris — 2022-10-24T20:03:57.560Z
Link: https://github.com/gladstoneai/POWERplay

We're open-sourcing POWERplay, a research toolchain you can use to study power-seeking behavior in reinforcement learning agents. POWERplay was developed by Gladstone AI for internal research.

POWERplay's main use is to estimate the instrumental value that a reinforcement learning agent can get from a state in an MDP. Its implementation is based on a definition of instrumental value (or "POWER") first proposed by Alex Turner et al. We've extended this definition to cover certain tractable multi-agent RL settings, and built an implementation behind a simple Python API.

We've used POWERplay previously to obtain some suggestive early results in single-agent and multi-agent power-seeking. But we think there may be more low-hanging fruit to be found in this area.

Beyond our own ideas about what to do next, we've also received some interesting conceptual questions in connection with this work. A major reason we're open-sourcing POWERplay is to lower the cost of converting these conceptual questions into real experiments with concrete outcomes, that can support or falsify our intuitions about instrumental convergence.

Ramp-up

We've designed POWERplay to make it as easy as possible for you to get started with it. Follow the installation and quickstart instructions to get moving quickly. Use the replication API to trivially reproduce any figure from any post in our instrumental convergence sequence. Design single-agent and multi-agent MDPs and policies, launch experiments on your local machine, and visualize results with clear figures and animations.

POWERplay comes with "batteries included", meaning all the code samples in the documentation should just work out-of-the-box if it's been installed successfully. It also comes with pre-run examples of experimental results, so you can understand what "normal" output is supposed to look like. While this does make the repo weigh in at about 500 MB, it's worth the benefits of letting you immediately start playing around with visualizations on preexisting data.

If we've done our job right, a smart and curious grad student (with a bit of Python experience) should be able to start reproducing our previous experiments within an hour, and to have some new — and hopefully interesting! — results within a week.

We're looking forward to seeing what people do with this. If you have any questions or comments about POWERplay, feel free to reach...
Consider trying Vivek Hebbar's alignment exercises
by Orpheus16 — 2022-10-24T19:46:40.847Z

Vivek Hebbar recently developed a list of alignment problems. I think more people should try them. I'm impressed with how well they (a) get people to focus on core problems, (b) encourage people to come up with their own ideas, (c) encourage people to notice and articulate confusions, and (d) accomplish a-c while also providing a fair amount of structure and guidance.

You can see the problems in this google doc or pasted below.

Note that Vivek is also a mentor for SERI-MATS. These exercises are also the questions that people need to answer to apply to work with him and Nate Soares. Applications are due today.

Problems:

These problems are basically research questions — we expect them to be difficult, and good responses will be valuable as research in their own right. MIRI will award prizes (likely on the order of $5000 for excellent submissions).

Instructions: We recommend that you focus on 1 or 2 of the hard questions and leave the other hard questions blank. It is mandatory to attempt either #1a-c or #2.

Note on word counts: These are guidelines for how long we think a typical good response will be, but feel free to write more. If you have lots of ideas, it’s great to write them all, and don’t bother trying to shorten them.

Hard / time-consuming questions (contest problems):

Problem 1

Pick an alignment proposal[footnote 1] and specific task for the AI.[footnote 2]

First explain, in as much concrete detail as possible, what the training process looks like. Then go through

Eliezer’s doom list

.

Pick 2 or 3 of those arguments which seem most important or interesting in the context of the proposal.
(~250 words per argument) For each of those arguments:

What do they concretely mean about the proposal?
Does the argument seem valid?

If so, spell out in as much detail as possible what will go wrong when the training process is carried out

What flaws and loopholes do you see in the doom argument? What kinds of setups make the argument invalid?

(~200 words) For one argument only, suggest a modification to the proposal, or a research direction, to get around the doom argument.

Do you think your proposed modification/direction will work?

If so, explain why.
If not, do you think your modification is in a space where continued brainstorming and iteration will lead to a correct solution,

...
Education not meant for mass-consumption
by Tolo — 2022-10-24T19:45:09.165Z

Say you wanted to raise a genius. How would you go about doing that?

Assuming that your starting with an above average intelligence child with a greater likelihood of naturally becoming a "genius", can appropriate environmental/educational interventions substantially increase the child's potential for becoming one?

A couple of anecdotes say yes:

(1) Scott's review of "The Man From The Future" shows that Von Neumann, besides from obvious his obvious genetic advantage, benefited from growing up in a very resourceful family environment with rich exposure to top intellectuals & domain experts from all kinds of fields and private tutors.

(2) Laszlo Polgar's "Raise A Genius" suggests that the key is "early specialization" in a subject chosen under the parent's discretion (as long as the child enjoys the subject) in conjunction with 1:1 tutoring and having access to peers that are "mentally appropriate partners."

The field of behavioral genetic paints a somewhat different picture:

Most if not all psychological traits including intelligence being highly heritable, as long as you don't severely mess up your parenting (eg abuse, malnutrition), the specifics of your parenting strategy won't influence your child's outcome that much. Social/educational interventions just ... aren't as effective as one may expect them to be a priori.

But these two views aren't incompatible; some possibilities include:

Tail-effects in education: Since interventions have to scale, they end up being mediocre to "what could be possible." Perhaps having a Really Good Education (like what Von Neumann had) has disproportionate effects on one's life outcomes/genius-ness in a way that standard social/educational intervention RCTs can't capture.

Tail-effects in genetics: Just like how people with very low-intelligence seem to be fundamentally bounded in terms of the tasks they're capable of (regardless of the education they receive), a similar effect might be going on with normal intelligence folks where there's rapidly diminishing return to improved education—i.e. education scales better with individuals genetically predisposed to be smarter.

Both (1) and (2), where geniusness is a "leaky pipeline" in which you need both tail-end education and tail-end genetics to become a "genius."

What is the actual underlying dynamic of "genius production"? My hunch is that things are closer to (3), considering how Vo...
Realizations in Regards to Masculinity
by  — 2022-10-24T19:42:28.603Z

This isn't a bait of some sort, but rather just a self-realization on how we view masculinity. I, for context, am a man, rather a boy who was not treated fairly in regards of being a man.

Growing up, I wasn't treated well by my father and I would often get the belt whenever I showed signs of femininity or any emotions that could lead me into crying. I had thoughts whether or not I was wrong for having those things, or if being sensitive was wrong. The truth is farther from that.

The idea of masculinity isn’t necessarily about being the biggest and the greatest to the point wherein natural selection won’t affect you due to how godly you are in any sense. Moreover, it is about the responsibilities, not how you present yourself. Yes, you could exhibit feminine traits, but that does not make you less of a man.

There is no reason for you to be dictated by traditionalist ideas archaic in the contemporary world. One must go against the flow of the seas, for getting dragged along with rough waves makes you less of a person with an identity. Going with the flow is a sentiment I undeniably loathe due to how pretentious it sounds. Your life is malleable by you. Just because many people are going through with one thing, that doesn’t mean you must follow suit as well. It makes you no different from the people of the previous years and those in the current world.

Similarly, masculinity is not about being the strongest physically but can also mean being strong mentally. Being strongly willed is something most people are not used to hearing. Yes, you could argue that the sentiment “ mind over matter “ overcomes this argument, but I digress. Mind over matter often refers to situations that are supposed to be succeeded in a physical form, but not over the concerns of the heart. We have been trained to go beyond the feeling of physical pain but not to go beyond and understand the problems of the heart. The heart is often disregarded by the concept of “masculinity” because the physical form is superficial and easily noticed. That, in itself, hurts the innate emotions and feelings people have.

We must not let the previous ideologies hold us back. We might not even think about it, but maybe in a few years time we will be the ones who are unreasonable to the current generation.

Masculinity is not about being all physical, it is also about knowing ...
The Futility of Religion
by  — 2022-10-24T19:42:28.520Z

As someone surrounded by religion, this mysterious and non-arbitrary belief system seemed convenient to believe during those times; but I was gravely mistaken. According to the Ontological, Teleological, and Cosmological Arguments, a god must exist.

These arguments provide the ideology of a supreme being as the stopping point for all unanswered questions. “How was our universe formed? The Big Bang Theory, but how did The Big Bang Theory happen? Well, there was a singularity that exploded, but who caused such an event?” This, in particular, revolves around the idea that someone is the supreme cause of everything.

The Causality Principle of Philosophy also comes to play here. If everything has a cause and an effect, then there must be a god before the god we currently have; but we have no such information regarding such ideology. Perhaps, there is a god, but where are they? I acknowledge these arguments, but it does not take hold of me. Why aren’t they here helping humanity surpass its devils if there is a god?

The devil is the personification of the grief of humanity. Moreover, a god is the personification of the true good we all inherently have. I argue that God was created as a concept for the people afraid of the inevitable death we face. This belief system rationalizes the idea of a supreme being who molded the world through words and general power. This idea in itself is faulty. How come such beings have these powers? Humans weren’t the smartest at one point, but we could adapt and learn things through evolution. How about god? Was he also, at one point, a being of minor powers that evolved to the point of being all-powerful and omnipotent?

Moreover, religion alludes to a pseudo-reality that we have. Nothing is truly justified we have no absolute reason to believe in it. God is an ideology and not an actual being. It is a being that humans have fathomed in order to make sense of the senseless. It is a being that humans have created to help those who are trying to find sense in their lives that are slowly dwindling. If He truly does exist, where was he when wars broke out? Where was he when His people were placed under scrutiny during WWII? Were the prayers not enough? Were the cries of help not enough?

Freedom in Religion

Religion...
The optimal timing of spending on AGI safety work; why we should probably be spending more now
by Tristan Cook — 2022-10-24T17:42:05.865Z

Summary

When should funders wanting to increase the probability of AGI going well spend their money? We have created a tool to calculate the optimal spending schedule and tentatively conclude that funders collectively should be spending at least 5% of their capital each year on AI risk interventions and in some cases up to 35%, depending on views about AGI timelines and other key variables.

This is likely higher than the current AI risk community spending rate which we estimate to be at most 3%[1]. In most cases, we find that the optimal spending schedule is between 5% and 15% better than the ‘default’ strategy of just spending the interest one accrues and generally between 5% to 35% better than a naive projection of the community’s current spending rate[2].

We strongly encourage users to put their own inputs into the tool to draw their own conclusions.

The key finding of a higher spending rate is supported by two distinct models we have created, one that splits spending of capital into research and influence, and a second model (the ‘alternate model’) that supposes we can spend our stock of things that grow on direct work. We focus on the former with the latter described in the appendix since its output is more obviously action-guiding[3].

The table below shows our best guess for the optimal spending schedule using the former model when varying the difficulty of achieving a good AGI outcome and AGI timelines. We keep other inputs, such as diminishing returns to spending and interest rate constant.[4]

Median AGI arrival
2030[5]	2040[6]	2050[7]
Easy[8][9]

Medium[10]

Hard[11]

How much better the optimal spending schedule is compared to the 2%+2% constant spending schedule (within-model lower bound

)

[12]

Median AGI

2030	2040	2050
Easy	37.6%	18.4%	11.8%
Medium	39.3%	14.9%	12.0%
Hard	12.3%	5.85%	1.55%

Some of the critical limitations of our model include: poorly modelling exogenous research, which is particularly important for those with longer timelines, and many parts of the model - such as diminishing returns - remaining constant over time.

Further, we find that robust spending strategies - those that work in a wide variety of worlds - also support a higher spending rate. We show the results of a Monte Carlo simulation in the appendix.[13]

Introduction

Humanity might be living at a hinge moment in history (

MacAskill, 2020

)....