trentmkelly/LessWrong-43k huggingface dataset:
Aura as a proprioceptive glitch
by pchvykov — 2023-06-12T19:30:46.448Z

So I recently went on a Vipassana retreat, which is a fun little experience that involves being silent for 10 days and meditating 11 hours each day. Needless to say, that sort of situation necessarily leads to all sorts of interesting experiences and ideas. Among them, one I found especially peculiar is that I distinctly started feeling some sort of sensations outside of my body. Aside from my rational mind, which kept saying that this should not be possible, the truth of my subjective experience of this was clear. Of course, I got really excited and started exploring these sensations more systematically. It seemed that when I was sitting quietly and peacefully, these sensations were like subtle uniform tingling in the space around my body, though with a non-uniform shape. In front of me, this “field” felt like it extended about 2-3 feet, while behind my back it was very thin, if not missing entirely.

If this seems oddly specific for such a strange subjective phenomenon, I need to give a bit more context of what Vipassana is all about. The premise of this meditation technique is that we spend so much of our lives caught up in our imaginary stories and projections into the future or memories of the past, that we start to confuse our reality and our imagination, forgetting what is the true actual real present “here” experience. And the only thing that is really here and true are our pure sensory inputs in this moment. When you are sitting in a largely dark, quite, clean hall, the only sensory input you actually have is that of touch. So the focus of the meditation is just to observe these tactile sensations for hours and hours on end. The key point here is not to imagine them, and to constantly check yourself that you’re really not imagining them, and then notice that you, in fact, are imagining them and to stop imagining anything! Turns out it’s nearly impossibly to observe sensations without imagining anything. If I want to feel the sensations on my left leg, I somehow need to imagine my left leg, where it's currently at, and what it's doing.

So this is roughly how I realized that I could feel sensations outside my body – as I was doing my body scans, my attention kept drifting off of my physical body parts and onto sensations I "imagined" in the space around me. The most obvious was that every time I got to my shoulders and scanned down my arms, I’d often sca...
Aligning Mathematical Notions of Infinity with Human Intuition
by London L. — 2023-06-12T19:19:07.931Z
Link: https://medium.com/@london-lowmanstone/comparing-infinities-e4a3d66c2b07

If you took half the stars, how many would remain? (Photo by

Greg Rakozy

on

Unsplash

)

Mathematicians have come up with a way of comparing infinities that doesn’t make sense to most people. I want to help.

So, I’ve written this to do the following:

Explain how you probably compare infinities
Explain how mathematicians compare infinities
Provide a mathematical framework for how people usually compare infinities (rather than the weird mathematician way)

The reason why I think this work is important is because computers tend to be based in mathematics. If there’s an artificial intelligence (AI) that is talking to humans about infinity, I want to make sure that you know how the AI is thinking. Also, if an AI (or a mathematician) happens to read this article, I want them to be able to understand how regular people think about infinity.

If you’re new to comparing infinities, I’d recommend reading this whole article. If you’re a mathematician or an AI, I’d recommend skipping to “The Bruce Framework” section.

How You Compare Infinities

Which is bigger, the amount of all positive whole numbers (1, 2, 3, 4, 5, …) or the amount of all positive even numbers (2, 4, 6, 8, …)?

If you’re not a mathematician, my guess is that you think the amount of all positive whole numbers (1, 2, 3, 4, 5, …) is bigger than just the evens (2, 4, 6, 8, …).

Why is this? Well, it’s pretty obvious. If you were to take away all of the evens from all of the positive whole numbers, you’d have some left over, namely, the positive odd numbers (1, 3, 5, 7, …). So if one set of things is literally contained within another set, and there are elements left over, it just makes sense to say the other set is bigger.

How Mathematicians Compare Infinities

Which is bigger, the amount of all positive whole numbers (1, 2, 3, 4, 5, …) or the amount of all positive even numbers (2, 4, 6, 8, …)?

If you’re a mathematician, you’ll say that they’re both the same size! What?!

Mathematicians really care about having clear rules about how they define things. So, they came up with rules for how to determine whether two sets of things have the same size.

Let’s call the first set A and the second set B. Each thing inside of a set we call an “element” of that set. (So for a set of numbers, each number in the set is an element of that set.)

Mathematicians say that A is smaller than or equal to B when you can take every element in A and...
ARC is hiring theoretical researchers
by paulfchristiano — 2023-06-12T18:50:08.232Z

The Alignment Research Center’s Theory team is starting a new hiring round for researchers with a theoretical background. Please apply here.

Update January 2024: we have paused hiring and expect to reopen in the second half of 2024. We are open to expressions of interest but do not plan to process them until that time.

What is ARC’s Theory team?

The Alignment Research Center (ARC) is a non-profit whose mission is to align future machine learning systems with human interests. The high-level agenda of the Theory team (not to be confused with the Evals team) is described by the report on Eliciting Latent Knowledge (ELK): roughly speaking, we’re trying to design ML training objectives that incentivize systems to honestly report their internal beliefs.

For the last year or so, we’ve mostly been focused on an approach to ELK based on formalizing a kind of heuristic reasoning that could be used to analyze neural network behavior, as laid out in our paper on Formalizing the presumption of independence. Our research has reached a stage where we’re coming up against concrete problems in mathematics and theoretical computer science, and so we’re particularly excited about hiring researchers with relevant background, regardless of whether they have worked on AI alignment before. See below for further discussion of ARC’s current theoretical research directions.

Who is ARC looking to hire?

Compared to our last hiring round, we have more of a need for people with a strong theoretical background (in math, physics or computer science, for example), but we remain open to anyone who is excited about getting involved in AI alignment, even if they do not have an existing research record.

Ultimately, we are excited to hire people who could contribute to our research agenda. The best way to figure out whether you might be able to contribute is to take a look at some of our recent research problems and directions:

Some of our research problems are purely mathematical, such as these matrix completion problems – although note that these are unusually difficult, self-contained and well-posed (making them more appropriate for prizes).
Some of our other research is more informal, as described in some of our recent blog posts such as Finding gliders in the game of life.
A lot of our research occupies a middle ground between fully-formalized problems and more informal questions, such as fixing the

...
Introduction to Towards Causal Foundations of Safe AGI
by tom4everitt — 2023-06-12T17:55:24.406Z

By Tom Everitt, Lewis Hammond, Rhys Ward, Ryan Carey, James Fox, Sebastian Benthall, Matt MacDermott and Shreshth Malik representing the

Causal Incentives Working Group

. Thanks also to Toby Shevlane, MH Tessler, Aliya Ahmad, Zac Kenton, Maria Loks-Thompson, and Alexis Bellot.

Over the next few years, society, organisations, and individuals will face a number of fundamental questions stemming from the rise of advanced AI systems:

How to make sure that advanced AI systems do what we want them to (the alignment problem)?
What makes a system safe enough to develop and deploy, and what constitutes sufficient evidence of that?
How do we preserve our autonomy and control as decision making is increasingly delegated to digital assistants?

A causal perspective on agency provides conceptual tools for navigating the above questions, as we’ll explain in this sequence of blog posts. An effort will be made to minimise and explain jargon, to make the sequence accessible to researchers from a range of backgrounds.

Agency

First, with agent we mean a goal-directed system that acts as if it it is trying to steer the world in some particular direction(s). Examples include animals, humans, and organisations (more on agents in a subsequent post). Understanding agents is key to the above questions. Artificial agents are widely considered the

primary existential threat

from AGI-level technology, whether they

emerge

spontaneously

or through

deliberate design

. Despite the

myriad risks to our existence

, highly capable agents pose a distinct danger, because many goals can be achieved

more effectively

by accumulating influence over the world. Whereas an asteroid moving towards earth isn’t intending to harm humans and won’t resist redirection, misaligned agents might be distinctly adversarial and active threats.

Second, the preservation of human agency is critical in the approaching technological transition, for both individuals and collectives. Concerns have already

been

raised

that manipulative social media algorithms and content recommenders undermine users’ ability to focus on their long-term goals. More powerful assistants could exacerbate this. And as more decision-making is delegated to AI systems, the ability of society to

set its own trajectory

comes into question

Human agency can also be&nbs...
Manifold Predicted the AI Extinction Statement and CAIS Wanted it Deleted
by David Chee — 2023-06-12T15:54:51.699Z

Crosspost from https://news.manifold.markets/p/manifold-predicted-the-ai-extinction

History may look back on CAIS’s AI risk statement as a pivotal moment IF we survive AI.

But, what if I told you that AI researchers “leaked” info about the statement on Manifold Markets over a week before it was published? This justifiably led to CAIS requesting the market to be deleted, concerned that it could damage the impact of the statement.

To provide some background, the statement was signed by the likes of Sam Altman, Demis Hassabis, Dario Amodei and other prominent AI figures. Short and with gravitas it read:

“Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”

Imagine if the CEOs of major oil companies banded together in the 1900s and told their respective governments, “We need to be careful about building infrastructure relying heavily on fossil fuels. It could be catastrophic for the environment long term.”

This may seem like an extreme comparison to make, but the reality is leading AI companies consider the risk significant enough to compromise on profit and progression. Well, that’s what their words say, time will tell how it is reflected in corporate decision-making.

I had an opportunity to talk to some of the insider traders and Dan Hendrycks from CAIS, and hope you enjoy a breakdown of one of our craziest markets and what we’ve learnt from it.

Background on Manifold Markets

Prediction markets allow people who lack expertise about certain topics to form precise models of what the future could look like thanks to the live-updating probabilities that are generated by traders. Trades can buy YES or NO shares, which fluctuate in price depending on the current probability (similar to sports betting odds).

Manifold Markets uses play money, which leads some to express skepticism of its efficacy. However, data suggest Manifold Markets are incredibly accurate!

Here is a graph from an analysis of Manifold Market’s calibration carried out by Vincent Luczkow. This places markets into buckets based on their probability at the middle of their lifespan. It then looks at the percentage of markets in each bucket that resolve to yes. To be well-calibrated, you want markets in the 10% bucket at their mid-point to resolve yes 10% of the time and no 90% of the time (this ideal is represented by t...
Explicitness
by TsviBT — 2023-06-12T15:05:04.962Z

[Metadata: crossposted from https://tsvibt.blogspot.com/2023/03/explicitness.html. First completed March 3, 2023.]

Explicitness is out-foldedness. An element of a mind is explicit when it is available to relate to other elements when suitable.

Thanks to Sam Eisenstat for related conversations.

Note: The ideas of explicitness and inexplicitness require more explication.

Explicitness and inexplicitness

Elements can be more or less explicit, less or more inexplicit.

(This statement wants to be unpacked.)

In general, inexplicitness is the lack of explicitness, and explicitness is when elements that have good reason to be related, are able to be related. That is, when structure is explicit, it can be brought into relation with other structure when suitable.

Structure is explicit when it's out-folded: when it already makes itself available (visible, applicable, informative, copyable, tunable, directable, modifiable, predictable, combinable, interoperable), so that nothing is hidden or blocked.

An explicit element is an element high in explicitness, i.e. it can be brought into relation with other elements when suitable.

Explicitizing

Elements can become more explicit.

By default, structure is fully inexplicit for a mind. That is, it's fully ectosystemic for the mind: it's not available for elements of the mind to relate to.

Structure can be brought into explicitness.

For example, these processes make structure more explicit: reflection, analysis, description, expression, joint-carving, separating, factoring, refactoring, modularization, indexing, interfacing, connecting, disentangling.

The early stages of explicitizing involve incomplete or deficient participation——like a blind man touching an elephant's tail, or entering the outer regions of a nexus of reference. E.g., the relationship that the Ancient Greek mathematicians had to Cartesian algebraic geometry.

A diagram:

Examples

An example of explicitizing also furnishes examples of inexplicitness (before the explicitizing) and explicitness (after the explicitizing), and likewise an example of inexplicitizing also furnishes examples of explicitness and inexplicitness.

Classes of examples of explicitizing

Internal sharing of elements. See here for examples of inexplicitness.

Making an analogy. By grasping a partial isomorphism, the partially isomorphic aspects of the analogands can be transferred back and

...
If you are too stressed, walk away from the front lines
by Neil  — 2023-06-12T14:26:44.030Z

tl;dr: If your reason for cramming AI knowledge into your brain is stress, then don't do it. You can still be useful, but walk away from the front lines where people are directly responsible.

Disclaimer: 1) This is an obvious problem that has already been noticed and addressed by many LessWrong users 2) This is not an original solution but rather a specific framing of the problem and some food for thought 3) I could be gravely mistaken, and your best bet might be putting your all into research after all. However, you might just want to emerge from lurking and actually do something if that is the case. 4) The rather romantic and optimistic tone employed here is fait exprès and is not meant as an accurate description of the reality in which we are in right now.

The heroes who will save the world will be alignment researchers: when, dot by dot, the stars will grow dark and a dark dawn will rise, we will all have to buy them a beer. If you are not among this small band of heroes who will guarantee us the best future possible, you may feel an urgent need to promptly join them: I beg of you only to consider whether this is a good idea.

If you think that you should become an alignment researcher in a matter of months,[1] I will not try to stop you. But it's probably worth a few days worth of cranial compute to establish whether you are exploiting yourself in the best way you could.

I'll set the parameters of the problem. "Becoming an alignment researcher" is a spectrum: the more you learn about alignment, the more capable you are at navigating the front lines of the alignment project. Certainly, understanding the tenets of alignment is a laudable goal; but at what point will you be faced with diminishing returns? If you are not planning on single-handedly solving the alignment problem, are there not better uses of your time?[2]

There are many instrumental goals that serve the terminal goal of completing the alignment project, and they might be more worthy of your time:

The logistics of the alignment project: educating and hiring all the AI experts out there who could be working directly on the front lines. How many Von Neumanns are locked in Subsaharian Africa that could be shipped to the front lines of alignment in a few years? How many engineers at OpenAI could be hired to work on the alignment problem instead of on the next va

...
UK PM: $125M for AI safety
by Hauke Hillebrandt — 2023-06-12T12:33:21.372Z
Link: https://twitter.com/Tom_Westgarth15/status/1668169997443530753

The UK PM recently announced $125M for a foundation model task force. While the announcement stressed AI safety, it also stressed capabilities. But this morning the PM said 'It'll be about safety' and that the UK is spending more than other countries on this and one small media outlet had already coined this the 'safer AI taskforce'.

Ian Hogarth is leading the task force who's on record saying that AGI could lead to “obsolescence or destruction of the human race” if there’s no regulation on the technology’s progress.

Matt Clifford is also advising the task force - on record having said the same thing and knows a lot about AI safety. He had Jess Whittlestone & Jack Clark on his podcast.

If mainstream AI safety is useful and doesn't increase capabilities, then the taskforce and the $125M seem valuable.

We should use this window of opportunity to solidify this by quoting the PM and getting '$125M for AI safety research' and 'safer AI taskforce' locked in, by writing and promoting op-eds that commend spending on AI safety and urge other countries to follow (cf. the NSF has announced a $20M for empirical AI safety research). OpenAI, Anthropic, A16z and Palantir are all joining DeepMind in setting up offices in London.

This might create an AI safety race to the top as a solution to the tragedy of the commons (cf. the US has criticized Germany for not spending 2% of GDP on defence; Germany’s shot back saying

the US should first meet the 0.7% of GNI on aid target

).
Could induced and stabilized hypomania be a desirable mental state?
by MvB — 2023-06-12T12:13:29.837Z

This may be a bad idea that acted upon may do more harm than good, but I propose that we should find a way to induce a controlled and stable state of hypomania in people who are willing to enhance their functional capabilities.

Hypomania is a state that frequently occurs in people with bipolar disorder, which in various variants affects up to 3% of the population. In contrast to full-blown mania, characterized by delusions, mind racing and intense euphoria and/or dysphoria, hypomania does not incapacitate a person on the individual or social level.
Instead, creative thinking and cognition as well as overall energy level are considerably enhanced, while the need for sleep and rest is reduced significantly. Mood is well above the baseline of the given individual. Often, hypomania is not even recognized as such, as the person sometimes appears perfectly healthy to the outsider who isn‘t familiar with the „normal“ state of the affected person. The major drawbacks are a tendency towards absent-mindedness and a certain loss of social inhibitions. Increased risk-taking can also pose a problem.

I have experienced hypomania myself some years ago.
During that time, I experienced a motivation doing everyday chores, learning and sports like never before. I had never been as productive, creative and enthusiastic as during that few months.
I had to pay a price for this: Severe psychosis followed by long depression and rehabilitation.

Now what if we could find a way to experience the benefits of hypomania without the negative consequences?
Surely, this could improve the intellectual and artistic output of many people struggling, for example, with motivational issues or procrastination.

It is still a pressing and unsolved problem to find a lasting remedy against depression. I have not yet met a psychiatrist (or patient) who is satisfied with the state of the art. But from the point of view of utility, even the mentally healthy usually experience subjective shortcomings in their capabilities on an everyday basis, and improving upon what are now supposedly healthy states seems to be a worthwhile undertaking.
In my opinion, it would useful to examine the mechanisms of hypomania for exploitation of this mental state.

Lacking a background in neurology and psychopharmacology, I cannot outline a research program leading into this direction.
What I know is, from my subjective experience...
Non-loss of control AGI-related catastrophes are out of control too
by Yi-Yang — 2023-06-12T12:01:26.682Z

For the

Open Philanthropy AI Worldview Contest

Executive Summary

Conditional on AGI being developed by 2070, we estimate the probability that humanity will suffer an existential catastrophe due to loss of control (LoC) over an AGI system at ~6%, using a quantitative scenario model that attempts to systematically assess the relative probabilities of all AGI-driven catastrophes in a collectively exhaustive manner (

details here

;

spreadsheet here

).
We also estimate the overall probability that humanity will suffer an existential catastrophe due to an AGI-related catastrophe at ~12%. The remaining 12% - 6% = 6% of the probability mass derives from

non

-LoC AGI catastrophes (e.g. AI-exacerbated x-risks).
Given our estimate that p(existential catastrophe | non-LoC) ≈ p(existential catastrophe | LoC), we argue that Open Philanthropy (OP) should consider broadening their priorities to include

non

-LoC scenarios as well (

more

), instead of focusing mainly on what we think is OP’s core threat model of misaligned power-seeking AI. (

more

)

In particular, we suggest that OP look into approaches to safeguard against non-LoC AI-exacerbated development and deployment of full-scale nuclear war and engineered pandemics. (

more

)
We think the ‘AI-exacerbated’ tie-in requires a cross-domain approach that makes OP best positioned to work on this issue, given OP’s

existing grantmaking

on both AI and biosecurity GCRs.

In tabular format, our estimates are as follows:

Failure mode / recoverability of catastrophe

p(LoC → AGI catastrophe)

p(non-LoC → AGI catastrophe)

Total p(AGI catastrophe)

Unrecoverable
(i.e. existential)

5.7%

6.0%

11.7%

Recoverable

3.3%

10.9%

14.3%

Total p(AGI-related catastrophe)

9.1%

16.9%

26.0%

Table 1

Introduction

We broadly agree with the following picture from Metaculus’

Ragnarok Question Series

, even if we disagree with exact estimates:

While in most possible futures (proxying by snapshot at year 2100) humanity will not experience global catastrophes (i.e. leading to the death of >10% of the world population), enough possible futures do experience such global catastrophes as to warrant societal-scale concern and immediate collective action.
In possible futures with global catastrophes, the most likely failure mode will be related to AI, capturing nearly half of the probability mass; nuclear war and global catastrophic (GC) biorisk account for mo

...