trentmkelly/LessWrong-43k huggingface dataset:
Frames in context
by Richard_Ngo — 2023-07-03T00:38:52.078Z

In my previous post, I introduced meta-rationality and frames, and described some examples of frames and some of their properties. In this post I’ll outline some of the limitations of existing ways of thinking about cognition, and some of the dynamics that they can’t describe which meta-rationality can. This post (especially the second half) can be seen as a summary of the key ideas from the rest of the sequence; if you find it too dense, feel free to skip it and come back after reading the next five posts. To quickly list my main claims:

Unlike logical propositions, frames can’t be evaluated as discretely true or false.
Unlike Bayesian hypotheses, frames aren’t mutually exclusive, and can overlap with each other. This (along with point #1) means that we can’t define probability distributions of credences over frames.
Unlike in critical rationalism, we evaluate frames (partly) in terms of how true they are (based on their predictions) rather than just whether they’ve been falsified or not.
Unlike

Garrabrant traders

and

Rational Inductive Agents

, frames can output any combination of empirical content (e.g. predictions about the world) and normative content (e.g. evaluations of outcomes, or recommendations for how to act).
Unlike model-based policies, policies composed of frames can’t be decomposed into modules with distinct functions, because each frame plays multiple roles.
Unlike in multi-agent RL, frames don’t interact independently with their environment, but instead contribute towards choosing the actions of a single agent.

I’ll now explain these points in more detail. Epistemology typically focuses on propositions which can (at least in principle) be judged true or false.

Traditionally

, truth and knowledge are both taken as binary criteria: each proposition is either true or false, and we either know which it is or we don’t. Intuitively speaking, though, this doesn’t match very well with our everyday experience. There are many propositions which are kinda true, or which we kinda know: cats are (mostly) carnivorous (I think); Bob is tall(ish, if I’m looking at the right person); America is beautiful (in some ways, by my current tastes).

The most straightforward solution to the problem of uncertainty is to assign credences based on how much evidence we have for each proposition. This is the bayesian approach, which

solves

a ...
Meta-rationality and frames
by Richard_Ngo — 2023-07-03T00:33:20.355Z

How should we think about thinking? In this sequence I outline an epistemology called meta-rationality, which in my opinion improves on existing epistemologies in a number of important ways. This first post introduces some of the key concepts in meta-rationality (in particular the concept of frames), provides some of the intellectual context behind my conception of meta-rationality, and lists the main claims I’ll be making in the next half-dozen posts. Those posts, which focus on introducing meta-rationality, constitute the first half of the sequence, and will be posted over the next month; the second half of the sequence focuses on more complex aspects of meta-rationality, and will be posted over the following month or two.

The traditional approach to epistemology is to focus on our knowledge of propositions like “there is a red car in the garage” or “I’m feeling thirsty”, which can in principle be evaluated as true or false. At a high level, meta-rationality is about making epistemology less reductionist by focusing less on assigning credences to isolated propositions like these, and more on the larger-scale mental entities which we actually use when thinking about complex domains—entities including:

Ideologies like environmentalism, neoliberalism, communism, longtermism, etc
Scientific paradigms like darwinism, keynesianism, quantum physics, deep learning, etc
Life philosophies like stoicism, conformism, careerism, etc
Moral drives like egalitarianism, patriotism, compassion, etc
Epistemologies like empiricism, scientism, various schools of rationalism, etc
Persistent personality traits like openness to experience, ambition, narcissism, etc
Wide-ranging heuristics like “follow common-sense advice” or “move as fast as possible”

I’ll call these frames.[1] I’ll very roughly define a frame as a cluster of mental entities and processes (such as concepts, beliefs, heuristics, instincts, habits, skills, mental models, desires, values, etc) which tend to be active in conjunction with each other. Under this definition, the extent to which a group of traits qualifies as a frame is a matter of degree (although for convenience I'll mostly talk about frames in binary terms), and we might focus on frames at different levels of abstraction in different contexts. For example, the concept of the voltage across a component in an electrical circuit tends to be as...
VC Theory Overview
by Joar Skalse — 2023-07-02T22:45:59.974Z

In this post, I will give a brief overview of VC theory and computational learning theory. The reason for this is that I will later write a post about singular learning theory, in which I will refer back to this post. However, this post is self-contained, and may be interesting for anyone interested in theoretical machine learning.

What is Computational Learning Theory?

In short, computational learning theory (CLT) is what happens when you combine complexity theory with machine learning. Just like complexity theory is concerned with understanding what kinds of problems are hard to compute, or easy to compute, CLT is concerned with understanding what kinds of things are easy to learn, or hard to learn. This means that CLT seeks to identify classes of problems that you can learn with a small amount of data, and classes of problems that no algorithm can learn unless it gets a large amount of data. This also means that CLT produces generalisation bounds for different kinds of learning algorithms, and other such results.

Probably Approximately Correct Learning

In order to study what kinds of problems are hard to learn, and easy to learn, we first need a mathematical formalism that captures what it means to "learn a problem". The most popular model in CLT is given by the probably approximately correct (PAC) learning framework. This is a model of supervised learning for classification. It uses the following components:

We have an instance space X, which contains all the data that we might observe. For example, X may be a vector space, or it may be the set of all bit strings, etc. If we are doing image classification, then X may be the set of all RGB images of a certain size, and so on.

We have a set of concepts C, where every element of C is a function c:X→{0,1}. This is the set of functions that we might be trying to learn. For example, if X is a vector space, then C could be the set of all linear classifiers on X. Alternatively, if X is {0,1}n, then C could be the set of all Boolean functions over n variables with circuit complexity ≤m. And so on.

We assume that we have a (potentially unknown) distribution D over X, and that there is an unknown function c:X→{0,1} that we are trying to learn. To do this, our learning algorithm will observ...
Sources of evidence in Alignment
by Martín Soto — 2023-07-02T20:38:34.089Z

Summary: A short epistemological study to discover which sources of evidence can inform our predictions of action-relevant quantities in alignment.

This post follows Quantitative cruxes, although reading that first is mostly not required. Work done during my last two weeks of SERI MATS 3.1.

Sources of evidence

No researcher in any field ever makes explicit all of their sources of evidence. Let alone in a field as chaotic and uncertain as ML, in which hardly-earned experience and intuitions play a central role in

stirring the tensor pile

. And even less in a field with as many varied opinions and confusing questions as alignment. Nonetheless, even when researchers are just “grokking some deeper hard-to-transmit structure from familiar theory and evidence”, they need to get their bits of information from somewhere. Knowledge doesn’t come for free, they need

entanglement with observed parts of reality

.

Getting a better picture of where we are and could be looking, brain-storming or deepening existing sources, and understanding methodology limitations (as

Adam Shimi

’s efforts already pursue) can dissolve confusions, speed progress forward, help us calibrate and build common knowledge.

In reality, the following sources of evidence motivating any belief are way less separable than the below text might make it seem. Nonetheless, isolating them yields more conceptual clarity and is the first step for analysis.

1. Qualitative arguments

One obvious, theoretical source, and the most used in this community by far. The central shortcoming is that their abstractions are flexibly explanatory exactly because they abstract away detail, and thus provide more information about the existence of algorithms or dynamics, than about some relevant related quantities like how prevalent they actually are in a certain space, when do these dynamics actually start to appear and with how much steering power, etc.

Sometimes a tacit assumption might seem to be made: there are so many qualitative arguments for the appearance of these dynamics (and so few for the appearance of rectifying dynamics), that surely one of them will be present to a relevant degree, and early on enough! This seems like a sort of

presumption of independence

about yet unobserved structure: a priori, we have no reason to believe any one of these qualitative arguments have higher or lower quantitative impact,...
Quantitative cruxes in Alignment
by Martín Soto — 2023-07-02T20:38:18.534Z

Summary: Sometimes researchers talk past each other because their intuitions disagree on underlying quantitative variables. Getting better estimates of these variables is action-guiding. This is an epistemological study to bring them to the surface, as well as discover which sources of evidence can inform our predictions, and which methodologies researchers can and do use to deal with them.

This post covers some discussion of quantitative cruxes. The second one covers sources of evidence (a topic interesting on its own). The whole write-up can also be read in this document. I also did a short presentation of it. And it might be expanded with a benchmark of thought experiments in alignment.

Work done during my last two weeks of SERI MATS 3.1. Thanks to Vivek Hebbar, Filip Sondej, Lawrence Chan and Joe Benton for related discussions.

Many qualitative arguments urge worries about AI. Most of these argue shortly and easily for the possibility (rather than necessity) of AI catastrophe. Indeed, that’s how I usually present the case to my acquaintances: most people aren’t too confident about how the future will go, but if some yet unobserved variables turn out to not line up well enough, we’ll be screwed. A qualitative argument for non-negligible chance of catastrophe is enough to motivate action.

But mere possibilities aren’t enough to efficiently steer the future into safety: we need probabilities. And these are informed by quantitative estimates related to training dynamics, generalization capacity, biases of training data, efficiency of learned circuits, and other properties of parameter space. For example, a higher credence that training runs on task A with more than X parameters have a high chance of catastrophe would make us more willing to risk shooting for a pivotal act. Or, a lower credence on that would advise continuing to scale up current efforts. Marginal additional clarity on these predictions is action-relevant.

Indeed, the community has developed strong qualitative arguments for why dangerous dynamics might appear in big or capable enough systems (the limit of arbitrary intelligence). But it’s way harder to quantitatively assess when, and in what more concrete shape, they could actually arise, and this dimension is sometimes ignored, leading to communication failures. This is especially notable when researchers talk past each other because of different ...
Going Crazy and Getting Better Again
by Evenstar — 2023-07-02T18:55:25.790Z

[Epistemic status: Processing after a long period of work and recovery, which is not yet over]

Background

Four years ago, I was in university. I was a casual rationalist, having been introduced by HPMOR, been active on rationalist tumblr, and then joined a loosely allied fiction writing community known as Glowfic. Things seemed pretty okay. I had known for a while that I was not neurotypical, having been diagnosed with ADD in high school and also having been in a gifted program.

I was also familiar, casually, with weird things happening to me and people around me. I was active in the hypnosis scene recreationally, and was fascinated by how people transformed under the influence of trance. I am also trans, and thus had experienced a major identity shift from my "cis male" self to a transfeminine identity.

That feminine identity was based on a female roleplay character that I had played as obsessively for several years - not an uncommon occurrence in trans people. She talked to me in my head - not an uncommon occurrence for authors. Eventually I decided I'd be her because she was cooler than my (dysphoric, beaten-down) self. And so we went on in a new identity, changing from our deadname to our new name. There was an attempt at coming out which... did not go well. I'll spare the details. I ended up closeted and under great stress to finish my degree.

I reached out to writing as my source of comfort, as I always had. I poured my hurt out into a newly-turned girl being abused by her vampiric sire. I started getting feelings and emotions from her - a sense of attraction to gothic and old-style clothing, a distinct fondness for lists and checkboxes. It was useful to me to like lists and checkboxes - after all, I was failing my degree from ADD - so I encouraged the persona, using a half-understood concept of Internal Family Systems and my experience with altered states achieved under hypnosis to try and develop the identity.

It worked! And we remain plural to this day, with a dozen or so other identities we've developed over time. We use these identities as different lenses to view the world through, ways of helping us not get too attached to any particular identity. We took the prescription to be a fox and not a hedgehog very seriously to heart. As far as we can tell this is both stable and useful; it's not what this post is about. It's simply necessary background so ...
Shall We Throw A Huge Party Before AGI Bids Us Adieu?
by GeorgeMan — 2023-07-02T17:56:48.372Z

I don't think there is much more to this post then what it says in the title, but I'll add more details anyway.
Essentially, it's become increasingly obvious that despite the best efforts, the progress in AI alignment and other safety efforts has been, well... minimal. Yet the predictions are validated more and more and even previously ignorant public figures are starting to be aware of the issue.
Hence, to prevent future ai-written history books recording how "all they did on lesswrong was writing increasingly lengthy shitposts about how I will kill them and then I killed them", I suggest, with all seriousness, that we should at least once consider using this site for something actually useful. I suggest we use it to plan and throw a huge party each year while we wait for our destiny. Let's party while we can!
Besides a wild rave with Eliezer, Jeffrey, Elon and Joshua, we could extend the invitation to sceptics like Yann and show to the outside world that we are not just a bunch of crazy lunatics, but an actually sensible, open and welcoming community.
To foster the sense of community we could also dress ourselves in paperclip costumes, sending a strong signal that we are not scared, but instead fully reconciled with our fate.
And who knows? It might actually turn out to be a successful safety strategy - If we show the AGI that we are not just a bunch of useless boring atoms, but can also throw a great party and enjoy life, it might decide to keep us around for fun.
Please comment bellow if interested. This post is serious - I would actually quite enjoy a party with my fellow lesswrong comrades.
Why it's so hard to talk about Consciousness
by Rafael Harth — 2023-07-02T15:56:05.188Z

[Thanks to Charlie Steiner, Richard Kennaway, and Said Achmiz for helpful discussion. Extra special thanks to the Long-Term Future Fund for funding research related to this post.]

[Epistemic status: confident]

There's a common pattern in online debates about consciousness. It looks something like this:

One person will try to communicate a belief or idea to someone else, but they cannot get through no matter how hard they try. Here's a made-up example:

"It's obvious that consciousness exists."

-Yes, it sure looks like the brain is doing a lot of non-parallel processing that involves several spatially distributed brain areas at once, so-

"I'm not just talking about the computational process. I mean qualia obviously exist."

-Define qualia.

"You can't define qualia; it's a primitive. But you know what I mean."

-I don't. How could I if you can't define it?

"I mean that there clearly is some non-material experience stuff!"

-Non-material, as in defying the laws of physics? In that case, I do get it, and I super don't-

"It's perfectly compatible with the laws of physics."

-Then I don't know what you mean.

"I mean that there's clearly some experiential stuff accompanying the physical process."

-I don't know what that means.

"Do you have experience or not?"

-I have internal representations, and I can access them to some degree. It's up to you to tell me if that's experience or not.

"Okay, look. You can conceptually separate the information content from how it feels to have that content. Not physically separate them, perhaps, but conceptually. The what-it-feels-like part is qualia. So do you have that or not?"

-I don't know what that means, so I don't know. As I said, I have internal representations, but I don't think there's anything in addition to those representations, and I'm not sure what that would even mean.

and so on. The conversation can also get ugly, with boldface author accusing quotation author of being unscientific and/or quotation author accusing boldface author of being willfully obtuse.

On LessWrong, people are arguably pretty good at not talking past each other, but the pattern above still happens. So what's going on?

The Two Intuition Clusters

The basic model I'm proposing is that core intuitions about consciousness tend to cluster into two camps, with most miscommunication being the result of someone failing to communicate with the other camp. I will call the camp o...
How Smart Are Humans?
by Joar Skalse — 2023-07-02T15:46:55.309Z

Epistemic status: free speculation

How intelligent should we expect AI to be, compared to humans, and how quickly should we expect it to reach this level of intelligence? This is of course an important strategic question, which has a large impact on our AI threat models. At the moment, the most common answers to these questions are "much more intelligent", and "very quickly". In this post, I will describe an arguably realistic scenario in which this would not be the case.

First of all, why should we expect AI to end up much more intelligent than humans? A common argument goes roughly like this: humans are much, much smarter than animals. We can build computers, and send things to space, but they can't. This proves that the range of possible intelligence is very wide. Moreover, it would be prima facie very surprising if human intelligence is at the top of this range. Therefore, we should expect it to be possible for AI systems to get much smarter than humans. Moreover, there is no reason to think that AI progress would slow down around human intelligence in particular. Therefore, we should expect AI intelligence to quickly far outstrip our intelligence.

This argument relies very crucially on the assumption that humans are much smarter than animals. But is this actually true? I'm not entirely convinced. First of all, there have been "feral" humans that grew up surrounded by animals. As far as I know, these humans are not obviously much more intelligent than animals (in terms of their ability to solve problems). This already casts some doubt on the notion that humans are much, much smarter than animals.

It is important to remember that humans, unlike all other species, are able to use complex language. This is a huge confounding factor, when we try to compare the intelligence of humans and animals. It is obviously very powerful to be able to exchange complex ideas, and build up knowledge intergenerationally. This would probably be enough to give humans a very large advantage, even if our intelligence was otherwise exactly the same as that of other primates.

Therefore, consider the following hypothesis: humans have an innate ability to use complex, combinatorial language, but all other species lack this ability (in the way Noam Chomsky thinks). In addition to this, humans are also somewhat (but not hugely) more intelligent than other primates (eg, 1.5x as intelligent...
Through a panel, darkly: a case study in internet BS detection
by jchan — 2023-07-02T13:40:48.186Z

I set out to answer a simple question: How much energy does it take to make solar panels? A quick DuckDuckGo search led me to this website [archive] which says:

it would cost about 200kWh of energy to produce a 100-watt panel

But something about this website seemed "off". I suspected that it was an AI-generated SEO page designed specifically to appear in search results for this question. If so, then that gives me reason to doubt the truth of the answer. I checked their About us page [archive] and my suspicions grew.

The page lists four names: "Elliot Bailey, Brad Wilson, Daniel Morgan, Joe Ross", which sound like they were selected from a list of the most common first and last names. The portraits also look a lot like the output of thispersondoesnotexist.com. (What kind of strange jacket is "Elliot Bailey" wearing?) Furthermore, if I search Google Images for them, I find no photos of these people except the exact same ones from that website. One would think, if these people are leading solar energy magnates, that at least one picture of at least one of them from another angle would exist.

The page also gives the address "1043 Garland Ave, San Jose, CA". When I look this up on Google Maps, I find a tiny strip mall containing a tattoo parlor, a ballet studio, a food bank, and a martial arts school - but no sign of a solar panel manufacturer. And if I go to their supposed LinkedIn profiles, I get three pages that you need to sign in to view, and one "Profile Not Found".

So, I'm now pretty convinced that "Sol Voltaics" is not actually a real company and that none of these people actually exist. But what incentive could someone have had to set up such a deception? Are they trying to sell me something?

I go to their Products page [archive]. Interestingly, it seems like they don't actually sell any of their own products. Instead, the page consists of affiliate links to products sold by other companies: Bluetti, Anker, Rich Solar, etc. Are these also "fake" companies? I don't have the time or wherewithal to look into it. But at least it seems I can actually buy products from them if I wanted to, which is more than I can say for "Sol Voltaics".

My best guess is that "solvoltaics.com" is really just an elaborate marketing campaign for those companies. I imagine some savvy hustler approached them with a proposition of driving clicks to their websites in exchange for a cut...