trentmkelly/LessWrong-43k huggingface dataset:
Public selves
by KatjaGrace — 2021-01-18T21:10:20.159Z

A question that I return to in life strategy is whether to lean heavily on ‘spending one’s weirdness points wisely’—otherwise put, cowering lonely behind a cardboard cutout of the most forgettable person while proffering optimized propaganda through carefully selected slots—or whether to offer the world a fuller view of oneself.

A few arguments as I see them:

Hiding allows you to be strategic, showing anything that is good to show, hiding anything that is not. Surely that is better then than any alternative, that must involve showing things that are bad to show, or not showing things that are good to show?

Not necessarily! People can tell which strategy you are using, and usually the things that are ‘bad to show’ are bad for you to show, but other people would be perfectly interested to see them. So it is less cooperative, and people may respond to that, which may on a longer term view be bad for you.

Also, which strategy you are enacting overall, or what you are doing in the past or future, can change whether something is good or bad to share. For instance, maybe you have personal problems that it would be both nice to have in the open, and helpful for others to know that you also face. If you are usually open about things, mentioning these might be no big deal, and so worth it on net. Whereas if you tend to be private, then suddenly announcing a personal problem will seem like a bigger deal, so the costs might outweigh the benefits.

There is something good about actually knowing other people - being part of a global intellectual society of real people, not of robotic fictions created by people. Being open contributes to this world being actual.

There are intermediate options too, of course. Are there good principled ones?

What considerations am I missing?
Short summary of mAIry's room
by Stuart_Armstrong — 2021-01-18T18:11:36.035Z

This is a shorter summary of the post "mAIry's room: AI reasoning to solve philosophical problems". Its aim is to explain the key concepts of that post without going into full technical details.

Can you feel my pain?

People can be more or less empathetic, but we have trouble really feeling the pain of another person. If the person is close to us, and we get more details of what they're going through, then we can start to feel a sympathetic pain; but almost nobody reacts viscerally to a sentence like "and then someone in the country started to feel a lot of pain."

So, though we can feel our own pain, and though we can to some extent feel the pain of others if given enough detail, the information that someone else is in pain is very different from the experience of pain.

Now, this might be where you could start taking about "qualia"; but the key insight is that this behaviour is very akin to reinforcement learning agents.

The "pain" of RL agents

You can see "low reward" as a crude analogue of "pain" for a reinforcement learning agent.

It has some of the features mentioned above. An RL agent's own reward channel is much more important to it than the reward channel of another agent. If it gets some information about low reward for other agents, this will not be of any interest to it unless this affects its own reward.

We can make the analogy closer if we imagine more limited, boundedly rational RL agents[1]; maybe powered by neural nets under the hood. These neural nets are function approximators; but we can also think of them as heuristic generators. They generate approximate answers to specific questions (eg: "what action best increases my reward?") and update themselves based on inputs and outputs.

Now let's imagine this agent has been in a rather unchanging environment for some time, getting medium rewards, with the neural net refining itself but not changing much, as its predictions are very decent. In such an environment, the agent has the luxury of making long-term plans.

And then, suddenly, the rewards drop to something very low. The agent's neural net starts to update massively, since its predictions are now quite wrong. Weights and connections that haven't changed in a long time are suddenly shifting. Meanwhile, the agent's behaviour changes radically, as it tries to figure out a behaviour, any behaviour, that will bring its rewards back up. The long-term is...
Birds, Brains, Planes, and AI: Against Appeals to the Complexity/Mysteriousness/Efficiency of the Brain
by Daniel Kokotajlo — 2021-01-18T12:08:13.418Z

[Epistemic status: Strong opinions lightly held, this time with a cool graph.]

I argue that an entire class of common arguments against short timelines is bogus, and provide weak evidence that anchoring to the human-brain-human-lifetime milestone is reasonable.

In a sentence, my argument is that the complexity and mysteriousness and efficiency of the human brain (compared to artificial neural nets) is almost zero evidence that building

TAI

will be difficult, because evolution typically makes things complex and mysterious and efficient, even when there are simple, easily understood, inefficient designs that work almost as well (or even better!) for human purposes.

In slogan form: If all we had to do to get TAI was make a simple neural net 10x the size of my brain, my brain would still look the way it does.

The case of birds & planes illustrates this point nicely. Moreover, it is also a precedent for several other short-timelines talking points, such as the human-brain-human-lifetime (HBHL) anchor.

Plan:

Illustrative Analogy
Exciting Graph
Analysis

Extra brute force can make the problem a lot easier
Evolution produces complex mysterious efficient designs by default, even when simple inefficient designs work just fine for human purposes.
What’s bogus and what’s not
Example: Data-efficiency

Conclusion
Appendix

1909 French military plane, the Antionette VII.

By Deep silence (Mikaël Restoux) - Own work (Bourget museum, in France), CC BY 2.5, https://commons.wikimedia.org/w/index.php?curid=1615429

Illustrative Analogy

AI timelines, from our current perspective	Flying machine timelines, from the perspective of the late 1800’s:
Shorty: Human brains are giant neural nets. This is reason to think we can make human-level AGI (or at least AI with

strategically relevant

skills, like

politics and science

) by making giant neural nets.	Shorty: Birds are winged creatures that paddle through the air. This is reason to think we can make winged machines that paddle through the air.

Longs: Whoa whoa, there are loads of important differences between brains and artificial neural nets: [what follows is a direct quote from the objection a friend raised when reading an early draft of this post!]
- During training, deep neural nets use some variant of backpropagation. My understanding is that the brain does something else, closer to Hebbian learning. (Though I vaguely rememb

...
Literature Review on Goal-Directedness
by adamShimi — 2021-01-18T11:15:36.710Z

Introduction: Questioning Goals

Goals play a central role in almost all thinking in the AI existential risk research. Common scenarios assume misaligned goals, be it from a single AGI (paperclip maximizer) or multiple advanced AI optimizing things we don’t want (Paul Christiano’s

What Failure Looks Like

). Approaches around this issue ask for learning the right goals (

value/preference learning

), allowing the correction of a goal on the fly (

corrigibility

), or even removing incentives for forming goals (

CAIS

).

But what are goals, and what does it mean to pursue one?

As far as we know, Rohin Shah’s

series

of

four

posts

were the first public and widely-read work questioning goals and their inevitability in AI Alignment. These posts investigate the hypothesis that goals are necessary, and outline possible alternatives. Shah calls the property of following a goal “goal-directedness”; but he doesn’t define it:

I think of this as a concern about long-term goal-directed behavior. Unfortunately, it’s not clear how to categorize behavior as goal-directed vs. not. Intuitively, any agent that searches over actions and chooses the one that best achieves some measure of “goodness” is goal-directed (though there are exceptions, such as the agent that selects actions that begin with the letter “A”). (ETA: I also think that agents that show goal-directed behavior because they are looking at some other agent are not goal-directed themselves -- see this

comment

.) However, this is not a necessary condition: many humans are goal-directed, but there is no goal baked into the brain that they are using to choose actions.

Later on, he explains that his “definition” of goal-directedness relies more on intuitions:

Not all behavior can be thought of as

goal-directed

(primarily because I allowed the category to be defined by fuzzy intuitions rather than something more formal)

Clearly, fuzzy intuitions are not enough to decide whether or not to focus on less goal-directed alternatives, if only because we can’t define the latter. We thus need a more advanced understanding of the concept. What’s more, deconfusing this concept would probably move forward many existing research approaches, as defended in the

first post

of this sequence.

That being said, intuitions shouldn’t be thrown out of the window either. As

suggested

by Vanessa Kosoy, they provide the analogue of experimental data fo...
(USA) N95 masks are available on Amazon
by Wei Dai — 2021-01-18T10:37:40.296Z

My family has been relying on a supply of off brand "procedure masks" that I got at the start of the pandemic, and many of my neighbors are still wearing cloth masks. After reading "A good supply of KN95 masks is available from China, with many supermarkets and pharmacies now selling them for a couple of dollars each." in Why Aren’t We Wearing Better Masks? I decided to try to buy some of these, but then found something apparently even better: N95 masks of a new design made by a reputable American company, currently available on (and sold directly by) Amazon at a very reasonable price (even cheaper than most KN95 masks). Here's a Rolling Stone article I found that goes into the design/features of this mask.

(I also bought a full face mask/respirator along with a pair of 3M P100 filters, in case I have to go into some really high risk environments.)

Edit: Added a "USA" tag to the post title. Also, in case it's not clear, I emphasized brand in this post because "Tragically, America is swamped with fraudulent medical-grade masks, some of which are only 1 percent effective." according to Why Aren’t We Wearing Better Masks?.
Actually possible: thoughts on Utopia
by Joe Carlsmith — 2021-01-18T08:27:39.428Z

(Cross-posted from Hands and Cities)

Life in the future could be profoundly good. Many people accept something like this in principle. But I think it often goes underestimated in practice, especially once we imagine society’s most glaring problems fixed, and ask where we might go from there. The difference in quality of life between a fixed-up version of our current world and the best possible future is, I think, less like the difference between a mediocre job and a beach vacation, and more like the difference between being asleep and being awake; between blindness and seeing; a droplet and an ocean; a cave and an open sky.

Following Bostrom (2008), let’s call a profoundly good future “Utopia.” The most important fact about Utopia, I think, is that we (that is, humanity and our descendants) could, with enough patience and wisdom, actually build it. This is a real thing we could actually do; a way the future could really be. It’s a thing the way laptops were a thing before they were invented; the way you could, if you really wanted, find a way to get some ice cream by the end of next week. It’s not some empty fantasy or symbol. It’s a physically-available path that, modulo certain exotic scenarios, is actually open to us, if we don’t mess up.

This post describes some different ways of thinking about Utopia, and what I think matters most about the differences. It also discusses some common objections to different types of “Utopian” thought and practice. My focus is on the quality of life in Utopia; see Ord (2020), Chapter 8, for more on just how much of that life there could be.

I. Concrete utopias

We can think of visions of Utopia in two broad categories: concrete, and sublime.

Concrete Utopias are imagined in specific and often human-scale terms. Most literary depictions of Utopia fall into this category, as does the vision offered by a friend of mine who hopes, if he makes it to the Utopian future, to “sit on the giant pile of pizza and play video games all day” (I think this is at least partly tongue in cheek, but it’s a bit hard to tell).

One issue people raise about these Utopias — and which the pizza example, for me, illustrates — is that they don’t always sound particularly appealing, especially not to everyone, or after scrutiny. Orwell, for example, complains that “all efforts to describe permanent happiness, on the other hand, have been failures.” Se...
Are the consequences of groups usually highly contingent on their details?
by KatjaGrace — 2021-01-18T01:30:16.236Z

How much of the impact of an organization is covered by it being ‘a group of size M working on X’, relative to the specifics of how and what they do in working on X? What if we also include a single scale of how functional they are?

For instance, does it mostly matter that my research group, AI Impacts, is a certain size force for AI risk related thinking (with the size determined by the number and competence of people and the functionality of the organization, say), or does it matter whether we write journal articles or news stories or blog posts or research pages, or whether we choose our projects collectively vs. individually, or whether we get most of our feedback internally vs. externally? Maybe most of these things can mostly be translated into ‘functionality’. But the ‘type of thing we are producing’ one doesn’t seem to as easily.

How much does what exactly you are producing matter? It could matter almost entirely or not at all, to my knowledge. For instance, I have some intuition that ‘there are about three small orgs in that space’ is a reasonable description of how much effort is going into a goal, but I also have intuitions that, say, minor differences in the responsiveness or navigability or style of a website can make the difference between it seeming great or annoying or crankish, and being read or not, or liked by different people. Which seems like it should just have a fairly different effect. These seem vaguely in conflict.

I originally meant for AI Impacts to be an unusual kind of entity, and the form of the output (website of ongoingly updated research pages on modular questions, hierarchically supporting one another) was a key part of the experiment. Which doesn’t imply a strong view on the importance of format—experimentation might make sense if format is so important it is worth searching alternative ones, but it might also make sense if it is so unimportant that it won’t mess anything up.

But there are costs to doing unusual things (e.g. people are confused about what you are doing, other entities aren’t organized to interface with you), so if format is totally unimportant, maybe we should move to more normal things. (Or the same goes if format is important, and this one is clearly worse.)

This question is also relevant in how you pick organizations to donate to, so maybe people who regularly do that know the answers. Does it mostly matter ...
The Ecological Perspective
by Chris_Leong — 2021-01-18T01:19:34.745Z

One of the shifts I've made in recent years is adopting a more ecological perspective of society. Even if you find a particular group of people frustrating, they may very well be serving an important function in the same way that an ecosystem requires many different plants and animals in order to thrive (this may not be a fully accurate picture of how ecosystems work, but this doesn't matter as I'm only using it by way of illustration).

So if you're conservative, you may be frustrated with progressives and their tendency to be uncompromising in their desire for reform, but at the same time you might be able to appreciate that it's this very attribute that allowed them to push through the changes in the past which you appreciate now. On the other hand, if you're a progressive, then you might be frustrated with conservatives and their unwillingness to let go of past traditions and practices, but you might be able to appreciate the tendency of your group to get carried away if left to its own devices and so you might be able to appreciate how this quality reduces the adverse impact from everyone jumping on the latest intellectual fashion. Or you might be annoyed with libertarians and how everything is theft, but at the same time, you might be able to appreciate the role they play in pointing out when regulation has become stifling.

A few key ideas emerge from this perspective:

• Firstly, that someone can be playing an important role in society, not just in spite of their flaws, but sometimes precisely because of their flaws

• Secondly, that it suggests that unless we have a society of superhumans capable of playing every role well, then the best society wouldn't consist of everyone believing the same thing or acting the same way, but rather people playing a multiplicity of roles

• Thirdly, from this perspective, when societal dysfunction is not so much a result of dysfunctional individuals as a dysfunctional and out of balance system.

(This argument applies to groups as well. Maybe I'll develop it more in the future).
DALL-E does symbol grounding
by p.b. — 2021-01-17T21:20:45.567Z

I don’t want to write much more than the title already expresses. But considering the constantly shifting goalposts when it comes to progress towards AGI, I think this bears writing down.

In the abstract of the seminal paper [1] on the topic, the symbol grounding problem is defined as follows:

„How can the semantic interpretation of a formal symbol system be made intrinsic to the system, rather than just parasitic on the meanings in our heads? How can the meanings of the meaningless symbol tokens, manipulated solely on the basis of their (arbitrary) shapes, be grounded in anything but other meaningless symbols?“

Good questions. And also valid objections against assigning intelligence or understanding to GPT-3. And a solution is sketched:

„Symbolic representations must be grounded bottom-up in nonsymbolic representations of two kinds: (1) iconic representations, which are analogs of the proximal sensory projections of distal objects and events, and (2) categorical representations, which are learned and innate feature-detectors that pick out the invariant features of object and event categories from their sensory projections. Elementary symbols are the names of these object and event categories, assigned on the basis of their (nonsymbolic) categorical representations. Higher-order (3) symbolic representations, grounded in these elementary symbols, consist of symbol strings describing category membership relations (e.g., An X is a Y that is Z).“

(1) The „iconic representations“ are pictures in the case of DALL-E, certainly „analogs of the proximal sensory projections of distal objects and events“.

(2) „Categorical representations are the text token embeddings that constitute DALL-E's input layer and the „innate feature-detectors“ are provided by the VQ-VAE that decomposes images into image tokens.

(3) According to Harnad higher order symbolic representation can now be grounded in these grounded symbols. I.e. a DALL-E scaled to the text context size of GPT-3 trained on GPT-3 training data in addition to text-image pairs.

[1] Harnad, Stevan (1990) The Symbol Grounding Problem.

http://cogprints.org/3106/
D&D.Sci II Evaluation and Ruleset
by abstractapplic — 2021-01-17T16:58:40.087Z

This is a followup to the D&D.Sci post I made earlier this week; if you haven’t already read it, you should do so now before spoiling yourself.

Here is the web interactive I built to let you test your solution; below is a complete explanation of the rules used to generate the dataset. You’ll probably want to test your answer before reading any further.

(The generation process is more complex than last time, so I’m leaving some extremely minor and irrelevant details out: feel free to dig through my code for the full story if this bothers you.)

Ruleset

Item Types

A magic item is either a Weapon (Sword, Longsword, Warhammer, Battleaxe), Tool (Plough, Saw, Hammer, Axe), or Trinket (Ring, Pendant, Amulet).

Abstractions

Every item has an abstraction (Wrath, Prosperity, etc.) assigned to it. Weapons are randomly assigned Weaponish abstractions, Tools are randomly assigned Toolish abstractions, and Trinkets are randomly assigned Weaponish or Toolish or Trinketish abstractions. These abstractions have no effect on other features, and are only relevant to this exercise insofar as they make the demarcation of item types more obviously meaningful.

Modifiers

Weapons are assigned modifiers (+1, +2, etc.) explaining how much more damage the enchantment lets them inflict. This is both easily testable and tightly-regulated in-universe: warriors can just swing their new Battleaxe of Wrath +2 at a training dummy and confirm it does 2 extra damage. This is potentially relevant to us because the amount of extra damage is equal to mana/10 rounded down.

Non-Weapon items are also sometimes assigned modifiers; this is a marketing tactic performed ad-hoc by salesmen with shaky or nonexistent justification, and provides no actual information.

Color and Mana

Enchanting an item randomly assigns it a color, and then randomly assigns it an amount of mana via a process based on that color:

Red-enchanted items have 1d4*1d4*1d6 mana.
Blue-enchanted items have 1d6*1d10 mana.
Yellow-enchanted items have 17+1d4 mana.
Green-enchanted items have 2*1d20 mana.

The mean amount of mana for each color is near 20, but the variance varies a lot.

The Thaumometer

Wakalix did not read the instructions which came with his Thaumometer, and does not realize it needs to be calibrated based on the size and color of an item. Unfortunately, your lack of magical ability prevents you from adjusting it yourself.

It is currently opt...