trentmkelly/LessWrong-43k huggingface dataset:
How Can One Tell What Is Beautiful?
by Erich_Grunewald — 2021-05-15T12:55:25.109Z
Link: https://www.erichgrunewald.com/posts/how-can-one-tell-what-is-beautiful/

The starting-point for all systems of aesthetics must be the personal experience of a peculiar emotion. The objects that provoke this emotion we call works of art [...] if we can discover some quality common and peculiar to all the objects that provoke [it], we shall have solved what I take to be the central problem of aesthetics. We shall have discovered the essential quality in a work of art, the quality that distinguishes works of art from all other classes of objects.[1]

– Clive Bell

Take any object and you will find a person who thinks it beautiful and another one who does not. Take any two people and you will find that they have different taste. A picture can be beautiful to my eyes but barren or even repulsive to yours. But some things, say the music of Johann Sebastian Bach or the paintings of Vincent Van Gogh, have the approval of nearly all who hear or see them. All well-functioning humans having some things in common, it stands to reason that our judgments should sometimes converge.

But the question is not just whether some things generally strike more people as endowed with beauty than do other things. The answer to that question is trivially yes. The question is rather whether the thing really is more beautiful than the other thing: if that is somehow written in the stars, so to put it. If I say that some or another painting by Vincent Van Gogh is beautiful, is it possible that I am stating a fact? Is what I say the sort of thing that can be true or false?

Christopher Mole makes some useful distinctions here.[2] He calls anti-realism the view that any statement such as "the Mona Lisa is a beautiful painting" means not that the Mona Lisa is objectively beautiful, but only that the speaker of said statement is enjoying experiencing it; in other words, it is not intended as a statement of fact. There are two versions of realism. Subjectivity is the view that all facts involved in aesthetic assessments are subjective, in other words they are facts about the way in which a thing appears to the subject. Objectivity is the view that some of those facts are facts about the object being perceived. (A stronger version of objectivity would hold that all such facts are about the object being perceived.)

About the anti-realist view I will only say the following. It involves making the distinction between two sorts of judgments of beauty. One is a judgment of t...
Technocratic Plimsoll Line
by Martin Sustrik — 2021-05-15T08:34:51.630Z

Plimsoll line is a marking where the hull of a ship meets the surface of the water.

I've just read a discussion between Glen Weyl and Scott Alexander about technocracy. It's really hard to figure out what they are actually disagreeing about. My naive understanding is that Weyl may be saying: "When we design mechanisms, we have to pay attention to make them legible to the political class and/or to the general public," and Scott Alexander responding "Sure, but why do you believe the rationalist community to be opposed to the idea?"

And while I have little to add to that discussion - besides maybe adding a case study by pointing out that the gripes that many Europeans have with the European Union seem to be mostly focused on the legibility rather than on the actual performance of the institution - it made me write down couple of thoughts on technocracy that I never bothered to put on paper before.

For the purpose of this article, technocracy, bureaucracy and meritocracy are the same thing: It's a class of people who got their jobs based on their capabilities. The counterpart is the political class, the people who get their jobs based on their political views and/or their loyalty. They may be elected by the public or get selected by a higher-up. I guess there isn't a short word for the latter kind of person, so I'll just go with "political appointee".

Viewed this way, a president or a prime minister are political appointees. So is, say, a director of the tax authority, who may be literally appointed by the prime minister. A head of a department at a ministry may, on the other hand, apply for the position based on a job ad and manage to get hired by passing an interview testing his professional skills. That makes him a technocrat - or bureaucrat, if you will. In the same way, Chinese emperor, even if it sounds weird, is a political appointee and the mandarins are technocrats.

The line tends to be blurry. Even the experts are often hired based on the recommendation of their peers and it's often hard to tell whether a person was hired because they were technically superior or whether they were just buddies with the interviewers.

In any case, the point I am trying to make is that in every state, in every business or organization there's a certain horizontal line separating political appointees on the top from the technocrats on the bottom. Depending on your political pre...
Intermittent Distillations #3
by Mark Xu — 2021-05-15T07:13:24.438Z

Mundane solutions to exotic problems (Paul Christiano)

Mundane solutions to exotic problems

Summary

Thinking about AI safety often leads to considering exotic problems: models purposefully altering their gradients, agents hiding their capabilities to defect when an opportunity arises, or humans being vulnerable to side-channel attacks. These exotic problems might seem like they require exotic solutions. However, the author points out that the origin of these exotic problems is the models themselves having exotic capabilities. If one is able to train a model to use whatever capabilities it has for a good purpose, then if the model gains exotic capabilities, it'll also be able to use those to avoid exotic problems.

As an analogy, if one hires a mercenary to help them when a war, they might be worried that the mercenary develops some weapon that means they are no longer motivated by your money. Since the mercenary has different goals than their employer, the employer must be worried about the incentive structure that they set up being broken. However, if one has a soldier that is fundamentally loyal, one is not at all worried about this soldier developing a powerful weapon, since their loyalty ensures they'll use this new exotic capability in service of your goals. If it turns out that the weapon requires special containment procedures, the soldiers' loyalty will ensure that they'll use their weapons expertise to help you contain it.

Opinion

This framing on capabilities redirecting is similar to framings provided in The strategy-stealing assumption and Universaliity Unwrapped. Empirically, it's taken me a while to understand the author's framing of "build an overseer that 'knows' everything the model knows" as a sufficient solution to the alignment problem, but I think it makes sense to me now.

However, I still don't really understand why the author thinks this is a tractable problem. To be fair, I'm not sure why people think value learning is tractable either. I'd blame this on my lack of understanding more than anything.

Elsewhere, the author has said their goal is to provide a clean solution to the alignment problem. I think this post gives intuition for why the messiness that sometimes appears at the surface of AI safety problems might be solvable with some deep solution.

Low-stakes alignment (Paul Christiano)

Low-stakes alignment

Summary

A model is operating ...
How to compute the probability you are flipping a trick coin
by lsusr — 2021-05-15T00:02:55.476Z

Suppose you flip a coin n times and get n heads in a row. What is the probability the next flip will land heads?

Suppose the coin is either a fair coin with one heads or a trick coin with two heads. Let X denote our training data of n heads. We want to find P(fair|X) and P(trick|X). Let P(trick)=ϵ. It follows that P(fair)=1−ϵ.

We use Bayes theorem P(A|B)=P(B|A)P(A)P(B).

P(fair|X)=P(X|fair)P(fair)P(X)=P(X|fair)P(fair)P(fair)P(X|fair)+P(trick)P(X|trick)=2−n(1−ϵ)(1−ϵ)2−n+ϵ1=(1−ϵ)(1−ϵ)+2nϵ

P(trick|X)=P(X|trick)P(trick)P(X)=P(X|trick)P(trick)P(trick)P(X|trick)+P(fair)P(X|fair)=1ϵϵ1+(1−ϵ)2−n=ϵϵ+(1−ϵ)2−n=2nϵ(1−ϵ)+2nϵ

I have flipped perhaps a hundred coins. One was double-headed trick coin. On the one hand, I flip trick coins with anomalously high frequency. On the other hand, double-headed trick coins are more likely than regular coins to get flipped than fair coins. I estimate the flip frequency of double-headed trick coins to be one in ten thousand.

ϵ=trick coinstotal coins=110,000=10−4

What does it look like when we graph our probabilities with ϵ=10−4?

For the first 5 heads you can remain confident you are flipping a regular coin. Around 10 heads the exponential takes off. You quickly become confident you are not flipping a regular coin. At 20 heads in a row you can be confident you are not flipping a regular coin.

The Inflection Point

The inflection point occurs when the probabilities are equal.

P(fair|X)=P(trick|X)(1−ϵ)(1−ϵ)+2nϵ=2nϵ(1−ϵ)+2nϵ1−ϵ=2nϵ1ϵ−1=2n1ϵ=2n+1ϵ=12n+1

A linear increase in your data has predictive power equal to an exponential increase in the strength of your prior.
Under Pressure
by Alex Hollow — 2021-05-14T22:04:06.017Z
Link: https://alexhollow.wordpress.com/2021/05/14/under-pressure/

One morning, I was making espresso and also had to fill up my water bottle. I ground the beans, tamped them down, pulled the shot, and then started filling up my water bottle in the sink. The espresso machine finished spitting out coffee, and all that was left to do was empty out the portafilter.

I looked down at my water bottle, listened to the change in the noise of water splashing that tells you how full the bottle is and figured I had time to set the water bottle down, dump the portafilter into the trash, and then come back before the water bottle overflowed. (I believe in being impatient.)

So, I do! And then I realize that something weird has just happened – the coffee grounds flew out on the first try. Normally, it takes me several rounds of waving the portafilter through the air, tapping it on the side of the trash can, and trying to figure out what I’m doing wrong before I get the grounds out. That day, I got it on the first try, because I had to.

Over the next few days, I explored portafilter emptying strategies. The general strategy remained the same – lift up the portafilter, swing down quickly, then slow down and let the coffee’s inertia carries it out. My first contender was raising it higher before swinging it down, but that didn’t replicate. I tried it a few times, increasing the level of stress I was under, hoping to recreate the feeling of “needing to get it right on the first try”, but still nothing. I tried to force myself to forget the whole “raising it higher” thing, and just try what feels natural. Halfway through that swing, I noticed I was holding the portafilter much further up the handle, closer to the metal part. That let me slow down my swing faster, because the portafilter had less leverage on me.

The object-level lesson is to hold the portafilter further up the handle. The meta-level lessons are to experiment more with procedures you invented yourself (no one taught me how to empty a portafilter, so I should have realized I was likely in a local maximum), and that putting some fake pressure on yourself (where the stakes are actually low, but you instinctively believe they are high) can make you actually try to solve your problems.
Who has argued in detail that a current AI system is phenomenally conscious?
by Robbo — 2021-05-14T22:03:12.882Z

I am working on a talk in which I will try to present the strongest possible case that, by the lights of some plausible and widely accepted scientific theory (or theories) of consciousness [1], some current AI system is phenomenally conscious.

I am curious what work like this, if any, already exists.

I am aware of many pieces on the possibility of AI consciousness in general, or on the importance of AI sentience and possible moral patienthood. But I am curious who has given a detailed argument concerning an existing system or systems.

[1] Examples would include: global workspace theory, attention schema theory, self-model approaches, or higher order theories.
Utopic Nightmares
by Jozdien — 2021-05-14T21:24:09.993Z

Utopias are hard to believe in.

Our basis for normality - and thus for our frame of reference for the world - obviously follows from what we’ve seen and experienced in the world around us. But utopias are hard to write, so, never seeing them, we’re left with a glaring emptiness in place of cached thoughts about a future that actually hits ceilings of hedonism.

When you’re a writer trying to create a utopian world, historically, you’ll end up at one of a few kinds of stories. You either envision a world that seems perfect to its people - as in Brave New World, for the most part - but that we can see is flawed from our morality; thus hardly qualifying as a utopia to us. Or you create a seemingly utopian world, filled with a hedonic people, but which harbors one sinister secret that breaks the illusion - in some stories, these arise as a natural consequence of trying to build a utopia. If you really tried to think of a perfect world, you’d probably realize you were wasting your time; utopias are boring. Writing needs challenges for the characters.

This criticism follows through to the real world. The idea of eternal boredom is commonly touted as an argument against life extension, transhumanism, and sometimes the ideal of a better future. At first glance, it may sound like understandable rhetoric - people routinely get bored with things even today, where opportunities for happiness are far lower than their upper limits (though still higher than all of history).

But the future wouldn't resemble the present, in less obvious ways. The scientific prowess we would have to develop before we can viably imagine actually creating a utopia is still leagues beyond us (from a technical perspective - from a chronological perspective, it might only be a few decades before we have AGI). Humans are marvelously bad at predicting the growth of new technologies, but there are a few we can speculate of today that we have reason to believe can’t be solved with time and sufficiently large amounts of resources (which we could obtain through interstellar exploration or on a long enough timeline, stuff like Dyson spheres). Some modern ideas may run against what we consider today to be fundamental laws of physics, but those are outside the scope of this post.

The technologies we would conceivably need to buil...
Concerning not getting lost

by Alex Flint — 2021-05-14T19:38:09.466Z

Content warning: striving, awkwardness, expressions of personal insecurities

Outline

This post is somewhat personal.

I try to convey why I think it’s important to stay in touch with a certain personal voice.

The reason I give is that without this personal voice, we are liable to lose track of our terminal goals and get lost.

By "persona voice", I don’t mean "emotional" or "following the gut" or "following the heart". This post is my own attempt at writing from this personal voice. I hope that this conveys what I mean by "personal voice" even if the explication I give does not.

Concerning not getting lost

There was something on the tip of my tongue that I wanted to write, just before I sat down. There was a monologue playing out in my mind, and it was beautiful, and it was relevant, and I thought I would write it down and share it with you all so that it wouldn’t just be me alone hearing it in my head.

It had to do with the agent model, and with notions of self. It had to do with loneliness. It was personal. And it was relevant to the conversation happening here in the place where we discuss AI and the future of life on the planet.

It is so very difficult to speak about that which is personal. But actually this is all personal to me — this whole journey we’re on to navigate the development of AI. Every one of the small number of posts I’ve written here over the past ten years has been personal to me, but I’ve mostly hidden that personal dimension behind a certain kind of wall.

And that’s fine. Our purpose here is to navigate a very real and very threatening state of affairs out in the world. We should do whatever needs to be done in order to navigate this threat, and that may mean putting aside that which is personal in order to get the job done. But it seems to me that when we leave aside that which is personal we actually forego a certain kind of power. We lose the compass within ourselves that separates that which is important from that which is a distraction. And in a world rife with distracting things, there is no place in the world where this personal compass is more important than within the community of people working to navigate the most significant threats to life on this planet.

As I write this, I am finding it difficult to stay with it. I am getting up and moving about and thinking of going someplace else where I don’t have to open up in the...
D&D.Sci May 2021 Evaluation and Ruleset
by abstractapplic — 2021-05-14T11:37:14.328Z

This is a followup to the D&D.Sci post I made last week; if you haven’t already read it, you should do so now before spoiling yourself.

Here is the web interactive I built to let you evaluate your solution; below is an explanation of the rules used to generate the dataset. You’ll probably want to test your answer before reading any further.

Ruleset

(Note: to make writing this easier, I’m using standard D&D dice notation, in which “4d8+3” means “roll four eight-sided dice, sum the results, then add three”.)

Species

The carcasses brought back by adventurers are 2/13 Yetis, 5/13 Snow Serpents, and 6/13 Winter Wolves.

Days Since Death

The days since each creature’s death is modelled by rolling two d12s, taking the lowest result, and subtracting 1.

(For the rest of this post, let “[DSD]” stand in for “Days Since Death”.)

Butchery

Carver takes a “collect anything the local alchemists might pay for, then carve out all the technically-edible meat you can still access” approach to her vocation.

The amount Carver gains from a Yeti carcass is given by 72+1d6-[DSD]d6, the amount from a Snow Serpent is 20+2d6 (she can’t prepare snakemeat, so all the profit comes from non-degrading components like scales and fangs; ergo, no time effects), and the amount from a Winter Wolf is 25-2*[DSD]+4d6.

Bidders

There are three bidders at a given auction: Alistair, Betty, and - except for today's auction, where you take her place - your employer Carver.

Carver the Butcher

Carver bids 32-2*[DSD]+2d20 on Yeti carcasses, 7+2d10 on Snow Serpent carcasses, and 31-3*[DSD]+2d8 on Winter Wolf carcasses.

Alistair the Butcher

Alistair has a different business model to Carver, focusing on extracting the highest quality cuts and selling them to rich clients; as such, his utility sharply decreases with time since death.

He is very predictable, bidding 55-6*[DSD] for Yetis, 60-20*[DSD] for Snow Serpents, and 50-12*[DSD] for Winter Wolves.

Betty the Necromancer

Betty doesn’t care how long something’s been dead, so long as no-one’s interfered with the body.

She bids 29+1d6 on Yetis, 9+1d8 on Snow Serpents, and 19+1d4 on Winter Wolves.

Strategy

The relevant factors are summarized in these graphs.

In almost all cases where rivals aren’t making bids greater than the Expected Value of the lot to Carver, the EV-maximizing choice is to bid the lowest amount that would guarantee winning the lot. The one exception is with lo...
Why Democracy?
by sxae — 2021-05-14T11:02:13.241Z

I am an avid and radical believer in the systemic property of democracy. But if you had asked me (before I wrote this, anyway) why I hold such a strong and deeply-held belief, I would have been uncomfortable with the amount of cultural conditioning that would come to mind. I grew up in in the West where you are saturated with a nominally pro-democracy viewpoint for your whole life, and so it is easy to endorse it as an ethical axiom, as opposed to in support of ethical axioms. It isn't enough for me to just feel strongly in support of radical democracy - I need to be able to tell you why.

Originally posted here.

What is Democracy?

Some definitions focus on implementation, stressing that this property lies in the ability for a population to choose their representatives in an electoral system:

Democracy is a form of government in which the people have the authority to choose their governing legislators.

I dislike this focus on implementation - surely there is a more general property we're attempting to describe here? I want a definition which is clearly phrased as a heuristic that can be maximized. We want something with some dimensionality, some ability to place different systems of governance on an axis of exactly how democratic they are, which doesn't seem to be the case with definitions that focus on implementation. We know too well that merely the act of having elections doesn't make a system democratic.

Other definitions are much more dimensional, but they describe the distribution of power like "everyone is equal on everything" or are generally vague around the specific rules about how power is distributed:

[Democracy is] a system of government based on a [belief in freedom and equality], in which power is either held by elected representatives or directly by the people themselves

But I find this unsatisfying as well, because "everyone getting power over every decision" seems like an obviously poor choice for getting things done. Referendums are more often used as binary bludgeons for loaded questions as they do ways of genuinely capturing a group's preferences. There are certainly situations where a decision will affect everyone equally (e.g. the adoption of a standard or new law, environmental costs, the fate of large public goods) but it seems like some decisions should be local. I don't think I should have power over what colour someone's bins on the other ...