trentmkelly/LessWrong-43k huggingface dataset:
High-precision claims may be refuted without being replaced with other high-precision claims
by jessicata — 2020-01-30T23:08:33.792Z
Link: https://unstableontology.com/2020/01/30/high-precision-claims-may-be-refuted-without-being-replaced-with-other-high-precision-claims/

There's a common criticism of theory-criticism which goes along the lines of:

Well, sure, this theory isn't exactly right. But it's the best theory we have right now. Do you have a better theory? If not, you can't really claim to have refuted the theory, can you?

This is wrong. This is falsification-resisting theory-apologism. Karl Popper would be livid.

The relevant reason why it's wrong is that theories make high-precision claims. For example, the standard theory of arithmetic says 561+413=974. Not 975 or 973 or 97.4000001, but exactly 974. If arithmetic didn't have this guarantee, math would look very different from how it currently looks (it would be necessary to account for possible small jumps in arithmetic operations).

A single bit flip in the state of a computer process can crash the whole program. Similarly, high-precision theories rely on precise invariants, and even small violations of these invariants sink the theory's claims.

To a first approximation, a computer either (a) almost always works (>99.99% probability of getting the right answer) or (b) doesn't work (<0.01% probability of getting the right answer). There are edge cases such as randomly crashing computers or computers with small floating point errors. However, even a computer that crashes every few minutes functions very precisely correctly in >99% of seconds that it runs.

If a computer makes random small errors 0.01% of the time in e.g. arithmetic operations, it's not an almost-working computer, it's a completely non-functioning computer, that will crash almost immediately.

The claim that a given algorithm or circuit really adds two numbers is very precise. Even a single pair of numbers that it adds incorrectly refutes the claim, and very much risks making this algorithm/circuit useless. (The rest of the program would not be able to rely on guarantees, and would instead need to know the domain in which the algorithm/circuit functions; this would significantly complicate the reasoning about correctness)

Importantly, such a refutation does not need to come along with an alternative theory of what the algorithm/circuit does. To refute the claim that it adds numbers, it's sufficient to show a single counterexample without suggesting an alternative. Quality assurance processes are primarily about identifying errors, not about specifying the behavior of non-functioning products.

A Baye

...
how has this forum changed your life? 
by Jon Ronson — 2020-01-30T21:54:56.037Z

Hello! I am totally new here so please bear that in mind in the event I commit faux pas! I'm a writer who has written a LOT about rationality and when rationality eludes us. These are books like So You've Been Publicly Shamed, Them, The Psychopath Test and The Men Who Stare At Goats, among others.

I don't know if this question has been asked elsewhere, but I'd love to know: has learning to be more rational impacted your everyday lives, in small or (perhaps more interestingly) BIG ways? Have there been occasions when you've put these skills (perhaps learned at a workshop) into practice in your domestic or work lives and the ripples have been surprising/consequential in positive or negative ways?

I'm thinking of writing about what happens when a person learns to be more rational, hence my question.

Hope this is the start of a wonderful thread!

Jon Ronson
Artificial Intelligence, Values and Alignment
by Gordon Seidoh Worley — 2020-01-30T19:48:59.002Z
Link: https://deepmind.com/research/publications/Artificial-Intelligence-Values-and-Alignment

I haven't yet had a chance to read the whole thing, but DeepMind has a new paper up from Iason Gabriel titled "Artificial Intelligence, Values and Alignment". Here's the abstract:

This paper looks at philosophical questions that arise in the context of AI alignment. It defends three propositions. First, normative and technical aspects of the AI alignment problem are interrelated, creating space for productive engagement between people working in both domains. Second, it is important to be clear about the goal of alignment. There are significant differences between AI that aligns with instructions, intentions, revealed preferences, ideal preferences, interests and values. A principle-based approach to AI alignment, which combines these elements in a systematic way, has considerable advantages in this context. Third, the central challenge for theorists is not to identify 'true' moral principles for AI; rather, it is to identify fair principles for alignment, that receive reflective endorsement despite widespread variation in people's moral beliefs. The final part of the paper explores three ways in which fair principles for AI alignment could potentially be identified.

As I said, I haven't read the whole thing, but I'm somewhat suspicious of moving in the direction of principles as discussed in this paper since, based on my skimming, it seems to be suggesting we pick the method of aggregating human preferences an aligned AI would use with a higher level of specificity than I suspect is safe (given the level of confidence we would need to have to make a particular choice).

Nonetheless this paper looks interesting and covers a number of topics in detail I haven't seen show up much in the academic literature even if we talk about them here, so I look forward to looking at this paper more closely and possibly having more detailed comments to offer then.
If brains are computers, what kind of computers are they? (Dennett transcript)
by Ben Pace — 2020-01-30T05:07:00.345Z

In 2013, Daniel Dennett gave this talk at an Oxford conference, and it was uploaded to FHI's YouTube channel. I remember this being one of my favourite talks by Dennett. I rewatched it yesterday, and found it had a surprising amount to say about subjects we've discussed lately on LessWrong, pertaining to cultural evolution, embedded agency, and design principles in biology.

I've made a transcript and edited it together below, with some section titles and images and light editing for clarity. (If Dan or someone else would like this edited/removed, I'll of course do that, I just thought it was really interesting and wanted to discuss it on LessWrong.) Dennett later turned these ideas into a book and gave more talks, but they were all directed at a less technical audience than LessWrong, so I found I enjoyed this talk more.

Here is the transcript.

Thanks.

I want to apologize for not attending the afternoon sessions. I was back in my room trying to digest and accommodate and incorporate a few little items from what I've been hearing today. I figured that would happen, and sure enough it did. So you will probably see some little fossil traces of things you said or heard in earlier talks.

Thanks to the organizers for putting this together and for inviting me. It's been as informative as I expected and hoped it would be and I certainly enjoyed the whole thing.

Competence Without Comprehension

We've been talking about “Yeah, of course the brain is a computer!” for a long time, and we run into a stone wall of disbelief and fear and loathing in many quarters. I think one thing is you have to stop and ask why that is. Why is this such a repellent idea to so many people? Of course, one reason might be because it's false. Brains really aren't computers, but that's not the new wrinkle that I'm going to be putting forward today. But I do hope there are some new bits.

Some of you will have heard parts of this before or seen parts of this before, I dare say, but rest assured, there's some stuff you haven't seen before. So don't just do your email or something. You might be surprised.

Think of all the people that say this “Brains aren't computers”. They're not idiots. There are some good scientists. Roger Penrose insists on this, Gerry Edelman insists on this, Jaak Panksepp. These are industrial strength scientists, and they all dislike the idea tha

...
Value uncertainty
by MichaelA — 2020-01-29T20:16:18.758Z

Epistemic status: This is basically a new categorisation scheme for, and analysis of, ideas that other people have proposed previously (both in relation to moral philosophy and in relation to AI alignment). I’m not an expert on the topics I cover here, and I’d appreciate feedback or comments in relation to any mistakes, unclear phrasings, etc. (and just in general!).

We are often forced to make decisions under conditions of uncertainty. This may be empirical uncertainty (e.g., what is the likelihood that nuclear war would cause human extinction?), or it may be moral uncertainty (e.g., is the wellbeing of future generations morally important?).

But what if you don’t believe that “morally important” is a coherent concept? What if you’re a moral antirealist and/or subjectivist, and thus reject the idea that there are any (objective) moral facts? Would existing work on moral uncertainty (see my prior posts) still be relevant to you?

I think that a lot of it is, to a large extent, for the reasons discussed in this footnote.[1] But I think that, to directly discuss how that work is relevant to antirealists and/or subjectivists, it would help to speak not of moral uncertainty but of value uncertainty (VU; i.e., uncertainty about what one “values” or “prefers”). Doing so also helps us to categorise different types of VU, and potential ways of resolving each of these types of VU. A final benefit is that such an analysis of VUs also has substantial relevance for moral realists, and for work on AI alignment.

So this post will:

Clarify what I mean by “values” in this context

Name, briefly describe, and briefly suggest responses to four types of VU, and two types of situations that actually aren’t VU, but could look as though they are

Return to the question of who and what these ideas are relevant for

Values

I should clarify a few points about what I mean by “values” in this post:

I mean what a person actually values (or would if they knew more, or will in the future, or would if “idealised”), not what a person’s explicitly endorsed moral theory suggests they should value.

For example, after a bunch of caveats and moral uncertainty, I roughly identify as a classical utilitarian. However, in reality, psychologically speaking, I also value things other than whatever maximally increases the wellbeing of conscious beings.

On the other hand, I don’t believe that people act

...
Towards deconfusing values
by Gordon Seidoh Worley — 2020-01-29T19:28:08.200Z

NB: Kaj recently said some similar and related things while I was on hiatus from finishing this post. I recommend reading it for a different take on what I view as a line of thinking generated by similar insights.

One of the challenges with developing a theory of human values is dealing with the apparent non-systematic nature of human decision making which makes it seem that human value are not consistent, coherent, or rational. One solution is to build or discover mechanisms by which they can be made legible and systematic. Another is to embrace the illegibility and inconsistency and find ways of working with it. This is a short start towards doing the latter because I believe the former cannot be made to work well enough to stand up against Goodhart effects under extreme optimization by superintelligent AGI that we want to align with human values.

I've been thinking a lot about what values are, and in particular looking for phenomena that naturally align with the category we variously call values, preferences, affinity, taste, aesthetics, or axiology. The only thing I have found that looks like a natural kind (viz. a model that cuts reality at its joints) is valence.

Valence on its own doesn't fully explain all the phenomena we want to categorize as values, especially things like meta-preferences or "idealized" values that are abstracted away from the concrete, embedded process of a human making a choice at a point in time. Instead it gives us a mechanism by which we can understand why a human makes one choice over another at some point in their causal history. And decisions are not themselves preferences, because decisions are embedded actions taken by an agent in an environment whereas preferences are, as typically considered, generators of decisions. I think we need to flip this notion of preferences as generators on its head, and in so doing we move towards becoming less confused about preferences.

So let me describe my current model of how this works, and let's see if it explains the world we find ourselves in any better than existing theories of values and preferences.

The New Model Axiology

Humans are embedded agents. They carry out processes that result in them causing events. We call this process of an agent causing events acting and the process that leads to taking one action rather than any other possible action a decision. Al...
[AN #84] Reviewing AI alignment work in 2018-19
by Rohin Shah — 2020-01-29T18:30:01.738Z

Find all Alignment Newsletter resources

here

. In particular, you can

sign up

, or look through this

spreadsheet

of all summaries that have ever been in the newsletter. I'm always happy to hear feedback; you can send it to me by replying to this email.

This is the summary of a

review post

of public work in AI alignment over 2019, with some inclusions from 2018. The full post has a preamble (~700 words), this short version / summary (~1.6k words), and a long version (~8.3k words). It is also available as a Google Doc

here

.

While the full post tries to accurately summarize different points of view, that is not a goal in this summary. Here I simply try to give a sense of the topics involved in the discussion, without saying what discussion actually happened. I'd strongly recommend reading the full post; I would have put it in full in this email, but 8,300 words seemed a bit too long, even for this newsletter.

Basic analysis of AI risk. Traditional arguments for AI risk argue that since agentic AI systems will apply lots of optimization, they will lead to extreme outcomes that can’t be handled with normal engineering efforts. Powerful AI systems will not have their resources stolen from them, which by various dutch book theorems implies that they must be expected utility maximizers; since expected utility maximizers are goal-directed, they are dangerous.

However, the VNM theorem

does not justify

the assumption that an AI system will be goal-directed: such an assumption is really based on intuitions and conceptual arguments (which are still quite strong).

Comprehensive AI Services

(CAIS) challenges the assumption that we will have a single agentic AI, instead suggesting that any task will be performed by a collection of modular services.

That being said, there are several other arguments for AI risk, such as the

argument

that AI might cause “lock in” which may require us to solve hard philosophical problems before the development of AGI.

Nonetheless, there are

disjunctive reasons

to expect that catastrophe does not occur: for example, there may not be a problem, or ML researchers may solve the problem after we get “warning shots”, or we could coordinate to not build unaligned AI.

Agency and optimization. One proposed problem is that of

mesa optimization

, in which an optimization algorithm used to train an AI creates an agent t...
Slide deck: Introduction to AI Safety
by Aryeh Englander — 2020-01-29T15:57:04.979Z
Link: https://drive.google.com/file/d/1-Mb2_h5UTp0wxObdXn-NZndrfEBa8b3j/view?usp=sharing

This is from a presentation that I've been giving over the past few months to various groups within the Johns Hopkins University Applied Physics Laboratory (APL), where I work. I'd be interested to hear people's comments, critiques, suggestions, etc.

Note that this is a somewhat older version of the presentation, and I've made several improvements since then. But this is the version that finally made its way through our Office of Technology Transfer and was approved for public release. At some point I'll submit an updated version for public release approval, but at the moment this is the version I can share.
The Skewed and the Screwed: When Mating Meets Politics
by Jacob Falkovich — 2020-01-29T15:50:31.681Z

Cross-posted, as always, from Putanumonit.

If an alien learned about our world from browsing the internet of beefs, they would be under the impression that a bitter war is raging between the sexes. All you read about are toxic men and manipulative women fighting to entrench the patriarchy or gynocracy (respectively) and subjugate the opposite gender forever. Men and women are not merely failing to connect, they are fighting a zero-sum conflict without compromise.

This sentiment seems to be coming from the extremes of the political spectrum, the very progressive left and the very conservative right. Humans aren’t naturally drawn to cooperate with those of other nations, races, classes, and tribes. But most of us are drawn to seeking mates of the opposite sex. So those who fight the culture, class, and identity wars, why do they also fight the gender war?

This question touches on every major theme of this blog: my deep fascination with romance and dating, my model of politics as a horseshoe, my entreaties to seek cooperation and resisting polarization. Above all, my penchant for explaining complex phenomena using simple mathy models.

And so: I will try and explain what happens when mating meets politics using a speculative and overgeneralized model. This is a descriptive account of what I think is actually happening, not a moral sermon. There will be charts, and memes, and dating stories.

Woke Women, Dangerous Men

The immediate trigger for this post is the recent article about The Dangerous Rise Of Men Who Won’t Date “Woke” Women by Vicky Spratt. The article lists the signs by which ye shall know them: men who list “no psychos” as a requirement on their dating profiles, this one guy who described Jordan Peterson as his “dream dinner guest”. But a few paragraphs later, Ms. Spratt is talking about MAGA, 8chan, and mass shooters.

I know quite a few guys who are terrified of dating woke women. Let me paint you a picture: nerd, works in tech or journalism, voted for Hillary and is now debating between Bernie and Yang, hooked up with a trans woman once, thinks that Sam Harris pwned Peterson because religion is dumb, asks for verbal consent before kissing on the first date. Not quite the stereotypical right-winger. The real right-wingers can barely distinguish between the “woke” and the moderate libe...
Mod Notice about Election Discussion
by Vaniver — 2020-01-29T01:35:53.947Z

TL;DR: Posts discussing the presidential election may be hidden, not appearing in recent discussion, on the home page, or on the allPosts page.

Ben Landau-Taylor wrote The Four Year Locusts to describe the periodic problem of politics consuming everything within reach. From it:

While the swarm is too big to defeat entirely, local defense is possible. Some areas can be kept clear. What is precious can be guarded. The swarm is temporary; you need only outlast it.

Even in the best of times, Less Wrong discourages posts on politics; Politics is the Mind Killer, as the saying goes, a post that turned into a norm a bit more expansive than the original post suggests. We are not currently in the best of times.

I see three main reasons why this matters:

Object-level harms to the discourse from using political examples. It's both harder for people to discuss politics, and harder for them to agree on the right abstractions. If you discuss the abstractions directly, you can avoid those issues.
Pull factors for the wrong crowd. Suppose some people care 100% about rationality, and discuss rationality things; people who care 80% about rationality and 20% about politics join in and talk about rationality. But if the people who care a little about politics start discussing politics with each other, then they pull in people who care 60% about rationality, and 40% about politics, and perhaps things spiral out of control, especially if it's somehow easier or more rewarding to talk about politics.
Push factors for the right crowd. Suppose people want to avoid politics, because of disinterest, reputation harms, or the increased surface area for enemy action. Then if your forum becomes about that thing, people who would otherwise be interested in your thing will leave, or be harassed.

So back to the locusts. How to respond? People like Bryan Caplan take a social media hiatus. Here on LW, we're going to try to minimize the visibility of political discussion, especially with regards to presidential politics. We're interested in raising the sanity waterline more broadly, but also in picking our battles, and presidential politics is a battle we definitely are not interested in picking.

We're considering several different ways of minimizing visibility, and will deploy them as they seem necessary. You'll still be able to write personal blog posts on topics as you see fit with the normal restrict

...
Survival and Flourishing grant applications open until March 7th ($0.8MM-$1.5MM planned for dispersal)
by habryka — 2020-01-28T23:36:40.191Z

The Survival and Flourishing Fund has taken over most of the grantmaking of the Berkeley Existential Risk Initiative (BERI) and just opened up another grant round (after the last round in September):

http://survivalandflourishing.fund/#2020-h1-grant-round

The plan is to make a total of $0.8MM-$1.5MM in grants to organizations working on the long term flourishing and survival of humanity.

At this point in time SFF can only make grants to charities and not individuals, so if you have a project or organization that you want to get funding for, you will have to either already be part of an established charity, found one, or be sponsored by an existing charity. We have some initial plans to run a grant round for more project and individual-based grants, but nothing concrete yet.

Also happy to again answer any questions you have in the comments, to the best of my knowledge.