trentmkelly/LessWrong-43k huggingface dataset:
Constraining narrow AI in a corporate setting
by MaximumLiberty — 2022-04-15T22:36:46.757Z

I have a question that is extremely practical.

My company is working with AI that is frankly not all that capable. A key example includes text classification that humans do, but that would let us avoid the expense of having a human do it. Off-the-shelf machine learning works fine for that. But we're starting the long process of learning and using various AI techniques. Over time, that should become much more sophisticated as we solve easy problems with very limited capabilities.

I'm in a position to erect prohibitions now on any future AI work that we do. These would be in the nature of "don't do X without escalated approvals." Should I require such a list and, if so, what should I put on it? For reference, our business is mainly acquiring and processing data. And we don't have the resources of a Google to put on research that doesn't quickly pay off. So we won't be doing anything cutting edge from the perspective of AI researchers. It's mainly going to be application of known techniques.

I can harmlessly put, "don't use or try to build an artificial general intelligence," because we'll never have the capability. So if that's all I put, there's no point.

Should I put "don't use or build any AI technique in which the AI engine recodes itself"? That's probably too simple an expression of the idea, but I don't want these prohibitions to be tied to particular versions of current technology.

Should I put "use additional security around AI engines to prevent external users from combining the capabilities of our AI engines with theirs" on the theory that a growing AI would imperialistically grab hold of lots of other AI engines to expand its capabilities?

I'm obviously out of my depth here, so I'd be grateful for any suggestions.
Summary: "How to Read Books and Write Essays" by OSP's Blue
by Pablo Repetto — 2022-04-15T21:17:30.590Z
Link: https://pabloernesto.github.io/2022/04/15/summary-ospblue-essays.html

This post is part of my "research" series.

Blue. How to Read Books and Write Essays. Overly Sarcastic Productions, 2019. https://www.youtube.com/watch?v=E3v8-j1znXM.

Why

This got recommended at the end of
Red's video on how to do research
and can be considered part of the same series.

In one paragraph

Academic reading is often boring reading, which is hard. Summaries, audiobooks, and limited or "focused" reading will help you get through a boring book, while annotating it will help you re-find things. Literary analysis accommodates as many meanings as you can sell your audience on. Good project management helps with writing too: schedule abundant time, make a point of taking breaks and revisiting drafts, and seek early feedback. Also texts should flow, try reading your writing aloud.

Table of Contents

Table of Contents in the original is embedded as text on the video.

Part 1: Reading
1.1 Reading for meaning 0:29
1.2 Getting the Most from Summaries 0:59
1.3 Reading efficiently 2:44
1.4 Understanding Content 3:24
1.5 Sourcing a Thesis 3:53
1.6 Annotation 4:34

Part 2: Argumentation
2.1 Relativism 5:16
2.2 Authorial Intent and You! 6:14
2.3 Good and Bad Logic 6:49
2.4 Interpreting a Text 7:24
2.5 Supporting an Interpretation 7:49
2.6 Crafting an Argument: Change 8:52

Part 3: Writing
3.1 Scheduling Your Writing 9:41
3.2 The Good, the Bad, and the Outline 10:55
3.3 Writing Lean Essays 11:35
3.4 Writing Like You Speak 12:37
3.5 Editing Drafts 13:10
3.6 In-class Essays 13:55
3.7 In Videos and Essays, Conclusions are Key 14:54

An alternative outline

Broadly, how this article reads from my perspective.

Prologue 0:00
Introduction 0:20

Part 1: Reading
1.1 Do not rely on summaries 0:29
1.2 Academic reading as a skill 1:16
1.3 Using reading aids
1.3.1 Summaries, their purpose and mode of employ 1:49
1.3.2 Audiobooks 2:44
1.4 Academic reading
1.4.1 Plot, character, theme, symbolism, style 3:24
1.4.2 Thesis LAST 3:36
1.4.3 Reading with a focus 4:01
1.4.4 Annotation 4:34

Part 2: Argumentation
2.1 Relativism
or "Why there are infinitely many right answers" 5:16
2.2 Evidence: the Foundation of Argumentation 6:31
2.3 The danger of circular logic 6:49
2.4 Evidence 2: Electric Boogaloo
or "Why there are als

...
Everything I Need To Know About Takeoff Speeds I Learned From Air Conditioner Ratings On Amazon
by johnswentworth — 2022-04-15T19:05:46.442Z

I go to Amazon, search for “air conditioner”, and sort by average customer rating. There’s a couple pages of evaporative coolers (not what I’m looking for), one used window unit (?), and then

this

:

Average rating: 4.7 out of 5 stars.

However, this air conditioner has a major problem. Take a look at this picture:

Key thing to notice: there is one hose going to the window. Only one.

Why is that significant?

Here’s how this air conditioner works. It sucks in some air from the room. It splits that air into two streams, and pumps heat from one stream to the other - making some air hotter, and some air cooler. The cool air, it blows back into the room. The hot air, it blows out the window.

See the problem yet?

Air is blowing out the window. In order for the room to not end up a vacuum, air has to come back into the room from outside. In practice, houses are very not airtight (we don’t want to suffocate), so air from outside will be pulled in through lots of openings throughout the house. And presumably that air being pulled in from outside is hot; one typically does not use an air conditioner on cool days.

The actual effect of this air conditioner is to make the space right in front of the air conditioner nice and cool, but fill the rest of the house with hot outdoor air. Probably not what one wants from an air conditioner!

Ok, that’s amusing, but the point of this post is not physics-101 level case studies in how not to build an air conditioner. The real fact of interest is that this is apparently the top rated new air conditioner on Amazon. How does such a bad design end up so popular?

One aspect of the story, presumably, is fake reviews. That phenomenon is itself a rich source of insight, but not the point of this post, and definitely not enough to account for the popularity of this air conditioner. The reviews shown on the product page are all “verified purchase”, and mostly 5-stars. There are only 4 one-star reviews (out of 104). If most customers noticed how bad this air conditioner is, I do not think a 4.7 rating would be sustainable. Customers actually do like this air conditioner.

And hey, this air conditioner has a lot going for it! There’s wheels on the bottom, so it’s very portable. Setup is super easy - only one hose to the window, much less fiddly than those two-hose designs where you attach one hose and the other pops off.

Sure, the air...
Feature proposal: Close comment as resolved
by Viliam — 2022-04-15T17:54:06.779Z

There is an article. It contains a typo. Someone mentions it in a comment. The comment is upvoted as helpful. Author fixes the typo. Now there is a no-longer-useful comment in the discussion, sometimes with replies (the author saying "thanks, fixed", etc.). Sometimes it sits at the top of the discussion, if it was sufficiently upvoted.

If I try to avoid this, I send the author a private message. This does not clutter the discussion, but the disadvantage is that other people don't see me doing this -- so they are likely to also send private messages to the author, and maybe someone will make a comment anyway.

Downvoting the comment gets it out of the way, but it's not fair to the person who made it. We should encourage reporting typos, definitely not punish it.

*

My proposal is to make a new functionality that approximately means "this comment was useful at the moment it was made, but it has outlived is usefulness". Available in the three-dots menu in the upper-right corner of the comment.

A comment can be marked like this by the person who made it, or by the author of the article. (Or by the moderators, of course.) A comment marked like this will be moved to the bottom, regardless of the sorting mode, and will by default appear collapsed. (Maybe also displayed using a different color.)

Unlike retraction, this has no impact on voting, etc. (I am not 100% sure how retraction works.) You can still read the comment, even upvote it if you wish. It just doesn't get in the way. Also, semantically, there is a difference between "I no longer support this" and "this is no longer relevant, but it was when I wrote it".
Convince me that humanity *isn’t* doomed by AGI
by Yitz — 2022-04-15T17:26:21.474Z

Earlier this week, I asked the LessWrong community to convince me that humanity is as doomed by AGI as Yudkowsky et al., seems to believe. The responses were quite excellent (and even included a comment from Yudkowsky himself, who promptly ripped one of my points to shreds).

Well, you definitely succeeded in freaking me out.

Now I’d like to ask the community the opposite question: what are your best arguments for why we shouldn’t be concerned about a nearly inevitable AGI apocalypse? To start things off, I’ll link to this excellent comment from Quintin Pope, which has not yet received any feedback, as far as I’m aware.

What makes you more optimistic about alignment?
Feature proposal: Shortform reset
by Viliam — 2022-04-15T15:25:10.100Z

Shortform comments are displayed separately on the main page, but technically they are implemented as top-level comments of an automatically created article that has a title "{userName}'s Shortform" and an empty body. You can find this article either directly in the list of user's articles, or indirectly by clicking on a shortform comment's permalink.

(By the way, it seems like you can edit the shortform article's title and body afterwards. No idea whether changing the title would interfere with its functionality.)

This is a smart solution that adds extra functionality by reusing the existing mechanism, so the shortform comments automatically inherit all functionality of the regular comments, such as replies, voting, etc. That is great!

My objection is that the current mechanism does not scale well. Some people use the shortforms more frequently than others. Some people's shortform comments start long discussions. As a result, although most people's shortform articles are easy to view, some people's shortform articles have over thousand comments (including the replies to the shortform comments), which exceeds the longest debates under regular articles. And of course, the length of a shortform debate will only keep increasing.

Now, maybe this is actually not a problem. With regular articles, if I am interested in the article, I often want to see all the comments, no matter how many of them there are. With shortforms, this makes less sense. Maybe. Depends on how other people feel about this.

*

Anyway, my proposal to address this problem (of too many comments under one shortform article) is the following:

When the user clicks New Shortform, the algorithm considers the situation, and optionally decides to automatically perform a "shortform reset", that is to create a new shortform article called "{userName}'s Shortform #2". The new shortform comment would then get posted in the new article, and so would the following shortform comments... until at some moment in the future another "shortform reset" happens and a new shortform article "{userName}'s Shortform #3" is created.

On the main page, the shortforms will look exactly the same. Under the old shortform article, it will still be possible to write replies to the already existing shortform comments; the only change will be that the new shortform comments would now no longer appear here, but at the new place.

The new sho...
Some reasons why a predictor wants to be a consequentialist
by Lauro Langosco — 2022-04-15T15:02:43.676Z

Status: working notes

Here's an exercise I've found very useful for intuition-building around alignment:

Propose a solution to the alignment problem.

Dig into the details until you understand why the proposal fails.

If there is an obvious fix, go back to step 2 and iterate.

In this post I'll go through an example of this type of exercise applied to oracle AI. The ideas in here are fairly standard, but I haven't seen them written up all together in one place so I'm posting this for easy reference. Some obvious-to-mention other posts on this topic are Dreams of Friendliness, The Parable of Predict-O-Matic, and Why Tool AIs want to be Agent AIs.

Predictors

Proposal: Instead of building a system that acts in the world, build a system that just tries to make good predictions (a type of oracle). This should avoid existential risk from AGI, because the system will have no agency of its own and thus no reason to manipulate or otherwise endanger us.

There are many things that can go wrong with this approach. Broadly, the common thread of the problems I list here is that instead of "just" making good predictions, the system acts as a consequentialist.[1] By this I mean that it 1) pursues its objective like an expected utility maximizer and 2) considers a large space of possible actions and doesn't ignore any particular pathway towards optimizing its objective (like producing manipulative outputs or hacking the hardware it's running on).

First problem: 'pick out good predictions' is a problematic objective. For example, imagine a model that is trained to predict camera inputs, and scored to maximize predictive accuracy. The model that actually maximizes predictive accuracy is one that takes control of the camera inputs and provides a null input forever.

This produces all the problems that come up with agents that maximize expected utility, such as the instrumental goal of killing of all humans to defend the camera setup.

Fix: we can define a notion of 'predictive accuracy' that doesn't have this problem. A candidate: predict what would have happened if the model only ever outputs a null prediction, and doesn't otherwise affect the world. (ETA: in addition, the predictor needs to be myopic, i.e. care only about the current prediction. Otherwise it will still be incentivized to return outputs that manipulate humans into e.g. given the model more resources, or make future qu...
Why Go is a Better Game than Chess
by monkymind — 2022-04-15T13:32:50.369Z

Go is a better game than chess because its rules are simpler and more aesthetic, and lead to an emergent complexity and strategy that is at worst equal to the emergent complexity and strategy of chess.

I am no chess hater (I own a $500 chess board), but I've long thought that go is the better game.

The rules of go are more simple than the rules of chess. If you don't believe me, compare the rules of each game.

Rules of Chess

Rules of Go

I understand simplicity is not a well defined concept, but for most naïve heuristics I could think of, go scored better: number of characters of the rules (in English), number of distinct concepts in the rules, amount of time to explain the rules to a new player, amount of time for beginner to only play legal moves, etc.

The emergent complexity and strategy of go is at worst equal to the emergent complexity and strategy of chess. It is no accident that an AI, Deep Blue, beat the Chess World Champion Gary Kasparov in 1996, but an Al, AlphaGo, only beat the Go World Champion Lee Sedol in 2016.

The number of distinct positions and possible moves in go far outnumber those of chess. Back of the envelope calculations put the number of distinct positions in chess at 10^120, and in go at 10^360. While both numbers are incomprehensible to our monkey minds, it is clear that go is in its own complexity category.

But maybe you're not convinced that go having simpler rules (axioms) and being more complex makes it a better (more fun) game.

Games are supposed to be fun to play. And I've found that the most fun games are ones that can be played with other people, and continue to stay fun even as you get better at them.

There are many reason both go and chess are wonderful games,

Practically infinite emergent complexity and strategy.
Beautiful strategy.
Large population of players

Reasons why chess is a more fun game than go,

More people in the Western world already know the rules of chess.
Chess involves pieces that can move, and pieces with distinct movements.

Reasons why go is a more fun game than chess,

Go has simpler and more aesthetic rules than chess. If I have to explain en passant to one more chess beginner, I'm going to cry.
Go is more complex than chess. And go strategy is more beautiful to me than chess strategy.
Go's varying board sizes and adjustable scoring make it a better game to play with a worse player than chess.

I love both ...
How to become an AI safety researcher
by peterbarnett — 2022-04-15T11:41:51.082Z

Crossposted from the EA Forum.

Reminder that you can listen to this post and other LessWrong posts on your podcast player with The Nonlinear Library.

What skills do you need to work on AI safety? And what can we learn from the paths people have taken into the field? We were inspired by the

80,000 Hours podcast with Catherine Olsson and Daniel Ziegler

, which had great personal stories and advice about getting into AI safety, so we wanted to do it for a larger sample size.

To better understand the lives and careers of AI safety researchers, I talked to eleven AI safety researchers in a variety of organizations, roles, and subfields. If you’re interested in getting into AI safety research, we hope this helps you be better informed about what pursuing a career in the field might entail, including things like:

How to develop research taste
Which specific technical skills to build
What non-technical skills you’ll need

The first section is about the general patterns we noticed, and the second section describes each person’s individual path.

Of note, the people we talked with are not a random sample of AI safety researchers, and it is also important to consider the effects of survivorship bias. However, we still think it's useful and informative to hear about how they got into the field and what skills they have found valuable.

This post is part of a project I’ve been working on at

Nonlinear

.

Paths into AI safety

What degrees did people get?

Perhaps unsurprisingly, the researchers we talked to universally studied at least one STEM field in college, most commonly computer science or mathematics. Most had done research as undergraduates, although this often wasn’t in AI safety specifically; people often said that getting early research experience was valuable.

It is sometimes joked that the qualification needed for doing AI safety work is dropping out of a PhD program, which three people here have done (not that we would exactly recommend doing this!). Aside from those three, almost everyone else is doing or has completed a PhD. These PhD programs were often but not universally, in machine learning, or else they were in related fields like computer science or cognitive science.

All of the researchers we talked with had at least familiarity with Effective Altruism and/or Rationality, with most people being actively involved in at least one of these communitie...
Announcing the Forecasting Wiki 
by nikos — 2022-04-15T09:52:33.167Z

We're excited to announce the Forecasting Wiki, the (future) go-to place for all information related to forecasting.

The goal of this wiki is to become a comprehensive repository that collects our shared knowledge on forecasting. We aim at a broad audience, ranging from newcomers who want to learn about forecasting and how they can improve to forecasting researchers who are looking for up to date research information.

At the moment we are just getting things of the ground. We will have a first meetup on Sunday, April 24 at 6pm CET / 5pm UK time / 12pm Eastern Time and are happy about everyone who would like to join us.

When you go to the wiki main page you will find
- the zoom link to the meetup
- an invite to our Discord channel
- Information on how to join our mailing list
- a link to our gather town workplace

We're looking forward to building the Forecasting Wiki together with you!
How path-dependent are human values?
by Ege Erdil — 2022-04-15T09:34:23.280Z

In the Pointers Problem, John Wentworth points out that human values are a function of latent variables in the world models of humans.

My question is about a specific kind of latent variable, one that's restricted to be downstream of what we can observe in terms of causality. Suppose we take a world model in the form of a causal network and we split variables into ones that are upstream of observable variables (in the sense that there's some directed path going from the variable to something we can observe) and ones that aren't. Say that the variables that have no causal impact on what is observed are "latent" for the purposes of this post. In other words, latent variables are in some sense "epiphenomenal". This definition of "latent" is more narrow but I think the distinction between causally relevant and causally irrelevant hidden variables is quite important, and I'll only be focusing on the latter for this question.

In principle, we can always unroll any latent variable model into a path-dependent model with no hidden variables. For example, if we have an (inverted) hidden Markov model with one observable xk and one hidden state hk (subscripts denote time), we can draw a causal graph like this for the "true model" of the world (not the human's world model, but the "correct model" which characterizes what happens to the variables the human can observe):

Here the hk are causally irrelevant latent variables - they have no impact on the state of the world xk but for some reason or another humans care about what they are. For example, if a sufficiently high capacity model renders "pain" a causally obsolete concept, then pain would qualify as a latent variable in the context of this model.

The latent variable hk at time k depends directly on both xk and hk−1, so to accurately figure out the probability distribution of hk we need to know the whole trajectory of the world from the initial time: x1,x2,…,xk.

We can imagine, however, that even if human values depend on latent variables, these variables don't feed back into each other. In this case, how much we value some state of the world would just be a function of that state of the world itself - we'd only need to know xk to figure out what hk is. This naturally raises the question I ask in the title: empirically, what do we know about the role of path-dependence in human values?

I think path-dependence comes up often...