trentmkelly/LessWrong-43k huggingface dataset:
AI Writeup Part 1
by SNl — 2022-02-04T21:16:27.345Z

Aims

In this post I summarize my discoveries from a semester of AI readings and discussions drawn primarily from the

2022 AGI Safety Fundamentals Alignment Curriculum

. I am grateful to Professor Anton Korinek for his generous advising in my independent study which lead to this writeup. I am also grateful to my fellow AI reading group discussants at UVA (and especially our discussion leader Ryan Bloom) for their thoughtful contributions to our group which informed my thinking for this post. While this post was primarily written as a means to clarify my own thinking as I learned more about AI Safety, I also hope to use it as a reference point for facilitating my university’s AI discussion group. I also aim to write a Part 2, covering the second half of the 2022 AGI Safety Fundamentals Alignment Curriculum and discuss any updates to my understanding as I begin facilitating.

Technical Background: What is Artificial Intelligence?

Definitions

Artificial Intelligence (AI) is both the study of intelligent algorithms and the intelligent algorithms themselves.[1] For the purposes of this post, we’ll hold that intelligence measures one’s ability to achieve one’s goals across a wide range of environments.[2] An algorithm is a “step-by-step procedure” for solving a problem, typically written on a computer.[3] In turn, intelligent algorithms are computational procedures that can achieve their goals across many environments.

A very brief history of AI

Beginning in the 1950s, “Good Old Fashioned AI” (GOFAI), also known as symbolic AI, used search and logic to solve high-level mathematical equations. In 1997 Deep Blue beat chess Grandmaster Garry Kasparaov by using GOFAI—Deep Blue searched over millions of positions to find the optimal play.[4] In the 1960s, a series of criticisms argued GOFAI could never handle the complexities of the real world. This burst the building AI hype and led to an AI winter, in which funders pulled out and AI progress slowed.[5] In the 1990s, AI research made a comeback as researchers shifted from symbolic AI to machine learning, which remains the dominant paradigm today. [6]

Machine Learning basics

Unlike symbolic AI, Machine Learning (ML) is adept at addressing real world complexity. Rather than searching through every possible configuration to find the optimal algorithm, ML relies on using statistical te...
Do mesa-optimization problems correlate with low-slack?
by sudo — 2022-02-04T21:11:10.315Z

My understanding is that the primary problem with inner-alignment is that even outer-aligned AIs may be incredibly unpredictable due to unpredictable mesa-optimization. Put another way, there is a possibility that even air-tight objective functions could be ignored by a hypothetical superintelligent AI to a sufficient degree such that the AI still ends up misaligned.

It seems as if the general consensus on LessWrong (and possibly the larger AI safety community) is that increasing slack in AI training creates safer models. I might be misremembering, but I believe Paul Christiano actually specifically argued that higher pressure to optimize to an objective function is one of the most likely sources of unsafe AI.

However, one would imagine that the more slack is present in an AI model, the harder the inner-alignment problems are to resolve.

I'll like to pose a hypothetical question to help me understand this dilemma a little more clearly. Imagine a world where we largely solve outer-alignment problems (while remaining unsure about inner-alignment problems) and are forced to deploy AGI models (to beat Dr. Amoral, perhaps). Do you think that reducing slack would be an effective way to balance inner and outer alignment risks?
Impactful Forecasting Prize for forecast writeups on curated Metaculus questions
by elifland — 2022-02-04T20:06:16.408Z

cross-posted to EA Forum

TLDR

We’re giving out $4,000 to the best forecast writeups submitted via this form on these Metaculus questions by March 11 to encourage more people to forecast on impactful questions and write up their reasoning.

Motivation

We believe that forecasting on impactful questions is a great candidate for an activity more EA-interested people should partake in, given that it:

Provides direct value when done on decision-relevant questions.

Improves and demonstrates good judgment and research skills.

Can be fun: provides a concrete and gamified framing for activities that look similar to research.

Leads to generally learning more about the world.

Helps match up bright, like-minded collaborators.

This is informed by personal experience: we have learned a lot and found great collaborators through forecasting.

However, it’s difficult to start doing impactful forecasting right now. Metaculus has lots of questions, most of which aren’t selected for impact. It can be overwhelming to navigate and find questions that are both impactful and interesting. It may also be difficult to know where to start in an analysis. Additionally, it can be scary to share your thoughts publically without a push and if it’s good reasoning, the incentives are usually against you.[1]

How to participate

We curated 25 Metaculus questions for prediction in this Airtable. Either Eli Lifland or Misha Yagudin has forecasted on each of these to provide a starting point for further analysis. The table has cause area tags to allow filtering for interest and expertise.

To participate: do the following by March 11 2022, 11:59 PM Anywhere On Earth:

Make a forecast on one of the selected questions and write up your reasoning.

Write up the reasoning for your forecast, on Metaculus or elsewhere (e.g. a forum/blog post).

We require you to share your reasoning publically, unless there are infohazards in which case we will consider private submissions.

Fill out this form with a link to your writeup and contact info; anonymous writeups and email addresses are okay.

You may submit up to one writeup per question; if you submit multiple, your final one will be used. You may submit entries for as many questions as you please.

We will host meetups in this gather.town space to chat about the contest and facilitate forecasting on the questions together. The meetups will be on Wed Feb 16 and W...
Money-generating environments vs. wealth-building environments (or "my thoughts on the stock market")
by Nicole Dieker — 2022-02-04T17:37:33.612Z

Crossposted.

Very interested in whether you think I'm right; also interested in the ways you think I'm wrong.

This post is going to include a discussion of the stock market, which means that I am obligated to begin with a statement that you should not take any of this as investment advice.

The real question is whether I will.

Sooooo about a month ago I told you all [ON MY BLOG, WHERE THIS IS CROSSPOSTED FROM] that I wanted to write about

money-generating environments vs. wealth-building environments

, with the primary distinction between the two defined as follows:

Money-generating environments are controlled by someone else (and the value that they are willing to release into the environment).
Wealth-building environments are controlled by you (and the value you bring to the environment).

If you’re writing for a content site that offers defined jobs at a preset value per job, you’re working within a money-generating environment.

If you’re writing for clients who are reaching out to you based on your expertise and are giving you the opportunity to either name or negotiate your rate, you’re working within a wealth-building environment.

It’s worth noting that the latter environment has an upper bound; if someone asked me to write a 1,200-word article about how to choose a CPA, for example, and I said I’d only do it for $2 a word, they’d probably go find someone else.

But it’s also worth noting that in a wealth-building environment, the wealth that you’re building goes beyond money. You’re also building your reputation, your network, your portfolio, and so on.

Plus, you’re building a level of expertise that you will be unlikely to get if you only write low-paying content site stuff. When I was giving a presentation on freelancing to grad students at Texas Tech earlier this week, for example, I explained to the students that when the CARES Act was released in 2020, I read the entire thing — and was able to use my knowledge of what was actually in the CARES Act (in combination with the financial knowledge I already had in my toolkit) to pitch high-value work.

I created value for readers by being able to provide a professional-level analysis of a document I had actually taken the time to read.

I created value for clients by giving them the kind of article that attracted readers who were looking for specific information about the CARES Act.

I created value for myself by building ...
Transferring credence without transferring evidence?
by Kaarel — 2022-02-04T08:11:48.297Z

Agent X has evidence for proposition P that agent Y does not have, and X wants to convince Y that P is true (or more precisely, increase Y's credence in P as well). However, there are the following unfortunate conditions:
1) Y doesn't trust X, meaning that X just telling Y that they have evidence will do little to convince Y. (In particular, they don't have common knowledge of their credences.)
2) X wants to reveal as little as possible about the evidence they have.
Is there anything clever X and Y can do here? Or is there a reason why this is impossible? Can one prove a tradeoff between the update on Y's credence and the amount of details about the evidence revealed? If this is too vague/does not make formal sense for some reason, as well as for illustration, here are some particular examples of what I have in mind:
Example 1: X has a picture of the Moon and wants to convince Y that the Moon exists. The picture is a selfie taken by Adolf Hitler living on the far side of the Moon; X doesn't want Y to know the various other things of which this picture or X's possession of the picture would be evidence (e.g. because X wants to maintain their edge in the "space Nazi invasion of Earth by 2030" prediction market), so just handing the picture to Y is not a good option for X.
Example 2: The State Department wants to convince the public that Russia is in the process of making a specific disinformation video while revealing as little as possible about how they found out that this is the case. See https://www.youtube.com/watch?v=8G3WtjU9XA4 and https://www.youtube.com/watch?v=rU1_BHMHsI8 (but let's ignore the question of whether they actually, in reality, in this particular case, have information or not; this is just supposed to be a hypothetical example for the purpose of this discussion).
My observations thus far & more refined versions of the previous questions:
This seems obviously close to zero-knowledge proofs, but I haven't been able to flesh out a correspondence to any satisfactory degree. Should X be trying to give something like the evidential analogue of a zero-knowledge proof of P? Or should X be trying to give something like a zero-knowledge proof of "I have evidence that P"? Maybe these are equivalent, but one is a better way to think about it than the other? What would such a thing look like in the above examples? Is the best X can do to just genera...
Positly covid survey 2: controlled productivity data
by KatjaGrace — 2022-02-04T07:30:16.934Z

(This continues data from the survey I did the other day on Positly (some other bits here and here.)

Before asking everyone whether they had had covid, or have ongoing problems from it, or anything else, I asked them about their recent productivity relative to 2019. I was hoping to minimize the influence of their narratives about their covid situation and how it should affect their productivity on their productivity estimates.

The clearest takeaway, I think: people who would later report ongoing post-covid symptoms also reported being way less productive on average than people who didn’t report any covid.

Perhaps stupidly, I asked people about their productivity in one of three randomly selected ways, to better tell if results were sensitive to misunderstandings, wording or other vagaries of survey taking. While I stand by this for higher commitment surveys, here it meant trying to aggregate a bunch of slightly mismatched claims about how unproductive a year people are having, when I could have just been averaging a column of numbers. And people did give different-looking distributions of answers for different questions, and different numbers of people with covid got randomized into different kinds of questions, so combining them is messy.

The different questions were these:

How productive were you this week, relative to your average in 2019?

More than twice as productive

About 1.5x as productive

About as productive

About three quarters as productive

Less than half as productive

How productive were you this week, as a percentage of average for you in 2019?

(slider: 0-200%; instruction: For more than 200%, just put 200%)

Were you much less productive in 2021 than in 2019?

Yes

No

Below are the answers, divided up by question type. Note that the numbers of respondents involved are tiny and things are noisy. For example, the category question only got two long covid sufferers, and one of them was the nurse apparently working 200% as much as in 2019 but fatigued to the point of crashing her car. (In general, productivity varying because of demand as well as supply is an obvious-in-retrospect theme, according to my reading of the long answers to later questions.)

0-200% slider (average %)
Not ‘much less’ productive (fraction of people)
Category

...
Playing With a Talk Box
by jefftk — 2022-02-04T02:50:05.105Z

Sometimes I describe my goal in playing around with different ways of
making music as trying to maximize my musical information rate: how
many bits per second? With my hands I'm generally playing keyboard or
mandolin, and they are very busy; compare to trumpet, where one hand
is choosing combinations of only three buttons and the other hand is
just holding the horn in position. My feet are generally occupied
with

drums, which
leaves the mouth as the remaining high-bandwidth option.

I've played with controlling a synthesizer
by whistling and using a breath
controller to control volume/expression, but recently I got a talk
box. This is a speaker with tube to pipe the output
of an instrument into your mouth, where you adjust the shape of the
mouth to filter the sound. They are most commonly used to make an
instrument pretend to talk, but you can also use it as an effects
pedal.

I'm still learning how to make a good use of it, but here's where I am
right now:

Comment via: facebook
How can I see a daily count of all words I type?
by Alex_Altair — 2022-02-04T02:05:04.483Z

I'd like to have a way to see a daily count of all words I type. What are some good technical solutions to this?

In the ideal case, it counts all words that I type in any application, whether I'm using my laptop keyboard or external keyboard, and also track words typed on my phone. It intelligently tracks backspacing and option-backspacing, et cetera. But that's asking for a lot. I'm also on Ubuntu, which could make this easier or harder.

But for compromises, are there keyloggers that are known to be really secure? Are there great browser extensions that could do this for me? Is there some kind of device that I could plug my keyboard into, which would just read the count directly from the stream of key press signals?
Covid 2/3/22: Loosening Bounds
by Zvi — 2022-02-03T23:00:01.542Z

In better discourse this week, I have a long reply to Scott Alexander’s Bounded Distrust. I also spun off another piece about a toolkit that’s advocating for schools to return to normal, The Urgency of Normal: An Exercise in Bounded Distrust.

As a result of all that, I ran out of time before I could get to everything this week, so this post does not include any coverage of The Convoy, which is getting pushed into either its own post or next week, depending on what I find when I dig into it more.

Executive Summary

Cases declining quickly.
My people are being let go in some places but not many yet, mostly slowly.
Looks like we’ve got us a convoy up in Canada as a protest, will cover later.

Let’s run the numbers.

The Numbers

Predictions

Prediction from last week: 2.85mm cases (-30%) and 20,000 deaths (+25%).

Results: 2.48mm cases (-39%) and 17,669 deaths (+10%)

Prediction for next week: 1.49mm cases (-40%) and 18,000 deaths (+2%).

Basically the whole nation is in rapid decline in cases so no reason not to expect the trend to continue another week. At some point the decline will slow, but it should be a while before there’s enough adjustments for too much of that. Deaths were a pleasant surprise after an unpleasant one, which usually means things are evening out from some wonkiness. The death peak should be approaching soon given the curve of cases. On reflection I do expect one final increase, and we might see two or possibly a small third one, but that should be it and I doubt we ever hit 20,000 in a week.

Deaths

Worth noting that recent rise in UK deaths is due to a backlog, actual new deaths are falling.

Also figured I’d post this here.

Cases

That’s an even bigger decline than I expected, and it is happening across 49 states and all four regions. You love to see it. Now that the trend is clear, no reason not to expect it to continue.

New York ER doctor who has given us several updates in the past reports no Covid patients his entire shift.

Vaccinations

You know. For kids. Pfizer is officially applying to the FDA to allow them to vaccinate children as young as six months.

This is vitally needed, because everyone involved is responding to the inability to vaccinate those children by acting completely crazy given that these children are at essentially zero risk.

Let’s be honest. We are not going to be v...
Forecasting Newsletter: January 2022
by NunoSempere — 2022-02-03T19:22:00.438Z

Highlights

Polymarket now operates out of Panama
Aver, a new Solana-based betting site, raised $7.5M in seed funding
Announcing the Forecasting Newsletter $10,000 Micro-Grants Program
Forecasting meetup next Wednesday (9th Feb) at 19:00 UTC in the LessWrong online garden

Index

Prediction Markets & Forecasting Platforms
Forecasting Newsletter Micro-Grants Program
Blog Posts

You can sign up for this newsletter on

substack

, or browse past newsletters

here

. If you have a content suggestion or want to reach out, you can leave a comment or find me on

Twitter

.

Prediction Markets & Forecasting Platforms

Polymarket

Since Polymarket settled with the CFTC last month (a), they have moved out of US jurisdiction. They are now operating from Panama (a). Their terms of use now prohibit users from the US from participating on the platform, and likewise prohibits use of a VPN to circumvent this prohibition (a). They have now started using UMA (a) as a market resolver. Personally, I am glad Polymarket has survived.

Manifold Markets

Manifold Markets (a), a new play-money prediction market, has kept its frantic development pace. They will be "launching" on February the 8th, from their current open beta. They outline some of their hopes and plans in their substack newsletter (a).

I created some markets on whether I will consider the Principles of Intelligent Behavior in Biological and Social Systems fellowship to be "a success" (a), on the number of subscribers to this newsletter (a), and on whether my current employer will still be alive by the end of 2022 (a). The process was extremely painless, and I recommend that readers give it a try.

CSET-Foretell is now INFER

CSET-Foretell moved from being hosted by CSET (a) at Georgetown to being hosted by ARLIS (a) at Maryland. ARLIS also received an $8M grant (a) from Open Philanthropy for forecasting specifically.

This is a fairly large amount of EA funding for forecasting projects. For reference, Metaculus recently received $250k (a), and Phil Tetlock and associates previously received two (a) ~$500k (a) grants. Open Philanthropy also previously gave ~$55M CSET (a), as well as later top-up (a) grants (a). But this was for a broader think tank, rather than for forecasting specifically.

In the December issue of this newsletter, I assessed that the move from CSET to ARLIS was probably a negative development, partially because I thought that ...
The Urgency of Normal: An Exercise in Bounded Distrust
by Zvi — 2022-02-03T18:10:00.683Z

[NOTE: This post, except for the introductory section here in brackets, was written continuously as I investigated the situation, then split off from the weekly post. Except for adding this first paragraph to provide context I did not go back and edit based on later observations. You can choose whether to look at the toolkit in question first, or whether to proceed as I did and look at it after. I hope people do a mix of those two things.]

This attack is an excellent example of the rules of bounded distrust, and what happens when the ‘experts’ say something that someone thinks the Narrative should continue to dislike, so they get turned on. What matters isn’t truth, what matters is whether you are being loyal to Narrative, or giving aid and comfort to ‘the enemy’ with wrongfacts.

I wrote that before looking at the toolkit, because I used the rules of Bounded Distrust or anything other than the Twitter thread above. This is a hit job, so one can assume this is the best they’ve got to hit them with. When ‘the first problem’ is that something is ‘being picked up by’ the wrong people, you make your True Objection very clear. It’s that this supports conclusions you don’t like, and you are in pure Soldier Mindset. When your second problem is not an error but warning that ‘when you misrepresent science to further a cause trust suffers’ you doth protest way too much. What you’re saying is ‘look what you made me do, forcing me to misrepresent the situation and forcing me to label you as providing misinformation. This will destroy trust.’

Or more bluntly, ‘Credentialed scientists going against other credentialed scientists will destroy trust in science, so you’re not allowed to do that.’ Or at a minimum, oh boy are we going to use isolated demands for rigor.

The last note, about them being ‘somewhat receptive’ to correcting errors makes it clear that they’re willing to correct such errors (‘somewhat’ being a clear insinuation word), but that this does not matter because accuracy never mattered anyway, only Narrative.

And then the kicker, that seals it all in place and shows exactly how this mindset works: “But this is a Pandora’s Box situation, and it will set us back in our collective goal of getting schools back to normal.”

Advocating for getting things back to normal, you see, is going to delay getting things back to normal, because now the wrong peop...