trentmkelly/LessWrong-43k huggingface dataset:
AI alignment landscape
by paulfchristiano — 2019-10-13T02:10:01.135Z

Here (link) is a talk I gave at EA Global 2019, where I describe how intent alignment fits into the broader landscape of “making AI go well,” and how my work fits into intent alignment. This is particularly helpful if you want to understand what I’m doing, but may also be useful more broadly. I often find myself wishing people were clearer about some of these distinctions.

Here is the main overview slide from the talk:

The highlighted boxes are where I spend most of my time.

Here are the full slides from the talk.
Prediction Markets Don't Reveal The Territory
by SebastianG  — 2019-10-12T23:54:23.693Z

[A draft section from a longer piece I am writing on prediction and forecasting. Epistemic Status: I don't know what I am missing, and I am filled with doubt and uncertainty.]

If the notion of professional forecasters disturbs you in your sleep, and you toss and turn worrying about the blight of experts brooding upon the world, perhaps the golden light of distributed information systems have peaked out from beyond these darkest visions, and you have hope for the wisdom of crowds.

Prediction markets aggregate information by incentivizing predictors to place bets on the outcomes of well-defined questions. Since information can be both niche and useful, prediction markets also incentivize the development of specialized expertise that is then incorporated into the general pool of information in the form of a bet. When this works, it works very well.

When information is not widely distributed or discoverable, prediction markets are not useful. Prediction markets for WMDs, Pope Francis’ next pronouncement, or which celebrity couple will choose to live in a van probably will not work. Or consider a public prediction market about what percent of the current freshman class at California public universities will make it to graduation. Such a market would be pretty distorted if all the registrars and admissions councillors were betting as well.

Prediction markets do have some wicked clever uses too. For example, a prediction market can also act as a proxy for some other event. That is to say, that through some ingenious design one can correlate a prediction market’s assessment of an event to another measurable outcome. Here is one instance in which researchers used a prediction market about the probability of war in Iraq, correlated it to the price of oil, and estimated the effect of war on oil prices. This provided the very useful information telling us what % of the price of oil is caused by the threat of war in Iraq. At an even broader level, this prediction market design allows us to study the effects of war on economies.

On the other hand, an additional limitation to prediction markets is that people have to be interested enough to take part in them, which is a real bummer. Intelligent quantitative people might enjoy researching to gain some betting leverage in prediction markets qua prediction markets. But even then, most people want to research que...
Planned Power Outages
by jefftk — 2019-10-12T14:10:01.395Z

With the dubiously motivated PG&E blackouts in California there are

many
stories

about
how

lack of
power

is
a serious problem, especially for people with medical dependencies
on electricity. Examples they give include people who:

Have severe sleep apnea, and can't safely sleep without a CPAP.

Sleep on a mattress that needs continous electricity to
prevent it from deflating.

Need to keep their insulin refrigerated.

Use a medicine delivery system that requires electricity every
four hours to operate.

This outage was dangerous for them and others, but it also seems like
a big problem that they're in a position where they need absolutely
reliable grid power. Even without politically motivated outages, the
grid isn't built to a standard of complete reliabilty.

There's an awkward valley between "reasonably reliable, but with a major
outage every few years in a storm or something" and "completely
reliable, and you can trust your life on it" where the system is
reliable enough that we stop thinking of it as something that
might go away but it's not so reliable that we should.

We can't get California out of this valley by investing to the point
that there won't be outages; earthquakes, if nothing else, ensure
that. So instead we should plan for outages, and make outages
frequent enough that this planning will actually happen.
Specifically:

Insurance should cover backup power supplies for medical
equipment, and they should be issued by default.

When there hasn't been an outage in ~1y, there should be a test
outage to uncover unknown dependencies.

While this outage was probably not done for good reasons, the problems
it has uncovered are ones we need to fix.
A simple sketch of how realism became unpopular
by Rob Bensinger — 2019-10-11T22:25:36.357Z

[Epistemic status: Sharing current impressions in a quick, simplified way in case others have details to add or have a more illuminating account. Medium-confidence that this is one of the most important parts of the story.]

Here's my current sense of how we ended up in this weird world where:

I still intermittently run into people who claim that there's no such thing as reality or truth;
a lot of 20th-century psychologists made a habit of saying things like 'minds don't exist, only behaviors';
a lot of 20th-century physicists made a habit of saying things like 'quarks don't exist, only minds';
there's a big academic split between continental thinkers saying (or being rounded off to saying) some variant of "everything is culture / perception / discourse / power" and Anglophone thinkers saying (or being rounded off to saying) "no".

Background context:

1. The ancient Greeks wrote down a whole lot of arguments. In many cases, we're missing enough textual fragments or context that we don't really know why they were arguing — what exact propositions were in dispute, or what the stakes were.

2. In any case, most of this is screened off by the fact that Europe's memetic winners were Christianity plus normal unphilosophical beliefs like "the sky is, in fact, blue".

3. Then, in 1521, the Protestant Reformation began.

4. In 1562, the Catholics found a giant list of arguments against everything by the minor Greek skeptic Sextus Empiricus, got very excited, and immediately weaponized them to show that the Protestant arguments fail (because all arguments fail).

5. These soon spread and became a sensation, and not just for being a useful superweapon. Plenty of intellectuals were earnest humanists used to taking arguments at face value, and found Sextus' arguments genuinely upsetting and fascinating.

I trace continental thinkers' "everything is subjective/relative" arguments back to a single 1710 error in George Berkeley:

[...] I am content to put the whole upon this Issue; if you can but conceive it possible for one extended moveable Substance, or in general, for any one Idea or any thing like an Idea, to exist otherwise than in a Mind perceiving it, I shall readily give up the Cause[....]

But say you, surely there is nothing easier than to imagine Trees, for instance, in

...
A Short Introduction to Theory of Change
by Ian David Moss — 2019-10-11T19:00:06.321Z

(Cross-posted from LinkedIn.)

At the heart of any strategy are two questions: what do we want to accomplish? And how are we going to do it?

In many situations, answering these questions might not seem difficult. We may already have a mission statement or set of values that guides all of our actions, addressing the first question. Likewise, we may already have a plan of action in place, a set of activities that seems to match the goals we’ve set out. Problem, meet solution. Done and done.

As intuitive as it is to imagine the beginning and the end of that process, though, all too often the devil is in the details—or more specifically in this case, all of the pesky steps in between. Figuring out what those ought to be takes real work, and is generally not something that can be done in one’s head. And because it takes work, a lot of times we don’t bother to do it.

Fortunately, there is a tool called theory of change that provides a means of figuring out all the steps. A theory of change is a visual depiction of your strategy. You probably already have a notion in your head of what your strategy is, but a theory of change gives you a means of articulating that strategy in a form of a diagram.

Why would you want to do this? Making your theory of change explicit accomplishes several things:

Developing a theory of change is a great way to get big questions about program or organizational strategy out in the open. In some cases, these questions might not have ever been considered before, or they’ve been thought about but never discussed among the team.
A finished theory of change diagram is a useful shorthand for explaining the mechanics of how a program works to someone who’s not that familiar with it. It’s not a marketing document, but it is a communication tool for people who want to understand your strategy in depth.
Theories of change are great for helping you decide what to measure. What information is most important for your program or organization to have? Are there any assumptions underlying your success that seem especially vulnerable or uncertain? Does the theory of change show that certain objectives have to be met before any others are possible? If so, you might want to focus measurement or tracking efforts on those objectives or assumptions.
Walking through a theory of change with new staff members can be a gr

...
Social Class
by jenn — 2019-10-11T18:46:44.228Z

Notes from the Salon

ETA: This is a write-up of some interesting points raised at an in-person rationalist meetup on October 6th, 2019. It is not intended to be a comprehensive overview of the topic. It is conventional for attendees to do all the suggested readings before salon starts, so some parts of the write-up might not make sense without that context.

Assigned readings: Siderea on Class, Thoughts on the "STEM" class

Economic Class vs. Social Class

Economic class and social class are not the same thing. The two are decoupled, but only partly. You can be wealthy but lower class (e.g. a skilled tradesman making 6 figures) or poor but upper class (e.g. an adjunct professor at a prestigious university). Everyone at the salon was in the same social class: the class that attends semi-ironic and entirely sober literary salons, likely G2 on Church's ladder. But there's a much broader distribution of economic classes represented in the room.

Performing upper-(middle)-classness grants you privilege. It costs some amount of money, but not as much as you would think, to perform professional-classness. The harder part by far is knowing what to buy in the first place. A clean-ish hoodie emblazoned with your university seal does the job. A blazer with a loud animal print pattern does not.

Most people want to make more money, but few people are interested in climbing the social ladder. When you find the class that you belong to and join it, you tend to go “ew” at the people in the classes below you, and “yikes”* at the people in the classes above you.

*Or the equivalent of “yikes”, if you’re in a class that doesn’t say “yikes”

Everyone is "Middle-Class"

The middle class is the only virtuous class - both "lower class" and "upper class" are insults. This results in basically everyone except billionaires and the homeless identifying as middle class. Premium mediocrity is what middle-classness could look like for millennials.

Class is a culture and a performance. For example, someone who makes six figures may choose to get their morning coffee at Tim Horton's instead of Starbucks or an indie coffee shop. They might make this decision even if they like their coffee not terrible, because it's more important to them to be seen as a salt-of-the-earth sort.

A more extreme example of this was outlined in this piece, where a Yale student who was actually broke didn't realize that all of he...
Rent Needs to Decrease
by jefftk — 2019-10-11T12:40:01.798Z

Here's part of a comment I got on my

housing coalitions post:

I consider it extremely unlikely you have found renters
with the expectation of rent going down. Assuming they want to live in
a well maintained building, I consider unlikely they even desire it,
once they think about it. What renters hope for in general is
increases that are less than their increases in income. Landlords
mostly do expect that rents will go up, but the magnitude of their
expectations matters, many have the same expectations as renters for
moderate increases. Others will have short term/transactional thinking
and will want to charge what the market will bear.

This seems worth being explicit about: when I talk about how I think
rents should be lower, I really mean lower. I'm not trying to say
that it's ok if rent keeps rising as long as incomes rise faster, but
that rents should go down.

Here are Boston rents in June 2011:

And in
June 2019:

These are on the same scale, though not adjusted for inflation (13%
from 2011-06 to 2019-06).

In 2011 a two-bedroom apartment in my part of Somerville would have
gone for $1800/month, or $2050/month in 2019 dollars. In 2019, it
would be $3000/month. Compared to 13% inflation we have 67% higher
rents.

Another way to look at this is that for what you would pay now for an
apartment a ten-minute walk from Davis would, in 2011, have covered an
apartment a ten-minute walk from Park St. And what you would have
been paying for a Harvard Sq apartment in 2011 wouldn't get you an
East Arlington apartment today.

These large increases have been a windfall for landlords.
Property taxes haven't risen much, upkeep is similar, but because
demand has grown so much without supply being permitted to rise to
meet it the market rent is much higher. If we build enough new
housing that rents fall to 2011 levels, landlords will make less money
than they had been hoping, but they'll still be able to afford to
keep up their properties.

I'll support pretty much any project that builds more bedrooms: market
rate, affordable, public, transitional. Rents are so high that all of
these would still be worth building and maintaining even if everyone
could see were were building enough housing that rents would fall next
year to 2011 levels.

As a homeowner and a landlord, I know that this means I would get less
in rent and I'm ok ...
Sets and Functions
by countedblessings — 2019-10-11T05:06:23.711Z

Sets and functions are two of the most basic ideas in mathematics, and we'll need to know what they are to discuss some things about categories rigorously. Normally you'd learn about sets and functions way before encountering category theory, but in the spirit of assuming as little math as possible, we should write this post. It's also worth addressing a few matters for their conceptual relevance.

Sets are imaginary bags we put things into. For example, you can take a dog, a cat, and a shoe, put them in an imaginary bag, and now you have a set consisting of {dog,cat,shoe}. The members of the set—dog, cat, and shoe—are called the elements of the set.

A subtle but important aspect of a set is that the imaginary bag has to be defined by a rule. This rule can be pretty much anything, like "put into a bag everything I'm pointing at," but it does have to be a rule. Typically, sets can fit pretty much anything in, and so you can often just say "here is my set" rather than having to be explicit about the rule. We'll get back to why the rule matters at the end. For now, sets are imaginary bags that you can put pretty much anything into.

What are we putting into these bags, exactly? Pretty much anything, yes—but clearly we aren't actually putting dogs, cats, and shoes into bags. Mathematically, what are these things?

That is to say, what's the difference between the set {dog} and the set {cat}?

Well, what's the difference between the equations x+2=3 and y+2=3? Nothing but the name of the variable—which does not matter at all. We could call x anything. We could represent it with a thirty-foot tall watercolor of a fire truck.

So what's the difference between the set {dog} and the set {cat}? Only the name of the element—which does not matter at all. Just like we can take posets like 1→2→3 and a→b→c and represent their common structure abstractly as ∙→∙→∙ , we can do the same for {dog} and {cat} with this set: {∙}.

The set {∙} is what a one-element set like {dog} or {cat} really is. There's no mathematical meaning to {dog} that actually makes the element of this set a four-legged barking creature. It's just an element that happens to be labeled "dog."

So why do we care about sets? Set theory is really important to mathematics, but from a category-theoretic perspective, they actually aren't very important at all. Instead, sets serve one major purpose: sets let us define functio...
Reflections on Premium Poker Tools: Part 4 - Smaller things that I've learned
by Adam Zerner — 2019-10-11T01:26:40.240Z

Previous posts:

Part 1 - My journey
Part 2 - Deciding to call it quits
Part 3 - What I've learned

In the previous post, I talked about what I've learned. That post focused on bigger things. But there were a lot of smaller, more miscellaneous things that I've learned too. Those are the things that I want to talk about in this post.

People think of a mobile app when you say you're building an "app"

Even when I clarify and try to explain that it's a web app, most people are still confused. So sometimes I call it a website, which I hate because that sort of implies that it's static. Sometimes I describe it as poker software. I still haven't found a good solution to this. I think "website" is probably best.

Ghosting is normal

This is a huge pet peeve of mine. I hate it. But apparently it's just a thing that many people do. At least in the business world. Let me give you some examples.

1) I reached out to this vlogger on Twitter. I asked if she'd be interested in working with me. She said she is. Then she proposed that I pay her as an advertiser to promote the app. I said in response that I'm only looking for revenue share partnerships right now, and asked if she's interested in that. No response. I follow up in a few days. No response. Follow up again. No response.

2) There was a guy who I would study poker with via Skype every week. I swear, we had a good relationship, and had productive study sessions. At some point he was going to be away for some trip for a few weeks, so we said we'd resume when he gets back. After the trip I reach out to set up a time for our next session. No response. I reach out again. No response. Again. No response. I eventually start spacing it out over months, but I never get a response from him. Eventually he signs up as my second paid user. I email him to thank him and ask if he wants to catch up. No response. At this point maybe he just feels too awkward to respond. I'm really confused though. I have no clue what happened.

3) There've been a few times where I'd try to set up lunches with people for advice. Many times the conversation would go:

Me: Want to get lunch?

Them: Sure, how about next week.

Me: Sounds good, what would be a good time for you?

Them: Silence.

Me: Just checking in for lunch next week.

Them: Silence.

Me: Still want to get lunch?

Them: Silence.

Me: Sorry we missed each other last week. Want to reschedule?

Them: Silence.

Them: Hey,

...
Reflections on Premium Poker Tools: Part 3 - What I've learned
by Adam Zerner — 2019-10-11T00:49:10.739Z

Previous posts:

Part 1 - My journey
Part 2 - Deciding to call it quits

I finally made the decision to call it quits. Now I think it would be a good time to reflect on my experiences and see if I could learn something from them.

Market size

As I talked about in the previous post, initially, I thought that the market size was hundreds of thousands of users, and that I could make something like $100/user. After talking to people in the industry, I now believe that the market is more like 5-10k users, and not all of them are willing to pay $100.

This is incredibly important! Going after a $20M market is very different from going after a $200k one. If I knew it was the latter in the beginning, I wouldn't have pursued this as a business. Spending 2+ years for the chance at making maybe $200k just isn't worth it, given the inherent uncertainty with startups, and the alternatives of pursuing a startup with a higher upside, or getting a job that pays approximately that much but does so with 100% certainty.

So what happened? Where did I go wrong? Let's see. This was (roughly) my initial logic:

100k subscribers to the poker subreddit. Educational YouTube videos gets 100k+ views. Popular posts on TwoPlusTwo (big poker forum) have 1M+ views over the years.

The kind of person to subscribe to the poker subreddit, watch an educational YouTube video, or spend their time on TwoPlusTwo is probably somewhat serious about poker. They're trying to get better at the game. All of this is indicative of a market size of hundreds of thousands of users. Possibly more. And poker is expanding. So there seems to be a big market here.

And poker players are probably pretty willing to spend money. Investing in software is +ROI. Poker players love ROI! And they tend to be on the wealthy side.

Maybe I really underestimated the divide between passive + free and active + paid. Watching a YouTube video is something that is passive. You sit there and consume information. Using poker software is something that is active. You have to sit and think and mess around with numbers. Watching a YouTube video is free. Poker software costs money.

But then what about the existence of poker books? There around 500 poker books on the market. The top ones get up to 100k sales, and many others get in the tens of thousands, I think. Maybe it's that with books, someone is telling you what the right answers are, but with softwa...