trentmkelly/LessWrong-43k huggingface dataset:
Voluntourism
by jefftk — 2019-12-17T00:40:01.552Z

Someone recently asked me what I thought of the idea of using extra
vacation days for volunteering, from an effective altruism
perspective. Here's what I wrote to them:

In the effective altruism community people tend to be relatively down
on this approach. It's relatively common for, say, a church group to
raise funds to fly a bunch of people to a poor country for a week or
two where they do unskilled manual labor (painting orphanages, picking
up trash, building housing). A typical effective altruist response
would be something like:

Poor countries are not lacking in workers!

The money spent on flying you there would easily have funded several
times as much work as you were able to do, and from much more
experienced people! Usually it makes more sense to use your job to
make things better (through donations or directly valuable work) and
use your vacation to make yourself happy.

The idea is, if you're just optimizing for helping others this doesn't
make sense.

There are two cases where I see this as more compelling, though:

When you're using skills that are
undersupplied where you're going to. Things like doctors traveling
to work at clinics, structural engineers reviewing building plans,
or programmers teaching people to code. The main downside is that
this can be similar to regular work, and so not be much of a
vacation! And most people's skills aren't a good fit for this.

When you mostly want to take a vacation but would enjoy helping some
along the way. If you compare voluntourism to regular tourism it
looks decent!

So, why is this common? Some guesses:

Traveling together and working hard together is really good
for building community. So church groups that organize this kind of
trip are likely to be stronger, more stable, and expand more.

Not only do these build community, they build committment and
motivation. People who've been on these trips are more likely to want
to help others go on them, and, I suspect, are more likely to
volunteer in ways that make their local community go better.

They also build connection. People who help make beds at a
clinic for a week might be more hassle than they're worth on average,
but if a few of them develop a personal connection to the clinic and
donate to it over the years it could easily be worth it for the clinic
to try to attract a lot of them.

Possibl...
T-Shaped Organizations
by Vaniver — 2019-12-16T23:48:13.101Z

Back in 2012, the Valve

New Employee Handbook

made the rounds on the internet, and claimed that Valve was looking for T-shaped individuals, who were broad generalists (the horizontal top of the T) with a depth of expertise in a particular topic (the vertical base of the T). I was reminded by this by thinking about

Your Price for Joining

. Eliezer argues that people (and skeptics / individualists in particular) likely overestimate the amount of ‘negotiating pull’ they should have in interacting with groups, and set standards that are too high, especially for incidental factors (like use of a subpar font or money storage method).

According to me, a T-shaped organization should be ‘broadly generalist’, in that it needs to do lots of different things in order to exist at all (like keep money in a bank, and comply with regulations, and have an office, and a website, and so on), and ‘narrowly expert,’ in that it performs exceptionally well on the functions related to succeeding at its primary mission. But with limited attention and resources, whenever the organization is given a choice between doing a lot better on a secondary factor or a little better on a primary factor, it should choose the primary factor. The organization should be

half-assing everything

; ideally in the way where the derivative of the goal with respect to each input is the same, and failing that hitting each quality threshold with minimal effort, and failing that being a trier at the primary goals and a slacker at all the other goals. The difference between a good bank and an okay bank probably has almost no effect on organizational success, whereas the difference between having a bank at all and having no bank has a huge effect on organizational success.

This means that you should think very differently about organizational performance on primary tasks and non-primary tasks. You almost want to cultivate a sort of

Gell-Mann Amnesia

, where you see a restaurant’s menu use a font no designer would be caught dead using and forget about it as actually not very informative about the quality of the food. You don’t want to forget it all the way, because underlying generators do cause things to be correlated; just as real budgeting constraints mean that money spent on the menu can’t be spent on the food, real variation in wealth and profitability means that some restaurants can spend more on both than oth

...
How’s that Epistemic Spot Check Project Coming?
by Elizabeth — 2019-12-16T22:50:01.675Z

Quick context: Epistemic spot checks started as a process in which I did quick investigations a few of a book’s early claims to see if it was trustworthy before continuing to read it, in order to avoid wasting time on books that would teach me wrong things. Epistemic spot checks worked well enough for catching obvious flaws (*cou*Carol Dweck*ugh*), but have a number of problems. They emphasize a trust/don’t trust binary over model building, and provability over importance. They don’t handle “severely flawed but deeply insightful” well at all. So I started trying to create something better.

Below are some scattered ideas I’m playing with that relate to this project. They’re by no means fully baked, but it seemed like it might be helpful to share them. This kind of assumes you’ve been following my journey with epistemic spot checks at least a little. If you haven’t that’s fine, a more polished version of these ideas will come out eventually.

A parable in Three Books.

I’m currently attempting to write up an investigation of Children and Childhood in Roman Italy (Beryl Rawson) (affiliate link) (Roam notes). This is very slow going, because CaCiRI doesn’t seem to have a thesis. At least, I haven’t found one, and I’ve read almost half of the content. It’s just a bunch of facts. Often not even syntheses, just “Here is one particular statue and some things about it.” I recognize that this is important work, even the kind of work I’d use to verify another book’s claims. But as a focal source, it’s deadly boring to take notes on and very hard to write anything interesting about. What am I supposed to say? “Yes, that 11 year old did do well (without winning) in a poetry competition and it was mentioned on his funeral altar, good job reporting that.” I want to label this sin “weed based publishing” (as in, “lost in the weeds”, although the fact that I have to explain that is a terrible sign for it as a name).

One particular bad sign for Children and Childhood in Roman Italy was that I found myself copying multiple sentences at once into my notes. Direct quoting can sometimes mean “there’s only so many ways to arrange these words and the author did a perfectly good job so why bother”, but when it’s frequent, and long, it often means “I can’t summarize or distill what the author is saying”, which can mean the author is being vague, eliding over important points, or letti...
To Be Decided #4
by Ian David Moss — 2019-12-16T18:41:27.983Z

TBD is a quarterly-ish newsletter about deploying knowledge for impact, learning at scale, and making more thoughtful choices for ourselves and our organizations.

On Stories vs. Data

The reason stories work for us as human beings is because they are few in number. We can spend two hours watching a documentary, or a week reading a history book, and get a deep qualitative understanding of what was going on in a specific situation or in a specific case. The problem is that we can only truly comprehend so many stories at once. We don’t have the mental bandwidth to process the experiences of even hundreds, much less thousands or millions of subjects or occurrences. To make sense of those kinds of numbers, we need ways of simplifying and reducing the amount of information we store in each case. So what we do is we take all of those stories and we flatten them: we dry out all of the rich shape and detail that makes up their original form and we package them instead in a kind of mold: collecting a specific and limited set of attributes about each so that we can apply analysis techniques to them in batch. In a very real sense, data = mass-produced stories.

It sounds horrible when I put it like that, right? But make no mistake: stories are never incompatible with data. When you or someone you know has an indelible experience at a spiritual retreat, or when a child’s life is saved through involvement with your nonprofit, or when people are brought together who wouldn’t otherwise meet because of an event you organized, those are all great stories — and they’re also data.

(

Keep reading

)

What I've Been Reading

Fostering Government Evidence Use in the Global South
Our first item is a celebration of Results for All, a global initiative to improve the use of evidence in government decision-making led by Abeba Taddese. Over the course of four years, Results for All produced a panoply of information resources worth checking out, including a landscape analysis of 100+ government initiatives to accelerate evidence use in policymaking, case studies of some very creative incentive programs to encourage better evidence use, a collection of 50+ evidence-sharing networks, a survey of funders who support evidence in international development, and more. Sadly, Results for All shut down and laid off its staff...
How time tracking can help you prioritize
by lynettebye — 2019-12-16T17:07:17.045Z

This piece is cross-posted on my blog here.

Ann told her supervisor that she would get a draft of her research proposal to them by Friday. However, it’s Sunday afternoon and Ann is just now finishing up her draft. She has to force herself to even look at the draft because Ann is dreading her supervisor being disappointed that she is late again.

She finally sends in the draft Sunday evening. However, she’s not happy with it. She can picture the improved version she could have created with just a couple more hours of work. She determines that next time she will meet the deadline, come hell or high water.

I would be surprised if she meets the next deadline.

I don’t doubt Ann’s resolve to meet the next deadline. I’m sure she truly wants to do so. But she isn’t changing any of the processes that lead her to be late. She has a track record of missing the deadlines she sets for herself, so the issue probably won’t go away by default. Yet she hasn’t identified why she missed the last deadline or made a different plan to correct the issues.

If Ann consistently misses deadlines, she needs to try a new approach - more determination rarely works. In this case, she probably needs better prioritization, more frequent and more realistic deadlines, and some more motivation.

For these issues, Ann would probably benefit from a good outside view on how long similar tasks take. But that information is hard to remember off the top of her head. However, if Ann started tracking her time for a few weeks, I expect the story would continue something like this:

Ann started tracking her time and was surprised at how long it took to run her experiment. The first step had an error in the data collection, so she had to redo it. Then it took her twice as long to write the introduction to her paper as she expected. At the end of the project, it had taken her twice as long as she had originally planned, but she thought it probably took longer than normal. Other projects would go faster.

After two more projects, she decided that it actually was normal for projects to take that long. So for her next paper, she quickly drafted the entire introduction, instead of spending hours trying to get each word perfect. She had less time to polish it up afterwards than she had hoped, but moving the ideal-but-not-necessary steps to the end meant she ha...
A parable in the style of Invisible Cities
by Daniel Kokotajlo — 2019-12-16T15:55:06.072Z

Cities & Belief: Goreme

The city of Goreme hides a terrible secret. At first, innocent visitor, you will admire the old bridge over the river, the colorful graffiti on alley walls, and the flags waving prettily in the public square. You will be impressed and humbled by how passionate and well-informed the locals are, and you will sign up to attend one of their protest-rallies.

Eventually, however, you will see the demons. A local guide will casually pull one out of his ear and offer it to you, or a dinner party host will unwrap a boxful. Picture it! With supernatural speed the tiny, squirmy yellow creatures clamber up your leg and perch on your shoulder. They whisper to you, and try to enter your ear canal. You wipe them off in disgust, but your local friends assure you the things are harmless—all they do is talk, and they are quite helpful. There are good demons and bad demons, your friends explain, and the good ones help you resist the bad. The bad ones have infected other people in the city—selfish people, hateful people, and also people who just don't know any better. The bad demons want power; they lie and deceive their hosts so that they can multiply and spread to more people. Demons are very persuasive, very clever. This is why the good demons are so helpful, your friends say. The good demons care about humans; they can help you identify bad demons inside other people, and they'll provide facts and arguments to counter enemy misinformation.

Does this story disturb you, traveller? Are you suspicious of these good demons? Good. You should be. Most visitors to Goreme are, at least at first. Eventually, though, everyone lets at least a few demons into their head. Hard to believe, but I assure you it is true. The demons are very persuasive, especially when all your friends agree with them. All in all, it isn't so bad. The city of Goreme is run by warring tribes of demons, yes, but they run it pretty much just like any other city.

This brings me back to the beginning: I said that Goreme hides a terrible secret. You are probably thinking the demons are the secret. No. The secret is this: The difference between Goreme and every other city is that in Goreme the demons are visible.
Taxi Industry Regulation, Deregulation, and Reregulation: The Paradox of Market Failure
by Orborde — 2019-12-16T08:05:01.044Z
Link: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2241306

(from the March 2018 Gwern.net newsletter)

Uber/Lyft are disrupting the near-total regulatory capture of the taxi industry. But why does that regulatory mess exist in the first place? As it turns out, the regulations are trying to solve a bunch of specific market failures in the legacy taxi industry, and this paper thoroughly explains what those failures are.
Boston Solstice 2019 Retrospective
by jefftk — 2019-12-15T21:20:01.767Z

This was the second time we hosted the Boston Secular Solstice at our
house (

2018
retrospective), and my first time running it. I'd been in a
"music director" sort of role for Boston Solstices going back to 2013,
but with our main organizers now living in the Bay Area I decided to
have a go at putting something together myself.

I decided pretty early that I wanted to build something around themes
of coordination across boundaries and existential risk. I thought
Christmas in the Trenches would potentially be good for the
dark portion, either sung solo or read like a poem, as well as maybe
reading letters people had sent home from the front.

In September I picked the songs and wrote a new melody for
Somebody Will. I recorded rough demos so I could show
people how I was thinking the songs would go, and started trying to
find people who were interested in helping play and sing. Johnson
volunteered to play piano for ~5 songs, we did two unaccompanied, and
I did the rest. Unlike previous years I decided not to have a leader
for each song, and instead we'd get a group of people to learn all the
songs and be distributed around the audience. I felt like this worked
pretty well, and made for more of a community sort of feeling. We also
got together for a full music rehearsal, something I'd wanted to do in
previous years but generally hadn't been able to schedule, and am very
glad we did.

About two weeks out I wrote transition words and picked (what I
thought) were final readings, and sent around a draft to people for
review. Al gave me an enormous number of good suggestions, and in
response I completely swapped around the order of songs and wrote new
transitions that better showed how everything fit into a coherent
whole. Al also found new first-person readings to replace the adapted
version of the World
on the Brink readings we'd been intending to use. This was
incredibly useful, and the to top it off they volunteered to MC, which
was awesome.

I tried a new layout, without any board-benches, and I think it worked
well:

It helped that we hadn't gotten our Christmas tree yet, so we had a
bit of extra space.

Johnson came early and helped set up, which was super helpful. We
took the apartment door off its hinges,
disassembled the dining room table, moved two futons into the dining
room, set up tons of chairs, and generally tri

...
Approval Extraction Advertised as Production
by Benquo — 2019-12-15T14:51:44.878Z
Link: http://benjaminrosshoffman.com/approval-extraction-advertised-as-production/

Paul Graham has a new essay out, The Lesson to Unlearn, on the desire to pass tests. It covers the basic points made in Hotel Concierge's The Stanford Marshmallow Prison Experiment. But something must be missing from the theory, because what Paul Graham did with his life was start Y Combinator, the apex predator of the real-life Stanford Marshmallow Prison Experiment. Or it's just false advertising.

As a matter of basic epistemic self-defense, the conscientious reader will want to read the main source texts for this essay before seeing what I do to them:

The Lesson to Unlearn
The Stanford Marshmallow Prison Experiment
Sam Altman's Manifest Destiny
Black Swan Farming
Sam Altman on Loving Community, Hating Coworking, and the Hunt for Talent

The first four are recommended on their own merits as well. For the less conscientious reader, I've summarized below according to my own ends, biases, and blind spots. You get what you pay for.

The Desire to Pass Tests hypothesis: a Brief Recap

The common thesis of The Lesson to Unlearn and The Stanford Marshmallow Prison Experiment is:

Our society is organized around tests imposed by authorities; we're trained and conditioned to jump through arbitrary hoops and pretend that's what we wanted to do all along, and the upper-middle, administrative class is strongly selected for the desire to do this. This is why we're so unhappy and so incapable of authentic living.

Graham goes further and points out that this procedure doesn't know how to figure out new and good things, only how to perform for the system the things it already knows to ask for. But he also talks about trying to teach startup founders to focus on real problems rather than passing Venture Capitalists' tests.

The rhetoric of Graham's essay puts his young advisees in the role of the unenlightened who are having a puzzling amount of trouble understanding advice like "the way you get lots of users is to make the product really great." Graham casts himself in the role of someone who has unlearned the lesson that you should just try to pass the test of whoever you're interacting with, implying that the startup accelerator (i.e. combination Venture Capital firm and cult) he co-founded, Y Combinator, is trying to do something outside the domain of Tests.

In fact, Graham's behavior as an investor has perpetua...
When would an agent do something different as a result of believing the many worlds theory?
by mako yass — 2019-12-15T01:02:40.952Z

One of the things impeding the many worlds vs wavefunction-collapse dialogue is that nobody seems to be able to point to a situation in which the difference clearly matters, where we would make a different decision depending on which theory we believe. If there aren't any, pragmatism would instruct us to write the question off as meaningless.

Has anyone tried to pose a compelling thought experiment in which the difference matters?
Is the term mesa optimizer too narrow?
by Matthew Barnett — 2019-12-14T23:20:43.203Z

In the post introducing mesa optimization, the authors defined an optimizer as

a system [that is] internally searching through a search space (consisting of possible outputs, policies, plans, strategies, or similar) looking for those elements that score high according to some objective function that is explicitly represented within the system.

The paper continues by defining a mesa optimizer as an optimizer that was selected by a base optimizer.

However, there are a number of issues with this definition, as some have already pointed out.

First, I think by this definition humans are clearly not mesa optimizers. Most optimization we do is implicit. Yet, humans are the supposed to be the prototypical examples of mesa optimizers, which appears be a contradiction.

Second, the definition excludes perfectly legitimate examples of inner alignment failures. To see why, consider a simple feedforward neural network trained by deep reinforcement learning to navigate my Chests and Keys environment. Since "go to the nearest key" is a good proxy for getting the reward, the neural network simply returns the action, that when given the board state, results in the agent getting closer to the nearest key.

Is the feedforward neural network optimizing anything here? Hardly, it's just applying a heuristic. Note that you don't need to do anything like an internal A* search to find keys in a maze, because in many environments, following a wall until the key is within sight, and then performing a very shallow search (which doesn't have to be explicit) could work fairly well.

As far as I can tell, Hjalmar Wijk introduced the term "malign generalization" to describe the failure mode that I think is most worth worrying about here. In particular, malign generalization happens when you trained a system with objective function X, that at deployment has the actual outcome of doing Y, where Y is so bad that we'd prefer the system to fail completely. To me at least, this seems like a far more intuitive and less theory-laden way of framing inner alignment failures.

This way of reframing the issue allows us to keep the old terminology that we are concerned with capability robustness without alignment robustness, but drops all unnecessary references to mesa optimization.

Mesa optimizers could still form a natural class of things that are prone to malign generaliz...