trentmkelly/LessWrong-43k huggingface dataset:
Saint Louis Meeting Notes 11/23/2019
by SebastianG  — 2019-11-24T21:36:20.093Z

In attendance: 6 people

Collective Food: Cashew Chicken and Rice, Devilled Eggs, Pumpkin Bread, Apple Pie, Apple Cider Cinnamon Ice Cream, Cheese and Crackers, Tea, Pepsi, Red and White Wine

1. Member Presented Argument: Are Podcasts locking away information?

a. Podcasts take content away from the written word, which is a more efficient way to consume information - all else being equal.

b. Without transcripts podcasts are not term searchable.

c. The momentary convenience of podcasts while walking and driving diminishes the collective pool of good written content, and good written content has more staying power than a good podcast episode.

2. Conversation: Practical knowledge in science

a. Blacksmiths and Engineers

i. Iron Working, Guild and Trades, etc. functioned through an unbroken apprentice system. It takes a blacksmith to make a blacksmith. The non-academic approach makes learning what was done at any particular time difficult. High iteration processes generally have this feature. What fields today are very difficult to learn about without doing?

ii. NASA’s Apollo Program was so “do engineering” focused that it took a long time after the fact to figure out exactly how all the parts fit together.

iii. Nuclear scientists today are monitored so that their know-how is put to use in approved countries, same for WMD specialists.

iv. International Students in the US have limited access to highly advanced fields and the most cutting edge. Sometimes for National Security reasons, but mostly for the reason of patent protection and intellectual property rights for professors and grant making agencies.

v. How can we make sure that knowledge passed on this way does not get lost? Post-mortems? Proper Latin: Post Mortes?

3. Presented Lecture: The Grand Plan of Rome by Giambattista Nolli, Art and Science of

a. Commissioned by Pope Benedict XIV to district the city into political wards. The 1757 map employs New techniques of cartography and surveying. This is the first map comparable to the accuracy of Google Maps.

i. New Idea: Measurement of everything.

ii. Figure-Ground distinction for depiction of buildings

iii. Floor Plans of Public Spaces with exact colonnades.

iv. Data collection of all agricultural fields; the fields are depicted on map with the crop type visually in

...
Explaining why false ideas spread is more fun than why true ones do
by Matthew Barnett — 2019-11-24T20:21:50.906Z

As typical for a discussion of memes (of the Richard Dawkins variety), I'm about to talk about something completely unoriginal to me, but that I've modified to some degree after thinking about it.

The thesis is this: there's a tendency for people to have more interest in explaining the spread of ideas they think are false, when compared to ideas they think are true.

For instance, there's a lot written about how and why religion spread through the world. On the other hand, there's comparatively little written about how and why general relativity spread through the world. But this is strange -- they are both just ideas that are spread via regular communication channels.

One could say that the difference is that general relativity permits experimental verification, and therefore it's no surprise that it spread through the world. The standard story here is that since the idea is simply true, the explanation for why it became widespread is boring -- people merely became convinced due to its actual veracity.

I reject this line of thought for two reasons. First, the vast majority of people don't experimentally verify general relativity, or examine its philosophical basis. Therefore, the mechanism by which the theory spreads is probably fairly similar to religion. Secondly, I don't see why the idea being true makes the memetic history of the idea any less interesting.

I'm not really sure about the best explanation for this effect -- that people treat true memes as less interesting than false ones -- but I'd like to take a guess. It's possible that the human brain seeks simple single stories to explain phenomena, even if the real explanation for those phenomena are due to a large number of factors. Furthermore, humans are bored by reality: if something has a seemingly clear explanation, even if the speaker doesn't actually know the true explanation, it's nonetheless not very fun to speculate about.

This theory would predict that we would be less interested in explaining why true memes spread, because we already have a readily available story for that: namely, that the idea is true and therefore compels its listeners to believe in it. On the other hand, a false meme no longer permits this standard story, which forces us to search for an alternative, perhaps exciting, explanation.

One possible takeaway is that w...
RAISE post-mortem
by  — 2019-11-24T16:19:05.163Z

Edit November 2021: there is now the Cambridge AGI Safety Fundamentals course, which promises to be successful. It is enlightening to compare this project with RAISE. Why is that one succeeding while this one did not? I'm quite surprised to find that the answer isn't so much about more funding, more senior people to execute it, more time, etc. They're simply using existing materials instead of creating their own. This makes it orders of magnitude easier to produce the thing, you can just focus on the delivery. Why didn't I, or anyone around me, think of this? I'm honestly perplexed. It's worth thinking about.

Since June, RAISE has stopped operating. I’ve taken some time to process things, and now I’m wrapping up.

What was RAISE again

AI Safety is starved for talent. I saw a lot of smart people around me that wanted to do the research. Their bottleneck seemed to be finding good education (and

hero licensing

). The plan was to alleviate that need by creating an online course about AI Safety (with nice diplomas).

How did it go

We spent a total of ~2 years building the platform. It started out as a project based on volunteers creating the content. Initially, many people (more than 80) signed up to volunteer, but we did not manage to get most of them to show up consistently. We gradually pivoted to

paying people

instead.

We received a lot of encouragement for the project. Most of the enthusiasm came from people wanting to learn AI Safety. Robert Miles joined as a lecturer. When we reached out to some AI Safety researchers for suggestions on which topics to cover, we readily received helpful advice. Sometimes we also received some funds from a couple of prominent AIS organizations who thought the project could be high value, at least in expectation.

The stream of funding was large enough to sustain about 1 fte working for a relatively low wage. Obtaining it was a struggle: our runway was never longer than 2 months. This created a large

attention sink

that made it a lot harder to create things. Nearly all of my time was spent on overhead, while others were creating the content. I did not have the time to review much of it.
About 1 year into the project, we escaped this poverty trap by moving to the EA Hotel and starting a content development

team

there. We went up to about 4 fte, and the production rate shot up leading to an

MVP

relatively quickly.

How did it end

Before launc...
Solution to the free will homework problem
by Erik Jenner — 2019-11-24T11:49:51.072Z

At the last meetup of our local group, we tried to do Eliezer's homework problem on free will. This post summarizes what we came up with.

Debates on free will often rely on questions like "Could I have eaten something different for breakfast today?". We focused on the subproblem of finding an algorithm that answers "Yes" to that question and which would therefore - if implemented in the human brain - power the intuitions for one side of the free will debate. We came up with an algorithm that seemed reasonable but we are much less sure about how closely it resembles the way humans actually work.

The algorithm is supposed to answer questions of the form "Could X have happened?" for any counterfactual event X. It does this by searching for possible histories of events that branch off from the actual world at some point and end with X happening. Here, "possible" means that the counterfactual history doesn't violate any knowledge you have which is not derived from the fact that that history didn't happen. To us, this seemed like an intuitive algorithm to answer such questions and at least related to what we actually did when we tried to answer them but we didn't justify it beyond that.

The second important ingredient is that the exact decision procedure you use is unknown to the part of you that can reason about yourself. Of course you know which decisions you made in which situations in the past. But other than that, you don't have a reliable way to predict the output of your decision procedure for any given situation.

Faced with the question "Could you have eaten something different for breakfast today?", the algorithm now easily finds a possible history with that outcome. After all, the (unknown) decision procedure outputting a different decision is consistent with everything you know except for the fact that it did not in fact do so - which is ignored for judging whether counterfactuals "could have happened".

Questions we haven't (yet) talked about:

Does this algorithm for answering questions about counterfactuals give intuitive results if applied to examples (we only tried very few)? Otherwise, it can't be the one used by humans since it would be generating those intuitions if it were
What about cases where you can be pretty sure you wouldn't choose some action witho

...
New MetaEthical.AI Summary and Q&A at UC Berkeley
by June Ku — 2019-11-24T01:47:39.117Z

Previous Intro: Formal Metaethics and Metasemantics for AI Alignment

I’m nearing the completion of a hopefully much more readable version of the ideas previously released as set-theoretic code. This takes the form of a detailed outline, currently in WorkFlowy, in which you can easily expand/collapse subsections which elaborate on their parents’ content. You can find the current draft here.

Although it’s not polished, I’m releasing it in preparation for a Q&A I’ll be holding at the University of California Berkeley AI and Philosophy working group, which I hope you will attend. I’ll likely make some brief introductory remarks but reserve most of the time for answering questions. The working group is part of the UC Berkeley Social Science Matrix and will be held at:

Barrows Hall, 8th Floor, Mezzanine Level
Wed, Dec 4th 12:30-2:30pm
(only the first hour is reserved for this Q&A)

Here I’ve reproduced just the first few levels of the outline. Click here to see their elaboration (currently ~4,800 words).

Given mathematical models of the world and the adult human brains in it, an ethical goal function for AI can be constructed by applying a social welfare function to the set of extensional rational utility functions of the brains.

The mathematical model of a world or brain is to be given as a causal Markov model.

A causal Markov model is a convenient model for generating a causal model.

The notion of a causal model is taken directly from Judea Pearl.

A causal model is composed of:

A causal Markov model is composed of:
A causal Markov model (cmm) generates a causal model (cm) as follows:

A brain’s rational utility function is the utility function that would be arrived at by the brain’s decision algorithm if it were to make more optimal decisions while avoiding unrelated distortions of value.

A brain’s decision algorithm is the one that best satisfies these desiderata:

First, it must take the mathematical form of a decision algorithm, which is a tuple composed of:
Next, there must be an implementation function which maps brain states to decision states such that these two routes from a brain state to a decision event always arrive at the same result:
It achieves a high rate of compression of the brain’s causal transition function.
It is probabilistically coherent, including with its represented c

...
Thoughts on Robin Hanson's AI Impacts interview
by Steven Byrnes — 2019-11-24T01:40:35.329Z

There was already a LessWrong Post here. I started writing this as a comment there, but it got really long, so here we are! For convenience, here is the link to interview transcript and audio, in which he argues that AGI risks are modest, and that EAs spend too much time thinking about AGI. I found it very interesting and highly recommend reading / listening to it.

That said, I disagree with almost all of it. I'm going to list areas where my intuitions seem to differ from Robin's, and where I'm coming from. Needless to say, I only speak for myself, I'm not super confident about any of this, and I offer this in the spirit of "brainstorming conversation" rather than "rebuttal".

How likely is it that the transition to superhuman AGI will be overwhelmingly important for the far future?

Robin implies that the likelihood is low: "How about a book that has a whole bunch of other scenarios, one of which is AI risk which takes one chapter out of 20, and 19 other chapters on other scenarios?" I find this confusing. What are the other 19 chapter titles? See, in my mind, the main categories are that (1) technological development halts forever, or (2) AGI is overwhelmingly important for the far future, being central to everything that people and societies do (both good and bad) thereafter. I don't immediately see any plausible scenario outside of those two categories ... and of those two categories, I put most of the probability weight in (2).

I assume Robin would want one of the 20 chapters to be about whole-brain emulation (since he wrote a whole book about that), but even if whole-brain emulation happens (which I think very unlikely), I would still expect fully-artificial intelligence to be overwhelmingly important in this scenario, as soon as the emulations invent it—i.e. this would be in category 2. So anyway, if I wrote a book like that, I would spend most of the chapters talking about AGI risks, AGI opportunities, and what might happen in a post-AGI world. The rest of the chapters would include things like nuclear winter or plagues that destroy our technological civilization forever. Again, I'm curious what else Robin has in mind.

How hard is it to make progress on AGI safety now? How easy will it be in the future?

I could list off dozens of specific open research problems in AGI safety where (1) we can make real progress right now; (2) we are making real progress r

...
Acting without a clear direction
by Chris_Leong — 2019-11-23T19:19:11.324Z

One of the key questions that we all face is to figure out a purpose in life, a direction, a goal. However, that is not an easy question. In fact, it's not even easy to say what kind of thing human values are in general. Our brain is fragmented in so many different ways: past, present and future preferences; emotional, intuitive and cognitive systems and multiple layers of meta-preferences. Given the tangle of confusion and that a solution that seems right for me might not seem right for you, I would suggest that finding a pragmatic approach to this problem might be even more important than actually trying to solve it.

I would propose that contrary to current rationalist wisdom trying to pull some kind of consistent utility function out of this can be counterproductive. I've honestly burned up far too many brain cycles trying to do this; sometimes there is value in just doing something and forgetting about optimality. After all, utility pumps are quite rare and people tend to catch on when they are being pumped anyway.

Consider the following: What should we optimise for personal utility or our values? Assume that we include the utility we gain from achieving our values in personal utility. If you believe in moral realism, then you have an obvious reason to pursue your values even when it doesn't benefit you, but what about otherwise? Should you take the self-centered approach of only caring about your values insofar as they seem likely to provide you with utility?

Your hedonic component (or components)would be quite satisfied with this solution, but the part of you that has values outside of yourself would not be. Each part self-affirms its own viewpoint. If we have no real resolution about which part deserves precedent, then a sensible default would be to assign value to both.

This gives us a reason to move past pure hedonism (or hedonism + values as instrumental for hedons), but do we have a reason to go any further? After all, there's a significant difference between merely attempting to realise your other-directed values and being deeply committed to achieving them.

Maybe we don't have any reason from a principled perspective, so I suppose we'll now have to fallback to the instrumental (and admittedly self-directed) perspective. Firstly, if we aren't committed to a goal, we'll be unlikely to achieve it even when we easi...
Market Rate Food Is Luxury Food
by jefftk — 2019-11-23T15:10:02.344Z

We're in the middle of a major food crisis. It is so expensive that
people have little left for shelter, clothing, or other necessities.
We cannot let this continue, but that doesn't mean every proposal is a
good one. Specifically, I want to address the extreme position of
eliminating

marketing
orders and allowing unrestricted production of market rate food.
This has historically been pushed by farmworkers, who are far from
unbiased, but lately some "

Yes In My BarnYard"
farmer groups have been advocating for this as well.

Let me be clear: new market rate food is luxury food. You cannot
solve a problem that disproportionately affects low-income people by
producing more food they clearly cannot afford. The market cannot
grow its way out of this crisis.

With deregulation, farmers would massively shift to luxury crops, and
we would have shortages of bread, milk, eggs, and other staples.
While high-margin crops like nuts, oranges, and arugula would get
somewhat cheaper, that's no help to low-income families that can
barely afford the basic calories they need.

While YIMBYs claim insufficient supply is the underlying problem, the
real issues are much more complex than you learn in Econ 101. Lack of
supply is only a symptom of a fundamentally broken system where food
is a commodity, for sale to the highest bidder. We cannot leave
something as fundamental as food to capitalism.

Since deregulation is clearly not the answer, what do we do instead?
The number one thing we need is more and better public food. Public
Food Authorities provide a critically important service for
food-insecure people, but the Faircloth
limit caps production at October 1, 1999 levels. Our PFAs are
also chronically underfunded for the vital work they do, and are not
able to the produce the nutritious food our low-income families
deserve. We need to remove the cap, and reverse decades of
underinvestment and neglect.

We should also fully fund SNAP.
The waiting-list for benefits can be multiple years, which is
incredibly damaging. We also need to fully enforce the Small Area Fair
Market Food rule to make sure that grocers are fairly compensated for
their participation in SNAP, but do not make a windfall from the
program.

We also need far more affordable
food. Most regions still do not require new market-rate farms to
reserve any of their production for low-in...
Gears-Level Models are Capital Investments
by johnswentworth — 2019-11-22T22:41:52.943Z

Mazes

The usual method to solve a maze is some variant of

babble-and-prune

: try a path, if it seems to get closer to the exit then keep going, if it hits a dead end then go back and try another path. It's a black-box method that works reasonably well on most mazes.

However, there are

other methods

. For instance, you could start by looking for a chain of walls with only one opening, like this:

This chain of walls is a

gears-level insight

into the maze - a piece of the internal structure which lets us better understand “how the maze works” on a low level. It’s not specific to any particular path, or to any particular start/end points - it’s a property of the maze itself. Every shortest path between two points in the maze either starts and ends on the same side of that line, or passes through the gap.

If we only need to solve the maze once, then looking for a chain of walls is not very useful - it could easily take as long as solving the maze! But if we need to solve the same maze more than once, with different start and end points… then we can spend the time finding that chain of walls just once, and re-use our knowledge over and over again. It’s a capital investment: we do some extra work up-front, and it pays out in lower costs every time we look for a path through the maze in the future.

This is a general feature of gears-level models: figuring out a system’s gears takes extra work up-front, but yields dividends forever. The alternative, typically, is a black-box strategy: use a method which works without needing to understand the internals of the system. The black-box approach is cheaper for one-off tasks, but usually doesn’t yield any insights which will generalize to new tasks using the same system - it’s context-dependent.

Marketing

Suppose we work with the marketing team at an online car loan refinance company, and we're tasked with optimizing the company's marketing to maximize the number of car loans the company refinances. Here's two different approaches we might take:

We

a/b test

hundreds of different ad spend strategies, marketing copy permutations, banner images, landing page layouts, etc. Ideally, we find a particular combination works especially well.
We obtain some anonymized data from a credit agency on people with car loans. Ideally, we learn something about the market

...
Solar One Year In
by jefftk — 2019-11-22T15:20:01.839Z

Last fall we had solar panels installed. Our roof is pretty marginal
for solar, large parts blocked by trees and the remainder mostly
facing West, but incentives were high enough that it looked decent.
And even if it only broke even I still liked it for the

resiliency advantages. It's
currently doing slightly better than expected, so I'm happy!

We have fourteen LG Neon-R 360 watt panels, three facing South and
eleven facing West:

The installers predicted we would see 3.45MWh in the first year,
slowly ramping down to 3.25MWh/y over the next twenty five years as
the panels degrade. After a year of operation, we've seen 3.95MWh.
This includes three short periods in March, April, and August where
the system turned itself off and needed to be reset by the installer.
This is covered by our extended warranty, but does make me nervous
about its long-term robustness.

One of the bigger risks I saw going into this was that the installers
had a lot of incentive to overestimate our production, so I'm glad
that while it was only in operation about 96% of the year it still
made ~14% more power than expected.

Four MWh is not a huge amount of electricity, however. At wholesale
rates of ~$40/MWh (pdf)
this would be just $160. MA has net metering, however, which means we
can subtract the power generated from our bill. Our (retail) rate is
$214/MWh, so this is $844 for the year. The whole system cost $26k,
so this doesn't break even within the system's lifetime, and that'a
ignoring opportunity cost.

There are additional incentives, however: a 30% federal tax credit, a
$1k MA tax credit, and SREC II generation incentives. The tax credits
bring the effective cost down to $17k, and the generation incentives
give us ~$500/y-$700/y for the first ten years. This gives 11-13
years until payback, and if we figure a 25 year lifetime (the length
of the warranty) and electricity going up 3%/y it's equivalent to an
investment returning a nominal ~9%.

So while I don't think rooftop solar makes much sense for society
(unless we required systems to function during power outages, which we
don't) and our roof is marginal for solar, it does look like our
system is working out from a self-interested perspective.

Comment via: facebook