trentmkelly/LessWrong-43k huggingface dataset:
Results of $1,000 Oracle contest!
by Stuart_Armstrong — 2020-06-17T17:44:44.566Z

Almost a year ago, I posted a contest to find the best questions to ask an Oracle AI. I've been very slow in grading it, apologies for that, but here are the final results and the sumptuous cash prizes.

First of all, thanks for everyone who wrote an entry; even if your suggestion didn't win, most of them had interesting ideas that made me think. But here are the prizes:

Counterfactual Oracle

$350 for the best question(s) to ask a counterfactual Oracle, awarded to Wei_Dai for any one of these posts.

The best counterfactual Oracle suggestions revolved around using the Oracle to automate or hasten difficult human work, by getting the Oracle to tell us what humans would have achieved had they done the work. Wei_Dai's suggestions in this area were the most useful, and he also had an interesting iterated Oracle idea that suggests some other avenues to explore.

Low-Bandwidth Oracle

$350 for the best question(s) to ask a low-bandwidth Oracle: the joint winners are Wei_Dai, for his idea on using the Oracle to predict crime, and William_S, for his idea to get the Oracle to critique plans.

Generally good ideas

There's a total of $300 "to be distributed as I see fit among the non-winning entries; I'll be mainly looking for innovative and interesting ideas that don't quite work." This award is split among several people:

$150 to cousin_it for finding the "bucket chain" flaw if there are Oracles whose time-ranges of prediction overlap.

$50 to evhub's idea to use Oracles to do iterated amplification and distillation.

$50 to paulfchristiano for a useful critique of evhub's idea, the thread starting from here. It increased my understanding both of Paul's approach and my own.

$25 to Gurkenglas for using Oracles to see how predictable history was; I have no idea how to make this work precisely, but it is a fascinating idea.

$25 to romeostevensit for a very thorough schema of issues around Oracles.

Thanks again to all who participated, and the winners can contact me to receive their ill-gotten gains.
Memory is not about the past
by simbyotic — 2020-06-17T16:43:22.995Z

Crossposted from Lettuce Be Cereal

We spend a large part of our lives reminiscing about the past.

Few activities are as quintessentially human as being on the cusp of falling asleep and suddenly be assaulted by a memory that has us relive an embarrassing episode that we thought long forgotten. Yet this ability to store and recall the past didn't emerge so that we could cringe at our past ineptness. We use our memories all the time, from remembering where we parked our car, to cook today's dinner, or to try and put a name to the face of the person who's now talking to us as if we'd met before, even though we could swear up and down to never have seen that person before.

Our memory is a crucial aspect of our cognition, and we often bemoan when it fails us. Ask the student who failed an exam, or the husband who forgot to buy his wife a gift for her birthday and now finds himself having to sleep on the couch in repentance.

To remember the past seems like an obvious boon to our ability to comport ourselves in the world. The student wouldn't have failed the exam if his memory was flawless, and certainly the husband would have been in less trouble - perhaps even have a happy relationship - were he not so forgetful. Yet despite these obvious advantages, our memory is, nonetheless, faulty. It seems like a patently evident design flaw that this is the case. Remembering is good. Forgetting is bad. Our memory regularly forgets. Therefore, our memory is flawed. Away to the trash can does our memory go.

Having recognized the failure of our memory, we say to it:

"Pack your bags and leave this place! Don't even think of coming back until you've gotten yourself cleaned up. I don't want to see your face again until you've turned yourself into a less defective piece of cognitive machinery. Go!"

It's not surprising then that Silicon Valley millionaires are trying to create technology that can be embedded into our brains to augment, and ultimately replace, our cognition. Indeed we have already solved the problem of memory, it just hasn't made its way into our brains. Our computers, smartphones, even fridges and autonomous vacuum cleaners have chips in them capable not only of storing information but also to have that information perdure, if not forever, then for a lot longer than our faulty memory is able to. If these science fictional dreams come to pass, legions of people will have t

...
Set image dimensions using markdown
by Alex Flint — 2020-06-17T12:37:54.198Z

When embedding an image using the markdown editor, is it possible to specify the image dimensions? It seems that both of these do not work:

Inline HTML:

<img src="https://i.imgur.com/25Magmb.png" width="123" height="123">

Some markdown variant I found on stackoverflow:

![alt text](https://i.imgur.com/25Magmb.png =123x123)
Our take on CHAI’s research agenda in under 1500 words
by Alex Flint — 2020-06-17T12:24:32.620Z

This work was supported by OAK, a monastic community in the Berkeley hills. It could not have been written without the daily love of living in this beautiful community.

Last week I attended the annual workshop of Stuart Russell’s research lab at UC Berkeley — the Center for Human-Compatible AI (CHAI). There were talks by Russell himself, as well as several graduates of the lab who now have research positions of their own at other universities. I got the clearest picture that I’ve yet encountered of CHAI’s overall technical research agenda. This is my take on it.

Assistance games

Traditionally, AI researchers have formulated problems assuming that there will be a fixed objective provided by a human, and that the job of the AI system is to find a solution that satisfies the human’s objective. In the language of sequence diagrams this looks as follows:

For example, in a search problem the objective specification might be a graph over which the system is to search, a cost for each edge, and a goal state that terminates the search. The AI researcher then needs to to develop optimization algorithms that efficiently find a minimum-cost path to a goal state. Or in a supervised learning problem the objective specification might consist of a dataset of labelled examples and the AI researcher needs to develop optimization algorithms that efficiently find function approximations that extrapolate these labelled examples to future unlabelled examples.

CHAI’s basic insight is to ask: why limit ourselves to a one-time objective specification event? We know that it is difficult to capture everything we care about in a formal metric (c.f. Goodhart’s law). We know that humans aren’t very good at foreseeing the strange and sometimes deranged ways that powerful optimization can give you what you asked for but not what you wanted. Why should information about the human’s objective be transmitted to the machine via a one-time data dump, after which it remains fixed for all time?

There are many alternative interaction patterns by which information about the human’s objective could be transmitted to the machine. The human could observe the machine and provide it with feedback as it works. The machine could ask the human questions about its objective. The machine could observe the human and deduce its objective from its behavior. And so on.

CHAI calls this an assistance game: the human ...
What is meant by Simulcra Levels?
by Chris_Leong — 2020-06-17T02:20:19.620Z

Simulcra levels are a concept that have seen a lot of play on Less Wrong. I suspect that different people are using these concepts in slightly different ways, so I thought it might make sense to ask a question to provide a central location for recording these theories and helping people disambiguate. For this reason, I don't think that there should be a single answer, but rather multiple answers. If you propose a theory, I'd suggest ideally providing a descriptive title or label.
Relating HCH and Logical Induction
by abramdemski — 2020-06-16T22:08:10.023Z

I'd like to communicate a simple model of the relationship between logical induction and HCH which I've known about for some time. This is more or less a combination of ideas from Sam, Tsvi, and Scott, but I don't know that any of them endorse the full analogy as I'll state it.

HCH as Bounded Reflective Oracles

An HCH is a human consulting the computational power of an HCH. It's very natural to model this within a reflective oracle setting, so that we can talk about computations which have oracle access to the output of any other computation. However, reflective oracles (ROs) are too powerful to capture a realistic HCH: access to an RO is tantamount to logical omniscience, since you can ask the RO the output of any computation, no matter how large.

Instead, we can think of HCH as a human with access to a bounded reflective oracle. A bounded reflective oracle (BRO) limits the time spent by the Turing machine (similar to how HCH requires the human to return an answer within a set time period, such as an hour, exporting any additional computation required to answer the question to the HCH calls the human makes in that time), plus, limits the size of the call trees which can be created as a consequence of the recursive calls an oracle machine makes (similar to how some versions of HCH give the human a limited number of recursive calls, which the human must then allocate between the HCH calls which the human makes, bounding the total size of the HCH call tree).

So, we can think of HCH and BRO as interchangeable: an HCH is just a BRO computation which starts with a simulation of a human with access to a BRO, and with the property that the human only ever makes calls to simulations of themselves-with-access-to-BRO, recursively. Similarly, a BRO-machine can be computed within an HCH if we have the human carry out the steps of the BRO-machine. Whenever the BRO machine makes a BRO call, the human in the HCH makes a corresponding HCH call, asking another human to simulate another BRO machine and report back with the results.

BROs Learning to Think

Bounded Oracle Induction (BOI) is a variant of logical induction based on BROs. Standard logical induction constructs market prices each day by finding the equilibrium prices, where all bets are balanced by opposite bets (or every "buy" has a corresponding "sell"). BOI uses BROs to find this equilibrium, so we can think of trad

...
Where to Start Research?
by Elizabeth — 2020-06-16T18:10:06.689Z

When I began what I called the knowledge bootstrapping project, my ultimate goal was “Learn how to learn a subject from scratch, without deference to credentialed authorities”. That was too large and unpredictable for a single grant, so when I applied to LTFF, my stated goal was “learn how to study a single book”, on the theory that books are the natural subcomponents of learning (discounting papers because they’re too small). This turned out to have a flawed assumption baked into it.

As will be described in a forthcoming post, the method I eventually landed upon involves starting with a question, not a book. If I start with a book and investigate the questions it brings up (you know, like I’ve been doing for the last 3-6 years), the book is controlling which questions get brought up. That’s a lot of power to give to something I have explicitly decided not to trust yet.

Examples:

When reading The Unbound Prometheus, I took the book’s word that a lower European birth rate would prove Europeans were more rational than Asians and focused on determining whether Europe’s birth rates were in fact lower (answer: it’s complicated), when on reflection it’s not at all clear to me that lower birth rates are evidence of rationality.
“Do humans have exactly 4 hours of work per day in them?” is not actually a very useful question. What I really wanted to know is “when can I stop beating myself up for not working?“, and the answer to the former doesn’t really help me with the latter. Even if humans on average have 4 hours, that doesn’t mean I do, and of course it varies by circumstances and type of work… and even “when can I stop beating myself up?” has some pretty problematic assumptions built into it, such as “beating myself up will produce more work, which is good.” The real question is something like “how can I approach my day to get the most out of it?”, and the research I did on verifying a paper on average daily work capacity didn’t inform the real question one way or the other.

What would have been better is if I’d started with the actual question I wanted to answer, and then looked for books that had information bearing on that question (including indirectly, including very indirectly). This is what I’ve started doing.

This can look very different depending on what type of research I’m doing. When I started doing covid research, I generated a long list of fa

...
What are the high-level approaches to AI alignment?
by Gordon Seidoh Worley — 2020-06-16T17:10:32.467Z

I'm writing a post comparing some high-level approaches to AI alignment in terms of their false positive risk. Trouble is, there's no standard agreement on what various high-level approaches to AI alignment there are today, either in terms of what constitutes these high-level approaches or where to draw the line in categorizing various specific approaches.

So, I'll open it up as a question to get some feedback before I get too far along. What do you consider to be the high-level approaches to AI alignment?

(I'll supply my own partial answer below.)
Creating better infrastructure for controversial discourse 
by Rudi C — 2020-06-16T15:17:13.204Z

Currently there are three active forks of Lesswrong; Itself, the alignment forum and the EA forum. Can adding a new fork that is more focused on good discourse on controversial/taboo topics be a good idea?

I am an Iranian, and have a fair share of experience with bad epistemic conditions. Disengagement with politics is always a retreat, though it can be strategic at times. As time goes on, the politics of a memetic-based elite will increasingly infringe on your rights and freedoms. Aside from this, as your community’s status grows, your epistemic freedoms get a lot worse. The community itself will be increasingly infiltrated by the aggressive memetic structure, as the elite control the education, the media, and distribution of status/resources.

The fact (IMHO) that the current object-level neo-religion(s) is also vastly superior in its dogma is also bad news for us. The better a religion is, the harder fighting it is. I, myself, believe a bit in the neo-religion.

The rationalist/EA community has not invested much in creating a medium for controversial discourse. So there should be quite a bit of low-hanging fruit there. Different incentive mechanisms can be tried to see what works. I think anonymity will help a lot. (As a datapoint, Quora had anonymous posting in the past. I don’t know if they still do.) Perhaps hiding all karma scores might help, as well; (An anonymized score of the user’s karma on Lesswrong can be shown to distinguish “the old guard.”) Outsiders won’t have as much of an incentive to engage without a karma system. Posts that are, e.g., just ripping on the outgroup won’t be as attractive without the scores.

A big issue is distancing the controversial forum from tainting Lesswrong’s brand. I don’t have good ideas for this. The problem is that we need to somehow connect this forum to Lesswrong’s community.

Another direction to tackle the controversy problem is introducing jargon. Jargon repels newcomers, attracts academic-minded people, gives protection against witch-hunters, and raises the community’s status. (Its shortcomings should be obvious.) One of the ways it protects against witch hunters is that it signals that we are politically passive and merely “philosophizing.” Another reason (possibly what gives the signal its credibility) is that we automatically lose most of the population by going down this route. We have introduced artificial inf

...
Is utilitarianism the result of a Conflict Theory singularity?
by spkoc — 2020-06-16T13:29:39.269Z

Gross buzzwords but correctly used?

This all started with a comment on https://www.lesswrong.com/posts/6eNy7Wg7X7Mb622D7/two-kinds-of-mistake-theorists?commentId=TdF8Ra9dGG6ZfaQMf#TdF8Ra9dGG6ZfaQMf which led to me getting a link to https://www.lesswrong.com/posts/no5devJYimRt8CAtt/in-defence-of-conflict-theory and I started writing a reply to that but realised it was extremely long so I made it a separate post.

Intro

Here's my main argument against emphasising mistake theory over conflict theory: you're only able to be a mistake theorist after the conflict theorists have done most of the hard work.

Very nice and succinct. This immediately made me think of art vs engineering. Ambiguous domains where people can't quite serialize how they produce good outputs(though we can recognize good output when we see it, wafts of P vs NP) vs domains where we can algorithmize the production of good outputs and take the ... human flair out of it.

This post is me trying to work out what I think the 'hard work' is.

Is there a point at which conflict theory makes itself obsolete?

Is it still useful in the present, at least for people with utilitarian ethical axioms(all humans' utility is valuable, and aggregate utility should be maximised)?

Mistake theory is about maximising utility for a set of entities, conflict theory is about expanding the set of entities under consideration. If you already want to maximize total human utility(do you though? and do your actions reflect this truly?), what can conflict theory do for you?

Mood-setting story: let's pretend we believe history advances over time by some metric

History(including biological history) progresses by expanding the circle of concern, making an individual's utility function include more and more entities outside the individual. It starts as a singular point, the individual itself(say among pre-social animals), grows to include the parent and children(in animals that take care of their own young), grows to include the expanded family group(most famous in social mammals and eusocial insects etc), grows to include the very expanded family group(tribes and proto-states and city-states).

Within human history you expand the circle to include women, slaves, children, poor people, people of the same religion or appearance or language group, people absolutely everywhere currently alive, people alive in the past(ancestor worship),

...
Superexponential Historic Growth, by David Roodman
by Ben Pace — 2020-06-15T21:49:00.188Z
Link: https://www.openphilanthropy.org/blog/modeling-human-trajectory

This is research trying to do a similar analysis to Hanson's paper Long-Term Growth as a Sequence of Exponential Modes, and coming to different conclusions in some areas and the same conclusions in others. It also discusses Scott's post 1960: The Year the Singularity Was Cancelled. The name of the post/paper is "Modeling Human Trajectory".

From the summary.

One strand of analysis that has caught our attention is about the pattern of growth of human society over many millennia, as measured by number of people or value of economic production. Perhaps the mathematical shape of the past tells us about the shape of the future. I dug into that subject. A draft of my technical paper is here. (Comments welcome.) In this post, I’ll explain in less technical language what I learned.

It’s extraordinary that the larger the human economy has become—the more people and the more goods and services they produce—the faster it has grown on average. Now, especially if you’re reading quickly, you might think you know what I mean. And you might be wrong, because I’m not referring to exponential growth. That happens when, for example, the number of people carrying a virus doubles every week. Then the growth rate (100% increase per week) holds fixed. The human economy has grown super-exponentially. The bigger it has gotten, the faster it has doubled, on average. The global economy churned out $74 trillion in goods and services in 2019, twice as much as in 2000.1 Such a quick doubling was unthinkable in the Middle Ages and ancient times. Perhaps our earliest doublings took millennia.

If global economic growth keeps accelerating, the future will differ from the present to a mind-boggling degree. The question is whether there might be some plausibility in such a prospect. That is what motivated my exploration of the mathematical patterns in the human past and how they could carry forward. Having now labored long on the task, I doubt I’ve gained much perspicacity. I did come to appreciate that any system whose rate of growth rises with its size is inherently unstable. The human future might be one of explosion, perhaps an economic upwelling that eclipses the industrial revolution as thoroughly as it eclipsed the agricultural revolution. Or the future could be one of implosion, in which environmental thresholds are crossed or the creative process that drives growth runs amok, as in a

...
How to analyze progress, stagnation, and low-hanging fruit
by jasoncrawford — 2020-06-15T21:02:46.969Z
Link: https://rootsofprogress.org/teasing-apart-the-s-curves

Is progress slowing down?

Those who favor the stagnation hypothesis point out that, other than computing, most fields have seen only modest, incremental improvements. Our cars, planes, factories, food supply, antibiotics, and so forth are just improved versions of what we had in, say, 1960.

The counter-argument is: Wait, what do you mean, “other than computing?” How can you just ignore the area where we have seen revolutionary progress? Saying, “other than orders of magnitude increases in capacity, performance, and cost; revolutionizing all of communications; connecting every human being on Earth; and putting all the world’s knowledge and culture in every pocket; what has technology done lately?” sounds a lot like “what have the Romans ever done for us?”

The rebuttal to the counter-argument is: Computing is just one area. We used to have revolutionary changes going on in multiple areas at once:

1870–1920 saw the the electric generator, electric motor, and light bulb; the telephone, wireless, phonograph, and film; the first automobiles and airplanes, and the assembly lines to build them; the first synthetic plastic (Bakelite); the Panama Canal; the Haber-Bosch process; and the germ theory, along with its applications to public health.
1920–1970 saw radio, television, radar, and the first computers; the invention of nylon and other plastics; the expansion of mass manufacturing with an explosion of consumer products; penicillin and the golden age of antibiotics; Norman Borlaug’s Green Revolution in agriculture; nuclear power; the interstate highway system, jet airplanes, and the Moon landing.
1970–2020 saw … the PC, Internet, smartphone, and GPS; and genetic engineering, including GMOs. It also saw the Apollo project ended, the first supersonic passenger jet launched and then canceled, the promise of nuclear power unfulfilled, a War on Cancer with lackluster results, and similarly modest progress against heart disease. Yes, there were many incremental improvements in a variety of areas that are easy to forget about, but that’s the point—there were fewer revolutionary breakthroughs.

Another way of putting this is that if you look at a 1970 living room vs. a modern one, not much has visibly changed—again, other than the computer (and related technology, such as the big flat screen TV that has replaced your clunky CRT).

If progress is slowing down, what’s causing it?

One h

...