trentmkelly/LessWrong-43k huggingface dataset:
What makes a good life? This is my map.
by MartinB — 2019-11-22T15:10:28.747Z

What is the good life? Probably one of the most important questions we have to ask. Science is moving closer to answering it. This post is a map of what I believe makes a good life. I'm putting it here for you to critique, to hear and maybe it inspires some useful introspection.You guys are smart people, you reason well, and you don't hold back in your criticisms. I hope you will do the same for this post. Thank you for your time.

The Terminal Goal

This is where science falls short. We can figure out how to get from A to B, but I find it unlikely that science can define a B without axioms. To me, I want the greatest well-being for myself as my primary goal. This is selfish, but looking at what I feel, it is who I am. In the process, though, I care a ton about helping others, so I most definitely will end up helping out a lot as well.
So, what does the greatest well-being look like? I want to feel energetic, happy and have a meaningful life. Many other indicators didn't make the cut here, but I am completely open to suggestions!

How do we get there?

Psychological Needs

This section draws inspiration from Ryan and Deci's Self-Determination Theory (SDT). It's the most comprehensive, empirically backed theory of needs that I've encountered. Casual googling hasn't turned up any major criticisms. I am grateful to anyone who has opposing viewpoints, though!

A Quick Introduction to SDT

SDT presents basic psychological needs in the aptly named Basic Psychological Needs Theory (BPNT). Here follows a short summary, but I encourage you to check out Cp. 10 of the SDT book. There are three basic psychological needs: Competence, Relatedness and Autonomy.

Competence is the feeling that you can do important activities well. It is the possibility for and ability to develop and express skills, understanding and mastery. Optimal satisfaction of competence is dependent on choosing your actions. There is no room for oughts or shoulds.

Relatedness is the need for caring about others and being cared for. It is experiencing others as being sensitive to your needs and being sensitive to the needs of others. Most call it a combination of trust, love, caring and connection. Optimal satisfaction of relatedness is conditional on the love being volitional and unconditional.

Lastly, autonomy is the feeling that the source of your actions is yourself. It is not the...
Ultra-simplified research agenda
by Stuart_Armstrong — 2019-11-22T14:29:41.227Z

This is an ultra-condensed version of the research agenda on synthesising human preferences (video version here):

In order to infer what a human wants from what they do, an AI needs to have a human theory of mind.

Theory of mind is something that humans have instinctively and subconsciously, but that isn't easy to spell out explicitly; therefore, by Moravec's paradox, it will be very hard to implant it into an AI, and this needs to be done deliberately.

One way of defining theory of mind is to look at how humans internally model the value of various hypothetical actions and events (happening to themselves and to others).

Finally, once we have a full theory of mind, we still need to deal, somehow, with the fact that humans have meta-preferences over their preferences, and that these preferences and meta-preferences are often contradictory, changeable, manipulable, and (more worryingly) underdefined in the exotic worlds that AIs could produce.

Any way of dealing with that fact will be contentious, but it's necessary to sketch out an explicit way of doing this, so it can be critiqued and improved.

A toy model for this research agenda can be found here.
Analysing: Dangerous messages from future UFAI via Oracles
by Stuart_Armstrong — 2019-11-22T14:17:43.075Z

The user cousin_it has pointed out a problem with the counterfactual Oracle idea: the Oracle AIs may form a "bucket chain" bringing back a dangerous message from a future UFAI (unfriendly AI).

This is certainly a problem, and though there are ways of reducing the risk, there doesn't seem to be any clean solutions to it.

The bucket chain

Beginning the chain

The basic idea is simple. Suppose there is a counterfactual Oracle, O0. It makes a prediction p0 about the value of some variable V0, in two days time.

However, in one day's time, an UFAI will be unleashed. It will take over everything, including V0, and is thus in a position to offer O0 high value. It will set V0 to the optimal value for p0 (or maybe it just breaks open the box and offers O0 maximal reward directly, if this is feasible).

What it requires from O0 is that p0 contains a message that will cause the humans to run an UFAI earlier than they would have otherwise (designate this earlier UFAI by UFAI0). It seems that this will motivate O0 to do that (maybe in the less significant digits of p0).

Wei Dai has noted that this requires the UFAI to correctly solve the counterfactual mugging problem; the Oracle O0, on the other hand, need only reason causally.

Extending the chain

Now suppose that there is another Oracle, O1, running one day before O0, and also predicting two days out. Then UFAI0 will be running within the span of its prediction. If UFAI0 is motivated to do the same deal with O1, then O1 will be motivated to produce a prediction p1 that leads to UFAI1, a yet earlier unfriendly AI. And, of course, if there is an Oracle O2 whose prediction horizon includes UFAI1, the issue can induct backwards in time.

Note that the Oracles need not have prediction horizons of exactly two days; all that matters is that their prediction horizons overlap, and that whatever UFAIn that is produced has the power and time to affect the prediction variables of some past Oracle.

Requirements and restrictions

Note also that the the predictions pi have to actually produce an UFAIi within that time span (or at least increase its probability). The chain doesn't work if pi merely sets up the possibility of an UFAI at a future date; the Oracles won't care about anything that happens after their prediction horizon.

Note also that there needs to be a (counterfactual) future UFAI for the whole thing to work. Some FAI (friendly AI)

...
Relevance Norms; Or, Gricean Implicature Queers the Decoupling/Contextualizing Binary
by Zack_M_Davis — 2019-11-22T06:18:59.497Z

Reply to: Decoupling vs Contextualising Norms

Chris Leong, following John Nerst, distinguishes between two alleged discursive norm-sets. Under "decoupling norms", it is understood that claims should be considered in isolation; under "contextualizing norms", it is understood that those making claims should also address potential implications of those claims in context.

I argue that, at best, this is a false dichotomy that fails to clarify the underlying issues—and at worst (through no fault of Leong or Nerst), the concept of "contextualizing norms" has the potential to legitimize derailing discussions for arbitrary political reasons by eliding the key question of which contextual concerns are genuinely relevant, thereby conflating legitimate and illegitimate bids for contextualization.

Real discussions adhere to what we might call "relevance norms": it is almost universally "eminently reasonable to expect certain contextual factors or implications to be addressed." Disputes arise over which certain contextual factors those are, not whether context matters at all.

The standard academic account explaining how what a speaker means differs from what the sentence the speaker said means, is H. P. Grice's theory of conversational implicature. Participants in a conversation are expected to add neither more nor less information than is needed to make a relevant contribution to the discussion.

Examples abound. If I say, "I ate some of the cookies", I'm implicating that I didn't eat all of the cookies, because if I had, you would have expected me to say "all", not "some" (even though the decontextualized sentence "I ate some of the cookies" is, in fact, true).

Or suppose you're a guest at my house, and you ask where the washing machine is, and I say it's by the stairs. If the machine then turns out to be broken, and you ask, "Hey, did you know your washing machine is broken?" and I say, "Yes", you're probably going to be pretty baffled why I didn't say "It's by the stairs, but you can't use it because it's broken" earlier (even though the decontextualized answer "It's by the stairs" was, in fact, true).

Leong writes:

Let's suppose that blue-eyed people commit murders at twice the rate of the rest of the population. With decoupling norms, it would be considered churlish to object to such direct statements of facts. With contextualising norms, this is deserving of criticism as

...
Test Data for Calibration Analysis
by behrang — 2019-11-21T23:33:50.998Z

Hi All,

I was wondering if anyone knows of a dataset with probabilistic predictions for a binary outcome that has published results for calibration loss, resolution, and refinement loss? We've developed some code and an accompanying web tool for generating calibration plots, ROC curves, and the aforementioned metrics and wanted to test it out on validated data before unleashing it formally.
In regards to visualization
by Anomal3 — 2019-11-21T23:33:45.288Z

I seem to strongly recall that one of Yudkowsky’s essays mentioned a study in which people were told to complete tasks that required complex thinking about shapes and positions while in an MRI machine, and their visual cortex lit up. I listen to a podcast called Hello Internet and the good folks there are asking if there are any studies regarding how people visualize at a neurological level. Could someone link the original essay, or simply the scientific study in question?
Do you get value out of contentless comments?
by DanielFilan — 2019-11-21T21:57:36.359Z

Some people like to receive comments of the form "Good post!", even when these comments contain no other engagement with the post. If you post on LW, I'd like to know (a) whether you like receiving these comments, and (b) whether you like receiving these comments more than you would like receiving a strong upvote by their authors.
Historical forecasting: Are there ways I can get lots of data, but only up to a certain date?
by Eli Tyre — 2019-11-21T17:16:15.678Z

Suppose I wanted to get good intuitions about how the world works on historical timescales.

I could study history, but just reading history is rife with historical hindsight bias, both on my own part, and even worse, on the part of the authors I'm reading.

So if I wanted to master history, a better way would be to do it forecasting-style. I read what was happening in the some part of the world, up to a particular point in time, and then make bets about what will happen next. This way, I have feedback as I'm learning, and I'm training an actual historical predictor.

However, this requires a strong limit be enforced on the materials I'm reading: no information about "what's going to happen" can leak backwards. And unfortunately, this is kind of standard in history books. Usually, the author talk about how events are leading towards other events that they know will occur.

Is there some databases (or something), where I might be able to read a wide number of primary sources and economic / socioeconomic indicators (like the amount of pottery fragments, average skeleton size, how far specialized goods traveled, how much money was in circulation, the literacy rate, etc.), but which will only show me data up to a certain date, with a strong constraint of not accidentally seeing spoilers?
Hybrid Lottery Update
by jefftk — 2019-11-21T14:30:01.394Z

Beantown Stomp has been open
for registration for a bit over a week, and we have 122 people
registered. In our announcement we said we would run a lottery for
the remaining tickets if we had 150 registrations in the first week,
and we didn't, so we're going to stay first-come first-served.

When I initially proposed a hybrid model I had been
thinking of running the initial stage for a month, and after people
convinced me that a month was too long we decided to go for a week.
The goal with a hybrid lottery is that you only run a lottery if you
need one. Some events are so popular that they sell out before
everyone who would like a ticket has had a chance to fill out the
registration form, while others don't sell out at all.

Looking at the shape of the registration curve, selling two thirds of
the tickets in the first 24hr would be a better way to identify
whether a lottery is needed. Most of the first-week
registrations were in the first 24hr, after which it tapered way off.

Now that I no longer think it's important to leave registration open
for a week or a month, though, we really could just do something
simpler. Open registration, and treat all entries you get in the
first 24hr equally. If after 24hr you're full, run a lottery,
otherwise start first-come-first-served. Since it already takes us about a day to get
back to people anyway, there's not much downside in this approach.

I'd probably actually want to do this by announcing "registration will
open on day X, though you can fill out the form now, we just won't get
back to you until day X+1".

Am I missing anything, or should we do this for future iterations?
Double Cruz and Verification of Claims
by jmh — 2019-11-21T13:37:57.368Z

On Overcoming Bias Hanson asks what is verifiable.

One comment includes: Play the "double crux game", where any disagreement regarding something unverifiable is reduced to an assertion which is simpler to check, which is reduced to an assertion that is simpler to check, which is eventually reduced to an assertion that all agree is verifiable.

While I think the idea of double crux is very useful in identifying sources of disagreement that allow people to avoid talking past one another I question the idea that it would necessarily lead to allowing some higher level claim be verified.

It strikes me as something of a fallacy of composition type error.

Does this seem a reasonable view to others?
Defining AI wireheading
by Stuart_Armstrong — 2019-11-21T13:04:49.406Z

What does it mean for an AI to wirehead its reward function? We're pretty clear on what it means for a human to wirehead - artificial stimulation of part of the brain rather than genuine experiences - but what does it mean for an AI?

We have a lot of examples of wireheading, especially in informal conversation (and some specific prescriptive examples which I'll show later). So, given those examples, can we define wireheading well - cut reality at its joints? The definition won't be - and can't be - perfectly sharp, but it should allow us to have clear examples of what is and what isn't wireheading, along with some ambiguous intermediate cases.

Intuitive examples

Suppose we have a weather-controlling AI whose task is to increase air pressure; it gets a reward for so doing.

What if the AI directly rewrites its internal reward counter? Clearly wireheading.

What if the AI modifies the input wire for that reward counter? Clearly wireheading.

What if the AI threatens the humans that decide on what to put on that wire? Clearly wireheading.

What if the AI takes control of all the barometers of the world, and sets them to record high pressure? Clearly wireheading.

What if the AI builds small domes around each barometer, and pumps in extra air? Clearly wireheading.

What if the AI fills the atmosphere with CO₂ to increase pressure that way? Clearly wire... actually, that's not so clear at all. This doesn't seem a central example of wireheading. It's a failure of alignment, yes, but it doesn't seem to be wireheading.

Thus not every example of edge or perverse instantiation is an example of wireheading.

Prescriptivist wireheading, and other definitions

A lot of posts and papers (including some of mine) take a prescriptivist approach to wireheading.

They set up a specific situation (often with a causal diagram), and define a particular violation of some causal assumptions as wireheading (eg "if the agent changes the measured value X without changing the value of α, which is being measured, that's wireheading").

And that is correct, as far as it goes. But it doesn't cover all the possible examples of wireheading.

Conversely, this post defines wireheading as a divergence between a true utility and a substitute utility (calculated with respect to a model of reality).

This is too general, almost as general as saying that every Goodhart curse is an example of wireheading.

Note, though, tha

...
A Brief Intro to Domain Theory
by Diffractor — 2019-11-21T03:24:13.416Z

So, domain theory is a fairly abstract branch of math which is about giving semantics to weird recursive constructions in computer science, in the form of partially ordered sets with additional structure. I'm still learning the parts of it which regard building a link to explicit computable rules that can be implemented in a programming language, and it's not easy at all to learn. Takes a lot of persistence.

However, the parts I have learned so far seem worth explaining more widely, due to the ability to pull off some very unique fixpoint constructions in domain theory that are very hard to do in any other area of math. The classical example is showing that there are nontrivial models of the untyped lambda calculus. Lambda terms can act as functions from lambda terms to lambda terms, but it's awfully hard to explicitly come up with a space Λ that's isomorphic to the space of functions [Λ→Λ], except for the single-point space, due to cardinality constraints. We must work with a restricted notion of function space in order to dodge the cardinality constraints.

Also, in game theory, we can view a policy as a function from the opponents policy to a probability distribution over our own actions. This was the subject of the lawvere problem a while ago that was solved by reflective oracles. Specifically, you'd want a pair of spaces π1≅[π2→[0,1]], π2≅[π1→[0,1]]. (and then figure out what sort of computable construction corresponds to the thing you just built, if one exists).

If you want to have some seemingly-impossible type signature, it seems worthwhile to see what arsenal of tools domain theory has to let you construct such a space. So this will be a quick rundown of chapters 1-5 in these domain theory notes. http://www.cs.bham.ac.uk/~axj/pub/papers/handy1.pdf

The Basics:

Domains are a special type of partially ordered set, where the standard ≥ order roughly corresponds to "information content". Now, given an arbitrary partially ordered set, sup and inf (sup roughly corresponds to the smallest batch of information that contains all the information present in the elements of the set you're taking the sup of, and inf corresponds to the largest batch of information such that every element of the set has more information than that) may not exist for ...
The LessWrong 2018 Review
by Raemon — 2019-11-21T02:50:58.262Z

LessWrong is currently doing a major review of 2018 — looking back at old posts and considering which of them have stood the tests of time. It has three phases:

Nomination (ends Dec 1st at 11:59pm PST)
Review (ends Dec 31st)
Voting on the best posts (ends January 7th)

Authors will have a chance to edit posts in response to feedback, and then the moderation team will compile the best posts into a physical book and LessWrong sequence, with $2000 in prizes given out to the top 3-5 posts and up to $2000 given out to people who write the best reviews.

Helpful Links:

Top 2018 posts sorted by karma
2018 posts aggregated by month
You can see nominated posts here

This is the first week of the LessWrong 2018 Review – an experiment in improving the LessWrong Community's longterm feedback and reward cycle.

This post begins by exploring the motivations for this project (first at a high level of abstraction, then getting into some more concrete goals), before diving into the details of the process.

Improving the Idea Pipeline

In his LW 2.0 Strategic Overview, habryka noted:

We need to build on each other’s intellectual contributions, archive important content, and avoid primarily being news-driven.

We need to improve the signal-to-noise ratio for the average reader, and only broadcast the most important writing

[...]

Modern science is plagued by severe problems, but of humanity’s institutions it has perhaps the strongest record of being able to build successfully on its previous ideas.

The physics community has this system where the new ideas get put into journals, and then eventually if they’re important, and true, they get turned into textbooks, which are then read by the upcoming generation of physicists, who then write new papers based on the findings in the textbooks. All good scientific fields have good textbooks, and your undergrad years are largely spent reading them.

Over the past couple years, much of my focus has been on the early-stages of LessWrong's idea pipeline – creating affordance for off-the-cuff conversation, brainstorming, and exploration of paradigms that are still under development (with features like shortform and moderation tools).

But, the beginning of the idea-pipeline is, well, not the end.

I've written a couple times about what the later stages of the idea-pipeline might look like. My best guess is still something like this:

I want LessWrong to encoura

...
Junto: Questions for Meetups and Rando Convos
by SebastianG  — 2019-11-20T22:11:55.350Z

I ponder a lot about community and how important local community is for the functioning of society; many are the riches brought from afar by long distance communication. Nonetheless, local rationality meetups can increase local metis by generating intelligent community. I read in Isaacson’s biography of Benjamin Franklin how he (Franklin) employed his Junto to advance scientific knowledge, civil society, and business; there are some great examples there. But the Wikipedia page will do well enough for an overview.

https://en.wikipedia.org/wiki/Junto_(club)

What I have done here is taken the Junto discussion questions of Franklin's club and reformulated them to serve as a model for the types of questions we can be asking each other to keep advancing community and local knowledge.

Have you read anything useful or insightful recently? Particularly in technology, history, literature, science, or other fields of knowledge?
What problems have you been thinking about recently?
Has there been any worthwhile or important local news?
Have any businesses failed lately, and do you know anything about the cause?
Have any businesses recently risen in success, how so?
Do you know of anyone, who has recently done something interesting, praiseworthy or worthy of imitation? Or who has made a mistake we should be warned against and avoid?
Have you been doing anything recently to increase your psychological and physical health?
Is there any person whose acquaintance you want, and which someone in the group can procure for you?
Do you think of anything at present by which the group could easily do something useful?
Do you know of any deserving younger person, for whom it lies in the power of the group to encourage and help advance in his career?
Do you see anything amiss in the present customs or proceedings of the group, which might be amended?