trentmkelly/LessWrong-43k huggingface dataset:
Meditation Retreat: Immoral Mazes Sequence Introduction
by Zvi — 2019-12-28T00:50:01.078Z

I just got home from a six day meditation retreat and began writing.

The catch is that I arrived at the retreat yesterday.

I knew going in that it was a high variance operation. All who had experience with such things warned us we would hate the first few days, even if things were going well. I was determined to push through that.

Alas or otherwise, I was not sufficiently determined to make that determination stick. I didn’t have a regular practice at all going in, was entirely unfamiliar with the details of how this group operated, and found the Buddhist philosophy involved highly off putting, in a ‘no that’s not how it works that’s not how any of this works nor would we want it to’ kind of way. I am no negative utilitarian. I couldn’t focus, my meditations were entirely unproductive compared to past experience and were increasingly focusing on how terrible things were.

I am highly, highly confident that none of the people who warned me would be surprised by those developments, and similarly confident that they would all tell me to push through it. And will tell me, when I see them, that if I can do so I should try again. But I also realized that the anticipated reaction from others saying I didn’t give it a proper chance was the only reason I was considering not leaving. So I left.

To my surprise, those there said I was likely making a mature decision and were sympathetic. They spun it a little to try and get me not to give up in the future, but that was it, which was a really good look. It did not go unnoticed.

I took the bulk of the day to get home and relax, play a game, saw the excellent movie Knives Out. What I did realize was that yes, some combination of the Solstice plus the meditation retreat, even if I only did a few hours worth of sessions, did have a clarifying and motivating effect to get me back on track. I’m not unhappy I went, even though I bailed, because I was, in a much more practical and everyday very small sense, enlightened.

I’m also leaning towards being happy I left when I did. I do buy that there are serious effects that can only be gained from being out of feedback loops and in silence for several days, but my instincts (however motivated they may be) are strongly telling me that this is not the way for me to do that.

The other motivating part of this is that, while I will absolutely take the majority of tomorrow to enjoy the College Foot...
Critiquing "What failure looks like"
by Grue_Slinky — 2019-12-27T23:59:50.440Z

I find myself somewhat confused as to why I should find Part I of “What failure looks like” (hereafter "WFLL1", like the pastry) likely enough to be worth worrying about. I have 3 basic objections, although I don't claim that any are decisive. First, let me summarize WFLL1 as I understand it:

In general, it's easier to optimize easy-to-measure goals than hard-to-measure ones, but this disparity is much larger with ML models than with humans and human-made institutions. As special-purpose AI becomes more powerful, this will lead to a form of differential progress where easy-to-measure goals become optimized well past the point when they correlated with what we actually want.

(See also: this critique, although I agree with the existing rebuttals to it).

Objection 1: Historical precedent

In the late 1940s, George Dantzig invented the simplex algorithm, a practically efficient method for solving linear optimization problems. At the same time, the first modern computers were coming around, which he had access to as a mathematician in the US military. For Dantzig and his contemporaries, a wide class of previously intractable problems suddenly became solvable, and they did use the new methods to great effect, playing a major part in developing the field of operations research.

With the new tools in hand, Dantzig also decided to use simplex to optimize his diet. After carefully poring over prior work, and putting in considerable effort to obtain accurate data and correctly specify the coefficients, Dantzig was now ready, telling his wife:

whatever the [IBM] 701 says that's what I want you to feed me each day starting with supper tonight.

The result included 500 gallons of vinegar.

After delisting vinegar as a food, the next round came back with 200 boullion cubes/day. There were several more iterations, none of which worked, and after everything Dantzig simply went with a "common-sense" diet.

The point I am making is, whenever we create new methods for solving problems, we end up with a bunch of solutions looking for problems. Typically, we try to apply those solutions as widely as possible, and then quickly notice when some of those solutions don't solve the problems we actually want to solve.

Suppose that around 1950, we were musing about the potential consequences of the coming IT revolution. We might've noticed th...
More on polio and randomized clinical trials
by jasoncrawford — 2019-12-27T21:07:02.340Z
Link: https://rootsofprogress.org/more-on-polio-and-randomized-clinical-trials

My post and Twitter thread about the controversy over the 1954 polio vaccine trials generated many replies on Twitter, so here is a followup.

First, I’m very sympathetic to the dilemma that Salk faced. I think it’s a tough problem, and it’s worth thinking about different ways to approach it. I didn’t mean to cast aspersions on Salk.

One way in general to improve this situation is to make sure that all the controls get the treatment immediately after the trial, if it is proved safe and effective. But in this case, that wouldn’t have changed anything. Polio was a seasonal disease, peaking each summer. Getting the vaccine after the trial meant getting it the next season.

Some people have suggested that trials can be ended early if the data clearly shows a conclusion. This is true, although it’s trickier than it appears—if you do it in a naive way, you are prone to reaching false conclusions. The statistics of doing this properly is sophisticated. This is also difficult in the case of a vaccine, where the outcome is binary (you get the disease or you don’t) and you have to wait for a certain period of exposure. This wasn’t like a blood pressure medication, where you can constantly measure a continuous variable.

One idea occurred to me that I haven’t heard anyone suggest: the trial didn’t have to be 50-50. With a large enough group, you could hold back a smaller subset as the control (80-20?). Again, you need statistics here to tell you how this affects the power of your test.

Returning to the issue of: was an RCT needed at all? Again it’s a tough call, but I still think it was, for two reasons: one scientific/epistemological, and one social/political.

Epistemologically, it’s easy to say in hindsight that an observed-control trial would have been conclusive. Here’s the data (copied from Oshinsky’s book):

Placebo-control areas:

• Vaccinated: 200,745 subjects / 33 cases = 1 per 6,083
• Placebo: 201,229 subjects / 115 cases = 1 per 1,750

Observed-control areas:

• Vaccinated: 221,988 subjects / 38 cases = 1 per 5,842
• Observed: 725,173 subjects / 330 cases = 1 per 2,197

But think of how it might have turned out.

One scenario: You run the trial, and it’s not conclusive. Maybe it shows a slight reduction ...
Vaccine... Help? Deprogramming? Something?
by Golden Delicious — 2019-12-27T20:27:27.367Z

I need help. Pretty much the entire scientific community and everyone I trust as an intellectual role model has said that vaccines are an almost entirely good thing, yet a close member of my family has made a somewhat convincing argument they are dangerous, and I’m terribly confused. I’ve been trying to figure this issue out for months now, and I just can’t. I’ve seen some (a lot of) dark side epistemology used by the more… out there antivaxers (i.e. homeopathy and essential oil people), but although I have a creeping sense some of what my family member is saying is bullshit, I’m still really confused overall, and some of the stuff they are saying seems reasonable, like family history screening.

They also say some stuff I’m pretty sure is total bullshit, like thimerosal levels in vaccines being a problem, or aluminum content.

I don’t think vaccines are linked to autism, and I don’t think the relative in question does either.

In addition, a very young nephew of mine (with a medium degree of confidence as to whether this story was overinflated by the family member) fell ill with severe seizures and almost died less than 24 hours after vaccination.

Can anyone help me sort out this problem?

I’m honestly confused and I think I might be being programmed by my relative.

Edit: Thanks to everyone here. I now realize that what my relative was saying (not my uncle lol) is total bullshit, not just mostly bullshit. They've been hit by a bad belief, and while I don't think I'll be able to talk them out of it, I'll ignore their rants about it in the future.

Thanks again,

-Golden.
Expected utility and repeated choices
by Marco Discendenti — 2019-12-27T20:26:17.465Z

Maybe this is a well known kind of problem but I am a novice and it looks puzzling to me.

Here is a lottery: I have these two choices:

(a) get 0.5$ for sure
(b) win 1$ with probability 23 or nothing with probability 13

My utility function is U(x)=√x .

What should I choose?

Let's compute the expected utilities:

expected utility for one single game is for (b) 23⋅√1=23≈0.67 while for (a) is √0.5≈0.7 so I have maximized expected utility with choice (a)
if I compute expected utility for two games I get a different prescription:

utility for chosing (a) two times is U(1)=√1=1
the expected utility for chosing (b) two times is

P(2 wins)U(2)+P(1 win)U(1)=(23)2⋅√2+2⋅23⋅13⋅√1

This last computation is equal to 49(√2+1)≈1.07 which is greater than the utility of double (a) (i.e. 1) so in order to maximize expected utility I should actually prefer to play (b) two times rather than playing (a) two times.

So we have this apparent inconsistency:

for one single game it's better to choose (a)
for two games it's better to choose (b) both times

This result is puzzling to me because I would expect that utility maximization for one single game should be enough in order to take the decision regardless of what I am allowed to do in future choices. It seems instead that the mere possibility that I could play this same lottery another time changes the convenience of the choices about what to play in the first game. If this is the case then utility theory seems almost useless: I would be forced to put in my computation the whole list of my possible future choices!

Am I missing something or is this an actual problem?
We need to revisit AI rewriting its source code
by ryan_b — 2019-12-27T18:27:55.315Z

It feels like community discussion has largely abandoned the topic of AGI having the self-modifying property, which makes sense because there are a lot of more fundamental things to figure out.

But I think we should revisit the question at least in the context of narrow AI, because the tools are now available to accomplish exactly this on several levels. This thought was driven by reading a blog post, Writing BPF Code in Rust.

BPF stands for Berkeley Packet Filter, which was originally for network traffic analysis but has since been used for tracing the Linux kernel. The pitch is that this can now be used to let userspace code run in the kernel, which is to say change the way the kernel is working.

The ability to use code to write code is very old; it is arguably the core insight of LISP. But this is becoming increasingly common as a practice now, including things like writing OCaml or Haskell for generating correct C code, and increasingly powerful code generation tricks in compilers.

It's also possible to change how the compilers work now. The Julia language has a package named Cassette.jl, which allows dynamically injecting behavior into the Just-in-Time compiler. As it happens both this Julia trick and the BPF trick for the Linux kernel rely on LLVM.

All of this means at present we can in fact write code that modifies itself, and that modifies the way it is compiled, and that modifies the way the environment runs it (assuming the environment is Linux). The infrastructure seems to be present for large-scale, multi-layer self modification to take place. This seems like it makes questions about self modification testable in a way they weren't before. The BPF and Cassette.jl tricks don't even require software written for them explicitly, they work on previously existing code whose authors had no idea such capability existed. These methods are independent of ideas like Software 2.0/Differentiable Programming, and combined they make me wonder if the safety problems we are concerned with might actually start appearing at the level of applications first.
Firming Up Not-Lying Around Its Edge-Cases Is Less Broadly Useful Than One Might Initially Think
by Zack_M_Davis — 2019-12-27T05:09:22.546Z

Reply to: Meta-Honesty: Firming Up Honesty Around Its Edge-Cases

Eliezer Yudkowsky, listing advantages of a "wizard's oath" ethical code of "Don't say things that are literally false", writes—

Repeatedly asking yourself of every sentence you say aloud to another person, "Is this statement actually and literally true?", helps you build a skill for navigating out of your internal smog of not-quite-truths.

I mean, that's one hypothesis about the psychological effects of adopting the wizard's code.

A potential problem with this is that human natural language contains a lot of ambiguity. Words can be used in many ways depending on context. Even the specification "literally" in "literally false" is less useful than it initially appears when you consider that the way people ordinarily speak when they're being truthful is actually pretty dense with metaphors that we typically don't notice as metaphors because they're common enough to be recognized legitimate uses that all fluent speakers will understand.

For example, if I want to convey the meaning that our study group has covered a lot of material in today's session, and I say, "Look how far we've come today!" it would be pretty weird if you were to object, "Liar! We've been in this room the whole time and haven't physically moved at all!" because in this case, it really is obvious to all ordinary English speakers that that's not what I meant by "how far we've come."

Other times, the "intended"[1] interpretation of a statement is not only not obvious, but speakers can even mislead by motivatedly equivocating between different definitions of words: the immortal Scott Alexander has written a lot about this phenomenon under the labels "motte-and-bailey doctrine" (as coined by Nicholas Shackel) and "the noncentral fallacy".

For example, Zvi Mowshowitz has written about how the claim that "everybody knows" something[2] is often used to establish fictitious social proof, or silence those attempting to tell the thing to people who really don't know, but it feels weird (to my intuition, at least) to call it a "lie", because the speaker can just say, "Okay, you're right that not literally[3] everyone knows; I meant that most people know but was using a common hyperbolic turn-of-phrase and I reasonably expected you to figure that out."

So the question "Is this statement actually and literally true?" is itself potentially ambiguo

...
What spiritual experiences have you had?
by Gordon Seidoh Worley — 2019-12-27T03:41:26.130Z

The recent adversarial collaboration on spiritual experiences on Slate Star Codex includes this paragraph:

It was also discovered that people in the United States, Australia, the United Kingdom, and Scandinavia do not tend to share their spiritual experiences with others. Hood et al. wonder if this is why such spiritual experiences are thought to be uncommon (as fewer people in these societies might have heard reports of others’ spiritual experiences).

This naturally lead me to wonder, what spiritual experiences have LessWrong readers have that they are willing to share, since the readership of LW is (very likely) overwhelmingly from Nordic and Anglophone countries?

NB: If you're like me, you find the term "spiritual experience" awkward because it suggests these are discrete events somehow separate from the rest of normal experience. The linked post mentioned the alternative term "mystical perception", which to me seems better on the "perception" front but worse on the "mystical" one. So please be a bit generous in what you count as "spiritual experience", noting it reasonably includes not just rare, special, and highly memorable experiences but also regular, ongoing experiences and common, easily-forgettable experiences because they blend together in your memory.
Naming Rooms
by jefftk — 2019-12-27T02:40:01.644Z

Growing up, the bedrooms in the house had clear names: Jeff's room,
Rose's room, Alice's room, Rick and Suzie's room, the Au Pair room,
and the guest room. But people have moved around a lot: later
occupants of "my" room have included Rose, Stevie, then later me,
Julia, and Lily, and then even later Alice, Alex, and their children.
Other rooms had a similar range of people rotating through ("the Wyman
St home for itinerant folk-dancing youth") and referring to rooms
became really difficult.

Around this time last year we decided to give names to the rooms:
England, Scotland, Wales, and Italy. We painted the flags on the
doors:

I was initially worried that the names wouldn't stick, and we'd be
stuck talking about
"Alice's—no—Rose's—no—Stevie and Val's old
room", but now that's just "Scotland". Some of the names get used
more than others—Rick's room is just called Rick's
room—but having an unambiguous option for each room is super
useful.

Possibly we should preemptively do something similar with our house,
but we're probably ~15y off from this sort of room rotation and for
now occupant-based naming is clearer.
Link: Does the following seem like a reasonable brief summary of the key disagreements regarding AI risk?
by Davidmanheim — 2019-12-26T20:14:52.509Z

This is a link to a question asked on the EA Forum by Aryeh Englander. (Please post responses / discussion there.)

Does the following seem like a reasonable brief summary of the key disagreements regarding AI risk?

Among those experts (AI researchers, economists, careful knowledgeable thinkers in general) who appear to be familiar with the arguments:

Seems to be broad (but not universal?) agreement that:

Superintelligent AI (in some form, perhaps distributed rather than single-agent) is possible and will probably be created one day
By default there is at least a decent chance that the AI will not be aligned
If it is not aligned or controlled in some way then there is at least a decent chance that it will be incredibly dangerous by default

Some core disagreements (starred questions are at least partially social science / economics questions):

Just how likely are all of the above?
Will we have enough time to see it coming, and will it be obvious enough, that people will react appropriately in time to prevent bad outcomes?

Still might be useful to have some people keeping tabs on it (Robin Hanson thinks about 100), but not that many

How hard is it to solve?

If easy then less time needed to see it coming, or inventors more likely to incorporate solutions by default
If really hard then may need a long time in advance

How far away is it?
Can we work on it profitably now given that we don't know how AGI will work?

If current ML scales to AGI then presumably yes, otherwise disagreement

Will something less than superhuman AI pose similar extreme risks? If yes: How much less, how far in advance will we see it coming, when will it come, how easy is it to solve?
Will we need coordination mechanisms in place to prevent dangerous races to the bottom? If yes, how far in advance will we need them?
If it's a low probability of something really catastrophic, how much should we be spending on it now? (Where is the cutoff where we stop worrying about finite versions of Pascal’s Wager?)

What about misuse risks, structural risks, or future moral risks?
Various combinations of these and related arguments result in anything from "we don't need to worry about this at all yet" to "we should be pouring massive amounts of research into this"
New paper: (When) is Truth-telling Favored in AI debate?
by VojtaKovarik — 2019-12-26T19:59:00.946Z
Link: https://medium.com/@RyanCarey/new-paper-when-is-truth-telling-favored-in-ai-debate-8f58f14562e5

An introduction to a recent paper by myself and Ryan Carey. Cross-posting from Medium.

For some intellectual tasks, it’s easy to define success but hard to evaluate decisions as they’re happening. For example, we can easily tell which Go player has won, but it can be hard to know the quality of a move until the game is almost over. AI works well for these kinds of tasks, because we can simply define success and get an AI system to pursue it as best it can.

For other tasks, it’s hard to define success, but relatively easy to judge solutions when we see them, for example, doing a backflip. Getting AI to carry out these tasks is harder but manageable — we can generate a bunch of videos of an AI system making some motion with a simulated body. Then we can give these videos to some people who allocate “approval” to the best-looking motions, and train the AI to maximize that approval until it does a backflip.

What makes AI really hard are tasks for which we have no definition of success, nor any timely way to evaluate solutions. For example: to which school should I send my kids? And should I accept this job or that one? One proposal for these cases is to use AI Debate. The idea is to ask AI systems how to perform a task, and then to have them debate about the virtues of different possible decisions (or answers). The question could be how to win a game of Go, how to do a backflip, which school to send your kids to, or, in principle, basically anything. The hope is that observing an AI debate would help the human judge to better-understand different possible decisions and evaluate them on-the-fly, even if success can’t yet be quantified.

One concern with such a scheme is whether it would be safe, especially if the AI systems used are super-smart. Critics ask: “How can we be confident that in AI Debates, the true answer will win?” After all, in human debates, rhetorical tools and persuasive techniques can cause an audience to be mislead.

To us, it seems wrong to imagine all debates will be safe. But it seems equally wrong to expect that none can be. A better question, to us, is: “In which debates will the winner be the true answer?” In our recent paper, we (Vojta and Ryan) have taken a first stab at making mathematical models that address this question.

So what is a debate exactly? According to our model, every debate revolves around some question posed by a human and c

...
Finding a quote: "proof by contradiction is the closest math comes to irony" 
by Hazard — 2019-12-26T17:40:41.669Z

I've got a paraphrased quote floating around in my mind, and I'm trying to track down the source. I think it was an article online but I have no idea where.

There was a sentence like "proof by contradiction is the closest math comes to irony." They then laid out a demonstration of a polynomial root that was imaginary and said "We've found a number that, when squared, is negative! These numbers are quite peculiar and further study is required of them."

It was then paralleled with a standard proof by contradiction of the irrationality of √2 , except the proof was ended with "We've found a number that is both odd and even! These numbers are quite peculiar and further study is required."

Does anyone know the source I'm referring to? Thanks!