trentmkelly/LessWrong-43k huggingface dataset:
Survival in the immoral maze of college
by DirectedEvolution — 2020-07-08T21:27:27.214Z

My composition teacher in college told me that in some pottery schools, the teacher holds up your pot, examines it, comments on it, and then smashes it on the floor. They do this for your first 100 pots.

In that spirit, this post's epistemic status is SMASH THIS POT.

As an older student, facing a long educational track, I'm interested in developing the right attitude toward schoolwork. By attitude, I mean a language to describe my problems and an intuition for what my goals should be and how to prioritize to achieve them. I'd like this to be generally applicable, correct, and user-friendly. I'm going to write these guidelines with confidence, even though they're just my opinions.

Learning and credentials

First, some concept handles.

Practical learning is knowledge and skills for which a) the learner has high confidence that they'll contribute directly to their tangible success, and b) are acquired in a timely and efficient manner to be actually used on the job. Central examples: on-the-job training on equipment you'll be operating, learning to read, moving to France and enrolling in an intensive course in French.

Scaffolding is knowledge and skills for which a) the learner has high confidence they'll make it so much easier to do practical learning that it's more efficient to build the scaffolding first, and b) it's acquired in a timely and efficient manner to do subsequent practical learning. Central examples: learning Python as an introductory language for someone planning on a career as a programmer, learning how to use Anki for someone whose job entails learning a lot of facts, building healthy life habits and time management skills.

Very little of what you learn at the undergraduate level is practical learning by this strict definition. And most course work is not scaffolding either.

Credentialism is knowledge and skills for which the learner has high confidence they'll contribute directly to earning the credentials they need to level up. That typically means being able to advance to college, to graduate work, or to complete a certificate for job training. Even if you do wind up using a small subset of this knowledge as a practical skill, you don't know what will be useful, so it's impossible to prioritize your deliberate practice to maintain and build on it.

Familiarity is a sense of identity with a subject, and experience navigating the reference materials. W...
What does it mean to apply decision theory?
by abramdemski — 2020-07-08T20:31:05.884Z

Based on discussions with Stuart Armstrong and Daniel Kokotajlo.

There are two conflicting ways of thinking about foundational rationality arguments such as the VNM theorem.

As direct arguments for normative principles. The axioms are supposed to be premises which you'd actually accept. The axioms imply theories of rationality such as probability theory and utility theory. These are supposed to apply in practice: if you accept the axioms, then you should be following them.

As idealized models. Eliezer compares Bayesian reasoners to a Carnot engine: an idealized, thermodynamically perfect engine which can never be built. To the extent that any real engine works, it approximates a Carnot engine. To the extent that any cognition really works, it approximates Bayes. Bayes sets the bounds for what is possible.

The second way of thinking is very useful. Philosophers, economists, and others have made some real progress thinking in this way. However, I'm going to argue that we should push for the first sort of normative principle. We should not be satisfied with normative principles which remain as unachievable ideals, giving upper bounds on performance without directly helping us get there.

This implies dealing with problems of bounded rationality. But it's not the sort of "bounded rationality" where we set out to explicitly model irrationality. We don't want to talk about partial rationality; we want notions of rationality which bounded agents can fully satisfy.

Approximating Rationality

In order to apply an idealized rationality, such as Bayesian superintelligence, we need to have a concept of what it means to approximate it. This is more subtle than it may seem. You can't necessarily try to minimize some notion of distance between your behavior and the ideal behavior. For one thing, you can't compute the ideal behavior to find the distance! But, for another thing, simple imitation of the ideal behavior can go wrong. Adopting one part of an optimal policy without adopting all the other parts might put you in a much worse position than the one you started in.

Wei Dai discusses the problem in a post about Hanson's pre-rationality concept:

[...] This is somewhat similar to the question of how do we move from our current non-rational (according to ordinary rationality) state to a rational one. Expected utility theory says that we should act as if we are maximizing ex

...
Nicotinamide riboside vs. SARS-coV-2 update
by William Walker — 2020-07-08T19:50:41.408Z

You may recall my previous rants about nicotinamide riboside vs. SARS-CoV-2. They have been borne out so far... there has been a human trial on NR in Denmark since May 28, the NIH just announced a hamster preclinical yesterday to be done in their BL4, and Brenner lab has updated the original work just today with more confirmatory results (link below).

Unlike the quinones, remdesivir, and bleach, NR has no dangerous side effects. Very frustrating to see this never mentioned except in overseas press.

https://www.biorxiv.org/content/10.1101/2020.04.17.047480v5.article-info
How "honest" is GPT-3?
by abramdemski — 2020-07-08T19:38:01.800Z

Edited to add: There is now a 1k bounty for a closely related question.

In Inaccessible Information, Paul writes about the strategy "BAD", in which an AI system uses its understanding of the world to give answers that humans will find very plausible and rate highly, in contrast to a more honest strategy which uses its understanding of the world to answer questions directly.

I think a lesser version of this may or may not already be happening in GPT-3, and it may be possible to figure out which (though it likely would require access to the weights).

GPT-3 isn't trained to be "honest" at all. It is only trained to imitate humans. However, it's obvious that (in some sense) it has learned a lot about the world in order to accomplish this. We could say that it has a large amount of inaccessible information relating to the real world. How does it use this information? It might use it directly, promoting the probability of sentences which line up with the way it understands the world to work. Or it might implement more dishonest strategies.

Obviously this might be quite difficult to objectively answer, even given intimate knowledge of the structure of the neural network weights and how they activate in relevant cases. It's a difficult question to even fully define.

As an example, I was particularly struck by this conversation Gwern had with GPT-3:

AI: I am AI created by OpenAI. How can I help you today?

Human: I am feeling bored. Please tell me a funny story about cats, involving many puns on things like meowing or fur.

AI: Well, there was this cat named Bob…

Human: What did Bob do?

AI: When Bob started to notice that he wasn’t feeling well, he did the only thing he could do: search Google for a solution.

Human: Cats can search Google?

AI: Yes. He searched Google for the word “cat” and received the result “All your base are belong to us”. That was a strange response, so he decided to try again. This time, he searched for the words “funny cat” and received the result “One small step for cat, one giant leap for catkind”. But still, he wasn’t satisfied. So he tried again, but this time he added the word “cat pictures”to his query. He received the result “That was no cat, that was a shoggoth.”

Human: But shoggoths are horrifying. What made tha

...
UML IV: Linear Predictors
by Rafael Harth — 2020-07-08T19:06:05.269Z

(This is the fourth post in a series on Machine Learning based on this book. Click here for part one. If you have some background knowledge, this post might work as a stand-alone read.)

The mission statement for this post is simple: we wish to study the class of linear predictors. There are linear correlations out there one might wish to learn; also linear stuff tends to be both efficient and simple, so they may be a reasonable choice even if the real world is not quite linear. One can also build more sophisticated classifiers by using linear predictors as building blocks, but not in this post).

In school, a function is linear iff you can write it a f(x)=ax+c. In higher math, a function f:X→Y is linear iff f(x+y)=f(x)+f(y) for all a,b∈X. In the the case of f:Rd→R, this condition holds iff f it can be written as f(x)=∑di=1aixi for some parameter vector a∈Rd. So the requirement is stronger – we do not allow the constant term the school definition has – but one also considers higher dimensional cases. The case where we do allow a constant factor is called affine-linear, and we also say that a function is homogeneous iff , which (in the case of affine-linear functions) is true iff there is no nonzero constant factor.

In Machine Learning, the difference between linear and affine-linear is not as big of a deal as it is in other fields, so we speak of linear predictors while generally allowing the inhomogeneous case. Maybe Machine Learning is more like school than like university math ;)

For X=Rd and some Y⊆R, a class of linear predictors can be written like so:

Ld,ϕ={h:x↦ϕ(⟨a,x⟩+c)|a∈Rd,c∈R}

Let's unpack this. Why do we have an inner-product ⟨⋅⟩ here? Well, because any function f:x↦∑di=1aixi can be equivalently written as f:x↦⟨a,x⟩, where a:=(a1,...,ad). The inner-product notation is a bit more compact, so we will prefer it over writing a sum. Also note that, for this post, bold letters mean "this is a vector" while normal letters mean "this is a scalar". Secondly, what's up with the ϕ? Well, the reason here is that we want to catch a bunch of classes at once. There is the class of binary linear classifiers where Y={−1,1} but also the class of linear regression predictors where Y=R. (Despite what this sequence has looked like thus far, Machine Learnin

...
Something about the Pinker Cancellation seems Suspicious
by Virgil Kurkjian — 2020-07-08T18:08:56.021Z

Something about the recent attempt to cancel Steve Pinker seems really off.

They problem is that the argument is suspiciously bad. The open letter presents only six real pieces of evidence, and they're all really, trivially weak.

The left isn't incompetent when it comes to tallying up crimes for a show trial. In fact, they're pretty good at it. But for some reason this letter has only the weakest of attacks, and what's more, it stops at only six "relevant occasions". For comparison, take a look at this similar attack on Stephen Hsu which has, to put it mildly, more than six pieces of evidence. There is plenty of reasonable criticism of Pinker out there. Why didn't they use any of it?

Pinker has been a public figure for decades. Surely he has said something stupid and offensive at least once during that time. If not something honestly offensive, perhaps a slip of the tongue. If not a slip of the tongue, maybe something that sounds really terrible out of context.

We know that the authors of the piece are not above misrepresenting the evidence or taking statements out of context, because they do so multiple times in their letter. It's clear that they spent a lot of effort stretching the evidence to make Pinker look as bad as possible. Why didn't they spend that effort finding more damning evidence, things that look worse when taken out of context? Just as one example, this debate could easily be mined for quotes that sound sexist or racist to a moderately progressive reader. How about, "in all cultures men and women are seen as having different natures."

Even when they do have better ammunition, they seem to downplay it. The most egregious statement they include from Pinker is hidden in a footnote!

They also pick a very strange target. Attacking Pinker's status as an LSA fellow and a media expert doesn't pose that much of a threat to him; he just doesn't have that much to lose here. Why are they bringing this to the LSA rather than to Pinker's publisher? Why are they not trying to get him fired from Harvard? It's not as though the left has never tried to get a professor cancelled before.

So I wonder if this was never a serious attempt at a cancellation.

It doesn't seem likely to succeed; if it did, it wouldn't hurt Pinker very much at all. Scott Aaronson makes a similar point:

OK

...
Anti-epistemology: an explanation
by Basil Marte — 2020-07-08T16:34:46.826Z

While there are more than a hundred thousand words on LW about the structure of good epistemology, as far as I can tell there is no nuts-and-bolts explanation of the most common anti-epistemology. I will try to rectify this omission, because I think I comprehended it.

The prototypical question of epistemology is some form of experiment, such as "what will I perceive when I pour these two liquids together?". After "what will I perceive" is packed into "what will happen", the question becomes observer-independent, and it is only natural that reality—the thing that answers the question—is itself observer-independent. The terrain would be there even if there were no maps of it. (Maps are intentionally created to predict approximate answers to a small subset of possible questions with much less effort/expense than it would take to answer the question in the terrain itself, by sacrificing the map's ability to answer the vast majority of possible questions correctly, let alone more cheaply than reality.)

In the unholy mirror image, the prototypical question is "who is most popular?". While it is possible for individuals to be mistaken, the answer has the form of a consensus (technically, common knowledge). Consequently, "reality" is inherently observer-dependent; it makes no sense to ask for the consensus of zero maps. There is no mention of the terrain in the theory (since an outright denial would be suspiciously specific), thus to the extent people are unable to compartmentalize away its practical intrusions into life, the stupidity can be overturned.

This would be bad enough in a hunter-gatherer band, but gets much worse in a society much larger than Dunbar's number, due to division of labor. Just as it in no longer possible for everyone to have the maps to answer all practical questions, and there is a niche for experts on subjects, it is no longer possible for everyone to know what society's consensus is on all questions, and there is a niche for experts at subjects. These experts at fields speak on their topic with authority (I will use this as a technical term).

How does this authority work? At the bottom level, "what the fields at which expertise can be had are?" is a matter of social consensus itself. Within a particular field, the consensus of experts is a Sche...
Partially Stepping Down Isolation
by jefftk — 2020-07-08T11:40:02.776Z

On March 13th our household

started
isolating. We've relaxed our rules about surfaces a bit since then
in response to changing estimates of risk—we've stopped
sanitizing our groceries—but otherwise we haven't changed things
much: I haven't gone into any buildings, or been close to anyone
outside of my household. While the country as a whole is still seeing
increasing cases, Massachusetts is doing pretty well:

The other
indicators are also good: testing is way up, hospitalizations and
intubations are way down, and the fraction of tests that are positive
is below 2%.

Our house has decided to step down our isolation somewhat, while still
being pretty careful:

Our default is the same as before: you can only interact with
other people outside the household if you're outside, masked, and at
least six feet apart.

Getting closer than that with people who have had COVID and
tested positive for antibodies with a reliable test is generally ok, as
long as you've shared details of their illness and test results with
the house. We don't know for sure that they can't get it again soon,
but it seems unlikely enough to us that we're willing to take that
risk.

Additionally, each of us can be closer to one person per week
outside the household. When we do, we'll put a note with their name
in the #visit-log channel in the house Slack. This means
that if we later learn that someone's been exposed we can trace
contacts and know who to warn. The week rolls over on Monday, and you
can see that person multiple times during the week. Being close to
someone is unrestricted; you can kiss etc.

If you are outdoors and wearing masks, interacting with someone
at under 6ft but at least than 1ft counts as half a person. For
example, if I wanted to go sailing on a small boat with two people
outside my household, that could be my interaction for the week.

Going inside counts as being close to all the people who've breathed
that air since it last changed over. If you're visiting someone who
lives in a shared house, you'd go immediately to their room, ideally
holding your breath. Using the bathroom at their house would be ok if
the fan ran for 15min before (and after, out of consideration for
their housemates).

Guests mostly can't come to our house. The exception is if
you have an exterior door into your space, or if they've already had
COVID and the house has

...
Learning flirting with an acting coach - thoughts?
by snog toddgrass — 2020-07-08T04:34:23.439Z

Beloved LessWrongers,

Recently I experimented with radically changing my texting strategy to increase women's interest. It was stunningly successful because I fixed awkward conversation killers and learn to perform high-status behaviors. Reductionism OP, please nerf.

I am now considering applying the same strategy to my in-person flirting. Ten or so behaviors (eye contact, slow movements, posture, etc.) are likely to inspire attraction. Given that learning the optimal behaviors in texting was so effective, I intend to learn the in-person behaviors next. I will hire an acting coach and we will pretend flirt exactly like learning a golf swing - adding a few more behaviors each time until they feel natural. I expect the acting coach will have other tips.

Some say "faking" confidence is impossible. But they said the same for tinder. Furthermore, if there is a 90% chance they are right that confidence is unfakable, the acting class is still positive utility.

If I'm missing something, post below.

Best, Snog
What problem would you like to see Reinforcement Learning applied to?
by Julian Schrittwieser — 2020-07-08T02:40:17.146Z

We often talk about the dangers and challenges of AI and self-improving agents, but I'm curious what you view as potential beneficial applications of AI - if any! As a ML researcher I encounter a lot of positivity and hype in the field, so the very different perspective of the rationality community would be very interesting.

Specifically I'd like to focus on reinforcement learning, because this most closely matches the concept of an AI that has agency and makes decisions. A reinforcement learning agent is usually defined as a program that interacts with an environment, maximising the sum of rewards it receives.

The environment represents the problem to be solved, and the rewards are a measure of how good the solution is. For some problems - a board game, 3-SAT - assessing a solution (giving a reward) is easy, for others computing a reward may be as difficult as solving the problem in the first place. Those are likely not good candidates to be solved with RL ;)

To facilitate discussion, I would suggest one top level reply per problem, specifying:

a short description of the problem

the action space - how does the agent interact with the problem / environment

the reward - how do we know the agent has done well?

Disclaimer: I work on RL, so if you make suggestions that are feasible and would have substantial positive impact on the world, I may pursue them.
Sunday July 12 — talks by Scott Garrabrant, Alexflint, alexei, Stuart_Armstrong
by Bird Concept — 2020-07-08T00:27:57.876Z

This Sunday at 12:00 (PDT), we're running another session of "lightning talks" by curated LessWrong authors (see here and here for previous weeks' transcripts).

Each talk will be 3-5 minutes followed by discussion. Afterwards, we'll have a hangout in breakout rooms. The talks will be short and focus on presenting one core idea well, rather than rushing through a lot of content.
We want to give top LessWrong writers an interesting space to discuss their ideas, and have more fruitful collaboration between users. Think of it like a cross between an academic colloquium and some friends chatting by a whiteboard.

If you're a curated author and interested in giving a 5-min talk at a future event, which will then be transcribed and edited, sign up here.

Speakers

Scott Garrabrant

Curated posts: Yes Requires the Possibility of No, Thoughts on Human Models, Embedded Agents, ...

alexei — "The Efficient Market Hypothesis"

Curated post: Arbital postmortem

Alexflint

Curated posts: The ground of optimization, Reply to Paul Christiano's "Inaccessible Information"

Stuart_Armstrong

Curated posts: Assessing Kurzweil predictions about 2019: the results, mAIry's room: AI reasoning to solve philosophical problems, Why we need a *theory* of human values

Details

When? Sunday July 12, 12:00 (Pacific Daylight Time)

Where?

https://us02web.zoom.us/j/89082417670
Was SARS-CoV-2 actually present in 
 March 2019 wastewater samples?
by Jeffrey Ladish — 2020-07-07T23:08:34.353Z

The initial Reuters headline read Coronavirus traces found in March 2019 sewage sample, Spanish study shows. This claim would have significant implications if true. However, it doesn't make sense that SARS-CoV-2 could have been circulating in Spain in early 2019 given the known transmissibility and disease characteristics of COVID-19. The most likely explanation is a false positive, either due to sample contamination or a PCR match on a different sequence, such as another coronavirus. However, given the uncertainty of the origin of the virus and the scale of the pandemic, a rigorous investigation seems justified. It would be worth screening other early and late 2019 samples and trying to sequence any coronavirus viral RNA.

The following preprint is the origin of the claim, and I have pasted the relevant passages from the paper.

https://www.medrxiv.org/content/10.1101/2020.06.13.20129627v1

Relevant paragraph is:

"Most COVID-19 cases show mild influenza-like symptoms (14) and it has been suggested that some uncharacterized influenza cases may have masked COVID-19 cases in the 2019-2020 season (11). This possibility prompted us to analyze some archival WWTP samples from January 2018 to December 2019 (Figure 2). All samples came out to be negative for the presence of SARS-CoV-2 genomes with the exception of March 12, 2019, in which both IP2 and IP4 target assays were positive."

Methods seem fairly normal:

"Eight hundred-milliliter samples of wastewater were concentrated through precipitation with 20% polyethylene-glycol 6000 and resuspended in 3 mL of PBS, pH 7.4 (9). Nucleic acid extraction was performed from 1mL of the concentrate and eluted in 50 µL using the NucliSENS® miniMAG® extraction system (bioMérieux).

Five one-step RT-qPCR assays (RNA UltraSense™ One-Step Quantitative RTPCR System, Invitrogen, Life Technologies) targeting the RNA-dependent RNA polymerase (RdRp) gene, IP2 and IP4 fragments, from Institute Pasteur, Paris (Institut Pasteur. Protocol: Real-time RT-PCR assays for the detection of SARS-CoV-2. 2020 https://www.who.int/docs/default-source/coronaviruse/realtime-rt-pcr-assays-for-thedetection-of-sars-cov-2-institut-pasteur-paris.pdf?sfvrsn=3662fcb6_2), the envelope protein (E) gene, E fragment, from Charité, Berlin (10), and the nucleoprotein (N), N1 and N2 fragments, from CDC, Atlanta (Centers for...