trentmkelly/LessWrong-43k huggingface dataset:
Deliberately Vague Language is Bullshit
by lsusr — 2021-05-14T09:17:23.640Z

Bullshit is what comes out of the mouth of someone who values persuasion over truth. Truth is about communicating information. Vague language communicates less information than precise language.

It is important to distinguish vague language from general, poetic and simple language.

Vague language is distinct from general language. Vague language communicates less information than regular language. General language communicates more information than regular language. Broader claims are more easily falsifiable than narrow claims. The more falsifiable a claim is the stronger a claim it is. The stronger a claim is the more information it communicates.

Vague language is distinct from poetic language. Poetic language has very high information density. High information density is the opposite of vagueness.

Vague language is distinct from simple language. Simple language is easy to understand. Simple language should be used most of the time because simple language is more convenient than complicated language. In disagreements, simple vague language should be replaced with precise language—even at the cost of some complexity.

Deliberately vague language increases vagueness without increasing generality, poetry or simplicity. It obscures the truth.

"No evidence" is vaguer than "evidence of absence".

How many shares of a company you own is vaguer than what percentage you own.

The people with a need to obscure the truth are those with a political or social agenda. Vague language is endemic to taboo subjects. Deliberately vague language is such a reliable signal of taboo subjects that you can use it to discover hidden taboos. If you find yourself or others using deliberately vague language around a topic it means there is something you can't say.

The most precise language is numerical. If numbers could be used but aren't then you are dealing with a taboo subject.

"Smart" is vaguer than IQ thresholds.

"Fit" is vaguer than BMI and bodyfat percentage.

You can even use vague language to figure out which subjects are bullshit and which people are bullshitters.

Modern art uses vague language. Drawing manuals like Scott McCloud's Understanding Comics: The Invisible Art use precise language.

Literary theory is notorious for using vague language. Brandon Sanderson's lectures on creative writing use precise language
Zvi's Law of No Evidence
by lsusr — 2021-05-14T07:27:51.474Z

Update: Zvi wrote his own post.

Law of No Evidence: Any claim that there is “no evidence” of something is evidence of bullshit.

―Covid 5/13: Moving On by Zvi

But in probability theory, absence of evidence is always evidence of absence

―Absence of Evidence Is Evidence of Absence by Eliezer Yudkowsky

This is not a contradiction. Zvi and Yudkowsky are both correct.

"No evidence" is a vague term. Consider the phrase "There is no evidence of Bigfoot". It could mean one of two things.

Explorers looked for Bigfoot and didn't find him. This is evidence Bigfoot doesn't exist.

Nobody looked for Bigfoot and nobody found him. This is not evidence Bigfoot doesn't exist.

It is true "no evidence" is evidence of absence insofar as "no evidence" means "we ran good experiments and they failed to produce evidence for the hypothesis". But in this case you shouldn't say "no evidence". You should say "evidence of absence".

Inference starts with priors and are then updates them according to evidence. If there is no evidence related to a hypothesis then your beliefs reflect your original priors. If "no evidence" refers to true "absence of evidence" then everyone needs to retreat to their Bayesian priors. But this not what anyone means when they say "no evidence" because your Bayesian priors for one context are informed by transfer learning of evidence from other contexts e.g. I have a computer programmer friend who thinks about everything in terms of software metaphors.

If someone deliberately uses vague[1] language instead of precise language then it is because they are trying to conflate two ideas which should not be conflated. If someone says there is "no evidence" of something then it is because they are trying to pass off "nobody looked for Bigfoot and nobody found him" as "explorers looked for Bigfoot and nobody found him".

Saying "there is no evidence of UFOs" communicates nothing useful. You must explain why absence of evidence constitutes evidence of absence.

If you want to prove that red light doesn't cause cancer you don't say there is "no evidence". You should explain the frequency dependence of the photoelectric effect.

If someone deliberately uses general language instead of precise language then that is good because general language is easier to falsify than precise language. Vague language is bullshit because it is hard to falsify. ↩︎
Hypothesis Space Entropy
by lsusr — 2021-05-14T07:21:21.349Z

In Why quantitative finance is so hard I explained why the entropy of your dataset must exceed the entropy of your hypothesis space. I used a simple hypothesis space with n equally likely hypotheses each with m tunable parameters. Real life is not usually so homogeneous.

No Tunable Parameters

Consider an inhomogeneous hypothesis space with zero tunable parameters. Instead of H=logn which works for homogeneous hypothesis spaces, we must use more complicated entropy equation.

H=−n∑i=1ρilnρi

This equation makes intuitive sense. It vanishes when one ρi=j equals 1 and all other ρi≠j equal 0. Our equation is extremized when all ρi are equal at 1n. H=logn is the maximal case when ρi=1n∀i∈{1,…,n}[1].

With Tunable Parameters

Suppose each hypothesis i has mi tunable parameters. We can plug mi into our entropy equation.

H=n∑i=1ρi(mi−lnρi)

Our old equation H=m+logn is just the special case where all ρi are homogenous and mi are homogeneous too.

We have so far treated mi as representative of each hypothesis's tunable parameters. More generally, mi represents each hypothesis's internal entropy. If we think of hypotheses as a weighted tree, mi is what you get when you iterate one level down the tree. Our variable H identifies the root of the tree. Suppose ith branch of the next level down is called Hi.

H=n∑i=1ρi(Hi−lnρi)

We can define the entropy of the rest of the tree with a recursive equation.

Hμ=n∑i=1ρi(Hμ,i−lnρi)=n∑i=1(ρiHμ,i−ρilnρi)

There are two parts to this equation: the recursive component ρiHμ,i and the branching component −ρilnρi.

Branching component −ρilnρi

The −ρilnρi component is maximized when ρi=1e.

−ρilnρi=−1eln1e=1elne=1e

The branching component tops out at 1e. It can never contribute a massive quantity of entropy to your hypothesis space because it is limited to 1e entropy per level of the tree.

0≤−ρilnρi≤1e

The branching factor is mostly unimportant. The bulk of our entropy comes from the recursive component.

Recursive component ρiHμ,i

Fix ρi at a positive value. There is no limit to how big Hμ,i can become. You can make it arbitrarily large just by adding parameters. Consequently ρiHμ,i can become arbitrarily large too. In real world situations we should expect the recursive components of our hypothesis space to dominate the branching components.

If ρi vanishes then the recursive component disappears. This might explain why human minds like to round "extremely unli...
Does butterfly affect?
by pchvykov — 2021-05-14T04:20:58.374Z

Can a butterfly really cause a hurricane?

Having been working in complexity science, I realize that I have a problem with the conventional understanding of the butterfly effect. Sure, in ideal deterministic chaotic systems, the butterfly effect does a great job illustrating the quasi-stochastic nature of chaos and sensitivity to small perturbations. But in the real complex world, far from a mathematically idealized deterministic model, there is no reasonable sense in which we can say that a butterfly’s wings can cause a hurricane. Even if this isn’t surprising, as intuitively people won’t attribute a hurricane to a butterfly, I would argue that causally attributing World War I to the assassination of Archduke Ferdinand follows the same fallacy.

Consider a pot of supercooled water. Any minúte impurity in it can seed the freezing transition of the entire pot. But can we really say that this impurity “caused” the transition? This seems to depend on the counterfactual world: what would have happened otherwise? If the pot is sitting in an isolated clean room where there would not typically be any dust other perturbations, then this one particular impurity may be to blame for the phase-transition. But out in the messy “real world,” abundant with impurities of all kinds, supercooled water would not survive long anyhow — if not this impurity, then another would have seeded the transition a moment later.

Now, this counterfactual definition of causality,

formalized by Judea Pearl

using tools information theory, similarly gives problems for the classic interpretation of the butterfly effect. Would the hurricane have happened if not for the butterfly? To answer this question carefully, it isn’t enough to look at what the world would look like if we removed the butterfly, but held everything else fixed. Instead, we must consider the full statistical ensemble of possible world, and quantify to what extent the butterfly shifts that ensemble. To model this in a mathematically idealized setting, we could take some deterministic chaotic system, add some small stochastic noise to it at all times t to generate the statistical ensemble of possibilities, and then consider the effect of some slight “butterfly” perturbation at a given time t*. In the typical scenario, the impact of this single perturbation will not rise above the impact of the persistent background noise inherent to a...
AXRP Episode 7 - Side Effects with Victoria Krakovna
by DanielFilan — 2021-05-14T03:50:11.757Z

YouTube link

This podcast is called AXRP, pronounced axe-urp and short for the AI X-risk Research Podcast. Here, I (Daniel Filan) have conversations with researchers about their papers. We discuss the paper and hopefully get a sense of why it’s been written and how it might reduce the risk of artificial intelligence causing an existential catastrophe: that is, permanently and drastically curtailing humanity’s future potential.

One way of thinking about how AI might pose an existential threat is by taking drastic actions to maximize its achievement of some objective function, such as taking control of the power supply or the world’s computers. This might suggest a mitigation strategy of minimizing the degree to which AI systems have large effects on the world that are not absolutely necessary for achieving their objective. In this episode, Victoria Krakovna talks about her research on quantifying and minimizing side effects. Topics discussed include how one goes about defining side effects and the difficulties in doing so, her work using relative reachability and the ability to achieve future tasks as side effects measures, and what she thinks the open problems and difficulties are.

Daniel Filan:
Hello everybody. Today, I’ll be speaking to Victoria Krakovna, a research scientist at DeepMind, working on designing incentives for AI systems to promote safety. She also co-founded the Future of Life Institute, a nonprofit that focuses on reducing existential risks from advanced technologies, while working on her PhD in statistics at Harvard. Today, we’ll be talking about her work on mitigating side effects from AI systems, and in particular, the papers Penalizing Side Effects Using Stepwise Relative Reachability and Avoiding Side Effects by Considering Future Tasks. For links to these papers and other useful things, you can check the description of this podcast, and you can read a transcript at axrp.net. Victoria, welcome to AXRP.

Victoria Krakovna:
Thanks Daniel. I’m excited to be here.

Daniel Filan:
Great to hear. We’re talking about your work on side effects, can you tell us what is a side effect?

Victoria Krakovna:
It’s when the agent has an impact on the environment that’s unnecessary for achieving its objective. A classic toy example is, if the agent’s objective is to cross the room and there happens to be a vase in the middle of the room, then if the direct path ...
Understanding the Lottery Ticket Hypothesis

by Alex Flint — 2021-05-14T00:25:21.210Z

Financial status: This is independent research. I welcome financial support to make further posts like this possible.

Epistemic status: The thread of research I’m reviewing here has been contentious in the past so I expect to make updates based on the comments.

Outline

This is my attempt to understand the lottery ticket hypothesis.

I review the original paper, as well as two follow-up papers and one related paper.

I also go over four previous lesswrong posts on this subject.

Introduction

The lottery ticket hypothesis is a hypothesis about how neural network training works. It was proposed in 2018 by Jonathan Frankle, a PhD student at MIT, and Michael Carbin, a professor at MIT. It suggests that, in a certain sense, much of the action in neural network training is during initialization, not during optimization.

Research that sheds light on neural network training is relevant to alignment because neural network architectures may eventually become large enough to express dangerous patterns of cognition, and it seems unlikely that these patterns of cognition can be detected by input/output evaluations alone, so our only choices seem to be (1) abandon the contemporary machine learning paradigm and seek a new paradigm, or (2) augment the contemporary machine learning paradigm with some non-input/output method sufficient to avoid deploying dangerous patterns of cognition. Insights into contemporary machine learning effectiveness is relevant both to determining whether course (1) or (2) is more promising, and to executing course (2) if that turns out to be the better course.

The lottery ticket hypothesis

The lottery ticket hypothesis (or LTH), as originally articulated by Frankle and Carbin, says:

A randomly-initialized, dense neural network contains a subnetwork that is initialized such that—when trained in isolation—it can match the test accuracy of the original network after training for at most the same number of iterations.

A "subnetwork" means that you take the original neural network and clamp some of the weights to zero, so a network with N weights has 2N subnetworks. The lottery ticket conjecture, which is an extension of the lottery ticket hypothesis but which Frankle and Carbin are careful to point out is not tested directly in their original paper says:

SGD seeks out and trains a subset of well-initialized weights. Dense, randomly-initialized n

...
Vaccine Rollout as Wheeled-Luggage Problem
by DirectedEvolution — 2021-05-13T21:04:04.363Z

According to FiercePharma, Pfizer, AstraZeneca, Moderna, Johnson & Johnson, and Novavax plan to produce a combined ~9 billion doses this year. They are collaborating with other companies to scale up production, and have facilities all over the world. They're licensing their product to nonprofit producers, and some have committed to providing the vaccines at cost.

The fact that these companies project to produce enough vaccine to vaccinate the entire world this year is great. Why didn't they have it ready to go earlier?

Note that this is a similar question to "Why did it take so long to invent wheeled luggage?" Why has it taken so long to ramp up production, when the first mRNA vaccines were created in January 2020?

Jason Crawford proposes epistemic standards for this sort of question:

On the other hand, if you want to argue that something could have been invented much earlier than it was, you have to do better than glancing at its high-level concepts or components. You need to rigorously examine every part, material, and manufacturing process, and rule them all out as gating technologies. Any detail, even a minor one, can become crucial—especially when we remember that inventions need not only to work but to be practical, which includes performance, reliability and cost.

We want to identify the fundamental bottleneck for ramping up vaccine supply. I've seen lots of attempts. They usually boil down to the idea that with better planning and less regulation, we could have spent a relative pittance for more production capacity early on, and might have been able to produce the billions of needed doses many months sooner.

That explanation does not meet the Crawford Standard. Instead, we need to rigorously examine every aspect of vaccine manufacturing and rule them all out as bottlenecks.

First, we need to consider planning and regulation failures prior to 2020. What failures of planning and regulation prevented us from setting up sufficient emergency vaccine production capacity before COVID-19 emerged? Could we have had a global network of pandemic vaccine factories on standby back in 2015?

Secondly, we need to consider the practical and material challenges producers must have faced once they'd developed their vaccines. If they'd had the money and governmental support, could they have ramped up production faster than they did? Or would they have faced problems sourcin...
Covid 5/13: Moving On
by Zvi — 2021-05-13T14:30:01.849Z

For over a year, Covid-19 has been the central fact of life.

The goal now is to make that no longer true.

If you’re reading this, chances are very high you are vaccinated. If you’re not reading this, but you live in the United States, chances are still pretty good you’re vaccinated.

The question everyone is asking is now, can life return fully to normal?

Yes.

Well, almost. We’ll never be psychologically quite the same. We’ll never unlearn the lessons we’ve learned over the past year – nor would we want to – about the way our civilization and its institutions work, or about what matters in life. And at least for now, we’ll need to continue to worry about how others will interact with us and how to navigate people’s concerns and various governmental restrictions, as well as the Covid-19 conversations that they’ll doubtless want to have for a long time. If others don’t return to normal, there’s no fully normal to return to yourself.

And that doesn’t mean quite fully normal quite yet, in the sense that there’s an amount of indoor crowding I’d still be inclined to avoid for the next few weeks or months.

And of course, things may be going well in America and most other highly vaccinated places, but the worldwide pandemic is far from over. Things in many other places remain quite bad, and will be quite bad for some time.

But… mostly? Yes. For those who are fully vaccinated, life can safely return to normal.

This column can also, events willing, start winding down or transitioning to other matters. If things go as planned, there will steadily be less Covid-19 news to talk about each week, and I can shift my blogging time into other, longer-term pursuits once more.

For now, let’s run the numbers.

The Numbers

Predictions

Prediction from last week: Positivity rate of 3.5% (down 0.4%) and deaths decline by 7%.

Result:

Things improved faster than I expected, which is great. The fall in deaths makes perfect sense, as I was adjusting for strangely small drops in deaths from previous weeks. The fall in cases is mostly in line with expectations.

Prediction for Next Week: Positivity rate of 3.0% (down 0.4%) and deaths decline by 10%.

This looks like the endgame. The control system is the potential threat to that, as we’re likely not near full herd immunity if everyone went fully back to the old normal, but I do not expect people to return to the full old no...
We can understand value only through endless self-reflection
by Riccardo Volpato — 2021-05-13T10:22:17.626Z
Link: https://volpato.io/articles/2105-why_doing_something.html

Once I was talking with a friend about an upcoming career decision. I said I wanted to do something "valuable" and he said that he recognises value when he sees it. Understanding values has never been easy for me, so I was intrigued by his sentence. I started asking myself some questions. Why doing something valuable felt much harder to me than to him? How did our approach to understanding what is valuable differed? Is determining value an objectively hard problem or not?

To start, I reviewed and reflected on some common ways in which we attempt to understand what we and others value.

Common ways of understanding value

Words

How many times when you asked a friend for help on a decision they answered you "it depends on what you want"? How many times after that question you started describing that wanted to start that business or how you would feel incredibly good if only you found a partner that was right for you?

These are examples of explicitly describing what we value in words. It is the most common way in which we find out what we value: by directly answering "what do you want/value?". Sadly, determining value in this way is often inaccurate. Very often, we say we want things that we don't really want, for many reasons. Other times, we want things that we can't admit we want. Derek Sivers gives a great example of this from his personal life. This is also why surveys and interviews are often a poor way to understand what someone wants.

Similarly, when people engage in life coaching, they usually start by finding out what they value. There are different ways to do this: shortlisting people we look up to and working out what they represent for us, go through a list of value words and pick what resonates with us, comparing values we like and selecting which ones we would choose even over another value we care about and many other methods.

I tried several of them. The result is a list of abstract ideas I felt somewhat emotionally attached to. However, I rarely refer back to it. It is rare that I have to decide between an option that closely matches my values and one that doesn't. More often, I have to decide between options that contain some mixed elements of most of my values.

This is true for companies as well. Startups are encouraged to clarify their purpose and values early on to establish a strong culture. At large corporations, values often take up a large po...