trentmkelly/LessWrong-43k huggingface dataset:
[AN #116]: How to make explanations of neurons compositional
by Rohin Shah — 2020-09-09T17:20:04.668Z

HIGHLIGHTS

Compositional Explanations of Neurons (Jesse Mu et al) (summarized by Robert): Network dissection is an interpretability technique introduced in 2017, which uses a dataset of images with dense (i.e. pixel) labels of concepts, objects and textures. The method measures the areas of high activation of specific channels in a convolutional neural network, then compares these areas with the labelled areas in the dataset. If there's a high similarity for a particular channel (measured by the intersection divided by the union of the two areas), then we can say this channel is recognising or responding to this human-interpretable concept.

This paper introduces an extension of this idea, where instead of just using the basic concepts (and matching areas in the dataset), they search through logical combinations of concepts (respectively areas) to try and find a compositional concept which matches the channel's activations. For example, a channel might respond to (water OR river) AND NOT blue. This is still a concept humans can understand (bodies of water which aren't blue), but enables us to explain the behaviour of a larger number of neurons than in the original network dissection method. Their work also extends the method to natural language inference (NLI), and they interpret neurons in the penultimate layer of a BiLSTM-based network trained to know whether a sentence entails, contradicts, or is neutral with respect to another. Here they create their own features based on words, lexical similarity between the two sentences, and part-of-speech tags.

Using their method, they find that channels in image classifiers do learn compositional concepts that seem useful. Some of these concepts are semantically coherent (i.e. the example above), and some seem to have multiple unrelated concepts entangled together (i.e. operating room OR castle OR bathroom). In the NLI network, they see that many neurons seem to learn shallow heuristics based on bias in the dataset - i.e. the appearance of single words (like nobody) which are highly informative about the classification.

Finally, they use their method to create copy-paste adversarial examples (like in Activation Atlas (AN #49)). In the Places365 dataset (where the goal is to classify places), they can crudely add images which appear in compositional concepts aligned with highly contributing neurons, to mak...
Loneliness and the paradox of choice
by DirectedEvolution — 2020-09-09T16:30:20.374Z

Loneliness is a problem of decision-making. Being around other people, whether in a social or work setting, comes with many supports to our decision-making process. In an idealized process of decision-making, each time the question "What's next?" pops up, we imagine a menu of options. We choose one of them, and then we go through a psychological process of committing to it. This leads to activity, and both within that activity and after it, we repeat this algorithm many more times.

Think of all the ways that the presence of other people can support this process. Just being around others comes with cultural expectations about the options we choose from in response to that constant question, "What's next?" Settings where two or more people are gathered also typically have some implicit or explicit structure around how options will be chosen. And once an option is chosen, the simple fact that each individual knows that it has been registered as the group's choice acts as a commitment device.

By contrast, when an individual is on their own, that simple fact does not apply. There is no group to have registered a collective choice. Have you ever had a week entirely to yourself, perhaps when you were single, and had the experience of making choices about what to do, only to spend a lot of time revisiting that choice? Considering other options even after you've selected one? Asking whether you picked the highest-priority option, even though in retrospect it didn't really matter very much which you chose?

Even more difficult for the individual is the fact that because there is no second person present, you are unable to take their preferences into consideration as you invent and weigh the options for what to do. If you do not have a strong sense of your own preferences and needs, or have some other binding constraint such as a high-pressure work schedule, then this might pose a real problem. The options might all feel a bit arbitrary. Witnessing yourself as a person who does arbitrary activities simply to fill the time may not be a pleasant reflection to see in the mirror.

Finally, the whole process of making decisions, or a particular step in it, can feel fraught. Rather than appearing to the mind as a matter of fact process, a puzzle to solve with clear steps, sometimes the mind simply flails. Ever had a dream where you had to play in a major league sports match for a ...
Leslie's Firing Squad Can't Save The Fine-Tuning Argument
by dadadarren — 2020-09-09T15:21:19.084Z
Link: https://www.sleepingbeautyproblem.com/7-1-leslies-firing-squad/

This post argues against fine-tuning, especially the fine-tuning argument for design. It again shows the importance of perspectives in reasoning as discussed in a previous post.

I previously argued why fine-tuning is a misconception. Reasoning from a first-person perspective, I endorse the Weak Anthropic Principle (WAP) response. Leslie’s Firing Squad is perhaps the most famous argument against this response. I think it would be interesting to examine it in detail.

It should be pointed out my objection to fine-tuning differs from most, if not all, anthropic objections since perspective-based argument (PBA) rejects the notion of observation selection effect (OSE). The WAP should only be regarded as a logical truth, not as selection bias. Nonetheless, the firing squad argument can still be used to counter PBA. Therefore it deserves an explanation.

The Firing Squad

Suppose you are a prisoner facing the death penalty. You are blindfolded and placed against a wall. A firing squad consist of a dozen expert marksmen stands just a few meters away to carry out the deed. The order is given. The shots ring out. Yet somehow you find yourself alive after the whole ordeal. Every single bullet has missed.

Considering you are still alive you should be vastly more confident that the marksmen intentionally spared you, i.e. by design. After all, the probability of all 12 marksmen missing by chance is immeasurably small.

The Rebuttal To WAP

The firing squad scenario may seem analogous to the fine-tuning argument. WAP suggests all fundamental constants being compatible with our existence is a logical truth, i.e. to be expected. Therefore it is no evidence supporting a designed universe. Then by the same logic, after surviving the firing-squad I should also conclude that is to be expected. Because being alive is the only observation I can make, a logical truth. Consequently, it is not evidence suggesting the marksmen intentionally missed either. That is obviously irrational.

I suggest the analogy drawn between the firing squad and the fine-tuned universe is flawed. There is a critical difference between the two cases. Based on my research, this rebuttal has not been published before.

Analysis of the Firing Squad

The core of my argument (PBA) is to reason from one consistent perspective. So I am going to be explicit about it in the analysis.

Let’s take the prisoner’s first-person pers...
What's the CFAR position on how the workbook can be used?
by ChristianKl — 2020-09-09T13:56:52.514Z

The CFAR handbook is a great resource for applied rationality. I talked with another rationlist in Berlin about creating a reading group around it. Did CFAR articulate a position around how the handbook can be used and whether such a group would be okay? If so, are there any norm such a reading group should fulfill (e.g. making it clear it's independent of CFAR)?
How To Fermi Model
by habryka — 2020-09-09T05:13:19.243Z

[Note from Eli in 2020: I wrote this document in 2016, in conjunction with two workshops that I helped Oliver Habryka run. If I were to try and write a similar document today, it would likely be substantially different in form and style. For instance, reading this in 2020, I’m not very compelled by some of the argumentation that I used to justify this technique, and I think I could have been clearer about some of the steps.

Nevertheless, I think this is some useful content. I’m not going to take the time to write a new version of this document, so it seems better to share it, as is, instead of sitting on it.]

Oliver Habryka provided the seed material and did most of the development work on this technique. He gets upwards of 90% of the credit, even though I (Eli) wrote this document. Thanks Oli!

Introduction:

Rationale for Fermi Modeling:

Making good decisions depends on having a good understanding of the world: the better one’s understanding the better one’s decisions can be. Model-building procedures allow us to iteratively refine that understanding.

Using any model-building procedure at all is a large step up from using no procedure at all, but some procedures are superior to others. If possible, we would want to use techniques that rely on verified principles and are based on what we know about how the mind works. So, what insights can be gleaned from the academic social and cognitive sciences that is relevant to model-building?

First, Cognitive psychology has shown, many times over, that very simple algorithmic decision rules frequently have just as much predictive power, and even outperform, human expert judgment. Deep, specific models that take into account many details specific to the situation (inside views) are prone to overfitting, and are often inaccurate. Decision rules combat biases like the Halo effect and consequently tend to produce better results.

For instance, a very simple equation to predict the probability that a marriage will last is:

Frequency of lovemaking / Frequency of fights

(Where a higher number represents a more stable marriage. Example taken from Thinking Fast and Slow ch. 21)

This assessment measure is intuitive and uncomplicated, and it predicts length of marriage about as well as any other method, including expert evaluation by experienced couples counselors. Most of the relevant information is encapsulated in j...
Updates Thread
by Adam Zerner — 2020-09-09T04:34:20.509Z

If you've updated your belief about something you think is worth noting, post it here.

It doesn't have to be a full blown "mind change", just an incremental update to your beliefs.
I'm thinking it'd be good to have a low bar for what is "worth noting". Even if it's something trivial, I figure that the act of discussing updates itself is beneficial. For rationality practice, and for fun!
That said, I also expect that browsing through updates that other people on LessWrong make will lead to readers making similar updates themselves a decent amount of the time.
I've been developing a strong opinion that journaling and self-reflection in general is incredibly useful. Significantly underrated even among those that preach it. This thread is a way to perform such journaling and self-reflection.
CTWTB: Paths of Computation State
by johnswentworth — 2020-09-08T20:44:08.951Z

This is the second post of Category Theory Without The Baggage;

first post here

. You can probably follow most of this post without reading the first one. Be warned that I am a novice when it comes to category theory; please leave a comment if you see a substantive error or oversight.

Suppose we want to evaluate x(x+3)x+y at the point x = 2, y = 5. We have a choice about the order in which to perform the operations. One possibility:

Add x+y
Add x+3
Multiply x*(x+3)
Divide

Another possibility:

Add x+3
Add x+y
Multiply x*(x+3)
Divide

This hopefully seems trivial in such a simple example, but we want to be able to easily talk about this sort of thing in more complicated problems. To that end, we’re going to visualize these possible computation-orders in terms of graphs.

We’ll start with a graph representing the expression itself - i.e. the usual visual for a circuit:

We can represent “computation state” as a cut through this DAG. For instance, we start out in this state:

The state (x, x, x, y) gives the data carried by each edge we cut through. If we imagine that the computation at each node is performed by a different person, then at the very beginning of the computation, these would be the messages sent from our first two people (i.e. the people with our input values x = 2 and y = 5) to the people downstream. Alternatively, if a single CPU were performing this computation, it would probably save some memory by only storing x once rather than keeping three separate copies; more on that later.

From there, we have two possible next steps:

On the left, we compute x+3 first, so our state updates to (x+3, x, x, y). On the right, we compute x+y first, so our state updates to (x, x, x+y).

After performing either of these operations, we can perform the other, leaving us in the state (x+3, x, x+y). We can visualize this as two computation-paths in a computation-graph:

Or, dropping all the visuals of the circuit-cuts:

The two paths from (x, x, x, y) to (x+3, x, x+y) correspond to the two possible orders of the addition operation: x+3 followed by x+y, or x+y followed by x+3.

Here’s the full graph of computation states, with the multiplication and division operations added in.

Any allowed order of the operations in our original circuit corresponds to a path from the upper-left to lower-right in this computation-state graph. If that makes sense, then you’ve probably understood the bas...
Indignation in response to the 1890 census
by jasoncrawford — 2020-09-08T20:14:30.619Z
Link: https://rootsofprogress.org/indignation-at-the-1890-census

In reading about the development of technology, I keep an eye out for changes in society as well. I commented recently that we don’t seem to celebrate major achievements as much anymore. But it’s not just technology that Americans used to view differently. It’s growth of all kinds.

The book Computer: A History of the Information Machine tells the story of the 1890 census. It was the first census to be computed, not by hand, but with tabulating machines, developed by Herman Hollerith. On August 16, 1890, the grand total was announced: the population of the United States was 62,622,250.

“But”, it says, “this was not what the allegedly fastest-growing growing nation in the world wanted to hear.” It quotes a contemporary account in a periodical, The Electrical Engineer, from 1891 (emphasis added):

The statement by Mr. Porter [the census director] that the population of this great republic was only 62,622,250 sent into spasms of indignation a great many people who had made up their minds that the dignity of the republic could only be supported on a total of 75,000,000. Hence there was a howl, not of “deep-mouthed welcome,” but of frantic disappointment.

The book continues:

The press loved the story. In an article headlined “Useless Machines” the Boston Herald roasted Porter and Hollerith; “Slip Shod Work Has Spoiled the Census,” exclaimed the New York Herald; and the other papers soon took up the story.

“Spasms of indignation” because population growth was too low for “the dignity of the republic”. Americans were proud of being the fastest-growing country. Today, in contrast, people fear overpopulation, and the general slowing of world population growth is generally considered to be good news.

Something changed in American attitudes in the last 100+ years, not just toward technology or the economy as such, but more fundamentally toward growth itself.
Escalation Outside the System
by jefftk — 2020-09-08T18:20:09.275Z

Transcript of a discussion on a friend's wall on the merits of

responding
"guillotines" to union text-bankers when asked what the country
needs more of:

Me: When you respond "guillotines" what do you expect the campaign
volunteer reading the response to think you're advocating for?

Them: Murder.

Me: Whose murder do you expect them to think you are advocating for?

Them: The richest people in the US.

Me: 0.1%, 1%, 5%?

Them: I think billionaires is a good cutoff. There are 540 in
the US. So, the richest 0.000164%.

Me: Is executing them your first choice? Or would you prefer to see
non-violent redistribution?

Is your objection that having this much money is immoral when others
need it so much more, that you can't be this rich without having
committed serious crimes, or something else?

Them: I'd certainly prefer a non-violent
solution. Redistribution sounds lovely.

My objection is that we do not have anything remotely resembling a
democracy. And I think that having that much money is actually
immoral, while 21% of the children in the US are below the poverty
line. Putting them together, I think it's a fine solution to kill them
off until they can figure out how to release their chokehold on our
government.

And it seems to me, solutions within the system have been adequately
tried.

(I do think that keeping millions—let alone billions—for
yourself is immoral when the money could do so much more, but that is
the extent of my agreement.)

I see this perspective often in leftist spaces: the system has failed
its most vulnerable, it cannot be fixed, we must escalate violently
to transcend the system and find new solutions outside.

There have been many successful leftist revolutions, at least if you
define success as gaining power. (And that gap is one reason why I
wouldn't support violent revolution regardless.) What I don't
understand is how leftists could look at the current political climate
in the US and think that violent revolution would work out well for
them?

It's not
clear that Trump will leave office if he loses in November:

Crowd: Four more years! Four more years! Four more years!

Trump: Now if you really want to drive them crazy you say
"twelve more years"

Crowd: [cheers]

Crowd: "Twelve more years! Twelve more years! Twelve more
years!"

—Trump at the
Republican National Convention, 2020-08-24

Then co...
Efficacy of Vitamin D in helping with COVID
by df fd — 2020-09-08T10:49:58.319Z

so a new research found that high dose of Vitamin D significantly improve out comes for COVID patient:

https://www.sciencedirect.com/science/article/pii/S0960076020302764?via%3Dihub

another source on facebook claim that:

- A study in Indonesia found that out of the patients that died from COVID-19, 98.9% of them were deficient in vitamin D, while only 4% of the patients with sufficient vitamin D died.

-A study of patients in New Orleans found that 84.6% of the COVID-19 patients in the ICU were deficient in Vitamin D while only 4% of the patients in the ICU had sufficient levels of Vitamin D.

-A study in the Philippines found that for every standard deviation increase in vitamin D people were 7.94 times more likely to have a mild rather than severe COVID-19 outcome and 19.61 times more likely to have a mild rather than critical outcome.

I couldn't find any mention of this on lesswrong, [granted I haven't look very hard], anyone who have done their reseach on this can help me determine the import of Vitamin D in fighting this pandemic?

and if it's true, anyway we can profit from this? any stock or index fund?
Loneliness
by DirectedEvolution — 2020-09-08T08:14:56.557Z

Loneliness is an inevitable part of the human condition, no matter your culture or set of relationships. You have to spend time alone. And sometimes, you'll spend time not only alone, but undistracted. The work is done for the day. You have no plans. There are no pressing errands or chores. You are not tired. No activity - not reading or writing, not cooking or playing music, not anything - calls out for your attention.

You feel aware that all activities have a shared characteristic, which is that they can distract you from your loneliness. Or at least numb it somewhat, bring it to a tolerable level. So much of our work, too, is done in the long run to protect us from loneliness, or with the goal of helping us escape it.

Some people seem to revel in being alone. They feel as if they have too many demands on their attention, and crave some time to themselves. They have so many books or hobbies to pursue. Or maybe they just want to sleep or get some exercise. They are almost tormented by the extreme pressure to relate. They are more in demand than they can handle.

Others might look at these people with envy. For some, loneliness is a feeling of alienation and rejection. Or perhaps of neglect, of being left out. It might be caused not even by being ignored, but due to life circumstances, such as a transition or a move that leads to distance from loved ones. Some people might have friends, family, even a romantic partner and children, and yet still feel consumed by loneliness, out of touch with the people they believe care about them the most. It's hard to know who has it worst. But everyone has this loneliness in some form or another. I imagine that even the people who can't seem to escape the ceaseless demands for their attention must experience a special form of loneliness within that dynamic.

Now, we all know that there are experiences, or practices, that are said to help us escape from loneliness, or numb it, or at least push the boundaries of it back a little bit. But these fixes are often temporary. An isolated individual who begins to make friends may find themselves surprised to learn that the loneliness comes right back, this time in a new form. Yes, the new friends help - in the right moment, when the interaction is desired and both people are at their best - but often, even two good friends will not see eye to eye or have a meeting of the minds. A person...