trentmkelly/LessWrong-43k huggingface dataset:
Report from a civilizational observer on Earth
by owencb — 2022-07-09T17:26:09.223Z

[translated into English by the author]

I have arrived in [2022]. There is one species, humans, that is building civilization. (A few other species exhibit potential but have not crossed the threshold of runaway economic development or cultural accumulation; it is now unlikely that they ever will unless a deliberate choice is made to allow them to do so.)

There are around 129.2 humans. They are organized into loose and overlapping political structures, including firms, [countries with systems of law one cannot opt out of], and [religions and hobby groups]. There is a degree of [law, treaties, and custom] constraining these groups, but there is no strong political unification, and sometimes there is significant [violence causing destruction of persons or property].

Most of the physical activity of civilization is done by machines operated by humans, although a significant minority is still done directly by humans, and a significant-and-increasing minority is done by machines operated by [rule-governed thinking-machines].

In some domains a large majority of the intellectual labour of civilization is done by thinking-machines, but these tend to be [rule-governed thinking-machines] following precise instructions laid down by humans. Most key intellectual work is still done by humans (although a lot of the communication of this is mediated by [rule-governed thinking-machines]).

In the last econoyear[1], civilization has built nontrivial [patterning thinking-machines]. And in the last economonth[2], these have been applied to human language with significant success. [Patterning thinking-machines] are now capable of doing sophisticated tasks that humans might previously have been paid to do. And since the language they learned encoded a lot of civilizational knowledge, these [patterning-thinking-machines] now implicitly contain a large fraction of civilizational knowledge (at least at a [shallow/indexing] level).

Intellectual automation

The next several economonths will likely represent a period of acceleration for civilization. It is unclear whether they will contain an emergent [post-civilization], though it is most likely that that will occur within the next 24 econoyears, and maybe much sooner. It is currently unclear what the nature of any emergent [post-civilization] will be.

The reason for acceleration is:

They can now build

...
Grouped Loss may disfavor discontinuous capabilities
by Adam Jermyn — 2022-07-09T17:22:23.947Z

Thanks to Evan Hubinger and Beth Barnes for comments on these ideas.

Language models exhibit scaling laws, where the loss is a power-law in model size. This offers a lot of predictive power, and seems like a useful thing to know. By contrast, individual capabilities can exhibit sharp discontinuities in performance as a function of model size and training time.

It would be great if individual capabilities just gradually improved like the broader loss. Then we wouldn’t have to worry quite so much about surprising new capabilities emerging suddenly during training.

Is there a way to change the loss function so that it incentivizes more gradually capability improvements?

Grouped Loss

Imagine grouping training examples by the kind of capability they exhibit. For instance arithmetic problems go in one group, “parse json” could go in another, and so on. With these groups, we could define a new loss function

L=∑g∈groupsℓ2g

where ℓ is the loss function we originally used (e.g. cross-entropy loss) and ℓg means to compute the sum of ℓ over examples from group g, e.g.

ℓg=1len(g)∑example∈gℓ(example)

which may be estimated by using random examples drawn from g.

Because we have squared the group losses, the overall loss is dominated by the worst group. As a result, the model is incentivized to develop capabilities in each group at comparable rates, and so has little incentive to e.g. finely hone its poetry skills while being unable to multiply numbers.

Challenge: Defining Groups

It’s possible that using grouped loss results in smooth development of capabilities that aren’t represented in the groups. For instance, it seems plausible that if “adding arabic numerals” and “translating words into arabic numerals” are two groups but “adding numbers written as words” is not, performance on the latter could nonetheless develop smoothly as the model gets better at the others. It would certainly be weird if performance ”adding numbers written as words” advanced as a sudden leap in this case.

This points to a general problem though, which is that if we have to define the groups manually we have to foresee the capabilities we’re worried about. That seems bad.

Gradient Cluster Grouping

If we could automatically group examples we wouldn’t need to do it manually. How could we do this?

I think the key feature of a group is that when the model updates, the loss of m...
Train first VS prune first in neural networks.
by Donald Hobson — 2022-07-09T15:53:33.438Z

This post aims to answer a simple question about neural nets, at least on a small toy dataset. Does it matter if you train a network, and then prune some nodes, or if you prune the network, and then train the smaller net.

What exactly is pruning.

The simplest way to remove a node from a neural net is to just delete it. Let yj=f(∑ixiwij+bj) be the function from one layer of the network to the next.

Given I and J as the set of indicies that aren't being pruned, this method is just

~wij={wiji∈I and j∈Jnothingelse

~bj={bjj∈Jnothingelse

however, a slightly more sophisticated pruning algorithm adjusts the biases based on the mean value of xi in the training data. This means that removing any node carrying a constant value doesn't change the networks behavior.

The formula for bias with this approach is

~bj={bj+∑i∉I¯xiwijj∈Jnothingelse

This approach to network pruning will be used throughout the rest of this post.

Random Pruning

What does random pruning do to a network. Well here is a plot showing the behavior of a toy net trained on spiral data. The architecture is

And this produces an image like

In this image, points are colored based on the network output. The training data is also shown. This shows the network making confidant correct predictions for almost all points. If you want to watch what this looks like during training, look here https://www.youtube.com/watch?v=6uMmB2NPv1M

When half of the nodes are pruned from both intermediate layers, adjusting the bias appropriately, the result looks like this.

If you fine tune those images to the training data, it looks like this. https://youtu.be/qYKsM29GSEE

If you take the untrained network, and train it, the result looks like this. https://www.youtube.com/watch?v=AymwqNmlPpg

Ok. Well this shows that pruning and training don't commute with random pruning. This is kind of as expected. The pruned then trained networks are functional high scoring nets. The others just aren't. If you prune half the nodes at random, often a large chunk of space is categorized wrongly. This shows that the networks aren't that similar. This is kind of to be expected. However, these networks do have some interesting correlations. Taking the main 50x50 weight matrixes from each network and plotting them against each other reveals.

Same plot, but zoomed in to sho...
Visualizing Neural networks, how to blame the bias
by Donald Hobson — 2022-07-09T15:52:55.031Z

Background

This post is strongly based on this paper, which it calls the LRP algorithm, https://arxiv.org/pdf/1509.06321v1.pdf [1]

I later learned of the existence of this paper, which is even more similar to the ideas discussed here. https://arxiv.org/pdf/1704.02685.pdf [2]

In it, I examine 2 of the methods for neural network visualization, and show that they have structural similarities. I show that these algorithms only differ by a difference in how they treat the biases, and (possibly a difference in getting started)

The second algorithm obeys conservation laws, it tries to parcel credit and blame for a decision up to the input neurons.

Intro

The task we want is to assign importance to different inputs of a neural network in production of an output. So for example, in the case of a trained image classifier, the visualization method would take in a particular image, and highlight the parts of the image that the network thought were important.

A general method for visualizing neural networks is back-propagation. First evaluate the network forwards. Then work backwards through the network by using some rule about how to reverse each individual layer.

One example of this is differentiation. Finding the rate of change of the output, with respect to each input. But there are others.

Firstly, lets pretend biases in the network don't exist. We are allowed non-linearities, so long as they satisfy the equation f(0)=0 .

Lets look at various layer types and the different back-propagation rules.

Maximum

Most often found in the form of max pooling.

b=maxiai

Gradient

The rule used in gradient descent, and I think the only rule used in the paper above for back propagating maximum. (Notation note. R here isn't exactly a function. Its more like ddx , its output is related to the context in which the input occurs. Think of every number in the forward net having an associated number )

R(ai)={R(b) ai=b0else

Ignoring the case of an exact tie. Exact ties, and what to do on that kind of singular point will be ignored in general, because they are fiddly and unimportant.

Radial.

Treats 0 as a special point.

R(ai)=siR(b)

Where the plot above shows si and the si are non-negitive and sum to 1.

Case. ai<0, b>0⟹si=0 . Relative to 0, a ne...
Using Ngram to estimate depression prevalence over time
by David Gross — 2022-07-09T14:57:55.313Z
Link: https://www.pnas.org/doi/10.1073/pnas.2102061118

Historical language records reveal a surge of cognitive distortions in recent decades

My summary:

People diagnosed with depression tend to exhibit characteristic patterns of language use that demonstrate the underlying cognitive distortions associated with depression.

For example... individuals label themselves in negative, absolutist terms (e.g., “I am a loser”). They may talk about future events in dichotomous, extreme terms (e.g., “My meeting will be a complete disaster”) or make unfounded assumptions about someone else’s state of mind (e.g., “Everybody will think that I am a failure”). Typologies of cognitive distortions generally differentiate between a number of partially overlapping types, such as “catastrophizing,” “dichotomous reasoning,” “disqualifying the positive,” “emotional reasoning,” “fortune telling,” “labeling and mislabeling,” “magnification and minimization,” “mental filtering,” “mindreading,” “overgeneralizing,” “personalizing,” and “should statements.”

The researchers looked for these language patterns in 14 million books, published over the past 125 years in English, Spanish, and German, that are available via Google Ngram, to see how their prevalence has changed over time.

They found that in general the prevalence of such language patterns decreased or stayed stable over the course of the 20th century up until around 1978. There were some local and temporary spikes (e.g. German-language texts between the world wars and after World War II, English-language texts in 1899 for some reason). In 1978, prevalence began to rise slowly, and then in 2000 more rapidly, leveling out again around 2008 at a historically-high level.

The authors conclude that there has been a recent rapid and strong rise in the use of language patterns that suggest the cognitive distortions associated with depression, in recent years in published books.

Some Concerns

Could it be that language fashions change in ways that are independent of depression but that overwhelm the effect of depression on the data? For example, if hyperbole goes in and out of fashion for purely aesthetic reasons, so will "catastrophizing," "overgeneralizing," "magnification and minimization."
Similarly, writing today seems more direct, less baroque. A writer today might simply say "Everyone thinks I should exercise if I want to look better," while a writer 125 years ago might say "It is a truth unive

...
Making it harder for an AGI to "trick" us, with STVs
by Tor Økland Barstad — 2022-07-09T14:42:29.803Z

Summary / Preamble

AI Alignment has various sub-areas. The area I focus on here is ways we might use a superintelligent AGI-system to help with creating an aligned AGI-system, even if the AGI we start out with isn’t fully aligned.

Imagine a superintelligence that “pretends” to be aligned. Such an AI may give output that seems to us like what we want. But for some types of requests, it’s very hard to give output that seems to us like what we want without it actually being what we want (even for a superintelligence). Can we obtain new capabilities by making such requests, in such a way that the scope of things we can ask for in a safe way (without being “tricked” or manipulated) is increased? And if so, is it possible to eventually end up with an aligned AGI-system?

One reason for exploring such strategies is contingency planning (what if we haven’t solved alignment by the time the first superintelligent AGI-system arrives?). Another reason is that additional layers of assurance could be beneficial (even if we think we have solved alignment, are there ways to relatively quickly add additional layers of alignment-assurance?).

When dealing with a genie/oracle, we may not want to ask it to provide some direct solution/answer to what you want (and have it argue why it’s a good idea). Instead we obtain what we want more indirectly, by having different instances of the genie/oracle aid us in the construction of more narrow/specialized programs (that score high in terms of verifiability).

We could call such programs STVs, where STV is an abbreviation for Specialized Transparent Verifiable Program.

STV is a blurry concept (much like how AI and AGI are blurry concepts).

An STV could be (but would not have to be) a narrow AI. More colloquially, STVs could also be referred to as “generators”.

In this post I explore principles/techniques/strategies for using STVs in ways that enable us to get things we want from an AGI without being “tricked”.

I hope and think this article reads well in isolation, even though it's part 2 of

a series

.

Future posts in this series will include discussion of:

AI-generated computational proofs/argument-trees, and methods/strategies for verifying such proofs/arguments.
Ideas for how formalisms for computational proofs could try to incorporate human cluster-like concepts, concept vagueness, formalism-to-reality mappin

...
Ars D&D.sci: Mysteries of Mana
by aphyer — 2022-07-09T12:19:36.510Z

This is an entry in the 'Dungeons & Data Science' series, a set of puzzles where players are given a dataset to analyze and an objective to pursue using information from that dataset.

STORY (skippable)

For thousands of years the Order of Athena has preserved magical knowledge, passing it down from masters to apprentices, in a lineage of knowledge unmarred by countless centuries.

Those hidebound old fools! Your thesis, 'Calculating Levels of Ambient Mana,' could have revolutionized all magic! You found a way to determine how powerful various types of mana used to power spells were at various times, via bold modern Data Science techniques these antediluvian wizards would never have dreamed of.

Sadly, these senile old mages failed to appreciate your contribution, and have assigned you to a standard apprenticeship under a training mage, intending to have you spend the next fifteen years studying primeval books under his tutelage before they let you call yourself a mage. Ridiculous! Modern techniques are far more applicable, and only by looking forward can we hope to become more powerful than the past! These mages are doomed to only ever pass on the knowledge of the past, but you can learn more and grow stronger than they could ever be!

Of course, first you'll need to convince your new 'master' to let you pass early. He has at least agreed to give you a chance to demonstrate the use of your new techniques. Among these mages' traditions is to hold frequent non-lethal duels between one another, and he has one such duel coming up. He's offered to allow you to use your new techniques to choose which spells he should prepare for that duel.

With your new knowledge of mana levels, and an extensive history of duels, you expect you can choose excellent spells and maximize his odds of winning. Then, perhaps, you'll be able to spend the next fifteen years pursuing your own research instead of learning Latin so that you can read cobwebbed old books!

DATA & OBJECTIVES

The dataset contains a list of past duels between mages. Each row contains:

The mana levels at the time of the duel.
The spells each mage brought to that duel.
Which mage won that duel.

Using that dataset, you need to select which spells your master should prepare for this imminent duel to maximize his odds of winning. He can prepare

...
I’ve become a medical mystery and I don’t know how to effectively get help
by CraigMichael — 2022-07-09T06:58:48.688Z

To put it very mildly, I’m in a really bad way emotionally because of this and I’m running out of ideas. Any ideas are welcome—specific to symptoms, or just generally how to deal with having a problem and not being able tot get diagnosis or treatment.

TL;DR:
I’ve been having paresthesias and neuropathic pain over most of my body, but in shifting locations, since March 30th of 2022. (Paresthesia is like the sensation of a limb falling asleep, neuropathic pain is like that but most of a painful prick or electric shock-like feeling). I’ve seen my PCP, ENT, psychiatrist, neurologist, dentist, physical therapist and message therapist, but I don’t have a diagnosis.

Longer version:
On March 30th I work up with a knot (TMJ) in my left jaw and paresthesia in my left leg. I also had dry needling done on my left hip that previous day but a professional licensed practitioner (someone who should have not done a lot of damage).

My PCP did a lot of blood tests (including b12, b6 and ferritin serum). All of these were normal. He able to feel the knot in my left masseter (jaw muscle). Referred to ENT, Neurologist, Dentist.

I got an acrylic night guard for the TMJ from my dentist.

ENT was kind of a jerk and was like “I don’t believe in TMJ treatments other than night guards—not sure what paresthesias are about.”

PT was most helpful. Found limited mobility C3-C5 on left side of neck and limited mobility L5-S1. Thought to could be Radiculopathy—which still makes the most sense. He did some massage kind of things to try for improve the mobility, but didn’t seem to make much of a difference.

Neurologist— suggested it was because of bad sleep hygiene and anxiety (both of which I have and take Rxs for). and x-rayed my neck. While I was bent in on the tight side the radiologist didn’t see anything worrying. Doesn’t think it’s radiculopathy, but I don’t know that he’s really done enough imaging to rule this out.

I have an appointment to see a TMJ specialist but can’t get in to see him until December.

Psychiatrist is nice about wanting to give me more medication, but doesn’t have many ideas.

Message therapist gave me a nice neck massage, but I don’t think it helped much.

I have a good relationship with my PCP, psychiatrist, physical therapist l and neurologist from previous issues. Have seen the ENT before and he’s always kind of been a jerk, but he must be a good enough doctor otherwise.

Circa 3...
Some thoughts on Animals
by nitinkhanna — 2022-07-09T02:11:09.272Z

I came across this post today where the author talks about happiness and evolution. The author seems to have ambivalent feelings towards happiness. On the one hand, it helps them keep on living. On the other, not having happiness seems to be an evolutionarily good thing to the author.

The author seems to compare two extremes - the human experience, and that of a single-celled organism, and asks the question - “Do single-celled organisms experience happiness?”, and answers, “Maybe.”

What I wonder when reading such posts is, why do we disregard the large and varied group of organisms in between?

What about dolphins, which are known to be extremely intelligent and expressive animals and are known to use pufferfish to gain a high? What about when they foster relationships and become happy when they spend quality time with friends and family?

What about chimpanzees laughing at each other and getting surprised by amateur magicians performing for them? What about elephants displaying empathy and anger?

Forget all these wild ones, what about dogs displaying happiness by wagging their tails when they see their “dog parents” day after day?

Why did the author discount these amazing examples and go directly towards the single-celled organism or the computer program?

Please note - I’m not trying to attack the author, rather ask them this question - “should we disregard every other intelligent species because we feel special in some way since we’re humans?”

And I’ll answer it too - no, we shouldn’t. We should look at the myriad ways evolution allows for happiness and conclude that happiness and generally feeling good has a special place in our evolutionary path and we should accept it.

I also linked one other idea while reading this post. I came across this post yesterday where the author links to a Kurzgesagt video about the future of humans and charges that the video ignores transhumans and genetic engineering, which the author supposes will be very common in the future. In fact, the video acknowledges this too and says that from that point on, it’ll not talk about “humans” but “people”, encompassing all the varied “types” of humans that may exist.

While thinking about this, the thought came to me asking, “what about animals?”

While I accept that we’re on track to killing basically every non-human species of the animal kingdom, efforts are being made to store their DNA and get bet...