trentmkelly/LessWrong-43k huggingface dataset:
Convolution as smoothing
by Maxwell Peterson — 2020-11-25T06:00:07.611Z

Convolutions smooth out hard sharp things into nice smooth things. (See previous post for why convolutions are important). Here are some convolutions between various distributions:

Separate convolutions of uniform, gamma, beta, and two-peaked distributions. Each column represents one convolution. The top row is f, the middle row g, and the bottom row f∗g.

(For all five examples, all the action in the functions f and g is in positive regions - i.e. f(x) > 0 only when x > 0, and g too - though this isn't necessary.)

Things to notice:

Everything is getting smoothed out! Even the bimodal case, in the column all the way to the right, has a convolution that looks like it's closer to something smooth than its two underlying distributions are. You might say the spike that uniform ∗ uniform gives doesn't seem more smooth, exactly, than the uniform distributions that led to it. But notice the second column, which convolves a third uniform distribution with the result of the convolution from the first column. That second convolution leads to something more intuitively smooth (or if not smooth, at least more hill-shaped). We can repeat this procedure, feeding the output of the first convolution to the next, and each time we get a black result distribution that is a little bit shorter and a little bit wider. Thus we get a smooth thing by repeatedly convolving two input sharp things (f and g):

The more we do this (for these particular shapes f and g), the more the final distribution will resemble a Gaussian. In other words, the central limit theorem applies to sets of uniform distributions.

Even the more difficult bimodal distributions smooth out under repeated convolution, although it takes longer:

I'm not sure whether this goes into a Gaussian or not. But the shape is suggestive.

f∗g (for positive f and g) is always to the right of f and g both. Because all these example f and g are positive, their means are positive. And it's a theorem that the mean ⟨f∗g⟩ is always equal to the sum of the underlying means ⟨f⟩+⟨g⟩. So the location of the distributions is always increasing. Of course, for f and g with negative first moments, the location of f∗g would go leftward; a

...
Evading Mind Control
by lsusr — 2020-11-25T05:01:32.404Z

A couple days ago I surveyed readers for deviant beliefs. The results were funny, hateful, boring and bonkers. One of the submissions might even be useful.

If you care a lot about your mind, it is not unreasonable to avoid advertisements like plague rats, up to and including muting your speakers and averting your gaze.

This extremist position caught my eye because humans have a tendency to underestimate the effect advertising[1] has on us. I never realized how much advertising affected me until I got rid of it.

For nearly a year I have been avoiding junk media. I thought this would make me happier, healthier and more productive—which it has—but the most surprising effect is how much the reduction in advertising affects my behavior.

When I give up junk media, I stop thinking about politics, videogames and celebrities. I think less about products in general. Important things eventually expand to fill this void. But for the first week or so, my head just feels empty.

Tim Ferris doesn't just avoid news, television and social media. He even avoids reading books—especially nonfiction. When I first read that, I thought he was an Eloi. Having blogged regularly for the past year myself, I now sympathize with him.

If you are young then you should read lots of books because you need to amass information. Eventually you hit diminishing returns. Reading more books fills fewer conceptual holes per unit time invested.

You cannot discover new knowledge for humanity by reading a book written by a human.

But there is a bigger problem. It is easy to look up answers to common questions in a book. It is harder to look up answers to esoteric questions. It is impossible to look up answers to open problems. The difficulty of looking up important things you don't know answers to increases the more low-hanging fruit you pick from the Tree of Knowledge.

As your power waxes, it becomes easier to invent answers to your own questions. Eventually the trajectories cross. It becomes easier to figure things out yourself than to look up the answer. The comparative value of reading books goes negative. Books, once guides, become reference material. It is more efficient to write your own book than to read someone else's.

I used to read a lot of books. I finished 18 books in the first 4.5 months of 2020.

Date
Title
Author
Page Count

January 1
The Trouble with Physics
Lee Smolin
392

January 17

...
What to put in my time capsule?
by justinslud — 2020-11-25T04:21:51.052Z

In a few weeks I will open a time capsule I set for myself 5 years ago. I'm extremely excited and want to celebrate by setting myself a new time capsule to open in 5 years from now.

The difficult part about making the time capsule is what to include. Since I have not yet opened the original capsule, I don't know how myself from 5 years ago chose what "artifacts" to include.

I feel like whatever goes inside the capsule should be a "snapshot" of life at this moment for me. However, I will still have access to all my writings from around this time that are not in the capsule. So I want to include things from my current life and also things from recent past (anything within 2 years is in this "stage of life").

Here is a breakdown of what I plan to include:

school papers
various small items and papers from recent past
photos
USB stick with a video to my future self
document detailing all the artifacts listed above
letters from other people. I have not yet asked them to write anything but it could be fun.
letters to other people (that I write down and deliver to them in 5 years) Like the previous idea, I have no idea how it might go.
list predictions about future self
letter to future self

my hopes for future self
detailed description about my current mental models of the world
what activities, hobbies, habits I'm currently engaged in (and have done in recent past). For example, what I do to exercise, what songs I listen to, what foods I always eat, etc.
things I may forget or would love to be reminded of. For example, inside jokes with friends.
questions I want future self to answer upon opening the capsule. For example, "How have your thoughts on X changed in 5 years?"

What do you think of this list? What do you recommend adding or doing differently? I'd appreciate any ideas on how to carry out experiments like this over long time frames.
Transparent Technologies
by lsusr — 2020-11-25T02:19:46.209Z

I cook my own meals because restaurant food is expensive. But there are many activities I would prefer to cheap versions independent of price.

I Like
I Don't Like

bicycles
cars

entrepreneurship
jobs

solo adventure
guides

barbells
exercise machines

meditation
psychoactive drugs

Linux, LineageOS
Windows, Mac, iPhone

i3, CLI
GUI, IDE, desktop metaphor

autodidacting
school, tutors

From the perspective of someone with unusually high intelligence, activities involving skill tend to be cheap because our civilization has adequate material capital, The limiting factor of our industrial economy is intelligence.

I have to manually override my instinctual feeling that civilization is materially limited. It is true there aren't enough spaceships and private airplanes to go around. Otherwise, we have plenty of material goods. Many people own too much stuff.

Since the limiting factor on wealth is humans' inadequate cognitive ability to utilize our stuff, companies often create value by making things easier to use.

If the process ended here then we would live in utopia. But when a company provides the service of making something easier to use, it takes effective control of the distribution channel. Centralized control of distribution channels inhibits competition. Weaker competition gives companies leverage over consumers. The Invisible Hand compels companies to extract maximum profit from whatever leverage they have.

A prepackaged commercial product designed to make things easier for consumers tends to contain anti-features. Vehicles require special tools to interface with their computers. Apple employs pentalobe security screws. Universities charge fees for services you don't use and require you to take classes you're not interested in. Android distributors often won't let you root your phone.

Expending effort to worsen a product is productively inefficient. An efficient market is a market that does not do inefficient things. Anti-features are market inefficiencies because anti-features deliberately make a product worse. Anti-features are negative sum.

Rational negative sum activities are often coercive. When coercive behavior goes one direction it indicates an exploitative relationship. Anti-features are the most flagrant signal that an organization is exploiting you.

Even touching a product with anti-features makes me feel unclean. Like many moral princ...
Latest & Greatest Wiki-Tag Updates: Import, Dashboard, and Feed
by Ruby — 2020-11-24T21:35:06.656Z

It's been a little while since we've posted an update about Wiki-Tag [1] stuff, but there is now news. Good news, even.

Import of the Old Wiki: Complete 🎉

It was a large project requiring dozens of hours of post-processing clean-up even after the automatable parts, but we got it done! Over five-hundred pages transferred from wiki.lesswrong.com to the main LessWrong site for unprecedented knowledge-base unity.

I want to give huge thanks to those who gave their time generously to make this happen.

Major Upgrades to the Wiki-Tag Dashboard

The dashboard is a power tool for those magnanimous souls who contribute to the ongoing maintenance and expansion of our corpus of wiki-tags.

Index of the expanded Wiki-Tag Dashboard

New Tag Flags

Tag Flags, i.e. flags set on wiki-tags, were introduced with the wiki import to mark post-processing tasks. With that done, I've set up long-term flags for tracking the work to be done. They go hand in hand with wiki-tag Talk/Discussion sections.

I've automatically populated a few of these tag-flag lists, now we need to apply other flags where needed.

Information about the meaning and purpose of each flag is provided upon hover-over.

Any user can set or remove these flags from posts, either from a tag's main edit page or tag edit mode on the Dashboard page itself. For now, the mod team controls the flags available, but we're open to suggestions.

On the Wiki-Tag Dashboard, clicking a Tag Flag's item will give you a list of only the tags with that item set. If you click on a tag on the list, it will open the edit page in Playlist Mode. When you're done editing, you iterate onto the next wiki-tag on your current list.

View the Tags You Created

If you've created any wiki-tags, you can get a list of just those wiki-tags from the dashboard.

Now Included: Tag Voting Activity and Wiki-Tag Activity Feed

To date, we've had a separate page that displays all recent tagging activity as well as newly created tags. That's now rolled into Wiki-Tag dashboard so all the info is conveniently in one place.

Further, we've recently launched a new feed of Wiki-Tag activity: creation, edits, and discussion. While it's available on its own page too, we've also rolled into the Wiki-Tag Dashboard.

Now in one place you've got: 1) wiki-tags in need work, 2) newest wiki-tags, 3) tag voting activity, 4) combined feed of creation, edit, discussion.

Wiki-Tag Activity Feed

Just s...
Pain is not the unit of Effort
by alkjash — 2020-11-24T20:00:19.584Z
Link: https://radimentary.wordpress.com/2020/11/24/pain-is-not-the-unit-of-effort/

(Content warning: self-harm, parts of this post may be actively counterproductive for readers with certain mental illnesses or idiosyncrasies.)

What doesn't kill you makes you stronger. ~ Kelly Clarkson.

No pain, no gain. ~ Exercise motto.

The more bitterness you swallow, the higher you'll go. ~ Chinese proverb.

I noticed recently that, at least in my social bubble, pain is the unit of effort. In other words, how hard you are trying is explicitly measured by how much suffering you put yourself through. In this post, I will share some anecdotes of how damaging and pervasive this belief is, and propose some counterbalancing ideas that might help rectify this problem.

I. Anecdotes

1. As a child, I spent most of my evenings studying mathematics under some amount of supervision from my mother. While studying, if I expressed discomfort or fatigue, my mother would bring me a snack or drink and tell me to stretch or take a break. I think she took it as a sign that I was trying my best. If on the other hand I was smiling or joyful for extended periods of time, she took that as a sign that I had effort to spare and increased the hours I was supposed to study each day. To this day there's a gremlin on my shoulder that whispers, "If you're happy, you're not trying your best."

2. A close friend who played sports in school reports that training can be harrowing. He told me that players who fell behind the pack during for daily jogs would be singled out and publicly humiliated. One time the coach screamed at my friend for falling behind the asthmatic boy who was alternating between running and using his inhaler. Another time, my friend internalized "no pain, no gain" to the point of losing his toenails.

3. In high school and college, I was surrounded by overachievers constantly making (what seemed to me) incomprehensibly bad life choices. My classmates would sign up for eight classes per semester when the recommended number is five, jigsaw extracurricular activities into their calendar like a dynamic programming knapsack-solver, and then proceed to have loud public complaining contests about which libraries are most comfortable to study at past 2am and how many pages they have left to write for the essay due in three hours. Only later did I learn to ask: what incentives were they responding to?

4. A while ago I became a connoisseur of Chinese webnovels. Among those written for a ma...
The next AI winter will be due to energy costs
by hippke — 2020-11-24T16:53:49.923Z

Summary: We are 3 orders of magnitude from the Landauer limit (calculations per kWh). After that, progress in AI can not come from throwing more compute at known algorithms. Instead, new methods must be develloped. This may cause another AI winter, where the rate of progress decreases.

Over the last 8 decades, the energy efficiency of computers has improved by 15 orders of magnitude. Chips manufactured in 2020 feature 16 bn transistors on a 100mm² area. The switching energy per transistor is only 3×10−18 J (see Figure). This remarkable progress brings us close to the theoretical limit of energy consumption for computations, the Landauer principle: "any logically irreversible manipulation of information, such as the erasure of a bit or the merging of two computation paths, must be accompanied by a corresponding entropy increase in non-information-bearing degrees of freedom of the information-processing apparatus or its environment".

Figure: Switching energy per transistor over time. Data points from Landauer (1988), Wong et al. (2020), own calculations.

The Landauer limit of kTln(2) is, at room temperature, 3×10−21 J per operation. Compared to this, 2020 chips (tsmc 5nm node) consume a factor of 1,175x as much energy. Yet, after improving by 15 orders of magnitude, we are getting close to the limit – only 3 orders of magnitude improvement are left. A computation which costs 1,000 USD in energy today may cost as low as 1 USD in the future (assuming the same price of USD per kWh). However, further order-of-magnitude improvements of classical computers are forbidden by physics.

At the moment, AI improves rapidly simply because current algorithms yield significant improvements when increasing compute. It is often better to double the compute than work on improving the algorithm. However, compute prices will decrease less rapidly in the future. Then, AI will need better algorithms. If these can not be found as rapidly as compute helped in the past, AI will not grow on the same trajectory any more. Progress slows. Then, a second AI winter can happen.

As a practical example, consider the training of GPT-3 which required 3×1023 FLOPs. When such training is performed on V100 GPUs (12 nm node), this would have cost 5m USD (market price, not energy p...
Snyder-Beattie, Sandberg, Drexler & Bonsall (2020): The Timing of Evolutionary Transitions Suggests Intelligent Life Is Rare
by Kaj_Sotala — 2020-11-24T10:36:40.843Z
Link: https://www.liebertpub.com/doi/full/10.1089/ast.2019.2149

Copying over Anders Sandberg's Twitter summary of the paper:

There is life on Earth but this is not evidence for life being common in the universe! This is since observing life requires living observers. Even if life is very rare, the observers will all see they are on planets with life. Observation selection effects need to be handled!

Observer selection effects are annoying can produce apparently paradoxical effects such that your friends on average have more friends than you or that our existence "prevents" recent giant meteor impacts. But one can control for them with some ingenuity!

Life emerged fairly early on Earth: evidence that it is easy and common? Not so fast: if you need multiple hard steps to evolve an observer to marvel at it, then on those super-rare worlds where observers show up life statistically tend to be early.

If we have N hard steps (say; life, good genetic coding, eukaryotic cells, brains, observers) as difficulty goes to infinity in cases where all steps succeed before biosphere ends they become equidistant between first and last habitability.

That means that we can take observed timings and calculate backwards to get probabilities compatible with them, controlling for observer selection bias.

Our argument builds on a chain from Carter's original argument and its extensions to Bayesian transition models. I think our main addition here is using noninformative priors.

https://royalsocietypublishing.org/doi/10.1098/rsta.1983.0096

https://arxiv.org/abs/0711.1985v1

http://mason.gmu.edu/~rhanson/hardstep.pdf

https://www.pnas.org/content/pnas/109/2/395.full.pdf

The main take-home message is that one can rule out fairly high probabilities for the transitions, while super-hard steps are compatible with observations. We get good odds on us being alone in the observable universe.

If we found a dark biosphere or life on Venus that would weaken the conclusion, similarly for big updates on when some transitions happened; we have various sensitivity checks in the paper.

Our conclusions (if they are right) are good news if you are worried about the Great Filter. we have N hard filters behind us, so the empty sky is not necessarily bad news. We may be lonely but have much of the universe for ourselves.

Another cool application is that this line of reasoning really suggests that M-dwarf planets must be much less habitable than they seem: otherwise we should

...
How should OpenAI communicate about the commercial performances of the GPT-3 API?
by Maxime Riché — 2020-11-24T08:34:08.988Z

OpenAI published their GPT-3 model on mai 2020 and gave access to it through an API to selected developers on June. On September, OpenAI licensed GPT-3 to Microsoft:

OpenAI has agreed to license GPT-3 to Microsoft for their own products and services. The deal has no impact on continued access to the GPT-3 model through OpenAI’s API, and existing and future users of it will continue building applications with our API as usual.

And finally some product using the API are promoted by OpenAI on their website.

I am interested to hear your thoughts on the following:

Given that the OpenAI API is a commercial (success/failure/neutral), how should they display it (as a success/failure/remaining silent/etc.)?

A naive view is that displaying success will encourage competitors to investor more in the short term in AI capability development and large model training.
Manifesto of the Silent Minority
by lsusr — 2020-11-24T05:30:59.099Z

Yesterday I asked readers the Thiel Question. 40 people responded. I have combined the responses into a single political platform. You can view the raw responses here.

This country is tired of the Democrats, Republicans, Libertarians and human politics in general. I, a human-aligned AGI dictator, promise to solve this problem once and for all. If I cannot remove humans from government entirely then I will attempt to abolish democracy. If I cannot abolish democracy then I will repeal the 19th amendment and leave the rest to OpenAI.

Morality is not real. Words have no meaning. I tell lies of every color except white. I actively avert my gaze from all advertising and never read the news.

My careful abstinence from biased information informs me of what's really going on. Like how the Boston Marathon bomb plot was an FBI sting operation that got out of hand. And how most adults born after 1980 in North America have been poisoned by [REDACTED].

Price gouging does not exist. I will clean up Wall Street by legalizing insider trading. I will fix the media ecosystem by abolishing copyright. This will improve public health via music's placebo-like healing effects. I will repeal the 21st amendment. I have drafted a 28th amendment prohibiting civilian smartphone use. Difference between social classes, races, sexes in achievement are significantly genetically based. I will remedy the gap via magick.

I will replace Shakespeare with Zettai Shōnen in English curricula and then abolish all public schools with large class sizes. This is the first step of a long-term strategy to liberate children from adults. The plan includes a political compromise legalizing infanticide of children below the age of 4.

Status is unsatisfying but mating success is the strongest determinant of life satisfaction in humans. I will leave the status quo unchanged because humans mostly just execute social programs and do not seek happiness. Meditation is pointless. "Enlightenment" in the Buddhist sense is a real and non-mystical thing that anybody can attain.

Life requires death. The silent minority is unequivocal in its endorsement of chemical, biological AND nuclear weapons.

Physicists are stuck because reality is uncomputable, we lack hypercomputers and we have no good ways to figure out what a universe is. A renewable, low-carbon economy is difficult if not impossible. MIRI isn't even wrong. Our bright...