trentmkelly/LessWrong-43k huggingface dataset:
Useless knowledge; why people resist education improvement
by Andrew Vlahos — 2021-02-26T03:43:03.794Z

I was a physics major, but there were still some general education courses that were required. Although a few had some useful information, most of them were worthless. I went to the first few classes, and if I found that they contained no useful information, I would not read the books or go to the lectures except when needed. One of these courses was called “Survey of the Arts.”

For my midterm essay, I argued that art movements typically have patterns opposing societal change, such as the “Romanticism” art movement standing against increasing industrialization and a decreasing sense of individual dignity.

For my final essay, I argued that art movements typically have patterns reflecting societal change, such as the “Realism” art movement being caused by increasing industrialization and a decreasing sense of individual dignity.

I got good grades on both.

Most of you, just from reading this description, would immediately see some of the problems with this course. The standards of claims are so weak that you can say "most art movements have trends opposite to society's" and "most art movements have trends in the same direction as society" and have both be considered true. (You're already past 100%, and now what do you do when you find a trend unconnected to society?). Many of the things taught are empty labels that neither help predict future trends nor help students make better art themselves. There were a lot more problems with the course than what I said, but this is a good start.

This post isn't just venting off steam, though. The point is to show the mindset of why this persists. Consider the comments that appeared when I posted this story here. They were brutal.

There were a few common threads though. One is that people thought that rejecting these courses means that I am a total bore who has no interest in learning anything except STEM. Another is that people complained about STEM elitism. Most tellingly, however, is that a lot of them said "look, the poster did learn something! They practiced building strong arguments and things like that" or "There's no such thing as useless knowledge". Putting aside for the moment the fact that much of the information is redundant such as creating an argument, explaining why this is wrong seems worthwhile.

My mom has read dozens of books about racism. She follows blogs about it and regularly has meetings with like-mind...
What is the VIX?
by Annapurna — 2021-02-26T01:05:06.420Z

Original Post Here

The VIX is a unit of information within the financial market system that investment professionals use to make investment decisions. There are derivatives (options and futures) of that unit of information that investment professionals (and regular people) can use as tools to trade and/or invest within the financial market ecosystem.

What unit of information does the VIX track? the volatility1 of the S&P 500 index over the next 30 days, annualized. What does this mean?

Volatility

is a measure of the severity of the return of an asset, regardless of whether the return is positive of negative. Lets say that on Monday Apple stock goes down 6%, Tuesday up 5%, Wednesday down 7%, Thursday up 8%, and Friday down 10%. Since Apple stock typically moves under 2% per day, we can infer that this week Apple stock was volatile. The higher the volatility of an asset, the riskier it is perceived.

The S&P 500 index

is a stock market index composed of the 500 largest American companies that trade in public exchanges. It is probably the most common financial market index today.

So what the VIX tracks is what market participants are implying the S&P 500 Index volatility will be over the next 30 days. The VIX presents that number in percentage terms, which is then annualized. The higher the VIX percentage, the higher it is implied that the risk of the S&P 500 index is over the next 30 days. This is why the VIX is considered the fear gauge. It informs market participants how risky it is to invest in the S&P 500 index in the near term.

How market participants derive future implied volatility of the index and how the Chicago Board Options Exchange (The creators of the VIX) calculate the VIX is somewhat complex and beyond the scope of this post. You can read the VIX whitepaper

here

. I would suggest also looking at

The Black-Scholes Model

formula for pricing options before looking at the whitepaper. I am happy to discuss in more detail the complexity of the VIX and options pricing in private.

You can read more about the VIX here.

This is not investment advice. For informational purposes only. I hold a long position on the VIX through call options.

1 To be more specific, the VIX tracks the variance, or volatility squared. Non quantitative financial professionals use variance and volatility interchangeably, ...
What are a rationalist's best tools for better decision making?
by acomeyer — 2021-02-25T22:06:55.810Z

I'm curious what others here use for improving their decision making skills. For something so important (making good decisions), society doesn't seem very interested in collectively improving our decision making. This is likely due to the fact, right or wrong, that we collectively suffer from "resulting" (or judging our decisions based on their outcomes alone). It just amazes me that you can go through 15+ years of education and not have a single course on how to make better decisions. Perhaps it suffers from the same fate other practical courses (like personal finance) do or that the idea is you learn the details of a subject and leave the decisions up to the individual? But nonetheless seems important enough to bring some attention to it.

I personally try to improve my decision making by trying to add skin in the game by placing bets and investments where possible and use a decision journal, which has arguably been the best tool for me personally. What do you use or recommend?
Is there any serious attempt to create a system to figure out the CEV of humanity and if not, why haven't we started yet?
by Jonas Hallgren — 2021-02-25T22:06:04.695Z

Hey, fellow people, I'm fairly new to LessWrong and if this question is irrelevant i apologise for that, however I was wondering whether any serious attempts to create a system for mapping out the CEV of humanity have been started yet?

Since the CEV ismore of a democratic process than any other alignment system I can think of, it would make sense to try to create a database for the purpose of a future AGI to calibrate itself on. If we where to create the database now we could explore if it suited our needs through trial and error (testing whether it would predict moral decisions), which would mean that we would get a more functional alignment system than we could otherwise get. Also as a follow up, if we where to create a central authority for creating this database, there's a possibility that the authority could become a central alignment checking facility meaning that we could avoid potential misalginment disasters.

There are therefore quite clear reasons for me why it would be a good idea to start this project which is why I'm wondering if there are any such plans.

Thank you for your time.
My Thoughts on the Apperception Engine
by J Bostock — 2021-02-25T19:43:55.929Z

Epistemic status: I am writing mainly to clarify my own thoughts on this, but I think it is definitely worth sharing what I have here. This is not meant to be a review of the paper itself, rather a discussion of its implications.

So:

We note in particular that in the sequence induction intelligence tests, our system achieved human-level performance.

This is a very flashy claim from the authors of this paper, which has been discussed briefly on LW. They are referring to the Apperception Engine (surely a name Babbage would be proud of), which they claim is able to create human-readable causal models, and perform at human-level in certain domains. Wow! Or perhaps not.

The AE itself resembles a small portion of AIXI if it was implemented by someone who hadn't heard of Bayes' theorem (or AIXI), which acts on an incomplete sequence of "sensory inputs" (e.g. 123456789 of which only --34-6-8- might be shown to the system). It has a set of meta-rules which define a large series of what we might call "hypotheses" about the world. Each of these consists of a set of initial conditions at timestep 0; and a list of rules which both relate each timestep to the next, and determine the predicted sensory input. These are iterated to create a list of sensory inputs over time, which is compared with the "visible" elements of the sequence. The most important part of this process is that these hypotheses generally posit some unseen objects with various states: they are models of the world with internal structure and rules.

To choose between multiple hypotheses which perfectly fit the visible elements of the sequence, a cost function is used, playing the role of a complexity-penalizing prior. (There are also some other constraints on hypotheses, but I think these are mostly irrelevant here) These simplifications compared to AIXI mean it is computable (although it presumably sometimes fails to find a hypothesis) but this does not necessarily mean fast.

Computable does not necessarily mean fast

One way of thinking about this is that the AE is an example of a class of systems, analogous but (in many ways) orthogonal to the "class" of neural networks. It seems relevant that the AE is good at tasks that NNs are bad at, especially working with very small amounts of data. It also appears that the AE is not the first system in this class, in fact in the paper it is compared to similar programs d...
Compromising with Compulsion
by matejsuchy — 2021-02-25T16:43:25.000Z

Xposted from my substack.

Epistemic Status: Highly Speculative, Possibly Useful

Do I contradict myself?
Very well then I contradict myself,
(I am large, I contain multitudes.)

— some guy

Whence arises contradiction? This is an urgent question, especially for the many who sour at Whitman’s suggestion that it is in fact very well that it does so at all. In this post, I’ll introduce a toy model which has suggested itself to me recently as a bridge between various theories of cognition (Friston & Guyenet) and Jungian psychology.

Predictive coding tells us that our sensory function is the product of a bayesian process that combines prior beliefs with raw data. The Fristonian formulation uses minimal assumptions to formalize this idea and cast agent action as an optimization algorithm that minimizes the divergence between the actual distribution p on prior beliefs and raw data we experience and a target distribution t. But what are p,t really?

Let’s imagine we have an agent subject to a simple survival pressure: the agent is subject to a repeated game for which in every iteration he is shown an array of images. If he is able to point to the image that cannot be construed to match the others, he gains one util; otherwise, he loses one util. Assume for the sake of argument that in every case the images will be of either n-1 rabbits with one non-rabbit duck or n-1 ducks with one non-duck rabbit (if asserting that the duck must be non-rabbit seems silly, sit tight for a paragraph or so). Because target distributions can be associated with preferences, we infer that his target distribution is one where the images are available and usefully distinguishable in the sensory function. That is, where raw data from the images are available and matched to priors that allow for the appropriate odd-one-out to be identified. To get there, the agent applies some RL algorithm to update his distribution p on raw data (by updating his actions — for example, we can expect that he will converge to turning his head toward the images in question) and on priors (going from blank slate to having some maximally useful tiling of the latent space). Once at t, the agent sits pretty.
Ok, so far so good. The agent starts out with p0 initialized at some random point in the distribution of distributions, applies an RL algorithm, and ends up at t (or t’ in t’s level-set).
Now, let’s make things int...
Covid 2/25: Holding Pattern
by Zvi — 2021-02-25T14:30:01.836Z

Scott Alexander reviewed his Covid-19 predictions, and I did my analysis as well.

It was a quiet week, with no big news on the Covid front. There was new vaccine data, same as the old vaccine data – once again, vaccines still work. Once again, there is no rush to approve them or even plan for their distribution once approved. Once again, case numbers and positive test percentages declined, but with the worry that the English Strain will soon reverse this, especially as the extent of the drop was disappointing. The death numbers ended up barely budging after creeping back up later in the week, presumably due to reporting time shifts, but that doesn’t make it good or non-worrisome news.

This will be a relatively short update, and if you want to, you can safely skip it.

If anyone knows a good replacement for the Covid Tracking Project please let me know. Next week will be the last week before they shut down new data collection, and I don’t like any of the options I know about to replace them.

Let’s run the numbers.

The Numbers

Predictions

Last week: 5.2% positive test rate on 10.4 million tests, and an average of 2,089 deaths.

Prediction: 4.6% positive test rate and an average of 1,800 deaths.

Result: 4.9% positive test rate and an average of 2,068 deaths.

Late prediction (Friday morning): 4.5% positive test rate and an average of 1,950 deaths (excluding the California bump on 2/25).

Both results are highly disappointing. The positive test rate slowing its drop was eventually going to happen due to the new strain and the control system, so while it’s disappointing it doesn’t feel like a mystery. Deaths not dropping requires an explanation. There’s no question that over the past month and a half we’ve seen steady declines in infections, and conditions are otherwise at least not getting worse. How could the death count be holding steady?

One hypothesis is that weather messed with the reporting, but Texas deaths went down and the patterns generally do not match. Virginia saw a giant jump in the death rate and I’ve adjusted them down by 475 deaths to account for that, which seem to be holiday-related catchup even though it seems super late for that. Nothing else stood out in the charts. As per my rules, since the Virginia adjustment improves my accuracy and there’s a non-zero amount of ambiguity, I’m not including it in the results section. If included, that would move u...
Never Go Full Kelly
by SimonM — 2021-02-25T12:53:50.618Z

Introduction

I am assuming the reader is at least somewhat familiar with the Kelly Criterion. If not, I suggest you read this first. There is a sense in which I am mostly just adding meat to the bones of "Never Go Full Kelly". (Or more narrowly his two last two points):

You can't handle the swings
You have full knowledge of your edge

Hopefully I can give you some rules of thumb for how to handle both those issues:

Scale down all bets "fractionally"
Scale down by these specific fractions:

A function of your risk-adjusted edge (1-1/(1+SR^2))
A function of how much information is in your prediction vs your counterparty (your "information" / the "information" between you)

You want to do both if you can't handle the swings AND don't have full knowledge. (Which is basically everyone).

For this post, fis the Kelly fraction, p is your estimate of the probability of an event, q is the market / your counterparty's probability. In this notation:

f=p−q1−q

After being introduced to the Kelly criterion the next thing people are introduced to is "fractional Kelly". Specifically the idea is rather than wagering the Kelly fraction f on an event, you wager αf (typically with 0<α<1).

What's wrong with Kelly?

Kelly is pretty aggressive compared to most people's risk tolerance. Upon seeing a Kelly bet some people will be somewhat dubious as to how aggressive it is. If someone offers you evens on something you think is 60%, Kelly would have you betting 20% of your net worth. 20% seems like an awful lot to bet on something which is only going to happen 60% of the time. You can play this game yourself here and see how comfortable you are with the drawdowns with "fake" money. (This was originally from a paper where the participants did not perform well). If you don't fancy playing, consider this thought experiment: 3 tails is a 50% draw down, and this will happen ~once every 24 flips.

To give a more recent example, before the 2020 Presidential election, the betting markets were giving Biden a 60% change. If you believed 538, and put it at ~90%, would you have been comfortable staking 50% of your net worth on Biden?

Let's have a look at what the geometric growth rate looks like for different fractions of a Kelly bet. Growth is maximised at Full Kelly:

Returns vs Bet sizing

This is plotting fractional-Kelly coefficient (α) ...
The Inner Workings of Resourcefulness
by Nora_Ammann — 2021-02-25T09:15:35.919Z

Cross-posted to my personal blog.

Meta: If not for its clumsiness, I would have titled this piece “[some of] the inner workings of resourcefulness”. In other words, I do not want to pretend this piece offers a comprehensive account, but merely partially insights.

About this time a year ago - when for a lot of people, the world came crashing in on them - I started obsessing over an idea. Maybe, I thought to myself, one of the most valuable things I could be focusing on at the moment is becoming more resourceful.

In this time of crises, starting early 2020, what was, in a sense, most needed were people capable of dealing with flexibly new, difficult situations and of finding and enacting creative solutions despite constraint and uncertainty. People capable of staying oriented, in the face of unfolding chaos, while also being relentlessly effective in their action. People who can give, plentifully, when others are in need of help. Resourceful people

For much of March, I wasn't resourceful. I had shit on my own to figure out, and a lot of my energy flew (sunk!) into that, and into handling the wave of uncertainty that, for a couple of days, took away my breath.

With opportunities for meaningful mitigative action at my fingertips yet much of my resources gobbled up, I couldn't stop ruminating the question of what I could have done to be a more valuable actor now.

For some time now, I have been deliberately working on making myself more valuable to this world. Never before in my lifetime was the world so much in need of me. And yet I felt so utterly unprepared.

This is when it came to me that resourcefulness was an important concept in this quest of mine.

Turns out, I didn’t know - not really - what being resourceful means. Or rather, while I could give a description of what resourcefulness might look like, I wasn’t able to “pull back the curtains” and look at the “inner workings” of resourcefulness. I needed to remedy this situation, and I have been pondering the nature of resourcefulness ever since then.

This morning, a day in early 2021, like a bird flying up to my window bench in joyful chatter, the thought came to me: What have I learnt about resourcefulness over the last year? Let’s take stock.

I am confident I haven’t finished my quest to understand the inner workings of resourcefulness. In fact, the rest of this post will adopt only tw...
Against unstoppable crypto prediction markets
by lemonhope — 2021-02-25T06:02:23.102Z

As others have outlined, there are a million ways that prediction machines and betting markets can go wrong and do crazy things like spontaneously inducing unlikely senseless verifiable destruction. I am aware of interest in making better, faster, more expressive prediction markets. But if you make an unstoppable smart contract then those problems will surely manifest and there will be nothing to be done about it. So I urge future market makers to build robust moderation systems into the foundation of your prediction market, to the extent possible.
Tsuyoku Naritai
by Ruby — 2021-02-25T05:40:08.470Z

This is the fourth speech in the wedding ceremony of Ruby & Miranda. See the Sequence introduction for more info. The speech was given by Ruby.

Riva descends from podium.

Brienne: We have spoken of the horrors that exist, the threats that loom, and the good worth fighting for. We cannot know that we are equal to these challenges, but we won’t let that stop us. I call upon Ruby to speak of our determination to Become Stronger.

Ruby ascends podium.

IMAGE 3 PROJECTED ON PLANETARIUM

Composed of gas and dust, a pillar in the Carina Nebula

Ruby commences speech.

There is a cry among us, a cry dear to us. “Tsuyoku Naritai” – “I want to become stronger.” Embodied in these few foreign syllables is one of our deepest values and greatest virtues. We say these words because we know that the challenges we face today are great, perhaps too great, perhaps greater than we can handle – today. But that does not matter, because we want to become stronger.

Some believe that we were created in the image of a perfect, wise, and just god. We know that we were not. No, we are the product of a blind, uncaring, unfeeling, raw mechanical process. We were hacked together, a disparate assortment of adaptations just good enough to pass our genes onto another generation, without oversight, without foresight, and certainly without benevolence. We became whatever worked – but only whatever worked in a time and place we left long ago.

So when we are faced with the suffering of millions, when we are marching towards our deaths, and when threats to our existence looming on the horizon – we cannot expect to be equal to these trials. There is no magical reason why the universe ought to present us with challenges we are strong enough to overcome. But in this cold and uncaring universe, there is one mercy. One card we hold in our hands – the capacity to grow. The ability to become stronger.

[We will run rampant with this ability, for in it is our capacity to do anything.] We will grow until we have the might to banish the darkness, and until we have the power to seize the stars.

Already we have come so far: in wisdom, skill, and kindness. Once we accepted dogma, the stories which were told to us, but then we learnt to argue and to reason towards truth. Later we learnt that even reasoning can easily go astray, so we taught ourselves to look at the world, to let it tell us what was true. Afte...