trentmkelly/LessWrong-43k huggingface dataset:
Why do stocks go up?
by tryactions — 2021-01-17T16:36:57.447Z

Here are two stories about stocks that I find hard to reconcile:

Stock prices represent the market's best guess at a stock's future price.
Overall, stock prices tend to go up due to advances in technology.

But if stock prices tend to go up due to technology, why isn't that already priced in?

This seems relevant to investment strategy:

The more true #1 is, the less you should expect to beat the market over the long term: lots of other people with skin in the game have worked hard to produce that best guess.
The more true #2 is, the more you might expect to beat the market over the long term: you only need to know which technologies are likely to grow, you don't need to know better than everyone else.

As an example: if you think AI is going to accomplish big thing X, and you think the market already knows this, should you buy and hold relevant AI company stock? Or would you expect the anticipated growth to already be priced in?

Some potential answers I can think of:

The market is continually surprised by the pace of technology. (This seems unlikely to me.)
Significant technology gains are realized in private companies. Stock market indicators go up when these companies go public, but the gains are only realized by the private investors. (But we could then reframe the question as "why do index funds go up" and hit the same issues.)
Time-discounting: the market expects prices to go up, but they would rather have their money now than wait for the return and deal with volatility. As the technological gains are realized, the market expects the price to increase; individuals just can't benefit from that given their (aggregate) preferences. Knowledge of which technologies are likely to grow is already factored in, to the point where expected growth is distributed roughly evenly across all public companies. So story #1 is false, but to beat the market, you still have to know better than everyone else. (My current leading candidate.)
Either story #1 or story #2 is false in some other way.

Question: What are good ways to think about this? What evidence do we have?

EDIT - Summary of things I got from answers/comments:

ike points out that "why stocks go up" is at least partially an open problem known as the "equity premium puzzle". https://www.lesswrong.com/posts/4vcTYhA2X99aGaGHG/why-do-stocks-go-up?commentId=XtR6rrdTnXJ

...
CollAction history and lessons learned
by Ron — 2021-01-17T15:01:20.114Z

Hi all,

@Yoav Ravid mentioned that it might be interesting/useful to get a “retrospective” post on CollAction.org (an assurance-contract website or what we call a ‘crowdacting’ website) that me and a few friends started a while ago. I’ll share a little bit about the history and the challenges that we ran into along the way.

Three points before I dive in:

The reason I’m writing this post is two-fold: a) I hope it helps anyone that has a similar idea; and b) we’re actually looking for a team to take this platform to the next level. So please let me know if you’re interested to lead or be part of this team
The website is still up and running. But at the moment the focus and limited resources that we have are mostly spent on building one of our most successful crowdactions (the Slow Fashion Season) into a movement (appropriately named the Slow Fashion Movement). Which brings us back to point 1b: we’re looking for a new team to take the collaction.org forward
Apologies for writing a rather long post - I didn’t have the time to write a short one, as the saying goes ;)

Some history

A friend and I had this idea about 5 years ago. We were/are interested in collective action problems, more specifically of the Tragedy of the Commons kind. We saw that the solutions to collective action problems (regulation and privatization) weren’t always applicable, desirable, or working as they should. At the same time we saw all these platforms that used the power of the collective, whether it was related to buying (e.g. Groupon), funding (e.g. crowdfunding), advocacy (e.g Avaaz), or something else. So we thought: we should be able to use the internet to provide a new solution to collective action problems. Never before were this many people (3 billion at the time) connected with each other, through the internet. This should allow us to act collectively on an unprecedented scale. And thus the idea of crowdacting was born (well, born...As with most ideas, lots of other people probably had the same idea already as well. E.g. we later found out that

Pledgebank

did something similar in the UK a few years earlier, although they facilitated both crowdacting, petitions, and crowdfunding it seems. The site was taken down in 2015.)

We defined crowdacting as “Coordinated, conditional, collective action to achieve a positive social and/or ecological goal” . Simply put, with crowdacting we ...
How can I find trustworthy dietary advice?
by Daniel Kokotajlo — 2021-01-17T13:11:54.158Z

I currently think that the official dietary advice is sometimes untrustworthy, and I don't know when it's trustworthy and when it isn't, so I'm in a state of epistemic learned helplessness. Some reasons why I think this:

--I get the impression that much of the official advice of the past was actively harmful; the official advice has flip-flopped on many things over the past decades; things which were touted as healthy were later shown to be unhealthy, and vice versa. So presumably some of the current official advice is also actively harmful, and some of it is merely useless. But I don't know which.

--I get the impression that there are studies available supporting pretty much any conclusion; if I want to believe that red meat is bad for me, I can find mountains of studies showing that it is bad for me, and if I want to believe that it is good for me, I can go talk to carnivore diet people and they'll give me convincing evidence it's actually really good for me.

I'm optimistic that if I spent a few weeks of effort looking into the literature and reading the studies myself I could come to good opinions on this topic. But maybe someone here has already done this? Or maybe I'm just wrong about the official dietary advice?
What is going on in the world?
by KatjaGrace — 2021-01-17T11:30:12.275Z

Here’s a list of alternative high level narratives about what is importantly going on in the world—the central plot, as it were—for the purpose of thinking about what role in a plot to take:

The US is falling apart rapidly (on the scale of years), as evident in US politics departing from sanity and honor, sharp polarization, violent civil unrest, hopeless pandemic responses, ensuing economic catastrophe, one in a thousand Americans dying by infectious disease in 2020, and the abiding popularity of Trump in spite of it all.

Western civilization is declining on the scale of half a century, as evidenced by its inability to build things it used to be able to build, and the ceasing of apparent economic acceleration toward a singularity.

AI agents will control the future, and which ones we create is the only thing about our time that will matter in the long run. Major subplots:

‘Aligned’ AI is necessary for a non-doom outcome, and hard.

Arms races worsen things a lot.

The order of technologies matters a lot / who gets things first matters a lot, and many groups will develop or do things as a matter of local incentives, with no regard for the larger consequences.

Seeing more clearly what’s going on ahead of time helps all efforts, especially in the very unclear and speculative circumstances (e.g. this has a decent chance of replacing subplots here with truer ones, moving large sections of AI-risk effort to better endeavors).

The main task is finding levers that can be pulled at all.

Bringing in people with energy to pull levers is where it’s at.

Institutions could be way better across the board, and these are key to large numbers of people positively interacting, which is critical to the bounty of our times. Improvement could make a big difference to swathes of endeavors, and well-picked improvements would make a difference to endeavors that matter.

Most people are suffering or drastically undershooting their potential, for tractable reasons.

Most human effort is being wasted on endeavors with no abiding value.

If we take anthropic reasoning and our observations about space seriously, we appear very likely to be in a ‘Great Filter’, which appears likely to kill us (and unlikely to be AI).

Everyone is going to die, the way things stand.

Most of the resources ever available are in space, not subject

...
Wuwei
by lsusr — 2021-01-17T08:28:21.034Z

Wuwei (kanji: 無為, simplified: 无为, pinyin: wúwéi, hiragana: むい) denotes a political philosophy and a state of mind.

Wuwei combines wu 無 and wei 為. Wu is straightforward to translate. Wu means "don't" or "without". Wei is not straightforward to translate. I like to think of wei as "action" or "intention". The CC-CEDICT project defines wei in terms of performing a capacity. That is, to become by doing i.e. to take a role.

為 为 [wei2] /as (in the capacity of)/to take sth as/to act as/to serve as/to behave as/to become/to be/to do/by (in the passive voice)/

―CC-CEDICT

Political Philosophy

The political philosophy of wuwei can be crudely translated into "self-government".

The Emperor

In eastern political philosophy, the emperor is above running the government. The actual job of running the government is his ministers' responsibility. If anything goes wrong then it is the ministers' fault. If the emperor does make a proclamation then it should be vague and abstract. If things go right then it is due to the emperor's wise leadership. If things go wrong then it is the fault of corrupt incompetant bureaucrats. The emperor leads indirectly, by setting a example of model behavior.

The Shogun

"I will punish the bird if it does not sing," said Nobunaga.

"I will reward the bird it it does sing," said Hideyoshi.

Tokugawa waited for the bird to sing.

State of Mind

The wuwei state of mind is analogous to wuwei statecraft. It is the idea that the best way to act is by letting yourself act instead of compelling yourself to act.

I put almost no effort into actually doing things. If I keep away from time-wasting activities then I do worthwhile things by default.

In Zen, you attain enlightenment by default simply by not doing anything else.
Use conditional probabilities to clear up error rate confusion
by Maxwell Peterson — 2021-01-17T08:27:38.137Z

It's common in my part of the data science world to speak of model accuracy for classifiers in terms of four quantities: True Positive Rate (AKA Sensitivity, Recall), False Positive Rate, True Negative Rate (AKA Specificity), and False Negative Rate. But after more than 5 years in the field, I still have a hard time remembering which is which. Recently I found that writing conditional probabilities instead makes things clearer for me. For a binary classifier - where the response, or actual value, is one of {true, false} - the following rates and probabilities are the same:

True Positive Rate is the same as the probability of predicting "true" when the actual label is "true". Or: P(predicted true | actually true).
False Positive Rate: P(predicted true | actually false)
True Negative Rate: P(predicted false | actually false)
False Negative Rate: P(predicted false | actually true)

And too for the non-binary case. If I have k possible outcomes, then there is a False Positive Rate for each, FPRk = P(predicted to be outcome k | not actually k), and so on for the other rates.

I think those probability expressions are what I've been mentally translating the rate terms into, in my head, every time I hear or say something like "False Positive Rate". For me, that subconscious translation was slow, and annoying, and bogged up my thinking. Writing the probability expressions directly when displaying a confusion matrix, or the axes of a ROC curve, has cleared up my thinking and conversations about classifier error rates. You don't have to remember which is which - you can just read off the conditional probability.

Wait, are those the right probabilities?

All I meant to write for this post was the single section above. So you can stop here and still be good. But... the way these error rates are commonly calculated might not be quite right. I've always calculated e.g. True Positive Rate as sn, where n is "number of actual positives" and s is "number of those positives predicted correctly". But the most basic frequency expression for a rate, where one knows nothing - nothing! - except that it is physically possible for an outcome to be 1 or 0, and additionally that there have been s 1's in a sample of size n, is Laplace's rule of succession: s+1n+2. So we should probably be using the den...
Excerpt from Arbital Solomonoff induction dialogue
by Richard_Ngo — 2021-01-17T03:49:47.405Z
Link: https://arbital.com/p/solomonoff_induction/?l=1hh

This post contains an excerpt from a dialogue by Eliezer about why Solomonoff induction is a good answer to "how to do epistemology using infinite computing power". I'm link-posting it from Arbital because I found it really useful and wish I'd seen it on Less Wrong earlier. (Edit: it's now been cross-posted in full to LessWrong.) Eliezer covers a wide range of arguments and objections in a very accessible and engaging way. It's particularly interesting that, near the end of the dialogue, the two characters discuss an objection which feels very similar to what I was trying to convey in my post Against Strong Bayesianism - specifically my argument that

An ideal bayesian is not thinking in any reasonable sense of the word - instead, it’s simulating every logically possible universe. By default, we should not expect to learn much about thinking based on analysing a different type of operation that just happens to look the same in the infinite limit.

Given that both of the characters agree with the version of this argument made in the Arbital dialogue, I guess that my position is closer to Eliezer's than I previously thought. I suspect that the remaining disagreement is something like: given these problems, is it better to aim for "some future formalism that's better than Solomonoff induction", or instead to focus on thinking about how intelligence actually functions in practice? Reading this Arbital post has moved me slightly towards MIRI's position, mainly because it's evidence that Eliezer had considered this specific argument several years ago. However, I'm still more excited about the latter - in part because it seems that logical inductors are vulnerable to a similar type of objection as Solomonoff induction. The following excerpt is the (relatively small) section of the original dialogue which focuses on this type of objection.

Excerpt starts here, with Ashley (a fictional computer scientist) explaining one source of her skepticism about Solomonoff induction.

Ashley: The 'language of thought' or 'language of epistemology' seems to be different in some sense from the 'language of computer programs'. Like, when I think about the laws of Newtonian gravity, or when I think about my Mom, it's not just one more line of code tacked onto a big black-box computer program. It's more like I'm crafting an explanation with modular parts - if it contains a part ...
How do I improve at being strategic?
by Conor — 2021-01-17T01:36:14.070Z

After reading Humans Are Not Automatically Strategic I realized I am not very strategic, and I want to improve. As practice I'm been using the recipe for strategic planning from that post to create a strategy to become more strategic. Specifically, creating a system or framework for forming strategies.
That said, I'm new to LW and do not have rigorous science or math training, so perhaps strategy is background knowledge here? I've found little info on this topic on LW. Most of which is tactical not strategic. Have I missed something? It seems like training in strategy would be an extremely useful framework to hang all these instrumental rationality techniques from.
Status as a Service (Done Quick)
by SarahNibs — 2021-01-16T23:25:13.677Z

This is a summary of an excellent but long blog post, Status as a Service (StaaS) — Remains of the Day (eugenewei.com). I wanted everyone to have access to the perspectives and ideas laid out in the post, but the barrier to entry is currently a 20,000 word essay. The main thing knowingly missing from these summaries is that the original is humorously well-written, which will not come through.

Also included is a very brief look at LessWrong in light of the ideas, which I could not prevent myself from writing.

100x compression

List of takeaways:

Human beings chase status (social capital) and are good at recognizing efficient ways to gain it.
Social networks provide social capital opportunities, utility, and entertainment. Analyzing just utility/entertainment/network effects isn't enough.
Social network success is highly analogous to cryptocurrency. New capital (new status hierarchy), proof-of-work (sharable Tweet), built-in and increasing scarcity (harder to get the most followers once Twitter is saturated).
Young folks are the biggest target: they have more time and less existing efficient ability to gain social capital.
Proof-of-work is an asymptote of growth; only so many people will compose a video for TikTok. Social capital devaluation can also kill a network; classic example is parents-joining chased kids off Facebook.
Merely exposing status might be enough to provide social capital opportunities, a persistent profile + artifacts is not necessarily required.
Social capital can be exchanged for goods and services. Sometimes.
Unless you are status-poor, you probably don't understand the actions of those trying to gain social capital cheaply.

10x compression

Efficient Status Opportunities

Humans are status-seeking monkeys and continually seek out efficient paths to maximizing their social capital, but social networks are rarely analyzed on the dimensions of status or social capital. Let's analyze social networks in general, and many specific instances like Facebook or TikTok in particular, through a lens which emphasizes their status-seeking dynamics.

First and most importantly: social networks provide lucrative status-gaining opportunities. Some of the first good Twitter users have millions of followers today, far out of proportion to their likelihood of gaining millions of followers if they joined for the first time right now.

There's gold in them thar hills.

Network effe...
Why Productivity Systems Don't Stick
by Matt Goldenberg — 2021-01-16T17:45:37.479Z

Note: Experimental, trying to repost a Twitter thread as a Lesswrong post to see if people like it (so if you like it, or don't, tell me).

Let's talk a little bit about oscillating motivation, shadow values, non-coercion, and the problem of revolving productivity systems.

We've all had the feeling that new productivity system is working! Only to find that days, weeks, or months later it falls off, and we fall back into a period of being unproductive. Hell, I was stuck inside this revolving door for more than a decade.

I tried every planner, app, book, coach, journal, todo list, calendar, and retreat under the sun. But always, would find myself back in the revolving door, back to where I started.

Robert Fritz calls this "Oscillating Tension." in order to discover the root of it, we need to look at what procrastination actually is.

What is Procrastination?

It's trendy these days to say that "We procrastinate to avoid negative emotions." That's true, but what does that make procrastination? It's a defense mechanism against taking actions that hurt psychologically. Procrastination is a tool to prevent psychological self-harm.

So along comes this tool that we can use to get ourselves to act! It charges us money, or it gets us really excited about our todo list, or it gives us social pressure to take that action anyway.

But the underlying emotion is still (often) there. Every time we trick ourselves into doing the task without resolving that negative emotion, we're confirming to our defense mechanisms that they were right to worry.

And then, when our environment changes, or we miss that call, or we misplace that planner... guess what happens? Our procrastination knows using that tool leads directly to self-harm, so it causes you to procrastinate from ever using that tool again. Oscillating tension.

The Need for Autonomy

There's another problem here too. It's not just the task itself that felt bad - the very fact that you felt you had to force yourself (instead of acknowledging and processing the negative emotions) leads the task to be even MORE AVERSIVE!

According to self-determination theory, humans have an innate need for autonomy. When we feel coerced into action, we resent this need not being met. That resentment builds over time. This leads to the second pattern of oscillating tension.

The pattern: We find a new coercive method that gets us to do things through "self-disc...
Is it the case that when humans approximate backward induction they violate the markov property? 
by Quinn — 2021-01-16T16:22:21.561Z

Backward induction is the process of reasoning backwards in time, from the end of a problem or situation, to determine a sequence of optimal actions.

Backward induction is used to solve Bellman equations in dynamic programming, which is leveraged in reinforcement learning.

The markov property holds of a process if the probability of each event depends only on the state attained in the previous event, formally if P(st+1)=f(st) for some f.

For an example of a human approximating backward induction, I'll use the chocolate example from Instrumental and Terminal Values: Alice wants chocolate, so she computes that she needs to go to the store, so she computes that she needs to drive, so she computes that she needs to grab her keys. If it's not obvious how grabbing keys is related to obtaining chocolate, it should be obvious when you look a the whole chain.

Formally perhaps we might say for any goal An we can compute a list A1←A2←...←An−1←An, where A←B denotes "A leads to B and we can compute that fact from looking at B" or "from the desire for B we can compute the desire for A".

But! As Eliezer points out, if you're at the Agrabkeys←Adrive step, and an oracle tells you that the store is out of chocolate, you're not forced into some semi-myopia where you feel the need to start driving but you no longer know why. Instead, the whole chain might collapse at once, radically reconfigure itself from the initial desire for chocolate.

I feel like there's this possibility for a wrong interpretation of backward induction that's analogous to the markov property, where every desire in the chain is a function of the preceding desire to which it is instrumental. This wrong interpretation actually lies precisely in a blindspot of my previous formalization! Put much better, though a little difficult on the eyes:

Formally perhaps we might say that for any goal An we can compute a list of shrinking lists A1←(A2,...,An)←(A3,...,An)←...←(An−1,An)←An, where Ak←(Ak+1,...,An) denotes "Ak leads to Ak+1, which we can compute by looking at the sequence (Ak+1,...,An), (notice recursion, base case An−1←An)" or "from a desire for the list (Ak+1,...,An) we can compute the desire for Ak".

This formalism is robust to disruptions in the chain mid-planning, because the whole chain is input to compute a preceding instrumental goal.

So I have a basic belief about this, and I want to find a way to verify if it...