trentmkelly/LessWrong-43k huggingface dataset:
A Refutation of (Global) "Happiness Maximization" 
by fare — 2020-08-25T20:33:36.866Z

NB: This is a follow-up to my previous essay The Criminal Stupidity of Intelligent People, as cross-posted to my blog: http://fare.livejournal.com/168562.html.

As promised,
here is a detailed rational refutation of
the utilitarian concept of (Global) "Happiness Maximization" in general,
and its specific variant that concludes in favor of
wireheading,
the cultivation of individuals in vats
in a permanent state of artificially stimulated bliss.

Wireheading

To recall the background, I recently had a discussion with
a very intelligent and extremely well-meaning colleague of mine,
Jeff Kaufman,
who, as an aside during an evening he organized
to foster efficient philanthropy,
was seriously defending the utilitarian ideal
of "Happiness Maximization" to its wantonly absurd consequence,
wireheading:
putting humans, by force if needs be, into vats,
where the pleasure centers of their brains
will be constantly stimulated for lifelong bliss
through properly implanted electric wires and/or chemical injections.
Or perhaps instead of humans, the Utilitarian Elite would use
rats, or ants, or some brain cell cultures
or perhaps nano-electronic simulations of such electro-chemical stimulations;
in the latter cases, biological humans,
being less-efficient forms of happiness substrate,
would be done away with or at least not renewed
as embodiments of the Holy Happiness to be maximized.
He even wrote at least two blog posts on this theme:
hedonic vs preference utilitarianism in the Context of Wireheading,
Value of a Computational Process.

Failing the Smell Test

Now, in my previous post,
The Criminal Stupidity of Intelligent People,
I gave several blanket principles by which one might reject
such mass-criminal schemes.
Indeed, there is no question that Jeff is much more intelligent than most
(I would notably wager that
he's more intelligent than I, as measurable by an IQ test),
and much more knowledgeable of this topic than anyone;
few could oppose him a rational refutation like I'm going to offer
(no false modesty from me here);
yet one need not have identified the precise mistakes of a demonstration
to reject its provably absurd conclusions.
Just like you can without looking reject elaborate alleged proofs
of the squaring of the circle, you can without looking reject proofs
that some higher principle calls for mass murder:
that's the Libertarian principle,
recognizing that ...
Sunday, 20/8/30, 12pm PDT – Tagging Celebration: Habryka/Crawford + Party
by Ruby — 2020-08-25T20:21:37.473Z

FB Event
Woop! Woop! This past Saturday, gallant taggers brought our immense first tagging campaign to a close. The archives have been swept and every post with over 25 karma has been given at least one tag. To date, 15,440 tags have been applied to 8,022 posts.

See the full celebration post here.

While this is only the beginning for tagging, what a beginning it is! We think this deserves some celebration. We're preparing a two-part event:

Part 1: Oliver Habryka and Jason Crawford discuss Intellectual Progress

https://us02web.zoom.us/j/82547847213

Back in 2017, when the future of LessWrong was in doubt, Habryka led the revival efforts, pulling together a team and creating LessWrong 2.0. Central to Habryka's vision of LessWrong was that it would accelerate intellectual progress on the important problems facing humanity. Intellectual progress here simply meaning progress on building knowledge and understanding.

For several years, Jason has been seeking to answer the question of which factors have led to human progress in general: how did we go from living at the mercy of nature with merely stone tools and fire to buildings, electricity, medicine, legal systems, etc? Jason's research, and info about the broader Progress Studies movement, can be found at his blog, Roots of Progress (with many pieces crossposted to LessWrong).

Of course, the history of human progress, in general, is tightly woven with humanity's intellectual progress. When our understanding of the world increased, so did our ability to shape it (for better or worse).

To celebrate the new tagging system (itself designed with the goal of intellectual progress) we've decided to have Habryka and Jason chat about questions such as:

What historical factors were important for intellectual progress?
What conditions are most important to create now in order to get intellectual progress?

Starting at 12:00 PDT, Habryka and Jason will chat on Zoom for ~1 hour, including some Q&A

Part 2: Party in Rational Woods (our Topia location)

https://topia.io/rational-woods (feel free to check it out now)

Following the talk, we will migrate to a social environment that more easily allows people to strike up small group conversations. Raemon has created the marvelous Rational Woods on the Topia platform. You can check it out now for social hangouts. There are some neat features, so make sure to click on things.

This sec...
Mathematical Inconsistency in Solomonoff Induction?
by curi — 2020-08-25T17:09:49.832Z

What counts as a hypothesis for Solomonoff induction? The general impression I’ve gotten in various places is “a hypothesis can be anything (that you could write down)”. But I don’t think that’s quite it. E.g. evidence can be written down but is treated separately. I think a hypothesis is more like a computer program that outputs predictions about what evidence will or will not be observed.

If X and Y are hypotheses, then is “X and Y” a hypothesis? “not X”? “X or Y?” If not, why not, and where can I read a clear explanation of the rules and exclusions for Solomonoff hypotheses?

If using logic operators with hypotheses does yield other hypotheses, then I’m curious about a potential problem. When hypotheses are related, we can consider what their probabilities should be in more than one way. The results should always be consistent.

For example, suppose you have no evidence yet. And suppose X and Y are independent. Then you can calculate the probability of P(X or Y) in terms of the probability of P(X) and P(Y). You can also calculate the probability of all three based on their length (that’s the Solomonoff prior). These should always match but I don’t think they do.

The non-normalized probability of X is 1/2^len(X).

So you get:

P(X or Y) = 1/2^len(X) + 1/2^len(Y) - 1/2^(len(X)+len(Y))

and we also know:

P(X or Y) = 1/2^len(X or Y)

since the left hand sides are the same, that means the right hand sides should be equal, by simple substitution:

1/2^len(X or Y) = 1/2^len(X) + 1/2^len(Y) - 1/2^(len(X)+len(Y))

Which has to hold for any X and Y.

We can select X and Y to be the same length and to minimize compression gains when they’re both present, so len(X or Y) should be approximately 2len(X). I’m assuming a basis, or choice of X and Y, such that “or” is very cheap relative to X and Y, hence I approximated it to zero. Then we have:

1/2^2len(X) = 1/2^len(X) + 1/2^len(X) - 1/2^2len(X)

which simplifies to:

1/2^2len(X) = 1/2^len(X)

Which is false (since len(X) isn’t 0). And using a different approximation of len(X or Y) like 1.5len(X), 2.5len(X) or even len(X) wouldn’t make the math work.

So Solomonoff induction is inconsistent. So I assume there’s something I don’t know. What? (My best guess so far, mentioned above, is limits on what is a hypothesis.)

Also here’s a quick intuitive explanation to help explain what’s going on with the math: P(X) is both shorter and less probable than P(X o...
The Best Toy In The Park
by jefftk — 2020-08-25T14:30:06.615Z

One of the

playgrounds
near our house has an

angled
ring shaped merry-go-round:

There are lots of different toys at the park, but this one is
different. It has depth.

It's a freely spinning ring, with a 10deg tilt. As you walk up, it
spins you back down again. It has enough momentum, however, that you
can get yourself to higher positions on the ring by working it back
and forth. Because of the tilt, your speed changes as you move around
the ring, and you have to adjust your angle relative to the ring as
well. First is the challenge of staying upright as you walk at the
bottom. Then the challenge of getting yourself to the top. Then the
challenge of getting around it at different speeds, changing
orientation, riding with other people, rolling on it, jumping, etc.

There are shallow games, like Tic-tac-toe, that are fun for a bit, and
deeper games, like Go, that
can be enjoyed for years. While I haven't spent years on any of these
toys, this angled spinning ring has the most depth of any I've
seen.

Comment via: facebook
Why hasn't there been research on the effectiveness of zinc for Covid-19?
by Rudi C — 2020-08-24T20:58:33.962Z

There was significant interest in zinc early on. It seemed like one of the most promising avenues for protecting oneself against Covid-19. Searching on both Google and Lesswrong itself, I do not see new research on it. (This is the closest thing I found; A metareview that says there is almost no research whatsoever.) Its epistemic status seems frozen in the pre-Covid past: promising experimental treatment with no significant evidence confirming or disconfirming it, and no largescale commercialization and deployment.

This seems quite an inefficient result, if my understanding is correct. What has gone wrong?
Socially-distanced outdoor Petrov Day ceremonial manual
by jchan — 2020-08-24T14:11:49.804Z

For Petrov Day 2020, we of the Austin Less Wrong community decided that we would modify the usual ceremonial procedure to accommodate COVID-19 social distancing guidelines - the ceremony will be held outdoors, with everyone wearing masks and keeping 6 feet of distance from others. The modified Petrov Day manual can be read here:

Organizer guide (PDF)

Double-sided booklet (PDF)

Mobile-friendly (PDF)

We are making this modified manual available in case other Less Wrong meetup groups are considering holding an outdoor Petrov Day ceremony and would like to use it. If so, please note that obtaining all the materials and finding a suitable location may take a few weeks, so make sure to begin your preparations well in advance.

On that note, if you are in Austin or within driving distance, please save the date: Saturday, September 26, 2020, 6:30pm, at an outdoor location in Austin, TX. Petrov Day is our premiere event of the year and we would be delighted to see you there. I will post more information here on Less Wrong as the date approaches; or, message me on LW or email (jchan107@protonmail.com) and I'll keep you informed of any updates. In the meantime, you can sign up for our mailing list and join us for our regular virtual and in-person meetups, which are variously announced there. Our virtual meetups have been going strong since the lockdown, with activities and social meetups each week.
When can Fiction Change the World?
by Timothy Underwood — 2020-08-24T13:47:31.359Z

I suspect that a nontrivial percentage of the people reading this became involved with the community because of Harry Potter and the Methods of Rationality.

So to the extent that those who were drawn to join the community because of that source are making the world a better place, we have at least one clear example of a novel having an important impact

I’ve made a living through self publishing novels for the last five years (specifically Pride and Prejudice variations, that is Jane Austen fan fiction). Recently inspired by conversations at EA Virtual and worries made more emotionally salient by GTP-3 examples, I decided that I wanted to put part of my professional time towards writing novels that might have a positive impact on conversations around AI.

As part of this I did some thinking about when fiction seemed to exert an influence on public policy, and then I looked for academic research on the subject, and I think there are people in the community who will find this write up about the subject interesting and useful.

Theoretical Model

I identified four common mechanisms that seemed to be involved when fiction had a large impact on opinions. This is not an exhaustive list, and there is some overlap and fuzziness around the boundary of each concept.

Radicalizing the already convinced:

A classic example is Uncle Tom’s Cabin, a novel about a slave unjustly suffering written in the 1850s that was credited with helping to spark the Civil War. Uncle Tom’s Cabin did not introduce anyone to the idea that slavery was bad, or convince anyone who thought that slavery was a fine peculiar Southern institution that it was actually evil. However it seems to have radicalized Northern attitudes towards slavery, and it was part of the moment when enough one issue voters on slavery existed that the party system broke down and allowed the new abolitionist Republican party to win congress and the presidency in 1860.

Research has been done via surveys to find out if readers of popular novels about climate change have changed their views about climate change relative to similar readers who did not read any of them. Concerned readers become alarmed by climate change after reading, and those who were aware but not concerned become concerned. However, readers who think that climate change is a hoax usually don’t read ‘Cli-fi’ and when they do r...
Do you vote based on what you think total karma should be?
by Rafael Harth — 2020-08-24T13:37:52.987Z

I recently strong-downvoted a post that I would have weak-upvoted if it had been at a lower karma. In general, I usually vote primarily based on what I think the total karma should be. I'm curious whether other people do similar things.

This is both a question and a poll. The poll is in the comments; it works via upvotes but there is a karma balance comment. (Note that one can recover the non-weighted results (i.e., number of votes) by hovering one's mouse over the current score.) This is about votes on LessWrong only.

I'm also wondering whether this behavior is, in some sense, anti-virtuous. If everyone votes based on what they think the total karma should be, then a post's karma reflects [a weighted average of opinions on what the post's total karma should be] rather than [a weighted average of opinions on the post]. This feels worse, though I'm not entirely sure that it is.

Correction: as jimmy points out, voting independently of current karma does not give you a weighted average of opinions on the post because there are only a limited number of ways you can vote.

Meta: There's been some speculation about this (maybe read after voting), but nothing conclusive.

Current non-weighted results (08/28 07:05 EDT) (TK is 'target karma'.)
Is 'satificing' optimisation?
by Riccardo Volpato — 2020-08-24T11:51:40.246Z

It seems to me that the behavioural science research around 'the trade-off between maximising and satisficing' in terms of well-being relates with human beings being 'optimisers' only sometimes. Also, it seems to suggests that there can be alternative and better approaches to optimisation? Is this true? Or is 'satisficing' just a hidden multi-objective optimisation that factors in time and effort costs?
See what trigger my question: A decision-maker’s dilemma: search for the best option or settle for ‘good enough’?
Learning human preferences: black-box, white-box, and structured white-box access
by Stuart_Armstrong — 2020-08-24T11:42:34.734Z

This post is inspired by system identification; however, I'm not an expert in that domain, so any corrections or inspirations on that front are welcome.

I want to thank Rebecca Gorman for her idea on using system identification, and her conversations developing the concept.

Knowing an agent

This is an agent:

Fig. 1

We want to know about its internal mechanisms, its software. But there are several things we could mean by that.

Black-box

First of all, we might be interested in knowing its input-output behaviour. I've called this its policy in previous posts; a full map that will allow us to predict its output in any circumstances:

Fig. 2

I'll call this black-box knowledge of the agent's internals.

White-box

We might be interested in knowing more about what's actually going on in the agent's algorithm, not just the outputs. I'll call this white-box knowledge; we would be interested in something like this (along with a detailed understanding of the internals of the various modules):

Fig. 3

Structured white-box

And, finally, we might we interested in knowing what the internal modules actually do, or actually mean. This is the semantics of the algorithm, resulting in something like this:

Fig. 4

The "beliefs", "preferences", and "action selectors" are tags that explain what these modules are doing. The tags are part of the structure of the algorithm, which includes the arrows and setup.

If we know those, I'd call it structured white-box knowledge.

Levels of access

We can have different levels of access to the agent. For example, we might be able to run it inside any environment, but not pry it open; hence we know its full input-output behaviour. This would give us (full) black-box access to the agent (partial black box access would be knowing some of its behaviour, but not in all situations).

Or we might be able to follow its internal structure. This gives us white-box access to the agent. Hence we know its algorithm.

Or, finally, we might have a full tagged and structured diagram of the whole agent. This gives us structured white-box access to the agent (the term is my own).

Things can more complicated, of course. We could have only access to parts of the agent/structure/tags. Or we could have a mix of different types of access - grey-box seems to be the term for something between black-box and white-box.

Humans seem to have a mixture of black-box and structured white-box acce...
"Good judgement" and its components
by owencb — 2020-08-24T10:32:50.445Z

Epistemic status: Sharing a personal ontology I've found useful. I expect most claims to be fairly uncontroversial, but maybe the perspective will be interesting/clarifying for some readers. (Cross-posted, except for this paragraph, from the EA forum.)

Good judgement is about mental processes which tend to lead to good decisions. (I think good decision-making is centrally important for longtermist EA, for reasons I won't get into here.) Judgement has two major ingredients: understanding of the world, and heuristics.

Understanding of the world helps you make better predictions about how things are in the world now, what trajectories they are on (so how they will be at future points), and how different actions might have different effects on that. This is important for helping you explicitly think things through. There are a number of sub-skills, like model-building, having calibrated estimates, and just knowing relevant facts. Sometimes understanding is held in terms of implicit predictions (perhaps based on experience). How good someone's understanding of the world is can vary a lot by domain, but some of the sub-skills are transferrable across domains.

You can improve your understanding of the world by learning foundational facts about important domains, and by practicing skills like model-building and forecasting. You can also improve understanding of a domain by importing models from other people, although you may face challenges of being uncertain how much to trust their models. (One way that models can be useful without requiring any trust is giving you clues about where to look in building up your own models.)

Heuristics are rules of thumb that you apply to decisions. They are usually held implicitly rather than in a fully explicit form. They make statements about what properties of decisions are good, without trying to provide a full causal model for why that type of decision is good. Some heuristics are fairly general (e.g. "avoid doing sketchy things"), and some apply to specific domains (e.g. "when hiring programmers, put a lot of weight on the coding tests").

You can improve your heuristics by paying attention to your experience of what worked well or poorly for you. Experience might cause you to generate new candidate heuristics (explicitly or implicitly) and hold them as hypotheses to be tested further. They can als...
Split-a-Dollar Game
by Martin Sustrik — 2020-08-24T04:54:22.313Z

When researching the topic of distributing government seats in Switzerland. I've chanced on the so-called split-the-dollar game, which is kind of simplified, game-theoretical model of the process. Given that despite having some basic knowledge of the game theory I have never heard of the game, it may be worth making a short blog post about it.

In the game, three players are asked to vote on splitting a dollar. The game is iterative, it has multiple rounds. At the beginning, a third of a dollar may be assigned to each player. The players, unanimously, vote for the arrangement.

However, the arrangement is not stable. It's a game about forming coalitions. Two players can conspire and vote in a arrangement where they split the dollar fairly among themselves and give nothing to the third player.

But even then the arrangement isn't stable. Now the third player can approach one of the other two coallition parties and propose a better deal: Instead of 50 cents they would get 75 cents and the proposing party would content itself with just 25 cents. Their current coalition partner would get nothing. The party is incentivized to accept not only because of sheer greed, but also because it is aware that if they rejected the proposal, the proposing party can make the same offer to their coalition partner and thus lock them out of the coalition.

But even now, with two players sharing the dollar in 75:25 ratio, the arrangement is not stable. The senior partner can blackmail the junior partner by threatening to make a coalition with the opposition party. The junior partner, in their turn, can do exactly the same thing.

As can be seen, any arrangement is, in principle, unstable. The negotiation game can go on forever without reaching a stable state.

We can see the game in action in the case of Switzerland. In 1953, despite FDP (liberals), CVP (christian democrats) and SP (social democrats) getting very much the same amount of votes, the seats in government were distributed among them in 4:2:0 ratio. (There was also one seat going to SVP, which matched their election result and is not relevant to this story.)

The interesting aspect here is that unlike in other countries with the same problem, Switzerland has been able - after more than 100 years of playing the game - to reach a stable state. They did so by adopting a cultural norm, so called "Magic Formula", which splits the seats in...
Investment is a useful societal mechanism for getting new things made. Stock trading shares some functionality with investment, but seems very very inefficient, at that?
by mako yass — 2020-08-24T01:18:19.808Z

Jack Bogle, the creator of the first index fund, says

The stock market has nothing—n-o-t-h-i-n-g—to do with the allocation of capital. All it means is that if you’re buying General Motors stock, say, someone else is selling it to you. Capital isn’t allocated—the ownership just changes. I may be an investor, you may be a speculator. But no capital goes anywhere. This is basically a closed system. You have new IPOs and whatnot, but they’re very small compared to this vast thing we call a market

My response to this has always been... if that's true, what is the point in all of this? It's a mechanism that predicts the success of companies, but plays only a very small role in investment? Could we get that money to do something better, then?