trentmkelly/LessWrong-43k huggingface dataset:
A Map of the Lesswrong-O-Sphere of Interest
by Elo — 2019-05-04T04:19:25.345Z

It's clear to me that we on lesswrong operate on a pile of different interest areas. In my head I am doing a sorting method that divides topics. There's not just topics I (personally) like and topics I don't like but also many sub-topics within those. Here's a list of the domain areas as I see them:

Lesswrong:

Artificial intelligence

Programming (not just AI)
Machine learning
Ethics
Math

Decision theory

Philosophy

Effective Altruism (mostly on the EA forum)
Ethics
Scientific method and crisis
Decision problems

Human Psychology

Therapy and related personal work
Biases and related thinking
Group thinking and how to work with people
Personal development (other than therapy)
How humans think (consciousness - overlapping with Post-rationality area)

Guides for doing things, like problem solving
Thinking exercises to try
reports/retrospective reflection on some event that has happened

Post-rationality

Buddhism
Mysticism
Woo
map/territory blurring

Human health and medical

Wellness
Longevity
Diet
Cryonics (uncommon)

Society

Economics

forecasting/prediction

History
Culture

The state of culture in other places virtually or physically

Book reviews

Meta: how lesswrong works or could work.
Original sequences and relevant theory or ongoing development (as opposed to new sequences)
Uncommon areas

Physics
Politics (discouraged)
Big/small (new) ideas and theories

--------------

We’re not doing one thing. We’re not even doing three things, we are interested in a whole lot more topics. I could reformat the list to include more questions, along the lines of, “what are we trying to answer when we ask a history question?” but that’s not it. My point here is that we are investigating a diverse bunch of topics, and there’s a structure to our pattern. Even if it might look like a mess.

For me, I know AI is not my domain. EA, decision theory are not my domain. My domain is Humans and how they work, how they think, and (personally) how I relate to the world around me.

Personally, in regard to archipelago of forums or dividing up topics, I know I don’t comment on AI information, but I do comment on human, PR and a few other specific topics.

This post is to follow

this

comment because I wanted to expand on my understanding of where lesswrong has moved since the original sequence materials.

Maybe you have a better classification of the many diverse topics we cover, but as a start, here’s what’s floating in...
[Meta] Hiding negative karma notifications by default
by habryka — 2019-05-04T02:36:43.919Z

A few weeks ago we launched karma notifications, and I've been overall pretty happy with the feature. However, we and many other users noticed that while the current system treats upvotes and downvotes as symmetric, humans tend to definitely not perceive those two as symmetric and so the lived experience of having a single day with a slightly negative karma total tends to outweigh many days of strongly positive karma scores.

I also noticed that this experience has backpropagated into me being much more hesitant to downvote users or even to remove my upvotes since I know they will perceive this as a downvote and if they are not a frequent user might feel quite punished as a response.

As a consequence, we decided to hide negative karma notifications for users by default, though you can change that in your settings. My current model of users suggests it is very unlikely for users to not notice if they are getting significantly downvoted, so I am not very worried about that information getting lost, and I expect it will overall make people's site-experience better (and their beliefs about the value and reception of their content more, not less, accurate)

For some more context, see this discussion on my short-form feed.
Totalitarian ethical systems
by Benquo — 2019-05-03T19:35:28.800Z
Link: http://benjaminrosshoffman.com/totalitarian-ethical-systems/

(Excerpt of another conversation with my friend Mack.)

Mack: Do you consider yourself an Effective Altruist (capital letters, aligned with at least some of the cause areas of the current movement, participating, etc)?

Ben: I consider myself strongly aligned with the things Effective Altruism says it's trying to do, but don't consider the movement and its methods a good way to achieve those ends, so I don't feel comfortable identifying as an EA anymore.

Consider the position of a communist who was never a Leninist, during the Brezhnev regime.

Mack: I am currently Quite Confused about suffering. Possibly my confusions have been addressed by EA or people who are also strongly aligned with the stated goals of EA and I just need to read more. I want people to thrive and this feels important, but I am pretty certain that "suffering" as I think the term is colloquially used is a really hard thing to evaluate, so "end suffering" might be a dead end as a goal

Ben: I think the frame in which it's important to evaluate global states using simple metrics is kind of sketchy and leads to people mistakenly thinking that they don't know what's good locally. You have a somewhat illegible but probably coherent sense that capacity and thriving are important, and that suffering matters in the context of the whole minds experiencing the suffering, not atomically

There's not actually a central decisionmaker responsible for all the actions, who has to pick a metric to add up all the goods and bads to decide which actions to prioritize. There are a lot of different decisionmakers with different capacities, who can evaluate or generate specific plans to e.g. alleviate specific kinds of suffering, and counting the number of minds affected and weighting by impact is one thing you might do to better fulfill your values. And one meta-intervention might be centralizing or decentralizing decisions.

Since you wouldn't need to do this if the info were already processed, the best you can do really is try to see (a) how different levels of centralization have worked out in terms of benefiting from economies of scale vs costs due to value-distortion in the past, and (b) whether there's a particular class of problem you care about that requires one or the other.

So, for instance, you might notice that factory farming creates a lot of poin...
Authoritarian Empiricism
by Benquo — 2019-05-03T19:34:18.549Z
Link: http://benjaminrosshoffman.com/authoritarian-empiricism/

(Excerpts from a conversation with my friend Mack, very slightly edited for clarity and flow, including getting rid of most of the metaconversation.)

Ben: Just spent 2 full days offline for the holiday - feeling good about it, I needed it.

Mack: Good!

Ben: Also figured out some stuff about acculturation I got and had to unlearn, that was helpful

Mack: I'm interested if you feel like elaborating

Ben: OK, so, here's the deal.

I noticed over the first couple days of Passover that the men in the pseudo-community I grew up in seem to think there's a personal moral obligation to honor contracts, pretty much regardless of the coercion involved. The women seem to get that this increases the amount of violence in the world by quite a lot relative to optimal play, but they don't really tell the men. This seems related somehow to a thing where the men feel anxious about the prospect of modeling people as autonomous subjects - political creatures - instead of just objectifying them, but when they slap down attempts to do that, they pretend they're insisting on rigor and empiricism.

Which I'd wrongly internalized, as a kid, as good-faith critiques of my epistemics.

Story 1:

I was talking with my father about Adorno, the Enlightenment, and anti-Semitism, and the conversation was doing a reasonable-seeming thing, UNTIL he brought up the issue of high-fertility ethnic minorities with distinct political loyalties in democracies. So, naturally, first I explored the specific thing he brought up, which was that this strategy exploits a real security flaw in the democratic setup, and (since this came up in the context of Israel) that hypocritical ethnic majorities willing to occasionally violate their "standards" do a lot better patching the security flaw, than do ethnic majorities who insist on ACTUALLY having structurally neutral liberalism that takes care of and empowers everyone.

But, then, since we'd been talking about anti-Semitism, I had to point out that there's a structurally similar thing going on with Jews and credit-allocation systems in early financialized states like pre- and interwar Germany. If there had been actual coordination and an actual agenda, it would have been trivial to take over the state. (There wasn't and there wasn't, it's a trope in pre-WWII-era Jewish humor that the anti-Semitic newspapers kin...
The State of Affairs
by Jacob Falkovich — 2019-05-03T16:18:31.706Z

Cross posted from Putanumonit.

The Modern Model

Marriage is a lot like physics – it runs into trouble when a third body is introduced.

Both fields also experienced a revolution in the last century or two, with old ideas being replaced by new conceptions of how things work. For marriage, what was once a mostly economic and societal arrangement is now a romantic one, and is subject (at least in the West) to the “Modern Model of Marriage”.

The modern model goes something like this: a couple meets, feels strong chemistry, discovers shared interests, and starts having sex. The next step is emotional intimacy, as the partners are expected to become each other’s best friends and confidants. Next comes economic partnership: living together, making and spending money together, getting ready for parenthood. A spiritual dimension is added as each partner finds meaning and transcendence in their shared love. Finally, the wedding vows lock in the final requirement: that all of the above will now be provided exclusively within the couple, forever.

This exclusivity is meant to provide security, which has to take precedence in the trade-off against the other benefits of marriage. A union that scores a B grade on chemistry, engagement, sex, meaning, intimacy, and economics with an A+ on security is considered a great marriage, about as good as one can hope for. We accept that couples may face financial troubles, a drifting apart of interests and hobbies, decreased intimacy, and lackluster sex. But it’s an oxymoron to say: “my marriage is great, there’s just a bit of infidelity”.

And yet, as Esther Perel quips:

Infidelity has a tenacity that marriage can only envy.

The quote is from Perel’s book, The State of Affairs: Rethinking Infidelity. I bought it on audio (Perel narrates it herself with passion and a delightful Belgian accent), then immediately ordered a paperback copy for my wife, who read it in one day.

Relationship science seems to lag about 24 centuries behind physics, the modern model of marriage is quite reminiscent of Aristotelian physics. It is a neatly organized package of rules that makes sense when you see it on the page. But it is not built on solid first principles, and it certainly doesn’t hold up to a close inspection of reality.

The State of Affairs is a close inspection of the reality of marriage and adultery. For the most part it relies not on p<0.05 studies ...
Oracles, sequence predictors, and self-confirming predictions
by Stuart_Armstrong — 2019-05-03T14:09:31.702Z

My counterfactual Oracle design uses a utility function/reward function in order to train it to give the right predictions. Paul Christiano asked whether the whole utility function approach was necessary, and whether it could be replaced with a simple sequence predictor.

It turns out the answers are no, and yes. The counterfactual nature of the approach can be preserved, without needing a utility function.

Counterfactual sequence prediction

There is a boxed Oracle, O, which gets fed a series of observations o0,o1,…on−1, and must make a prediction pn about the next observation.

If the prediction was sent out into the world, then O is attempting to make pn into a self-confirming prediction, with all the problems that this could entail.

However, we make O into a counterfactual Oracle; on some occasions, the output pn is erased, and not seen by anyone. In that case, the Oracle will get a special message e (for "erasure"), that will be added on after of on−1.

Then the job of the counterfactual Oracle is, given a history hn−1=o0p1o1…pn−1on−1 (including past "e"s), to produce a prediction pn that is the best prediction for on given the history hn−1e. In terms of words, the Oracle is trying to best predict the next observation, given that erasure is assumed to happen (the definition of "best" depends on the details of the design).

If our setup is well designed and there is no information leakage, then any on has a probability that is independent of the prediction pn:

∀on,hn−1,pn,p′n:P(on∣hn−1epn)=P(on∣hn−1ep′n).

Meaning of predictions

If we ran the oracle without the assumption that e would be observed, we'd have the standard self-confirming predictor (that being the only correct predictions); with that assumption, it becomes a counterfactual predictor.

Though counterfactuals are conceptually tricky, it's actually easier to understand what a counterfactual predictor is doing than a general sequence predictor Oracle. We understand that a general sequence predictor is trying to find a self-confirming prediction - one where pn is the best estimate of on, given hn−1pn - but unless we understand the structure of the world very well, we don't have a clue where that self-confirming prediction might be.

In contrast, the counterfactual prediction is easy to interpret: "had we not seen pn, that is what on would have been"; there are no loops in the definition.

Extent of manipulati

...
Self-confirming predictions can be arbitrarily bad
by Stuart_Armstrong — 2019-05-03T11:34:47.441Z

Predicting perverse donors

There is a rich donor who is willing to donate up to £2,000,000 to your cause. They’ve already written a cheque for £1,000,000, but, before they present it to you, they ask you to predict how much they'll be donating.

The donor is slightly perverse. If you predict any amount £P, they’ll erase their cheque and write £(P-1) instead, one pound less than what your predicted.

Then if you want your prediction to be accurate, there’s only one amount you can predict: £P=£0, and you will indeed get nothing.

Suppose the donor was perverse in a more generous way, and they’d instead write £(P+1), one more than your prediction, up to their maximum. In that case, the only accurate guess is £P=£2,000,000, and you get the whole amount.

If we extend the range above £2,000,000, or below £0 (maybe the donor is also a regulator, who can fine you) then the correct predictions get ever more extreme. It also doesn’t matter if the donor subtracts or adds £1, £100, or one pence (£0.01): the only accurate predictions are at the extreme of the range.

Greek mythology is full of oracular predictions that only happened because people took steps to avoid them. So there is a big difference between “prediction P is true”, and “prediction P is true even if P is generally known”.

Continuity assumption

A prediction P is self-confirming if, once P is generally known, then P will happen (or P is the expectation of what will then happen). The previous section has self-confirming predictions, but these don’t always exist. They exist when the outcome is continuous in the prediction P (and a few technical assumptions, like the outcome taking values in a closed interval). If that assumption is violated, then there need not be any self-confirming prediction.

For example, the generous donor could give £(P+1), except if you ask for too much (more than £1,999,999), in which case you get nothing. In that case, there is no correct prediction £P (the same goes for the £(P-1) donor who will give you the maximum if you’re modest enough to ask for less than £1).

Prediction feedback loops

But the lack of self-confirming prediction is not really the big problem. The big problem is that, as you attempt to refine your prediction (maybe you encounter perverse donors regularly), where you end up at will not be determined by the background facts of the world (the donor’s default generosity) but i

...
Episode 3 of Tsuyoku Naritai! (the 'becoming stronger podcast): Nike Timers
by Bae's Theorem — 2019-05-03T02:53:07.600Z

Latest episode is up! In this episode, we learn rationality from Shia LeBeouf.

https://anchor.fm/tsuyokunaritai/episodes/Episode-3---Nike-Timers-e3trl2

http://bit.ly/2GZnB0E
Functional Decision Theory vs Causal Decision Theory: Expanding on Newcomb's Problem
by Geropy — 2019-05-02T22:15:39.128Z

I’ve finished reading through Functional Decision Theory: A New Theory of Instrumental Rationality, and in my mind, the main defense of causal-decision-theory (CDT) agents in Newcomb’s Problem is not well-addressed. As they stated in the paper, the standard defense of two-boxing is that Newcomb’s Problem explicitly punishes rational decision theories, and rewards others. The paper then refutes this by saying: “In short, Newcomb’s Problem doesn’t punish rational agents; it punishes two-boxers”.

It is true that FDT agents will “win” at Newcomb’s Problem, but what the paper doesn’t address is that it is quite easy to come up with similar situations where FDT agents are punished and CDT agents are not. A simple example of this is something I’ll call the Inverse Newcomb’s Problem since it only requires changing a few words relative to the original.

Inverse Newcomb’s Problem:

An agent finds herself standing in front of a transparent box labeled “A” that contains $1,000, and an opaque box labeled “B” that contains either $1,000,000 or $0. A reliable predictor, who has made similar predictions in the past and been correct 99% of the time, claims to have placed $1,000,000 in box B iff she predicted that the agent would two-box in the standard Newcomb’s problem. The predictor has already made her prediction and left. Box B is now empty or full. Should the agent take both boxes, or only box B, leaving the transparent box containing $1,000 behind?

The bolded part is the only difference relative to the original problem. The situation initially plays out the same, but just branches at the end:

1. The prediction is made by a reliable predictor (I’ll use the name Omega) for how the agent would respond in a standard Newcomb’s dilemma (“one-box” or “two-box”)

2. Armed with this prediction, Omega stands in front of box B and decides whether to place the $1,000,000 inside.

3. The $1,000,000 goes in the box iff:

a. The prediction was “one-box” (Standard Newcomb’s Problem)

b. The prediction was “two-box” (Inverse Newcomb’s Problem)

Because FDT agents would one-box in a standard Newcomb’s problem, their box B in the inverse problem is empty. In this inverse problem, there is no reason for FDT agents not to two-box, and they end up with $1,000. In contrast, because CDT agents would two-box in a standard Newcomb’s problem, their box B in the inverse problem has $1,000,000. They two...
What is corrigibility? / What are the right background readings on it?
by Ruby — 2019-05-02T20:43:45.303Z

Ryan Carey asks in a related question "What are some good examples of incorrigibility?" He provides the following overview:

The idea of corrigibility is roughly that an AI should be aware that it may have faults, and therefore allow and facilitate human operators to correct these faults. I'm especially interested in scenarios where the AI system controls a particular input channel that is supposed to be used to control it, such as a shutdown button, a switch used to alter its mode of operation, or another device used to control its motivation.

What's a more detailed understanding? What are the right things to read? I believe there's at least one MIRI paper, some Arbital posts. Writing to this question to center my inquiry.
Neural networks for games
by bipolo — 2019-05-02T04:28:06.234Z

Recently, I've read tutorial about neural network in a snake game.
(you can see it here - https://towardsdatascience.com/today-im-going-to-talk-about-a-small-practical-example-of-using-neural-networks-training-one-to-6b2cbd6efdb3)
But, the way it train the neural network it bit funny - in practice, it depends on knowing what the snake should do in every step, and when he didn't, train him. Not very useful(cause, you know, I could just program him to do that).

The other way, I guess, is to use log of the game, assume that the players doing the optimal steps (lets say, exept of the end of there game😉) and then train the network on this situations.

But, when I haven't log of the game, is there a way I can use neural networks for a game? And if there is, how?
Swarm AI (tool)
by Alexei — 2019-05-01T23:39:51.553Z

https://unanimous.ai/swarm/

I remember playing with this a while back, answering random questions. I guess now they've released it as a business tool for companies to run their own voting rooms.

Quick overview:

You create a room.

You invite 4-200 participants.

You ask questions.

Participants vote.

You get prediction results using their AI.

Their video and website claims they got a lot of hard predictions right. Of course, they aren't saying how many other things they guessed. So it's hard to say how magical it is, but it seems worth trying out. I'm up for joining people's room if they want to run some experiments.
Open Thread May 2019
by ryan_b — 2019-05-01T15:43:23.982Z

If it’s worth saying, but not worth its own post, you can put it here.

Also, if you are new to LessWrong and want to introduce yourself, this is the place to do it. Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are welcome. If you want to explore the community more, I recommend reading the Library, checking recent Curated posts, and seeing if there are any meetups in your area.

The Open Thread sequence is here.