trentmkelly/LessWrong-43k huggingface dataset:
The Curse of Cursory Idealism
by Noah Blaff — 2020-07-28T19:56:09.253Z

In schools, children are often taught, “it’s the thought that counts.” This phrase implies that kind intent is of paramount importance. This can be a valuable lesson for children as they often make mistakes and the objective is to develop an inherent kindness within their thoughts and actions. But this adage can carry terrible consequences for the population. Vast amounts of the younger populations now carry the torch for social justice activism. False idealism can lead to surface level policy support that lacks efficacy but seems well-intentioned.

Pragmatism is a lost art. Identifying as a pragmatist in social justice circles today might lead to doubt over one’s support for a certain cause.

Some think these decisions should be empathetic. After all, empathy has its merits and ideally, one should make decisions knowing what it feels like to be oppressed. “But on the whole,” Paul Bloom, author of “Against Empathy” writes, “it’s a poor moral guide.” It supports foolish judgments and reactionary politics. Empathetic thought usually contradicts the cold, calculated nature of critical analysis, therefore leading to irrational decisions.

The insidious thing about empathy and idealism isn’t that they are malevolent, it’s that they are basic cognitive functions – our default approach to dealing with social issues.

A recent NPR / PBS Newshour poll finds that 50% of democrats positively associate with socialism compared with 7% of republicans. The same poll finds that support for socialism declines throughout generations with only 20% of Baby Boomers positively identifying. The younger generations have spoken. They want to see more equity in society, this is a noble cause.

But how do you achieve such ambitious goals? Surely, there are policies aimed at leveling the playing field within society. The trap to avoid, however, is letting empathy usurp efficacy. Good intentions aren’t enough. If you care about a cause, the only measure of success is the impact. Milton Friedman famously remarked that, “One of the great mistakes is to judge policies and programs by their intentions rather than their results.”

It makes sense that we show kindness in our solutions, it’s a moral litmus test to ensure we’re on the right path. Intentions, however, ofte...
Does the lottery ticket hypothesis suggest the scaling hypothesis?
by Daniel Kokotajlo — 2020-07-28T19:52:51.825Z

The lottery ticket hypothesis, as I (vaguely) understand it, is that artificial neural networks tend to work in the following way: When the network is randomly initialized, there is a sub-network that is already decent at the task. Then, when training happens, that sub-network is reinforced and all other sub-networks are dampened so as to not interfere.

[EDIT: This understanding goes beyond what the original paper proved, it draws from things proved (or allegedly proved) in later papers. See thread below. EDIT EDIT: Daniel Filan has now convinced me that my understanding of the LTH as expressed above was importantly wrong, or at least importantly goes-beyond-the-evidence.]

By the scaling hypothesis I mean that in the next five years, many other architectures besides the transformer will also be shown to get substantially better as they get bigger. I'm also interested in defining it differently, as whatever Gwern is talking about.
Probability that other architectures will scale as well as Transformers?
by Daniel Kokotajlo — 2020-07-28T19:36:53.590Z

GPT-1, 2, and 3 have shown impressive scaling properties. How likely is it that, in the next five years, many other architectures will also be shown to get substantially better as they get bigger? EDIT I am open to discussion of better definitions of the scaling hypothesis. For example, maybe Gwern means something different here in which case I'm also interested in that.
How should I back up and redo, in a publicly-edited article?
by Jameson Quinn — 2020-07-28T19:07:13.046Z

I've been working on the article https://www.lesswrong.com/posts/Rh8v4ba9w5KTb6onD/ongoing-thoughts-on-proportional-voting-methods for some time now. From the beginning, I was driving towards defining a metric for proportional voting method quality that could be connected through some chain of reasoning and approximations to utilitarianism. I have now achieved such a definition, and am beginning to apply my metric to real-world voting scenarios.

And I am realizing that the connection to utilitarianism is too tenuous and abstract to ever hope to get ordinary people to even understand it exists. Even though I'm no stranger to explaining difficult and abstract concepts in simple concrete terms, this is just too much of a stretch.

I'd like to use the same building blocks to construct a different metric of "vote wastage" which is easier to explain and to grasp. I have a good idea of what that would look like, and estimate that means rewriting about 1/5 to 2/5 of what I have so far.

My question is: how, if at all, should I preserve the article as it currently stands? Is it important for somebody reading the article to understand how I came to this new idea through the old one, or should I focus on clarity and sweep any complexities of the old idea under the rug?

I imagine that at least some of the people reading this question have an obvious answer in mind: "just relentlessly rewrite, don't worry about the current ideas". That answer would be helpful for me to hear, but more helpful would be if you have any advice on specific rhetorical strategies I could use in this situation.
Rereading Atlas Shrugged
by Vaniver — 2020-07-28T18:54:45.272Z

This post will not attempt to avoid spoilers, and will be much more comprehensible if you've read the book or are familiar with its basics, but I also hope it'll be somewhat understandable if you haven't read the book at all; to aid with that I'll put summaries after all the names.

I first read Atlas Shrugged as a teenager, I think for an essay contest. I was already a libertarian from reading Free to Choose, and found Rand's moralism offputting and her characters strange. I was a 'technical' libertarian,[1] in that I was convinced that decentralization led to better decision-making and better results, and didn't see how the moral libertarians made a better case than the moral statists. And even when it came to morality, the people I saw at church were putting in significant effort to try to be better, and yet Rand's heroes didn't seem to have any sort of moral development; the good people were good, and the bad people were bad, and there wasn't any engagement with the question of how to become good. I think that was the main content of my essay, and unsurprisingly it didn't win anything.

But a friend recently mentioned that they had read it and were surprised about how much it was about rationality; I remembered some bits and said "yeah, that checks out," but when I reread it recently was surprised at just how much there was, and how topical much of it was to current events and decisions I'm facing.

A truly great book should be read in youth, again in maturity and once more in old age, as a fine building should be seen by morning light, at noon and by moonlight. ― Robertson Davies

For this post I expect to slip between "how I saw Atlas Shrugged as a youth" and "how I see it now", with the first mostly to explain by contrast.

Creators revisited

In youth, I thought Galt's Gulch (a hideout in the Rockies accessible only to the creators on strike) was ridiculous. You have people whose primary skills are being executives, and they become manual laborers, and they're better off? Why think a mining executive would be any good at digging copper himself, or an aircraft executive would be good at raising hogs?

I think I was confused instead of enlightened by having the category of "executive." James Taggart (the 'villain' railroad executive) would be denied entry to Galt's Gulch, and starve if he ended up there. The primary characteristic of the creators is that they ...
FHI Report: How Will National Security Considerations
Affect Antitrust Decisions in AI? An
Examination of Historical Precedents
by Cullen — 2020-07-28T18:34:23.537Z
Link: https://www.fhi.ox.ac.uk/wp-content/uploads/How-Will-National-Security-Considerations-Affect-Antitrust-Decisions-in-AI-Cullen-OKeefe.pdf

Part I lists and analyzes the cases I identified, then lists several conclusions I was able to draw from my case studies. Those conclusions are:

National security considerations have entered the antitrust enforcement process numerous times over the past 100 years.

It is rare for the USG to actively use antitrust enforcement to advance unrelated national security objectives.

In cases where national security and economic considerations conflict, economics has been given more weight over time.

The president plays an important role in reconciling conflicting considerations.

Part II discusses how these conclusions might apply to AI firms in the coming decades.
Delegate a Forecast
by Amandango — 2020-07-28T17:43:58.461Z
Link: https://forum.effectivealtruism.org/posts/GKnXGiobbg5PFikzJ/delegate-a-forecast

EDIT: We've stopped answering questions for now, sorry if we didn't get to your question! We're still really interested in what kinds of questions people want forecasted, feedback on how useful it is to delegate forecasts, and Elicit as a tool, so feel free to keep commenting these thoughts. We also forecasted questions on the EA Forum version of this post.

Hi everyone! We, Ought, have been working on Elicit, a tool to express beliefs in probability distributions. This is an extension of our previous work on delegating reasoning. We’re experimenting with breaking down the reasoning process in forecasting into smaller steps and building tools that support and automate these steps.

In this specific post, we’re exploring the dynamics of Q&A with distributions by offering to make a forecast for a question you want answered. Our goal is to learn:

Whether people would appreciate delegating predictions to a third party, and what types of predictions they want to delegate
Whether a distribution can more efficiently convey information (or convey different types of information) than text-based interactions
Whether conversing in distributions isolates disagreements or assumptions that may be obscured in text
How to translate the questions people care about or think about naturally into more precise distributions (and what gets lost in that translation)

We also think that making forecasts is quite fun. In that spirit, you can ask us (mainly Amanda Ngo and Eli Lifland) to forecast any continuous question that you want answered. Just make a comment on this post with a question, and we’ll make a distribution to answer it.

Some examples of questions you could ask:

When will I be able to trust a virtual personal assistant to make important decisions for me?
I live in the US. How much happier will I be if I move to Germany?
How many EA organizations will be founded in 2021?
I live in New York. When will I be able to go to the gym again?
In 2021, what percentage of my working hours will I spend on things that I would consider to be forecasting or forecasting-adjacent?

We’ll spend <=1 hour on each one, so you should expect about that much rigor and information density. If there’s context on you or the question that we won’t be able to find online, you can include it in the comment to help us out.

We’ll answer as many ques...
Asset Prices Consistently Violate Efficient Market Hypothesis
by Liron — 2020-07-28T14:21:15.220Z
Link: https://lt3000.blogspot.com/2020/07/market-inefficiency-liquidity-flywheels.html

Lyall Taylor points out two reasons why asset prices consistently violate the EMH:

(1) Liquidity flywheels make the market inefficient from a long-term value perspective, because shorter-term momentum dynamics exist for structural reasons decoupled from underlying asset values

(2) The cost of capital in various assets is determined by a coarse-grained bucketing into leaky abstractions like “listed equities”, and there aren’t many participants capable of arbitraging when the specifics of an asset defy their category bucket, due to structural forces of short time horizons and fund management principal-agent problems
Billionaire Economics
by Virgil Kurkjian — 2020-07-28T03:01:33.743Z

I have progressive friends in my circles and so I see a lot of stuff like this:

As well as somewhat more eloquent takes like this one, and a number of similar things I haven't bothered digging up.

But I notice that I'm still confused. The entry-level mistake (which some progressives do make) is thinking that having a net worth of $48 billion means that you have $48 billion sitting in a bank account somewhere. It's clear that none of them have that kind of liquidity. I've even seen, very occasionally, back and forth takes on these issues. So not everyone is making this mistake.

Ok, I'm still confused. What is going on?

Possibly estimates like this are just way off on how much money it takes to solve problems. Is $20 billion really enough to end homelessness in the US? End for how long? The NYT suggests that we already spend about $12 billion (though perhaps not very well), and that's

per year

! $300 billion does seem awfully optimistic for halting climate change. Maybe no one's personal fortune is actually enough to do any of these things.
Still, with several billion dollars, you could probably do something. Let's assume that there is at least one major, important thing you could do with a one-time investment of $20 billion (seems reasonable). Billionaires are well-informed, they probably know about whatever that thing is. There are several who could apparently afford it. Surely one of them would "defect" and burn half of their fortune on it? This suggests that either 1) billionaires really are universally evil just as progressives think, 2) no opportunity like this exists and billionaires are not, in practical terms, all that rich, or 3) billionaires can't access all of their wealth for some reason.
(Or 4) billionaires do this already and I don't know about it? IDK)
One angle on that last point is like this: If Mark Zuckerberg wanted to liquidate all $54.7 billion of his wealth to give it away to some cause (give every American... wow only $182.33, can that be right?), how quickly / easily could he do that? Would he be able to do it without any trouble? What are the barriers in his way? Does he simply not have access to that kind of liquidity?

I know that's not especially clear but I hope that y'all can help me out. Any thoughts or discussion on these points would be welcome and appreciated. Thank ...
What specific dangers arise when asking GPT-N to write an Alignment Forum post?
by Matthew Barnett — 2020-07-28T02:56:12.711Z

Last year Stuart Armstrong announced a contest to come up with the best questions to ask an Oracle AI. Wei Dai wrote,

Submission. For the counterfactual Oracle, ask the Oracle to predict the n best posts on AF during some future time period (counterfactually if we didn’t see the Oracle’s answer).

He later related his answer to Paul Christiano's posts on Human-in-the-counterfactual-loop and Elaborations on apprenticeship learning. Here I'm interested in concrete things that can be expected to go wrong in the near future if we gave GPT-N this task.

To provide a specific example, suppose we provided the prompt,

This is the first post in an Alignment Forum sequence explaining the approaches both MIRI and OpenAI staff believe are the most promising means of auditing the cognition of very complex machine learning models.

If by assumption, GPT-N is at least as good as a human expert team at generating blog posts, we could presumably expect this GPT-N to produce a very high quality post explaining how to inspect machine learning models. We would therefore have a way of to automate alignment research at a high level. But a number of important questions remain, such as,

How large would GPT-N need to be before it started producing answers comparable to a human expert team, and
Given the size of the model, what high-level incentives should we expect to guide the training of the model? In other words, what mesa optimization-like instantiations can we expect to result from training, exactly, and
Is there a clear and unambiguous danger that the model would be manipulative? If so, why?
Is the threat model more that we don't know what we don't know, or that we have a specific reason to believe the model would be manipulative in a particular direction?
The Future of Science
by Richard_Ngo — 2020-07-28T02:43:37.503Z

(Talk given at an event on Sunday 19th of July_. Richard Ngo is responsible for the talk, Jacob Lagerros and David Lambert edited the transcript. _

If you're a curated author and interested in giving a 5-min talk, which will then be transcribed and edited, sign up here_.) _

Richard Ngo: I'll be talking about the future of science. Even though this is an important topic (because science is very important) it hasn’t received the attention I think it deserves. One reason is that people tend to think, “Well, we’re going to build an AGI, and the AGI is going to do the science.” But this doesn’t really offer us much insight into what the future of science actually looks like.

It seems correct to assume that AGI is going to figure a lot of things out. I am interested in what these things are. What is the space of all the things we don’t currently understand? What knowledge is possible?

These are ambitious questions. But I’ll try to come up with a framing that I think is interesting.

One way of framing the history of science is through individuals making an observation and coming up with general principles to explain it.
So in physics, you observe how things move and how they interact with each other. In biology, you observe living organisms, and so on. I'm going to call this “descriptive science”.

More recently, however, we have developed a different type of science, which I'm going to call “generative science”. This basically involves studying the general principles behind things that don’t exist yet and still need to be built.

This is, I think, harder than descriptive science, because you don't actually have anything to study. You need to bootstrap your way into it. A good example of this is electric circuits. We can come up with fairly general principles for describing how they work. And eventually this led us to computer science, which is again very general. We have a very principled understanding of many aspects of computer science, which is a science of things that didn't exist before we started studying them.

I would also contrast this to most types of engineering such as aerospace engineering. I don't think it's principled or general enough to put it in the same class as physics or biology and so on.

So what would it look like if we took all the existing sciences and made them more generative? For example, in biology, inste...
Basic Conversational Coordination: Micro-coordination of Intention 
by Eli Tyre — 2020-07-27T22:41:53.236Z

Follow up to: Something simple to try in conversations

In this essay, I want to describe a simple, but important, abstraction for how to think about what’s happening in a conversation, and how to make use of that abstraction for helping a conversation go more smoothly. This idea applies at multiple time-scales: in this post, I’ll describe the simpler case of second-to-second interactions. In some future post, I’ll explain the broader (and more complicated) minute-to-minute case.

Let’s consider some two-person epistemic conversation, say two people attempting to converge on some topic they disagree about.

There are two roles (moves/ strategies) that one can play in a two-person epistemic conversation.

One can either be listening [/ be the listener], trying to understand their partner’s point.
Or one can be explaining [/ be the explainer], trying to convey a point to their partner.

Both participants will almost certainly do both over the course of the conversation, and each may switch very rapidly between explaining and listening, but for any discrete time-slice of the conversation, each person will be holding an intention either to convey, or to come to understand.

It’s useful to track conversations in terms of which participant is implementing which role, second to second. Thinking about conversations this way, and being conscious of what intention you are implementing, as a participant, is useful because it enables conversational coordination.

At any given time, no more than one person should be explaining, and at least one person should be trying to understand what is being said. Specifically, this means that it should never be the case that both participants are in the explaining mode at the same time.

At any given moment, one person should be explaining, and the other person should be listening and trying to understand.

This may seem like an obvious point, but conversations frequently fail to meet this standard. A common failure mode: both parties are excited about the point they have to make, and have a visceral compulsion to make that point clear to the other person. And so both people are explaining at each other. When both people are in the explaining role at the same time, communication usually fails.

(The way Double Crux has been traditionally described aggravates this issue. The concept of a Double Crux can l...
[updated] how does gpt2′s training corpus capture internet discussion?  not well
by nostalgebraist — 2020-07-27T22:30:07.909Z

[Updated to correct my earlier claim that this doesn't affect GPT-3. Apparently it does?]

I’m out sick today, but had enough energy to do some GPT-related fiddling around.

This time, I was curious what “internet discussions” tended to look like in the original training corpus. I thought this might point to a more natural way to represent tumblr threads for @nostalgebraist-autoresponder​ than my special character trick.

So, I looked around in the large shard provided as part of https://github.com/openai/gpt-2-output-dataset.

Colab notebook here, so you can interactively reproduce my findings or try similar things.

—–

The results were … revealing, but disappointing. I did find a lot of discussion threads in the data (couldn’t find many chatlogs). But

almost all of it is from phpBB-like forums (not bad per se, but weird)
it chooses a single post from each page and makes it “a text,” ignoring all the other posts, so no way for GPT2 to learn how users talk to each other :(
sometimes the post quotes another user… and in that case, you can’t see where the quote starts and the post begins
lots of hilarious formatting ugliness, like “Originally Posted by UbiEpi Go to original post Originally Posted by”
about 0.28% of the corpus (~22000 docs in full webtext) consists of these mangled forum posts
also, just as a chilling sidenote, about 0.30% of the corpus (~25200 docs in full webtext) is badly mangled pastebin dumps (all newlines removed, etc).no overlap between these and the mangled forum threads, so between them that’s ~0.58% of the corpus.
remember: the vast majority of the corpus is news and the like, so these percentages aren’t as small as they might sound

For example, from this thread it picks the one post

and renders it as

“ Pillowapnts
tho the gem doesnt specifically say that you need to crit with something linked, i thought it was just crit in general ahhh. alright i get it thxtho the gem doesnt specifically say that you need to crit with something linked, i thought it was just crit in general
That would be OP That would be OP Posted by Lordsidro
on on Quote this Post

This is apparently standard behavior for the newspaper text cleaner they used, and I could reproduce it exactly. (Its heuristics grab a single...