trentmkelly/LessWrong-43k huggingface dataset:
Where's my magic sword?
by lsusr — 2021-09-27T07:26:28.446Z

When performing first aid, you must never leave a patient until you have passed them off to someone more qualified than yourself. This is a corollary to primum non nocere (first do no harm). Helping someone and then abandoning them can be worse than just being a bystander.

Once upon a time, when I was at the lightrail station in the International District, a young woman approached my brother and me. She was holding the hand of a blind elderly lady. The old woman did not speak English. The young woman did not speak Chinese. This was obviously a quest hook. I could almost see the giant yellow question mark floating above her head. My brother and I accepted the escort mission.

If this were a videogame, we'd get a quest marker indicating on our map where we're supposed to take the blind woman. Real life isn't so kind. My brother and I speak enough Chinese to ask "Where do you want to go?" Unfortunately, the dialect of Chinese the woman spoke was not the dialect we spoke.

My brother and I speak Mandarin. The lady spoke Cantonese. The difference between Mandarin and Cantonese isn't like the difference between British English and American English. It's more like the difference between English and Russian. You know the saying "a language is a dialect with an army"? Well, a language without an army is a dialect. Chinese so-called "dialects" are mutually unintelligable languages when spoken.

We asked where she wanted to go. She said "lusi". [Click here for an approximate pronunciation]. We were confused. "Lusi" isn't a word in Mandarin. "Lu" and "si" are words in Mandarin but that's less helpful than you might expect. "Lu" has like 50 different meanings and "si" is just as bad. To make matters worse, "lu" sounds a lot like "lü" and "si" might mean "shi", depending on one's accent. "Lusi" could mean anything from "aluminum lion" to "lawyer". (In case you've ever wondered why Chinese doesn't use phonetic writing, this is one of the reasons.)

Fortunately, there is a large Cantonese-speaking population in the International District. (This is not a coincidence. The lady was part of that Cantonese population.) We visited a tea shop to find someone who spoke Cantonese. Our translator confirmed that the blind lady did indeed want to to go to "lusi" and was confused as we were what "lusi" was. But the translator did provide an additional clue: the woman wanted to go to the pharmacy...
Updates from Leverage Research: History and Recent Progress
by LarissaRowe — 2021-09-27T05:23:30.353Z

As many LessWrong readers may have seen there was recently

this post

ostensibly aimed at creating common knowledge about Leverage Research.

While Leverage Research is no longer actively involved in the Rationality community, from the comments and upvotes it seems that there is both continued interest in Leverage Research within the Rationality community and a substantial lack of information—even about our more recent work—and context about our history. This post, along with my reply to the recent post, aims to correct some of that.

As relevant background, my name is Larissa Hesketh-Rowe. I work as the Public Engagement Specialist at Leverage Research and am posting on the organization’s behalf.

I’ve worked at Leverage Research since 2019. Before that, I worked at the Centre for Effective Altruism, from 2016 - 2019, the final year of which I was the CEO.

This means I wasn't involved with Leverage Research during the initial tumultuous phase of research which is sometimes called "Leverage 1.0," (I’m not sure I’d even heard of them until 2017) but I do know many of the people involved and I feel heartfelt gratitude to them for the work they did that makes our present work possible. I have also only been tangentially involved in the Rationality community, primarily through my involvement in Effective Altruism (EA).

About Leverage Research

Leverage is a non-profit research institute that studies and supports scientific advance. We run long-term research programs in areas related to epistemology, sociology, and psychology and apply findings from our research to support efforts by others to make progress in science and technology.

We are a four-person

team

based in the US. We are currently working remotely due to the ongoing COVID-19 pandemic, although we hope to get a new office and return to having at least some staff working in person at some point towards the end of this year, or early 2022.

You can find out more about Leverage Research via:

Our Website:

http://leverageresearch.org/

Our quarterly newsletter:

https://www.leverageresearch.org/updates

Our 2019-2020 Annual Report:

https://www.leverageresearch.org/annual-report-2019-2020

Twitter:

https://twitter.com/leverageres

Facebook*:

https://www.facebook.com/LeverageResearch/

LinkedIn*:

https://www.linkedin.com/company/leverage-research

(*we’re not that active on social media but I try to post oc...
How to measure (human) impact?
by T431 — 2021-09-27T02:32:00.908Z

I would like to more thoroughly investigate the impact I have on the world. This includes determining roughly how my interactions with other people affect their well-being; how much energy is required to sustain my habits (this would be the energy expenditure involved in the transportation, distribution, production, etc... of the goods and services I consume or utilize); and what my carbon/emissions footprint is.

Before I act on this, I would like to further reduce the likelihood that I waste my time doing something intractable or unuseful. So, as a first step, I ask:

What metrics might be useful for measuring an individual's short-term and long-term impact on civilization and the biosphere?

I have already thought about estimating my impact in terms of emissions, energy expenditures, and well-being fluctuations in others, but would like to know if this list should be expanded or pruned.

While there are more questions I could ask (e.g., how should I proceed in measuring metric x), I think the question I have asked above is a decent first step in this project.

In addition to any answers to this question, a pointer to resources or existing literature on the topic of "human impact quantification" (or an equivalent term) would be highly appreciated. I do not know if similar projects have been carried out, save for posts similar to Scott Alexander's Carbon Cost Quantified. An interesting prototype question I have thought of for my investigation is: How costly (in terms of energy expenditure or emissions) is it for me to use a single square of toilet paper?.

A tangential question to this one that focuses more on the successful execution of a project on impact quantification might be: What are some products or services that more easily allow one to quantify the impact they make by using them? Any advice (questions or other considerations) of this nature would also be useful to me.
100 Years Of Existential Risk
by jdp — 2021-09-27T00:31:53.667Z
Link: https://www.wrestlinggnon.com/extropy/2021/09/26/100-years-of-existential-risk.html

100 years ago Alfred Korzybski published The Manhood Of Humanity, the first book I'm aware of to analyze existential risks as a general category and try to diagnose their root causes. Using the primitive analytic tools available to him, Korzybski created a movement called General Semantics, which he hoped would raise the sanity waterline in a self improving process far into the future. General Semantics largely petered out a few decades after Korzybski's death in 1950, but its spirit has been recaptured in contemporary movements like LessWrong rationality, Effective Altruism, etc. In typical histories of existential risk human extinction is taken as becoming an urgent concern after the invention of the atomic bomb. This is true, but Bostrom's definition of existential risk doesn't just concern extinction, it also considers regression and stagnation:

Existential risk – One where an adverse outcome would either annihilate Earth-originating intelligent life or permanently and drastically curtail its potential.

Human extinction may have become an urgent concern with the invention of the atomic bomb, but the permanent regression and stagnation of civilization became an urgent question sooner than that. Conservatively it became urgent after the end of World War I, and World War II simply escalated it from a question of regression to extinction. In fact if we examine the hundred years that have passed since Korzybski's first investigation, what we find is a century of anxious discourse about the future of human technology and society. Each generation seems to rediscover and reiterate the same theme of exponential growth: in scientific knowledge, in firepower, in resource consumption, in population, in computer intelligence, and in the ability to finely manipulate the physical environment, as threats to humanity's continued development and existence. Even as these things bolster and uplift us they threaten to destroy us in the same stroke.

The regression and stagnation of civilization is not just a technological question, but a sociological one. This means that the reiterated question of threats from exponential growth are inseparable from ideas like The Great Stagnation, which are fundamentally about why we see exponential growth in some areas but not others. This essay will outline a progression from democratic republics ending the feudal era by inventing mass mo...
Chris Dixon's Crypto Claims are Logically Flimsy
by Liron — 2021-09-26T21:50:47.132Z
Link: https://medium.com/bloated-mvp/chris-dixons-crypto-claims-are-logically-flimsy-a26cc8906b3f

In this post, I make heavy use of the power to demolish bad arguments in the domain of crypto claims. The crypto world seems rich with areas where you can save a lot of wasted time and money by simply asking for a specific example of how a crypto project beats the best available alternative from the non-crypto world.
Does anyone have the reference to the "researchers should be spending more time on literature reviews" study?
by tb_throwaway — 2021-09-26T21:47:16.732Z

This has been driving me crazy: about a month or two ago, I started (and kept) seeing various references to a study that basically argued that in many/some fields, the incremental benefit of researching a thing X are lower than doing a literature review of all the studies on X to date. IIRC, the idea was that there is not enough systematization of knowledge going on, and consolidating evidence from various places currently has higher marginal gains than adding yet another individual study...

Or, at least, that's what I think the paper said, because that's the context I kept seeing the references to it in, but I never actually bothered to click on it. And now that I want to, I can't find it! Does anyone happen to know what I'm talking about? I may have seen references to this from:

Astral Codex Ten
José Luis Ricón Fernández de la Puente (@ArtirKel/Nintil)
Other people on Twitter

But my Googling skills have been failing me. Thanks a million!

EDIT: typo in the title
The Paradox of Expert Opinion
by Emrik — 2021-09-26T21:39:45.752Z

The best-informed opinions tend to be the most selection-biased ones.

If you want to know whether string theory is true and you're not able to evaluate the technical arguments yourself, who do you go to for advice? Well, seems obvious. Ask the experts. They're likely the most informed on the issue. Unfortunately, they've also been heavily selected for belief in the hypothesis. It's unlikely they'd bother becoming string theorists in the first place unless they believed in it.

If you want to know whether God exists, who do you ask? Philosophers of religion agree: 70% accept or lean towards theism compared to 16% of all PhilPaper Survey respondents.

If you want to know whether to take transformative AI seriously, what now?

The people who've spent the most time thinking about this are likely to be the people who take the risk seriously. This means that the most technically eloquent arguments are likely to come from the supporter side, in addition to hosting the greatest volume of persuasion. Note that this will stay true even for insane causes like homeopathy: I'm a disbeliever, but if I were forced to participate in a public debate right now, my opponent would likely sound much more technically literate on the subject.

To be clear, I'm not saying this is new. Responsible people who run surveys on AI risk are well aware that this is imperfect information, and try to control for it. But it needs to be appreciated for its generality, and it needs a name.

Sampling bias due to evidential luck is inevitable

This paradox stays true even in worlds where all experts are perfectly rational and share the same priors and values.

As long as

experts are exposed to different pieces of evidence (aka evidential luck), and
decide which field of research to enter based on something akin to Value of Information (even assuming everyone shares the same values), and
the field has higher VoI the more you accept its premises,

then the experts in that field will have been selected for how much credence they have in those premises to some extent.

But, as is obvious, experts will neither be perfectly rational nor care about the same things as you do, so the real world has lots more potential for all kinds of filters that make it tricky to evaluate expert testimony.

Adversarial Goodhart amplifying deception

There are well-known problems with the incentives experts face, especially in academia....
My Hypermind Arising Intelligence Forecasts and Reflections
by elifland — 2021-09-26T20:47:28.267Z
Link: https://www.foxy-scout.com/my-hypermind-ari/

Context

The Arising Intelligence tournament is being run on Hypermind, consisting of "questions about progress toward general artificial intelligence". There are versions of each question resolving in each year from 2022 to 2025. There are $30,000 worth of prizes spread across the 4 years, increasing from $3,000 to $12,000 between 2022 and 2025.

Selected 2025 forecasts

Note that as discussed in the reflections section, Hypermind's interface for inputting distributions is fairly inflexible. I'd like to make all these forecasts substantially wider, so mostly pay attention to the median.

US vs. China experiment size

My median is that the US's ML experiments will remain larger than China's (though not by as much; the current multiplier is -6x), while the crowd expects China's to be larger by 2025.

Most computing power from non-Chinese non-giant

I'm less confident than the crowd that the 3.5 month compute doubling time identified in AI and Compute will hold up (or close), mostly due to this insightful Metaculus comment by krtnu about how companies can't continue to scale up cash investments in experiments without hitting limits soon. Note that I wish I could add a longer right tail to this forecast, but have a bit of a lower mean. I couldn't figure out how to do this with the Hypermind interface.

MATH dataset SOTA

Similar to Jacob Steinhardt (whose group commissioned these questions), I'm skeptical of how much progress the crowd is projecting on MATH by 2025. It seems like an extremely challenging dataset. The current SOTA is 6.9%.

Massive Multitask dataset SOTA

While the difference isn't as dramatic, I'm also more pessimistic than the crowd about advancements on the Massive Multitask Language Understanding. The current SOTA is 49.8%.

Reflections

My forecasts relative to the crowd

I’m predicting slower progress than the crowd on all questions about progress speed (which are all but US vs. China), similar to Jacob Steinhardt. I think he overrates how much he should defer to the crowd here, given that the questions are open for anyone to predict on AFAICT.

What does this generally mean re: AI timelines? It’s plausible that I’m more uncertain and/or generally expect slower progress than the median EA/rationalist focused on AI risk. I'm hoping to do a more extensive project where I attempt to form my overall views on Transformative AI timelines soon.

Impact of this and future tourna

...
Transformative AI and Compute [Summary]
by lennart — 2021-09-26T11:41:44.731Z

Cross-posted here on the EA Forum.

This is the summary of the series Transformative AI and Compute - A holistic approach. You can find the sequence here and the links to the posts below:

Compute [1/4]

Forecasting Compute [2/4]

Compute Governance and Conclusions [3/4]

Compute Research Questions and Metrics [4/4]

0. Executive Summary

This series attempts to:

Introduce a simplified model of computing which serves as a foundational concept (Part 1 - Section 1).

Discuss the role of compute for AI systems (Part 1 - Section 2).

In Part 1 - Section 2.3 you can find the updated compute plot you have been coming for.

Explore the connection of compute trends and more capable AI systems over time (Part 1 - Section 3).

Discuss the compute component in forecasting efforts on transformative AI timelines (Part 2 - Section 4)

Propose ideas for better compute forecasts (Part 2 - Section 5).

Briefly outline the relevance of compute for AI Governance (Part 3 - Section 6).

Conclude this report and discuss next steps (Section 7).

Provide a list of connected research questions (Appendix A).

Present common compute metrics and discusses their caveats (Appendix B).

Provide a list of Startups in the AI Hardware domain (Appendix C).

Abstract

Modern progress in AI systems has been driven and enabled mainly by acquiring more computational resources. AI systems rely on computation-intensive training runs — they require massive amounts of compute.

Learning about the compute requirements for training existing AI systems and their capabilities allows us to get a more nuanced understanding and take appropriate action within the technical and governance domain to enable a safe development of potential transformative AI systems.

To understand the role of compute, I decided to (a) do a literature review, (b) update existing work with new data, (c) investigate the role of compute for timelines, and lastly, (d) explore concepts to enhance our analysis and forecasting efforts.

In this piece, I present a brief analysis of AI systems’ compute requirements and capabilities, explore compute’s role for transformative AI timelines, and lastly, discuss the compute governance domain.

I find that compute, next to data and algorithmic innovation, is a crucial contributor to the recent performance of AI systems. We identify a doubling time of 6.2 months for the compute requirements of the final training run...
From Manchester to Beirut: The power of education to change lives
by gareth — 2021-09-26T11:40:07.869Z
Link: https://medium.com/@gareth_fr/from-manchester-to-beirut-the-power-of-education-to-change-lives-dc26689f7610

This post is about our pilot digital literacy program in the Shatila refugee camp in Beirut. I am interested in the concepts of effective altruism and how to maximize the impact of the project, so any feedback about how to do this or other programs or resources would be welcome.

Most people can name one or two teachers that had a profound influence on their lives. For me, my secondary school physics teacher, Ray Davies, was one of them. Even before he became my physics teacher, he struck me as a curious, somewhat eccentric figure. Walking his 2 dogs every lunchtime around the school grounds, hobbling slightly due to a 10 metre fall whilst rock climbing. A fall that nearly killed him and left him with dozens of plates and screws in his legs after months of operations and convalescence. His occasional talks at school assembly made me sometimes question his mental state. “Thoughts on a world without friction” is one that comes immediately to mind. Some unkind souls quietly asked if he also hit his head in the fall.

Ray Davies — Photo courtesy MGS Archives

In his lessons we learned about Brownian motion and Van der Graaf generators. He managed to make what can sometimes be a dry and daunting subject into something that if not always fun, was at least interesting, and I passed my physics exams with flying colours. But it wasn’t those lessons that made him — for me — such an outstanding teacher. “Inventive planning” and “the risk factor” were 2 courses he taught as part of our final year extra-curricular optional lessons.

Inventive planning would probably be called life-skills for entrepreneurs nowadays. We learned about brain-storming techniques, imagining alternate realities, and how to change your life to get to where you want to be.

The risk factor was, unsurprisingly, about taking risks, because 1. If you don’t take at least a few big risks in your life you’ll probably regret it, and 2. If you don’t take risks, you’ll never get what you really want out of life. And let’s face it, who doesn’t want to live a little bit outside of their comfort zone?

I don’t think it’s an exaggeration to say that what I learned during those 2 courses changed the way I think and — like those smoke particles we saw in that Brownian motion experiment — has probably been nudging my life in a different, more interesting direction ever since.

So here I am, sitting in a plane on the way ...
Beyond fire alarms: freeing the groupstruck
by KatjaGrace — 2021-09-26T09:30:17.288Z

Crossposted from AI Impacts

[Content warning: death in fires, death in machine apocalypse]

‘No fire alarms for AGI’

Eliezer Yudkowsky wrote that ‘there’s no fire alarm for Artificial General Intelligence’, by which I think he meant: ‘there will be no future AI development that proves that artificial general intelligence (AGI) is a problem clearly enough that the world gets common knowledge (i.e. everyone knows that everyone knows, etc) that freaking out about AGI is socially acceptable instead of embarrassing.’

He calls this kind of event a ‘fire alarm’ because he posits that this is how fire alarms work: rather than alerting you to a fire, they primarily help by making it common knowledge that it has become socially acceptable to act on the potential fire.

He supports this view with a great 1968 study by Darley and Latané, in which they found that if you pipe a white plume of ‘smoke’ through a vent into a room where participants fill out surveys, a lone participant will quickly leave to report it, whereas a group of three (innocent) participants will tend to sit by in the haze for much longer1.

Here’s a video of a rerun2 of part of this experiment, if you want to see what people look like while they try to negotiate the dual dangers of fire and social awkwardness.

A salient explanation for this observation3 is that people don’t want to look fearful, and are perhaps repeatedly hit by this bias when they interpret one another’s outwardly chill demeanor as evidence that all is fine. (Darley and Latané favor a similar hypothesis, but where people just fail to interpret a stimulus as possibly dangerous if others around them are relaxed.)

So on that hypothesis, thinks Eliezer, fire alarms can cut past the inadvertent game of chicken produced by everyone’s signaling-infused judgment, and make it known to all that it really is fire-fleeing time, thus allowing face-saving safe escape.

With AI, Eliezer thinks people are essentially sitting by in the smoke, saying ‘looks fine to me’ to themselves and each other to avoid seeming panicky. And so they seem to be in need the analogue of a fire alarm, and also (at least implicitly) seem to be expecting one: assuming that if there were a real ‘fire’, the fire alarm would go off and they could respond then without shame. For instance, maybe new progress would make AI obviously an imminent risk to humanity, instead of a finicky...