trentmkelly/LessWrong-43k huggingface dataset:
Possible Worlds after Prediction Take-off 
by isaduan — 2021-10-03T10:26:34.832Z

This is the final piece of a blog post series that explores how AI prediction services affect the risks of war. It is based on my 10-week summer research project at Stanford Existential Risk Institute. Here's a summary of my project and my epistemic status.

In previous posts, I have surveyed different prediction technologies and analyzed what implications follow for the risk of war. In this concluding post, I will describe three possible “world order” scenarios that seem internally consistent. They differ in how prediction capability distributes across nation-states and within nation-states.

Three Possible Worlds

Intelligence Singularity

In this world, one nation-state government gets the best AI prediction services. With a deeply integrated industrial-military complex, it achieves both intelligence and technology dominance. The two reproduce each other: control over the best information technology enables worldwide surveillance, and this intelligence access gives the government an overall best chance to keep such control - it runs the most profitable business, preempts threats and peer competitors, and finds technical and managerial solutions the fastest. The world is largely peaceful for most periods of time. World politics runs with the unipolar’s ideology and rules, more effectively than they are today. Other governments mostly accommodate the unipolar’s interests but occasionally find rooms to restrain it. Only an extremely small number of highly determined subscribers to marginalized, “extremist” ideologies are interested in violence. Most things are dealt with behind the scene, and truth tactfully disguised. The public image of “what is happening” can depart dramatically from reality.

Competitive Regional Orders

In this world, several most powerful nation-state governments are the oligarchies of the best AI prediction services as they enjoy a large amount of closed-source data and access to consumer data across different technology corporations. They intensely compete for citizens and their data as key strategic resources. They are likely to become more ideological, for ideology serves as a good strategy for retaining citizens and data. Competitive dynamics among them change little, although prediction services make international policy coordination—especially on topics of clear mutual interests, urgency, and expert consensus like climate change—easie...
What makes us happy and depressed?
by Marius Hobbhahn — 2021-10-03T06:25:03.822Z

This post is a cross-post from my personal blog.

What is this post about?

This post is the result of a long inquiry into happiness and depression and was kicked off by two other resources. The first is Luke Muehlhauser's 2011 post How to Be Happy which convinced me that there are some aspects of happiness that you can improve. The second is the 80k podcast episode with Howie which convinced me that there are many aspects of your mental health that you have absolutely no control over. To integrate these two points into one overarching model, I tried to understand happiness and depression better and answer two simple questions.

How much control do you have over your happiness and depression?
What can you do to improve them or prevent worst-case scenarios?

DISCLAIMER: I'm not a trained professional in medicine, psychiatry, or anything alike. After listening to personal accounts of depression, I'm pretty convinced that I have never been depressed myself and that my worst periods were mostly prolonged sadness mixed with unhealthy expectation management. Thus, this post should be seen as the result of ca. 100 hours of reading, listening to personal stories and asking people who have more expertise rather than an expert review. I’m certain, I got some things wrong and missed important context, etc. If you notice such an omission, please contact me.

There are two resources I can really really recommend to learn more about depression. Firstly, Scott Alexander's website (and his blog) and secondly the Our World in Data article on Depression by @saloni.

Summary of my results

Happiness and depression are very multifactorial. Some of the causes are easy, some are hard and some are impossible to control.

Ultimately, I find it easiest to think about the amount of control you have with an analogy. Imagine you ride on a canoe. You have control over your navigation and your stroke and can improve them easily through practice and training. You also have some control over the quality of the boat depending on your building skills and wealth. In the end, though, some people have a worse or better boat than others or face much rougher waters. These are the things you have no control over. But you can still mitigate their impact - even if you were given a bad boat, better handling reduces your chances of falling over.

For the rest of the article, I will categorize the level of control into t...
Engaging with hypotheticals as they are defined
by damiensnyder — 2021-10-03T05:22:17.232Z

Some arguments against with total utilitarianism

Utility monsters

"Utility monster" is a term coined by Robert Nozick as part of a critique of total utilitarianism. He explains the term as such:

"Utilitarian theory is embarrassed by the possibility of utility monsters who get enormously greater sums of utility from any sacrifice of others than these others lose ... the theory seems to require that we all be sacrificed in the monster's maw, in order to increase total utility."
—Robert Nozick, as quoted in the Wikipedia article on utility monsters

Some utilitarian frameworks, like average utilitarianism and maximin, are designed to prevent utility monsters from dominating the equation. It's an unappealing thought to consider that the well-being of billions of people could be sacrificed to appease a single being. Yet seemingly total utilitarianism promotes that exact outcome.

The "repugnant conclusion"

Another charge leveled against total utilitarianism is the so-called repugnant conclusion. The repugnant conclusion argues that the "optimal" state of a population (under total utilitarianism) will contain extraordinarily vast numbers of people living barely positive lives, rather than a large number of people living highly positive lives.[1] This as well is unappealing: I would much rather live one of ten billion joyous lives than one of ten trillion barely positive lives.

Imagining these arguments at human scale

Both of these criticisms do not demonstrate failures of total utilitarianism, but rather failures of the critic to fully comprehend their hypotheticals.

Utility monsters

Consider again the quote on utility monsters:

the theory seems to require that we all be sacrificed in the monster's maw, in order to increase total utility.

This sounds pretty bad. But if it were written by a pig raised for human consumption, it would truthfully describe a situation most humans seem fine with.

If you are not one of the humans that endorses animal slaughter, then you might still find a hypothetical where this makes sense. For example, if hundreds of wealthy people forgo a $60 video game purchase, then give those $60 to a poor individual to pay for basic necessities, that would make a lot of sense. If $12,000 allows a single person to afford much better food, shelter, and medical care, that is probably more valuable than if it allows 200 people to buy a new video game...
2021 Darwin Game - Contestants
by lsusr — 2021-10-03T05:09:25.987Z

123 players submitted a total of 556 species. Most players submitted just 1 species. One player submitted 11. (I disqualified his[1] 11ᵗʰ submission.) 555 species qualified for entry. I expected more players would submit 10 species compared to 8 or 9 in order to max out their submissions but that didn't happen.

The most popular name was Krill, which three separate people submitted. This does not count Yellow Krill, Green Krill, Red Krill, Nano-krill, Fornicacious Krill, Orange-Krill and Orange Krill.

Other duplicate names included Armadillo, Bear, Desert Tortoise, Forest Tribble, Kraken, Trash Panda and Flesh-Eating Clam.

A couple species were placed in the Tundra/Desert without the necessary temperature adaptations. I modified them so they wouldn't instantly die. Other people gave cold/heat adaptations to water-breathing species, which are useless. I left those unaltered.

Speaking of useless adaptations, Weapons + Armor is only useful up to 10. If your Weapons + Armor exceeds 10 the extra armor is useless. 40 submissions (7%) had Weapons + Armor more than 10. I think the idea here was to create big exciting powerful monsters. This is the realm of Dragons, Forest Dragons, Basilisks, Sandworms, White-Whales, Tundrus Rex, Humans, All-eating Leviathans and the so-called "Ultimate Lifeform". I expect[2] none of them will survive to the end.

I tried to render a food web but it didn't work because there were 54,867 connections. I let graphviz run for ten hours before putting it out of its misery.

Stats

One sixth of the entries were venomous. One third had antivenom.

Unlike their number of entries, many players did choose to max out weapons, armor and speed.

Spawning Locations

It makes sense that the ocean is popular because of how much food it supports. I was surprised by how popular the human garbage dump was. The Desert and Tundra were the least popular, presumably due to the temperature hazards.

Foraging

The most common foraging adaptations were algae, seeds and carrion.

Algae is on top because there is the most algae.

Seeds were popular because they offers lots of nutrition compared to the digestive adaptation.

Carrion it is most nutritionally dense option.

Forage or Hunt?

Half of all organisms were pure foragers. Of the rest, the majority had a foraging adaptation. Only a tiny minority were specialized hunters.

Social Media

I promised link to the winners' socia...
Occam's Razor and the Universal Prior
by Peter Chatain — 2021-10-03T03:23:14.581Z

It can scarcely be denied that the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience. - Albert Einstein (Nature 2018).

In this post, I'll walk you through how you could have created an algorithm for assigning prior probabilities to all hypotheses called the Universal Prior. It's really a universal prior, since there are many. The formalization that uses this prior to learn about the world is called Solomonoff induction, which Alex and Luke wrote a great post giving an intuitive introduction here. Marcus Hutter took Solomonoff induction a few steps forward and defined the super-intelligent agent

AIXI

in his textbook Universal Artificial Intelligence (2008). AIXI, like solomonoff induction, is impossible to compute. So, Hutter defined AIXI-tl, a time bounded AIXI that outperforms, in a loose mathematical sense, all other possible algorithms. While there are philosophical concerns that the prior is malign1, it’s the closest we’ve come to understanding a general and formal definition of intelligence. The hope is that a fundamental understanding of intelligence will help guide AI safety development, even if it’s not the solution to safe artificial general intelligence (it isn’t)2.

Structure: Part 1 aims to build up intuition by examining what a universal prior would look like in english. Part 2 translates this intuition to the actual universal prior. The last section connects the universal prior to AIXI and AI safety.

TLDR: Instead of defining a uniform prior over

observation space

, assume all Turing machines in

hypothesis space

are generated bit by bit from a uniform prior over 0 and 1 (fair coin flips). Then, each event in observation space receives the probability of all Turing machines that predict it. Before any observations, all Turing machines are possibly the correct one underlying the world. As observations come in, we can learn by ruling out Turing machines that disagree with reality.

Introduction:

How can you best learn a concept? In math, I would often struggle the most when I tired to simply memorize and hammer out the details. I've found much more success by first building intuition with examples, or when examples won’t do, analogies of the concept. For example, unless you have seen it before, it's hard t...
[linkpost] Vitalik Buterin on Nathan Schneider on the limits of cryptoeconomics

by mike_hawke — 2021-10-02T19:11:06.108Z
Link: https://vitalik.ca/general/2021/09/26/limits.html

Kinda long, but I found it to be worthwhile. I expect that I'll end up reading this one multiple times.

Selected quotations:

When there's only two people, more coordination can only be good. But once there's three people, the wrong kind of coordination can be harmful, and techniques to prevent harmful coordination (including decentralization itself) can become very valuable. And it's this management of coordination that is the essence of "politics".

"Finance" is a category of patterns that emerge when systems do not attempt to prevent collusion. When a system does not prevent collusion, it cannot treat different individuals differently, or even different numbers of individuals differently: whenever a "position" to exert influence exists, the owner of that position can just sell it to the highest bidder.
From The Great Man to The Great Founder Theory: Bezos, Musk, and Systrom
by hivemindsingularity — 2021-10-02T17:07:22.919Z

This particular blog is motivated by my observation of the tech journalism-sphere. I cannot help but notice the deficiency of analytical rigour in explaining the 'cult of founder' in Silicon Valley.

On the cult of founders and theories of history; this is a long-form blog.
Prosocial Capitalism
by scafaria — 2021-10-02T17:06:27.609Z
Link: https://prosocialcapitalism.com/

This is my first LessWrong post. I read Inadequate Equilibria by Eliezer Yudkowsky and was inspired to research and articulate additional solutions to get our civilization "un-stuck". I would love feedback from the community on the essay I've posted to www.prosocialcapitalism.com. I can re-post it here if preferable for guidelines, just let me know. Thank you!

Prosocial Capitalism: How positive sum networks will out-compete communism and antisocial capitalism, reshape the Internet, preserve liberty, and stave off dystopia
The underappreciated value of original thinking below the frontier
by Emrik — 2021-10-02T16:03:14.300Z

When is it more valuable to do original thinking? Probably the most common assumption/implied advice here is that it's pointless to try to come up with anything original until you've consumed the entire field and can work on the frontier of knowledge. I meet this assumption sometimes when I try to do original thinking (I have no credentials), and I see it in other shy but otherwise really bright community members when they decide to hold back from thinking for themselves.

This is terrible advice.

First, it's a myth that the global frontier of knowledge is the only frontier that matters. Different communities have different frontiers, and if you want to do valuable knowledge-work you should aim to push the frontiers that matter. This can be done either by importing knowledge or by thinking originally, and the latter strategy is sometimes more cost-effective (especially in areas with immense research debt). Model the value of your knowledge-work based on who you think it'll reach.

Second, the value of original thinking increases based on how unusual the values you're optimising for are. Imported knowledge, when it exists, is often optimised for something else. A great example here is Holden Karnofsky's History of empowerment and wellbeing. There are thousands of professional historians with more field knowledge than Holden, but because Holden is optimising for something different, he's still able to produce valuable original knowledge-work for our community.

This is broadly true of academia where the name of the game is to accumulate prestige. Prestige often involves optimising for being prolific, how difficult the work appears, number of citations, working on fashionable topics, using fashionable methods/concepts, and more pointless things. These incentives are stuck in an inadequate equilibria enforced by journals: you get prestige by publishing to journals and by citing research that's been published there. Unilaterally trying to do something different within the system is hard, but you might be able to beat it from the outside.

Third, before you've reached the frontier of knowledge you have in mind, it can be really usefwl to spend some time trying to develop your own unique perspectives/tools/angles of attack. This becomes a lot harder once you've already consumed other peoples' perspectives/tools in your haste to get to the frontier, due to the Einstellung ef...
Forecasting Compute - Transformative AI and Compute [2/4]
by lennart — 2021-10-02T15:54:54.231Z

Cross-posted here on the EA Forum.

Transformative AI and Compute - A holistic approach - Part 2 out of 4

This is part two of the series Transformative AI and Compute - A holistic approach. You can find the sequence here and the summary here.

This work was conducted as part of Stanford’s Existential Risks Initiative (SERI) at the Center for International Security and Cooperation, Stanford University. Mentored by Ashwin Acharya (Center for Security and Emerging Technology (CSET)) and Michael Andregg (Fathom Radiant).

This post attempts to:

Discuss the compute component in forecasting efforts on transformative AI timelines (Section 4)

Propose ideas for better compute forecasts (Section 5).

Epistemic Status

This article is Exploratory to My Best Guess. I've spent roughly 300 hours researching this piece and writing it up. I am not claiming completeness for any enumerations. Most lists are the result of things I learned on the way and then tried to categorize.

I have a background in Electrical Engineering with an emphasis on Computer Engineering and have done research in the field of ML optimizations for resource-constrained devices — working on the intersection of ML deployments and hardware optimization. I am more confident in my view on hardware engineering than in the macro interpretation of those trends for AI progress and timelines.

This piece was a research trial to test my prioritization, interest and fit for this topic. Instead of focusing on a single narrow question, this paper and research trial turned out to be more broad — therefore a holistic approach. In the future, I’m planning to work more focused on a narrow relevant research questions within this domain. Please reach out.

Views and mistakes are solely my own.

Previous Post: What is Compute?

You can find the previous post "What is Compute? [1/4]" here.

4. Forecasting Compute

Highlights

For transformative AI timeline models with compute milestones, we are interested in how much effective compute we have available at year Y.

We can break this down into (1) compute costs, (2) compute spending, and (3) algorithmic progress.

Hardware progress: For forecasting hardware progress, no single model can explain the improvements of the last years. Instead, a mix of Moore’s Law, chip architectures, and hardware paradigms are applicable models and categories to think about progress.

Performance improvements

...
Why We Should Always Distrust Our Certainties
by Ted Reynolds — 2021-10-02T06:05:44.075Z

While just thinking around, which I often do when nothing else offers, I recently came across a line of thought which presented an insight into a possible cause of much of the misunderstanding which arises within and between human minds. Since I have not seen this concept mentioned before, and since others may also find it worth considering, I here put it out for what it may be worth.
Both in our communications with others and in our internal monologues with ourselves, we often employ the words or concepts of “certainty”. I here wish to examine the meaning that “certainty” holds for others and for ourselves, as well as the effects that it may cause for the accuracy of our mental conclusions. I will come to the conclusion that in some rather clearly distinguishable conditions, the use of the concept “certainty” serves to obscure, rather than to clarify, the objects of thought. What it conveys at these times is the following:
“I am firmly convinced of the truth of this. I have no doubt about it. I have examined it in every way and cannot be mistaken. I certainly do not need to examine it again.”
This feeling will reappear instantaneously whenever this concept enters one’s thoughts. The result will be that this concept will not be reexamined in any way. None of any possible observations or reasonings which may have once, perhaps years ago, have supported the belief, will be brought before the mind. Instead a very strong feeling of being correct, being justified, not being mistaken will both reinforce the belief, and provide a very pleasant feeling that the mind will quite willingly return to on future occasions. In other words, when the “certainty” feeling is once activated, it becomes more and more unlikely that it ever will be discarded. If it was once in any way an error, that error will continue to remain and strengthen.
A less insistent form of what I am pointing out is, of course, already familiar to us all in the form “we tend to remember what attracts us and to forget what we do not like to think about,” The less dogmatic concepts “I think that,” “It appears to me,” or “I am of the opinion,” escape the dangerous magnetism of “I know that,” "I am certain." But I think the present analysis brings home more sharply what it means for the high possibility that just those of our beliefs...
[Event] A Metaculus Open Panel Discussion: How Forecasts Inform COVID-19 Policy
by ChristianWilliams — 2021-10-02T00:37:49.421Z

Linkpost and announcement for this event on 10/5 at 3pm ET with Virginia Department of Health officials and University of Virginia Biocomplexity Institute researchers, presented by Metaculus

From the event page:
"Since March 2020, Metaculus has provided forecasting and modeling resources to public health professionals as they've made crucial decisions in tracking and combating COVID-19. These efforts include the ongoing Keep Virginia Safe and the recently concluded Virginia Lightning Round forecasting tournaments, which were developed in partnership with the Virginia Department of Health and the University of Virginia Biocomplexity Institute, and were designed to enhance COVID-19 modeling efforts while contributing to the ongoing public health policy conversation in Virginia.

But how exactly are these forecasts used to inform COVID-19 policy? In this panel discussion and Q&A, presenters will share how the VDH and UVA Biocomplexity Institute teams first convened and how they currently collaborate, offering a behind-the-scenes look at how tournament questions are formulated—and ultimately, how they support public health policy. During Q&A, the audience will have an opportunity to engage with the public health officials and researchers who make use of forecaster predictions.

In addition to other panel members from the VDH and UVA teams, attendees will hear from

Justin Crow, VDH Director of the Division of Social Epidemiology and superforecaster
Bryan Lewis, Computational Epidemiologist and Public-Health Lead at the Network Systems Science and Advanced Computing Division of the Biocomplexity Institute
Srinivasan Venkatramanan, Research Assistant Professor at the Network Systems Science and Advanced Computing Division of the Biocomplexity Institute"

RSVP here!

This online event is scheduled for October 5th, 3:00PM - 4:00PM ET.

NOTE: Attendees will have the opportunity to receive some great UVA Biocomplexity Institute swag!

A Zoom link will be added to the page as the event approaches. For now, register a spot, and we’ll look forward to seeing you there!