trentmkelly/LessWrong-43k huggingface dataset:
A short introduction to machine learning
by Richard_Ngo — 2021-08-30T14:31:45.357Z

Despite the current popularity of machine learning, I haven’t found any short introductions to it which quite match the way I prefer to introduce people to the field. So here’s my own. Compared with other introductions, I’ve focused less on explaining each concept in detail, and more on explaining how they relate to other important concepts in AI, especially in diagram form. If you're new to machine learning, you shouldn't expect to fully understand most of the concepts explained here just after reading this post - the goal is instead to provide a broad framework which will contextualise more detailed explanations you'll receive from elsewhere.

I'm aware that high-level taxonomies can be controversial, and also that it's easy to fall into the illusion of transparency when trying to introduce a field; so suggestions for improvements are very welcome!

The key ideas are contained in this summary diagram:

First, some quick clarifications:

None of the boxes are meant to be comprehensive; we could add more items to any of them. So you should picture each list ending with “and others”.
The distinction between tasks and techniques is not a firm or standard categorisation; it’s just the best way I’ve found so far to lay things out.
The summary is explicitly from an AI-centric perspective. For example, statistical modeling and optimization are fields in their own right; but for our current purposes we can think of them as machine learning techniques.

Let’s dig into each part of the diagram now, starting from the top.

Paradigms of artificial intelligence

The field of artificial intelligence aims to develop computer programs that are able to perform useful tasks like answering questions, recognizing images, and so on. It got started around the 1950s. Historically, there have been several different approaches to AI. In the first few decades, the dominant paradigm was symbolic AI, which focused on representing problems using statements in formal languages (like logic, or programming languages), and searching for solutions by manipulating those representations according to fixed rules. For example, a symbolic AI can represent a game of chess using a set of statements about where the pieces currently are, and a set of statements about where the pieces are allowed to move (you can only move bishops diagonally, you can't move your king into check, etc). It can then play chess by searc...
Beware of small world puzzles
by mukashi — 2021-08-30T06:00:55.054Z

A very well known puzzle in probabilities says:

Mr. Smith has two children. At least one of them is a boy. What is the probability that both children are boys?

There is much discussion about this puzzle because depending on how you find out that one of the kids is a boy, the probability changes. I understand this, but this is not what I wanted to discuss. If you want to know more about this, you can read this Wikipedia page.

What I wanted to discuss is that for me to give the answer that I am expected, I need to be aware that the person posing the question is expecting me to write a tree with all the possible combinations etc. The tacit assumption about the problem is that it is a puzzle, i.e a problem of a small world with very clear rules. There are many puzzles such as this one that are used to illustrate things such as: humans behave irrationally, humans are very bad at judging probabilities, humans lack a basic understanding of the most fundamentals concepts in economics, etc. I feel that many of these puzzles rush too quickly into those conclusions and we should always treat them with a pinch of salt.

The world is not a small world

A small world is an artificial environment created with well-defined rules and probabilities. Usually, small worlds are inhabited by agents that are uniform (meaning that all of them behave the same under the same circumstances), have perfect knowledge, lack any feelings and are perfect reasoners; if the problem has a solution (in their small world environment) they will reach the solution.

We don't live in a small world. We don't live in a world where most problems can be assigned a clear probability such as the ones you can compute when flipping a coin.

Why is this relevant?

In the last two hundred years, a large part of economic thinking, now considered to be the orthodoxy, has used simplified models of the world to understand the complex system of economics. In these models, every agent of the system has perfect information about the system, takes decisions in a way that maximizes a well-defined utility function and every agent is equivalent to every other agent.

In the 60s, the American thinker Herbert Simon conceived the idea of Bounded rationality, in which humans are not perfect thinkers but often take sub-optimal decisions, as a response to the Homo economics model of humans. A few years later, Kahnem...
Humanity is Winning the Fight Against Infectious Disease
by lsusr — 2021-08-30T01:31:24.107Z

One of my favorite things about living in the 21st century United States is how so few people die of infectious disease. Here's a graph of deaths showing how infectious diseases decreased by an order of magnitude between the turn of the century and Salk's Polio Vaccine.

In 1900, 12% of deaths were caused by pneumonia and influenza. 11% were caused by tuberculosis. 8% were caused by of diarrhea. A hundred years ago infectious diseases killed over one third of everyone. In contrast, not a single person in my extended social circle has ever died of an infectious disease. I know more people who were killed by airplane crashes than by infectious diseases.

The raw 10× reduction undersells the progress we've made against infectious disease. We have more people overall and we live together in cities. Death due to infectious disease should have skyrocketed over the last century. We would have been fortunate if medicine and sanitation merely kept infectious disease at 1900 levels.

The emergence of new diseases like HIV and COVID-19 has had a tiny impact compared to the overall trend.

In 2019, there were 15,815 deaths among adults and adolescents with diagnosed HIV in the United States and 6 dependent areas. These deaths may be due to any cause.

―hiv.gov

2,855,000 Americans died in 2019. Those with HIV were 0.5% of the total. Even if we are maximally pessimistic and attribute to HIV every single death of every single American with HIV, this is a tiny fraction compared to how many people used to die of the plagues of 1900.

As of July 20, 2021, COVID-19 killed 600,000 Americans over the course of 1.5 years. That's 400,000 deaths per year. At 13% of total deaths, COVID-19 temporarily increased deaths due to infectious disease back to what we had in the 1940s. But the increase is temporary. We have vaccines. Even if we didn't have vaccines, deaths due to COVID-19 would decrease naturally after it burned through the vulnerable population.

If new diseases emerged while technology stagnated then we'd be living in a temporary golden age―and maybe we are. But biotechnology is advancing faster than any technology ever.

The COVID-19 vaccines were invented and deployed with unprecedented speed. Civilization will only get better at inventing vaccines. I mean, can you plausibly imagine a worse response to COVID-19 than what we experienced in the West?

With biotechnology advancing so quick...
A Small Vacation
by DanB — 2021-08-29T19:55:34.970Z

(note: I have been thinking about this essay for several months; I did not realize it would turn out to be
so timely).

Many people in the rationality community
and adjacent corners of the
intellectual world are sympathetic to the idea of
Open Borders.
This concept can be explained easily.
People in the developing world are suffering because of poverty
and government oppression.
When a migrant moves from a country like Guatemala to a country like the US,
her economic prospects brighten dramatically.
At the same time, according to OB proponents,
she does not substantially reduce the well-being of the previous inhabitants of her adopted country,
and may actually improve it.
Even if the net benefit to the current inhabitants is zero,
the fact that the immigrants accrue so much benefit,
should be enough to persaude most fair-minded people that immigration is good -
and in fact most people are in favor of modest immigration.
The OB position takes this logic far further than most people are comfortable with,
but the rationale is clear.

Unfortunately, most people won't accept "scaled-up" economic arguments.
If the evidence shows that immigration is mostly good,
normal people will accept a modest level of immigration,
but not the complete abolition of immigration restrictions.
In recent times this issue has become politically polarized,
but it wasn't always a clear-cut Left vs Right battle -
here's
Bernie Sanders deriding Open Borders as a Koch brothers proposal.
Depending on your own views,
you might regard the mainstream attitude as sensible,
or you might regard it as cold-hearted and self-destructive.
Either way,
it's clear that full Open Borders isn't politically feasible in the near term.

I believe there is a policy that is more likely to be politically acceptable,
while also capturing many of the ethical benefits of Open Borders.
That policy is Vacation -
the US should vacate small strips of territory
to make a space where refugee and migrant populations can construct
independent city-states.
A key piece of the argument is the fact that small nations are in fact quite viable,
so let me talk about Singapore for a moment.

The Singaporean Miracle

Singapore is a city-state in Southeast Asia.
Formerly a part of Malaysia,
SG was expelled from the mother country in 1965,
largely due to ethnic strife between the Chinese and Malays.
Under the pragmatic - and somewhat autho...
Virtual Machine Learning Conferences: The Good and the Bad
by libai — 2021-08-29T19:26:57.159Z

We are now almost two years into the COVID-19 pandemic: international travel has slowed to a trickle and all machine learning conferences have moved online. By now, most of my PhD research has taken place during the pandemic, which I’ve presented at four online conferences (ACL ’20, EMNLP ’20, NAACL ’21, and ACL ’21). I’ve also had the fortune to attend a few in-person conferences before the pandemic, so here I'll share some of my thoughts on the advantages of each format.

Travel

One of the perks of choosing grad school for me was the chance to travel to conferences to present my work. Typically the first author of each paper gets funded by the university to present their work. The university pays for the conference fees, hotels, and airfare, which adds up to several thousand dollars per conference. With all conferences online, the school only pays for the conference fee (a trivial amount, about $25-$100). Effectively, a major part of grad student compensation has been cut without replacing it with something else.

There are some advantages though. Before the pandemic, it was mandatory to travel to present your work at the conference. This can be at an inconvenient time or location (such as a 20-hour flight to Australia), so I avoided submitting to certain conferences because of this. With virtual conferences, I can submit anywhere without location being a factor.

Another factor is climate change. The IPCC report this year announced that the earth is warming up at an alarming rate, and at an individual level, one of the biggest contributors to greenhouse gas emissions is air travel. Thousands of grad students travelling internationally several times a year adds up to a significant amount of carbon emissions. Therefore, unless there are substantial benefits to meeting in-person, the climate impact alone is probably worth keeping to virtual conferences post-pandemic.

Talks and Posters

Typically in conferences, paper presentations can be oral (12-minute talk with 3 minutes Q/A), or poster (stand beside a poster for 2 hours). Online conferences mimiced the format: oral presentations were done in Zoom calls, while poster sessions were done in Gather Town (a game-like environment where you move around an avatar). Additionally, most conferences got the authors to record their presentations in advance, so the videos were available to watch at any time during the conference...
The Solow-Swan model of economic growth
by Matthew Barnett — 2021-08-29T18:55:34.848Z

Note: This is the first post in a sequence I announced over a year ago. The sequence is intended to serve as a primer to the field of economic growth for those interested in the intersection between growth theory and artificial intelligence. See the announcement for more details.

Economic growth theory concerns the long-run behavior of economies, such as the United States, or the world economy. An "economy" here simply means the collective behavior of many agents interacting in a universe with scarce resources. Since economies are very complex, economists tend to focus on a small set of summary statistics to characterize them. For our purposes, we are most interested in the behavior of gross domestic product (GDP) over time as it interacts with factors such as labor, capital, and technology. The Solow-Swan model is the benchmark model in mainstream economics for understanding this process, so it is the first model we will look at in this sequence.

Why GDP?

Technically speaking, GDP is the market value of all final goods and services produced within a given nation, in one year. The meaning of "final good" is crucial here: if someone is producing a good for an intermediate purpose—such as smelting metal to be used in a smartphone—only the value added to the smartphone is counted for the purpose of GDP, not the total money exchanged during the entire process of making the smartphone. This concept is crucial for understanding the so-called production approach to calculating GDP.

GDP is useful for characterizing a large economy because, in two senses, it represents both the output and input to a market economy. GDP represents the output of an economy for a reason I have already alluded to: it is estimated by counting up all production within the economy, with only those interactions between agents that add value to end consumers counting in the calculation.

GDP also represents the input to a market economy because each good produced will at some point be bought by a consumer. There are two approaches to calculate GDP which take this fact into account. The first is the expenditure approach, which adds up the market value of all goods and services sold to consumers within an economy over a year. The second is the income approach, which is based on the fact that each good and service bought by a consumer also represents income taken home by a seller.

From the perspective ...
Has Van Horn fixed Cox’s theorem?
by  — 2021-08-29T18:36:46.280Z

What do you think about Kevin S. Van Horn’s formulation of probability as extended logic? It looks interesting because it doesn’t seem to rely on the not-so-intuitive Universality axiom, removing which leads to Halpern’s counter-example, or conventions, axioms/desiderata which are not required by common sense, like representing plausibility using real numbers. Even David Chapman, who is highly critical of probability theory as extended logic, seems to be okay with it. Is it widely accepted or does it have conceptual issues like the original formulation?
The Governance Problem and the "Pretty Good" X-Risk
by Zach Stein-Perlman — 2021-08-29T18:00:28.190Z

No longer endorsed.

Like a child suddenly given awesome strength, I could have pushed too hard, and left the world a broken toy I could never repair. —The Hero of Ages

I hope we create superintelligence, and I hope it does what we want. But this would not suffice to make the future great. For a great future from superintelligence, a third condition is necessary: what we want is great. This post is about what the controllers of a superintelligence will want and how we can improve that.

Epistemic status: confident about the problem, not confident about specific failure modes or solutions.

I. Introduction

If it were available to them, I think current human elites (or almost any other group of humans) would overwhelmingly choose this bargain over the status quo:

Earthly Utopia. Human civilization will last on Earth for ten billion years. During that time, almost every person's life will be better than the best life ever lived before now. Earth will be a place of great learning and discovery, personal excellence and achievement, expansive freedom, extraordinary beauty, wonderful experiences, deep relationships, harmony with nature, meaningful projects, and profound joy. However, we may never use resources outside our solar system.

It may be that more intelligent, thoughtful, and wise versions of ourselves would take such a bargain. But more likely, I think, our enlightened counterparts would regard it as the worst mistake and the greatest catastrophe ever. If an astronomical number of wonderful planets is astronomically better than one, we have a problem:

Our decisionmakers and decisionmaking institutions would take a terrible bargain because they neglect nontraditional sources of value (viz., optimizing the rest of the universe) that happen to be overwhelmingly important (in expectation). This is not surprising — these same processes egregiously fail to appreciate the value or risk of AI. So if we solve alignment and create superintelligence, it will not just take care of the rest. It's not enough to have nice people control the superintelligence. To adapt from The Rocket Alignment Problem:

Superintelligence may be developed by ill-intentioned people. But that's not the failure mode I'm worried about right now. I'm more worried that right now, even nice and generous potential-controllers-of-superintelligence would want it to do non-optimal things. Whether G

...
What will be the most important resource after the information age? 
by actuallynotaustralian — 2021-08-29T10:23:50.644Z

The value of resources is defined by them being scarce and unequally distributed. But when nearing an equilibrium state, new resources emerge through the synthesis of previously available ones. One can go as far as saying, and even claim there is a hierarchy of resources. Starting with bare natural resources which can be found in nature, to processed resources that are required to manufacture goods, to any form of manufactured goods self, to information that is required to manufacture and manage the production goods in the first place. Admittingly, as we all know, we live in the "information age", centered around the latter stage. But what resource will emerge and becomes most important after the information age?
[ACX Linkpost] Highlights From The Comments On Missing School
by mike_hawke — 2021-08-29T08:01:59.534Z
Link: https://astralcodexten.substack.com/p/highlights-from-the-comments-on-missing

It's a really good post, and worth reading the whole thing. In section V, Scott gives an extended response to the general category of claim that school provides some kind of illegible benefit that doesn't show up in test scores.

[...]I get nervous when I see a giant institution with lots of really legible costs, whose legible benefits don’t withstand scrutiny, and people proposing a bunch of very diverse, kind of flaky sounding illegible benefits that mean we should still force everyone to participate in it.

...

There’s this weird trap a lot of adults fall into where anything a kid does on their own, however interesting, is “wasting away”, and anything they do at school, however ridiculous, is Exciting Prosocial Learning Fun Glowing Childhood Memories. I think this might be entirely a function of whether the parents can spectate and take pictures that look good on a mantlepiece: easy with hobbyhorsing, harder with learning C++.

The section strays a bit from the narrower topic of school absences, but I'm really glad it does, since he articulates the status quo bias really well.
How much should we still worry about catching COVID? [Links and Discussion Thread]
by MichaelStJules — 2021-08-29T06:26:04.292Z

[Cross-posted to the EA Forum. Consider checking comments there, too.]

I've decided to collect research on the risks of COVID, especially among vaccinated people and for long COVID. Please share more in the comments, and it would be helpful to categorize them according to the section headings of this post. Other discussion about the evidence and research is also welcome in the comments.

I've included quotes of estimates for the three links in the section "Overall how bad it is to catch COVID".

I'm not committing to updating the post with more links beyond the ones I've already included (although I might); I encourage readers to check the comments instead.

EDIT: Check the COVID-19 tag on LessWrong.

Overall how bad it is to catch COVID

I've come across three estimates for the number of expected quality of life adjusted days lost to (long) COVID for fully vaccinated people. I think the main reasons for the discrepancies between the first and second are higher risks of long COVID conditional on COVID and higher risks of long COVID becoming basically permanent conditional on long COVID in the first link. Some potential symptoms like fatigue and brain fog would reduce your productivity.

1. https://www.mattbell.us/delta-and-long-covid/

For fully vaccinated people catching Delta:

If you’re a 35 year old woman, and your risk of ending up with lifelong long COVID from catching COVID is 2.8%, then catching COVID would be the same, statistically speaking, as losing (50 years * 0.18 * 0.028 * 365 days/year) = ~90 days of your life.

(...)

A 35 year old woman runs about an 0.5% chance of the "worst case scenario" outcome if she gets Delta. For comparison, 0.5% is about 42x your chance of dying in a car crash in the next year.

Also, see these two figures (derived in the article) for risks of multi-year long COVID or worst case multi-year long COVID by age and gender among vaccinated people, and the rest of the key takeaways at the start of the article.

2. https://www.lesswrong.com/posts/9xd5ArGud8897q5on/delta-strain-fact-dump-and-some-policy-takeaways?commentId=rGZiHcuzt7bh7sXmL

For fully vaccinated people:

My new estimate from the calculation is 3.0 to 11.7 quality-adjusted days lost to long-term sequelae, with my all-things-considered mean at 45.

3. https://www.lesswrong.com/posts/9xd5ArGud8897q5on/delta-strain-fact-dump-and-some-policy-t...