trentmkelly/LessWrong-43k huggingface dataset:
What can we learn from traditional societies?
by mukashi — 2021-10-05T00:24:35.098Z

I have been reading books on Anthropology for a few years now and the one that ignited my passion on the topic was Jared Diamonds' very popular Guns, Germs and Steel. Jared Diamond is a polymath, historian, anthropologist and physiologist, fluent in many languages, a Pulitzer prize-winning and a very cool guy. He has followed a very heterodox academic pathway that inspired me to try weird things. His book "The world until yesterday" is a gem that has not received all the attention it deserves, so I wanted to do a bit of justice here and share some of its most interesting ideas and some of my thoughts after reading it.

State societies vs traditional societies

5000 years ago, the vast majority of people lived in small groups composed of 20-100 individuals. But at several points in human history as a result of the domestication of crops (which boosted an explosive population growth), some of these groups started coming together and created the first cities. It is possible to sit in a big circle and discuss the best ways to organize a society when your group has 40 people. If your group is composed of several thousand individuals, it is not: this gave rise to the centralization of power and the birth of the first states.

Today, the situation is totally different: a vast majority of people are under the rule of a state and just a few preserved the ancient way of life. These traditional societies retain behaviours, moral codes and societal structures that have not been modified for thousands of years, and resemble more what we could call a "natural state" of humans. These people face challenges more similar to the challenges we had to face in the ancestral environment where we evolved. They can be found in very diverse places such as the Amazon forest, New Guinea, Australia, Alaska or Subsaharan Africa. So what are some of the most striking aspects of their ways of living? Can we learn valuable things from them?

Childbirth

In the Western world, most births occur nowadays in hospitals, with professional assistance from doctors and nurses. The attitude towards childbirth varies a lot from culture to culture. Often, childbirth takes place with the assistance of other women. On occasions, childbirth is a public event; for instance, in the Agta people in the Philippines, a woman gives birth in a public house where everyone can visit and shout out instructions to the deliver...
Nuclear Espionage and AI Governance
by Guive — 2021-10-04T23:04:14.253Z

ABSTRACT:

Using both primary and secondary sources, I discuss the role of espionage in early nuclear history. Nuclear weapons are analogous to AI in many ways, so this period may hold lessons for AI governance. Nuclear spies successfully transferred information about the plutonium implosion bomb design and the enrichment of fissile material. Spies were mostly ideologically motivated. Counterintelligence was hampered by its fragmentation across multiple agencies and its inability to be choosy about talent used on the most important military research program in the largest war in human history. Furthermore, the Manhattan Project’s leadership prioritized avoiding domestic political oversight over preventing espionage. Nuclear espionage most likely sped up Soviet nuclear weapons development, but the Soviet Union would have been capable of developing nuclear weapons within a few years without spying. The slight gain in speed due to spying may nevertheless have been strategically significant.

Based on my study of nuclear espionage, I offer some tentative lessons for AI governance:

The importance of spying to transformative AI development is likely to be greater if the scaling hypothesis is false than if it is true.
Regardless of the course that AI technology takes, spies may be able to convey information about engineering or tacit knowledge (although more creativity will be required to transfer tacit than explicit knowledge).
Nationalism as well as ideas particularly prevalent among AI scientists (including belief in the open source ideal) may serve as motives for future AI spies. Spies might also be financially motivated, given that AI development mostly happens in the private sector (at least for now) where penalties for spying are lower and financial motivations are in general more important.
One model of technological races suggests that safety is best served by the leading project having a large lead, and therefore being secure enough in its position to expend resources on safety. Spies are likely, all else equal, to decrease the lead of the leader in a technological race. Spies are also likely to increase enmity between competitors, which seems to increase accident risk robustly to changes in circumstances and modeling assumptions. Therefore, it may make sense for those who are concerned about AI safety to take steps to oppose espiona

...
How good is security for LessWrong and the Alignment Forum?
by Quintin Pope — 2021-10-04T22:27:24.604Z

As far as I can tell, LessWrong/Alignment Forum haven’t been noticeably attacked or disrupted. However, I’m concerned that could change because:

Attack command, control and automation will definitely improve, making more sophisticated attacks easier to deploy at scale.

If AI becomes as important as we think, and the Rationality community succeeds at influencing its trajectory, LW/AF may become targets for surveillance/influence operations.

So my questions are:

How much focus does Lightcone Infrastructure and put on security?

Does Lightcone contract with any external security experts or penetration testers?

Are the any plans to implement two factor authentication for LW/AF?

Are there any planned responses if automated trolling/astroturfing attacks become much more common/advanced, as seems plausible with the rise of strong language models?

Are there plans for secondary hosting providers, in case Amazon/the US become hostile?

Is there some way we can download and backup all public conversations hosted on LW/AF?

Relatedly, how are backups handled for LW/AF?

Thank you very much to Lightcone Infrastructure and the LessWrong team for your work. I’d be glad for any insight they (or anyone else) can share.
How much memory is reserved for cute kitten pictures?
by Bezzi — 2021-10-04T21:30:39.646Z

In my social circles, I frequently tell a joke making fun of the awful lot of cute kitten pictures available on the internet ("somewhere in the world, a whole server farm is doing nothing but storing pictures of cute kittens"). Joking apart, there are thousands of data centers around and the world's total data storage capacity is measured in Zettabytes.

How much of this memory do you think to be actually occupied by cute kitten pictures? What could be an effective way to make a Fermi estimate?
Open & Welcome Thread October 2021
by qazzquimby — 2021-10-04T19:52:40.691Z

(
I saw October didn't have one. First post - please let me know if I do something wrong.
To whoever comes after me: Yoav Ravid comments that the wording could use an update.
)
If it’s worth saying, but not worth its own post, here's a place to put it.

If you are new to LessWrong, here's the place to introduce yourself. Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are invited. This is also the place to discuss feature requests and other ideas you have for the site, if you don't want to write a full top-level post.

If you want to explore the community more, I recommend reading the Library, checking recent Curated posts, seeing if there are any meetups in your area, and checking out the Getting Started section of the LessWrong FAQ. If you want to orient to the content on the site, you can also check out the new Concepts section.

The Open Thread tag is here. The Open Thread sequence is here.
[Book Review] "I Contain Multitudes" by Ed Yong
by J Bostock — 2021-10-04T19:29:55.205Z

I have read this book once before, just before I went to university, and I found it immensely enlightening at the time. Now I return to it as an older man, and find it has lost a lot of its shine. I don't know if this is reflective of my loss of naivete, or increase in biological knowledge.

Core Thesis

The subject of this book is the microbiome. This is the community of micro-organisms (although it's almost all focused on bacteria with a small amount of emphasis on viruses) that live around us, on us, and inside us.

The book is mostly a long series of interesting investigations into the microbiomes of humans and animals. What is implied (and sometimes said outright) is that biologists have neglected studying it.

The history of the study of microbiology is explored in detail. As I am not a huge fan of scientific history literature I mostly skimmed this but I expect it would be interesting to many people. Some key points here are the origins of the focus on microbes as pathogens and parasites, and how this idea has changed to focus more and more on symbiosis over time. We need our microbes and they need us!

Microbes do not have the same properties when it comes to genes as do large organisms. They can and do frequently exchange genetic material with other microbes. This means that in a microbiological community, the set of all genes present is more important than the set of specific microbes.

Holobionts and their Hologenomes

This culminates in the concept of the "hologenome". This is the set of genes in both an organism and its microbes, which are together referred to as a "holobiont". Hologenomes have some properties common to mere "genomes", such as the construction of biological pathways (which are like manufacturing lines, the product of one reaction being the starting-point of the next) using genes from many different bacteria and the host.

On the other hand, genomes of macroscopic organisms are generally static, only changing upon being transferred to the next organism. Hologenomes contain the "core" genes of the main organism, and it's necessary symbionts. Other genes are part of slow-changing microbes, and others still are ephemeral. Microbial genes are subject to different types of natural selection depending on their relationship to the host.

This perspective is genuinely very interesting. It's a different way of thinking about organisms and their bacterial c...
Andrew Yang on "The Priests of the Decline"
by Timothy Johnson — 2021-10-04T19:24:43.014Z
Link: https://www.andrewyang.com/blog/g3snyneafaf817fkkdsvpnf7w611w4

In this post excerpted from his new book, Andrew Yang uses his experience in politics to explain why the US political system rarely accomplishes much. It's strikingly similar to another recent post someone shared from Dominic Cummings: Dominic Cummings : Regime Change #2: A plea to Silicon Valley - LessWrong.

I've copied some of the key sections below:

"I call this dynamic constructive institutionalism — a tendency among leaders to state publicly and even hold the belief that everything will work out, despite quantitative evidence to the contrary, coupled with an inability to actually address a given institution’s real problems...

Indeed, two groups that are especially prone to constructive institutionalism are those that we rely upon both to give us a sense of the problems and to solve them — journalists and politicians.

Journalists are typically trained to be impartial observers, which inhibits them from expressing emotion or opinion. They are supposed to calmly document and present the news. For many, there is an implicit perch of authority and stability. Unfortunately, this has also turned many into market-friendly automatons and cultural guardians who make pro forma gestures about decorum, virtue, and propriety while ignoring the disintegration of trust, the dissipating integrity of their own organizations, or the decline of the American way of life...

If journalists are conditioned to calmly document dispassionately, politicians are conditioned to invoke profundity, resilience, and the greater good at every turn. As a politician you’re like a totem or shaman. You show up to a gathering or charity event to speechify and elevate the proceeding: “Thank you for the incredible work that you’re doing. It’s so important.” Which it is, of course. Though it would be if you didn’t show up too.

You are meant to embody the concerns of the community. You listen patiently to all. You are present. If someone asks you a question, you answer it reassuringly. You express values and aspirations. You are a human security blanket, and your job is to make people feel better."
To every people according to their language
by Yair Halberstadt — 2021-10-04T18:42:09.573Z

Here is an approximate transcript of a conversation I had with a friend, who is an Orthodox Jew.

Transcript

ME: Imagine the US government announced a Manhattan project to reverse aging and end death forever. It's estimated this project will cost 5% of GDP. There's no guarantees it will be successful, or if successful how long it will take, but the majority of domain experts agree that 50 years is a pretty good median estimate.

Suddenly all the people between the ages of 20 and 50, who currently accept their death as inevitable, will do a quick bit of mental arithmetic and realize that 50 years is a bit too close for comfort. There'd be calls to spend 10% of GDP and get it done in 25 years, or even 50% of GDP and get it done in 5 years! I'm not saying that either of those strategies would work, but they highlight how all of a sudden a light bulb will go off in peoples minds, and people would be frantically trying to escape death and aging.

So if we know that once we realize there's a possibility to escape death we will be desperate to clutch at it, and right now there's at least some chance that such a Manhattan project could be feasible, we should push for this Manhattan project right away. Let's not wait till the government announces it and we realize how much we always wanted it!

FRIEND: I'm not sure I would be frantic to support such a project - in fact I'd probably oppose it. I believe it would fail anyway since God wants people to die.

ME: But God doesn't inherently want people to die - (according to Orthodox Judaism) before the fall from Eden people were meant to live forever, and people only die as a punishment for Adam and Eve's sin. And in the time of the Messiah, people will live forever.

FRIEND: Yep, but God hasn't bought the Messiah yet.

ME: Do you believe that the Messiah is meant to be bought by people, or by God?

FRIEND: I've only ever heard that people are meant to bring the Messiah through creating world peace, and building the State of Israel. It's always been clear that it's God that's meant to end death.

ME: Well obviously people have always thought that, since it seemed that ending death would be impossible for humanity to achieve. I assume everyone assumed the in-gathering of exiles would be miraculous before Zionism and the creation of the state of Israel. But now that it finally might be within Humanity's grasp, wouldn't it make most sense, and...
Candy Innovation
by jefftk — 2021-10-04T13:50:03.541Z

What innovations in candy have there been since the '90s? Are there
new flavors? Better imitations of existing flavors? New textures?

So far, all of the candy my kids have brought home seems to be things
we could get 25 years ago. Though possibly flavors have improved,
since I've only been trying what they've decided to share with me.

There have been some gains due to globalization, where candy that was
previously hard to get in the US or unknown here is now more widely
available, but has there been development beyond that?

(This post brought to you by yesterday's neighborhood, pinata)

Comment via: facebook
What role should LW play in AI Safety?
by Chris_Leong — 2021-10-04T02:21:53.190Z

Many people on LW consider AI Safety either the most, or one of the most, important issues that humanity has to deal with. Surprisingly, I've seen very little discussion about how the LW community slots in here. I'm sure that the Lightcone team has discussed this extensively, but very little of their discussions have made it onto the forum. I hope that they write up some more of their thoughts at some point, so that the community can engage with them, but since there hasn't been much written on this topic, I'll focus mostly on how I see this topic.

I think a good place to begin would be to list the different ways that the Less Wrong community contributes or has contributed towards this project. By the LW community, I mean the broader rationalsphere, although I wouldn't include people who have just posted on LW once or twice without reading it ir itherwise engaging with the community:

a) By being the community out of which MIRI arose
b) By persuading a significant number of people to pursue AI safety research either within academia or outside of it
c) By donating money to AI Safety organisations
d) By providing a significant number of recruits for EA
e) By providing an online space in which to explore self-development
f) By developing rationality tools and techniques useful for AI safety (incl. CFAR)
g) By improving communication norms and practices
h) By producing rationalist or rationalist-adjacent intellectuals who persuade people that AI Safety is important
i) By providing a location for discussing and sharing AI Safety research
j) By creating real-world communities that provide for the growth and development of participants
k) By providing people a real-world community of people who also believe that AI safety is important
g) By providing a discussion space free from some of the political incentives affecting EA
h) More generally, by approaching the problem of AI safety with a different lens than other concerned communities

Some of these purposes seem to have been better served by the EA community. For example, I expect that the EA community is currently ahead in terms of the following:

a) Building new institutions that focus on AI safety
b) Donating money to AI Safety organisations
c) Recruiting people for AI Safety research
The rationality community may very well be ahead of EA in terms of having produced intellectuals who persuaded people that AI Safety is importan...
Trust and The Small World Fallacy
by DPiepgrass — 2021-10-04T00:38:45.208Z

When people encounter other people they recognize, they exclaim "small world!"

I suspect that most people have 300 acquaintances or less. I probably have under 100. Still, sometimes I run into people I know and I'm tempted to say "small world".

But it's not actually a small world, is it? It's an unimaginably enormous world.

I mean that literally. You cannot imagine how big the world is.

You're not likely to meet a million people in your life. If you were to meet 100 strangers in 8 hours, you would have less than 5 minutes to spend with each person. If you met 100 strangers every day including weekends, with no vacation days, it would take over 27 years to meet a million people.

How many of those million people would you be able to remember after you've been meeting 100 of them every day for 27.4 years? A few hundred, maybe? A few thousand if you have an especially good memory? It seems to me that even after you've met a million people, your brain is already too small to properly comprehend the thing you just accomplished.

And a million people is nothing in this world. This world has over 7,000 million people. It's truly beyond imagination.

There was a time when the entire global anti-vax movement was centered around a single man who wrote a single paper citing the opinion of 12 parents that perhaps the combination MMR (measles, mumps, and rubella) vaccine caused a combination autism and bowel disease, or as the paper put it, "chronic enterocolitis in children that may be related to neuropsychiatric dysfunction." Among other anomalies, this man took unusual steps like holding a press conference about his N=12 study "Early Report", having a "publicist" answer his phone, and filing a patent for a measles vaccine months before publishing his paper.

At that time you could argue that we should Beware The Man of One Study. Science produces many studies, including many that suffer from a small sample size, and even some with large biases. Some studies are even fraudulent. Did you know that over 100,000 papers have been published on the topic of climate change? The point is, any reasonable person won't take a single study as proof (though it is still evidence).

Of course, it's not as if "Beware The Man of One Study" would have ever been an effective argument against an anti-vaxxer, even back then. Somehow, the original claim that "the combination MMR vaccine is related t...
An analysis of the Less Wrong D&D.Sci 4th Edition game
by Maxwell Peterson — 2021-10-04T00:03:44.279Z

This is an analysis of the game described in this post - you might want to read that first. Or if you don't want to, here's a quick summary: There's a team-based competitive video game, the Green team has a game coming up against the Blue team, and we're supposed to advise the Green team which team to use. There are 19 characters and each team is composed of 5 of them. We know which team the Blue team is going to use, so want to find the Green team best suited to this known opponent team. To this end we have data on 65K historical games. The data is just the character picks for each team, and the game result.

I trained a predictor on most of the games, and used the rest to validate. The predictor has an out-of-sample AUC of 0.79, which is not fantastic, but if these are really supposed to be games played by people, we can hardly hope that using just the team compositions to predict who won would get close to perfect accuracy. The post describing the game gives a single Blue team composition to optimize against. Once I had the predictor, I generated all possible Green teams, and evaluated how each would do against that composition. That gave me win probabilities for every Green team against the one single Blue team.

The predictor has some calibration problems:

Although the model has OK calibration, it's pretty underconfident at the low end, and pretty overconfident at the high end. But I'm sick of dinking with the model, so let's push on, pretend it's perfectly calibrated, and talk about its predictions as win probabilities.

Blue has a good team

So, first, Blue has quite the team. There are 11,628 (choose-5-from-19) possible team comps for Green, but against the Blue comp from the original post, less than 700 of these have a better-than-even win probability:

This is really surprising. More on this later.

How Green can beat Blue anyway

So here's my recommendation to the Green team: The team with the highest probability of winning (against the Blue team Dire Druid, Greenery Giant, Phoenix Paladin, Quartz Questant, Tidehollow Tyrant) is Arch-Alligator, Greenery Giant, Landslide Lord, Nullifying Nightmare, and Phoenix Paladin. The predicted win probability for that Green team is around 75%. (Considering the calibration plot above, maybe we should move this to 78%.)

Let's see what some winning Green compositions look like. There are 132 teams with a predicted win pro...