trentmkelly/LessWrong-43k huggingface dataset:
The harms you don't see
by ViktoriaMalyasova — 2022-10-16T23:45:31.113Z

Падшие ангелы рядом с лицами старых солдат

Ждут одного лишь приказа: вернуться… Вернуться назад

Чтобы собрать все знамёна, стрелы, обломки мечей,

Вновь пережить вместе битву ста дней и ночей.

Катарсис, "Крылья"
I often see people in the West argue that Ukraine ought to cede some territories to Russia so that peace is achieved and people are kept safe. While perharps well intentioned, I think these suggestions are based on a distorted perception of reality. There are harms that you notice, because they happen in a big, loud fight that draws everyone attention, and there are harms that happen not so visibly, in dark overcrowded cells, in dusty courtrooms, in people's heads. You may not hear about them in the news, but it is still important to know and account for them.

In 2014, when Russia annexed Crimea, it started destroying all Crimean Tatar civil society institutions. Mejlis, the executive representative body of Crimean Tatars since 1991, was outlawed as an "extremist organization" in 2016. Russia banned Crimean Tatar national events: the rally on the Deportation of Crimean Tatar People Day on May 18th and the Crimean Tatar National Flag day on June 26th.

Over 340 people are being persecuted for their alleged participation in Hizb ut-Tahrir. Over 230 people are already sentenced, and most of them got prison terms of 10 years or more. Hizb ut-Tahrir is an Islamist political organization that aims...
Maximal lotteries for value learning
by ViktoriaMalyasova — 2022-10-16T23:44:12.957Z

Suppose Ann asks robot Bob to buy her favorite fruit. Bob doesn't know what Ann's favorite fruit is, but he knows that Ann likes either apples, bananas or oranges, each with probability 1/3, and doesn't like the other two fruits at all. At the groceries, for his money Bob can buy 10 apples, 10 bananas, 10 oranges or a basket with 1 apple, 1 banana and 1 orange. What should he do (assuming that asking Ann is not an option)?

One way for Bob to approach this dilemma is to imagine he playing a game against a robot Carl in a world that is exactly like ours except that Carl may have a different policy than Bob. In this game, both robots choose and execute a policy, and the robot who produced a better outcome according to Ann's utility function wins. If they did equally well, the winner is determined randomly. Bob should execute the mixed strategy Nash equilibrium policy in this game. In our example, this is the policy of buying the gift box: it has a 2/3 chance of winning against other pure policies.
This game is known as maximal lottery. It was introduced[1] by G. Kreweras as a social decision scheme - a way to aggregate preferences of voters over candidates. In our case, Ann's possible utility functions are voters, their prior probabilities are voter fractions, and policies are candidates.

Learning utility

Imagine now that an option of asking Ann about her favorite fruit and buying 10 of it became available. Clearly this would be the winning policy. In a more general case, learning about the user's utility function can only help the robot win, so the robot always wants to do it if it can be done for free.

Theorem

Suppose we are playing the maximal lottery game, we believe that a human's utility function is Ui with probability qi, i=1,…,N. We know there is a switch X, which is initially off, such that if a human has utility function Ui, human sets X on with probability pi. (We assume at least one of pi is not 0 and at least one is not 1, or there would be nothing to learn from looking at X.) Player doesn't yet know if X is on, but can find it out for free (as measured by every Ui). Suppose that in a game where X is known to be on and prior beliefs for utility Ui are piqi/∑ipiqi, π1 is a winning strategy. Suppose in a game where X is known to be off and prior beliefs for utility Ui are&...
Popular Personal Financial Advice versus the Professors (James Choi, NBER)
by BrownHairedEevee — 2022-10-16T22:21:09.012Z
Link: https://spinup-000d1a-wp-offload-media.s3.amazonaws.com/faculty/wp-content/uploads/sites/27/2022/08/Popular-personal-financial-advice-2022.08.10.pdf

This econ paper about personal finance was featured in a recent Freakonomics Radio episode. It compares the financial advice in popular personal finance books with what economics research says is optimal financial behavior.

Some of the most surprising (to me) recommendations in the paper:

It's actually suboptimal to put away a fixed percentage of your income as savings every year. Instead of smoothing your savings, you should smooth your consumption so that you spend the same amount of money every year. Since most people's earnings potential peaks in midlife, it's best to have a small or even negative savings rate early in your career, and save the largest percentage of your income toward the midpoint of your career.

Smoothing consumption over time makes sense to me. If you know that you will be able to earn a fixed amount of money over your lifetime, then you ought to spend it evenly over time, as this maximizes your utility at each point in time given diminishing marginal utility of spending.
However, consumption smoothing implies a different optimal strategy for those interested in jumping from a high-paying career to a less highly paying one such as entrepreneurship, nonprofit work, or the arts. In general, your savings rate should be highest when your earnings potential is highest. But if you expect to earn the most early in your career, then that's when your savings rate should be the highest. For these people, saving a fixed % of your income is probably closer to the right move.

Many popular finance books recommend overweighting U.S. stocks relative to the international stock market, since U.S.-based multinational companies provide exposure to international markets and international stocks carry a number of risks such as currency risk and weaker accounting and financial transparency standards. However, most economists reject arguments for overweighting the U.S. market, recommending instead that "every investor should hold each country’s securities in proportion to its market capitalization" (p. 16).

First, the costs and perceived risks of exposure to international stocks are too small to justify the amount of home bias that we see among investors.
Second, "the correlation of multinationals’ stock returns with their domestic stock market is very high, limiting the international diversification benefit obtained by buying the multinational stocks of one’s own c

...
Life, Death, and Finance in the Cosmic Multiverse 
by peterb — 2022-10-16T18:57:07.082Z

I published this report a while ago and would love to get some feedback from the experts here.

https://www.bcaresearch.com/public/content/GIS_SR_2021_12_21.pdf

Highlights

• The idea of parallel worlds seems like the stuff of science fiction. Yet, most leading cosmological theories suggest that the universe is vast enough to contain multiple copies of everything, including multiple copies of you.• The existence of the “multiverse” may account for why the parameters of physics seem highly fine-tuned to support life and why the earth has escaped so many existential threats – major asteroid strikes, World War III, a more deadly strain of Covid, etc. The resulting survival bias helps explain why stocks have historically performed so well.• Randomness is an illusion. Everything that could possibly happen to the stock market over the next 12 months will happen. As an investor, the best you can do is estimate the relative frequencies of various outcomes, as given by the Born Rule for quantum mechanics.• Subjectively, you will feel immortal. When life-threatening events occur, it will always seem as though you miraculously pull through. This has major implications for financial decisions, not to mention how you live your life and see the world.
Significance of the Language of Thought Hypothesis?
by DrFlaggstaff — 2022-10-16T18:09:29.083Z

I've recently read a few books on cognition and psycholinguistics which has put a name to a concept I've become increasing the familiar with, the Language of Thought Hypothesis; wherein researchers theorize that mentalese, and subsequently most of human thought, follow syntactical rules.

They attempt to translate sentences into language independent symbolic propositions, for example:
"Sam spray-painted the walls." Becomes
"(Sam spray paint) cause (paint go to (on wall))"

I find this endlessly fascinating but I struggle to think of an actual application of this kind of knowledge. What could we actually do with an understanding of how thoughts are structured syntactically?

The one and only thing that comes to mind would be intelligently designing future languages to fit more snugly with cognition, which is no small thing admittedly. But I would love to discuss the implications of this research if anything comes to mind to anyone.
Luck based medicine: my resentful story of becoming a medical miracle
by Elizabeth — 2022-10-16T17:40:03.702Z
Link: https://acesounderglass.com/2022/10/13/my-resentful-story-of-becoming-a-medical-miracle/

You know those health books with “miracle cure” in the subtitle? The ones that always start with a preface about a particular patient who was completely hopeless until they tried the supplement/meditation technique/healing crystal that the book is based on? These people always start broken and miserable, unable to work or enjoy life, perhaps even suicidal from the sheer hopelessness of getting their body to stop betraying them. They’ve spent decades trying everything and nothing has worked until their friend makes them see the book’s author, who prescribes the same thing they always prescribe, and the patient immediately stands up and starts dancing because their problem is entirely fixed (more conservative books will say it took two sessions). You know how those are completely unbelievable, because anything that worked that well would go mainstream, so basically the book is starting you off with a shit test to make sure you don’t challenge its bullshit later?

Well 5 months ago I became one of those miraculous stories, except worse, because my doctor didn’t even do it on purpose. This finalized some already fermenting changes in how I view medical interventions and research. Namely: sometimes knowledge doesn’t work and then you have to optimize for luck.

I assure you I’m at least as unhappy about this as you are.

Preface to the Preface

I’ve had nonspecific digestive issues since before I have memories. In pre-school my family joked that I would die as a caveman because there were so few things I would eat, and they were mostly grains. This caused a bunch of subclinical malnutrition issues that took a lot of time to manage and never got completely better. And while I couldn’t articulate this until it went away, food felt gross all the time

It’s hard to convey just how bad this was for me, because it feels like it undermines everything I did to work around it. I’ve always been functional but decidedly less healthy than my friends. I got sick more often and it hit me harder. I was slower to heal from injuries and scrapes and that limited my interest in the more athletic sort of hobbies. I couldn’t work the same hours, and working hours traded off really sharply against energetic hobbies. I had to spend a lot of time managing food where other people can just show up and eat, which was a constant source of social stress. My genetics say I was destined to h...
Age changes what you care about
by Dentin — 2022-10-16T15:36:36.148Z

[Possible trigger warning for discussion of mortality.]

I've been on LessWrong for a very long time. My first exposure to the power of AI was in the mid 1980's, and I was easily at "shock level 4" at the turn of the millennium thanks to various pieces of fiction, newsgroups, and discussion boards.

(If you're curious, "The metamorphosis of prime intellect", "Autonomy", and "A fire upon the deep" all had large impact on me.)

My current best guess is that there's a double-digit percentage chance the human race (uploads included) will go extinct in the next century as a result of changes due to AI.

With that background in mind, existential AI risk is not my highest priority, and I am making no meaningful effort to address that risk other than sometimes posting comments here on LW. This post is an attempt to provide some insight into why, to others who might not understand.

Put bluntly, I have bigger things to worry about than double digit odds of extinction in the next century due to AI, and I am a rather selfish individual.

What could possibly be more important than extinction? At the moment, that would be an extremely solid 50% chance of death in the next 3-4 decades. That's not some vague guesswork based on hypothetical technological advances; it's a hundred thousand people dying per day, every day, as they have been for the last century, each death contributing data to the certainty of that number. That's two decades of watching the health care system be far from adequate, much less efficient.

Consider that right now, in 2022, I am 49 years old. In 2030 I'll be 57, and there's good reason to believe that I'll be rolling 1d100 every year and hoping that I don't get unlucky. By 2040, it's 1d50, and by 2050 it's 1d25. Integrate that over the next three decades and it does not paint a pretty picture.

Allow me a moment to try to convey the horror of what this feels like from the inside: every day, you wonder, "is today the day that my hardware enters catastrophic cascading failure?" Every day, it's getting out a 1d10 and rolling it five times, knowing that if they all come up 1's it's over. I have no ability to make backups of my mind state; I cannot multihost or run multiple replicas; I cannot even export any meaningful amount of data. The high reliability platform I'm currently running on needs to fail e...
Cruxes in Katja Grace's Counterarguments
by azsantosk — 2022-10-16T08:44:56.245Z

In the recent counterarguments to the basic AI x-risk case, Katja Grace mentions a basic argument for existential risk from superhuman AI, consisting of three claims, and proceeds to explain the gaps she sees in each on them. Here are the claims:

A. Superhuman AI systems will be "goal-directed"

B. Goal-directed AI systems' goals will be bad

C. Superhuman AI systems will be sufficient superior to overpower humanity

I personally consider all three claims to be likely, and changing my mind regarding any of them would change my overall assessment of AI existential risk, so I decided to evaluate each counterargument carefully for a potential change of opinion.

A. Superhuman AI systems will be "goal-directed"

Contra A, Katja first argues that there are several different concepts of "goal-directedness", and that AI's may have weaker version of rationality/goals that are still economically useful, but without the "zealotry" of utility-function maximization. Here is a future she envisions:

It often looks like AI systems are ‘trying’ to do things, but there’s no reason to think that they are enacting a rational and consistent plan, and they rarely do anything shocking or galaxy-brained.

While I agree with the possibility of such pseudo-agents, they seem like the sort of first-generation AGIs developed by humans, and not like the superintelligent AIs that will be developed through intelligent self-modification.

Katja mentions the coherence arguments as a "force" in the direction of utility maximization, but she argues this may not be enough to get there:

Starting from a state of arbitrary incoherence and moving iteratively in one of many pro-coherence directions produced by whatever whacky mind you currently have isn’t obviously guaranteed to increasingly approximate maximization of some sensical utility function.

I am not very convinced by this argument, and I hold the position that superintelligent AIs will be rational (as in the utility-maximization sense). The superintelligent AIs I am worried about are the ones who got where they are through intelligent self-modification. I don't think we should try to look at each individual self-modification step and trying to determine whether they make the system significantly "more" utility-maximizing, because the AI we care about has the potential to make many such steps very quickly.

So either the AI is randomly modifying itself in a c...
Building the Loft Beds
by jefftk — 2022-10-16T01:10:01.513Z

A few weeks ago I posted

some plans
for

kids' loft beds; they're now
built! Not exactly following the plans, but that's ok. Pictures:

The main change from the plans is that I used less wood and more
metal: instead of a three-way box
joint I used (thanks
Emery!) a tie
corner:

And instead of a complex arrangement of
wooden angles I used large L-brackets:

And another on the inside of each:

I stuck with the plywood base as originally planned instead of
switching to bed slats: it's only slightly more expensive and makes it
far sturdier. Both beds feel as solid as part of the house.

The ladders are well-sanded 2x3s with 1 1/8" dowel rungs. I clamped
them together and drilled the holes with a spade bit. This would have
worked better with either a drill press or more care: a few of the
angles weren't quite right and Lily's came out warped after assembly.
I needed to cut an inch off the top of one side to get it to fit
properly:

In retrospect it would have been less visible to trim the bottom than
the top, though it was easier to get it right doing it this way.

I used bunk bed
ladder hooks to attach the ladders to the beds, and they're not
fixed to the floor. The post on Anna's bed isn't fixed either. I'd
rather not put holes in the floor!

The railings are a 2x3 frame with 1x2 furring strips for spindles.
They're glued and screwed at the top, and screwed at the bottom.
They're not as sturdy as the rest of the beds, but they're still good.

The kids are both very happy with their new beds. Anna was excited
enough that even though her new twin mattress isn't here yet she
wanted me to move her crib-sized mattress up so she could start using
the loft right away:

Lily's loft is a bit longer than a twin bed, so I boxed in the area at
the end to make a shelf:

I ran a power strip up to it so she can plug in her clock and tablet:

Anna's shelf is still in progress; hers will be wall-mounted near the foot.

Comment via: facebook
Best resource to go from "typical smart tech-savvy person" to "person who gets AGI risk urgency"?
by Liron — 2022-10-15T22:26:52.377Z

Background

Most tech-savvy people today see AI as a cool tech trend like the internet / smartphones / crypto / virtual reality. The claim "AGI is pretty likely to kill you and your loved ones in your lifetime" has never been a belief they take seriously, or think that many other people take seriously. They also don't intuitively perceive AGI as a risk that's relevant in their own lifetime; only as a low-probability long-term risk like an asteroid impact.

The Needed Resource

Something on the internet they can read/watch to try to guide such a person to the point where they realize maybe AI existential risk is a real urgent thing, at least as much so as climate change. Someone who gets that they might be ringing in the new decade as a nonhuman configuration of atoms.

Benefit of Having This

The benefit is that a significant fraction of the people who get AGI risk urgency will be motivated to do something helpful to the cause, much more so than the fraction of people who don't get AGI risk urgency.

Some Candidate Resources

Two off the top of my head:

[WaitButWhy] The AI Revolution: Our Immortality or Extinction

[LessWrong] AGI Ruin: A List of Lethalities

The WaitButWhy post does a great job of being generally accessible, while List of Lethalities packs in more of the arguments why the danger level seems high. It seems like there's a gap for a resource that's more suitable to the task of making a normal person realize this is a real and urgent thing, despite the social Overton Window not yet giving them that message.

If there's no good resource, I might be up for working on something myself. Maybe a good format would be similar to that of a technical documentation site, where it's kind of a linear walkthrough but also lets you skip around to sections you're interested in.
Bounded distrust or Bounded trust?
by M. Y. Zuo — 2022-10-15T16:41:22.085Z

There seems to be an upper limit to how much any randomly selected individual would trust another randomly selected individual. Real world observation of society shows that this upper limit varies between different societies and social/environmental conditions.

When the potential pool is narrowed down to the immediate family and closest friends, it becomes normal to claim the capacity for unlimited trust, however this does not seem credible when looking at the revealed preferences of large groups of middle aged individuals.

Therefore it stands to reason that the degree of mutual trust, between individuals, is bounded by some practical upper limit.

Conversely, there appears to be no lower limit, other than zero, for the degree of lack of trust. In fact this lower limit of zero is practically demonstrated in the real world via prisons that maintain solitary confinement in padded cells for their most troublesome prisoners.

Does this then imply that instead of speaking of a bounded distrust, which appears to not have any meaningful lower bound other than zero, we should speak of bounded trust?

For real world examples, the most useful would be those demonstrated in monetary terms, as other types of expression of trust are difficult to measure or compare on a YoY or cross-cultural basis.

For example,

There's the widely recognized notion of a 'handshake-deal' where significant transactions, in monetary terms, are settled via simple handshakes and word of mouth, as opposed to contracts. So among certain members of the population, the upper limit of trust can be quite high. The most extreme example would be of one famous individual, Warren Buffett, who regularly boasts of having conducted multi-billion dollar deals on a handshake.

However there are no cases, that I know of, for deals worth multiple tens-of-billions done on a handshake basis, even Warren Buffet gets lawyers and relies on contracts when his deals get that large. Which implies that the present upper limit of trust in human society, as valued in revealed monetary terms, would be <$10 billion USD.

Prenups are also very common among engaged couples, even of quite modest means, at least in the U.S., which suggests that the practical limit of mutual trust to be in range of a few millions of USD for the majority of the population, at least going by revealed preferences. (As it's unlikely that the ...