trentmkelly/LessWrong-43k huggingface dataset:
How feeling more secure feels different than I expected
by Kaj_Sotala — 2021-09-17T09:20:05.294Z

This year, I’ve been feeling more emotionally secure, thanks to various kinds of internal and external work (the stuff at https://attachmentrepair.com/ being one notable example).

The effect of this doesn’t always feel like I expected it to feel.

I once thought that in order to not worry so much about whether people like me, I would need to become convinced that they do like me. Or at least I’d need to convince myself that at least some people like me, strongly enough that it would reassure me in general.

That does play a certain role. I probably wouldn’t feel the way I do now, if I hadn’t managed to internalize evidence about some people actually liking me quite a bit. And if I ended up in a conversation where it was obvious that someone hated me, yeah, that wouldn’t be fun.

But my actual internal experience of feeling more secure in interacting with other people, isn’t necessarily that I put a higher probability on the other person liking me than I did before. Often it’s more like noticing that thinking about how much the other person likes me, isn’t a particularly rewarding frame of mind to be in. And that it’s possible to sometimes at least drop that frame and let a more rewarding frame arise.

If I had to verbalize it, there’s sometimes a shift in stances that’s accompanied by a thought that goes along the lines of “it’s possible that I’m secretly annoying this person and they totally hate me without telling me that and I’m doing nothing to deny that possibility, but I’m going to get more out of this interaction if I just focus on something more rewarding – such as the general flow of this conversation – unless I get a clear indication that I’m doing something wrong”.

Except that that’s not quite right, because what I do then isn’t me trying to focus on something more rewarding. Nor is it an attempt to suppress the analysis about what the other person’s opinion about me is. Rather it’s just a remembering to inhibit the part of me that’s about to start focusing on that analysis, and then letting something else arise from that space on its own.

And that’s becoming more automatic, so that I don’t necessarily even need to do that anymore. If the thought of “it’s possible that this person secretly hates me” crosses my mind at all, it may do so very quickly and then be gone.

(I originally wrote large parts of this a month ago, for the forum of Michael Ashcroft’s Alexa...
Research speedruns
by drossbucket — 2021-09-17T06:54:12.663Z

The 'research speedrun' is a format that I've been playing with on my blog for the last year or so. It's been more popular than I expected and it looks like there's a lot more that could be done with the idea. So I thought I'd write it up here and see if anyone else wants to experiment with it themselves, or suggest different things to try.

The format

It's a very simple format, so this section will be short:

Pick a topic

Set a one hour timer

Find out as much as possible about the topic before the buzzer goes off while writing up a live commentary

Do a very quick editing pass to fix the worst typos and then hit Publish

So far I've done speedruns on Marx on alienation, the Vygotsky Circle, sensemaking, the Prussian education system, abacus schools, Germaine de Staël, and mess.

What I've used it for so far

Obviously, there's only so much you can learn in an hour - calling this 'research' is a little bit of a stretch. Sometimes I don't even manage to leave Wikipedia! Even so, this technique works well for topics where the counterfactual is 'I don't read anything at all' or 'I google around aimlessly for half an hour and then forget it all'. Writing notes as I go means that I'm making enough active effort that I end up remembering some of it, but I know the process is timeboxed so it's not going to end up being one of those annoying ever-expanding writing projects.

Here are a few rough categories of topics I've tried so far:

'Sidequests'. Speedruns are great for topics that you find interesting but are never going to devote serious time to. I have a very minor side interest in the history of schools and universities, so if I come across something intriguing, like Renaissance abacus schools, it's a good way to learn a few basic things quickly. I have one or two more ideas for speedruns in this area.

Historical background. An hour is quite a good length of time to pick up a few fragments of background historical context for something you're interested in. One hour won't get you far on its own, but the good thing about historical context is that it builds nicely over time as you get a better picture of the timeline of different events and how they affect each other.

Finding out what something is at a basic level. I did the 'sensemaking' speedrun because I'd heard that term a lot and had very little idea what it referred to.

Dubious or simplistic claims. The

...
Lessons from covid predictions: Always use multiple models
by pku — 2021-09-17T03:19:29.451Z

Epistemic status: This point isn't novel (it's one of the the ten commandments of superforecasting, more or less), but it's often underappreciated. Crossposted here.

I went out of my way to make two explicit predictions about covid over the course of the pandemic. The first, back around January 2020, was a vague claim that internet people's worrying about the pandemic was probably overblown and it'd probably level off before it got global. The second one (dated March 12 2021) was more explicit:

Israel opened vaccines up to general availability at about 0.7 SPP (Shots Per Person). Biden wants to open them on May 1st, which will be just about 0.7SPP at current rates.
Overall, this seems pessimistic (we should expect more acceleration by then), so it does seem to be more of Biden's "underpromise and overdeliver" media strategy. But this one's likely to not be too far off, since there's much less time for acceleration now.
(A few complications: The US vaccinated slower, so you might need a higher threshold for demand shock. But also US rules are more convoluted, so you'd expect demand shock to hit in earlier - but the more convoluted rules also mean there's more people in the various priority groups, so less pressure to open up to general availability. Overall I think these are mostly a wash).

In the spirit of testable predictions: My 90% confidence interval for the median (that is, 25th/50) state making vaccines generally available is 4/17-5/2.

My reasoning behind the first prediction was that we'd had pandemic scares every few years for a while - swine flu, SARS 1, Ebola, Zika, and they'd all fizzled out. So there's an argument that your prior should be that most pandemic scares fizzle out.

The first prediction, obviously, was wrong. The second was technically correct (the actual date of the median state making vaccines available was 4/19), but just barely, and thanks to an unprincipled extension of my error bounds (I'd run a spreadsheet with a few different scenarios for the rate of vaccine acceleration, then added a few days to each side to get my interval. The US was nowhere at only 0.63 SPP at the time.) I'd give myself a 0.5/2 here, which isn't a great track record.

Where did I go wrong?

My biggest mistake was only having one model for each prediction. With the first one, I didn't even have a confidence interval. With the second one, I acco...
Let Us Do Our Work As Well
by jsteinhardt — 2021-09-17T00:40:18.443Z

As each new writing deadline approaches, I return to The Builders, a poem by Longfellow:

All are architects of Fate,

Working in these walls of Time;

Some with massive deeds and great,

Some with ornaments of rhyme.

Nothing useless is, or low;

Each thing in its place is best;

And what seems but idle show

Strengthens and supports the rest.

For the structure that we raise,

Time is with materials filled;

Our to-days and yesterdays

Are the blocks with which we build.

Truly shape and fashion these;

Leave no yawning gaps between;

Think not, because no man sees,

Such things will remain unseen.

In the elder days of Art,

Builders wrought with greatest care

Each minute and unseen part;

For the Gods see everywhere.

Let us do our work as well,

Both the unseen and the seen;

Make the house, where Gods may dwell,

Beautiful, entire, and clean.

Else our lives are incomplete,

Standing in these walls of Time,

Broken stairways, where the feet

Stumble as they seek to climb.

Build to-day, then, strong and sure,

With a firm and ample base;

And ascending and secure

Shall to-morrow find its place.

Thus alone can we attain

To those turrets, where the eye

Sees the world as one vast plain,

And one boundless reach of sky.

Each year the exhortation to Let us do our work as well becomes harder to follow, both because the pace of ML keeps increasing, and because more students show up to my lab so my attention is split between more projects. This creates pressure to triage and to leave some broken stairways as they are.

I'm sure that I inevitably do leave some stairways broken--questioning experiments in less detail than I would have as a PhD student, or leaving weak or ambiguous arguments in a paper because we ran out of time before the deadline. That being said, I strongly desire to build a culture that is worthy of the elder days of Art.

Is it possible to do this without permanently sacrificing scalability? I think yes, for the following reasons:

Standards propagate into culture. If you take the time to teach high standards to junior students, they will eventually become senior students and pass those standards down to the next generation. So, mending broken stairways is an investment that lets newcomers climb higher faster.

Scaling through processes. As a manager, catching bugs by exhaustively examining each experiment is not only time-consuming, it's also error

...
Writing On The Pareto Frontier
by johnswentworth — 2021-09-17T00:05:32.310Z

I have a personal rule: don’t write something which someone else has already written better.

This is easier than it sounds. For instance, suppose I’m writing an intro to systems biology. I don’t need it to be the most comprehensive intro ever written, or the most accessible intro. I just need it to be good enough on each of these axes that no other work is better on both at once.

In other words, I try to always write things which are the

Pareto Best In The World

.

Of course this generalizes to more dimensions as well: I might also care about writing something rigorous, or communicating intuitions, or making the piece enjoyable to read, or including good visualizations, or …. I don’t need to write the best piece in the world along any particular dimension. I just need to write a piece good enough on enough dimensions that nothing else beats it on every dimension which I care about.

Ways To Be On The Pareto Frontier

Novel Ideas

One natural way to be on the Pareto frontier is to write about a new idea, or at least an idea unusual enough that few people have written about it.

As with writing, new ideas are not necessarily that difficult to find. The trick is to combine things: finding a novel and interesting idea in complexity theory is a lot harder than finding a novel and interesting application of complexity theory to caricature art. On the LessWrong frontpage right now, there’s an

interesting post

about applying the idea of film study (i.e. athletes watching films of their games) to become better at research. I’ve been writing a

whole series of posts

intended to practice applying frames from math in unusual ways, and some interesting novel ideas have already come out of them - e.g.

optimizing multiple imperfect filters

.

Distillation

Research goes through a pipeline. First, researchers write papers, packed with jargon and assuming lots of background context. As results come together, they get boiled down into overview papers. Then come textbooks and programming libraries. Eventually, there might even be courses.

At each of those steps, work is done to distill the results - to explain them, draw analogies, add visuals, suggest exercises and applications, etc. In general, there’s a Pareto frontier with new cutting-edge results along one axis, and well-distilled results along the other. Pushing that Pareto frontier outward means finding some result which hasn’t been explai...
Jitters No Evidence of Stupidity in RL
by 1a3orn — 2021-09-16T22:43:57.972Z

Epistemic status: Pretty sure about core, not about edges

A while ago, I noticed a possible bias in how I evaluated reinforcement learning agents. It tended to cause me to revise my estimation of their intelligence downwards, after I viewed a video of them in action.

I've seen other people fall into what I believe to be the same error. So I'm writing this to correct myself if I am wrong and to alert others if I am right.

The Bias

Many reinforcement learning agents have "jitters." They alternate actions quickly, looking nearly palsied, apparently nullifying the effects of earlier actions with later ones. This is true across a wide variety of reinforcement learning agents.

Many people see these jitters as evidence of the relatively primitive nature of these agents. These actions look clearly stupid and sub-optimal.

For instance consider the original Deep Q Network paper. Even after training for some time on Breakout, it still erratically moves the paddle back and forth when the ball is not near it. One person mentions that it makes "erratic jerky movements that obviously could not in principle be optimal," which was once my impression as well.

Similarly, much more recently, consider DeepMind's recent work on generally capable agents. In the show reel the movement of the agents often looks erratic. Conversation around LessWrong sometimes alluded to these erratic movements as evidence against the intelligence of the agents.

Jitters Non-Optimal For Energy-Constrained Agents

Evolved intelligence on earth has energy conservation as a fundamental part of its optimization function.

Unnecessary movements spend energy. Spent energy must be recovered, at the cost of reproductive fitness. So generally only sick animals, insane animals, and so on, have the shakes or tremble continuously. Energy conservation applies to every animal on earth, which is why we probably feel intuitively confident applying this rule across the broad variety of animals.

Additionally, extremely erratic movements can result in injury to the animal which is making them. So this is another reason why, for creatures that are a result of evolution, erratic movements are a sign of insanity or injury.

RL Agents Are Not Energy-Constrained

Reinforcement learning agents are not energy-constrained. They do not draw on a finite store of glucose when acting. Nor do they have any possibility of injuring themselves. A...
Comments on Jacob Falkovich on loneliness
by Alex Flint — 2021-09-16T22:04:58.773Z

Two years ago, Jacob wrote an essay about dating, sex, and loneliness. I found this essay recently because I am extremely interested in loneliness. It seems to me that a great deal of work is needed to resolve existential risk, and that loneliness is one of the top practical impediments to that work. I think that a good understanding and significant resolution to loneliness would be of enormous value to this community.

And so with that purpose in mind, I discovered Jacob’s essay. Jacob’s basic point, as I understand it, is that a trade-off exists between spending time opening to connection via self-work, and spending time opening to connection via relating. He sees a lot of his friends trying to open to connection via self-work alone, and in his essay he exhorts us to mix in some relating, too.

Jacob actually talks mostly about dating, not relating, but I do not think he would disagree with my re-characterization in terms of relating because he writes:

This essay is to tell you: [...] ask someone on a date. At the very least, invite someone to hang out and ask them what they’re struggling with.

Jacob’s essay is tenacious and charged, which I appreciate very much, but I am aiming for a different kind of charm in this post, so I will cut out some of the more charged language when I quote him, and I will indicate changes with square brackets. I promise to do my best not to skew the meaning of anything with my cuts.

Jacob's essay is at times harsh, at times gentle, at times crude, and at times sweet. All of these provoked things in me, which I appreciated very much. I have therefore found it most helpful to write this essay by simply quoting Jacob in full, and adding commentary. What follows is most of the original essay, with my replies inline. Jacob is in blockquotes, I am not.

Commentary

From Tokyo to TriBeCa, people are increasingly alone. People go on fewer dates, marry less and later, have smaller families if at all. People are having less sex, especially young people. The common complaint: it’s just too hard. Dating is hard, intimacy is hard, relationships are hard. I’m not ready to play on hard mode yet, I’ll do the relationship thing when I level up.

Jacob says that loneliness exists in the world, and is getting worse. Right in the second sentence he moves from loneliness to dating, marriage, and sex. I think this is significant. It is extremely common...
Economic AI Safety
by jsteinhardt — 2021-09-16T20:50:50.335Z

Crossposted from my blog, Bounded Regret.

There is a growing fear that algorithmic recommender systems, such as Facebook, Youtube, Netflix, and Amazon, are having negative effects on society, for instance by manipulating users into behaviors that they wouldn’t endorse (e.g. getting them addicted to feeds, leading them to form polarized opinions, recommending false but convincing content).

Some common responses to this fear are to advocate for privacy or to ask that users have greater agency over what they see. I argue below that neither of these will solve the problem, and that the problem may get far worse in the future. I then speculate on alternative solutions based on audits and “information marketplaces”, and discuss their limitations.

Existing discourse often mistakenly focuses on individual decisions made by individual users. For instance, it is often argued that if a company is using private data to manipulate a user, that user should have the right to ban the company’s use of the data, thereby stopping the manipulation. The problem is that the user benefits greatly from the company using their private data---Facebook’s recommendations, without use of private information, would be mostly useless. You can see this for yourself by visiting Youtube in an incognito tab. Most users will not be willing to use severely hobbled products for the sake of privacy (and note that privacy is not itself a guaranteed defense against manipulation).

A second proposal is to provide users with greater agency. If instead of passively accepting recommendations, we can control what the algorithm shows us (perhaps by changing settings, or by having recommendations themselves come with alternatives), then we can eventually bend it to satisfy our long-term preferences. The problems here are two-fold. First, even given the option, users rarely customize their experience; a paper on Netflix’s recommender system

asserts

:

Good businesses pay attention to what their customers have to say. But what customers ask for (as much choice as possible, comprehensive search and navigation tools, and more) and what actually works (a few compelling choices simply presented) are very different.

Thus to the extent that user agency is helpful, it would be from providing information about a small and atypical subset of power users, which must then be extrapolated to other users. In addition, even those...
The Metaethics and Normative Ethics of AGI Value Alignment: Many Questions, Some Implications
by Eleos Arete Citrini — 2021-09-16T16:13:38.510Z

(first published on the EA Forum)

Short Summary

This is a commentated research agenda on the metaethics and normative ethics of AGI (artificial general intelligence) value alignment. It aims to show that various crucial aspects of this interdisciplinary Herculean task hinge on
1) numerous metaethical (and metaethics-related) questions;
2) on the moral implications of different answers to these questions;
3) on how relevant actors end up deciding to take the uncertainty surrounding these (meta)ethical questions into account.

My key takeaways (for the short summary):
1) At least some of the questions about metaethics and normative ethics briefly discussed in the six main parts and listed in the two appendices merit further attention.
2) Let us take seriously Nick Bostrom’s verdict that we have to do philosophy with a deadline.
3) Let us get a better, more exhaustive understanding of what work on AGI value alignment currently does involve, and what it could involve, and what it should involve, as well as the most decision-relevant uncertainties, and how to mitigate these uncertainties, and how to handle the remainder.
4) Let us give not only the intradisciplinary work but also the interdisciplinary work that the challenge of AGI value alignment requires its due attention.

Longer Summary

(I assume reading the short summary preceded starting to read this sentence.)

Artificial general intelligence (AGI) is a form of artificial intelligence exhibiting the capacity to apply its problem-solving skills in a broadly human-comparable cross-domain fashion (as opposed to its intelligence being forever narrowly restricted to a given set of tasks).

Since AGI, although currently hypothetical, is thought of by most pertinent researchers as “eventually realised” instead of “forever mere sci-fi”, there is ample need to research and discuss the implications this emerging technology might have. One of the most important questions arising in this context is how to go about AGI value alignment. This is among the most prominent interventions for increasing AGI safety and involves researching how to make AGI(s) “aligned to humans by sharing our values”. Incidentally: What exactly this even means gives rise to foundational questions that are far from settled.

There is a beautiful Value Alignment Research Landscape map by the Future of Life Institute giving an impressive overview of th...