trentmkelly/LessWrong-43k huggingface dataset:
COVID Era: Updating On Life Risks
by jmh — 2021-11-29T17:27:28.354Z

I've been struggling in posing the questions/thoughts I am currently having an written, then deleted, a few versions of this question post. I've decided I will just keep it simple and just offer some illustrative examples that I hope extend into a more complex, realistic setting.

I was wondering about the impact of the pandemic on life expectancy. Looks like we've lost about 15 years of gains and the current estimates are about what were held in 2005. One might then ask "Why are we not living our lives as if it were 2005 rather than as we are now?" One answer might just be that our life risks are not what they were in 2005 -- they are MUCH worse so we have to take more extraordinary steps. The other might be that we're only able to use that 2005 baseline in the context of the current risk controls. But at some point we still need to accept that the risk profile to being alive now is different that it was prior to the pandemic and I don't think it's going back to what it was before. The virus is here to stay as part of the territory but it seems like many want to apply maps that are more fitting to the pre-pandemic territory.

To put this in a slightly different light, consider motor vehicle transportation. We know it kills some people. It seems we have some risk control social rules but they seem rather light and in many ways more about coordination than outright risk mitigation. They seem just different than how the risks from COVID are getting managed. That seems to be true of other disease risk mitigation compared to COVID as well. Mostly we seen to let people individually and privately (as opposed to large public policy) manage the risks while socially producing the infrastructure that deals with the risk events -- hospital emergency room, EMV response and the like -- and that is balanced with the volume of events occurring. Supply and Demand equilibration as it were. We don't seem to really be going in that direction though.

Even though it seems we're getting closer to entering the 3rd year of the pandemic the discussion seems to be on public policy on controlling the risk exposure by individuals and not on social infrastructure to dealing with the new, riskier world we're in.

Going back to the life expectancy is now 2005, what might be a reasonable live risk profile expectations to hold for the new COVID world and how does the way ou...
Snoo App Thoughts
by jefftk — 2021-11-29T16:40:05.361Z

With our

youngest child we got a
Snoo automated bassinet, and overall we've been

very happy with it. On
the other hand, the corresponding phone app has several missing
features. Since all of this is software, it should be fixable, but
the current state is a hassle.

When the baby is fussy, one of the first things I want to know is how
long they've been up: the longer they've been awake, the more likely
it is that they need sleep. If I open the app I see:

The number in the upper left-hand corner is how long their most recent
sleep was, which is a fine number to include, but much less useful
than how long they've been up. If I switch to the history tab I don't see
anything useful:

The problem is that it's morning, and the last sleep finished
"yesterday". Swipe back, and I can see the most recent waking is 8:25:

Now I can do some math in my head and see it's been almost an hour.
Instead, it would be great to have this number calculated
automatically, and made available on the first screen:

A related problem is that the information about soothing status is not
very useful. Ideally, it would detect whether the baby was awake or
asleep, perhaps with a camera, but since it only has a microphone to
work with I understand how it can't do that. Instead, they mark
"soothing" in red and "baseline" in blue. Once the baby falls asleep,
however, it takes quite a long time for the Snoo to gradually work
it's way back down to baseline. Manually triggering the highest
setting (while empty) and waiting for it to return to baseline, it
took 21 minutes. The whole time it said the baby was "calm", but
still marked the time as red on the chart and didn't count it towards
daily sleeping time:

I would find the chart and timings much more useful if it colored by
whether the baby was calm (whether it had recently detected crying)
than by whether it is still in soothing mode (responding to earlier crying).

Then there's a layout issue, worse in combination with the two above,
which means you can't even always tell when the most recent nap ended:

The problem is that if it labeled all three of the nap start, soothing
end, and nap end, the labels would overlap. They could do something
fancier with their layout, where the line from the timestamp to the
event doesn't have to be horizontal, or they could make it so that
when you...
Omicron Variant Post #2
by Zvi — 2021-11-29T16:30:01.368Z

It’s now been three days since Post #1. The situation is evolving rapidly, so it’s time to check in. What have we learned since then? How should we update our beliefs and world models? There will inevitably be mistakes as we move quickly under uncertainty, but that’s no reason not to do the best we can.

Update Update

What should we look for here and in the coming days?

No news is good news.Omicron is scary because many scary things are possible. The worse things are going to get, the sooner they will make themselves known. When we get news that something has happened, especially news that isn’t the result of a lab or statistical analysis, that will typically be bad news, but we expect a certain rate of such bad news. If we get less than expected, that’s good news.
The pattern of where and how we find cases, and the details of those cases, will give us better insight into how widely and quickly, and in which directions, Omicron has spread. This will tell us how far along things are, and give a better estimate and narrower range of how infectious Omicron can realistically be.
Information on how deadly Omicron is should update us quickly, and matters a ton for how to react, but beware of confounding factors, motivated statements and misunderstandings in both directions, and small sample sizes. Hospitalizations are not a natural category and at this stage deaths will be rare unless things are much worse than we expect. How mild the cases we find are will largely depend on how we are testing. Details matter.
The reaction of various countries, the stock market and others will continue to tell us what they think is going on, what they expect to happen next, and how they respond when such things happen. The general vibe of elite and media messaging also reveals much information, with a focus on what kinds of actions they will try to engineer rather than the ‘facts on the ground.’
Information on immune erosion will come from results from laboratory analysis and other looks at Omicron’s mutations, and other physical detail information, such as testing antibodies, and from statistics on who gets infected or sick once we have enough cases for that to mean something. It also comes from knowing how infectious Omicron is, since if we know how fast it’s spreading and we know how fast it spreads to the immunologically naive, we also then know how immune erosive it i

...
Comments on Allan Dafoe on AI Governance
by Alex Flint — 2021-11-29T16:16:03.482Z

Financial status: This is independent research, now supported by a grant.

Epistemic status: Views here are almost entirely my own.

There are some think pieces that lay out a bunch of perspectives with which we might think about a thing. This can be either terrible or excellent. At the terrible end, there are certain philosophical styles that describe endless possible views that one might take without actually saying anything real. One can draw out endless matrices of possible conjunctions and give names to them and describe their implications without actually making a point. But on the excellent end, when we have been using one perspective for a long time and have started taking it for granted then it can be helpful to give a name to that perspective and to think through some other perspectives, if only so that we can be sure that we are using this particular perspective for good reasons rather than out of habit.

And sometimes there really are good reasons to use a single perspective over and over. A physicist might model interactions between billiard balls using the perspective of elastic collisions. This might well be an excellent perspective to use, and the physicist might know this, and might keep choosing to use the same perspective over and over for well-calibrated reasons. Or one might choose to view the real-world phenomenon of machines that maintain credences in their beliefs about the world through the perspective of probability theory. For many jobs, probability theory is really an excellent choice of perspective, and it might be fine to use it over and over, especially if that choice is made in awareness that it is, in fact, a choice.

In AI Governance: Opportunity and Theory of Impact, Allan Dafoe asks how we should understand what AI actually is in a way that is conducive to effective oversight by our institutions. Dafoe describes the "Superintelligence perspective" like this:

Many longtermists come to the field of AI Governance from what we can call the superintelligence perspective, which typically focuses on the challenge of having an AI agent with cognitive capabilities vastly superior to those of humans. Given how important intelligence is---to the solving of our global problems, to the production and allocation of wealth, and to military power---this perspective makes clear that superintelligent AI would pose profound opportunities and risk

...
How to measure FLOP/s for Neural Networks empirically?
by Marius Hobbhahn — 2021-11-29T15:18:06.999Z

Experiments and text by Marius Hobbhahn. I would like to thank Jaime Sevilla, Jean-Stanislas Denain, Tamay Besiroglu, Lennart Heim, and Anson Ho for their feedback and support.

Summary:

We measure the utilization rate of a Tesla P100 GPU for training different ML models. Most architectures and methods result in a utilization rate between 0.3 and 0.75. However, two architectures result in implausible low utilization rates of lower than 0.04. The most probable explanation for these outliers is that FLOP for inverted bottleneck layers are not counted correctly by the profiler. In general, the profiler we use shows signs of under- and overcounting and there is a possibility we made errors.

Findings:

Counting the FLOP for a forward pass is very simple and many different packages give correct answers.
Counting the FLOP for the backward pass is harder and our estimator of choice makes weird overcounting and undercounting errors.
After cleaning mistakes, it is very likely that the backward/forward ratio is 2:1 (at least for our setup).
After correcting for the overcounting issues, we get empirical utilization rates between 0.3 and 0.75 for most architectures. Theoretical predictions and empirical measurements seem very consistent for larger batch sizes.

Estimated GPU utilization rates on different architectures, using four different estimation setups.

Introduction

In the “Parameter, Compute and Data Trends in Machine Learning" project we wanted to estimate GPU utilization rates for different Neural Networks and GPUs. While this sounds very easy in theory, it turned out to be hard in practice.

Utilization rate = empirical performance / peak performance

The post contains a lot of technical jargon. If you are just here for the results, skip to the Analysis section.

I don’t have any prior experience in estimating FLOP. It is very possible that I made rookie mistakes. Help and suggestions are appreciated.

Other work on computing and measuring FLOP can be found in Lennart Heim's sequences Transformative AI and Compute. It's really good.

Methods for counting FLOP

In this post, we use FLOP to denote floating-point operations and FLOP/s to mean FLOP per second.

We can look up the peak FLOP/s performance of any GPU by checking its datasheet (see e.g.

NVIDIA’s Tesla P100

). To compare our empirical performance to the theoretical maximum, we need to measure th...
Watching Myself Program
by matto — 2021-11-29T13:44:18.268Z

Michael Malis described an interesting technique for improving his software development workflow:

One incredibly useful exercise I’ve found is to watch myself program. Throughout the week, I have a program running in the background that records my screen. At the end of the week, I’ll watch a few segments from the previous week. Usually I will watch the times that felt like it took a lot longer to complete some task than it should have. While watching them, I’ll pay attention to specifically where the time went and figure out what I could have done better. When I first did this, I was really surprised at where all of my time was going.

(It's from his How to Improve Your Productivity as a Working Programmer, which I found through Dan Luu's 95%-ile isn't that good, itself an excellent description of how getting better at small things compounds into larger improvements).

I've read that some people who play sports or games record themselves to find opportunities for improvement. But it never crossed my mind that the same could be applied to programming. After all, sports and games are designed around making players' performance legible and comparable, and in contrast, programmers' productivity is notoriously difficult to measure. My gut feeling was that something like this could never work.

But what if it could? What if it really did bestow the user with productivity gainz?

Well, I could meditate on this and do some research. But being a programmer, I could also throw some code together and experiment myself, which would be interesting and fun.

So I wrote a quick and dirty script that captures a screenshot of my screen(s) every 30 seconds and, on the next day, compiles all the screenshots into an mp4 video. (Here's the source code - it should work on any *nix system that has ffmpeg and scrot installed).

After a few days, I settled on reviewing 1-2 videos from the work week, focusing on days that felt particularly good or bad or interesting. This allowed me to avoid watching myself type Slack messages or check the weather and instead focus on periods of important work. (It just occurred to me that I should occasionally review a random day just to make sure I can notice patterns I'm not explicitly looking for). It would take some short notes on anything that stood out to me.

Here's what I found from a handful of reviews after about three weeks:

I switch to Slack every

...
Question/Issue with the 5/10 Problem
by acgt — 2021-11-29T10:45:12.845Z

I'm not sure if the 5/10 problem and the surrounding Löbian uncertainty is still an issue/area of research, but I've been struggling with the validity of this argument lately - I doubt this is a novel point so if this is addressed somewhere I'd appreciate being corrected. On the off chance that this hasn't been explained elsewhere I'd be really interested to hear peoples' thoughts

The proof as I understand it is roughly as below, with "A" referring to the agent's output, "U" referring to the environment's output, and "□" standing for the provability predicate:

(A = 5) →(U= 5)
(A = 5) →((A = 10)→(U=0))
□(((A = 5) → (U = 5) ∧ ((A = 10) → (U = 0))) → (A = 5)
□(((A = 5) → (U = 5) ∧ ((A = 10) → (U = 0))) → ((A = 5) → (U = 5) ∧ ((A = 10) → (U = 0))
((A = 5) → (U = 5)) ∧ ((A = 10) → (U = 0))
A = 5

(I have probably butchered the parentheses sorry, I'm on mobile)

The key to this argument seems to be step 3, which is the step that seems flawed to me. Informally this is stating "If I prove that choosing $5 leads to $5 and choosing $10 leads to $0, then I will choose $5". This seems like a reasonable claim and is what allows the vacuously true conditionals that cause the whole Löbian troubles, but I don't see how an agent will be able to prove this. Step 3 seems like it can be interpreted in 2 ways, neither of which appear to work:

This seems to me to be the more natural interpretation. The agent can prove (3.) because it can prove of itself that, when it finds a proof showing that it gains more utility on one path, it will halt its proof search and take the higher-utility path. So the agent reasons that, if "((A = 5) → (U = 5)) ∧ ((A = 10) → (U = 0))" is provable then it will eventually find this proof, halt, and output $5. But this halting aspect introduces a difficulty in that the agent must prove not only that the above is provable, but that no contradictory proof will be found earlier in the proof search. Even if the above is provable, if there is a proof earlier in the ordering of the statement "((A = 5) → (U = 0)) ∧ ((A = 10) → (U = 10))" (or any other statement of this form where $10 is valued higher than $5) then (3.) is false. The agent needs to be able to prove not only that the proof exists but that it will find this proof before it proves something else that halts it looking at proofs. The obvious way to guarantee this won't happen is for the agent to prove that if i

...
Redwood Research is hiring for several roles
by Jack R — 2021-11-29T00:16:32.650Z

Redwood Research

does applied AI alignment research. We’re led by

Nate Thomas

(CEO),

Buck Shlegeris

(CTO), and

Bill Zito

(COO), and our board is Holden Karnofsky, Paul Christiano, and Nate Thomas. To learn more about our current project, you can read

a project update from September

or Paul’s “

Why I’m excited about Redwood Research’s current project

.”

In addition to our primary work of applied AI alignment research, Redwood hosts and operates Constellation, a shared office space we created to foster knowledge-sharing and collaboration between longtermist organizations. People from the following organizations often work from the space: Alignment Research Center, MIRI, and Open Philanthropy.

We’re hiring for several roles, listed below in order of which roles we’re excited to get more applications for right now (subject to change within a month):

Redwood / Constellation Generalist

Research and Engineering Staff

Constellation Operations Manager

Constellation Member Experience Manager

Constellation Sous Chef

If you have any questions, feel free to email Bill (

bill@rdwrs.com

) about generalist/ops roles or Buck (buck@rdwrs.com) about technical roles.

Below is a picture taken from the office -- we have beautiful views in all directions!
Compute Research Questions and Metrics - Transformative AI and Compute [4/4]
by lennart — 2021-11-28T22:49:56.591Z

Cross-posted here on the EA Forum.

Transformative AI and Compute - A holistic approach - Part 4 out of 4

This is the Appendix (part four) of the series Transformative AI and Compute - A holistic approach. You can find the sequence here and the summary here.
This appendix attempts to:

Provide a list of connected research questions (Appendix A).

Present common compute metrics and discusses their caveats (Appendix B).

Provide a list of Startups in the AI Hardware domain (Appendix C).

Previous Post: Compute Governance and Conclusions

You can find the previous post "Compute Governance and Conclusions [3/4]" here.

Appendix

This appendix lists research questions, some thoughts on metrics related to compute, and a list of AI hardware startups.

These paragraphs ended up in the Appendix, as I have spent less time on them and it is more of a draft. Nonetheless, I still think they can be potentially useful.

A. Research Questions

This is not a research agenda or anything close to it. This is me doing research and thinking “Oh, I’d be curious to learn more about this *scribbling it down*”.

The questions rank from specific to nonspecific, from important to “I’m just curious”. There are a couple of questions which I think are more important, I’ve marked them with a ★.

This list is also published as a Google Doc, so one can add comments to individual items.

In general, this domain of research requires more:

Data Acquisition: Collect data in a machine-readable format on compute trends, hardware performances, algorithmic efficiency,

Parameter, Compute and Data Trends in Machine Learning (Sevilla et al. 2021) is such an example. Work with us!

Data Analysis: Data analysis is a crucial ingredient: from trend graphs and doubling times up to new measurements of AI systems’ capabilities and AI hardware.

Data Interpretation: Once we have available data and analyzed general trends, we require interpretation for informing forecasts and potential policies.

Figure A.1: Sketch of research domains for AI and Compute.

Compute Governance

For a list of research questions, see “Some AI Governance Research Ideas” (Anderljung and Carlier 2021).

What is the current budget of public research organizations for compute?

Should we lobby for an increase in compute budget?

If yes, how?

How does this compare to corporate AI research labs?

Scaling Hypothesis

GPT-3 is a leading language model i

...
Anthropics and the Universal Distribution
by Joe Carlsmith — 2021-11-28T20:35:06.737Z

(Cross-posted from Hands and Cities. Content warning: especially niche topic.)

Some readers of my recent sequence on anthropics suggested that I consider an approach that they find especially plausible — namely, UDASSA (or the “Universal Distribution” plus the “Absolute Self-Sampling Assumption”). So, partly on this prompting, and partly from pre-existing interest, I spent some time learning about UDASSA, and talking to people who like it.

What is UDASSA? Roughly, UDASSA is anthropics for people who really like the Universal Distribution (UD), which is a way of assigning prior probabilities to hypotheses (see my previous post for details). UDASSA’s hope is that, having discovered the UD, it has done a lot to solve anthropics, too: all you need to do is to apply the UD (plus some Bayesianism) to hypotheses about who you are (and in particular, which observer-moment you are).

In practice, and granted some further assumptions (what I call the “world and claw assumptions”), this means that UDASSA expects to find itself in “simpler” combination of (a) worlds, and (b) “locations” in a world (where the relevant notion of “location” is importantly up for grabs, and itself biased towards simplicity). We can think of this move as an extension of Occam’s Razor from objective worlds to centered worlds: “you know how you wanted to weight hypotheses about the world according to some notion of simplicity? Well, I want to do that with hypotheses about who I am, too.”

Is UDASSA a good approach? I think it’s important to distinguish between two claims:

The UD is cool.
The UD helps a lot with anthropics.

As I indicated in my previous post, I’m open to (1): I think the UD might well be cool, despite various issues (e.g., uncomputability, the arbitrary choice of UTM, ruling out uncomputable worlds, and so on).

But I’m more skeptical of (2). Granted the world and claw assumptions, I think the UD helps with some problems in anthropics: in particular, relative to SIA and SSA, it seems better positioned – at least in principle — to deal with infinities, and with cases that make it unclear how to count observers (some other candidate pluses, like “there are limits to my dogmatism,” seem to me less exciting).

But these benefits come with serious costs. Applying the UD to anthropics leads to unappealing combinations of: (a) extreme disconnection from normal patterns of reasoning (and/or ethical...