trentmkelly/LessWrong-43k huggingface dataset:
Covid 9/9: Passing the Peak
by Zvi — 2021-09-09T16:10:01.451Z

Labor day muddies the data a bit, but it seems that Alex Tabarrok was correct and the current wave has peaked. We could well be facing another peak in December due to seasonality. We might also have issues from schools, although as you’ll see later they’re taking extreme precautions and also we didn’t see any sign of a school effect last year.

The primary question now is how and if we return to normality. It’s no longer a question of when. We’re going to be dealing with a substantial amount of Covid for quite a while, and a large number of unvaccinated people for quite a while, and our lives are ending one minute at a time. Whatever we are going to do to return to normal life, we need to start doing it, and if we’re not doing it, accept that actually we are and whatever we are doing is now normal. Either reclaim your life and the things that bring you joy, or accept you’re not getting them back.

Note: This week’s post was written using LessWrong’s editor. Hopefully this solves the issue people were reporting with being unable to load images. Be quick to point out any remaining issues. I’m working on being able to paste Excel charts properly with the new tech stack.

Executive Summary

Top points this week:

Covid case numbers peak.
Australia’s dystopian nightmare deepens, hopefully we can avoid this.
Schools are acting crazy in name of prevention.

Let’s run the numbers.

The Numbers

Predictions

Once again, I forgot about an upcoming holiday, in this case Labor Day, and its tendency to screw up reporting. I think it’s because holidays don’t seem ‘real’ to me as I’ve been working from home on my own schedule most of my life.

Prediction from last week: 1.1mm cases (+5%) and 11,150 deaths (+20%).

Result: 940k cases (-9%) and 10,272 deaths (+10%).
My guess is that the 10% drop on deaths is a reporting issue, and thus the cases also have a similar reporting issue and were about flat. This is still good news, but we shouldn’t expect a bigger drop next week or anything like that.

Prediction for next week: 940k cases (+0%) and 10,800 deaths (+5%).

Deaths

Sorry that this is an image, couldn’t get this to look right otherwise.

This is an excellent deaths number, but the difference from my prediction is likely the result of Labor Day rather than less people dying. We’ll find out next week.

Cases

Again, sorry this is an image. Working on it.

Was this the peak? I thi

...
D&D.Sci Pathfinder: Return of the Gray Swan Evaluation & Ruleset
by aphyer — 2021-09-09T14:03:56.859Z

This is a followup to last week's D&D.Sci scenario; if you haven’t already read it, you should do so now before spoiling yourself.

RULESET

Code used to generate the dataset is available here, with my apologies for the somewhat-spaghetti nature of it. The master voyage and encounter logs (including more detail and including destroyed ships) are available here and here.

MAP

The master map is:

Also available at https://github.com/aphyer1992/dndsci-pathfinder/blob/main/ship_map_master_version.png

Features to note:

Light Blue and Dark Blue show 'Depth' of water hexes. Water hexes adjacent to land are Shallow, and have more danger from e.g. Reefs. Water hexes very far from land (3 hexes or more) are Deep, and have more danger from e.g. Kraken.
Orange=Merfolk. The circles are their attack ranges. The triangle is the migration route of Atlantean merfolk.
Blue=the Maelstrom.
Red=the Dark Warlock's towers.
Black=Captain Calamity's Cove.
Brown=False Lighthouse.

Note also that there is a secret island in hex L16. This island is not on the Admiralty maps. (Congratulations simon for noticing this in the data).

SHIPS

A ship has several relevant statistics:

Ship	Hull	Guns	Speed	Size
Galleon	30	4	1	Large
Barquentine	20	4	3	Large
Carrack	30	2	3	Large
Dhow	20	2	3	Small

A Dhow has generally poor stats but is Small. This gives it some protection from reefs and running aground, but more importantly means that the Admiralty tends to assign it to short voyages, and so its survival rate in the data is higher than it should be. My condolences to anyone tricked by this.

Hull is your effective HP. Damage done is a percentage of this - a Barquentine or Dhow that takes 4 damage will have taken 20% damage, while a Galleon or Carrack will only have taken 13.3% damage (rounded up to 14% in the Admiralty logs you saw). A Barquentine has low Hull, which makes it more vulnerable to encounters that directly damage Hull without interacting with Guns or Speed (e.g. Reefs, Icebergs).

Guns help defend you against certain encounters. A Carrack has low Guns, which makes it more vulnerable to encounters that need to be fought off (e.g. Krakens, Dragons, Harpies).

Speed helps you avoid certain encounters. A Galleon has low Speed, which makes it much more likely to have encounters that can be outrun (e.g. Pirates, Dragons).

CAPTAINS

A ship also has a Capt...
Pleasure and Pain are Long-Tailed
by lsusr — 2021-09-09T05:10:18.883Z

This post is derived from Logarithmic Scales of Pleasure and Pain by the Qualia Research Institute. Thank you Romeo Stevens for introducing me to the cool work you're doing over there.

People have a certain baseline amount of happiness. Fix their problems, and they’ll be happy for a while, then go back to baseline. The only solution is to hack consciousness directly, to figure out what exactly happiness is – unpack what we’re looking for when we describe some mental states as having higher positive valence than others – and then add that on to every other mental state directly.

―Fear and Loathing at Effective Altruism Global 2017 by Scott Alexander

If we want to cure suffering then we need a way to measure suffering.

The human brain does not, by default, reason in absolutes. We reason via ratios. The difference between 1 and 2 feels about the same as the difference between 100 and 200 even though the difference between 100 and 200 is 100× as large as the difference between 1 and 2. This is known as Weber's Law.

To put this in mathematical terms, we map everything onto a logarithmic scale and then compare distances on the log plot. On a log plot, the distance between 1 and 2 equals the distance between 100 and 200.

Weber's Law doesn't just apply to measurements of our external environment. It applies to subjective experience itself. When you combine Weber's Law with subjective reports of conscious experience we find that valence (the intensity of an experience) is long-tailed. Consciousness itself may be long-tailed too.

The Valence Scale

Are people intersubjectively consistent on the relative pleasure and pain caused by different things? Yes we are.

Pleasure

“Ok,” you might say, “you’re just telling me that pleasure and pain can be orders of magnitude stronger than I can even conceive of. What do you base this on?” The most straightforward way to be convinced of this is to literally experience such states.

You know how every instant is a priceless gift but it's hard to appreciate that fact? When I'm meditating regularly I consistently, after thirty minutes of focused attention, drop into a state where I appreciate it. (My default mode network turns off too.) I don't know where my experiences fall on the nth Jhana ladder but I suspect there are meditative states way higher than what I've experienced. There are people who meditate for decades.

When I'm in a st...
Countably Factored Spaces
by Diffractor — 2021-09-09T04:24:58.090Z

A followup to Scott's "Finite Factored Sets", specifically the Applications part at the end where he talked about the infinite case.

As it turns out, there's a natural-seeming (at least to my mathematical tastes) way to generalize all the finite factored set stuff to the infinite case.
To be perfectly clear about what is being claimed:
1: It is possible to deal with arbitrary compact metric spaces, instead of just finite products of finite sets, with a minimum of fuss. In particular this means that we can have countably many axes/basis factors/coordinates now!
2: Orthogonality, time, history, and all that remain perfectly well-defined in the infinite-coordinate case, no meaningful differences crop up in comparision to how the finite case works.
4: Of the three ways to implement the extension to the infinite case which were suggested in Scott's post, it is closest in spirit to the second approach, with having the history of X be the intersection of all the sets C⊆B s.t. C⊢X. The proffered counterexample from Scott's post is forbidden by restricting the partitions to "sufficiently nice" ones, which makes everything Just Work.
5: Yes, the semigraphoid axioms, and everything not specifically mentioned, work out, with absolutely no special tricks needed besides the single starting restriction on only using "sufficiently nice" partitions and sets.
6: I was only able to get one direction of the fundamental theorem to work out (the conditional orthogonality to conditional probabilistic independence direction), since the nice attributes of ordinary vanilla polynomials don't last when you start mucking about with uncountable sums of countable products of variables. It's fairly plausible that the missing direction of the fundamental theorem does work out and I'm just not a good enough mathematician to show it, feel free to try it yourself. I suspect it'll require a different proof path than the original, though.
7: Daniel Filan has, completely independently of me, generalized all results in "Finite Factored Sets" to countable sets (note that in that case, there can still only be finitely many factors, while this version can have countably many factors. Daniel's version of things is unconstrained by the compactness restriction that I have, though.) Apparently everything works out perfectly, including the fundamental theorem, although the direction of it...
Selecting the Default Browser in Facebook on Android
by jefftk — 2021-09-09T00:50:03.665Z

On mobile, when an app wants to open a web page, it has three options:

Use the user's default browser. You leave the app entirely,
and switch to the browser. History, logins, autofill, etc work as the
user expects.

(Android only) Use a "custom
tab". Same as #1, except the app has a bit of control over
appearance (bottom toolbar, custom menu, etc). Also some apps [shakes
fist] don't respect your default browser, and always use Chrome for
the tab.

Use an "in-app browser". It is as if the app has implemented
their own web browser, where you are logged out of every site, history
is not shared, and they are able to inject arbitrary JavaScript into
the pages you visit.

I mildly prefer #1 over #2, but strongly prefer either to #3. In-app
browsers are add friction with no benefit, don't respect user choice,
make it harder for Firefox etc, and are insecure (more).

If you click on an external link in Facebook for Android, you'll get something like:

That's their in-app browser. You might think to turn it off you
could click the three dots:

And "Go to Settings":

There used to be an option here to turn off the in-app browser, but
now the only options are to configure it. Dead end.

To turn it off you need to go back to the Facebook home screen, go
into "Settings & Privacy", and click "Media":

At the bottom of the page there's an option for "Links open externally":

While I wish Facebook just used the default browser, moving the
setting to somewhere unreachable from the in-app browser is somewhere
between "intentionally user-hostile" and "failure to think about how
users interact with browser choice settings".

Comment via: facebook
How to Find a Problem
by J Bostock — 2021-09-08T20:05:45.835Z

There are many ingredients for creativity. One is to have a lot of knowledge saved up, so it can combined in novel ways. One is to actually think about things for five minutes. One is to not flinch away from problems when you find them.

But the most important is to have a problem to solve. Or if you're Feynman:

You have to keep a dozen of your favourite problems constantly present in your mind, although by and large they will lay in a dormant state.

Importance

To have a problem in the back of your mind constantly is somewhat taxing. You need to find a problem which isn't emotionally draining. For this reason the problems need to be important. They don't have to be the most important thing in the world, but they do have to be important enough to you to be somewhat self-motivating. If your day-to-day life involves so many interactions with doorbells that you can think about doorbell design all day, then that's enough.

In some ways, I have it easy on this front, which makes it hard for me to give advice. My domain is biological chemistry, so I can do a good job just by pointing myself at the closest ageing-related issue. Ageing is very salient to me, and as I get older I will either succeed or it will become painfully more salient.

Size

The problems need to be of the correct size. I don't hold the problem "cure ageing" in my head all day. That wouldn't work, because anything could in theory be related to "cure ageing" but there's no clear way for my brain to make the right connection. Instead you have to find a correct balance. The problem "replace lost stem cells in human tissue" is about the right level. There are lots of possible answers, but also the line between ideas I come across and that is not too many steps long.

I suspect that the right level is different for different people. This makes my advice somewhat unhelpful. Still, just knowing there is a correct level might be helpful. Perhaps a good idea is to try and keep a few different problems of different levels around, at least until you find the right one.

How to actually find the problem

Hopefully the two issues above have already clued a few readers into my method.

First off, think of something you want, or want to know. Just wanting to "discover things" is not enough. You have to work one level down to achieve that goal. It can even be silly or useless. As long as your emotional brain will let you focus on ...
A Taxonomy of Research
by J Bostock — 2021-09-08T19:30:52.194Z

When doing research you have methods of doing things. These differ from field to field according to the sort of thing you're investigating.

You also have overarching principles which direct your attention and resources along certain lines. The latter are much more interesting to talk about.

One Solution vs Many Solutions

For some avenues of research, there is only one solution. Think discovering Newton's Laws, or finding the protein which is responsible for breakdown of proteins in the stomach. Research is therefore a process of ruling out answers until you get to the only one which is correct. This often comes down to finding the best model. Eventually it comes down to finding a model so good it's the best model possible, which is "the truth".

For others, there are many solutions. Think drug discovery, or developing a new technology. Research here is often a process of search for a decent design, followed by gradual improvement. Other times, it's coming up with a new insight as to how existing things can be used. This is more like problem solving.

Known Solution vs Unknown Solution

For some problems, the general form of the solution is likely to be known. If you're working in battery technology, you probably know the next battery will have an anode, cathode, and an electrolyte which can accept and give up electrons. If you're trying to make a new antibiotic you probably know what sort of molecule it will be.

For other problems, you might have no idea. Until they were discovered, nobody knew what the laws of quantum field theory would look like, nobody expected the world to be made out of weird functions in 4d. Research in gene sequencing has come up with lots and lots of wildly different approaches.

The Grid

Looking for a single answer that you don't know what it looks like is discovery.

Looking for a single answer that you have a pretty good idea of what it will look like is investigation.

Trying to solve a problem with a solution you know the form of is engineering.

Trying to solve a problem with a solution you don't know the form of is invention.

Here's some examples which I've tried to put roughly in the right place

Depending on where you're aiming, you need different tools.

The further down, and also the further right, the more you need to pay attention to unexpectedly unexpected results. These are results which vary orthogonally to the axis of variation you're study...
[AN #163]: Using finite factored sets for causal and temporal inference
by Rohin Shah — 2021-09-08T17:20:04.522Z

Alignment Newsletter is a weekly publication with recent content relevant to AI alignment around the world. Find all Alignment Newsletter resources here. In particular, you can look through this spreadsheet of all summaries that have ever been in the newsletter.

Audio version here (may not be up yet).

Please note that while I work at DeepMind, this newsletter represents my personal views and not those of my employer. This newsletter is a combined summary + opinion for the Finite Factored Sets sequence by Scott Garrabrant. I (Rohin) have taken a lot more liberty than I usually do with the interpretation of the results; Scott may or may not agree with these interpretations.

Motivation

One view on the importance of deep learning is that it allows you to automatically learn the features that are relevant for some task of interest. Instead of having to handcraft features using domain knowledge, we simply point a neural net at an appropriate dataset and it figures out the right features. Arguably this is the majority of what makes up intelligent cognition; in humans it seems very analogous to System 1, which we use for most decisions and actions. We are also able to infer causal relations between the resulting features.

Unfortunately, existing models of causal inference don’t model these learned features -- they instead assume that the features are already given to you. Finite Factored Sets (FFS) provide a theory which can talk directly about different possible ways to featurize the space of outcomes and still allows you to perform causal inference. This sequence develops this underlying theory and demonstrates a few examples of using finite factored sets to perform causal inference given only observational data.

Another application is to embedded agency (AN #31): we would like to think of “agency” as a way to featurize the world into an “agent” feature and an “environment” feature, that together interact to determine the world. In Cartesian Frames (AN #127), we worked with a function A × E → W, where pairs of (agent, environment) together determined the world. In the finite factored set regime, we’ll think of A and E as features, the space S = A × E as the set of possible feature vectors, and S → W as the mapping from feature vectors to actual world states.

What is a finite factored set

Generalizing this idea to apply more broadly, we will assume that there is a set o...
[External Event] 2022 IEEE Conference on Assured Autonomy (ICAA)
by Aryeh Englander — 2021-09-08T16:31:29.265Z
Link: https://iaa.jhu.edu/icaa/

From the website:

Important Dates

10/18/2021 (Anywhere on Earth) Paper Submission Deadline

11/22/2021 Acceptance Notification

12/30/2021 Camera-ready Papers Due

3/22/2022 - 3/24/2022 Conference

Call for Papers

ICAA seeks contributions on all aspects of AI safety, security, and privacy in autonomous systems. Papers that encourage the discussion and exchange of experimental and theoretical results, novel designs, and works in progress are preferred. Topics of interest include (but are not limited to):

Autonomous System and AI Safety

Detecting dataset anomalies that lead to unsafe AI decisions
Evaluating safety of autonomous systems according to their potential risks and vulnerabilities
Resilient, explainable deep learning, and interpretable machine learning
Verification, testing and acceptance of machine learning models and autonomous systems
Autonomic computing for autonomous system safety
Standards, ethics, and policies for autonomy and AI safety
Ethics of autonomous system behaviors, algorithms and implementations
Safety and assurance of human-autonomy teaming

Security and Privacy of Autonomous Systems and AI

Detecting dataset anomalies that lead to autonomous system security and privacy violations
Differential privacy and privacy-preserving learning and generative models
Adversarial attacks on AI and autonomy, and defenses against adversarial attacks
Improving resiliency of AI and autonomous system methods and algorithms to various forms of attacks
Engineering trusted autonomous system and AI software architectures
Gradient descent is not just more efficient genetic algorithms
by leogao — 2021-09-08T16:23:46.996Z

I think one common intuition when thinking about gradient descent (GD) is to think about it as more efficient genetic algorithms (GAs). I certainly used to think this way—after all, taking a bunch of random mutations and keeping the ones that work well sounds basically just like a slower version of finding the steepest direction in the first place. GAs are also much more intuitive than GD, so it's tempting to think about GD in terms of them. Unfortunately, there are some instances where they behave completely differently, which is really relevant for thinking about things like gradient hacking. This post provides one such example.

Redundancy and Submodule Protection

Imagine you're given a network that has two identical submodules, and some kind of combining function where if it detects the outputs from both submodules are the same it passes the value through but if they're different it explodes and outputs zero or something totally random, or whatever. This is a natural idea to come up with if your goal is to ensure the optimizer doesn't mess with these modules, for example, if you're trying to protect the mesaobjective encoding for gradient hacking.

It's easy to convince yourself that this is effective against GAs: since mutations are inserted at random, the probability that both modules are mutated the same way is very small, and so any model that changes either of these modules will do very poorly since the output will be totally useless.

GD on the other hand will happily change both modules at the same time, since the direction of steepest descent is the one where both submodules change at the same time. This is even despite the fact that partial derivatives are taken assuming all other parameters stay the same, because partial derivatives don't care about the cliff just epsilon away but only the slope at the exact point where both modules are the same. Another way of thinking about it is the gradient necessarily points in the direction of steepest ascent, even if that direction isn't axis-aligned. If you're still not convinced, see this comment for a proof that there exists no combining function (under modest constraints) that can protect submodules this way.

Takeaways

The main takeaway from this is that gradient descent is really unintuitive sometimes. When in doubt, it's probably a good idea to work out the gradient by hand.