Ask HN: What medical advice did you have to learn the hard way? - <AUTHOR> <comment><USER2> You have to be the expert in you.

You should be the one that cares the most about your health. Yeah your wife cares, your mother cares, and even your doctor cares. But you must care the most.

<My story> When you are told what to do about your health care, and you follow it without understanding then you have abdicated your caring to another.

I had a thyroid issue. I complained to my doctor. He said I was fine, I accepted it. He then decided I had other issues. Issues that he himself had and was very sensitive to the symptoms. He then went heavy handed in curing it. As opposed to the common light handed initial treatments to fix it.

A different doctor then found the thyroid problem. Turned out my thyroid was 'technically' putting out enough, 1% lower and it would have been too low. But my pituitary gland (which drives the thyroid) was pushing the maximum to the thyroid to produce. Turns out doing the ratio between the two numbers is a common practice, which my original doctor did not do.

From that I learned to understand my blood work. I get copies of it and check for items slightly outside the norm and discuss with doctor. I also read up on all medication the doctor gives me. Some meds I have been given will address the symptoms, but can actually make the problem worse.

To be fair, most doctors only have a limited time with you. You have to be the expert in you.

</comment><comment><AUTHOR> I had knee surgery. After weening off opioids, I was put on three different NSAIDs simultaneously -- meloxicam, ibuprofen, and aspirin for blood clots. Two and a half weeks later I was admitted to the hospital for three days with micro-perforations and an infection in the sigmoid colon (diverticulitis). It's impossible to prove causation, but this is not something that young people usually get.

Medical advice: Be _very_ careful with NSAIDs. And be careful when multiple doctors are prescribing you medications.</comment>


Ask HN: Does working remotely hinder the ability to get a mortgage? - <AUTHOR> Consider a person A who works regular 9-5 job vs a person B who works for a company in another country remotely, does the bank see person A and person B differently at all provided that they make the same salary? <comment><USER2> I suspect remote/on-prem would be much less important than the difference between permanent employee with predictable salary versus freelancer/contractor with less predictable income.

If the bank were really on the ball they might try to estimate the probability of you losing your income stream and the probability you'd be able to replace it with a new one to keep paying off the mortgage. Then where you lived and what local opportunities for work were available might come into it. Giving a loan to a person in a small town with only one big employer / one industry is riskier than giving a loan to someone in a location where there are many alternative opportunities for employment. But on the other hand, land is probably cheaper in the small town, so the loan might be smaller..

</comment><comment><USER3> Based on my experience, no. They look at the documentation of your income and debts. Where I physically sit when I work simply wasn't asked.

</comment><comment><USER4> Interesting question.

I can't say for sure, but my take is it probably wouldn't, based on a couple things:

1\. I've known people who work sales for companies located across the country. We don't consider these jobs "remote" like the tech world, but they're similar in most respects. They own a home in a different city than their employer.

2\. Banks and brokers are, for the most part, dying to give away money. The employment section of the mortgage application often doesn't ask much more than how long you've worked at your employer and what your salary is (sometimes a letter from your employer is required, but should be no problem).

Just apply, it doesn't hurt.

</comment><comment><USER5> I'm not sure they would even know / bother to check.

I've regularly worked for companies located far from where I bought a home. The topic never came up outside of income.

Feel free to ask a mortgage broker or rep from the mortgage company, they are happy to answer questions.

</comment><comment><USER6> I had to have my manager sign a letter that I was a full time remote employee.</comment>


Show HN: React Drag-N-Drop Email Editor - <AUTHOR> https://github.com/unroll-io/react-email-editor <comment><USER2> Hey, developer of this React component here. I run several SaaS startups and we always wanted a good email editor that we can quickly embed in our applications. Couldn't find one that's free and solid so we decided to build one.

It takes less than 5 minutes to get started with this React component. Would love to hear your thoughts and feedback.</comment>


The Complete Guide to Know How New-Found Stem Cell Helps Regenerate Lung Tissue - <AUTHOR> https://www.creative-bioarray.com/services/stem-cell-research.htm <comment><AUTHOR> Morrisey, leader of the research, the a professor of Cell and Developmental Biology and director of the Penn Center for Pulmonary Biology and scientific director of the Institute for Regenerative Medicine, said that, 'One of the most important parts of understanding lung regeneration is the alveoli, the blood in the lungs that absorbs oxygen and the tiny ticks that exhale carbon dioxide. To better understand these subtle structures, we have been drawing different types of cells in the alveoli. Understanding cell-cell interactions should help us discover new players and molecular pathways that will provide targets for future treatments.

This study explored whether epithelial cells on the alveolar surface have stem cell behavior that may restore normal respiratory function after serious injury caused by influenza infection or in diseases such as chronic obstructive pulmonary disease (COPD). Although some organs, such as the intestine, replace the entire intestinal epithelial wall with the resident stem cells every five days, organs such as the lungs show a very slow replacement and contain stem cells that are only activated when they are damaged to regenerate this damaged tissue.

</comment><comment><AUTHOR> The Morrisey team identified an alveolar epithelial progenitor (AEP), in which the AEP was shed in larger cell populations known as alveolar type 2 cells (AT2 cells). These cells produce pulmonary surfactant (wetting agent) that prevents the lungs from collapsing with each breath. Dr. Will Zacharias, co- author of the paper and a postdoctoral researcher at Morrisy Labs, said, 'AEP is a stable cell lineage in the lung that changes slowly, but rapidly proliferates after injury, regenerating the alveolar walls and restoring gas exchange.'

AEPs exhibit their own unique set of genes and contain unique epigenetic markers. The Morrisey laboratory used the genomic information obtained from mouse AEPs to identify a conserved cell surface protein called TM4SF1 and ultilize this protein to separate AEP from human lungs. The Morrisey team then developed a three-dimensional lung organoid by taking advantage of the ability to isolate mouse AEP and human AEP. Mprrisy claimed that, 'From our organ-like culture system, we were able to confirm that AEP is an evolutionarily conserved alveolar progenitor cell that represents a new target for human lung regeneration strategies.'

</comment><comment><AUTHOR> The development of the lung is an evolutionary adaption to life on land. In order to survive on land, the lungs are necessary for most large animals. Its complex structure is partly determined by its integration with the cardiovascular system, which makes it an attractive organ that is difficult to study from a perspective of regenerative medicine. In addition, lung disease is one of the leading causes of death in the world, second only to cardiovascular disease and cancer.

In a new study, researchers from the Perelman School of Medicine at the University of Pennsylvania identified a lung progenitor cell that repairs alveoli (a gas exchange compartment in the lungs). They isolated these progenitor cells from mice and human lungs, described their characteristics, and confirmed that they are essential for lung tissue damage caused by severe flu and other respiratory diseases. Related research results were published online in the Journal of Nature.

</comment><comment><AUTHOR> In the further study, the Morrisey team aimed to study influenza-induced lung tissue injury and other lung disease models to determine where and when AEP responds to acute lung injury or more chronic conditions . Given the severity of the current influenza season, this study provides new insights into human lung regeneration and provides new important genetic and epigenetic pathways for lung regeneration. The Morrisey team is now exploring which of these molecular pathways may contribute to the function of AEP in mice and human lungs, including understanding whether activating Fgf signaling (one of the key pathways conserved in mouse AEP and human AEP) may promote lung regeneration. We will keep updating more related and further information for you.</comment>


Probability Theory for Scientists and Engineers - <AUTHOR> https://betanalpha.github.io/assets/case_studies/probability_theory.html <comment><USER2> This is excellent. Amazing (equation + picture)/text ratio.

Complaint: you don't define your notion of "space". In chapter 1 it's some informal notion that you use to motivate the definition of a set (??), in 1.3 and 1.4 it becomes clear by space you mean "set". Then later you start talking about dimension of spaces, implying not only do they come with a topology now they have a well defined dimension, so a locally Euclidean Hausdorff space or something - but maybe you just mean R^n.

Comment for other commentators in this thread: not all expositions is tailored for the masses. A piece of pedagogical literature that does not appeal to your background doesn't mean it's not good. There's a very clear need for exposition on basic structures in probability theory and this fits there.

<comment><USER3> It's like the definition of set defined here should really be a subset, and the definition of space should be a set. Maybe just say a set is any collection of objects?

<comment><USER4> Agreed!

A sample space would be relevant in probability theory and is often helpful to calculate for your denominator, but this definition is rather vague.

</comment></comment></comment><comment><USER5> I'm not really sure who the intended audience is here. There's a lot of material covered very briefly in a very short space, and not enough details that anyone who doesn't already know it would be able to pick up anything substantive.

<comment><USER6> Seriously. Content like this is only useful for people who already know probability.

<comment><USER7> Do you know of anything which can help people get over the hurdle to know enough to use this content?

For me it's only worked when colleagues have explaining concepts to me when they were needed, after a several occurrences of this everything finally started to make sense and I could then make use of material like this.

<comment><USER6> There are no shortcuts with math. If you really want to learn it, you must be willing to put in a large number of hours over a long period time in order to master it. Are you willing to do that?

</comment></comment></comment></comment><comment><USER8> This is an excellent book by L. V. Tarasov on probability.

[https://archive.org/details/TheWorldIsBuiltOnProbability](https://archive.org/details/TheWorldIsBuiltOnProbability)

I have always found Russian math book writers to be on point, not going too much over your head, also respecting the reader's intelligence. If you like it, then you will love his calculus book, that one is also a real gem.

<comment><USER9> Wow, that's pretty expensive book [https://www.amazon.com/World-Built- Probability-L-Tarasov/dp/...](https://www.amazon.com/World-Built-Probability- L-Tarasov/dp/5030011242)

<comment><USER8> Yeah, his books are out of print. I have been looking for a hard copy of his calculus book. Really expensive.

<comment><USER9> Thanks for the reference anyway! I downloaded the Russian version in form of PDF document and enjoy reading it:-) What was great about USSR is its level of popularization of science. All these books written for kids or high school students were amazing.

<comment><USER10> Yes, and a number of them are now available for free over the internet. In Russian...

</comment></comment></comment></comment></comment><comment><USER11> Echoing other comments here, this seems like a hard way to start learning probability. It sounds like the goal is to make probability easier to understand based on what you say here ([https://betanalpha.github.io/writing](https://betanalpha.github.io/writing))

> In this case study I attempt to untangle this pedagogical knot to illuminate > the basic concepts and manipulations of probability theory and how they can > be implemented in practice

But I think this is too hard. I really loved "Probability For The Enthusiastic Beginner" [http://a.co/2kp5PZd](http://a.co/2kp5PZd)

<comment><USER12> "For Scientists and Engineers" sounds to me like it's targeting people who already have a strong background in more advanced mathematics, but not necessarily probability and measure theory. If so I think this is a decent way to go about it.

I'm an engineer and sometimes mathematician that works with fairly in-depth probability theory related things and this looks to be a condensed version of a lot of the basic stuff I had to self learn when I was getting into what I work on now. I'm in a niche area though, and I do wonder if this really is that useful to most scientists and engineers.

<comment><USER13> Reading 'All of statistics' by Wasserman would be a much better bet than diving deep into measure-theoretic probability though.

<comment><USER12> Depends on your goal, statistics and probability theory are separate (though of course related) fields with different applications. For me I really needed the measure-theoretic bits because I was (am) working on modeling ergodic processes. This article honestly doesn't go into enough detail to be especially useful but I like the direction the author approaches it from.

I'm familiar with the text you mentioned, it's certainly good and would be better than this article for most, but comparing a textbook to a short web article isn't exactly fair.

</comment></comment></comment></comment><comment><USER14> The goal is worthy, but the product is inadequate to say the least. This thing is littered with typos, and enough of the exposition is sufficiently irrelevant or incorrect to be unintuitive. That said, I like the graphics and layout.

For example, when he discusses power sets in order to introduce sigma algebras, he implies that a sigma algebra is a better-behaved alternative to a power set. However, a power set is always itself a sigma algebra (after all, even a power set of an uncountable set still is closed under complements and countable unions).

Later, when discussing probability distributions, he writes:

> [W]e want this allocation [of a conserved quantity] to be self-consistent – > the allocation to any collection of disjoint sets, A_n ∩ A_m=0, n≠m, should > be the same as the allocation to the union of those sets, > ℙπ[∪(n=1 to N) > A_n]=∑(n=1 to N)ℙπ[A_n].

The condition `A_n ∩ A_m=0, n≠m` is actually incorrect, since A_n and A_m are sets and 0 is an integer. The author means the empty set, but typo'd.

Sometimes he frequently uses words like "conserved" or "well-defined" without giving us a clue as to what these mean. In what context are probabilities "conserved"? What distinguishes "well-defined" from "not well-defined"?

I'm a software engineer. A non-trivial amount of my time is devoted to reading code and finding bugs. Sloppy reasoning, inconsistencies and outright errors like that are big red flags to me. It doesn't help that the whole section on sigma algebras is somewhat irrelevant, since he doesn't really explore measure theory as the basis for modern probability.

IMO a better resource is the series of "Probability Primer" videos from mathematicalmonk on YouTube[1]. He does an excellent job (IMO) of covering all pertinent pre-requisites and being mostly rigorous without necessarily proving every single fact or exhaustively covering all edge and corner cases. He also makes a good effort to recommend advanced (and rigorous) treatments of the subject (and ancillary ones like measure theory). A readable version of this YouTube series would be a great resource, and if Michael Betancourt is reading, I'd encourage him to pursue that in his next iteration of this product.

[https://www.youtube.com/playlist?list=PL17567A1A3F5DB5E4](https://www.youtube.com/playlist?list=PL17567A1A3F5DB5E4)

<comment><USER13> > It doesn't help that the whole section on sigma algebras is somewhat > irrelevant, since he doesn't really explore measure theory as the basis for > modern probability.

Christ, practically all of measure theory is irrelevant for applied work, in much the same way that an engineer shouldn't care about the definition of a real number.

There's a model of real analysis due to Solovay that used the axiom of dependent choice instead of the full axiom of choice. In the Solovay model, _all sets are measurable_. Thus any results that _require_ measure theory inherently depend on the axiom of choice.

I'd be worried if I was relying on Choice as an applied scientist.

Edit: same goes for Lebesgue vs. Riemann integration. To quote Richard Hamming: _Does anyone believe that the difference between the Lebesgue and Riemann integrals can have physical significance, and that whether say, an airplane would or would not fly could depend on this difference? If such were claimed, I should not care to fly in that plane._

<comment><USER15> It matters very much in computational / quant finance

<comment><USER13> It matters by convention, because the textbooks are written that way.

My point is that _you don 't need that level of formal rigour_ to do applied work. You can derive the Feynman-kac formula via a scaling limit of discrete- time Markov chains. Add some levy process (a.k.a compound Poisson processes) and you're basically done.

If you want to be ultra-rigourous in your definitions, then you need measure theory, yes. But even Einstein didn't need that for his description of Brownian motion. If a scaling limit is good enough for him, it's good enough for me.

</comment></comment></comment></comment><comment><USER16> I confused this with the book "Probability and Statistics for Engineers and Scientists" by Anthony Hayter and I got excited.

I am kind of a beginner in Machine Learning and was struggling badly with basic probability and Statistics concepts. I went through so many resources and somehow none of them clicked. Then I stumbled upon this book and I realized this is exactly the kind of book I needed. It assumes no prior knowledge and is very heavy on examples. Other books just dive into jargon/symbol laded theory without giving simple examples or building concepts from ground up.

I mentioned this because I feel someone might benefit from this suggestion.

</comment><comment><USER17> Wow, this seems like a particularly hard way to learn probability.

One thing I noticed about myself as I did more and more work with probability is that I started thinking in terms of distributions a lot more.

These days I find it very difficult to think without using them. In just about everything I do now I tend to think about moving probability mass around.

</comment><comment><USER18> Here's my non-standard, nutshell, IMHO advice in using probability theory:

(1) Random Variables. Go outside. Observe a number. Then that is the value of a _random variable_. To have a random variable, that the number be _random_ in the sense of unpredictable is not needed. For the phrase and/or criterion "truly random", mostly f'get about it, but we return to that for the subject of random number generation below. So, net, your data, all your data, are the values of random variables.

(2) Distributions. Sure, each random variable has a distribution. And there is the Gaussian, uniform, binomial, exponential, Poisson, etc. distributions.

Sometimes in practice can use some assumptions to conclude that a random variable has such a known distribution; this is commonly the case for exercises about flipping coins, rolling dice, shuffling cards.

For another example, suppose customers are arriving at your Web site. Well maybe the number of arrivals since noon have stationary (over time) independent increments -- maybe you can confirm this just intuitively. Then, presto, bingo, the arrivals are a Poisson process, and the times between arrivals are independent, identically distributed exponential random variables -- see E. Cinlar, _Introduction to Stochastic Processes_. Further, since might be willing to assume that the arrivals are from many users acting independently, the _renewal_ theorem says that the arrivals will be approximately Poisson, more accurately for more users -- see W. Feller's second volume.

Sometimes the central limit theorem can be used to justify a Gaussian assumption.

Still, net, in practice, mostly we don't and can't know the distribution. To have much detail on a distribution of one variable takes a lot of data; the joint distribution on several variables takes much more data; the amount of data needed explodes exponentially with the number of joint variables. So, net, don't expect to know or find the distribution.

Often you will be able to estimate mean and variance, etc. but not the whole distribution. So, usually need to proceed without knowing distributions. In simple terms: Distributions -- they exist? Yup. We can find them? Nope!

(3) Independence. Probability theory is, sure, part of math, but, really, the hugely important, unique feature is the concept of independence.

One of the main techniques in applied math is divide and conquer. Well, where you can make an independence assumption lets you so divide.

Independence? A simple criterion for practice is, suppose you are given random variables X and Y. You are even given their probability distributions (but NOT their _joint_ probability distribution). Then X and Y are independent if and only if knowing the value of one of them tells you nothing more than you already know about the value of the other one.

The hope here is that often in practice you can check this criterion just intuitively from what you know about the real situation. E.g., does a butterfly flapping its wings in Tokyo tell you more about weather tomorrow in NYC? My intuitive guess is that this is a case of independence which means that for predicting weather of NYC tomorrow, we can just f'get about that butterfly.

(4) Conditioning. For random variables X and Y, can have the conditional expectation of Y given X, E[Y|X]. Such conditioning is the main way X tells you about Y. Then there is a function f(X) = E[Y|X], and f(X) is the best non- linear least squares estimate of Y. Note that E[E[Y|X]] = E[Y] which means that E[Y|X] is an _unbiased_ estimate of Y.

(5) Correlation. If you don't have independence, then likely use the Pearson correlation -- it works like the cosine of an angle. If random variables X and Y are independent, then their Pearson correlation coefficient is 0 -- proof is an easy exercise just from the basic definition and properties of independence.

(6) The Classic Limit Theorems. Pay close attention to the central limit theorem (CLT) and the weak and strong laws of large numbers (LLN). The CLT is the main reason we get a Gaussian distribution, and the LLN is the main reason we take averages.

(7) Random Number Generation. A sequence of random numbers are to look, for some practical purposes, like a sequence of random variables that are all independent and have uniform distribution on [0,1]. Are they "truly random"? Maybe not. But if they are, then they are independent and identically distributed (i.i.d.) on [0,1] -- and that's all there is to it, and don't have to struggle to say or understand more.

<comment><USER19> Probability theory expositions, especially for [software] engineers, would be better served if they were well typed. What is the type of a random variable, E[Y|X], E[E[Y|X]]? Hint, a random variable is not a scalar, but rather a function, the probability distribution.

<comment><USER20> Hmm, a random variable (in the sense of measure theory, as in OP) is indeed a function - but it's not a probability distribution.

An R.V. is a measurable function from the sample space into the reals. A probability distribution is a function assigning probabilities to measurable sets, formally, a function from the sigma-algebra into [0,1].

So in particular, a R.V. (like a gaussian) can take on negative values. A probability distribution cannot.

Also, the domain of the R.V. is the sample space. But the domain of the probability distribution is the sigma-algebra over that sample space.

<comment><USER18> A distribution is a real valued function of a real variable. The domain of the function is the whole real line.

Note: Below, borrowing from D. Knuth's TeX, we use the underscore character '_' to denote the start of a subscript.

Details: For real valued random variable X, probability measure P, and the set of real numbers R, the _cumulative distribution_ of X is the function F_X: R --> R where, for x in R, F_X(x) = P(X <= x).

If F_X is differentiable, then the _probability density distribution_ of X is the real valued function of a real variable f_X: R --> R where, for all x in R, f_X(x) = d/dx F_X(x) where d/dx is the calculus derivative.

For the connections with sigma algebras, that is more advanced than most engineers care about, but here are some of the details:

For real numbers a and b with a < b, there is the _open interval_

(a,b) = {x|a < x < b}

A topology on R is a collection of subsets regarded as _open_ and that satisfy the axioms for a topology -- the sets in a _topology_ are closed under finite intersections and arbitrary unions and both R and the empty set are open. The _usual topology_ on R is the smallest (a short argument shows that this "smallest" is well defined) topology that has each open interval an element of the topology -- right, the topology regards the open intervals as open.

The usual reason to discuss a topology is to have a means of defining continuous functions, a means more general than from the usual "for each epsilon greater than zero, there exists a delta greater than zero such that ..." or in terms of limits of sequences. Indeed, there are advanced situations where we can use topologies to define continuous functions where epsilon and delta and where converging sequences don't work. If curious, look up Moore- Smith convergence, nets, and filters or just Kelley, _General Topology_.

Well, a _sigma algebra_ is like a topology, that is, is a collection of subsets: A sigma algebra is closed under countable unions and relative complements. Right, we avoid uncountable unions because otherwise we will get stuck in a big mud hole. It is an early exercise that there are no countably infinite sigma algebras.

The reason for sigma algebras is to permit defining a _measurable_ function, that is, one where we can apply the Lebesgue integration theory. The integral of calculus is due to B. Riemann and is the _Riemann_ integral. W. Rudin, _Principles of Mathematical Analysis_ shows that for a continuous real valued function with domain a _compact set_ (closed and bounded, where _closed_ is the complement of an open set) has a Riemann integral. Well in this case, the Lebesgue integral gives the same numerical answer -- same thing. The advantage of the Lebesgue approach is that the function can be even bizarre and its domain can be much more general. Indeed, in probability theory, expectation is just the Lebesgue integral. In simple terms, Riemann partitioned on the X axis, and Lebesgue partitioned on the Y axis.

Well, given the usual topology on R, we can ask for the smallest sigma algebra on R that has the topology as a subset. That sigma algebra is the _Borel_ sets of R. Uh, Lebesgue was a student of E. Borel. In Rudin will find the Heine- Borel theorem.

So, in probability theory, we have a _sample space_. Each point in the sample space is a _trial_ , i.e., essentially a real world experimental trial (note: really our attitude is that in all the universe we see only one such trial -- if this seems far out, then blame the Russians, e.g., A. Kolmogorov, E. Dynkin, etc.!). Well, an _event_ is a subset of the sample space, that is, a set of trials. So, flip a coin. Let H be the event, the set of all trials where, that the coin comes up heads.

Well, to apply Lebesgue's theory of integration, we want the set of all events to be a sigma algebra.

Then a _probability_ measure is a measure in the sense of Lebesgue's measure theory, that is, a real value function, in the case of probability taking values in [0,1], and with domain the sigma algebra of events. So, for the event H, we can ask for the probability of H, that is, P(H), which is a number in [0,1]. For a _fair coin_ tossed by an honest member of the FBI we have P(H) = 1/2.

Then a real valued _random variable_ X is just a real valued function with domain the sample space and also _measurable_ : This part about being _measurable_ is that for each Borel set A, a subset of R, the set of all trials w so that X(w) is in A is an event, that is, an element of the sigma algebra on the set of trials. That is, the inverse image under X of the Borel sets are events, elements of the sigma algebra on the sample space.

So, with X measurable in this way, we have a near perfect shot at defining the expectation of X, E[X]. For this we have a little two step dance:

First we look at X^+ ('^' denotes a superscript) where X is >= 0. So, X^+ is the _positive_ part of X. Similarly X^- is the _negative_ part of X. So, X = X^+ + X^-. Uh, I'm working quickly from memory; maybe we want X^- to be -X where X < 0 and 0 otherwise. So both X^+ and X^- are >= 0 and we have X = X*+ - X^-. Either way.

Well, we can use Lebesgue's theory to integrate X^+ and X^-. Biggie stuff: The X need only be measurable, and that admits lots of really wildly bizarre functions. We've got great generality, and that's good to have in various limiting arguments. Uh, we like limiting arguments because that is our main way to approximate which our main way to being healthy, wealthy, and wise!

So the Lebesgue integral of X^+ we write as E[X^+]. Similarly for X^-. Now no way do we want to be subtracting one infinity from another since permitting that would trash the usual laws of arithmetic.

So, for our second step, in the case X^- >= 0, if at least one of E[X^+] and E[X^-] is finite, then we define E[X] = E[X^+] - E[X^-].

Now we've defined expectation ("average") of a real random variable X. Our definition is just the Lebesgue integral. For the Lebesgue integral, we wanted the sigma algebras.

On the real line we can consider the sigma algebra of Lebesgue measurable sets; that's larger than the Borel sets. Then we just ask, assume, assert, believe, ..., that our random variables are measurable with respect to the sigma algebra of Lebesgue sets and the sigma algebra of the events. Uh, right, Lebesgue measure on R assigns Lebesgue measure b - a to interval (a,b) and extends from there. Fine details are in various texts by Rudin, Royden, etc.

That's the beginnings of the role of sigma algebras in advanced approaches to probability, statistics, and stochastic processes. It turns out, the sigma algebra approach is for several parts of what we want in probability, much nicer, e.g., for defining independence and conditional expectation. E.g., if we want to know that some set of uncountably infinitely many random variables are independent, we can. Same for conditioning on uncountably infinitely many random variables, e.g., the past history of a stochastic process.

</comment></comment></comment></comment><comment><USER21> I don't see the point of introducing sigma-algebras if you're not doing probability based on measure theory.

As others have said I wouldn't suggest this exposition to someone learning probability for the first time, but it's not as bad if you're familiar with the material and need a quick review.

</comment><comment><USER22> > The set of all sets in a space, X, is called the power set, P(X). The power > set is massive and, even if the space X is well-behaved, the corresponding > power set can often contain some less mathematically savory elements. > Consequently when dealing with sets we often want to consider a restriction > of the power set that removes unwanted sets.

I wish people could teach math in plain English. I don't know why the math and physics world refuses to write for the reader. I took this class before, and I still don't know what the author means by "less mathematically savory elements".

Here's you explain things to humans:

> There is a set called the power set that contains all the sets in a space. > This set is huge, and it contains [less mathematically savory elements]. > This is why we usually use a restricted version that removes the unwanted > sets.

Seriously, there's no point to this sort of fancy language. Math is already hard. No need to make it harder.

<comment><USER23> "There is a set called the power set that contains all the sets in a space"

I don't think he's the best expositor and some of his terminology is crappy, but I understood the author from what I've read so far. I literally have no idea what you're trying to say; this has no meaning

<comment><USER22> There is a set. It's called the power set. It contains all the sets in a space.

<comment><USER23> The terms space and set are not synonymous. A space is defined on a set.

The collection of all subsets of a set X is called the power set and is typically denoted as P(X).

Replacing something confusing but loosely understandable with something even more confusing isn't an improving anything

</comment></comment></comment></comment><comment><USER24> I thought this was really nice, strange to see that so many people dislike it.

</comment><comment><USER25> I find that this guide unhelpfully conflates probability and inference in a few places. Probability theory on its own is interesting but not terribly useful without the infrastructure of estimation.

</comment><comment><USER26> i think these are mistakes

>2.5 Conditional Probability Distributions As we saw in Section 3.4,

>It turns out that in this case a σ-algebra on Z naturally defines a σ-algebra

</comment><comment><USER27> NO NO NO!!! Don't start with Venn diagrams, sets, and other such fluff. Reminds me of the thin, little book they tried sticking on us in my probability class; undergrad EE. It was meant for math majors.

There is a book "Probability and Statistics for Engineers and Scienctists" by Raymond Walpole. That book is excellent. Rolling dice and pulling colored marbles from jars is how you teach probability.

<comment><USER28> I studied probability during my undergrad (and high school) using dice, coins and other such things. It made sense to me but there was a dark area in my understanding. It felt like a blind spot and I could never get into it. In the final year of engineering, we had someone do a quick refresher on probability as a prelude to a longer course on pattern recognition and he described the whole thing using set theory (Venn diagrams, functions mapping from one space to another etc.) and I felt that the blind spot was illuminated. So, I don't know if starting from there would make sense but I do think it's useful, atleast sometime in your studies, to look at the whole system through this lens.

I've been working through [http://www.greenteapress.com/thinkbayes/](http://www.greenteapress.com/thinkbayes/) and am quite enjoying it. My only complaint is that he, as intended, teaches using programs and a computer and I learn better by doing stuff by hand. He also has a think stats book at [http://www.greenteapress.com/thinkstats/](http://www.greenteapress.com/thinkstats/) which people might find interesting.

<comment><USER18> There is a good connection between probability and Venn diagrams: Both are about area. Probability is about area where the area of everything under consideration is 1. So, there is a set of _trials_. It has area 1. Each subset of the set of trials is an _event_ and has an area, its _probability_. Then we can move on to random variables, distributions of random variables, independence of events and random variables, the event that a random variable has value <= some real number x, etc.

In pure math, since H. Lebesgue in about 1900, the usual good theory of area is Lebesgue's _measure theory_. The ordinary ideas of area we learned in grade school, plane geometry, and calculus are all special cases. But Lebesgue's theory of area handles some bizarre, pathological, extreme cases. And we can show that there can be no really _perfect_ theory of area -- e.g., there have to be some bizarre subsets of the real line to which no nice theory of area can assign a length. But, once we have the Lebesgue theory, the usual way to show that there is a subset of the real line without an area uses the axiom of choice.

Well, in 1933, A. Kolmogorov wrote a paper showing how Lebesgue's theory of area would make a solid foundation for probability, and that approach is the standard one for advanced work in probability, statistics, and stochastic processes.</comment></comment></comment>


Whitepages Caller ID is now Hiya - <AUTHOR> http://www.hiya.com <comment><AUTHOR> Hiya Caller ID & Block is a mobile app that identifies unknown numbers and automatically blocks scam and spam across the globe so that your phone experience is safe and secure.

With a database of more than 600 MM mobile and landline phone numbers in the US, and over 1.5 billion unique numbers globally, Hiya leverages its expansive algorithms to identify unknown calls and texts and to monitor phone-based threats for consumers and businesses. Hiya screens more than 400 MM incoming calls per month, and has detected more than one billion robo, telemarketing and scam calls and texts to date. Hiya is available as a consumer app on Google Android and is integrated into the phone experience for T-Mobile and Samsung Galaxy S7 users worldwide. For more information, please visit www.hiya.com.</comment>


Show HN: There are 100s of similar Monitor Arms so I built this - <AUTHOR> https://monitorarms.co/ <comment><USER2> I've bought six monitor stands in the past two months (don't ask) and in no case did this tool surface any of the stands I actually bought, all of them very popular models (i.e., high sales rankings, lots of reviews).

On another note, the competition in monitor stands has been _great_ for consumers. A decade ago Ergotron was basically the only option for high- quality stands. Now, I just bought stands that cost 1/3 or less of what an Ergotron goes for and the quality is _at least_ as good as the two Ergotrons I have. The quality is astounding for the price.

<comment><USER3> Any manufacturer you can recommend as a competitor to Ergotron? I was looking at their stands because my monitor is on the heavier side (~26 lbs / 11kg) and expensive so I want an adjustable arm stand that I can trust to hold it securely.

A lot of cheaper stands say they only hold up to about 20 lbs.

<comment><USER2> I've never looked for that because I don't have any monitors that weigh anywhere close to that. (The heaviest monitors I have are 27" 4K monitors weighing 12 lbs each.) The brands I bought are WALI and Vivo, and they're both excellent, but I don't know if they have any stands that will support 26 lbs.

<comment><USER3> Thanks, I'll look more into those brands.

If anyone is in a similar position, I've got a 34" ultrawide, so it's a bit heavy. I was looking at the Ergotron HX Monitor Arm as it can hold 20 to 42 lbs, but it's about $280. The best I could find on Vivo (STAND-V101H) has a maximum capacity of 26 lbs, so it might just work out.

Of course my monitor comes with what is actually a very nice stand, but it just takes up so much room.

</comment></comment></comment></comment><comment><USER4> \- What if I want a monitor arm that's more than quadruple? (A colleague has 6 screens, stacked 3 on top of 3).

\- What is a grommet mount? Maybe some images?

\- Why can't I answer "no" to "do I have a color preference"?

\- "What size Vesa mount", would love an illustration image here

\- There doesn't seem to be any Quadruple + adjustable stands in your database

\- What about accessories?

\- I can't filter nor sort on price?

\- When showing the list it doesn't show me a number telling me how many results there are.

\- Can I filter on brand?

\- Can I convert currency into Euro?

\- What about shipping prices?

\- What about availability?

\- What about rating?

\- Some monitor arms have built in USB-hubs, but I can't filter on this.

Otherwise a good site, just missing some features. The database over available stands also feels a bit small, but I'll bookmark it for later.

</comment><comment><USER5> Please extend this ... to every product on Amazon.

Please also treat identical images as the same product. Copycat products on Amazon are the bane of my existence. I hate scrolling through pages of "products" and seeing the same "product" listed repeatedly even though they are simply different listings.

</comment><comment><USER6> Just saw a similar Show HN post [1] comparing usb-c hubs [2]. Am I spotting a new trend ?

[1]: [https://news.ycombinator.com/item?id=22152736](https://news.ycombinator.com/item?id=22152736)

[2]: [https://www.usb-hubs.org/](https://www.usb-hubs.org/)

<comment><USER7> Not a Show HN trend I'm liking, seems lazy.

<comment><USER8> Why is coding something that could easily add functionality to, say, PC Part Picker, a lazy trend?

<comment><USER9> This one seems to somehow not come up with a lot of options that are out there ... a number of products that I know I found on Amazon aren't on there no matter what I answer on this system.

</comment></comment></comment></comment><comment><USER10> Completely unusable on iPhone.

<comment><AUTHOR> We are currently working on it :/ Thanks for the patience

</comment></comment><comment><USER11> The viewport doesn't allow a usable interface on iPhone XS. E: Starting from first question

</comment><comment><USER12> UX feedback: Dont make me click "Continue" when all I have is one choice! If there's only 1 possible option, proceed to the next question after I have made the selection.

<comment><AUTHOR> Fixed. Thanks for the feedback

</comment></comment><comment><USER3> A picture of a white, single monitor, adjustable arm stand links to this: [https://www.amazon.com/gp/product/B00C5H5DN0](https://www.amazon.com/gp/product/B00C5H5DN0)

I wish I could tell you what monitor it was showing me, but the site hides all information that would let me skip the affiliate links.

<comment><AUTHOR> Can you tell which one has the wrong link, there might be a wrong link on the data. I would like to fix it.

Edit: I checked the link on the white single arm, It seems correct. Please let me know if its a different one.

Edit 2: I mostly built it for myself, because I was going crazy over seeing 100s different options and trying to filter . Then realized that I could add links and give it a try (never done it before). Didn't mean to be aggressive. And if you look at the product names they are all useless and make the page look very ugly, that why I decided to not display them.

<comment><USER3> Single -> Adjustable -> Desk -> Not Sure -> 100x100

Second option for $54.99. Maybe it's technically silver, but looked white to me.

<comment><AUTHOR> Thanks for the info. Seems like it was broken. Fixing it.

And here is the non affiliate link for that product

[https://www.amazon.com/dp/B07BF76ZG1](https://www.amazon.com/dp/B07BF76ZG1)

</comment></comment></comment></comment><comment><USER13> If you're looking for feedback, I'd suggest that the prices have two decimal points. A price of $49.9 looks wrong.

<comment><AUTHOR> Should be fixed soon, Good catch :)

</comment></comment><comment><USER14> Hi, congrats What is the stack?</comment>


Stackexchange network is down. - <AUTHOR> https://twitter.com/StackStatus

<comment><USER2> _This page ([https://superuser.com/users/login?returnurl=http%3a%2f%2fsup...](https://superuser.com/users/login?returnurl=http%3a%2f%2fsuperuser.com%2f)) is currently offline. However, because the site uses CloudFlare's Always Online™ technology you can continue to surf a snapshot of the site. We will keep checking in the background and, as soon as the site comes back, you will automatically be served the live version. Always Online™ is powered by CloudFlare | Hide this Alert_

So there are some issues</comment>


Account Deletion - <AUTHOR> I don't want my HN account anymore and leaving a dead account hanging around like a bad toenail is simply untidy.
Am I missing the delete button somewhere or has the most straightforward DELETE operation in history still not been implemented on hacker news? <comment><USER2> As a counterpoint, it drives me nuts when I see a graveyard of deleted comments on reddit. Do you want all your activity to vanish or do you just want to disown the username?

I think a fair compromise would be to simply delete the _account name_ associated with comments.

<comment><AUTHOR> A fair point, and honestly I wasn't even thinking about my posts one way or the other. But yes reddit's thread quality is unfortunate. The writing on the wall here in a similar sense is actually why I'm trying to make things expedient and delete it now so I don't have old things to look back on or any compulsion to come back.

And seriously, I'm guessing many users here have password managers. Eventually one sees it and gets curious. I'd rather just avoid the imminent depression.</comment></comment>


Are We Still Thinking? - <AUTHOR> https://firstname.lastname@example.com/are-we-still-thinking-795bd9f4a658 <comment><USER2> Made me think :)

Good point about 'Describe yourself' Labeling is a known way of getting people to judge each other. And good observation about the ideology machine responding with the mindless "Everything is Political". I can't stand that too.

Solutions do exist. And we will see them showing up in the coming years. Everyone I know who can still think is pretty sick of what is going on.

This write up reminded me of an old Boss I had, who would tell the team anytime an argument broke out - "Solutions over Reactions please" \- and that would be the que to just stop and go off and think.</comment>


Enloya - <AUTHOR> http://enloya.com <comment><AUTHOR> Enloya is a legal marketplace for startups. Enloya matches specific legal solutions to specific legal problems through the use of computer algorithms and carefully designed user-experience. In doing so, we have in mind what typical and atypical startups struggle with, from opening up a simple brick- and-mortar business to selling sophisticated goods and services across borders. In a world where legal fees are obscure and expensive, and knowing what specific legal solutions need to be sought is confusing, Enloya provides transparency, affordability and simplicity to the legal industry by enabling a community where fixed-fee services, superior albeit affordable lawyers, and specific legal solutions are prioritized. In parallel, Enloya provides lawyers with a platform to manage their schedules and to create teams with other lawyers and legal professionals on the platform (e.g. with economists, accountants, engineers, or even marketers).</comment>


Show HN: Simple multiline find and replace with sed - <AUTHOR> https://replace.sh <comment><AUTHOR> Hi HN,

I've made a website for generating sed scripts with Rust and Typescript+React. Built this in a week to learn Rust with GraphQL and Diesel, and a bit of front end with React. Feedback is much appreciated.

Github: [https://github.com/esemeniuc/replace.sh/](https://github.com/esemeniuc/replace.sh/)

<comment><USER2> The example is not really helpful because you can do it with

      sed s/findo/replaco/g

<comment><AUTHOR> That's true, I've considered that but I haven't found an example that is as easy to scan for quick demos.

</comment></comment></comment><comment><USER3> It looks like you're not escaping single quotes in the find pattern, so the command output gets malformed.

Also, you're reading in the whole file before replacing, even though most uses of sed will probably be per line. Including an option for line-by-line (maybe as default) would be much better for large files.</comment>


Ask HN: Which proprietary SaaS can be replaced with FOSS cron jobs? - <AUTHOR> I am thinking of using `curl` for healthcheck, `lighthouse` for webpage performance analysis, `ab` for load-testing etc.<p>Which other open-source tools are useful to run as cron jobs? <comment><USER2> The classic dropbox -> rsync, of course.

</comment><comment><USER3> Not a cron job per se... But people overlook Redmine as a fantastic alternative to project management systems like Asana or bug trackers like Jira.

</comment><comment><USER4> I think a lot of popular services are just enhancements to open source tools like git, selenium, elasticsearch, etc.

But to answer your question selenium/puppeteer looks like a good tool for automatic website monitoring and testing.

<comment><USER5> Running Puppeteer for monitoring is 50% of the SaaS I run. You can totally run it in a cronjob. Getting it to behave predictably and consistently and delivering actionable results is a different thing though.

<comment><USER4> Yes I think browserless.io is a very good site if you want to avoid those headaches with puppeteer.

But it may dilute your profits though since you'll need to buy a commercial license for it even though the project itself is open source. Depends on the money vs time you want to invest in doing it.

</comment></comment></comment><comment><USER6> maybe [https://cronhooks.io](https://cronhooks.io) ?</comment>


From Python Script to Maintainable Package – Podcast - <AUTHOR> https://testandcode.com/80 <comment><AUTHOR> This episode is a story about packaging, and flit, tox, pytest, and coverage. And an alternate solution to "using the src".

Python makes it easy to build simple tools for all kinds of tasks. And it's great to be able to share small projects with others on your team, in your company, or with the world.

When you want to take a script from "just a script" to maintainable package, there are a few steps, but none of it's hard.

Also, the structure of the code layout changes to help with the growth and support.

Instead of just talking about this from memory, I thought it'd be fun to create a new project and walk through the steps, and report back in a kind of time lapse episode. It should be fun.

Here are the steps we walk through:

0.1 Initial script and tests

0.2 build wheel with flit

0.3 build and test with tox

0.4 move source module into a package directory

0.5 move tests into tests directory</comment>


Show HN: Graph Processing with Postgres and GraphBLAS - <AUTHOR> https://github.com/michelp/pggraphblas <comment><USER2> Since you are mentioning Redis Graph and I'm not really into academic side of graphs, I want to ask - how do you use it from the user perspective, who wants to do some simple graph traversals using Cypher or some equivalent? For Redis Graph you have some nice higher-level API, is something similar planned for pggraphblas or outside of scope for this project?

<comment><AUTHOR> There is work being done by a standards committee on a common high level Cypher like language:

[https://www.gqlstandards.org](https://www.gqlstandards.org)

Long term that's maybe a promising target. That's out of scope of pggraphblas now, but like RedisGraph uses GraphBLAS internally, one could implement a Cypher or GQL look alike using the foundational bits of pggraphblas.

Edit: and to answer your actual question, if Cypher, or SQL/GQL for that matter, is "high-level" user experience then the GraphBLAS API experience is sort of aiming for the "mid-level". Still quite technical, but more powerful tools than for "low-level" which would be writing graph algorithms by hand in some procedural language.

One of the key goals of GraphBLAS is this sort of meeting of the minds between the hardware experts and the graph problem solvers. This one page poster kind of sums that up nicely:

[https://resources.sei.cmu.edu/asset_files/Poster/2016_020_00...](https://resources.sei.cmu.edu/asset_files/Poster/2016_020_001_484268.pdf)

</comment></comment><comment><USER3> I am following your discussions on the GraphBLAS email list. Are you interested in the distributed-memory API of GraphBLAS (for clusters, typically over MPI) or can you make use of it if it were available (you may know, CombBLAS already has a distributed-memory implementation)?

<comment><AUTHOR> Yes, I have only reviewed ComBLAS's documentation, but would eventually like to dig into it deeper. A thought I had for distributed graphs would be something like support on top of a distributed store like citus, where shards hold subgraphs. This may be an interesting way to sidestep Postgres' 1GB limit on varlena objects as well.

Edit: I'd forgot to ask, do you have some ideas on the subject? I'm all ears.

<comment><USER3> Thanks for the clarification. I basically wanted to know if database systems could make use of existing distributed-memory HPC programming models, and it seems that there is a possibility, however engineering efforts might be significant. I really have no experience with database engineering, and I am not sure if MPI and Citus will play out well. I hope you are considering submitting a paper/poster at HPEC'19 for your current work ([http://www.ieee- hpec.org/](http://www.ieee-hpec.org/)), I think that's a good venue to engage with the GraphBLAS community in person.

<comment><AUTHOR> Good idea! Hopefully I can put something together.

</comment></comment></comment></comment><comment><USER4> Has anyone done a similar thing with Spark and Postgresql for large graphs ? I'm curious to know how you write and structure your algorithms, given that graph partitioning is quite tricky (and not possible for a lot of algorithms).

I'm guessing Google uses a variant of this at scale - but not sure if their label propagation algorithms can be implemented on something like Spark and Postgres.

</comment><comment><USER5> A slightly off topic question regarding the mxm (matrix times matrix), mxv (matrix times vector), and vxm (vector times matrix) functions. AFAIK, Postgres supports function overloading. With the new types of matrix and vector, would it be easier to have a single function for multiplication, than three separate ones?

Also, can we overload operators in Postgres?

<comment><AUTHOR> The * operator can be overloaded, and it is already done s so for the mxm, mxv and vxm combinations. The catch is, all these AxB functions have more arguments than just their operands (mask, accumulator, semiring, descriptors). So you can `a_matrix * b_vector` and get "classical" matrix multiplication, but if you want accumulation, masking, a non-default semiring etc, you have to use the functional notation.

</comment></comment><comment><USER6> I found this project on Github a few weeks ago and have been quietly following it.

It looks really cool but I didn't get from the readme how I would best take my relational tables and make a set of matrices from them.

I'll take another look this weekend

<comment><AUTHOR> Matrices are constructed from arrays of rows, columns, and values. The simplest way to pivot a table into a matrix is to use array_agg()

      select matrix(array_agg(row), array_agg(col), array_agg(val))) from table;

For the inverse operation there are set returning functions called 'matrix_elements_<type>(m)' which return the graph elements as row tuples.

From a table management standpoint, matrices are varlena objects much like arrays. Triggers can be used to call set_element() to keep matrices and tables sync'ed, although like arrays, this has a high re-serialization cost for large arrays. Because of this it's best to modify matrices in bulk operations.</comment></comment>


iCloud Drive can strip metadata from documents - <AUTHOR> https://eclecticlight.co/2018/01/06/icloud-drive-can-strip-metadata-from-your-documents/ <comment><USER2> The original source for the article is a bit clearer on what's actually happening, and the blog post for the HN link omits some details:

[https://eclecticlight.co/2018/01/06/icloud-drive-can- strip-m...](https://eclecticlight.co/2018/01/06/icloud-drive-can-strip- metadata-from-your-documents/)

I recommend read the original (and that the submission be changed to the original source instead), but a more accurate description here is that two Macs running two different versions of macOS will see the file differently once it's copied to iCloud. If a High Sierra Mac sends a file with a custom icon, for example, to iCloud, the High Sierra Mac will see the file as expected in iCloud. If a Mac on Sierra looks at the same file on the same iCloud share, it will see the base file without the custom icon. Other xattr items are removed as well, as documented in the original source.

<comment><USER3> Thanks, we've updated the link from [https://mjtsai.com/blog/2018/01/08/icloud-drive-can-strip- me...](https://mjtsai.com/blog/2018/01/08/icloud-drive-can-strip-metadata- from-your-documents).

</comment></comment><comment><USER4> This title sounds like the actual files get modified by iCloud. This is not the case. Only external data ('extended attributes') in the filesystem gets stripped. It is true though that this is called metadata, but so is EXIF in photos, or OCR text in PDF scans, which definitely do not get stripped.

Also note extended attributes do not survive most file transfers.

<comment><USER5> This is what I'm wondering. Does Skim (the example used) actually store the annotations inside of the original document in a standard fashion or is it stored in some kind of additional file or as a standard xattr. If the latter is the case, then the OP needs to check to see if High Sierra is using APFS or HFS because High Sierra should have automatically converted the filesystem to APFS which has xattr support natively while Sierra, by default, would still have used HFS+ which stores xattrs in the B*-tree and not in the standard metadata for the file. Since iCloud would only sync over filesystem supported metadata, the xattrs would be dropped by Sierra.

This is like complaining that a file created in Windows 95 called FileWithAReallyLongName.doc would be truncated when saved in MS-DOS as FILEWI~1.DOC.

<comment><USER6> The internal storage of xattrs is an implementation detail. They look the same to applications, and copying from APFS to iCloud Drive should function identically to copying from HFS+.

</comment></comment></comment><comment><USER7> It's tough not to scoff. In years past it was a relief to see iCloud Drive/Mobile Me didn't simply strip out an entire directory. It was honestly a relief when Dropbox came along as at least you felt you could recover with their 30 day retention.

There must be mid-tier oldies like me who stopped using iCloud Drive due to those early loss of trust issues?

<comment><USER8> I don't think I lost any data on my iDisk, but the connection wasn't always reliable. IIRC it was mounted over WebDAV, but may have been AppleShare before they switched to that. Flash drives were definitely safer (but more expensive) option. I can't remember ever losing one while I needed it.

</comment></comment><comment><USER9> Although I haven't used this feature, the behaviour with the custom icon and filesize seems extremely unusual and I'd be quite puzzled if I came across this without knowing about it first --- imagine opening a text file whose size is shown as 500KB+, but discovering it has a tiny fraction of that --- I'd probably be shocked and, if it was a file someone else gave me, wondering "where's the rest of the data? Is my filesystem corrupt?"

Of course, if it doesn't count xattr sizes, it would be equally confusing in the other direction. Maybe showing "1372 bytes (and xxxxxx bytes extended attributes)" would've been better.

But from past experiences, IMHO xattrs and similar "bonus filesystem features" are not something you should rely on in general for portable data exchange. The name (or a slightly modified variant thereof) and the associated sequence of bytes are all that can really be relied upon to reliably survive transfers, since it's the lowest-common-denominator abstraction.

PDF has annotation features in the file format itself; perhaps Skim should use those instead --- in which case annotations will also survive and be visible in a Windows or Android PDF reader.

I'd also consider things like icons to be "display metadata", in the same way as the size of the icon or its position in the GUI use to display it.

</comment><comment><USER5> I wonder if the OP and the article linked in one of the other comments looked at whether or not the Sierra install was on APFS or not. High Sierra automatically converts all drives to APFS during install while Sierra does not. Since Sierra stores xattrs in a special B*-tree node rather than as extended filesystem attributes, it's possible that the High Sierra install is saving them to iCloud Drive properly with all the filesystem attributes and the Sierra install is dropping them, for security reasons, as they would have been invalid on an HFS+ filesystem.

That should be viewed as a bug from the user-perspective but, because those extended attributes literally did not exist when Sierra was released, it's hard to say that it should be considered a bug for Apple when it's functioning exactly as intended in terms of the filesystem. It's almost like complaining when MS-DOS saved files as NAMEOF~1.DOC instead of "Name of Really Long File.doc" from Windows 95.

<comment><USER10> > Since Sierra stores xattrs in a special B*-tree node rather than as extended > filesystem attributes...

I think you've misunderstood something.

"Extended filesystem attributes" is literally just what xattrs stands for -- "eXtended ATTRibuteS". "A special B-tree node" is one way that a filesystem can choose to internally store xattrs. They aren't two separate things.

<comment><USER5> I haven't misunderstood it (I don't think). B*-tree is basically a hack to store extended attributes that weren't initially included in HFS. APFS includes those extended attributes natively. I'm just wondering if the difference in how they're stored is the reason for the loss.

<comment><USER9> I doubt it, since the APIs to access them are the same.

</comment></comment></comment></comment><comment><USER11> Unrelated, but IMO, the clients for iCloud Drive on both iOS and OS X are very poorly implemented. I used to have a 16 GB of iPhone that would magically go from 8GB of available storage to 0 despite me not downloading anything onto it. It turns out, if you have iCloud Drive enabled on iOS, it automatically downloads the entire content of your drive to your phone and in the settings app, it classifies it under the "other" so there was no way for me to figure out what was eating up all of my phone's storage.

The mac client isn't any better, it sometimes utilizes 100% of CPU to sync a few files (bird process on OS X) with iCloud and there's no way of stopping the sync or slowing it down.

</comment><comment><USER12> Slightly Off Topic Question.

Why is Apple continuously pushing its customers towards cloud, when many ( more then 40% ) in developed world dont even have access to 10Mbps + Internet Connection. ( And that is just download speed )

</comment><comment><USER13> Not related, but when I adjust chmod for files in iCloud, they revert back (during next sync cycle I presume). Just something that may or may not be interesting for folks looking at iCloud.

</comment><comment><USER14> I wonder if its a security precaution.</comment>


Show HN: Scrapp – a lightweight bookmark manager PWA - <AUTHOR> https://scrapp.it <comment><AUTHOR> This has been a labour of love to create a web based bookmark manager that I can use across all devices without having to install anything. I've learnt a lot about building this PWA with web components as a well as more than I care to mention about the intricacies and inconsistencies of site meta data! I hope other people find it useful.</comment>


Interview with Don Chamberlin, designer of SQL - <AUTHOR> https://www.mappingthejourney.com/single-post/2017/10/12/episode-11-interview-with-don-chamberlin-designer-of-sql-database-language/ <comment><USER2> Disappointingly low on technical detail... For me the most interesting part of an interview with the original author of some well-established technology is when he or she is discussing something that you take for granted and says something along the lines of "oh, that thing is more of a historical accident, I would design it a bit differently today". These nuggets can really broaden your perspective, but I couldn't find any of them in this interview.

<comment><USER3> That's a problem with a lot of interviews. Journalists almost always go for the "human" side but I enjoy reading about the technical evolution and how things developed.

I just read a book about the development of windows NT. Same thing, a lot about people but not much about technical stuff.

</comment></comment><comment><USER4> As someone who uses SQL extensively every day at my job, I really appreciate the thought that went into it's design. It is by far the most sane part of what I do every day.

<comment><USER5> Completely agree! Although most of the developers I interface with look at me like I'm an alien. So I don't know if SQL being the sanest part of my day is a point _for_ SQL or a point _against_ the general sanity of my day.

</comment></comment><comment><USER6> Took an undergrad comp sci class taught by Don Chamberlin at UCSC in 2009. He is a fascinating professor!

</comment><comment><USER7> I've been subscribed to this podcast since his 2nd or 3rd episode. He's had some interesting guests.

I think the podcast is a one-man show. He could probably use some help with mixing/production (and a new intro theme), but for a one-man show it's pretty darn good.

Edit: I'm an idiot and just realized OP is the podcaster.

<comment><AUTHOR> Thank You. Yes, I'm a one-man army and would use some help. Finding interesting guests have not been hard but doing this week after week has been hectic, but I love it. Maybe someday I can run the podcast as a team.

<comment><USER7> Well considering you're doing it all by yourself, you are doing an awesome job! I look forward to each new interview. I like that you give the guests time to provide an in-depth answer without peppering them with too many questions.

When you're a famous podcaster you can do a Mapping the Journey episode about Mapping the Journey! :)

</comment></comment></comment><comment><USER8> I was glad for a transcript!

I find it interesting that the technology of the time meant it was hard to write ∀ and ∃ (though APL users seemed to go OK) and so they chose a complex syntax that reminds me of COBOL. The early database applications were business applications (SABRE being #1 I believe) so this wouldn't have been at all an unreasonable basis for SQL's syntax.

</comment><comment><USER9> Mr. Chamberlin was my dad's roommate at Harvey Mudd and worked with him at IBM. It amazes him how much Sequel runs the world now.

</comment><comment><USER10> One thing I thought about after reading this interview was just how much these grassroots projects (at the time) were done just as a labor of love. Also, just how free companies were with information. Publish a paper and share it with the world.</comment>


Israel co claims to have a 100% defense.Offers $200K to anyone who can hack it - <AUTHOR> https://www.cyber20.com/ <comment><USER2> > _\- Our challenge includes computers and servers, configured for file > sharing_

> _-No other special configuration: no firewall and no antivirus or other > defense programs are installed_

> _\- The only defense mechanism is Cyber 2.0 in System_

> _\- Hackers will gain access to the network via a Wi-Fi access point_

> _\- They will also receive the admin user name and password for each of the > computer and servers participating in the challenge_

The news video on the page also says hackers were given just 5 hours on site.

How does this compare to other security challenges? 5 hours seems like a very short amount of time to me. No doubt someone genuinely interested in hacking your corporate network is going to put more than 5 hours effort it.

</comment><comment><USER3> Israeli startups are always over confident about everything. They have a track record of delivering but also making bogus claims like this.

</comment><comment><USER4> It's a pretty shitty looking website considering the confidence. Maybe there's just nothing to actually hack.

</comment><comment><USER5> They don't really expect it to happen, don't they?

</comment><comment><USER6> Offering bounties for hacking your system is a good idea.

Claiming to be unhackable seems dishonest. How would they know?</comment>


Libgphoto2 Port for iOS Released - <AUTHOR> https://github.com/touchbyte/GPhoto2Framework <comment><AUTHOR> touchbyte today released an iOS version of libgphoto2 - a library to access digital cameras. libgphoto2 has been adopted to run on iOS and allows controlling supported digital cameras using the PTP/IP protocol over Wi-Fi. The adopted library is licensed under LGLP2 and used in PhotoSync 4.0 for iOS ([https://www.photosync-app.com](https://www.photosync-app.com)), a product by touchbyte which enables powerful and flexible photo & videos transfers and backup. The full source code including an example app is available at [https://github.com/touchbyte/GPhoto2Framework](https://github.com/touchbyte/GPhoto2Framework)</comment>


Ask HN: Resources on modern SE best practice? - <AUTHOR> I'm looking for recommendations on modern best practices in:
Software engineering; Software project management; How to properly, efficiently, and clearly document software; Best practices for programming in teams; etc.
I realise there is no one-size-fits-all solution, but I'm hoping to find a variety of sources that I can filter through for gems.
Backstory: I'm moving from academia (robotics & ML research) to industry in 4 months and am looking for some resources to help me transition from working on single-person/small-team research projects and codebases, to those of a 30-engineer company.
The company is in the embedded systems & robotics space and has recently found success and is looking to grow. They have engineers who are technically excellent but lack formal training in software engineering and project management. They have a few guidelines in place for source control (Git) usage, and have some quality control & testing processes implemented, including running integration tests of releases on the physical systems continuously throughout the day.
I'm entering the company at a rather senior level, since my research work focused on exactly an area that they're looking to expand in. I will spend the first half year getting to know their robotic systems. Longer term they would like me to hire and manage a small team working in my field as part of their expansion strategy.
I've had masters-level courses in Software Engineering and Project Management, but this was many years ago and I expect the material of these courses to be very outdated. <comment><USER2> As no-one else has replied, randsinrepose always used to be popular around here:

[https://hn.algolia.com/?query=randsinrepose.com&sort=byPopul...](https://hn.algolia.com/?query=randsinrepose.com&sort=byPopularity&prefix=false&page=0&dateRange=all&type=story)

But these days I don't know.

Everyone says they follow agile, or agile methodology, but it can mean everything from actually following the [https://agilemanifesto.org/](https://agilemanifesto.org/) and letting the engineers determine their own working practices, to extremely prescriptive process like Scrum and Extreme Programming (XP). Personally I'd be wary about any overly prescriptive processes like scrum, which tend to treat grown adults like children and is often little more than thinly disguised micro-management.

<comment><USER3> The biggest issues I've run into with scrum and agile has been the disconnect between upper management both wanting transition to agile, and a firm commitment on what exactly is going to built over the next 3month/year.

</comment></comment><comment><USER4> On the project and management level read Rapid Development by Steve McConnell. On the detailed technical level read Code Complete by same author.

Then read some blog posts on agile, scrum and kanban only to get familiar with the jargon.</comment>