skysight-inc/HackerNews huggingface dataset:
I just did a simple benchmark: 67 million rows, integers, 4 columns wide, with postgresql 10 and pandas.

pg 10

huge=# \timing on
Timing is on.
huge=# copy lotsarows from '~/src/lotsarows/data.csv' with csv header;
COPY 67108864
Time: 85858.899 ms (01:25.859)
huge=# select count(*) from lotsarows;
count
----------
67108864
(1 row)

Time: 132784.743 ms (02:12.785)
huge=# vacuum analyze lotsarows;
VACUUM
Time: 185040.485 ms (03:05.040)
huge=# select count(*) from lotsarows;
count
----------
67108864
(1 row)

Time: 48622.062 ms (00:48.622)
huge=# select count(*) from lotsarows where a > b and c < d;
count
----------
16783490
(1 row)

Time: 48569.866 ms (00:48.570)

pandas

In [2]: import pandas as pd

In [3]: %time df = pd.read_csv('data.csv')
CPU times: user 34.1 s, sys: 4.49 s, total: 38.6 s
Wall time: 38.7 s

In [4]: %time len(df)
CPU times: user 125 µs, sys: 19 µs, total: 144 µs
Wall time: 166 µs
Out[4]: 67108864

In [5]: %time ((df['a'] > df['b']) & (df['c'] < df['d'])).sum()
CPU times: user 1.74 s, sys: 135 ms, total: 1.88 s
Wall time: 1.88 s
Out[5]: 16783490
Voice certainly seems like a viable solution to some of that. In some ways it’s imminent if the accessibility improvements pushed to iOS, iPadOS and macOS are advanced just a bit further.

However, we should also expect to see PC use pushed into more advanced and specialized territory on our most powerful and versatile devices as phones and tablets assume more traditional PC work. That specialized use will advance as fast as the most attuned interface for that platform (keyboard+mouse) and the others will necessarily lag behind.
I'm not the person you replied to, but is this new info? I could have sworn I read all this last year...
Australia has in some states added nCov-19 to the standard testing panel for all respiratory illnesses, and we haven't seen a spike in detection so far. I'd expect to see an increased detection rate if it was in-fact endemic.
Yeah but it was 1 year ago...
I’m just here to support the four people that have a sense of humor on HN.
Sprint/T-Mobile merger gets federal judge approval
Reverie Labs (YC W18) Is Hiring Cloud Architects in Boston to Help Cure Cancer
I found it interesting from a legal point of view when someone pointed out that the recent "AI dungeon generator" that was using BERT to act like a game master was in some occurrences basically copying (relevant) excerpts from books.

Can an AI commit copyright infringment? BERT probably "knows" that Cthulhu is a giant thing evoking squids, tentacle and non-orthonormic dimensions. These are facts based on books, but you can produce copyright infrigement based on those facts. It is called "producing a derived work".

In the past years I never managed to get anyone with legal knowledge interested in what they saw as a totally impossible scenario: the idea that AI could one day produce original work y learning its craft, like human do, from copyrighted works. Their criterion was "if you fed copyrighted work into an algorithm to produce a new work, then that's a derived work".

Humans are somehow imbued with a magic property that allows them to watch read WH40K books, alien and predator movies, then produce the Starcraft universe, and have it count as original work.

We do have a philosophico-legal discussion to have there. And way overdue, if I may. The state of copyright is already late in acknowledging internet, DL-generated work will be even more of a conundrum for it.
Collocation providers now have options to put your physical environment on network with your amazon account to avoid egress fees.
Using audio channels as a social signal for remote workers
In don’t think anyone mentioned a listing of all possible melodies either, and if you think something they did mention is functionally equivalent, it would help if you made that explicit.

Algorithmically generated music is not such a list, and does not involve publishing such a list, as no one would want that. They typically involve a lot of selection and choices by the author. They’re only algorithm-generated on the sense that the Sistine Chapel’s ceiling was “brush-generated”.
When it comes to actual recipes, I find that most things I like come from a lot of repetition to adjust the details and figure out the pitfalls, so even if it's generally similar to what was published, a lot of value has been added from my perspective.

Code can be written deliberately in a kind of unfinished, unrefined way, and let those who would use it go through a similar process.

But the GPL seems significantly different from the absence of copyright.
> An engineer should know better

Maaaaybe? I'm not a hydraulic engineer and won't pretend to be one. So I hire plumbers when things go wrong. If they screw up, I don't tend to blame myself though. I think the same logic would apply to the deceased.
I wouldnt put infectious disease epidemiologists on a pedestal either, whilst they can use plain old Police-like detection methods to identify so called Super Spreaders, the egit's can not identify what makes someone a Super Spreader. Considering the medical profession are so limited in their ability to measure the content of every cell and condition of every cell, cell sensing, signalling, enzymatic activity and radical activity in-vivo, eg localised hydroxyl radical activity in the presence of pathogens, infectious disease epidemiologists are no better than social commentators if society is considered an organism, or economists who swear to an economic school of thought on a par with a religion of your choice.

All experts have a common trait, they espouse their virtues when they are right and slink into the shadows when they are wrong, sometimes, if they have the audacity they wait for the next opportunity before popping up like a jack in a box.
Ask HN: When did you decided to start freelance/consulting or start SaaS and why
Seasonality is a...vexing problem. It seems to straight forward, and then...
Digital Hoarders
Herb Sutter also has presentation on this: https://www.youtube.com/watch?v=ARYP83yNAWk
> I really like seeing the word "for" twice when you're iterating over two things.

You will if you write a Python list or generator comprehension that iterates over two things, won't you?
From the using the right tool for the right job perspective, can any expert let me know when is the best time to use awk or sed if other tools are also available? I know awk and sed are different tools so maybe I'll ask a more general question: given other tools e.g. Python are available, what's the suitable scenario to use cl tools?
The solution manuals for the 1st and 2nd edition are both easily found online with a quick search. As for the 3rd edition, I was able to find this [0], although I haven't verified if their answers are correct.

But it would appear that you're correct that Pearson doesn't seem interested in selling the 3rd edition solution manual to anyone that isn't an instructor [1]. IMO, fuck their antiquated gate-keeping bullshit.

[0] https://dreamanddead.gitbooks.io/csapp-3e-solutions/content/

[1] https://www.pearson.com/us/higher-education/product/Bryant-I...
Sort of relevant: https://www.gwern.net/GPT-2#random-samples
And you can see this structure if you add to any "git log" command "--graph --oneline --decorate --color". IIRC some of those are unnecessary in recent versions of git, I just remember needing all of them at the point I started using it regularly.

I have a bash function for it (with a ton of other customizations, but it boils down to this):

function pwlog() {
git log "$@" --graph --oneline --decorate --color | less -SEXIER
}
pwlog --all -20

(...in that "less" command, "S" truncates instead of wraps lines, one "E" exits at EOF, "X" prevents screen-clearing, and "R" is to keep the color output. The second "E" does nothing special, it and "I" (case-insensitive search) are just to complete the word)
Just a few days ago I wrote a simple awk script to parse some log files but it was horrendously slow. I had to replace understandable loops with weird calls to builtin functions to make it fast enough for my usecase.
bad_alloc doesn't just naturally spring into being. If you override operator new, your implementation may not use it (for example tcmalloc does not throw if it can't allocate, it logs a message and terminates the program). Many systems of the type you mention, embedded control systems and the like where correctness actually matters, don't use new and delete anyway. They often use placement new with space statically allocated at program startup. These are the same kinds of projects that often decide to not use exceptions.
Unintended accelerations in Nissans were blamed on bit-flips due to cosmic rays. I think that one actually was true in the end. So unless you'd like to test the CPUs near nuclear reactors and electron guns, I'd say that there is a (cosmically) small risk of it turning on despite not enabling it.
Mythical man month: 10 lines per developer day
The obvious dividing line is, the copyright holder wants their data (creative work) to be spread as widely as possible (to paying customers), whereas someone with a secret potentially wants no one in the world to receive that data.

I don't think anyone is claiming that composers should be forced to publish their songs against their will.
/R/Datahoarder
Well... fancy features/abstractions are pretty low cost, if you use just one. It's when they interact that they really get expensive unless they are almost flawlessly designed to work together (and even then, they just get somewhat more expensive).

It's like the first donut has 100 calories, the second has 200, the third has 400, and the fourth has 800. If you eat the whole box, it's really going to hurt. Rather than eat them all, you need to pick the one or two that you really want, and leave the rest in the box.
The History Of WD-40
Use one of the many tools for customizing your keyboard layout. {} are Alt+J and Alt+K for me.
All Programming Languages are Wrong (2018)
> The worst part is that you can’t get rid the incontinent piss dribbles by whacking the reader mode icon

Eh, they don't animate in Firefox, so, a simple solution presents itself. :)
It's a good-enough description of SVN, where branches exist in the same directory tree, commits are tied to the branch by way of the path, and the merge tools are "merge this batch of commits from branch A to trunk" (you don't have to take the whole branch at once).

One of the biggest hurdles my co-workers have had learning git after having used svn for years is the "bucket of commits" mental model they've built up for branches. A common question is how to merge a single commit.
SaaS is harder to startup but I enjoy building a business where the we can have revenue without having to find clients / perform labor. SaaS is hard but rewarding when you hit X recurring revenue. I can travel the world and just have to deal with support and sales.
What's the difference between a stash and a "working tree as a commit"?
The amount of horseshit that's exploded all over any simple-ass system that could fit comfortably on one real mid-range server, these days, is truly astounding. I thought I was used to churn and such in this field, but the last five or so years are really starting to strain me. The amount of junk one increasingly must know to work on these over-engineered systems—and they never have enough personnel to keep everyone from having to know & constantly work with a dozen different tools and interfaces just to get anything done, on top of what you need for the work of actually writing and working with code—is getting to be more than I can handle.

I only hope if anything good comes out of the next bubble burst it's that some of this piles-of-cash burning counterproductive insanity gets reigned in.
> Rewrite the slow bits in C++/Rust/Go and wrap it.

For reposurgeon, you can't. Not the author, but I have done some hacking on it. "The slow bits" are not things you can rewrite in a different language and wrap. They are much too integral to the code.

> That's how all major Python components like Numpy, Scipy, Tensorflow, PyTorch etc. does it.

And what all of these have in common is that the "slow bits" are not like those in reposurgeon. The wrapped speeded-up things are basically fast implementations of appropriate basic data types, like Numpy arrays and vectors. Those aren't the kinds of things that are slow in reposurgeon.
Ah yes, the reliable BackBlaze folks. That they've out-Googled Google in a niche using mostly commodity infrastructure and kept their business alive for so long is a testament to their ingenuity (I wonder how their operating costs compare with AWS Glacier which has a theoretical advantage of unpowered disks.). And the releasing of this proprietary operational business data is a testament to their coolness factor.

It's a timely article as I'm looking at HC530's (WUH721414ALE6L4 / WUH721414ALN6L4 (wiredzone carries it)) for a home FreeNAS box:

- any relatively-modern enterprise 4U 3.5" storage box with Xeon 4 cores or so

- quieter, high-volume fan mod

- RAM: 64-128 GiB, beyond that isn't useful unless deduping

- NIC: X710-T4L 4x 10GbE copper NIC

- ZIL: mirrored pair of high-endurance, write-intensive, reliable SSD like Optane 900p/905p 280-480GB

- L2ARC: striped pair of read-intensive/larger SSDs like the Gigabyte Aorus Gen4 1 TB

This will fit nicely as my home NAS for a water-cooled dual EPYC virtualized server/workstation build underway. I managed to get a single water block with (3) G1/4 connections that will cool both CPUs and the VRM chokes/converters.

If anyone has better suggestions, please chime in.
There are a few DIY, template-based video-making tools such as GoAnimate (now Vyond) and Biteable, that are pretty good. They are cost-effective and serve a certain need. They are in fact, as you mentioned, like a Canva for videos.

Custom videos made by animation studios serve a different requirement, because of the creativity, quality and effectiveness they bring in. We are making Homer for this market.
I don't like to say this but Mozilla leadership sucks. In the light of previous layoffs, there was a discussion that the CEO, who doesn't even have a proper technical background, is getting too high of compensation. I have also heard Mozilla has become a Suits-first company lately.
As a software engineer it's hard to trust code that you know has flaws, because there could be a flaw that causes autopilot to automatically turn on. Extremely unlikely, I know, but it's definitely a nagging thought. It would be nicer if there were a mechanical cutoff (not really possible in this case) so you could be absolutely sure.

I know, I know -- you can design software so that this type of flaw is completely impossible, but as a user I have no way of verifying this, and my trust has already been eroded by the feature I'm trying to disable so I really want to verify it.
> Align with the core strengths of Python's philosophy and its toolset and get the benefit. Why fight it ?

In this particular case, it's worth noting that reposurgeon highlights some core weaknesses in Python's philosophy and its toolset. The two biggest are the amount of memory required for even basic objects and the GIL.
I don't know if I'm an "expert" but I only use awk and/or sed for simple shell scripts, when I'm munging around in a terminal, or for simple CLI tools I write for myself. If I'm writing code that I need to maintain, could grow, or needs to be contributed to in a team setting...I write it in a real programming language.
For me, no. At least, not yet.

The 16 inch requires macOS Catalina, which kills all 32-bit apps, among other things.

My employer still has some critical apps we use that are 32-bit, and while I wouldn’t be buying a 16 inch MacBook Pro for work, I don’t want to be forced to use Catalina at home. At least, not yet.

Maybe in six months. Or maybe not. But by then, it may be too late and many people may have moved on to other machines. I know I probably will have done so by then.
Be Optimistic About the Future [video]
Is this a reference to something? I can't imagine it would take 500mb to reimplement git in JavaScript. This one's <75kb and full of explanations: https://github.com/maryrosecook/gitlet
You wouldn't take the huge pile of features in an all or nothing. You get to pick and choose. I think the authors wish list is similar to most people who are experienced with more expressive languages. I don't see calls for Python f-strings or async syntax to end up in Go. But something like a list-comprehension syntax I think would be really popular (based on what I saw when it was introduced to Python, originally 'why' but now many people's favorite feature).

Interesting how Python is now considered a huge pile of features, considering how long it took for things like a ternary operator to end up in the language. Maybe Go is in it's Python 1.5.2 lifecycle stage ;)
My 5-star system:

: I want to give this more than 5-stars

: This is pretty good, but not MORE than 5 stars

: I've enjoyed this.

: I need to keep this for completist reasons

*: Delete this.

Oh wait, that's my iTunes rating system
Financial Position of the United States
tl;dr - It depends on who's asking.
I posted this two days ago. I believe a moderator must have rescued this post today to bring it to the front page.

By the way, I posted this link because I've been running my own mail transfer agent (MTA) for my personal vanity domain to send/receive emails for about 10 years now and I often forget why I need to edit /etc/mailname while setting up the MTA, so I posted this link to HN as a bookmark for myself. (Who else uses HN submissions as bookmarks?)

I use Exim 4 as my MTA on Debian GNU/Linux. During its setup, among many other things, I need to configure /etc/mailname with the domain name I want to see in the "From:" field of outgoing security notification emails, e.g., email notifications for failed sudo attempts.

If you want to take a look at my MTA setup, see this Makefile: https://github.com/susam/susam.in/blob/master/mta/Makefile

Related Show HN post: https://news.ycombinator.com/item?id=22299169
I would, but I expect my HR rep would instruct me not to.
I recall a couple of efforts to write a BLAS in pure Julia. https://discourse.julialang.org/t/we-can-write-an-optimized-...
We need to invent a quantum ratings system.
What's the complication on the fitting in cache? The error handling is taken out of the code that's loaded in icache. When you throw, a runtime library call is made that walks your stack. Until throw, that code has no reason to be in your icache.
I think the problem is the idea of 'tooling'.

A table saw by itself is mostly useless. With a fence and a miter gauge, it becomes useful. With a push block, stop block, subfence, outfeed table, infeed table, featherboard, crosscut sled, tenon jig, and dado set, it is the single most useful tool in a woodshop. Keep in mind it is still one tool and all those accessories are not "tooling", they are accessories to a single tool that increase what you can do with it. The tool always works the same way, and anyone can use it with any of those accessories in any woodshop in the world. In effect it becomes a new, larger solution, made up of many features that extend the utility of the tool.

That's not really what we have. Mostly what we have are jigs. A jig isn't an outfeed table or a crosscut sled. It's a hack for a particular job. If you need to make one specific cut 300 times, you nail together some scrap wood, dial in the miter gauge, angle the saw blade, and make your cuts. And the jig is scrap once again.

But in the heady new world of "DevOps engineering", the jig is now "tooling", and we pat ourselves on the back that we were able to nail some scrap wood together and claim it created business value. Of course, it's not a shitty jig like in the bad old days of shell scripts ("ha ha! remember when we were productive with this simple code that was portable and not gigantic or complicated? how foolish!"), because instead of making it out of scrap wood, we now make it out of scrap steel with a MIG welder. We're advanced now.

And I'll go further. The fact that most woodworkers make their own tablesaw extensions is illustrative of the problem: craftspeople like having fun with their toys. Is there value and experience and dollar savings you get out of making your own crosscut sled? Sure! But it'll also take you 1-2 days of buying parts, measuring, cutting, gluing, clamping, drying, aligning, and finishing. Any business with any sense should have paid $100 to just buy a complete crosscut sled made of aluminum with a good design that will last forever. But they are too dumb to notice they're spending an inordinate amount of time and money on craftspeople making jigs.

I wish the ghost of W. Edwards Deming would rise from the grave and call us what we are: bullshit artists.
Is being pleasing a requirement of copyrighted material?
This is closer to how I would describe HG and SVN. Branches can be traced from start to merge, they are heavy. You know which commits came from what.

In git you can lose track of what came from what branch when you start merging multiple back and forth, this does happen with svn.
> Where does the faith in the tech and its future come from though?

https://economictimes.indiatimes.com/small-biz/security-tech... (The Cambrian Explosion in Technology, and How It's Affecting Us |
Seeing the progress made across various fields over the last few years, we can ask if we are witnessing a Cambrian explosion in technology today)
Ex-HSBC customer in France here.
It was one of the worst banking experiences possible - incredibly expensive debit cards with very low ceilings, etc.

The "plus" that you mention, having a dedicated account guru can swing both ways: mine left my local branch for another branch 10km away, and I would not perform any significant operation without their presence.. which was no longer in the branch.

Boursorama or N26.
Fennec Textwrap addon for Firefox on Android will slightly improve pinch zoom text reflow.
"Thus, "Humans are much safer than people on average" is nonsensical."

Does it make more sense as "Humans, when driving in conditions suitable for Autopilot, are much safer than people on average"?

"I've chosen to take them at face value with a grain of salt, and to believe that for the data they've collected from the hundreds of thousands of Tesla's with millions of hours of data using Autopilot, it's fair to say they have a large enough sample to draw conclusions about the safety of their cars vs. any incident rates from pretty much any other distribution"

You seem to be saying that if you have a lot of data it doesn't matter what you compare it to. That seems wrong to me. Also, I don't have this data, and you are not bothering to help me find it.
I think the classic answer to that question involves taking a lot of time, doing some reading, and having lots of spare time for your brain to wander to interesting places, rather than the stuff you normally immerse yourself in every day.
I'm not an expert by any means but I suspect that the Mortality rate is lower than what they have said.

We've been told that the majority of people will get very mild symptoms if they have it. So the actual number of people that have been infected is way higher since many will never even think of going to a doctor.

So the 2-3% case fatality rate is higher than the actual number. If the number of deaths is correct or close to correct.

The scary part is that the virus is so easily spread between humans so the number of deaths will skyrocket when compared to other epidemics. Let's hope they bring it under control soon and there's a vaccine in the near future.
TransmogrifAI – AutoML for Apache Spark
What you described seems like it would have been dead simple in Flash.
no answer in the article. because this is the maximized value for items bought * avg paid?

also, popcorn COGS is nowhere near $0.90, if we are talking about goods only and packaging(as in COGS), its less than 10 cents, throwing markups numbers in completely different tier.