skysight-inc/HackerNews huggingface dataset:
> "I don't really see what there is to be gained by making scheduling part of the remit of pipelines."

I think it depends on the use case, sometimes the components of the pipeline aren't necessarily running on the same machine, and they don't know where and how to get access to data and artifacts generated by previous steps, and so scheduling and orchestration becomes an important component of the pipeline itself.

> "I'd rather assemble together a set of tightly scoped "UNIX philosophy" libraries and tools rather than try and use an all encompassing framework and be straitjacketed by its imposed structure."

I think the idea behind building such frameworks is to help people avoid going through the same steps of building such tool internally by "assembling together a set of tightly scoped "UNIX philosophy" libraries". In general these frameworks are using libraries and tools, and exposing an easy way to leverage them instead of spending time doing that over and over.
reminded me of: https://twitter.com/pasiphae_goals/status/923835427995508741...
That just sounds like a cynical thing to say, not a fact backed up by evidence.
I’ll admit that I’m probably biased, and I haven’t had enough experience living outside of the US to have gained a different perspective.

But, now I’m curious. Which countries are currently and historically known for being tolerant and accepting of dissent? Honest question - I’ve been considering retirement lately, and I’m up for a new adventure.
In my opinion, this won't stop you from from enjoying Infinite Jest. There are a few arcs that might require some insight into American culture (Professional Football, the US drug rehabilitation system). But the big picture themes are either very universal or so particular you aren't really expected to be keyed into them (you don't need to be a competitive tennis player or a Quebecois separatist to enjoy it).

Pale King might be a bit of a different beast. The focus on the IRS is somewhat particular, and you may miss big beats because of ignorance about the IRS. But at the same time, some of the accounting minutiae are such that I don't think you're expected to understand them.

Anyway! I wouldn't let this steer you away. To me, the joy of DFWs writing is the individual sentences. The mannerisms and humanity of the characters. If you miss some details because they're US focused, I don't think it'll be anything important.
Added support for this to git.sr.ht a while ago. It's used to set your preferred tab width and to show a ruler at the max line limit (which default to 8 and 80 columns respectively if you don't have an editorconfig file).
Show HN: RunMyCode Online – Run Code Online, Directly from GitHub
They print money and own many segments of smartphone that really matter. They can cruise for a decade doing nothing.
Mine will be a scripting language called Compiler
The fips example seems like a good one. "Don't care how, tick this box so I'm compliant with my government contract".

Not arguing it's a good idea, just that it might be wanted.
Unless the giant whiteboarding companies are also hiring people who fail the whiteboard interviews so they can see to what extent (or not) success in whiteboarding correlates with good employee, I'm unconvinced.
We have C, a language named after a letter, and Go, a language named after a verb, and your issue is with "Kernel"?

Just fucking add fucking "programming language" as the suffix to your fucking query about Kernel the PL, to mimic the tone of the original comment.
Hey HN, the author here. This is the second iteration of what I thought out to be a platform (then named CodeSheets) for code snippet repos with a built-in code runner. Instead of building my own code repo platform, I am utilizing GitHub, GitLab and BitBucket and have added runners on each of them through a browser extension. In addition to browsing and storing snippets on GitHub, now you can even execute them without leaving your browser! You can run a Hello World program in every language[1] right from your browser without installing anything! (Okay, not every yet, but you get the point.)

Due to the extensible architecture of the code, it is easy to add support for any website which has standalone code snippets. With the optional permissions, adding new websites (read host permissions) doesn't block extension updates. I have added support for Go by Example[2] and Tutorials Point[3] with just a few lines of code.

Even though it has been actually public for more than a year now, only recently I became confident in advertising it to a broader audience. Currently, it has more than 1000 weekly active users across Chrome and Firefox.

For more info, check out the FAQ[4]. Any feedback is welcome.

PS: I am currently a graduate student at ASU, looking for a Summer internship.

[1] https://github.com/leachim6/hello-world

[2] https://gobyexample.com

[3] https://www.tutorialspoint.com

[4] https://runmycode.online/faq.html
The editor can detect all of these for each file. If the editor doesn't, it probably don't support editorconfig either.
I absolutely love your comment on statistics.

But on the notion that large companies specifically don't have hit or miss design or redesigns: I'd argue there's more than enough evidence to the contrary. And, personally, I find Atlassian products to be a very good example of that.

Note: In this regard, I focus mainly on Don Norman's view on "design", which is largely orthogonal to aesthetics. It's cool for software to look nice. But if it's laid out in a non-intuitive way or doesn't prevent (or even provokes) user errors, it's not good design, period.
Cool article. My knowledge of statistics is really rusty, but isn't this another way approaching the topic of "Bayesian Thinking"? If you think about the scenarios in the article from the standpoint of predicting any given outcome in advance, male vs. female and hard department vs. easy department should be treated as "priors". Or to put it another way, Bayesian thinking means asking the question "What is the chance of X happening given Y?"

A nice intro to the topic: https://betterexplained.com/articles/an-intuitive-and-short-...

Which explains why a positive test on a mammogram means you only have an 8% chance of having breast cancer:

>The chance of getting a real, positive result is .008. The chance of getting any type of positive result is the chance of a true positive plus the chance of a false positive (.008 + 0.09504 = .10304).

>So, our chance of cancer is .008/.10304 = 0.0776, or about 7.8%.

>Interesting — a positive mammogram only means you have a 7.8% chance of cancer, rather than 80% (the supposed accuracy of the test). It might seem strange at first but it makes sense: the test gives a false positive 9.6% of the time (quite high), so there will be many false positives in a given population. For a rare disease, most of the positive test results will be wrong.
But communication on-the-fly, on-your-feet, in a high pressure environment, which is what a whiteboard test is, is not most communication.

Most communication gets thought about, drafted, revised, critiqued, socialized, ahead of distribution.

If your whiteboarding comes after hours or days of consideration and review, that sounds like a great way to vet good communicators. Otherwise, aren't you optimizing only for speed? If someone cracks a joke in the moment, they're a wit; but we only pay money to and call comedians those who worked on it for a while.
I was responding more to the idea that they're discriminating against PhDs who've left academia. If it's not meant for anyone who's graduated, then it's discriminating equally against those who've stayed.
I feel so frustrated by the emergence of these things and the constant attempt to brand them or stylize them towards data science.

There are just engineering projects. There are not any other things.

For some engineering projects, you need to support dashboard-like, interactive interfaces that depend on data assets or other assets (like a database connection, a config file, a static representation of a statistical model, whatever). Sometimes you need a rapid feedback system to investigate properties of the engineering project and deduce implications for productively modifying it. These are universal requests that span tons of domains, and have very little to do with anything that differentiates data science from any other type of engineering.

At the level of an engineering project, you should use tools that have been developed by highly skilled system engineers, for example like Make or Bazel, or devops tools for containers or deployment and task orchestration, like luigi, kubernetes tools, and many others.

For a web service component, you should use web service tooling, like existing load balancing tools, nginx, queue systems, key value stores, frameworks like Flask.

For continuous integration or testing, use tools that already exist, like Jenkins or Travis, testing frameworks, load testing tools, profilers, etc.

Stop trying to stick a handful of these things into a bundle with abstractions that limit the applicability to only “data science” projects, and then brand them to fabricate some idea that they are somehow better suited for data science work than decades worth of tools that apply to any kind of engineering project, whether focused on data science or not.
How to clean up Ubuntu software package repositories
The Most Important Skill
No way. There’s a huge market for medical monitoring that is dominated by big dumb players.

Your insurance pays like $3k to wear a heart monitor for a day or two.
No, I'm not making the argument that there aren't coercive social forces. Peer pressure is a real thing, and I've had to endure it, just like everyone else. In my opinion, what social pressure tends to do is exaggerate existing biological differences, enforce conformity (pressure you into behaving more like others). I'm not arguing that this is a good thing either. I didn't like that kind of social pressure growing up.

The argument I'm making is that pretending that typical male/female behavior is exclusively a product of our culture is wrong. It doesn't have to be all social, or all biological. It most probably is a mixture of both (just like in every other nature vs nurture debate). If we understand the biological root well, then we have a more solid basis for understanding the social forces that operate on top of that.
> The UI is the interface between your app and the customer. It's like an API. No one arbitrarily changes APIs

Love it . I’m stealing this one.
Interesting you bring this up because even as someone who has passively read bell hooks and been familiar with her work (not intimately so but by proxy), it was through this article that I learned for the first time that the de-capitalization of her pen-name was a conscious choice, and even thought both titles-here and on the NYT-site had committed an interesting grammatical mistake not capitalizing a proper noun.
It's pretty obvious that Oracle has been milking the government for decades. Those contracts and license fees that they get are just free money from taxpayers. Does any software developer still use Oracle products? Even Java started to lose momentum after Oracle acquired it.
Another reason is it’s possible the same author writes an update or new article on the same topic. The year helps disambiguate that.
It's great that this is reported by The Guardian. What's missing is a way to communicate the same thing to the masses in a way that motivates them, i.e. somehow linking knowledge of this to a sense of loss or impending loss to the reader or user of Facebook.

Just on Sunday I had lunch with a friend who works at the ECB and asked disbelievingly whether phones and apps can really track your location or not.

So if Guardian readers with TWO PhDs don't get it, how are we going to educate a woman on the breadline with two kids, living in a shanty on the outskirts of Lilongwe?
Mine will be called lang. The docs can be found on the official website, langlang.org of course.
By 2025 most IC/semiconductors will be aimed at AI market
Counter product:

Internet Archive for tweets.
He says he sold off Oracle for precisely that reason.
I don't actually use cron directly. What I do use is capable of scheduling and error detection (nonzero errors). Even if it weren't, the script it invoked could do both with < 4 lines of code.

I think it would actually be professionally negligent to introduce coupling at this point.
I used to work at RS. If the person had been there for years, they probably do still care. Having also worked at Wells Fargo back in the day it almost sounds like they are tying sales referrals into support performance metrics. Sounds aweful.

The culture started going downhill around the time they were bringing Bank Of America employees into cloud operations upper management and they were chain hiring their pals.. Lot of talent was disease franchised and leaving.
> We have put a lot of effort into making this library useful to you. To help us make this library even better, it collects ANONYMOUS error messages and usage statistics. See d6tcollect for details including how to disable collection. Collection is asynchronous and doesn't impact your code in any way.

That seems really out of place. I'm somewhat used to automatic data collection from applications, but automatic data collection from programming libraries / frameworks? Really?
There are many ways to combine results during training in a federated workflow. One method I'm currently using on healthcare imaging data is to simply average each client's weights after every epoch. My server gathers all client updates, averages the weights, and sends the new calculated wights back to the clients to continue training.
The giant companies who whiteboard are full of people who have a strong incentive to reach the conclusion that the process that selected themselves over other people is a good one.

Unconcious bias is a tough thing to control for.
> The aim of this exercise is to prove that interpreted languages can be just as fast as C.

From my limited understanding of Python, it compiles into bytecode that is fed into a virtual machine doesn't it? How can that ever be just as fast as C? Unless the C programmer does a really bad job?
>Buying a $40,000 car, for instance, he says, would require a year’s worth of work if you’re earning $20 per hour after taxes and working 200 hours annually.

I understand how he can claim to make that much money,last time I worked 200 hours at $20 dollars net I was strangely paid US$ 4,000 (and NOT $40,000)
Would be interested to find out what other tools are available in this space and how this compares with them. What’s the difference beteeen d6tflow, Luigi, Airflow?
I am completely fine with the final result, except the three columns. That stuff is completely useless to me. What I want is this: README and Issues/PRs front and center. If I want to view the code I'll use OctoTree or the `T` shortcut. There's absolutely no reason to be viewing the code from the main page in my opinion. Most of the time I'm visiting GH is to view the readme, the issues, the prs, or the wiki. I never navigate through the code using their 'file browser', as it's terrible and you can't even see the hierarchy. OctoTree is much better for that.
You’re right about ML. But you’re mistaken about software engineering—though in good company with most software engineers.

Of denotation, cache access, confidentiality, authentication, integrity, non-repudiability, performance, thread safety, memory overhead—only denotation and some parts of memory overhead allow composition of abstractions.
> RTLinux is only barely soft realtime.

Linux used to run as a non-privileged process in hard real-time kernel, which taken together was guaranteeing responses in nanoseconds range.

If this is still the case (along with what Wikipedia states) , the whole system is by definition hard real-time.
Might be for use on web and email advertisements where inline styling is a necessity.
I'm currently working with on-site hospital data and have been designing our own in-house federated library on top of Keras to train in-aggregate while leaving all data on-site.
Part of the problem is the area is like the ultimate backwater.
>At work I have what are essentially cron jobs running scripts which invoke sklearn pipelines.

If cron works for you, that’s great, and you should continue to use it. However, I would be interested to know how many data sources you have, how you handle failures in pipe segments, and your general throughput.

In more complicated flows, ones that require different different data sets to to be combined, or lots of data flows that depend on each other, moving to a DAG with event triggering is a much better setup in my experience. Data is generated faster, and errors are handled more gracefully, and recovery much faster since data is only recalculated when needed.
I recently read that there are over 100 workflow engines [1], it is still often difficult to make the case that Make is insufficient

[1] https://vatlab.github.io/blog/post/sos-workflow-engine/
I'd say you can have substantial freedom (not luxury) with less.

But 7% yield is a bit optimistic, because you pay tax on nominal growth. Say, if you achieve 8% nominal growth, have a 37.5% tax rate, you're left with 5%. Now deduct 2% inflation, and you can only take out 3% per year.

Thus, your $10m only give you $300k a year (after tax, real), which is enough for 5, I'd say (maybe not in NY, but in many other nice places). However, that assumes 8% nominal return, which is not something I'd bet my retirement on, given current elevated asset values.
Ask HN: Kickstarter for Data Science?
With OpenAI's decision not to release the fully-trained model for their GPT-2 NLP model, some ambitious people have attempted to 1) build a similar dataset (40G of text), and 2) find funds to train the model (~$40,000).

1) https://github.com/jcpeterson/openwebtext

2) https://www.reddit.com/r/MachineLearning/comments/av7s38/d_i_won_20000_maybe_120000_of_google_cloud_credit/

Over the past year I've noticed that the best ML work requires scale that is impractical to individuals working without corporate funding. If this trend grows, it seems to me that innovation will be stifled. OpenAI wasn't wrong to be cautious about releasing an OP NLP model, because innovation is intrinsically amoral. A fine-tuned GPT-2 model for spear-phishing attacks could cause havoc. However, a GPT-2 model for identifying spear-phishing attacks could be inversely useful. But the question of distributing power verses centralizing power is settled among Hackers: distributing power (i.e., solutions) is always better, because "no problem should be solved twice" and "drudgery is evil".

Which is why I offer this question to Hacker News.

Would a "Kickstarter for Data Science" work? What would it have to look like to work? And by "work", I mean that models like the GPT-2 would be efficiently funded, trained, and released to the public.
Observation #2: the paradox is essentially describing statistical gerrymandering. :)
We already knew men are taller on average. If an explanation can replace “amygdala” with “height” and lose no plausibility, I prefer to set it aside.
Here’s the whole non existent category:

https://octave.org/doc/v4.2.0/Optimization.html
I think that correlation would be impossible to interpret due to the obvious confounding factors. A randomized controlled trial would be interesting though...
I was sort of kidding earlier. I tried registering with my current, academic, address and was rejected. I'm not surprised because it ends in "mx". The "joke" was that a PhD student at my university would have a similar looking email and I don't think they would have been accepted either. (I did have an "edu" when I was a student in the US and that address would probably have been accepted.)
Depending on the level and type of support you need, you can either go for AWS’s business support plan (10% of your bill with $100 minimum) but they won’t do anything involving anything hosted on your actual VM besides maybe help you with one of their software agents. On the other hand, you can always pay a manager service provider to pick up the slack if you don’t have the expertise in house.
Except the original statement didn't make any claim about "not slicing", so neither does mine.
Wanted the same way HIGH:MEDIUM:LOW was wanted, with a new recipe for late binding of noncomposable abstractions. Same outcome expected.
No offense, but you're exceptionally wrong.

Go through their financial statements. They were barely profitable on a GAAP basis for a long time. Barely profitable is entirely different from massively bleeding money. Barely profitable means they were making profits but reinvested them into their business.

The stories are entirely different.
I'm unclear: what was the incorrect claim you're saying the author made?
Yeah, that jumped out at me too. I don't know how you get that fact wrong.
Circular reasoning isn't data, though.

"The procedures at companies X, Y and Z are correct because their operations require their procedures to be correct" sounds entirely accurate, but doesn't prove that the procedures are correct (or optimal, which is another question not covered by your argument: the status quo can be sufficient but not optimal)
I hear you, but the implication that you make is that these writers would not merit making it to these lists were it not for a quota. My take is a lot of these female, non-white writers have not made it due to the echo-chamber that is the Western Canon. This list effectively rehashes this canon.

Here's an example from my life. Many years ago, I was a music student. Part of being a music student means taking theory and history. We learned about all of the Western Classical composers, and I remember learning, very briefly, about Clara Schumann. She was taught as a footnote, as the object of desire of Robert Schumann, Brahms, and others. I never knew, until recently, what a gifted and talented composer she was. Her compositions, imo, stand at the same level as her male contemporaries.
dot com boom was pretty big. A lot of people who could barely program were getting big money. The nasdaq was at 5000 when the cash flow couldn't justify it.

There was a bust and I am sure it will happen again. History is cyclical. Facebook, google, apple etc. will be fine because they generate so much cash. They can afford to pay people 300k. I don't see how companies like Uber and Lyft which generate no cash can afford to pay similar salaries.
Location: West Palm Beach, FL

Remote: Yes

Willing to relocate: No

Technologies: Android, Java, Kotlin, iOS, Swift

Résumé/CV: https://www.linkedin.com/in/nersa-acosta/

Interested mainly in mobile, but have also done web development. I like to ship software well organized and that can be maintained. I can also take existing code and maintain it. I have 8 years of software development experience and I have spent the last 4 years working with Android and iOS.

Email: nersa.acosta@gmail.com

I am open for hire, contract work or freelance projects right now.
SOSUS (according to the public maps) is very focused on the Pacific and Atlantic oceans and a lot around Russian territorial waters. The network is probably a bit denser than represented on the map but there's not much need for it in the Indian Ocean.

https://upload.wikimedia.org/wikipedia/commons/f/fc/Sosus_ma...
No disrespect taken. Thanks for raising this point — one that I’m often frustrated by. (Yes, I’m over 50.) I’ve resorted to using Safari’s Reader mode on the vast majority of the sites I visit because the respective designer’s idea of type face, size, and contrast is so unreadable to me. Obviously this doesn’t work for interactive sites like maps, though.
You should ask a PhD student at your university to try. It'll be interesting if they get accepted.
Look up dollar street by the same people who made gap minder. It’s a wonderfully simple look into the differences in wealth and living, which shows you things like “favorite thing” filtered along an axis of wealth.
De-Nerding Programming: Jonathan Edwards [audio]
I wonder if there’s a database of websites with known but unresolved security flaws?