skysight-inc/HackerNews huggingface dataset:
Question: How did they obtain the Colossal Clean Crawled Corpus (C4) they mention in the article?

Options:

1. "Mechanical Turk" style, a massive undertaking to manually clean up Common Crawl, perhaps using underpaid labor in third world countries (such as samasource.com does)

2. By means of somehow getting the internet to do it for them with something like reCAPTCHA

3. With the help of machine learning / traditional text processing

4. Some other way

Anyone has any ideas? I'm intrigued. The paper [https://arxiv.org/pdf/1910.10683.pdf] and the website [https://www.tensorflow.org/datasets/catalog/c4] mention almost nothing, except for an option to switch off the cleaning & deduplication, which hints at option number 3.
The Spectrum Next is finally here – the evolution of the original 80s computer
You're definitely right that there are a number of simple-ish ways to represent the data, but in a tradition regression model the 102nd attribute being the letter A could mean something totally different than the 103rd attribute equaling A.

So therefore the authors had a challenge having sequences that are different lengths. In the past, one possible workaround that people have done with some success is taking some fixed length summary (say, the proportion of As, Cs, Gs, and Ts). Then you can control the order and avoid having to consider a possibly intractable number of sub-sequence comparisons, but you also lose information that might be important.
"On the internet, nobody knows you're a bison."
Change the title to "Ask HN: Why do I feel like a script kiddie?" to make it more clear that you are not submitting an article for discussion but asking the audience a question.
Would be interesting to see the same benchmarks for an AMD Threadripper or Epyc with 64 cores.
Having papers published in journals (especially prestigious ones like Nature) is how you attain status in the research community. Also most research institutions have some expectation that you will publish N papers a year on average (because it also enhances their status).
My interpretation of the comment is that it is criticizing the somewhat baseless claim. I find it hard to believe that even internally at Google the statement is considered an indisputable truth. I agree programmer productivity is a complex management challenge that everyone would be happy to have a silver bullet for.
As many of us doing research, writing, reviewing, typesetting, checking proofs, organizing their dissemination conferences, administrative tasks, filling up IP forms, returning them, getting them checked by attorneys paid on our side, etc., etc., etc., do not get absolutely anything remotely in value to the price it costs to our institutions or peers to access our work.

We force ourselves though to do something so illogical and nonsensical because we are evaluated according to inefficient and silly performance bars that emerge from a herd of seemingly intelligent people each acting in their best interest (both those that seek showing their merit, and those that seek evaluating the merit of others), most of which do not have enough power and/or leadership to be the first ones to disrupt the current status quo. ['Tradegy of the Commons' theme]

Of course, I would not suggest any student to try to subvert it, as doing it "heroically" will only constitute an obstacle for themselves without doing anything to convince your rightly-so competitive peers to join your efforts rather than to overtake you. [edited: sentence corrected]

However, things are slowly changing, as people & institutions with the right amount of power and leadership are starting to take action.
Author here: you got here before me but this is precisely the right answer, although it’s more about not needing the full contents to start _compiling_, rather than executing necessarily. Supporting streaming execution parallel to compilation is absolutely possible, but a tougher challenge that isn’t necessarily for our purposes.
Can anyone explain how the US got to such ridiculously high salaries for Tech compared to most other countries? As someone working in the UK I'm more than a little jealous and tempted to try the visa gauntlet though it seems like a long shot.
> The research is already funded differently

Yes, I understand the hatred for academic publishers b/c they don't fund the research.

The comment above said 'knowledge should be free' and that is what I was responding to.
> Internet became a garbage dump of humanity

I really like it, I am one of those filthy ones.

> anonymity, poor trace-ability (of source), inability to confirm attribution, etc.

I don't think this is a problem and it cheapens the arguments in #2 and #3. Still an important fact to be able to weight information. But even so I don't think the quality of information is that much worse than comparable "official" sources and that more people are informed to a degree never seen before.

> What will likely happen [...] is [...] "sub-Internets".

On this one I agree and think it is a problem. This will mainly happen due to the subjective content classification of several platforms that try to contain certain content and in doing so will create the basis for further content control by third parties such as companies and countries. A grave mistake.

> re-invented concept of on-line identity

I disagree profoundly, since this would increase ambitions identified as problems in #2 and #3. This will most likely supress voices that aren't a problem. If you think there are too many problematic voices, you should work on yourself, since that is entirely your problem. Real criminal activity on the net are foremost identity theft and phishing. Both issues are reinforced by the ideas around better identification of users. And let us be honest, that is the ambition proposed.

Compared to "conventional" crime, internet crime is actually pretty low since there is a high barrier for entry.
But we do know who moved to and did reduce the funding on different levels in recent times.
I love how an article about knitting gets related so quickly to other domains (by reading the comments). I had to search on that blog to understand that it wasn't related to a technical domain.

However, from the title I thought it was about the law (if it's not a law, it should be) that says that if a pair of sockets in a drawer are put together you can easily find the pair. Instead if you find one of the socks of the pair, the other sock of the pair, while still in the drawer, is mysteriously teleported into another dimension, never to be found again.
What's a morality rate? Is it a typo or an actual indicator? I found this: https://www.hsph.harvard.edu/population-development/tag/mora... but it seems to be a typo.
An average takes a whole lot of data and compresses it in to one number.

Why not present more of the data so we can understand the specifics?
> things are slowly changing, as people & institutions with the right amount of power and leadership are starting to take action.

Where exactly, what people and institutions?
Ok, is now a good time to start taking this seriously?

Spanish Flu had an R0 of about 2[1]. Current published estimates for 2019-ncov are 2.3-3.8 for this paper [1] and there's another floating around which gets as high as 6-8.

1. https://www.ncbi.nlm.nih.gov/pubmed/19545404/

2. https://www.sciencedirect.com/science/article/pii/S120197122
Let me know if you could use a partner.
Web Browser Privacy: What Do Browsers Say When They Phone Home? [pdf]
“The mortality rate of patients with 2019-nCoV infection was lower than that of Severe Acute Respiratory Syndrome (SRAS) and Middle East Respiratory Syndrome (MERS).”

Is this a typo or are they politicizing SARS here so people think it’s something else?
Well, it might have been obvious but it was certainly not a useless exercise. Some users were really interested and wanted to experiment with it.

Transaction data is read and never stored, just used to processing matches.

The flow was relatively simple, link an account during signup.

The regulation was hard and expensive.
Be transparent and open right from the start,Interview with 16 YO Ethan,
I agree that my earlier argument is simplistic in its presentation, (here I would treat 'construction costs' as to build and operate an apartment, amortized over lifetime and apartments in a building -- I understand I could have phrased that better but terseness is necessary lest every comment devolve into an essay) but is still correct in its assessment: CA rents are far in excess of any reasonable economic equilibrium, per 'amortized cost of providing the rental property, including land acquisition, construction, maintenance, etc.' Given in particular that in CA (the bay area and SF too) land is in surplus[0].

[0]Regulatory-allowed access to said land, on the other hand, is less permissive.
Reactance
What’s so special about women here? I’ve spent a decent amount of time in prison where men raping men is very much a thing, rape jokes were common even among those who found it absolutely awful. (Actual rape jokes, not expressions like “ear rape”)

Is there a reason why you think that women would be especially susceptible to this besides some bizarre sexist fantasies of yours?

What you’re suggesting here is far more offensive than any rape jokes could ever be.
My last project QuoraScanner was a browser addon that helped marketers decide what questions to answer next on Quora based on views and tracking users.

https://24hrstartup.com/products/quora-scanner

It failed as Quora changed how much info could be seen/scraped on the frontend without a login. Even if they hadn't I likely wouldn't have been able to get enough LTV from each user to spend enough acquire them at a decent pace.

No regrets though, it was my first experience doing user feedback sessions over Skype, which I can't recommend highly enough vs. asking them to fill a form.
Author compensation depends on the author and their ability to strike a good deal for themselves, just like the rest of us when we apply for jobs, they do it per piece of work.

As for journals, I understand the distaste academics have for these. This thread was about whether all published information should be free, not just journals.
I still don't understand how Apple does noet update their guidelines. Navigation and the top bar are sch a pain to use.
Reachability helps, but it's just bandaid around bad decisions
Yeah but Kubernetes "self-healing" a lot of times isn't. Or it just trips over itself (stuck pods, stuck cronjobs, etc).
A relevant factor could be in the other con

Con: None of our current end-developers use Rust.
Cognito as the Worse AWS Product?
You didn't even need first class functions for setInterval. setInterval also accepted a string!
Please update the title to remove the typo (s/morality/mortality), from the abstract: "the mortality rate of patients with 2019-nCoV infection was 4.8%".
Sorry, I thought I read somewhere that a foundation might be started eventually.

Edit: here: https://news.ycombinator.com/item?id=22008887 but I think it's not official.
Sure it does. Retention is discounted if people are desperate to work specifically at your company.

(Not commenting on the underlying discussion, just the particular logical point.)
I still would like to see the age repartition and most importantly a study on unreported cases.

From what I’ve heard, CIVID-19 has symptoms very similar to the flu.
Building a Roguelike from Scratch – Unity Tutorial Part 1 – Setup
Apologies, try here - https://community.couchdrop.io/hc/en-us/articles/36000135857...

I'll submit again using this link
> The answer is never bet against the Internet.

I normally tend to agree with this sentiment, but if that were always the case, please explain the prevalence--and quite honestly the very existence--of services like Redbox (2002-Present).

With the advent of torrents/newsbin/High speed broad band etc... any and all media is available in the comfort of your home, with no worries of region blocks or another BS and yet there they are.

I grew up on the Internet and quite honestly half the fun of DLing and such was being able to say I had already watched something to someone who had to rely on traditional outlets: the last season of Sopranos comes to mind.

Some try to rationalize the non-tech savvyness of its target demographics (Boomers) and that could be a part of it, but I have seen young people(looked like young millennials or gen Z) waiting in lines at these things.

For a while I rationalized it must have been an elaborate DNM drop-scheme, but then I met someone who said they use it regularly and was akin to a 24 hour Blockbuster. Which seems to have left a hole in some people who never even experienced it when it was alive and well (80-90s).

If I'm honest, a part of me still has some longing nostalgia for running into those things wanting/waiting for a new SNES/PS game all day at school only to be elated that I got the last one and it felt like I won the lottery.

The mom and pop ones were even funnier evoking the sounds of the chiming doors and the rush of adrenaline you got trying to peek into the beaded curtain area for the porn in the back as a kid. Also fumbling and wondering where you left with the plastic cases with a 'be kind and rewind' smiley on them.

But we live in an era of convenience, and comfort creatures where none of that is a part of the equation anymore.
I'm on their free plan and have never had an issue. They also send me warnings letting me know I'm at 50% of my free usage, which is a nice touch.
Yeah, that's a typo, it should be mortality rate.
> What is ELO?

https://en.wikipedia.org/wiki/Elo_rating_system

In this instance, it's presumably a measure of a person's holistic attractiveness - say the average willingness of prospective partners to accept another date.
Show HN: Mount S3 cloud storage directly to Windows/Linux using native SMB/NFS
Herga adds variable output option to standard range 6227 and 6228 footswitches
Extending Vue Component Templates
Can't edit the title anymore. Could an admin eg: @dang edit it, thank you.
If they did, they did it very quickly as https://hackernewstitles.netlify.com/ didn't track a change on it.
How to Duplicate a WordPress Page or Post – Simple Steps
JS is used on the server because there are tons of JS developers thanks to it being basically the only language used on the frontend.
* Ocaml's more straightforward and conventional - it's impure, you can mutate with abandon, it's eagerly evaluated for more predictable performance, no one's doing wild stuff with arrows or profunctor lenses or f-algebras or whatever. All this could be either a pro or a con, depending on your perspective.

* Ocaml has an extremely powerful module system that puts that of almost every other language, including Haskell's, to shame.

* Ocaml also has a very powerful object system that goes unused 95% of the time, but could potentially be useful for that other 5%; there's nothing comparable in Haskell.

* Ocaml has no ad-hoc polymorphism, whereas Haskell has typeclasses. The module system substitutes for this in many cases, but it's arguably less ergonomic. In other cases you just need to, say, use different operators to add floats than integers, which is definitely less ergonomic.

* Ocaml has comparable - probably superior - single-threaded performance (very good), but unlike Haskell there's no real parallelism story to speak of. Multicore support has been in the works for over five years and still hasn't arrived.

* Ocaml has a smaller community and ecosystem. Yes, smaller than Haskell.

* As a matter of personal opinion, I prefer Ocaml's straightforward and widely-loathed syntax to Haskell's weird, whitespace-sensitive one.
Hey there varlock! It's a good thing you're asking this question here.

Now the answer to your question[s] will really depend on what it is you're trying to build (and more importantly, sell).

Take for instance a website like Stackoverflow, it will be quite hard for anyone to develop and get people to use an alternative for the same because the entire value the website delivers is derived from its user base.

However, if your idea is something like Mailchimp, it will be a lot easier for you to build and sell the same because the value delivered by the product depends mostly on whether it works for prospective users and your ability to convince them about the same.

> What would you do to differentiate the product? What steps would you follow?

I would pick a niche (subset) from the market the incumbent player is catering to and build something superior catered only towards them and then grow horizontal from there.

> Or would you drop the ball altogether (hopefully not!)?

Hello no!

Depends of whether networks effects are an important aspect of the incumbent's success and the resources that are at my disposal. So if I don't already have a bunch of money or risk appetite (even in terms of time), I wouldn't really invest myself in doing a marketplace sort of thing (where network effects matter i.e. product value is derived from user base).[1]

If network effects don't matter and I can build and sell this thing on my own or if I at least have access to the sort of people that can help me, I would go ahead. This might be a good read: https://invertedpassion.com/copying-ideas-is-highly-underrat...

> What dimensions would you (or did you) consider to differentiate your product?

Pick a niche from your competitor's market. Get into the skin of your user / customer from this niche. You'll start finding aspects which you can improve and build upon to make a prospective user possibly switch. Don't dismiss things like being a small bootstrapped company with an accessible team can be a differentiator too.

[1] - However, this will differ a lot from person to person because everyone's risk assessment in life is quite different based on their goals and current situation.
That's great. But do they have any wild animals left in China?
How to Increase Your Credit Limit on Your Credit Card (5 Step)
Great concept, although worried about uploading datasets with sensitive data to your cloud.

I usually use this for Pandas Profiling to accomplish EDA tasks https://github.com/pandas-profiling/pandas-profiling
You can check for yourself - on providing and dissemination, check Arxiv & Siblings. On institutions, check e.g., University of California, or search, e.g., "Elsevier's Paywall" using some internet search service to find names.
How to Rank My Link
Hi All. let me know how can I rank my link here and get more traffic?
darknet.py provides ease of use and security that in some cases is not easy to recreate by simply using torrc.

Features:

- Compatible with any SystemD based Linux

- Makes TOR work with SELinux on RHEL distro

- Secures Linux Kernel preventing IPv4 Forwarding, etc...

- Makes DNS leaks almost impossible

- Redirects all traffic thought TOR

- Works Out of the Box

- MAC Address Spoofing

This software was created to have a Transparent Proxy based on TOR as an alternative to VPN and easy to use in any Linux distribution mainly, especially those based on RHEL or with SELINUX.

Another reason for its creation is that it can be easily combined with i2p increasing the anonymity of the latter network.

As for the article, I know it, in fact, I relied on it for the creation of darknet.py, and the text clearly states that it is not a discouraged only method but rather a method to be reproduced only if you have knowledge and ability to limit the Packet Forwarding of the Linux Kernel, Dropping packages that could create a leaky situation and other practices that could compromise anonymity.

Currently, there are few cases where it is possible to have a DNSLeak with darknet.py and obliviously, it's up to the end user to take the right precautions for their need for anonymity.

darknet.py is an experimental yet stable software that will surely be subject to improvements over time, one on all the different rules of IPTables that are currently good but not excellent.

For any improvement I am at your disposal.
Well, sci-hub's one.
What FUD? And what aversion to "googling stuff"? The Start Menu Personalization is only one of the ad delivery channels, and only one about which I (literally) forgot.

Do you still mean to say that there are no ads on a legitimately-purchased Windows 10 copy, after literally writing that you, yourself, have seen at least one?
What are you referring to? I've only ever had it play in a YouTube tab in the background.
I can't see any typos in your quoted text. Am I missing something obvious?
I can't give specific advice without knowing more about the idea, so I'll keep it general.

I think pretty much whatever you do, if you do it long enough, at some point you are going to discover there is a competition. It is a validation that your idea makes sense.

In your case you discovered it early, even before you started. But does that really matter? What would your decision be if you have been working on the project for 1, 3 or 6 months and then discovered it? I would ask myself that question in your situation.

If somebody had the same product idea, it is not like they "own" it now and you are not an original anymore. If your intrinsic motivation for doing it is strong enough and you want to do it long-term, it shouldn't really matter. Of course, I am assuming here your competitor is also in the early stage and has not overtaken 90% of the market.

You don't even have to differentiate in the beginning - why would you? You can both offer the same service - there are Pepsi and Coke, Intercom and Drift and they basically offer the same service.

As the time progresses, it is likely both of you will differentiate more and more, and orient toward a specific niche of the customers.

You even have an advantage coming in the second - you can learn from the existing solution and make it better.

So alltogether, I would say that the main question is whether this is something you personally want to do long-term or not, whether you have the internal motivation for it.
If it was just a "quick" idea you had and the main motivation was to make a quick buck, then probably not. But if this is something you are interested in and want to keep doing it, then why not.
> Now, ordinarily these computations are run on a supercomputer and cost thousands of dollars per solution, or you’d need to build a cluster for $15-20K or more. But with 28 cores and the ability to handle up to 1.5TB of memory, the Mac Pro is a competitive alternative.

If a desktop is being considered, aren't desktops with similar performance from Dell, HP or whoever just as competitive?

(Also, that cluster is cheaper than this Mac.)
Well I think it is more about the human power sources.

Only the usage of more efficient crops like potatoes and the import of fertilizer like guano enabled the people of the industrial revolution to work in factories instead on the fields.

No pre industrial civilization was able to feed so many people not working in agriculture.
When I first brought my puppy home I set up a Raspberry Pi with a Camera module to spy on him during the day. I had it stream to a private YouTube channel, but there's no reason it couldn't stream to anywhere you like.

You could also use the IR camera and a good enough IR lamp to give you coverage at night as well. Use a PIR if you want motion sensing added on.

https://projects.raspberrypi.org/en/projects/infrared-bird-b...
Can You Build Muscle Without Weights?
Is this 4.8% mortality number from cured/dead patients or also counting ongoing cases?
I.e. If they could build them faster they would likely be the best selling.
I bought an Alexa.