skysight-inc/HackerNews huggingface dataset:
> For example, "All CSVs contains a single consistent encoding". If this is not the case I'd (rightfully) reject the file as being a proper CSV file.

"CSV" only specify the metacharacters (at best), unless it's explicitly stated that the CSV is a single consistent encoding it's no more improper than inconsistently encoded Linux paths. It's stupid and a pain in the ass to be sure, but it's not improper.

And when you don't control the entire pipeline, you will eventually hit this issue.

In fact the first falsehood programmers believe about CSVs is that it's a good serialisation, interchange (between separate systems) or human-edition format. While it's routinely used for all of those it's also absolute garbage at all of them.
I also don't fully understand the need for laws to fix the "planned obsolescence" issue - if a manufacturer's products consistently broke shortly after the warranty expired (well before expected end of lifetime), why would anyone continue to purchase from that crap manufacturer...

Not knowing upfront, for example. Assuming their one failure was simply unlucky. Pessimism (deciding all manufacturers design products this way). Lack of money. Seldom used product. Lack of selection. Assuming that it was that one product from manufacturer x instead of their entire line of 200+ products (and of course a few will miss). Researching and realizing product B was really manufactured by Y instead of X, unlike most of their other product lines. Realizing product B was made to be cheaper to sell at Walmart, so you weren't getting the "good" brand you thought you were. Negative review overload (all of them have faults, it seems).
For the record: HBO was apparently supposed to do this a couple years ago [1].

I wonder how well the Foundation books will translate to the screen. Given the segmented format of the first three Foundation novels, it almost seems like a mockumentary-style exposition would be necessary (which could preserve things like the cool Encyclopedia Galactica quote epigraphs).

[1] http://www.player.one/foundation-tv-show-whatever-happened-h...
>The deal with RHEL is that they will no longer include python2 packages at all.

Generally, if you are installing your own application you will use a virtualenv with all the packages closed off from the base operating system and dependencies installed with pip (we're all doing this, right folks!?).
If they're going to put it in Excel, just use their formats[1], [2].

[1]: https://news.ycombinator.com/item?id=15476951

[2]: https://www.google.com/search?q=simple+xlsx+writer

If it's something you're streaming, using a binary messaging format is always faster/better. I tend to use Kx's format since a lot of my messages go from to their systems, but Erlang's binary term format is good too -- even if you're not using Erlang.
I use git sub-trees. Actually I love the thing. They give you a 'linear' history, and allow you to merge/pull/push into their original tree, keeping the history (if you require it).
Where do you see google calling IBM a "Computer manufacturing company" if you search for IBM? I'm not saying they don't. I just want to see examples of what you are talking about.

I googled IBM, and did not see it classified as such.
Which reinforces the position that git should have stayed a Linux kernel specific DVCS, as the Bitkeeper replacement it is, instead of forcing its use cases on the rest of us.
Well, Hari Seldon - a recording of him at least - is a constant presence. There are lots of episodic tv series with varying cast. For example, Twilight Zone, Black Glass, etc. I'm sure all of the ages treated in the series can be fleshed out in sufficient detail to make for entertaining tv. As another example, Tolkiens original lore is not exactly ... visual ... but it gained lots of artist output in the decades between the publication of the series and Jackson's movies. Jackson's movies own cinematically almost as much to the artistic output by cadres of fantasy artists as much as to Tolkiens original work.
You can certainly reject the file, but that's not going to make your user happy. The (sad) reality is that you have often to deal with these edge cases in software.

The user does not care that the CSV is not proper. He just wants to open it, and if it doesn't work, they will blame your program, not the source of the CSV.

And you only control the former. I have had to deal with a CSV export from a web service that was "not proper". I notified their support, but that problem was never fixed.

Luckily, the user in that case was just me. But if I was making software for someone else, blaming the right source of the problem would have not taken me anywhere.
dude IQ is as legit as horoscope
JSON is pretty extensively specified.

Saying that CSV is not is generous.
It really depends on the purpose. For scientific computing, for instance, CSV is used as a "standard" for quick tables of numbers. Sufficiently "normal" files can be read by almost all scientific tools (speaking of libraries such as numpy, systems such as Mathematica, Maple, or ecosystems such as R, basically even C++ and Fortran allow this task to be implemented in a couple of lines).

However, also in this context CSV has major drawbacks, for instance not defining column headers, comment lines, number formatting. A proper drop-in replacement in my subject is HDF5 (https://www.hdfgroup.org/) which is mostly used for being a binary representation of n-dimensional tables with couplings to major programming systems.

However, I never heard of HDF5 outside of science. Therfore I make this example here.
People used to complain about the way Python changes were breaking their code. So it was decided to restrict all the non-reverse compatible changes to a separate version called 3 and to stop breaking things in 2. That probably allowed Python to become more popular than it would of otherwise, but at the cost of a significant fork.

So what are you going to do? Either way has downsides.

And it isn't like developers really have a lot of power to guide the course of a language. Lua is an excellent example of this. If you want to write a Lua program you probably want to target version 5.1 . The developers have released 5.2 and 5.3 but uptake has been fairly minimal and it's a problem because 5.3 is a Python3 like fork. You can lead but that doesn't mean that anyone will follow. The Python 3 thing could of gone a lot worse (or better) depending on what you think of Python 3.

Ultimately the issue is that different people have different ideas about where a language should go (if anywhere) and no faction of developers or users have ultimate control. It isn't anyone's fault, it is everyone's fault.
Many of the complaints stem from the fact that many CSV writers do not follow any spec and produce garbage data. Kind of like when the industry decided that reading badly formed HTML is beneficial.

For the sake of sanity I'd recommend splitting the logic of reading csv into two parts. One that takes garbage and produces a correctly formatted file (e.g. properly quoted values, equal number of columns, single encoding) and the second part that actually does CSV parsing adhering to strict rules.
You need to reach a certain karma threshold, I don't remember the exact value.
Exactly. Maybe we should not ask them to regulate what they do not understand?
52. All tools working with CSV follow RFC 4180[1]

[1] https://tools.ietf.org/html/rfc4180
I couldn't agree more, the world wide web is definitely just a piece of the internet. I remember the battles over chat protocols, the scourge of p2p protocols and even the move from email to webmail and back again.

The web has always been a bit contentious for instance the browser wars are still being fought in some way even today. The problem is that the web is where developers were able to build without concern over who the powers that be even were but now the web has been turned into a creepy surveillance machine.

Our current president even paid Cambridge Analytica to socially engineer all of us. Turns out the tricks that hackers used to employ have been appropriated for use by the web monopolies. I can't think of a way in which this turns into anything good for the average person.
They were breaking many things, maybe they should have rewritten the C API? From what I remember that's the blocker.

That would have broken all C extensions, but it's not like Python 3 at launch was a raving success regarding library porting.
My one trick there is to just not ask for CSV, and import Excel files directly. Most languages have libraries to read basic (non-formula) excel files, they work fine, and provide richer data models than CSV (though somewhat risky as you have to deal with formattings and finding e.g. numbers where you expected strings and the other way around).
HN. Not really a social network, and that's one of the main reasons why I like it.

Besides HN... I don't really curate my other social networks. My FB has been disabled for newarly two years now, i never really understood what is the interesting/useful part of twitter and snapchat/instagram... I really don't see the point.
JavaScript's Tilde: Its Real Use Is No Mystery
True, but is there a situation where upgrading will hurt instead of benefit?

Not every solution is just as good for everyone. Think of the delayed upgrade game that some people play. They wait for others to upgrade first to get rid of new bugs at their expense. Now take this game at a 10 year extreme.

For them, the new branch (e.g. new language etc.) is simply too risky. It's not that they are not smart to upgrade, they simply have different opinions on what is valuable than you do.

It
"In June 1970, E. F. Codd of IBM Research published a paper [1] defining the relational data model and
introducing the concept of data independence. Codd's thesis was that queries should be expressed in terms
of high-level, nonprocedural concepts that are independent of physical representation."

The key, the whole key, and nothing but the key so help me Codd.

Also said as... "In Codd we trust."

If none of these DB jokes mean anything to you, take a DB concepts class at a CS university. There's a lot of great research going back 50 years and you can learn a great deal about why things are the way they are (tuple algebra and calculus). And before changing anything for something you think may be better, you should fully understand what you are giving up.

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.86....
That's not fair. Forcing them to update me, and only me, via a second channel isn't "care".
I barely ever see an engineer knowing everything that happens from the top to the bottom on any platform. Very few companies are forced to get to know their stack in depth, usually they throw more money to the problem.
I use Gmail over IMAP and SMTP, only logging in through the web to change server-side mail filtering rules.

But yes, as much as I love the web, it's funny how people forget (or never realized) there are other protocols.
Japan team maps 'semi-infinite' trove of rare earth elements
But RDF was pretty good with that, because it didn't force records into hierarchies. You could just define relationships, and then create them freely between records. Some people created heavy ontologies, but it wasn't a result of the data model.
On Google Knowledge Graph https://youtu.be/mmQl6VGvX-c
Those are the reasons I think it could work... yes, the stories span centuries, but they are laid out in specific chapters, which not only are short enough to work in the scale of a TV show, but would allow for cast changes on a regular basis, and could be more of a Doctor Who-style series, with different eras expressed by different show-runners, directors and casts.

And it likewise works OK for the characters to not be fully fleshed out immediately - they aren't on TV either, as the actors and writers get used to the characters.
Recurring charges are more often than not automatically updated when the card expires.

https://mobile.datamation.com/columns/executive_tech/article...

Even when you report the card as lost or stolen, sometimes recurring charges can somehow still be charged. Hulu was able to change my Citi card that I reported lost for three months. They sent me an email saying it was a courtesy and please update my payment to ensure no loss of service. I let it go to see how long it would change it.
That's most "SRE" -- it's a title arms race in that field between the underqualified and those that wish to convey they know how to do more than write system scripts in DSLs
But it is the 'Go' in the headline that probably got this article promoted to the frontpage of HN ..
> Protobufs are usually almost a drop in replacement [for csv]

I'm sorry, what? In what way is it a drop in replacement? Csvs are editable in a text editor and natively imported and exported by a wide variety of tools.
> Any alternatives to CSVs?

Depends on the context. If it's data-exchange and the data is going to be edited by humans, just take excel files directly as input. That'll save you tons of trouble.
Sometimes you have to, or customers will hate you for it
United Kingdom Relative Deformation Map
Lately there have been attacks on BankID in Sweden.

They call you up and pretend to be from your bank, then ask you to open your BankID app and verify your identity so that they can share some very important information with you.

You open your app, and ID yourself, thereby logging them into your bank account on their end.
More power to you, but my initial argument was contingent on "for your definition of works".

Eventually, your client will see that money spent on maintaining old (more appropriately rusty) packages will better be spent on upgrading.
Based on his testimony today, Mark Zuckerberg puts users last during a crisis
It also thwarts phishing and MitM attacks (assuming the browser is not evil), which OTPs do not.
Lol "reject" the file. The author in this story could be me and the reason we both have so much experience working with that long list of fucked up CSV files is because we don't have the power to 'reject' anything. Our client gives us the data and tells us to fuck off until we solve their problem. Professional services != consumer services in servicer/servicee power disparity.
Ah, you can learn the beauty of merge AND rebase at the same time then...

Here to 'present' feature branches, we take a feature development branch will all the associated crud... Once it's ready to merge, the dev checkouts a new 'please merge me' branch, resets (or rebase -i --autosquash) to the original head, and re-lay all the changes as a set of 'public' commits to the subsystems, with proper headings, documentation etc.

At the end, he has the exact same code as the dirty branch, but clean...
So he merges --no-ff the dirty branch in (no conflicts, same code!) and then the maintainer can merge --no-ff that nice, clean branch in the trunk/master.

What it gives us is a real, true history of the development (the dirty branch is kept) -- and a nice clean set of commits that is easy to review/push (the clean branch).
Sure, of course. I was hoping to point out that there's an increasingly overlooked value in having someone question complexity. The "no, you don't need React" of the frontend dev or the "our data is actually relational" of the back end dev.
Atos Claims Breakthrough in Quantum Computing Simulation
The idea that a senate hearing could be anything but a sham is rather innovative. Are the authors naive, stupid, or evil?
Yes yes, it's a set-word! type in the "do" dialect. That's not how it's learned when first starting, though, and not usually the thing you're thinking about when using it moment-to-moment.
Interesting. Do you think they miss out on a fair amount of customers at that price point?
Quantum simulator offers faster route for prime factorization
How Can You Tell Someone Is Truly a Bad Boss?
Swift is Apple’s baby and has a captive audience. Since Apple cannot even update their example code, I don’t think it proves any points about keeping up with a moving target.
>So I have currently zero empathy for the "victims" of this leak. Quite the contrary, I feel a little angry. Because I have to leave in a world that could be much more beautiful if only people, sometimes, just sometimes, though about the consequences of their life instead of just going with the motion. And this affects much more than privacy. And it's unfair.

And the very fact that you insist on blaming the individuals for a collective problem is why problems like this are so hard to solve.

If so many people don't care about this issue, despite its importance, maybe you should stop blaming them and expecting them to magically start caring, and start thinking about how our society leads them to believe it's not an issue?
If you do use an ORM, you should still know SQL and know what raw queries it actually generates, and you should make sure it doesn't do anything cute like falling back to concatenating strings or some other unsafe practice for the sake of having a convenient API. Everyone assumes that using an ORM means better security than raw SQL, and maybe in the general case that's true, but the more abstract and complex a framework is, the more room there is for unknown and unwanted behavior.

The one thing you probably should never do is use an ORM because you don't want to learn SQL or you just can't be bothered to care.
>Facebook welcomes regulation in order to suffocate would-be competitors with it

Sounds like a libertarian conspiracy theory to me, especially when we still don't know what these hypothetical regulations would look like. For instance if they make it easier to access your data, download and delete it they might somewhat weaken the lock-in effect of Facebook and level the playing field somewhat.

Beyond that and even if they make it slightly harder to build a social network in the future it doesn't mean that a reasonable amount of regulation is not worthwhile. If you want to start a bridge-building company you'll have to abide by a massive amount of rules, yet I'm not really sure I want to let the "completely free market" decide what a safe bridge looks like.
For identical quantum channels, order matters
I do not think double standard means what you think it means.
That's true. Fortunately, if everything else fails, we should always be able to connect on 443 via stunnel for example, no matter what protocol we use. That's the price to pay for the net getting more complex and more abused.
I like to run no script or ghostery, I recommend uBlock for ads, a user agent switcher, cookie auto delete.

I'm going to have to get Decentraleyes and Don't touch my tabs, and try Disconnect. I don't bother with containers but I'll use private browsing if I need one.
This is unacceptable in large business settings today. A list of assets alone can be tens of millions of lines. A list of transactions? Oh boy time to break out Rust if you want your database to be finished this month.
New NIST chip hints at quantum sensors of the future
Curious:

How many levels of indirection from end users do you need to be, in order to consider installing an arcane linux tool and then launching `protoc --decode_raw < message.bin` from the command line an acceptable drop-in replacement for CSVs?

I'll take the generous interpretation and assume OP meant some very specific developer use-cases :)
> For example, "All CSVs contains a single consistent encoding". If this is not the case I'd (rightfully) reject the file as being a proper CSV file.

That's literally the only one on there I felt that way about. All the other ones can and should be accounted for in your application if it accepts arbitrary CSVs.

Encoding is a special case because it's impossible to do anything other than guess at a plaintext document's encoding. If you're getting byte code parsing errors, the code should try a few different encodings before rejecting the document.

This won't help if there's a mix of encodings in the document. If this is the case, nothing can help and you will have garbage going in.
Except for the sites which prohibit special characters, or limit to a max of 16 characters (I ran into this last week, was genuinely surprised it was still a thing).
I would need to dig it up from my old harddrive, if it still exists. To be honest not sure it still exists, as I said it was a long time ago.
The difference is that the video model abstracts over the encoding, the git model does not abstract over the storage model, it exposes it. git commands are operations on a versioned blob store.
Steam's new privacy update will apparently ruin Steam Spy – PC Gamer
It depends on the context. If you're pulling in data from a source whose format the user can't control, then you have to deal with the broken CSV emitter. But if you're pulling in data from Excel, you're better off telling the user to save as xlsx because Excel makes such a hash of CSVs. In Python, I've had way fewer problems using xlrd and openpyxl than I have with any of the ways you can read CSVs.
The thermodynamics of computing
About as well, but in a number of cases signicantly worse. I am not saying people shouldn't upgrade, but it's not without issues.
The system we're building (and selling to big US administration groups) uses certificates to authenticate. I've never found installing a certificate hard.
Try incognito mode
The reason CSV files are used everywhere is not because of lack of alternatives, but because business people understand what CSV files are and feel safe using them, plus you're not relying on any technology or more complex protocol, it's really just the "natural form" to handle table data.

If you can successfully convince the business to first always use some command line tool to convert data before they can look at it in Excel, then you're in a very lucky position.
It doesn't have to be a contract, it can just be trust, of the same type that people use to tell friends their personal feelings or whatever. If you know someone's who's a gossip, they might be fun to talk to but you wouldn't tell them you cheated on your wife because you don't trust them in that way.
Why Americans avoid the doctor-Despite having insurance, the costs are too high
Number of VPN users strongly increase
I would be shocked if the government was not a major customer of FB.
There are couple projects that try to tackle this problem by providing an alternative CLI (on top of git's own plumbing), like gitless and g2. Haven't used any of them myself, but would be interested in experience of others.
That's crazy protobuf are a massive pain. I was doing a ctf recently using protobuf and it was incredibly anonying decoding or reversing it. You lose the schema you're shit out of luck.
> Congress doesn't need to understand anything in order to pass a law concerning it.

They have made that abundantly clear.
Wow, that's an amazingly prescient post.
>> that filters out all the bullshit on the web?

The problem is that your definition of bullshit might be different than mine, so any filter you come up with will have issues.
Given this massive list of falsehoods I have to ask: what can programmers believe about CSVs?

How do you go about writing a good CSV parser given that you can't assume anything about your input data? Are there examples of safe, robust CSV parsers that deal with all these falsehoods?
Charlie Munger: Investing in AI [video]
If there's any Sci-Fi story that will easily adapt to the screen, its gonna be "Permutation City"
You shouldn't assume you know what I believe.

That being said, the vast majority of CSV files I've dealt with are very simple and are plain ASCII separated by a comma (or whatever). But that doesn't mean I don't know there could be more to a CSV file.
CakePHP 2.0 with Twitter Bootstrap?
Or PayPal. When NYT wouldn't let me cancel without a phone call, I simply canceled the sub via PayPal. They sent me nasty collection agency-style "your account is past due" emails before finally canceling, 2 months after I stopped paying them.

I now pay careful attention to cancellation procedures when signing up for stuff.
JavaScript performance tricks
Decoding is going to have be supported in hardware for any sort of mass adoption anyway.