skysight-inc/HackerNews huggingface dataset:
To me, this is very simple:

File-level encryption has total format flexibility. A filesystem can store arbitrary metadata. It turns out that strong encryption schemes want internal metadata. You can go through contortions --- contortions no block encryption scheme currently goes through --- to get some sort of space for metadata at the block layer, at significant expense. Or you can get it for free with files.

File-level encryption is also message-aware. Encryption cares about the boundaries between messages (just for instance: because it leaves no ambiguity about the set of disk sectors that makes up any particular file at any particular moment in time). Block-level crypto can't provide this; it's by definition oblivious to messages.

We have block-level crypto because it's a cheap easy win, for the minimal security it provides. If the FBI is interdicting you, block crypto isn't going to help you: they incurred weeks worth of planning before anyone with handcuffs approached you, and the extra planning it takes to make sure they grab you when your key is resident is a rounding error. But if you leave your laptop full of financial secrets in a taxi cab, block crypto will probably keep you from having to roll all your keys, change your account numbers, notifying your clients, etc.
If someone approaches me to talk somewhere just because I’m a woman, they haven’t done their job of finding what my expertise is. Therefore, I am going to insta-decline.

This is a really good policy. She's 100% right.
We have those in Europe? I had no idea, sign me up!
Yet the information density of the parent comment is so much higher. You have 3 statements that can be reduced to "Things that work in programming don't necessarily work in other fields." No kidding? Then what's your solution for operational efficiency in government?
Since those all have a lot more code than is needed to show the basic idea of an NFA-based matcher: https://github.com/darius/sketchbook/blob/master/regex/nfa_s...
Chrome devtools tip o' the day: “continue to here”
Yeah, Nick "deleted 20507 characters in body" five hours ago:
http://stackoverflow.com/posts/38484433/revisions
Thanks. I appreciate that. I think you actually have a keen eye for the issues.

Since my time in government, I've become more sympathetic to agency employees, contractors, management - pretty much everyone. The entire environment is really challenging to do the right thing.

As an executive, the typical tools you have at your disposal, budget and HR, are circumscribed by congressional appropriations and federal hiring guidelines. Your ability to promote and demote and reorganize are limited by laws and unions and you definitely can't grant stock options. The metrics that you typically use to measure success are much fuzzier because there isn't a PNL but it's really easy for the press to find a single case where your org does a poor job.

I still maintain hope that good things can be done - I have seen it - and good management makes a huge difference.

Disclosure: I'm an engineer at USDS and these are my own opinions.
> If the string to be matched against contains 20,000 space characters in a row, but not at the end, then the Regex engine will start at the first space, check that it belongs to the \s character class, move to the second space, make the same check, etc. After the 20,000th space, there is a different character, but the Regex engine expected a space or the end of the string. Realizing it cannot match like this it backtracks, and tries matching \s+$ starting from the second space, checking 19,999 characters. The match fails again, and it backtracks to start at the third space, etc.

That's not how backtracking works. A regex engine will only backtrack to try and make the rest of the regex match, i.e. it will take characters of the RHS of the string, not try and start "from the second character off the start of the string". I mean, if the engine tried matching from the second space, what would be matching the first space? Something has to.

Which meant, that even if the regex engine was incredibly stupid and could not figure out that a greedy block of \s was never going to contain a non-\s, it would only have to check 20,001 times, not 199000 (or whatever it was).

I can't reproduce this "bug" in either Perl or Python. The time taken to match a 30,000 block of space either followed by $ or XX$ was basically identical for \s+$.

There does appear to be normal backtracking going on, roughly doubling the search time for large strings terminating in non-\s. This is expected, as it has to check 20,000 during the first gobble, then 20,000 as it backtracks from the right 20,000 times.

$ time perl -e '(" " x 100000000 . "X") =~ /\s+$/ && print "MATCH"'

real	0m0.604s
user	0m0.509s
sys	0m0.094s

$ time perl -e '(" " x 100000000) =~ /\s+$/ && print "MATCH"'
MATCH
real	0m0.286s
user	0m0.197s
sys	0m0.089s
being able to see pull requests deployed live without compromising production changed drastically the way we work as a team and we build our front-end applications. I can't wait to get other people's feedback.
Bunt is a purely in-memory database, but it also persists to disk so that the database can be reopened. It's a lot like Redis in this manner.

Basically, BuntDB requires that data be persisted prior to completing a transaction. There is no window of time where there is data in memory and not on disk. It's designed so that there is no way for data to exist in memory and not be on disk.

I decided that strings were a better way to go because 1) the string is the most common type in a key/value database, 2) strings take up less memory than a byte slice, and 3) strings are just bytes anyhow so they can always be converted using []byte(str).

Thanks for the kind words and I hope you give it a try.
This feels like a really weird situation for you guys. You want the selling point for your software to be that you're more open than proprietary platforms like GitHub but you're unwilling to actually commit to that and have to hide behind terms like 'open core' and 'source available'.

> When we mention that GitLab is available in an open source edition people frequently ask "Isn't GitHub open source?""'?

* I think even more people are surprised to find out that everything except Gitlab CE is closed source. If someone were choosing between hosting with GitHub or Gitlab.com then you're only competing only on features and community since they're both equivalent in terms of openness.

* And I hate nitpick on word choice but you don't get to so shamelessly hide behind your brand. There is no such thing as Gitlab; Gitlab Inc. releases a product called Gitlab CE with a similar but more limited feature set to Gitlab EE under an open source license. If Mercedes released a $10,000 car by stripping out features and using a cheap Volvo engine I know their marketing campaign, "Mercedes is now available for an affordable price" would leave a bad taste in your mouth.

I know you've been getting a lot of negative feedback recently on HN, at it seems, at least for me, that there is a huge demand for a truly Free-with-a-capital-F alternative to GitHub and you're falling short. A thousand Gitlab CE instances hosting a few OSS projects is nothing compared to your potential if you really embrace openness.
You may already be aware of this, but, from the experience of watching old home movies while my mom digitized them: if you're recording video or taking pictures at least in part for your kids, be aware that they may care more about seeing old you and other relatives than they will old them, so try to turn the camera on yourselves once in a while.
Amazon wants to use lampposts, churches as drone ‘perches'
For less than 20$ i get 40GB here in Denmark

EDIT: this one https://www.oister.dk/mobilabonnement/taletidsabonnementer/o... and then I took advantage of a "double your data" offering they had. It's 120DKK for 20GB, usually.
That's just special casing the end of line. If it was "\s+x" you'd still have the same problem.
> 1. smartphones will finally have decent batteries (e.g. lasting at least 1 day for the avg PokemonGo player, aka days/weeks for the rest of the world)

This is blocked upon a breakthrough in battery technology, not any (in)action on the part of handset manufacturers as you are lightly implying. Drones are more my wheelhouse but phones and automobiles are similar; we've reached roughly the top end of energy density with current known technology and chemistry. The fact that your phone can talk for several hours while operating multiple radios is already quite impressive given battery size, so I'm not sure why you classify phone batteries as anything but "decent."

Pokémon Go isn't going to revolutionize our understanding of chemistry and physics, unfortunately. If you want a phone the size of an old brick you can have longer battery life, with the risk that a battery of that size will react very violently to being punctured or abused and swells frequently. Even current ones do.

(You should see my drone's LiPos after I pull them out of a particularly demanding run [i.e., sport mode]; they're often hot to the touch and swelling, and this is somewhat expected depending on circumstances. Battery health is something you have to take seriously with drone batteries because they're surprisingly sensitive and don't have a long useful life. "Invent a better battery" is a meme among drone folks.)
It's used in lots of places. Internet Explorer uses it to save whether a file was downloaded via IE. They are just only useful on NTFS, and often not even then because file hating utilities like Dropbox don't store them. So if you upload a file with an ADS to Dropbox, then copy it back again, you'll have lost that data.

One thing I'm not sure about is whether it appears in the file size when using dir. And if you apply a file hashing algorithm to generate a hash and you only use the file attributes, base file name and $DEFAULT data stream then you can append to the file via another data stream. So hash utilities need to be AFS aware to be truly useful in Windows.

Unless you are calling data an "attribute" though, it's really a bit of a silly comparison. Literally it's a seperate namespace in which you store data. The standard tools and utilities provided by Windows generally only look at $DEFAULT. The article is correct, git is pretty much doing something very similar, only the data is stored in .git (or specified somewhere else) and you use a tool like git to get access to that data, but you can also dive into the directory directly with any other tool. In Windows you use streams.exe, and it's a. generalised, b. non-portable as it's an intrinsic part of NTFS, and c. denoted as part of the NTFS filename by the delimiter ":", which is a reserved character and documented as such.

https://blogs.technet.microsoft.com/askcore/2013/03/24/alter...
Why would you need the Speech API if you already have a known-good transcript?
Alexa puts it at rank 50 globally. Honestly this seems suspiciously high and I wonder if their sampling method is biased. Regardless, it's a lot of traffic.

http://www.alexa.com/siteinfo/stackoverflow.com
TIL what language Stack Overflow is written in.
either overreaching or possibly doing some work for lobbyists for the telecoms?
And 10-20MB/hr is a massive overestimation. I've been playing virtually nonstop for a week and Android calculates its usage at 193MB...
I think this[0] summarises the main objective of Dart. Having said that I fail to see how it succeeded. Maybe it helped JavaScript evolve a bit but IOS development experience and performance is still vastly superior to Dart/JavaScript/Web.

[0] `The emergence of compelling alternative platforms like iOS has meant that the web platform must compete on its merits, not just its reach. Javascript as it exists today will likely not be a viable solution long-term. `
Sadly not much Thompson's libraries are implemented. I have tried to find one for F# but are just toy projects.
The Guatemala-Mexico security cordon has no structures or deployed patrols?
Yeah, we were told not to worry too much, just to be wary since someone had died 30 years prior. What I've read is that if you're in an enclosed space with a lot of rodent droppings(say an abandoned cottage or something) then there is some risk, but open air is safe.
The onus is on the uninformed customers to inform themselves.
Quoting is a much bigger problem than differentiating flags from paths, and on Unix that's a solved problem: the shell always handles quoting, and unix programs only expect a list of words which can contain arbitrary characters (except NUL, of course). If you invoke a program directly using the exec-family of syscalls, you don't need to quote anything.

Whereas AFAIU Windows programs expect quoted words to be passed via main(), and must parse them. The only benefit is that you can disambiguate a filename with a dash (or slash) based on whether it was escaped, but that's a quite rare necessity, and of course still relies on the caller quoting them. (Does cmd.exe quote pathname expansions?)

The dash problem is also solved as long as programs use getopt() or getopt_long(). First, getopt() knows which flags take arguments and which don't. Knowing this, if a flag takes an argument it doesn't matter whether the argument begins with a dash or not. One consequence is that there's no such thing as an "optional" argument to a flag when using getopt and friends, as that ambiguity cannot be handled cleanly. People who roll their own argument processing code just so they can get "optional" arguments to flags invariably don't appreciate the security problem.

Second, a double-dash (--) terminates the argument list. getopt stops consuming command-line arguments at that point, and optind will index the first non-flag argument. So if passing a list of filenames to a command, the correct idiom in Unix is something like, `foo -- /path/to/*`. Of course, that presumes that the foo is using getopt or getopt_long, or a compatible argument processing implementation. Fortunately the vast majority do.

Smart programmers should rarely if ever roll their own argument processing code. Any headaches (real or imagined) related to a mismatch between the semantics offered by getopt and what the application might want is usually dwarfed by the usability and security benefits of adhering to the system facilities.

On a related note, I've always disliked the way GNU's getopt and getopt_long permuted (reordered) argument lists. I have an inkling it could introduce needless security issues, though I haven't thought it through carefully.
> Interesting that you had no issues without ECC in the first place. Some ZFS boards are pretty against trying ZFS without.

ZFS without ECC is no better or worse than any other filesystem without ECC, in that silent data corruption is possible. It is for this reason just a lot worse than ZFS with ECC, introducing a fault condition normally not seen on ZFS.

But that fault condition does not go away because you chose to skip ZFS due to no ECC. And you will get a lot of other problems on top that ZFS would have helped against, even without ECC.

Lastly, you'll end up back in the horrible land of static partitioning schemes with questionable tooling. Why suffer through that if you do not have to?
Yes that's the other option I considered, after I had already written my comment. Imagine an omnipresent voice assistant/AI that you can summon from anywhere (home, office, business) and which projects things on whatever screen you happen to be looking at at the moment (as judged by eye tracking or gestures). No personal device needed!
ADPlus + WinDBG + SOS possibly.
I realize that it seems different to you but that's (understandably) because you're so close to the story. To the typical HN reader this is the same topic as https://news.ycombinator.com/item?id=12128546. It isn't in HN's interest to have two articles about that on the front page; nor is it in your interest, because readers will get ornery about it.

The thing to do instead is post a link in the original thread to let interested readers know that you put up a follow-up post. You can use the HN link if you want; the discussion will remain open even though we've marked the item as a dupe.
Your dog's poop in someone else's hands
Most simple regular expression evaluators are basically state machines. They hold a few variables:

a) what part of the regex am I currently trying to match

b) what point in the string am I currently starting at

c) how much of the string has this piece of the regex consumed so far

Then the state machine basically has three transitions:

* if (b+c) terminally matches (a), increment both (a) and (b) and reset (c)

* if (b+c) matches (a), but more could be consumed, increment (c) and check again

* if (b+c) doesn't match (a), increment (b) and reset (c)

So yeah, you could put in short cuts for things like "well this didn't match because the next piece of the regex is matching on a line ending", but keeping it simple is straightforward and generally smiled upon.
Disclosure: I'm an engineer at USDS and these are my own opinions.

I'm personally appreciative of the work that you've done and your continued work, in spite of the many challenges and frustrations. There are many ways to serve. It's always incredibly rewarding to be working along side dedicated and talented public servants and contractors. Thank you.
Even with Dartium it didn't reach a speed close to native. I don't doubt that if more effort would be put in the VM the perf could get better but as it is now Dartium is no longer an option.
I remember the day I learned that Python's "re" module uses backtracking for non-extended regexes. My tests covered lots of corner cases in the regex logic, but were too short for me to notice the performance penalty. Luckily I only caused a partial outage in production.

I actually got to talk to Raymond Hettinger (Python core team) about why re uses a potentially exponential-time algorithm for regexes when there is a famous linear-time algorithm, and (I suspect) most programmers would assume linear complexity. As it turns out, there was an attempt to re-write re to fix this, but the re-write never managed to present exactly the same (extremely large) API as the existing module. He advised me that "the standard library is where code goes to die."
The key quote here is:

"Regular expressions are one of computer science's shining examples of how using good theory leads to good programs ..."

"Today, regular expressions have also become a shining example of how ignoring good theory leads to bad programs. The regular expression implementations used by today's popular tools are significantly slower than the ones used in many of those thirty-year-old Unix tools."

The alarming thing is that regex are supposed to be compiled before use, and the class of expressions that need quadratic or exponential time is distinct from the class that needs linear time. Why don't these popular, widely-used regex implementations perform any optimizations?
Emin supports some variant of a hard fork regardless.
A few months ago, a Stack Overflow representative asked me if their presence at a dev conference was justified. My positive answer more or less revolved around the importance SO took in the daily life of programmers everywhere.

If only she was there to witness the effect of a 34 minute downtime on an open space full of mobile/back/front developers.
> deleted 20507 characters in body - Nick Craver

You are very likely right. Though it looks like they permanently deleted those characters because they don't show up on the edit history.
The only thing more infuriating than the fact that this letter was sent out...is that taxpayer money went to some incompetent person who couldn't be bothered to use a proper small-caps font.
Yes, by default the module requires that you explicitly whitelist the domains you want to allow certificate registration for. So while you could allow any domain to be registered, that's not recommended for this precise reason. But the whitelist is defined as a Lua function, so you have quite a bit of flexibility in integrating the whitelist logic with other sources of information.

See https://github.com/GUI/lua-resty-auto-ssl#precautions and the "allowed_domain" configuration for a bit more detail.
The answer is literally bigger batteries. I'd rather have a phone that lasts all day than one that's a millimeter thinner than it was last year. I carried around a 14oz 1" deep Cassiopeia Pocket PC in my pocket for years, a smartphone half that size would last days... so where is it?
It's hard not to read gSSk there. I scratch my head.
I tried both TS and Dart and Dart has just way better developer exp. especially when using IntelliJ and doing lot of small apps which have to be mantained.
In classical logic your statement entails that there is a proof that cannot be checked (marked "true") by an algorithm (note, that the proof itself cannot be checked, not a fact, for which there is no proof) or that there is an algorithm marking something "true" that doesn't constitute a proof (trivially the thing that is marked "true" is proven). I doubt you'll find counterexamples to either of them, as an infinite, yet valid, proof is hardly believable (nobody of us could ever finish reading it) and a terminating ("marking true") algorithm always computes something.
The Million Dollar Homes of SF Tech Execs Have Slower Internet Than You
This is far more likely a move precipitated by wireless carriers attempting to pull some sort of ripcord rather than a House committee noting the existence of a fad game.
The house should be asking questions. But it shouldn't be of Niantic; it should be of the carriers, who are treating a not-very-scare resource as if it were far more valuable than it is, thanks to their poor management of a government-granted monopoly.
was in code tags, so had a very long horizontal scroll bar :P
Perhaps it is perfectly fine for people to comment on the fact that women whining about this stuff is the worst possible approach to solving it. That the author's desire to be brought in to actually talk about tech is the correct position if you want to make real headway and that her insta-delete policy is brilliant.

One of the problems with being a woman in a tech environment is that there is already too much focus on the fact that you are a woman. Like that is something bizarre. Putting additional focus on it and allowing you to only talk about that and not about your actual professional expertise is all kinds of problematic.
Was a similar inquiry conducted with Snapchat? I've read several stories of 4-figure data bills having been run up by teenagers addicted to the service, but have not read about Congressional investigations into that issue. This letter seems like a desperate attempt by its authors to ride the wave of publicity currently surrounding Pokemon Go - it's clickbait for Congresspeople.
How would you suggest running a user-created-content site without ever fudging/sanitizing the data being posted to the server?
It definitely is in some places. I was unable to log in at the Santa Monica pier on Saturday night with T-Mobile, but those in my party with Verizon were unaffected.
I didn't say anything contrary to that.
I'm sorry if I was unclear. If I used the Cloud Speech API to produce the transcript, is there another library/service/etc that would align it to the original audio again.
Great! I've deleted my parent comment and marked the subthread off-topic, so it won't distract from what you posted.

Btw we're probably going to build moderation tools so users can give each other feedback about the guidelines like this without it having to take the main discussion off topic.
Don't you get a breakdown of what every app has used on Android? Why does the app need to repeat that?
I've been playing this game quite a bit (level 20) and my data usage is a mere 122MB since July 1. It ranks 4th among apps in my overall data usage.
> But he lies so much and makes so many factual errors how can you trust him on anything?

This applies to Hillary too, you know? The most awful thing about this election is that neither candidate is trustworthy. They are both blatant liars.