skysight-inc/HackerNews huggingface dataset:
LMDB is a COW tree but it is emphatically not an append-only tree. Append-only sucks, for all of the reasons already spelled out in the blog post.

The write amplification in LMDB is bounded by the height of the tree, so it's O(logN) with N=number of records. The write amplification in all WAL-based designs is O(N) where N=size of the records. With a 4K page size, to get a write amp of 64K in LMDB would require a tree depth of 16. At 100 bytes per record, 40 records per page, that would require 40^16 records, or 42949672960000000000000000 records. IMO your figure of 64K is vastly overestimated.

I've done more detailed investigation of write amplification in various DB engines here http://symas.com/mdb/ondisk/ and you can see that the break-even point relative to WAL-based engines is around record size of 2KB. Above that size LMDB has lower write-amp than WAL-based approaches. Coincidentally (hah) LDAP entries tend to be >= 2KB in size.

But most importantly for real-time services - LMDB write latency is tightly bounded, and never suffers from stop-the-world compaction pauses. This was a design requirement from day 1. (See section 3.2 of the 2011 LMDB design paper http://symas.com/mdb/20111010LDAPCon%20MDB.pdf ) E.g., see the comparison of write latency in LMDB vs HyperLevelDB here http://symas.com/mdb/hyperdex/#100M - we can reliably compute/predict LMDB's I/O latency and our prediction perfectly matches the real world results - 33ms avg latency on a disk with 16ms avg seek time. You cannot do that with any of the other DB engines.
> There is a fantastic context menu on the Start button with quick access to various administrative functions.

Never heard about this one before, thanks for the tip!
I'd like to phrase dustingetz' question slightly differently: if you're building an app that does not have multiple relatively independent parts as dustin describes, does Relay still have a major advantage over what's currently common? (i.e. if one page is only ever one "thing", how is it better than just calling the right APIs from the router be done with it?)
A file system is not a data structure, it is an interface.
Show HN: HTML to Hiccup Converter
Any remote data science/ analytics bootcamps?
Is this a case of right tools for the right job? Are there special cases where immutability is superior?
From the author of `How I Penetration Tested Your Mother`?
7 Best Twitter Tools to Dominate Twitter Marketing in 2015 – Pro Blog Tricks
Webseclab – Web security test cases and a construction toolkit
High tech swingers
Let's say your kernel currently compiles N bits of entropy per second from various sources. If you add a new hardware RNG that produces 1024N bps, you won't be able to simply append the two streams together to get 1025N bps, because that would be extremely unsafe like you pointed out. The best you can do is periodically xor the output of the hardware RNG onto itself ten times so that you get N bps, and you xor that with the original trusted entropy stream. By the end of the day you still only have N bps, it's just that these new N bps have "better randomness".

This is why hardware RNG is such a hard sell. Constructs like clockspeed, FLOPS, memory bandwidth, I/O throughput are easy to understand when compared to the more abstract idea of "better randomness".
How well does it need to play? You could just do random moves withinh the allowed rules.
I didn't literally mean to say that a majority of the country watches football. Even if I had (incorrectly), It wouldn't change the fact that it's a huge portion of the country watching the same thing at the same time. And as for "real" football, that's really neither here nor there.
Yes, you're right, sorry. There is one example, maybe it's cherry-picked, maybe it's just one example that happens not to have this issue. What I'm saying is, it absolutely is a real issue, which is contrary to the thesis of this article. What's more, it is easy to mitigate the issue. I've spent more time writing these replies than wiping dust off my lenses ;-)
Thanks for your reply kephra! That's good info to know.
On the RethinkDB thread yesterday there seemed to be a popular misconception that MVCC provides rewindable versioning capabilities -- it does not. Good to see this article getting some traction.
Build your own column-oriented time series database
Atmospheric Vortex Engine
Joystiq is closing and I'll miss them like hell – Polygon
Top Super Bowl Commericals of All Time
Somewhat off-topic, but why are we lionizing Reid Hoffman? It's well-documented that LinkedIn uses extremely shady growth hacking tactics. One of my friends granted LinkedIn access to her Gmail contact list, and LinkedIn proceeded to spam every single contact with an invitation to connect.

[0] http://marknugent.tumblr.com/post/98203319356/linkedins-evil...
[1] https://gigaom.com/2013/09/21/linkedin-is-breaking-into-user...
3 TOP KNOWLEDGE MANAGEMENT TOOLS
What is the user supposed to learn from the visualization? Is it high level understanding what the text is about? Is it some support to make the text easier to read and understand (like syntax highlighting is for programmers)? Is it something in between, something else?
Concurrency is also a problem if you access or lock resources blindly. I once sped up a Java webapp by factor 10 by removing an unnecessary lock in the authentication and authorization code.
Great answer; thanks for the clarification!
> I can't find it, but I remember a popular post that talked about this with respect to iOS' better performance than Android (while having less resources available)

This may have been it, and is a great read anyway if it wasn't:

http://sealedabstract.com/rants/why-mobile-web-apps-are-slow...
Screamed at JavaScript for failing at big integer multiplications. Got it after rewriting in python. Here is my poorly written explanation.

--------- MAJOR SPOILERS ---------

Let's multiply unique prime numbers and count their factors.

n Number of factors
2 2 (1, 2)
2x3 4 (1, 2, 3, 6)
2x3x5 8
2x3x5x7 16

Notice that each time you add a new prime number, the factor count doubles

i.e. 2x3x5x...x500500th prime will have 2^500500 factors

The 500500th prime is 7376507 (thanks to wolframalpha), and it's not very big.

We can easily get the product of the above with 500500 multiplications while modding result of each step by 500500507 to get an answer.

Except that the product of first 500500 primes isn't the smallest number containing 2^500500 factors. This can be made smaller.

Observe that

(2x3x5x...x500499th prime)x2

has

2^500499 + 2^500498 factors.

This is because with the extra 2 we added, it produces additional factors with existing factors that are a multiples of 2.

Continuing with this, we see that

(2x3x5x...x500499th prime) x (2 x 2)

has

2^500499 + 2^500498 + 2^500498 = 2^500500 factors.

This is a better solution, since 2x2 is smaller than the 500500th prime.

If we want continue doubling factor count by multiplying 2s, we will need to multiply by 2x2x2x2 next, and 2^8 after, which becomes inefficient quickly.
Instead, why not use 3? Multiplying (2x3x5x...x500499th prime) by (3 x 3) also doubles the factor count.

So at this point, we think about doubling factor count and the ways we can do it. Out options are

<1>. Multiply by a prime that we have never used so far

<2>. Multiply by an existing prime k + 1 times, where k is the number of times it has been used

We repeat this 500500 times, using rule <1> and <2> (which can be generalized to one rule) and the result is the final answer.

factor count n
2 2
4 2*3 <1>
8 2*3*2*2 <2>
16 2*3*2*2*5 <1>
32 2*3*2*2*5*7 <1>
64 2*3*2*2*5*7*3*3 <2>
128 2*3*2*2*5*7*3*3*11 <1>

For 16 factors the number works out to be 120 (just like the example!). For numbers shown in the questions, sieving the prime takes some time, I also found it helpful to use a binary heap for speeding up finding the next smallest factors.
As someone who is not a heavy coffee drinker can anyone explain what the attraction of Keurig is without ranting about something dumb like "Everyone's a sheeple!" or "Because the masses don't care about flavor!"?

Is it because it's quick? Or just hassle-free? According to some comments here their coffee pods also aren't recyclable, which seems like it wouldn't mesh well with coffee drinkers in SF.
> Even if we had the perfect computer, at the end of the day it's got to get to your average person through a local news weather reporter.

An average person maybe, but not us if you use Dark Sky for iOS or its web equivalent http://forecast.io/

I'm simplifying but it essentially plugs your GPS coordinates into a weather model and gives you the output.
Seahawks Predicted to Win Another Super Bowl Must See Video
Wiretaps 2013 – 41 wiretaps encountered encryption in 2013 up from 15 in 2012
Guide to installing OpenBSD with full disk encryption
I've always wondered why freelance developers compare to plumbers/electricians. You will hire the plumber for 3-4 hours of work and that's it - he'll be on his way. But we will be there for days/weeks/months.

I agree in charging as much as you want if you can solve a problem noone else can. In ~2002 a company had serious database issues with reporting queries running for over 24 hours. So they hired 2 people who charged 600 pounds each an hour. They both worked on the problem for 2 days and brought down the execution time to 3 minutes. In that case yeah, I think they can charge as much as they want.

Don't get me wrong, I'm a developer as well and I know the difference between a 40/h and a 80/h developer. But I believe an hourly rate must depend on how long you'll be working on that project, what you'll be doing as well and how critical it is.
As a dane who moved to NYC, it was really confusing. Only an hour into my stay, the subway lady asked how i was, and though i thought it was overly kind of her to ask, I told her of my exhaustion from my flight, just to be polite. she abruptly cut me off. Funny in hindsight.
DMCA only works in the USA. I work for a Portuguese record label and we get ignored when we ask the local Youtube copycats to remove our copyrighted content. I even suspect they download the videos from Youtube to upload on their services. And I'm talking about sites owned by the biggest telecommunications providers.
Achieving Growth with No Marketing Is Not Impressive
Agreed. None of the ideas are new, and append-only doesn't make life easier.
Many MVCC systems can. I personally have never had a reason to want to.
Obligatory quote: http://dilbert.com/strip/2001-10-25 - "9, 9, 9, 9... -Are you sure that's random? -That's the problem with randomness: You can never be sure."
Aquapods Are the Future of Oceanic Fish Farming [video]
Google docs for Data
مسلسل من النظرة الثانية الحلقة 23
Modeling words co-occurrence graph and then pruning "weak" edges (or achieving similar pruning by using community detection to find clusters) works kind of like a "feature selection" based on something that resembles a bare mutual information or tf*idf.

I'm not entirely familiar with LDA, but from what I was able to understand from their intro, it feels like their LDA application could have used some feature selection.
Is it really that different (than the US)? How did you get a work visa?

I can't even express how many "geniuses" I've tried to work for only to find out they completely crumble under the pressure of the expectations they build for themselves, and then are unable to accept help or work with other reasonably intelligent people (which might threaten their ego).

Seriously getting burnt out by tech, which is sad, so if it is really as different as you say I would love to make a move.
Nobody is squeaky clean. Therein lies the problem.

On a devils advocate note. Maybe the erosion of privacy will lead to more informed voters and honest politicians. For this I don't hold much hope though.
To clarify for others who might not see this: Where do you think sensitive data - private keys, passwords, etc. - should be kept? For instance, when setting up infrastructure for a company, how would you desire that data be shared across users?

Once might have the same reservations about something like Heroku - or really any cloud provider - given that at some point, you are pushing code to a server that is owned by another company whose security you cannot audit.
It's hard to understand why spoofing would be illegal. If it fools traders who are relying on buy orders to judge the price, then why don't those traders just accept spoofing exists and not rely so heavily on the unfilled buy orders?

If it's illegal to cancel a buy order immediately after placing it, then perhaps exchanges shouldn't provide that facility? Perhaps there's some legal reason you would immediately cancel an order, and this action somehow doesn't destabilize the market or screw over other traders?
For Glide, video is the new text
How about "exposing cron as a filesystem"?
Really glad you did! Thanks--what an interesting find.
The Bitcoin Foundation’s View and Policy on Trademarking “Bitcoin”
I tried to google Atmel backdoor and I didn't find anything...
I've not had any issues with SBCL on Windows except with the thread support.
This thread was spinned off from such a comparison: comparison of cybercrime conviction rates to conviction rates of the Spanish Inquisition. That sounds anachronistic indeed in my opinion.

I'm also curious if someone has good sources pointing out that the conviction rate of the Inquisition really was 99.6 %.

But of course, if we go back to cybercrime, a conviction rate of 99.6 % may be indicative of two different things: either the courts will convict almost any and every prosecution, or then only rock-solid cases are even prosecuted. I tend to think that it's mostly the latter: a small proportion of all cases is ever prosecuted, but those cases are then so clear regarding "what happened" that conviction rates are high. It is then another matter "what it means", i.e. the facts are not disputed but the interpretation of law is, and this is why convictions may be repealed on appeal.
I was just about to chime in here regarding Twitter too. I recall it blew up in 2007 not 2006. Also, the winning factor of Twitter back then was indirect group texting. Once you signed up for Twitter with your phone, you could send a text to a specific number and it got blasted out to everyone following you. Instead of sending 10 texts to 10 friends, you just sent one. Anyone who bothered to respond, could just directly text your cell or tweet it themselves to carry on the group conversation. It was great for "Hey, I'm hungry... who wants Thai?"
Perhaps random number aren't in demand, beyond the resources currently supplied?
Everything is a file doesn't scale to more than one system without resorting to hacks (Using NFS or the likes).

One condition for a scalable system is a network interface imho.

Cron doesn't have that as well, which usually leads to application level complexity to avoid running the same cron on multiple machines.
We live in a world where a single publication is less and less meaningful. The writers from Joystiq will likely continue writing about games. Find your favorite ones and follow them, and you'll still get the same quality of content.
Sounds like a Fox News title, where they expose cron to secretly be a filesystem. They'll call it "crongate".
My first computer too. I remember I managed to create a Frogger like game in ascii art, before turning off the computer and deleting the game for ever.
Intuit Embraces the Disruptors--Partners with Uber
> github must present ssh fingerprint into their web page and user must check that fingerprint

yet another case where DNSSEC secured SSHFP records would automate this. However, given that people currently commit and push passwords, private keys and who knows what else to places like GitHub, it seems unlikely these people would recognise why a connection might refuse (e.g. because of an invalid fingerprint) anyway

> authtoken is supposed to be an unique identifier and github server knows that it's associated with your account

ah sorry I thought "authtoken" was meant to be some command to run on the server.

Frankly I think things like adding a public key (whether to GitHub or a system that allows SSH logins) over the internet, are probably safer behind a double factor auth system (e.g. password + otp or client cert + otp) - the people who need to use it can be shown how to copy their public key quite easily (if they can't open Terminal.app, type "cat ~/.ssh/id_rsa.pub | pbcopy" and then paste the result into a web form, can they really handle Git, or even SSH for that matter?)
What!? Did you even read the article? It covers the fact that files are network transparent...i.e. every file is available everywhere on the network. You really should give the article a read and maybe read up on Plan 9 a bit. You're missing out on some really interesting stuff.
Same thing happened to me, despite living a few hours' drive away from this Nebra archaeological site.
imo, fortran90 is about the easiest language to follow - if language is all that is stopping you (and you know some other language already), then you should just pile straight in.

From looking at some of the code, what is hard for me to follow is the 'overall view'. The documentation (of the model) looks quite nicely done, but there must be a huge amount of fluid dynamics etc etc knowledge, that would take me years to learn.
This makes me think of a 4th point:

Performance problems aren't always tied to CPU usage. Waiting for disk IO is the most common example, but, as you say, coarse locks can also be problematic. Probably the worst I've seen are threaded web frameworks used for making external API calls (to facebook, say). You can a machine doing 0 CPU and Disk, but completely locking up.
Gin – A Fast, Martini-like Web Framework in Go
Alternatively, the companies that have products with DRM could be honest in their advertisements and call them what they are: leased products.

If someone rented a Keurig and then went to add modifications, no one here would cheer it on. It's Keurig's machine after all, and private property law dictate the social rules. Renting and buying therefore have different social rules, laws, and morality attached and companies that try to blur the lines should rightfully be ignored.
Think you replied to the wrong comment.
That's a fair point. I sat down and tried to sketch it on paper after posting my comment, and found there was too much domain knowledge involved to do versioning directly in the database.

My comment can be rephrased that it's still painful to do this at the application level: for such a common task, the frameworks/libraries/software I work with don't have anything baked in to handle it.

I've started trying to use Git as an additional data backend to hold the versioning, but changing from a JSON/Database storage structure to one Git will happily diff and reconcile is in itself not pretty.
What are the best productivity tools for programmers?
I am not sure of the exact behaviour. I only read something Lattner wrote as a comment on very slow behaviour when passing things back and forth into CoreGraphics that relied on bridging Swift arrays/dictionaries into their CF counterparts.

In the debug trace, it appeared that the Swift hash map got bridged over and over again every time it was to be read. Since this was a set of attributes for CoreGraphics graphics rendering, the dict/array conversion ended up costing magnitudes more than what the rendering itself did.

From what I remember, Lattner (might have been someone else on the compiler team) said that such conversion should only have needed to happen once, and not at every access to an element. Something like that.

This issue started to occur when it was no longer possible to directly create NSDictionary/NSArray from Swift.

I may also misremember things. Best would be to ask on the Apple developer forums.
Databox could one day both safeguard your personal data and sell it
perhaps avoutthere was also being slightly hyperbolic.

The fact remains that OP chose to to write an entire article about this, someone chose to share it on HN, and many of us are taking the time to share our opinions in these here comments.

I'm not bothered by any of these things, including the original 'problem' and people using hash tags in real life, but it really fascinates me that other people are (whether they're also being hyperbolic or not).

I'm wondering if this is an age thing, perhaps. I'm in my late twenties and I distinctly remembered being bothered quite often by many things like this that I now consider trivial. At some point I just stopped caring.

I'm trying to 'toot my own horn here', by the way. I get annoyed way too much by other kinds of 'trivialities', like people standing too close to me in public spaces, or people standing in front of public transport vehicle doors without letting others out first. I'm just wondering whether there's any particular logic to how my frustrations shifted over time, and whether the same logic applies to other people.
> Whatever happened to rather that 10 guilty men walk free rather than that one man be jailed innocent?

That has always been bullshit, the way you and most of the Internet are misusing this quotation.

Why 10 to 1? Why not 1 to 1 or 1e100000 to 1?

This ratio is purely arbitrary.

Obviously, if you really followed through with the gist of your argument, we should never convict anyone, because better to let everyone go than miss the one injustice in a thousand years.

Unless you can make a case for a specific ratio, 10 to 1 has no special meaning.

The quotation only argues that the balance should be biased towards aquittal. Nothing more.
Is it that 'physics simulation' forecasting is best for short timescales (e.g. < 1 week in the uk), whilst ML can be better for estimating next year's weather statistics in some particular place? (Because your butterfly effect means that simulating the actual weather out to 1 year 'accurately' is not possible).

Presumably any short-term ML forecasting would use the physics simulation as it's main input, and then try to improve slightly on it (e.g. maybe you observe that in a particular small area, the rainfall is on average 10% more than the physics simulation, so your ML method would add 10%, giving you a slight forecast improvement)?
Looks like a really nice book, but I don't know a thing about Erlang.
I love Project Euler. In this thread I've found references for similar projects but focussed more on actual coding (http:(http://oj.leetcode.com) or on BioInformatics (http://rosalind.info/problems/locations/).

Any chance you guys know of a similar project for Computer Vision or general AI topics?
Im such a user. Sometimes I feel the urge to comment, but it happens so rarely that I won't recall my username or password. So I probably have 20 accounts on HN and 40 on Reddit, each with just a few posts.
Misfire: Fighting Fire with Fire on Tinder
Should you perform a quick internet search using the keywords "Tinder Troll," the immediate results document the exploits of men as they prank women on the popular matchmaking app. Interestingly enough, there are extremely few instances of men themselves being pranked over the network.

Until now.

In an attempt to give Tinder women a reprieve, a team at SB Hacks decided to give the men of Tinder what they want most... themselves.

When two men match with the female Misfire account, the Misfire bot redirects all of the messages it receives from each man to the other--thus engaging the two unwitting men in conversation.

The men who take interest in their partner usually discover the deception. The men whose aim is focused elsewhere tend to misfire.

The ensuing hilarity is recorded on their website (misfire.me) for the rest of the internet to enjoy.