skysight-inc/HackerNews huggingface dataset:
Seems like you'd have to construct a pretty weird situation in order to blow up your cache, particularly if your load is enough that you can max out a server. Like, if you're running a lot of instances of the same app, deduplication should work fine, right? It only would show up if you've built a bunch of different applications that use the same libraries and consume roughly similar amounts of CPU; if you're running multiple copies of the same application, the deduplication should work just fine.

And that's assuming that it'd work exactly the way you're thinking.

I feel like the win over running VMs (which incur something like a 12% overhead compared to both Docker and running right on the machine for a single application), plus flexibility, plus ease of deployment is worthwhile. I mean, the current situation is running VM images anyway, right? This is a step in the right direction over that, even you must admit.
Distance shouldn't matter. Assuming you can pick up the (extremely) redshifted signal. Also assuming it's headed directly away from Earth.

What'll happen is that it'll be slowed down by a constant factor dependent on speed. How? Depends on the video player.

The factor should be, if I remember my relativity right, sqrt((1 + v/c) / (1 - v/c)). The formula for relativistic doppler shift of wavelength.

Note that "what factor does the video slow down by" is equivalent to saying "if you have two points a certain time apart in the video, what factor is it stretched out by on reception". Which is equivalent to redshift in wavelength terms where the time is just wavelength / c.

There's also a gravitational effect, but it should be relatively negligible.
Gah that autoplayed and I forgot I had my headset still on. Wow. At least I'm fully awake again. Recommended but turn down the volume on your computer before clicking because that's really loud (un-muffled V8 on a tracked vehicle that makes most Bombardier stuff look very tame in comparison).
I wish they would find a better way to sort their themes. Alphabetical is not helping.
Thanks — fixed!
Stripe enters Asia with Japan beta launch
Do you want to design for failover of a complex system? Then containers are your friend.

Every dedicated server that you set up has an opportunity to not be duplicated perfectly. Every system that knows about your dedicated server has an opportunity to hard code what it shouldn't. Which makes these a potential point of failure. Add enough of those, and you're statistically guaranteed that the careful architecture that you have for failover is a pipe dream.

If everything is deployed with containers and discovery and the correct provisioning, then dealing with the fact that containers move around forces you to solve all of your other problems. And containers provide an abstraction layer that makes the rest of it straightforward.

Let me illustrate with an example.

When I worked at Google in 2010, I remember reading an article from eBay about how they finally manage to transition everything off of a running data center, without interrupting live traffic, and how much planning it took them. And they were congratulating themselves on what a heroic feat they had managed. Most companies today would still consider that a pretty amazing feat, and would find that challenging.

At the same point time I was learning how things were set up at Google so that you could drop any data center at random with barely any interruption of live traffic, and with no manual intervention required. And Google occasionally does this without warning to important data centers just to be sure that it works.
I don't really understand this article.

The only example of industrial design and mechanical engineering being fused given in the article is the heat vent one where you took the vents off the top of the sweat-lodge shaped computer and put them on the bottom because it looked "like a large salt shaker"...

But doesn't heat rise? And now the vents are in the thermodynamically opposite place they should be, and simultaneously sheltered from crosswind by the gumdrop shape? That doesn't sound like much of a field-mesh rather than an intentional crippling of the engineering portion...
Fooled by randomness: Tail risk of violence and its underestimation: NN Taleb [pdf]
Worthless, unless you want to be beholden to 21's mining pool. The minimum block-chain transaction size is 546 Satoshis.
thats a pretty good comment actually. there's quite a bit of similarity.

we generally tend to jump into these as "omg awesome new tech" with a very narrow view. But it also helps boosting more though-out techs (even thus it feels less efficient to go through that route first, its perhaps the only route that works with human: try, fail, try again, etc.)
A Look into Netty’s Recent Security Update
Good thinking, you two! The link shows that the black magic (spindle control, etc) is in a dedicated chip controlled by another chip. The "black box" controlling the "black box." I thought they'd be on the same chip and software package. That would increase risk. I didn't anticipate them putting it on a dedicated chip. It makes sense along some lines.

So, that counts out at least one HD vendor from my claim. More could follow if they do something similar. Thanks for the article, too, as I enjoyed reading it.
Been on HN Front Page? Heavybit Is Hiring a PR Person
[...] since the whole userland that you need is included in the binary (worth noting)

you just dont get the debugging stuff (which is okay as long as you can choose)
I agree, this sounds more like a few people with an idea who are looking for someone to build it for them in exchange for equity. Regardless though if I were looking for a position I would want to know more about who they are and what their background is before I proceeded.
If the code is stinky, it's bad code. It can still be have business value, which is why somebody coined the term "technical debt".
Seems MRAPs are still allowed. That's fair I suppose, depending on need. I just wish they'd attach a condition that recipients must not plaster "Special Operations" (or similar bullshit) on the vehicles.

Fortunately surplus camouflage uniforms were restricted, though there's nothing stopping police from purchasing their own camouflage uniforms independently.
Containers are simply a generalisation of python virtualenvs or ruby rbenvs, to encapsulate the entire environment. If you asked a Python programmer "well, why don't you just buy a new laptop for each project?" you might get strange looks.
> However, since it's not buffering and just playing as it streams in, and because time has slowed for those on the ship, I will posit that the video would play at relatively the normal speed to the observers on the ship as both their time and the stream itself have slowed significantly.

I don't think this is correct. If you're traveling near the speed of light, light from behind you is drastically redshifted from your point of view, which implies that the wavelength is longer from your point of view. So, for example, if Earth sends out a 1 Hz signal, you'll see it at much less than 1 Hz. So it shouldn't play at normal speed. It should be slowed down.
Not always, no. Aptitude !== Experience
I wasn't referring to ToF measurement, I was referring to two diodes measuring the difference in phase of a subcarrier intensity modulated signal. Essentially the subcarrier is a intensity modulation of the light. At XX MHz the optical output power will go from 100% to 0% and then back. If two diodes receive the light at the same distance then difference in phase will be zero (i.e. correlating the two signals will have a 0 lag). If the two diodes are separated by a distance, then one diode will receive a greater optical signal and one will receive a smaller optical signal. Correlating the two received signals (or measuring the phase difference) will directly correspond to the difference in distance between the two diodes - this is because the light travels at a fixed speed and the intensity of the signal directly corresponds to the physical distance. Obviously there isn't a unique solution since the diodes could be spaced at multiples of the wavelength, but for a fixed area this isn't a problem.

This only requires knowledge of the subcarrier modulation frequency, as any diode can be picked as the reference and the others matched to it. This isn't an ideal way of doing distance measurement because noise will greatly effect the estimate, but it's certainly do-able.
I wished VMs would only incur a 12% overhead (that's assuming absolutely optimal configuration and a fairly static load), it can be substantially more than that, especially when people go 'enterprise' on you for what would otherwise be a relatively simple setup.

But you've made me curious enough that I'll do some benchmarks to see how virtualization compares to present day containers for practical use cases faced by mid-size and small companies, my fooling around with this about a year ago led to nothing but frustration, it's always a risk to argue from data older than a few months in a field moving this fast and more measurements are the preferred way to settle stuff like this anyway.
Why? Are they more frequent? Anyway, how frequent it is, is irrelevant. Cyclists can and do cause fatalities (sometimes their own). To mitigate this they should follow the rules.

There are rules of the road for everyone's safety. I obey them when I walk, bike and drive. If we're going to go Stasi on drivers, let's go Stasi on everyone.
My Experience with KetoSoy
Apart from myself, anyone else hoping to see downloading straight into an S3 bucket? (Seeding from a bucket has been around for some time, you could imagine my disappointment when clicking the link)
The Land of No RF
I am referring to subcarrier intensity modulation. If you were to turn the optical power from 100% to 0% and then back to 100% in a sinusoid fashion, you would be modulating the intensity of the output. The diode observes the extremely high optical frequency as DC signal, onto which you are modulating another signal at a lower frequency. e.g. I(t) = cos(f_opticalt)cos(f_modulation*t), where I(t) is intensity vs time, f_optical is extremely high (THz) and f_modulation is the lower modulation frequency (MHz)
BTW, you should mention https://github.com/fish-shell/fish-shell/wiki/Nightly-builds after "You can try the beta now", because I cannot get beta at http://fishshell.com/beta/
I love the idea of using containers. Due to linux popularity and google's backing, containers will be next.

But FREEBSD had jails since back in the day. What's the benefit of containers over bsd jails?
College roommate scares easily
Look Before You Leap – The Coming Leap Second and AWS
Sounds like a more succinct language that compiles down to (unreadable) pcap filters could be made, no?
I'll definitely concede the 'mistakes are made' point because I recently spent a month untangling 8 supposedly identical VMs which really weren't but it took a lot of sleuthing to spot the differences. Oh, and to make life more interesting, not all those differences were made to the configuration files (making the systems not quite reboot safe), quite a few of them were only made to certain kernel settings in /proc and /sys (who thought that we should have two of those systems anyway...?).

I like the test-the-panic-button attitude that google brings to these things, I've yet to get someone to accept my challenge to power down their supposedly automated fail-over solution, it's supposed to work but they usually can't be sure or management would surely not allow such a rash thing as a live test and it's bad form for me to then walk up to the switchboard and trip the breaker. Verry tempting...
This seems to be similar to the Costco / Sam's Club membership model, but only on certain items. Costco's profits are largely from memberships, as items are sold close to cost. I imagine Amazon is selling these items at lower margin to the Prime members.
Yeah we do. My only guess is it's two things: (a) our industry is horrendous at communicating previous generation's wisdom in a usable way; (b) a social phenomenon. Quick example of the first are industry pro's locking up their good advice in obscure, expensive books and cutting edge research silo'd into ACM, IEEE, etc.

The other is a social thing that leads to the "network" effect. People flock to something for whatever reason. This builds a community (or network) that entices others to join. That also tend to forget about other things and reinvent the wheel. Example: much of current work in Web applications aims to solve problems already solved in client-server apps with better efficiency, security, reliability, and portability. Even Facebook went back to that model for mobile IIRC. Good luck convincing most Web technologists to switch to client-server, though.

Whoever solves both these problems will create ripple effects that grow innovation at a heightened, maybe exponentially better, pace. The reason will be a combination of avoiding wasted effort plus visibility into best efforts. I got ideas on Problem 1 but the best minds need to get on Problem 2: it's a gold mine if it's solved.
This basically for batch-processing. That means that the lack of availability isn't such an issue because this lets you trade speed-of-processing for cost.

Say you have a few hundred terabytes of images to process. You can prioritize images by pushing them to the head of the queue, but you don't really care how long the complete batch takes.

If you are happy to wait for your job to complete you pay less. Otherwise, pay more and guarantee completion.
> When you start typing a command-line in Fish that matches a previously-typed command, it automatically shows the rest of the line as an available completion, and pressing ^F or the right arrow will complete it for you.

^R is more powerful, though; it matches at any point in previous commands, not just at the beginning.

> The lack of this functionality actually really kills my ability to use any other shell.

You can achieve that behavior in readline (which is used by bash, python shell, etc.) by changing the up/down arrows from previous-history/next-history to history-search-backward/history-search-forward. In ~/.inputrc:

"\e[A": history-search-backward
"\e[B": history-search-forward

Note that you can still use ^P/^N for previous-history/next-history (or you can remap "\C-p"/"\C-n" as well).
I didn't read the whole blog series, but this data structure seems like it could support O(log(n)) insert/remove in middle or front. The equivalent operation in an ArrayList would take O(n).

Why not show off this performance advantage in the benchmark?
I can't comment on the particulars, because the document doesn't mention this name (maybe it changed since it was originally posted?), but, whatever you feel about the justification for the original article, it seems clear that nothing is served by your re-posting the information.

At the moment, for me, your post is the only hit on Google for the query you suggest.
Is it possible that has been removed (in which case, kudos to the author)? I didn't see a name in the document, and the name that Lorento mentions doesn't appear.
Backlash Against Facebook’s Free Internet Service Grows
Inside Google's Secret War Against Ad Fraud
You have some interesting points, however Eric Brewer isn't just 'some Google guy discussing CAP'. He actually invented the theorem... It's also known as 'Brewer's Theorem'.

FWIW, CAP is not about NoSQL or the 'NoSQL movement', it's about distributed systems and distributed shared memory, which applies to a whole range of computing problems.
Yes i have been meaning too
Sandstorm is interesting, but I didn't realize it had this sort of DNS requirement. This will also mean it will essentially require a wildcard ssl certificate for people who want to completely host it themselves, which could seriously limit its uptake.
It is depressing. Networks are broken to the point where we have services resold back to us to provide the basic functionality that should be inherent in the protocol.
I think your second point is getting closer to where 21 is actually going. If you look at some of Balaji's tweets from a few weeks ago he was talking about 4-factor authorization. Financial fraud is hugely expensive, a huge market opp for a solution. Maybe 21 is working toward a bullet proof system/gadget that will eliminate fraud. Maybe BTC chips in every device is part of that solution.
> That argument doesn't work because that was already possible with TCP

TCP: low level network protocol, HTTP: high level network protocol, HTML: high level layout description (ok, maybe not that high ;) ).

> and it doesn't explain the rise of "apps".

Actually, you can explain apps (and silos) as a way to undo all that open goodness and to bring control back to the large companies that are (rightly) frightened out of their wits by what an open internet and a peer-to-peer world actually means.

Now if we could get providers to play ball and to allow servers to co-exist with residential services on ports 80 and 25 (somebody think of the spam...) and hand out symmetric bandwidth by default we might be able to undo some of the damage.
It is inherent in the protocol, but providers are more than happy to deny you the option to use your local connection to host services and NAT took away the peer-to-peer nature of the internet before it really took off.

See: https://www.fourmilab.ch/documents/digital-imprimatur/ (prescient and very depressing read, but read it anyway if you haven't done so).

UPNP and some other firewall traversal tricks restored some of that but asymmetric bandwidth is the norm these days. Maybe with IPv6 we'll see a reversal of these trends.
"Your terminal never felt _this_ slow before."
The borgmon process would be a completely separate job on separate machines (generally with a lot fewer instances).

The logsaver would also be a separate job, although typically running co-located 1:1 with instances of the actual service job. The service and the logsaver would have access to the same chunk of disk (where the logs were generated) but otherwise they were separate as far as the kernel was concerned. (As far as Borg was concerned they were very much related, but that was at a much higher level than the kernel).
I believe it's been discussed many times before. Developers who use jQuery do so because it provides a good API / abstraction layer, as dealing with browser specific issues could be non-trivial.
> What's the benefit in this?

You're not buying a dedicated instance for a month, you're renting it as long as you need it. If you only need a core for an hour, your bill will be $0.01.
Exactly my point. Dropbox shoudn't even exist as a business model.

edit - not crapping on dropbox particularly. Lots of things should be basic functionality by this point, sharing files being one of them. If stuff was more like plan9, for instance, things like skype wouldn't exist either.
Show HN: Wallapatta static blog generator
It's not really that different: If your preemptible GCE instance gets killed, you can try restarting it as a 'normal' (non-preemptible) instance. But in Google's case, both prices will be predictable.
At first sight, this looks like a competitor to json based data stores such as mongodb. However, it looks like json is just a line format and you still need to write T-SQL.
8 things I learned from building a startup in China
GitHub [1] and Bitbucket [2] do that.

[1] https://help.github.com/articles/closing-issues-via-commit-m...

[2] https://confluence.atlassian.com/display/BITBUCKET/Resolve+i...
Because rails app 1 needs version x.y.z and rails app 2 needs version a.b.c and RVM sucks and managing gems sucks.

This is, literally, the reason.

You can replace rails with any similarly bad technology. I got this explanation a few weeks ago at my job:

the java build process (I'm not kidding) has such a complex dependency graph that it must spin up full containers to do each build.

----

If dynamic languages supported better packaging/isolation this entire thing would be research projects.
Data Mining: 5 tools and techniques for text analytics
Spotify Inks Deal with Starbucks Tasking Customers with Picking In-Store Music
If you are using it for your own use, you can use a self-signed certificate. I think that browsers will warn you if your self-signed certificate changes, even in the absence of certificate pinning (because it is a new-untrusted certificate). If not, then something like certificate patrol[1] should help.

And let's face it - if you are using Sandstorm for the next year or so, you probably understand what a self-signed certificate is.

[1] http://patrol.psyced.org/
It's nice to see Stripe coming here, but I wonder how much demand there really is for it. Many (Japanese) people I know would rather stick to bank transfers and payment at convenience stores than use credit cards. All of our company-to-company business is handled with bank transfers. The only reason we have a credit card is for international transactions, I guess.

On a semi-related side note, I saw Square out in the wild for the first time the other week -- I went to a newish bar and asked the owner if he took credit card; he said he'd bought something and would try it out with my card and it turned out to be Square's dongle, which I've seen in convenience stores here.
> See: chef, configuration management and various deployment services

A chef script is basically the automation of "I need to copy every library of the OS and every line of code for the kernel and redeploy it every time I add a node?" I'm sorry if you didn't pick up on my implied remark. Two problems are then introduced when automating those actions: (1) it doesn't negate the fact that I need to store and deploy a 700M sized OS layer every time I want to add a node (which takes minutes, not seconds with non-containerized configs) and (2) maintaining config scripts can (not always) be painful (version control, rollbacks, etc)

> unless you have very specific use cases.

> Containers are like sandboxes with significant overhead for applications

Again, do you have experience using containers? You seem awfully dismissive ("you're doing it wrong!") in a way that suggests that you might not entirely understand how they actually work...
That is good to hear, thanks.
If you have a distributed system and need to create a transaction across multiple nodes 2PC has some shortcomings[1], the 3PC is addressing this.

[1] https://en.wikipedia.org/wiki/Three-phase_commit_protocol#Mo...
Better support for DNS SRV records[1] in client apps would help here and outside of Sandstorm, too.

[1] http://en.wikipedia.org/wiki/SRV_record
I think it's all practice, and a little natural talent. Then again I can't say I'm the fastest thinker. That is, I don't know. I've never compared myself.

What you're talking about here takes practice and a super powered brain. But at the same time, maybe you need to be more specific. I ask myself these questions sometimes as well. Try learning a diverse skill set, rounding yourself out. At the same time, know and accept the brain's limitations. For example, you wouldn't start a car in 5th gear. But those are the kinds of things we do with our psychology all the time. Like multitasking, stressing ourselves out, fatiguing our brains, eating shitty food, breathing poorly.

But why is quickness so important?

Drugs, in the right doses, can sometimes help you do this.

Maybe understanding "why" would help as well.
No false dichotomy, I didn't suggest that it's either crooks or those who "can't discern the difference between punishment of speech and punishment for consequences of speech".

The point was just that there are much more obvious problems with people in positions of power than not being able to distinguish with punishment for using one's vocal chords and punishment for whatever it caused.