skysight-inc/HackerNews huggingface dataset:
I've ended up using "big data" tools like Spark for only 32GB of (compressed) data before, because those 32GB represented 250 million records that I needed to use to train 50+ different machine learning models in parallel.

For that particular task I used Spark in standalone mode on a single node with 40 cores, so I don't consider it Big Data. But I think it does illustrate that you don't have to have a massive dataset to benefit from some of these tools -- and you don't even need to have a cluster.

I think Spark is a bit unique in the "big data" toolset, though, in that it's far more flexible than most big data tools, far more performant, solves a fairly wide variety of problems (including streaming and ML), and the overhead of setting it up on a single node is very low and yet it can still be useful due to the amount of parallelism it offers. It's also a beast at working with Apache Parquet format.
> One could argue not all laws are moral and that following immoral laws is emphasising your own interests over those of humanity.

This here is one of my all time favorite false-equivalences.

People performing sit-ins during the Civil Rights era, or women standing up for their right to vote are equivalent to a corporation that wants to run without paying for its drivers health-insurance.

It's another example of the common trope in the news media of "we must cover both sides of the issue". Yes a cashier skimping you on one penny in your change is technically a moral issue - but it's really not.

It's absurd how much our country is going to profit models that consist of nothing more than "make the same revenue, and externalize the costs onto society."
The people trying to get out of Vim probably don't care about having to press one extra key to escape it.
Generally LIDAR outputs a point cloud.

Velodynes tend to output very stripey point clouds.

Edit to add more info: the Puck's data sheet [0] claims a range of 100m and 300,000 points per second.

[0] http://velodynelidar.com/docs/datasheet/63-9229_Rev-F_Puck%2...
Good luck with that. It appears hard to know when the getting is good.

I'm stuck in the rental loop. Looking outward and seeing rents have increased by at least 200-300 dollars a month for similar sized units is disheartening. It's hard enough to save as it is.
It's no concept my friend. Sure, the students need nurturing. With the world each of them is surrounded by, and their own life outside school, they are bound to behave badly. But it sounds like you are saying schools shouldn't have authority. Schools with well-behaved students have an inertia, but the school will also ultimately exercise control over them.

edit: didn't realise who I was talking to, more respect is due (or less opining on 'baltimore' and 'teaching'). I've read some of your journal.
Disruption! Creative destruction!
At first, I thought the same thing. But if they ask that question, then no, I'm not hiding it from the border agent. I'm disabling a feature while I travel so that nobody has the potential to get to it.

Edit: Besides, if I ever travel out of country with my work phone, if anyone wants access to it they'll need to call my work's legal office as I'm not allowed to let anyone access that phone without their permission.
It's that kind of rare blog post written by someone who actually has expert knowledge on the subject.
Clustering suggestion for CoAP server
Hello Hackers!

Has anyone tried connecting more than 65K clients to a CoAP server such as Californium? During our scale tests at iotify we found that a single instance of CoAP server can not handle more than 65535 unique client IP/port combination. We use multiple source IPs so 16 bit UDP port shouldn't be a limitation. I haven't seen any clustering solution for CoAP so far except Eclipse Leshan, which unfortunately requires LWM2M protocol on top.

Any help would be appreciated.
10 years ago I was working in Canada; couple of friends and I (Australian, British and Québécois) decided to go and ski in Montana for a few days. We had a few beers on the way down and stopped just before customs to drop off open cans before we crossed the border. Being 11pm, we were the only people at the crossing. As we circled round they decided something wasn't right (probably justified although not in their jurisdiction) - 4 hours later we were allowed into the US having been fingerprinted and our car searched on a ramp for what I assume was explosives or drugs.
3 days later we returned to the border travelling the other direction - the CBSA officer looked at the cover of all three different nations' passports before saying "I'm sure there's a visa in there somewhere, have a nice day."
> (kill-emacs) C-x C-e

> Pretty straightforward

If this is a joke, it's totally whooshing over my head right now
....and then $NATION_STATE puts enough nodes on the network to execute a 51% attack, and we're back where we started.

Basing anything on proof-of-work puts you in a perpetual race to control more compute than your adversaries.
But are you concerned with border agents searching your phone? If you are then any time spent on this is time well spent. Although protecting your password manager is obviously of vital importance, there's a lot more to be concerned about sitting around on your phone if they can get in.

There's also the general concern -- although I don't know if it's ever been proven to have happened anywhere -- of border agents installing tracking software / malware. They often take the phone out of sight for a while. This is probably more of an issue with Android phones but again if you are a journalist or human rights activist or anyone with legitimate reasons to be concerned, I would absolutely want to wipe the phone as soon as possible after a border crossing if agents had forced me to hand it over for inspection.
At least in San Francisco, there's far more of them. The extra wear and tear on the roads ends up as a road repair bill that taxpayers pick up.

If the TNCs were a 1:1 replacement for traditional taxis, this wouldn't be a legitimate argument, but the evidence out there suggests that TNCs have grown the size of the car hail market at the expense of off-peak transit, so the TNCs really are putting more cars on the streets.
you didn't specify anything about RAM/resources...
Cool, as a story writer sometimes it really helps me just make a voice memo! I've actually mocked out a full 3 Act feature script on a casual walk. Granted, I had the characters and idea already, but it was just a nice use of time and surprisingly beneficial. Keep it up!
More whispy-lines.

https://youtu.be/ZiscokCGOhs?t=1m36s
Nocturnal animals function by having a large proportion of rods. This was true of our anthropoid ancestors, resulting in humans having very limited high acuity color vision. We get round this by rapidly scanning and stitching the world together (the inability to do this is a sore problem for visual prosthesis). That the world seems sharp and rich in color is a testament to the feats of predictive inference our brains regularly perform.
Pit vipers too. The pits are used to 'see' infrared.
Q2: THis isn't a syntax thing, but it does require knowledge of python queue objects to get right

Q3: This requires knowledge around the semantics of argument passing, which vary from language to language.

Q4: this again requires some understanding of how values are stored. Some languages will copy, while others will use a reference to the originally inserted value.

Q5: python closure syntax. python scoping rules.

Q8: unless you know what "bytearray.decode" does, the best you can do is make an educated guess from the context.
The article is unbearable to read.
As a general comment to so many of the follow-ups to this post:

You really, really don't want to get into a rules-lawyering match with Federal fucking prosecutors over whether "clever technological solution" counts as "hiding" something or not. They have all of the guns in this situation, and you have a demonstrably inaccurate understanding of the relevant statute.

You WILL lose.
Yes, I think it can. However, this type of content makes up a smaller percentage of HN content, and lack of video and images makes it less visceral. Also, HN commenters are fairly good at acknowledging this and helping each other counteract the distortion.
I have joined many kickstarters. I received all of the rewards promised to me, some with a lot of delay but no kickstarter I joined was ever cancelled.

People approach kickstarter with the wrong mindset. It is not an alibaba/aliexpress/taobao market where you can buy interesting things. When you're joining a campaign, you're investing on makers, helping them develop an idea. You are not buying stuff, ideas might not pan out, you're in essence an investor.

If you approach it like an investor and analyze each campaign with cold eyes, looking if the team behind it can actually deliver the goods, if it is worth the risk of investing and if you actually have a personal investment in the idea as in "I want this product to exist", then you will not be frustrated, even when things don't pan out.

I am somewhat tired of people approaching kickstarter with the idea of "I am buying stuff and they turn out to be vapor". This is not understanding what kickstarter is.
For all the guys (and gals) out there, let me introduce you to the Diesel JoggJeans. They look like jeans, coming in various washes, cuts and styles, but they are sweatpants. May you never go back.
Postgresql, SQL Server, Oracle, or MySQL, properly configured can handle larger loads than some people assume. Even SQLite is pretty performant; you just need to write some kind of front end to make it usable as a service.

Of course, it depends on the precise nature of your workload.
Show HN: Muuri – Responsive, sortable, filterable and draggable grid layouts
Heh, yeah, that was sort-of what I wanted to emphasize. I feel like some of the appeal of high-priced denim is for the fashion aspects, even if the quality is a little better than Levi's or similar mass-produced stuff.

> Even the founder/designer jokes that he wouldn't pay full price for his own designs.

I really don't blame him. A lot of it seems like jokey conceptual art stuff. Cute to look at/think about but I probably wouldn't wear.
I know very little about blockchain technology other than the high points, but this statement really intrigued me:

"The defining characteristic of anything digital is its zero marginal cost…Bitcoin and the breakthrough it represents, broadly speaking, changes all that. For the first time something can be both digital and unique, without any real world representation."

My question is this: is this the solution to our ongoing problem of not being able to properly reward the creation of digital goods because they can be infinitely reproduced ? I know that this could be considered bad news by a lot of people that use such digital goods for free, but this type of technology can't help but be a net positive benefit for everyone, if true. Especially since we're seeing a future in which more and more people will experience the ill effects of automation on their jobs and careers.
And telling yourself that on such a small sample (200years) Captialism is a success is lying to yourself. We're already facing the hard consequences of capitalism and consumerisms, and with no real change in sight all the QOL improvement in the world will not matter when this planet is mostly destroyed.

But who cares, we'll go trash Mars with Elon.
Lying to a federal employee is a felony; if you know you are answering untruthfully and the USG can prove it then you are probably going to prison.
Easy multi datacenter high availability can benefit smaller datasets as well.
...and all the supervillains are human.

My first rule of debugging: Always fix the known bugs before trying to fix the unknown bugs.
Both pickled and the canned German varieties I grew up with, e.g. with cream, mustard or tomato sauce.
XML? It's not very hard to parse texts for the necessarily present section titles. Codification of legalese and logical formulas on the other hand would not really be facilitated by xml, unless I miss something because I never used it.
Not sure where you work - but not anymore... most kids out of college willing to work for places with "management" pay attention to 9 to 5 thing much more these days.

And they'd be stupid if they didn't.
And I could argue that my intent was to protect myself from identity theft, should my device be stolen. It's a reasonable explanation for having deleted the data.

I'd also argue that I could've purchased a completely new device immediately prior to travel and brought that along, with the intention of being able to say that the device had never contained information that I was trying to hide.

I think that part of the point is that they'd have to prove intent. It's easiest to prove if you refuse to unlock the device, harder to prove if you provide a destrutive password, harder still if you remove your passwords and keys before leaving, and essentially impossible if you have a device that never contained sensitive data in any form.
Relative deprivation certainly seems a better explanation than requiring some mysterious "fundamental universal rights" that people never knew they had until 18th Century Westerners discovered them. Maybe they're a bit like "souls" for atheists?
Yes... "Go's package manager is bad." That's a fact...
The author successfully demolishes a strawman. He's right, the problem was not caused by supply side limitations, but a demand side surged. But the people he's demonizing aren't arguing that. They're saying that the problem can be solved supply side. Build enough housing and prices will stabilize. Easier said than done, but that's the only true long term solution.
Can the border patrol ask you to sign into any online service? Because that's essentially what this is.

The data isn't on the computer they are searching, it's on a server thousands of miles away. The data was erased from the device. If they can force you to sign into that service, they could also force you to sign into your bank, github, etc.
I think some people (like grandparent) truly don't realize when their (well-meaning) efforts to empower some people unjustly invalidate other people.

I doubt that grandparent would use the words "comical hyberbole" if exactly the same treatment had been doled out to someone because they were female, gay, trans, etc. In my opinion, such people have forgotten the concept of justice being blind.
Send Invoices to Facebook messenger, Telegram, Viber and get paid faster
I have some ideas I think will improve our security in this direction. Apple seeks to make it technically impossible to extract iPhone data and I've been wondering how we can do the same with using someone's credentials to enter the systems we build.

One idea is to allow users to define how many concurrent sessions they can have so they can manage those slots and require something sign out before their credentials can sign in again.

The other is to allow users to configure a schedule when their credentials work so you can block most of the world and probably most of most days too.
2FA
Seems like it's a tough argument though, I never have all my email on my phone, or all my dropbox files, etc. If I choose to not sync certain GMAP labels to IMAP, does that mean I am 'hiding' them?
Backup codes are exactly that, though they're more in case you don't have access to your 2FA device.
Have to provide enough bits and twiddles to allow the dept head to hire his favorite people and you're right. This is basically what Tableau et al have done. There are enough buttons that you can have a few "Tableau guys", and you don't have to hire custom grapher guys.
If you aren't aware of a matter concerning your data you can't intend to impede that matter by choosing not to transport your data on your person.
Thanks! This looks like a pretty simple solution to me.
There are a bunch of very popular web development frameworks where pretty much all the functionality comes from the javascript. Angular and React are some of the better known ones.
One option is to lock out the passwords for some amount of time, or to do geolocation.

Both can be defeated (they can detain you at the airport for a whole day, or they can spoof GPS) but neither of these mechanisms holds up to mass surveillance: you can't detain everyone who goes through the airport, or even all people with 1Password, for a day, nor can you spoof GPS at the security checkpoint because it'll probably leak to airplanes. You have to pick individual travellers and put them in a Faraday cage with a Stingray and an internet connection.

I'm not sure what the threat model really is, but it's possible that this will require enough time and resources to disincentivize asking for even more passwords when there's not a very specific suspicion, which might be good enough.
Not really, I've seen amateur 1-2 Dan insulting the other player before leaving a game (online), or just trolling to piss off the other player. People get really childish in Go because it's hard to accept defeat in this game.
I'm sitting here staring at my US keyboard and scratching my head. All of the Menu shortcuts align perfectly with the UK keyboard's iconography but not the US keyboard which opts for labels. And why do we have a key labeled both enter and return?
I read that he said he was selling it because of his severely diminished income, which is true. But I guess he had enough saved and it was just for charity.
this work was well underway before zuckerberg got involved yet the headline implies (suggests?) that it happened _because_ of zuckerberg.

i'm torn; it smacks of getting your name associated with something good after it gets off the ground but it is allowing something good to continue. reminds me of VC funding mentality...
Very true, but that argument was lost over 20 years ago.
Why not just a webcam and a touch screen? Seems like it won't last, the pricing is outrageous.
The last two questions are little more than "what is the name of the algorithm I should use to solve this problem?".

This doesn't test understanding in any way. How is it useful to determine whether someone can work on the examples you give, let alone the average software engineering job?
> Most of the other things I said aren't even controversial

Most of the things you said were a straw man and nothing more. Go and Python are good at what they do. I see them as tools in a vast tool box. You think everything is a nail. Enjoy thinking that's a valid viewpoint.
> Today, we’re very happy with Redshift, and we have 5 billion rows stored in our database.

1.4 billion of our connection logs (24 fields) takes up 89G on my clickhouse VM. 5 billion records would take ~320G.

Based on http://tech.marksblogg.com/benchmarks.html a 6-node ds2.8xlarge redshift cluster is about as fast as clickhouse on a single i5.
> Can the border patrol ask you to sign into any online service?

If you're a non-citizen attempting to enter the US under a visa waiver program, from certain countries, yes, they can.
That doesn't solve the problem, because you could be detained until the data is accessible again.
> They stop just short of suggesting that only 'spatially literate' Americans should be able to vote.

Where are you getting this from? Nothing in the article implies restricting voting rights.

It calls out a correlation between a specific piece of geographic knowledge and a preference for economic sanctions. It implies a causality between them (that less geographic knowledge lead to more violent preferences). And a secondary implication that less knowledgeable/educated Americans preferred a violent interaction.

And in their "conclusive paragraphs", they summarize that American geography skills are poor, they discuss a book "why geography matters", and finally speculate that being less knowledgeable about geography means people can be more easily misled.

The entire article is a critique of our american education / awareness compared to the rest of the world. Voting restriction has terrible history in our country, but I think you're reading into things that aren't there.
The Uberpreneur: How an Uber Driver Makes $252,000 a Year (2015)
Google’s new conference room tool Jamboard is now available
I would say that "buzzword bingo" is even more important for older candidates. The younger candidates will be perceived as more current, so the older ones will need to make clear that they are not "dinosaurs" stuck on 20 year old tech.
Seems extremely language-semantics specific.
https://www.youtube.com/watch?v=hJhZhLg3obk "Amjith Ramanujam Awesome Command Line Tools PyCon 2017" goes through some nice examples: mycli, pgcli, fish shell and few others
I think you are vastly underestimating how fast trends in the market come out.

USB-A was two years ago.

We're currently in the "having tons of dongles" state. In a year, it'll be 50% USB-C 50% other stuff. In 3 years, it'll be 100% USB-C, with high probability given the current market trends. Even Apple is adopting the USB-C standard. That speaks volumes to the coming trends, and how fast they're coming.
That's a good point, but also reminds me of moths circling a flame or porch light. The vast majority of light we see in the world comes from the sun, but when the sun is elsewhere, our artificial light triggers changes in how insects behave.

I wonder if ground-source lidar at night would have similar effects on animals that are "used to" IR coming from above.
You're basically describing two factor authentication, when you have not authorized the particular computing device in question to skip it.
Or Uniqlo jogger jeans.

https://www.uniqlo.com/us/en/men/jeans/joggers
I started learning ember because it was fast and cool and it was my first MVC but with a php background I should have played safe and choose a framework I was comfortable. My lesson: play safe and learn something that you would enjoy and not because it's cool.
Or literally anyone else in the country to which I'm travelling. Perhaps I feel I'm at an increased likelihood to either lose or have my phone stolen? Saving my passwords from a potential thief (who may or may not have tools to break into the device assuming I secured it) is a reasonable precaution.
AFAIK the pattern just gets translated into a pin
The fact that this is a necessary feature today is a horrible thing, something I would never have thought about a few years ago.
It's not unlikely that knowing the distance to North Korea, and hence the incredibly slim chance that a North Korean missile could make it all the way, factors into this opinion.
Well, fundamental human rights were, by the 18th century "discoverers", usually attributed to endowment by a Divine Creator, so maybe not particularly "for atheists".

But, yeah, they are certainly non-physical non-demonstrable entities that are presumed to exist based on faith alone.
The OS/Platform/Ecosystem was a reference to the post I was replying to.

To be clearer, I don't think you need such things when you just need one or maybe a few servers talking together when you remove the cost of parallelism (see article about COST) and all the hadoop machinery. I have an OS, platform and ecosystem, it's called unix and works for 95% of my needs.

Frankly I've seen my current project be implemented partly on single node (to be replicated for later scalability) and partly multiprocess python. The python is maybe 10x speedier and has 10% the code and run complexity. We tried to run it on a shared cluster of hundred nodes for giggles, the python finished running before the other one started launching. Not taking into account the setup / maintenance.

The COST model is a good one indeed.

So I'm roughly paraphrasing the article (which I point anyone talking about needing big data to, just before asking how big their data exactly is.)
An AI Will Decide Which Criminals in the UK Get Bail
A good portion of the trouble retail is in is due to leverage. Neither they nor their lenders baked online in to their models. The leverage kills the flexibility to adapt to new trends, or in some cases even survive a 10% sag in earnings.

Amazon is terrible for clothes, but non-Amazon is fantastic.