skysight-inc/HackerNews huggingface dataset:
> Instead, you can just wrap the DataGenerator in a simple function that lazily outputs the next batch of training examples.

You probably know those simple generators aren't recommended to be used by Keras, instead keras.utils.Sequence is preferred due to (Keras doc): "Sequence are a safer way to do multiprocessing. This structure guarantees that the network will only train once on each sample per epoch which is not the case with generators."

I couldn't see any equivalent of this for estimators, sadly, and wrapping it up in a naive generator seemed like a functionality downgrade.

> because this is part of the compiled Keras model, before ever converting anything to TensorFlow Estimator

Right, you specify a loss function before compiling, however if it is a custom one and you for some reason need to reload a model snapshot (i.e. resuming training), you need to provide it separately or the loading fails. I haven't found any docs on this. Imagine your training optimizer automatically generating loss functions by means of function composition, e.g. you put a mix of +-*/,log,exp,tanh etc. based off some past training experience of what helped in individual cases/literature, then taking 1000s of these loss functions and pushing them to a large cluster where they are scored on how well did they perform, keeping only the best performing ones.

Class weights are specified in fit_generator(), not in compile time; again, here I couldn't find any description on how to convert Keras' weight dictionary to what TensorFlow needs.

> Callbacks... "Penalizing" Keras because TensorFlow offers less functionality doesn't seem right.

The thing here is that some of those callbacks are mandatory for a training to converge, e.g. decreasing learning rate, escaping plateau situations, computing various stats that aren't provided by Keras (outside loss/accuracy; you might want F1, Fleiss/Cohen's Kappa, Matthews correlation coefficient, AUC ROC etc.) that might be decisive for keeping/discarding a model; then also multi GPU callbacks; some people even use callbacks to perform the whole distributed computation as well. In my examples, if I remove any of those callbacks, my models won't achieve any kind of usability but with those callbacks I match world-class results. I couldn't find any non-insane way to map them to TensorFlow prior to our conversation.

As I mentioned, I have a very large cluster, each node with multiple GPUs, so I need an orchestration on both hyperparameters/loss functions per node as well as within each node to run on multiple GPUs.

The page from Keras you mentioned was precisely my starting point and from those tf.Estimator seemed the last devops-intense way to go (Horovod needs MPI and CERNDB/Keras Spark).

I'll take a deeper look into SessionRunHook you mentioned - thanks! ;-)
Internet is losing its memory: Cerf
> they don't want to pay for good talent in security

To be fair, I'm not sure they made the wrong choice.
`gv` for reselecting the last selection

Assigning different functions to the arrow keys helps when learning vim navigation
Portugal did exactly the contrary.

We have put activities in place to try to save Mirandese.

In Spain likewise, after the civil war, regions got active to ensure their local languages strive alongside Spanish.

In Germany some states have the option of children having after school activities in the regional dialects.
I disagree with calling bugs "errors" that the programmer made. A program has a context, and bugs depend on the context, hence the old joke about "features". For a really simple example, suppose I write a program to do mail merge for myself and it has my name hard-coded in it. If you try to use it, now it has a bug, because it prints the wrong name. But it doesn't have a bug when I use it.

And if the context determines whether a bug exists, one can deduce that even if a program is provably correct by some standard and works for a billion years, the context could then change and oops, it has a bug.
Well I don't blame python. It's just that the stack python/Django/DRF is not more efficient for us now as our requirements changed spa/ws/asynchronous tasks. Investing in this stack is more and more difficult as our requirements don't fit.
But yes we can also check flask/sqlalchemy . I'm just afraid of the stuff needed for ws/asynchronous task. This will not simplify our architecture.
I see this point coming up a lot. But if you consider the car a medium of transport for the individual and how it affects the collective it makes more sense. Think of it as "build cities for people, not individuals"
The laundering will work the opposite way. Instead of hiding behind one big entity, political advertiser's will hide behind a multitude of smaller entities to make themselves look like grassroots organizations.

You won't see "promoted by [Koch|Soros], who spent $8.5m on ads this year", you'll see things promoted by "milualkee citizens for freedom, who spent $1500 this year". But the money will be coming from the same place.

If you give advertisers more space, they'll figure out a way to use it effectively. Even if the point of that space was to try and keep them honest.
Could someone shed the light on how useful this concept is? Aren't the data densities we're working with in modern silicon already better than what DNA storage could offer us (or at least in the same ballpark)?
I bought a car earlier this year. Exciting purchase. We had got to the final bit before they hand over the keys and there was some paperwork to sign. On page 3 was the small print about us agreeing to give our data to everyone.

So I refused and made it clear I would walk away. The sales guy went though the whole ‘it’s not a problem, I’ve bought cars from here and haven’t got spammed’. In the end he had to get a manager and it turned out that the option could be removed from the contract, three menus down in the system.

Sounds like no one had ever asked before. I imagine GDPR will have changed this to opt-in.
>The single monetary policy hasn't prevented most countries from having growth.

Many of these are the counties that are beneficiaries of the policy. Ireland is a more reasonable counter- similar problems to Spain, but a good recovery.
Which version were you installing? Version 0.4 released in April added Windows support.
It's probably because you have the NHS and socialized medicine, which would probably house the mentally ill homeless in some sort of mental hospital? I'm guessing the OP post lives in SF or maybe LA, where the mental hospitals were dismantled because of scandals in the 70s or 80s.

There are 2 kinds of homeless, normal people down on their luck, and people who need psychiatric or addiction care. The normal people go use the bathroom in an alleyway or a bush somewhere, the ones off their rocker piss themselves while sleeping on the train. There are far more 'normal' homeless vs. mental homeless, and it's the mental homeless that cause the problems.
Product Update: Introducing Subtasks in OrangeScrum
The reasons for this isn't your privacy. But still it can match you as a person even if your other data is defect or incomplete. A good primary key.
I'm not going to question how and why this way but I'm pretty sure it's something CSS preprocessors like SASS would do efficiently without sending an overblown css file to the client. Compression helps though, so It might just even out!
Would you please not post snarky dismissals to Hacker News? Even if you're right, you damage the community by doing this—and it's fragile, so we all need to take care of it.

https://news.ycombinator.com/newsguidelines.html
these seem like solid advice, especially about writing tests.
You don't see how that is bad for people in demand of living space right now / in the near future?
You could do business with places in the EU, where you are covered by EU's stronger data protection law. If an EU based company do stuff with the personal data of US citizens in the US, then the GDPR (etc) applies to that EU company.
i dont know why i found this so so funny but i did.
At least many many years ago, it was just groups of people with a common interest finding each other. And then within those groups, you had people with a more loose moral compass, who ended up finding each other.

Also, not that many black hat groups are publishing papers, and the days of full disclosure are long past us.
Should be "Juan de Fuca", not "Jaun de Fuca"
god i wish they would just relase something already.
OTOH it might be a way to filter for cheapest candidates. At places where mental capacity is prized much less, and perseverance over long dull shifts is more important, it could even make sense. E.g. for supermarket cashiers or warehouse workers.
> I do operations. I can tell you exactly how not to make rookie mistakes like this

But you guys are expensive and management can't tell what you do, so we invented devops to make the developers do it. It worked perfectly until it didn't.

Seriously though, as a dev with script kiddie levels of pen-testing skills it's amazing the amount of potential exploits out there. Even where I work with sensitive data it's assumed that the only attack vector is external.
Oh god the skytrain is miles better than the situation with BART & Muni in SF.
There will always be outliers in any discipline. I do things 100% or not at all, and seeing others being significantly better than me is a goalpost. It's a mindset, and as long as I'm improving why bother that someone who dedicated their life to something is better? Of course they are.

The thing with climbing is that it's a combination of physical and technique. You can train your physique, but you need to be of a certain build to be one of the greatest. And technique is something you never stop improving. So being good in high school, with the right build and perseverance you could have been significantly better in your mid twenties. But it would have been at the cost of the rest of your youth.
I wrote a package that creates a mock object that can be used for pretty much anything. It's a joy to use when testing.

It's called @adrianhelvik/mock

https://www.npmjs.com/package/@adrianhelvik/mock
Dart has database drivers, and I'm using the Appengine and the Postgres one in server-side Dart apps.

But Google's Dart team is not actively involved in developing e.g. the Postgres driver, it is done by an independent company who is building their stack on server-side Dart.

Btw. I agree: the isolate-based concurrency + language tools is really great for server-side Dart apps. I use it in a highly concurrent data setup, and I wouldn't switch to other language.
There is currently 904.8 TB of data available on Internet-exposed Elastic clusters. Here is an overview of where these servers are located:

https://www.shodan.io/report/yhaN9gje
Or release the source. I'm good at backend but only a hobbyist with node. Perhaps I could do something with your ready-made frontend code.
Honda reportedly retires the Asimo robot
Never heard that. You must be living in the Romandie ;-)
How to write your first Serverless Component
>Our government takes money through fines and taxes and uses it to build infrastructure and provide services.

Using fines to fund public services creates perverse incentives though, especially where the fines go directly to the agency that brings the case.
MonetaryCoin Distribution: How It Works? Video and How-To Instructions
> The funny thing was how people sent out those emails to their newsletter subscribers and I do wonder how decimated those lists became. People got GDPR fatigue and could not be bothered with the emails after a while. This particularly affected smaller businesses who had legitimate reasons to email customers.

We've had data protection laws for 20 years. Those companies were already breaking the law if they didn't have permission to send email, and if they had a legitimate need to send email they don't need consent under GDPR.
Video direct link:
https://youtu.be/t-xkKHQ4204
Why is it not nicer than Germany? :D
> > fines don't usually go to the people injured by it,

> Well they do. Our government takes money through fines and taxes and uses it to build infrastructure and provide services.

That's only true if you consider the public at large to be equivalent to any individual member of the public, or if you believe only the government is "injured" by a data breach.

If I stole all your money and repaid it in fines to the government instead of directly to you, would you consider the matter settled?
Please can you link to the "lawsuits"?
Ask HN: Can one detect hacked Alexas stealing info?
how safe is to buy a used Alexa?

Is there a way to detect if a used Alexa, bought off eBay, was previously hacked to steal data from new owner?
The funds are best disbursed through a non-profit because most high school students do not know about legal and accounting procedures. Say an after-school club raises 20k intended for the annual budget. Sponsors which might be individuals but more than likely are companies will not write a check to any student but rather write a check to the school. Schools alone have many bureaucratic hurdles that make managing club funds inefficient. Therefore, although Clubs might have in paper 20k or say even 50k that money is being held by the school or another organization whose priority is not acting as the middleman, therefore, causing delays and all sorts of problems. Tackling a very cumbersome challenge, congrats Hack Club!
Thoughts and prayers.
I was calling attention to the vast difference between what the status quo security industry considers white hat and what free individuals should consider white hat - low corporate pain versus actually closing the hole.

As I and others have said elsewhere, the data was leaked the moment it was collected and priced for selling to attackers. Forgoing full disclosure is really just blunting the truth, giving corporate whitewash a leg up, and delaying society learning the lesson of what we're up against. As (presumably) individuals and not owners of surveillance companies, we shouldn't bless this behavior as being in the public interest.
Bathroom Decorating Tips
I agree, if elected officials want to reduce traffic, they would put barrier separated bike lanes and keep the trains clean and safe.

But the bay area local governments are self-sabotaging parody where it's difficult for anything like that to get done.
What does easily mean? Not even notice the extra expenditure on the car? When was that ever the case for most people? It's always been a big outgoing, second only to housing.
Not sure it needs to mean firing people as much as giving any constructive feedback to an experienced engineer can be difficult, even if done perfectly.
??? What transit is this? Commercial airline?
Complete nonsense.

Look at all the food serving establishments: almost none of them have a lawyer to check they're compliant with food hygiene regulation.

Look at all the companies who advertise: almost none of them have a lawyer to check if they're compliant with advertising regs.
GDPR would have allowed you to ask the company for it. And to some extent remove it.
Given the military realities of that time, I‘d say it was both smart and inevitable. But definitely disastrous.
Great PoC, but please, don't do this. At my first gig I had to work on a huge Lua code base, where the original author did tricks like this, effectively creating a new language. Learning by example how things should not be done is effective, but the suffering wasn't worth it.
It's only been law for almost thirty years in some places, but pray Europe be prudent!

And what is this "any cost" rubbish?
What's interesting is that I don't think any of the problems you describe with mass transit are inherit to or really even related to mass transit. I think they are just our cultural problems, spilling over into public spaces, such as mass transit.

For example, in Japan there are many less homeless people on the buses and trains. No surprise there though, as there are very few homeless people in Japan. Even in Tokyo there only about 0.1 per 100k, compared with 795 per 100k in San Francisco.

If someone is in a car, that does naturally insulate them from the homeless people, though it certainly does nothing to alleviate the root problem of there being a lot of homeless people.

If our homeless epidemic is so bad that it is ruining our public parks, transit, and sidewalks, one would hope fixing it would become a political priority. Unfortunately, there doesn't seem to be much political will in the US these days to take on big, difficult issues. :/

Also, just curious, where did you see the strategy of cracking down on public transit behavior that was effective?
EU law states this though.

> If a country in the EU choose to make an example of my company for whatever reason would would be preventing them from doing that?

GDPR states all fines need to be proportionate.
I think it can be explained with game theory. Right now customer data is massively valuable to a company for all sorts of reasons. A data breach is unlikely to occur and if it does occur then the financial loses to the company are much less (on a risk-adjusted basis) compared to the benefits. Every company has every incentive to collect as much consumer data as possible.

It will only change when consumers demand it to change or outright refuse to give their personal information away. I think everyone should adopt pseudonyms for everything and to be constantly changing their pseudonyms regularly.
"The original idea was actually completely different to what we ended up building."

Nearly every internet-startup, ever
Also ctrl+[ , if that's any easier.
For me, Roomcarnage is one of the best Dwarf Fortress stories. It's presented as a set of imgur albums, listed here: http://www.bay12forums.com/smf/index.php?topic=139393

It's also a recent enough version, with gameplay similar to the latest ones, and I find the illustrations and writing style to make the whole thing very enjoyable. It might still get updated in the future, though the pace of updates has now slowed down considerably.
> That's only true...

Nonsense.

> If I stole all your money and repaid it in fines to the government instead of directly to you, would you consider the matter settled?

I mean, that's typically how it works.

People get robbed by people who don't have the ability to directly restore what they've taken, so the state takes them into custody and makes them a productive member of society.

Would I consider the matter settled? Gee, I've seen some really stupid arguments on the Internet that have made me wish I could punch someone over TCP/IP, but while I'm wishing I'm not going to wish for that either.
Pokémon is the highest-grossing franchise of all time, even more than Star Wars or any of the Marvel/DC characters:
- https://en.wikipedia.org/wiki/List_of_the_highest-grossing_m...

It was obvious that the popularity was going explode.
Excellent work!

From the article:

> I’ve worked with a lot of data visualization libraries over the years, both proprietary and open source, and I don’t know of any that can both display a million points this quickly, and support zooming, panning, and selection at interactive speed. (And now that I’ve I said this, I look forward to learning about some in the Hacker News discussion of this post )

FYI: https://github.com/maartenbreddels/ipyvolume does this in 3d already for quite a while, and based on this I've created a PR for bqplot (2d) https://github.com/bloomberg/bqplot/pull/661
That last one is not released though.
If every city had dedicated lanes and parking spots for scooters (like we do for cars) - electric scooters would be an obviously amazing compliment to city travel.
We’re told not to talk about any details (even high level) of this sort by legal because anything we put on the internet could be used against the company in court even if it seems harmless to us. If you want to find out about the difficulties just find an engineer working at a major company and ask them about it in person.
A solution that is as simple: Pay a deposit.

If you don't pay the bill, they use your deposit and terminate your service.

My current ISP didn't even ask for a deposit. I just visited one of their storefronts so they could check my ID to verify my name.
Also ctrl-v allows you to manipulate a block of text, or insert text on multiple lines at once using I or A for before or after the block.
Well, this certainly happens at small scales: Police who issue speeding tickets in the US to fund the police force; but it's not clear it happens at large scales.
It is actually many people's full time job to 'solve the problem'. Homeless services in SF alone get funding in terms of billions of dollars, which pay many full time jobs. On top non-profits which get private donations.
There are a whole host of compliance and security issues that you don’t have to (and shouldn’t) worry about when you’re small. Employee background checks, documented disaster recovery plan, 3rd party pen testing, etc. Those are things you worry about once you have built something. It’s different than just application security which itself can always be improved over time even if it wasn’t an afterthought.
they're gone?
Nice! I never thought I'd see mention of something close to home on HN.

I grew up in Port Angeles. Ocean in your front yard, mountains in your backyard, rivers, lakes, hunting. The works.

It was a beautiful place.
Viewership statistics of movies do not take into account pirated movies mostly because it's difficult to count. Tax offices might only care about movies sold, but producers regularly try to estimate their losses due to pirate channels, and I am sure that people who measure viewership as a proxy for impact on the popular culture would love to have to be able to count everyone.

The article's author makes some observation on employment volumes in Southern Europe, then decries "the Eurozone's failure to implement meaningful pro-growth reforms".
In that context, It makes sense to try to estimate how much of the economic activity has simply been reclassified. The fact that it isn't taxed makes it, once again, more difficult to measure (if that's what you meant by your "Taxes exist for a reason" truism) but not irrelevant.
> were more interested in the status of becoming a manager instead of building something great.

It is not the status. For long being a manager was the only way of getting a reasonable salary and to get to older age with a job. Developers were young and badly paid. It existed a generation for whom being a developer or just knowing about computer was a high pay job, but that disappeared fast.

Nowadays is changing fast. The main factor is companies that create product instead of consultancy companies that sell labour. For a consultancy, their profit is the difference between what a company pays and what the employee gets a salary. For a company with a product, the product quality, features and development time is important, so they pay and respect developers a lot more.

I have been part of the indie game development scene in Barcelona. I have met several people in their twenties, half my age, with published successful games in mobile stores and Steam.

Barcelona, the city I know, is changing fast for the better. Even that it is going to take time to get rid of the old habits. But I can see myself going back in a decade if they want to hire a developer in his fifties.
I've lived on the strait named after him my entire life and didn't know about him and his controversy. There's also a wonderful hiking trail named after him (mentioned in the article) which is well worth visiting if you're ever on Vancouver Island. It takes most people a couple days, but it's beautiful, not too difficult, and not too busy.

So fascinating to think people explored these waters in hopes that it would connect to the Atlantic. Not even close.
> Nonsense.

Can you please detail why paying a fine to the government is the same thing as paying a fine to the people injured by a crime?