skysight-inc/HackerNews huggingface dataset:
Always fascinated by how diverse the discussion and answers is for HN threads on web-scraping. Goes to show that "web-scraping" has a ton of connotations, everything from automated-fetching of URLs via wget or cURL, to data management via something like scrapy.

Scrapy is a whole framework that may be worthwhile, but if I were just starting out for a specific task, I would use:

- requests http://docs.python-requests.org/en/master/

- lxml http://lxml.de/

- cssselect https://cssselect.readthedocs.io/en/latest/

Python 3, AFAIK, doesn't have anything as handy as Ruby/Perl's Mechanize. But using the web developer tools you can usually figure out the requests made by the browser and then use the Session object in the Requests library to deal with stateful requests:

http://docs.python-requests.org/en/master/user/advanced/

I usually just download pages/data/files as raw files and worry about parsing/collating them later. I try to focus on the HTTP mechanics and, if needed, the HTML parsing, before worrying about data extraction.
This type of thing needs to be considered, because I think people are really fooling themselves if they think Apple, et al won't be required to do this within the next 5-10 years.

There will be an incident that turns public opinion on this to such a degree that it will be inevitable.
Microsoft has so mismanaged WPF over the years that most developers want nothing to do with it. That means the dot net world lacks any sort of real UI building capability, forcing us int HTML/CSS/JS front ends (at least until Mono goes live with their WebAssembly port of dot net, at which point we can write our front ends in C# again, hopefully not far into 2018)
Report on Manipulating Social Media to Undermine Democracy
the author of TST is refusing donations AFAIK. He also said a lot of blockages came from Firefox and not from himself. I find it a bit sad that Firefox is not making TST a prime feature of Firefox. Are no Firefox developers using TST? I find that surprising.
It also doesn't help that almost 100% of the time, when I'm Googling for docs, I get sent to a page for something deprecated that may or may not still exist in the version of .NET I'm using. It seems like more often than not, I have to really hunt for up-to-date information, and even then I'm never really sure. MS direly needs to do some SEO work on MSDN.

That and the alarming amount of mixed information you get on whether or not using X is a better idea than Y, for ill-defined (from all angles, including MS), deprecation-related reasons. At least that was my experience a couple weeks ago when I got placed on a one-off .NET project.

I'll admit that I'm not terribly experienced with .NET's standard library. But I am very familiar with C# as a language and I'm comfortable using it. It was rather frustrating for the stumbling block to be what it was.
Shameless plug: https://github.com/noam3127/github-code-folding
If drivers were the entire reason, why do they need Uber at all?
Or you can spend $200 and get a pretty good, non-flagship phone, with software that's (usually almost) as up-to-date as the flagships.
A democratically run company is still an owner of private property, the fact that citizens have a say is irrelevant to the idea that property is privately owned. This is evident in the case in which the State is an actor on the global stage and defends its private property from other nations; whether that be land or means of production.
The "typical" American taco is probably meat, lettuce and cheese in a U shell. Sour cream and extra vegetables is "deluxe" or whatever.

I guess most grocers devote more shelf space to uncooked tortillas these days than to the shells though.
Simply false. You don't appear to know what you're talking about, so maybe you should stop pretending.

https://bugzilla.mozilla.org/show_bug.cgi?id=1215061
Thanks, that's a lot better than what I was doing before.
Yea, I mean wasn't the default provider of Yahoo before because they paid for it specifically.

It'd be nice if DuckDuckGo could afford it, but keep in mind, the privacy aspect of their service is a marketing bent. They still run mostly on AWS. If a government agency wants your DuckDuckGo data, they can just serve a warrant to Amazon instead.
My favorites when I was a undergrad:

Architectural Styles and the Design of Network-based Software Architectures (the paper that invented REST):

http://jpkc.fudan.edu.cn/picture/article/216/35/4b/22598d594...

Solving the Dating Problem with the SENPAI Protocol:

http://sigtbd.csail.mit.edu/pubs/veryconference-paper10.pdf
I have my density set to compact, changing it affects the vertical space of everything except the bookmarks and menu bars. They remain incredibly tight.
KDE handles Gtk3 CSD well. I think that might work. I haven't tried it on KDE though.
For anyone else who doesn't speak British, Chippy = Carpenter
... which is why I noted the importance of using well-supported hardware, ie Thinkpad or Dell XPS which are both very well documented in the ArchLinux wiki and well supported by the community with drivers. This solved a big chunk of the issues I was experiencing.

I've found the people who attack Linux for lack of graphics/network driver support are people who haven't used it in recent years. This was a far bigger problem in the past than it is today. But if you're planning to use Linux for your desktop, with the latest software, not 2yr old distros, it's essential to purchase a laptop with Linux in mind. Much like how MacOS is limited to particular hardware, it's good to view Linux for desktop similarly.
I would really love to know what the Blood Alcohol Content of the Yahoo representatives that signed that contract was. Those terms seem absolutely braindead stupid.
I don't think judges should necessarily be punished for bad decisions. The judiciary is a mechanism for allowing society to get along peacefully with one another (ie you can take your grievance to a judge instead of to the mob). I don't really want judges having to worry about punishment when doing their job.

On the other hand. Incompetent and corrupt judges have no place in the judiciary because it's basically an attack on the citizenry's ability to live peacefully. Now you need money, power, or violence to live security and that doesn't go well. Incompetent results in dismissal forever (sorry I don't have a good test for this). Corruption results in loss of past, present, and future benefits and significant jail time.

Third parties abusing the judiciary to abuse the citizenry is just as bad as a corrupt judge. Their end should be in a jail cell.
I completely disagree that the issue is JavaScript here.

In my opinion, the issue is the DOM. It's API is massive, there is decades of cruft and backwards compatibility to worry about, and it's codebase is significantly larger in all major open source browsers out there.
Blackmail https://en.wikipedia.org/wiki/Blackmail is a crime
yeah, wasn't this (or related to it) at the top of HN just yesterday?

https://news.ycombinator.com/item?id=15686653
Sure. But it would be nice to be able to cheese between go and python for the UI, as well.
Things like this illustrate why it's a bit disingenuous to praise the nonprofit status of Mozilla as some huge distinguishing factor. Not only do the executives pay themselves more than generous salaries (1m for chair, 874k for director, 908k for treasurer) but nonprofit doesn't mean free from commercial concerns. They still need to make money to pay those salaries and support their projects. It seems wiser to judge the ethos of a company based on their actions and decisions instead of their corporate tax status.

[1] - https://static.mozilla.com/moco/en-US/pdf/2014_Mozilla_Found...
it's uber slow
I don't think it's a 2nd amendment issue, per se. But it's very similar. If you're afraid of your freedom to stand up against a tyrannical government being taken away, you should be especially afraid of your right to talk freely about those issues being taken away. That's what the NRA people should be thinking about.
Yep, but you're not going to do something like that with all the embedded CPUs out there, and they're still quite capable of crypto.
THe value would in the "Studio" app which is the hardest portion to replicate (if it's well done).

But I would prefer paying a license for the SDK and host the API myself than having to rely on a third party for something so crucial to a business.
As an introvert prone to social anxiety, a lot of this resonated with me. I however, am not subject to betrayal by my own voice, as a stutterer must be. This passage really stood out;

Absent the context afforded by weekly papers and emails, silence acts as a blank slate onto which people tend to project their own insecurities, expectations, even paranoia. It can masquerade as apathy, or rudeness

I've always found that keeping quiet was just easier in so many ways. Consequently I've always been seen as quiet, reserved, stoic at best or cold and judgemental at worst. I've only recently come to realize what a disservice I've done myself by keeping quiet all these years. No one knows what I'm about if I don't tell them. If I'm kind and generous, I have to tell people what I'm about, who I am, through words and actions.
These seem like reasonable precautions that don't disproportionately affect womens' access to your time and expertise. They probably help some women feel more comfortable as well.
Wow, no one's talking about mobile? Have they fixed the lagginess and inaccurate click detection that routinely crops up on the Android browser?
I suspect this happens because what makes the original dish special is the delicate balance of local ingredients, expert skills, and local taste which evolved together over centuries. Take the dish to a new context, with different quality of ingredients and less skilled cooks, and you lose that balance. You then have to compensate by piling on other flavors – ideally popular local ones – to make the dish interesting again.
"Or, dare I dream, one that can import everything from ScrapBook?". Let's hope dreams are answered. You can add me to that probably long list of hopefuls who have huge records of online life in scrapbook for pre-57 Firefox.
This comment is predicated on the idea that each problem can be solved/handled in isolation. Each isolated solution can be weighed against another and deemed better, vastly better, etc.

I think in many cases an optimal product can only be produced when there is a holistic vision that is kept in view. And in this paradigm, is is much more difficult to say A is vastly better than B.

Here's just one example--> many software systems that I come to have trended into maintenance nightmares. When you trace back all of the steps from inception to the present moment, you realize that it is due to a series of these kind of A/B decisions that boiled the frog. In each case A wasn't vastly better than B. But the sum of all the A decisions... And this is where it gets harder to make a case... where you may have to take a long walk to make the case...
Airbus will test its Vahana electric ‘flying car’ by the end of 2017 – The Verge
The underdogs don't have hundreds of millions of dollars to pay Mozilla. They do have to keep the lights on.

But, Mozilla does often do partnerships with smaller, local search engines in small markets.
> It's like when you see the meme/trope about Ikea furniture being hard to assemble.

Oh dear, I always thought that was about missing parts or pieces not fitting. This gives a new perspective …
The emergency move away from nuclear has been incredibly short sighted. I understand not wanting to build new reactors, but shutting down running reactors, with all the capital investment involved, just doesn't make any sense. Especially when there is little risk of natural disasters in Germany.

If people are serious about maintaining the same quality of lifestyle that we have today without burning as much coal, the current solution is Nuclear Energy. Yes it does pose many risks but so does burning coal, and the latter seems to be destroying our environment.
E.g. intuitively grokking the Maillard reaction, fond, deglazing, etc.

In my experience with beginners - the number one error they make is not understanding their heat source and pans. For example, they'll put oil in a thin pan, turn on the burner and drop in a chicken breast right from the fridge and wonder why its not done in 6 minutes.

My advice to beginners cooking on stove tops: get a cast iron skillet - treat it with love and kindness. Let it warm up for a bit before you start cooking.
All great advice. I've written dozens of small purpose-built scrapers and I love your last point.

It's pretty much always a great idea to completely separate the parts that perform the HTTP fetches and the part that figures out what those payloads mean.
Well, I'm not exactly a random guy. I have a lot of involvement in open source out there plain to see, and sr.ht itself is open source so you can just run it yourself.

>Crucially, there's no imperative to leave a platform just because it might be slowly dying. If it does what you need it to do, and do it well, the effort to switch is unlikely to be worth it. SourceForge is still around, and still (strangely enough) servicing some projects -- it's not like anyone got left behind because they failed to see that SF was "slowly dying" in time.

SF has also been injecting adware into software hosted on it. You stay behind at the expense of everyone around you.

And honestly - it's a lot easier, at least for me, to trust a person whose name and email address I know than a faceless company with interests that don't align with mine.
The download prompt mostly. Although the context option was sometimes useful it wasn't very reliable. Sites often don't give you direct links to download files that the context menu would help with.
Wow, that is really cool actually. Impressive performance on my OG Pixel XL too.
Just updated it on Android... and I wish I hadn't.

The very first thing I was greeted with on the "top sites" page was a bunch of "Recommended by Pocket" links, and no way to disable that.

I can't find anything in about:config to disable this either. How about NO, goddammit?

Shame on you, Mozilla.
night light mode:

https://addons.mozilla.org/en-US/firefox/addon/night-light-m...
This makes me want to write a geofencing phone app that vibrates when you get within ~1 mile of your station.
Indeed, under Settings → Notifications → Custom Routing.
I think this just applies to the commercialized versions of those foods in the foreign countries. The pizza you linked to from Mexico looks like something you would get from any fast food pizza place in America and the Italian one looks like a margherita slice from any half decent pizza place in NYC. Same goes for the tacos. The "American" one just looks like something you get from Taco Bell while the "Original" looks like something I would get from a local taco truck.

The burrito is an interesting case. I do think that's evolved a bit in America to the point where everyone is copying Chipotle and Qdoba and trying to make the biggest burritos.
Yeah let's just walk around with our pants down because belts can be used to strangle people.
I just watched the first video. Thank for sharing.
Manuscript – Project management, issue tracking, and support
I noticed the domain when logging in to Kiln, Fog Creek Software's hosted version control app/service, and decided to checkout the website. Based on the screenshot fragments, it appears to be a newer version of FogBugz. The marketing text even includes "Based on the engine of FogBugz".

My company's FogBugz sub-domain also seems to be already registered. Maybe they're going to migrate everyone from FogBugz automatically.

Anyone have an info about this? I haven't noticed anything in any emails from Fog Creek. Nor do I remember seeing anything else about this anywhere.
Can totally relate. Your comment made my day.
First Time Home Buyers Should Avoid Using a Cash Gift
Thirty countries use 'armies of opinion shapers' to manipulate democracy
Is this as good as Play! framework?
Switch to tab: https://chrome.google.com/webstore/detail/switch-to-tab/gbfh...
Its not an argument. Its an observation of the existing monetary system.

When you borrow on your future earnings you are essentially putting yourself in indentured servitude to the banks.... and this is your choice.

No one forces you to do this.
Not at all!

I will gladly use Win Forms, WPF or UWP over anything resembling Electron.
Thanks! Here's how I did it on macOS:

1. Go to about:support in Firefox URL bar.

2. the address is in "Profile Folder" row.
I quite liked it but there was one annoying thing: occasionally, new sections would open in between the lines, after the part is was reading, stopping me from reading. That was actually quite painful and annoying.
The S3 and S4 were pretty nice! Very slow flash, like every other Android back then, but good screens and very capable GPUs. (Since then screen size increases have outpaced GPU speed bumps, so modern phones don't always draw any faster.)
I wouldn't call the python page a cheat sheet. It's more of a concise index into the official docs.
That's not how things work in the real world. This is a blatant conflict of interest, and its ironical inspite of the open source and privacy song and dance by Mozilla there is no transparency on the deal.
> It is likely some outcomes were worsened by that waste of resources

I think the more interesting concept is the second look. Doctors are trained to be sensitive, then specific. Perhaps it would be easier to black tag people if you knew the next thing that would happen is they would go the red tag area for a second call of "black tag".
"Terms didn't read" is useful extension
> I'm thinking about reverse-engineering the way ScrapBook stores data so I can write my own migration to something else, but...oof.

Looking around on the ScrapBook website[0] reveals that "gomita", the author of the plugin, has a GitHub profile[1]. Sadly without a repository of ScrapBook source code. Still, you could try contacting gomita and ask if they want to put it there to help with reverse engineering it? Or maybe even document how it works.

[0] http://www.xuldev.org/scrapbook/

[1] https://github.com/gomita/
I find it hard to imagine you couldn't get a better ROI from just sitting down with "Cracking the Coding Interview" and tediously completing each exercise.
Third party software is always a bumpy ride. Something is going to not work out of the box, optimization will be less than stellar, battery usage won't be as good. I cannot tolerate my cellphone not working 100% of the time.
You can't take a gun, put it into a machine, push a button, and have the machine make you a million guns. If you could, gun control would go from difficult and incomplete to downright impossible. That's how encryption is now.
You might also enjoy similar documentaries Cloverfield, The Blair Witch Project, and This is Spinal Tap ;-)
How is uboot fundamentally different than BIOS or UEFI/EFI? It's not like all the PC clones through the ages had the same bootup sequence for RAM and peripheral init.
I personally have already purchased a Kano Complete kit, which comes with a sound sensor, plus separate purchase of an additional motion sensor.
This whole thing is a straw man.

It is we the people who should be dismantling the security state, in order to solve the terror/war problem - we sure as hell don't need career experts dictating this. The public truly needs to fight back on this.

Crypto is Math.

Broken crypto is broken Math. But Math is broken anyway, because we can't really stop people using Math to hurt us.

After Crypto being banned, what next? The Golden Rule?

We can't have Math, because "Terrorism".

The only solution is an open and honest society, in which sufficient resources are applied, at the human being level, to ensure that terrorism doesn't happen.

Like, we could deploy battleships with bombs all we want, but those same battleships would go a long way towards actually making peace in the world, by healing it instead.

But, we "can't do that", apparently, because "Threats/Force/Terror are more important".

Let we, the people, continue to use crypto to undo the need for such power structures as the NSA, and all the rest of the spooks ..
Until you can kill someone with encryption this is absolute bullshit FUD. Are you a paid shill?
Best Paper Awards in Computer Science (since 1996)
http://jeffhuang.com/best_paper_awards.html

Papers I Read
https://shagunsodhani.in/papers-I-read/
In the mean time, they've lost one of those power users (likely more, going by some of the discussion in issues for various vim-like plugins) and will, in the future, need to do something to entice me back. This experience has left a bad taste in my mouth and I'm not going to be rushing to go back to a browser that breaks functionality on me without prompt.
I'm not sure why you would trust any statements from Pai's office given their history of partisanship and misinformation.

I'm pretty cynical about all politician's motivations, but this statement doesn't hold up. If the FCC really is cheating to benefit one political party or company, then it would be impossible to accuse them of cheating without implicitly targeting that party or company. (Edit: In other words, their statement gives no information that helps distinguish whether the FCC is cheating or whether they are just being unfairly targeted by Democrats. It only points fingers.)

The statement is focusing the issue on politics in order to distract attention from the actual issue. Then at the end of the statement, it accuses their opponents of doing the same thing. Very scummy.
I agree that universities are not going to tighten the purse strings unless they have no choice. The cause, as you mention, is the feds throwing money at the education system. Therefore, the only logical solution is to remove that. You say you don't want to limit access to education, but the world isn't perfect. If you want everyone to have access (meaning having access to funding via loans) then you must accept the consequences of that. I, personally, don't have an issue with limiting access. Education that involves universities with buildings, faculty, land to maintain, etc. are not infinite resources. The natural check on the usage of resources is cost. Presumably, you don't sit around letting your sink run all day because you would wind up with a massive water bill. If people want to go to college and get an education, there is a cost involved. There are always going to be those who are unable to pay due to extreme financial situations. But that's why there are scholarship programs, grants, etc. that exist.

The education system used to work fairly well in this country several decades ago. You used to be able to go to college and pay for it in a very reasonable amount of time even through minimum wage jobs. Loans weren't even necessary. And when the end user pays the bills, the connection to cost is there so people pay attention to what it really costs because they bare the brunt of it right away and it's not hidden (so to speak) from them through loans that are paid off over 10-20 years.

It was only when the government decided that something was "wrong" and needed to be improved that the system started going to shit and getting ever more expensive and the product (education) getting worse and worse - the quality of the education people are paying for nowadays is abysmal half the time.

I discussed this recently in another thread with some specific numbers attached to my explanation:
https://news.ycombinator.com/item?id=15647956
That's right - the notification itself tells you to get up and walk around for a minute.
Tampermonkey: https://chrome.google.com/webstore/detail/tampermonkey/dhdgf...

(greasemonkey for Chrome)
disable autoplay:

https://github.com/Eloston/disable-html5-autoplay

(until chrome gets their shit together)
https://news.ycombinator.com/item?id=15696184