teleren/devto huggingface dataset:
How to web scrapping using C#
Tags: webscrapping, csharp, dotnet, tutorial
Published: 2020-03-14 20:20:18

---
title: How to web scrapping using C#
published: true
description: 
tags: #webscrapping #csharp #dotnet #tutorial
---

This article is a part of web scrapping series using c#:

| **How to web scrapping using C#**  |
|---|
|  [Speed up web scrapping using C#] (https://dev.to/lleonardogr/parallelizing-web-scrapping-using-c-1d1p) |


Web scraping is a useful skill to learn for getting data. We can get data from the web, to use for Data Analysis, Data Science or even to Learn some features of Web. So what is necessary to get data from Web?

## HtmlAgilityPack
![HtmlAgilityPack](https://dev-to-uploads.s3.amazonaws.com/i/bydbgsfljjj9y32ym8w9.jpeg)

[HtmlAgilityPack](https://html-agility-pack.net/) is a library to help getting information from Html pages. With this library, we can transform Html pages into strings and get some part of the entire text, using string methods or Xpath to search specific CSS classes. Here is an example to get page info using then.

```csharp

Console.WriteLine("Getting page from Lord of the rings...");

//Download Html from a Url:
var HtmlRequestResult = Client.DownloadString("https://www.rottentomatoes.com/m/the_lord_of_the_rings_the_return_of_the_king");

//Load HtmlString to AgilityPack Document
var Document = new HtmlDocument();
Document.LoadHtml(HtmlRequestResult);
Console.WriteLine("Gettting data from page...");

//Get movie title, critic score and user score
var MovieTitle = Document.DocumentNode.Descendants("h1").FirstOrDefault()?
    .InnerText.Trim();
var CriticScore = Document.GetElementbyId("tomato_meter_link")?
    .InnerText.Trim();
var UserScore = Document.DocumentNode.Descendants("a")
    .FirstOrDefault(x => x.GetAttributeValue("href", "") == "#audience_reviews")?
    .InnerText.Trim();

//Show the results
Console.WriteLine(string.Format(" Title:{0} \r\n Critic Score:{1} \r\n User Score:{2}", MovieTitle, CriticScore, UserScore));

Console.WriteLine("Press any key to close the program...");
Console.ReadKey();

```


This is a simple example to get data from rotten tomatoes. It's common to see in the internet other examples, generally in Python using [Beaultiful Soup](https://www.crummy.com/software/BeautifulSoup/bs4/doc/) to get data. Here, the strategy is the same as Python. The results are showed bellow and you can see the original page in this [link](https://www.rottentomatoes.com/m/the_lord_of_the_rings_the_return_of_the_king)...

![Results](https://dev-to-uploads.s3.amazonaws.com/i/gey9kqhtmzgriksc8j9j.png)


So.. after this, you can improve your strategy to web scrapping, but this is a good start point if you want to learn how to get data from the web.
Instant Arrays, Javascript
Tags: javascript, arrays, programming, tricks
Published: 2020-03-13 23:58:26

---
title: Instant Arrays, Javascript
published: true
description: Some quick tricks for making arrays in Javascript
tags: javascript, arrays, programming, tricks
---

Here's a cool way to quickly make arrays in Javascript.  

```javascript
let array = Array(10).fill()
```
This creates an array with ten slots in it. I usually find this useful if I need to quickly iterate over a list in React. For me, this is a nice quick way to check if my styling is working. For example, what if we wanted to know what a component would look like with 10 list items compared to 20. 

**Example below** 

```javascript
const List = () => <div>{Array(10).fill().map((_, i) => <p key={i}>Text</p>)}</div>;
```

What's great about this is that I just change one number and get an entirely different numbered array.  No need to manually type out the array yourself.

Sometimes though it's also useful to have an ordered array.  This is also quite easy to achieve.

```javascript
let array = Array(5).fill().map((_, index) => index +1); // 1,2,3,4,5
```

Here we're just using the **index** of the array and returning it after adding **1**.  This gives us an array with five elements ranging from 1 to 5 

If we didn't need an ordered array we could just provide **.fill** an **argument** and fill the entire array with that value.

```javascript
let array = Array(100).fill(2); // [2,2,2,2,2...etc]
```
So there we have it, I hope that you learned something from this article. I love learning new coding tricks so definitely let me know if you have anything cool to share in the comments!
Corona projects?
Tags: corona
Published: 2020-03-14 19:46:47

---
title: Corona projects?
published: true
description: 
tags: corona
---

I suspect that quite a few of us are working on projects related to corona virus. If you need chat or activity feed technology for a non-profit project that's helping to mitigate the coronavirus impact [DM](https://twitter.com/tschellenbach) me and I'll set up a free Stream account. EDtech, telemedicine, DMV, groceries, social distancing, contact research. Anything goes #flattenthecurve

There's quite a few cool projects popping up so far:

* https://github.com/mattroconnor/deep_learning_coronavirus_cure
* https://github.com/MhdHejazi/CoronaTracker
* https://github.com/JohnCoene/coronavirus
* https://github.com/RamiKrispin/coronavirus
* https://github.com/alfianlosari/CoronaVirusTrackerSwiftUI
* https://github.com/AaronWard/covid-19-analysis
* https://www.instructables.com/id/The-Pandemic-Ventilator/
Getting started with Rust on Mac
Tags: rust
Published: 2020-03-14 19:40:03

## The Easy Way<sup>TM</sup> to Install Rust (on a Mac) 

The easiest way to install Rust on a Mac is to head on over to https://www.rust-lang.org/tools/install and use the shell command there - 

`curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh`

But...

Sometimes we get the following error -

> Unknown SSL protocol error in connection to static.rust-lang.org

Sometimes.  I got this on my work machine but not my personal one.

You can read a bit about this issue here - https://www.wormly.com/answers/sid/142/topicid/19

There is a fairly straightforward solution

## The Fairly Easy Way<sup>TM</sup> to Install Rust (on a Mac)

(See Update below before trying this)

We want to use `rustup` so let's get that first.

`brew install rustup`

Next let's setup cargo and rustc,

`rustup-init`

Finally we can check our installation with - `rustc --version`

## UPDATE

[jeikabu](https://dev.to/jeikabu) on Dev.to has given this tip to try.

The old way of installing rustup - `curl sh.rustup.rs -sSf | sh`

I haven't tested this but it is worth trying before falling back on brew.
Generics in Typescript

Tags: typescript, codenewbie, 100daysofcode, generics
Published: 2020-03-14 18:59:38

**Introduction**

A major part of software engineering is building components that not only have well-defined and consistent APIs but are also reusable. Components that are capable of working on the data of today, as well as the data of tomorrow, will give you the most flexible capabilities for building up large software systems.
In languages like C# and Java, one of the main ways for creating reusable components is generics, that is, being able to create a component that can work over a variety of types rather than a single one. This allows users to consume these components and use their own types.

*Interested to know more? Read the complete article from here*
{% medium https://medium.com/@avishekp86/generics-in-typescript-69586b13bbfd %}

If you want me to write the entire article on dev.to then please let me know via comments here or twitter. My twitter handle is https://twitter.com/avishekp86
This is a new one
Tags: discuss
Published: 2020-03-14 18:21:09

So this is a weird situation with so much going on in the world. Being a new Bootcamp grad, I'm still in the process of finding my first coding job. I've looked at so many different areas including web design, graphic design and more multimedia focused jobs. 

But with the scare of how things are continuing to grow, what is this new struggle going to present itself with? For me, constant interview reschedules or even worse yet, cancellations. 

But this makes me wonder, why cancel when video conferences are such a advantage for people. Is it that people don't want to continue their work on this time? 

It's a struggle for sure, on top of that, they ask for professional projects done, when they feel bootcamp projects aren't good enough.

I guess the two week quarantine they're asking people to have is going to really take its toll on everyone.

Stay safe and wash your hands everyone!
Removing older BOOT entries (Windows Version)
Tags: windows, operatingsystems, troubleshooting
Published: 2020-03-14 18:20:04

---
title: Removing older BOOT entries (Windows Version)
published: true
description: Medium Article on bootloader entries
tags: windows,operatingsystems,troubleshooting
---
New article on Medium for removing older boot entries if you are on Windows.. 

https://medium.com/red-flag-studio/removing-older-boot-entries-windows-version-3c03454b9a9e
Bug in article conversion from RSS feeds
Tags: meta, syndication, bug
Published: 2020-03-14 17:51:41

When you import an article via RSS, the following HTML markup at the source leads to words mushed together (like `… tothe …`), because the line endings are eaten up instead of replacing them with (white-)space.

```html
<p>Basically, ‘fpm’ allows you to deploy any software via OS packages,
from an installation tree on disk or from already built artifacts specific to
the chosen implementation language.</p>
```
How I stumbled into Data Science & Python
Tags: python, datascience
Published: 2020-03-14 16:53:05

The year was 2013, and I was fresh out of university. I had graduated as an economics major, and was clueless. Turns out, an economics degree does not make one an economist by career. If only life was that simple. Jobs at bulge bracket banks were fiercely competitive and I was late into the game. The hiring spree was over for research roles at banks, and even then you needed at least a Masters to get your foot in the door. 

Overnight, I was a student turned adult out of work, and so the search began for jobs. Entry-level jobs. Everyone's got to start somewhere. And so I landed my first uneventful job but I had seen and experienced how office work would look like. Meetings, conference calls, e-mails. So this was how it is. Also, data! For the former student, the word data just simply meant how much more can I surf the net before I burst my data - mobile data that is. 

Perhaps it was the nature of my job(s), or something cosmic which led me in 2015 to another job with crazy big datasets, and I had only basic knowledge of Microsoft Excel to get through it all. It was daunting to say the least. Still, I was up for the challenge. A couple of Google searches here and there led me to Data Science, and Python programming and all its wonders. I needed some guidance in the very information-heavy data science world so I enrolled into a reputable Data Science Programme which ran for 3 months to get my data science journey going. It was one of the best investments I made early in my career to date, not that I am a data scientist by the time I finished the course, but for the fact that it started the ball rolling. 

Maybe in the near future, I would go for a postgraduate course in data science just to 'seal the deal' that this is what I will branch into hence forth. 

I hope to make something good although small or incremental in the field of economics and data science. If you have or have come across any interesting #econdatsci projects or initiatives, feel free to connect!
OverTheWire Bandit Level 8 Level 9 walk-through
Tags: security, beginners, infosec, ctf
Published: 2020-03-14 16:47:57

Challenge page => https://overthewire.org/wargames/bandit/bandit9.html

*Level Goal*
> The password for the next level is stored in the file data.txt and is the only line of text that occurs only once

*Commands you may need to solve this level*

> grep, sort, uniq, strings, base64, tr, tar, gzip, bzip2, xxd

This challenge is similar to the last one as well, as it involves searching for the password in a file that already contains a huge amount of text, making manual searching impractical.

Let's take a look at the file in question shall we?

![file contents](https://dev-to-uploads.s3.amazonaws.com/i/tivx5gm6000u6hembrt2.png)

Normally this is where I would put the *Hint 1* part. But if you've been reading the previous walk-throughs and trying to solve this on your own, Google or DuckDuckGo should have been your first goto step. So its a pretty obvious hint.

Do not hesitate to use the search engines! Its not possible for one person to know everything. Besides, this is a learning process. You will naturally remember the commands after enough practice.


So, after some web searches, we can safely narrow down the commands required to `sort` and `uniq`

`sort` well, sorts the text files in alphabetical order. 

`uniq` (from the DESCRIPTION section in its man file) 

> Filter adjacent matching lines from INPUT (or standard input), writing to OUTPUT (or standard output).
> With no options, matching lines are merged to the first occurrence.

So `uniq` cannot work on its own, as it needs a file with identical lines adjacent to each other, which will only happen if the file was sorted beforehand.

So we will need to pipe the output of sort to uniq to get the unique lines. Let's try it out shall we?

![unique lines of data.txt](https://dev-to-uploads.s3.amazonaws.com/i/1po64wp676orwkwfiyx6.png)

Well, that didn't go as planned. Looks like we should have paid more attention to the documentation!

Turns out, `uniq` filters out duplicate lines that are adjacent to each other, and sends the original to `stdout`. But the password is on the *only line of text that occurs only once*
___

How do we get `uniq` to print out only those lines that occur only once in its input stream?

*Hint:* Look in the man page :)
___

Turns out `uniq` has a pretty convenient flag `-u` that ensures *only* unique lines are printed.

So the final command would be `sort data.txt | uniq -u`

![Solution](https://dev-to-uploads.s3.amazonaws.com/i/vdceuvocthez2bl7xp5b.png)

Onto the next challenge!
Data structures in daily programming!
Tags: datastructures, algorithms, frontend, javascript
Published: 2020-03-14 16:43:27

While I studying in computer science graduation there was a separate subject called data structures. But I don't understand what is it till last week. 

I wanted to explain the way I needed while at my college. I wanted to explain the real-world example first and **how can we use it in software development.**


## **What is data structures?**
### **Official**
In computer science, a data structure is a data organization, management, and storage format that enables efficient access and modification. More precisely, a data structure is a collection of data values, the relationships among them, and the functions or operations that can be applied to the data.

I understand this explanation is not feasible. Let me explain in layman terms.

### **Layman term**
![Alt text of image](https://c8.alamy.com/comp/BW2W55/image-of-several-kinds-of-raw-and-boiled-vegetables-and-eggs-BW2W55.jpg)
In real-world, imagine you have some items like eggs, vegetables, vessels, etc., And you need to store all the items. How would you do it? It's not possible to store all the items in the same storage. If you do means eggs will break. So, you need to come up with an efficient solution that the items can be easily retrieved whenever you need and also you can add a certain item easily. How would you do it?

This is where the data structures come to the game. So now you are using different storage structure for different items that the items can be added/removed/accessed from the data storage efficiently. 

### **How can we use it in daily programming?**
Good question! In javascript, there are two data structures we are using array and objects both are good in their purposes. In some cases, you might be needed to insert the value in front of the array instead of to the last. In this case, inserting value to the front is very costlier (we'll discuss it later). So array won't be helpful in this case but there are other data structures will be helpful eg: Linked List. Inserting a value to the front is constant time in the linked list. 

We'll discuss more on different data structures and how to implement in daily development in upcoming blogs.

Simple right?

### **Type of data structures:**
1. Singly Linked List
2. Doubly Linked List
3. Stacks
4. Queues
5. Binary Search Trees
6. Tree Traversal
7. Binary Heaps
8. Hash Tables (Javascript Objects)
9. Graphs
10. Graph Traversal

I'll write a separate blog post for each data structures. Hope you understand what is data structures and why we are using it. Please comment on your feedback about my blogs.

Thanks!
Reactjs Login
Published: 2020-03-14 16:34:42

Hi, how are you? I am just starting to develop an application with ReactJS, with a backend connected to a BD Sql Server. What I want is for you to help me with the logic when I start the session. Currently I keep Locally with the LocalStorage the token generated by the backend and besides this I handle in a global state with redux a variable "is_logged" that takes me true when the start is correct and false when it is not correct. What I do not know is how to handle, from that variable "is_logged" the redirection to, for example: / dashboard, and to protect all the routes within that dashboard, and also, if it is located in the login and is already logged ignore and stay on the dashboard the same. I hope you can help me ! Greetings from Venezuela.
Building Decentralised Link Shortner on Ethereum blockchain using Truffle
Published: 2020-03-14 16:27:09

Link: https://blog.joshsoftware.com/2019/07/22/building-decentralised-link-shortner-on-ethereum-blockchain-using-truffle/

Blockchain is emerging technology, it needs no introduction. By any chance if you are left behind and don’t know about blockchain then I recommend reading about blockchain first before reading this article. You can read about blockchain here or here or search on internet and you will find plenty of article to read from.
Reading helped me become a better developer
Tags: reading, writing, growth
Published: 2020-03-14 16:12:05

---
canonical_url: https://www.jenkens.dev/blog/reading-helped-me-become-a-better-developer/
---

Since I was younger I've always loved reading. I was able to explore new places and learn so many new things without having to experience these things myself. I'm convinced my love of reading has helped me level up as a developer. Here are my three favorite sources to read from. 

## Documentation 
I'd like to think we all read the documentation, but let's be honest sometimes Stack Overflow can be simpler and faster when we are looking for the answer. Over the years I've started reading more and more documentation and I feel like my grasp on certain topics has increased. I really enjoy reading the documentation, especially when I first start learning a new technology since they tend to have a great 'Getting Started' section that isn't too robust or complicated. In recent times there's been a big push to make good readable documentation. It's also a great place to make your first open source pull request. Documentation is great because it tends to get updated as the technology changes while other resources can be out of date. We can often save ourselves a lot of time and futures headaches by reading documentation or other resources instead of finding the first answer that solves the problem on Stack Overflow.   

## Newsletters 
I seem to get most of my information through newsletters these days. I love them because I just sign up once and interesting resources come right to me. I don't have to put any work in and because these newsletters come weekly there's always a lot of recent, relevant information. We can't always be looking out for all the new technology emerging so newsletters help me stay up to date. Any time there's a new exciting technology I can at least hear the name and know it's purpose or advantages. This helps reduce my imposter syndrome because I don't feel as overwhelmed by all the new technology. Newsletters tend to be shorter and have a higher density of information.

Some of my favorite newsletters:
[Cassidy Williams](https://cassidoo.co/newsletter/)
[Alex Kallaway](https://tinyletter.com/dotheopposite)
[Veni Kunche](https://www.diversifytech.co/)
Do you have a favorite newsletter?

## Articles 
Last but definitely not least is articles. I personally love articles. While newsletters help sift through all the information out there, articles are what makes newsletters so useful. A lot of my learning is done through articles. Because everyone has a different experience learning I find articles helpful after struggling with the documentation. They can be especially helpful for us beginners since they tend to break things down one step further and really get to the why behind some of the concepts. 

Before I started programming I had no idea how much reading was involved. I thought you were taught everything in classes and didn't have to do so much extra research. I enjoy reading on my commute because it gets me excited to start the day. As soon as I sit down at my computer I'm ready to work!
Sesion de Usuario y Redireccionamiento
Tags: sesiondeusuarioenreactjs
Published: 2020-03-14 16:08:49

Hola que tal. Recien empiezo a desarollar una aplicacion con ReactJS, con backend conectado a una BD Sql Server. Lo que quiero es que por favor me ayuden con la logica a la hora de Iniciar sesion. Actualmente guardo de forma Local con el LocalStorage el token generado por el backend y ademas de esto manejo en un estado global con redux una variable "is_logged" que me toma true cuando es correcto el inicio y false cuando no es correcto. Lo que no se es como manejar, a partir de esa variable "is_logged" el redireccionamiento a , por ejemplo: /dashboard, y a proteger todas las rutas dentro de ese dashboard, y ademas, si se ubica en el login y ya esta logueado lo ignore y se quede en el Dashboard igual. Espero me puedan ayudar ! saludos desde Venezuela.