teleren/devto huggingface dataset:
Scraping Corona Virus data with Python and Beautiful Soup
Published: 2020-10-04 16:40:02

Today we are going to see how we can scrape Corona Virus data using Python and BeautifulSoup in a simple manner.

The aim of this article is to get you started on a real-world problem solving while keeping it super simple so you get familiar and get practical results as fast as possible.

So the first thing we need is to make sure we have Python 3 installed. If not, you can just get Python 3 and get it installed before you proceed.

Then you can install beautiful soup with:

We will also need the libraries requests, lxml, and soupsieve to fetch data, break it down to XML, and to use CSS selectors. Install them using:

Once installed open an editor and type in:

Now let's go to the Corona Virus data listing page at the ECDC website and inspect the data we can get.

This is how it looks:

Back to our code now. Let's try and get this data by pretending we are a browser like this:

If you run it.

You will see the whole HTML page

Now, let's use CSS selectors to get to the data we want. To do that let's go back to Chrome and open the inspect tool.

We notice that all the individual product data are contained in a with rows containing individual country info and the tags containing the specific fields of data.

So we can extract them like this:

If we run it we will get all the data we need.

We got them all.

If you want to use this in production and want to scale to thousands of links then you will find that you will get IP blocked easily by several websites. In this scenario using a rotating proxy service to rotate IPs is almost a must. You can use a service like Proxies API to route your calls through a pool of millions of residential proxies.

If you want to scale the crawling speed and don't want to set up your own infrastructure, you can use our Cloud base crawler crawltohell.com to easily crawl thousands of URLs at high speed from our network of crawlers.
Nullish coalescing operator
Tags: javascript, webdev, codenewbie, beginners
Published: 2020-10-04 16:32:41

```
const height = 0;

console.log(height || 100); // 100
console.log(height ?? 100); // 0
```

`Nullish coalescing operator` (??) returns the right-hand side value only if the left-hand side value is `undefined` or `null`

---

**Thanks for reading 💙**  
Follow **@codedrops.tech** for daily posts.  
[Instagram](https://www.instagram.com/codedrops.tech/) ● [Twitter](https://twitter.com/codedrops_tech) ● [Facebook](https://www.facebook.com/codedrops.tech/)

Micro-Learning ● Web Development ● Javascript ● MERN stack ● Javascript  
**[codedrops.tech](https://www.codedrops.tech/)**
Day 20 - CSS Drawing
Tags: beginners, css, codenewbie
Published: 2020-10-04 16:16:39

[Here](https://codepen.io/josno/pen/vYGoarP) is a CSS drawing of a mojito!

### What I Learned
I learned to build a leaf SVG and use a path builders! I also learned to make asymmetrical borders.

### How This Will Help Me
Understanding the power of SVGs will hopefully help me build more flexibility in frontend development.

Onto the next challenge!
Differentiating Authorization from Authentication
Tags: tutorial, auth, todayilearned, codenewbie
Published: 2020-10-04 16:08:08

Authentication and Authorization are 2 confusing terms in the world of tech. It is not difficult to use them interchangeably when you do not understand their meanings. This is a very common problem.

In this tutorial, I will be teaching you the difference between these terms using a real life scenario. Let's get to it.

## Real Life Scenario 

> Scenario 

Imagine that you are visiting a house and you knock or ring the door bell.

Someone comes and open the door, seeing that you are someone they know, they let you in **(i.e. authentication)**

But While in the house, you can only seat on the floor and no where else **(i.e. authorization)**

> Other Explanations

1. You can login to an App **(i.e. authentication)** but you can access only the features of a “user” because you are not **“authorized”** to use other features.

2. **Authentication** tells that you are an approved user of an app while **Authorization** tells if you can use a particular feature. 

### Demonstrating Authentication
I have shown how Authentication can be achieved using nodejs and mongoDB in my series entitled: *"Authentication with Nodejs and mongoDB"*. Check them out below:

1. [Part 1](https://dev.to/ebereplenty/authentication-with-nodejs-and-mongodb-part-1-10pn)
2. [Part 2](https://dev.to/ebereplenty/authentication-with-nodejs-and-mongodb-part-2-1hg9)
3. [Part 3](https://dev.to/ebereplenty/authentication-with-nodejs-and-mongodb-part-3-3mic)
4. [Part 4](https://dev.to/ebereplenty/authentication-with-nodejs-and-mongodb-part-4-4p37)

## Conclusion

We have so far seen that even though these terms are related, they are indeed different. I hope it makes sense now. I will be making a code example of authorization in the future hopefully.

Meanwhile, see you around.
Avoid Losing Errors in a Bash Pipe
Tags: bash
Published: 2020-10-04 15:48:28

---
canonical_url: https://kbravh.dev/writing/avoid-losing-errors-in-bash-pipe
---

Have you ever wanted to string together some bash commands using pipes, but weren't sure how to not lose the error codes in the process?

---

tl;dr Add `set -o pipefail` to the beginning of your script!

---

For the [CircleCI Rome Orb](https://circleci.com/developer/orbs/orb/kbravh/rome) that I created for Hacktoberfest, I was seeing very odd output to the pipeline logs for the commands. This was due to the way that the Rome linter streams its output. Each message was appearing one letter per line instead of showing the full message.

![Line output letter by letter](https://dev-to-uploads.s3.amazonaws.com/i/t02fqt3ddaqw73zc82mh.png)

In order to alleviate this, I wanted to pipe my linting command through to `cat` to be displayed all at once instead of being streamed in character by character.

`npx rome check | cat`

![Lint output line by line](https://dev-to-uploads.s3.amazonaws.com/i/ii2863898mev5nc07xdm.png)

---

Are you curious as to why we can't use `echo` here? The `echo` command only takes input from the arguments passed to it directly, not from stdin. If we used `echo` here, the console would only output a blank line!

---

However, just doing this piping would cause a problem in our script execution; if linting failed, the error code would just disappear! The problem stems from the fact that Bash does not always act like higher level programming languages. If a command fails, Bash will continue execution like normal. For a linting command, we want to get back our exit code so we know if linting failed without losing it in the pipe.

Luckily, there's a handy option for this. 

`set -o pipefail`

By running `set -o pipefail` in our script before our pipe command, bash will finish executing the entire pipe and throw the error code of the right-most error that occurs, if any. So if `npx rome check` fails, that error code will still be thrown even though `cat` succeeds.

If you want to get some more in depth information and see some other good options to set for your bash scripts, check out [vaneyckt's great article](https://vaneyckt.io/posts/safer_bash_scripts_with_set_euxo_pipefail/).

Hero photo by Valentin Petkov https://unsplash.com/@thefreak1337
Key-Value store implementation in C++ : A series 
Published: 2020-10-04 14:31:08

I will be posting a series of posts where I will be documenting a project inspired by Emmanuel Goossaert’s Code Capsule blog post : http://codecapsule.com/2012/11/07/ikvs-implementing-a-key-value-store-table-of-contents/



An example of a KVS is dict in Python, or HashMap in JAVA. It’s a simple database format, where we can store a value attached to a key. It’s also considered a NoSQL type of database, much like MongoDB.

The interesting part of the project is that unlike other KVS projects, we’re targeting specific data representations, operations (read/write..), and It will be applicable to respond to a specific issue. We also want it to allow different access methods other than the simple get method (sorting keys like TreeSet in JAVA..).

It's not completely original, since as I already mentioned, is basically my documentation of a project made by Emmanuel Goossaert. 
(I tried to document this on Tumblr lol, but the dev community is non-existent so I would never get any input from anybody)
Write a simple loop with fix
Tags: haskell
Published: 2020-10-04 15:24:31

`fix` (defined in `Data.Function`) is a function that calculates the least fixed point of a function.

```hs
fix :: (a -> a) -> a
fix f = f (fix f)
```

For example, the fixed point of a function that always returns "hello" is

```hs
> fix (const "hello")
"hello"
```

just "hello". This is because a fixed point is a point where the input and the output of a function become the same. 

It is well known that recursive functions can be written as least fixed point of higher-order functions in domain theory.

```hs
fact :: Int -> Int
fact = fix $ \f n -> if n == 0 then 1 else n * f (n-1)
```

## Write a simple loop with fix

You can also use `fix` to write code in your everyday life. For example, let's implement a program that repeats the action of inverting the user's input and outputting it until the input "quit" is received.

```hs
main :: IO ()
main = do
    putStrLn "Start"
    fix $ \loop -> do
        str <- getLine
        unless (str == "quit") $ do
            putStrLn (reverse str)
            loop
    putStrLn "Bye"
```

If you don't use `fix`, you can use `let` to define the recursive function once and then execute it, but I think using fix is cleaner because you can define and execute it at once.

## Give the loop an initial value

Next let's consider an example of setting a limit on the number of loops. Let's make a loop that terminates automatically after 3 loops, even if you don't type "quit".

```hs
main :: IO ()
main = do
    putStrLn "Start"
    flip fix 3 $ \loop n -> do
        unless (n == 0) $ do
            str <- getLine
            unless (str == "quit") $ do
                putStrLn (reverse str)
                loop (n-1)
    putStrLn "Bye"
```

You can give an initial value by flipping a fix. You can see this in the type, as shown below.

```hs
> :t flip
flip :: (a -> b -> c) -> b -> a -> c

> :t fix
fix :: (a -> a) -> a

> :t flip fix
flip fix :: b -> ((b -> c) -> b -> c) -> c
```

In other words, you can think of the `fix` type as `((b -> c) -> b -> c) -> b -> c` and `flip`.
#100DaysOfCode Thoughts?
Tags: 100daysofcode, python, datascience
Published: 2020-10-04 15:19:31

---
title: #100DaysOfCode Thoughts?
published: True
description: 
tags: 100DaysOfCode , Python , DataScience
//cover_image: https://unsplash.com/photos/wh-RPfR_3_M
---

![pic](https://images.unsplash.com/photo-1554306274-f23873d9a26c?ixlib=rb-1.2.1&ixid=eyJhcHBfaWQiOjEyMDd9&auto=format&fit=crop&w=2850&q=80)



I have always been interested in computers and programming from a young age, but never supported my direction.

Python courses always try to give you the basics and nothing more; most courses nowadays do a lot better when it comes to the content. 

I wanted to learn programming on a deeper level, so I decided to buckle down and learn Python from the top. I've made some small python projects in the past, but nothing too significant, just enough to connect the dots. 

Then cam the [Python for Everybody](https://www.coursera.org/specializations/python) and [Data Science applied with Python](https://www.coursera.org/specializations/data-science-python) from the University of Michigan; this was a big game-changer. 

These courses brought you through everything you would need to get started as a python programmer, from understanding the workflow and how to think about problems from an engineering standpoint. 

I will say that after completing this challenge, I do think that you could probably learn a lot more, focusing on a project-based capacity.

Even if you do not know 100 percent of what you are doing, this will allow you to research and find out what you need for a specific problem; I think this correlates well to real-world experiences / Projects.

100 Days Of Coding can be an excellent resource and or habit for coding, but I think that moving on to something like 100DaysOfCoingProjects would be a good idea as well.


### Project ideas

https://github.com/karan/Projects

https://www.codementor.io/@npostolovski/40-side-project-ideas-for-software-engineers-g8xckyxef

https://www.reddit.com/r/learnprogramming/comments/2a9ygh/1000_beginner_programming_projects_xpost/

https://www.freecodecamp.org/news/more-project-ideas-to-improve-your-coding-skills-99f48d09bb4b/
How to create recursive multi-columns in react-beautiful-dnd? 
Tags: react
Published: 2020-10-04 15:15:28


in the following code sandbox I tried to create a sortable tree with `react-beautiful-dnd` but I don't know to finish it.
[codeSandBox] (https://codesandbox.io/s/dnd-zjdjd?file=/src/App.js)
Rotate on a periodic basis using Terraform
Tags: terraform
Published: 2020-10-04 15:15:26

<sub><span>Photo by <a href="https://unsplash.com/@jeremyperkins">Jeremy Perkins</a> on <a href="https://unsplash.com/s/photos/circle?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></span></sub>

One key component of modern security is rotating secret.

Hashicorp has a good product that can generate secrets based on a master one : Vault.

I though it was very difficult to achieve this through Terraform.
But Terraform has a provider that can provide a change at the defined frequency basis : [time_rotating](https://registry.terraform.io/providers/hashicorp/time/latest/docs/resources/rotating).

Example:

```Terraform

terraform {
  required_providers {
    time = {
      source = "hashicorp/time"
      version = "0.5.0"
    }
  }
}

resource "time_rotating" "example" {
  rotation_days = 30
}

resource "random_id" "server" {
  keepers = {
    # Generate a new password each time time rotates
    rotation = time_rotating.example
  }

  byte_length = 8
}

```


Hope this helps !
Spam posts in Dev
Tags: spam, discuss
Published: 2020-10-04 15:11:36

Suddenly, I see a lot of spam posts here with services promoting their customer care numbers like this one

https://dev.to/roygorav56/mayra-studio-customer-care-number-7318720711-7384823438-4198

Just because you want SEO ranks, you can post anything. What the heck? 

You are not adding anything meaningful to this community with your customer care numbers

Cant we do something about this?
Check If It Is a Straight Line : LeetCode
Tags: java
Published: 2020-10-04 15:04:31

Problem Link: https://leetcode.com/problems/check-if-it-is-a-straight-line/

class Solution {

    public boolean checkStraightLine(int[][] a) {

        double m = ((((a[1][1]-a[0][1]) * 100.0) / (a[1][0]-a[0][0]))*100.0);
        for(int i=2;i<a.length;i++){
            if((a[i][0]-a[i-1][0])==0){
               if(m!=Math.abs((((a[i][1]-a[i-1][1]) * 100.0) / (a[i][0]-a[i-1][0]))*100.0))
               return false; 
            }else{
                if(m!=((((a[i][1]-a[i-1][1]) * 100.0) / (a[i][0]-a[i-1][0]))*100.0))
               return false;
            }
            
        }
        
               return true;
        
    }
}
YAHP (Yet another Hacktoberfest post)
Tags: hacktoberfest
Published: 2020-10-04 13:44:01

I got my 4 PRs approved as of this morning. 

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/9kak2bihall6zbpbk5xn.png)

It's the first time I participate in Hacktoberfest because I was scared to death about it. Every single time I knew it was coming I thought that in order to participate I'd have to fix a bug on a really advanced repository, such as git, vue, javascript or something like that.

But I was really wrong! There's tons and tons of repositories and there's infinite ways of contributing wether if you are an experienced developer or not.

So give it a go! I already did, it's your turn now

If you want to participate and yet you don't know where to begin, head over to [Hacktoberfest](https://hacktoberfest.digitalocean.com/) and sign up. You still have a month to go
Microsoft Azure hack for social justice
Tags: azure, hackathon, beginners
Published: 2020-10-09 18:17:15

---
title: Microsoft Azure hack for social justice
published: true
date: 2020-10-04 15:00:33 UTC
tags: Azure, hackathon, beginners
canonical_url: https://am2.co/2020/10/microsoft-azure-hack-for-social-justice/?utm_source=rss&utm_medium=rss&utm_campaign=microsoft-azure-hack-for-social-justice
---

[![](http://am2.co/wp-content/uploads/cody-pulliam-GGKVGSkKIzM-unsplash.jpg)](https://socialjusticehack.devpost.com/)<figcaption id="caption-attachment-4895"><a href="https://unsplash.com/photos/GGKVGSkKIzM">Photo via unsplash</a></figcaption>

Microsoft is going to be running a virtual hackathon, running from October 12 through November 20. The theme of the hack will be to use Microsoft Azure to address some social justice issue. I’m thrilled to be joining [the judging panel](https://socialjusticehack.devpost.com/#judges) for this event, which is at the intersection of two of my greatest passions–social justice and technology.

> Microsoft invites you to use your creativity, passion, and cloud expertise to create solutions that empower communities and help address a timely and relevant social justice issue.

If you’re excited to use technology to help advance social justice, [check out the event page](https://socialjusticehack.devpost.com/), and find some friends to create your hack team. There are even [swag & cash prizes](https://socialjusticehack.devpost.com/#prizes)!

Hope to see you there!

The post [Microsoft Azure hack for social justice](https://am2.co/2020/10/microsoft-azure-hack-for-social-justice/) appeared first on [Andy M Mallon - AM²](https://am2.co).
Changed my Linux "stack" after 10 years
Tags: linux
Published: 2020-10-04 14:58:59

After 10 years and a simple post on dev.to I completely changed my Linux distro and tools.

I wonder if you did something like this and what was your change? I want to hear long time Linux users, not a fresh ones. I'm interested about a change, not tools themselves.

My change was:
- Mint/Cinnamon to Arch/Xmonad (after 10 years)
- WebStorm to Vim (after 2 years, was Sublime before)
- MC to Vifm (after probably 30 years and Norton Commander in DOS)

... Or in general: Mouse to keyboard :)
Notes on sorting [Double]
Tags: haskell
Published: 2020-10-04 14:58:47

**If NaN is included in the list, it's not sorted as expected.**

```hs
> import Data.List
> let nan = 0/0 :: Double
> sort [3,2,nan,1,5,nan,4]
[1.0,4.0,NaN,5.0,NaN,2.0,3.0]
```

This behavior seems to be due to the strange behavior when comparing nan to itself or to a number.

```hs
> nan < nan
False
> nan == nan
False
> nan > nan
False
> compare nan nan
GT
> nan < 0
False
> nan == 0
False
> nan > 0
False
> compare nan 0
GT
```

Wat a horror.

When sorting a list that is likely to contain NaN, it is better to use `isNaN`

```
isNaN :: RealFloat a => a -> Bool
```

to remove the NaN or to sort it after proper processing.

```hs
> sort . filter (not . isNaN) $ [3,2,nan,1,5,nan,4]
[1.0,2.0,3.0,4.0,5.0]

> sort . map (\x -> if isNaN x then -1 else x) $ [3,2,nan,1,5,nan,4]
[-1.0,-1.0,1.0,2.0,3.0,4.0,5.0]
```
REST Client for your early Rest API-based project using Visual Studio Code
Tags: vscode, rest, http, api
Published: 2020-10-04 14:34:17

Imagine if your API project still in early phase development and you only code one endpoint at the moment, using Postman must be 'overkill' for the project. If you're using Visual Studio Code, you must be lucky because there is an extension that can do HTTP requests based on a file configuration named [REST Client](https://marketplace.visualstudio.com/items?itemName=humao.rest-client). With this extension, you can test your API project without having more HTTP client programs open in your machine, just only your beloved Visual Studio Code editor. So, let's install it in the VSCode:

<img src="https://i.ibb.co/sH1m0kM/rest-client-vscode.png" alt="rest-client-vscode" border="0">

## Basic GET Requests

We will be using a free public API for the demonstration, let's start by creating a file named `swapi.http` and let's fill the file with these requests:

```
### Resources List
GET https://swapi.dev/api HTTP/1.1
content-type: application/json

### People List
GET https://swapi.dev/api/people HTTP/1.1
content-type: application/json

### Planet List
GET https://swapi.dev/api/planets HTTP/1.1
content-type: application/json

### Film List
GET https://swapi.dev/api/films HTTP/1.1
content-type: application/json

### Vehicle List
GET https://swapi.dev/api/vehicles HTTP/1.1
content-type: application/json

### Starship List
GET https://swapi.dev/api/starships HTTP/1.1
content-type: application/json
```
The `Send Request` button above each URLs should be visible when you already installed the extension, feel free click any of it to get the result like below:

<img src="https://i.ibb.co/6Yg37JS/rest-client-star-wars-api.png" alt="rest-client-star-wars-api" border="0">

## Basic POST requests

Because the star wars API doesn't provide any POST request endpoint, let's try another public API named `reqres.in` and create another file named `reqres.http` and fill it with this one example POST request:

```
### Create An User
POST https://reqres.in/api/users HTTP/1.1
content-type: application/json

{
    "name": "morpheus",
    "job": "leader"
}
```

Send the request and you have demonstrated the POST request :)

<img src="https://i.ibb.co/3FzJ6mP/rest-client-reqres.png" alt="rest-client-reqres" border="0">

I will cover more of the extension features later to make a request collection that more manageable and DRY, have fun to explore it first by yourself!
Media Queries Code Snippet
Tags: css, codenewbie, beginners
Published: 2020-10-04 14:19:26

*MQs - Media Queries*

Media Queries are the bee's knees when coding [responsive websites](https://www.w3schools.com/html/html_responsive.asp). As you may know, determining the best breakpoint for your site isn't an exact science but there are a few MQs I use for every site that I make. When setting up scss/css files I always set these MQs at the bottom of the file in anticipation that I'll be using them as the project moves along:

```
// MD devices (tablets, 768px and up)
@media (min-width: 768px) { }

// LG devices (desktops, 1024px and up)
@media (min-width: 1024px) { }

// XL devices (large desktops, 1200px and up)
@media (min-width: 1200px) { }

// XXL devices (large desktops, 1500px and up)
@media (min-width: 1500px) { }
```

A few things to note here: 
* I always write code beginning at the mobile size and then gradually moving to XXL screen sizes. This is why you won't see any (max-width: ####) on this code snippet. 

* As I mentioned earlier determining the breakpoint for your website is not an exact science. Do not be beholden to this snippet. Adjust as needed. Never be afraid to experiment to get the results that you need. 

For more on media queries check out this article: [Media Query CSS Tutorial](https://www.freecodecamp.org/news/css-media-queries-breakpoints-media-types-standard-resolutions-and-more/) 

I hope this helped :)