teleren/devto huggingface dataset:
如何获取Google Serp页面
Published: 2021-05-23 07:02:16

在前面内容中，我们重点在解析html页面，获取想要的信息。那么这个页面如何去获取，第一种方法是引入`requests`包，例如下面这样。

```python
def get_source(url: str) -> BeautifulSoup:
    """获取网页内容
    Args:
        url (str): 要爬取的URL

    Returns:
        BeautifulSoup: The soup of the page source
        """
    content = requests.get(url)
    return BeautifulSoup(content.text, 'html.parser')
```

按照这样的方法，我们获取到的html源码并不完整，缺少右侧的Wiki内容，如图。

![image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hefo3smw32180k3oh2jk.png)

原因是因为很多网页需要js代码执行来渲染内容，也就是说我们需要等待页面执行完js代码，才能够爬取到相关的内容。

## 使用pyppeteer获取源码

这里我们采用pyppeteer的方案来解决，代码如下：

```python
async def get_source( url: str):
    browser = await launch({'headless': True})
    page = await browser.newPage()
    await page.goto(url)
    try:
        # 等待知识页面加载，超时时间设置为3s
        await page.waitForSelector('.kp-wholepage', timeout=3000)
    except TimeoutError:
        print('Wait for knowledge failed')
    
    # 获取网页内容
    content = await page.content()
        
    
    await browser.close()

    return BeautifulSoup(content, 'html.parser')
```

## 插件机制

这是两种获取源码手段.
Best IDEs
Tags: coding, watercooler, tools
Published: 2021-05-23 06:28:19

<div class="modal-body">
<p>There are tools to every trade and writing code is no different. I’ve posted in the past about the best code diff tools, but that’s only a small part of the equation. Arguably the biggest part of the job is which IDE are you using. IDE stands for Integrated Development Environment and it handles a lot more than you would think. Apart from the basics of file management, a good IDE should prioritize helping you write good code fast. Any basic text editor can help you write text, but an IDE is more about the bundle of tools that accompanies it.</p>
<h3><strong>Old School:&nbsp;VIM</strong></h3>
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*WImHOy2Op3LJ1qt1"></figure><p>What is beloved and also hated by the same amount of people? <a href="https://www.vim.org/">Vim</a>. Known for it’s complicated command interface to know when typing versus running commands, Vim provides extremely powerful text editing capabilities for those who know how to use&nbsp;it.</p>
<h3><strong>GOAT: Visual&nbsp;Studio</strong></h3>
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*cwuiPWS8DUVRftNU"></figure><p>This is the elephant in the room. <a href="https://visualstudio.microsoft.com/">Visual Studio</a> has dominated so much the code editing space for so long that it’s hard to ignore. And with the introduction of Visual Studio Code, the offering are stronger than ever now. It’s hard to even put all of the offers that visual studio has into words, but you miss the feature set when you go to other editors. In particular, I think that visual studio has the best debugger of any IDE out there. It’s just super intuitive and allows you to fly around the stack trace super&nbsp;easily.</p>
<h3><strong>Underdog: Atom</strong></h3>
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*YmqugWvcQRFXyRh5"></figure><p><a href="https://atom.io/">Atom</a> is a hackable text editor for the 21st century, built on <a href="https://github.com/electron/electron">Electron</a>. From the GitHub team (which is owned by Microsoft now right?) comes an extremely versituatly and lightweight IDE. Before Visual studio code came out, Atom was my go to option for a lightweight editor that was also cross platform. It allows extensions that allow you to customize almost everything about your experience. It also has an extremely deep community around it so it’s known for being reliable and friendly. If you’ve never used it before, I highly recommend checking it&nbsp;out!</p>
<p><strong>All in all, which is your favorite IDE? Are there any IDEs that you think should be on this&nbsp;list?</strong></p>
<img src="https://medium.com/_/stat?event=post.clientViewed&amp;referrerSource=full_rss&amp;postId=92d925d5c443" width="1" height="1" alt="">
</div>
Automapper in .NET Core
Tags: automapper, dotnetcore, csharp, dotnet
Published: 2021-04-24 06:51:11

---
title: Automapper in .NET Core
published: true
description: Using Automapper in .NET Core
tags: automapper, dotnetcore, csharp , dotnet
//cover_image: https://direct_url_to_image.jpg
---
GitHub link for the project:
{% github https://github.com/aman-singh-parihar/Automapper no-readme %}

We will create a project in .NET Core using dotnet cli and will use visual studio code as the IDE.

Go to any directory where you want to create this project. 

We are using powershell to setup this project.
> `PS E:/VSCODE/NETProjects>mkdir Automapper`
> `PS E:/VSCODE/NETProjects>cd Automapper`

Lets create a solution in Automapper folder.
>`PS E:/VSCODE/NETProjects>Automapper>dotnet new sln`

Lets create a project in the Automapper folder.
>`PS E:/VSCODE/NETProjects>Automapper>dotnet new webapi -o Automapper.API`

Lets add the project in the solution file because as of now they are disconnected to each other.
>`dotnet sln add Automapper.API/Automapper.API.csproj`

Now lets open Automapper folder in vscode.
>`PS E:/VSCODE/NETProjects>Automapper>code .`

Switch to project folder and install automapper.
>`PS E:/VSCODE/NETProjects>Automapper>cd Automapper.API`

>`PS E:/VSCODE/NETProjects>Automapper>Automapper.API>dotnet add package AutoMapper.Extensions.Microsoft.DependencyInjection`

We setup our project, now lets write some code. First of all delete the weather api controller and weather api class as we don't need this.

Lets create two folders Models and DTO.
Models will have classes that will exactly match with the database tables and DTO will have classes which need to pass to the client.

In our case we are using Employee class in Models folder and EmployeeDTO in DTO folders.

##Employee.cs
{% gist https://gist.github.com/aman-singh-parihar/014dc55ff53e3fbae39594fabc3f32be %}

##EmployeeDTO.cs
{% gist https://gist.github.com/aman-singh-parihar/09484b534bd00770e888d31d7503c87f %}

You can see that we are not passing employee salary to the client.

Now we will create profile which will map from Employee to EmployeeDTO.

Lets create a folder AutoMapperProfile in the project and add a class AutomapperProfile.cs.

##AutomapperProfile.cs
{%gist https://gist.github.com/aman-singh-parihar/c48716748e799221b9740cb53d343edd %}

Lets setup the ConfigureServices method of startup class for automapper.

##Startup.cs
{% gist https://gist.github.com/aman-singh-parihar/29e774a4e2dc7b9752492675e2a92a34 %}

Now lets create a EmployeeController in Controllers folder.

##EmployeeController.cs
{% gist https://gist.github.com/aman-singh-parihar/83977c7f9bfdeab2a3483a0bfe42b855 %}

We are almost done with this, lets run this project and see the output.
>```PS E:/VSCODE/NETProjects>Automapper>Automapper.API>dotnet run```

Type this url in the browser to view the output: https://localhost:5001/api/Employee

{% gist https://gist.github.com/aman-singh-parihar/e2555802fad9da984ab74b2ad2edfc9f %}
Laravel Custom Getter Tutorial Example

Tags: laravel, programming, php, tutorial
Published: 2021-05-23 06:27:03

Hi Artisan,

In this blog, i will learn you how to laravel custom getter tutorial in your laravel project.The laravel custom getter tutorial is so easy to use.so you can just follow my step by step and learn laravel custom getter tutorial.

In this article we have three subject now we are getting three subject total marks how this possible now we can use laravel Getter function let's implement and here we go jump to full example follow my bellow step

In this example now define getter attribute getStudentTotalMarkAttribute public function in Product model and sum of total marks of student profile.

if you want to see full example follow this link..
https://codingtracker.blogspot.com/2021/05/laravel-custom-getter-tutorial-example.html
Working with Multiple Elements in Cypress
Tags: javascript, testing, webdev, tutorial
Published: 2021-05-23 06:13:18

In this Cypress tutorial, we will cover how you can work with multiple elements. We will verify the length of the list as well as assert the text of all the list items.

### Test Scenario
In this test scenario, we will try to get the length of the menu link items below as well as get the text of each of the list items.

![menu-links](https://automationbro.com/wp-content/uploads/2021/05/Screen-Shot-2021-05-14-at-3.17.56-AM.png)

---

### .each()

Since we will be working with the list items, we will need to iterate through the list. Cypress provides a handy `.each()` function to iterate through Array-like structures. Let’s see how we can use it –

![.each](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gwnr5mh90mxivo3hs4wz.png)

---

### Asserting the list text

Once we are able to iterate through the list using the .each() command, we can get access to each individual list item. We can then use the .text() command to access the text of the list items.

![assert](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8yxspt0twmj1isu1gsi9.png)

In the above code, we are asserting the text using 2 ways –

- Via the `.text()` command – this way is useful if you would like to modify the text in some way before asserting it
- Via the `.wrap()` and `.should()` command – this is the natural way to assert the text in Cypress
Depending on your own use case you can pick either way to assert the text.

---

#### Check out the video below to see learn how to work with multiple elements in Cypress –

{% youtube eMLVTX7AyNk %}

---

📧 Subscribe to my [mailing list](https://bit.ly/sdet-u-mailing-list) to get access to more content like this 

👍 Follow [SDET Unicorns](https://bit.ly/sdet-u-twitter) on Twitter for the latest updates

...

I love coffees! And, if this post helped you out and you would like to support my work, you can do that by clicking on the button below and buying me a cup of coffee -

<a href="https://www.buymeacoffee.com/dilpreetjohal">![Buy me a coffee](https://dev-to-uploads.s3.amazonaws.com/i/4aw2ub3f4qkyjk0ivwt4.png)</a>

You can also support me by liking and sharing this content.

---

#### Enjoyed this read? Discover more insightful articles on software testing and development on my [blog](https://sdetunicorns.com/blog). 🌐
Day 70 of 100 days of SwiftUI
Tags: swift, 100daysofcode
Published: 2021-05-23 06:10:48

I just completed day 70 of 100 days of SwiftUI. Today i learnt some advanced features of the MapKit by adding annotation pins to maps. I also learnt how to customizing  the annotation pins by showing a alert that changes the title of the annotation pins.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/as4s166qp7yrkei4gl4y.png)
Generating QR Codes in Google Sheets
Tags: googlecloud, watercooler
Published: 2021-05-23 05:39:53

Being a developer comes down to using the right tool for the right job. And sometimes, it can be a Google sheet or Excel sheet that can automate a simple task for you.

Have you ever needed to generate QR codes from a whole list of URL's for instance?

Well, Google has your back on that.
Today we'll be using Google Sheets and Google Charts to generate QR Codes!

![Generating QR Codes in Google Sheets](https://cdn.hashnode.com/res/hashnode/image/upload/v1621491034345/0q398HEE4.gif)

## Generating a QR code with Google Charts

Google Charts is an amazing tool that can generate several types of charts for us.
One of the charts we can generate is a QR code.

The benefit of this is that we get a dynamic endpoint where we can change the encoded QR element to be something variable.

[Read more on the Google Chart QR Code](https://developers.google.com/chart/infographics/docs/qr_codes)

If you are following along with my article, this is the end URL we will be using.

```text
https://chart.googleapis.com/chart?chs=200x200&cht=qr&chl=https://daily-dev-tips.com
```

Where the variables we use are:

- `chs`: Chart size, we defined a 200x200 square
- `cht`: Chart type, in our case QR
- `chl`: This is the data we want to encode

If we use that URL as an image, we get the following result.

![Daily dev tips QR code](https://chart.googleapis.com/chart?chs=200x200&cht=qr&chl=https://daily-dev-tips.com)

## Google Sheets dynamic QR codes

Now that we know about creating QR codes let's put this into action as we generate them dynamically in Google Sheets.

Open up a new Google Sheets and fill out column A with random websites.

Then in column B, we will add the following formula.

```text
=IMAGE("https://chart.googleapis.com/chart?chs=200x200&cht=qr&chl="&A1&"")
```

Then we can drag this formula down to the other cells!

Make the rows a bit bigger so you can see the image correctly.
And that's it we now have a dynamic QR generator in Google Sheets.

You can view the [Google QR Sheet here](https://docs.google.com/spreadsheets/d/1qcpM5jyOC2bb8Yg3V3JfvnSefo9yzmb_B1UNUgu3Hyc/edit?usp=sharing).

### Thank you for reading, and let's connect!

Thank you for reading my blog. Feel free to subscribe to my email newsletter and connect on [Facebook](https://www.facebook.com/DailyDevTipsBlog) or [Twitter](https://twitter.com/DailyDevTips1)
Responsive menu with only HTML and CSS (part 2)
Tags: css, codenewbie, beginners, todayilearned
Published: 2021-05-23 05:27:58

In this project, you will learn how to make a responsive menu with only HTML and CSS, But keep in mind this is not a burger menu. The main feature of the menu is, it will look in a row at the desktop device and when we open this project on small screen device like smartphone or mobile. Then it will rearrange menu links in columns.

I have used a media query to make this menu responsive 

![media query](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2evwh2riql84bvpoaidg.png)

But you can make this menu with flexbox and grid too.

---

[![CSS course eduonix](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/az71k2lp2eeyll27izng.png)](https://www.eduonix.com/css-bootcamp-master-in-css-including-css-grid-css-flexbox/UHJvZHVjdC0xMzE1MDIw)

---

So this is the final result of our responsive menu. I recommend you to play with this codepen project. It will help you to clear your thoughts.

{% codepen https://codepen.io/atulcodex/pen/ExWWgmY %}

You can comment below if you are facing any issue with the project, I would love to help you.

{% link https://dev.to/atulcodex/responsive-navbar-with-only-html-and-css-269 %}
How to setup Cmder in VSCode in 2021
Published: 2021-05-23 03:28:50

```json 
"terminal.integrated.profiles.windows": {
    "Cmder": {
      "path": "${env:windir}\\System32\\cmd.exe",
      "args": ["/k", "C:\\cmder\\vendor\\bin\\vscode_init.cmd"]
    }
  },
  "terminal.integrated.defaultProfile.windows": "Cmder",
```

Step 1: Download Cmder
Step 2: Save Cmder to C:\ drive
Step 3: Open Settings.json in VSCode
Step 4: Create VSCode Integrated Terminal Settings

```json 
"terminal.integrated.profiles.windows": {
    "Cmder": {
      "path": "${env:windir}\\System32\\cmd.exe",
      "args": ["/k", "C:\\cmder\\vendor\\bin\\vscode_init.cmd"]
    }
  },
```
This will initialize the default command prompt with the argument of vscode_init.cmd

**OPTIONAL:** Set Cmder as default integrated terminal
```json
  "terminal.integrated.defaultProfile.windows": "Cmder",
```
## Common Use Case: Multiple Integrated Terminals
For using multiple integrated terminals Powershell, Git Bash, or the regular command prompt
```json 
"terminal.integrated.profiles.windows": {
    "PowerShell": { "source": "PowerShell", "icon": "terminal-powershell" },
    "CommandPrompt": {
      "path": [
        "${env:windir}\\Sysnative\\cmd.exe",
        "${env:windir}\\System32\\cmd.exe"
      ],
      "icon": "terminal-cmd"
    },
    "GitBash": {
      "path": ["C:\\Program Files\\Git\\bin\\bash.exe"],
      "source": "Git Bash",
      "icon": "terminal-bash"
    },
    "Cmder": {
      "path": "${env:windir}\\System32\\cmd.exe",
      "args": ["/k", "C:\\cmder\\vendor\\bin\\vscode_init.cmd"]
    }
  },
// Choose Default Terminal
// "terminal.integrated.defaultProfile.windows": "Cmder"
// "terminal.integrated.defaultProfile.windows": "GitBash"
// "terminal.integrated.defaultProfile.windows": "CommandPrompt"
// "terminal.integrated.defaultProfile.windows": "PowerShell",
```

// End

## To Learn More:
- [How to set Git Bash as integrated terminal in VSCode in 2021](https://dev.to/andrewriveradev/how-to-set-git-bash-as-integrated-terminal-in-vscode-2k31)
5/22 TIL: Processes, Contrapositive, etc.
Published: 2021-05-23 02:55:21

<strong>--Systems--</strong>

<strong>Processes</strong>

- A process is an instance of an executing program; each process has a unique process id & its parent process id
- Virtual memory of a process is logically divided into:
    - Text initialized & uninitialized
    - Data
    - Stack - a series of frames with a new frame being added as a function is invoked & removed when the function returns; each frame contains local variables, function arguments, and called linkage information for a single function implication
    - Heap
- Command line arguments supplied when a program is invoked are made available via argc and argv arguments to the main function

```

#include<unistd.h>

int proc_id = getpid();

int par_proc_id = getppid();

```

<strong>Memory Allocation</strong>

```

malloc() // memory allocation

calloc() // contiguous allocation

free() // de-allocate

realloc() // re-allocate

// example

int* ptr

int num=8;

ptr = (int*)malloc(num*sizeof(int))

```

<strong>System Limits</strong>

- SUSV3 (Single Unix Specification V3) adds a lot of new functions into C/C++ library
- SUSV3 specifies limits that an implementations may enforce and system options that an implementation may support

```

#include <limits.h>

LONG_MIN, LONG_MAX, CHAR_MIN, CHAR_MAX // etc

```


<strong>--Discrete Math--</strong>

if p then q (p → q)
![image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5c23ic9t5v6lf45xdlvj.png)
- p → q ≡ ~p V q

~(p → q) ≡ ~(~p V q) ≡ (~~p ʌ ~q) ≡ p ʌ ~q

- apply DeMorgan's Law
- the only scenario where p → q is false is when p is T and q is F. Negating that p → q would be F. So this is equivalent to p ʌ ~q where p is T and q is F.

<strong>Contrapositive of a conditional</strong>

p → q ≡ ~q → ~p because

- p → q ≡ **~p V q** and
- ~q → ~p ≡ **q V ~p**
- with **~p V q** ≡ **q V ~p** being true

Converse: p → q is the statement q → p (not logically equivalent)

Inverse: p → q is the statement ~p → ~q (contrapositive of a converse)

- conclusion: converse ≡ inverse since contrapositive statements are logically equivalent

<strong>Biconditional p ↔  q</strong>

- a conjunctive statement where we have both p → q and q → p implications
- "if and only if"
Mobile Table of Contents Component for React | What are the best component libraries?
Tags: react, javascript, css, webdev
Published: 2021-05-23 02:14:33

Hey hey ladies and gents, and whomever else might be reading this piece.

I’m new to React and love the component-based builds, but I frequently get into situations where I’m building components from scratch when I just *know* that someone else has probably built something similar.

At the moment, I’m looking at the table of contents button on the React website - it’s fantastic. Now, I could build this myself, or get it from open source libraries that more talented developers have already made available.

But since I’m new, I don’t know where the best libraries are or if they exist.

I’d love to get some advice from the community!

Can anyone point me in the right direction?

Thanks 🙏,
Sah
Search Engine Ranking
Tags: seo, seotips, webtool
Published: 2021-05-23 01:58:27

The position of a page on the search engine results page (SERP) is what is referred to as search engine ranking. There is a relationship [between](https://money.usnews.com/investing/investing-101/articles/etf-vs-index-fund-the-difference-and-which-to-use) traffic to a site and its ranking on the SERP. Many factors influence a page’s position on the SERP based on the relevant content search phrase or the quality of backlinks from trusted sites pointing to the website. 

## Good Rankings Increases Traffic

The main benefit of good rankings in SERP is that the page receives more visitors from the organic search channels. A user will most likely click on the result if it ranks high on the search query results. It explains the relationship between high SERP rankings and the increased traffic to the site; it is more prominent in the top three search results.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pig2xrtp0dthakh94x7s.jpg)

## Different Rankings for Different Search Engines 

The keyword or website ranking for URLs or the keyword combination varies from one search engine to another. A domain may rank highly for a specific keyword in one search engine but not feature at the top of the results page on another search engine. Search engine optimization is not an exact process. Still, there are many ways to carry out a website check using an [**all-in-one website analyzer**](https://sitechecker.pro/) to improve a website’s visibility and climb higher in search rankings. Every search engine applies its methods to calculate rankings and, therefore, rank pages differently. 

## Factors that Influence Rankings

Search engine rankings are affected by search engine optimization because search engines use algorithms to calculate keyword search results. Search engines use their unique factors to rank and weight pages while calculating the rankings; these factors are guarded intellectual property and are not shared with anyone. It is presumed that search engines consider the length of time a user spends on a page before returning to the SERP, keywords in metadata descriptions, and backlinks to trusted sites. Also, the structure of URLs, loading speed of pages, traffic to a site, number of times a result is clicked, the content on the page, and how up-to-date the page are used to calculate rankings. In the end, rankings prove if the company’s SEO strategy is working or not; therefore, a brand should regularly revise its [SEO strategy](https://blog.hubspot.com/marketing/seo-strategy) to ensure that the page remains at the top of the SERP.

### Conclusion

The work and attention determine a website’s success. A page ranking constantly and quickly changes in time, especially when dealing with highly competitive keywords. There are unlimited keywords that a brand can select from for their SEO campaign, but not all words are appropriate. Gains in ranking mean that the website performs well, while a loss shows that the brand should review its SEO strategy.