teleren/devto huggingface dataset:
How to build a web crawler as a beginner? 
Tags: beginnerwebscraping
Published: 2019-05-07 09:25:21

---
title: How to build a web crawler as a beginner? 
published: true
description: 
tags: beginner; web scraping
---



Using Computer Language (Example: Python)

For any non-coders who wish to build a web scraper using a computer language, Python might be the easiest one to start with comparing to PHP, Java, C/C++. Python's grammars are rather simple and readable for anyone that reads English.

Here is a simple example of a web crawler writing with Python.

--------------------------------------------------------------------


 import Queue

       initial_page = "http://www.renminribao.com"

            url_queue = Queue.Queue()

            seen = set()

            seen.insert(initial_page)

            url_queue.put(initial_page)

  while(True):

       if url_queue.size()>0:

            current_url = url_queue.get()

            store(current_url)

            for next_url in extract_urls(current_url):

                  if next_url not in seen:

                       seen.put(next_url)

                       url_queue.put(next_url)

       else:

              break

---------------------------------------------------------------------------
 

As beginners without knowing how to program, we are absolutely required to spend time and energy in learning Python and then writing a web crawler ourselves. The whole studying process might last several months.

 

 

Using Web Scraping Tool (Example: Octoparse)

When a beginner wants to build a web crawler within a reasonable time, a visual web scraping software like Octoparse is a good option to consider. It is a coding-free web scraping tool that comes with a free version. In comparison with other web scraping tools, Octoparse can be a cost-efficient solution for anyone looking to quickly scrape some data off a website.[Top 5 Web Scraping Tools Comparison](https://www.octoparse.com/blog/top-5-web-scraping-tools-comparison).

 

How to“Build a web crawler” in Octoparse.

1.[Wizard Mode for easy scraping](https://www.octoparse.com/tutorial-7/wizard-mode)

Wizard Mode which will guide users step by step in scraping data in Octoparse provides three pre-built templates – “List or Table”, “List and Detail” and “Single Page”. Providing the pre-built templates were able to satisfy our need, we can easily to build a “web crawler” in Octoparse within clicks after downloading Octoparse.

 

 

2.[Advanced Mode for complex web scraping](https://www.octoparse.com/tutorial-7/advanced-mode)

Since some websites are built with complex structures, Wizard Mode cannot help us scrape all the data we want. Thus, we’d better use Advanced Mode which is more powerful and flexible in scraping data.

Here is an example that how to build a web crawler by using Octoparse.[VEDIO: Scrape product information from Amazon (Octoparse 7.X)](https://www.youtube.com/watch?v=vEGFe6shbac)
Fresh Web Design and UI Trends for 2019
Tags: webdesigntrends2019
Published: 2019-05-07 08:34:38

---
title: Fresh Web Design and UI Trends for 2019
published: true
description: Constant innovation has become requisite in achieving an edge over other competitors’ website.
tags: web design trends 2019
---
![web design trends 2019](https://thepracticaldev.s3.amazonaws.com/i/wbruv83htpbuk4rpyxus.jpg)
For years, fresh UI trends and web design are an integral part of the alteration. To stay ahead, websites strictly need to follow the updated trends recognizing the improvement.

Peculiar past elements, that gained popularity in the last ending year will undoubtedly secure the spot in next. From color selection, voice to virtual reality or VR interfaces - these core elements must be carried forward for better optimization.

Secured UI trends and web design, either from past analysis or present, are hard to break down, but, we are presenting you the top-spotted web design and UI trends for 2019. Either with the help of the web design company or other helping hands, these web design trends 2019 must be carried forward to be updated and stay ahead. Have a look below to check out our complete listings.

#webdesign #webdevelopment

Read the full post here: https://www.cswebsolutions.ca/blog/best-web-design-and-ui-trends-for-2019
My top 5 playlists for coding
Tags: music, coding, concentration
Published: 2019-05-07 08:24:51

---
title: My top 5 playlists for coding
published: true
description: Spotify playlists for long coding sessions
tags: music, coding, concentration
---

Following [Ben Halpern's post](https://dev.to/ben/the-best-deep-work-soundtrack-437k) about the best deep work soundtrack, I'm going to share my 5 most listened playlists for deep work, no special preference.

Here we go! (For some reason I can't embed Spotify URIs sorry)

|Title        | Description        |
|-------------|--------------------|
|[Moodboard](https://open.spotify.com/user/rfpp/playlist/3a3EWf2nNagQSQLvILkMCN?si=fWSTKZazQqOHUWwrfc9H3Q) | Deep atmospheric sounds, good for relaxing. |
| [Coding Session](https://open.spotify.com/user/rfpp/playlist/1fUYR6bummmWIFymwXB8Vl?si=T0ng-CMjQSWkVrvDeMJqkw) | When I need something more "Electric". |
| [Coffee break table Jazz](https://open.spotify.com/user/spotify/playlist/37i9dQZF1DWVqfgj8NZEp1?si=aN3U-XrtSOmSRAN76YX9VA) | Another great playlist, I like Jazz when I'm doing creative work. For some reason Jazz helps me with my crappy ideas. |
| [Coding / Hacking / Feeling like a 80's scifi superhero](https://open.spotify.com/user/dewanderer/playlist/601WLbJ3Vj91XIugGUJNUe?si=vgTaQ3MiRtSbRgT2NADWfA) | The title says it all. |
| [Paceful Piano](https://open.spotify.com/user/spotify/playlist/37i9dQZF1DX4sWSpwq3LiO?si=CSPNPmlYQfqGM2uGMp_Z8A) | Another favourite when I need to do creative work. |

And this is it, if you feel in the mood, share your preferences in comments!
React app with Apollo APQ
Tags: help
Published: 2019-05-07 08:03:05

---
title: React app with Apollo APQ
published: true
tags: help
---

const GET_NODES = {
operationName: 'nodeById',
variables: null,
extensions: {
persistedQuery: {
version: 1,
sha256Hash: 'hashOfQuery'
}
}
};
I want to export this but it is a object. What needs to be done for converting it into string.
React APP with Apollo APQ 
Tags: react
Published: 2019-05-07 08:02:13

---
title: React APP with Apollo APQ 
published: true
description: I am new to React and trying APollo APQ but i stuck with it
tags: react
---

const GET_NODES = {
	operationName: 'nodeById',
	variables: null,
	extensions: {
		persistedQuery: {
			version: 1,
			sha256Hash: 'hashOfQuery'
		}
	}
};


I want to export this but it is a object. What needs to be done for converting it into string.
英国留学如果挂科，怎么办?
Tags: 英国留学如果挂科
Published: 2019-05-07 06:19:27

---
title: 英国留学如果挂科，怎么办?
published: true
description: 英国留学如果挂科，怎么办?咨询联系q1252839746。 这可能是大部分同学会遇到的情况，特别是英国读研的同学，学习压力大，如果还挂科的话，其实可能会影响整个人的一生，那遇到这种情况应该怎么办呢? 
tags: 英国留学如果挂科
---

 
英国留学如果挂科，怎么办?咨询联系q1252839746。 这可能是大部分同学会遇到的情况，特别是英国读研的同学，学习压力大，如果还挂科的话，其实可能会影响整个人的一生，那遇到这种情况应该怎么办呢?

所获学分决定最后的证书

英国硕士一般情况下只需要读一年，所以其实英国硕士不存在劝退说法，一般也不存在重修的说法。

学生在学年末获得自己的成绩，一般如果挂科一两门有机会申请重考，最后所获得学分决定了最后的证书。

一般英国获得180学分(其中毕业论文占60学分)即授予master degree;120学分授予postgraduate diploma;60学分授予postgraduate certificates。

因此第一个硕士课程若没有成功获得硕士学位，可以申请第二个硕士，重新就读，这就不属于转学范畴，而是重新申请新的课程，当然第一个课程的失败也会影响第二硕士的申请，建议申请相关专业。

有些学校不支持补考或重修的

像爱丁堡大学的一些专业就不能够“补考”毕业论文。

对于研究生，必须均分达到50%，并且满足到达50%的科目加起来的学分在80分以上，毕业论文的分数也不能低于50%。

如果论文挂了，那么不好意思，你就与硕士学位无缘了，因为明文规定“not allowed to re-sit a paper”(不能够“补考”论文)。

但也有特殊情况，遇上比如亲友过世这类的重大变故，造成论文失利，也许可以找学院商议，再次提交论文，但仍旧机会渺茫。

一般Diploma指的是结业证书，Degree才是学位证书，Degree with Distinction (优等生)一般均分要在70%，并且通常考官也并不会真的铁面无私，在2%以内，都有商量余地。比如68%-69.99%就可以拿到distinction，48%-49.99%就可以拿到硕士学位，38%-39.99%也能算你通过，拿到结业证书。

有些学校是可以补考或重修的

有的学校是支持补考和重修的，但是不同学校之间的规章制度不同，所以开学前仔细研究Handbook是非常重要的!

但是如果你没有仔细阅读你的Handbook，又恰巧在期末的时候找不到了，怎么办?你可以在大学的官网，查看相关的补考信息。

一般你登入搜索网站，在搜索栏中输入“Reassessment postgraduate”或者“Resit an exam” + “输入你所在的学校”。就可以查询到你所在学校关于补考的措施。

如果你挂科了，或者预知这次会挂科，你需要向学校提出补考申请，当学校收到你的申请，会联系你所在的院系，确定补考时间，以及采取何种补考形式。

注意核实提交补考的deadline

每个学校的政策都有所不同，理论上部分学校要求修满180个学分，最后修满160，也给发了学位证书。但另一些学校真的需要一本正经的满打满算。

持Tier4签证的学生，如果需要补考或者重修(包括不能在规定的时间内完成学业的)，允许在英国境内申请签证延期，不受Academic Progression规则的限制。

无论签证到期与否，学生都必须在补考前申请签证延期，补考后再申请将不能获得免修资格。如果学生已经参加了补考，然后发现需要更多的滞留时间，则必须回国重新申请签证。

如何避免挂科

笔记是“救命稻草”

自己如果因为语言不通，那就去找同学借，但是笔记一定要有!

不要死记硬背

期末考试通常是2-3小时的大型考试，只有准确地理解题目、严谨地回答才有成功的可能，所以同学们要有对整个学科的清晰条理性和严密的思考能力。

重视答题技巧

当你收集好考试复习的重点内容，甚至是题库后(开放性问答题)，你需要把这些内容都复习一次，然后挑选出几个重点题目重点复习。
QQ1252839746解决挂科GPA低被学校开除退学等问题专业经验秉直严谨的态度应对问题风险。提供完整的技术分析案例和操作方案。
Setup Tailwind@next in Vue CLI 3 project
Tags: vue, tailwindcss, css
Published: 2019-05-07 06:05:05

Setting up Tailwind is really an easier process consist of few simple steps. But, developers who are new to Webpack or common CSS configuration like PostCSS (like me) might feel it difficult to join all the parts. This post will help to set up and run tailwind with basic configuration in a Vue CLI 3 project.

## Create a new Project
Create a new Vue project using Vue CLI 3 using any of your presets.

```bash
vue create my-app
```

## Install Tailwind (@next)

```bash
# Using npm
npm install tailwindcss@next --save-dev

# Using Yarn
yarn add tailwindcss@next --dev
```

## Load all the Tailwind defaults
Load tailwind defaults in a `.css` file. Create a new `css` file (say, `src/assets/css/tailwind.css`) and load the defaults

```css
/* tailwind.css */

@tailwind preflight;
@tailwind components;
@tailwind utilities;
```

Import this `css` file inside `main.js` entry file.

```js
// main.js

// other imports
import '@/assets/css/tailwind.css'
```

## Configure PostCSS
Configur PostCSS to use tailwind styles

```js
// postcss.config.js

module.exports = {
  plugins: [
    require('tailwindcss'),
    require('autoprefixer'),
  ]
}
```

Now restart the vue server and start working with Tailwind 🎉 
Watch this series for more Tailwind and Vue related tips 😉
Sublime 3: Goto Anything... Tip
Tags: sublime, productivity, css, javascript
Published: 2019-05-07 05:18:36

---
title: Sublime 3: Goto Anything... Tip
published: true
description: Exclude directories when using Goto Anything in Sublime
tags: sublime,productivity,css,javascript
---

If you use Sublime 3 you're likely used to using `Goto Anything...` (CMD+P Mac, CTRL+P Windows) to quickly bring up files. One downside is this will bring up vendor and compiled files as well. This really isn't ideal. So to fix this you can exclude these directories. 

Open up your user settings and add this line:

```
"binary_file_patterns": ["node_modules/", "public/", "vendor/", "*.jpg", "*.jpeg", "*.png", "*.gif", "*.ttf", "*.tga", "*.dds", "*.ico", "*.eot", "*.pdf", "*.swf", "*.jar", "*.zip"],
``` 
If you add other folders, just be sure to suffix them with a "/".

Then save the file and you should be good to go.

As always, happy coding! 🤓
The Single CSS File Manifesto
Tags: css, webdev, ux, manifesto
Published: 2019-05-07 04:32:07

---
title: The Single CSS File Manifesto
published: true
description: Brevis is a CSS toolkit engineered for high performance and scalable web applications.
tags: css,webdev,ux,manifesto
---

NO preprocessors
NO javascript
NO npm
NO grunt
NO yarn
NO bower
NO travis
NO cdn

Just a [SINGLE CSS](https://getbrevis.com) file.
Today I Learned: How to Delete Git-Tag Remotely.
Tags: gitlab, todayilearned, github, git
Published: 2022-01-24 09:28:51

---
title: Today I Learned: How to Delete Git-Tag Remotely.
published: true
tags: gitlab,todayilearned,github,git
canonical_url: https://medium.com/easyread/today-i-learned-how-to-delete-git-tag-remotely-caa95dc3a495
---

#### A simple command to save your days :)

**TLDR;**

```
# delete remotely
$ git push --delete origin tagName

# delete tag in your local
$ git tag -d tagName
```

![](https://cdn-images-1.medium.com/max/1024/0*euENSFNA22UQNJ_j)<figcaption>Photo by <a href="https://unsplash.com/@yancymin?utm_source=medium&amp;utm_medium=referral">Yancy Min</a> on <a href="https://unsplash.com?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption>

My daily life is working with Git, in my case Github. Both in my works, and also in my spare time(doing a side project with open-source). Currently, I maintain 2 of my open source projects, they are:

- faker ([https://github.com/bxcodec/faker](https://github.com/bxcodec/faker))
- gotcha([https://github.com/bxcodec/gotcha](https://github.com/bxcodec/gotcha))

There is a time that I made countless mistakes when releasing a new feature of my projects both in my current job and in my side project. Just like just a few minutes ago before I write this story.

It happens when I want to release one of my open-source projects faker. When releasing it, I accidentally hit my enter keyboard, but I’m not finished yet to specify the version. And that causes the Github to create a new tag there and ruin the order of the semantic versioning from the previous version.

So to fix this, since in the Github we can’t change the tag-name, it’s either to delete it and create one, or just let it be like that. And that’s how I learn a thing about this git tag deletion.

With just a simple command, I can delete a tag remotely and locally.

```
# delete remotely
$ git push --delete origin tagName

#delete tag in your local
$ git tag -d tagName
```

But, we also can do this in the Github itself, \*TBH, I just figured this, and I think it’s a new feature in Github that I just realized. Since a year ago, I can’t delete a tag in Github, except using my terminal command.

Here’s how to delete a tag with the Github web interfaces.

![](https://cdn-images-1.medium.com/max/1024/1*1HDqKLhD3VcoQiFOvjNXGA.gif)<figcaption>demo delete Tag with Github</figcaption>

Just it. Hopefully, it can help others :)

<iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2Fc36271%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2Fc36271%2F&amp;image=http%3A%2F%2Fapi.screenshotlayer.com%2Fapi%2Fcapture%3Faccess_key%3Dfe59908dad3baab69ffab249a2224b03%26viewport%3D1024x612%26width%3D1000%26url%3Dhttps%253A%252F%252Fupscri.be%252Fc36271%253Fscreenshot&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/81d478571152ec502750529a22011bf8/href">https://medium.com/media/81d478571152ec502750529a22011bf8/href</a></iframe>
* * *
Analyzing Dev.to Git History with Elasticsearch
Tags: git, elasticsearch, kibana
Published: 2019-05-07 04:15:20

Importing Git History into Elasticsearch is [not very much fun](https://analytics.codingblocks.net/blog/analyzing-git-history-with-elasticsearch//blog/importing-git-history-to-elasticsearch/) thanks to the lack of git formatting options, but it's pretty cool once you get it working!

The commits that I imported has 22 data points, including a one-to-many list of changed files. I imported the git history from [dev.to](https://github.com/thepracticaldev/dev.to) to see how things looked.

Now I can run simple queries like this to see things like how many commits were made by each contributor:

![Query for Commits by Author in Kibana Dev Tools](https://analytics.codingblocks.net/img/devtools.png)

That's nice and all, but I'd prefer something visual so I put together a simple dashboard in [Kibana](https://www.elastic.co/products/kibana).

This is a screenshot of the charts I put together. I can apply filters and searches at the top of the dashboard and it will affect all the charts.

![Unfiltered dashboard](https://analytics.codingblocks.net/img/kibana-1.png "Unfiltered dashboard")

I can (for example) look at commits by Ben Halpern in the last 90 days that involve "Algolia". 

Check out the video to see a bit more about the visualizations:

{% youtube ejeHzxOBiOQ%}

Pretty cool right?

## Conclusion
Eventually I'd like to take some time to clean-up the import process so it's easier to run, but for now I'm going to take some time exploring a couple of my own projects to see if I can dig anything interesting up.
Day 13 Learning Javascript
Tags: beginners
Published: 2019-05-07 02:59:33

---
title: Day 13 Learning Javascript
published: true
description: 
tags: #beginner
---

"let" is the new "var"....

Or so I've heard. I was both pleased and slightly frustrated with learning ES6. On the one hand, I love that'll make coding Javascript simpler. On the other hand, it'll take a little getting used to learning new syntax after getting so used to typing "var" for everything.

Oh well, shouldn't take long. By the way, cheers to ES6 template strings! Soooo much better.
Bringing uxJS (A Simplified MVVM Framework) To A Modern Standard.
Tags: javascript, showdev, webdev, opensource
Published: 2019-05-07 02:57:32

---
title: Bringing uxJS (A Simplified MVVM Framework) To A Modern Standard.
published: true
description: 
tags: javascript, showdev, webdev, opensource
---

At the point at which I am writing this, uxJS (https://github.com/ua1-labs/uxjs) released some 5 years ago as another framework (https://github.com/joshualjohnson/jquery.x), has yet to be released as a 1.0 production ready library. uxJS was a hard fork of jQuery.x as an attempt to provide an simplified version of an MVVM Framework without using jQuery as the primary force for providing the functionality. This framework has been used in production for creating such user experiences like https://www.thetvshield.com/the-tv-shield-product-selector/.

As rapid as we move in the JS world, I've noticed that it's once again time to update the concepts in uxJS and prepare it for a 1.0 release into the world. In its current version, you can read all about the concept here https://github.com/ua1-labs/uxjs/blob/master/README.md. 

Here is what I'm thinking we need to do to modernize the framework:

1. Move away from controllers all together.
2. Keep the plugins concept but refactor it so that you can call it a "ux" instead of a plugin. 
3. Remove data-ux-plugins="plugin1 plugin2" and reuse construct data-ux="ux1 ux2" so that you can layer in multiple "ux" definitions to one DOM node.
4. To register new ux definitions, the api should be refactored from `ux.plugin()` to `ux.define(name)`
5. Update the entire library to use TypeScript
6. Create an abstract UX class that can be extended can be used to register "ux" definitions.

These are some of the things I am thinking at this point...

I know, I already hear you saying it, "Why yet another JS framework?" and the answer is simple...Because we as humans have the ability push software further along. This framework has lived with me for the past 6 years and will continue to do so and evolve as time moves on. It will remain uxJS for the rest of it's life, but will iterate as we modernize its functionality.

I think that's all I've got! You wanna join me in this refactoring effort?
Hello World! 
Tags: introduction, web, android, development
Published: 2019-05-07 02:49:02

---
title: Hello World! 
published: true
description: About me :)
tags: introduction, web, android, development
---

My name is Shahid Shaikh and I have been coding for the past 4 years now. I live in one of the busiest yet interesting city in India known as Mumbai. 

I started my journey by creating Icon Packs for Android Devices in 2015, where I used Open Source Icon Packs dashboard and compiled it along with icons that I made using Photoshop. We had a small community of icon pack designers on Google Plus (RIP) contributing and supporting each other's work. 

Later in 2016, I decided to learn more about Android Development and took a few basic to an advanced course on Udacity. In a year's time, I was able to design and develop multiple applications on the Play Store. One of my favorite and successful app is called as "Scrittor". It's a simple/minimal note-taking application. 

In 2017, I started working as a Web Solution Specialist in an MNC called "Endurance Internation Group". I was fortunate enough to work on some really great web development projects where my contribution was in the Front End. I learned a lot about HTML, CSS (including SASS) and JS. 

During this time my interest changed from Mobile to Web Development team and I am very excited to learn more about the same. One thing which overwhelms me about web development is the number of technologies that are being introduced each week. 

Currently, I am learning Node JS along with Express and MongoDB from Udemy and it has been such an interesting journey. 

Also, English is not my first language nor I am good at writing (need to fix this) and I am sure you have learned that by now.🤭 

I am hoping to learn a lot more via the "Dev.to" community and contribute to the same. 

Thank you for reading 😊
6 Collaborative Coding Tools for Remote Pair Programming
Tags: remote, vscode, atom, codepen
Published: 2019-05-07 01:17:30

---
title: 6 Collaborative Coding Tools for Remote Pair Programming
published: true
description: Whether you work on a distributed team or just have the occasional problem-solving session with a friend, you'll find a remote pair programming tool here.
tags: remote, vscode, atom, codepen
---

Fifteen years ago, most online collaboration and remote work involved email threads, Skype calls, and endlessly re-zipping projects for sharing after each iteration.

Things have changed a lot — most things for better (Google Docs!) and some things for worse ([always-on messaging](https://www.vox.com/recode/2019/5/1/18511575/productivity-slack-google-microsoft-facebook)). Either way, it's fair to say that a lot of the friction has been removed from remote work.

One traditionally high friction area has been cracking real-time code collaboration. But that's been changing in recent years as developers working on the world's top code editors have put forward their solutions.

Whether you work on a distributed team and need collaborative coding tools to serve you every day, or just need something for the occasional problem-solving session with a friend, you'll find something you can use here.

## 1\. Visual Studio Live Share

[![Visual Studio Live Share](https://www.sitepoint.com/wp-content/uploads/2019/04/1556692135visualstudio.jpg)](https://visualstudio.microsoft.com/services/live-share/)

[Visual Studio Live Share](https://visualstudio.microsoft.com/services/live-share/) is Microsoft's own real-time collaborative development solution for Visual Studio and Visual Studio Code.

Live Share allows teams to collaborate on a shared codebase easily — nobody needs to clone a repo, and everyone can use their own development environment. Each person can perform actions like navigating between files, making changes to code, selecting text, and see them reflected for everyone immediately.

Live Share shares everything from files to terminals and audio, offers features for collaborative debugging, and is free. If you're already a VS Code user, this choice is a no-brainer.

[Continue reading the original article to learn about the other 5 tools](https://www.sitepoint.com/collaborative-coding-tools-for-remote-pair-programming/).