teleren/devto huggingface dataset:
Scrapy: store files efficiently using folder trees
Tags: showdev, opensource, python, scrapy
Published: 2022-02-01 11:44:21

## Context

When it comes to scrapping, [`Scrapy`](https://scrapy.org) is one of the most known and used frameworks. The huge community and the large collection of third-party extensions are only few reasons to choose scrapy. You can find a scrapy extension for almost everything.

## File storing

As I have already said [here](https://simakis.me/posts/scrapy-store-images-efficiently-using-folder-structure/), storing large number of files is quite challenging, you should always have in mind storing large number of files in a single folder is not a good idea.

In case of `Scrapy`, all files are stored in a single folder. So I decided to implement a `Scrapy` pipeline extension in order to provide a way to store files in a more efficient way using folder trees.

## scrapy-files-hierarchy

{% github sp1thas/scrapy-folder-tree %}

This scrapy pipelines provides various ways to store your crawled files. Currently, given this scraped file: `05b40af07cb3284506acbf395452e0e93bfc94c8.jpg`, the following three folder structures are supported:

### Using the file name(hash)
```
full
├── 0
.   ├── 5
.   .   ├── b
.   .   .   ├── 05b40af07cb3284506acbf395452e0e93bfc94c8.jpg
```

### Using the crawling date
```
full
├── 0
.   ├── 11
.   .   ├── 48
.   .   .   ├── 05b40af07cb3284506acbf395452e0e93bfc94c8.jpg
```

### Using the crawling time
```
full
├── 2022
.   ├── 1
.   .   ├── 24
.   .   .   ├── 05b40af07cb3284506acbf395452e0e93bfc94c8.jpg
```

## Installation

```shell
pip install scrapy-folder-tree
```

## Usage

Use the following settings in your project:

```python
ITEM_PIPELINES = {
    'scrapy_folder_tree.FilesHashTreePipeline': 300
}

FOLDER_TREE_DEPTH = 3
```

Feel free to give a try and to provide your feedback.

## Future work

 - Support more folder structures
 - Parameterize folder structure
WordPress Black Friday / Cyber Monday Deals (Huge Savings)
Published: 2022-02-01 10:36:43


**WPForms**

Save 65% off WPForms – WPForms is the best WordPress form builder used by over 5 million sites. It comes with over 300+ pre-made form templates, smart conditional logic, surveys, conversational forms, payment and marketing integrations, and more.


**All in One SEO for WordPress**

Save 65% off AIOSEO – All in One SEO is the best WordPress SEO plugin used by over 3 million websites. It helps you get higher SEO rankings with SEO audit, TruSEO content analyzer, smart sitemaps, schema markup, local SEO tools, redirection manager, and more.


**SiteGround**

Save 80% off SiteGround – SiteGround is one of the best hosting companies around and an officially recommended WordPress hosting provider. They offer ultrafast hosting, built-in caching, an easy to use hosting dashboard, and heroic customer support.


**MonsterInsights**

Save 65% off MonsterInsights – MonsterInsights is the best WordPress analytics plugin used by over 3 million sites. It makes it easy to setup comprehensive website analytics, so you can see the website stats that matter and grow your business with confidence.


**OptinMonster**

Get 60% off OptinMonster – OptinMonster is the best lead generation and conversion optimization toolkit on the market. It helps you unlock the maximum revenue potential from every page and visitor on your website. Simply put, you get more email subscribers, leads, and sales. Used by over 1 million smart business owners.


**SeedProd**

Get 65% off SeedProd – SeedProd is the #1 drag and drop page builder plugin for WordPress. It allows you to quickly make landing pages that work independently of your WordPress theme or plugins. It comes with dozens of beautiful templates that are designed for speed, higher conversions, and more sales.



**Reference Blog**
[https://www.gomahamaya.com/black-friday-sale/](https://www.gomahamaya.com/black-friday-sale/)
Become persistent by keeping up with the streak
Tags: beginners, productivity, writing, motivation
Published: 2022-02-01 11:25:13

Thanks, everyone, for showing love and support to my work. 

It gives me immense joy to share that my articles have been read more than **5k** times on this platform and **10k** times across platforms. 

I started my writing journey this year, and I have already reached
- 250 followers on [DEV](https://dev.to/anshuman_bhardwaj)
- 150 followers on [Twitter](https://twitter.com/sun_anshuman)
- 4-weeks streak badge

<img width="100%" src="https://media1.giphy.com/media/U4DswrBiaz0p67ZweH/giphy.gif" />

It is such a milestone for me. It motivates me to write and share more with the community and gives new ideas to think about.

## My Process

I'm a fan of James Clear, and I followed his advice of repetitive action over perfection. He recommends building up a process rather than overdoing things. So, I set up a goal to write and publish one article per week. I've been able to keep up with it for over a month now, and I have published six posts, with 40 in the draft. 

If you are planning to start, my advice would be to

- fix a schedule for yourself and follow it without questioning
- increase the frequency when it starts feeling too easy
- focus on your progress only, everyone else had a different start
- keep an idea box to add new ideas as draft
- keep improving your content and listen to what the audience likes

## What's next
- I'll publish articles twice a week 
- I'll tweet once each day
- Aiming for the eight-weeks streak badge

---

I hope this was motivating and helpful for you. Please share your experience in the comments below, and for more such content, please follow me on [Twitter](https://twitter.com/sun_anshuman)

> Until next time

<img width="100%" src="https://media.giphy.com/media/kRkJXYahXjSE0/giphy.gif" />
Next generation Hello World program
Tags: programming, challenge, helloworld
Published: 2022-02-01 11:14:20

While learning a programming language, I am pretty sure you started with the basic "Hello World!" program :)

But once you advance in a particular programming language, the Hello World code would start seeming to be boring.

But here's an interesting challenge for you! My friend and I have created a challenge out of the basic Hello World code so you could stretch your coding muscles and judge how well you know about the language you specialize in!

Read [this article](https://genicsblog.com/next-generation-hello-world-program) to learn about the challenge and also how to solve the given problem.

Though it might feel it is for advanced coders, everything is explained in a beginner-friendly way so it's worth checking out!
Learn create-react-app 
Tags: javascript, react, beginners, tutorial
Published: 2022-02-01 10:46:04

Let's get straight to the point.
If you want to become a [React](https://reactjs.org/) Developer, you should know create-react-app

but...


## WHAT IS CREATE-REACT-APP
Create-react-app is a tool built by Facebook to help set up all the tools you need for your React applications.

Before we dive into this fantastic tool, be sure to know Javascript ES6 and to have node.js installed on your computer.

CSS and HTML are also required to build a react app.

Now it's time to open your terminal and type the command npx create-react-app followed by a space and your application's name.

```
npx create-react-app myapp
```

Once the installation is completed, you will get the success message in the terminal.

Navigate in your project folder and run the command ` npm start `
to start the development server on localhost:3000.

![react-localhost](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/z6hoki8q26r7nwiotnq1.jpg)

Open the project folder in your text editor and look at the folder structure created by create-react-app.

If it is your first time using a Javascript library, you can be intimidated by the number of files inside your project folder.

Let’s take a look at the most important files.

**THE PACKAGE.JSON FILE**

It Contains the dependencies needed to build your project, and it also describes your application, such as the name and the version.
You can easily change the name of your App inside this file.

**THE SRC FOLDER**

We will put all the JS and CSS files necessary to build the UI and your application's functionality inside this folder.
All the components we want to create must be inside the **src folder.**

The App.js file is the wrapper component of the App. 
It's the container for all other React components.
The index.js file tells React where to render the App component.

**THE PUBLIC FOLDER**

In the public folder, we will focus on the index.html file.
This file contains plain HTML.
React will inject the code inside the div with the id 'root' via the render function inside the **index.js** file.

Try to edit the App.js file, and let's see what happens!
Delete the code inside the wrapper div and also remove the logo.svg file from our project and don't forget to remove the import statement in the App.js.

Going back to the browser, you will see a completely blank page

Inside the App.js file, try to write an H1 HTML tag between the div with the class App.

```
function App() {
  return (
    <div className="App">
     <h1>Learning React</h1>
    </div>
  );
}
```
React will automatically update the page.

Back to the browser, and you will notice a nice **LEARNING REACT **header displayed on your page

The setup is finished, you are ready to code your App.
How to do EDB to PST Conversion
Tags: edb, pst, edbtopst, edbtopstconverter
Published: 2022-02-01 10:23:33

It is simple to migrate PST files and backup Exchange Server mailboxes. As a result, this article will recommend a professional EDB to PST converter to convert your Exchange EDB files to PST file format.

Weeom's competent [EDB to PST converter](https://www.weeom.com/edb-to-pst-converter.php) is the greatest programme. It checks for and discovers unavailable EDB files before converting them to Outlook PST. It is simple to use for everyone.

Key Characteristics:

- Prior to conversion, it allows flawless recovery of faulty EDB files with no data loss.

- It allows you to preview things from source files before saving them as Outlook PST files.

- Split option to create numerous smaller PST files as a consequence.

- It exports EDB file elements to MSG, EML, HTML, and vCard formats.

To assess the performance of Weeom EDB to PST converter, a free trial version that permits processing the first 50 items per folder can be used.

For more details, visit:- https://www.weeom.com/edb-to-pst-converter.php
JS problems
Published: 2022-02-01 10:18:44

**1. How to add array to an object**

```
<script>
// JavaScript program to add array into
// an object using push() function

// Creating a JS object to add array into
var Obj = {			
	arrayOne: ['Geeks', 'for', 'Geeks'],
	arrayTwo: []
};

// Array to be inserted
var arraynew = ['Hello', 'World', '!!!'];

// Pushing of array into arrayTwo
Obj['arrayTwo'].push(arraynew);	

alert(Obj.arrayTwo);
	
</script>					

```
**2. Add & Delete dynamic input fields**

```
 const inputArr = [
    {
      type: "time",
      id: 1,
      value: "",

    }
  ];

const [field, setField] = useState(inputArr);
const [tempfield, setTempfield] = useState("");

// Add input field
const addInputField = (e) => {
    e.preventDefault();
   
      setField(s => {
        const lastId = s[s.length - 1].id;
        const nextId = lastId + 1;
        console.log("lastId", lastId)
        return [
          ...s,
          {
            type: "time",
            id: nextId,
            value: tempfield
          }
        ];
      });
}

 // Delete the input field
  const deleteField = (fieldId) => {

  let filterField = field.filter(arr => arr.id !== fieldId + 1)
      setField(filterField)
}
```
**3. Checkboxes**

```
 const [checked, setChecked] = useState(["Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday"])

  // Add/Remove checked item from list
  const handleCheck = (event) => {
    var updatedList = [...checked];
    if (event.target.checked) {
      updatedList = [...checked, event.target.value];
    } else {
      updatedList.splice(checked.indexOf(event.target.value), 1);
    }
    setChecked(updatedList);
  };

<div>
              <input
                className="form-check-input"
                value="Monday"
                type="checkbox"
                defaultChecked
                onChange={handleCheck} />
              Monday
</div>
```
**4. Find & replace array object**

```
// can use find or map
 let find = cartData.find((o, i) => {
      if (o.product_id === pid) {
        // cartData[i] =
        //   { name: 'new string', value: 'this', other: 'that' };
        cartData[i].quantity += 1
        return true; // stop searching
      }
    });
    console.log("Replaced data", cartData)
```
Places to get free vector images for your designs
Published: 2022-02-01 10:09:47

**Undraw**

Katerina Limpitsouni has created a library of amazing and customizable MIT licenced illustrations. They’re free and there’s no need for attribution either.
Her whimsical and fun illustrations are also SVG — scalable at any size without losing any of the quality.


**Freepik**

Freepik boasts over 800,000 free illustration vectors. There’s everything to choose from. You need animals? What about silhouettes? You’ll find it on Freepik.


**Pixabay**

Pixabay is another image search engine with a ton of free vectors up for grabs. Pixabay also aggregates images from other free websites, making their collection even bigger.


**Vector Stock**

Vector Stock is home to over 200,000 free vector images. You can sort them by latest uploads or by trending vectors. As with the other sites, the formats include: AI, EPS, PDF and JPG. All free, all yours. However, the vectors are for personal use only, which is a bit of a limitation for freelance designers.


**Flaticon**

If you want access to over a million vector icons then Flaticon’s your best bet. It has collections grouped into packs, for easy browsing.
There’s the whole gamut of icons: strategy and management, friendship, ecommerce, marketing and SEO… you name it, Flaticon’s got it.


**Reference Blog**
[https://www.gomahamaya.com/vector-illustrations-images/](https://www.gomahamaya.com/vector-illustrations-images/)
Trending Now!! Watch and Learn Machine Learning Course From Experts & Excel in Your Career.
Tags: machine, machinelearning
Published: 2022-02-01 09:56:23

You will learn how to become a machine learning engineer and all areas of machine learning by the end of this course video. Machine learning algorithms, fields of use for machine learning, statistics and probability principles, deep learning frameworks, end-to-end projects, interview questions, and much more are covered in this machine learning lesson. If you're seeking for a machine learning training, this machine learning tutorial is exactly what you need.

Watch Now: [https://www.youtube.com/watch?v=zNhTOPQeRaU&t=8s](https://www.youtube.com/watch?v=zNhTOPQeRaU&t=8s)
Latest Data Science Course by Top Data Science Experts
Tags: datascience, course, training
Published: 2022-02-01 09:54:45

With this video about data science principles, skills, and careers that you need to know to become a data scientist, you may become a successful data scientist. You will master the principles of data science, the algorithms used by data scientists, and the roles and duties of data scientists in this comprehensive data science course. We'll show you how to read a data scientist's resume and educate you how to answer the most common data science interview questions.

Watch now : https://www.youtube.com/watch?v=a5KmkeQ714k
Easy Project That Will Make You Famous
Tags: sideprojects, 100daysofcode, career
Published: 2022-02-01 09:54:19

Are you a developer or CS student looking for an easy project that will make you famous?

Create a Zoom app that announces "${name}, you're muted", every time someone starts screen sharing.

Use Zoom's "user.presence_status_updated" webhook to trigger the alert whenever a user's "presence_status" is updated to "Presenting."

Here's the documentation you need: https://marketplace.zoom.us/docs/api-reference/zoom-api/events/#/paths/user.presence_status_updated/post

You're welcome!
NativeBase Clone Experiments
Tags: react, reactnative, clone, developer
Published: 2022-02-01 09:51:24

[NativeBase](https://nativebase.io/?utm_source=Devto&utm_medium=Social&utm_campaign=Article&utm_id=Main+Website) Clone Experiments is a showcase of work by developers who are pushing the boundaries of web & mobile technology with NativeBase, creating beautiful, unique experiences by taking inspirations from famous websites. You'll find helpful links throughout the blog for creating your own experiments, and you can also explore resources like using a single codebase to build web and mobile apps.

NativeBase uses React Native and React Native Web for the web support but without losing the best of both the worlds. For eg: We wrote our own mapping for accessibility that handles keyboard and mouse interactions and works seamlessly on all the platforms, we called it React Native Aria.
 
The idea behind these projects is to showcase a list of projects solving one critical problem each so that you won’t have to solve them.
 
You can browse the experiments below:

- [Food ordering clone experiment](https://nativebase.io/blogs/clone-experiment-food-ordering-app?utm_source=Devto&utm_medium=Social&utm_campaign=Article&utm_id=Food+ordering+app)
State of Frontend 2022
Published: 2022-02-01 09:36:23


#stateoffrontend 2022 is almost here! 😮I need your help – below you will find a survey about your experiences and opinions about #FrontendDevelopment. ⬇ Together, we create a huge report for the #frontend community, so every vote counts! 🤝

Link: https://tsh.io/state-of-frontend/

#frontend #survey #javascript #devs #community
Wordpress in docker is slow ... I have a solution for that!
Tags: wordpress, docker, programming, webdev
Published: 2022-02-01 09:32:46

Docker WordPress is slow, terribly slow. But I have a way.

**See one simple trick! WordPress and Docker haters hate it!**

In `docker-compose.yml`, in the place where we configure the volumes, we will add the flag `:delegated`.

```yml
    volumes:
      - my_wordpress_db:/var/lib/mysql:delegated

###
    volumes:
      - ./wp-content:/var/www/html/wp-content:delegated
```

These two sites in 
`docker-compose.yml`
 do the job. It is possible to work, even quite comfortably 💪🐳

![WordpressDocker](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/s3nys33wtvy6d1e5bel5.gif)
The shortest and best way to add two string numbers that you probably don't know
Tags: javascript, webdev, node
Published: 2022-02-01 09:26:09

### Hello Folks 👋
What's up friends, this is **SnowBit** here. I am a young passionate and self-taught developer and have an intention to become a successful developer.

Today, I am here with an amazing thing that you probably didn't know but now you will know about that. Happy reading

--- 

```js
const x = "5"
const y = "4"

const z = x + y
```
This won't work, because adding string concatenates the string and therefore the output of the code will  be `"54"` and not `9`

In this article, I will discuss two methods to add string numbers.

### Using `parseInt()`
```js
const x = "5"
const y = "4"

const z = parseInt(x) + parseInt(y)
```
Here, the string gets parsed to a number, therefore the output of this code must be `9` as both `x` and `y` variables are converted to a number.

If you use `parseInt()` with words and letters it will return - `NaN` and it stands for **Not a Number**.

This method was pretty simple to use, but now we go for a much simpler way.

### Using the unary plus operator - Best method
As discussed above, we can't just add two string numbers with the `+` operator. But there's a way to add two string numbers with the `+` operator.

Let me show you,
```js
const x = "5"
const y = "4"

const z = +x + +y
```
Using the `+` operator alone before an element indicates a math operation and tries to convert the element to a number, and if it fails to do - it will return `NaN`

That is it for this article. I regularly share articles so make sure to click the **follow button**.

---

Thank you for reading, have a nice day!
**Your appreciation is my motivation 😊**

- Follow me on Twitter - [@codewithsnowbit](https://twitter.com/codewithsnowbit)
- Subscribe to me on YouTube - [Code With SnowBit](https://www.youtube.com/channel/UCNTKqF1vhFYX_v0ERnUa1RQ?view_as=subscriber&sub_confirmation=1)

Cover photo by: <a href="https://unsplash.com/@timesnewroman14?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Roman Skrypnyk</a> on <a href="https://unsplash.com/s/photos/calculator?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>
Today's Fun Joke For Developers - Daily Developer Jokes
Tags: jokes, dailydeveloperjokes
Published: 2022-02-01 08:00:08

---
title: "Today's Fun Joke For Developers - Daily Developer Jokes"
description: "Check out today's daily developer joke! (a project by Fred Adams at xtrp.io)"
series: "Daily Developer Jokes"
published: true
tags: #jokes, #dailydeveloperjokes
---

Hi there! Here's today's Daily Developer Joke. We hope you enjoy it; it's a good one.

![Joke Image](https://private.xtrp.io/projects/DailyDeveloperJokes/public_image_server/images/5e125925337e5.png)

---

For more jokes, and to submit your own joke to get featured, check out the [Daily Developer Jokes Website](https://dailydeveloperjokes.github.io/). We're also open sourced, so feel free to view [our GitHub Profile](https://github.com/dailydeveloperjokes).

### Leave this post a ❤️ if you liked today's joke, and stay tuned for tomorrow's joke too!

_This joke comes from [Dad-Jokes GitHub Repo by Wes Bos](https://github.com/wesbos/dad-jokes) (thank you!), whose owner has given me permission to use this joke with credit._

<!--
Joke text:
___Q:___ Why did the constant break up with the variable?

___A:___ Because they changed.
 -->