teleren/devto huggingface dataset:
Data Pipelines with Apache Airflow - Book Review
Tags: python, dataengineering, books, bigdata
Published: 2022-06-13 22:12:14

### The Book:  

[Data Pipelines with Apache Airflow](https://www.manning.com/books/data-pipelines-with-apache-airflow) By Bas HarenSlak & Julian de Ruiter

### Short Summary:

"Data Pipelines with Apache Airflow" is an introductory and intermediate book about Apache Airflow. The book covers everything from introducing Airflow to giving some excellent ideas for generic use cases. The book has four parts: "Getting Started," "Beyond the Basics," "Airflow in Practice," and "In the Clouds." "Getting Started" and "Beyond the Basics," detail Airflow- such as how to use the framework and interacting with DAGs. "Airflow in Practice" also has some Airflow details but focuses more on the practical parts of Airflow, such as security. "In the Clouds" give examples of deploying a project using AWS, Azure, and GCP.

### What I liked:

The first two parts are excellent - the authors do a great job explaining Airflow. I learned so much that I was able to use immediately. Airflow is such a massive framework, but somehow, they were able to condescend all the essential concepts into these two parts without going into unnecessary topics. 

### What I disliked:
The last two parts went into some detail but didn't work for me as much as they assume a lot about your project. I understand the thinking behind these parts - they wanted to show examples and starting points - but for me, a lot of it was irrelevant. The only chapter I admit to skipping is chapter 17 because it discusses deployment in Azure, and I don't see myself using Azure anytime soon. 


### Review round-up:
This book is excellent for anyone working with Apache Airflow. Although I couldn't say the last two parts worked for me, the authors say in their "About this Book" that after chapter five, the reader can read what they feel is necessary. So perhaps, I overread what I needed/ overread what the Authors suggested, and I don't think it takes away from everything you can learn from this book. Again the first two parts are more than worth it, and if you decide to read the book, skim the chapters and see if you need to read the entire book. 


### Rating:
8/10 Python Snakes
The Genesis.Json file - A quick intro
Tags: web3, blockchain, tutorial, ethereum
Published: 2022-06-13 22:04:40

Hey it's been a while...

I have been doing a lot of learning in the blockchain space and I recently learnt about the **genesis.json** file, so i'm back here to share my learnings with you guys. If you are someone who is curious about the web3 space, i hope you find this post useful.

## The Genesis.json file

The Genesis.json file is a config file for Go-Ethereum (a blockchain node), that is needed in order to create a private blockchain network. It is the file that Go-Ethereum needs to create the very first block (Genesis block) of your private network.

## Some content of the file

1. <u>ChainId</u>: This can be set to any number except 1, 2 and 3, because 1 is reserved for the **Mainnet** while 2 & 3 is reserved for the **Testnets**.

2. <u>Homesteadblock</u>: Ethereum is separated into certain releases, the previous one was **Frontier network** launched on July 30th, 2015 and next release was called the **Homestead** launched on Mar-14-2016. It is usaually set to zero (0) in the Genesis.json file.

3. <u>eip155Block</u>: eip stands for Ethereum improvement protocol. It is usually set to zero (0) in the Genesis.json file.

4. <u>Difficulty</u>: This is very important if you want to mine ether on your network or create contracts. The lower it is, the faster mining will be, the easier it will be for your miner to solve the mathematical riddle in order to mine blocks. If you realize that mining takes too long, you can initialize 
a new private network and set the difficulty lower. 

5. <u>GasLimit</u>: This is necessary for deploying contracts, each contract written in solidity will be compiled to something like assembly or byte code. And each of them takes up some gas. The GasLimit is here to limit the amount of logic that you can run on a miner. The higher the GasLimit, the more complex your contracts can get. The current GasLimit on the Ethereum network is around 6 million or even more.

6. <u>Alloc</u>: This is set to pre-allocate ether to certain accounts. 
When you initialize your private network you might want to allocate ether to some accounts and this is where you set those accounts before initializing your private network. 


That's it for this post guy's, I hope I've been able to at least give you an idea about what the Genesis.json file is and why this space is so interesting.


Keep building, catch you later!
Binary Tree Level Order Traversal
Tags: leetcode, dsa, theabbie
Published: 2022-06-13 22:00:49

Given the `root` of a binary tree, return _the level order traversal of its nodes' values_. (i.e., from left to right, level by level).

**Example 1:**

![](https://assets.leetcode.com/uploads/2021/02/19/tree1.jpg)

**Input:** root = \[3,9,20,null,null,15,7\]
**Output:** \[\[3\],\[9,20\],\[15,7\]\]

**Example 2:**

**Input:** root = \[1\]
**Output:** \[\[1\]\]

**Example 3:**

**Input:** root = \[\]
**Output:** \[\]

**Constraints:**

*   The number of nodes in the tree is in the range `[0, 2000]`.
*   `-1000 <= Node.val <= 1000`

**SOLUTION:**

    # Definition for a binary tree node.
    # class TreeNode:
    #     def __init__(self, val=0, left=None, right=None):
    #         self.val = val
    #         self.left = left
    #         self.right = right
    class Solution:
        def levelOrder(self, root: Optional[TreeNode]) -> List[List[int]]:
            if not root:
                return []
            queue = [(root, 0)]
            levels = {}
            level = 0
            while len(queue) > 0:
                curr, level = queue.pop(0)
                levels[level] = levels.get(level, []) + [curr.val]
                if curr.left:
                    queue.append((curr.left, level + 1))
                if curr.right:
                    queue.append((curr.right, level + 1))
            return [levels[l] for l in range(level + 1)]
Multi Search LCCI
Tags: algorithms, cpp
Published: 2022-06-13 21:37:48

Given a string band an array of smaller strings T, design a method to search b for each small string in T. Output positions of all strings in smalls that appear in big, where positions[i] is all positions of smalls[i].


```
Input: 
big = "mississippi"
smalls = ["is","ppi","hi","sis","i","ssippi"]
Output:  [[1,4],[8],[],[3],[1,4,7,10],[5]]
```

- 0 <= len(big) <= 1000
- 0 <= len(smalls[i]) <= 1000
- The total number of characters in smalls will not exceed 100000.
- No duplicated strings in smalls.
- All characters are lowercase letters.



https://leetcode.cn/problems/multi-search-lcci/

```cpp
const int N = 100000 + 10;

class Solution {
private:
  int son[N][26], values[N], idx;
  vector<vector<int>> res;
  void insert(string &word, int val) {
    int p = 0;
    for (char &c : word) {
      int i = c - 'a';
      if (!son[p][i])
        son[p][i] = ++idx;
      p = son[p][i];
    }
    values[p] = val;
  }

  void search(int start, string &big) {
    int p = 0;
    for (int i = start; i < big.size(); i++) {
      int k = big[i] - 'a';
      if (!son[p][k])
        return;
      p = son[p][k];
      if (values[p] >= 0)
        res[values[p]].push_back(start);
    }
  }

public:
  vector<vector<int>> multiSearch(string big, vector<string> &smalls) {
    memset(son, 0, sizeof son);
    memset(values, -1, sizeof values);
    idx = 0;
    res.resize(smalls.size());

    for (int i = 0; i < smalls.size(); ++i)
      insert(smalls[i], i);

    for (int i = 0; i < big.size(); ++i)
      search(i, big);
      
    return res;
  }
};
```
Day 572 : Started
Tags: code, coding, hiphop, lifelongdev
Published: 2022-06-13 21:34:01

_liner notes_:
- Saturday : Been a couple of weeks since I was at the radio station and even longer with my co-host in the same room. Had a really good time as usual. The recording of this week's show is at https://kNOwBETTERHIPHOP.com

![Radio show episode image of a map of the United States with a red border around it with the text "Nearly 2 out of 3 people live within the 100-mile Border Zone" in the center of the map with the words June 11th 2022 Royal Blue](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kwreqn338zyjbsz342mk.jpg)

- Sunday : I wasn't traveling so I did the https://untilit.works cyphers. Actually had another person in the chat! I don't know if anyone else is on the site unless they say something in the chat. haha Got some some stuff done, like the mountain of laundry, post radio show promo, and research on my next project. Ended the night watching the latest "The Boys" episode. That show is wild! haha

- Professional : Today, I had a couple of meetings. Got to meet a new team member. Met up with my manager. Tested some stuff I worked on last week and set the tickets to done. Updated the documentation code snippet with some new functionality and got that done. Not a bad way to start the week.

- Personal : My goal this week is to get a quick working MVP of my next side project. Been thinking about different aspects for a minute and it seems like things are aligning so I want to take advantage of the things happening.

![Campagna, Italy countryside with rolling hills with trees, farmlands and homes](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/irnn4kwfpvlqfoovbxsa.jpg)

I think I figured out the technology I want to use for the front end of my next side project. I was thinking about creating a Web Component, but I'm not so sure right now. I'm just going to pull the trigger and get something started. Can always change it later.

Have a great night!

peace piece
Dwane / conshus
https://dwane.io / https://HIPHOPandCODE.com
{% youtube 6loygEEUvXg %}
Deconstruction to not work
Tags: help, react, javascript
Published: 2022-06-13 21:32:54

Deconstruction to not work

```
function QuziLogic({
  currentQuestion, questions, handleAnswerOptionClick, countDownBarWith, totalTime,
}) {
  return (
    <>
      <div className={styles.questionCount}>
        <p className={styles.questionText}>What is the output?</p>

        <p className={styles.questionText}>
          Question
          {currentQuestion + 1}
          /
          {questions.length}
        </p>
      </div>
```
Rearrange Array Elements by Sign
Tags: leetcode, dsa, theabbie
Published: 2022-06-13 21:30:40

You are given a **0-indexed** integer array `nums` of **even** length consisting of an **equal** number of positive and negative integers.

You should **rearrange** the elements of `nums` such that the modified array follows the given conditions:

1.  Every **consecutive pair** of integers have **opposite signs**.
2.  For all integers with the same sign, the **order** in which they were present in `nums` is **preserved**.
3.  The rearranged array begins with a positive integer.

Return _the modified array after rearranging the elements to satisfy the aforementioned conditions_.

**Example 1:**

**Input:** nums = \[3,1,-2,-5,2,-4\]
**Output:** \[3,-2,1,-5,2,-4\]
**Explanation:**
The positive integers in nums are \[3,1,2\]. The negative integers are \[-2,-5,-4\].
The only possible way to rearrange them such that they satisfy all conditions is \[3,-2,1,-5,2,-4\].
Other ways such as \[1,-2,2,-5,3,-4\], \[3,1,2,-2,-5,-4\], \[-2,3,-5,1,-4,2\] are incorrect because they do not satisfy one or more conditions.  

**Example 2:**

**Input:** nums = \[-1,1\]
**Output:** \[1,-1\]
**Explanation:**
1 is the only positive integer and -1 the only negative integer in nums.
So nums is rearranged to \[1,-1\].

**Constraints:**

*   `2 <= nums.length <= 2 * 105`
*   `nums.length` is **even**
*   `1 <= |nums[i]| <= 105`
*   `nums` consists of **equal** number of positive and negative integers.

**SOLUTION:**

    class Solution:
        def rearrangeArray(self, nums: List[int]) -> List[int]:
            n = len(nums)
            nums.sort(key = lambda num: -1 if num < 0 else 1)
            op = []
            for i in range(n // 2):
                op.append(nums[i + n // 2])
                op.append(nums[i])
            return op
Random CSS fish
Tags: codepen
Published: 2022-06-13 21:04:56

<p>This is an attempt to use JS to manipulate CSS, but still keep full control over the appearance within the CSS.
I'll use CSS variables as "parameters" to JS, do some calculations (randomize something), and return the result to another CSS variable.
Allowing me (in theory) to tweak the appearance of this pen, without touching the Javascript.</p>

{% codepen https://codepen.io/harunpehlivan/pen/abqPPmx %}
Replace All Digits with Characters
Tags: leetcode, dsa, theabbie
Published: 2022-06-13 21:00:46

You are given a **0-indexed** string `s` that has lowercase English letters in its **even** indices and digits in its **odd** indices.

There is a function `shift(c, x)`, where `c` is a character and `x` is a digit, that returns the `xth` character after `c`.

*   For example, `shift('a', 5) = 'f'` and `shift('x', 0) = 'x'`.

For every **odd** index `i`, you want to replace the digit `s[i]` with `shift(s[i-1], s[i])`.

Return `s` _after replacing all digits. It is **guaranteed** that_ `shift(s[i-1], s[i])` _will never exceed_ `'z'`.

**Example 1:**

**Input:** s = "a1c1e1"
**Output:** "abcdef"
**Explanation:** The digits are replaced as follows:
- s\[1\] -> shift('a',1) = 'b'
- s\[3\] -> shift('c',1) = 'd'
- s\[5\] -> shift('e',1) = 'f'

**Example 2:**

**Input:** s = "a1b2c3d4e"
**Output:** "abbdcfdhe"
**Explanation:** The digits are replaced as follows:
- s\[1\] -> shift('a',1) = 'b'
- s\[3\] -> shift('b',2) = 'd'
- s\[5\] -> shift('c',3) = 'f'
- s\[7\] -> shift('d',4) = 'h'

**Constraints:**

*   `1 <= s.length <= 100`
*   `s` consists only of lowercase English letters and digits.
*   `shift(s[i-1], s[i]) <= 'z'` for all **odd** indices `i`.

**SOLUTION:**

    class Solution:
        def replaceDigits(self, s: str) -> str:
            return "".join([chr(ord(s[i - 1]) + int(c)) if i & 1 else c for i, c in enumerate(s)])
Deploying WordPress on Kubernetes
Tags: kubernetes, wordpress, javascript, webdev
Published: 2022-06-13 21:00:17

Onboarding applications on Kubernetes can be complex. 

Understanding how deployment sets work, volumes, env variables, services, and networking works on Kubernetes can often take away the time from developers and impact application delivery.

Also, sharing application definitions with other teams can help speed up onboarding of new applications, new developers, and more.

Because of that, I started a personal project to help developers onboard their applications on Kubernetes, share those definitions with their teams, and publicly if desired.

![Searching saved app definitions](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5koof70bg7kv1r8uqjjo.png)

![Exporting app definitions for a provider](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ve52rxezjqyticojwgt6.png)

I’m building the project on top of shipa.io APIs. The purpose is to give developers a way to go from application definition to management without dealing with the underlying Kubernetes complexity. 

![App management with Shipa](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3s0kas38o2i0g5qty1to.png)

I’m just starting with this (I don’t even have a name for it yet), and bugs will be found :) but I wanted to get it out there early, get feedback, and see what could be added to help solve a big problem for different teams.

You can find the application available here: [https://you.defineyour.app/search](https://you.defineyour.app/search)

Please, give it a try, share some feedback, and share it with your teammates!
11ty Second 11ty: Basics of Collections
Tags: 11ty, jamstack, javascript, html
Published: 2022-06-13 20:52:45

Collections are the workhorse of most 11ty projects. They provide a way of grouping content or data in meaningful ways. By default, this is done by providing a `tag` property to a piece of content (such as in the front matter of your markdown)

{% embed https://www.youtube.com/watch?v=7G5m6RDZ6cU %}
Day 66: Hello again freecodecamp!
Published: 2022-06-13 20:49:19

Hi!
Today I got back to the freecodecamp, the first resources I used to learn programming I remember when I got started I did some challenges in the responsive web  design section basically HTML and CSS but I didn't finish even the first part of the section and that's the same for the section of JavaScript algorithm and data structures but for some reason I want these certificates and I want them soon. so today I returned where all started and done challenges about the responsive web design course I expected to know everything as it's just basics but I was suprised about how much I didn't know about basics and I'm glad I' taking this tutorial again as I can see some gaps I have in the basics about frontend development. I think freecodecamp is a great free resource for learning and I expect to continue using it this week and days to come. as I will mostly be working on UI/UX and frontend development for the upcoming few weeks I will be focusing more on frontend technologies mainly React and other JavaScript based libraries/frameworks  

Signing off...
📢 We're on DockerHub
Tags: docker, api, programming, news
Published: 2022-06-13 20:39:32

## ✨About

This week, I'd like to promote our public Docker images, and today is the intro : 

The first approach is to 

> look for them on Google to see how well they are indexed : 

{% youtube 9H60KADDJRo %}

## 🔖 Resources

[`opt-nc`](https://github.com/opt-nc) on DockerHub : 

{% embed https://hub.docker.com/u/optnc %}
Check if Array Is Sorted and Rotated
Tags: leetcode, dsa, theabbie
Published: 2022-06-13 20:30:42

Given an array `nums`, return `true` _if the array was originally sorted in non-decreasing order, then rotated **some** number of positions (including zero)_. Otherwise, return `false`.

There may be **duplicates** in the original array.

**Note:** An array `A` rotated by `x` positions results in an array `B` of the same length such that `A[i] == B[(i+x) % A.length]`, where `%` is the modulo operation.

**Example 1:**

**Input:** nums = \[3,4,5,1,2\]
**Output:** true
**Explanation:** \[1,2,3,4,5\] is the original sorted array.
You can rotate the array by x = 3 positions to begin on the the element of value 3: \[3,4,5,1,2\].

**Example 2:**

**Input:** nums = \[2,1,3,4\]
**Output:** false
**Explanation:** There is no sorted array once rotated that can make nums.

**Example 3:**

**Input:** nums = \[1,2,3\]
**Output:** true
**Explanation:** \[1,2,3\] is the original sorted array.
You can rotate the array by x = 0 positions (i.e. no rotation) to make nums.

**Constraints:**

*   `1 <= nums.length <= 100`
*   `1 <= nums[i] <= 100`

**SOLUTION:**

    class Solution:
        def check(self, nums: List[int]) -> bool:
            n = len(nums)
            pivot = 0
            for i in range(1, n):
                if nums[i] < nums[i - 1]:
                    pivot = i
                    break
            for i in range(pivot, pivot + n - 1):
                if nums[(i + 1) % n] < nums[i % n]:
                    return False
            return True
Free Hosting for Static Website
Tags: webdev, staticwebapps, react, angular
Published: 2022-06-13 20:29:32

🌐 [GitHub Pages](https://pages.github.com/)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mnq6xld12g03qwndda95.png)

🌐 [Netlify](https://www.netlify.com/)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/v9gw8dqocu95hh81o4p1.png)

🌐 [ Firebase Hosting](https://firebase.google.com/)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qzsjwelj9w4ukqner9b1.png)

🌐 [Vercel](https://vercel.com/)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0pzuapl4x7yijprpqlx0.png)

🌐 [Amazon S3 Hosting](https://aws.amazon.com/s3/)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1an8xnzd3ykwgprj7vff.png)

🌐 [Cloudflare Pages](https://pages.cloudflare.com/)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/x3b96gviyehttlvzb43h.png)

🌐 [Render.com](https://render.com/)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tapwie1zt3l1okd81kd8.png)

🌐 [Glitch.com ](https://glitch.com/)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/94ahhgdnvrgzww2ct6xq.png)

🌐 [Surge.sh](https://surge.sh/)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tnjvup4kqjbimqk4z97w.png)

🌐 [Fleek.co](https://fleek.co/)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/s2sf0pc9b2d4th8afi61.png)
Quick life update
Tags: beginners, devjournal, writing, codenewbie
Published: 2022-06-13 20:25:30

I’ve been trying to structure my DSA study, and [this video by Maya Bello](https://www.youtube.com/watch?v=dR6n0uLk1KQ) where she ranks the resources she used to study for technical interviews was immensely helpful.

I’m going to use the Technical Interview Study Guide by Eric Duran to track my progress, and I’ll tackle one DSA each week. The schedule I’ve worked out is as follows:

- Monday - watch lectures / videos (MIT Open Courseware, HackerRank, etc) + read relevant chapter of Crack the Coding Interview 
- Tuesday - do 2-3 problems on Leetcode 
- Wednesday - use practice questions from Crack the Coding Interview chapter to practice whiteboarding 
- Thursday - practice answering JS and React-specific interview questions 
- Friday - do more Leetcode problems 
- Saturday - practice writing DSAs from scratch 

I still have to work on my portfolio projects, so I’ll focus on DSAs in the morning and do project work in the afternoon. 

My first mini-project is to create my personal website with React and host it on a cloud service, like AWS Amplify. I had actually planned on completing that this past weekend, but there was an unforeseen setback. My MacBook sustained liquid damage and is being repaired right now, so I probably won’t have access to it until Wednesday… 😭

Even without my laptop, I can still watch videos on Youtube, read CCI, and do practice questions. The first topic on Duran’s study guide is Big O and that’s scary and intimidating, so I’ll start with the topic right after — strings, lmho. 

As far as my job search goes, I’ve been applying to software engineering internship programs (specifically ones that don’t require CS degrees), so hopefully I’ll hear something soon.
If you lose interest in programming, how do you usually get it back?
Tags: discuss, motivation
Published: 2022-06-13 20:06:50

Sometimes when I'm procrastinating and feeling that I completely lost interest in coding(temporarily, of course) I usually dive into something related to computers, like cyberpunk games/movies/comics, stuff about hackers and so on, to feel that there's not only the field I'm working within, but the whole world. Also it helps to do something completely different, like getting very deep inside *unix systems.
What's your way to get interest back?
Creating multiples directories from a file on bash terminal
Tags: bash, ansible
Published: 2022-06-13 19:56:25

## The problem
A repetitive task to do: create 111 directories with 2 sub directories each one. The structure is:

```
├── SMD0088
│   ├── files
│   └── tasks
```
The names are from a spreadsheet column.

## The motivation
I need to prepare 6 computer labs for next semester classes. Each professor has a small list of application to install on a right lab. 

## The Solution
Create an Ansible role to each subject and a playbook for each lab.

## The procedure
1. Copy and paste the spreadsheet column and save as 'd' file.
2. Navigate to 'playbook/roles' ansible directory and run the command:

`$ while read line; do mkdir -p $line/{tasks,files}; done < d`

## The result

The line 'SMD0088' on "d" turns on to 'SMD0088/files' and 'SMD0088/tasks' directories.