teleren/devto huggingface dataset:
AWS Services | AWS CodePipeline | AWS CodeDeploy | AWS CodeBuild | CodeCommit | AWS DevOps
Published: 2022-12-20 06:26:26

This Edureka "CodeBuild CodePipeline CodeDeploy CodeCommit in AWS” video will give you a thorough and insightful overview of all the concepts related to CI/CD services in AWS.
{% embed https://www.youtube.com/watch?v=h0p4dxuwv1s %}
841. Leetcode Solution in Java
Tags: java
Published: 2022-12-20 06:24:17

```java
class Solution {
    public boolean canVisitAllRooms(List<List<Integer>> rooms) {
        Stack<Integer> stack = new Stack<>();
        Set<Integer> set = new HashSet<>();
        set.add(0);
        stack.add(0);
        while (!stack.isEmpty()) {
            int keyPopped = stack.pop();
            List<Integer> keys = rooms.get(keyPopped);
            for (Integer key : keys) {
                if (!set.contains(key)) {
                    stack.push(key);
                    set.add(key);
                }
            }
        }
        
        return set.size() == rooms.size();
    }
}
```
#leetcode
#challenge
Here is the link for the problem:https://leetcode.com/problems/keys-and-rooms/
🏎️ Yearly GitHub.com repos review (as a movie) 📽️
Tags: github, datascience, showdev, management
Published: 2022-12-20 06:22:42

## 🎯 About this series

What you'll discover in this series is **the knowledge we got by working on the data** our activity did generate on GitHub.com, only by using its [(always richer) set of APIs](https://docs.github.com/en/rest/issues?apiVersion=2022-11-28).

## ❔ Intro

GitHub.com is a complete platform on which people :

- 🧑‍🤝‍🧑 Build & **socialize** around code
- 🎁 **Contribute**
- 📅 Manage **projects**
- 📦 **Deliver** software
- 🚀 Trigger **deployments**

**Each of these activities generate structured data** that describe what you did, when, with who... and (hopefully 😅) why.

Some reporting is already delivered through charts by GitHub:


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/r7lfsnts4ro6eaunlsaf.png)

... but sometimes you can be challenged by many questions, including (but not only) one of the most crucial one : 

> _"How much time (and where) do you spend on `BUILD` vs. `RUN`"_

or this one:

> _"You talk about BOTs and third party services... but how can you tell me more about the ROI ... and how it helps achieve more things ?")_

👉 **In this episode**, we'll only **focus on reporting which repositories have been the most active** during the past year... through a **format anyone can immediately understand.**

## 🍿 Demo

The **chart race of the most active repositories**, based on issues activity :

{% youtube aFtJx-dEqug %}

## 🧰 Stack

Here is the flow we used to produce the content :

1️⃣ Dump GitHub issues to `csv` with Python [`invoke`](https://www.pyinvoke.org/) tasks
2️⃣ Use [`Jupyter Notebook`](https://jupyter.org/)
3️⃣ Generate Race Chart with [`Bar Chart Race`](https://www.dexplo.org/bar_chart_race/)

## 💰 Benefits

By a **single movie** we could show how our most trendy activity evolved over the months since our migration from onPrem GitLab to GitHub.com... without having ever asked anyone to fill a single timesheet. 😅
An introduction to Data Engineer Azure
Tags: beginners, tutorial
Published: 2022-12-20 06:21:23

The Azure Data Engineers are responsible for data-oriented tasks in implementation and can ingest streams and batch data, offer provisioning storage, transform data into data that meets security demands, implement retention policies for data, identify issues with performance, and solve bottleneck issues, and is able to access external data. In recent years numbers of Azure data engineers has been growing steadily. Therefore, it is essential to understand the requirements to be an effective Azure information engineer.  If you're seeking in knowing more about Microsoft Azure underwater datacenter you can find more details with [Data Engineer Course](https://www.edureka.co/microsoft-azure-data-engineering-certification-course).

For all types of engineers working in data, fundamental knowledge of programming languages, coding analytics, and management of databases are typical. The most sought-after skills are in Azure data engineer must have an understanding of the changes in the landscape of data and its effects for data engineers. They should be aware of the fundamental distinction between cloud-based solution for data as well as on-premises. He must be well-versed regarding Azure services.

Azure data engineers must have expertise in ETL tools, data warehouses and databases such as NoSQL and SQL Machine Learning, Python, Scala, and Java programming languages and Data APIs and understanding the fundamental notion of distributed systems. must have a solid understanding of algorithms and data structures.

To be a successful data engineer on Microsoft Azure which fits the Azure associate data engineer role the candidate must be proficient in Azure fundamentals. He should also be able to pass two tests to qualify for the job of a data engineer. Unexpectedly, the role-based certification may also help to be a data engineer on Azure. Azure platform.

The two most important exams required to be an Azure associate data engineer are DP-201 and DP 200. In order to pass the certification the candidate must have an appropriate year of experience in the field of data engineering and also an extensive hands-on experience with Azure.

An IT professional looking to improve and enhance his career prospects in Azure associates for data engineering will be able to study well for these tests and be successful in achieving the position. The pay for this job is the highest and is worth the effort and dedication.
Poetry v1.3 broke my pipeline
Tags: python, docker, devops
Published: 2022-12-20 05:41:47

`poetry install` abruptly exited with Error code 1 in Buildkite but you cannot reproduce it on your local machine?! It took my team around half day to track down the root cause and I hope this post can save you some time here.


![The screenshot for the odd peotry output](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jxm4s59ll1t60trdya7x.png)

For your information, we use docker to build our images for the production environment and all tests are executed inside the docker container in Buildkite. This gives us the confidence if we have an issue in our CI/CD pipeline, we can reproduce it locally. 

Apparently, we take this too far, variety of base OS (Linux for Buildkite, macOS for local) plus other hardware disparity could easily put us off. And this time, it's TTY.

Normally when you run a test locally, we tend to use an interactive shell whereas in the pipeline it is normally discouraged for sake of performance and cost. It's useless as well given it's uncommon for developers to connect to a build machine and give the build extra input. 
However, this difference can mask issues that can be exposed easily and earlier.

We initially observed an inconsistent build outcome. Due to unpinned Poetry version. Everything is fine before Poetry v1.22. But once the docker cached layer expired and the latest Poetry kicked in, you'll see a broken `poetry install`. It's so sudden that even if you turn on `-- verbose`, there's no extra insight you'll get around the stack trace.

This issue has been recorded [here](https://github.com/python-poetry/poetry/issues/7184). Essentially, the `stdlib` method used in `cleo` acts differently in the newer version and it's sneaked into Poetry v1.3 without being caught by tests. Who will come up with a test case like that?

Anyway, if you meet the same issue and want to avoid this issue in your CI/CD pipeline, please 
- use 
`poetry --quiet` or `poetry --no-ansi` if you still want this Poetry version 
- pin your Poetry to version v1.2.2

**Discussion time**
Considering we treat this as a surprise, what will you recommend to remediate similar issues in the future?
For example, do you always recommend pin Poetry and any other relevant tools in the CI/CD pipeline for a consistent build outcome or will you advocate a fast fail approach so that problems can be spotted earlier?
Leave your thoughts in the comment section because I'm keen to know your opinion!
Day 20: Skeletons are good, but can be misleading - CI for the draftjs_html Ruby Gem
Tags: ruby, programming, ci, devops
Published: 2022-12-20 06:19:00

---
title: Day 20: Skeletons are good, but can be misleading - CI for the draftjs_html Ruby Gem 
published: true
description: There are many projects that don't have working CI (maybe have a Travis-CI configuration, but that does not do anything now), and their tests fail.
tags: ruby, programming, ci, devops
# cover_image: https://direct_url_to_image.jpg
# Use a ratio of 100:42 for best results.
published_at: 2022-12-20 06:19 +0000
canonical_url: https://code-maven.com/ci-for-draftjs-html
series: ci-2022-12
---




It took me quite a few failed attempts on various Ruby Gems found on the [Ruby Digger](https://ruby-digger.code-maven.com/) till I found one that passed its tests. I reported the failures for the others, but this is getting a bit frustrating.


## Default setup

I noticed that many Ruby Gems have the same instructions to set up the development environment and to run the tests. I guess this means they used some tool to generate the skeleton of the project that automatically created the `bin/setup` shell script and the test spec files that  could be filled.

One one hand this is good as it probably encourages people to start writing test-specs.

On the other hand having a bash script to execute `bundle install` means we cannot use this on Windows natively. So for Windows we need different instructions. I think this isn't such a good idea.

## Travis-CI

It seems that the script creating the Ruby Gem skeleton also include a configuration file for Travis-CI. Unfortunately Travis-CI does not work any more for free so this file just sits there and has no purpose. It even causes confusion because it gives the feeling that the the tests worked at least once in Travis, even for Ruby projects that were created after Travis-CI stopped its public service.

## GitHub Actions

Anyway, for this particular Ruby Gem the instructions worked so I could configure GitHub Actions and send a [pull-request](https://github.com/dugancathal/draftjs_html/pull/4)

```yaml
name: CI

on:
  push:
  pull_request:
  workflow_dispatch:
#  schedule:
#    - cron: '42 5 * * *'

jobs:
  test:
    strategy:
      fail-fast: false
      matrix:
        ruby: [ '3.0' ]

    runs-on: ubuntu-latest
    name: Ruby ${{matrix.ruby}}
    container: ruby:${{matrix.ruby}}

    steps:
    - uses: actions/checkout@v3

    - name: Show ruby Version
      run: |
        ruby -v

    - name: Install Modules
      run: |
        ./bin/setup

    - name: Run tests
      run: |
        rake spec
```

## Conclusion

There are many projects that don't have working CI (maybe have a Travis-CI configuration, but that does not do anything now), and their tests fail. I am actually a bit surprised about this even though this is one of the reasons I think it is important to have a functioning CI set up, so that you will be reminded if you forget to run the tests and they break.
How a simple left join can be your biggest nightmare.
Tags: spark, bigquery, python
Published: 2022-12-20 06:13:49

What if someone ask you a basic question - You have a table item_store having stores and there corresponding item details and table item_store_old that is having legacy data for all the item's and stores, now I need to find out if item_store_old is having all the stores which are present in item_store ?

The first thought is ahh that's pretty simple I will write a query something like this 

`Select * from item_store store
 left join
 item_store_old old_store
 on store.store_id = old_store.store_id
 where old_store.store_id is null`

In case the above query return's 0 row's I can confirm that item_store_old is having all the store_id's which are present in item_store.

When I ran the above query on BQ it kept on running for 3 hrs, and I have to kill the job.

Reason - item_store_old was having 400 million records
 and item_store was having 15 million records and I was trying to join these tables which had no partition.

This is the problem statement which I faced recently at my day to day work, interested to know what all I tried before I got a working solution. I will come up with another post, let me know in the comments what all you would have tried.

**Note** the data was in the form of parquet files and I loaded those in BigQuery tables in order to query that data.
Unable to fetch firebase realtime database data into website using reactjs.
Tags: firebase, firebaserealtimedatabase, react, javascript
Published: 2022-12-20 06:09:32

I am unable to fetch data from firebase realtime database version 9 using reactjs , i can view the data in the console , but failing to render it into my website , please help...

below is my code snippet.

```
import React, {useState, useEffect} from 'react';
import  { db } from '../firebase-config';
import { Link } from 'react-router-dom';
import { ref, onValue } from 'firebase/database';




const ViewAll = () => {
  const [posts, setPosts] = useState([]);
  useEffect(() => {
    const posts = ref(db, 'superadmin/admins/')
    onValue(posts,(snapshot) => {
      const json = ({...snapshot.toJSON()})
      const keys = Object.keys(json);
      const postJSON = keys.map(key => {
        const element = json[key];
        return [element.FullName,element.doctors, element.technicians, element.patients]
      });
      setPosts(postJSON);
      console.log({...snapshot.val()})
    })
  }, []);

```

 

`
return (
    
    <div style={{marginTop: "100px"}}>
      <table className='styled-table'>
        <thead>
          <tr>
            <th style={{textAlign: "center"}}>No.</th>
            <th style={{textAlign: "center"}}>Name</th>
            <th style={{textAlign: "center"}}>Action</th>
          </tr>
        </thead>
        <tbody>
          {Object.keys(posts).map((id,index)=>{
            return (
              <tr key={id}>
                <th scope='row'>{index + 1}</th>
                <td>{posts[id].name}</td>
                <td>
                  <Link to={'/SuperAdmin/ViewAll'}>
                  <button className='btn btn-view'>View</button>
                  </Link>
                </td>
              </tr>
            );
          })}
        </tbody>
      </table>
        
    </div>
  )
}

export default ViewAll
`
Objectives Of Testing
Tags: testing, istqb
Published: 2022-12-20 06:08:46

1. Work Product Evaluation.
2. Requirement Fulfilment
3. Building Confidence in Components
4. Providing Info to stakeholders
5. Preventing Defects
6. Finding Defects
7. Reduce Risks
8. Compliance with Laws
9. Objectives may vary

Work Product Evaluation:-
Anything we develop while we are going through in order to develop the final product is called work product evaluation.

Requirement Fulfilment:-
Say a client gave us a requirement and as a tested you need to make sure that final product is same as the requirement you got from the client

Building Confidence:-
Test Not to find defect but to build confidence on the product

Provide Information to stake holders:-
Inform stake holder when there's an issue or anything which has to be reported

Preventing Defects:-
Here the main goal is to find defects and reduce the risks

Compliance with Laws:-
Follow the rule and regulations as per the requirement, Contract has to be checked

Objectives may vary:-
providing info to stake holders objectives may vary
A Love Letter
Published: 2022-12-20 05:58:05

Dear Mackachi,

I hope this letter finds you well. I just wanted to take a moment to tell you how much I love you and how much I miss you. My love for you grows stronger with each passing day and being away from you is torture.

Ever since we first met, I knew there was something special about you. Your kindness, your intelligence, and your beauty have captured my heart and I can't imagine my life without you.

I miss the way your eyes light up when you laugh, the way you always know just what to say to make me feel better, and the way you hold my hand and make me feel like the luckiest person in the world.

I miss being able to spend time with you, to see your smile and hear your voice. But even when we are apart, I carry your love with me and it fills my heart with warmth and joy.

I wanted to include a little poem with this letter to express my love for you. I hope you like it:

Your love is like a rose,
So beautiful and pure.
It fills my heart with joy
And makes my life secure.

Your touch is gentle,
Your kiss is sweet.
I love you more each day,
My love for you can't be beat.

I hope this letter finds you well and that it brings a smile to your face. Please know that you are always on my mind and in my heart.

Yours always,
Ichachan
MongoDB Interview Questions for Professional
Published: 2022-12-20 05:37:05

Introduction to MongoDB
When dealing with data, there are two types of data as we know – (i) structured data and (ii) unstructured data. Structured data is usually stored in a tabular form whereas unstructured data is not. To manage huge sets of unstructured data like logs or IoT data, a NoSQL database is used.

**What is MongoDB?**
MongoDB is an open-source NoSQL database written in C++ language. It uses JSON-like documents with optional schemas.
It provides easy scalability and is a cross-platform, document-oriented database.
MongoDB works on the concept of Collection and Document.
It combines the ability to scale out with features such as secondary indexes, range queries, sorting, aggregations, and geospatial indexes.
MongoDB is developed by MongoDB Inc. and licensed under the Server Side Public License (SSPL).
What are some of the advantages of MongoDB?
**Some advantages of MongoDB are as follows:**

MongoDB supports field, range-based, string pattern matching type queries. for searching the data in the database 
MongoDB support primary and secondary index on any fields
MongoDB basically uses JavaScript objects in place of procedures
MongoDB uses a dynamic database schema
MongoDB is very easy to scale up or down
MongoDB has inbuilt support for data partitioning (Sharding).
For more follow this link https://amlogy.com/mongodb-interview-questions-for-professional/
POST Request in Javascript with Fetch() and XMLHttpRequest
Tags: javascript, postrequest, fetch, xmlhttpreque
Published: 2022-12-20 05:35:24

To make a POST request in JavaScript, you can use the **fetch()** method or the **XMLHttpRequest** object.

Here's an example using fetch():

```
fetch('https://example.com/api/create', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    name: 'John Smith',
    email: 'john@smith.com'
  })
})
.then(response => response.json())
.then(data => console.log(data));
```

And here's an example using XMLHttpRequest:

```
var xhr = new XMLHttpRequest();

xhr.open('POST', 'https://example.com/api/create', true);
xhr.setRequestHeader('Content-Type', 'application/json');

xhr.onload = function () {
  if (this.status === 200) {
    console.log(this.response);
  }
};

xhr.send(JSON.stringify({
  name: 'John Smith',
  email: 'john@smith.com'
}));
```
Top 10 React Libraries in 2023
Tags: reactlibraries, npm, javascript, react
Published: 2022-12-20 05:28:13

React libraries are collections of code that are specifically designed to be used with the React JavaScript library. These libraries contain pre-written code that can be imported into a React project to provide additional functionality or features. Some examples of popular React libraries include React Router, React Redux, and React Bootstrap.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/z5rdg6b6fk94b2q6oho0.png)

**React Router** - a library for managing client-side routing and navigation in a React application


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vwhyuazkq4zcfbgi5u3u.png)

**Redux** - a state management library for managing complex application state in a predictable and organized manner


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yhhxxw906j8sswetzubk.png)

**Axios** - a library for making HTTP requests in a React application


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6eazin7xit1trl61q1f9.png)

**Material-UI** - a library for implementing Google's Material Design in a React application

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1vzc980qfhtzmy3qo2q3.png)

**Jest** - a testing library for React applications

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lhzwgs3ba89yzsbpb2rm.png)

**Enzyme** - a library for testing and manipulating React components

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/960geg5cshwttfxepefj.png)

**Formik** - a library for managing and validating form inputs in a React application

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1tflav9rtc2fx9ej41l7.png)

**React Bootstrap** - a library for implementing the Bootstrap framework in a React application

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/z89w6zb9ol6169bqgv23.png)

**Styled Components** - a library for writing and styling React components using CSS-in-JS

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lr0zkvz6yod1859ew8om.png)

**React-Lazyload** - a library for implementing lazy loading in a React application to improve performance.
The tilde(~) operator: the secret weapon for efficient integer conversion
Tags: javascript, programming, productivity
Published: 2022-12-20 05:15:02

In JavaScript, the tilde `(~)` is a bitwise `NOT` operator. It performs a bitwise NOT operation on its operand, which inverts the bits of the operand.

For example:

```
let x = 5;  // 00000000000000000000000000000101
let y = ~x; // 11111111111111111111111111111010
```
It's important to note that the bitwise NOT operator only works on integers. If you try to use it on a non-integer value (such as a string or a floating-point number), it will be converted to an integer before the operation is performed.

The tilde operator can also be used as a shorthand for `Math.floor()` to round a number down to the nearest integer. For example:

```
let x = 5.7;
let y = ~~x;  // y is 5
```
In this example, the double tilde operator `~~` rounds the number `x` down to the nearest integer, resulting in `y` being set to 5. This is equivalent to using the `Math.floor()` function.

Just be mindful with the size of your integers using the double NOT operator

```
// 10 digits
> parseInt(9999999999)
9999999999
> ~~9999999999
1410065407

// 9 digits
> parseInt(999999999)
999999999
> ~~999999999
999999999
```
Unable to start namenode with Hadoop 3.3.0 on WSL 2
Tags: hadoop, wsl2
Published: 2022-12-20 05:01:28

I'm running Hadoop on WSL 2, I couldn't open the namenode interface on port 9870I'm using hadoop 3.3.0

After checking service with jps, there is only secondary namenode and data node running, while the name node service is gone.

Here is my core-site.xml:

```
<configuration>
<property> 
<name>fs.defaultFS</name> 
<value>hdfs://localhost:9000</value> 
</property> <property> 
<name>hadoop.tmp.dir</name> 
<value>file:/hadoop/hadoop-3.3.0/tmp</value> 
<description></description>
 </property> 
</configuration>
```
While running format for namenode, this query showed up:
`Re-format filesystem in Storage Directory root= /hadoop/hadoop-3.3.0/tmp/dfs/name; location= null ? (Y or N)`
Then I check the log of namenode, it seems unable to read from filepath:
`org.apache.hadoop.util.ExitUtil: Exiting with status 1: java.io.IOException: Could not parse line: Filesystem 1024-blocks Used Available Capacity Mounted on`

How could I configure the namenode to make it run?
Thank you for all the help!