teleren/devto huggingface dataset:
How to Install Overlord on Linux
Tags: overlord, linux, automationtool, redteamingautomation
Published: 2021-04-26 04:27:53

In this article, we'll learn more about how to install Overlord on Linux.

<a href="https://github.com/qsecure-labs/overlord" target="_blank" rel="noopener">Overlord – Red Teaming Automation</a> provides a python-based console CLI which is used to build Red Teaming infrastructure in an automated way. The user has to provide inputs by using the tool’s modules (e.g. C2, Email Server, HTTP web delivery server, Phishing server etc.) and the full infra / modules and scripts will be generated automatically on a cloud provider of choice. Currently supports AWS and Digital Ocean. The tool is still under development and it was inspired and uses the Red-Baron Terraform implementation found on Github.

A demo infrastructure was set up in our blog post <a href="https://blog.qsecure.com.cy/posts/overlord/" target="_blank" rel="noopener">https://blog.qsecure.com.cy/posts/overlord/</a>.
<p style="font-size: 17px; font-weight: 500;"><a href="https://www.hostnextra.com/kb/how-to-install-overlord-on-linux/">Install Overlord on Linux</a></p>
Let's get started with the installation.

Run following commands to install Overlord on linux:
```
git clone https://github.com/qsecure-labs/overlord.git
cd overlord/config
chmod +x install.sh
sudo ./install.sh
```

<h4>Advanced Configuration</h4>

<strong>Installation Templates</strong>

In the c2 module the user has the ability to install tools from a list. The tool automatically loads the scripts from the <em>./redbaron/data/scripts/tools</em>. By adding a new script in the directory, you can install by the <em>tools</em> variable in the c2 module. We encourage though to use the ansible module.

<strong>Default Configuration File</strong>

The <em>./config/config.json</em> file contains the default configuration on each module and the providers that are used by overlord. By changing each object, you can customize the default values of each module when it loads to Overlord.
   ```
// Default
    "mod_c2": {
        "module": "c2",
        "type" : "http",
        "redirectors": 1,
        "tools": [],
        "region": "LON1",
        "provider": "digitalocean",
        "size": "s-1vcpu-1gb",
        "id": ""
    }

    // Customized
    "mod_c2": {
        "module": "c2",
        "type" : "http",
        "redirectors": 0,
        "tools": ["metasploit","empire"],
        "region": "us-east-2",
        "provider": "aws",
        "size": "t2.nano",
        "id": ""
    }
```
In this article, we have seen more about how to install Overlord on Linux.

Check out our high performance <a href="https://www.hostnextra.com/dedicated-server.html" target="_blank" rel="noopener">dedicated servers</a> and <a href="https://www.hostnextra.com/vps-hosting.html" target="_blank" rel="noopener">KVM VPS</a>.
Python 'for'
Tags: python
Published: 2021-04-26 04:08:40

stocks = []
    get_stocks = item_infos.select('.col3mobile > input')
        
    for get_stock in get_stocks:
        stock = get_stock.get('value')
        stocks.append(stock)

    print(stocks)

[ '40', '1', '0', '41', '1', '0', '42', '1', '0', '43', '2', '0', ' 44 ','1 ','0 ','45 ','1 ','0 ','46 ','1 ','0 ']

I only need the result of the 1st, 4th, 7th, 10th, and 13th......elements in the result. How do I write "for"?
for example 
for example i want result 
['1' , '1', '1', '2', '1', '1', '1'] 
!!! The result of the list is always changing.!!!
SPRING BOOT
Tags: java, webapplication, configuration, springboot
Published: 2021-04-25 09:04:19

**Spring Boot** is an open-source **Java framework** that was developed by **Pivotal in 2014** and simplifies the task of deploying Java enterprise web applications. It is a project built on top of the Spring framework that provides an efficient way to set up and run applications.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kcix84l2oiozr2hftm60.png)
Spring Boot consists of the **Spring Framework** and **Embedded Servers.** Because it uses convention over configuration software design, it does not require XML configuration.

**Spring Boot was created to achieve the following goals:**

Simplify the process of developing production-ready Spring 
  applications.
Avoid XML configuration in Spring.
Reduce development time by decreasing the number of needed 
  import statements.
Provide an opinionated development approach.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/srh4n9a5b9jgjt2kz0sk.jpg)

**Advantages of using Spring Boot:**

Creates stand-alone Spring applications that only need to be 
  started using Java -jar.
Provides features for products such as health checks and 
  externalized configuration.
Provides a CLI tool for developing and testing your application
Rapidly increases productivity.
Reduces development time.
Easy to learn and build spring applications.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9aecwff4qtue02aqcp6e.jpg)

**Limitations of Spring Boot:**

Sometimes, Spring Boot will use dependencies that are not actually used in the application. This can increase the size of your application.
Though Spring Boot comes with some basic tools for logs and your app health monitoring, these aren’t sufficient. Tools like Retrace help teams monitor Java apps with ease. This tool helps to detect slow SQL queries, provides performance and CPU usage reports, and shows the most common errors by interpreting the logs.
What Software Engineering Universities Don't Teach You
Tags: productivity, management, codenewbie, startup
Published: 2021-04-26 03:22:19

With almost 10 years of working experience, I played several roles as a software engineer. And I am currently CTO of a video game company. Thinking back about my time at university, there are tons of subjects I wish I had been taught properly.  Here is my list... 

## Time and task management ⏲️

- Partitioning time (ex Pomodoro) 
- Avoid distractions, getting into the flow state
- Inbox Zero

## Tooling 🛠️

- Hardware: computers, input devices, and network devices
- Tools to increase productivity, shortcuts
- Shell: bash, zsh, vi, or others
- Mastering IDE
- Test framework and test automation
- IT: resources inventory, resources management

## Communication 💬

- With your technical colleagues
- With your non-technical colleagues
- With clients and customers: be confident and assertive
- Marketing: Self and Company promotion

## Project Management 💼

- Agile: Scrum, Kanban
- Stories, task definition 
- Task estimation
- TDD, DDD
- Performance monitoring

## Leading / Mentoring 🦸

- Onboarding people
- Mentoring people
- Documentation
- Code reviews
- Peer reviews, Pair programming

## Management 👨‍💼

- 1-on-1
- Delegation
- Feedback and Performance reviews
- Career management, coaching
- Personalized communication based on DISC assessment or others
- Objectives definition: OKR, KPI, SMART, etc.

## Budget Management 💰

- Definition of budget
- Expenses monitoring
- Accounting

## Entrepreneurship 🏢

- Corporate governance and the basics of corporate law
- Business development
- Business plan
- Business monitoring

Do you agree? Share your comments below. I also invite you to make your list and share it as well.  

-----------------------------------------

Photo by <a href="https://unsplash.com/@awcreativeut?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Adam Winger</a> on <a href="https://unsplash.com/s/photos/boy-reading?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>
从国内访问返回503错误
Tags: gfw
Published: 2021-04-26 03:20:49

做实验的时候访问一个服务返回503错误，查了好一会儿，没发现问题，后来想到应该尝试从国外直接访问，结果正常。后来发现去掉 http_proxy 变量就好了，http_proxy 指向了翻墙用的代理
Do you need express for a Airtable project? My .env file doesn't work. 
Tags: react, cms, help, devops
Published: 2021-04-26 03:12:51

I’m creating a Airtable project, that I use CRA, and then I add Airtable and Styled Components. Then I create the .env file, but it’s not connecting the AIRTABLE_API_KEYS to where there suppose to go.
12 Simple CSS Hover Effects
Tags: css, beginners, codenewbie, webdev
Published: 2021-04-26 03:12:13

Here is the list of 12 simple beginner level CSS menu button hover  animation, it use simple CSS animation techniques. I tried to explain how the animation works as comments inside the code. It is not made responsive and the styling of components is minimal so as to avoid complexities. I just want to show that these techniques can be used as hover effect, use your creativity to make it awesome. If you find any errors let me know, Happy Coding.

{% codepen https://codepen.io/kiran-r-raj/pen/yLgZmJp  %}
{% codepen https://codepen.io/kiran-r-raj/pen/abpXeBP default-tab=result %}
{% codepen https://codepen.io/kiran-r-raj/pen/poRGMPa default-tab=result %}
{% codepen https://codepen.io/kiran-r-raj/pen/MWJLNoX default-tab=result %}

{% link https://dev.to/kiranrajvjd/19-simple-hover-animation-using-css-1l36 %}
{% link https://dev.to/kiranrajvjd/10-simple-navigation-bar-hover-animations-1980 %}
{% link https://dev.to/kiranrajvjd/awesome-css-layout-resources-for-beginners-508a %}
{% link https://dev.to/kiranrajvjd/15-awesome-css-button-generator-websites-17hj %}
{% link https://dev.to/kiranrajvjd/10-best-javascript-based-github-repos-for-web-developers-49pc %}
Converting Markdown to Epub or Mobi 
Tags: pandoc, kindle, ebook
Published: 2021-04-26 03:09:55

---
title: Converting Markdown to Epub or Mobi 
published: true
description: Converting Markdown to Epub or Mobi 
tags: pandoc, kindle, ebook
cover_image: https://themythicalengineer.com/assets/images/converting-markdown-to-epub-mobi/workflow.png
---
In this blog post, I will show you the process of combining and converting a list of markdown files into kindle supported ebook (.mobi) format. 

In this example I would be converting official documentation of Docker website into an ebook for offline use.

I will use Pandoc to combine and convert the markdown (.md) files into .epub format. Then I will use Calibre to convert it to .mobi format.

![workflow](https://themythicalengineer.com/assets/images/converting-markdown-to-epub-mobi/workflow.png)

You can follow the steps and modify the linux commands according to your needs.

If you don't have a linux machine, you can use [Deepnote](https://deepnote.com/) platform to run linux commands in jupyter notebook.

<small>Note: If you're running this in Deepnote, you need to append "!" as prefix to each command, so that deepnote can identify it as bash command. If running in linux machine, remove "!" from all commands written below.</small>

Here is a small video tutorial of the working script.
Video Link: https://www.youtube.com/watch?v=L4hGDPLMPcw

Steps to be followed:

```bash
# Download Pandoc executable
!wget https://github.com/jgm/pandoc/releases/download/2.11.3.2/pandoc-2.11.3.2-1-amd64.deb
```

```bash
# Install pandoc
!sudo dpkg -i pandoc-2.11.3.2-1-amd64.deb
```

```bash
# Clone Docker CLI documentation github repository
!git clone https://github.com/docker/cli.git
```

```bash
# Verify the list of markdown files and contents of files
!cd cli/docs/reference/commandline && ls -la
```

```bash
# Generate ebook from markdown files, *.md picks up all the files with .md as extension
!cd cli/docs/reference/commandline && pandoc -o docker_cli.epub --metadata title="Docker CLI Docs" *.md
```

```bash
# Move generated file to required location
!mv cli/docs/reference/commandline/docker_cli.epub ~/work/docker_cli.epub
```

```bash
# Install required dependencies for Calibre
!apt update -y
!apt install libgl1-mesa-glx -y
```

```bash
# Download and install calibre command line utility
!wget -q -O- https://download.calibre-ebook.com/linux-installer.sh | sudo sh /dev/stdin
```

```bash
# Convert ebook to kindle supported .mobi format
!ebook-convert docker_cli.epub docker_cli.mobi
```

Now you can transfer generated .mobi file to your kindle device.

There is also a browser extension called [EpubPress](https://epub.press/), which can convert any webpage into a kindle book in a single click. You can use it convert any webpage or blogs to read on Kindle.

You can follow the discussion thread on [Hackernews](https://news.ycombinator.com/item?id=26898591)

*Originally published at [https://themythicalengineer.com](https://themythicalengineer.com) on Apr 21, 2021.*
Handle async code in node.js using PROMISES
Tags: javascript, node, promise
Published: 2021-04-26 02:46:07

Today we going to take a look at how to handle async code in nodejs using Promises.


While doing projects, we all have to use promises at some point of time sooner or later. regardless of what type of API we use, (whether it is 3rd party API like mentioned above or your own back end API) Promises are always *Swiss army knife's* for us.


As of now, many HTTP libraries gives us the elegant way to handle async code using promises, the **Axios** is my preferred candidate. Because, it gives us the flexibility and provides built-in support for many things like promises. 


While handling promises, it primarily have to three states of of execution where it lives once we started Promise execution.

**Pending**  -->  Currently executing 
**Resolved** -->  Executed successfully
**Rejected** -->  Failed to Execute

```javascript

let fakeAsyncCall = (ms) => {
  return new Promise((resolve, reject) => {
   try {
    setTimeout(() => {
      console.log("executed at", Date());
      resolve(true);
    }, ms);
   } catch (error) {
     console.log("err", error.message)
     reject(false)
   }
  });
};
 
```

Above snippet returns the value as true by using the callback function resolve the promise if the timeout executed successfully else it will reject it with false value.

This is the basic implementation of promises, we can replace setTimeout() with any API call. 

```javascript
//calling fakeAsyncFunction()

fakeAsyncCall()
.then( data => console.log("success"))
.catch(err => console.log("Promise failed"))

``` 

This is how we execute handelAsynCall(). As it returns the Promise object, we are handling Promise using *then.... catch* block.
(we can use async --- await also) 

So, then() block executes if promise is resolved successfully otherwise catch() block is executed if any exception occurs.

--------------------------------

# ** Final Thoughts **

I hope this post will help someone to gain better understanding regarding promises.

if you liked the post, please consider sharing it.

Thank you
NestJS & Heroku Boilerplate
Tags: nest, javascript, heroku, boilerplate
Published: 2021-04-26 02:21:19

I spent this evening creating a clean boilerplate project to combine: 

- [NestJS](https://nestjs.com/)
- [Heroku](https://heroku.com/)
- Hot Reload Feature

I started by following the official [Nest Guide](https://docs.nestjs.com/first-steps) and then googled to setup Nest correctly with Heroku. I had to adjust some scripts in the `package.json`, add a `Procfile` file as well as setup webpack for hot-reload during local development.

I hope this saves someone time. Here you go: [Github Repo](https://github.com/mathiasklenk/nest-heroku-boilerplate)
LeetCode 46. Permutations (javascript solution)

Tags: algorithms, javascript
Published: 2021-04-26 01:56:17

###Description:
Given an array nums of distinct integers, return all the possible permutations. You can return the answer in any order.

###Solution:
Time Complexity : O(n!)
Space Complexity: O(n)

```javascript
var permute = function(choices, temp = [], permutations = []) {
    // Base case 
    if(choices.length === 0){
      permutations.push([...temp]);
    } 
    
    for(let i = 0; i < choices.length; i++){
        // Create new array without current letter
        let newChoices = choices.filter((choice, index) => index !== i)
        // Add current to the temp array which is our current permutation
        temp.push(choices[i])
        permute(newChoices, temp, permutations)
        // Once we have explored options remove the current letter from our current permuataion
        temp.pop()
    }
    
    return permutations
};
```
Using LocalStorage to improve your webpage
Tags: webdev, javascript, html
Published: 2021-04-26 01:55:44

Localstorage is a simple way to save user data on your website, by letting them save it to their browser.  All it takes is a little JSON and you'll be on your way to a friendlier user experience.  

# Getting input
Once you've got some input from a user via Form or browsing data, You'll need to save that to the browser.  Localstorage manages its storage based on a key value pair.  So take the array or object your data is stored in, assign it to a variable, and stringify it with JSON.  The save operation will look something like this 
```
localStorage.setItem("keyAsAString", JSON.stringify(myDataObjOrArray));
```
You can decide when the localStorage should be updated, but generally, anytime the user resubmits a form or some kind of data, its a good idea to call this method again.  

# Retrieving values
When the user comes back to the page, check local storage for the data. Make sure to parse it using JSON or you will just get a string.  
```
data = JSON.parse(localStorage.getItem("keyAsAString)) || []
```
It's very helpful to add a default if this fails, as I did in the above example. 

That's all there is to localstorage.  Where have you used this in a personal project?
LeetCode 208. Implement Trie (Prefix Tree)
(javascript solution)
Tags: algorithms, javascript
Published: 2021-04-26 01:48:04

###Description:
A trie (pronounced as "try") or prefix tree is a tree data structure used to efficiently store and retrieve keys in a dataset of strings. There are various applications of this data structure, such as autocomplete and spellchecker.

Implement the Trie class:

Trie() Initializes the trie object.
void insert(String word) Inserts the string word into the trie.
boolean search(String word) Returns true if the string word is in the trie (i.e., was inserted before), and false otherwise.
boolean startsWith(String prefix) Returns true if there is a previously inserted string word that has the prefix prefix, and false otherwise.

```javascript
/**
 * Initialize your data structure here.
 */
var Trie = function() {
    this.root = {};
};

/**
 * Inserts a word into the trie. 
 * @param {string} word
 * @return {void}
 */
Trie.prototype.insert = function(word) {
    let node = this.root;
    for(let letter of word) {
        if (node[letter] === undefined) node[letter] = {};
        node = node[letter]
    }
    node.isEnd = true;
};

/**
 * Returns if the word is in the trie. 
 * @param {string} word
 * @return {boolean}
 */
Trie.prototype.search = function(word) {
    let node = this.root
    for(let letter of word) {
        // check if current letter is in the node
        if(!node[letter]) {
            return false;
        } else {
            node = node[letter];
        }
    }

    return node && node.isEnd === true;
};

/**
 * Returns if there is any word in the trie that starts with the given prefix. 
 * @param {string} prefix
 * @return {boolean}
 */
Trie.prototype.startsWith = function(prefix) {
    let node = this.root;
    for(let letter of prefix) {
        if(!node[letter]) {
            return false;
        } else {
            node = node[letter];
        }
    }
    return true;
};
```
Audit Your Website!
Tags: seo, a11y, webdev, performance
Published: 2021-04-26 00:52:21

Today I will be auditing the website of a Penn State Student Organization I have proudly dedicated over 3 years to, Engineering-Consulting Collaborative. 

[Web.dev](https://web.dev/) (aka Lighthouse) and [WAVE](https://wave.webaim.org/) are the services I will be using to analyze the accessibility, SEO, and performance of [http://eccpsu.weebly.com/](http://eccpsu.weebly.com/). 

#Accessibility / a11y
Providing and supporting access to the web and technologies regardless of ability, connection quality, device capabilities, or human capabilities. 

#Search Engine Optimization / SEO
The discoverability of a website, based on how high it ranks on Google, Bing, and other search engines. This also comes into play with how often it shows up on social media feeds. Good SEO means more traffic. 

###Check out the [video](https://youtu.be/gNSzi8REJlQ) of the results of the audit, where I also share the steps we plan to take to improve the website.
A Custom React Hook that handles duplicate API call
Tags: react, hook, javascript, webdev
Published: 2021-04-26 00:39:44

Today, we are going to create a custom hook that solve this problem:
* We have multiple components that calling same API.
* Component may not always be on same page, therefore, they have to call the API by themselves.

Here is a example hook I think can handle this problem:

```javascript
let isCalling = new Map(); // This can be replace by some global state. I use this for the sake of simplicity.
function useFetch(url) {
  const [data, setData] = useState("");
  useEffect(() => {
    let isThisAPICalling = isCalling.get(url);
    if (!isThisAPICalling) {
      console.log("new");
      isThisAPICalling = fetch(url).then((response) => response.json());
      isCalling.set(url, isThisAPICalling);
      // The body can only be parsed once.
    }
    isThisAPICalling.then((json) => {
      console.log("done");
      console.log(json.title);
      isCalling.delete(url);
      setData(json.title);
    });
  }, []);
  return data;
}
```

Here is a demo.

{% codesandbox o9sxf %}



Thank you all! Please let me know if you have any suggestion.
How to convert an HTML string into real HTML or DOM using JavaScript?
Tags: javascript
Published: 2021-04-26 00:00:00

---
  title: How to convert an HTML string into real HTML or DOM using JavaScript?
  published: true
  tags: JavaScript
  date: Mon Apr 26 2021 05:30:00 GMT+0530 (India Standard Time)
  canonical_url: https://melvingeorge.me/blog/convert-html-string-into-real-html-or-dom-javascript
  cover_image: https://melvingeorge.me/_next/static/images/main-e34754b466f7b92f5ae0f3ab9fa3678e.jpg
  ---
  [Originally posted here!](https://melvingeorge.me/blog/convert-html-string-into-real-html-or-dom-javascript)
  
To convert an HTML string into real HTML or DOM, you can use the `DOMParser` Web API using JavaScript. The `DOMParser` helps us to parse HTML or XML string into real Document or DOM nodes.

### TL;DR

```js
// html string
const htmlStr = "<h1>Hello World!</h1>";

// make a new parser
const parser = new DOMParser();

// convert html string into DOM
const document = parser.parseFromString(htmlStr, "text/html");
```

For example, let's say you have a HTML string of `h1` tag with the text of `Hello World!` like this,

```js
// html string
const htmlStr = "<h1>Hello World!</h1>";
```

Now, to convert this string into a real HTML tag we can use the `DOMParser` web API.

So first, we have to make a parser using the `new` keyword like this,

```js
// html string
const htmlStr = "<h1>Hello World!</h1>";

// make a new parser
const parser = new DOMParser();
```

After that, we can use the `parseFromString()` method in the `parser` object and pass:

- the raw HTML string as the first argument
- and the `mime` type or the type of the document contained in the string as the second argument. In our case, the mime-type value is `text/html`.

There are other mime types we can use such as:

- `text/xml`
- `application/xml`
- `application/xhtml+xml`
- `image/svg+xml`

So it will look like this,

```js
// html string
const htmlStr = "<h1>Hello World!</h1>";

// make a new parser
const parser = new DOMParser();

// convert html string into DOM
const document = parser.parseFromString(htmlStr, "text/html");
```

Now the HTML string is converted to an HTML DOM node. You can now use the usual methods and properties available on a DOM node such as `appendChild()`, `classList`, etc.

See the above code live in [JSBin](https://jsbin.com/zabahepozo/2/edit?js,console).

That's all 😃!

### Feel free to share if you found this useful 😃.

---