teleren/devto huggingface dataset:
Download a webpage and all of its assets using wget
Tags: linux
Published: 2022-01-26 02:30:32

---
title: Download a webpage and all of its assets using wget
published: true
description: Download a webpage and all of its assets, even when the assets aren't on the same domain.
canonical_url: https://tinkerlog.dev/journal/downloading-a-webpage-and-all-of-its-assets-with-wget
tags: linux
---

A friend of mine has a single-page website that he hasn't updated in over a year. He'd like to keep the website, but he'd _also_ like to save $20 a month. I told him I could probably help him get it on Netlify since he never changes it.

I needed a way to download the page with all of its assets. Chrome's "Save as..." menu option wasn't working: it wouldn't download content from the CDN because it was on a different domain. I thought `wget` might be a good option.

Here is the command I ultimately ended up using:

```sh
wget \
  --page-requisites \
  --convert-links \
  --span-hosts \
  --no-directories \
  https://www.example.com
```

To go through the arguments one-by-one:

- `--page-requisites` downloads the images, css and js files
- `--convert-links` makes the links "suitable for local viewing," whatever that means (thank you, `man` page)
- `--span-hosts` is the magic here: this tells `wget` to download the files from different hosts like the CDN
- `--no-directories` downloads the files into a single flat and messy directory, which is perfect for my needs

If you open `index.html` the assets will be broken: `--convert-links` doesn't seem to make these relative to the root directory. So to view the page, you'll need to start a webserver in the download directory. You can use the following command:

```sh
python3 -m http.server
```

The output is pretty messy, and it might be quicker to just build something with Tailwind than clean this download up. Even still, there are times when this could be useful.
Getting started with Git day 01.
Tags: git, beginners
Published: 2022-01-26 02:28:59

Three days before I have started learning git.
In **day 01** I have learned basic commands used in git

- git init

- git status

- git add **filename**/./--all

- git commit -m "message"

- git log


---------------------------------------------------
-------------:)-have a good day--------------------
100 Days of Learning - Day 15
Tags: 11ty, netlify
Published: 2022-01-26 02:13:57

---
series: 100 Days of Learning
---

Hello and welcome to my 100 Days of Learning Challenge! 

In the last few weeks, I've explored vanilla Javascript, learning SQL for work, and video editing, but this week I'm shifting my focus.

**I'm going to build a personal blog using Eleventy!**

Since I already host my personal site on Netlify, I thought using 11ty would be a great fit. I want to create my own platform for sharing blog posts, especially as I explore non-tech related topics that I don't think would be appropriate or relevant to share here on DEV.

I've tried to get an 11ty blog set up in the past, but I struggled to get it into a usable state. This is mainly because 1) I had no idea what I was doing and 2) I didn't know what I didn't know or where to get started. Looking back, the biggest blockers were probably not understanding how an 11ty project is structured, and then not knowing how to make it look good.

There's some knowledge gaps I've had to fill, and I'm in a much better place to tackle this project. I'm more comfortable with Git, deploying projects to Netlify, setting up dev environments, and learning how to learn. It also helps that there are even more resources for learning how to set up a blog with 11ty now.

I was even considering writing my own series on getting started with 11ty, but then I came across @raymondcamden's comprehensive guide to getting started in 11ty that he just posted about a week ago. Truly perfect timing! I will probably also take a stab at writing my own guide, since there are some things I'd like to highlight as a true beginner while they're fresh on my mind.

My plan for this project is to do some research (in progress) and make sure I understand the things that caught me up the last time I attempted this, and I'll get to building this out tomorrow. I intend to have something working, and looking pretty, by next week.

**Here are some of the resources I'm using:**
- [A Complete Guide to Building a Blog with Eleventy](https://cfjedimaster.github.io/eleventy-blog-guide/guide.html)
- [11ty Tutorial: Cranking Your Jamstack Blog Up to 11!](https://snipcart.com/blog/11ty-tutorial) (I didn't realize this tutorial was also by Raymond!)
- [Raymond Camden's Blog](https://www.raymondcamden.com/2022/01/19/a-guide-to-building-a-blog-in-eleventy)
- [11ty Docs]()
- [Let's Learn Eleventy! | Learn with Jason](https://www.learnwithjason.dev/lets-learn-eleventy)

-----
Header by <a href="https://unsplash.com/@craftedbygc?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Green Chameleon</a> on <a href="https://unsplash.com/s/photos/writing?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>
Day 516 : The Setup
Tags: hiphop, code, coding, lifelongdev
Published: 2022-01-26 00:49:09

_liner notes_:
- Professional : Had my weekly meeting with my manager. Spoke about different projects I've got coming up in like the next month. Then we had a company wide meeting. Cool to hear about the direction the company is going in. Helped troubleshoot a question in the community. Then I worked on creating some Web Components. I got a couple done and started working on converting another, but ran into a weird issue when I tried to incorporate another UI library. I looked it up and turns out that it's a known issue, and there's no solution. I'll have to see if I can find a workaround tomorrow.

- Personal : Last night, I worked out a new layout for the mobile view for the livestream on my side project. So now, the video player stays at the top and the user can swipe left or righ on the bottom to see the controls or the chat. First I had it to were it when you slide to see the chat, it would move the video player out of the way. Then I thought, maybe people would want to be able to view the video and chat at the same time. After that, I watched the latest "Attack on Titan" and a couple of episodes of "Demon Slayer".


![View looking out into the ocean from a cove in Phuket, Thailand with mountains almost encircling the area with a large rock formation covered with ferns sticking out of the ocean](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qppbn4l41y6g0mm6u3bm.jpg)

Tonight, I'm thinking I'll start working on the scenes for the livestream part of my side project. One includes letting the viewer add themselves to a map that they can see update live in the video feed. I'll work on getting that set up and then probably watch some more anime in my mini theater. Been really enjoying it. I even ordered an extension arm that I will attack to my van frame so I can just pull the projector down and just start watching. Trying to perfect the setup.

Have a great night!

peace piece
Dwane / conshus
https://dwane.io / https://HIPHOPandCODE.com
{% youtube A3_syznHL5Y %}
echo3D interviews with Columbia Engineering
Tags: metaverse, engineering, virtualreality, augmentedreality
Published: 2022-01-26 00:41:17


![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1643155839606/IzIJgiaOD.png)

Check out this latest interview with [echo3D](https://www.echo3d.co/)’s CEO Alon Grinshpoon and his alma mater, Columbia Engineering, about the Metaverse and our company’s entrepreneurial journey at echo3D.

Read the article and watch the videos it includes on how [echo3D](https://www.echo3d.co/) got started, where is the AR and VR industry heading, and how to start your own startup:

Read it [here](https://www.linkedin.com/pulse/metaverse-more-than-latest-buzzword-columbia-engineering/).

How Grinshpoon and echo3D Got Started

{% youtube Woju7XvrWlc %}

Where is the AR and VR Industry Heading?

{% youtube iXFkNgyQ5yw %}

What Challenges did echo3D face?

{% youtube ti0Mxxr13no %}

Advice for getting started

{% youtube lwC9b8EzbzM %}

Follow [Columbia Engineering on LinkedIn](https://www.linkedin.com/company/columbia-engineering/) and share [their original post](https://www.linkedin.com/feed/update/urn:li:ugcPost:6890342509831413760).


![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1643156082202/h_RYsjUy7.png)

---
>_**echo3D** ( [www.echo3D.co](https://medium.com/r/?url=http%3A%2F%2Fwww.echo3d.co%2F); Techstars 19') is a cloud platform for 3D/AR/VR that provides tools and network infrastructure to help developers & companies quickly build and deploy 3D apps, games, and content_

![14.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1642125660137/aejgid-pv.png)
How To Create Files Faster in Visual Studio Code With a Command
Published: 2022-01-26 00:16:33

Creating files using a mouse on Visual Code seems like a lot of work so I mostly use UNIX commands; `mkdir` to make directories and `touch` to make files.

 But the `touch` command only works when the file directory(s) (destination) exist(s).

So we will create a bash-alias to create a file and create the directory(s) from the path if they do not exist.

For instance:

```bash
[command] src/components/App.ts

# should create
- src/
  |
	-- components/
		 |
		 -- App.ts
```

- create a shell script and paste this code into it:

```bash
#!/bin/bash

function t() {
    regex="([^/]+\/)"
    f=$1
    temp="./"

while [[ $f =~ $regex ]]; do
    name="${BASH_REMATCH[1]}"
    temp=$temp$name
    mkdir $temp -p
    f=${f#*"${BASH_REMATCH[1]}"}
done

touch $temp$f    
}
```

- enable running commands from the file by adding it to bash aliases using this command below:

```bash
source ./path/to/your_bash_alias_file

# mine, for instance
source /temp/scripts/my_bash_alias.sh
```

- Now, we can make use of `t` to created files and automatically create the paths to the file even if they do not exist:

```bash
t src/components/App.js
```

## Conclusion

The alias we created above is temporal only for testing the script out. To create an alias permanently add the alias to your `.bashrc` file. 

And I also published an [npm](https://www.npmjs.com/package/touch-alias) package to make use of the `t` command. by installing it globally with

`npm install-g touch-alias`.

Please, kindly star the [repository](https://github.com/zt4ff/touch-alias) if you find this helpful and leave a comment.
How I configure my echo dot Alexa to turn on my computer
Published: 2022-01-26 00:13:47

## Introduction

I recently got the echo dot Alexa ([Echo Dot](https://www.amazon.com.br/Echo-Dot-3%C2%AA-Gera%C3%A7%C3%A3o-Cor-Preta/dp/B07PDHSJ1H)) for my birthday and I decided to play with Alexa trying to customize commands.

One of the commands that I found super cool on the internet was to turn on my computer using Alexa, so here is the step-by-step how I did it:

#### Considerations

I configured my Alexa using `windows`, if you want to configure using another operating system you can use this tutorial as well.

## Steps

1. Enter in [Wake on Lan Alexa Skill (wolskill.com)](https://www.wolskill.com/) and login in with amazon account

2. Create a new device in wolskill.com
- Add a name for your computer 
- Add the mac address the computer.

You can get the mac address, on `Windows` you need to go to `Settings > Network & Internet > Ethernet`. It's the option “Physical address (MAC)”

3. Add “AddWake on Lan (WoL)” as a skill into your Alexa using the app ([Amazon.com: Wake on Lan (WoL): Alexa Skills](https://www.amazon.com/Oscar-Penelo-Wake-Lan-WoL/dp/B07PGKK416))

4. All the configuratio are done, now to test “Alexa, turn on hopper”, the name_computer is the name you configured in step 2.
I choose Hopper to honor [Gracie Hopper](https://pt.wikipedia.org/wiki/Grace_Hopper) that contributed to advancing computer science as we know it today.

hope you can configure your Alexa too and impress your crush or your family, lol.

Playlist listen when I was writing this tutorial:

[https://open.spotify.com/playlist/7h0WH8cs8z76njzNN11fY6?si=8b3ca6ce1c4a430c](https://open.spotify.com/playlist/7h0WH8cs8z76njzNN11fY6?si=8b3ca6ce1c4a430c)

That's all folks
GitHub Codespaces creation with the CLI
Published: 2022-01-27 03:04:56

---
title: GitHub Codespaces creation with the CLI
published: true
date: 2022-01-26 00:00:00 UTC
tags: 
canonical_url: https://devopsjournal.io/blog/2022/01/26/GitHub-Codespace-creation-with-the-cli
---

This one took me some time to figure out, so I wanted to have something for the next time I need it :-). I needed to create repos for a team and then create a codespace for them. I already was using the [GitHub CLI](https://github.com/cli/cli) for my automation, so I wanted to use it as well to create the codespace. The documentation states that you can call [gh codespace create flags](https://cli.github.com/manual/gh_codespace_create) and you are good to go.

Well, this command in the CLI works **differently** then you might think: First of, most of the commands work and pickup the repo that you are currently in. This command does not: you need to pass in the `-r` parameter and specify the repo using the `<owner/repo>` notation.

Next up is the `-m` parameter to specify the machine you want to have as the compute option for the Codespace. In the [REST API](https://docs.github.com/en/rest/reference/codespaces) you do not need to specify this parameter and it will give you the type of machine in the response:

```
"machine": {
    "name": "standardLinux",
    "display_name": "4 cores, 8 GB RAM, 64 GB storage",
    "operating_system": "linux",
    "storage_in_bytes": 68719476736,
    "memory_in_bytes": 8589934592,
    "cpus": 4
},

```

The help for the CLI command doesn’t give any hints as well.

So, I tried to be smart and send in this json as the value, thinking they’ve missed that wrapper in the CLI. That fails with no extra information, probably due to some wrong string escaping on my end.

Then I entered ‘standardLinux’ as the value, thinking this will never work, since I want to create it with a certain `size`, and all flavors are Linux based….

This is then the response:

```
 gh codespace create -b main -s -r "globaldevopsbootcamp/team01" -m "standardLinux"

error getting machine type: there is no such machine for the repository: standardLinux
Available machines: [basicLinux32gb standardLinux32gb premiumLinux largePremiumLinux]

```

Finally some hints of the expected values! (even better: these are the expected values 😄).

Here you can see how those values match to a machine type:

![Screen shot of the UI options how they work towards the values from the commandline: they match! The sequence from top to bottom in the UI match the sequence of the command line response](https://devopsjournal.io/images/2022/20220126/20220126_UI_Options.png)

# Tips

1. Keep in mind that the API for Codespaces is still in beta.
2. Do note that for automating these API calls you cannot use a GitHub App: the scope for the Apps is not available (yet?). You can make this work with an OAuth App or a Personal Access Token (PAT).
How to create a new project or an app in Nestjs?
Tags: resource
Published: 2022-01-26 00:00:00

---
  title: How to create a new project or an app in Nestjs?
  published: true
  tags: Resource
  date: Wed Jan 26 2022 05:30:00 GMT+0530 (India Standard Time)
  canonical_url: https://melvingeorge.me/blog/create-new-project-nestjs
  cover_image: https://melvingeorge.me/_next/static/images/main-cc0578fa7d9b26bb4e085d66fb9a3fa9.jpg
  ---
  [Originally posted here!](https://melvingeorge.me/blog/create-new-project-nestjs)
  
To create a new project in Nestjs, you can use the Nestjs CLI command called `nest` followed by the keyword `new` then the name of the project you need to use.

For example, let's say we need to make a project called `todo` in Nestjs, so we can use the `nest` command like this,

```bash
# Create a new Nestjs
# project called `todo`
nest new todo
```

**Note: To know more about installing the Nestjs CLI see the blog on [How to install Nestjs CLI?](https://melvingeorge.me/blog/install-nestjs-cli).**

After executing the above command you can see that a directory called the `todo` is created and inside the directory, you can find the default boilerplate files and folders required for the Nestjs app to run.

See the default Nestjs app live on [codesandbox](https://codesandbox.io/s/default-nestjs-app-l4sj6t?file=/src/main.ts).

That's all 😃!

### Feel free to share if you found this useful 😃.

---
The first time I printed "Hello, world!"
Tags: javascript, beginners, webdev, help
Published: 2022-01-25 23:30:47

It was 2017, when I started using a computer for the first time in my life. That was the first time I found out what is internet and how it works. And the only website I knew was Google, lol.

**"HOW DID PEOPLE MAKE GOOGLE?"**, I used to ask myself this always.

My curiosity started then. I had no computer, no smartphone, I was just there. I only accessed computer using my school computers. It was also too hard because there was many students who needed to use the computers as I was doing. I used to go there in computer lab, and sat looking at other people watching videos on YouTube, doing their stuffs on Facebook, and I could not believe my eyes, because before that I didn't know any of that websites or even what website was.

And after a few days, I started using the computer myself, by looking at how others are doing, I could also navigate the internet and type in some keywords in Google and click "Enter". This is when my life changed in a way that I could not even remember who I was.

After a few days, I started searching, "How can I make my own website?" And that day, I printed my first "Hello, World!" I was like, "Wow, now I can make a website myself," with a big smile on my face "I can even make Google."

```
<html>
<head>
<title>My Website</title>
</head>
<body>
      <p>Hello, World!</p>
</body>
</html>
```

**So funny, right? I thought so.**

I started teaching myself about the web development using HTML and CSS. It was going so smooth and after a few days, one month, I was fluent with HTML and I could make a simple HTML static websites.

I didn't know about JavaScript or PHP, I thought I had learnt it all. I let my guard down, and I passed more than one year without doing anything else.

In 2020, I started again, I became a JavaScript developer, and now I use React JS, Node JS, React Native and PostgreSQL. Today, I was sharing my beginning journey, how I learnt JavaScript will be a story for next time. 

Let me know what you think in the comment and share this article. I hope my story can help anyone else who is want to code. If you need any help, text me on Twitter [@achille_songa](https://twitter.com/achille_songa) or check out my Github at [@songa1](https://github.com/songa1) to checkout my projects so far.
Lessons learned when building self-serve provisioning
Tags: devops, aws
Published: 2022-01-25 23:24:19

---
title: Lessons learned when building self-serve provisioning
published: true
description: 
tags: #devops #aws
//cover_image: https://direct_url_to_image.jpg
---
I've talked to ~20 companies that are at various stages of building data infrastructure (and sometimes ML, analytics and other infrastructure). Many of them have a step when signing up customers that require provisioning some cloud resources for the customers. 

I was at Confluent at the time when we moved from "talk to sales and wait a week" manual provisioning to self-serve provisioning. The way different companies go about this are different some of the lessons are generally applicable. Ram Subramanian, who was the VP Eng at Confluent who ran the entire cloud service recorded a vide with me where we discussed the lessons we've learned:

* Manual process, especially one when there are multiple humans between the customer and "get something done" introduces risk of getting things wrong and pissing off the customer.

* Not only does it piss off customers, a lot of mistakes means a lot of back-and-forth and is a huge waste of time.

* This gets much worse as growth accelerates. And customer expectations go up over time too.

* Since building actual product automation (as opposed to partially-automated collection of scripts) takes some time (especially if you need it to be well tested since your customers are already a bit annoyed), you actually need to start before this is a real painful problem.

* It doesn't take much to go from a point of "manual is fine" to "our engineers spend so much time on manual work that they don't have time to work on automation" and from there to "our best engineers quit".

* Another thing we learned is that there is a huge gap between "automated for internal use" and "self serve for customers". For real customer facing you need much better feedback loop around "what is happening? are there any issues? is it making progress? what does this error mean and what do I do now?" and you also need much better guard rails - customer mistakes can bring down your system or cost you a lot if you are not careful about how you build it.

{% youtube w9MaX2WuUGA %}

P.S
We started a series of "SaaS Stories" - I am trying to interview people who built great SaaS products to talk about how they built some of the foundational SaaS experience and share what they learned.  Especially about things that may look basic but turned out to be much harder than anyone expected. Warning each other about potential pitfalls and potential solutions seems like a great cause.  So, if you want to join the series and talk about how you built your on-boarding, authorization, multiple product tiers, billing, notifications, user and org management, etc... I'd love to chat. No vendors talking about their solutions - only people building SaaS products sharing their lessons. Ping me via @gwenshap on twitter. TY!
React Compnent Rerender won't rerender on State Change
Tags: javascript, react
Published: 2022-01-25 23:01:49

I have a component which takes in a prop "board" which is a piece of state and a useEffect which changes the value of board. However, even when I verify that the useEffect has been run, the component does not rerender. 
This is what my component looks like:

Table rows = {game.rows} cols = {game.cols} style = {style} board = {board} key = {board} setMoveHistory = {setMoveHistory} moveHistory = {moveHistory} />

This is my useEffect:

useEffect(() => {
         if(!game) return;
         console.log('useEffect for setting board running');
        setBoard(createFilledArray(game.rows, game.cols, moveHistory.slice(0, tempTurnNum -1)));
        setRun(true);
        console.log(moveHistory)
     },[tempTurnNum, moveHistory, aState])

Even when the useEffect successfully resets the state variable "board", the table component does not rerender. I would appreciate it if anyone knows why.