teleren/devto huggingface dataset:
Introducción al web scrapping
Tags: python, web, scrapping, webdev
Published: 2022-06-30 02:33:54

En internet existe infinidad de información a la que podemos acceder, ya sea en foros, blogs, wikis, etc.. todas nos proporcionan cualquier tipo de información que podría sernos o no de gran utilidad.

Algunas veces estos sitios nos ofrecen la facilidad de obtener su información mediante diferentes métodos, algunos de ellos pueden ser: 

- Archivos que se pueden descargar directamente (csv, json, etc..)
- Mediante una API que conecta con su base de datos.

Y finalmente la última opción, la cual es un formato muy cómodo para los usuarios que recurrentemente visitan la página web, pero un poco frustrante para extraerla a un programa.

Por fortuna en Python existen librerías que nos ayudan a extraer la información de cualquier página web, con el simple hecho de visualizar la información en un navegador de internet bastara para extraerla, es lo que se conoce propiamente con el termino de web scrapping.

Dentro de la última opción, tenemos dos posibilidades. En la primera, podemos acceder a la información visitando el sitio web mediante la URL. En este caso al descargar todo el contenido de la página usaremos la biblioteca BeautifulSoap, la cual nos permitirá buscar el contenido deseado dentro del sitio web.

La otra posibilidad es para sitios de mayor seguridad que detectan este tipo de librerías, por suerte tenemos la siguiente librería Selenium, que nos permitirá realizar todo lo que haríamos desde el navegador de internet.

Espero que les ayude esta pequeña introducción al web scrapping, estos días estaré subiendo más contenido sobre esto para que les pueda servir en sus próximos desarrollos.

Saludos.
Fazendo deploy com o kubernets no GCP
Tags: cloud, todayilearned, googlecloud, braziliandevs
Published: 2022-06-30 01:00:23

Com uma conta ativa e com faturamento ativo no google cloud provider podemos procurar pelo kubernets engine na barra de pesquisa e ativar essa opção.

Com ela ativa podemos criar um cluster do tipo standard, dentro das configurações do cluster q vou criar somente mudei o nome e o numero de nós de 3 para 2

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/h7y4b0g12wlmdntzekm1.png)


Como quero fazer esse teste com o kafka fui no confluent cloud e criei uma conta lá, e subi um cluster gratuito no google cloud por lá

Em seguida para continuar [instalei o cli do google cloud](https://cloud.google.com/sdk/docs/install#deb) mas não é obrigatório, se quiser abaixo de onde o google cloud coloca a string de conexão ele também dá a opção de usar o terminal online
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9f3netw3eidwaaezxvvv.png)

Depois de feita a conexão podemos testar rodar o `kubectl get nodes` e então o output deve ser os nodes lá do GCP

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qopiabw8chj20bbmp6cp.png)
 
depois disso criei uma pasta para colocar os arquivos k8s do meu simulador então criei um deploy.yml 

> se quiser saber mais sobre deployment pode ir no meu [post sobre kubernets](https://dev.to/yanpiing/usando-o-kubernets-pela-primeira-vez-1eoj)

```yml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: simulator
spec:
  replicas: 1
  selector:
    matchLabels:
      app: simulator
  template:
    metadata:
      labels:
        app: simulator
    spec:
      containers:
        - name: simulator
          image: yansb/codedelivery:latest
          volumeMounts:
            - name: simulator-volume
              mountPath: /go/src/.env
              subPath: .env
              

      volumes:
        - name: simulator-volume
          configMap:
            name: simulator-conf
            items:
              - key: env
                path: .env
          
````

e um configmap.yml que tem minhas variáveis de ambiente

```yml
apiVersion: v1
kind: ConfigMap
metadata:
  name: simulator-conf
data:
  env: |
    KafkaReadTopic=route.new-direction
    KafkaProduceTopic=route.new-position
    KafkaBootstrapServers=
    KafkaConsumerGroupId=simulator
    security.protocol=SASL_SSL
    sasl.mechanisms=PLAIN
    sasl.username=
    sasl.password=
````
 
depois de criados esses arquivos temos que rodar os comandos 

`kubectl apply -f k8s/simulator/configmap.yml`
`kubectl apply -f k8s/simulator/deploy.yml`

e agora nossos pods já estão subindo 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/m4cucpb45iiaafrp1uln.png)
 
E como a aplicação q eu subi foi o simulator que está conectado no kafka posso verificar que o pod está online pelo confluent cloud

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/y4u333hktfrg08jc06eq.png)

Ele já está conectado, recebendo e mandando mensagens.
How to work with multiple AWS CLI accounts
Tags: aws, cli, tutorial, productivity
Published: 2022-06-30 01:26:47

Sometimes it happens that one needs to work with multiple AWS accounts in the same computer.

If that’s your case, the way to do that is the following:

First if you don’t have AWS Cli V2 installed, you should do the following steps, as explained on the [AWS documentation](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html).

Once you have aws cli v2 installed in your computer, now you need to set up your AWS profile, as explained in [this link](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-quickstart.html).

There are two ways to do that.

If you don’t specify anything, you just write the following on your terminal, and then follow the steps as always:

```
$ aws configure
```
It will start a command prompt asking you to complete the details as following:

```
AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE
AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
Default region name [None]: us-west-2
Default output format [None]: json
```
Then if you write the following on your terminal:

```
$ aws configure list-profiles
```
It will list your profiles, and that profile will be named as “default”

Also you can write this command, and it will show you the profile that you have currently set as “default”:

```
$ aws configure list
```
The other way to work with that is naming your profiles when you first configure it, in this way:

```
$ aws configure — profile account1
```
Then if you want to switch between one of this profiles, you do it by switching them as your default profile, doing the following:

```
$ export AWS_DEFAULT_PROFILE=account1
```

Note: It’s important to write it just as appears above, in capital letters and without spaces, otherwise it won’t work.

Other way to work with the profiles are specifying with which profile you want to run certain command, like this:

```
$ aws dynamodb list-tables — profile account1
$ aws s3 ls — profile account2
```
Also another way to work with different AWS profiles, is to specify that on the .ENV variables of your project, like this:

```
AWS_ACCESS_KEY_ID=”your_key_id”
AWS_SECRET_ACCESS_KEY=”your_secret_key”
```
If for any reason you need to edit or view where all this magic is stored on your computer, this is the path where you can find the aws cli config and credentials:

```
~/.aws/credentials

~/.aws/config
```
Note:If you can’t see it, you need to allow seeing hidden files.

For more info related with that, you can read [this document](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html).

Hope it was clarifying and helped you.
If you have any comments or anything to add, please don’t hesitate to do it on the comment section!

Have a great day :)
Top 6 Databases and what each of them are used for?
Tags: webdev, programming, discuss, tutorial
Published: 2022-06-30 00:31:00

Top 6 Databases trending recently (as per stackoverflow developer report) and what each of them are used for? 

1. **MySQL , PostgreSQL** are the top 2 open source SQL databases used by most tech companies and great for interview preparation. They are mainly relational databases. To compare the syntax you can refer to (https://t.co/LRhrMBT2yo)

2. **SQLite:** Is a light weight database that is embedded into most mobile applications. If you are an Android/IOS developer definitely take a look at it.

3. **MongoDB:** Is widely using NoSQL database. Its a cross-platform document-oriented database program.

4. **Microsoft SQL Server:** Its a microsoft version of SQL implementation for relational databases. Great for Azure or .Net applications.

5. **Redis:** This is a must learn open source, in-memory data store used by millions of developers as a database, cache, streaming engine, and message broker. Great for caching . 

PS: For top interview questions: https://tipseason.com/coding-interviews.html

What DB do you use for devleopment ? 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/so4t37grgb8sml92gbuq.png)
Vue 3 / Vite on Cloudflare Pages
Tags: vue, vite, cloudflare
Published: 2022-06-30 00:09:59

A build recently failed in Cloudflare Pages and showed the following error.

```
19:09:34.631	Executing user command: npm run build
19:09:34.980	
19:09:34.981	> webapp@0.0.0 build /opt/buildhome/repo
19:09:34.981	> vite build
19:09:34.981	
19:09:35.288	failed to load config from /opt/buildhome/repo/vite.config.js
19:09:35.289	error during build:
19:09:35.289	Error: Cannot find module 'happy-dom'
```

Doing `npm run build` on my local machine worked fine so I went on to find out the difference between my machine and the Cloudflare Pages build environment.

It turned out that my machine was on Node 16.15.0 while Cloudflare Pages was at 12.18.0, which is quite old. So, based on the [build configuration docs](https://developers.cloudflare.com/pages/platform/build-configuration/), all we need to do is set an environment variable in the build settings of the Cloudflare Pages project for both production and preview `NODE_VERSION=16.15.0` so it matches the local build environment.

After saving the environment variable and retrying deployment, the build was successful. This tip would apply to any javascript project that uses npm or yarn for building.
Discord OG Logo | Pure CSS
Tags: codepen, css, showdev, tutorial
Published: 2022-06-30 00:07:43

In today’s tutorial, I finished creating a OG Discord logo with CSS. Simple css code for this iconic staple. Though this is just a fun tutorial, you can practice CSS art to improve your CSS skills. CSS art has been a fun way for me to learn new properties and improve my CSS skills. I have a bunch of CSS art tutorials that are CSS inspired. You can check them out [here](https://codepen.io/rembertdesigns).

For this tutorial, all we need is some basic HTML and CSS.

{% codepen https://codepen.io/rembertdesigns/pen/JjLoGbj %}

If you liked this post, make sure to follow me on [Twitter](https://twitter.com/RembertDesigns) where I post daily about Tech related things!

https://www.buymeacoffee.com/rembertdesigns

### 🌎 Let's Connect

- [Portfolio](https://www.rembertdesigns.co/)

- [Twitter](https://twitter.com/RembertDesigns)

- [Github](https://github.com/rembertdesigns)

- [Codepen](https://codepen.io/rembertdesigns)

- [LinkedIn](https://www.linkedin.com/in/rrembert/)
Day 82: frontend is doable afterall
Published: 2022-06-29 23:51:06

Hi!
so today the job continue but fortunately the more I work on the frontend the easier it get. I now appreciate React more and the power it gives to a developer and it makes some complicated task easier to do 

Today I looked up how to create a multi step form which I plan to implement tomorrow morning from what I researched I now got the idea but I think I will still need to go through it again as I implement it tomorrow

That's it, off to sleep

Signing off...
React Component Props
Tags: react, webdev, beginners, programming
Published: 2022-06-29 23:41:50

The most important concepts to understand React are components, props, state and hooks.

I will explain 'props', short for 'properties' here. Props are accessed as parameters in the Component function. Often times, they are destructured to keep the code cleaner.

I will assume that you know how to import and export the components in your project hierarchy but if not, you can check out the create-react-app [docs](https://create-react-app.dev/docs/importing-a-component/)

```jsx
//in the App.js file you would render the prop:

<Hello person={benjamin} />

//and in the Hello.js file, the person prop would be passed down:

function Hello(props) {
return <h1>Hello, {props.person}</h1>
}

//which would ouput:

<h1>Hello, benjamin</h1>

//alternatively, you could destructure the prop for cleaner code:

function Hello({props}) {
return <h1> Hello,{person}</h1>
}
```
You can have as many props as needed. For example:

```jsx
function Weather ({temperature, day} {
return <h2> It will be {temperature} degrees on {day}</h2>
}

<Weather temperature='84' day='Tuesday' />

//Which results in:

<h2> It will be 84 degrees on Tuesday </h2>

```

## Class components

It seems that developers are moving away from class-based components as it's unnecessarily verbose. However, if you are still using class components, the process is very similar, but you need to add a `this.props` instead of just `props`

For example:

```jsx
import { Component } from 'react'

class Greeting extends Component {
  render() {
    return <p>Hello, {this.props.who}!</p>;
  }
}
```
# Prop Types

A prop can have any value including strings, numbers, objects, arrays, booleans, variable, function references.


* String literal:

```<Component prop='this is a string'>```

* Template literal:

```<Component prop={`this is a string with a ${variable}`}```

* Number literal:

```<Component prop={14} />```

* Boolean literal:

```<Component prop={true} /}```

* Object literal:

```<Component pro={{property : 'value'}} />```

* Array literal:

```<Component prop={['item 1','item 2']} />```

* JSX

```<Component prop={Message who='Batman' />} />```

* Variables or function references

```<Component prop={functionReference} />```
⚖️ Kafka image : wurstmeister vs. bitnami
Tags: docker, kafka, security, devops
Published: 2022-06-29 23:21:02

## ❔ Context

We recenlty did operate maintenance on our interal (yet public repo) that helps us internally promote Kafka practices : 

{% github opt-nc/atelier-spring-kafka %}

☝️ We also do apply a maintenance strategy on this content like : 

- [Spring Boot upgrades](https://github.com/opt-nc/atelier-spring-kafka/pull/21)
- [Kafka upgrades](https://github.com/opt-nc/atelier-spring-kafka/pull/19)

We created a [dedicated issue to upgrade to Kafka `3.2`](https://github.com/opt-nc/atelier-spring-kafka/issues/18), we switched from [`wurstmeister/kafka`](https://hub.docker.com/r/wurstmeister/kafka/) to [`bitnami/kafka`](https://hub.docker.com/r/bitnami/kafka/) (by [vmWare](https://hub.docker.com/u/bitnami)) cf [our configuration change](https://github.com/opt-nc/atelier-spring-kafka/commit/7255813352133f76d205e7cec0cf3d25ee6a8dee).

## 🛡️ Security concerns

☝️ Each time we use a new Docker image, we also focus on the security part.

👉 **This post is about the comparison of these two images security levels.**

```shell
grype --add-cpes-if-none wurstmeister/kafka:latest \
    | grep Critical \
    | wc -l
# 14
grype bitnami/kafka:latest \
    | grep Critical \
    | wc -l
# 5
```

|Criticity|`wurstmeister/kafka:latest`|`bitnami/kafka:latest` |
|       ---     |          ---       |          ---         |
| ☣️`Critical`  | 14                 | 5                    |  
| ❗`High`      | 34                 | 17                   | 
| ❕`Medium`    | 16                 | 9                    | 
| ⚠️`Low`       | 9                  | 9                    | 
| 🪶`Negligible`| 68                 | 66                   |
| ❔`Unknown`   | 15                 | 13                   |


## 📊 Charts

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/uolz7iqqat8ye0fww7ki.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1zfkuqvqb12dziuemxur.png)

## 🤔 The `Unknown` Criticity
 
Wondering what means `Unknown` Criticity ? Check the answer below :


{% embed https://github.com/anchore/grype/issues/807 %}

## 📦 Apache Kafka packaged by `Bitnami`

{% github bitnami/bitnami-docker-kafka %}

## 🔖 Resources

- 🐦 [`@bitnami`](https://twitter.com/Bitnami)
- 🐳 [Bitnami containers](https://bitnami.com/stacks/containers)
Podcast – The Best Bits of Xcode 14?
Tags: podcast, xcode, tools
Published: 2022-06-30 03:38:53

---
title: Podcast – The Best Bits of Xcode 14?
published: true
date: 2022-06-29 22:37:26 UTC
tags: Podcasts,Xcode, Tools
canonical_url: https://peterwitham.com/podcasts/cs-s04-e12/
---

I have compiled a list of the important things I think we need to know about Xcode 14 based on available details. There are some great new features and changes in my opinion.

{% spotify spotify:episode:https://open.spotify.com/episode/2O0JeI0XqWLIqDmHVNaOP9?si=beec1a1f2dc74a3c %}

The post [Podcast – The Best Bits of Xcode 14?](https://peterwitham.com/podcasts/cs-s04-e12/) appeared first on [Peter Witham](https://peterwitham.com).
Day 583 : Visions
Tags: hiphop, code, coding, lifelongdev
Published: 2022-06-29 22:01:03

_liner notes_:
- Professional : Sat in on a couple of talk prep sessions and gave some feedback. They did a really good job. Then I basically worked on my videos for the rest of the day. The first video is taking a little bit because I'm hopefully setting it up so that I can reuse a lot of it for the next 2. We'll see.

- Personal : Did a little work on my side project. Then worked on the videos. Went to bed watching a couple of episodes of "The Umbrella Academy". It's really good!

![Looking down through some trees at Cape Schanck, Australia shore with some large stone structures coming out of the water at sunset.](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tppkhyntsmekigmvrjgl.jpg)

Going to keep working on the videos. I want to get them done so I can take it easy on Friday. I ordered some stuff for the van like a thing I can use to hang up shirts. That should let me get rid of the box of shirts I have on the floor. Then I'll start working on the shower I want to put in that place. The visions I have for the van are coming together. There's an online meetup tonight that will be talking about WebXR so that will be cool.

Have a great night!

peace piece
Dwane / conshus
https://dwane.io / https://HIPHOPandCODE.com
{% youtube MZXQQkZbgLo %}
I've got a YouTube channel to complement my writing - what do you want to see?
Tags: tutorial, webdev, programming, beginners
Published: 2022-06-29 21:56:43

---
title: I've got a YouTube channel to complement my writing - what do you want to see?
published: true
description: I've got a YouTube channel to complement my writing, what do you want to see?
tags: tutorial, webdev, programming, beginner
cover_image: https://dev-to-uploads.s3.amazonaws.com/uploads/articles/x3ybp2mrbsue8n0uto2m.png

---

Hey friends. I decided to spend some time on building up a YouTube channel. I've been writing on dev.to for a while and a love it. I also recognize that some of you want to see videos additionally to written pieces.

This is what I'm working on at the moment on my channel and I would love some comments on what's there and what I can create. 

Would really appreciate a subscribe as well.

> [Subscribe to my channel](https://www.youtube.com/channel/UC1Q3yUG4E0afRxccqqI3ycw?sub_confirmation=1)

- [Rust series](https://www.youtube.com/playlist?list=PLRbYLREdOhfnj0XxsrWlCiFlZRPjpz3Zd)
- [Go/Golang series](https://www.youtube.com/playlist?list=PLRbYLREdOhfnLJl0xJxKEPe4BhlDU66Zj) 
- [Azure series](https://www.youtube.com/playlist?list=PLRbYLREdOhfkaTEwJMYcEFj-xuyuWcbPN)
- [.NET series](https://www.youtube.com/playlist?list=PLRbYLREdOhfkJazwCu4nzNNEzYVLf4rmb)
- [Python series](https://www.youtube.com/playlist?list=PLRbYLREdOhflQi-hDZSFikezNqLUT9-HH)
- [JavaScript series](https://www.youtube.com/playlist?list=PLRbYLREdOhfnC5t6IMLnlxuxwnN76ozuX)
How to use .env file in node.js
Published: 2022-06-29 21:44:41

Using .env file to store secret variables in software applications is a good common practice in software development. These variables can be database credentials, urls, ip addresses or hosts, secret keys for third party integrations etc.

In this tutorial, we are going to look at how to store secret variables in .env file and make use of them in a node.js application.

First of all, install an npm package called **dotenv** using the following command in your node.js project root directory;
```javascript
npm install dotenv --save
```
dotenv package automatically loads environment variables from .env file into process object in node.js applications.

Create a .env file in your project root directory
```
DB_NAME=students
DB_USERNAME=dallington
DB_PASSWORD=fdggavcyyatexcda
```
In the above example of .env, I have database secret variables; database name (**DB_NAME**), database username (**DB_USERNAME**) and database password (**DB_PASSWORD**) but you can add as many variables as per your project needs.

You can then access your environment variables in any file of your node.js application as follows;
```javascript
require('dotenv').config()
console.log(`Database name is ${process.env.DB_NAME}`);
console.log(`Database username is ${process.env.DB_USERNAME}`);
console.log(`Database password is ${process.env.DB_PASSWORD}`);

```
In the above code example, we import and configure dotenv using require; which is a built-in node.js function used to load modules. We then access our environment variables through **process** which is a global object in node.js.

**Output**
```
Database name is students
Database username is dallington
Database password is fdggavcyyatexcda
```
**Note:**
Since this file contains secret variables, we don't push it to git/github so remember to include .env file in your .gitignore file under your project root directory.

This is the basic example or tutorial of how you use .env file in node.js application. Thank you for reading through this tutorial and happy coding!😊
Rainbow Drops Background Animation [No JS]
Tags: showdev, html, css, codepen
Published: 2022-06-29 21:31:45

I've been playing around with pure CSS/no JS background animations, inspired by all the beautiful creations that can be found on CodePen. 

Today I made a rainbow drops background animation that I'm really proud of. It was also my first time using Haml and SCSS - or any HTML/CSS preprocessors for that matter - so I learned a lot while making this (and had a ton of fun!). 

Hope you enjoy!

{% codepen https://codepen.io/natgeyzentech/pen/rNdaaeP %}

---

**Connect with me**
[CodePen](https://codepen.io/natgeyzentech)
[Twitter](https://twitter.com/Nat_A_Number)
[GitHub](https://github.com/NatGeyzenTech)
What is your favourite ES6 feature?
Tags: javascript, discuss, career, interview
Published: 2022-06-29 21:17:42

Helloooo everyone! 👋 It has been a while since I last posted, I have been rather busy the past 2 weeks!

As the title suggests, I am curious to know what your favourite JavaScript ES6 feature is...🤔 

I was recently asked this in an interview and it surprised me as I have been learning JS with latest features so I don't quite know which features are ES6 and which are newer... So I answered with what came to mind, and it was `Promises`. Since I have been learning about backend development with NodeJS, I have learnt a lot about asynchronous programming which is why I chose that answer (I also talked about async/await, which was not introduced until ES7, oops!). Of course, I had to explain why I chose that particular feature and I talked about the benefit of how it helps eliminate *callback hell*, improve error handling and how you can use Promise chaining to your advantage.

So, I present to you the same question, what is/are your favourite ES6 feature/s and why? (If you are preparing for an interview, this could be a good practice to answer out loud 😉).