teleren/devto huggingface dataset:
Speeding up geodata processing
Tags: geopandas, python, pyogrio, pickle
Published: 2021-12-25 21:12:13

I've been using the excellent [geopandas](https://geopandas.org/) for working with largish geodata sets and CSV files. While geopandas has been great for working with data, it is slow to ingest geodata. I ran a simple test to time reading a 1.2GB line shapefile into a dataframe.

```python
import geopandas as gpd
import time
import pickle

# read shapefile
read_start = time.process_time()
data = gpd.read_file("Streets.shp")
read_end = time.process_time()

read_time = read_end - read_start
print(str(read_time/60)+" minutes")
```

25.43723483333333 minutes

[Martin Fleischmann](https://twitter.com/martinfleis) suggested using [pyogrio](https://pyogrio.readthedocs.io/en/latest/index.html) instead of geopandas. The result was quite impressive.

```python
# alternate package for reading data
from pyogrio import read_dataframe
import time
import pickle

# read shapefile
read_start = time.process_time()
data = read_dataframe("Streets.shp")
read_end = time.process_time()

read_time = read_end - read_start
print(str(read_time/60)+" minutes")
```

2.9936875333333335 minutes

While going from 25 minutes to 3 minutes is quite an improvement, I'm building out a data processing pipeline and I want to reduce read time even more. My next experiment uses Python's [pickle](https://docs.python.org/3/library/pickle.html) which serializes the data frame into a byte stream and writes it to a file. Here are the results from pickling the dataframe and reading the pickled data.

```python
#create a file
picklefile = open('streets', 'wb')

#pickle the dataframe
pickle_write_start = time.process_time()
pickle.dump(data, picklefile)
pickle_write_end = time.process_time()

#close file
picklefile.close()

pickle_write = (pickle_write_end - pickle_write_start)/60
print(str(pickle_write)+" minutes")
```

4.362236583333333 minutes

```python
#read the pickle file
picklefile = open('streets', 'rb')

#unpickle the dataframe
pickle_read_start = time.process_time()
df = pickle.load(picklefile)
pickle_read_end = time.process_time()

#close file
picklefile.close()

pickle_read = (pickle_read_end - pickle_read_start)/60
print(str(pickle_read)+" minutes")
```

0.9217719833333339 minutes

Wow! We've gone from 25 minutes to read a 1.2GB shapefile to less than a minute. 

Finally, pickling the dataframe reduces the shapefile size from 1.2GB to 984MB. However, pickled data can be efficiently compressed.

```python
import gzip
import shutil
with open('streets', 'rb') as f_in:
    with gzip.open('streets.gz', 'wb') as f_out:
        shutil.copyfileobj(f_in, f_out)
```
The compressed file is 78.8MB, a bit more than a 10x reduction in file size.

## Conclusion

If you're working with geodata that remains static, you can improve geopanda's read times by using pyogrio and pickling the resulting dataframe. Additionally, pickle files can be compressed efficiently, which can lower costs for data egress when using cloud storage. 

> [Pickle image](https://en.wikipedia.org/wiki/Pickled_cucumber#/media/File:Pickle.jpg) by Renee Comet
TiDB: The Best and Sexy Distributed SQL Database Platform for Horizontal Scalability, Strong Consistency, and High Availability!
Tags: devops, database, datascience, mysql
Published: 2021-12-25 19:40:43

The simple and effective solution for Horizontally scalable Relational Databases and Real-time Analytics cases.

TiDB is an open source, MySQL-compatible NewSQL database created by PingCAP and its open-source community.

To learn more about, go to:

https://jlgjosue.medium.com/tidb-the-best-and-sexy-distributed-sql-database-platform-for-horizontal-scalability-strong-3964db5f466e
CSS Blogs
Tags: css
Published: 2021-12-25 19:29:28

Position: CSS Position property refers to the assigned position of an element. We can set different elements of our application in different places with positions. With different properties of position, we can set our elements right, left, top, bottom as required. There are five properties in the position they are: static, absolute, fixed, relative, sticky. These are used in different cases. Relativity property can be used to relate the current state of an element without changing its layout and can be related to its position by changing the layout of an element with absolute and the element can be set in different places.
Key Bert
Tags: bert, keybert, transformers, ai
Published: 2021-12-25 19:14:31

Please check out key bert : 

https://maartengr.github.io/KeyBERT/guides/embeddings.html

You can use Keybert for embedded sentences  and to embedded search queries & paragraphs.

Augmented SBERT is based on Bertcross-encoder  to label a larger set of input pairs to augment the training data for the SBERT bi-encoder.
Find No of days between 2 dates using moment.js
Tags: javascript, webdev, beginners, help
Published: 2021-12-25 19:00:39

Time and time again I'm reminded that you can't escape time and so are the problems related to date and time. One such very common problem is to find the `difference between 2 dates.`

Let me give you a very simple example where we will try to `find the how many days old are you?`
So it feels like not a such big deal but think of writing the entire logic on your own when you know you can write but it will much less of a hassle to do it via library like `moment.`

I'm writing this as an article here because it took me time to find the way to do it using moment itself. I'm writing this article more for myself than for others. So consider this as a treat.

---
> **Here's the code:**

```
  let myDob = moment("19/12/1997", "DD/MM/YYYY");
  let today = moment();
  let myAge = today.diff(myDob, "years");
  let noOfDays = today.diff(myDob, "days");
```


> **Below is the embedded sandbox to demonstrate it:**

{% codesandbox 79nux %}

> Check out the same in moment docs:
[moment](https://momentjs.com/docs/#/durations/subtract/)

 Thanks for your time reading this. I hope it helped you. Have a productive day :)

**PS:**_Moment is in maintenance mode._

You can read about the meaning of it in the article linked below.
[MOMENT.JS OFFICIALLY BECOMES A LEGACY PROJECT IN MAINTENANCE MODE](https://ilikekillnerds.com/2020/09/moment-js-officially-becomes-a-legacy-project-in-maintenance-mode/)
My blog has moved
Tags: website, security
Published: 2021-12-25 18:36:36

---
title: My blog has moved
published: true
description: My blog has moved, please visit me at https://shehackspurple.ca/blog/.
tags: website, security
//cover_image: https://direct_url_to_image.jpg
---
**My blog has moved! Please check it out here: **
[https://shehackspurple.ca/blog/](https://shehackspurple.ca/blog/)
Are you expected to know all http status codes?
Tags: interview, webdev, programming, http
Published: 2021-12-25 18:21:07

I just got an interview and I was asked different http status and what their meaning was.

Besides the typical ones , 200, 203, 300, 404, 500, I couldn't come up with anything because you can look this up and shouldn't remember it by heart. That would be a waste of mental capacity IMO. Or am I wrong? Is it expected to know these things as a web developer?
A gradle Task to Save your time
Tags: android, gradle
Published: 2021-12-25 18:07:39

Recently, while running Android Studio I got next error from time to time which waste much time which can be increased based on project size (at least 1 minute, so this amount is **multiplied** with each build we do): 

> The process cannot access the file because it is being used by another process.

This can be **caused** when some **plugins** run _another Java instance_ so they **lock** files from one to the other **So** solution is to kill java process by using: 
`TASKKILL /F /IM java.exe `. 

So I searched to make this automated somehow And found we can declare simple gradle task to do this job, it as next( write it at `build.gradle`) 
``` gradle 
task killJavaProcess(type:Exec ){
    commandLine 'cmd', '/c', 'TASKKILL /F /IM java.exe'
}
```

Then to make it be done before each build go to build task at right side of gradle and choose to run before build as next: 
![image with steps ](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ky2v78kfagpnvccqdn52.png)

Have nice coding. 

## Resources: 
[gradle Exec Task type](https://docs.gradle.org/current/dsl/org.gradle.api.tasks.Exec.html)
Advent.js, una iniciativa navideña para mejorar tu lógica de programación🎄Experiencia y conclusiones
Tags: javascript, adventofcode, spanish, webdev
Published: 2021-12-25 17:46:56

Este año participe por primera vez en un evento de Adviento de código, a continuación menconaré algunas experiencias de este divertido (y algo estresante) proyecto.

> *"Sin exámen la vida no merece ser vivida."*
Sócrates

A veces uno piensa que sabe algo hasta que lo pone a prueba y se da cuenta de que no lo domina como pensaba que lo dominaba. Y esto esta bien, mientras uno se de cuenta de estas cosas cuanto antes podrá hacer algo al respecto.

La mayoria de los retos faciles y medios del Advent los resolvi sin tanto sufrimiento, pero los dificiles si que no pude, y esta bien, me toco preguntar en foros, leer código de otras personas, analizar sus algoritmos y comprender como piensas devs mas experimentados que yo. Se aprende bastante, enserio!

En desarrollo frontend por ejemplo, rara vez se consumen APIs que vengan en formato de arreglos anidados, entonces uno suele descuidar y hasta olvidar como trabajar con matrices, este es un punto donde encontre muchas dificultades al resolver ciertos ejercios, por ello me hice una pequeña lista de puntos concretos del lenguaje que es necesad repasar y/o aprender bien:

* **`Sets`**: me impresiono mucho como los devs mas capos solucionan algoritmos usando `Sets`. Son mucho mas potentes de los que creia.
* **`reduce` a fondo**: dominar `Array.reduce()` es super importante, sobre todo trabajando con arreglos y objetos, y no quedarse solo en el típico ejemplo de sumar los valores de un arreglo.
* **`closures`**: igual de importantes, no se deben descuidar aspectos teoricos del lenguaje que ayudan a resolver bastantes bugs.
* **Ciclos anidados**: `for` dentro de `for` no es algo que se vea en programacion funcional, y como adoro Vue y Raact, pues olvide usarlos.
* **Arreglos anidados**: raros de ver en la practica pero importante de dominar para ejercicios de algoritmia.

Para finalizar, agradecer a @midudev por este hermoso proyecto altruista para retarte a ti mismo, y superarte a ti mismo. Es un gran referente de la comunidad y lo admiro mucho. 

Si lees esto, gracias midu! 
Gracias 2021!
Con todo al 2022!!

> Cristian Fernando Villca Gutierrez.
HTML Combobox with Syntax & Example
Tags: webdev, beginners, tutorial, css
Published: 2021-12-25 17:38:50


![HTML Combobox](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/juy8e34fw677b85wqvvi.jpg)
HTML Combobox is primarily used to create forms in HTML. In which users can select an option from the list of their choice. The combo box in HTML consists of the selected element and the input type= "text" element. Combobox functionality is like a select tag. It also has a tag attribute inside the tag to select the menu option from the list so that anyone can choose the option of their choice.

Combobox is an HTML element that can easily keep all types of browsers. It acts as the latest form of a Combobox as an object to fill in the data through the choices in the given text field.

##HTML Combobox Syntax

As we know, Combobox is a combination of tag and element, so that the syntax for Combobox is specified only in itself.

```HTML
<input type="text" id="box" name="comboboxbox"/>
<select name="list" onChange="combo(this, 'box)">
<option>option 1</option>
<option>option 2</option>
<option>option 3</option>
<option> option 4 </option>
<option> option 5 </option>
</select>
```

In the above-provided syntax, the input type is a component that will create input text where we can take input from the user and show it. Also we used `<select>` element to select the options.

##Example:

```HTML
<select class="combobox">
<option value="Select">Select....</option>
<option value="php">php</option>
<option value="html">html</option>
<option value="javascript">Javascript</option>
<option value="CPP">CPP</option>
<option value="Python">Python</option>
<option value="vue">Vue</option>
<option value="7">Cobobl</option>
<option value="9">TEXT 4</option>
<option value="CSS Text">CSS TEXT</option>
<option value="responsive">Responsive</option>
<option value="script">scripting</option>
<option value="preprocessors">Saas, Less</option>
<option value="csharp">C Sharp</option></select>
<style>
.combobox{
padding:8px;
}
</style>
```
HTML Combobox is one of the most significant parts in web development to create a selection list. It is most generally used in the most delinquent browsers.

I shared this post from [HTML Combobox](https://softcodeon.com/html/html-combobox.htm) you can read in detail from there.

I hope you like this post. If you have any question please discuss below help to improve. Thank you.
JavaScript Question: Help change the script (newbie!)
Published: 2021-12-25 17:28:51

This script parses instagram followers. You need to log into your account and open the list of subscribers / subscriptions and paste the script into the console. Then it will start parsing.

Help change the code so that it only parses those who have published a story.

`let ul_accounts = document.getElementsByClassName("jSC57  _6xe7A"); // класс тега ul списка аккаунтов
function parsing() {
	let accounts = ul_accounts[0].innerHTML
	// ------------------------------------------------------------------------------
	// Парсинг ников
	// ------------------------------------------------------------------------------
	let result_nick = accounts.match(/title="[^"]+"/g)
	let result_count = result_nick.length
	result_nick = result_nick.join(' ').match(/"[^"]+"/g).join(' ').match(/[^"]+/g).join('').match(/[^\s]+/g).join('\n')
	// ------------------------------------------------------------------------------
	// Печать ников
	// ------------------------------------------------------------------------------
    console.log(result_nick)
	console.log('%cАккаунтов собрано: ' + result_count + ' шт.', 'color: #13a555; font-size:18px;')
}
parsing()`

Or help me to realize the idea. You need to parse people who have published a storypublished a story.

#js #instagram #story #script
Advent.js🎅🏼| #25: El último juego y hasta el año que viene
Tags: javascript, adventofcode, spanish, webdev
Published: 2021-12-25 17:26:22

## [ El último juego y hasta el año que viene](https://adventjs.dev/challenges/25)

Ayer, en noche buena, una família cenó por todo lo alto... Con tanta copa 🍾 encima todavía no han retirado los platos y la comida de ayer...

Un ratoncillo llamado midurat 🐭, que vió ayer el festín escondido, está relamiéndose los bigotes al ver todos los manjares que hay en el comedor.

Eso sí, hay que tener cuidado 😶 y sólo hacer los movimientos correctos para comer algo. Por eso, el ratón, que se ha visto los vídeos de midudev, va a crear una función para saber si su próximo movimiento es correcto o no ✅.

El ratoncillo se puede mover en 4 direcciones: up, down, left, right y el comedor es una matriz (un array de arrays) donde cada posición puede ser:

-     Un espacio vacío es que no hay nada
-     Una `m` es el ratón
-     Un `*` es la comida

Vamos a ver unos ejemplos: 

```js
const room = [
  [' ', ' ', ' '],
  [' ', ' ', 'm'],
  [' ', ' ', '*']
]

canMouseEat('up',    room)   // false
canMouseEat('down',  room)   // true
canMouseEat('right', room)   // false
canMouseEat('left',  room)   // false

const room2 = [
  ['*', ' ', ' ', ' '],
  [' ', 'm', '*', ' '],
  [' ', ' ', ' ', ' '],
  [' ', ' ', ' ', '*']
]

canMouseEat('up',    room2)   // false
canMouseEat('down',  room2)   // false
canMouseEat('right', room2)   // true
canMouseEat('left',  room2)   // false

```

¡Ten en cuenta que el ratón quiere buscar comida en diferentes habitaciones y que cada una puede tener dimensiones diferentes!

Completa el reto!

---
Te dejo una posible solución:


{% jsitor https://jsitor.com/embed/axPP8etva %}

---
Puedes seguir a @midudev y estar pendiente de los retos de Advent.js 

{% twitter 1466086678507008003 %} 

-
What is IIFE in JavaScript ?
Tags: javascript, webdev, beginners, tutorial
Published: 2021-12-25 17:24:12

Sometime in JavaScript we need a function that executes only once and never again. Basically a function that disappears right after its called once. How can we achieve this? for example we could do this:

```
const func1 = function(){
             console.log('this function will never run again!`)
                }
 func1();
```
But we definitely can call this func1() again if we want. This is **NOT** what we want to do. We want to execute a function immediately without having to save it somewhere.
So this is how we do that,
we simply write the function expression itself without assign it to any variable.

```
function(){
console.log('This function will never run again')
}
```
if we run this, we would get an error *function statements require a function name* an that's because JavaScript expects a function statement, but we simply start a new line of code with the function keyword.

Here we can still trick JavaScript into thinking that this is just an expression. We do that by simply wrapping all of this into GROUPING OPERATOR().

```
(function(){
   console.log('This is will never run again!')
});
```
so we basically transform the previous statement into an expression. But the function didn't execute yet. To execute the function we need to call it like this (adding '()' next to it, like we do with regular function call).

```
(function(){
   console.log('This is will never run again!')
})();
```

So we create a function expression and we immediately call it. which is why this pattern is called **Immediately Invoked Function Expression(IIFE).**

The same would also work for an arrow function this way.

```
(() => console.log('This function will never run again!'))();
```
This will also never run again.

so this is not really a feature, of the JavaScript.

it's more of a pattern, that some developers come up with and it is widely used.

This design pattern is also known as a **Self-Executing Anonymous Function.**
Self-Executing -> it executes the moment it is created.
Anonymous -> it has no name and is not stored variable.
Advent.js🎅🏼| #23: ¿Puedes reconfigurar las fábricas para no parar de crear regalos?
Tags: javascript, adventofcode, spanish, webdev
Published: 2021-12-25 17:16:34

## [¿Puedes reconfigurar las fábricas para no parar de crear regalos?](https://adventjs.dev/challenges/23)

Estamos en la fábrica de Santa Claus 🎅 creando regalos como si no hubiera un mañana.

Pensábamos que no íbamos a llegar pero **Jelf Bezos** ha tenido una idea genial para aprovechar las máquinas y optimizar al máximo la creación de regalos. 🎁

La configuración de las máquinas es un **string**. Podemos reconfigurarla para que haga otro regalo y, para ello, podemos cambiar cada carácter por otro.

Pero **tiene limitaciones** 🥲: al reemplazar el carácter se debe mantener el orden, no se puede asignar al mismo carácter a dos letras distintas (pero sí a si mismo) y, claro, la longitud del string debe ser el mismo.

Necesitamos **una función que nos diga si podemos reconfigurar una máquina para que de un regalo pueda pasar a fabricar otro según las reglas mencionadas**. Lo mejor es que veamos un ejemplo:

```js
const from = 'BAL'
const to   = 'LIB'
const canReconfigure(from, to) // true
/* la transformación sería así:
B -> L
A -> I
L -> B
*/

const from = 'CON'
const to   = 'JUU'
const canReconfigure(from, to) // false
/* no se puede hacer la transformación:
C -> J
O -> U
N -> FALLO
*/

const from = 'XBOX'
const to   = 'XXBO'
const canReconfigure(from, to) // false
/* no se puede hacer la transformación:
X -> X
B -> X (FALLO, no mantiene el orden de transformación y la B no puede asignarse a la X que ya se asignó a otra) 
O -> B
X -> O (FALLO, la X no puede asignarse a la O que ya se asignó a la X)
*/

const from = 'XBOX'
const to   = 'XOBX'
const canReconfigure(from, to) // true

const from = 'MMM'
const to   = 'MID'
cons canReconfigure(from, to) // false
/* no se puede hacer la transformación:
M -> M (BIEN, asigna el mismo carácter a si mismo)
M -> I (FALLO, asigna el mismo carácter a dos letras distintas)
M -> D (FALLO, asigna el mismo carácter a dos letras distintas)
*/

const from = 'AA'
const to   = 'MID'
cons canReconfigure(from, to) // false -> no tiene la misma longitud

```
Completa el reto!

---
Te dejo una posible solución:


{% jsitor https://jsitor.com/embed/ySuYhR3dm %}

---
Puedes seguir a @midudev y estar pendiente de los retos de Advent.js 

{% twitter 1466086678507008003 %}
Advent.js🎅🏼| #21: La ruta con los regalos
Tags: javascript, webdev, adventofcode, spanish
Published: 2021-12-25 17:03:21

## [La ruta con los regalos](https://adventjs.dev/challenges/21)

Se están preparando las rutas para el trineo de Santa 🎅. Tenemos almacenes por todo el mundo para que Santa pueda recoger los regalos y entregarlos en el destino final. 🎁

Necesitamos saber si las rutas que estamos creando tienen sentido o si Santa va a tener que dejar tirados regalos por el camino. 🥺

Para eso vamos a crear una función que recibe dos parámetros:

- Un número con **la capacidad máxima de regalos en el trineo**.
- El viaje que es un array de arrays. Cada subarray contiene tres números que representan:

- `trip[0]` = número de regalos a transportar
- `trip[1]` = punto de recogida de los regalos
- `trip[2]` = punto de entrega de los regalos 

La ruta siempre va de izquierda a derecha (nunca volverá Santa hacia atrás) pero... ¡ten en cuenta que en mitad de la ruta puede tener que recoger regalos cuando ya tiene alguno encima!

Lo mejor es que veamos un ejemplo:

```js
canCarry(4, [[2, 5, 8], [3, 6, 10]]) // false
// En el punto 5 recoge 2 regalos...
// En el punto 6 recoge 3 regalos...
// Del punto 6 al 8 tendría 5 regalos en total
// Y su capacidad es 4... así que ¡no podría!

canCarry(3, [[1, 1, 5], [2, 2, 10]]) // true
// En el punto 1 recoge 1 regalo...
// En el punto 2 recoge 2 regalos...
// En el punto 5 entrega 1 regalo...
// En el punto 10 entrega 2 regalos...
// ¡Sí puede! Nunca superó la carga máxima de 3 regalos

canCarry(3, [[2, 1, 5],[3, 5, 7]]) // true -> nunca supera el máximo de capacidad
canCarry(4, [[2, 3, 8],[2, 5, 7]]) // true -> del punto 5 al 7 lleva 4 regalos y no supera el máximo

canCarry(1, [[2, 3, 8]]) // false -> no podría ni con el primer viaje
canCarry(2, [[1, 2, 4], [2, 3, 8]]) // false -> del punto 3 al 4 supera la capacidad máxima porque llevaría 3 regalos
```

Lo difícil, e importante, es que entiendas que Santa Claus **va entregando y recogiendo regalos y que a veces eso puede hacer que supere la carga máxima**.

Completa el reto!

---
Te dejo una posible solución:


{% jsitor https://jsitor.com/embed/C8YP1BSPS %}

---
Puedes seguir a @midudev y estar pendiente de los retos de Advent.js 

{% twitter 1466086678507008003 %}
tailwind css flexBox | flex row | flex col | justify center | items center
Tags: tutorial, beginners, webdev, css
Published: 2021-12-25 17:01:27

watch the video to understand the differences between justify-center-center utility classes in the two cases : a flex-row container or a flex-col container
https://www.youtube.com/watch?v=Tyo055GRu_w
If you liked our content help us by subscribing on the channel and leaving a nice comment