teleren/devto huggingface dataset:
Open Source Meaning: post-2000 evolutions
Tags: opensource, openeducation, openscience
Published: 2024-01-31 19:46:21

Since 2000, the meaning given to « open source » is moving towards new steps by the diversification of code sharing practices and the development of other open movements.

Back from its formalization in 1998, « open source » was associated with the idea of sharing software with its source code without restrictions, following the « Open Source Definition » adopted by the Open Source Initiative.

Different approach of code sharing exists since early days of computers and more of them are flourishing over the time for a variety of motivations with the democratization of « open source ». In some extent, limitations near an available source code have been around since a really long time through a traditional use of licenses.

Following the open source movement, Microsoft launched the Shared Source Initiative in 2001 to show that they are fine with collaboratives practices around Microsoft product. (Or how to treat linux as a cancer and create FUD while mimicking their practices.)

An approach with the code open is seen increasingly as viable for business but with now open source companies struggling with GAFAM monopolies using non-competitive licensing, open resources sustain also evil so the realisition of problems generated by openness lead to an ethical movement with potentialy usage restrictions.

Around this idea of source sharing being (re)popularized for different typologies of players, philosophies vary.

What is then « open source » when sources are being open in these heteroclite manners ? The question has been growing for the past 20 years, and political split around it appear with initiatives like the Organization for Ethical source created in december 2020.

The language becomes a place of conflict, used as a barrier between these philosophies: source available, ethical source, shared source, post-open source, fauxpen source... Contested words for overlapping ideas with open source remaining the main umbrella.

Within software communities, the notion of « open source » is challenged. By the end of 90' and early 2000', other open movement arrived, with open hardware (1997), open content and education (1998~2002), the Open Access/Science movement affirmed in 2002-2003 with Budapest, Berlin and Bethesda declarations and so on.

New kind of digital resources being shared openly, where there are challenges around sources of resources to enable modifications.

Arrived and thought within the software world, it's slowly becoming something else not only associated to software "source code".

Having matured for 25 years, « open source » is evolving and is being redefined over the years in this post-2000s era.

[Publication related to the project open-source-undefined.org and work on history]

Image: Toa Heftiba, Unsplash
A problem tracing experience of application support
Tags: java, mysql, jpa, springboot
Published: 2024-01-31 19:34:41

## Notice

I wrote this article and was originally published on [Qiita](https://qiita.com/saladlam/items/dd4c58c5ac59a13a169c) on 30 August 2022.

---

# Symptom
A user reported that in the saved report of a application, all fields related to time are changed, changed to 1 hour earlier to original value. This happened recently and there was no problem in the past.

Simple to say, the application is build by Spring Boot, and details are:
- Java 8
- Spring MCV
- Spring Data JPA
- JPA (By Hibernate)
- MySQL

# Investigation direction: Time zone
In the UK, time zone will be changed 2 times per year. And there is one hour difference between two time zones. So first of all I looked into the points related to time zone.

When the **default time zone** of MySQL server is the same as the time zone of Java application (British summer time when time of writing), time input is correctly saved into the database. But when the default time zone of MySQL server is set to one hour earlier (Greenwich Mean Time), time saved to the database is one hour earlier than inputed value.

# To find out which part is out of expectation
And then looked into MySQL. The table column stored time is defined as **TIME**, according to the document, MySQL will not alter the value before stored. And after testing, the value is correctly stored by running insert SQL statements without using **Prepared Statements**.

It is time to move back to the application. Following data process path

1. Entity instance created and binded with values of HTTP request in the controller. The time fields of the entity are LocalTime class, it is not affected by the time zone setting of the application. It is easy to check, just dump the values of the entity created to the logger. As expected, the value correctly passed.
1. Entity persisted in the database by generating SQL commands by JPA provider. By setting following values in *application.properties*.

```
logging.level.org.hibernate.SQL=debug
logging.level.org.hibernate.type.descriptor.sql=trace
```

The generated SQL and parameter binding can be viewed. And the value correctly passed to the MySQL JDBC driver.

# Unexpected behavior of MySQL JDBC driver
When I traced the program flow in MySQL JDBC driver, finally I found out where the value was altered (in *com.mysql.cj.ClientPreparedQueryBindings#setTime(int, java.sql.Time, java.util.Calendar)*). [Here](https://dev.to/saladlam/bug-value-of-time-is-altered-by-mysql-jdbc-driver-pcl) is the detail.
Linear Search
Tags: algorithms, beginners, programming, tutorial
Published: 2024-01-31 19:30:38

# Linear Search 

Linear search is the most common way we might think about searching for somehting in a continuous list of elements.
We usually have to resort to linear search whenever we want to find something inside a list but we are not sure about how the elements 
are structured inside said list. So for example, let's say we want to find a specific page in a book, but we are not sure where the page is, how would we do to find the page?

Since we can't tell anything about the list, there is no way to find the page without going through every single page, comparing the value to the value you have.
The algorithm is something like this:

1. Let i be 0
2. While i is less than the number of pages
    1. Check if the ith page is the same we are looking for
    2. If it is return true
    3. Otherwise increment i
3. If i = number of pages, we could not find the page, therefore the page must not be in the book

So let's say the number of pages is 120, and the page we want to find is the same as the last one, we would need to iterate all over the list
the find the page 120, so in the worst case scenario, we would have to go through all the elements in the list, which might seem really expensive (and it can be for huge lists), but it is usually really fast because of cache locality.
Because of this limitation, we say that linear search has a complexity of *O(n)* such that n is the number of the input list.

## Implementation

The implementantion is pretty much the same in every single language, but let us implement it in C:

```c
#include <stdint.h>

uint8_t linear_search(uint8_t page_to_search_by, uint8_t *pages, uint_8 pagenumber)
{
    uint8_t i = 0;
    while (i < pagenumber)
    {
        if (pages[i] == page_to_search_by) return 1;
        i++;
    }
    return 0;
}
```

## When should I use linear search?

Linear search should be used whenever we cannot apply some sort of sorting to all the elements of the list, or the list is not really big.
Small lists can fit in L3 cache and are really fast for the computer to access so it is a lot faster to just go through the whole list than doing something more sophisticated such as [binary search](https://dev.to/gustrb/binary-search-j5f).
If the list can be big and it is sorted, binary search is the best approach.
Introdução às Coroutines em Kotlin: Simplificando Operações Assíncronas
Published: 2024-01-31 19:28:02


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/cw98jgyx01dwbzxjokd1.gif)

**Introdução:**

Coroutines em Kotlin oferecem uma maneira eficiente de lidar com operações assíncronas, tornando o código mais legível e fácil de manter. Elas são uma alternativa leve às tradicionais callbacks ou às primitivas de concorrência, permitindo que você escreva código assíncrono de maneira sequencial. Neste tutorial, exploraremos uma introdução básica às coroutines em Kotlin, apresentaremos um exemplo prático e concluiremos com insights sobre seu uso e benefícios.

**Exemplo:**

Vamos criar um exemplo prático usando coroutines para gerar números de pedido de maneira assíncrona. Primeiro, definiremos uma função generate que simula uma operação assíncrona para obter um número de pedido:

```
import kotlinx.coroutines.*

suspend fun generate(): String {
    val currentLocale: java.util.Locale = java.util.Locale.getDefault()
    val countryCode: String = currentLocale.country
    val order = generateSequence(1) { it + 1 }
        .take(1)
        .distinct()
        .sorted()
        .toSet()

    val number = order.stream().findFirst().get()
    val orderNumber = countryCode + number
    println("Order number: $orderNumber")
    return orderNumber
}
```
Agora, usaremos runBlocking para criar um escopo de coroutine bloqueante e chamar a função generate de maneira síncrona:

```
import kotlinx.coroutines.runBlocking

fun main() {
    runBlocking {
        println("Start of execution ")
        val orderNumber: String = generate()
        println("Coroutine completed with order number: $orderNumber")
    }

    println("Main program continues while the coroutine is running.")
}
```

**Conclusão:**

Como podemos ver, as coroutines em Kotlin oferecem uma abordagem simples e poderosa para lidar com operações assíncronas. Elas simplificam o código, tornando-o mais compreensível e evitando o callback hell associado a abordagens tradicionais. A utilização de runBlocking para simplificar o uso de coroutines em funções main ou em scripts pode ser adequada em certos contextos, mas em aplicações mais complexas, é recomendável o uso de escopos de coroutine mais específicos. Ao integrar coroutines em seus projetos Kotlin, você pode aproveitar os benefícios da concorrência sem sacrificar a legibilidade do código.

**Fontes:**

- https://kotlinlang.org/docs/coroutines-overview.html

- https://www.baeldung.com/kotlin/coroutines
Hybrid And Private Cloud (Enter Softiron)
Tags: kubernetes, cloud, git, programming
Published: 2024-01-31 19:23:34

At what point do you sit back and think "perhaps running every workload in a public cloud makes no sense.".

Without getting too buzzy, the whole idea of "cloud repatriation", which is simply just saying "hey, we don't want to run everything in the cloud anymore. Let's bring some stuff back to our datacenters" is a real thing. Not because it's new (running on-prem was where tech started", but because it's beginning to make more and more sense.

Let's talk about how it can be done properly with private cloud.

## Why Private Cloud?

The idea of "creating your own cloud" started many years ago, but got pretty popular with OpenStack. The gist essentially is if you:

1. Don't want to run in a public cloud.
2. Can't run in a public cloud for whatever reason (like government reasons).
3. Don't want to spend money on public cloud.

You can create your own cloud-like experience. It'll be on your own hardware (vs the cloud provider's hardware) and run in your datacenters. It'll look, smell, and feel the same as a public cloud, but it'll just be in your datacenters.

### Security

Even in 2024, you'll notice that a lot of banks, healthcare, and government sections of the world are still running on-prem. The reason why is that they need to fully control their data, where the data is, and what systems can see the data. Using hybrid cloud (both on-prem and in the cloud) gives the banks, healthcare, and government sectors the best of both worlds.

## Building Your Own Hardware

As you're building out a Private Cloud, you're ultimately responsible for hardware components like:

1. CPU
2. RAM (memory)
3. Motherboards

And outside of that, you're responsible for the OS that you're running on. This opens up not only security issues but overall configuration and expertise around what's necessary.

Ironsoftware builds all of its servers and most of the chipset in-house without the need to outsource it.

## Scaling

Hypercloud, multi-cloud, hybrid-cloud, or whatever you'd like to call it needs one primary piece to the puzzle - scale out. For example, what do a lot of people want to utilize multi-cloud for? Well, they don't actually need to use multiple services. Really what they want is multi-region.

Furthermore, what they want is the ability to scale out to other services. For example, let's say you're in a hybrid cloud environment and you for whatever reason run out of storage for a particular workload on-prem. The ideal scenario would be to scale out from the on-prem storage to, for example, Azure storage or an AWS S3 bucket.

Ironsoftware gives the ability to scale out to public cloud providers in a scenario like this.

## What Works For You?

Regardless of Kubernetes, containerization, or the public cloud, there are still a massive amount of datacenters running. Bare metal still exists and VMs still exist. The truth is that cloud-native doesn't have anything to do with the public cloud. It has to do with how workloads are configured, how they scale, and the fact that it's API/programmatically driven.
Capitalisation of table name generated by Hibernate when using MySQL server
Tags: jpa, hibernate, mysql
Published: 2024-01-31 19:22:09

## Notice

I wrote this article and was originally published on [Qiita](https://qiita.com/saladlam/items/845175c631e2fb0151fe) on 17 September 2022.

---

# My problem

I am working on a Spring Boot project with JPA (Hibernate implementation), and the database underlying is MySQL running on Windows. The table names are in lower case. One day I moved the database from Windows to Linux, imported all data, and then restarted the application. Hibernate created some of the tables for me, with names same as entity class, including capitalisation. And it reads the data from the new table instead of the old table with lower case name. I tried to find out why and finally got the reason.

# Database and table name of MySQL

First I need to talk about history. In the time of MyISAM storage engine, a database is just a folder and a table is a group of files in that folder. The name of the database is the name of the folder. In Windows, when accessing a folder, the case of name is not considered, so "Student" and "student" means the same folder. But in Linux, the folder name is case sensitive. So when running two SQL query below, different behavior can be seen when running on Windows and Linux.

```sql
SELECT `name`, `class` from `student`;
SELECT `name`, `class` from `Student`;
```

And now **lower_case_table_names** system variable determines the behavior of case of table name. For details please read [this](https://dev.mysql.com/doc/refman/8.0/en/identifier-case-sensitivity.html).

# When Hibernate will help you generate table name

When an entity class is not specified a table name by @Table annotation, Hibernate will determine the table name. Hibernate first read the system variable **lower_case_table_names**, and then decided to use the name including capitalisation or not by this value.

# Back to my problem

In the newly setup Linux MySQL database, add the following line into my.cnf. Restart both MySQL server and my application, and the problem is solved.

```ini
[mysqld]
lower_case_table_names = 1
```
Network limitation of Blink security camera
Tags: networking
Published: 2024-01-31 19:16:24

## Notice

I wrote this article and was originally published on [Qiita](https://qiita.com/saladlam/items/2381d1cdeb07540c81cf) on 26 February 2023.

---

# Network limitation of Blink security camera

1. it supports IPv4 only
1. the MTU of all sections of network to its API server must equal or greater than 1500 (for example it will not work if a section is tunneled by pptp, which MTU is 1450)
1. it uses HTTPS (tcp port 443) to upload video stream (boths real time and motion detection) and thumbnail
Running network tunnel under TCP is horrible idea
Tags: networking
Published: 2024-01-31 19:12:17

## Notice

I wrote this article and was originally published on [Qiita](https://qiita.com/saladlam/items/7f92fe34304566e42b3d) on 26 February 2023.

---

This passage is to explain why running network tunnel under TCP (for example running OpenVPN under TCP) is horrible idea.

# Characteristic of TCP

1. **Stateful**: sequence number is marked on every packet, so receiver can reconstruct the data stream ever if packet is received out of sequence

1. **Lossless**: receiver need to acknowledge which packet is received, otherwise sender will resend it. Furthermore flow control mechanism is also implemented in TCP, sender will limit number of packet send if resend mechanism is triggered.

# Why it is horrible idea

Stateful and lossless are not necessary for network tunnel protocol. Sequence maintenance of packet of TCP increase the latency of tunnel traffic because tunnel destination must wait for packet with small sequence number before sending already received packet with large sequence number.

Lossless is useless because packet tracing will be handled by destination (if TCP is used).

Furthermore, flow control mechanism makes thing worse. For example, current throughput of tunnel is 1Mbps, if one of the packet is lost (packet loss is very common in network), the throughput will drop significantly due to number of packet send is limited in resend mechanism. As a result tunnel buffer is full and packet will be dropped.
Some useful OpenSSL command
Tags: linux, cryptography
Published: 2024-01-31 19:08:09

## Notice

I wrote this article and was originally published on [Qiita](https://qiita.com/saladlam/items/d048a7decc23f9928e3d) on 4 March 2023.

---

OpenSSL is a swiss army knife of cryptography.

# Help

```sh
# show option of enc command
$ openssl enc help
Usage: enc [options]
Valid options are:
 -help               Display this summary
 -list               List ciphers
 -ciphers            Alias for -list
 -in infile          Input file
 -out outfile        Output file
 -pass val           Passphrase source
 -e                  Encrypt
 -d                  Decrypt
 -p                  Print the iv/key
 -P                  Print the iv/key and exit
 -v                  Verbose output
 -nopad              Disable standard block padding
 -salt               Use salt in the KDF (default)
 -nosalt             Do not use salt in the KDF
 -debug              Print debug info
 -a                  Base64 encode/decode, depending on encryption flag
 -base64             Same as option -a
 -A                  Used with -[base64|a] to specify base64 buffer as a single line
 -bufsize val        Buffer size
 -k val              Passphrase
 -kfile infile       Read passphrase from file
 -K val              Raw key, in hex
 -S val              Salt, in hex
 -iv val             IV in hex
 -md val             Use specified digest to create a key from the passphrase
 -iter +int          Specify the iteration count and force use of PBKDF2
 -pbkdf2             Use password-based key derivation function 2
 -none               Don't encrypt
 -*                  Any supported cipher
 -rand val           Load the file(s) into the random number generator
 -writerand outfile  Write random data to the specified file
 -engine val         Use engine, possibly a hardware device
```

# Generate symantec key

```shell
# generates 16 bytes (128 bits) key in binary format
$ openssl rand 16

# in hex format
$ openssl rand -hex 16
88cdbf1e106334f1bca57f730758abad

# encoded by BASE64 
$ openssl rand -base64 16
tMa1eyjIZw3g3M0dhPC87A==

# show hex format of the binary key saved in a file
$ xxd -ps symantec.key
88cdbf1e106334f1bca57f730758abad
```

# Encode/decode file by AES

Assume using aes-128-cbc algorithm (128 bits key), with 128 bits initialization vector and no salt. Padding algorithm is PKCS#5.

```sh
# encode, -K is key in hex format, -iv is initialization vector in hex format
$ openssl enc -e -aes-128-cbc -K abcdef01234567890123456789abcdef -iv 0123456789abcdef0123456789abcdef -nosalt -in file.txt -out file.txt.encode

# decode
$ openssl enc -d -aes-128-cbc -K abcdef01234567890123456789abcdef -iv 0123456789abcdef0123456789abcdef -nosalt -in file.txt.encode -out file.txt
```

# Establish HTTPS connection

```sh
$ openssl s_client -connect www.google.com:443
```
Use cases of wget
Tags: linux
Published: 2024-01-31 19:03:34

## Notice

I wrote this article and was originally published on [Qiita](https://qiita.com/saladlam/items/7aee55b20f0e4dd2cf09) on 18 March 2023.

---

# download files from list

assumes list.txt contains following

```text
voice-record.mp3
agenda.docx
minutes.docx
```

after executes this command

```sh
wget -i list.txt --base="http://your.site/page/"
```

following files will be downloaded

```text
http://your.site/page/voice-record.mp3
http://your.site/page/agenda.docx
http://your.site/page/minutes.docx
```

# add headers of http request

```sh
wget --referer="https://www.bbc.co.uk/" --header="User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/110.0" "https://www.bbc.co.uk/weather"
```

# check web server is ready to serve after restart  

```sh
wget -O/dev/null --timeout=5 --waitretry=15 --tries=27 --retry-connrefused --quiet "http://your.site/page"
```

Options
- -O/dev/null: saves downloaded content to /dev/null, i.e. doesn't keep
- timeout=5: 5 seconds connection timeout
- waitretry=15: waits up to 15 seconds in next try. By default, extra 1 second waiting time is added on next try, start from 1 second  
- tries=27: tries to connect 27 times 
- retry-connrefused: retries even though destination refuses to connect. Without apply this flag program terminates when destination refuses to connect  
- quiet: doesn't output anything to console

wget returns 0 when success, otherwise returns 4.
Building and Running llamafile with a GPU
Published: 2024-01-31 18:46:43

Getting llamafile running with a GPU can be a little tricky. For reference, I have a four-year-old Kubuntu laptop with a Nvidia RTX 2060.

Check to see if the correct Nvidia driver is installed.

```
nvida-smi
```
If the driver is installed, you will see a similar output.

![nvidia-smi output](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0pa0yoq2t9pdsexpol14.png)


## Building llamafile

To build llamafile on Ubuntu, make sure you have the git, GCC, make, and curl installed.

```
sudo apt-get install -y curl git gcc make
```

Next clone the llamafile repository and download unzip, make it executable and move it to `/usr/local/bin` so it's in the PATH.

```
git clone https://github.com/Mozilla-Ocho/llamafile.git
curl -L -o ./unzip https://cosmo.zip/pub/cosmos/bin/unzip
chmod 755 unzip 
mv unzip /usr/local/bin
```

Build and install llamafile.

```
make -j8
make install PREFIX=/usr/local
```

## Running llamafile with the GPU enabled

[Llama.cpp](https://github.com/ggerganov/llama.cpp) will need [clang](https://clang.llvm.org/), cuda-toolkit, and nvidia-gds to enable the GPU. You'll need the kernel headers and development packages for your kernel if they are not installed. Additionally, you will need to remove the outdated signing key. 

```
sudo apt install linux-headers-$(uname -r)
sudo apt-key del 7fa2af80
sudo apt update
sudo apt install -y clang 
sudo apt install -y cuda-toolkit
sudo apt install -y nvidia-gds
```

To run llamafile with the GPU, add the option `--ngl 9999`. This example runs llamafile as a server with the GPU enabled. 

```
llamafile --server -ngl 999 --host 0.0.0.0 -m codellama-7b-instruct.Q4_K_M.gguf
```

The RTX 2060 with 6GB can run models with 7B parameters. Note that llamafile will core dump if you try to run a model larger than what the GPU memory can hold. 

Even with an older GPU, the number of tokens per second is almost two orders of magnitude than running with the CPU.

![llamafile log](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/dh0wd9svliv2u1l87pgg.png)