teleren/devto huggingface dataset:
Memory-Efficient 2D Array Creation for Lower Triangle Matrices
Tags: dsa, array, programming, algorithms
Published: 2023-08-12 14:14:12

This blog discusses a technique to reduce memory usage while creating lower triangle matrices. The technique involves storing the matrix elements in a single array, with each element’s index calculated based on its row and column position. This can significantly reduce memory usage, as only the non-zero elements of the matrix need to be stored. The technique is also relatively simple to implement, making it a valuable tool for programmers who need to create lower triangle matrices in a memory-efficient way.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/eqh8cb11nijuitzurfzi.png)

A lower triangular matrix is a square matrix in which all the elements above the main diagonal are zero, in the above diagram, all the red portions are zero while the remaining green portions are non-zero elements.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kjxxct6m2kym68f0he9t.png)

In this blog, we’ll learn how to transform the 2-D matrix to 1-D array and reduce the number of memory blocks which are unused. To do so, we need some kind to formula to transform it, so let’s learn how to implement the formula. First, we’ve to figure out the formula for number of zeros and non-zero’s element in 2-D array.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9k5tzjeloiz5syxs28pk.png)

In the above diagram, in 1st row, we’ve 1 non-zero block. In 2nd row, we’ve 2 non-zero blocks. Likewise for 4th row, we’ve 4 non-zero blocks. Therefore, for n x n array, we’ve 1 + 2 + 3 + 4 + … + n non-zero blocks. This is the series for sum of n natural numbers, so the formula for it is n * (n + 1) / 2.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/64nd32gr82l5uuc45liz.png)

So, to convert 2D matrix, we’ll use ROW-MAJOR formula, which we’ve learn in previous blog post. In the above diagram, we’ve split the array in different rows, 1st row contains 1 element, 2nd row contains 2 element, likewise nth row contains n element.

For accessing element at A [4][3], we’ve to generate a formula such that it points to index 8. So, let’s generate the formula.

```
A[4][3] = [1 + 2 + 3] + (3 - 1) = 8

A[3][2] = [1 + 2] + (2 - 1) = 4

General formula,
A[i][j] = [summation of all element till i-1] + (j - 1)

A[i][j] = [1 + 2 + 3 + ... i-1] + (j - 1)
        = [i * (i-1) / 2] + (j - 1)
```

I hope, you we’ve learn something new from this blog. Do like, share and comment on the blog post. Also, if you’ve some doubt, do comment down the blog, I’ll try my best to answer your questions.
Development life cycle productivity tools
Tags: productivity, management, pr, github
Published: 2023-08-12 14:22:03

---
title: Development life cycle productivity tools
published: true
description: few tools we added recently to our development life cycle
tags: productivity, management, PR, GitHub,
# cover_image: https://direct_url_to_image.jpg
# Use a ratio of 100:42 for best results.
# published_at: 2023-08-12 14:10 +0000
---
a few nice tools we adopted recently to ease development process and improve productivity - and are very happy with:

***Add context automatically to any PR [LinearB GitStream](https://github.com/linear-b/gitstream):*** labels a PR with some context, i.e: time-to-review, missing-tests, deleted-files, big-change etc. very helpful when having several open PRs and want to focus on the high-impactful ones.

![Context](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0mhbaxbsdf7aicqxqva2.png)



***Automatic PR code review [CodiumAI PR Agent](https://github.com/Codium-ai/pr-agent):*** performs automatic code reviews, suggests code changes and shortly describes any PR. biggest success so far.


![PR review](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9psey48njwzp7dos60d4.png)

![PR description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ijxv62lw9oul3s8sab60.png)


***Security & static analysis checks on PR [Jit.io](jit.io):*** runs dependency checks, static analysis and IaC misconfigurations on every new introduced changes. can easily trigger slack notification if there are any new “security” issues introduced during development.

![IAC Misconfig](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/b4eluv0vuebchjw8ghxe.png)


***Auto approve PR’s according to logic (GitStream):*** tests-only change, .yml changes, etc

![Auto approve as safe change](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bplr60j6njtaxta96ls5.png)


***GitHub discussions:*** using existing templates to perform service-related discussions. in our case, this was helpful when conducting technical design meetings, allowing offline process to occur (something similar to code review).

We’re looking for talented devs to join our team, feel free to PM.
Trying out new Github tools, the Github AI chat bot in VSCode
Tags: programming, github, ai, chatbot
Published: 2023-08-12 13:50:21

This post will be about my experience using the AI chatbot extension on vscode. This feature is available to all users that subscribed to Github co-pilot project. 

Recently , I signed up for Github chatbot preview, so one fine day I received email from Github notifying me that I am enrolled as a new user for the chatbot preview project.

So without wasting any more time, I followed their instructions on how to get the chatbot extension installed in vscode. 

The looks.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/f1bcns8mh14rno1c99jc.png)

The chatbot extension will sits somewhere on the left side of vscode. You can simply prompt the AI to generate code for you, In this case, I used it to generate html for a twitter button in Tailwind css. 

Overall. It is a good extension to have, the AI extension at times may be slow to respond, I suspect due to the bad wifi coverage in my workstation. 

As for the code generation, the code generated - almost similar to that code generated by ChatGPT - as a developer you still have to test and verify the code works as intended. 

No one wanted to debug bad codes that deployed into production  , no one wanted to have security risks using code that not properly tested. 

Next - I will try using it to generate js extension that will sync my blog , my Dev.To account , and my personal website. I will share the code - if it works :-)
Streaming Large Files with PHP to Save Memory
Tags: webdev, php, files
Published: 2023-08-12 13:24:52

Handling large files is a common task for web developers. However, if not done properly, it can lead to high memory usage and slow performance. In this tutorial, we will look at how to stream large files to the browser in a memory-efficient way using PHP.

## Prerequisites

- Basic knowledge of PHP
- An active PHP development environment

## Step 1: File Streaming Basics

Before we dive into the code, let's first understand what file streaming is. When you open a file in PHP with `fopen()`, you can read it line by line or character by character with `fgets()` or `fgetc()`, instead of loading the entire file into memory. This is known as file streaming.

## Step 2: Setting Up the PHP Script

Let's create a new PHP script, `download.php`. In this script, we will:

1. Open the file we want to stream.
2. Read a portion of the file and output it to the browser.
3. Repeat step 2 until the entire file has been read and sent.

Here's the code:

```php
<?php
$file = 'path/to/your/largefile.pdf';

// Make sure the file exists
if (!file_exists($file)) {
    die('File not found.');
}

// Set headers to tell the browser to download the file
header('Content-Type: application/octet-stream');
header('Content-Disposition: attachment; filename="'.basename($file).'"');
header('Content-Length: ' . filesize($file));

// Open the file in binary mode
$fp = fopen($file, 'rb');

// Output the file
while (!feof($fp)) {
    // Read and output a chunk of the file
    echo fread($fp, 8192);

    // Flush the output buffer to free up memory
    ob_flush();
    flush();
}

// Close the file
fclose($fp);
exit;
```

In the above code, we use [`fopen()`](https://php.net/fopen) to open the file and [`fread()`](https://php.net/fread) to read a chunk of 8192 bytes at a time (which is approximately 8KB). We then output this chunk using `echo` and `flush` the output buffer to free up the memory used. This process repeats until the end of the file (`feof($fp)` returns `true`).

In this tutorial, you've learned how to stream large files to the browser in a memory-efficient way using PHP. This method is very useful when dealing with large files that could otherwise consume significant server memory and lead to performance issues. Always remember to close any open file handles and flush the output buffer to free up memory.
createFactory in React. 🏭 important function in React
Tags: webdev, react, programming, frontend
Published: 2023-08-12 13:19:39

To create a factory in React, you can use the `createFactory` function from the React library. The `createFactory` function takes a component class as its argument and returns a factory function. The factory function can then be used to create new instances of the component class.


![createFactory ](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mygqgis7swi3t0yt2xd5.png)

Here is an example of how to create a factory in React:

```javascript
import React, { createFactory } from 'react';

const MyComponent = ({ name }) => {
  return (
    <div>
      Hello, {name}!
    </div>
  );
};

const MyFactory = createFactory(MyComponent);

export default function App() {
  return (
    <MyFactory name="John Doe" />
  );
}
```

In this example, we first create a component class called `MyComponent`. The `MyComponent` class takes a `name` prop as its argument and renders a greeting to the user.

Next, we use the `createFactory` function to create a factory function for the `MyComponent` class. The factory function is called `MyFactory`.

Finally, we use the `MyFactory` function to create a new instance of the `MyComponent` class and render it to the DOM.

The factory pattern is a useful design pattern in React when you need to create different objects based on different conditions or inputs. For example, you could use the factory pattern to create different types of buttons, or to create different types of forms.

The factory pattern can also be used to decouple the object creation logic from the client code. This can make your code more modular and easier to maintain.

Thank you for reading. I encourage you to follow me on Twitter where I regularly share content about JavaScript and React, as well as contribute to open-source projects. I am currently seeking a remote job or internship.

Twitter: https://twitter.com/Diwakar_766

GitHub: https://github.com/DIWAKARKASHYAP

Portfolio: https://diwakar-portfolio.vercel.app/
File and Directory Operations in Linux Commands
Tags: linux
Published: 2023-08-12 08:12:21

---
title: File and Directory Operations in Linux Commands
published: true
description: 
tags: Linux
---
## Creating and Removing
- `touch`: Create an empty file.
  ```bash
  $ touch newfile.txt
  ```

- `mkdir`: Create a new directory.
  ```bash
  $ mkdir new_directory
  ```

- `rm`: Remove files.
  - `-r (or --recursive)`: Remove directories and their contents.
  - `-f`: Force removal without confirmation.
    ```bash
    $ rm file_to_remove.txt
    $ rm -rf directory_to_remove/
    ```

- `rmdir`: Remove an empty directory.
  ```bash
  $ rmdir empty_directory
  ```

## Copying, Moving, and Renaming
- `cp`: Copy files or directories.
  - `-r`: Recursively copy directories.
  - `-i`: Prompt before overwriting files.
    ```bash
    $ cp source.txt destination.txt
    $ cp -ri source_directory/ destination_directory/
    ```

- `mv`: Move or rename files or directories.
  - `-i`: Prompt before overwriting files.
    ```bash
    $ mv -i oldname.txt newname.txt
    ```

## Listing and Navigating
- `ls`: List directory contents.
  - `-l`: Display in long format.
  - `-a`: Show hidden files.
  - `-h`: Display sizes in human-readable format.
    ```bash
    $ ls -lah
    ```

- `pwd`: Print the current working directory.
  ```bash
  $ pwd
  ```

- `cd`: Change the current directory.
  ```bash
  $ cd /path/to/directory/
  ```
## File Content Viewing
- `cat`: Display the entire content of a file.
  ```bash
  $ cat file.txt
  This is the content of file.txt.
  ```

- `less`: View file content with pagination. No direct output, but provides an interactive view.
  ```bash
  $ less file.txt
  ```

- `head`: Display the beginning of a file.
  ```bash
  $ head file.txt
  First line of the file.
  ```

- `tail`: Display the end of a file.
  - `-f`: Follow the content of a file in real-time.
    ```bash
    $ tail -f /var/log/syslog
    Aug 11 12:34:56 hostname program[1234]: Log message here.
    ```

## File and Directory Information
- `stat`: Display detailed information about a file or directory.
  ```bash
  $ stat file.txt
  File: file.txt
  Size: 1234       Blocks: 8          IO Block: 4096   regular file
  Device: 801h/2049d      Inode: 123456      Links: 1
  Access: (0644/-rw-r--r--)  Uid: ( 1000/    user)   Gid: ( 1000/    user)
  ```

- `file`: Determine the type of a file.
  ```bash
  $ file image.jpg
  image.jpg: JPEG image data, JFIF standard 1.01
  ```

## Searching for Files
- `find`: Search for files in a directory hierarchy.
  ```bash
  $ find /path/to/search/ -name "pattern*"
  /path/to/search/pattern1.txt
  /path/to/search/subdir/pattern2.txt
  ```
What you learning about this weekend?
Tags: codenewbie, discuss, learning, beginners
Published: 2023-08-12 13:00:00

Hey folks 👋

What y'all learning about this weekend?

Whether you're sharpening your [JS](https://dev.to/t/javascript) skills, making PRs to [your OSS repo of choice](https://github.com/forem/forem) 😉, sprucing up your portfolio, or [writing a new post](https://dev.to/new) here on DEV, we'd like to hear about it.

Expand your mind!

![A cartoon of a person reading books as their head grows larger like a balloon](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9l1td9wpgkfr6yxool6e.gif)
Text Processing in Linux
Tags: linux
Published: 2023-08-12 08:12:20

---
title: Text Processing in Linux
published: true
description: 
tags: Linux
---

## Searching within Text
- `grep`: Search for a pattern within files.
  - `-i`: Ignore case.
  - `-r` or `-R`: Recursively search in directories.
  - `-v`: Invert the search (return lines that don't match).
    ```bash
    $ grep "search_term" file.txt
    This line contains the search_term.
    ```
    ```bash
    $ grep -i "Search_Term" file.txt
    This line contains the search_term.
    ```

## Filtering and Transforming Text
- `awk`: A versatile programming language for working on files.
  ```bash
  $ echo -e "a 1\nb 2\nc 3" | awk '{print $2}'
  1
  2
  3
  ```

- `sed`: Stream editor for filtering and transforming text.
  ```bash
  $ echo "Hello World" | sed 's/World/Universe/'
  Hello Universe
  ```

## Sorting and Counting
- `sort`: Sort lines in text files.
  - `-r`: Reverse the result of comparisons (descending).
  - `-n`: Compare according to string numerical value.
    ```bash
    $ echo -e "banana\napple\ncherry" | sort
    apple
    banana
    cherry
    ```

- `uniq`: Report or omit repeated lines.
  - `-c`: Prefix lines by the number of occurrences.
    ```bash
    $ echo -e "a\na\nb\nb\nb\nc" | uniq -c
      2 a
      3 b
      1 c
    ```

  `wc`: Count words, lines, and characters in files.
    - `-l`: Count lines.
    - `-w`: Count words.
    - `-c`: Count bytes.
      ```bash
      $ echo "Hello World" | wc -w
      2
      ```

## Text Display
- `cat`: Concatenate and display the content of files.
  ```bash
  $ cat file.txt
  This is the content of file.txt.
  ```

- `tac`: Display files in reverse.
  ```bash
  $ echo -e "line1\nline2\nline3" | tac
  line3
  line2
  line1
  ```

- `nl`: Number lines of files.
  ```bash
  $ echo -e "line1\nline2\nline3" | nl
  1  line1
  2  line2
  3  line3
  ```

  - `head`: Display the beginning of a file.
    - `-n`: Specify the number of lines to show.
      ```bash
      $ echo -e "line1\nline2\nline3" | head -n 2
      line1
      line2
      ```

  - `tail`: Display the end of a file.
    - `-n`: Specify the number of lines to show.
    - `-f`: Follow the content of a file in real-time.
      ```bash
      $ echo -e "line1\nline2\nline3" | tail -n 2
      line2
      line3
      ```
Automating Fonts Previews with Python and Pillow
Tags: python, ux, automation, programming
Published: 2023-08-12 12:27:51

Selecting fonts from lists is a pain when you cant visualize how they'll look. Thats why I wrote a Python script using Pillow to auto-generate font previews. 

```python
# we're importing the needed libraries
import json
import requests
from PIL import Image, ImageDraw, ImageFont
from io import BytesIO
import os

# open fonts.json file to load fonts
with open('fonts.json') as f:
    fonts = json.load(f)

# set directory for font previews
outdir = './font_previews'

# creates previews directory if it doesn't exist
os.makedirs(outdir, exist_ok=True)

# for each font in the fonts list, it will attempt the following:
for idx, font in enumerate(fonts):

    try:
        # downloads the font file from the URL provided in the font list
        response = requests.get(font['url'])

        # create a bytesIO object with downloaded font. This object behaves like a file object.
        font_file = BytesIO(response.content)

        # if the response from the server is not successful (status code 200),
        # it will print a message and will skip the rest of the loop for the current font
        if response.status_code != 200:
            print(f"Skipping font {font['postscript_name']} due to unsuccessful download from {font['url']}.")
            continue

        # creates a new image with white background
        img = Image.new('RGB', (500, 200), color='white')

        # we can now do drawing operations on this opened image
        draw = ImageDraw.Draw(img)

        # loads the downloaded font file to be later used on the image
        loadfont = ImageFont.truetype(font_file, 36)

        # calculates size in pixels of the font family string
        left, upper, right, lower = draw.textbbox((0, 0), font['family'], font=loadfont)
        w, h = right - left, lower - upper

        # calculates the center position in the image for our string to be placed
        text_pos = ((img.width - w) / 2, (img.height - h) / 2)

        # places the text (the font family string) at the calculated position
        draw.text(text_pos, font['family'], fill='black', font=loadfont)

        # sets the output filename
        outfile = os.path.join(outdir, font['postscript_name'] + '.png')

        # and saves the image in a PNG format
        img.save(outfile)

        # notifies by printing
        print(f"Saved {outfile}")

    except Exception as e:
        # handles exceptions during font handling, notifies about the error.
        print(f"An error occurred with font {font['postscript_name']}: {e}")

```

Read the full article here: https://www.seeratawan.me/blog/generating-font-previews-with-python-pillow-for-enhanced-ux/
DIGITAL MARKETING
Published: 2023-08-11 18:47:31

[CBA IT & Freelancing institute](https://www.timemaghunt.com/), a leading IT training and Service institute in Bangladesh, has set its eye on providing the necessary support to individuals who want to exploit the opportunity and make their mark in digital marketing. CBA IT Search Engine Optimization solutions are usually unique to the industry along with provable results. Our SEO team techniques have been proven to increase website authority with on and off site optimization strategies which usually work in together with content marketing and social media marketing to boost your visibility on search engine result pages. Top 10 Search Engine Rankings are guaranteed for your website at an affordable price. Our digital marketing course will not only enable you to have a good understanding of the fundamentals of digital marketing but also generate a strategy for career growth. To help you understand the market behavior and formulate a productive approach, we have covered all the aspects of digital marketing in our course. After completing the course, you will have an in-depth understanding of SEO, Google Ranking Factors & Affiliate Marketing and how to envision the market scenarios and use them to good effect.
Built tic tac toe game using javascript
Tags: javascript, tictactoe, gamedev, programming
Published: 2023-08-12 12:09:19


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rju0uf87nt3ibrq6se2b.png)

Hello guys, if you like to try new stuffs in javascript and create something interesting this post is for you. Here we created a tictactoe game using javascript with little bit of UI designs.

Try it here - https://codepen.io/indrakantmishra/pen/bGQPOra

Hope you like it :)
Integrate Pyspark Structured Streaming with confluent-kafka
Tags: apachespark, pyspark, apachekafka, confluentplatform
Published: 2023-08-12 12:05:40

Apache Spark has revolutionized big data processing with its lightning-fast processing capabilities. With its built-in streaming library, Spark Streaming, developers can easily process and analyze streaming data. However, when it comes to integrating Spark Streaming with Apache Kafka, the process can be a bit challenging. Fortunately, the open-source community has come up with a solution: Pyspark Structured Streaming with confluent-kafka.

Pyspark Structured Streaming is a high-level API that simplifies the development of real-time data processing applications. It provides a DataFrame and SQL-like interface, making it easier for developers to express complex streaming queries. On the other hand, confluent-kafka is a Python client for Apache Kafka that provides a high-performance, low-latency, and fault-tolerant stream processing solution.

Integrating Pyspark Structured Streaming with confluent-kafka is straightforward. First, you need to install the required dependencies. You can use pip, the Python package manager, to install the necessary packages:

`pip install pyspark confluent-kafka`

Once the dependencies are installed, you can start writing your Pyspark Structured Streaming application. Here's a simple example that reads data from a Kafka topic and performs some transformations:

`from pyspark.sql import SparkSession spark = SparkSession.builder \ .appName("Pyspark Structured Streaming with confluent-kafka") \ .getOrCreate() df = spark.readStream \ .format("kafka") \ .option("kafka.bootstrap.servers", "localhost:9092") \ .option("subscribe", "my_topic") \ .load() # Perform transformations on the DataFrame query = df.writeStream \ .outputMode("append") \ .format("console") \ .start() query.awaitTermination()`

In this example, we create a SparkSession and read data from a Kafka topic called "my_topic". We can then perform various transformations on the DataFrame, such as filtering, aggregating, or joining. Finally, we write the transformed data to the console.

And there you have it! You have successfully integrated Pyspark Structured Streaming with confluent-kafka. Now you can process and analyze real-time streaming data with ease.

References:
-----------

*   Apache Spark - [https://spark.apache.org/](https://spark.apache.org/)
*   Apache Kafka - [https://kafka.apache.org/](https://kafka.apache.org/)
*   Pyspark Structured Streaming - [https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html](https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html)
*   confluent-kafka - [https://github.com/confluentinc/confluent-kafka-python](https://github.com/confluentinc/confluent-kafka-python)