teleren/devto huggingface dataset:
DevOps Interview: Replica sets vs Daemon sets
Tags: devops, sitereliabilityengineering, kubernetes, replicasets
Published: 2023-12-10 18:04:21

Replica sets and Daemon sets are controllers in Kubernetes used to manage the lifecycle of Pods. They are used to manage Pods.

Let's check the differences between them.

1. Replica sets ensure that a certain number of copies should be running on nodes. Daemon sets ensure exactly one copy running on all or certain nodes.

2. Replica sets can be scaled up or down by assigning "replicas" field in the manifest file. Daemon sets scales automatically when the node number changes.

3. Replica sets are used for stateless applications to maintain a certain number of instances of the application. Daemon sets are used for deploying backend services like monitoring agents, logging agents etc.

4. Let's see the manifest file for both.

Replica sets-

```
 apiVersion: apps/v1
 kind: ReplicaSet
 metadata:
   name: <any name>
 spec:
   replicas: <any number>
   selector:
     matchLabels:
       app: <any name> # any label based on your need
   template:
     metadata:
       labels:
         app: <webserver>
     spec:
       containers:
       - name: <any name>
         image: nginx:latest
         ports:
         - containerPort: 80

```
Daemon sets-

```
 apiVersion: apps/v1
 kind: DaemonSet
 metadata:
   name: <any name>
 spec:
   selector:
     matchLabels:
       app: < any name >  # any label based on your need
   template:
     metadata:
       labels:
         app: < any name>
     spec:
       containers:
       - name: <any name>
         image: fluentd:latest
         volumeMounts:
         - name: < your volume name>
           mountPath: /var/log   # any path directory on host 
       volumes:
       - name: < your volume name>
         hostPath:
           path: /var/log  # mounted volume on this path on host
node

```


Example of Replica sets - any web application with certain replicas etc.
Example of Daemon sets - agent deployed to collect logs from nodes.

Still confused !!

No need to worry. Let’s take daily life example for both.

Replica sets-

In a restaurant, the waiters (replicas) form a ReplicaSet. The manager, acting as the controller, ensures enough waiters are available to serve customers efficiently. If a waiter is absent, the manager replaces them with another waiter to maintain the desired number of serving staff.

Daemon sets-

In a city with a fleet of garbage trucks, each truck (pod) is assigned a specific area (node) to collect trash. The fleet manager, acting as the DaemonSet, ensures a truck is always present in every area (node) for efficient waste management.






That’s all. Pat yourself on the back. :)

Hope you enjoyed it. Don’t forget to like it.
My game engine: Rebel
Tags: showdev, gamedev, devjournal, programming
Published: 2023-12-10 17:09:16

Recently, after being super burned out with work, I've decided to make a game engine using LWJGL to relax (because nothing's more calming than OpenGL dev right?). Anyway, it's called Rebel, and it's currently on GitHub if you want to check it out. 

[](https://github.com/skyloft7/Rebel)

It currently only runs on Windows (just because the macOS/Linux LWJGL natives aren't added yet). And before someone points out my lack of a proper build system, I am planning to bring it to Gradle eventually.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zg0odnr5uxkcu02cjv0u.png)

Drop your thoughts below!
Cheers!
Casting Computer Network
Tags: network, networking, devops
Published: 2023-12-10 16:57:28

In computer networks, data transmission is the most important thing to pay attention to. Therefore we need a method for sending packets over a network which is usually called **Casting**

Casting is classified into three types which are **Unicast**, **Broadcast** and **Multicast**
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yarrfdxq4vzlqumrz72b.png)

## Unicast
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/t0hub14btgnwuxgvpdsm.png)

**Unicast** is a communication _one-to-one_ mode in computer networks where one sending device sends data to a particular receiving device.

In unicast, data packets are sent from one source to one destination, so that only one device receives and processes the data.

The advantage of using unicast is that the bandwidth used will be faster because it will only focus on one destination.

Examples of common uses of unicast include accessing web pages using the HTTP protocol and transferring files between two devices.

## Broadcast
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pf9t5nkqvp3alfrsitrp.png)
**Broadcast** is a _one-to-many_ communication mode in computer networks in which one device sends data to all connected devices in the same network.

The advantages of using broadcast include making it easier because it will send data to all devices at once.

An example of common use of broadcasting is the use of live streaming video which uses the UDP protocol.

In broadcast there are two types of methods that are often used that is _Limited Broadcast_ and _Direct Broadcast_.

### Limited Broadcast

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/aeujjy8oqmphheyowavi.png)

In **Limited Broadcast**, data is sent from one device to multiple devices on the same network.

### Direct Broadcast

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/e2trx4gfo7o6kukctol3.png)

In **Direct Broadcast**, data is sent from one device to multiple devices on the same network or can send to another network.

## Multicast

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7d9irag4iiv84yd7n13m.png)

**Multicast** is a _one-to-many_ communication mode in a computer network like _Broadcast_ but only a few devices can be allowed to connect to each other.

The advantages of using multicast include making it easier because it will send data to all those who are permitted to connect and the bandwidth is faster than broadcast.

An example of common use case for multicast is when a publisher only sends emails to subscriptions.
Nade - new PHP plateform
Published: 2023-12-10 16:56:31

Restez à l'affût ! Notre plateforme automobile arrive bientôt. Préparez-vous à une expérience inédite en rejoignant notre communauté dès maintenant. Soyez parmi les premiers à en bénéficier !
[nade](https://nade.ma/)
Integrate EventBridge with bunq Callbacks to track banking events.
Tags: aws, bunq, cdk, eventdriven
Published: 2023-12-10 16:51:58

The banking service [bunq](https://web.bunq.com/invite/sbstjn) supports webhooks for various account activities; they are named **bunq Callbacks**. Together with **API Gateway**, **SQS**, and **EventBridge Pipes** it's *simple* to [**integrate bunq with Amazon EventBridge**](https://sbstjn.com/blog/bunq-events-to-aws-eventbridge-bus-webhook-and-callbacks/). Based on existing code snippets, [this full-featured guide](https://sbstjn.com/blog/bunq-events-to-aws-eventbridge-bus-webhook-and-callbacks/) includes:

* [API Gateway with Service Integration for SQS](https://sbstjn.com/blog/aws-cdk-api-gateway-service-integration-sqs/)
* [EventBridge Pipe with SQS source](https://sbstjn.com/blog/cdk-eventbridge-pipes-event-bus-processing/)
* [EventBridge Bus with CloudWatch destination](https://sbstjn.com/blog/cdk-eventbridge-cloudwatch/)
* [Callback for bunq account activities](https://sbstjn.com/blog/bunq-webhook-callback-configuration-api/)

Configured using the **AWS Cloud Development Kit**, this explains how you [**setup a custom EventBridge Pipe to process and transform incoming bunq Callbacks**](https://sbstjn.com/blog/bunq-events-to-aws-eventbridge-bus-webhook-and-callbacks/) for your banking activity! [Read more …](https://sbstjn.com/blog/bunq-events-to-aws-eventbridge-bus-webhook-and-callbacks/)
How TaskWeaver Can Be Used To Create a Super AI Agent
Tags: datascience, machinelearning, programming, ai
Published: 2023-12-10 16:49:24

AutoGPT, ChatDEV, AutoGEN and metaGPT are some of the top multi-agent libraries released this year alone.

Microsoft has recently launched an exciting piece of software which is called TaskWeaver and there is so much talk about it in AI spaces.

specifically designed to allow users to turn their ideas into code with just a few commands.

TaskWeaver Microsoft is not just another tool; It’s a complex system that interprets your commands, converts them into code, and performs tasks with precision.

In this article, we are going to discuss what is TaskWeaver, what are the features of Taskweaver, the differences between Autogen and TaskWeaver and how TaskWeaver can be used to create a super AI Agent.

The Full Article can be found [here](https://quickaitutorial.com/how-taskweaver-can-be-used-to-create-a-super-ai-agent/)
Manual Testing -Benefits and Drawbacks with example
Published: 2023-12-10 16:13:49

**_Manual Testing:_**
	
   

- Manual testing is a software testing process which will done by testers manually without any code or tool to test the applications. 
-    Test cases should design for the given requirements. Test cases contains step by step instruction to proceed with the flow.
-    All test cases will be executed manually and Testers has to compare Expected vs actual results to ensure application working as expected. 
-    If  testers finds any discrepancy from the requirements and application then it should logged as defect and assigns to developers whose worked on that module. 
Once developers analyzed the issue and fixing from their end then inform to testers to retest the issue. 
   
- Testers will retest the issue if issues resolves then defect wil
l be closed else it will again back to developers queue.

Example - Login page

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wj9nsity02clh9u466mw.JPG)

Above screen requirements,
1.	User name and password is a mandatory field
2.	If both username and password is correct then login will happen
3.	If user doesn’t  register can signup

**Login Scenario:**

o	Without providing username password and click Login button should throw error message display.
o	Input wrong username and click Login button should throw error message display.
o	Input wrong password and click Login button should throw error message display.
o	If user input registered username and password and click Login button should redirect to Home screen.
o	Clicking on SignUp button should redirect to registration 


**_Benefits:_**
•	Easy to find every defects from the application 
•	No coding knowledge is needed to test
•	Compared to automation testing it is less expensive
•	Suitable when unplanned code change or deployment happen

**Drawback**:

•	It is not reliable because testers cannot cover all the aspects of the testing, Since Exhaustive testing is not possible
•	Need human intervention to test for all testing
•	Testers should have functional knowledge and experience in testing
•	Time consuming process
Best Practices for DevSecOps Implementation:
Tags: webdev, devops, devsecops, cybersecurity
Published: 2023-12-10 15:46:03

1️⃣ Shift Left, Think Secure: Start security considerations from the project's inception. By integrating security at the early stages of development, we identify and rectify vulnerabilities when they are less complex and costly to fix.

2️⃣ Automate Security Checks: Embrace automation for security testing. Automated security testing tools seamlessly integrated into the Continuous Integration/Continuous Deployment (CI/CD) pipeline allow for quick and consistent security assessments with every code commit, ensuring that security is not compromised at any stage.

3️⃣ Culture of Continuous Learning: Foster a culture of continuous learning and collaboration. Security awareness programs, workshops, and knowledge-sharing sessions empower developers, security professionals, and operations teams to stay updated with the latest threats and countermeasures.

4️⃣ Real-time Threat Detection: Implement continuous monitoring and real-time threat detection mechanisms. By actively monitoring applications and networks, we can promptly identify and respond to security threats, minimizing the potential impact on our systems.

5️⃣ Compliance and Beyond: Ensure compliance with industry standards and regulations, but don’t stop there. Go beyond compliance and strive for a security posture that exceeds the minimum requirements. This proactive approach ensures a robust defense against emerging threats.

Essential DevSecOps Tools:

1️⃣ OWASP ZAP: An open-source security testing tool that helps find security vulnerabilities in web applications during development and testing. Its dynamic application security testing (DAST) capabilities are invaluable.

2️⃣ Snyk: Snyk is a developer-first security solution that helps you use open source code and stay secure. It finds and fixes vulnerabilities for Node.js, Ruby, Python, Java, and more, empowering developers to write secure code.

3️⃣ SonarQube: SonarQube is an open-source platform for continuous inspection of code quality. It performs automatic reviews with static analysis of code to detect bugs, code smells, and security vulnerabilities on 25+ programming languages.

4️⃣ Docker Security Scanning: Docker Security Scanning automatically scans Docker images for vulnerabilities. It provides security intelligence about the software used in your application and its vulnerabilities.

5️⃣ HashiCorp Vault: HashiCorp Vault secures, stores, and tightly controls access to tokens, passwords, certificates, API keys, and other secrets in modern computing.

Incorporating these practices and tools into our development lifecycle isn’t just about ticking security checkboxes; it’s about fostering a security-first mindset that permeates every line of code we write. Let's make security not just a part of our process but a part of our DNA as technologists. Together, we can build a digital world that is not only innovative but also inherently secure. 💪
SQL - Optimizing Queries with Indexing
Tags: sql, programming, tutorial, beginners
Published: 2023-12-10 15:30:00

Effective indexing is crucial for optimizing the performance of SQL queries. Use indexes strategically on columns frequently used in WHERE clauses, JOIN conditions, and ORDER BY clauses. However, avoid over-indexing, as it can lead to increased storage requirements and slower INSERT/UPDATE operations.

Example:
Suppose you have a table called "products" with a large number of rows, and you often need to retrieve products by their category and sort them by price.

With Proper Indexing:
```sql
-- Create an index on the 'category' column if not already done.
-- Create an index on the 'price' column if not already done.
-- This is typically done once as a part of table design.
CREATE INDEX idx_category ON products (category);
CREATE INDEX idx_price ON products (price);

-- Now, use the indexes to optimize the query.
SELECT *
FROM products
WHERE category = 'Electronics'
ORDER BY price;
```

By creating indexes on both the 'category' and 'price' columns, you can significantly improve the performance of queries that filter by category and sort by price. Indexes allow the database to locate and retrieve the relevant rows more efficiently.

Remember that while indexes boost query performance, they come with storage overhead and may impact INSERT, UPDATE, and DELETE operations. Therefore, it's essential to strike a balance between indexing for read performance and maintaining reasonable write performance based on your specific database usage patterns.
Haskell
Tags: haskell, functionalreactiveprogramming, programming, watercooler
Published: 2023-12-10 15:28:03

A couple of weeks ago, I decided to learn Haskell, this beautifully designed programming language mainly by mathematicians, but I gave up after I tried to execute a certain program and the GHC refused to cooperate and compile the code !!

After a second thought, and thanks to a cool Russian programmer known as "tsoding" on YouTube, I was much more motivated to learn it, and I started watching University courses in Russian as I speak the language, and the lectures I watched were very detailed and very well constructed.

I am learning Haskell now, and I started writing all my beginner programs I wrote when I first learnt Python 8 years ago (I hate Cabal btw!).
I even have ordered this book from Empik in Poland :


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wz5lzxc6tble3pli40ao.png)


So I thought I might share with you guys who might be inspired and motivated by this little story of mine :)

#Haskell - What is it ?#
Haskell is a functional programming language, originated in the late 1980s through the collaborative efforts of a committee led by Haskell Curry. Renowned for its commitment to simplicity and mathematical precision, Haskell has evolved into a powerful and elegant programming language, challenging traditional paradigms and promoting a declarative programming style.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6t9sf7njare46l1z565q.png)

# Key Characteristics#

Haskell distinguishes itself by maintaining a delicate balance between simplicity and power. By emphasizing a declarative approach, it enables concise and readable code while facilitating abstraction and composition. Its unique design principles encourage developers to focus on the 'what' rather than the 'how,' fostering a more expressive and efficient coding experience.

#Historical Context#

Throughout its evolution, Haskell has been a focal point for innovative advancements in the programming language landscape. Notable contributions include the introduction of monads and advancements in type systems. The Haskell community, characterized by collaboration and intellectual curiosity, has played a pivotal role in shaping the language's trajectory.

#Functional Advantage#

Haskell's functional nature provides a robust foundation for programming, offering clear and concise solutions to complex problems. Leveraging a type system that catches errors at compile time, Haskell enhances code safety and instills confidence in the development process. Its functional paradigm aligns seamlessly with contemporary software engineering principles.

#Last word

In summary, Haskell stands as a testament to the marriage of simplicity and power in programming languages. As a functional language with a rich history of innovation, Haskell invites developers to embrace a declarative style, challenging conventional thinking and elevating the coding experience. It remains a key player in the realm of functional programming, offering a sophisticated platform for those seeking an elegant and efficient coding environment.
Inside Marker: A Guided Source Code Tour for an AI-powered PDF Layout Detection Engine
Tags: machinelearning, ai, python, datascience
Published: 2023-12-10 14:22:19

Last week, Marker, the PDF to Markdown converter, [topped](https://news.ycombinator.com/item?id=38482007) the Hacker News homepage for a while. As a curious student in the ML world, I thought it’d be a good opportunity to look under the hood, and learn more about how this awesome Document AI tool works.


## What is Marker?

As an analogy, think of marker as an intelligent transcriber, capable of reading through complex books and scientific article PDFs and converting them to clean text-oriented markdown files. Think of it as an intelligent digitization assistant for your document digitization needs.

The official description for the tool, is a bit more technical, which is as follows:

_Marker converts PDF, EPUB, and MOBI to markdown. It's 10x faster than nougat, more accurate on most documents, and has low hallucination risk._

- _Support for a range of PDF documents (optimized for books and scientific papers)_

- _Removes headers/footers/other artifacts_

- _Converts most equations to latex_

- _Formats code blocks and tables_

- _Support for multiple languages (although most testing is done in English). See settings.py for a language list._

- _Works on GPU, CPU, or MPS_


## Working Overview

![Marker Overview](https://journal-wa6509js.s3.ap-south-1.amazonaws.com/969bb02a362239d65a6a81d06dd9133ed1b1da265a34dce65825d7351f3cdcf4.png)
([Full Image](https://journal-wa6509js.s3.ap-south-1.amazonaws.com/969bb02a362239d65a6a81d06dd9133ed1b1da265a34dce65825d7351f3cdcf4.png))

_Marker_ functions in roughly 6 phases, as listed below:

[Continue reading this article at Hexmos Journal](https://journal.hexmos.com/marker-pdf-document-ai/)
Game Dev Journal: Press Start
Tags: learning, gamedev, gamedesign, programming
Published: 2023-12-10 14:07:03

I played my first video game when I was 4. I was fascinated when I found out that I could _move_ stuff on TV. I still remember what I felt and how I felt. It's like my strongest memory — and maybe the happiest. I was kinda used to the idea of television, my father was heavily addicted in action movies and we used to watch together a lot. But the fact of pressing some button and then stuff starts to move _as you wish_ opened a whole new perspective of life for me. I needed to know _how_ to make stuff like that.

I was 12 when I discovered that games are created with computers, and obviously I bothered my parents to buy one — even though I knew we couldn't afford one. But they actually did buy a computer, because they were convinced that it would be an investment on my future.

The first lesson that I learned about game development: it was hard. Not the kind of hard that I was used to. I learned english all by myself translating video games _before_ getting a computer, so I was able to deal with some pretty tough stuff. But I drowned in my first dive in this new ocean.

The second lesson was: it took time for a game to be made. A lot of it. And I was entering in a bad moment in my life, I needed to focus on other stuff, which lead me to the...

Third lesson: it was harder to get a job. They all expected someone with a few games made, and I couldn't finish a single one.

And then I gave up for a while, going in a different direction until the pandemics. I lost my job as soon as it started, and then I came back to coding, but now for a living. I'm currently working as a web developer, but my inner child deserves to have his dreams fulfilled.

And what would make this journey different than what it was fifteen years ago? I got a plan, and I'm more stubborn than ever.

Now, the fourth lesson: I need to be an AI specialist to make the games the way I want. I like to play hard games, and hard games requires good NPCs, which leads to a strong AI system.

I'm tired of writing, so this is it by now. Next time I'm here, I'll talk about what I've been doing with AI.
Hooked Book #2 - Trigger
Tags: design, books, ethic, resume
Published: 2023-12-10 14:00:00

_Facebook demonstrates the increasing power of habit-forming technology_

The chain reaction that forms a **habit** always **start** with a **trigger**. Triggers could be external or internal.

A **external trigger** is embedded with information, guiding users what to do next through a call to action.

Reducing the cognitive load required to take the next action increases the chances that users will do what the company wants.

External triggers could be of four types: paid, earned, relationship, or owned.

**Internal triggers** are tightly coupled with a thought, an emotion, or a preexisting routine. These kinds of triggers manifest automatically in our minds and are those that companies aim to create in users through their products.

The goal of a habit-forming product is to solve the user’s pain by creating an association between the problem and the solution solved by the product or service.

The author highlight six points:

1. Triggers are the first step in the Hooked Model.

2. Triggers come in two types: External and internal.

3. External triggers guide the user to the next action.

4. Internal triggers stored associations in user memory.

5. Negative emotions frequently serve as internal triggers.

6. Makers need to link their solution to a frequently felt internal trigger and leverage it by a external one.