teleren/devto huggingface dataset:
Enabling Graph Analytics with Apache AGE: Integrating Hadoop and Apache Kafka
Tags: kafka, apcahe, opensource, postgres
Published: 2023-08-06 16:27:02

Apache AGE (incubating) expands the PostgreSQL database system's capabilities by providing efficient graph analytics at scale. Its integration with the PostgreSQL ecosystem allows effortless storage, querying, and analysis of large-scale graph data. As the adoption of big data tools like Hadoop and Apache Kafka grows for data processing and analytics, the seamless integration of Apache AGE becomes vital for enhancing data interoperability. This step-by-step guide outlines the process of integrating Apache AGE with Hadoop and Apache Kafka, empowering organizations to leverage advanced graph analytics within their data infrastructure.

**Prerequisites**
Before proceeding with the integration, ensure the following prerequisites are met:
1. A functional PostgreSQL installation with the AGE extension.
2. An operational Hadoop cluster.
3. A properly configured Apache Kafka cluster.

## Integrating Apache AGE with Hadoop

**1. Installing Sqoop**
To facilitate efficient data transfer between Apache Hadoop and structured datastores, install Apache Sqoop on your Hadoop cluster.
**2. Configuring Sqoop**
Configure Sqoop to seamlessly connect to your PostgreSQL database, housing the AGE extension. This entails specifying the PostgreSQL JDBC driver, database URL, username, and password in the Sqoop configuration file.
**3. Importing and Exporting Data**
Leverage Sqoop's import and export commands to transfer data effortlessly between Hadoop and PostgreSQL, enhancing data interoperability.

## Integrating Apache AGE with Apache Kafka

**1. Installing Kafka Connect**
Install Kafka Connect, the framework that bridges Apache Kafka with external systems, to enable integration with Apache AGE.
**2. Configuring Kafka Connect**
Configure Kafka Connect to establish a connection with your PostgreSQL database containing the AGE extension. Specify the PostgreSQL JDBC driver, database URL, username, and password in the Kafka Connect configuration file.
**3. Creating a Connector**
Designate a source or sink connector to facilitate smooth data streaming between PostgreSQL and Kafka. For a source connector, define queries to select data from the PostgreSQL database, while for a sink connector, specify the target table and data format.
**4. Streaming Data**
Activate the configured connector to initiate a seamless data flow between Apache AGE and Apache Kafka. Monitor the connector's progress and status using the Kafka Connect REST API.

## Conclusion
By integrating Apache AGE with Hadoop and Apache Kafka, organizations can enhance their data infrastructure's analytical capabilities. This comprehensive guide empowers users to unlock advanced graph analytics, alongside their existing big data tools. The integration offers new insights and data-driven decision-making potential, enabling organizations to thrive in the era of big data and analytics.
Beautify Your Git Log with a Single Command
Tags: git, opensource, programming, beginners
Published: 2023-08-06 15:41:01

The standard `git log` command is functional, providing the necessary information, but it can come across as somewhat dull and verbose. What if there was a way to make the git log not just informative, but also visually appealing? Something like this:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nuudej4s8cndsp1h6t3v.png)

Yes, it's entirely possible! And the good news is that we can achieve this by simply using a bunch of flags and subcommands. There's no need to install or download anything.

Here's the command:

```bash
git log --color --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an>%Creset' --abbrev-commit --branches
```

But typing this command every time could be tiresome. A solution to this is using Git aliases, which allow you to create shortcuts for lengthy commands. If you're unfamiliar with Git aliases, I recommend reading my recent article that dives into the topic:

{% embed https://dev.to/pradumnasaraf/mastering-git-shortcuts-a-guide-to-git-aliases-324j %}

Now, let's set up an alias for our beautiful `git log` command:

```bash
git config --global alias.lg "log --color --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an>%Creset' --abbrev-commit --branches"
```

Now you can invoke the beautified `git log` using a simpler `git lg`.
How to resize your VM disk on Azure
Published: 2023-08-06 15:05:22

Today, I shall like to show you how to resize your Virtual Machine (VM) disk sizes on Azure in just few simple ways. It's worthy of note to let you know that VM disk can only be increase, reducing the size of your VM is not applicable.

To resize your VM disk capacity on Azure, you are to first create a VM, please review my article (https://dev.to/samuel_olo75614/steps-to-creating-azure-virtual-machines-18am) on how to create a VM.


Navigate to your VM by double clicking your VM from your home page.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/czzn6t3ui57eelgm4qq6.png)

**Note: VM disk cannot be resized while running, it has to be stopped or deallocate before carrying out this task.**


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ifgubghi8zo09i2sj2wm.png)

In the left menu under Settings, select Disks.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/sftvyslht7osm4jjfsqs.png)


Under Disk name, select the disk you want to expand, also take note of the initial disk size.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/e67xjydbg93z65ykvcm6.png)


In the left menu under Settings, select Size + performance.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/b88vkvsv76npmddultak.png)


In Size + performance, select the disk size you want. I am upgrading from 127Gib to 512Gib.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/usfk8umvqqvb84rt93fg.png)

Click on save then overview to check the current upgrade made.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/v0lqavw7n8vzi5guc2kp.png)

Upgrade confirmed.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4q1ra5ozaqlrcsixdg46.png)

This equally applies to when you want to decrease your VM size.

_Thanks for reading my article till end. I hope you learned something special today. If you enjoyed this article, then please share to your friends and if you have suggestions or thoughts to share with me then please write in the comment box._
Image Captcha Solver: Solve any image Captcha with a captcha solver
Published: 2023-08-06 14:51:21

![](https://assets.capsolver.com/prod/images/post/2023-06-20/0262a024-2c84-4b0a-ad7c-475929d94cc8.png)

# Solve Image Captchas with CapSolver

Image captchas are a common security measure used on websites to distinguish between human users and bots. They typically involve the user identifying specific elements within an image or series of images. In this blog post, we'll guide you through the process of solving image captchas using CapSolver.

## What is an Image Captcha?

An image captcha is a type of challenge-response test used in computing to determine whether the user is human. The test involves one or more images from which the user is asked to identify or select specific elements. The aim is to prevent automated software or "bots" from carrying out tasks that should be performed by humans.

## Solving Image Captchas with CapSolver

CapSolver provides a convenient and efficient way to solve image captchas. The process involves creating a task with the `createTask` method and providing the necessary parameters. Here's a step-by-step guide:

# Step 1: Creating the Task

To create a task, you need to make a POST request to the `https://api.capsolver.com/createTask` endpoint. The request should include your API key and the task details. For an image captcha, the task type is `ImageToTextTask`. The task details should also include the base64 encoded content of the image. Here's an example of the request:

```http
POST https://api.capsolver.com/createTask
Host: api.capsolver.com
Content-Type: application/json
{
 "clientKey": "YOUR_API_KEY",
 "task":{
 "type":"ImageToTextTask",
 "body": "base64_encoded_image_content"
 }
}
```

# Step 2: Receiving the Results
Unlike other task types, the ImageToTextTask returns the task execution result directly after createTask, rather than getting it asynchronously through getTaskResult. The response will include the solution to the captcha. Here's an example of the response:
```json
{
 "errorId": 0,
 "errorCode": "",
 "errorDescription": "",
 "status": "ready",
 "solution": {
 "text": "captcha_solution"
 },
 "taskId": "task_id",
}
```
# Conclusion
Solving image captchas can be a daunting task, especially when dealing with a large number of them. However, with CapSolver, the process becomes swift and efficient. By following the steps outlined above, you can easily solve image captchas and ensure your automated tasks run smoothly.
9 GitHub Repositories Every Frontend Developer Should Explore
Tags: webdev, github, opensource
Published: 2023-08-06 14:31:18

Attention all developers! 🚀

Are you ready to take your coding journey to the next level? I've got something special for you! 🎉

In my blog post, "9 GitHub Repositories Every Developer Should Explore," I've handpicked some of the most amazing repositories that are sure to ignite your passion for coding and boost your skills.

From powerful libraries and tools to creative projects and learning resources, this curated list covers it all. Whether you're a seasoned pro or just starting, these repositories have something unique to offer.

So, why wait? Head over to this [blog ](https://gaurav-shrivastava.hashnode.dev/9-github-repositories-every-developer-should-explore) and unlock a world of exciting possibilities. 🌟

Happy exploring, fellow developers! 🚀💻
LlamaIndex Overview
Tags: llamaindex, openai
Published: 2023-08-06 14:10:11

---
title: LlamaIndex Overview
published: true
description: description
tags: LlamaIndex, OpenAI
---

## LlamaIndex
LlamaIndex is a data framework designed to fully leverage LLMs (Language Models) pre-trained on vast amounts of public data.

## Why LlamaIndex?
LLMs offer a natural language interface between humans and inferred data. Most models are pre-trained on vast amounts of publicly available data such as Wikipedia, mailing lists, textbooks, and source code. LlamaIndex plays a pivotal role in supplementing these models with private or domain-specific data.

## How LlamaIndex Helps?
- **Data Connectors**: Imports existing data from its native source or format, including APIs, PDFs, SQL, etc.
- **Data Index**: Structures data into a format that's easy for LLMs to consume.
- **Engine**: Provides natural language access to the data.

## Who is LlamaIndex for?
LlamaIndex offers tools for beginners, advanced users, and everyone in between.

## Starting with LlamaIndex
To start using LlamaIndex, simply install it using pip.
Full installation instructions and starter tutorials are also provided.
```bash
pip install llama-index
```
## Ecosystem
You can download or contribute to LlamaIndex at:
- Github
- PyPi

Join the LlamaIndex community for assistance or to suggest features:
- Twitter
- Discord

There are related projects such as LlamaHub and LlamaLab.

## Quick Code Guide
1. Setting up OpenAI API key:
    ```python
    import os
    os.environ["OPENAI_API_KEY"] = 'YOUR_OPENAI_API_KEY'
    ```

2. Loading data and creating index:
    ```python
    from llama_index import VectorStoreIndex, SimpleDirectoryReader
    documents = SimpleDirectoryReader('data').load_data()
    index = VectorStoreIndex.from_documents(documents)
    ```

3. Executing a query:
    ```python
    query_engine = index.as_query_engine()
    query_engine.query("<question_text>?")
    ```

4. Persisting data:
    ```python
    index.storage_context.persist()
    ```

5. Loading data from disk:
    ```python
    from llama_index import StorageContext, load_index_from_storage
    storage_context = StorageContext.from_defaults(persist_dir='./storage')
    index = load_index_from_storage(storage_context)
    ```

Dependencies
The main third-party package requirements for LlamaIndex are `tiktoken`, `openai`, and `langchain`. All requirements can be installed with:
```bash
pip install -r requirements.txt
```
Firebase vs Appwrite
Tags: firebase, webdev, cloud, backend
Published: 2023-08-06 12:04:47

[Firebase](https://scientyficworld.org/cloud-computing/firebase/) and Appwrite are both backend-as-a-service (BaaS) platforms that provide developers with a range of features to build and manage their applications. However, there are some key differences between the two platforms.

**Firebase** is a proprietary platform owned by Google. It offers a wide range of features, including:

* Real-time database
* Cloud messaging
* Authentication
* Storage
* Machine learning
* Hosting

Firebase is easy to use and integrates well with other Google products. However, it can be expensive for large-scale applications.

**Appwrite** is an open-source platform that is self-hosted. It offers a similar range of features to Firebase, but with some key differences:

* Appwrite uses a custom NoSQL database, which is more flexible than Firebase's real-time database.
* Appwrite offers more control over authentication and authorization.
* Appwrite is more customizable than Firebase.
* Appwrite is cheaper than Firebase for large-scale applications.

**Here is a table that summarizes the key differences between Firebase and Appwrite:**

| Feature | Firebase | Appwrite |
|---|---|---|
| Ownership | Proprietary | Open-source |
| Hosting | Cloud-based | Self-hosted |
| Database | Real-time NoSQL | Custom NoSQL |
| Authentication | Simple | Flexible |
| Authorization | Basic | Advanced |
| Customization | Limited | Extensive |
| Cost | More expensive for large-scale applications | Cheaper for large-scale applications |

**Which platform is right for you?**

The best platform for you will depend on your specific needs and requirements. If you are looking for a platform that is easy to use and integrates well with other Google products, then Firebase is a good option. However, if you need more control over your data and authentication, or if you are on a budget, then Appwrite is a better choice.

Here is a table that summarizes the pros and cons of each platform:

| Platform | Pros | Cons |
|---|---|---|
| Firebase | Easy to use | Expensive for large-scale applications |
| Appwrite | More customizable | More complex to set up |

Ultimately, the best way to decide which platform is right for you is to try both of them out and see which one you prefer.
Understanding the Role of the `schema.rb` File in Ruby on Rails Development
Tags: rails
Published: 2023-08-06 13:08:37

---
title: Understanding the Role of the `schema.rb` File in Ruby on Rails Development
published: true
description: 
tags: rails
canonical_url: https://www.kartikey.dev/2023/08/06/understanding-the-role-of-the-schema-rb-file-in-ruby-on-rails-development.html
---
Hello, fellow developers! I've recently had a thought-provoking discussion on [LinkedIn Ruby on Rails group](https://www.linkedin.com/feed/update/urn:li:activity:7093289792351707136/) about the role of the `schema.rb` file in Rails development, particularly concerning its authority over the database schema. The conversation evoked mixed reviews, and I thought it might be interesting to bring the topic to dev.to and get more insights.

The Rails documentation has changed its stance over time. Initially, it suggested that `db/schema.rb` is the authoritative source for your database schema. However, later versions state that your database remains the authoritative source. [Here's an in-depth details](https://www.kartikey.dev/2023/08/06/understanding-the-role-of-the-schema-rb-file-in-ruby-on-rails-development.html) on this topic, where I've tracked the changes in Rails docs, and explored the implications.

**What's your take on this? Do you consider the `schema.rb` file as the definitive schema for your Rails apps, or do you rely on the database itself?**

In addition, I've been using a custom Rake task in my Rails projects, especially during the early phases of development:

```ruby
namespace :db do
  desc 'reset the database by dropping the schema'
  task complete_reset: :environment do
   raise unless Rails.env.local?
    
    FileUtils.rm_f('db/schema.rb')
    Rake::Task['db:drop'].invoke
    Rake::Task['db:create'].invoke
    Rake::Task['db:migrate'].invoke
    Rake::Task['db:seed'].invoke
    Rake::Task['dev:prime'].invoke
  end
end
```

This task helps to avoid conflicts and manage the fast-changing specifications by resetting the database, recreating it, and seeding it with test data. I'd love to know - how do you handle such changes in your projects? What are your thoughts on this `complete_reset` task?

Looking forward to some insightful discussion!
React/Vite Starter with TypeScript
Tags: react, typescript, webdev, javascript
Published: 2023-08-06 12:53:00

Hey Guys, Hope you all are doing well.

Whenever starting any new project, there are many things that need to deal with such as configuring the project, adding dependencies, deciding what the project code structure should look like and what not. So instead of initializing the project from scratch a good choice will be having starter kit that includes basic configuration and a proepr folder structure. So I've made such starter kit based on the requirements that helps you fastern your development workflow, saves your time and make sure you have everything you need at start of the project.

The boilerplate comes with **React**, Application scaffolded with **Vite**, Includes **TypeScript** and Basic **Redux** configuration. For styling purposes **Sass** is also included with some basic folder structure and styling.


![React/Vite Starter with TypeScript](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/l9yptjgc1nsfi9ypd6m8.png)

Tech stack:
- React.js
- TypeScript
- SASS
- Vite

## Get your starter kit 

-- [React/Vite Starter with TypeScript](https://github.com/HardikGohilHLR/react-vite-ts-boilerplate)


Thanks for reading, If you liked the post please share it and leave suggestions.

<hr /><a href="https://www.buymeacoffee.com/hardikgohilhlr" target="_blank"><img src="https://www.buymeacoffee.com/assets/img/custom_images/orange_img.png" alt="Buy Me A Coffee" style="height: 41px !important;width: 174px !important;box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;-webkit-box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;" ></a><hr />

## Connect With Me

Website: [Hardik Gohil](https://hardikgohilhlr.tech)

Github: [https://github.com/HardikGohilHLR](https://github.com/HardikGohilHLR)

Linkedin: [https://www.linkedin.com/in/hardikgohilhlr](https://www.linkedin.com/in/hardikgohilhlr)

Thanks ❤️
Create FAB
Published: 2023-08-06 12:43:58

FAB - Floating Action Button




```html
		
		<div class="fab-container">

			<div class="button iconbutton">
				<i style="font-size: 2.5em; color: #000;" class='bx bxs-message-square-add'></i> 

			</div>

		</div>	

```





```css


.fab-container{
	position:fixed;
	bottom:50px;
	right:50px;
	cursor:pointer;

}


.iconbutton, .button{
	width:50px;
	height:50px;
	border-radius: 100%;
	background:#fff;
	transition: all 0.3s ease; 


}

.button{
	width:60px;
	height:60px;
}



.iconbutton i{
	display:flex;
	align-items:center;
	justify-content:center;
	height: 100%;


}


.iconbutton:hover, .button:hover {
	width: 70px; /* Increase width */
	height: 70px; /* Increase height */
}


```
An Impressive 3D Card with Stunning Hover Effects using HTML and SCSS
Tags: codepen, css, webdev, ui
Published: 2023-08-06 12:29:27

{% codepen https://codepen.io/Smit29/pen/WNYmqbE %}

Today, I'm excited to share a fantastic 3D card design that I've created with some  hover effects using HTML and SCSS. This project was fun.
Feel free to share your creations or any questions you may have in the comments section below.
**Happy coding!**
dropdown arrows, css
Tags: webdev, css, html, javascript
Published: 2023-08-06 12:05:21

I wanna make my dropdown menu just like this and i spent days trying do so, I wanna add the black, hollow arrow that rotates and the animation when I click the button (slide up animation)

https://www.veed.io/view/308a765c-bb6e-4cbb-9611-64e37acbb17e?sharingWidget=true&panel=share
Why encourage to use JSTL in your legacy projects over JSP scriptlet?
Tags: jstl, java, jsp, idioms
Published: 2023-08-06 11:58:40

This article will be a short and will represent maybe just a memo on why you as Team Leader should impose (or encourage) to your team the use of JSTL or/and even migrate piece by piece your current JSP’s written with scriptlet into JSTL syntax.

The arguments:

## The human nature

If we write or maintain the .jsp pages in scriptlet code, believe me when I say that because of stressful situations (deadlines on projects or on features) or maybe because of commodity lot of things will be done in dumb ways. When I say dumb ways I mean that you will end up after a lot of time with a tone of things in FrontEnd written in .jsp as scriptlet that for sure should have been done in BackEnd. An example that I always saw in my time is that a lot of collections are sent in FrontEnd and the developer constructs a complete new collection from 0 and even does some calculations on it (that aren’t advanced but for sure they have no place in FrontEnd in this architecture).

Using jstl syntax is very limited (that’s the beauty) will impose to the developers to stop putting everything in FE and start thinking what should the BE provide already. Meanwhile the FE with the jstl will do small things in his limitation like iteration, conditional check, formatting some data, etc. nothing crazy that the classic scriptlet would have done.

## Visually pleasant

I don’t know about you but I never saw in my company on the legacy application (that is over 15 years) code written in .jsp that doesn’t do crazy stuff ([…] the worst I seen his connection to DB, recovery of data , transforming that data, displaying that data […]) and looks ugly. Instead after 1 years of putting my team to try to use jstl when possible some pages for us developers are now more clearer (even found old bugs).

BAD:

```
<html>
  <head>
    <title>Count to 10 in JSP scriptlet</title>
  </head>
  <body>
    <%
    for(int i=1;i<=10;i++)
    {%>
     <%=i%><br/>
     <%
     }
     %>
  </body>
</html>
```

NICE:

```
<%@ taglib uri="http://java.sun.com/jstl/core" prefix="c" %>
<html>
 <head>
  <title>Count to 10 Example (using JSTL)</title>
 </head>

 <body>
  <c:forEach var="i" begin="1" end="10" step="1">
   <c:out value="${i}" />
   <br />
  </c:forEach>
 </body>
</html>
```

## What is the cost ?

For you as developers ? nothing because we should know to do this, writing clean code instead of bad code shouldn’t even be an argument.

For the client ? if we don’t have experience in JSTL well some patience.

REMEMBER: Let the service layer have the business logic and determine what data the JSP needs.

---

If you liked the article please take a minute to offer me a clap 👏 or even buy me a coffee https://www.buymeacoffee.com/stefansplace (;

Based on the number of claps I will know if writing some tricks about this old concept is still wanted.

My main article: https://medium.com/@stefan.paladuta17/why-to-encourage-to-use-jstl-in-your-legacy-projects-over-jsp-scriptlet-4dd36e95ce00