teleren/devto huggingface dataset:
LISA integrato in SamGIS
Tags: llm, machinelearning, python, computervision
Published: 2024-05-26 14:38:43

# LISA integrato in SamGIS

La segmentazione d'immagine è un compito cruciale nella visione artificiale, dove l'obiettivo di fare ["instance segmentation"](https://www.ibm.com/topics/instance-segmentation) di un dato oggetto. Ho già lavorato ad un progetto, [SamGIS](https://trinca.tornidor.com/it/projects/samgis-segment-anything-applied-to-GIS), a riguardo. Un passo logico successivo sarebbe integrare la capacità di riconoscere gli oggetti attraverso prompt testuali. Quest'attività apparentemente semplice in effetti comporta però delle differenze rispetto a quanto fatto in SamGIS che utilizza [Segment Anything](https://segment-anything.com/) (il backend di machine learning usato da SamGIS). Mentre infatti "SAM" non categorizza ciò che identifica, partire da un prompt scritto necessita della conoscenza di quali classi di oggetti esistano nell'immagine in analisi. Un [modello di linguaggio visivo](https://arxiv.org/abs/2305.11175) (o VLM) che funziona bene per questo compito è [LISA](https://github.com/dvlab-research/LISA). Gli autori di LISA hanno basato il loro lavoro su [Segment Anything](https://segment-anything.com/) e [Llava](https://llava-vl.github.io/), un LLM con capacità multimodali (può elaborare sia istruzioni di testo che immagini). Sfruttando le capacità di "segmentazione ragionata" di LISA, SamGIS può eseguire analisi di tipo "zero-shot", ovvero senza addestramento pregresso specifico e specialistico in ambito geologico, geomorfologico o fotogrammetrico.

## Prompts testuali d'input e relativi geojson di output

Non riesco a mostrare qui su dev.to questa parte, quindi rimando alla [pagina dedicata sul mio blog](https://trinca.tornidor.com/it/projects/lisa-adapted-for-samgis#prompts-testuali-d-input-e-relativi-geojson-di-output).

## Durata dei task di segmentazione

Al momento, un prompt che richieda anche la spiegazione di quanto identificato nell'immagine rallenta notevolmente l'analisi. Lo stesso prompt d'analisi eseguito sulla stessa immagine però senza richieste di spiegazione viene elaborato molto più velocemente. I test contenenti richieste di spiegazioni vengono eseguiti in più di 60 secondi mentre senza la durata è intorno o inferiore a 4 secondi, utilizzando il profilo hardware HuggingFace "Nvidia T4 Small" con 4 vCPU, 15 GB RAM e 16 GB VRAM.

## Architettura software

Dal punto di vista tecnico e architetturale, la [demo](https://huggingface.co/spaces/aletrn/samgis-lisa-on-cuda) consiste di un frontend simile a quello sulla demo di [SamGIS](https://huggingface.co/spaces/aletrn/samgis). Niente barra degli strumenti per disegnare, sostituita dalla casella di testo per le richieste in linguaggio naturale. Il backend utilizza un'API basata su FastAPI e che invoca una funzione ad hoc basata su LISA.

Ho dovuto mettere in pausa la demo a causa del costo della GPU, ma sto richiedendo l'uso di una GPU gratuita da HuggingFace. Non esitate a contattarmi su LinkedIn per una dimostrazione dal vivo, chiedere maggiori informazioni o ulteriori chiarimenti.
LISA adapted to SamGIS
Tags: llm, machinelearning, python, computervision
Published: 2024-05-26 14:38:30

# LISA adapted to SamGIS

Image segmentation is a crucial task in computer vision, where the goal is to extract the [instance segmentation mask](https://www.ibm.com/topics/instance-segmentation) for a desired object within the image. I've already worked on a project, [SamGIS](https://trinca.tornidor.com/projects/samgis-segment-anything-applied-to-GIS), that focuses on this particular application of computer vision. A logical progression now would be incorporating the ability to recognize objects through text prompts. This apparently simple activity is actually different compared to what [Segment Anything](https://segment-anything.com/) (the ML backend used by SamGIS) does. In fact "SAM" does not outputs descriptions nor categorizations for its input images. Starting from a written prompt at the contrary requires understanding which classes of objects exist in the image under analysis. A [visual language model](https://arxiv.org/abs/2305.11175) (or VLM) that performs well for this task is [LISA](https://github.com/dvlab-research/LISA). LISA's authors built their work on top of [Segment Anything](https://segment-anything.com/) and [Llava](https://llava-vl.github.io/), a large language model with multimodal capabilities (it can process both text prompts and images). By leveraging LISA's "reasoned segmentation" abilities, SamGIS can now conduct "zero-shot" analyses, meaning it can operate without specific or specialistic prior training in geological, geomorphological, or photogrammetric fields.

## Some input text prompts with their geojson outputs

I can't show this part on dev.to, then I refer you to my [blog page](https://trinca.tornidor.com/projects/lisa-adapted-for-samgis#some-input-text-prompts-with-their-geojson-outputs).

## Duration of segmentation tasks

At the moment, a prompt that also requires an explanation about the segmentation task slows down greatly the analysis. The same prompt on the same image without "descriptive" or "explanatory" questions instead finish much faster. Tests with explanatory text perform in more than 60 seconds while without duration is between 3 and 8 seconds, using the HuggingFace hardware profile "Nvidia T4 Small" with 4 vCPU, 15 GB RAM and 16 GB VRAM.

## Software architecture

Technically and architecturally, the [demo](https://huggingface.co/spaces/aletrn/samgis-lisa-on-cuda) consists of a frontend page like [SamGIS](https://huggingface.co/spaces/aletrn/samgis) demo. Instead of the drawing tool bar there is a text prompt for natural language requests with some selectable examples displayed at the top of the page. The backend utilizes a FastAPI-based API that calls a custom LISA function wrapper.

Unfortunately I have to pause my demo due to GPU cost, but I am requesting the use of a free GPU from HuggingFace. Please feel free to reach out to me on LinkedIn for a live demonstration, ask for more information or further clarifications.
jewelry repair
Tags: jewelry, beginners, tutorial
Published: 2024-05-26 14:34:59

No matter what your jewelry or loan needs may be, DCL Jewelry & Loan is the clear go-to choicefor luxury pawn shops in San Diego. Don’t forget once you purchase your jewelry, they offer [jewelry repair](https://www.discreetcollateralloan.com/
) and watch repair along with jewelry appraisal if you so desire. 
With an unbeatable selection of quality items, a knowledgeable staff and competitive loans, DCL Jewelry &Loan  there's no better choice. Experience the difference for yourself and see why DCL Jewelry and Loan is San Diego's top luxury pawn shop.


Contract us:

Business Name;DCL Jewelry & Loan

BUSINESS EMAIL: info@dclsandiego.com

Address:12778 Rancho Peñasquitos Blvd, San Diego, CA 92129, USA

Phone:858-324-0064

Facebook: https://www.facebook.com/dclsandiego

Twitter: https://twitter.com/DCLSanDiego

Instagram: https://www.instagram.com/dclsandiego/
Lavish Beauty Studio - Another Frontend Gig.
Tags: nextjs, gsap, webdev, javascript
Published: 2024-05-26 14:31:00

Hey There ! 
On 22 May 2024, I picked up a challenge of building an awesome frontend website for an Australian-based Beauty Salon Shop. I was curious to work on this project as along the time I had learned many things with respect to Frontend and wanted to apply them in some real projects. 

On 26 May 2024, I finalized the project by deploying it at https://lavish-nine.vercel.app/ it was great working on this project.

![Lavish Beauty Studio Homepage](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xgg44gdssjo42r84k5lb.JPG)
Regards, 
Abhishek
abhinav210702@gmail.com
Behavioral u-loader
Tags: pkn, issn, viada, mchf
Published: 2024-05-26 14:21:34

Looks like a decent `enigma-mod_/(e|el)/-/\w{3,}/.un-splash.pinterest` split into a tandem of a coupled bed since _[Gan Bei](http://www.ganbeicity.lt/)_ **[PLC](https://www.akropolis.lt/lt/kaunas/pc-planas)** is looking turky compared to a general outpost ft. QWERTY &* WYSIWYG C/C++ call set **[XOR](https://www.debian.org)** _**MD5**_ hash basis bias target **[COMET](https://www.advansys.com/)** smartphone dater stalking towards the ex **WC**...

Nevertheless, the shortcomings of _[Mantic Minotaur](https://releases.ubuntu.com/mantic/)_ pullover backwards the _**[Havit](https://www.havit.hk/)**_ brand are accompanying [_Deltaco_](https://www.deltaco.lt/) and [**Westrom**](https://www.westromgroup.com/) **[PC](https://gitlab.com/e565-dev/nuclear-codes.git)** - _**[E](git@gitlab.com:e565-dev/atomic-fynish.git)**_ eras...

`bot-emoji_avg.option.head` and the controversy itself **[DSN](https://www.maastrichtuniversity.nl/)** drilling tier chocolate flavour card is what the oxymoron **GST** and **_SSM_** is usually **STEM** - **_[PROXY](https://www.php.net/manual/en/class.pdo)_** - _**VIA**_ is related Apple Inc. **_DVI_** - **HDMI** adapter stereo-mono **_[VGA](https://gadgetaz.com/Laptop/eMachines_E525--3202)_** spectrum splitter...
Reusing state management: HOC vs Hook
Tags: react, state, modularity, patterns
Published: 2024-05-26 14:14:00

I've been looking at alternative approaches to HoC wrappers, as the evolution of client/server components and React best practices has pushed the HoC pattern down the list of "good modularity patterns". As a comparative exercise, I took this wrapper, and re-wrote it as a hook, using useCallback and React.memo for re-render stability. The hook version seems more in line with modern React style, but otherwise doesn't seem to offer much over the HoC version. Has anyone else looked at these patterns, or otherwise have a better solution?

HoC:
```
"use client";

import { DtoUiComponent, Entity } from "../../types";
import React from "react";
import { useDtoStoreDispatchAndListener } from "../../hooks/main";
import { useDtoStoreDelete } from "../../hooks/main";

export function DtoComponentWrapper<T extends Entity>({
  entityClass,
  id,
  uiComponent: UiComponent,
}: {
  entityClass: string;
  id: string | number;
  uiComponent?: DtoUiComponent<T>;
}) {
  const { currentState, dispatchWithoutControl } =
    useDtoStoreDispatchAndListener<T>(
      id,
      entityClass,
      UiComponent?.name || "component",
    );
  const { dispatchDeletion, deleted } = useDtoStoreDelete(entityClass, id);

  return (
    UiComponent && (
      <UiComponent
        entity={currentState}
        entityClass={entityClass}
        dispatchWithoutControl={dispatchWithoutControl}
        deleted={deleted}
        dispatchDeletion={dispatchDeletion}
      />
    )
  );
}
```

Hook:

```
"use client";

import { DtoUiComponent, Entity } from "../../types";
import React, { memo, useCallback } from "react";
import {
  useDtoStoreDelete,
  useDtoStoreDispatchAndListener,
} from "../../hooks/main";

export function useDtoComponent<T extends Entity>(
  entityClass: string,
  UiComponent: DtoUiComponent<T>,
) {
  return useCallback(
    memo(({ id }: { id: string | number }) => {
      const { currentState, dispatchWithoutControl } =
        useDtoStoreDispatchAndListener<T>(
          id,
          entityClass,
          UiComponent?.name || "component",
        );
      const { dispatchDeletion, deleted } = useDtoStoreDelete(entityClass, id);

      return (
        UiComponent && (
          <UiComponent
            entity={currentState}
            entityClass={entityClass}
            dispatchWithoutControl={dispatchWithoutControl}
            deleted={deleted}
            dispatchDeletion={dispatchDeletion}
          />
        )
      );
    }),
    [entityClass, UiComponent],
  );
}
```
Top-Down Shooter Update: Day 2 (Mobile Game Testing Question)
Published: 2024-05-26 14:05:28

-Research question for today: how do developers normally test mobile games in Unity while developing on a computer?
I looked it up online, and found that to test mobile games in Unity, you need to download Android Build Support (~5GB and I don't have a great internet connection!). So... while it's downloading.. I'm writing this out! 
It's not as easy to test changes on a phone as it is for a computer game, it seems. I guess this is the case for all games where development happens on a different platform than the end user. I have a new appreciation for people who design non-computer games! Brave souls! 

My goals for today are:
-Add basic UI (health and score)
-Make main character and ninja star smaller
-Make star-throwing more directional

I'll check back with you later
Share Your AI Insights: How Do You Use ChatGPT for Work and Studying?
Tags: ai, productivity, chatgpt, discuss
Published: 2024-05-26 14:04:30

AI tools are transforming the way we work and study, making us more productive and efficient. These powerful tools can automate tedious tasks, provide quick information, and offer creative solutions. 

In my recent post, I shared my experiences and tips on using ChatGPT to enhance productivity and learning. If you missed it, check it out [here](https://blog.perstarke-webdev.de/posts/chatgpt-work-study).

Now, I want to hear from you! What are your favorite ways to use ChatGPT or other AI tools for work and studying? Whether it’s drafting emails, brainstorming ideas, or tackling complex problems, your insights can help create a valuable resource for the entire community.

Let’s build a comprehensive collection of AI usage ideas together. Your contributions will not only benefit you but also help fellow readers discover new and effective ways to leverage AI in their daily lives.

So, share your AI strategies in the comments below! How do you use ChatGPT or other AI tools to boost your productivity and learning? Let’s learn and grow together!
Cosa ho imparato durante lo sviluppo di SamGIS con LISA (finora)
Tags: machinelearning, learning, programming, llm
Published: 2024-05-26 13:57:08

# Cosa ho imparato durante lo sviluppo di SamGIS con LISA (finora)

## Leggere le pubblicazioni inerenti ai progetti su cui lavoro

Per migliorare la mia comprensione del mio progetto di machine learning ho deciso di leggere l'articolo su cui si basano LISA e Segment Anything. Oltre ad alcune informazioni teoriche su LLM, ho notato che l'architettura modulare di "SAM" consente di creare e riutilizzare gli image embedding. Dato che SamGIS non funzionava in questo modo inizialmente, ho formulato un'ipotesi al riguardo.

## Debug, misure ed ottimizzazione: ipotesi sul image embedding

A questo punto ho continuato il mio lavoro di debug misurando la durata dei singoli passaggi durante l'esecuzione delle funzioni di SamGIS. La creazione di un image embedding è un'operazione abbastanza onerosa, quindi è vantaggioso salvarlo e riutilizzarlo (ho verificato implementare la mia ipotesi migliorerebbe le prestazioni del software). Utilizzando il profilo hardware HuggingFace "Nvidia T4 Small" (con 4 vCPU, 15 GB RAM e 16 GB VRAM) è possibile risparmiare circa 1 secondo per ogni inferenza successiva alla prima, utilizzando la stessa immagine (quindi senza modificare il tile provider e l'area geografica).

## Il ruolo dei LLM con prompt aventi differenti caratteristiche

LISA eredita le capacità di generazione del linguaggio dei LLM multi-modali come [Llava](https://llava-vl.github.io/). Questi modelli eccellono nella gestione di ragionamenti complessi, conoscenza del mondo, risposte esplicative e conversazioni a più turni. Sono strumenti potenti per colmare il divario tra testo e comprensione visiva.

LISA permette di effettuare [ragionamenti piuttosto complessi](https://trinca.tornidor.com/it/projects/lisa-adapted-for-samgis#prompts-testuali-d-input-e-relativi-geojson-di-output) durante la segmentazione delle immagini (es. "identify the houses near the trees..." vs "identify the houses...") senza particolari peggioramenti prestazionali. Al contrario, richieste contenenti la spiegazione del motivo ("explain why") per cui il task di segmentazione sia fatto in un certo modo avranno tempi di esecuzione molto più elevati (nell'ordine di minuti).

Sono disponibili [maggiori dettagli qui](https://trinca.tornidor.com/it/projects/lisa-adapted-for-samgis#durata-dei-task-di-segmentazione) su questi miglioramenti in seguito alle modifiche descritte e relativamente alle differenti prestazioni dovute a diversi casi durante l'utilizzo di SamGIS con LISA.
How to create an Azure resource group from the Azure portal
Published: 2024-05-26 13:55:14

Azure Resource Group is a logical group of resources that includes virtual machines, databases, web application etc



  creating a resource group
1.Log in to Azure portal.
Search for “resource group” in the search bar.
It will display all the resource groups under your subscription Select “Resource groups”.
2.Click on + Create.
It will open a form to create a resource group.
Select your subscription.
3.Provide a global unique name for Resource group.
the next heading is the resource details with subtitle (region/location) Select the Region.
4. Next is tag- input name and tag to resource group
5.Lastly click on review and create where it validates the process
  If validation is successful, then click on Create.
  If validation fails, check the error and resolve it
What I learnt from development on LISA with SamGIS (So far)
Tags: machinelearning, learning, programming, llm
Published: 2024-05-26 13:52:16

# What I learnt from development on LISA with SamGIS (So far)

## Read publications related to the projects I work on

To improve my understanding of my machine learning project I decided to read the papers on which [LISA](https://arxiv.org/abs/2308.00692) and [Segment Anything](https://arxiv.org/abs/2304.02643) are based. Besides some theoretical informations about LLM, I noticed that the modular architecture of "SAM" permits to save and re-use image embeddings. Since SamGIS didn't work this way initially, I formulated an hypothesis about this.

## Debugging, measures and optimization: Image Embedding Hypothesis

At this point I continued my debugging work by measuring the duration of individual steps during the execution of SamGIS functions. Creating an image embedding is quite an expensive operation, so it is advantageous saving it and re-using it (I verified that implementing my hypothesis would improve the performance of the software). Using the HuggingFace hardware profile "Nvidia T4 Small" (with 4 vCPU, 15 GB RAM and 16 GB VRAM) it's possible to save almost 1 second on every inference after the first, using the same image (without change the geographical area tiles provider).

## The role of LLMs with prompts having different characteristics

LISA inherits the language generation capabilities of multi-modal LLMs such as [Llava](https://llava-vl.github.io/). These models excel at handling complex reasoning, world knowledge, explanatory answers and multi-turn conversations. They’re powerful tools for bridging the gap between text and visual understanding.

LISA allows you to perform [rather complex reasoning](https://trinca.tornidor.com/projects/lisa-adapted-for-samgis#some-input-text-prompts-with-their-geojson-outputs) during image segmentation (e.g. "identify the houses near the trees..." vs "identify the houses...") without any particular performance degradation. On the contrary, requests containing the explanation of reason ("explain why") the segmentation task is done in a certain way will have much higher execution times (in the order of minutes).

There are [more details here](https://trinca.tornidor.com/projects/lisa-adapted-for-samgis#duration-of-segmentation-tasks) about these improvements following the changes described and regarding different performance due to different cases when using SamGIS with LISA.
Code Spirited MVP - A P2P Coding Bootcamp Organizer
Tags: devchallenge, awschallenge, amplify, fullstack
Published: 2024-05-26 13:47:33

*This is a submission for the [The AWS Amplify Fullstack TypeScript Challenge ](https://dev.to/challenges/aws)*

## What We Built
Code Spirited MVP is a platform designed to revolutionize the way coding bootcamps in 🇵🇭 are organized and accessed. It allows users to create and manage custom bootcamps, seek out mentors, and offer their services as freelance instructors at affordable rates. Additionally, it features a marketplace for browsing and discovering various bootcamp programs, making it a comprehensive solution for both learners and educators in the coding community.

## Demo and Code

- Deployed Application: [Code Spirited MVP](https://main.d3agjn4aq682p5.amplifyapp.com/)
- Source Code: [Code Spirited MVP](https://github.com/futureworker10x/code-spirited-mvp)

Bootcamp Listing
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/63qgu9uq49u350fdawrk.png)

Bootcamp Information Page
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/es421qgbkefwc8xm29y4.png)

## Integrations
To build Code Spirited MVP, We leveraged several key AWS services to ensure a seamless and powerful user experience:

- **Data Management**: Utilized AWS Amplify to create curriculum data efficiently.
- **Authentication**: Implemented secure user authentication and authorization using AWS Amplify Auth.
- **Serverless Functions**: ~~Create a trigger to create a profile record after post confirmation.~~
- **File Storage**: Integrated AWS S3 for storing and retrieving profile pic, providing robust and secure storage solutions.
- **Connected Components**: Used Storage Manager to upload the profile pic to the cloud.

**Connected Components and/or Feature Full**
We used auth functions from AWS Amplify and connected a custom UI following the same UX flow.

## Team Member
- @deyanxxx
Backend cheatsheet suggestions (Upcoming)
Tags: webdev, backend, backenddevelopment, tutorial
Published: 2024-05-26 13:26:32

To the dev.to community, 

The title appears to be quite broad, and that is intentional! I was at my desk working away and brainstorming ideas expanding my own, as well as the community's knowledge development, when I thought about the idea of a cheatsheet, something I believe we are **_quite familiar_** with from school.

To create these, I would like to ask the awesome community for help! To anyone who sees this post, in particular to my fellow backend developers (love my frontend pals as well), I need a favour from you! Please leave in the comments on some of your answers to the questions below, they'll seriously aid me in helping to create something useful for the community~

1. What are some struggles you face in your daily lives as programmers? 

2. What are some things that you find yourself constantly having to search/Google up, that you wish you had a quick and reliable reference for?

3. Aside from a cheatsheet, what other mediums/bits of information do you feel will help yourself, or a programmer of a similar/junior level to you?

I appreciate all the feedback from the community, whether you're a frontend/backend developer! If you have any insights/inputs at all, please drop a comment and let's get this conversation going!