teleren/devto huggingface dataset:
Introduction to Scikit-Learn for Pandas Users: Your Data Science Toolkit
Tags: scikit, pandas, datascience, toolkit
Published: 2024-01-28 15:01:50

**Introduction:**
Hey there! Have you ever found yourself wrestling with a tricky data science problem, wishing there was an easier way to handle it? If you’re a Pandas users, you know it’s a fantastic tool for data manipulation and analysis. But, have you ever felt like you’re hitting a wall with what you can do with Pandas alone?

That’s where **Scikit-Learn** comes into play. Think of it as the superhero sidekick to Pandas. **Scikit-Learn** is this super cool library that takes your data science skills to the next level. It’s like having a turbo button for your data analysis – enabling you to do more advanced stuff like machine learning, which might seem daunting at first. But don’t worry, it’s not as complex as it sounds, especially if you already know your way around Pandas.

In this article, we’re going on a little adventure. I’ll be your guide, showing you how to seamlessly transition from using Pandas users to incorporating Scikit-Learn into your workflow. We’ll start with the basics of Scikit-Learn, then dive into some neat examples where we blend Pandas and Scikit-Learn together. It’s like making a delicious data science smoothie with all the right ingredients!

So, buckle up and get ready for a journey that will expand your data science toolkit, making you a more versatile and efficient data scientist. Let’s get started!

**Bridging the Gap between Pandas and Scikit-Learn:**
Alright, let’s chat about how we can connect the dots between Pandas and Scikit-Learn. Think of it like learning to ride a bike with training wheels (that’s Pandas) and then zooming off without them (hello, Scikit-Learn!).

**Data Preprocessing: Like a Data Chef!**
Certainly! Let’s create a practical example to illustrate how we can bridge Pandas and Scikit-Learn in a data preprocessing task. We’ll use a real-world dataset for this demonstration. Let’s say we’re working with a dataset about housing prices, a common scenario in data science. We’ll use Pandas for data manipulation and preparation, and then transition that data into a format suitable for Scikit-Learn.
more information about this post 
click [here](https://codewithnazam.com/introduction-to-scikit-learn-for-pandas-users-your-data-science-toolkit/)
What is Software Testing ? What do we need to know about Software Testing ? What is the relevance of Software Testing..
Published: 2024-01-28 15:01:10

As the name self explains, any new software that is developed by the software developers for a client as per their requirements is prone to have error or bugs and defects. To capture, convey, address and resolve these defects, Software Testing plays an essential role which helps create a bug free, defect less application that can be utilised at its best performance by the client. 

The process of evaluating and verifying the software for bugs and/or defects is called Software Testing.

First thing a Software Tester needs to know about Software Testing is if the software in question is adhering to all the client's requirements and is functional and not breaking on the ideal environment provided. Another thing is if it is doing exactly what it is expected to do. Also, if the purpose of Software testing is fulfilled, like ensuring that the software is functioning correctly, is helping to identify the issues and fixing early on during development thereby reducing the cost and efforts of fixing them later on.

Relevance of Software Testing is that it helps to point out errors and defects from the various phases of software development. Software testing ensures that the overall quality of the software including its functional aspects, performance, security, customer experience, etc are met as specified by the business or the clients.
OpenAI ChatGPT Embedding Example
Published: 2024-01-28 14:59:09

**OpenAI ChatGPT Embedding Example**.

import pandas as pd
from openai import OpenAI

# Initialize OpenAI client
client = OpenAI()

def get_embedding(text, model="text-embedding-ada-002"):
    text = text.replace("\n", " ")
    return client.embeddings.create(input=[text], model=model).data[0].embedding

# Load preprocessed dataset
df = pd.read_csv("fine_food_reviews.csv", index_col=0)

# Create the 'combined' column by concatenating 'Summary' and 'Text'
df['combined'] = "Title: " + df['Summary'].str.strip() + "; Content: " + df['Text'].str.strip()

# Get embeddings and save them for future reuse
df["embedding"] = df["combined"].apply(lambda x: get_embedding(x, model='text-embedding-ada-002'))
df.to_csv("fine_food_reviews_with_embeddings.csv")

fine_food_reviews.csv
OpenAI ChatGPT Embedding Search Example
Published: 2024-01-28 14:58:28

**OpenAI ChatGPT Embedding Search
Example**




import pandas as pd
import numpy as np
from ast import literal_eval
import openai

def get_embedding(text: str, model="text-similarity-davinci-001", **kwargs):
    text = text.replace("\n", " ")
    response = openai.embeddings.create(input=[text], model=model, **kwargs)
    return response.data[0].embedding

def cosine_similarity(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

# Load the dataset and convert embeddings
datafile_path = "fine_food_reviews_with_embeddings.csv"
df = pd.read_csv(datafile_path)
df["embedding"] = df.embedding.apply(literal_eval).apply(np.array)

# Function to search through the reviews
def search_reviews(df, product_description, n=3, pprint=True):
    product_embedding = get_embedding(
        product_description,
        model="text-embedding-ada-002"
    )
    df["similarity"] = df.embedding.apply(lambda x: cosine_similarity(x, product_embedding))

    results = (
        df.sort_values("similarity", ascending=False)
        .head(n)
        .combined.str.replace("Title: ", "")
        .str.replace("; Content:", ": ")
    )
    if pprint:
        for r in results:
            print(r[:200])
            print()
    return results

# Example searches
results = search_reviews(df, "tomato", n=1)
OpenAI ChatGPT RAG Retrieval Augmented Generation Example
Published: 2024-01-28 14:57:34

**OpenAI ChatGPT RAG Retrieval Augmented Generation Example**



import pandas as pd
import numpy as np
from ast import literal_eval
import openai

def get_embedding(text: str, model="text-similarity-davinci-001", **kwargs):
    text = text.replace("\n", " ")
    response = openai.embeddings.create(input=[text], model=model, **kwargs)
    return response.data[0].embedding

def cosine_similarity(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

# Load the dataset and convert embeddings
datafile_path = "fine_food_reviews_with_embeddings.csv"
df = pd.read_csv(datafile_path)
df["embedding"] = df.embedding.apply(literal_eval).apply(np.array)

# Function to search through the reviews
def search_reviews(df, product_description, n=3, pprint=True):
    product_embedding = get_embedding(
        product_description,
        model="text-embedding-ada-002"
    )
    df["similarity"] = df.embedding.apply(lambda x: cosine_similarity(x, product_embedding))

    results = (
        df.sort_values("similarity", ascending=False)
        .head(n)
        .combined.str.replace("Title: ", "")
        .str.replace("; Content:", ": ")
    )
    if pprint:
        for r in results:
            print(r[:200])
            print()
    return results

client = openai.OpenAI()

def generate_answer_with_chat(context, question):
    conversation = [
        {"role": "system", "content": "You are a knowledgeable assistant."},
        {"role": "user", "content": f"Context: {context}"},
        {"role": "user", "content": f"Question: {question}"}
    ]

    stream = client.chat.completions.create(
        model="gpt-4",
        messages=conversation,
        stream=True,
    )

    response_text = ""
    for chunk in stream:
        if chunk.choices[0].delta.content is not None:
            response_text += chunk.choices[0].delta.content

    return response_text

def rag_search(df, query, n=3):
    # Retrieve top N similar reviews
    top_reviews = search_reviews(df, query, n=n, pprint=False)

    # Combine the top reviews into a single context string
    context = " ".join(top_reviews)

    # Generate an answer based on the context using chat API
    answer = generate_answer_with_chat(context, query)

    return answer

# Example usage
answer = rag_search(df, "What do people think about the texture of whole wheat pasta?")
print(answer)

Embedding

    OpenAI Embedding

Search Embedding

    OpenAI Search Embedding
TaskWeaver Stock Price Plugin
Published: 2024-01-28 14:53:09

**	
TaskWeaver Stock Price Plugin**



stock_price.py

import yfinance as yf
from taskweaver.plugin import Plugin, register_plugin

# Define the detailed prompt for the plugin
stock_price_prompt = r"""
This plugin is designed to fetch real-time stock market data. It performs the following tasks:
- Retrieves the latest stock price information based on the provided stock symbol.
- Extracts key financial details such as the current market price.
- Finally, it returns this information in a structured dictionary format, as shown below:
  {
    "symbol": <stock_symbol>,
    "current_price": <current_market_price>
  }
"""

@register_plugin
class StockPricePlugin(Plugin):
    def __call__(self, stock_symbol: str):
        # Fetch stock data using yfinance
        stock = yf.Ticker(stock_symbol)

        # Check if stock exists and has info
        if not stock.info:
            return {"error": f"No data found for symbol: {stock_symbol}"}

        # Extract the current stock price
        current_price = stock.history(period="1d")['Close'].iloc[-1]

        # Check if current price is available
        if current_price is None:
            return {"error": f"Current price not available for symbol: {stock_symbol}"}

        # Return a dictionary containing the stock symbol and its current price
        return {
            "symbol": stock_symbol,
            "current_price": current_price
        }

stock_price.yaml

name: stock_price
enabled: true
required: false
description: >-
  The StockPricePlugin connects to the Yahoo Finance API using yfinance to fetch
  real-time stock market data. It retrieves and returns the latest stock price
  information based on the provided stock symbol, including the current market price.

parameters:
  - name: stock_symbol
    type: str
    required: true
    description: The stock symbol for which the current market price is to be retrieved.

returns:
  - name: stock_data
    type: dict
    description: >-
      A dictionary containing the stock symbol and its current market price. The format is:
      {
        "symbol": <stock_symbol>,
        "current_price": <current_market_price>
      }
TaskWeaver + Text Generation Web UI
Published: 2024-01-28 14:51:34

**TaskWeaver + Text Generation Web UI**


Text Generation Web UI

git clone https://github.com/oobabooga/text-generation-webui
cd text-generation-webui
bash start_macos.sh --api --listen --extensions openai

TheBloke/Orca-2-7B-GGUF
orca-2-7b.Q4_K_M.gguf

TaskWeaver

git clone https://github.com/microsoft/TaskWeaver.git
cd TaskWeaver
conda create -n taskweaver python=3.11
conda activate taskweaver
pip install -r requirements.txt

Config

{
  "llm.api_base": "http://0.0.0.0:5000/v1",
  "llm.api_key": "null",
  "llm.model": "orca2",
  "llm.response" : "text"
}
AutoGen AutoBuilder
Published: 2024-01-28 14:50:25

**AutoGen AutoBuilder**



import autogen
from autogen.agentchat.contrib.agent_builder import AgentBuilder

# 1. Configuration
config_path = 'OAI_CONFIG_LIST.json'
config_list = autogen.config_list_from_json(config_path)
default_llm_config = {'temperature': 0}

# 2. Initialising Builder
builder = AgentBuilder(config_path=config_path)

# 3. Building agents
building_task = "Find a paper on arxiv by programming, and analyze its application in some domain..."
agent_list, agent_configs = builder.build(building_task, default_llm_config)

# 4. Multi-agent group chat
group_chat = autogen.GroupChat(agents=agent_list, messages=[], max_round=12)
manager = autogen.GroupChatManager(groupchat=group_chat, llm_config={"config_list": config_list, **default_llm_config})
agent_list[0].initiate_chat(
    manager, 
    message="Find a recent paper about gpt-4 on arxiv..."
)

OAI_CONFIG_LIST.json

[
  {
    "model": "gpt-4-1106-preview"
  }
]
MLX Stable Diffusion UI
Published: 2024-01-28 14:49:27

import gradio as gr
from PIL import Image
import numpy as np
import mlx.core as mx
from stable_diffusion import StableDiffusion

def generate_images(prompt, n_images=4, steps=50, cfg=7.5, negative_prompt="", n_rows=1):
    sd = StableDiffusion()

    # Generate the latent vectors using diffusion
    latents = sd.generate_latents(
        prompt,
        n_images=n_images,
        cfg_weight=cfg,
        num_steps=steps,
        negative_text=negative_prompt,
    )
    for x_t in latents:
        mx.simplify(x_t)
        mx.simplify(x_t)
        mx.eval(x_t)

    # Decode them into images
    decoded = []
    for i in range(0, n_images):
        decoded_img = sd.decode(x_t[i:i+1])
        mx.eval(decoded_img)
        decoded.append(decoded_img)

    # Arrange them on a grid
    x = mx.concatenate(decoded, axis=0)
    x = mx.pad(x, [(0, 0), (8, 8), (8, 8), (0, 0)])
    B, H, W, C = x.shape
    x = x.reshape(n_rows, B // n_rows, H, W, C).transpose(0, 2, 1, 3, 4)
    x = x.reshape(n_rows * H, B // n_rows * W, C)
    x = (x * 255).astype(mx.uint8)

    # Convert to PIL Image
    return Image.fromarray(x.__array__())

iface = gr.Interface(
    fn=generate_images,
    inputs=[
        gr.Textbox(label="Prompt"),
        gr.Slider(minimum=1, maximum=10, step=1, value=4, label="Number of Images"),
        gr.Slider(minimum=20, maximum=100, step=1, value=50, label="Steps"),
        gr.Slider(minimum=0.0, maximum=10.0, step=0.1, value=7.5, label="CFG Weight"),
        gr.Textbox(default="", label="Negative Prompt"),
        gr.Slider(minimum=1, maximum=10, step=1, value=1, label="Number of Rows")
    ],
    outputs="image",
    title="Stable Diffusion Image Generator",
    description="Generate images from a textual prompt using Stable Diffusion"
)


iface.launch()
Google Gemini API Python Exampl
Published: 2024-01-28 14:47:31

**Google Gemini API Python Example**



import vertexai
from vertexai.preview.generative_models import GenerativeModel, Part

model = GenerativeModel("gemini-pro")
responses = model.generate_content(
    """Write a meal plan for today""",
    generation_config={
        "max_output_tokens": 2048,
        "temperature": 0.9,
        "top_p": 1
    },
    stream=True,
)

for response in responses:
    print(response.candidates[0].content.parts[0].text)

Gradio UI

import gradio as gr
import vertexai
from vertexai.preview.generative_models import GenerativeModel, Part

def generate(prompt):
    model = GenerativeModel("gemini-pro")
    response = model.generate_content(
        prompt,
        generation_config={
            "max_output_tokens": 2048,
            "temperature": 0.9,
            "top_p": 1
        },
        stream=False,
    )
    output = response.candidates[0].content.parts[0].text
    return output

iface = gr.Interface(
    fn=generate,
    inputs="text",
    outputs="markdown",
    title="Gemini Pro API UI"
)

iface.launch()
Delegate, Func and Action in C#
Tags: csharp, programming, tutorial, learning
Published: 2024-01-28 14:47:23

Hi there👋
welcome to my blog where I share my passion for coding in C#. Today, I want to talk about `delegate`, `Func` and `Action`. Sounds exciting, right? 😉


### Delegates 🚀

Delegate is a type that represents a method with a specific signature. It allows you to pass a method as a parameter to another method, or store it in a variable. Delegate is very useful when you want to create callbacks, events, or asynchronous operations.

Here is a simple example of a delegate declaration:
```csharp
public delegate void MyDelegate(string message);

// Method that matches the delegate signature
public static void MyMethod(string message)
{
    Console.WriteLine(message);
}

// Using the delegate
MyDelegate myDelegate = new MyDelegate(MyMethod);
myDelegate("Hello, Delegate!");

```

In the example above, `MyDelegate` is declared with a signature that takes a string parameter. The `MyMethod` method matches this signature, and an instance of the delegate is created and used to call the method.

But delegate has some drawbacks. It is verbose and requires you to define a delegate type for each method signature. 

![delegate is verbose](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ly2kwpjtodfefroezklz.gif)

That's why C# introduced two generic delegate types: `Func` and `Action`.

### Func and Action 🧩
Func and Action are predefined delegate types that can take any number of parameters and return a value or not. Func is used when you want to return a value, and Action is used when you don't. For example:

#### Func<T, TResult>:
```csharp
Func<int, int, int> add = (a, b) => a + b;
int result = add(5, 7);
```

#### Action<T>:
```csharp
Action<string> printMessage = message => Console.WriteLine(message);
printMessage("Hello, Action!");
```

Using Func and Action makes your code more concise and readable. You can use them with LINQ, lambda expressions, or any method that expects a delegate. They are also compatible with the built-in delegate types like EventHandler or Predicate.

### Why Func and Action? 🔎

- **Conciseness:** Func and Action provide a more concise syntax for defining delegates, especially when working with methods that take parameters or return values.
- **Readability:** The use of generic types in Func and Action makes code more readable, as you don't need to explicitly declare delegate types.
- **Compatibility:** Func and Action are compatible with LINQ expressions and other functional programming constructs, making them well-suited for modern C# development.


![Reading](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hj1lh6owe36ayqhvpg0d.gif)

I hope you enjoyed this post and learned something new about delegate, fun and action in C#. If you have any questions or feedback, feel free to leave a comment below. 
Happy coding!🌟
Google Gemini API Node.js Example
Published: 2024-01-28 14:46:34

**Google Gemini API Node.js Example**


Gemini Pro

import { GoogleGenerativeAI } from "@google/generative-ai";
import dotenv from 'dotenv';

dotenv.config();

// 1. Configuration
const api_key = process.env.GEMINI_API_KEY;
const genAI = new GoogleGenerativeAI(api_key);
const generationConfig = { temperature: 0.9, topP: 1, topK: 1, maxOutputTokens: 4096 };

// 2. Initialise Model
const model = genAI.getGenerativeModel({ model: "gemini-pro", generationConfig });

// 3. Generate Content
async function generateContent() {
  try {
    const prompt = "Create a Meal plan for today";
    const result = await model.generateContent(prompt);
    const response = await result.response;
    console.log(response.text());
  } catch (error) {
    console.error('Error generating content:', error);
  }
}

// Run the function
generateContent();

Gemini Vision

import { GoogleGenerativeAI } from "@google/generative-ai";
import { promises as fs } from 'fs';
import dotenv from 'dotenv';

dotenv.config();

// 1. Configuration
const api_key = process.env.GEMINI_API_KEY;
const genAI = new GoogleGenerativeAI(api_key);
const generationConfig = { temperature: 0.4, topP: 1, topK: 32, maxOutputTokens: 4096 };

// 2. Initialise Model for vision-based generation
const model = genAI.getGenerativeModel({ model: "gemini-pro-vision", generationConfig });

// 3. Generate Content with Image Input
async function generateContent() {
  try {
    // Load image
    const imagePath = 'image.jpeg';
    const imageData = await fs.readFile(imagePath);
    const imageBase64 = imageData.toString('base64');

    // Define parts
    const parts = [
      { text: "Describe what the people are doing in this image:\n" },
      {
        inlineData: {
          mimeType: "image/jpeg",
          data: imageBase64
        }
      },
    ];

    // Generate content using both text and image input
    const result = await model.generateContent({ contents: [{ role: "user", parts }] });
    const response = await result.response;
    console.log(response.text());
  } catch (error) {
    console.error('Error generating content:', error);
  }
}

// Run the function
generateContent();

Javascript
Google Gemini API Node.js Example
Published: 2024-01-28 14:46:27

**Google Gemini API Node.js Example**


Gemini Pro

import { GoogleGenerativeAI } from "@google/generative-ai";
import dotenv from 'dotenv';

dotenv.config();

// 1. Configuration
const api_key = process.env.GEMINI_API_KEY;
const genAI = new GoogleGenerativeAI(api_key);
const generationConfig = { temperature: 0.9, topP: 1, topK: 1, maxOutputTokens: 4096 };

// 2. Initialise Model
const model = genAI.getGenerativeModel({ model: "gemini-pro", generationConfig });

// 3. Generate Content
async function generateContent() {
  try {
    const prompt = "Create a Meal plan for today";
    const result = await model.generateContent(prompt);
    const response = await result.response;
    console.log(response.text());
  } catch (error) {
    console.error('Error generating content:', error);
  }
}

// Run the function
generateContent();

Gemini Vision

import { GoogleGenerativeAI } from "@google/generative-ai";
import { promises as fs } from 'fs';
import dotenv from 'dotenv';

dotenv.config();

// 1. Configuration
const api_key = process.env.GEMINI_API_KEY;
const genAI = new GoogleGenerativeAI(api_key);
const generationConfig = { temperature: 0.4, topP: 1, topK: 32, maxOutputTokens: 4096 };

// 2. Initialise Model for vision-based generation
const model = genAI.getGenerativeModel({ model: "gemini-pro-vision", generationConfig });

// 3. Generate Content with Image Input
async function generateContent() {
  try {
    // Load image
    const imagePath = 'image.jpeg';
    const imageData = await fs.readFile(imagePath);
    const imageBase64 = imageData.toString('base64');

    // Define parts
    const parts = [
      { text: "Describe what the people are doing in this image:\n" },
      {
        inlineData: {
          mimeType: "image/jpeg",
          data: imageBase64
        }
      },
    ];

    // Generate content using both text and image input
    const result = await model.generateContent({ contents: [{ role: "user", parts }] });
    const response = await result.response;
    console.log(response.text());
  } catch (error) {
    console.error('Error generating content:', error);
  }
}

// Run the function
generateContent();

Javascript
API Google Gemini API using API KEY
Published: 2024-01-28 14:45:09

**API Google Gemini API using API KEY**


Gemini Pro

import google.generativeai as genai
import os

# 1. Configuration
genai.configure(api_key=os.getenv("GOOGLE_API_KEY"))
generation_config = {"temperature": 0.9, "top_p": 1, "top_k": 1, "max_output_tokens": 2048}

# 2. Initialise Model
model = genai.GenerativeModel("gemini-pro", generation_config=generation_config)

# 3. Generate Content
response = model.generate_content(["Create a Meal plan for today"])

print(response.text)

# # For Streaming
# for chunk in response:
#     print(chunk.text, end="", flush=True)

Gemini Pro Vision

import os
from pathlib import Path
import google.generativeai as genai

# 1. Configuration
genai.configure(api_key=os.getenv("GOOGLE_API_KEY"))
generation_config = { "temperature": 0.4, "top_p": 1, "top_k": 32, "max_output_tokens": 4096 }

# 2. Initialise Model
model = genai.GenerativeModel( model_name="gemini-pro-vision", generation_config=generation_config )

# 3. Generate Content
image_path = Path("image.jpeg")
image_part = {
    "mime_type": "image/jpeg", 
    "data": image_path.read_bytes()
}
prompt_parts = [
    "Describe what the people are doing in this image:\n", 
    image_part
]

response = model.generate_content(prompt_parts)
print(response.text)