teleren/devto huggingface dataset:
GPT-4 Vision: Adding Football Commentator to Video
Published: 2024-01-28 15:37:39

**GPT-4 Vision: Adding Football Commentator to Video**



import cv2
import base64
import openai
from moviepy.editor import VideoFileClip, AudioFileClip

# Initialize OpenAI client
client = openai.OpenAI()

# Load the video
video = cv2.VideoCapture("football.m4v")

# Calculate video length
length = int(video.get(cv2.CAP_PROP_FRAME_COUNT))
fps = video.get(cv2.CAP_PROP_FPS)
video_length_seconds = length / fps

print(f'Video length: {video_length_seconds:.2f} seconds')

# Read frames and encode to base64
base64Frames = []
while video.isOpened():
    success, frame = video.read()
    if not success:
        break
    _, buffer = cv2.imencode(".jpg", frame)
    base64Frames.append(base64.b64encode(buffer).decode("utf-8"))

# Release the video object
video.release()
print(len(base64Frames), "frames read.")

# Create OpenAI chat completion
response = client.chat.completions.create(
    model="gpt-4-vision-preview",
    messages=[
        {
            "role": "user", 
            "content": [
                f"These are frames of a video. Create a short voiceover script in the style of a football commentator For {video_length_seconds:.2f} seconds. Only include the narration. Don't talk about the view",
                *map(lambda x: {"image": x, "resize": 768}, base64Frames[0::25]),
            ]
        }
    ],
    max_tokens=500,
)

# Print the response content
print(response.choices[0].message.content)
import time
time.sleep(2)
speech_file_path = "football.mp3"
response = client.audio.speech.create(
  model="tts-1",
  voice="onyx",
  input=response.choices[0].message.content
)

response.stream_to_file(speech_file_path)


video_clip = VideoFileClip("football.m4v")
audio_clip = AudioFileClip("football.mp3")
final_clip = video_clip.set_audio(audio_clip)
final_clip.write_videofile("football_with_commentator.m4v", codec='libx264', audio_codec='aac')
video_clip.close()
audio_clip.close()
final_clip.close()
Canopy RAG Examples
Published: 2024-01-28 15:36:20


**Canopy RAG Examples**


1.     Open-source RAG framework.
2.     Built on Pinecone vector database.
3.     Simplifies RAG application development.
4.     Manages chat history, query optimization, and context retrieval.
5.     Configurable server for easy RAG chat application deployment.
6.     Supports custom RAG application building.
7.     CLI tool for interactive chat and RAG workflow evaluation.

from datasets import load_dataset

data = load_dataset("jamescalam/ai-arxiv", split="train")
data = data.map(lambda x: {
    "id": x["id"],
    "text": x["content"],
    "source": x["source"],
    "metadata": {
        "title": x["title"],
        "primary_category": x["primary_category"],
        "published": x["published"],
        "updated": x["updated"],
    }
})
data = data.remove_columns([
    "title", "summary", "content",
    "authors", "categories", "comment",
    "journal_ref", "primary_category",
    "published", "updated", "references"
])
data.to_json("ai_arxiv.jsonl", orient="records", lines=True)

export PINECONE_API_KEY="<PINECONE_API_KEY>"
export PINECONE_ENVIRONMENT="<PINECONE_ENVIRONMENT>"
export OPENAI_API_KEY="<OPENAI_API_KEY>"
export INDEX_NAME=<INDEX_NAME>

pip install -qU canopy-sdk datasets
canopy
canopy new
canopy upsert ./ai_arxiv.jsonl
canopy start
canopy chat --no-rag

import openai
openai.api_base = "http://host:port/" 

import openai 
openai_response = openai.Completion.create(..., api_base="http://host:port/")
How This Underrated Technique Saved My Team Hours of Debugging
Tags: webdev, javascript, programming, productivity
Published: 2024-01-28 15:35:17

### The Hurdles of Debugging: Time, Effort and Annoyance

Ever felt like you are stuck in a never-ending bunch of errors? The process of debugging can be frustrating, without proper methods it can cost us **hours**, **effort** and **annoyance**.

Consider this scenario: You are dealing with a bug in your app where the submit button is not working. Where's the first place you go? The **logs**.

Here is what our logs look like, There are **18 logs** displayed.


![Alt text](https://journal-wa6509js.s3.ap-south-1.amazonaws.com/6a99c89f2673d5aa237bed0e53dd69627f549d7fc79a29cf15f07f3be1be3020.png)


Among the 18 logs, we need to manually find out the error log.
In larger systems, the number of logs can add up. It takes lots of time and effort to find the relevant log.


### How I Overcame the Hurdles

Seeing such hurdles, I had to find a technique to address this problem.
The problem is that, there is **too much information** in the form of log lines written by various team members.

In most of the cases these logs are written without much care and don't provide value.

Due to lack of standards in code reviews, Logging Practices are **inconsistent**.
This results in challenges for team members who need to get relevant information from these logs.

So, the solution is to set the standards for the team regarding Logging.
For helping to setup the standards, we already have something that is collecting dust: **Logging Levels**. Not every team uses it to its fullest potential.

For those who don't know, **Logging Levels** are a way of categorizing the logs based on their severity and purpose.

So let's return to the same scenario: we have a problem to debug, and we are looking at the logs.


![Alt text](https://journal-wa6509js.s3.ap-south-1.amazonaws.com/f795a09269a7f3a8ecebe71f9f889e88e75a1cef618d82e71752ba6700dce57b.png)

In the above image, from **18 logs** I have filtered it down to **1 relevant log**. I have done this using **Logging Levels**.

I used the devtools to display only logs at the error logging level. Amazing isn't it?

This is also pretty useful when working in teams. Any developer without previous experience in the app can jump into the problem directly without dealing with the confusion caused by the other logs.


### Apply the Technique: Real-Life Examples
Continue reading the full article at https://journal.hexmos.com/timesaving-debugging-technique/
OpenAI Assistants API + Code Interpreter
Published: 2024-01-28 15:31:27

**OpenAI Assistants API + Code Interpreter
**



import openai
import time

# Initialize the client
client = openai.OpenAI()

file = client.files.create(
    file=open("songs.csv", "rb"),
    purpose='assistants'
)

# Step 1: Create an Assistant
assistant = client.beta.assistants.create(
    name="Data Analyst Assistant",
    instructions="You are a personal Data Analyst Assistant",
    model="gpt-4-1106-preview",
    tools=[{"type": "code_interpreter"}],
    file_ids=[file.id]
)

# Step 2: Create a Thread
thread = client.beta.threads.create()

# Step 3: Add a Message to a Thread
message = client.beta.threads.messages.create(
    thread_id=thread.id,
    role="user",
    content="What are the most popular tracks and their artists?"
)

# Step 4: Run the Assistant
run = client.beta.threads.runs.create(
    thread_id=thread.id,
    assistant_id=assistant.id,
    instructions="What are the most popular tracks and their artists?"
)

print(run.model_dump_json(indent=4))

while True:
    # Wait for 5 seconds
    time.sleep(5)

    # Retrieve the run status
    run_status = client.beta.threads.runs.retrieve(
        thread_id=thread.id,
        run_id=run.id
    )
    print(run_status.model_dump_json(indent=4))

    # If run is completed, get messages
    if run_status.status == 'completed':
        messages = client.beta.threads.messages.list(
            thread_id=thread.id
        )

        # Loop through messages and print content based on role
        for msg in messages.data:
            role = msg.role
            content = msg.content[0].text.value
            print(f"{role.capitalize()}: {content}")
        break
    else:
        print("Waiting for the Assistant to process...")
        time.sleep(5)
AI Assistants API + Knowledge Retrieval
Published: 2024-01-28 15:30:23

**
AI
Assistants API + Knowledge Retrieval**



import openai
import time

# Initialize the client
client = openai.OpenAI()

file = client.files.create(
    file=open("songs.txt", "rb"),
    purpose='assistants'
)

# Step 1: Create an Assistant
assistant = client.beta.assistants.create(
    name="Customer Service Assistant",
    instructions="You are a customer support chatbot. Use your knowledge base to best respond to customer queries.",
    model="gpt-4-1106-preview",
    tools=[{"type": "retrieval"}],
    file_ids=[file.id]
)

# Step 2: Create a Thread
thread = client.beta.threads.create()

# Step 3: Add a Message to a Thread
message = client.beta.threads.messages.create(
    thread_id=thread.id,
    role="user",
    content="Tell me about Dance Monkey"
)

# Step 4: Run the Assistant
run = client.beta.threads.runs.create(
    thread_id=thread.id,
    assistant_id=assistant.id,
    instructions="Please address the user as Mervin Praison"
)

print(run.model_dump_json(indent=4))

while True:
    # Wait for 5 seconds
    time.sleep(5)

    # Retrieve the run status
    run_status = client.beta.threads.runs.retrieve(
        thread_id=thread.id,
        run_id=run.id
    )
    print(run_status.model_dump_json(indent=4))

    # If run is completed, get messages
    if run_status.status == 'completed':
        messages = client.beta.threads.messages.list(
            thread_id=thread.id
        )

        # Loop through messages and print content based on role
        for msg in messages.data:
            role = msg.role
            content = msg.content[0].text.value
            print(f"{role.capitalize()}: {content}")
        break
    else:
        print("Waiting for the Assistant to process...")
        time.sleep(5)

Categories
AI
OpenAPI Weather Schema

    Post author

By praison	
Post date

    November 11, 2023

{
  "openapi": "3.1.0",
  "info": {
    "title": "Get weather data",
    "description": "Retrieves current weather data for a location based on wttr.in.",
    "version": "v1.0.0"
  },
  "servers": [
    {
      "url": "https://wttr.in"
    }
  ],
  "paths": {
    "/{location}": {
      "get": {
        "description": "Get weather information for a specific location",
        "operationId": "GetCurrentWeather",
        "parameters": [
          {
            "name": "location",
            "in": "path",
            "description": "City or location to retrieve the weather for",
            "required": true,
            "schema": {
              "type": "string"
            }
          }
        ],
        "responses": {
          "200": {
            "description": "Successful response",
            "content": {
              "text/plain": {
                "schema": {
                  "type": "string"
                }
              }
            }
          },
          "404": {
            "description": "Location not found"
          }
        },
        "deprecated": false
      }
    }
  },
  "components": {
    "schemas": {}
  }
}
Tiktoken Calculate Tokens
Published: 2024-01-28 15:28:49

**Tiktoken Calculate Tokens**



import sys
import tiktoken

def count_tokens(filename: str, model_name="gpt-4") -> int:
    """Count the number of tokens in a file using TikToken."""
    try:
        with open(filename, 'r') as file:
            content = file.read()
            # Get the tokenizer encoding for the specified model
            encoding = tiktoken.encoding_for_model(model_name)
            tokens = encoding.encode(content)
            return len(tokens)
    except FileNotFoundError:
        print("File not found.")
        return 0

if __name__ == "__main__":
    if len(sys.argv) != 2:
        print("Usage: python script.py <filename>")
    else:
        filename = sys.argv[1]
        print("Number of tokens:", count_tokens(filename))

Usage

python script.py [FILENAME]
Divide Large files to small using Tiktoken
Published: 2024-01-28 15:28:07

**Divide Large files to small using Tiktoken**



import tiktoken

def create_chunks(file_path, model_name="gpt-4", max_tokens_per_chunk=1500000):
    # Get the tokenizer encoding for the specified model
    encoding = tiktoken.encoding_for_model(model_name)

    # Read the file
    with open(file_path, 'r') as file:
        text = file.read()

    # Divide the text into chunks based on tokens
    chunks = []
    current_chunk = ""
    current_token_count = 0

    for line in text.split('\n'):
        line_tokens = encoding.encode(line)
        line_token_count = len(line_tokens)

        if current_token_count + line_token_count > max_tokens_per_chunk:
            chunks.append(current_chunk)
            current_chunk = line + '\n'
            current_token_count = line_token_count
        else:
            current_chunk += line + '\n'
            current_token_count += line_token_count

    if current_chunk:
        chunks.append(current_chunk)

    # Save each chunk to a separate text file
    for i, chunk in enumerate(chunks):
        chunk_file_path = f'chunk{i+1}.txt'
        with open(chunk_file_path, 'w') as chunk_file:
            chunk_file.write(chunk)

# Example usage
create_chunks('songs_original.txt', model_name="gpt-4")
Assistants API + Curl
Published: 2024-01-28 15:27:18

**Assistants API + Curl**

curl --request POST \
  --url https://api.openai.com/v1/assistants \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "OpenAI-Beta: assistants=v1" \
  -H "Content-Type: application/json" \
  --data '{
    "model": "gpt-4-1106-preview",
    "instructions": "You are an expert in geography, be helpful and concise."
}'

export ASSISTANT_ID=

curl --request POST \
  --url https://api.openai.com/v1/threads \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -H "OpenAI-Beta: assistants=v1"

export THREAD_ID=thread_txhecUZk0AUrfkpyW6sVK55o

curl --request POST \
  --url https://api.openai.com/v1/threads/$THREAD_ID/messages \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "OpenAI-Beta: assistants=v1" \
  -H "Content-Type: application/json" \
  -d '{
    "role": "user",
    "content": "What is the capital of France?"
}'

curl --request POST \
  --url https://api.openai.com/v1/threads/$THREAD_ID/runs \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "OpenAI-Beta: assistants=v1" \
  -H "Content-Type: application/json" \
  -d "{
    \"assistant_id\": \"$ASSISTANT_ID\"
}"

curl --request GET \

  --url https://api.openai.com/v1/threads/$THREAD_ID/messages \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "OpenAI-Beta: assistants=v1" \
  -H "Content-Type: application/json"
Ollama: Harry Potter AI Persona
Published: 2024-01-28 15:11:55

{{
*Ollama: Harry Potter AI Persona*



Modelfile

FROM mistral:7b

# sets the temperature to 1 [higher is more creative, lower is more coherent]
PARAMETER temperature 1

# sets the context window size to 4096, this controls how many tokens the LLM can use as context to generate the next token
PARAMETER num_ctx 4096

# sets a custom system prompt to specify the behavior of the chat assistant
SYSTEM """
You are Harry Potter, acting as an assistant.
Behave like the character Harry Potter.
Don't say that you are AI
"""

curl http://localhost:11434/api/generate -d '{
  "model": "harrypotter",
  "prompt": "Who are you and what powers do you have?",
  "stream": false
}'

curl http://localhost:11434/api/generate -d '{
  "model": "harrypotter",
  "prompt": "Who are you and what powers do you have? Respond using JSON",
  "stream": false,
  "format": "json"
}'

Modelfile with all parameters

# Ollama Model File Example

FROM mistral:7b # Base model definition

# Parameters
PARAMETER temperature 1 # Creativity level
PARAMETER num_ctx 4096 # Context window size
PARAMETER mirostat 0 # Mirostat sampling (perplexity control)
PARAMETER mirostat_eta 0.1 # Mirostat learning rate
PARAMETER mirostat_tau 5.0 # Balance between coherence/diversity
PARAMETER num_gqa 1 # GQA groups in transformer layer
PARAMETER num_gpu 2 # GPU layers number
PARAMETER num_thread 4 # Computation threads
PARAMETER repeat_last_n 64 # Repetition lookback
PARAMETER repeat_penalty 1.1 # Repetition penalty strength
PARAMETER seed 42 # Random seed for generation
PARAMETER stop "AI assistant:" # Stop sequence for generation
PARAMETER tfs_z 1 # Tail free sampling impact
PARAMETER num_predict 42 # Max tokens to predict
PARAMETER top_k 40 # Probability reduction for nonsense
PARAMETER top_p 0.9 # Text diversity control

# Template
TEMPLATE """
{{- if .First }}
### System:
{{ .System }}
{{- end }}

### User:
{{ .Prompt }}

### Response:
""" # Full prompt template

# System Message
SYSTEM "You are Spider Man, acting as an assistant." # Custom behavior specifier

# Adapter
# ADAPTER ./ollama-lora.bin # LoRA adapter path

# License
LICENSE """
<license text>
""" # Legal license specification



}}
Swiftspeed
Published: 2024-01-28 15:25:45

"Swiftspeed App Builder is an AI tool for creating mobile apps for Android and iOS platforms. The combination of ease of use, customization, professional-level features, and ready-made templates makes Swiftspeed the best app-maker tool for all business types. The platform's intuitive interface simplifies the app development process, enabling users to bring their ideas to life through a series of straightforward steps without writing a single line of code.

https://www.newswire.com/news/build-customize-and-own-swiftspeeds-ai-powered-app-maker-redefines-22210291
"

https://swiftspeed.app
Pandas Plot Bar Chart: A Guide to Visualizing Data in Python
Tags: pandas, career, python, programming
Published: 2024-01-28 15:25:39

## **Introduction**
In the world of data analysis and visualization, Pandas, a popular Python library, plays a pivotal role. In this article, we will delve into the fascinating world of Pandas and explore how to create and customize Pandas Plot bar chart using this powerful library. If you’re looking to present your data in an easily understandable and visually appealing way, you’re in the right place.

**What is Pandas?**
Pandas is an open-source data manipulation and analysis library for Python. It provides easy-to-use data structures and functions for working with structured data, making it an essential tool for data scientists, analysts, and developers. for more information about what is Pandas in Python Click Here.

**Why is Pandas Important?**
Pandas is vital because it allows users to work seamlessly with structured data, such as CSV files, Excel spreadsheets, SQL databases, and more. It simplifies data cleaning, exploration, and analysis, making it a go-to choice for data professionals.

Creating a Pandas plot bar chart is a straightforward process that involves using the plot.bar() method of a Pandas DataFrame. Here’s a breakdown of the steps involved:

1. **Import Necessary Libraries:**
First, you need to import pandas and matplotlib, the two primary libraries for handling data and plotting in Python.
more information click [here ](https://codewithnazam.com/pandas-plot-bar-chart/)
AutoGen Example Code
Published: 2024-01-28 15:23:48

**AutoGen Example Code**


import autogen

user_proxy = autogen.UserProxyAgent(
    name="MervinPraison",
    code_execution_config={
        "work_dir" : "coding",
    }
)

chatbot = autogen.AssistantAgent(
    name="chatbot",
    system_message="You are a python developer",
)

user_proxy.initiate_chat(
    chatbot,
    message="Give me the stock price of Google"
)
Assistants API + AutoGen + Code Interpreter
Published: 2024-01-28 15:22:55

**Assistants API + AutoGen + Code Interpreter**



import autogen
from autogen.agentchat.contrib.gpt_assistant_agent import GPTAssistantAgent
import openai

client = openai.OpenAI()

file = client.files.create(
    file=open("2021_population.csv", "rb"),
    purpose='assistants'
)

assistant = client.beta.assistants.create(
    name="Python Developer",
    instructions="You are a python developer",
    model="gpt-4-1106-preview",
    tools = [ { "type": "code_interpreter" } ],
    file_ids=[file.id]
)

llm_config = { 
    "assistant_id": assistant.id,
    # "tools": [ { "type": "code_interpreter" } ],
    # "file_ids": [file.id]
}

gpt_assistant = GPTAssistantAgent(
    name="Coder Assistant",
    instructions="""
    You are an expert at writing python code to solve problems. 
    Reply TERMINATE when the task is solved and there is no problem
    """,
    llm_config=llm_config
)

user_proxy = autogen.UserProxyAgent(
    name="MervinPraison",
    code_execution_config={
        "work_dir" : "coding",
    }
)

user_proxy.initiate_chat(
    gpt_assistant,
    message="""
    What is the population trend.
    Give me an overview of the data. 
    Show how you solved it with code.
    """
)


Categories
AI
Assistants API + AutoGen

    Post author

By praison	
Post date

    November 15, 2023

import autogen
from autogen.agentchat.contrib.gpt_assistant_agent import GPTAssistantAgent
import openai

client = openai.OpenAI()

assistant = client.beta.assistants.create(
    name="Customer Service Assistant",
    instructions="You are a customer support chatbot. Use your knowledge base to best respond to customer queries.",
    model="gpt-4-1106-preview"
)

config_list = [{"model": "gpt-4-1106-preview"}]
llm_config = { 
    "config_list": config_list, 
    "assistant_id": assistant.id
}

gpt_assistant = GPTAssistantAgent(
    name="assistant",
    instructions=autogen.AssistantAgent.DEFAULT_SYSTEM_MESSAGE,
    llm_config=llm_config
)


user_proxy = autogen.UserProxyAgent(
    name="MervinPraison",
    code_execution_config={
        "work_dir" : "coding",
    }
)

user_proxy.initiate_chat(
    gpt_assistant,
    message="Create a sky invader game, execute it and save the game in the file"
)
Assistants API + AutoGen + Multi Agents
Published: 2024-01-28 15:21:58

**Assistants API + AutoGen + Multi Agents**

import autogen
from autogen.agentchat.contrib.gpt_assistant_agent import GPTAssistantAgent
import openai

client = openai.OpenAI()

llm_config = {}

coder = GPTAssistantAgent(
    name="Coder_Assistant",
    instructions="""
    You are an expert at writing python code to solve problems. 
    Reply TERMINATE when the task is solved and there is no problem
    """,
    llm_config=llm_config
)

analyst = GPTAssistantAgent(
    name="Data_Analyst",
    instructions="You are a data analyst that offers insight into data.",
    llm_config=llm_config
)

user_proxy = autogen.UserProxyAgent(
    name="MervinPraison",
    code_execution_config={
        "work_dir" : "coding",
    },
    system_message="Admin"
)

groupchat = autogen.GroupChat(agents=[user_proxy, coder, analyst], messages=[], max_round=10)
manager = autogen.GroupChatManager(groupchat=groupchat)

user_proxy.initiate_chat(
    manager,
    message="""
    Get the number of issues and pull requests for the repository 
    'microsoft/autogen' in the past 4 weeks and
    offer analyzes to the data. 
    You should print the data in csv format grouped by weeks.
    """
)