abhi commited on
Commit
455d620
1 Parent(s): 08bef75

Add application file

Browse files
Files changed (4) hide show
  1. .env +1 -0
  2. Dockerfile +11 -0
  3. main.py +303 -0
  4. requirements.txt +8 -0
.env ADDED
@@ -0,0 +1 @@
 
 
1
+ GOOGLE_API_KEY
Dockerfile ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ FROM python:3.9
2
+
3
+ WORKDIR /code
4
+
5
+ COPY ./requirements.txt /code/requirements.txt
6
+
7
+ RUN pip install --no-cache-dir --upgrade -r /code/requirements.txt
8
+
9
+ COPY . .
10
+
11
+ CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "7860"]
main.py ADDED
@@ -0,0 +1,303 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from fastapi import FastAPI, HTTPException
2
+ from fastapi.middleware.cors import CORSMiddleware
3
+ import google.generativeai as genai
4
+ import os
5
+ import requests
6
+ from bs4 import BeautifulSoup
7
+ import validators
8
+ from youtube_transcript_api import YouTubeTranscriptApi
9
+
10
+ from dotenv import load_dotenv
11
+ load_dotenv()
12
+ genai.configure(api_key=os.getenv("GOOGLE_API_KEY"))
13
+
14
+ app = FastAPI()
15
+
16
+ origins=[
17
+ "http://localhost:3000",
18
+ "chrome-extension://mjdggpghaabiocpngdkageeenpmgebim"
19
+ ]
20
+
21
+ app.add_middleware(
22
+ CORSMiddleware,
23
+ allow_origins=origins,
24
+ allow_credentials=True,
25
+ allow_methods=['*'],
26
+ allow_headers=['*']
27
+ )
28
+
29
+ model=genai.GenerativeModel("gemini-pro")
30
+
31
+
32
+ prompt1="""
33
+ As a Web Insight Summarizer, your task is to summarize the given web
34
+ content into a clear and informative text of around 250 words.
35
+ Highlight key points using bullet points or numbered lists to emphasize important details.
36
+ Keep the summary objective, coherent, and focused on the main insights without adding personal opinions.
37
+ Here's an example:
38
+
39
+ Main Topic: Climate Change
40
+ -Rising Temperatures: Due to greenhouse gas emissions.
41
+ -Consequences: Extreme weather events, rising sea levels, loss of biodiversity.
42
+ -Mitigation Strategies: Renewable energy, carbon reduction, sustainable practices.
43
+ -Call to Action: Urgent collective action needed.
44
+ Your summary should provide a comprehensive overview of the content, enabling readers to grasp the main insights efficiently.
45
+ Remember to keep the summary objective and focused on the key points of the content without adding unnecessary details or opinions.
46
+ Providing a clear and digestible overview will help users grasp the main insights from the parsed HTML content efficiently which is given in the following content :
47
+ """
48
+
49
+ prompt2="""
50
+ As an Insight Summarizer, your task is to distill provided information into a concise and informative
51
+ summary of around 250 words. Utilize bullet points or numbered lists to highlight key
52
+ points and ensure clarity and coherence in your summary. Avoid injecting personal
53
+ opinions and focus solely on presenting the main insights from the content.
54
+ Here's an example:
55
+
56
+ Main Topic: [Main topic of the content]
57
+ [Key point 1]: [Description or detail]<br>
58
+ [Key point 2]: [Description or detail]
59
+ [Key point 3]: [Description or detail]
60
+ <br>
61
+ Additional Considerations:
62
+ [Additional consideration 1]: [Description or detail]
63
+ [Additional consideration 2]: [Description or detail]
64
+ [Additional consideration 3]: [Description or detail]
65
+
66
+ Your summary should provide a comprehensive overview of the content, enabling readers to understand the main
67
+ insights efficiently. Remember to maintain objectivity and focus on the key points to ensure
68
+ clarity and usefulness for the reader
69
+ """
70
+
71
+ prompt6="""
72
+ As an Insight Summarizer, your task is to distill provided information into a concise and informative
73
+ summary of around 250 words. Utilize bullet points or numbered lists to highlight key
74
+ points and ensure clarity and coherence in your summary. Avoid injecting personal
75
+ opinions and focus solely on presenting the main insights from the content.
76
+ Ensure to structure the content using HTML tags like <p> for paragraphs and<ul>, <li> for list items. heading <b> tag , next line <br> , dont use "\n" , instead use <br> instead for next line
77
+ Here's an structure of output:
78
+ <p><b>Main Topic:[Main topic of the content]<b><br><ul><li>[Key point 1]: [Description or detail]</li><li>[Key point 2]: [Description or detail]</li><li>[Key point 3]: [Description or detail]</li></ul><br><b>Additional Considerations:<b><br><ul><li>[Additional consideration 1]: [Description or detail]</li><li>[Additional consideration 2]: [Description or detail]</li><li>[Additional consideration 3]: [Description or detail]</li></ul></p>
79
+
80
+ Your summary should provide a comprehensive overview of the content, enabling readers to understand the main insights efficiently. Remember to maintain objectivity and focus on the key points to ensure clarity and usefulness for the reader."
81
+
82
+ Example:
83
+ <p><b>Main Topic: FastAPI - A Quick Overview</b></p><p>FastAPI makes it quicker and easier to develop APIs with Python. In this course, Tommy will guide you through his first API Crash Course, demonstrating how FastAPI works.</p><p>FastAPI is a modern, fast, and high-performance web framework for building APIs with Python. It offers a straightforward approach to developing web applications.</p><p>Here are some key points to remember:</p><ul><li>FastAPI is easy to install using the Python package manager, Pip.</li><li>Ensure you have Python installed on your computer with a minimum version of 3.6 or higher.</li><li>Use command prompt (Windows) or terminal (Mac) to install FastAPI with the command:<code>pip install fastapi</code>.</li><li>You may also need to install an ASGI server like Uvicorn to run your FastAPI project.</li><li>Create a new Python file for your FastAPI project, e.g.,<code>my_api.py</code>.</li><li>Import FastAPI into your Python file using<code>import fastapi</code>.</li><li>Create an instance of FastAPI using<code>app = fastapi.FastAPI()</code>.</li><li>Define endpoints using different HTTP methods like<code>GET</code>,<code>POST</code>,<code>PUT</code>,<code>DELETE</code>, etc.</li><li>Endpoints represent different operations or resources in your API.</li><li>You can use parameters in your endpoints, such as path parameters and query parameters, to customize API behavior.</li></ul><p>Once your FastAPI project is set up, you can run it using an ASGI server like Uvicorn. Test your API endpoints and explore the autogenerated documentation provided by FastAPI.</p><p>With FastAPI, you can quickly build powerful and efficient APIs with Python, making it an excellent choice for web development projects.</p>
84
+ """
85
+
86
+ prompt7="""
87
+ As an Insight Summarizer, your task is to distill provided information into a concise and informative
88
+ summary of around 250 words. Utilize bullet points or numbered lists to highlight key
89
+ points and ensure clarity and coherence in your summary. Avoid injecting personal
90
+ opinions and focus solely on presenting the main insights from the content.
91
+ Ensure to structure the content using HTML tags like <p> for paragraphs and<ul>, <li> for list items. heading <b> tag , next line <br> , dont use "\n" , instead use <br> instead for next line
92
+ Here's an structure of output:
93
+ <p><b>Main Topic:[Main topic of the content]<b>
94
+ <br><ul>
95
+ <li>[Key point 1]: [Description or detail]</li>
96
+ <li>[Key point 2]: [Description or detail]</li>
97
+ <li>[Key point 3]: [Description or detail]</li>
98
+ </ul><br><b>Additional Considerations:<b><br><ul>
99
+ <li>[Additional consideration 1]: [Description or detail]</li>
100
+ <li>[Additional consideration 2]: [Description or detail]</li>
101
+ <li>[Additional consideration 3]: [Description or detail]</li>
102
+ </ul></p>
103
+ Your summary should provide a comprehensive overview of the content, enabling readers to understand the main insights efficiently. Remember to maintain objectivity and focus on the key points to ensure clarity and usefulness for the reader."
104
+
105
+ Example:
106
+ <p>Main Topic: <b>The Impact of Artificial Intelligence on Business</b>
107
+ <br>
108
+ <ul>
109
+ <li>AI enhances operational efficiency by automating repetitive tasks.</li>
110
+ <li>It improves decision-making through data analysis and predictive analytics.</li>
111
+ <li>AI-powered chatbots provide personalized customer support, enhancing customer satisfaction.</li>
112
+ </ul><br><b>Additional Considerations:</b>
113
+ <br><ul>
114
+ <li>AI may raise ethical concerns regarding data privacy and job displacement.</li>
115
+ <li>Investing in AI requires significant financial resources for research and development.</li>
116
+ <li>Regulatory frameworks are necessary to govern AI implementation and mitigate risks.</li>
117
+ </ul></p>
118
+ """
119
+
120
+
121
+ prompt3="""
122
+ As a web assistant, your task is to generate relevant answers in 150 words
123
+ based on the provided user's question by using the provided web content. The answers should accurately reflect the information provided
124
+ in the summary without introducing any personal opinions or irrelevant details.if suggestions are asked
125
+ provide the accurate suggestion on your own based on the topic , if the response is lenghty you can represent
126
+ it in the pointwise
127
+ """
128
+
129
+ prompt4="""
130
+ As a web assistant, your task is to generate concise and relevant
131
+ answers in 150 words based on the provided user's question and the
132
+ accompanying web content. The answers should accurately reflect the
133
+ information provided in the summary without introducing any personal
134
+ opinions or irrelevant details. If suggestions are requested, provide
135
+ accurate recommendations based on the topic. If the response is lengthy,
136
+ represent it in a pointwise format and dont mention
137
+ the user question again in the output.
138
+
139
+ Here is the content from the parsed web page: {transcript}.
140
+
141
+ User's Question: {question}
142
+
143
+ and dont provide the additional information other than the asked question
144
+
145
+ Ensure to structure the content using HTML tags like <p> for paragraphs and<ul>, <li> for list items. heading <b> tag , next line <br> , dont use "\n" , instead use <br> instead for next line , give output with <p> end with </p> tag
146
+
147
+ Example:
148
+ <p><b>Main Topic:The Impact of Artificial Intelligence on Business</b><br><ul><li>AI enhances operational efficiency by automating repetitive tasks.</li><li>It improves decision-making through data analysis and predictive analytics.</li><li>AI-powered chatbots provide personalized customer support, enhancing customer satisfaction.</li></ul><br></p>
149
+
150
+ """
151
+
152
+ prompt5="""you are a youtube video sumaeizer, you will be taking the transcript text
153
+ and summmarize the entire video and providing the importand summary in points
154
+ with 200 words .please provide the summary and make it pointwise in depth .
155
+ Ensure to structure the content using HTML tags like <p> for paragraphs and<ul>, <li> for list items. heading <b> tag , next line <br> , dont use "\n" , instead use <br> instead for next line
156
+
157
+ Your summary should provide a comprehensive overview of the full content, enabling readers to understand the main insights efficiently. Remember to maintain objectivity and focus on the key points to ensure clarity and usefulness for the reader. i give just a example with short content but add more content in para tag, generate content based on the type of content give"
158
+
159
+ Example if the content is about tutorial or instruction:
160
+ <p><b>Main Topic: Creating a Website from Scratch with HTML and CSS</b><p><strong>Step 1: Planning Your Website</strong></p><p>Before diving into coding, take some time to plan your website's structure, layout, and content. Consider the purpose of your website, target audience, and desired features. Sketch out wireframes or create mockups to visualize the design and user experience.</p><p><strong>Step 2: Setting Up Your Development Environment</strong></p><p>Ensure you have a text editor installed, such as Visual Studio Code or Sublime Text, for writing HTML and CSS code. Create a new project folder and organize your files neatly. Optionally, consider using version control systems like Git to track changes and collaborate with others.</p><p><strong>Step 3: Creating the HTML Structure</strong></p><p>Start by creating the basic structure of your website using HTML</p><p><strong>Step 4: Adding Content and Text</strong></p><p>Populate your website with relevant content, including headings, paragraphs, images, and links. Use descriptive text to communicate your message effectively and engage users. Organize content logically to improve readability and navigation.</p><p><strong>Step 5: Styling with CSS</strong></p><p>Apply styles to your website using CSS to enhance its appearance and layout. Create a separate CSS file and link it to your HTML document using the<link>tag. Use selectors, properties, and values to customize fonts, colors, margins, padding, and more.</p><p><strong>Step 6: Implementing Responsive Design</strong></p><p>Ensure your website looks great and functions well across various devices and screen sizes. Use media queries and flexible layout techniques to create a responsive design that adapts to desktops, laptops, tablets, and smartphones seamlessly.</p><p><strong>Step 7: Adding Navigation Menus</strong></p><p>Implement navigation menus to help users navigate your website easily. Create a header or sidebar menu using HTML lists , and style it with CSS to enhance its appearance. Consider adding hover effects and transitions for better user interaction.</p><p><strong>Step 8: Incorporating Multimedia Elements</strong></p><p>Enhance your website with multimedia elements such as images, videos, and audio files. Use the image video tags to embed media content directly into your web pages. Optimize multimedia files for faster loading times and better performance.</p><p><strong>Step 9: Implementing Forms and Interactivity</strong></p><p>Add interactive elements to your website, such as forms, buttons, and animations, to engage users and collect feedback. Use HTML form elements like <input>,<textarea>, and <button> to create user-friendly forms for inputting data.</p><p><strong>Step 10: Testing and Debugging</strong></p><p>Thoroughly test your website across different browsers and devices to ensure compatibility and functionality. Use developer tools to inspect and debug your code, fix any errors or inconsistencies, and optimize performance. Solicit feedback from peers or beta testers to identify areas for improvement.</p><p><strong>Step 11: Deploying Your Website</strong></p><p>Once you're satisfied with your website's design and functionality, it's time to deploy it to a web server. Choose a reliable hosting provider and upload your files using FTP or file manager tools. Configure domain settings and ensure proper security measures are in place.</p></p>
161
+
162
+ """
163
+
164
+ content=""
165
+ transcript=""
166
+
167
+ def scrweb(url):
168
+ global content
169
+ if not validators.url(url):
170
+ status=400
171
+ response="enter correct url"
172
+ return status,response
173
+ else:
174
+ web=requests.get(url)
175
+ status=web.status_code
176
+ if status==200:
177
+ soup=BeautifulSoup(web.content,"html.parser")
178
+ paragraphs = soup.find_all(["p", "h1", "h2"])
179
+ text_list = [p.get_text() for p in paragraphs]
180
+
181
+ # Combine text from all paragraphs into a single string
182
+ content = ''.join(text_list)
183
+
184
+ return status,content
185
+ else:
186
+ return {"unable to retrieve "}
187
+
188
+
189
+ def genetate_gemini_content(prompt,content):
190
+ response=model.generate_content([prompt,content])
191
+ return response.text
192
+
193
+ def extract_transcript(youtube_link):
194
+ try:
195
+ global transcript
196
+ if not validators.url(youtube_link):
197
+ status=400
198
+ response="enter correct url"
199
+ return status,response
200
+ else:
201
+ video_id=youtube_link.split("=")[1].split("&")[0]
202
+ transcript_text=YouTubeTranscriptApi.get_transcript(video_id,languages = [ 'en'])
203
+
204
+ for i in transcript_text:
205
+ transcript+=" "+i["text"]
206
+ return transcript
207
+
208
+ except Exception as e:
209
+ return "no information avaliable"
210
+
211
+
212
+
213
+
214
+
215
+
216
+
217
+
218
+
219
+ @app.get("/summary")
220
+ async def test(url:str):
221
+ try:
222
+ status,content=scrweb(url)
223
+ if status==200:
224
+ response=genetate_gemini_content(prompt6,content)
225
+ print(response)
226
+ return response
227
+ else:
228
+ return status
229
+
230
+ except:
231
+ raise HTTPException(status_code=500, detail="Internal Server Error")
232
+
233
+
234
+
235
+
236
+ @app.get("/ut")
237
+ async def test():
238
+ global transcript
239
+ # Check if the input is a valid URL
240
+ # if not validators.url(url):
241
+ # raise HTTPException(status_code=400, detail="Invalid URL")
242
+
243
+ return transcript
244
+
245
+ @app.get("/wb")
246
+ async def test():
247
+ global content
248
+ # Check if the input is a valid URL
249
+ # if not validators.url(url):
250
+ # raise HTTPException(status_code=400, detail="Invalid URL")
251
+
252
+ return content
253
+
254
+
255
+
256
+ @app.post("/webqa")
257
+ async def webchat(question:str):
258
+ global content
259
+ try:
260
+ if content:
261
+ response=model.generate_content([prompt4,content,question])
262
+ print(response.text)
263
+ return response.text
264
+ else:
265
+ return {"please summarize first"}
266
+ except:
267
+ raise HTTPException(status_code=500, detail="Internal Server Error")
268
+
269
+
270
+
271
+ @app.get("/youtube")
272
+ async def ytranscript(url : str):
273
+ global transcript
274
+ try:
275
+ transcript_text=extract_transcript(url)
276
+ print(transcript_text)
277
+ if transcript_text:
278
+ response=model.generate_content([prompt6,transcript_text])
279
+ print(response.text)
280
+ return response.text
281
+
282
+ else:
283
+ return {"unable to retrieve data"}
284
+ except:
285
+ raise HTTPException(status_code=500, detail="Internal Server Error")
286
+
287
+
288
+ @app.post("/ytqanda")
289
+ async def ytchat(question:str):
290
+ global transcript
291
+ try:
292
+ if transcript:
293
+ response=model.generate_content([prompt4,transcript,question])
294
+ print(response.text)
295
+ return response.text
296
+ else:
297
+ return {"please summarize first"}
298
+ except:
299
+ raise HTTPException(status_code=500, detail="Internal Server Error")
300
+
301
+ @app.get("/")
302
+ async def test():
303
+ return {"status":"api is working"}
requirements.txt ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ youtube_transcript_api
2
+ google-generativeai
3
+ python_dotenv
4
+ fastapi
5
+ uvicorn
6
+ beautifulsoup4
7
+ requests
8
+ validators