ncoop57 commited on
Commit
830243e
1 Parent(s): 52636a1

Update descriptions in both english and spanish and add page that describes project in spanish

Browse files
Files changed (4) hide show
  1. HOME.md +4 -2
  2. INICIO.md +8 -11
  3. __pycache__/clip.cpython-38.pyc +0 -0
  4. app.py +21 -12
HOME.md CHANGED
@@ -1,12 +1,14 @@
1
  ## Description
2
- This project is inspired by [@johanmodin](https://github.com/johanmodin)'s project [clifs](https://github.com/johanmodin/clifs) which allows you to search through a video using natural language. For example, if I'd like to find the frames of a video that contain the a pancake in the shape of an otter, I can search using the following prompt "pancake shaped otter" and pass the video that I want to search through and voila! I'll be able to find the frames of the video that contain the pancake shaped otter.
3
 
4
  This demo project makes this process easier by
 
5
  1. Hosting it on the awesome Huggingface's Spaces servers
6
  2. Allowing you to simply link to a Youtube video for specifying the video you want to search through
 
7
 
8
  ## Disclaimer!
9
  This project should not be used beyond educational/intellectual purposes. It is very much a work in progress and has not been tested on a large scale. Additionally, this type of technology can be misused for malicious purposes such as infringing on privacy through surveillance. Don't use this project for malicious purposes, please, be cool and ethical :).
10
 
11
 
12
- With all these potential downsides why did I make this project even easier to use? My main reason is that I see the potential good uses of this project. Namely, I believe this type of work can be used for helping students or learners in general to search through videos to find answers to their questions. Therefore, I created this project as a good way of showing the power of this technology and to learn more about how it works.
 
1
  ## Description
2
+ This project is inspired by [@johanmodin](https://github.com/johanmodin)'s project [clifs](https://github.com/johanmodin/clifs) which allows you to search through a video using natural language. For example, if I'd like to find the frames of a video that contain a pancake in the shape of an otter, I can search using the following prompt "pancake shaped otter" and pass the video that I want to search through and voila! I'll be able to find the frames of the video that contain the pancake shaped otter.
3
 
4
  This demo project makes this process easier by
5
+
6
  1. Hosting it on the awesome Huggingface's Spaces servers
7
  2. Allowing you to simply link to a Youtube video for specifying the video you want to search through
8
+ 3. Allowing you to use multiple languages. Currently supported languages can be found here: https://arxiv.org/pdf/2004.09813.pdf
9
 
10
  ## Disclaimer!
11
  This project should not be used beyond educational/intellectual purposes. It is very much a work in progress and has not been tested on a large scale. Additionally, this type of technology can be misused for malicious purposes such as infringing on privacy through surveillance. Don't use this project for malicious purposes, please, be cool and ethical :).
12
 
13
 
14
+ With all these potential downsides, why did I make this project even easier to use? My main reason is that I see the potential good uses of this project. Namely, I believe this type of work can be used for helping students or learners in general to search through videos to find answers to their questions. Therefore, I created this project as a good way of showing the power of this technology and to learn more about how it works.
INICIO.md CHANGED
@@ -1,16 +1,13 @@
1
  ## Descripción
2
- Este proyecto está inspirado por el proyecto de [@johanmodin](https://github.com/johanmodin) [CLIFS](https://github.com/johanmodin/clifs). CLIFS es una herramienta donde busque en una video usando linguaje de natural. Por ejeemplo, si quiero encontrar los cuadros de una panqueue en el forma de una nutria, puedo buscar usando esta consulta de busqueda: "panqueue en el forma de una nutria" y un video quiero buscar y viola! seré capaz de encontrar los cuadros en el video que contienen el panqueue en el forma de una nutria.
3
 
4
- Este proyecto de demostración hace busqueda de una consulta de busqueda en un video mas feliz. Especificamente:
5
- 1. Alojamiento en los servidores de "Huggingface Spaces"!
6
- 2. Busca una consulta de busqueda en un video por da un enlace de video en YouTube.
7
-
8
- ## Descargo de Responsabilidad!
9
- Este proyecto es una demostración de una herramienta de busqueda de video y no debe ser usado ademas para educación o cualquier otra finalidad. Es WIP y no tiene ninguna garantia de funcionamiento. Tambien, este tipo de tecnologia puede ser mal usado para propósitos maliciosos como infringir la privacidad a través de la vigilancia. Por favor, no lo use para ningún propósito.
10
 
11
- Con estes negativos, porque hago este proyecto? Porque quiero que el mundo sea un lugar mejor
12
-
13
- This project should not be used beyond educational/intellectual purposes. It is very much a work in progress and has not been tested on a large scale. Additionally, this type of technology can be misused for malicious purposes such as infringing on privacy through surveillance. Don't use this project for malicious purposes, please, be cool and ethical :).
14
 
 
 
15
 
16
- With all these potential downsides why did I make this project even easier to use? My main reason is that I see the potential good uses of this project. Namely, I believe this type of work can be used for helping students or learners in general to search through videos to find answers to their questions. Therefore, I created this project as a good way of showing the power of this technology and to learn more about how it works.
 
1
  ## Descripción
2
+ Este proyecto está inspirado a partir del proyecto de [@johanmodin](https://github.com/johanmodin) [CLIFS](https://github.com/johanmodin/clifs), el cual es una herramienta que permite buscar objetos usando lenguaje natural. Por ejemplo, si quiero encontrar los frames de un video para un pancake en forma de una nutria, puedo buscar usando la siguiente consulta: "pancake en forma de una nutria" y viola! La herramienta será capaz de encontrar los frames en el video que contiene el pancake en forma de una nutria.
3
 
4
+ Este proyecto de demostración hace búsqueda de una consulta de búsqueda en un video más feliz. Específicamente:
 
 
 
 
 
5
 
6
+ 1. Alojamiento en los servidores de "Huggingface Spaces"!
7
+ 2. Busca una consulta de búsqueda en un video para dar un enlace de video en YouTube.
8
+ 3. Busca en múltiples lenguajes. Actualmente soportado lenguajes se pueden encontrar aquí: https://arxiv.org/pdf/2004.09813.pdf
9
 
10
+ ## ¡Descargo de Responsabilidad!
11
+ Este proyecto es una demostración de una herramienta de búsqueda de video y no debe ser usado además para educación o cualquier otra finalidad. Es WIP y no hay garantía de funcionamiento. También, este tipo de tecnología puede ser mal usada para propósitos maliciosos como infringir la privacidad a través de la vigilancia. Por favor, no lo use para ningún propósito malicioso.
12
 
13
+ Con estos negativos, ¿por qué hice este proyecto? Porque creo que este tipo de tecnología puede ayudar a la gente como estudiantes y otras personas en general a buscar en videos para respuestas a sus preguntas como tutoriales. Por lo tanto, hice este proyecto para demostrar el poder de esta tecnología y aprender más sobre cómo funciona.
__pycache__/clip.cpython-38.pyc ADDED
Binary file (2.5 kB). View file
 
app.py CHANGED
@@ -29,7 +29,7 @@ def get_embedding(txt_model, vis_model, query, video):
29
  return text_emb, img_embs
30
 
31
  def find_frames(url, txt_model, vis_model, desc, seconds, top_k):
32
- text = st.text("Downloading video...")
33
  probe = ffmpeg.probe(url)
34
  video_stream = next((stream for stream in probe['streams'] if stream['codec_type'] == 'video'), None)
35
  width = int(video_stream['width'])
@@ -41,7 +41,7 @@ def find_frames(url, txt_model, vis_model, desc, seconds, top_k):
41
  .run(capture_stdout=True)
42
  )
43
 
44
- text.text("Processing video...")
45
  video = (
46
  np
47
  .frombuffer(out, np.uint8)
@@ -59,17 +59,25 @@ def find_frames(url, txt_model, vis_model, desc, seconds, top_k):
59
  with open("HOME.md", "r") as f:
60
  HOME_PAGE = f.read()
61
 
 
 
 
62
  def main_page(txt_model, vis_model):
63
  st.title("Introducing Youtube CLIFS")
64
 
65
  st.markdown(HOME_PAGE)
66
 
 
 
 
 
 
67
  def clifs_page(txt_model, vis_model):
68
  st.title("CLIFS")
69
 
70
- st.sidebar.markdown("### Controls:")
71
  seconds = st.sidebar.slider(
72
- "How many seconds of video to consider?",
73
  min_value=10,
74
  max_value=120,
75
  value=60,
@@ -82,17 +90,17 @@ def clifs_page(txt_model, vis_model):
82
  step=1,
83
  )
84
  desc = st.sidebar.text_input(
85
- "Search Description",
86
- value="Pancake in the shape of an otter", # panqueque en forma de nutria
87
- help="Text description of what you want to find in the video",
88
  )
89
  url = st.sidebar.text_input(
90
- "Youtube Video URL",
91
  value='https://youtu.be/xUv6XgPwGaQ',
92
- help="Youtube video you'd like to search through",
93
  )
94
 
95
- submit_button = st.sidebar.button("Search")
96
  if submit_button:
97
  ydl_opts = {"format": "mp4[height=360]"}
98
  with youtube_dl.YoutubeDL(ydl_opts) as ydl:
@@ -102,6 +110,7 @@ def clifs_page(txt_model, vis_model):
102
 
103
  PAGES = {
104
  "Home": main_page,
 
105
  "CLIFS": clifs_page
106
  }
107
 
@@ -112,8 +121,8 @@ def run():
112
  # main body
113
  txt_model, vis_model = get_model()
114
 
115
- st.sidebar.title('Navigation')
116
- selection = st.sidebar.radio("Go to", list(PAGES.keys()))
117
 
118
  page = PAGES[selection](txt_model, vis_model)
119
 
 
29
  return text_emb, img_embs
30
 
31
  def find_frames(url, txt_model, vis_model, desc, seconds, top_k):
32
+ text = st.text("Downloading video (Descargando video)...")
33
  probe = ffmpeg.probe(url)
34
  video_stream = next((stream for stream in probe['streams'] if stream['codec_type'] == 'video'), None)
35
  width = int(video_stream['width'])
 
41
  .run(capture_stdout=True)
42
  )
43
 
44
+ text.text("Processing video (Procesando video)...")
45
  video = (
46
  np
47
  .frombuffer(out, np.uint8)
 
59
  with open("HOME.md", "r") as f:
60
  HOME_PAGE = f.read()
61
 
62
+ with open("INICIO.md", "r") as f:
63
+ INICIO_PAGINA = f.read()
64
+
65
  def main_page(txt_model, vis_model):
66
  st.title("Introducing Youtube CLIFS")
67
 
68
  st.markdown(HOME_PAGE)
69
 
70
+ def inicio_pagina(txt_model, vis_model):
71
+ st.title("Presentando Youtube CLIFS")
72
+
73
+ st.markdown(INICIO_PAGINA)
74
+
75
  def clifs_page(txt_model, vis_model):
76
  st.title("CLIFS")
77
 
78
+ st.sidebar.markdown("### Controls (Controles):")
79
  seconds = st.sidebar.slider(
80
+ "How many seconds of video to consider? (¿Cuántos segundos de video considerar?)",
81
  min_value=10,
82
  max_value=120,
83
  value=60,
 
90
  step=1,
91
  )
92
  desc = st.sidebar.text_input(
93
+ "Search Query (Búsqueda de Consulta)",
94
+ value="Pancake in the shape of an otter",
95
+ help="Text description of what you want to find in the video (Descripción de texto de que desea encontrar en el video)",
96
  )
97
  url = st.sidebar.text_input(
98
+ "Youtube Video URL (URL del Video de Youtube)",
99
  value='https://youtu.be/xUv6XgPwGaQ',
100
+ help="Youtube video you want to search (Video de Youtube que desea búscar)",
101
  )
102
 
103
+ submit_button = st.sidebar.button("Search (Buscar)")
104
  if submit_button:
105
  ydl_opts = {"format": "mp4[height=360]"}
106
  with youtube_dl.YoutubeDL(ydl_opts) as ydl:
 
110
 
111
  PAGES = {
112
  "Home": main_page,
113
+ "Inicio": inicio_pagina,
114
  "CLIFS": clifs_page
115
  }
116
 
 
121
  # main body
122
  txt_model, vis_model = get_model()
123
 
124
+ st.sidebar.title("Navigation (Navegación)")
125
+ selection = st.sidebar.radio("Go to (Ir a)", list(PAGES.keys()))
126
 
127
  page = PAGES[selection](txt_model, vis_model)
128