Want to Search Inside Videos Like a Pro? CLIP-as-service Can Help
Tags: search, clip, opensource, mlops
Published: 2023-02-09 15:05:20

<section id="nice" data-tool="mdnice编辑器" data-website="https://www.mdnice.com" style="font-size: 16px; color: black; padding: 0 10px; line-height: 1.6; word-spacing: 0px; letter-spacing: 0px; word-break: break-word; word-wrap: break-word; text-align: left; font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, 'PingFang SC', Cambria, Cochin, Georgia, Times, 'Times New Roman', serif;"><p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">Wouldn’t it be great if you could search through a video the way you search through a text?</p>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">Imagine opening a digitized film, just hitting <em style="font-style: italic; color: black;">ctrl-f</em> and typing “Santa”, then getting all the parts of the video with Santa Claus in it. Or just going to the command line and using the <code style="font-size: 14px; word-wrap: break-word; padding: 2px 4px; border-radius: 4px; margin: 0 2px; background-color: rgba(27,31,35,.05); font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; word-break: break-all; color: #009688;">grep</code> command:</p>
<pre class="custom" data-tool="mdnice编辑器" style="margin-top: 10px; margin-bottom: 10px; border-radius: 5px; box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"><span style="display: block; background: url(https://files.mdnice.com/user/3441/876cad08-0422-409d-bb5a-08afec5da8ee.svg); height: 30px; width: 100%; background-size: 40px; background-repeat: no-repeat; background-color: #1E1E1E; margin-bottom: -7px; border-radius: 5px; background-position: 10px 10px;"></span><code class="hljs" style="overflow-x: auto; padding: 16px; color: #DCDCDC; display: -webkit-box; font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; font-size: 12px; -webkit-overflow-scrolling: touch; padding-top: 15px; background: #1E1E1E; border-radius: 5px;">grep&nbsp;<span class="hljs-string" style="color: #D69D85; line-height: 26px;">"Santa&nbsp;Claus"</span>&nbsp;Santa_Claus_conquers_The_Martians.mp4<br></code></pre>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">Normally, this would be impossible, or only possible if you had gone through the film and carefully labeled all the parts with a Santa in them already. But with Jina AI and CLIP-as-service, you can create a <strong style="font-weight: bold; color: black;">video grep</strong> command for MP4 film with just a few Python functions and a standard computer setup. There is no need for a GPU and no complex AI tech stack to install, just off-the-shelf and open-source Python libraries, with Jina AI Cloud doing all the heavy lifting.</p>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">This has immediate applications for anyone who has video data: film archivists, stock image vendors, news photographers, or even regular people who just keep videos from their cellphones around and post them to social media.</p>
<h2 data-tool="mdnice编辑器" style="margin-top: 30px; margin-bottom: 15px; padding: 0px; font-weight: bold; font-size: 22px; color: #009688; padding-left: 10px; margin: 1em auto; border-left: 3px solid #009688;"><span class="prefix" style="display: none;"></span><span class="content">Preliminaries</span><span class="suffix"></span></h2>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">You need Python 3, and you might want to create a new virtual environment before starting. Then, install a few components at the command line with <code style="font-size: 14px; word-wrap: break-word; padding: 2px 4px; border-radius: 4px; margin: 0 2px; background-color: rgba(27,31,35,.05); font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; word-break: break-all; color: #009688;">pip</code>:</p>
<pre class="custom" data-tool="mdnice编辑器" style="margin-top: 10px; margin-bottom: 10px; border-radius: 5px; box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"><span style="display: block; background: url(https://files.mdnice.com/user/3441/876cad08-0422-409d-bb5a-08afec5da8ee.svg); height: 30px; width: 100%; background-size: 40px; background-repeat: no-repeat; background-color: #1E1E1E; margin-bottom: -7px; border-radius: 5px; background-position: 10px 10px;"></span><code class="hljs" style="overflow-x: auto; padding: 16px; color: #DCDCDC; display: -webkit-box; font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; font-size: 12px; -webkit-overflow-scrolling: touch; padding-top: 15px; background: #1E1E1E; border-radius: 5px;">pip&nbsp;install&nbsp;clip_client&nbsp;<span class="hljs-string" style="color: #D69D85; line-height: 26px;">"docarray[full]&gt;=0.20.0"</span><br></code></pre>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">This installs:</p>
<ul data-tool="mdnice编辑器" style="margin-top: 8px; margin-bottom: 8px; padding-left: 25px; color: black; list-style-type: disc;">
<li><section style="margin-top: 5px; margin-bottom: 5px; line-height: 26px; text-align: left; color: rgb(1,1,1); font-weight: 500;">Jina AI’s <a href="https://docarray.jina.ai/" style="text-decoration: none; word-wrap: break-word; font-weight: bold; color: #009688; border-bottom: 1px solid #009688;">DocArray library</a></section></li><li><section style="margin-top: 5px; margin-bottom: 5px; line-height: 26px; text-align: left; color: rgb(1,1,1); font-weight: 500;">Jina AI’s <a href="https://clip-as-service.jina.ai/" style="text-decoration: none; word-wrap: break-word; font-weight: bold; color: #009688; border-bottom: 1px solid #009688;">CLIP-as-service</a> client</section></li></ul>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">You'll also need an account for CLIP-as-service. If you don't already have one, there are <a href="https://docs.jina.ai/jina-ai-cloud/login/" style="text-decoration: none; word-wrap: break-word; font-weight: bold; color: #009688; border-bottom: 1px solid #009688;">instructions in the Jina AI documentation</a>. Once you have an account, you will need a token. You can get one from <a href="https://cloud.jina.ai/settings/tokens" style="text-decoration: none; word-wrap: break-word; font-weight: bold; color: #009688; border-bottom: 1px solid #009688;">your token settings page at Jina AI Cloud</a>, or at the command line:</p>
<pre class="custom" data-tool="mdnice编辑器" style="margin-top: 10px; margin-bottom: 10px; border-radius: 5px; box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"><span style="display: block; background: url(https://files.mdnice.com/user/3441/876cad08-0422-409d-bb5a-08afec5da8ee.svg); height: 30px; width: 100%; background-size: 40px; background-repeat: no-repeat; background-color: #1E1E1E; margin-bottom: -7px; border-radius: 5px; background-position: 10px 10px;"></span><code class="hljs" style="overflow-x: auto; padding: 16px; color: #DCDCDC; display: -webkit-box; font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; font-size: 12px; -webkit-overflow-scrolling: touch; padding-top: 15px; background: #1E1E1E; border-radius: 5px;">&gt;&nbsp;jina&nbsp;auth&nbsp;login&nbsp;⤶<br><br>Your&nbsp;browser&nbsp;is&nbsp;going&nbsp;to&nbsp;open&nbsp;the&nbsp;login&nbsp;page.<br>If&nbsp;this&nbsp;fails&nbsp;please&nbsp;open&nbsp;the&nbsp;following&nbsp;link:&nbsp;https://jina-ai.us.auth0.com/au....<br>🔐&nbsp;Successfully&nbsp;logged&nbsp;<span class="hljs-keyword" style="color: #569CD6; line-height: 26px;">in</span>&nbsp;to&nbsp;Jina&nbsp;AI&nbsp;as....<br><br>&gt;&nbsp;jina&nbsp;auth&nbsp;token&nbsp;create&nbsp;video_search&nbsp;⤶<br>╭─────────────&nbsp;🎉&nbsp;New&nbsp;token&nbsp;created&nbsp;─────────────╮<br>│&nbsp;54f0f0ef5d514ca1908698fc6d9555a5&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;│<br>│&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;│<br>│&nbsp;You&nbsp;can&nbsp;<span class="hljs-built_in" style="color: #4EC9B0; line-height: 26px;">set</span>&nbsp;it&nbsp;as&nbsp;an&nbsp;env&nbsp;var&nbsp;JINA_AUTH_TOKEN&nbsp;&nbsp;&nbsp;│<br>╰──────&nbsp;☝️&nbsp;&nbsp;This&nbsp;token&nbsp;is&nbsp;only&nbsp;shown&nbsp;once!&nbsp;───────╯<br></code></pre>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">Your token should looks something like: <code style="font-size: 14px; word-wrap: break-word; padding: 2px 4px; border-radius: 4px; margin: 0 2px; background-color: rgba(27,31,35,.05); font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; word-break: break-all; color: #009688;">54f0f0ef5d514ca1908698fc6d9555a5</code></p>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">Keep your token in an environment variable, or a Python variable if you're using a notebook. You will need it later.</p>
<h2 data-tool="mdnice编辑器" style="margin-top: 30px; margin-bottom: 15px; padding: 0px; font-weight: bold; font-size: 22px; color: #009688; padding-left: 10px; margin: 1em auto; border-left: 3px solid #009688;"><span class="prefix" style="display: none;"></span><span class="content">Getting the Data</span><span class="suffix"></span></h2>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">Loading MP4 video takes just one line of code with DocArray:</p>
<pre class="custom" data-tool="mdnice编辑器" style="margin-top: 10px; margin-bottom: 10px; border-radius: 5px; box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"><span style="display: block; background: url(https://files.mdnice.com/user/3441/876cad08-0422-409d-bb5a-08afec5da8ee.svg); height: 30px; width: 100%; background-size: 40px; background-repeat: no-repeat; background-color: #1E1E1E; margin-bottom: -7px; border-radius: 5px; background-position: 10px 10px;"></span><code class="hljs" style="overflow-x: auto; padding: 16px; color: #DCDCDC; display: -webkit-box; font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; font-size: 12px; -webkit-overflow-scrolling: touch; padding-top: 15px; background: #1E1E1E; border-radius: 5px;"><span class="hljs-keyword" style="color: #569CD6; line-height: 26px;">from</span>&nbsp;docarray&nbsp;<span class="hljs-keyword" style="color: #569CD6; line-height: 26px;">import</span>&nbsp;Document<br><br>video_uri&nbsp;=&nbsp;<span class="hljs-string" style="color: #D69D85; line-height: 26px;">"https://archive.org/download/santa-clause-conquers-the-martians/Santa%20Clause%20Conquers%20The%20Martians.ia.mp4"</span><br><br>video_data&nbsp;=&nbsp;Document(uri=video_uri).load_uri_to_video_tensor()<br></code></pre>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">Loading trailer the to the 1964 film, <em style="font-style: italic; color: black;">Santa Claus Conquers the Martians</em></p>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">This downloads the trailer to the public domain film <em style="font-style: italic; color: black;"><a href="https://www.imdb.com/title/tt0058548/" style="text-decoration: none; word-wrap: break-word; font-weight: bold; color: #009688; border-bottom: 1px solid #009688;">Santa Claus Conquers the Martians</a></em> from the <a href="https://archive.org/details/santa-clause-conquers-the-martians" style="text-decoration: none; word-wrap: break-word; font-weight: bold; color: #009688; border-bottom: 1px solid #009688;">Internet Archive</a>. You can substitute another URL or a file path to a local file to use your own video instead.</p>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">The video itself is stored as a <code style="font-size: 14px; word-wrap: break-word; padding: 2px 4px; border-radius: 4px; margin: 0 2px; background-color: rgba(27,31,35,.05); font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; word-break: break-all; color: #009688;">numpy</code> array in the <code style="font-size: 14px; word-wrap: break-word; padding: 2px 4px; border-radius: 4px; margin: 0 2px; background-color: rgba(27,31,35,.05); font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; word-break: break-all; color: #009688;">Document.tensor</code> attribute of the object:</p>
<pre class="custom" data-tool="mdnice编辑器" style="margin-top: 10px; margin-bottom: 10px; border-radius: 5px; box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"><span style="display: block; background: url(https://files.mdnice.com/user/3441/876cad08-0422-409d-bb5a-08afec5da8ee.svg); height: 30px; width: 100%; background-size: 40px; background-repeat: no-repeat; background-color: #1E1E1E; margin-bottom: -7px; border-radius: 5px; background-position: 10px 10px;"></span><code class="hljs" style="overflow-x: auto; padding: 16px; color: #DCDCDC; display: -webkit-box; font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; font-size: 12px; -webkit-overflow-scrolling: touch; padding-top: 15px; background: #1E1E1E; border-radius: 5px;">print(video_data.tensor.shape)<br><br>(<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">4264</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">640</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">464</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">3</span>)<br></code></pre>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">The video itself is has 4,264 frames (the first dimension of the tensor), each measuring 640 pixels by 464 pixels (the second and third dimensions), and each pixel has three color dimensions (conventional RGB in the fourth dimension).</p>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">You can play the video in a notebook with the <code style="font-size: 14px; word-wrap: break-word; padding: 2px 4px; border-radius: 4px; margin: 0 2px; background-color: rgba(27,31,35,.05); font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; word-break: break-all; color: #009688;">Document.display()</code> method:</p>
<pre class="custom" data-tool="mdnice编辑器" style="margin-top: 10px; margin-bottom: 10px; border-radius: 5px; box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"><span style="display: block; background: url(https://files.mdnice.com/user/3441/876cad08-0422-409d-bb5a-08afec5da8ee.svg); height: 30px; width: 100%; background-size: 40px; background-repeat: no-repeat; background-color: #1E1E1E; margin-bottom: -7px; border-radius: 5px; background-position: 10px 10px;"></span><code class="hljs" style="overflow-x: auto; padding: 16px; color: #DCDCDC; display: -webkit-box; font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; font-size: 12px; -webkit-overflow-scrolling: touch; padding-top: 15px; background: #1E1E1E; border-radius: 5px;">video_data.display()<br></code></pre>
<figure data-tool="mdnice编辑器" style="margin: 0; margin-top: 10px; margin-bottom: 10px; display: flex; flex-direction: column; justify-content: center; align-items: center;"><a href="https://archive.org/download/santa-clause-conquers-the-martians/Santa%20Clause%20Conquers%20The%20Martians.ia.mp4" style="text-decoration: none; word-wrap: break-word; font-weight: bold; color: #009688; border-bottom: 1px solid #009688; border: none; display: flex; justify-content: center; align-items: center;"><img src="https://files.mdnice.com/user/39412/b88fca35-e102-4cd3-9291-858b9445062d.png" alt="click on the image to Watch the video" style="display: block; max-width: 100%; margin: 0px;"></a><figcaption style="text-align: center; font-size: 14px; display: flex; justify-content: center; align-items: center; width: 100%; margin-top: -35px; background: rgba(0,0,0,0.7); color: white; line-height: 35px; z-index: 20;">click on the image to Watch the video</figcaption></figure>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">You can also use DocArray to view individual frames by their frame number. For example, frame #1400:</p>
<pre class="custom" data-tool="mdnice编辑器" style="margin-top: 10px; margin-bottom: 10px; border-radius: 5px; box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"><span style="display: block; background: url(https://files.mdnice.com/user/3441/876cad08-0422-409d-bb5a-08afec5da8ee.svg); height: 30px; width: 100%; background-size: 40px; background-repeat: no-repeat; background-color: #1E1E1E; margin-bottom: -7px; border-radius: 5px; background-position: 10px 10px;"></span><code class="hljs" style="overflow-x: auto; padding: 16px; color: #DCDCDC; display: -webkit-box; font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; font-size: 12px; -webkit-overflow-scrolling: touch; padding-top: 15px; background: #1E1E1E; border-radius: 5px;"><span class="hljs-keyword" style="color: #569CD6; line-height: 26px;">import</span>&nbsp;numpy<br><br>Document(tensor=numpy.rot90(video_data.tensor[<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">1400</span>],&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">-1</span>)).display()<br></code></pre>
<figure data-tool="mdnice编辑器" style="margin: 0; margin-top: 10px; margin-bottom: 10px; display: flex; flex-direction: column; justify-content: center; align-items: center;"><img src="https://jina-ai-gmbh.ghost.io/content/images/2022/12/frame1400.png" alt="Frame 1400 of the trailer to _Santa Claus conquers the Martians_" style="display: block; margin: 0 auto; max-width: 100%;"><figcaption style="margin-top: 5px; text-align: center; color: #888; font-size: 14px;">Frame 1400 of the trailer to _Santa Claus conquers the Martians_</figcaption></figure>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">And we can extract clips from the video by giving specific start and end frame numbers:</p>
<pre class="custom" data-tool="mdnice编辑器" style="margin-top: 10px; margin-bottom: 10px; border-radius: 5px; box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"><span style="display: block; background: url(https://files.mdnice.com/user/3441/876cad08-0422-409d-bb5a-08afec5da8ee.svg); height: 30px; width: 100%; background-size: 40px; background-repeat: no-repeat; background-color: #1E1E1E; margin-bottom: -7px; border-radius: 5px; background-position: 10px 10px;"></span><code class="hljs" style="overflow-x: auto; padding: 16px; color: #DCDCDC; display: -webkit-box; font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; font-size: 12px; -webkit-overflow-scrolling: touch; padding-top: 15px; background: #1E1E1E; border-radius: 5px;">clip_data&nbsp;=&nbsp;video_data.tensor[<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">3000</span>:<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">3300</span>]<br>Document(tensor=clip_data).save_video_tensor_to_file(<span class="hljs-string" style="color: #D69D85; line-height: 26px;">"clip.mp4"</span>)<br>Document(uri=<span class="hljs-string" style="color: #D69D85; line-height: 26px;">"clip.mp4"</span>).display()<br></code></pre>
<figure data-tool="mdnice编辑器" style="margin: 0; margin-top: 10px; margin-bottom: 10px; display: flex; flex-direction: column; justify-content: center; align-items: center;"><a href="https://jina-ai-gmbh.ghost.io/content/media/2022/12/clip-1.mp4" style="text-decoration: none; word-wrap: break-word; font-weight: bold; color: #009688; border-bottom: 1px solid #009688; border: none; display: flex; justify-content: center; align-items: center;"><img src="https://jina-ai-gmbh.ghost.io/content/images/2022/12/media-thumbnail-ember331.jpg" alt="click the image, Watch the video" style="display: block; max-width: 100%; margin: 0px;"></a><figcaption style="text-align: center; font-size: 14px; display: flex; justify-content: center; align-items: center; width: 100%; margin-top: -35px; background: rgba(0,0,0,0.7); color: white; line-height: 35px; z-index: 20;">click the image, Watch the video</figcaption></figure>
<h2 data-tool="mdnice编辑器" style="margin-top: 30px; margin-bottom: 15px; padding: 0px; font-weight: bold; font-size: 22px; color: #009688; padding-left: 10px; margin: 1em auto; border-left: 3px solid #009688;"><span class="prefix" style="display: none;"></span><span class="content">Extracting Keyframes</span><span class="suffix"></span></h2>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">The procedure we’re using to search in videos is to extract <em style="font-style: italic; color: black;">keyframes</em> and then perform our search on just those still images.</p>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;"><strong style="font-weight: bold; color: black;">What is a keyframe?</strong></p>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">A keyframe is a frame in a video that is the first after a break from smooth frame-to-frame transition. This is the same as a <em style="font-style: italic; color: black;">cut</em> in film editing. We can identify them by going through the video frame by frame and comparing each frame to the next one. If the difference between the two frames is more than some value, we take that to mean that there was a cut, and the later frame is a keyframe.</p>
<figure data-tool="mdnice编辑器" style="margin: 0; margin-top: 10px; margin-bottom: 10px; display: flex; flex-direction: column; justify-content: center; align-items: center;"><img src="https://jina-ai-gmbh.ghost.io/content/images/2022/11/Keyframe.png" alt style="display: block; margin: 0 auto; max-width: 100%;"></figure>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">A visual illustration of keyframes.</p>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">DocArray will automatically collect keyframes as it loads the video with the method <code style="font-size: 14px; word-wrap: break-word; padding: 2px 4px; border-radius: 4px; margin: 0 2px; background-color: rgba(27,31,35,.05); font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; word-break: break-all; color: #009688;">Document.load_uri_to_video_tensor()</code> and store them in the <code style="font-size: 14px; word-wrap: break-word; padding: 2px 4px; border-radius: 4px; margin: 0 2px; background-color: rgba(27,31,35,.05); font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; word-break: break-all; color: #009688;">Document.tags</code> dictionary under the key <code style="font-size: 14px; word-wrap: break-word; padding: 2px 4px; border-radius: 4px; margin: 0 2px; background-color: rgba(27,31,35,.05); font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; word-break: break-all; color: #009688;">keyframe_indices</code>:</p>
<pre class="custom" data-tool="mdnice编辑器" style="margin-top: 10px; margin-bottom: 10px; border-radius: 5px; box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"><span style="display: block; background: url(https://files.mdnice.com/user/3441/876cad08-0422-409d-bb5a-08afec5da8ee.svg); height: 30px; width: 100%; background-size: 40px; background-repeat: no-repeat; background-color: #1E1E1E; margin-bottom: -7px; border-radius: 5px; background-position: 10px 10px;"></span><code class="hljs" style="overflow-x: auto; padding: 16px; color: #DCDCDC; display: -webkit-box; font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; font-size: 12px; -webkit-overflow-scrolling: touch; padding-top: 15px; background: #1E1E1E; border-radius: 5px;">print(video_data.tags[<span class="hljs-string" style="color: #D69D85; line-height: 26px;">'keyframe_indices'</span>])<br><br>[<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">0</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">25</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">196</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">261</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">325</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">395</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">478</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">534</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">695</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">728</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">840</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">1019</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">1059</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">1131</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">1191</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">1245</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">1340</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">1389</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">1505</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">1573</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">1631</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">1674</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">1750</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">1869</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">1910</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">2010</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">2105</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">2184</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">2248</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">2335</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">2585</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">2618</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">2648</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">2706</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">2756</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">2788</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">2906</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">2950</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">3050</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">3100</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">3128</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">3190</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">3216</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">3314</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">3356</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">3421</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">3514</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">3586</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">3760</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">3828</span>,&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">4078</span>]<br></code></pre>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">The frame numbers of the keyframes are stored in the <code style="font-size: 14px; word-wrap: break-word; padding: 2px 4px; border-radius: 4px; margin: 0 2px; background-color: rgba(27,31,35,.05); font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; word-break: break-all; color: #009688;">Document.tags</code> dictionary under the key <code style="font-size: 14px; word-wrap: break-word; padding: 2px 4px; border-radius: 4px; margin: 0 2px; background-color: rgba(27,31,35,.05); font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; word-break: break-all; color: #009688;">keyframe_indices</code>.</p>
<h2 data-tool="mdnice编辑器" style="margin-top: 30px; margin-bottom: 15px; padding: 0px; font-weight: bold; font-size: 22px; color: #009688; padding-left: 10px; margin: 1em auto; border-left: 3px solid #009688;"><span class="prefix" style="display: none;"></span><span class="content">Performing Search</span><span class="suffix"></span></h2>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">First, extract all the keyframes as images, and put each one into its own <code style="font-size: 14px; word-wrap: break-word; padding: 2px 4px; border-radius: 4px; margin: 0 2px; background-color: rgba(27,31,35,.05); font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; word-break: break-all; color: #009688;">Document</code>. Then compile all the frames into a <code style="font-size: 14px; word-wrap: break-word; padding: 2px 4px; border-radius: 4px; margin: 0 2px; background-color: rgba(27,31,35,.05); font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; word-break: break-all; color: #009688;">DocumentArray</code> object:</p>
<pre class="custom" data-tool="mdnice编辑器" style="margin-top: 10px; margin-bottom: 10px; border-radius: 5px; box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"><span style="display: block; background: url(https://files.mdnice.com/user/3441/876cad08-0422-409d-bb5a-08afec5da8ee.svg); height: 30px; width: 100%; background-size: 40px; background-repeat: no-repeat; background-color: #1E1E1E; margin-bottom: -7px; border-radius: 5px; background-position: 10px 10px;"></span><code class="hljs" style="overflow-x: auto; padding: 16px; color: #DCDCDC; display: -webkit-box; font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; font-size: 12px; -webkit-overflow-scrolling: touch; padding-top: 15px; background: #1E1E1E; border-radius: 5px;"><span class="hljs-keyword" style="color: #569CD6; line-height: 26px;">from</span>&nbsp;docarray&nbsp;<span class="hljs-keyword" style="color: #569CD6; line-height: 26px;">import</span>&nbsp;Document,&nbsp;DocumentArray<br><span class="hljs-keyword" style="color: #569CD6; line-height: 26px;">from</span>&nbsp;numpy&nbsp;<span class="hljs-keyword" style="color: #569CD6; line-height: 26px;">import</span>&nbsp;rot90<br><br>keyframe_indices&nbsp;=&nbsp;video_data.tags[<span class="hljs-string" style="color: #D69D85; line-height: 26px;">'keyframe_indices'</span>]<br>keyframes&nbsp;=&nbsp;DocumentArray()<br><span class="hljs-keyword" style="color: #569CD6; line-height: 26px;">for</span>&nbsp;idx&nbsp;<span class="hljs-keyword" style="color: #569CD6; line-height: 26px;">in</span>&nbsp;range(<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">0</span>,&nbsp;len(keyframe_indices)&nbsp;-&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">1</span>):<br>&nbsp;keyframe_number&nbsp;=&nbsp;keyframe_indices[idx]<br>&nbsp;&nbsp;&nbsp;&nbsp;keyframe_tensor&nbsp;=&nbsp;rot90(video_data.tensor[keyframe_number],&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">-1</span>)<br>&nbsp;&nbsp;&nbsp;&nbsp;clip_indices&nbsp;=&nbsp;{<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span class="hljs-string" style="color: #D69D85; line-height: 26px;">'start'</span>:&nbsp;str(keyframe_number),<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span class="hljs-string" style="color: #D69D85; line-height: 26px;">'end'</span>:&nbsp;str(keyframe_indices[idx&nbsp;+&nbsp;<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">1</span>]),<br>&nbsp;&nbsp;&nbsp;&nbsp;}<br>&nbsp;&nbsp;&nbsp;&nbsp;keyframe&nbsp;=&nbsp;Document(tags=clip_indices,&nbsp;tensor=keyframe_tensor)<br>&nbsp;&nbsp;&nbsp;&nbsp;keyframes.append(keyframe)<br></code></pre>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">The code above uses the <code style="font-size: 14px; word-wrap: break-word; padding: 2px 4px; border-radius: 4px; margin: 0 2px; background-color: rgba(27,31,35,.05); font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; word-break: break-all; color: #009688;">Document.tags</code> dictionary to store the frame number (as <code style="font-size: 14px; word-wrap: break-word; padding: 2px 4px; border-radius: 4px; margin: 0 2px; background-color: rgba(27,31,35,.05); font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; word-break: break-all; color: #009688;">start</code>) and the frame number of the next keyframe (as <code style="font-size: 14px; word-wrap: break-word; padding: 2px 4px; border-radius: 4px; margin: 0 2px; background-color: rgba(27,31,35,.05); font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; word-break: break-all; color: #009688;">end</code>) so that we can extract a video clip corresponding to that keyframe.</p>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">Then access CLIP-as-service, passing it the query text – in the example <em style="font-style: italic; color: black;">"Santa Claus"</em> – and the collection of keyframe images, and it will return the images ranked by how well they match the query text:</p>
<pre class="custom" data-tool="mdnice编辑器" style="margin-top: 10px; margin-bottom: 10px; border-radius: 5px; box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"><span style="display: block; background: url(https://files.mdnice.com/user/3441/876cad08-0422-409d-bb5a-08afec5da8ee.svg); height: 30px; width: 100%; background-size: 40px; background-repeat: no-repeat; background-color: #1E1E1E; margin-bottom: -7px; border-radius: 5px; background-position: 10px 10px;"></span><code class="hljs" style="overflow-x: auto; padding: 16px; color: #DCDCDC; display: -webkit-box; font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; font-size: 12px; -webkit-overflow-scrolling: touch; padding-top: 15px; background: #1E1E1E; border-radius: 5px;"><span class="hljs-keyword" style="color: #569CD6; line-height: 26px;">from</span>&nbsp;docarray&nbsp;<span class="hljs-keyword" style="color: #569CD6; line-height: 26px;">import</span>&nbsp;Document,&nbsp;DocumentArray<br><span class="hljs-keyword" style="color: #569CD6; line-height: 26px;">from</span>&nbsp;clip_client&nbsp;<span class="hljs-keyword" style="color: #569CD6; line-height: 26px;">import</span>&nbsp;Client<br><br>server_url&nbsp;=&nbsp;<span class="hljs-string" style="color: #D69D85; line-height: 26px;">"grpcs://api.clip.jina.ai:2096"</span><br><br><span class="hljs-comment" style="color: #57A64A; font-style: italic; line-height: 26px;">#&nbsp;substitute&nbsp;your&nbsp;own&nbsp;token&nbsp;in&nbsp;the&nbsp;line&nbsp;below!</span><br>jina_auth_token&nbsp;=&nbsp;<span class="hljs-string" style="color: #D69D85; line-height: 26px;">"54f0f0ef5d514ca1908698fc6d9555a5"</span><br><br>client&nbsp;=&nbsp;Client(server_url,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;credential={<span class="hljs-string" style="color: #D69D85; line-height: 26px;">"Authorization"</span>:&nbsp;jina_auth_token})<br>query&nbsp;=&nbsp;Document(text=<span class="hljs-string" style="color: #D69D85; line-height: 26px;">"Santa&nbsp;Claus"</span>,&nbsp;matches=keyframes)<br>ranked_result&nbsp;=&nbsp;client.rank([query])[<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">0</span>]<br></code></pre>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">We transmit the query and keyframe images to Jina AI Cloud, and CLIP-as-service calculates an embedding vector for the text query and for each keyframe. Then, we measure the distance between the keyframe vectors and text query vectors. We return a list of keyframes ordered by their proximity to the text query in the embedding space.</p>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">You can see this represented in the figure below.</p>
<figure data-tool="mdnice编辑器" style="margin: 0; margin-top: 10px; margin-bottom: 10px; display: flex; flex-direction: column; justify-content: center; align-items: center;"><img src="https://jina-ai-gmbh.ghost.io/content/images/2022/11/santa-and-martians-1.png" alt style="display: block; margin: 0 auto; max-width: 100%;"></figure>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">CLIP translates texts and images into vectors in a common embedding space where the distance between them reflects their semantic similarity. The embedding vectors of images with Santa Claus are much closer to the vector for the text "Santa Claus" than other images.</p>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">The query reorders the keyframe images in <code style="font-size: 14px; word-wrap: break-word; padding: 2px 4px; border-radius: 4px; margin: 0 2px; background-color: rgba(27,31,35,.05); font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; word-break: break-all; color: #009688;">Document.matches</code> in order from the best match to the worst.</p>
<pre class="custom" data-tool="mdnice编辑器" style="margin-top: 10px; margin-bottom: 10px; border-radius: 5px; box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"><span style="display: block; background: url(https://files.mdnice.com/user/3441/876cad08-0422-409d-bb5a-08afec5da8ee.svg); height: 30px; width: 100%; background-size: 40px; background-repeat: no-repeat; background-color: #1E1E1E; margin-bottom: -7px; border-radius: 5px; background-position: 10px 10px;"></span><code class="hljs" style="overflow-x: auto; padding: 16px; color: #DCDCDC; display: -webkit-box; font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; font-size: 12px; -webkit-overflow-scrolling: touch; padding-top: 15px; background: #1E1E1E; border-radius: 5px;">print(ranked_result.matches[<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">0</span>].tags)<br><br>{<span class="hljs-string" style="color: #D69D85; line-height: 26px;">'start'</span>:&nbsp;<span class="hljs-string" style="color: #D69D85; line-height: 26px;">'2105'</span>,&nbsp;<span class="hljs-string" style="color: #D69D85; line-height: 26px;">'end'</span>:&nbsp;<span class="hljs-string" style="color: #D69D85; line-height: 26px;">'2184'</span>}<br></code></pre>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">You can see in the <code style="font-size: 14px; word-wrap: break-word; padding: 2px 4px; border-radius: 4px; margin: 0 2px; background-color: rgba(27,31,35,.05); font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; word-break: break-all; color: #009688;">Document.tags</code> section that we've retained the information about this keyframe: It is frame #2105 and the next keyframe is at #2184. With this information, we can get the short video clip that this matches:</p>
<pre class="custom" data-tool="mdnice编辑器" style="margin-top: 10px; margin-bottom: 10px; border-radius: 5px; box-shadow: rgba(0, 0, 0, 0.55) 0px 2px 10px;"><span style="display: block; background: url(https://files.mdnice.com/user/3441/876cad08-0422-409d-bb5a-08afec5da8ee.svg); height: 30px; width: 100%; background-size: 40px; background-repeat: no-repeat; background-color: #1E1E1E; margin-bottom: -7px; border-radius: 5px; background-position: 10px 10px;"></span><code class="hljs" style="overflow-x: auto; padding: 16px; color: #DCDCDC; display: -webkit-box; font-family: Operator Mono, Consolas, Monaco, Menlo, monospace; font-size: 12px; -webkit-overflow-scrolling: touch; padding-top: 15px; background: #1E1E1E; border-radius: 5px;">match&nbsp;=&nbsp;ranked_result.matches[<span class="hljs-number" style="color: #B8D7A3; line-height: 26px;">0</span>]<br>start_frame&nbsp;=&nbsp;int(match.tags[<span class="hljs-string" style="color: #D69D85; line-height: 26px;">'start'</span>])<br>end_frame&nbsp;=&nbsp;int(match.tags[<span class="hljs-string" style="color: #D69D85; line-height: 26px;">'end'</span>])<br>clip_data&nbsp;=&nbsp;video_data.tensor[start_frame:end_frame]&nbsp;<br>Document(tensor=clip_data).save_video_tensor_to_file(<span class="hljs-string" style="color: #D69D85; line-height: 26px;">"match.mp4"</span>)<br>Document(uri=<span class="hljs-string" style="color: #D69D85; line-height: 26px;">"match.mp4"</span>).display()<br></code></pre>
<figure data-tool="mdnice编辑器" style="margin: 0; margin-top: 10px; margin-bottom: 10px; display: flex; flex-direction: column; justify-content: center; align-items: center;"><a href="https://jina-ai-gmbh.ghost.io/content/media/2022/12/match.mp4" style="text-decoration: none; word-wrap: break-word; font-weight: bold; color: #009688; border-bottom: 1px solid #009688; border: none; display: flex; justify-content: center; align-items: center;"><img src="https://jina-ai-gmbh.ghost.io/content/images/2022/12/media-thumbnail-ember857.jpg" alt="click the image, Watch the video" style="display: block; max-width: 100%; margin: 0px;"></a><figcaption style="text-align: center; font-size: 14px; display: flex; justify-content: center; align-items: center; width: 100%; margin-top: -35px; background: rgba(0,0,0,0.7); color: white; line-height: 35px; z-index: 20;">click the image, Watch the video</figcaption></figure>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">The top five clips all contain Santa Claus:</p>
<figure data-tool="mdnice编辑器" style="margin: 0; margin-top: 10px; margin-bottom: 10px; display: flex; flex-direction: column; justify-content: center; align-items: center;"><img src="https://files.mdnice.com/user/39412/98c20fcb-c940-4dd7-9ba5-9a9eed591579.png" alt style="display: block; margin: 0 auto; max-width: 100%;"></figure>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">This quick and simple technique brings very sophisticated multimodal AI to users with only a standard computer setup. CLIP-as-service is a powerful tool for anyone who needs to search through large volumes of digital media to find what they’re looking for. It saves time and helps you get the most out of your digital media collection.</p>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">So the next time you need to find Santa Claus, CLIP-as-service is here to help you look!</p>
<figure data-tool="mdnice编辑器" style="margin: 0; margin-top: 10px; margin-bottom: 10px; display: flex; flex-direction: column; justify-content: center; align-items: center;"><img src="https://jina-ai-gmbh.ghost.io/content/images/2022/11/DALL-E-2022-11-28-18.01.17---Santa-Claus-with-Martians--painted-in-the-style-of-Norman-Rockwell.png" alt="“Santa Claus with Martians, painted in the style of Norman Rockwell” according to DALL-E 2." style="display: block; margin: 0 auto; max-width: 100%;"><figcaption style="margin-top: 5px; text-align: center; color: #888; font-size: 14px;">“Santa Claus with Martians, painted in the style of Norman Rockwell” according to <a href="https://openai.com/dall-e-2/" style="text-decoration: none; word-wrap: break-word; font-weight: bold; color: #009688; border-bottom: 1px solid #009688; border: none; display: flex; justify-content: center; align-items: center;">DALL-E 2</a>.</figcaption></figure>
<h3 data-tool="mdnice编辑器" style="margin-top: 30px; margin-bottom: 15px; padding: 0px; font-weight: bold; color: black; font-size: 20px; margin: 0.6em auto; padding-left: 10px; border-left: 2px solid #009688;"><span class="prefix" style="display: none;"></span><span class="content">Autor:</span><span class="suffix" style="display: none;"></span></h3>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">Jie Fu,Scott Martens</p>
<h3 data-tool="mdnice编辑器" style="margin-top: 30px; margin-bottom: 15px; padding: 0px; font-weight: bold; color: black; font-size: 20px; margin: 0.6em auto; padding-left: 10px; border-left: 2px solid #009688;"><span class="prefix" style="display: none;"></span><span class="content">Original link:</span><span class="suffix" style="display: none;"></span></h3>
<p data-tool="mdnice编辑器" style="font-size: 16px; padding-top: 8px; padding-bottom: 8px; margin: 0; line-height: 26px; color: black; text-align: justify;">https://jina.ai/news/guide-using-opentelemetry-jina-monitoring-tracing-applications/</p>
</section>