Upload 19 files

hub/snakers4_silero-vad_master/.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,52 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: Bug report - [X]
+labels: bug
+assignees: snakers4
+
+---
+
+## 🐛 Bug
+
+<!-- A clear and concise description of what the bug is. -->
+
+## To Reproduce
+
+Steps to reproduce the behavior:
+
+1.
+2.
+3.
+
+<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->
+
+## Expected behavior
+
+<!-- A clear and concise description of what you expected to happen. -->
+
+## Environment
+
+Please copy and paste the output from this 
+[environment collection script](https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py)
+(or fill out the checklist below manually).
+
+You can get the script and run it with:
+```
+wget https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py
+# For security purposes, please check the contents of collect_env.py before running it.
+python collect_env.py
+```
+
+ - PyTorch Version (e.g., 1.0):
+ - OS (e.g., Linux):
+ - How you installed PyTorch (`conda`, `pip`, source):
+ - Build command you used (if compiling from source):
+ - Python version:
+ - CUDA/cuDNN version:
+ - GPU models and configuration:
+ - Any other relevant information:
+
+## Additional context
+
+<!-- Add any other context about the problem here. -->

hub/snakers4_silero-vad_master/.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,27 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: Feature request - [X]
+labels: enhancement
+assignees: snakers4
+
+---
+
+## 🚀 Feature
+<!-- A clear and concise description of the feature proposal -->
+
+## Motivation
+
+<!-- Please outline the motivation for the proposal. Is your feature request related to a problem? e.g., I'm always frustrated when [...]. If this is related to another GitHub issue, please link here too -->
+
+## Pitch
+
+<!-- A clear and concise description of what you want to happen. -->
+
+## Alternatives
+
+<!-- A clear and concise description of any alternative solutions or features you've considered, if any. -->
+
+## Additional context
+
+<!-- Add any other context or screenshots about the feature request here. -->

hub/snakers4_silero-vad_master/.github/ISSUE_TEMPLATE/questions---help---support.md

@@ -0,0 +1,12 @@
+---
+name: Questions / Help / Support
+about: Ask for help, support or ask a question
+title: "❓ Questions / Help / Support"
+labels: help wanted
+assignees: snakers4
+
+---
+
+## ❓ Questions and Help
+
+We have a [wiki](https://github.com/snakers4/silero-models/wiki) available for our users. Please make sure you have checked it out first.

hub/snakers4_silero-vad_master/CODE_OF_CONDUCT.md

@@ -0,0 +1,76 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, sex characteristics, gender identity and expression,
+level of experience, education, socio-economic status, nationality, personal
+appearance, race, religion, or sexual identity and orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+ advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+ address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+ professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community. Examples of
+representing a project or community include using an official project e-mail
+address, posting via an official social media account, or acting as an appointed
+representative at an online or offline event. Representation of a project may be
+further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project team at aveysov@gmail.com. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. The project team is
+obligated to maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
+
+[homepage]: https://www.contributor-covenant.org
+
+For answers to common questions about this code of conduct, see
+https://www.contributor-covenant.org/faq

hub/snakers4_silero-vad_master/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2020-present Silero Team
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

hub/snakers4_silero-vad_master/README.md

@@ -0,0 +1,113 @@
+[![Mailing list : test](http://img.shields.io/badge/Email-gray.svg?style=for-the-badge&logo=gmail)](mailto:hello@silero.ai) [![Mailing list : test](http://img.shields.io/badge/Telegram-blue.svg?style=for-the-badge&logo=telegram)](https://t.me/silero_speech) [![License: CC BY-NC 4.0](https://img.shields.io/badge/License-MIT-lightgrey.svg?style=for-the-badge)](https://github.com/snakers4/silero-vad/blob/master/LICENSE)
+
+[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/snakers4/silero-vad/blob/master/silero-vad.ipynb)
+
+![header](https://user-images.githubusercontent.com/12515440/89997349-b3523080-dc94-11ea-9906-ca2e8bc50535.png)
+
+<br/>
+<h1 align="center">Silero VAD</h1>
+<br/>
+
+**Silero VAD** - pre-trained enterprise-grade [Voice Activity Detector](https://en.wikipedia.org/wiki/Voice_activity_detection) (also see our [STT models](https://github.com/snakers4/silero-models)).
+
+This repository also includes Number Detector and Language classifier [models](https://github.com/snakers4/silero-vad/wiki/Other-Models)
+
+<br/>
+
+<p align="center">
+  <img src="https://user-images.githubusercontent.com/36505480/198026365-8da383e0-5398-4a12-b7f8-22c2c0059512.png" />
+</p>
+
+<details>
+<summary>Real Time Example</summary>
+  
+https://user-images.githubusercontent.com/36505480/144874384-95f80f6d-a4f1-42cc-9be7-004c891dd481.mp4
+  
+</details>
+
+<br/>
+<h2 align="center">Key Features</h2>
+<br/>
+
+- **Stellar accuracy**
+
+  Silero VAD has [excellent results](https://github.com/snakers4/silero-vad/wiki/Quality-Metrics#vs-other-available-solutions) on speech detection tasks.
+  
+- **Fast**
+
+  One audio chunk (30+ ms) [takes](https://github.com/snakers4/silero-vad/wiki/Performance-Metrics#silero-vad-performance-metrics) less than **1ms** to be processed on a single CPU thread. Using batching or GPU can also improve performance considerably. Under certain conditions ONNX may even run up to 4-5x faster. 
+
+- **Lightweight**
+
+  JIT model is around one megabyte in size.
+
+- **General**
+
+  Silero VAD was trained on huge corpora that include over **100** languages and it performs well on audios from different domains with various background noise and quality levels.
+
+- **Flexible sampling rate**
+
+  Silero VAD [supports](https://github.com/snakers4/silero-vad/wiki/Quality-Metrics#sample-rate-comparison)  **8000 Hz** and **16000 Hz** [sampling rates](https://en.wikipedia.org/wiki/Sampling_(signal_processing)#Sampling_rate).
+
+- **Flexible chunk size**
+
+  Model was trained on **30 ms**. Longer chunks are supported directly, others may work as well.
+
+- **Highly Portable**
+
+  Silero VAD reaps benefits from the rich ecosystems built around **PyTorch** and **ONNX** running everywhere where these runtimes are available.
+
+- **No Strings Attached**
+
+   Published under permissive license (MIT) Silero VAD has zero strings attached - no telemetry, no keys, no registration, no built-in expiration, no keys or vendor lock.
+
+<br/>
+<h2 align="center">Typical Use Cases</h2>
+<br/>
+
+- Voice activity detection for IOT / edge / mobile use cases
+- Data cleaning and preparation, voice detection in general
+- Telephony and call-center automation, voice bots
+- Voice interfaces
+
+<br/>
+<h2 align="center">Links</h2>
+<br/>
+
+
+- [Examples and Dependencies](https://github.com/snakers4/silero-vad/wiki/Examples-and-Dependencies#dependencies)
+- [Quality Metrics](https://github.com/snakers4/silero-vad/wiki/Quality-Metrics)
+- [Performance Metrics](https://github.com/snakers4/silero-vad/wiki/Performance-Metrics)
+- [Number Detector and Language classifier models](https://github.com/snakers4/silero-vad/wiki/Other-Models)
+- [Versions and Available Models](https://github.com/snakers4/silero-vad/wiki/Version-history-and-Available-Models)
+- [Further reading](https://github.com/snakers4/silero-models#further-reading)
+- [FAQ](https://github.com/snakers4/silero-vad/wiki/FAQ)
+
+<br/>
+<h2 align="center">Get In Touch</h2>
+<br/>
+
+Try our models, create an [issue](https://github.com/snakers4/silero-vad/issues/new), start a [discussion](https://github.com/snakers4/silero-vad/discussions/new), join our telegram [chat](https://t.me/silero_speech), [email](mailto:hello@silero.ai) us, read our [news](https://t.me/silero_news).
+
+Please see our [wiki](https://github.com/snakers4/silero-models/wiki) and [tiers](https://github.com/snakers4/silero-models/wiki/Licensing-and-Tiers) for relevant information and [email](mailto:hello@silero.ai) us directly.
+
+**Citations**
+
+```
+@misc{Silero VAD,
+  author = {Silero Team},
+  title = {Silero VAD: pre-trained enterprise-grade Voice Activity Detector (VAD), Number Detector and Language Classifier},
+  year = {2021},
+  publisher = {GitHub},
+  journal = {GitHub repository},
+  howpublished = {\url{https://github.com/snakers4/silero-vad}},
+  commit = {insert_some_commit_here},
+  email = {hello@silero.ai}
+}
+```
+
+<br/>
+<h2 align="center">VAD-based Community Apps</h2>
+<br/>
+
+- Voice activity detection for the [browser](https://github.com/ricky0123/vad) using ONNX Runtime Web 

hub/snakers4_silero-vad_master/examples/colab_record_example.ipynb

@@ -0,0 +1,241 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "bccAucKjnPHm"
+   },
+   "source": [
+    "### Dependencies and inputs"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "id": "cSih95WFmwgi"
+   },
+   "outputs": [],
+   "source": [
+    "!pip -q install pydub\n",
+    "from google.colab import output\n",
+    "from base64 import b64decode, b64encode\n",
+    "from io import BytesIO\n",
+    "import numpy as np\n",
+    "from pydub import AudioSegment\n",
+    "from IPython.display import HTML, display\n",
+    "import torch\n",
+    "import matplotlib.pyplot as plt\n",
+    "import moviepy.editor as mpe\n",
+    "from matplotlib.animation import FuncAnimation, FFMpegWriter\n",
+    "import matplotlib\n",
+    "matplotlib.use('Agg')\n",
+    "\n",
+    "torch.set_num_threads(1)\n",
+    "\n",
+    "model, _ = torch.hub.load(repo_or_dir='snakers4/silero-vad',\n",
+    "                              model='silero_vad',\n",
+    "                              force_reload=True)\n",
+    "\n",
+    "def int2float(sound):\n",
+    "    abs_max = np.abs(sound).max()\n",
+    "    sound = sound.astype('float32')\n",
+    "    if abs_max > 0:\n",
+    "        sound *= 1/abs_max\n",
+    "    sound = sound.squeeze()\n",
+    "    return sound\n",
+    "\n",
+    "AUDIO_HTML = \"\"\"\n",
+    "<script>\n",
+    "var my_div = document.createElement(\"DIV\");\n",
+    "var my_p = document.createElement(\"P\");\n",
+    "var my_btn = document.createElement(\"BUTTON\");\n",
+    "var t = document.createTextNode(\"Press to start recording\");\n",
+    "\n",
+    "my_btn.appendChild(t);\n",
+    "//my_p.appendChild(my_btn);\n",
+    "my_div.appendChild(my_btn);\n",
+    "document.body.appendChild(my_div);\n",
+    "\n",
+    "var base64data = 0;\n",
+    "var reader;\n",
+    "var recorder, gumStream;\n",
+    "var recordButton = my_btn;\n",
+    "\n",
+    "var handleSuccess = function(stream) {\n",
+    "  gumStream = stream;\n",
+    "  var options = {\n",
+    "    //bitsPerSecond: 8000, //chrome seems to ignore, always 48k\n",
+    "    mimeType : 'audio/webm;codecs=opus'\n",
+    "    //mimeType : 'audio/webm;codecs=pcm'\n",
+    "  };            \n",
+    "  //recorder = new MediaRecorder(stream, options);\n",
+    "  recorder = new MediaRecorder(stream);\n",
+    "  recorder.ondataavailable = function(e) {            \n",
+    "    var url = URL.createObjectURL(e.data);\n",
+    "    // var preview = document.createElement('audio');\n",
+    "    // preview.controls = true;\n",
+    "    // preview.src = url;\n",
+    "    // document.body.appendChild(preview);\n",
+    "\n",
+    "    reader = new FileReader();\n",
+    "    reader.readAsDataURL(e.data); \n",
+    "    reader.onloadend = function() {\n",
+    "      base64data = reader.result;\n",
+    "      //console.log(\"Inside FileReader:\" + base64data);\n",
+    "    }\n",
+    "  };\n",
+    "  recorder.start();\n",
+    "  };\n",
+    "\n",
+    "recordButton.innerText = \"Recording... press to stop\";\n",
+    "\n",
+    "navigator.mediaDevices.getUserMedia({audio: true}).then(handleSuccess);\n",
+    "\n",
+    "\n",
+    "function toggleRecording() {\n",
+    "  if (recorder && recorder.state == \"recording\") {\n",
+    "      recorder.stop();\n",
+    "      gumStream.getAudioTracks()[0].stop();\n",
+    "      recordButton.innerText = \"Saving recording...\"\n",
+    "  }\n",
+    "}\n",
+    "\n",
+    "// https://stackoverflow.com/a/951057\n",
+    "function sleep(ms) {\n",
+    "  return new Promise(resolve => setTimeout(resolve, ms));\n",
+    "}\n",
+    "\n",
+    "var data = new Promise(resolve=>{\n",
+    "//recordButton.addEventListener(\"click\", toggleRecording);\n",
+    "recordButton.onclick = ()=>{\n",
+    "toggleRecording()\n",
+    "\n",
+    "sleep(2000).then(() => {\n",
+    "  // wait 2000ms for the data to be available...\n",
+    "  // ideally this should use something like await...\n",
+    "  //console.log(\"Inside data:\" + base64data)\n",
+    "  resolve(base64data.toString())\n",
+    "\n",
+    "});\n",
+    "\n",
+    "}\n",
+    "});\n",
+    "      \n",
+    "</script>\n",
+    "\"\"\"\n",
+    "\n",
+    "def record(sec=10):\n",
+    "    display(HTML(AUDIO_HTML))\n",
+    "    s = output.eval_js(\"data\")\n",
+    "    b = b64decode(s.split(',')[1])\n",
+    "    audio = AudioSegment.from_file(BytesIO(b))\n",
+    "    audio.export('test.mp3', format='mp3')\n",
+    "    audio = audio.set_channels(1)\n",
+    "    audio = audio.set_frame_rate(16000)\n",
+    "    audio_float = int2float(np.array(audio.get_array_of_samples()))\n",
+    "    audio_tens = torch.tensor(audio_float )\n",
+    "    return audio_tens\n",
+    "\n",
+    "def make_animation(probs, audio_duration, interval=40):\n",
+    "    fig = plt.figure(figsize=(16, 9))\n",
+    "    ax = plt.axes(xlim=(0, audio_duration), ylim=(0, 1.02))\n",
+    "    line, = ax.plot([], [], lw=2)\n",
+    "    x = [i / 16000 * 512 for i in range(len(probs))]\n",
+    "    plt.xlabel('Time, seconds', fontsize=16)\n",
+    "    plt.ylabel('Speech Probability', fontsize=16)\n",
+    "\n",
+    "    def init():\n",
+    "        plt.fill_between(x, probs, color='#064273')\n",
+    "        line.set_data([], [])\n",
+    "        line.set_color('#990000')\n",
+    "        return line,\n",
+    "\n",
+    "    def animate(i):\n",
+    "        x = i * interval / 1000 - 0.04\n",
+    "        y = np.linspace(0, 1.02, 2)\n",
+    "        \n",
+    "        line.set_data(x, y)\n",
+    "        line.set_color('#990000')\n",
+    "        return line,\n",
+    "\n",
+    "    anim = FuncAnimation(fig, animate, init_func=init, interval=interval, save_count=audio_duration / (interval / 1000))\n",
+    "\n",
+    "    f = r\"animation.mp4\" \n",
+    "    writervideo = FFMpegWriter(fps=1000/interval) \n",
+    "    anim.save(f, writer=writervideo)\n",
+    "    plt.close('all')\n",
+    "\n",
+    "def combine_audio(vidname, audname, outname, fps=25): \n",
+    "    my_clip = mpe.VideoFileClip(vidname, verbose=False)\n",
+    "    audio_background = mpe.AudioFileClip(audname)\n",
+    "    final_clip = my_clip.set_audio(audio_background)\n",
+    "    final_clip.write_videofile(outname,fps=fps,verbose=False)\n",
+    "\n",
+    "def record_make_animation():\n",
+    "  tensor = record()\n",
+    "\n",
+    "  print('Calculating probabilities...')\n",
+    "  speech_probs = []\n",
+    "  window_size_samples = 512\n",
+    "  for i in range(0, len(tensor), window_size_samples):\n",
+    "      if len(tensor[i: i+ window_size_samples]) < window_size_samples:\n",
+    "        break\n",
+    "      speech_prob = model(tensor[i: i+ window_size_samples], 16000).item()\n",
+    "      speech_probs.append(speech_prob)\n",
+    "  model.reset_states()\n",
+    "  print('Making animation...')\n",
+    "  make_animation(speech_probs, len(tensor) / 16000)\n",
+    "\n",
+    "  print('Merging your voice with animation...')\n",
+    "  combine_audio('animation.mp4', 'test.mp3', 'merged.mp4')\n",
+    "  print('Done!')\n",
+    "  mp4 = open('merged.mp4','rb').read()\n",
+    "  data_url = \"data:video/mp4;base64,\" + b64encode(mp4).decode()\n",
+    "  display(HTML(\"\"\"\n",
+    "  <video width=800 controls>\n",
+    "        <source src=\"%s\" type=\"video/mp4\">\n",
+    "  </video>\n",
+    "  \"\"\" % data_url))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "IFVs3GvTnpB1"
+   },
+   "source": [
+    "## Record example"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "id": "5EBjrTwiqAaQ"
+   },
+   "outputs": [],
+   "source": [
+    "record_make_animation()"
+   ]
+  }
+ ],
+ "metadata": {
+  "colab": {
+   "collapsed_sections": [
+    "bccAucKjnPHm"
+   ],
+   "name": "Untitled2.ipynb",
+   "provenance": []
+  },
+  "kernelspec": {
+   "display_name": "Python 3",
+   "name": "python3"
+  },
+  "language_info": {
+   "name": "python"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}

hub/snakers4_silero-vad_master/examples/microphone_and_webRTC_integration/README.md

@@ -0,0 +1,28 @@
+
+In this example, an integration with the microphone and the webRTC VAD has been done.  I used [this](https://github.com/mozilla/DeepSpeech-examples/tree/r0.8/mic_vad_streaming) as a draft.
+Here a short video to present the results:
+
+https://user-images.githubusercontent.com/28188499/116685087-182ff100-a9b2-11eb-927d-ed9f621226ee.mp4
+
+# Requirements:
+The libraries used for the following example are:
+```
+Python == 3.6.9
+webrtcvad >= 2.0.10
+torchaudio >= 0.8.1
+torch >= 1.8.1
+halo >= 0.0.31
+Soundfile >= 0.13.3
+```
+Using pip3:
+```
+pip3 install webrtcvad
+pip3 install torchaudio
+pip3 install torch
+pip3 install halo
+pip3 install soundfile
+```
+Moreover, to make the code easier, the default sample_rate is 16KHz without resampling.
+
+This example has been tested on ``` ubuntu 18.04.3 LTS```
+

hub/snakers4_silero-vad_master/examples/microphone_and_webRTC_integration/microphone_and_webRTC_integration.py

@@ -0,0 +1,201 @@
+import collections, queue
+import numpy as np
+import pyaudio
+import webrtcvad
+from halo import Halo
+import torch
+import torchaudio
+
+class Audio(object):
+    """Streams raw audio from microphone. Data is received in a separate thread, and stored in a buffer, to be read from."""
+
+    FORMAT = pyaudio.paInt16
+    # Network/VAD rate-space
+    RATE_PROCESS = 16000
+    CHANNELS = 1
+    BLOCKS_PER_SECOND = 50
+
+    def __init__(self, callback=None, device=None, input_rate=RATE_PROCESS):
+        def proxy_callback(in_data, frame_count, time_info, status):
+            #pylint: disable=unused-argument
+            callback(in_data)
+            return (None, pyaudio.paContinue)
+        if callback is None: callback = lambda in_data: self.buffer_queue.put(in_data)
+        self.buffer_queue = queue.Queue()
+        self.device = device
+        self.input_rate = input_rate
+        self.sample_rate = self.RATE_PROCESS
+        self.block_size = int(self.RATE_PROCESS / float(self.BLOCKS_PER_SECOND))
+        self.block_size_input = int(self.input_rate / float(self.BLOCKS_PER_SECOND))
+        self.pa = pyaudio.PyAudio()
+
+        kwargs = {
+            'format': self.FORMAT,
+            'channels': self.CHANNELS,
+            'rate': self.input_rate,
+            'input': True,
+            'frames_per_buffer': self.block_size_input,
+            'stream_callback': proxy_callback,
+        }
+
+        self.chunk = None
+        # if not default device
+        if self.device:
+            kwargs['input_device_index'] = self.device
+
+        self.stream = self.pa.open(**kwargs)
+        self.stream.start_stream()
+
+    def read(self):
+        """Return a block of audio data, blocking if necessary."""
+        return self.buffer_queue.get()
+
+    def destroy(self):
+        self.stream.stop_stream()
+        self.stream.close()
+        self.pa.terminate()
+
+    frame_duration_ms = property(lambda self: 1000 * self.block_size // self.sample_rate)
+
+
+class VADAudio(Audio):
+    """Filter & segment audio with voice activity detection."""
+
+    def __init__(self, aggressiveness=3, device=None, input_rate=None):
+        super().__init__(device=device, input_rate=input_rate)
+        self.vad = webrtcvad.Vad(aggressiveness)
+
+    def frame_generator(self):
+        """Generator that yields all audio frames from microphone."""
+        if self.input_rate == self.RATE_PROCESS:
+            while True:
+                yield self.read()
+        else:
+            raise Exception("Resampling required")
+
+    def vad_collector(self, padding_ms=300, ratio=0.75, frames=None):
+        """Generator that yields series of consecutive audio frames comprising each utterence, separated by yielding a single None.
+            Determines voice activity by ratio of frames in padding_ms. Uses a buffer to include padding_ms prior to being triggered.
+            Example: (frame, ..., frame, None, frame, ..., frame, None, ...)
+                      |---utterence---|        |---utterence---|
+        """
+        if frames is None: frames = self.frame_generator()
+        num_padding_frames = padding_ms // self.frame_duration_ms
+        ring_buffer = collections.deque(maxlen=num_padding_frames)
+        triggered = False
+
+        for frame in frames:
+            if len(frame) < 640:
+                return
+
+            is_speech = self.vad.is_speech(frame, self.sample_rate)
+
+            if not triggered:
+                ring_buffer.append((frame, is_speech))
+                num_voiced = len([f for f, speech in ring_buffer if speech])
+                if num_voiced > ratio * ring_buffer.maxlen:
+                    triggered = True
+                    for f, s in ring_buffer:
+                        yield f
+                    ring_buffer.clear()
+
+            else:
+                yield frame
+                ring_buffer.append((frame, is_speech))
+                num_unvoiced = len([f for f, speech in ring_buffer if not speech])
+                if num_unvoiced > ratio * ring_buffer.maxlen:
+                    triggered = False
+                    yield None
+                    ring_buffer.clear()
+
+def main(ARGS):
+    # Start audio with VAD
+    vad_audio = VADAudio(aggressiveness=ARGS.webRTC_aggressiveness,
+                         device=ARGS.device,
+                         input_rate=ARGS.rate)
+
+    print("Listening (ctrl-C to exit)...")
+    frames = vad_audio.vad_collector()
+
+    # load silero VAD
+    torchaudio.set_audio_backend("soundfile")
+    model, utils = torch.hub.load(repo_or_dir='snakers4/silero-vad',
+                                    model=ARGS.silaro_model_name,
+                                    force_reload= ARGS.reload)
+    (get_speech_ts,_,_, _,_, _, _) = utils
+
+
+    # Stream from microphone to DeepSpeech using VAD
+    spinner = None
+    if not ARGS.nospinner:
+        spinner = Halo(spinner='line')
+    wav_data = bytearray()
+    for frame in frames:
+        if frame is not None:
+            if spinner: spinner.start()
+
+            wav_data.extend(frame)
+        else:
+            if spinner: spinner.stop()
+            print("webRTC has detected a possible speech")
+
+            newsound= np.frombuffer(wav_data,np.int16)
+            audio_float32=Int2Float(newsound)
+            time_stamps =get_speech_ts(audio_float32, model,num_steps=ARGS.num_steps,trig_sum=ARGS.trig_sum,neg_trig_sum=ARGS.neg_trig_sum,
+                                    num_samples_per_window=ARGS.num_samples_per_window,min_speech_samples=ARGS.min_speech_samples,
+                                    min_silence_samples=ARGS.min_silence_samples)
+
+            if(len(time_stamps)>0):
+                print("silero VAD has detected a possible speech")
+            else:
+                print("silero VAD has detected a noise")
+            print()
+            wav_data = bytearray()
+
+
+def Int2Float(sound):
+    _sound = np.copy(sound)  #
+    abs_max = np.abs(_sound).max()
+    _sound = _sound.astype('float32')
+    if abs_max > 0:
+        _sound *= 1/abs_max
+    audio_float32 = torch.from_numpy(_sound.squeeze())
+    return audio_float32
+
+if __name__ == '__main__':
+    DEFAULT_SAMPLE_RATE = 16000
+
+    import argparse
+    parser = argparse.ArgumentParser(description="Stream from microphone to webRTC and silero VAD")
+
+    parser.add_argument('-v', '--webRTC_aggressiveness', type=int, default=3,
+                        help="Set aggressiveness of webRTC: an integer between 0 and 3, 0 being the least aggressive about filtering out non-speech, 3 the most aggressive. Default: 3")
+    parser.add_argument('--nospinner', action='store_true',
+                        help="Disable spinner")
+    parser.add_argument('-d', '--device', type=int, default=None,
+                        help="Device input index (Int) as listed by pyaudio.PyAudio.get_device_info_by_index(). If not provided, falls back to PyAudio.get_default_device().")
+
+    parser.add_argument('-name', '--silaro_model_name', type=str, default="silero_vad",
+                        help="select the name of the model. You can select between 'silero_vad',''silero_vad_micro','silero_vad_micro_8k','silero_vad_mini','silero_vad_mini_8k'")
+    parser.add_argument('--reload', action='store_true',help="download the last version of the silero vad")
+
+    parser.add_argument('-ts', '--trig_sum', type=float, default=0.25,
+                        help="overlapping windows are used for each audio chunk, trig sum defines average probability among those windows for switching into triggered state (speech state)")
+
+    parser.add_argument('-nts', '--neg_trig_sum', type=float, default=0.07,
+                        help="same as trig_sum, but for switching from triggered to non-triggered state (non-speech)")
+
+    parser.add_argument('-N', '--num_steps', type=int, default=8,
+                        help="nubmer of overlapping windows to split audio chunk into (we recommend 4 or 8)")
+
+    parser.add_argument('-nspw', '--num_samples_per_window', type=int, default=4000,
+                        help="number of samples in each window, our models were trained using 4000 samples (250 ms) per window, so this is preferable value (lesser values reduce quality)")
+
+    parser.add_argument('-msps', '--min_speech_samples', type=int, default=10000,
+                        help="minimum speech chunk duration in samples")
+
+    parser.add_argument('-msis', '--min_silence_samples', type=int, default=500,
+                        help=" minimum silence duration in samples between to separate speech chunks")
+    ARGS = parser.parse_args()
+    ARGS.rate=DEFAULT_SAMPLE_RATE
+    main(ARGS)

hub/snakers4_silero-vad_master/examples/pyaudio-streaming/README.md

@@ -0,0 +1,20 @@
+# Pyaudio Streaming Example
+
+This example notebook shows how micophone audio fetched by pyaudio can be processed with Silero-VAD.
+
+It has been designed as a low-level example for binary real-time streaming using only the prediction of the model, processing the binary data and plotting the speech probabilities at the end to visualize it.
+
+Currently, the notebook consits of two examples:
+ - One that records audio of a predefined length from the microphone, process it with Silero-VAD, and plots it afterwards.
+ - The other one plots the speech probabilities in real-time (using jupyterplot) and records the audio until you press enter.
+
+## Example Video for the Real-Time Visualization
+
+
+https://user-images.githubusercontent.com/8079748/117580455-4622dd00-b0f8-11eb-858d-e6368ed4eada.mp4
+
+
+
+
+
+

hub/snakers4_silero-vad_master/examples/pyaudio-streaming/pyaudio-streaming-examples.ipynb

@@ -0,0 +1,331 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "62a0cccb",
+   "metadata": {},
+   "source": [
+    "# Pyaudio Microphone Streaming Examples\n",
+    "\n",
+    "A simple notebook that uses pyaudio to get the microphone audio and feeds this audio then to Silero VAD.\n",
+    "\n",
+    "I created it as an example on how binary data from a stream could be feed into Silero VAD.\n",
+    "\n",
+    "\n",
+    "Has been tested on Ubuntu 21.04 (x86). After you installed the dependencies below, no additional setup is required."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "64cbe1eb",
+   "metadata": {},
+   "source": [
+    "## Dependencies\n",
+    "The cell below lists all used dependencies and the used versions. Uncomment to install them from within the notebook."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "57bc2aac",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#!pip install numpy==1.20.2\n",
+    "#!pip install torch==1.9.0\n",
+    "#!pip install matplotlib==3.4.2\n",
+    "#!pip install torchaudio==0.9.0\n",
+    "#!pip install soundfile==0.10.3.post1\n",
+    "#!pip install pyaudio==0.2.11"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "110de761",
+   "metadata": {},
+   "source": [
+    "## Imports"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5a647d8d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import io\n",
+    "import numpy as np\n",
+    "import torch\n",
+    "torch.set_num_threads(1)\n",
+    "import torchaudio\n",
+    "import matplotlib\n",
+    "import matplotlib.pylab as plt\n",
+    "torchaudio.set_audio_backend(\"soundfile\")\n",
+    "import pyaudio"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "725d7066",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "model, utils = torch.hub.load(repo_or_dir='snakers4/silero-vad',\n",
+    "                              model='silero_vad',\n",
+    "                              force_reload=True)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "1c0b2ea7",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "(get_speech_timestamps,\n",
+    " save_audio,\n",
+    " read_audio,\n",
+    " VADIterator,\n",
+    " collect_chunks) = utils"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "f9112603",
+   "metadata": {},
+   "source": [
+    "### Helper Methods"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5abc6330",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Taken from utils_vad.py\n",
+    "def validate(model,\n",
+    "             inputs: torch.Tensor):\n",
+    "    with torch.no_grad():\n",
+    "        outs = model(inputs)\n",
+    "    return outs\n",
+    "\n",
+    "# Provided by Alexander Veysov\n",
+    "def int2float(sound):\n",
+    "    abs_max = np.abs(sound).max()\n",
+    "    sound = sound.astype('float32')\n",
+    "    if abs_max > 0:\n",
+    "        sound *= 1/abs_max\n",
+    "    sound = sound.squeeze()  # depends on the use case\n",
+    "    return sound"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "5124095e",
+   "metadata": {},
+   "source": [
+    "## Pyaudio Set-up"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "a845356e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "FORMAT = pyaudio.paInt16\n",
+    "CHANNELS = 1\n",
+    "SAMPLE_RATE = 16000\n",
+    "CHUNK = int(SAMPLE_RATE / 10)\n",
+    "\n",
+    "audio = pyaudio.PyAudio()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0b910c99",
+   "metadata": {},
+   "source": [
+    "## Simple Example\n",
+    "The following example reads the audio as 250ms chunks from the microphone, converts them to a Pytorch Tensor, and gets the probabilities/confidences if the model thinks the frame is voiced."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "9d3d2c10",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "num_samples = 1536"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "3cb44a4a",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "stream = audio.open(format=FORMAT,\n",
+    "                    channels=CHANNELS,\n",
+    "                    rate=SAMPLE_RATE,\n",
+    "                    input=True,\n",
+    "                    frames_per_buffer=CHUNK)\n",
+    "data = []\n",
+    "voiced_confidences = []\n",
+    "\n",
+    "print(\"Started Recording\")\n",
+    "for i in range(0, frames_to_record):\n",
+    "    \n",
+    "    audio_chunk = stream.read(num_samples)\n",
+    "    \n",
+    "    # in case you want to save the audio later\n",
+    "    data.append(audio_chunk)\n",
+    "    \n",
+    "    audio_int16 = np.frombuffer(audio_chunk, np.int16);\n",
+    "\n",
+    "    audio_float32 = int2float(audio_int16)\n",
+    "    \n",
+    "    # get the confidences and add them to the list to plot them later\n",
+    "    new_confidence = model(torch.from_numpy(audio_float32), 16000).item()\n",
+    "    voiced_confidences.append(new_confidence)\n",
+    "    \n",
+    "print(\"Stopped the recording\")\n",
+    "\n",
+    "# plot the confidences for the speech\n",
+    "plt.figure(figsize=(20,6))\n",
+    "plt.plot(voiced_confidences)\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a3dda982",
+   "metadata": {},
+   "source": [
+    "## Real Time Visualization\n",
+    "\n",
+    "As an enhancement to plot the speech probabilities in real time I added the implementation below.\n",
+    "In contrast to the simeple one, it records the audio until to stop the recording by pressing enter.\n",
+    "While looking into good ways to update matplotlib plots in real-time, I found a simple libarary that does the job. https://github.com/lvwerra/jupyterplot It has some limitations, but works for this use case really well.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "05ef4100",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#!pip install jupyterplot==0.0.3"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "d1d4cdd6",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from jupyterplot import ProgressPlot\n",
+    "import threading\n",
+    "\n",
+    "continue_recording = True\n",
+    "\n",
+    "def stop():\n",
+    "    input(\"Press Enter to stop the recording:\")\n",
+    "    global continue_recording\n",
+    "    continue_recording = False\n",
+    "\n",
+    "def start_recording():\n",
+    "    \n",
+    "    stream = audio.open(format=FORMAT,\n",
+    "                    channels=CHANNELS,\n",
+    "                    rate=SAMPLE_RATE,\n",
+    "                    input=True,\n",
+    "                    frames_per_buffer=CHUNK)\n",
+    "\n",
+    "    data = []\n",
+    "    voiced_confidences = []\n",
+    "    \n",
+    "    global continue_recording\n",
+    "    continue_recording = True\n",
+    "    \n",
+    "    pp = ProgressPlot(plot_names=[\"Silero VAD\"],line_names=[\"speech probabilities\"], x_label=\"audio chunks\")\n",
+    "    \n",
+    "    stop_listener = threading.Thread(target=stop)\n",
+    "    stop_listener.start()\n",
+    "\n",
+    "    while continue_recording:\n",
+    "    \n",
+    "        audio_chunk = stream.read(num_samples)\n",
+    "    \n",
+    "        # in case you want to save the audio later\n",
+    "        data.append(audio_chunk)\n",
+    "    \n",
+    "        audio_int16 = np.frombuffer(audio_chunk, np.int16);\n",
+    "\n",
+    "        audio_float32 = int2float(audio_int16)\n",
+    "    \n",
+    "        # get the confidences and add them to the list to plot them later\n",
+    "        new_confidence = model(torch.from_numpy(audio_float32), 16000).item()\n",
+    "        voiced_confidences.append(new_confidence)\n",
+    "    \n",
+    "        pp.update(new_confidence)\n",
+    "\n",
+    "\n",
+    "    pp.finalize()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "1e398009",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "start_recording()"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.10"
+  },
+  "toc": {
+   "base_numbering": 1,
+   "nav_menu": {},
+   "number_sections": true,
+   "sideBar": true,
+   "skip_h1_title": false,
+   "title_cell": "Table of Contents",
+   "title_sidebar": "Contents",
+   "toc_cell": false,
+   "toc_position": {},
+   "toc_section_display": true,
+   "toc_window_display": false
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

hub/snakers4_silero-vad_master/files/lang_dict_95.json

@@ -0,0 +1 @@
+{"59": "mg, Malagasy", "76": "tk, Turkmen", "20": "lb, Luxembourgish, Letzeburgesch", "62": "or, Oriya", "30": "en, English", "26": "oc, Occitan", "69": "no, Norwegian", "77": "sr, Serbian", "90": "bs, Bosnian", "71": "el, Greek, Modern (1453\u2013)", "15": "az, Azerbaijani", "12": "lo, Lao", "85": "zh-HK, Chinese", "79": "cs, Czech", "43": "sv, Swedish", "37": "mn, Mongolian", "32": "fi, Finnish", "51": "tg, Tajik", "46": "am, Amharic", "17": "nn, Norwegian Nynorsk", "40": "ja, Japanese", "8": "it, Italian", "21": "ha, Hausa", "11": "as, Assamese", "29": "fa, Persian", "82": "bn, Bengali", "54": "mk, Macedonian", "31": "sw, Swahili", "45": "vi, Vietnamese", "41": "ur, Urdu", "74": "bo, Tibetan", "4": "hi, Hindi", "86": "mr, Marathi", "3": "fy-NL, Western Frisian", "65": "sk, Slovak", "2": "ln, Lingala", "92": "gl, Galician", "53": "sn, Shona", "87": "su, Sundanese", "35": "tt, Tatar", "93": "kn, Kannada", "6": "yo, Yoruba", "27": "ps, Pashto, Pushto", "34": "hy, Armenian", "25": "pa-IN, Punjabi, Panjabi", "23": "nl, Dutch, Flemish", "48": "th, Thai", "73": "mt, Maltese", "55": "ar, Arabic", "89": "ba, Bashkir", "78": "bg, Bulgarian", "42": "yi, Yiddish", "5": "ru, Russian", "84": "sv-SE, Swedish", "80": "tr, Turkish", "33": "sq, Albanian", "38": "kk, Kazakh", "50": "pl, Polish", "9": "hr, Croatian", "66": "ky, Kirghiz, Kyrgyz", "49": "hu, Hungarian", "10": "si, Sinhala, Sinhalese", "56": "la, Latin", "75": "de, German", "14": "ko, Korean", "22": "id, Indonesian", "47": "sl, Slovenian", "57": "be, Belarusian", "36": "ta, Tamil", "7": "da, Danish", "91": "sd, Sindhi", "28": "et, Estonian", "63": "pt, Portuguese", "60": "ne, Nepali", "94": "zh-TW, Chinese", "18": "zh-CN, Chinese", "88": "rw, Kinyarwanda", "19": "es, Spanish, Castilian", "39": "ht, Haitian, Haitian Creole", "64": "tl, Tagalog", "83": "ms, Malay", "70": "ro, Romanian, Moldavian, Moldovan", "68": "pa, Punjabi, Panjabi", "52": "uz, Uzbek", "58": "km, Central Khmer", "67": "my, Burmese", "0": "fr, French", "24": "af, Afrikaans", "16": "gu, Gujarati", "81": "so, Somali", "13": "uk, Ukrainian", "44": "ca, Catalan, Valencian", "72": "ml, Malayalam", "61": "te, Telugu", "1": "zh, Chinese"}

hub/snakers4_silero-vad_master/files/lang_group_dict_95.json

@@ -0,0 +1 @@
+{"0": ["Afrikaans", "Dutch, Flemish", "Western Frisian"], "1": ["Turkish", "Azerbaijani"], "2": ["Russian", "Slovak", "Ukrainian", "Czech", "Polish", "Belarusian"], "3": ["Bulgarian", "Macedonian", "Serbian", "Croatian", "Bosnian", "Slovenian"], "4": ["Norwegian Nynorsk", "Swedish", "Danish", "Norwegian"], "5": ["English"], "6": ["Finnish", "Estonian"], "7": ["Yiddish", "Luxembourgish, Letzeburgesch", "German"], "8": ["Spanish", "Occitan", "Portuguese", "Catalan, Valencian", "Galician", "Spanish, Castilian", "Italian"], "9": ["Maltese", "Arabic"], "10": ["Marathi"], "11": ["Hindi", "Urdu"], "12": ["Lao", "Thai"], "13": ["Malay", "Indonesian"], "14": ["Romanian, Moldavian, Moldovan"], "15": ["Tagalog"], "16": ["Tajik", "Persian"], "17": ["Kazakh", "Uzbek", "Kirghiz, Kyrgyz"], "18": ["Kinyarwanda"], "19": ["Tatar", "Bashkir"], "20": ["French"], "21": ["Chinese"], "22": ["Lingala"], "23": ["Yoruba"], "24": ["Sinhala, Sinhalese"], "25": ["Assamese"], "26": ["Korean"], "27": ["Gujarati"], "28": ["Hausa"], "29": ["Punjabi, Panjabi"], "30": ["Pashto, Pushto"], "31": ["Swahili"], "32": ["Albanian"], "33": ["Armenian"], "34": ["Mongolian"], "35": ["Tamil"], "36": ["Haitian, Haitian Creole"], "37": ["Japanese"], "38": ["Vietnamese"], "39": ["Amharic"], "40": ["Hungarian"], "41": ["Shona"], "42": ["Latin"], "43": ["Central Khmer"], "44": ["Malagasy"], "45": ["Nepali"], "46": ["Telugu"], "47": ["Oriya"], "48": ["Burmese"], "49": ["Greek, Modern (1453\u2013)"], "50": ["Malayalam"], "51": ["Tibetan"], "52": ["Turkmen"], "53": ["Somali"], "54": ["Bengali"], "55": ["Sundanese"], "56": ["Sindhi"], "57": ["Kannada"]}

hub/snakers4_silero-vad_master/files/silero_vad.jit

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:082e21870cf7722b0c7fa5228eaed579efb6870df81192b79bed3f7bac2f738a
+size 1439299

hub/snakers4_silero-vad_master/files/silero_vad.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a35ebf52fd3ce5f1469b2a36158dba761bc47b973ea3382b3186ca15b1f5af28
+size 1807522

hub/snakers4_silero-vad_master/hubconf.py

@@ -0,0 +1,105 @@
+dependencies = ['torch', 'torchaudio']
+import torch
+import os
+import json
+from utils_vad import (init_jit_model,
+                       get_speech_timestamps,
+                       get_number_ts,
+                       get_language,
+                       get_language_and_group,
+                       save_audio,
+                       read_audio,
+                       VADIterator,
+                       collect_chunks,
+                       drop_chunks,
+                       Validator,
+                       OnnxWrapper)
+
+
+def versiontuple(v):
+    return tuple(map(int, (v.split('+')[0].split("."))))
+
+
+def silero_vad(onnx=False, force_onnx_cpu=False):
+    """Silero Voice Activity Detector
+    Returns a model with a set of utils
+    Please see https://github.com/snakers4/silero-vad for usage examples
+    """
+
+    if not onnx:
+        installed_version = torch.__version__
+        supported_version = '1.12.0'
+        if versiontuple(installed_version) < versiontuple(supported_version):
+            raise Exception(f'Please install torch {supported_version} or greater ({installed_version} installed)')
+
+    model_dir = os.path.join(os.path.dirname(__file__), 'files')
+    if onnx:
+        model = OnnxWrapper(os.path.join(model_dir, 'silero_vad.onnx'))
+    else:
+        model = init_jit_model(os.path.join(model_dir, 'silero_vad.jit'))
+    utils = (get_speech_timestamps,
+             save_audio,
+             read_audio,
+             VADIterator,
+             collect_chunks)
+
+    return model, utils
+
+
+def silero_number_detector(onnx=False, force_onnx_cpu=False):
+    """Silero Number Detector
+    Returns a model with a set of utils
+    Please see https://github.com/snakers4/silero-vad for usage examples
+    """
+    if onnx:
+        url = 'https://models.silero.ai/vad_models/number_detector.onnx'
+    else:
+        url = 'https://models.silero.ai/vad_models/number_detector.jit'
+    model = Validator(url, force_onnx_cpu)
+    utils = (get_number_ts,
+             save_audio,
+             read_audio,
+             collect_chunks,
+             drop_chunks)
+
+    return model, utils
+
+
+def silero_lang_detector(onnx=False, force_onnx_cpu=False):
+    """Silero Language Classifier
+    Returns a model with a set of utils
+    Please see https://github.com/snakers4/silero-vad for usage examples
+    """
+    if onnx:
+        url = 'https://models.silero.ai/vad_models/number_detector.onnx'
+    else:
+        url = 'https://models.silero.ai/vad_models/number_detector.jit'
+    model = Validator(url, force_onnx_cpu)
+    utils = (get_language,
+             read_audio)
+
+    return model, utils
+
+
+def silero_lang_detector_95(onnx=False, force_onnx_cpu=False):
+    """Silero Language Classifier (95 languages)
+    Returns a model with a set of utils
+    Please see https://github.com/snakers4/silero-vad for usage examples
+    """
+
+    if onnx:
+        url = 'https://models.silero.ai/vad_models/lang_classifier_95.onnx'
+    else:
+        url = 'https://models.silero.ai/vad_models/lang_classifier_95.jit'
+    model = Validator(url, force_onnx_cpu)
+
+    model_dir = os.path.join(os.path.dirname(__file__), 'files')
+    with open(os.path.join(model_dir, 'lang_dict_95.json'), 'r') as f:
+        lang_dict = json.load(f)
+
+    with open(os.path.join(model_dir, 'lang_group_dict_95.json'), 'r') as f:
+        lang_group_dict = json.load(f)
+
+    utils = (get_language_and_group, read_audio)
+
+    return model, lang_dict, lang_group_dict, utils

hub/snakers4_silero-vad_master/silero-vad.ipynb

@@ -0,0 +1,445 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "FpMplOCA2Fwp"
+   },
+   "source": [
+    "#VAD"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "heading_collapsed": true,
+    "id": "62A6F_072Fwq"
+   },
+   "source": [
+    "## Install Dependencies"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true,
+    "id": "5w5AkskZ2Fwr"
+   },
+   "outputs": [],
+   "source": [
+    "#@title Install and Import Dependencies\n",
+    "\n",
+    "# this assumes that you have a relevant version of PyTorch installed\n",
+    "!pip install -q torchaudio\n",
+    "\n",
+    "SAMPLING_RATE = 16000\n",
+    "\n",
+    "import torch\n",
+    "torch.set_num_threads(1)\n",
+    "\n",
+    "from IPython.display import Audio\n",
+    "from pprint import pprint\n",
+    "# download example\n",
+    "torch.hub.download_url_to_file('https://models.silero.ai/vad_models/en.wav', 'en_example.wav')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "id": "pSifus5IilRp"
+   },
+   "outputs": [],
+   "source": [
+    "USE_ONNX = False # change this to True if you want to test onnx model\n",
+    "if USE_ONNX:\n",
+    "    !pip install -q onnxruntime\n",
+    "  \n",
+    "model, utils = torch.hub.load(repo_or_dir='snakers4/silero-vad',\n",
+    "                              model='silero_vad',\n",
+    "                              force_reload=True,\n",
+    "                              onnx=USE_ONNX)\n",
+    "\n",
+    "(get_speech_timestamps,\n",
+    " save_audio,\n",
+    " read_audio,\n",
+    " VADIterator,\n",
+    " collect_chunks) = utils"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "fXbbaUO3jsrw"
+   },
+   "source": [
+    "## Full Audio"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "RAfJPb_a-Auj"
+   },
+   "source": [
+    "**Speech timestapms from full audio**"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "id": "aI_eydBPjsrx"
+   },
+   "outputs": [],
+   "source": [
+    "wav = read_audio('en_example.wav', sampling_rate=SAMPLING_RATE)\n",
+    "# get speech timestamps from full audio file\n",
+    "speech_timestamps = get_speech_timestamps(wav, model, sampling_rate=SAMPLING_RATE)\n",
+    "pprint(speech_timestamps)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "id": "OuEobLchjsry"
+   },
+   "outputs": [],
+   "source": [
+    "# merge all speech chunks to one audio\n",
+    "save_audio('only_speech.wav',\n",
+    "           collect_chunks(speech_timestamps, wav), sampling_rate=SAMPLING_RATE) \n",
+    "Audio('only_speech.wav')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "iDKQbVr8jsry"
+   },
+   "source": [
+    "## Stream imitation example"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "id": "q-lql_2Wjsry"
+   },
+   "outputs": [],
+   "source": [
+    "## using VADIterator class\n",
+    "\n",
+    "vad_iterator = VADIterator(model)\n",
+    "wav = read_audio(f'en_example.wav', sampling_rate=SAMPLING_RATE)\n",
+    "\n",
+    "window_size_samples = 1536 # number of samples in a single audio chunk\n",
+    "for i in range(0, len(wav), window_size_samples):\n",
+    "    chunk = wav[i: i+ window_size_samples]\n",
+    "    if len(chunk) < window_size_samples:\n",
+    "      break\n",
+    "    speech_dict = vad_iterator(chunk, return_seconds=True)\n",
+    "    if speech_dict:\n",
+    "        print(speech_dict, end=' ')\n",
+    "vad_iterator.reset_states() # reset model states after each audio"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "id": "BX3UgwwB2Fwv"
+   },
+   "outputs": [],
+   "source": [
+    "## just probabilities\n",
+    "\n",
+    "wav = read_audio('en_example.wav', sampling_rate=SAMPLING_RATE)\n",
+    "speech_probs = []\n",
+    "window_size_samples = 1536\n",
+    "for i in range(0, len(wav), window_size_samples):\n",
+    "    chunk = wav[i: i+ window_size_samples]\n",
+    "    if len(chunk) < window_size_samples:\n",
+    "      break\n",
+    "    speech_prob = model(chunk, SAMPLING_RATE).item()\n",
+    "    speech_probs.append(speech_prob)\n",
+    "vad_iterator.reset_states() # reset model states after each audio\n",
+    "\n",
+    "print(speech_probs[:10]) # first 10 chunks predicts"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "heading_collapsed": true,
+    "id": "36jY0niD2Fww"
+   },
+   "source": [
+    "# Number detector"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "heading_collapsed": true,
+    "hidden": true,
+    "id": "scd1DlS42Fwx"
+   },
+   "source": [
+    "## Install Dependencies"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true,
+    "id": "Kq5gQuYq2Fwx"
+   },
+   "outputs": [],
+   "source": [
+    "#@title Install and Import Dependencies\n",
+    "\n",
+    "# this assumes that you have a relevant version of PyTorch installed\n",
+    "!pip install -q torchaudio\n",
+    "\n",
+    "SAMPLING_RATE = 16000\n",
+    "\n",
+    "import torch\n",
+    "torch.set_num_threads(1)\n",
+    "\n",
+    "from IPython.display import Audio\n",
+    "from pprint import pprint\n",
+    "# download example\n",
+    "torch.hub.download_url_to_file('https://models.silero.ai/vad_models/en_num.wav', 'en_number_example.wav')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "id": "dPwCFHmFycUF"
+   },
+   "outputs": [],
+   "source": [
+    "USE_ONNX = False # change this to True if you want to test onnx model\n",
+    "if USE_ONNX:\n",
+    "    !pip install -q onnxruntime\n",
+    "  \n",
+    "model, utils = torch.hub.load(repo_or_dir='snakers4/silero-vad',\n",
+    "                              model='silero_number_detector',\n",
+    "                              force_reload=True,\n",
+    "                              onnx=USE_ONNX)\n",
+    "\n",
+    "(get_number_ts,\n",
+    " save_audio,\n",
+    " read_audio,\n",
+    " collect_chunks,\n",
+    " drop_chunks) = utils\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "heading_collapsed": true,
+    "hidden": true,
+    "id": "qhPa30ij2Fwy"
+   },
+   "source": [
+    "## Full audio"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true,
+    "id": "EXpau6xq2Fwy"
+   },
+   "outputs": [],
+   "source": [
+    "wav = read_audio('en_number_example.wav', sampling_rate=SAMPLING_RATE)\n",
+    "# get number timestamps from full audio file\n",
+    "number_timestamps = get_number_ts(wav, model)\n",
+    "pprint(number_timestamps)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true,
+    "id": "u-KfXRhZ2Fwy"
+   },
+   "outputs": [],
+   "source": [
+    "# convert ms in timestamps to samples\n",
+    "for timestamp in number_timestamps:\n",
+    "    timestamp['start'] = int(timestamp['start'] * SAMPLING_RATE / 1000)\n",
+    "    timestamp['end'] = int(timestamp['end'] * SAMPLING_RATE / 1000)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true,
+    "id": "iwYEC4aZ2Fwy"
+   },
+   "outputs": [],
+   "source": [
+    "# merge all number chunks to one audio\n",
+    "save_audio('only_numbers.wav',\n",
+    "           collect_chunks(number_timestamps, wav), SAMPLING_RATE) \n",
+    "Audio('only_numbers.wav')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true,
+    "id": "fHaYejX12Fwy"
+   },
+   "outputs": [],
+   "source": [
+    "# drop all number chunks from audio\n",
+    "save_audio('no_numbers.wav',\n",
+    "           drop_chunks(number_timestamps, wav), SAMPLING_RATE) \n",
+    "Audio('no_numbers.wav')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "heading_collapsed": true,
+    "id": "PnKtJKbq2Fwz"
+   },
+   "source": [
+    "# Language detector"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "heading_collapsed": true,
+    "hidden": true,
+    "id": "F5cAmMbP2Fwz"
+   },
+   "source": [
+    "## Install Dependencies"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true,
+    "id": "Zu9D0t6n2Fwz"
+   },
+   "outputs": [],
+   "source": [
+    "#@title Install and Import Dependencies\n",
+    "\n",
+    "# this assumes that you have a relevant version of PyTorch installed\n",
+    "!pip install -q torchaudio\n",
+    "\n",
+    "SAMPLING_RATE = 16000\n",
+    "\n",
+    "import torch\n",
+    "torch.set_num_threads(1)\n",
+    "\n",
+    "from IPython.display import Audio\n",
+    "from pprint import pprint\n",
+    "# download example\n",
+    "torch.hub.download_url_to_file('https://models.silero.ai/vad_models/en.wav', 'en_example.wav')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "id": "JfRKDZiRztFe"
+   },
+   "outputs": [],
+   "source": [
+    "USE_ONNX = False # change this to True if you want to test onnx model\n",
+    "if USE_ONNX:\n",
+    "    !pip install -q onnxruntime\n",
+    "  \n",
+    "model, utils = torch.hub.load(repo_or_dir='snakers4/silero-vad',\n",
+    "                              model='silero_lang_detector',\n",
+    "                              force_reload=True,\n",
+    "                              onnx=USE_ONNX)\n",
+    "\n",
+    "get_language, read_audio = utils"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "heading_collapsed": true,
+    "hidden": true,
+    "id": "iC696eMX2Fwz"
+   },
+   "source": [
+    "## Full audio"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "hidden": true,
+    "id": "c8UYnYBF2Fw0"
+   },
+   "outputs": [],
+   "source": [
+    "wav = read_audio('en_example.wav', sampling_rate=SAMPLING_RATE)\n",
+    "lang = get_language(wav, model)\n",
+    "print(lang)"
+   ]
+  }
+ ],
+ "metadata": {
+  "colab": {
+   "name": "silero-vad.ipynb",
+   "provenance": []
+  },
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.8.8"
+  },
+  "toc": {
+   "base_numbering": 1,
+   "nav_menu": {},
+   "number_sections": true,
+   "sideBar": true,
+   "skip_h1_title": false,
+   "title_cell": "Table of Contents",
+   "title_sidebar": "Contents",
+   "toc_cell": false,
+   "toc_position": {},
+   "toc_section_display": true,
+   "toc_window_display": false
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}

hub/snakers4_silero-vad_master/utils_vad.py

@@ -0,0 +1,488 @@
+import torch
+import torchaudio
+from typing import List
+import torch.nn.functional as F
+import warnings
+
+languages = ['ru', 'en', 'de', 'es']
+
+
+class OnnxWrapper():
+
+    def __init__(self, path, force_onnx_cpu=False):
+        import numpy as np
+        global np
+        import onnxruntime
+        if force_onnx_cpu and 'CPUExecutionProvider' in onnxruntime.get_available_providers():
+            self.session = onnxruntime.InferenceSession(path, providers=['CPUExecutionProvider'])
+        else:
+            self.session = onnxruntime.InferenceSession(path)
+        self.session.intra_op_num_threads = 1
+        self.session.inter_op_num_threads = 1
+
+        self.reset_states()
+        self.sample_rates = [8000, 16000]
+
+    def _validate_input(self, x, sr: int):
+        if x.dim() == 1:
+            x = x.unsqueeze(0)
+        if x.dim() > 2:
+            raise ValueError(f"Too many dimensions for input audio chunk {x.dim()}")
+
+        if sr != 16000 and (sr % 16000 == 0):
+            step = sr // 16000
+            x = x[::step]
+            sr = 16000
+
+        if sr not in self.sample_rates:
+            raise ValueError(f"Supported sampling rates: {self.sample_rates} (or multiply of 16000)")
+
+        if sr / x.shape[1] > 31.25:
+            raise ValueError("Input audio chunk is too short")
+
+        return x, sr
+
+    def reset_states(self, batch_size=1):
+        self._h = np.zeros((2, batch_size, 64)).astype('float32')
+        self._c = np.zeros((2, batch_size, 64)).astype('float32')
+        self._last_sr = 0
+        self._last_batch_size = 0
+
+    def __call__(self, x, sr: int):
+
+        x, sr = self._validate_input(x, sr)
+        batch_size = x.shape[0]
+
+        if not self._last_batch_size:
+            self.reset_states(batch_size)
+        if (self._last_sr) and (self._last_sr != sr):
+            self.reset_states(batch_size)
+        if (self._last_batch_size) and (self._last_batch_size != batch_size):
+            self.reset_states(batch_size)
+
+        if sr in [8000, 16000]:
+            ort_inputs = {'input': x.numpy(), 'h': self._h, 'c': self._c, 'sr': np.array(sr)}
+            ort_outs = self.session.run(None, ort_inputs)
+            out, self._h, self._c = ort_outs
+        else:
+            raise ValueError()
+
+        self._last_sr = sr
+        self._last_batch_size = batch_size
+
+        out = torch.tensor(out)
+        return out
+
+    def audio_forward(self, x, sr: int, num_samples: int = 512):
+        outs = []
+        x, sr = self._validate_input(x, sr)
+
+        if x.shape[1] % num_samples:
+            pad_num = num_samples - (x.shape[1] % num_samples)
+            x = torch.nn.functional.pad(x, (0, pad_num), 'constant', value=0.0)
+
+        self.reset_states(x.shape[0])
+        for i in range(0, x.shape[1], num_samples):
+            wavs_batch = x[:, i:i+num_samples]
+            out_chunk = self.__call__(wavs_batch, sr)
+            outs.append(out_chunk)
+
+        stacked = torch.cat(outs, dim=1)
+        return stacked.cpu()
+
+
+class Validator():
+    def __init__(self, url, force_onnx_cpu):
+        self.onnx = True if url.endswith('.onnx') else False
+        torch.hub.download_url_to_file(url, 'inf.model')
+        if self.onnx:
+            import onnxruntime
+            if force_onnx_cpu and 'CPUExecutionProvider' in onnxruntime.get_available_providers():
+                self.model = onnxruntime.InferenceSession('inf.model', providers=['CPUExecutionProvider'])
+            else:
+                self.model = onnxruntime.InferenceSession('inf.model')
+        else:
+            self.model = init_jit_model(model_path='inf.model')
+
+    def __call__(self, inputs: torch.Tensor):
+        with torch.no_grad():
+            if self.onnx:
+                ort_inputs = {'input': inputs.cpu().numpy()}
+                outs = self.model.run(None, ort_inputs)
+                outs = [torch.Tensor(x) for x in outs]
+            else:
+                outs = self.model(inputs)
+
+        return outs
+
+
+def read_audio(path: str,
+               sampling_rate: int = 16000):
+
+    wav, sr = torchaudio.load(path)
+
+    if wav.size(0) > 1:
+        wav = wav.mean(dim=0, keepdim=True)
+
+    if sr != sampling_rate:
+        transform = torchaudio.transforms.Resample(orig_freq=sr,
+                                                   new_freq=sampling_rate)
+        wav = transform(wav)
+        sr = sampling_rate
+
+    assert sr == sampling_rate
+    return wav.squeeze(0)
+
+
+def save_audio(path: str,
+               tensor: torch.Tensor,
+               sampling_rate: int = 16000):
+    torchaudio.save(path, tensor.unsqueeze(0), sampling_rate)
+
+
+def init_jit_model(model_path: str,
+                   device=torch.device('cpu')):
+    torch.set_grad_enabled(False)
+    model = torch.jit.load(model_path, map_location=device)
+    model.eval()
+    return model
+
+
+def make_visualization(probs, step):
+    import pandas as pd
+    pd.DataFrame({'probs': probs},
+                 index=[x * step for x in range(len(probs))]).plot(figsize=(16, 8),
+                 kind='area', ylim=[0, 1.05], xlim=[0, len(probs) * step],
+                 xlabel='seconds',
+                 ylabel='speech probability',
+                 colormap='tab20')
+
+
+def get_speech_timestamps(audio: torch.Tensor,
+                          model,
+                          threshold: float = 0.5,
+                          sampling_rate: int = 16000,
+                          min_speech_duration_ms: int = 250,
+                          min_silence_duration_ms: int = 100,
+                          window_size_samples: int = 512,
+                          speech_pad_ms: int = 30,
+                          return_seconds: bool = False,
+                          visualize_probs: bool = False):
+
+    """
+    This method is used for splitting long audios into speech chunks using silero VAD
+
+    Parameters
+    ----------
+    audio: torch.Tensor, one dimensional
+        One dimensional float torch.Tensor, other types are casted to torch if possible
+
+    model: preloaded .jit silero VAD model
+
+    threshold: float (default - 0.5)
+        Speech threshold. Silero VAD outputs speech probabilities for each audio chunk, probabilities ABOVE this value are considered as SPEECH.
+        It is better to tune this parameter for each dataset separately, but "lazy" 0.5 is pretty good for most datasets.
+
+    sampling_rate: int (default - 16000)
+        Currently silero VAD models support 8000 and 16000 sample rates
+
+    min_speech_duration_ms: int (default - 250 milliseconds)
+        Final speech chunks shorter min_speech_duration_ms are thrown out
+
+    min_silence_duration_ms: int (default - 100 milliseconds)
+        In the end of each speech chunk wait for min_silence_duration_ms before separating it
+
+    window_size_samples: int (default - 1536 samples)
+        Audio chunks of window_size_samples size are fed to the silero VAD model.
+        WARNING! Silero VAD models were trained using 512, 1024, 1536 samples for 16000 sample rate and 256, 512, 768 samples for 8000 sample rate.
+        Values other than these may affect model perfomance!!
+
+    speech_pad_ms: int (default - 30 milliseconds)
+        Final speech chunks are padded by speech_pad_ms each side
+
+    return_seconds: bool (default - False)
+        whether return timestamps in seconds (default - samples)
+
+    visualize_probs: bool (default - False)
+        whether draw prob hist or not
+
+    Returns
+    ----------
+    speeches: list of dicts
+        list containing ends and beginnings of speech chunks (samples or seconds based on return_seconds)
+    """
+
+    if not torch.is_tensor(audio):
+        try:
+            audio = torch.Tensor(audio)
+        except:
+            raise TypeError("Audio cannot be casted to tensor. Cast it manually")
+
+    if len(audio.shape) > 1:
+        for i in range(len(audio.shape)):  # trying to squeeze empty dimensions
+            audio = audio.squeeze(0)
+        if len(audio.shape) > 1:
+            raise ValueError("More than one dimension in audio. Are you trying to process audio with 2 channels?")
+
+    if sampling_rate > 16000 and (sampling_rate % 16000 == 0):
+        step = sampling_rate // 16000
+        sampling_rate = 16000
+        audio = audio[::step]
+        warnings.warn('Sampling rate is a multiply of 16000, casting to 16000 manually!')
+    else:
+        step = 1
+
+    if sampling_rate == 8000 and window_size_samples > 768:
+        warnings.warn('window_size_samples is too big for 8000 sampling_rate! Better set window_size_samples to 256, 512 or 768 for 8000 sample rate!')
+    if window_size_samples not in [256, 512, 768, 1024, 1536]:
+        warnings.warn('Unusual window_size_samples! Supported window_size_samples:\n - [512, 1024, 1536] for 16000 sampling_rate\n - [256, 512, 768] for 8000 sampling_rate')
+
+    model.reset_states()
+    min_speech_samples = sampling_rate * min_speech_duration_ms / 1000
+    min_silence_samples = sampling_rate * min_silence_duration_ms / 1000
+    speech_pad_samples = sampling_rate * speech_pad_ms / 1000
+
+    audio_length_samples = len(audio)
+
+    speech_probs = []
+    for current_start_sample in range(0, audio_length_samples, window_size_samples):
+        chunk = audio[current_start_sample: current_start_sample + window_size_samples]
+        if len(chunk) < window_size_samples:
+            chunk = torch.nn.functional.pad(chunk, (0, int(window_size_samples - len(chunk))))
+        speech_prob = model(chunk, sampling_rate).item()
+        speech_probs.append(speech_prob)
+
+    triggered = False
+    speeches = []
+    current_speech = {}
+    neg_threshold = threshold - 0.15
+    temp_end = 0
+
+    for i, speech_prob in enumerate(speech_probs):
+        if (speech_prob >= threshold) and temp_end:
+            temp_end = 0
+
+        if (speech_prob >= threshold) and not triggered:
+            triggered = True
+            current_speech['start'] = window_size_samples * i
+            continue
+
+        if (speech_prob < neg_threshold) and triggered:
+            if not temp_end:
+                temp_end = window_size_samples * i
+            if (window_size_samples * i) - temp_end < min_silence_samples:
+                continue
+            else:
+                current_speech['end'] = temp_end
+                if (current_speech['end'] - current_speech['start']) > min_speech_samples:
+                    speeches.append(current_speech)
+                temp_end = 0
+                current_speech = {}
+                triggered = False
+                continue
+
+    if current_speech and (audio_length_samples - current_speech['start']) > min_speech_samples:
+        current_speech['end'] = audio_length_samples
+        speeches.append(current_speech)
+
+    for i, speech in enumerate(speeches):
+        if i == 0:
+            speech['start'] = int(max(0, speech['start'] - speech_pad_samples))
+        if i != len(speeches) - 1:
+            silence_duration = speeches[i+1]['start'] - speech['end']
+            if silence_duration < 2 * speech_pad_samples:
+                speech['end'] += int(silence_duration // 2)
+                speeches[i+1]['start'] = int(max(0, speeches[i+1]['start'] - silence_duration // 2))
+            else:
+                speech['end'] = int(min(audio_length_samples, speech['end'] + speech_pad_samples))
+                speeches[i+1]['start'] = int(max(0, speeches[i+1]['start'] - speech_pad_samples))
+        else:
+            speech['end'] = int(min(audio_length_samples, speech['end'] + speech_pad_samples))
+
+    if return_seconds:
+        for speech_dict in speeches:
+            speech_dict['start'] = round(speech_dict['start'] / sampling_rate, 1)
+            speech_dict['end'] = round(speech_dict['end'] / sampling_rate, 1)
+    elif step > 1:
+        for speech_dict in speeches:
+            speech_dict['start'] *= step
+            speech_dict['end'] *= step
+
+    if visualize_probs:
+        make_visualization(speech_probs, window_size_samples / sampling_rate)
+
+    return speeches
+
+
+def get_number_ts(wav: torch.Tensor,
+                  model,
+                  model_stride=8,
+                  hop_length=160,
+                  sample_rate=16000):
+    wav = torch.unsqueeze(wav, dim=0)
+    perframe_logits = model(wav)[0]
+    perframe_preds = torch.argmax(torch.softmax(perframe_logits, dim=1), dim=1).squeeze()   # (1, num_frames_strided)
+    extended_preds = []
+    for i in perframe_preds:
+        extended_preds.extend([i.item()] * model_stride)
+    # len(extended_preds) is *num_frames_real*; for each frame of audio we know if it has a number in it.
+    triggered = False
+    timings = []
+    cur_timing = {}
+    for i, pred in enumerate(extended_preds):
+        if pred == 1:
+            if not triggered:
+                cur_timing['start'] = int((i * hop_length) / (sample_rate / 1000))
+                triggered = True
+        elif pred == 0:
+            if triggered:
+                cur_timing['end'] = int((i * hop_length) / (sample_rate / 1000))
+                timings.append(cur_timing)
+                cur_timing = {}
+                triggered = False
+    if cur_timing:
+        cur_timing['end'] = int(len(wav) / (sample_rate / 1000))
+        timings.append(cur_timing)
+    return timings
+
+
+def get_language(wav: torch.Tensor,
+                 model):
+    wav = torch.unsqueeze(wav, dim=0)
+    lang_logits = model(wav)[2]
+    lang_pred = torch.argmax(torch.softmax(lang_logits, dim=1), dim=1).item()   # from 0 to len(languages) - 1
+    assert lang_pred < len(languages)
+    return languages[lang_pred]
+
+
+def get_language_and_group(wav: torch.Tensor,
+                           model,
+                           lang_dict: dict,
+                           lang_group_dict: dict,
+                           top_n=1):
+    wav = torch.unsqueeze(wav, dim=0)
+    lang_logits, lang_group_logits = model(wav)
+
+    softm = torch.softmax(lang_logits, dim=1).squeeze()
+    softm_group = torch.softmax(lang_group_logits, dim=1).squeeze()
+
+    srtd = torch.argsort(softm, descending=True)
+    srtd_group = torch.argsort(softm_group, descending=True)
+
+    outs = []
+    outs_group = []
+    for i in range(top_n):
+        prob = round(softm[srtd[i]].item(), 2)
+        prob_group = round(softm_group[srtd_group[i]].item(), 2)
+        outs.append((lang_dict[str(srtd[i].item())], prob))
+        outs_group.append((lang_group_dict[str(srtd_group[i].item())], prob_group))
+
+    return outs, outs_group
+
+
+class VADIterator:
+    def __init__(self,
+                 model,
+                 threshold: float = 0.5,
+                 sampling_rate: int = 16000,
+                 min_silence_duration_ms: int = 100,
+                 speech_pad_ms: int = 30
+                 ):
+
+        """
+        Class for stream imitation
+
+        Parameters
+        ----------
+        model: preloaded .jit silero VAD model
+
+        threshold: float (default - 0.5)
+            Speech threshold. Silero VAD outputs speech probabilities for each audio chunk, probabilities ABOVE this value are considered as SPEECH.
+            It is better to tune this parameter for each dataset separately, but "lazy" 0.5 is pretty good for most datasets.
+
+        sampling_rate: int (default - 16000)
+            Currently silero VAD models support 8000 and 16000 sample rates
+
+        min_silence_duration_ms: int (default - 100 milliseconds)
+            In the end of each speech chunk wait for min_silence_duration_ms before separating it
+
+        speech_pad_ms: int (default - 30 milliseconds)
+            Final speech chunks are padded by speech_pad_ms each side
+        """
+
+        self.model = model
+        self.threshold = threshold
+        self.sampling_rate = sampling_rate
+
+        if sampling_rate not in [8000, 16000]:
+            raise ValueError('VADIterator does not support sampling rates other than [8000, 16000]')
+
+        self.min_silence_samples = sampling_rate * min_silence_duration_ms / 1000
+        self.speech_pad_samples = sampling_rate * speech_pad_ms / 1000
+        self.reset_states()
+
+    def reset_states(self):
+
+        self.model.reset_states()
+        self.triggered = False
+        self.temp_end = 0
+        self.current_sample = 0
+
+    def __call__(self, x, return_seconds=False):
+        """
+        x: torch.Tensor
+            audio chunk (see examples in repo)
+
+        return_seconds: bool (default - False)
+            whether return timestamps in seconds (default - samples)
+        """
+
+        if not torch.is_tensor(x):
+            try:
+                x = torch.Tensor(x)
+            except:
+                raise TypeError("Audio cannot be casted to tensor. Cast it manually")
+
+        window_size_samples = len(x[0]) if x.dim() == 2 else len(x)
+        self.current_sample += window_size_samples
+
+        speech_prob = self.model(x, self.sampling_rate).item()
+
+        if (speech_prob >= self.threshold) and self.temp_end:
+            self.temp_end = 0
+
+        if (speech_prob >= self.threshold) and not self.triggered:
+            self.triggered = True
+            speech_start = self.current_sample - self.speech_pad_samples
+            return {'start': int(speech_start) if not return_seconds else round(speech_start / self.sampling_rate, 1)}
+
+        if (speech_prob < self.threshold - 0.15) and self.triggered:
+            if not self.temp_end:
+                self.temp_end = self.current_sample
+            if self.current_sample - self.temp_end < self.min_silence_samples:
+                return None
+            else:
+                speech_end = self.temp_end + self.speech_pad_samples
+                self.temp_end = 0
+                self.triggered = False
+                return {'end': int(speech_end) if not return_seconds else round(speech_end / self.sampling_rate, 1)}
+
+        return None
+
+
+def collect_chunks(tss: List[dict],
+                   wav: torch.Tensor):
+    chunks = []
+    for i in tss:
+        chunks.append(wav[i['start']: i['end']])
+    return torch.cat(chunks)
+
+
+def drop_chunks(tss: List[dict],
+                wav: torch.Tensor):
+    chunks = []
+    cur_start = 0
+    for i in tss:
+        chunks.append((wav[cur_start: i['start']]))
+        cur_start = i['end']
+    return torch.cat(chunks)