CPU usage is very high and can't seem to fix

username: Brian_Bakerman
cooked: <p>I’ve been running a workflow that is both Whisper and Pyannote.audio diarization. I’ve been running into an issue where CPU usage keeps spiking, particularly with the diarization model. I have tried to combat this issue with torch.set_num_threads(1) and other settings like OMP_NUM_THREADS, but that has either been ineffective, or spiked memory.</p>
<p>Here is an end to end repro of the workflow I’ve been running:</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://github.com/bakermanbrian/gcpwhisppyan">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="32" height="32">

      <a href="https://github.com/bakermanbrian/gcpwhisppyan" target="_blank" rel="noopener nofollow ugc">GitHub</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/345;"><img src="https://opengraph.githubassets.com/8d0e4b8c73aa29db4ceeed2f8593ed14f5c548dcbf92b5224890b1d175c2c71c/bakermanbrian/gcpwhisppyan" class="thumbnail" width="690" height="345"></div>

<h3><a href="https://github.com/bakermanbrian/gcpwhisppyan" target="_blank" rel="noopener nofollow ugc">GitHub - bakermanbrian/gcpwhisppyan</a></h3>

  <p>Contribute to bakermanbrian/gcpwhisppyan development by creating an account on GitHub.</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>Minimum example:</p>
<pre><code class="lang-auto">import whisper
import os
from stable_whisper import modify_model
from pyannote.audio import Pipeline
import time
import torch

torch.set_num_threads(1)
model = whisper.load_model("base")
modify_model(model)


diarizationModel = Pipeline.from_pretrained("pyannote/speaker-diarization",
	                                    use_auth_token="&lt;AUTH TOKEN&gt;")
diarizationModel=diarizationModel.to(0)

def transcription(filePath):
	modelRes=model.transcribe(filePath).to_dict()
	res = {"text": modelRes["text"], "language": modelRes["language"]}
	toAdd=[]
	diarizationResult=diarizationModel(filePath)
	for turn, _, speaker in diarizationResult.itertracks(yield_label=True):
		toAdd.append({"startTime": turn.start, "stopTime": turn.end, "speaker": speaker})
	res["diarization"]=toAdd
        return res
    
print(transcription(""))
</code></pre>
<p>Requirements.txt</p>
<pre><code class="lang-auto">asteroid-filterbanks &gt;=0.4
einops &gt;=0.6.0
huggingface_hub &gt;= 0.13.0
lightning &gt;= 2.0.1
omegaconf &gt;=2.1,&lt;3.0
pyannote.core &gt;= 5.0.0
pyannote.database &gt;= 5.0.1
pyannote.metrics &gt;= 3.2
pyannote.pipeline &gt;= 2.3   # 2.4
pytorch_metric_learning &gt;= 2.1.0
rich &gt;= 12.0.0
semver &gt;= 3.0.0
soundfile &gt;= 0.12.1
speechbrain &gt;= 0.5.14
tensorboardX &gt;= 2.6
torch &gt;= 2.0.0
torch_audiomentations &gt;= 0.11.0
torchaudio &gt;= 2.0.0
torchmetrics &gt;= 0.11.0

#replaced torch with one above
numba
numpy
tqdm
more-itertools
tiktoken==0.3.3


fastapi
uvicorn
librosa
python-multipart
audioread
gunicorn
</code></pre>
<p>Docker file for the env</p>
<pre><code class="lang-auto">FROM nvidia/cuda:12.0.0-runtime-ubuntu22.04

EXPOSE 8000
RUN apt-get update &amp;&amp; apt-get install -y --no-install-recommends \
    python3.9 \
    python3-pip \
    &amp;&amp; \
    apt-get clean &amp;&amp; \
    rm -rf /var/lib/apt/lists/*

WORKDIR /
COPY . .
RUN apt-get update
RUN apt-get install --assume-yes git
RUN apt update &amp;&amp; apt install -y ffmpeg
RUN pip install --upgrade pip
RUN pip install git+https://github.com/openai/whisper.git

RUN pip install git+https://github.com/jianfch/stable-ts.git

RUN pip install -qq https://github.com/pyannote/pyannote-audio/archive/refs/heads/develop.zip

RUN pip install -r requirements.txt
</code></pre>