Error while doing diarization
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

I am following the ASR with speaker diarization on some wav files. It fails on some files giving the following error.




_LinAlgError
----> 1 diar_hyp, diar_score = asr_diar_offline.run_diarization(cfg, word_ts_hyp)

File /opt/conda/lib/python3.8/site-packages/nemo/collections/asr/parts/utils/diarization_utils.py:238, in ASR_DIAR_OFFLINE.run_diarization(self, diar_model_config, word_timestamps)
    235     diar_model_config.diarizer.vad.external_vad_manifest = oracle_manifest
    237 diar_model = ClusteringDiarizer(cfg=diar_model_config)
--> 238 score = diar_model.diarize()
    239 if diar_model_config.diarizer.vad.model_path is not None and not diar_model_config.diarizer.oracle_vad:
    240     self.get_frame_level_VAD(vad_processing_dir=diar_model.vad_pred_dir)

File /opt/conda/lib/python3.8/site-packages/nemo/collections/asr/models/clustering_diarizer.py:424, in ClusteringDiarizer.diarize(self, paths2audio_files, batch_size)
    419 embs_and_timestamps = get_embs_and_timestamps(
    420     self.multiscale_embeddings_and_timestamps, self.multiscale_args_dict
    421 )
    423 # Clustering
--> 424 all_reference, all_hypothesis = perform_clustering(
    425     embs_and_timestamps=embs_and_timestamps,
    426     AUDIO_RTTM_MAP=self.AUDIO_RTTM_MAP,
    427     out_rttm_dir=out_rttm_dir,
    428     clustering_params=self._cluster_params,
    429 )
    431 # TODO Resegmentation -> Coming Soon
    432 
    433 # Scoring
    434 score = score_labels(
    435     self.AUDIO_RTTM_MAP,
    436     all_reference,
   (...)
    439     ignore_overlap=self._diarizer_params.ignore_overlap,
    440 )

File /opt/conda/lib/python3.8/site-packages/nemo/collections/asr/parts/utils/speaker_utils.py:346, in perform_clustering(embs_and_timestamps, AUDIO_RTTM_MAP, out_rttm_dir, clustering_params)
    343 else:
    344     num_speakers = None
--> 346 cluster_labels = COSclustering(
    347     uniq_embs_and_timestamps=embs_and_timestamps[uniq_id],
    348     oracle_num_speakers=num_speakers,
    349     max_num_speaker=max_num_speakers,
    350     enhanced_count_thres=clustering_params.enhanced_count_thres,
    351     max_rp_threshold=clustering_params.max_rp_threshold,
    352     sparse_search_volume=clustering_params.sparse_search_volume,
    353     cuda=cuda,
    354 )
    356 base_scale_idx = max(embs_and_timestamps[uniq_id]['scale_dict'].keys())
    357 lines = embs_and_timestamps[uniq_id]['scale_dict'][base_scale_idx]['time_stamps']

File /opt/conda/lib/python3.8/site-packages/nemo/collections/asr/parts/utils/nmesc_clustering.py:663, in COSclustering(uniq_embs_and_timestamps, oracle_num_speakers, max_num_speaker, min_samples_for_NMESC, enhanced_count_thres, max_rp_threshold, sparse_search_volume, fixed_thres, cuda)
    651 nmesc = NMESC(
    652     mat,
    653     max_num_speaker=max_num_speaker,
   (...)
    659     cuda=cuda,
    660 )
    662 if emb.shape[0] > min_samples_for_NMESC:
--> 663     est_num_of_spk, p_hat_value = nmesc.NMEanalysis()
    664     affinity_mat = getAffinityGraphMat(mat, p_hat_value)
    665 else:

File /opt/conda/lib/python3.8/site-packages/nemo/collections/asr/parts/utils/nmesc_clustering.py:486, in NMESC.NMEanalysis(self)
    484 self.p_value_list = self.getPvalueList()
    485 for p_value in self.p_value_list:
--> 486     est_num_of_spk, g_p = self.getEigRatio(p_value)
    487     est_spk_n_dict[p_value] = est_num_of_spk
    488     eig_ratio_list.append(g_p)

File /opt/conda/lib/python3.8/site-packages/nemo/collections/asr/parts/utils/nmesc_clustering.py:550, in NMESC.getEigRatio(self, p_neighbors)
    530 """
    531 For a given p_neighbors value,
    532 calculates g_p, which is a ratio
   (...)
    546         The ratio between p_neighbors value and the maximum eigen gap value.
    547 """
    549 affinity_mat = getAffinityGraphMat(self.mat, p_neighbors)
--> 550 est_num_of_spk, lambdas, lambda_gap_list = estimateNumofSpeakers(affinity_mat, self.max_num_speaker, self.cuda)
    551 arg_sorted_idx = np.argsort(lambda_gap_list[: self.max_num_speaker])[::-1]
    552 max_key = arg_sorted_idx[0]

File /opt/conda/lib/python3.8/site-packages/nemo/collections/asr/parts/utils/nmesc_clustering.py:329, in estimateNumofSpeakers(affinity_mat, max_num_speaker, is_cuda)
    319 """
    320 Estimate the number of speakers using eigen decompose on laplacian Matrix.
    321 affinity_mat: (array)
   (...)
    326     if cuda availble eigh decomposition would be computed on GPUs
    327 """
    328 laplacian = getLaplacian(affinity_mat)
--> 329 lambdas, _ = eigDecompose(laplacian, is_cuda)
    330 lambdas = np.sort(lambdas)
    331 lambda_gap_list = getLamdaGaplist(lambdas)

File /opt/conda/lib/python3.8/site-packages/nemo/collections/asr/parts/utils/nmesc_clustering.py:304, in eigDecompose(laplacian, cuda, device)
    302 else:
    303     laplacian = torch.from_numpy(laplacian).float()
--> 304 lambdas, diffusion_map = eigh(laplacian)
    305 lambdas = lambdas.cpu().numpy()
    306 diffusion_map = diffusion_map.cpu().numpy()

_LinAlgError: torch.linalg.eigh: The algorithm failed to converge because the input matrix is ill-conditioned or has too many repeated eigenvalues (error code: 566).

**Steps/Code to reproduce bug**
Following this notebook exactly : https://github.com/NVIDIA/NeMo/blob/main/tutorials/speaker_tasks/ASR_with_SpeakerDiarization.ipynb


**Expected behavior**

Expected to run diarization with ASR successfully


**Environment overview (please complete the following information)**

 - Environment location: Docker
 - Method of NeMo install: RUN pip install nemo_toolkit[asr]
 - If method of install is [Docker], provide `docker pull` & `docker run` commands used

**Environment details**
Using nvidia Image : FROM nvcr.io/nvidia/pytorch:21.04-py3

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
- OS version
- PyTorch version
- Python version

**Additional context**

Add any other context about the problem here.
Example: GPU model