Title: The Omni-Expert: A Computationally Efficient Approach to Achieve a Mixture of Experts in a Single Expert Model
Venue: NeurIPS 2025
Accepted: yes
Keywords: speech enhancement

mixture-of-experts

omni-expert

feature transformation

cochlear implants
Abstract: Mixture-of-Experts (MoE) models have become popular in machine learning, boosting performance by partitioning tasks across multiple experts. However, the need for several experts often results in high computational costs, limiting their application on resource-constrained devices with stringent real-time requirements, such as cochlear implants (CIs). We introduce the Omni-Expert (OE) – a simple and efficient solution that leverages feature transformations to achieve the 'divide-and-conquer' functionality of a full MoE ensemble in a single expert model. We demonstrate the effectiveness of the OE using phoneme-specific time-frequency masking for speech dereverberation in a CI. Empirical results show that the OE delivers statistically significant improvements in objective intelligibility measures of CI vocoded speech at different levels of reverberation across various speech datasets at a much reduced computational cost relative to a counterpart MoE.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis submission provided a new omni-expert (OE) model, a more efficient alternative to the mixture-of-experts model (MoE) by creating a subtask-specific feature transformation block so that all the features are fed to a single model. The OE model is evaluated against MOE model in the context of cochlear implant (CI), where multiple tasks are performed. The OE model is found to have a better performance and is more computationally effective.\n\nSTRENGTHS AND WEAKNESSES:\nQuality: The submission presents an OE model architecture, and its performance is rigorously compared to the MOE model variant in the context of phoneme prediction of CI. However, in some of the results, i.e. table 2, where the OE did not show any performance improvement in terms of the STOI metric, or in figure 4, where both the MOE and OE models show minimal improvements to the phoneme independent (PI) model. The submission could benefit from the authors justifying that the minimal improvement is meaningful and discussing the result more, where STOI is not improved by using OE compared to MOE.\n\nClarity: The article is well-written and organized, although some of the specific terms could use a bit more explanation to be clearer for people outside the field of audio processing field, such as the difference between a phenome-known model and a phenome-predicted model. \n\nSignificance: The submission is proposing a new model for CI data processing that is less computationally expensive and performs better. It can be very beneficial for CI manufacturing. \n\nOriginality: This submission proposes a new OE model that attempts to improve upon MOE by using a feature transformation block. However, existing work exists that attempts to reduce the computational cost of MOE systems, albeit using a different strategy. (See the paper titled \"Merging Experts into One: Improving Computational Efficiency of Mixture of Experts\") The authors should mention previous work and consider explaining why this new strategy could be a better approach.\n\nQUESTIONS:\nWhat are the reasons that the MOE and OE models perform similarly in terms of the STOI metric?\n\nAre the relatively small improvements from MOE and OE models meaningful from using a PI model?\n\nHow would other strategies of improving MOE perform in this task, such as merging the expert models (i.e. from existing work \"Merging Experts into One: Improving Computational Efficiency of Mixture of Experts\")?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a speech dereverberation framework targeting Cochlear Implant(CI) device. It attempts to improve upon an existing MoE approach by proposing a single expert model and a phoneme-based feature transformation block. The feature transformation block learns a scale-shift operation to transform the input speech features based on the predicted phoneme label and the expert model works on these features to estimate the masks for speech dereverberation task.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\nThe paper is well motivated and easy to follow. \nThe paper targets a very important application of Cochlear Implant for hearing impaired individuals.\nSubstantial memory and computational complexity reduction compared to baselines. \nAblation experiments are performed to establish understanding of the proposed method. \n\nMajor Weaknesses:\nPerformance improvement is minor from provided baselines specially in more realistic noisy conditions. The much simpler PI model outperforms the proposed method in most of the SNR conditions (Table A7.1). \nLimited performance comparisons. Comparisons with existing dereverberation baselines such as [1], [2] not provided. \nSince the model is built for practical purposes,I feel a user subjective quality test under real conditions is necessary. \n\n\n[1] Zhao et al.  A deep learning based segregation algorithm to increase speech intelligibility for hearing-impaired listeners in reverberant-noisy conditions. J Acoust Soc Am. 2018 Sep\n[2] Healy et al. A deep learning algorithm to increase intelligibility for hearing-impaired listeners in the presence of a competing talker and reverberation. J Acoust Soc Am. 2019 Mar\n\nMinor weaknesses:\nNo demo samples are provided. In the absence of subjective quality tests from human users, I would have appreciated some samples of the model's produced output and baselines to better judge the performance.\n\nQUESTIONS:\nI have concerns about the proposed scale-shift feature transformation. As I expected, the shift operation would try to learn shift vectors for each phoneme so that the transformed feature clusters are pushed far apart and it gets easier for the model to reduce the training loss (Fig. 5). I suspect this will lead to poor generalization as the clusters grow apart with enough training. Was there any constraint/regularization on the shift vectors? Would it also help to use some kind of contrastive/regularization loss to keep the clusters close by and to maintain continuity in the feature space?\n\nIs the phoneme classifier well trained? Why is the phoneme classification accuracy (Table 1) so low? In the HINT dataset, for some RIRs, accuracy is lower than 30% (whereas the phoneme /sil/ itself is around 31% of the test data). \n\nIs the Fig A6.1.1correct? Are the shift and scale images swapped? It’s counter intuitive to me  that shift is not able to change the cluster locations but scale alone can.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a variant or alternative to mixture of experts (MoEs) but constrain their analysis to dereverbration of audio signals for cochlear implants and the associated compute and memory constraints. The authors address dereverberation by estimating a time frequency mask which then gets multiplied with the spectrogram representation. The authors propose to have task-specific scale and shift operations and a shared backbone model instead of multiple experts. This idea is validated by comparing first a single model against MoE and then against the proposed task-specific feature transformation with a single model. Experiments are thorough and contain intelligibility metrics obtained after synthesis from the cochlear implant output electrode signals.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper title suggests an alternative to MoEs widely applicable and studied on various domains. However, the abstract then disappoints a little by reducing the scope to audio, to be precise, dereverberation, to be precise, under limited compute, to be precise, on cochlear implants. The contribution is meaningful nonetheless but the title should be toned down or contain \"dereverberation\".\n\nThe experimental validation is very well done. Laudable that you use recorded room impulse responses (RIRs). Even more so that you take test RIRs from a different database than the train RIRs. Good experimental study with three different oracle conditions, good use of direct path signal to get, e.g., alignments and oracle. Often enough I had seen papers where people use the source signal as reference instead of direct path signal (and then wonder why timing is not right).\n\nThe references for alternatives to MoEs are a bit sparse. I suggest the conditional scale and shift parameters should cite https://arxiv.org/abs/1707.00683, in particular conditional batch normalization in Section 3. Also https://www.ecva.net/papers/eccv_2022/papers_ECCV/papers/136870299.pdf is a paper in which the backbone is shared and then only task specific (output) branches are used to make this behave like MoE.\n\nQUESTIONS:\nPlease clarify if the scale matrix A_k is a dense matrix or just a diagonal matrix. Was not fully clear.\n\nPlease clarify why you need a network to predict A_k and b_k from a one-hot vector. It seems wasteful. If you have one hot input and an MLP, the MLP is not required. You can just look up the end result because there are anyway only 40 possible vectors. (line 177). A frequently used term for this is \"embedding layer\".\n\nFigure 2: Why are true labels with probabilities 0<p<1 instead of either 0 or 1?\n\nConfidence intervals in Table3/4 are odd. Why are they symmetric? Bootstrap-based confidence intervals should be shown as separate columns or smaller numbers but not relative with +/-. Further, are the confidence intervals characterizing the test example variance or the variance between multiple repeated training runs?\n\nLIMITATIONS:\nThe biggest limitation of this work is the evaluation on such a niche application. The reader can hardly infer if this Omni-Expert approach applies to their domains and can replace their expensive MoE.\n\nIt is also highly likely that the single model capacity and consequentially the MoE model capacity is just much too high for this application. Thus, compressing it into only task-specific feature transformation works. Imagine a more complex task like language modeling where really huge MoEs are used. Given your experimental results, would you advise them to exchange their MoEs with Omni-Expert?", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-10"}