<figure><img src="image_1.png" /></figure>

HAL
open science

# Zita-at2 Autotune

Fons Adriaensen

► To cite this version:

Fons Adriaensen. Zita-at2 Autotune. Proceedings of the 19th Linux Audio Conference, Jun 2025, Lyon, France. hal-05096064v2

HAL Id: hal-05096064

https://hal.science/hal-05096064v2

Submitted on 16 Jun 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# ZITA-AT2 AUTOTUNE

Fons Adriaensen

Zita Audio

fons@linuxaudio.org

## ABSTRACT

The *zita-at1* autotuner (by the author) was first released in 2011 as a Jack application, and it became more popular as one of the x42 LV2 plugins created by Robin Gareus. While it works reasonably well, its main drawback has been that it doesn't preserve the formant spectrum when changing the pitch of a voice signal. The new *zita-at2* to be launched at LAC 2025, should fix this issue. It does this by using the PSOLA algorithm instead of just resampling. This paper documents the main design of and the algorithms used in this application.

## 1. INTRODUCTION

Autotuning has become one of the standard processes used in the production of popular music ever since it was introduced by Antares Audio Technologies in 1997. It can be used to subtly correct a singer's small pitch errors, or as an effect of its own, creating a sort of robotic voice that has such perfect pitch that it no longer appears natural. In particular the latter use has become a matter of controversy and some people <fnref n="1" /> really hate it.

Developing a usable autotune application is of course a challenge to anyone interested in audio DSP, and that's why the author wrote *zita-at1* many years ago, and decided to have a fresh look at it more recently.

## 2. THE NATURE OF VOICE SIGNALS

The singing or speaking human voice waveform can be of two types: *voiced* or *unvoiced*.

A *voiced* signal is a periodic impulsive waveform generated by the vocal cords and filtered by the upper respiratory tract, the larynx, the pharynx and the oral and nasal cavities. Voiced signals include all vowels and also sounds such as a sustained "mmm".

The *pitch* is the fundamental frequency of the impulsive waveform, controlled by the tension of the vocal cords. The pitch frequency range for humans depends on the type of voice: approx. 80 to 330 Hz (E2 to E4) for a male bass singer and 250 to 1050 Hz (C4 to C6) for a soprano.

The waveform generated by the vibrating vocal cords is rich in higher harmonics, and the filtering by the vocal tract produces a spectrum that has a number of distinct peaks (at least two) called *formants*. The actual formant frequencies are typical for each vowel, and while they are slightly different between male and female voices they do not depend on the voice pitch. The lower formant is usually in the range from 200 to 900 Hz, the upper ones from 500 to 3500 Hz. Trained singers tend to have more pronounced higher formants.

An *unvoiced* signal is one that is produced without vibrating vocal cords. It doesn't have any defined pitch and consists of filtered noise, mostly produced by air flowing around the tongue and teeth and through the lips, or for whispered vowels, deeper down the vocal tract. Although they can be sustained, in normal speech and singing unvoiced sounds tend to be mostly transient.

## 3. THE ALGORITHMS

Three steps are required to modify the pitch of a voice in a controlled way. These will be the subject of the following sections.

### 3.1. Pitch Detection

The first task is to separate the voice signal into voiced and unvoiced parts, and to determine the exact pitch of the voiced ones.

Many methods have been developed to do this, see e.g. [1], [2] or [3]. Among the most practical ones is to compute the *autocorrelation* of a short fragment of the input signal, and this is what is actually used in *zita-at2*.

The autocorrelation function indicates how similar a signal is to a time-shifted version of itself. For a periodic waveform it will have a maximum value for shifts equal to an integer multiple the waveform period. For unvoiced input, which will mainly be filtered wideband noise, its value will drop down rapidly even for small time shifts.

The autocorrelation of a real valued fragment $X[i]$ is defined as $R[k] = \sum_{i} X[i]X[i-k]$. Dividing this by $R[0]$, which is just the total energy of the fragment, makes the result independent of signal amplitude.

The most efficient way to compute $R[k]$ is not by using the time domain definition above, but as the inverse FFT of the power spectrum of $X$, itself computed by a forward FFT. Some care is required when using this method as the FFT operations interpret time in a cyclic way – for an N-point FFT a shift of $k$ samples is equivalent to one of $N - k$ samples. So for values of $k$ approaching $N/2$ two smooth peaks will overlap which will offset the apparent peak position. To avoid this a time domain window with a length of twice the longest expected period should be applied before the FFT, and the FFT size should be somewhat larger than this window.

A bandpass filter (80 Hz to 5 kHz) is applied before computing the autocorrelation, anything outside this range doesn't contribute to a correct result. Given $R[k]$, the basic algorithm is to find the highest peak which also exceeds a minimal value. The exact position and magnitude of the correlation peaks is obtained by fitting a parabola to a small region around the maximum value.

There are some corner cases, mainly for signals having a single very strong harmonic. Figure 1 shows such a 'near miss'. This is for a synthetic signal with a frequency of 120 Hz (400 samples at 48 kHz sample rate), and a third harmonic 25 dB stronger than the fundamental. The first two peaks of the autocorrelation function are due to this strong third harmonic. The third one, which is only marginally stronger, corresponds to the actual signal period.

This does happen also for real-life signals, but usually in those cases there will be more than one strong harmonic. This results in a much less ambiguous correlation, see Fig. 2 for an example.

Most errors occur at the onset of a voiced fragment before it fills the entire window. The *zita-at2* autotuner computes the

<footnote n="1">¹including the author</footnote>
<figure><img src="image_3.png" /><figcaption>Figure 1: Ambiguous autocorrelation of a waveform with a single strong harmonic. Top left: waveform and window, top right: spectrum, bottom: autocorrelation of windowed waveform, X-axis in samples. The blue dot at 400 samples indicates the estimated pitch period</figcaption></figure>

autocorrelation every 5 milliseconds. This is more than strictly necessary, but allows the inclusion of some logic that helps to reject wrong estimates. Also allowing the user to select the expected pitch range and using a smaller window size for a high (female) voice helps to reduce false estimates.

<figure><img src="image_4.png" /><figcaption>Figure 2: A more complex spectrum removes the ambiguity. Top left: waveform and window, top right: spectrum, bottom: autocorrelation of windowed waveform, X-axis in samples.</figcaption></figure>

## 3.2. Computing the Corrected Pitch

The correct retuned pitch will usually be one of the notes of a 12-note per octave scale.<fnref n="2" /> The default scale is the equal temperature one, but others can be used by importing a Scala<fnref n="3" /> file. Also the user can set the frequency (normally 440 Hz) corresponding to A4 or MIDI note 69.

In *Auto Mode* the set of allowed notes is selected using the one octave virtual keyboard in the GUI. The enabled note nearest to the detected pitch is selected regardless of in which octave it is, so the pitch change will never be more than half an octave up or down. The *Bias* control allows to add some preference for

the current note, allowing the pitch to go off-tune more before a different one is selected.

<figure><img src="image_5.png" /><figcaption>Figure 3: Real life example. X-axis: time (s). Top Y: signal level (dB). Bottom Y: MIDI note number, blue: estimated pitch, red: correct pitch.</figcaption></figure>

Auto mode can work well for a trained singer who has good pitch control to start with, but it has its limitations. Figure 3 shows an example of what can be seen in real life. Here the correct melody is [Bb, F, D], [Bb, C, Bb, A, F#, D], as shown by the red lines. The blue line is the measured pitch. Note how the second Bb starts almost an octave below the correct pitch, and how this singer has a general tendency to slide up towards a note as it starts and to drop the pitch at the end. Strange enough all this doesn't sound as bad as it looks, but it will if the sliding pitch is quantised. Auto mode will not handle such cases very well.

In *MIDI mode* the allowed note selection is taken from MIDI input. This can be used 'live' by playing along, but it is most useful when the data is coming from a MIDI track in a DAW such as Ardour.<fnref n="4" /> MIDI mode can be used in two ways. If enabled together with Auto mode, any MIDI input takes precedence over the GUI, and the nearest active note, modulo octaves, will be selected. When used on its own, the *last* played MIDI note will determine the output pitch. In this case the absolute pitch (not ignoring octaves) is used. Also any pitch bend input will be taken into account. When there is no active MIDI note, the input will be considered *unvoiced* and just copied to the output.

A few more controls will affect the final corrected pitch. These mainly allow a choice between the very obvious autotune effect and a more natural sounding result The *Speed* control sets how fast the quantised pitch will slide from one note to the next, this is similar to portamento in a synth. The *Corr* control determines how much of the pitch error is corrected. This also has a *Var* option. When *Var* is on, only the variation of the error is used while any constant part is removed. The result is that the average pitch will be correct while at the same time any expression (e.g. vibrato) is passed through.

Finally there are two controls to add a constant pitch offset (± one octave), and a constant formant shift (± a third).

## 3.3. Generating the Retuned Signal

### 3.3.1. The PSOLA Algorithm

In *zita-at1* the retuning was done by resampling the original signal, skipping forward or back by an integer number of periods to

<footnote n="2">²Currently *zita-at2* only supports 12-note scales, but others could be added in later releases.</footnote>

<footnote n="3">³Scala is a program to define and analyse musical scales. See https://www.huygens-fokker.org/scala</footnote>

<footnote n="4">⁴A nice YouTube video demonstrates this using the LV2 version of *zita-at1*. See https://www.youtube.com/watch?v=ttXCsIb_2L8</footnote>
preserve the length. This does not only shift the pitch, but the entire spectrum – and hence the formant frequencies – up or down. The consequence is that only relatively small pitch corrections will sound natural.

The method used in *zita-at2* instead is known as **PSOLA** – Pitch Synchronous OverLap and Add. [4, 5] This is in fact a form of granular synthesis: the output signal is formed by adding overlapping *grains*, each of them representing one period of the waveform.

Given the estimated pitch period $P_E$, we can take a fragment of the input signal of length $2P_E$, and apply a raised cosine window. When this grain is added to the output every $P_E$ samples the result is a perfect reconstruction of the input. This is shown in the top part of Fig.4. In this example $P_E$ is 240 samples (200 Hz at a sample rate of 48 kHz).

<figure><img src="image_6.png" /><figcaption>Figure 4: Top: reconstruction with original pitch, bottom: reconstruction at 1.5 times original frequency</figcaption></figure>

Changing just the distance between the grains still produces a periodic signal but with a different pitch. The bottom part of Fig.4 shows the result when the original grain is output every 160 samples. This corresponds to a pitch change with ratio 3/2 – a musical fifth. In practice the amplitude of the grains will need to be adjusted in order to preserve the original loudness, but this is easy enough.

For best results the position of a grain in the input stream should be selected such that most of its energy is near the center of the window. Of course this can be done only for the first grain of a voiced fragment. The position of the following grains is then determined by the measured pitch period.

The main advantage of PSOLA over resampling is that it preserves the formants of the original signal. Figure 5 shows the spectra of the waveforms in Fig. 4. The retuned signal still has the same spectral peaks at 1 kHz and 3 kHz as the original, even if the fundamental frequency has changed by much more than would normally be required for autotuning.

The reason why PSOLA tends to preserve the formants is that the spectral envelope is a property of the basic grain. Resampling the grain will shift the formant frequencies, and this can be an interesting artistic effect. Moving the formants up in frequency produces a sort of 'squeaky' quality, while lowering them results in a 'hollow' voice.

On the downside, PSOLA may distort waveforms with a very narrow spectrum. Take the extreme case of a pure sine wave of frequency *F*. It can be said to have a strong 'formant' at *F*. When this is tuned down PSOLA will preserve this 'formant' and add harmonics around *F* even if these were not present at

<figure><img src="image_7.png" /><figcaption>Figure 5: Spectra of the waveforms shown in Fig.4</figcaption></figure>

all in the input. This is in practice not a problem with voice signals as these tend to be wideband and rich in harmonics.

### 3.3.2. Implementation of PSOLA

Although the basic PSOLA algorithm is not really complicated, any real-life implementation will require some nontrivial code.

In general neither the input nor the output pitch frequency will correspond to a period equal to an integer number of samples. This means we will need to shift the grain waveform by fractions of a sample. In practice this doesn't need to be exact, as long as the error does not accumulate over time. The current implementation uses eight-point interpolation. The delays are quantised to 1/32 of a sample, so all the necessary coefficients can be precomputed. If resampling is required in order to shift the formants, this can be done at the same time, it just means that the delay changes as the algorithm iterates over the samples of a grain. In *zita-at2* a single *Grain* class combines the fractional sample shift, the resampling and the windowing in a single highly optimised method.

In addition to that, there is a lot of rather tricky bookkeeping required in order to correctly select and position grains in the input and output streams, in particular if minimal latency is required.

## 3.4. Latency Issues

The current *zita-at2* implementation has a latency of around 20 ms, reducing to around 15 ms if configured for a female voice.

While latency is more or less irrelevant when using autotuning on a recorded track (as long as it is reported correctly so the DAW can compensate for it), it seems that some people want to use an autotuner for live performance.<fnref n="5" />

The minimal latency that can be achieved consistently depends strongly on the lowest pitch frequency that can be handled. Autocorrelation requires at least two periods to detect the pitch with any level of confidence. When configured for a minimum pitch frequency of 80 Hz (for a 'bass' male voice) the window size used is 32 ms, so the estimate is already 16 ms behind 'real time'. Then selecting the grain such that most of its energy is near the center of its window may add additional delay. Also at the same lower limit of 80 Hz, a single output grain can be as long as 25 ms. Only configuring the process for a higher minimum pitch frequency will allow lowering these figures.

<footnote n="5">⁵What good can come from that remains a mystery to the author.</footnote>
What can be done is to ensure that latency is as low as possible for the *unvoiced* fragments of the input signal. Since these tend to be transient, they probably contribute more to the perceived latency than the voiced parts.

Anyway, if someone claims that a derived version of *zita-at2* provides just a few milliseconds of latency, there are only two possibilities: either this is pure snake oil, or other aspects of the original performance will be seriously affected.

## 4. CONCLUSIONS

The preceding sections provide an overview of the algorithms used in *zita-at2*. It should be emphasised that this paper only scratches the surface, but the author hopes it will provide a useful starting point for anyone interested in developing the code, or creating a similar application.

Without doubt someone will turn the original Jack application into an LV2 plugin.<fnref n="6" /> This will certainly make *zita-at2* more accessible to most users, as well as provide new functionality, e.g. by allowing automation of the parameters.

## 5. ACKNOWLEDGMENTS

Many thanks to the anonymous reviewers for their comments and suggestions for improving this paper.

## 6. REFERENCES

[1] D. Gerhard, “Pitch extraction and fundamental frequency: History and current techniques,” Tech. Rep., Dept. of Computer Science, University of Regina, 2003, https://www2.cs.uregina.ca/~gerhard/publications/TRdbg-Pitch.pdf.

[2] Philip McLeod, “Fast, accurate pitch detection tools for music analysis,” 2008, https://www.cs.otago.ac.nz/research/publications/oucs-2008-03.pdf.

[3] Adriano Mitre, Marcelo Queiroz, and Regis R. A. Faria, “Accurate and efficient fundamental frequency determination from precise partial estimates,” 2006, http://www.ime.usp.br/~mqz/Mitre_AESBR2006.pdf.

[4] Eric Moulines and Francis Charpentier, “Pitch-synchronous waveform processing techniques for text-to-speech synthesis using diphones,” *Speech Communication*, vol. 9, pp. 453–467, 1990, https://courses.grainger.illinois.edu/ece420/sp2017/PSOLA.pdf.

[5] Sami Lemmetti, “Review of speech synthesis technology,” M.S. thesis, Helsinki University of Technology, Helsinki, Finland, 1999, http://research.spa.aalto.fi/publications/theses/lemmetty_mst/thesis.pdf.

<footnote n="6">⁶There are some 'usual suspects' expected to do this.</footnote>