Complex Nonlinearities Episode 10: Gated Recurrent Distortion

Today we’re going to be discussing an interesting type of distortion effect, based around the idea of a Gated Recurrent Unit (GRU). First introduced by Cho et al in 2014, GRUs have become a basic building block of Recurrent Neural Networks. However, when analyzing the structure of these units, I started wondering about the possibility of using the GRU architecture for an interesting type of distortion effect.

GRU Architecture

A GRU is made up of two simple nonlinearities: the sigmoid and tanh nonlinearities, both shown below. While these curves look similar, note that the sigmoid function goes from 0 to 1, while the tanh function goes from -1 to 1.

Using these basic nonlinear building blocks we can construct a simple type of GRU known as a “minimal gated unit” (introduced by Heck and Salem, 2017). The signal processing architecture for this unit is shown below, with “Sg” denoting the sigmoid nonlinearity.

In this writing, I’m not going to discuss the mathematics of GRUs and minimal gated units in depth, but for more information, definitely take a look at the linked papers, as well as the Wikipedia article linked here

Distortion Curve and Parameters

What makes this type of distortion so interesting, is its wonderful potential for parameterization. Specifically, the architecture we use here has 5 parameters: Wf, Wh, Uf, Uh, and bf. Below we show how these parameters can change the shape of the distortion curve.

The simplest parameter is Wh, which acts as a basic “drive” parameter:

Wf and Uf change the shape of the top half of the distortion curve, to have more saturation and more width between the forward and backward parts of the curve.

Uh and bf affect the overall width of the central part of the distortion curve.

As is evidenced by the above plots, the GRU is an extremely flexible nonlinearity when the parameters are used to their fullest extent.

Harmonic Response

As shown in the section above, GRU distortion can be very different depending on the parameters of the distortion. We show below how various parameters can get different types of harmonics responses.

Conclusion

As usual, we have implemented the above distortion unit as an open-source audio plugin (VST, AU). The source code is available on GitHub, and a video demo can be found on YouTube.

Thanks as always for taking the time to read about nonlinear audio signal processing! Not sure what will be in store for out next episode, but stay tuned.