Complex Nonlinearities Episode 10: Gated Recurrent Distortion
Today we’re going to be discussing an interesting type of distortion effect, based around the idea of a Gated Recurrent Unit (GRU). First introduced by Cho et al in 2014, GRUs have become a basic building block of Recurrent Neural Networks. However, when analyzing the structure of these units, I started wondering about the possibility of using the GRU architecture for an interesting type of distortion effect.
A GRU is made up of two simple nonlinearities: the sigmoid and tanh nonlinearities, both shown below. While these curves look similar, note that the sigmoid function goes from 0 to 1, while the tanh function goes from -1 to 1.
Using these basic nonlinear building blocks we can construct a simple type of GRU known as a “minimal gated unit” (introduced by Heck and Salem, 2017). The signal processing architecture for this unit is shown below, with “Sg” denoting the sigmoid nonlinearity.
In this writing, I’m not going to discuss the mathematics of GRUs and minimal gated units in depth, but for more information, definitely take a look at the linked papers, as well as the Wikipedia article linked here
Distortion Curve and Parameters
What makes this type of distortion so interesting, is its wonderful potential for parameterization. Specifically, the architecture we use here has 5 parameters: Wf, Wh, Uf, Uh, and bf. Below we show how these parameters can change the shape of the distortion curve.
The simplest parameter is Wh, which acts as a basic “drive” parameter:
Wf and Uf change the shape of the top half of the distortion curve, to have more saturation and more width between the forward and backward parts of the curve.
Uh and bf affect the overall width of the central part of the distortion curve.
As is evidenced by the above plots, the GRU is an extremely flexible nonlinearity when the parameters are used to their fullest extent.
As shown in the section above, GRU distortion can be very different depending on the parameters of the distortion. We show below how various parameters can get different types of harmonics responses.
Thanks as always for taking the time to read about nonlinear audio signal processing! Not sure what will be in store for out next episode, but stay tuned.