Some Thoughts on Virtual Analog Modelling
In my work as an audio plugin developer, I spend a decent amount of time doing what some would call “virtual analog modelling,” or “physical modelling”. The main idea behind these types of modelling is that you have some physical device that exists in the world (maybe a tape machine or a guitar pedal), and you want to create an “emulation” of that device, so that you can achieve an identical sound in a digital environment.
Within the world of virtual analog modelling, folks often distinguish between “white-box” modelling, where the emulation is created from some physics-based understanding of how the device works; and “black-box” modelling, where the emulation is primarily data-driven. Emulations that are created using a mix of white-box and black-box techniques are sometimes referred to as “grey-box” models. In recent years, black-box modelling has become quite popular, in part due to the rise of neural networks, which can be used to emulate a wide variety of analog devices.
Over the past few years, I’ve developed some thoughts on virtual analog modelling, mostly via conversations that I’ve had with folks both inside and outside the audio industry. I wanted to use this article to share those thoughts, in particular a defense of why I believe that white-box modelling is a worthwhile endeavour, as well as some thoughts on why focusing on physical modelling can be quite limiting to our field.
Physical Modelling as Deep Work
“All things excellent are as difficult as they are rare, said a wise man. If so, what happens to excellence when we eliminate the difficulty and the rarity?”
- Edward Abbey
Why Bother?
As neural networks have become popular solutions to problems in nearly every field, they’ve certainly made an impact in the world of physical modelling. Several companies today, including Neural DSP, Native Instruments, BABY Audio, and more, offer audio plugins and other products which use neural networks to model guitar amplifiers and other musical equipment. I myself have developed several plugins that use neural networks for virtual analog modelling, along with tools to help other developers create their own software using neural networks in real-time.
With the success of black-box modelling with neural networks, it may seem that white-box modelling is no longer worth the effort. After all, if I can get a great sounding model of my guitar amplifier simply by recording a brief snippet of audio through the amp, and then training a neural network to instantly “capture” that sound, why go through the painstaking effort of constructing a physics-based model, with all the nasty math and physics that entails.
Deep Work
I recently watched a talk by game developer Jonathan Blow, where he talks about the idea of “Deep Work”. Blow defines Deep Work as work which fundamentally changes you, so that the person you are after having completed the work is a different person than the one who started.
This type of thinking may sound a little bit pretentious, but I think it’s worthwhile to consider the amount of time we spend at work. Assuming a 40 hour work week, and a working career spanning multiple decades, wouldn’t you want to spend that time doing work that changes you, hopefully in a positive way?
Learning from the model
A few years ago, I worked on a project where I attempted to create a model of the Klon Centaur guitar pedal, using both a “white-box” physics-based model, as well as a “black-box” neural-network-based model.
The Centaur is a distortion pedal, so I had assumed that most of the pedal’s circuit would be responsible for creating some kind of nonlinear process, thus generating harmonic distortion. However, while analyzing the circuit, I noticed that the primary distortion mechanism in the circuit is a simple diode clipper. Surrounding the diode clipper were several linear sub-circuits, primarily responsible for filtering the sound and shaping the frequency response of the signal.
Looking back on it now, the most important thing that I learned from creating that circuit model is that to create a distortion effect that sounds good it’s vitally important to consider the frequency response of the signal being distorted, both before and after the distortion occurs. From a “Deep Work” perspective that lesson changed me, it changed the way I think about distortion, and the way that I think about audio effect design overall.
I’m not sure I would have learned that same lesson from only constructing the neural network model. With the black-box approach, I wasn’t forced to spend much time with the circuit schematic (and if I actually owned the pedal, I wouldn’t have needed the schematic at all). It was only by analyzing the circuit at the level of depth necessary to create a white-box model, that I was able to learn more about the way the effect was designed.
Why Not?
I’m not trying to say that developers should only create white-box models, because black-box modelling is “lazy”, or that it’s impossible learn anything from creating neural network models. In contrast, there are many technical and logistical reasons to choose either a white-box or a black-box model for a given circuit. What I am trying to say, is that when I start working on a new circuit model, I try to think about what I want out of the process for myself. And when I do choose to engage with the often “deeper” work of white-box modelling, I almost always find the effort worthwhile.
Don’t Limit Yourself
You get a little jaded after a while… there’s so many bands that are coming out and doing their classic album from 25 years ago… It might be really exciting for the fans and pretty cool for them, but to me it’s almost a bit sad… it’s almost like being resigned that the best has come and gone.
- DJ Shadow
Cover Bands
From a musician’s perspective, creating a physical model of some musical equipment is somewhat analogous to a musician performing a cover song. In both cases, you have an original creator (the songwriter/composer :: the circuit designer), some process for creating the “cover” (transcribing the song :: constructing the physical model), and some form of presenting the cover (playing the song live or recording it for an album :: releasing an audio plugin).
Now, I understand that cover bands have a little bit of a negative connotation in some circles, and I’m certainly not trying to diminish the efforts of either cover bands or engineers who create physical models. After all, I’ve played in several cover bands, and many of my favorite musical artists have recorded or performed many cover songs. I’m just saying that I find it a bit sad how many audio software developers limit themselves to only thinking about physical modelling. For me personally, I feel that if I were to only make physically modelled audio effects, I would basically be admitting to myself that I’m not capable of designing my own effects that could be as good (if not better) than the devices I’m modelling.
Make It Your Own
Another useful way to think about the “cover songs” analogy, is to think about how many cover songs are not meant to be identical to the original. Often times the artist will make some changes to the arrangement or instrumentation of the song, or even try to alter the emotional tone of different sections of the song. I feel like this is something I would like to see more of in the physical modelling world. Rather than trying to make a direct emulation of some analog device, I sometimes try thinking about how I would have made the device differently if I were the one designing it. Or maybe, try emphasizing some aspects of the device that I really like, and ignoring or de-emphasizing some parts that I don’t.
For example, I was recently working on a model of the Crowther “Hot Cake” guitar pedal. In some versions of the circuit, there is a “switched” capacitor, to adjust the the high-frequency response of the circuit. Personally, I would rather have a continuous control over the high frequency response rather than a switched control, so in my model, I replaced the switched capacitor with a variable capacitor. This is a pretty small example of “making it my own”, but once you break out from the limitation of trying to do everything exactly like the original device, you can end up with some surprising and fascinating results!
Nothing New Under The Sun
Trying to make something completely new and original is a daunting task. And while many creative folks have attempted to do so, it often proves to be a fruitless and unforgiving endeavour. So here’s some ideas for creating new audio effects that are not direct emulations of existing devices:
- Think about some of your favorite effects. Now try to figure out what gives them their characteristic sound (similar to my analysis of the Centaur). Finally, design your own effect, using those same ideas. For bonus points, try merging ideas from several of your favorite effects.
- Start from the basics: think about basic elements of audio signal processing: filters, waveshapers, delay lines, oscillators. Now think about ways of combining these basic elements in creative and interesting ways. For some examples of this kind of thinking, I’d suggest reading the excellent SignalSmith blog.
- Find a sound that you like (maybe a guitar tone, or a fat drum kit sound). Now think about the controls that you would want over that sound as a musician or mixing engineer. Is it possible to design an effect which can create that sound while also giving you the musical controls you desire?
- Try some “model bending”. Dan Gillespie (of Newfangled Audio) and Sam Schachter have a fascinating DAFx paper from a couple years ago, introducing model bending as a way to coax new and unique sounds out of existing circuit models.
Wrapping Up
Writing this article has been a bit difficult for me. A lot of these ideas are things that I’ve thought about for a long time, but haven’t really known how to put into words. Hopefully this effort has been helpful for some readers. Onward!
