The golden test framework is already paying off. Today I was working on creating a golden test for the tempo-synced LFO feature, and while creating the preset and MIDI file for the test, I noticed a bug with the MIDI note ON/OFF implementation. It turns out that if a note OFF and a note ON event (for the same note) were handled in a single call to the audio block processing function, it would always treat them as ON-then-OFF despite the actual order in which they appeared. I'm not sure how I didn't notice this before, because it's not at all an edge case.
Anyway the golden test framework let me create a test that passes with the fixed code and fails with the old code, in just a few minutes, and also guarantees that this won't happen again. And, because it's a black-box test, it will automatically validate that the Metal and CUDA implementations also don't have this problem, once they come into existence.
