Charles Petzold

Many, many years ago when multimedia was new to Windows, I wrote a bunch of articles for PC Magazine about sound and music programming. Many of those sample programs ended up in Chapter 22 of Programming Windows, 5th edition, including KBMidi ("keyboard MIDI"), a program that let you play the sound board's MIDI synthesizer through your PC's keyboard.

KBMidi featured a diagram of your PC's keyboard labeled with the notes of the octave, which are highlighted when the key is pressed. Here's me playing a C-minor chord:

I decided to try something similar for Silverlight 3 but with a completely reconceived screen image. The SimpleKeyboardSynthesizer program displays a music keyboard labeled with the letters and numbers of your PC keyboard, which you can run from here:

SimpleKeyboardSynthesizer.html

Keys are colored gray when they're pressed. If the Shift key is down when you press a key, the pitch is raised an octave; the Ctrl key drops an octave. The ScrollBar is a volume control.

SimpleKeyboardSynthesizer has several problems and issues, some of which are my fault (because I've only recently started messing around with Silverlight's real-time sound capabilities and obviously I haven't gotten very far), and some of which are intrinsic to Silverlight:

• There is a delay between the time you press a key and the sound starts, and when you release a key and the sound ends. Obviously, this makes actually playing the keyboard with any type of musical sense extremely difficult, and (some would even say) renders the entire exercise useless. Unfortunately, the delay seems to be part of the buffering logic that the MediaPlayer uses and I don't see any way to shorten it.

• The KBMidi program had a four-octave range. This range on this Silverlight version is barely more than two octaves. This difference is related to the information (or rather, the lack of information) available through the Silverlight KeyDown and KeyUp events, which is in turn related to Silverlight's need to be platform independent. Even so, I'm assuming a US keyboard layout. The program won't work well with some other layouts.

• The keyboard is polyphonic (that is, you can play multiple notes at the same time) but with limits: The program itself allows a maximum of 8 oscillators, but the rollover characteristics of your PC keyboard will probably kick in before you reach that limit. Depending on the keyboard and the particular keys that are pressed, you may only be able to play three notes simultaneously.

• The notes themselves are unfiltered unadorned unenveloped algorithmic waveforms. The process of turning waveforms into interesting sounds is the real challenge of electronic music synthesis, and which I hope to tackle over the remainder of the summer and — who knows? — perhaps beyond.)

Here's the source code. Compared with my first blog entry on Silverlight Sound Synthesis, you'll notice that the Petzold.SoundSynthesis library now contains a Mixer module that combines outputs from multiple oscillators. Here's the Mixer class in its entirety:

public class Mixer : ISampleProvider
{
    public IList<ISampleProvider> Inputs { get; protected set; }

    public Mixer()
    {
        Inputs = new List<ISampleProvider>();
    }

    public short GetNextSample()
    {
        int sample = 0;

        if (Inputs.Count > 0)
        {
            int multiplier = 65536 / Inputs.Count;

            foreach (ISampleProvider input in Inputs)
                sample += multiplier * input.GetNextSample();
        }

        return (short)(sample >> 16);
    }
}

The Mixer class implements the ISampleProvider interface, which means that it includes a method named GetNextSample that returns a short. This method is called asynchronously (that is, not in the UI thread) 44,100 times per second. The class also includes a public property named Inputs of type IList<ISampleProvider>. In this program, these inputs are Oscillator objects.

It is tempting to write code for a music synthesizer using mostly floating point and then convert to 16-bit samples at the very tail of the process. However, if you consider that 16-bit samples need to be produced at the rate of 44,100 per second (for so-called CD Quality), it's best to minimize the use of floating point throughout the sample pipeline. (I still have floating point in the generation of the sine curve, but I'll be switching to a lookup table at some point.)

Because samples are only 16 bits wide, you can use use a 32-bit int to represent a 16-bit whole number with a 16-bit fractional part. This is basically what the Mixer module is doing: It's effectively multiplying each incoming 16-bit sample by 1/N, where N is the number of inputs, and then summing the results. But 1/N is really the integer 65536/N.

It seems reasonable that a program of this sort could dynamically create Oscillator objects and add these inputs to the Mixer as the user is playing the music. But this won't work right: Suppose the user presses a key for C. The program responds by creating a new Oscillator object, setting the correct frequency, and adding it to the Inputs collection of the Mixer. Then, without releasing the C, the user presses a key for an E. Now the program responds by creating a second Oscillator object, setting the correct frequency, and adding that the Inputs collection. But now there are two inputs to the Mixer, and it's multiplying the samples from each input by 1/2, and the sound from the first Oscillator suddenly drops in volume!

The correct approach is to create N Oscillator objects at the outset, set all the frequencies to zero, and add all N to the Inputs collection of the Mixer. Then as the user is playing the keyboard, just pick an available oscillator when needed. (That's also why I didn't implement any thread synchronization in the access of the List object, but I really should have.) In SimpleKeyboardSynthesizer, N is hard-coded as 8.