Charles Petzold

Sometimes I thiink of myself as a scientist who studies operating systems. Although these operating systems form the structure and foundation of the natural world in which I exist, for the most part I can't study the internals. I'm pretty much limited to the external physical manifestion of this world, ie, the API. But that's often enough. APIs can be poked and prodded. Experiments can be performed, and conclusions drawn. Like a real scientist I sometimes develop hypotheses about how these API worlds work, and then I must write some code to test if these hypotheses are valid.

And sometimes I jump the gun — at least in my own head. I stumble upon something that I just know will make a great article or blog entry. If I don't have time to work on it right away, I start mentally coding the programs I'll need to demonstrate the concept, and I'll be writing the article in my head, and even graciously acknowledging the accolades that are sure to follow.

And then — usually about the time I'm coding the samples that will prove the point I'm trying to make — reality simply does not oblige. That's too bad, because reality, for real scientists as well as the rest of us, must be the final arbiter.

So I'll begin by admitting that what I wanted to demonstrate in this blog entry is simply not the case. And that's good news and bad news. The good news is that the technique you use to update the pixels of a WriteableBitmap from a Windows 8 C++ program does not have any performance benefit over the technique you use in a C# program. But of course, that could also be the bad news.

WriteableBitmap is all about getting access to the bitmap pixels, either for modifying an existing bitmap image, or creating one from scratch. For convenience to the application programmer, a Windows 8 WriteableBitmap always has four bytes per pixel in the order Blue-Green-Red-Alpha. The colors are based on an sRGB color space with pre-multiplied alpha. There is no row padding.

If you've read my earlier blog entry on the Windows 8 WriteableBitmap or if you download the the release preview ebook of Programming Windows 6th edition in a couple weeks you know that a C# program must create a byte array for the pixels of the WriteableBitmap. If bitmap is an object of type WriteableBitmap, this array can be created like so:

byte[] pixels = new byte[4 * bitmap.PixelWidth * bitmap.PixelHeight];

Of course, in some cases you need to get pixels out of an existing WriteableBitmap to manipulate, and you always have to get those pixels into the WriteableBitmap. For that, WriteableBitmap defines a PixelBuffer property of type IBuffer.

IBuffer is defined in the Windows.Storage.Streams namespace and is documented as representing "a referenced array of bytes used by byte stream read and write interfaces." However, the IBuffer interface only defines two properties named Capacity and Length and not the array of bytes itself. I suspect that's because in most cases this array of bytes occupies operating system memory rather than application memory.

You can create a Buffer object that implements the IBuffer interface, but in doing so you don't specify an array of bytes and you don't have access to the array of bytes. So why would you want to create such a thing? Because it provides a convenient — and probably very efficient — way to transfer bytes from one stream to another. The ReadAsync method defined by the IInputStream interface reads from a stream into an IBuffer and the WriteAsync method defined by the IOutputStream interface writes from the IBuffer into the stream.

You can create a DataReader from an IBuffer using the static FromBuffer method, and this allows you to read primitive types from the IBuffer. However, DataWriter doesn't allow you to go in the opposite direction.

So how can we get pixels in and out of the WriteableBitmap through this IBuffer object? Fortunately for the C# programmer, there is an extension method defined in the System.Runtime.InteropServices.WindowsRuntime namespace that lets you access the IBuffer as a .NET Stream:

Stream pixelStream = bitmap.PixelBuffer.AsStream();

And that Stream allows you to call Read and Write (or ReadAsync or WriteAsync) to transfer the byte array in and out of the bitmap. For an image that requires dynamic updating, generally the WriteableBitmap, the pixels array, and the pixelStream object are stored as fields. After setting the new pixel values in the pixel array, the entire bitmap is updated like so:

pixelStream.Seek(0, SeekOrigin.Begin);
pixelStream.Write(pixels, 0, pixels.Length);
bitmap.Invalidate();

Let's see how this works in the context of a real Windows 8 app written in C#. This project is called TwisterCSharp, and it runs under the Release Preview of Windows 8. The XAML file simply defines an Image element waiting for a bitmap:

<Page
    x:Class="TwisterCSharp.MainPage"
    IsTabStop="false"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:local="using:TwisterCSharp">

    <Grid Background="{StaticResource ApplicationPageBackgroundThemeBrush}">
        <Image Name="image"
               Stretch="None"
               HorizontalAlignment="Center"
               VerticalAlignment="Center" />
    </Grid>
</Page>

The code-behind file downloads a bitmap from my web site using BitmapDecoder and saves the result in an array of bytes. It also creates a WriteableBitmap of the same size, sets that to the Image element, and creates an array of bytes for that bitmap. During each call to a CompositionTarget.Rendering event handler, the original pixel bytes are transfered to the second pixel array based on an animated bitmap "twister" algorithm, which bases the amount of twist on the degree to which a pixel is closest to the center of the bitmap. Here's the entire MainPage class:

public sealed partial class MainPage : Page
{
    static readonly long cycleDuration = TimeSpan.FromSeconds(3).Ticks;

    // Bitmap information
    WriteableBitmap bitmap;
    Stream pixelStream;
    byte[] srcPixels, dstPixels;
    int width, height, xCenter, yCenter;

    public MainPage()
    {
        this.InitializeComponent();
        Application.Current.DebugSettings.EnableFrameRateCounter = true;
        Loaded += OnMainPageLoaded;
    }

    async void OnMainPageLoaded(object sender, RoutedEventArgs args)
    {
        Uri uri =
                new Uri("http://www.charlespetzold.com/pw6/PetzoldJersey.jpg");
        RandomAccessStreamReference streamRef
                RandomAccessStreamReference.CreateFromUri(uri);

        using (IRandomAccessStreamWithContentType fileStream =
                        await streamRef.OpenReadAsync())
        {
            BitmapDecoder decoder = await BitmapDecoder.CreateAsync(fileStream);
            BitmapFrame frame = await decoder.GetFrameAsync(0);

            // I know the parameterless version of GetPixelDataAsync works for this image
            PixelDataProvider pixelProvider = await frame.GetPixelDataAsync();
            srcPixels = pixelProvider.DetachPixelData();

            // Create the WriteableBitmap
            bitmap = new WriteableBitmap((int)frame.PixelWidth,                                          (int)frame.PixelHeight);
        }

        // Set the bitmap to the Image element
        image.Source = bitmap;

        // Save some information in fields
        width = bitmap.PixelWidth;
        height = bitmap.PixelHeight;
        xCenter = width / 2;
        yCenter = height / 2;

        // Get Stream and create array for updating bitmap
        pixelStream = bitmap.PixelBuffer.AsStream();
        dstPixels = new byte[4 * width * height];

        // Let's go!
        CompositionTarget.Rendering += OnCompositionTargetRendering;
    }

    void OnCompositionTargetRendering(object sender, object args)
    {
        // Get elapsed time
        TimeSpan timeSpan = (args as RenderingEventArgs).RenderingTime;

        // Calculate twistAngle from -180 to 180 degrees
        double t = (timeSpan.Ticks % cycleDuration) / (double)cycleDuration;
        double tprime = 2 * (t < 0.5 ? t : 1 - t);
        double twistAngle = 2 * (tprime - 0.5) * Math.PI;

        for (int yDst = 0; yDst < height; yDst++)
        for (int xDst = 0; xDst < width; xDst++)
        {
            // Calculate length of point to center and angle
            int xDelta = xDst - xCenter;
            int yDelta = yDst - yCenter;
            double distanceToCenter = Math.Sqrt(xDelta * xDelta +
                                                yDelta * yDelta);
            double angleClockwise = Math.Atan2(yDelta, xDelta);

            // Calculation angle of rotation for twisting effect
            double xEllipse = xCenter * Math.Cos(angleClockwise);
            double yEllipse = yCenter * Math.Sin(angleClockwise);
            double radius = Math.Sqrt(xEllipse * xEllipse +
                                      yEllipse * yEllipse);
            double fraction = Math.Max(0, 1 - distanceToCenter / radius);
            double twist = fraction * twistAngle;

            // Calculate the source pixel for each destination pixel
            int xSrc = (int)(xCenter + (xDst - xCenter) * Math.Cos(twist)
                                     - (yDst - yCenter) * Math.Sin(twist));
            int ySrc = (int)(yCenter + (xDst - xCenter) * Math.Sin(twist)
                                     + (yDst - yCenter) * Math.Cos(twist));
            xSrc = Math.Max(0, Math.Min(width - 1, xSrc));
            ySrc = Math.Max(0, Math.Min(height - 1, ySrc));

            // Calculate the indices
            int iDst = 4 * (yDst * width + xDst);
            int iSrc = 4 * (ySrc * width + xSrc);

            // Transfer the pixel bytes
            dstPixels[iDst++] = srcPixels[iSrc++];
            dstPixels[iDst++] = srcPixels[iSrc++];
            dstPixels[iDst++] = srcPixels[iSrc++];
            dstPixels[iDst] = srcPixels[iSrc];
        }

        // Transfer the pixels to the bitmap
        pixelStream.Seek(0, SeekOrigin.Begin);
        pixelStream.Write(dstPixels, 0, dstPixels.Length);
        bitmap.Invalidate();
    }
}

This program has a combination of number-crunching and bitmap-updating. The CompositionTarget.Rendering method is called at the frame rate of the video display (often 60 times per second) but notice that the constructor has enabled the frame-rate counter to determine how many frames per second the program can really manage. The frame-rate counter is displayed in the upper-left corner of the window:

In testing the programs for this blog entry, I alternately compiled for Debug and Release configurations, and then launched the programs from the Windows 8 Start Screen so they were not running under the Visual Studio debugger, and I waited until the number stabilized. I ran the programs on the Samsung tablet distributed at the Build 2011 conference installed with Windows 8 Release Preview Build 8400. This program showed the following frame rates:

It's easy to verify that the performance issues are due almost entirely to the number crunching and not to the code at the bottom that transfers the pixels to the WriteableBitmap: Simply remove most of the CompositionTarget.Rendering method so it looks like so:

void OnCompositionTargetRendering(object sender, object args)
{
    for (int yDst = 0; yDst < height; yDst++)
    for (int xDst = 0; xDst < width; xDst++)
    {
        // Calculate the indices
        int iDst = 4 * (yDst * width + xDst);
        int iSrc = iDst;

        // Transfer the pixel bytes
        dstPixels[iDst++] = srcPixels[iSrc++];
        dstPixels[iDst++] = srcPixels[iSrc++];
        dstPixels[iDst++] = srcPixels[iSrc++];
        dstPixels[iDst] = srcPixels[iSrc];
    }

    // Transfer the pixels to the bitmap
    pixelStream.Seek(0, SeekOrigin.Begin);
    pixelStream.Write(dstPixels, 0, dstPixels.Length);
    bitmap.Invalidate();
}

The frame rate goes up to 60 but, of course, the visuals aren't nearly as interesting.

Despite that, I'm sure I'm not the only person who feels a little quesy seeing a Stream object in this context. I think it's a psychological block of sorts. When we see a Stream object we usually think about file I/O or network I/O, and thus we feel that any operation involving a Stream must be inherently slow.

Not necessarily so! This particular Write call probably just transfers one array into another array — from the local pixels array in the program to the array of bytes actually belonging to this WriteableBitmap. Memory-to-memory copies can be extremely fast, and are typically performed by a single optimized microprocessor instruction. (Indeed, memory block transfers represent some of the first "high-level" instructions implemented in microprocessors. I fondly remember them from coding for the Zilog Z80 in the late 1970s.)

But can we verify this? Yes, but in preparation let's look at the same program written in C++. This project is called TwisterCPlusPlus. The MainPage.xaml file is basically the same as the one shown earlier. Here's a good chunk of the MainPage.xaml.h file:

namespace TwisterCPlusPlus
{
    public ref class MainPage sealed
    {
    private:
        static const int64 cycleDuration = 30000000;   // 3 seconds

        // Bitmap information
        Windows::UI::Xaml::Media::Imaging::WriteableBitmap^ bitmap;
        Platform::Array<byte>^ srcPixels;
        byte* pDstPixels;
        int width, height, xCenter, yCenter;

    public:
        MainPage();

    private:
        void OnMainPageLoaded(Platform::Object^ sender, Windows::UI::Xaml::RoutedEventArgs ^ args);
        void OnCompositionTargetRendering(Platform::Object^ sender, Platform::Object^ args);
    };
}

And here are the implementations of the constructor and two methods in MainPage.xaml.cpp:

MainPage::MainPage()
{
    InitializeComponent();
    Application::Current->DebugSettings->EnableFrameRateCounter = true;
    Loaded += ref new RoutedEventHandler(this, &MainPage::OnMainPageLoaded);
}

void MainPage::OnMainPageLoaded(Object^ sender, RoutedEventArgs^ args)
{
    Uri^ uri = ref new Uri("http://www.charlespetzold.com/pw6/PetzoldJersey.jpg");
    RandomAccessStreamReference^ streamRef = RandomAccessStreamReference::CreateFromUri(uri);

    task<IRandomAccessStreamWithContentType^> (streamRef->OpenReadAsync()).
    then([](task<IRandomAccessStreamWithContentType^> thisTask)
    {
        IRandomAccessStreamWithContentType^ fileStream = thisTask.get();
        return BitmapDecoder::CreateAsync(fileStream);
    }).
    then([](task<BitmapDecoder^> thisTask)
    {
        BitmapDecoder^ decoder = thisTask.get();
        return decoder->GetFrameAsync(0);
    }).
    then([this](task<BitmapFrame^> thisTask)
    {
        BitmapFrame^ frame = thisTask.get();   

        // Save some information as fields
        width = frame->PixelWidth;
        height = frame->PixelHeight;
        xCenter = width / 2;
        yCenter = height / 2;

        return frame->GetPixelDataAsync();
    }).
    then([this](task<PixelDataProvider^> thisTask)
    {
        PixelDataProvider^ pixelProvider = thisTask.get();
        srcPixels = pixelProvider->DetachPixelData();

        // Create the WriteableBitmap 
        bitmap = ref new WriteableBitmap(width, height);

        // Set the bitmap to the Image element
        image->Source = bitmap;

        // Get access to the pixels
        IBuffer^ buffer = bitmap->PixelBuffer;

        // Obtain IBufferByteAccess
        ComPtr<IBufferByteAccess> pBufferByteAccess;
        ComPtr<IUnknown> pBuffer((IUnknown*)buffer);
        pBuffer.As(&pBufferByteAccess);
    
        // Get pointer to pixel bytes
        pBufferByteAccess->Buffer(&pDstPixels);

        // Let's go!
        CompositionTarget::Rendering += ref new EventHandler<Object^>(this, &MainPage::OnCompositionTargetRendering);
    });
}

void MainPage::OnCompositionTargetRendering(Object^ sender, Object^ args)
{
    // Get elapsed time
    TimeSpan timeSpan = dynamic_cast<RenderingEventArgs^>(args)->RenderingTime;

    // Calculate twistAngle from -180 to 180 degrees
    double t = (timeSpan.Duration % cycleDuration) / (double)cycleDuration;
    double tprime = 2 * (t < 0.5 ? t : 1 - t);
    double twistAngle = 2 * (tprime - 0.5) * 3.14159;

    for (int yDst = 0; yDst < height; yDst++)
    for (int xDst = 0; xDst < width; xDst++)
    {
        // Calculate length of point to center and angle
        int xDelta = xDst - xCenter;
        int yDelta = yDst - yCenter;
        double distanceToCenter = sqrt(xDelta * xDelta +
                                       yDelta * yDelta);
        double angleClockwise = atan2(yDelta, xDelta);

        // Calculation angle of rotation for twisting effect
        double xEllipse = xCenter * cos(angleClockwise);
        double yEllipse = yCenter * sin(angleClockwise);
        double radius = sqrt(xEllipse * xEllipse +
                             yEllipse * yEllipse);
        double fraction = max(0.0, 1 - distanceToCenter / radius);
        double twist = fraction * twistAngle;

        // Calculate the source pixel for each destination pixel
        int xSrc = (int)(xCenter + (xDst - xCenter) * cos(twist)
                                 - (yDst - yCenter) * sin(twist));
        int ySrc = (int)(yCenter + (xDst - xCenter) * sin(twist)
                                 + (yDst - yCenter) * cos(twist));
        xSrc = max(0, min(width - 1, xSrc));
        ySrc = max(0, min(height - 1, ySrc));

        // Calculate the indices
        int iDst = 4 * (yDst * width + xDst);
        int iSrc = 4 * (ySrc * width + xSrc);

        // Transfer the pixel bytes
        pDstPixels[iDst++] = srcPixels[iSrc++];
        pDstPixels[iDst++] = srcPixels[iSrc++];
        pDstPixels[iDst++] = srcPixels[iSrc++];
        pDstPixels[iDst] = srcPixels[iSrc];
    }

    // Invalidate the bitmap
    bitmap->Invalidate();
}

In theory, we should get better performance from this program because C++ programs for Windows 8 are compiled to native code whereas C# programs are compiled to managed code.

However, there is also a significant difference in how a C++ program updates a WriteableBitmap. The C++ program does not need to create a local byte array for the pixels and then transfer this array of pixels to the bitmap through a Stream. Towards the bottom of the constructor, you can see how the program gets an IBuffer from the PixelBuffer property of the WriteableBitmap but then obtains an IBufferByteAccess object and from that a pointer to the pixels:

// Get access to the pixels
IBuffer^ buffer = bitmap->PixelBuffer;

// Obtain IBufferByteAccess
ComPtr<IBufferByteAccess> pBufferByteAccess;
ComPtr<IUnknown> pBuffer((IUnknown*)buffer);
pBuffer.As(&pBufferByteAccess);

// Get pointer to pixel bytes
pBufferByteAccess->Buffer(&pDstPixels);

I believe this is a pointer to the actual pixel array associated with the WriteableBitmap. This means that the C++ program can use this pointer to transfer new pixels directly to the bitmap!

Interestingly, the C++ program only has better performance than the C# program when compiled in the Release configuration, and then it had demonstrably better performance. Here are the frame rates:

The difference between Debug and Release emphasizes even more that the performance improvements in the C++ program are due to the number crunching in CompositionTarget.Rendering and not how the pixels are transferred into the WriteableBitmap across the API. But still, wouldn't it be great to be able to get ahold of that pixel pointer in a C# program just to see what it would be like?

It is indeed possible — as long as you're not opposed to using the unsafe keyword — and the technique is demonstrated in the TwisterCSharpUnsafe project. I won't show you the whole MainPage file again because only a few things are different. You need to define an IBufferByteAccess interface that mimics the one available in C++:

[Guid("905a0fef-bc53-11df-8c49-001e4fc686da"),
   InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
interface IBufferByteAccess
{
    unsafe void Buffer(out byte* pByte);
}

Notice the unsafe keyword required because the Buffer method returns a pointer. I lifted the GUID from robuffer.h, the header file that defines the IBufferByteAccess interface for C++ programs. You can then cast the IBuffer object obtained from the PixelBuffer property of WriteableBitmap to this interface, and call the Buffer method to get a pointer to the pixel array:

// Get a pointer to the bitmap pixels
IBuffer iBuffer = bitmap.PixelBuffer;
IBufferByteAccess bufferByteAccess = (IBufferByteAccess)iBuffer;
unsafe
{
    bufferByteAccess.Buffer(out pDstPixels);
}

Just as in C++, you can then use this pointer to copy bytes into the bitmap, just as long as you put the code in an unsafe block:

// Transfer the pixel bytes
unsafe
{
    pDstPixels[iDst++] = srcPixels[iSrc++];
    pDstPixels[iDst++] = srcPixels[iSrc++];
    pDstPixels[iDst++] = srcPixels[iSrc++];
    pDstPixels[iDst] = srcPixels[iSrc];
}

It certainly seems as if this should be faster than using a Stream to transfer a local array of bytes into the WriteableBitmap, but I couldn't see a performance difference:

To me this indicates that despite our quesiness about using a Stream in connection with a WriteableBitmap, internally it's very fast; regardless how we update a WriteableBitmap, performance is the same. In my experience, this is true of the rest of the Windows 8 API. The API itself is language independent. For a program that doesn't do a lot of heavy processing on its own but instead mostly interacts with the API, it makes very little difference whether you code it in C# or C++.

However, if your program itself needs to perform a lot of heavy lifting, C++ offers definite performance advantages.

All three projects are downloadable here.