While I should have been testing more images, to prove my concept, I really have been busy with maturing my compression project.

For first time readers, a little flashback. Filesize is highly important when creating browser games. Small data size is required to get faster download times. Download time is a crucial factor for people to decide whether they will play your game or not.

So how to crop lots of images in your game, while keeping the download size acceptable? Well, use a very good compression format for your images. PNG has a good compression, however many PNG’s altogether make a slow download. With Inkscape, it is possible to trace PNG bitmaps into vector graphics and save them as XAML. But these vector graphics are usually much larger than the PNG you started with. I have been busy with a method to compress these Inkscape XAML’s, with some success.

The following picture illustrates different quality results, so they are easy to compare:

From left to right.

The first film-strip is the original PNG-24, 15.3KB, which is used in the game Slengo.

The second film-strip is the original PNG, but now saved as PNG-8, which is 4.59Kb.

The third filmstrip is an Inkscape result and then compressed with my compression technique. The original PNG-24 is input. Trace bitmap, smooth=false, remove background=true, scans=11. The size of this file is reduced to 11Kb, when winzipped.

The fourth filmstrip is an Inkscape result, compressed with my technique. Input is a regenerated image, similar to the PNG-24 original, but now twice its size. The original uses 40x40 frames, while this image uses 80x80 frames. The Inkscape steps taken are:

1. Trace bitmap, smooth=false, remove background=true, scans=11.
2. Simplify.

The size of the compressed result is 7.16Kb.

The fifth filmstrip is like the fourth filmstrip, but Trace bitmap has been performed with scans=10; one color-layer less than the fourth filmstrip. The resulting file can be compressed to 6.36Kb with Winzip.

An honest conclusion is that I did not beat PNG-8. However, I did beat PNG-24 with interesting results. The loss of quality, between original and result is clearly visible. But the quality of the fourth and fifth filmstrips are still quite acceptable.

I still need to investigate this method with more images. Even though it does not outperform PNG-8, it still may be an interesting method to compress vector-based images. 

The motive for smaller image size is evident: faster download time. But how low can you go? Inkscape is an OpenSource drawing program which you can download for free. It has a brilliant feature to convert bitmap images (PNG’s) to vector graphics. Silverlight used to advertise that vector graphics have higher quality as you can zoom in without quality loss. This is true, however PNG’s and vector graphics are not very well exchangeable.

I took the following steps to convert a PNG to vector graphics:

1. Import PNG from the File menu;
2. Convert it to vector graphics, Trace Bitmap in the Path menu. Select “Colors”, Scans=12, Remove Background checked. Click Update then click OK.
3. Reduce vector points, Simplify in Path menu;
4. Save as XAML, File menu;

This is the result in Inkscape:

The left is the original, the right is converted to vector graphics and Simplified only once. Now obviously, there is quite some quality loss in the vector representation. We deal with quality later, this article is about file size.

Open the resulting XAML in Visual Studio. Now go to the Edit menu, Advanced, Format Document. Remove empty tags “Resources” and “RenderTransform”. Remove the “<?xml” line at the top. This result can be copied into a new Silverlight UserControl.

If the new UserControl is now compiled, the resulting binary is larger in file size than the original PNG you started with. The reason for this is that Visual Studio does not convert XAML to binary code. If you open a Silverlight binary in for example Notepad, you will notice that a complete copy of the original XAML is somewhere within the dll. In most cases, text is larger in size than the binary equivalent.

Compression 1: text-float to binary float

The Path geometry data from your Inkscape XAML contains many floating point numbers, in text representation, while a binary float is only 4 bytes long. As these floats occupy the larger part of the Inkscape XAML, we could write a little program to convert this Inkscape result into something binary.

Well, that is what I did experimentally. And although this converter does not support the full Path data mini-language yet, the lion-part is already converted to a binary representation. And if you have the lion-part, then there is a good moment to determine whether your experiment will ever have any success: get smaller file sizes.

These are my findings:

The original PNG is 16KB. The product from Inkscape is 50KB. This will also be the size when you compile the UserControl. When zipping this to a xap will never be better than the original 16KB.

Converting the Path textual data to binary .Net floats, the product is now 22Kb (the .XZ file). When adding more features to the converter, this size will be a little larger; I estimate about 1Kb, because these features are rarely used.

The next step is to look at the size of the XZ file when zipped: 11Kb. Zipping the original PNG will result in a zipfile with the same size. So with this immature and still-under-development compression technique, we already gain 35% in filesize.

Compression 2: text-float to binary short

I also tested another technique for further compression. Float’s are 4 bytes long, while shorts are 2 bytes long. A technique learned from the past: multiply the float with a factor to gain precision, and cast it to short to discard remaining precision. Evaluation of this technique is out of scope here. The result is however that a number no longer occupies 4 bytes, but only 2 bytes.

To only know the results of this technique, I did a test, having factor = 100. These are my findings:

The XZ file, the result of my compression program, is 12KB, already smaller than the original PNG. Compressed, the XZ file is only 6KB, giving a gain of 62%.

Discussion

Now, let’s not get too excited. I have cheated this discussion on two important points. Yes I have converted the Inkscape XAML to a binary representation, but I never decompressed it. I don’t know what the results will be when I decompress the XZ file and construct a UserControl in my Silverlight application. It could turn out to look very bad. This is the next research step.

Secondly, after I converted the PNG to vector graphics, I used the Simplify function in Inkscape, to decrease the amount of floats in the resulting XAML. How far does this Simplify step affect the compression results? Well, quite a bit.

This is the result of converting PNG to vector graphics, without simplify function:

Here are the compression results for text-float to float:

And here are the compression results for text-float to short:

So when it is forbidden to use Inkscape’s simplify function, the conclusion is that the compression 1 method is not worth implementing, while the compression 2 method is worth investigating.

Ofcourse it is not forbidden to use the simplify function. You can even create graphics which anticipate this function in the art process.

Many are still looking for a solution for the Silverlight sound problem. What are the problems and what is the solution?

People who tried to create a casual game in Silverlight, encountered problems when they added sound effects to the game. The time between starting a sound effect and actually hearing it, is unacceptably long. This is called “audio latency”.

Short effects, typically less than a second, sound good the first time, while they sound bad the second time when they are played. The source of this problem is not clear, however long sounds don’t have this problem. This is called the “short sound problem”.

Also, it is very hard in Silverlight to play music in an idle loop. An idle loop with the out-of-the-box MediaElement, plays a silence every time the music is repeated. The silence at the beginning is actually caused by the same audio latency described above.

The above problems exist in Silverlight 4 and lower versions.

The audio gap
The music compression format MP3, is standard encoded with a silence at the beginning – the audio gap. This is because the decompression code needs some time to get ready, like an engine which needs time to get started. Tools like WinLame may encode an MP3 without an audio gap at the beginning, but this does no good to Silverlight’s decompression code. An MP3 created with WinLame will sound bad.

The audio buffer
Silverlight uses an audio buffer internally, which needs to be filled with audio data before any sound is played. The size of the audio buffer is measured in time, and it costs exactly that time to fill the audio buffer, before any sound is heard.

Reducing audio latency
The audio latency is caused by the audio gap from the music format and because of the audio buffer. So if we want to decrease the time between calling the Play function and actually hearing sound, we need to handle both the audio gap and audio buffer. Now obviously, a latency of 0.0 is impossible, otherwise Microsoft would have provided this with Silverlight.

The solution: VorbisPlayer
The solution is to choose for an audio format which is gapless. Second, the default length of the audio buffer must be decreased. And for a seamless idle loop, we must stream the next music part into the audio buffer, right after the previous music part has finished.

Ogg Vorbis is a music format which can be encoded without audio gap, however it is not supported by Silverlight. However, Silverlight does allow developers to implement their own audio decoder.

The VorbisPlayer provides exactly the solutions described above. More important, the VorbisPlayer is easy to use, for not so technical people.

The VorbisPlayer allows low latency for sound effects, much lower than Silverlight provides out of the box. Because of the used technique, the short sound problem is also solved. Also, the VorbisPlayer has an easy function to invoke a playlist of music, which will be played seamless without gaps in between.

The Silverlight VorbisPlayer can be found on Codeplex: http://vorbisplayer.codeplex.com/

Update 11 jun 2010.

For those interested, the VorbisPlayer has now been published on Codeplex. Find the source code here.

----

I have finished the playing part of Slengo. Next are the titlescreen, the highscore, making some levels and… adding sound effects. When I started adding the sound effects, I instantly remembered why I use little effects in my games: the slow latency.

It is like this. On the screen you push an icecube, and half a second later (or longer) you hear the sound effect. The visual and audio are so much out of sync that it breaks game-playing experience.

In Avios, I solved this problem by delaying the graphics. The event of a bomb falling into the sea works with this principle. This was possible because the sounds in Avios are not bound to user-events. They are bound to in-game events. With user-bound events, this has the drawback that the game will appear to react slowly on user input.

The delay has two causes.

1) Mp3’s are standard encoded with a gap at the beginning;
2) The MediaElement has an internal buffer which is filled before even one sound is played.

Also Silverlight has problems with short sounds (less than a second), they make the delay worse, or you don’t hear them at all.

Countering the audio gap
I have thought of two ways to deal with the gap, encoded in audio files. Either find a gapless format or gapless encoder, or skip the gap when starting the sound.
If you encode Mp3’s gapless with WinLame, then the audio quality is very bad (I’ve tested a short sound).

In Silverlight, it is possible to write your own sound decoder. Larry Olson wrote Managed Media Helpers which inspired about everybody else. What I understand from this project is that only the Mp3 frame headers are decoded, while the compressed sound data is send to the MediaElement, making use of the decoder already in Silverlight. If this is true, then this scheme is not sufficient to skip the gap programmatically, because you cannot set a start pointer in the middle of compressed data.

DDtMM wrote a library for seamless audio loop, for Mp4 encoded files. Expression Blend Encoder can encode gapless. I had many troubles downloading the library in the past (timeouts), so I was unable to investigate this code. Recently I had more luck, but being too occupied with MoonVorbis (see below) I haven’t studied the code very well yet.
Ahura Mazda wrote the Saluse Media Kit, fully decoding Mp3’s and sending the raw audio to the MediaElement. This code is a good option to skip the gap at raw-sound level. Unfortunately I had problems playing short sounds repeatedly, so I looked further.

A project which really drew my attention was MoonVorbis from Atsushi Eno. Casy Wireman wrote on GameDev that Ogg Vorbis is a pretty good format for games. It seems that Ogg Vorbis is encoded without audio gap. MoonVorbis decodes to raw sound samples. So if there is a gap we can skip it.

Countering the audio buffer

MoonVorbis is not production ready. I had problems with the pitch of my test-sound, and problems with repeatedly playing a short sound. However, I’ve worked a solution. But still, there is latency between user-event and sound being played.

I have introduced a Play function on the stream. The MediaElement’s source is set to the stream and starts playing directly. The stream produces silence when the sound is not played. When the Play function is called on the stream, it starts to send sound samples. With this approach, the short-sound problem is solved, because the standard silence extends the length of your sample.

Now to reduce audio buffer latency. The MediaStreamSource has a property called AudioBufferLength, which is measured in milliseconds. The standard setting is 1000 (1 second), making the latency very slow. If you set this property to its minimum, 15 milliseconds, the latency is much better, but the sound begins to stutter. A setting of 50ms gives a little more latency, but with good sound quality. I am still investigating whether this is the best we can get.

Further

Solving the short-sound problem is a major improvement. I would still like to reduce the sound latency a little more. However, I still have the option of delaying the graphics, but now for a much lower latency. That is an improvement. Still, I need to find out how all this works out in practice when I bring it to the game. Although I have a preference for WMA audio, Ogg Vorbis is a good enough choice for me.

Avios is a great game if you liked Wings of Fury during the nineties. It took 5 months to develop, but it isn’t very well appreciated. There are different aspects for improvement. One aspect is to get a better loading time experience.

Players of online games don’t like to wait very long for a game to load. I have seen different approaches for better a better game-loading experience.

1. Load everything all in one
The game exists of only one xap, which is loaded at once. At Mashooo and Silverarcade they’ve found a way to improve experience. Each game is loaded by a generic preloader, displaying the current percentage. This is very well done. The drawback is however that the game will be loaded completely, you cannot split it into parts. This approach cannot distract the player’s attention if the game is very large.

2. Partial preload
A preloader loads the basic data. The rest of the data is loaded while the game is playing. This is the scheme I’ve been using for most of my games. In my case, the preloader has a list of images and sound effects to be loaded. Any music which is played during the game, is played from the server during the game. Music files can be quite large and they’re not always played completely. So this approach has an advantage over loading it all at once.
You also have more control of what will be loaded in the preloaded, and what will be loaded during the game. Nokola’s Shock for example loads the most basic things to play the first level in the preloader and everything else during the game. Avios however loads all images and sound effects for all levels in the preloader.
An improvement for Avios could be to load everything to play the first level, while loading everything else when the first level is played. No problem for Avios. But in a different game, in such a scheme, the loading and playing of music could compete with loading other data. It could interfere with the music being played. So this needs to be tested.

3. Lower quality
It isn’t surprising that visuals in many games are very small. Smaller graphics reduce the data volume and give better loading experience. For the same reason, players experience lower sound quality for the same reason. Lower quality improves game load experience but may reduce playing experience. This balance is the choice.

4. Change the Encoding
Unfortunately, using vector graphics doesn’t increase quality, while data volume is reduced. I have been playing around with Inkscape 1), to convert bitmap images into vector graphics, exporting it in XAML. In many cases, where I replaced a bitmap image for a vectorized version, the data volume increased. If you want to preload such resources, you need to dynamically load a dll, instead of loading a list of resources on the server.

For my games, I usually use PNG’s. PNG supports alpha channel –  transparency. JPG’s do not support this. In Photoshop you can export JPG’s at low quality. One could use these low-quality JPG’s, assign a transparency color to it and process the images using WriteableBitmap, to support alpha channel. I have done a little study to this option, but I found the image-quality so very low that I decided it was not worth it.

There is also a gain for sound encoding. MP3’s are usually larger than WMA’s. So it does pay to encode for WMA. It is worth it to play around with the encoding settings. I was able to encode an original MP3 file of 1.8 MB, to a 491 KB WMA file, without significant loss of quality.