Loading Microsoft RIFF Palette (pal) files with C#

At the start of 2014, I published an article describing how to read colour palettes from BBM/LBM files. At the end of that article I noted that Microsoft palette files used a similar format, but I didn't investigate that at the time. Since then I followed up with articles on reading and writing Adobe's Color Swatch and Color Exchange format files and I also posted code for working with JASC, Gimp and a couple of other palette formats.

Now, finally, I decided to complete the collection and present an article on reading Microsoft's palette files. These files are RIFF forms containing colour data, similar to a BBM palette being an IIF form.

The RIFF File Format

The Resource Interchange File Format (RIFF), a tagged file structure, is a general specification upon which many file formats can be defined. The main advantage of RIFF is its extensibility; file formats based on RIFF can be future-proofed, as format changes can be ignored by existing applications.

The above paragraph is taken verbatim from the Multimedia Programming Interface and Data Specifications 1.0 document co-produced by Microsoft and IBM around the time of Windows 3.0.

The RIFF format shares allows different file types to use the same underlying structure. For example, as well as the palettes we'll cover in this article, Wave audio (.wav) files are RIFF forms as are some MIDI (.mid) and device independent bitmap (.dib) files.

A RIFF form is comprised of chunks of data tagged with an ID and a size. Some chunk types are globally defined and can apply to all resource types, while others are resource specific. Global tags include the ability to specify meta data, such as artist information or to specify language options such as a character set.

The screenshot below shows the structure of a Wave file containing fmt and data chunks and then a list of meta tags. Notice how the meta tags are in upper-case but the fmt and data tags are in lower-case. By convention, RIFF suggests that global tags used by more than one form type are in upper-case, whilst those specific to a single form type are in lower-case. An ISFT tag in a Wave file means exactly the same thing as an ISFT tag in a palette file, but the Wave's data tag does not correspond with a palettes data tag.

Chunks are word-aligned, so if the size of a chunk is odd, an extra padding byte must be added at the end of the chunk. Note that the chunk size does not include this alignment byte, so you must manually check if the size is odd and handle this accordingly.

The nature of the chunk format means a program can scan a file, process the chunks it recognises, and ignore those it doesn't with relative ease.

Most of the binary formats I've previously covered use big-endian ordering (including the original EA IFF 85 Standard for Interchange Format Files that RIFF is derived from), however RIFF is a noticeable exception as it uses little-endian (which the spec refers to as Intel byte-ordering. There is a counterpart format, RIFX that uses big-endian (referred to as Motorola byte-ordering). I don't think I've ever come across this variant, so I won't be covering it in this article.

A more advanced version of RIFF exists which makes use of compound elements and content tables, but that is also far out of the scope of this article.

Obtaining the specification

Unless you happen to have a hard-copy of the book lying around, you can get an electronic version from Nicholas J Humfrey's WAVE Meta Tools page.

About the RIFF Palette

There are actually two variants of RIFF palettes, simple and extended. As I've only come across simple palettes in the wild, this article will concentrate only on the former.

If anyone does have extended versions, please let me know, would be interesting to test these.

The simple format is an array of RGB colours, easily earning the simple moniker.

The extended variant includes extra header data describing how the palette should be used, and can include either the basic RGB palette, or palettes using YUV or XYZ colour data.

The following form-specific chunk types are supported

The screenshot below shows a basic palette file loaded into a chunk viewer. Unlike the Wave screenshot above, only a single format-specific tag is present.

Reading a RIFF file

Reading the form type

The header of a RIFF file is 12 bytes comprised of the following information

• Four bytes containing the signature RIFF
• 32-bit unsigned integer which contains the size of the document
• Four bytes containing the form type, for example WAVE or MIDI

The form type for a palette is PAL. As this is less than four characters, it is padded with trailing spaces to make up the difference.

We can test to see if a file is a valid RIFF form using code similar to

stream.Read(buffer, 0, 12);
if (buffer[0] != 'R' || buffer[1] != 'I' || buffer[2] != 'F' || buffer[3] != 'F')
{
  throw new InvalidDataException("Source stream is not a RIFF document.");
}

if (buffer[8] != 'P' || buffer[9] != 'A' || buffer[10] != 'L' || buffer[11] != ' ')
{
  throw new InvalidDataException("Source stream is not a palette.");
}

In the above example, I'm ignoring the size read from the header. If you wanted to perform some extra validation, you could always check the read value against the size of the file you are processing - the read value should match the file size, minus 8 bytes to account for the RIFF signature.

I'm also comparing each byte to a character as that is more readable, but you could always treat the 12 bytes as 3 unsigned 32-bit integers and compare the numbers - 1179011410 for RIFF and and 541868368 for PAL (don't forget the trailing space!).

if (buffer.ToInt(0) != 1179011410)
{
  throw new InvalidDataException("Source stream is not a RIFF document.");
}

if (buffer.ToInt(8) != 541868368)
{
  throw new InvalidDataException("Source stream is not a palette.");
}

Not quite as readable and so I'll just stick with looking at the individual characters.

Reading the chunks

Although most palettes probably only contain the data chunk, additional chunks (such as meta data) could be present, and I have seen some RIFF files where custom chunks were present before the main data. For this reason, I'm not going to blindly assume that the palette is the first chunk and will iterate over each one searching for palette data.

In a RIFF file, a chunk is identified by a four byte character code, followed by a 32-bit unsigned integer describing the size of the data. This means we can read the 8 byte header, decide if we support the chunk or not, and if we don't we can simply skip over the number of bytes identified by the size.

while (!eof)
{
  if (stream.Read(buffer, 0, 8) == 8)
  {
    chunkSize = buffer.ToInt(4);

    // see if we have the palette data
    if (buffer[0] == 'd' && buffer[1] == 'a' && buffer[2] == 't' && buffer[3] == 'a')
    {
      // we have a RGB palette, process the data and break

      if (stream.Read(buffer, 0, chunkSize) != chunkSize)
      {
        throw new InvalidDataException("Failed to read enough data to match chunk size.");
      }

      // TODO: Extract palette from the buffer

      eof = true;
    }
    else
    {
      // not the palette data? advance the stream to the next chunk

      // advance the reader by a byte if the size is an odd number
      if (chunkSize % 2 != 0)
      {
        chunkSize++;
      }

      stream.Position += chunkSize;
    }
  }
  else
  {
    // nothing to read, abort
    eof = true;
  }
};

Reading the palette

Once you have the chunk data, this needs converting into something usable. For an RGB palette, the data is actually a LOGPALETTE structure containing an array of PALETTEENTRY values. While this probably means there's a cool way of converting that byte data directly into a LOGPALETTE, we'll construct a Color[] array manually.

typedef struct tagLOGPALETTE {
  WORD         palVersion;
  WORD         palNumEntries;
  PALETTEENTRY palPalEntry[1];
} LOGPALETTE;

typedef struct tagPALETTEENTRY {
  BYTE peRed;
  BYTE peGreen;
  BYTE peBlue;
  BYTE peFlags;
} PALETTEENTRY;

If you want more information on Windows data types, you can find it on MSDN, but suffice to say WORD is a 16-bit unsigned integer, BYTE is as named, and DWORD is an unsigned 32-bit integer.

Reading the palette is therefore as easy as pulling out the number of colours and processing the bytes for each colour.

Color[] palette;
ushort count;

count = buffer.ToInt16(2);
palette = new Color[count];

for (int i = 0; i < count; i++)
{
  byte r;
  byte g;
  byte b;
  int offset;

  offset = (i * 4) + 4;
  r = buffer[offset];
  g = buffer[offset + 1];
  b = buffer[offset + 2];

  palette[i] = Color.FromArgb(r, g, b);
}

Although I included the PALETTEENTRY structure definition above, I thought it was worth pointing out - each palette entry is comprised of four bytes, but the fourth byte is not an alpha channel, it is a set of flags describing how Windows should process the palette.

And that's pretty much all you need to handle reading a RIFF palette file, although as usual I've included a sample application for download.

Update History

• 2017-02-18 - First published
• 2020-11-22 - Updated formatting

Like what you're reading? Perhaps you like to buy us a coffee?

Files

• RiffPaletteLoader.zip (15.41 KiB)