adpcm compression - developement stuffs

[Zenon]

Hi, I am currently developing a renoise to mod/xm format utility converter in C#.

There's already a beta release here, so my next purpose is to make a full conversion possible.

To doing this I should convert all sample data, but I'm currently stucked because I can't figure out how convert sample (wav stream data) in adpcm xm standard compression.

I read a document at ufMod but honestly I don't understand that section.

Any help, advice (even in which lib modplug uses to do this) would be really appreciated.

Thanks
Zenon

[Saga Musix]

XM samples are not ADPCM-compressed*; in fact, they are not compressed at all. However, XM samples are stored as delta values, meaning that sampling point x is saved as the difference of sampling point x-1 and x. If x = 1, the previous sampling point is assumed to be 0. So if your WAV sample looks like this:
13,26,39,26,1,0
the XM equivalent would be:
13,13,13,-13,-25,-1

In the modplug code, this is done in CSoundFile::WriteSample (Sndfile.cpp). Look for RS_PCM16D and RS_PCM8D.

* ModPlug introduced ADPCM-compressed samples to various module formats; however, those additions are neither official nor supported by a wide variety of trackers/players. And to be honest, ADPCM-compressed samples sound extremely shitty. If you simply want to convert your XRNS files to XMs, don't even think about ADPCM.

PS: Oh, and in case you're going to stick to uFMOD file description of the XM format:

Code Select Expand

38     20    char Tracker name 'FastTracker v2.00   '
Please write your own applications name there (f.e. "XRNS2XM v0.1"), not "FastTracker v2.00". That way, people know who to blame if they find weird XM files.

[Zenon]

Jojo thanks a lot for the info, I'll see what I can do.  


I tried first to put a wav without any compression but it seems modplug didn't recognize it, therefore I should try with several players (i.e. xmplay).

Btw UFMod document is my only reference, but on the sample info section at +offset 14 of sample header (loop and sample type) I have found a strange contraddiction with reality values because sometimes while reading xm files I find a 49 byte value in that offset which is not expected as described on manual (talks only of bit 0-1 and 4).

oh sorry for the excursiveness

meanwhile I'll gonna change that caption..

Greets!

[Saga Musix]

there's plenty of documentation about the XM format, check this (the only official document, many mistakes!) and this (many mistakes of the original document are explained here).

I tried first to put a wav without any compression but it seems modplug didn't recognize it

A computer only recognizes what you teach it to recognize, it can't think for itself. MPT of course expect delta-encoded values here, as I explained above.

but on the sample info section at +offset 14 of sample header (loop and sample type) I have found a strange contraddiction with reality values because sometimes while reading xm files I find a 49 byte value in that offset which is not expected as described on manual (talks only of bit 0-1 and 4).

I have a hard time understanding what you are actually trying to say here, mostly because of the lack of punctuation. Can you please clarify which parts of the sample header you mean?

[Sam_Zen]

Those two wotsit links are bad.
Maybe you could use the textfile here

[Saga Musix]

Whoops... No clue why they're invalid, but just search for "XM" on wotsit and you'll find the two docs immediately.

[Zenon]

Thanks to all,

I'll try to put value as described in those files and then I'll post if I find any problem.

Jojo, the byte I found strange was this

Type: Bit 0-1: 0 = No loop,
                                   1 = Forward loop,
                                   2 = Ping-pong loop;
                                   4: 16-bit sampledata

Because once a time I remember when I was reading a XM file and found there a value of 49, but I was using unsigned byte so I don't know if this could be the problem.

[Saga Musix]

When refering to numbers, please always use hex (49= 0x31 in your case) unless there's a good reason not to, as that's less confusing. We're talking about binary operations here, and those are a lot more obvious in the hexadecimal system than in the decimal system.

There are more unofficial XM extensions (extended module extensions, lol...) by ModPlug, for example including stereo samples. In your case, 0x31 = 0x01 | 0x10 | 0x20, that means: looped sample (0x01), 16-bit (0x10), stereo (0x20). If you want to convert XRNS to XM, this inofficial stereo addon might be interesting for you. The channels are not stored interleaved, but in the WAV format (which you most likely have as source), they are interleaved. That means that you first have to split the source data into two channels and then storing the left channel first (delta-encoded, of course) and then the right channel.

[Zenon]

another two questions..

1) For an 8 bit sample range of encoded delta values go from -255 to 255.

-255 is more than a signed byte, so encoded data length may be larger than real data.

Is it right ?

2)what for a 16bit sample?

I wonder if for a 16 bit data I shoud encode every single byte or the entire signed int value (range from -32768 to 32767 means a value encoded from -65536 to 65536).

Maybe I've completely mess up ?

[Saga Musix]

Does C# handle overflows? I hope it does not. Since the source is 8-bit, you only need 8-bit delta values, ever. If you have an extreme case like, say, 127 -> -128 (max to min), you would not do -255 but +1 - since it just wraps around. A simple way to do this is to first use an int (32bit) to determine the delta and then convert the int to an unsigned 8-bit (or 16-bit) integer.

[Zenon]

Hi, looks like my delta encoding won't work.. why?

this is the code:

//C# code
public static Stream Encode8BitWav(byte[] wav, int bit)
       {
//encoded stream wav
           MemoryStream stream = new MemoryStream(wav.Length);

           BinaryWriter writer = new BinaryWriter(stream);

           writer.Write(wav[0]);
           for (uint i = 1; i < wav.Length; ++i)
           {
               writer.Write((byte)(wav - wav[i - 1]));
           }


           return stream;
       }

// encodedWav result as a scratchy wav
// Sure it's not a programming masterpiece, but why would not work ?

[Saga Musix]

this might help:

Code Select Expand

int diff = ((int)wav[i]) - ((int)wav[i - 1]);
writer.Write((byte)diff);

Is your sample data signed or unsigned? if adding 0x80 to every wav[] value helps, then the sample data was in the wrong format (unsigned).

[Zenon]

For what I know, sample data is good (I also tried to store original byte[] wav to a file and sounds nice), furthermore according to wav specs 8bit samples are stored as signed.

I tried as you said, with 0x80 too, but nope.

I also tried with this:
for (uint i = 1; i < wav.Length; i++)
           {
               int diff = ((int)wav) - ((int)wav[i - 1]);
               if (diff < 0)
               {
                   writer.Write((sbyte)(diff));
               }
               else
               {
                   writer.Write((byte)(diff));
               }
           }

nothing again, scratchy sound

PS
Thanks for your very quick response

[Saga Musix]

if (diff < 0)
               {
                   writer.Write((sbyte)(diff));
               }
               else
               {
                   writer.Write((byte)(diff));
               }

well, that would be utterly wrong - if the actual representation was that important. if c# behaves like c++, it would not matter at all if the variable is signed or not, but i dunno if that is the case (it will overflow anyway, and the result is the same).

Is this any good?

Code Select Expand


int s_old = 0;
for (uint i = 0; i < wav.Length; ++i)
{
int s_new = wav[i];
byte s_diff = (byte)(s_new - s_old); // should be unsigned; if possible, use uint8 or uint8_t or whatever it's called in c# for more clarity.
writer.Write(s_diff);
s_old = s_new;
}


return stream;
}


[Zenon]

Here I come again.

The best result I had was adding:

if (s_diff < -128) s_diff = -128;
if (s_diff > 127) s_diff = 127;

Meanwhile, I wonder if I was wrong because starting from offset 0 I encoded header data too, and I think it's wrong.

Btw I tried to put data with all possibile solutions (only wave data encoded, all wave stream encoded, etc etc)

gosh