ModPlug Central

OpenMPT => Help and Questions => Topic started by: Waxhead on January 23, 2006, 21:03:58

Title: Feature suggestion: Create exportet wav as sparse
Post by: Waxhead on January 23, 2006, 21:03:58
If ModPlug would try to create exported wav files as sparsefiles my life would improve considerably. The air would be fresher, food would taste better, my shoes would feel more confortable and I'm pretty sure I would stop loosing my hair also :)
Title: Re: Feature suggestion: Create exportet wav as sparse
Post by: Matt Hartman on January 24, 2006, 06:45:40
Quote from: "Waxhead"If ModPlug would try to create exported wav files as sparsefiles my life would improve considerably. The air would be fresher, food would taste better, my shoes would feel more confortable and I'm pretty sure I would stop loosing my hair also :)

I do say, HeadWax, what is this, "sparse" you speak of man? I'm quite intrigued to know.

Rioght.
Title: Feature suggestion: Create exportet wav as sparse
Post by: speed-goddamn-focus on January 24, 2006, 06:53:07
I think it means that "irrelevant" data isn't written to the file and set to 0 when loading it, thus saving disk space. I'm not quite sure what would be irrelevant in a wave file tho... Then again, I probably misunderstand everything as per usual.
Title: Feature suggestion: Create exportet wav as sparse
Post by: LPChip on January 24, 2006, 08:55:21
Seems that we're not the only one. I'm wondering if Waxhead is the only one that knows what he means. :P

(Finally someone else that can say anything here that no one understands :P)
Title: Feature suggestion: Create exportet wav as sparse
Post by: rewbs on January 24, 2006, 10:03:05
Does it mean silence would take up no diskspace?
Title: Feature suggestion: Create exportet wav as sparse
Post by: Waxhead on January 24, 2006, 15:54:40
Oh sorry guys... I guess I was a bit to technical for you except for Rewbs :)

If you save a sample it will be lot's of numbers. Most often there are lot's of zero (silence) in many tracks and using a sparsefile will (very basically) save this data

123000000000000000000000000000000000000000000000004567

like this:

12304567

some sort of 0-compression. ;)

You can create sparsefiles yourself from "dos" with fsutil (native winxp app)
Title: Feature suggestion: Create exportet wav as sparse
Post by: rewbs on January 24, 2006, 16:50:31
I imagine this could be handy when rendering channels/instruments individualy, where some tracks will have long periods of silence. Out of curiosity, do you know any other applications that save wavs as sparse files? I hadn't heard the term before, so I'm guessing its not a very common practice and therefore might have drawbacks (difficult to implement? compatibility issues?)
Title: Feature suggestion: Create exportet wav as sparse
Post by: Matt Hartman on January 24, 2006, 16:50:37
Quote from: "Waxhead"Oh sorry guys... I guess I was a bit to technical for you except for Rewbs :)

If you save a sample it will be lot's of numbers. Most often there are lot's of zero (silence) in many tracks and using a sparsefile will (very basically) save this data

123000000000000000000000000000000000000000000000004567

like this:

12304567

some sort of 0-compression. ;)

This makes me wonder why it isn't like this by default? What's the advantage of having 00000000 when there's no information contained in them? There has to be a specific reason, no?
Title: Feature suggestion: Create exportet wav as sparse
Post by: Squirrel Havoc on January 24, 2006, 16:59:55
Well the 0's are part of the audio, just silence im sure. And what he's talking about is RLE (run-length encoding compression). Just zip the wave file :)
Title: Feature suggestion: Create exportet wav as sparse
Post by: Waxhead on January 24, 2006, 17:07:40
Quote from: "Squirrel Havoc"Well the 0's are part of the audio, just silence im sure. And what he's talking about is RLE (run-length encoding compression). Just zip the wave file :)

Yes or turn on compression on NTFS volumes.

But let me try to be a bit clearer. A sparsefile works in such a way that (total) silence doesn't use loads of precious disk-space. When the file is loaded the packed zero's are regenerated by the filesystem. the advantages are that a saved track takes less time to load and will use less diskspace it should also be quite easy to implement since the encoding is done by the filesystem. However this is a feature of NTFS (not FAT) and if present I can't think of any good reason not to try and save as sparsefile if the filesystem is NTFS.
Title: Feature suggestion: Create exportet wav as sparse
Post by: Waxhead on January 24, 2006, 22:11:39
Quote from: "rewbs"I imagine this could be handy when rendering channels/instruments individualy, where some tracks will have long periods of silence. Out of curiosity, do you know any other applications that save wavs as sparse files? I hadn't heard the term before, so I'm guessing its not a very common practice and therefore might have drawbacks (difficult to implement? compatibility issues?)

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/fs/sparse_files.asp
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/fs/fsctl_set_sparse.asp

As you can se the the sparsefilemanagement is done by the filesystem and is transparent to applications. The only potential drawback I can think of is that IF a file have frequent (very) short periods of scilence in it it might need some processing power to load/save... I'm unsure of this tough...

btw: when the new forum is up perhaps it would be an idea to create a "technical" forum for coders to discuss various things ;)

I know I have encountered some apps that creates sparsefiles but I Can't remeber which ones... :(
Title: Feature suggestion: Create exportet wav as sparse
Post by: Sam_Zen on January 25, 2006, 01:50:40
A few aspects are skipped here so far. The matter of Time and the question : do you want to be able to recover the original wav-file with the silences included ?
A gif-file is non-destructive, because it not only skips unnecessary zeroes, but saves the number of them.
So 123000000000000000000000000000000000000000000000004567
would become something like
123 47x0 4567
Title: Feature suggestion: Create exportet wav as sparse
Post by: Waxhead on January 25, 2006, 16:28:20
Time:
 most sparsefiles should load and save faster

Recover:
Sparsefiles are as non-destructive ;)

The only reason I suggested this was that sparsefiles are quite handy for samples. When exporting all tracks as wav and loading them in a multitrack sampleeditor (Audition (coolEdit)  for example) you will probably have a large portion of some tracks as silence. My point is why waste perfectly good disk space when just setting a flag for the NTFS filesystem will solve that problem ;)

Meanwhile let's just use NTFS compression and be happy with it :)