Feature suggestion: Create exportet wav as sparse

Waxhead · January 23, 2006, 21:03:58

If ModPlug would try to create exported wav files as sparsefiles my life would improve considerably. The air would be fresher, food would taste better, my shoes would feel more confortable and I'm pretty sure I would stop loosing my hair also

Matt Hartman · January 24, 2006, 06:45:40

Quote from: "Waxhead"If ModPlug would try to create exported wav files as sparsefiles my life would improve considerably. The air would be fresher, food would taste better, my shoes would feel more confortable and I'm pretty sure I would stop loosing my hair also

I do say, HeadWax, what is this, "sparse" you speak of man? I'm quite intrigued to know.

Rioght.

speed-goddamn-focus · January 24, 2006, 06:53:07

I think it means that "irrelevant" data isn't written to the file and set to 0 when loading it, thus saving disk space. I'm not quite sure what would be irrelevant in a wave file tho... Then again, I probably misunderstand everything as per usual.

LPChip · January 24, 2006, 08:55:21

Seems that we're not the only one. I'm wondering if Waxhead is the only one that knows what he means.

(Finally someone else that can say anything here that no one understands )

rewbs · January 24, 2006, 10:03:05

Does it mean silence would take up no diskspace?

Waxhead · January 24, 2006, 15:54:40

Oh sorry guys... I guess I was a bit to technical for you except for Rewbs

If you save a sample it will be lot's of numbers. Most often there are lot's of zero (silence) in many tracks and using a sparsefile will (very basically) save this data

123000000000000000000000000000000000000000000000004567

like this:

12304567

some sort of 0-compression.

You can create sparsefiles yourself from "dos" with fsutil (native winxp app)

rewbs · January 24, 2006, 16:50:31

I imagine this could be handy when rendering channels/instruments individualy, where some tracks will have long periods of silence. Out of curiosity, do you know any other applications that save wavs as sparse files? I hadn't heard the term before, so I'm guessing its not a very common practice and therefore might have drawbacks (difficult to implement? compatibility issues?)

Matt Hartman · January 24, 2006, 16:50:37

Quote from: "Waxhead"Oh sorry guys... I guess I was a bit to technical for you except for Rewbs

If you save a sample it will be lot's of numbers. Most often there are lot's of zero (silence) in many tracks and using a sparsefile will (very basically) save this data

123000000000000000000000000000000000000000000000004567

like this:

12304567

some sort of 0-compression.

This makes me wonder why it isn't like this by default? What's the advantage of having 00000000 when there's no information contained in them? There has to be a specific reason, no?

Squirrel Havoc · January 24, 2006, 16:59:55

Well the 0's are part of the audio, just silence im sure. And what he's talking about is RLE (run-length encoding compression). Just zip the wave file

Waxhead · January 24, 2006, 17:07:40

Quote from: "Squirrel Havoc"Well the 0's are part of the audio, just silence im sure. And what he's talking about is RLE (run-length encoding compression). Just zip the wave file

Yes or turn on compression on NTFS volumes.

But let me try to be a bit clearer. A sparsefile works in such a way that (total) silence doesn't use loads of precious disk-space. When the file is loaded the packed zero's are regenerated by the filesystem. the advantages are that a saved track takes less time to load and will use less diskspace it should also be quite easy to implement since the encoding is done by the filesystem. However this is a feature of NTFS (not FAT) and if present I can't think of any good reason not to try and save as sparsefile if the filesystem is NTFS.

Waxhead · January 24, 2006, 22:11:39

Quote from: "rewbs"I imagine this could be handy when rendering channels/instruments individualy, where some tracks will have long periods of silence. Out of curiosity, do you know any other applications that save wavs as sparse files? I hadn't heard the term before, so I'm guessing its not a very common practice and therefore might have drawbacks (difficult to implement? compatibility issues?)

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/fs/sparse_files.asp
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/fs/fsctl_set_sparse.asp

As you can se the the sparsefilemanagement is done by the filesystem and is transparent to applications. The only potential drawback I can think of is that IF a file have frequent (very) short periods of scilence in it it might need some processing power to load/save... I'm unsure of this tough...

btw: when the new forum is up perhaps it would be an idea to create a "technical" forum for coders to discuss various things

I know I have encountered some apps that creates sparsefiles but I Can't remeber which ones...

Sam_Zen · January 25, 2006, 01:50:40

A few aspects are skipped here so far. The matter of Time and the question : do you want to be able to recover the original wav-file with the silences included ?
A gif-file is non-destructive, because it not only skips unnecessary zeroes, but saves the number of them.
So 123000000000000000000000000000000000000000000000004567
would become something like
123 47x0 4567

Waxhead · January 25, 2006, 16:28:20

Time:
most sparsefiles should load and save faster

Recover:
Sparsefiles are as non-destructive

The only reason I suggested this was that sparsefiles are quite handy for samples. When exporting all tracks as wav and loading them in a multitrack sampleeditor (Audition (coolEdit) for example) you will probably have a large portion of some tracks as silence. My point is why waste perfectly good disk space when just setting a flag for the NTFS filesystem will solve that problem

Meanwhile let's just use NTFS compression and be happy with it