text-based tracker formats support

Started by nitrofurano, June 22, 2017, 14:23:26

Previous topic - Next topic

nitrofurano

since i can't use or create an account in the issue tracker, i guess i can start this thread here

it's known that most trackers, OpenMPT included, supports binary tracker files, like .mptm, .mod, .s3m, .xm and .it

but there are some other few trackers, like VortexTracker that supports txt-based formats

and as far as i know, it seems to be not that hard having equivalent txt files to binary .mptm, .mod, .s3m, .xm and .it formats, like .mptm.txt, .mod.txt, .s3m.txt, .xm.txt and .it.txt

an example of a .mod.txt , with 8 channels, would be like https://pastebin.com/raw/LxPCZkgu - as seen from there, the samples are stored as hexdumps at the file bottom, as we have from the binary format (i'm mentioned it here because i heard from some people that it is "impossible" to have txt-based tracker formats because of the samples, which is not true as we can see...)

and here an example of the txt format used on VortexTracker: http://pastebin.com/raw/hYVm4AdY

i'm really curious for comments related to this (specially if such feature is indeed going to be supported), and thanks in advance

Saga Musix

Is it really necessary to create this thread on several forums?

Quotespecially if such feature is indeed going to be supported
No. You are practically the only person who ever asked for a text-based format because they want to edit module files in a text editor. If you want to edit your songs in a text editor, you can make up your own format and write a converter between that format and MOD/XM/S3M/IT/whatever.

Quoteand as far as i know, it seems to be not that hard having equivalent txt files
Designing any good file format is hard (no matter if text-based or not). And there is no good reason to design yet another broken format.
In particular, text formats are quite often difficult and slow to parse, and once there is more than one application using such a format, it's only a matter of time until deviations of the format show up which are only supported by one program but not others, etc. (just look at the sfz format, and then try writing a parser for it).
» No support, bug reports, feature requests via private messages - they will not be answered. Use the forums and the issue tracker so that everyone can benefit from your post.

nitrofurano

#2
Quote from: Saga Musix on June 22, 2017, 19:38:16
Is it really necessary to create this thread on several forums?
i wouldn't ask if it wasn't! :o)


Quote
Quotespecially if such feature is indeed going to be supported
No. You are practically the only person who ever asked for a text-based format because they want to edit module files in a text editor. If you want to edit your songs in a text editor, you can make up your own format and write a converter between that format and MOD/XM/S3M/IT/whatever.
would be great if we could have some scripting support, like running lua or python scripts, for that, as plugins (like you develop and run picture filters for GrafX2, for example) - we would run these converters inside OpenMPT, and that would feel almost like it was part of OpenMPT! ;)

Quote
Quoteand as far as i know, it seems to be not that hard having equivalent txt files
Designing any good file format is hard (no matter if text-based or not). And there is no good reason to design yet another broken format.
In particular, text formats are quite often difficult and slow to parse, and once there is more than one application using such a format, it's only a matter of time until deviations of the format show up which are only supported by one program but not others, etc. (just look at the sfz format, and then try writing a parser for it).
i confess i need to do my part on keep testing and improving it, thanks for the heads up reminding! ;)

Saga Musix

Quote from: nitrofurano on June 22, 2017, 21:09:47
would be great if we could have some scripting support, like running lua or python scripts, for that, as plugins (like you develop and run picture filters for GrafX2, for example) - we would run these converters inside OpenMPT, and that would feel almost like it was part of OpenMPT! ;)
There will be scripting support eventually, but don't ask when that is going to be (I've had it on my nice-to-have list for 7 years now). Designing a good scripting API is also hard. ;)
» No support, bug reports, feature requests via private messages - they will not be answered. Use the forums and the issue tracker so that everyone can benefit from your post.

manx

Quote from: nitrofurano on June 22, 2017, 21:09:47
Quote from: Saga Musix on June 22, 2017, 19:38:16
Quotespecially if such feature is indeed going to be supported
No. You are practically the only person who ever asked for a text-based format because they want to edit module files in a text editor. If you want to edit your songs in a text editor, you can make up your own format and write a converter between that format and MOD/XM/S3M/IT/whatever.
would be great if we could have some scripting support, like running lua or python scripts, for that, as plugins (like you develop and run picture filters for GrafX2, for example) - we would run these converters inside OpenMPT, and that would feel almost like it was part of OpenMPT! ;)

The essential key ingredient for featuers like scripting support or text module export/import is NOT the fact that the representation is text. The essential bit here is that the information is STRUCTURED.
The same thing applies to the module diffing functionality suggested here: https://forum.openmpt.org/index.php?topic=5481.0 .


Quote from: nitrofurano on June 22, 2017, 21:09:47
Quote from: Saga Musix on June 22, 2017, 19:38:16
Quoteand as far as i know, it seems to be not that hard having equivalent txt files
Designing any good file format is hard (no matter if text-based or not). And there is no good reason to design yet another broken format.
In particular, text formats are quite often difficult and slow to parse, and once there is more than one application using such a format, it's only a matter of time until deviations of the format show up which are only supported by one program but not others, etc. (just look at the sfz format, and then try writing a parser for it).
i confess i need to do my part on keep testing and improving it, thanks for the heads up reminding! ;)

Also, something like JSON or XML is essential here, in order to at the very least have at the very least the syntax of the format consistent and defined.

Quote from: Saga Musix on June 22, 2017, 21:16:00
Quote from: nitrofurano on June 22, 2017, 21:09:47
would be great if we could have some scripting support, like running lua or python scripts, for that, as plugins (like you develop and run picture filters for GrafX2, for example) - we would run these converters inside OpenMPT, and that would feel almost like it was part of OpenMPT! ;)
There will be scripting support eventually, but don't ask when that is going to be (I've had it on my nice-to-have list for 7 years now). Designing a good scripting API is also hard. ;)

I think, designing the data structure is actually harder than designing the API here. In the case of modules, the overall structure is pretty clear though. Nevertheless, one must be cautious to avoid redundant or ambiguous data (like some module formats have internally).