"CC #38 is optional"... have at least 2 meanings.
The term "optional" can have a lot meanings. But as the terms "should" and "should not" in RFCs do not have the "usual" ("common sense") meaning, it is with other standards as well. It is not about what a normal human being would think about the meaning of "optional" but what was defined in the technical papers and why.
Is it the device choice to decide if it needs it or not and write it down in its midi implementation doc? or does the device should be able to accept without CC#38 in any case.
Is it "the devices choice" weather it needs CC#38 or ot?
Clearly: No.
MIDI-standard defines the term "optional" in the regard of control-messages like this: The MIDI-device shall still work without any flaws, but with reduced precision, even if the message for setting the LSB never arrives.
So, "optional" in this context means: It's upon anyone (including, but not exclusively to the playback-routine(!) producing the MIDI-data) to decide, that currently there is "just" not enough time to send it in full lenght and so it's LSB-value is left out to gain some time. All general controler values shall be defined accordingly.
The later doesn't make any sense.
Sorry, but it does. I just have given a meaningful senario for it. The reason for the irritation might be what a "controller" in terms of MIDI-spec really is: It's the value of a physical device like a rotary-encoder, a poti, a switch or a multi-switch. It's clearly not intendet as a general-purpose comms for the device.
How does the device know if a last LSB CC#38 will arrive after a CC#6 ?
It can't "know" and it should neither be dependend of nor wait for it. In terms of software development: Access to MSB and LSB are not atomic. They are thought to be absolutely independent from each other. And yes, this will produce artifacts. This is why it is not meant as a method of precise data-comm with the device.
It has to wait for the following byte ?
No. If the command for setting the MSB arrives, it sets the MSB, leaving the LSB untouched. If the command for setting the LSB arrives it sets the LSB, leaving the MSB untouched. It is not atomic. It can and it will contain garbage in between. It is not suited for precise data-communication with the device. That's what SYSEX is for...
The protocol in pseudo-code (for the sender side) including some errors or weirdo-things and how to deal wih them from the device-side is as follows:
set( NRPN_REG_MSB, 10 ); // set the MSB of the register-address. Block access to data-entry.
set( NRPN_REG_MSB, 0 ); // set the MSB of the register-address. Block access to data-entry. Overrides previous MSB-set
set( DATA_ENTRY_MSB, 10 ); // plainly ignored...
set( DATA_ENTRY_LSB, 0 ); // ignored as well...
set( NRPN_REG_LSB, 2 ); // set the LSB of the register-address. Release access to data-entry.
set( DATA_ENTRY_LSB, 10 ); // set the LSB to 10 whatever is in MSB... weird ... but OK...
sendOtherDataMaybeNoteOnOffs/(); // not a good idea but allowed
set( DATA_ENTRY_MSB, 0 ); // set MSB to 0, leaving LSB at 10
set( DATA_ENTRY_MSB, 10 ); // OK,... set the MSB again...
set( DATA_ENTRY_MSB, 9 ); // OK,... and again...
set( DATA_ENTRY_MSB, 8 ); // OK,... and again...
sendOtherDataMaybeNoteOnOffs/(); // Perfectly OK to do this
set( DATA_ENTRY_MSB, 7 ); // OK,... set the MSB again...
set( DATA_ENTRY_MSB, 6 ); // OK,... and again...
sendOtherDataMaybeNoteOnOffs/(); // Perfectly OK
set( DATA_ENTRY_LSB, 8 ); // weird ... but still OK...
set( DATA_ENTRY_LSB, 9 ); // even more weird ... but still OK...
set( DATA_ENTRY_LSB, 10 ); // even more, more weird ... but still OK...
...
What if none arrive ?
Then LSB is untouched. Whatever was in before...
How long to wait before considering it will never arrive ?
There is no passive or active waiting for setting Data-Entry-LSB. Realtime protocols simply do not permit this in different messages. I say it again: Access to MSB and LSB is completely indepentent from each other.
Latency will quick become unacceptable.
As there is no waiting...
If the value is between 0 and 200. Is CC#38 mandatory ?
No. According to MIDI-spec, there is not a single allowed situation, where CC#38 is mandatory.
What if a device has a new version, and this new version accepts value between 0 and 150 although the previous one had a 100 upper limit. Changing the meaning of the Data MSB break all compatibility.
I have not heard that a Montage is compatible to a DX-7 within the NRPN range of settings... Or that a Korg E2S is compatible to an Korg ES.
NRPNs only makes sense if that's the device (the preenfm) in its spec that decide how the DATA (LSB/MSB) must be sent and if CC#38 is used.
No... NRPNs are still "general controllers" in a MIDI-sense. Just like volume, panning and/or filter-cutoff. They may also carry the information "switch is in position Nr 0...127" (an no, 0...16129 is NOT allowed in this case, as you cannot prevent illegal intermediate switch-positions due to non-atomic access to LSB/MSB).
As you can guess, i disagree with the title of this thread
Xavier, it's your synth, so it's your decision. I have found a work-around which is not nice but works for me.
And I owned (and still own) "some" other midi-devices with grossly more serious glitches/flaws/bugs in their MIDI implementation. Most of the time this was done due to the precise and understandable wording in the MIDI spec (which in it's original 1.0 form excluded dealing with any corner-case...) leaving the manufacturers (too) much of room for own creativity. But also, sometimes breaking the standard was done by intention.
As you can guess, for me it is a bug in the PreenFM's firmware, still. I could tell you what I would do about it, but I can't tell what you will do about it -- or IF you will do something about it at all.
best,
Stefan
PS: Do I understand it correctly, that the PreenFM2 does actively wait for the LSB in this case? If so, what will happen if it never arrives? Maybe because the DAW died and was restarted?
PPS: If I had to fix it: I would use SYSEX for things which must be changed in an atomic way. I would keep using full-scale (MSB+LSB) NRPNs only for parameter-changes where small audible "glitches" are allowed. For that I would upscale the range (say you have a range of [-1; +22.3], maybe?) to the full 14 bits, so "leaving" off the LSB makes it less precise but it would still work...
