NRPN and MIDI documentation questions and issues part 3

[feijai]

[I see you're moving the messages, so I'll do the same here to save you time. :-)]

I've found a significant problem with NRPN/RPN parsing.  :-(  In my testing, it appears that the PreenFM2 cannot accept RPN messages at full speed (31500 bps).  I have only tested over USB.

If I send an NRPN message of the form: 99, 98, 6, 38

It seems to be able to parse it in time.

But if I send a properly nulled NRPN(+RPN) message, that is, one of the form: 99, 98, 6, 38, 101(127), 100(127)

Then the PreenFM2 cannot parse faster than about 1ms per message, and starts dropping MIDI bytes if I send faster than that.  :-( :-(  That's not good, it should never be dropping any MIDI bytes ever.  And it means I can't send a nulled message.

I suspect what's happening is that the PreenFM2's NRPN/RPN parser expects to see a 6 or 38 (or an INC or DEC) after the opening 99/98 or 101/100, and if it doesn't it might be freaking out.  But NRPN and RPN doesn't have to send a 6 or 38; RPN NULL is an example.  Anyway, that's a guess.

[feijai]

Another oddity.  The valid characters with which you can make a name appear to include space (' ') twice: once after 'Z' and once after 'z' when you scroll the leftmost encoder when choosing a filename.  Is this the same character and just an oddity of the UI, or are they different characters being represented in the same way?

[Xavier]

But if I send a properly nulled NRPN(+RPN) message, that is, one of the form: 99, 98, 6, 38, 101(127), 100(127)

Then the PreenFM2 cannot parse faster than about 1ms per message, and starts dropping MIDI bytes if I send faster than that.  :-( :-(  That's not good, it should never be dropping any MIDI bytes ever.  And it means I can't send a nulled message.

The preenfm2 expectes NRPN messages (not RPN).
It expects 99 and 98 to be sent.
Then :
. 6 AND 38
. or 96
. or 97
You can see the code here :
https://github.com/Ixox/preenfm2/blob/master/src/midi/MidiDecoder.cpp#L550

Don't hesitate to submit patch if you modify the code and think you improved the behaviour.

Thanks,

Xavier

[feijai]

The preenfm2 expectes NRPN messages (not RPN).

Looking through the code solves the mystery of why Arpeggiator Pattern and Arpeggiator Direction aren't working.  And it's not a good result.  :-( :-(

The CC for these two (CC_ARP_PATTERN, CC_ARP_DIVISION) is 100 and 101, the two values for RPN.  This means that if I send an RPN NULL, the PreenFM2 is presently interpreting it as changing the ARP pattern and ARP division (to 127!).

This is a problem.  The industry best practice for NRPN is to send an NRPN message and then terminate it with an RPN NULL to make sure it's closed.  See here for an example:  http://www.philrees.co.uk/nrpnq.htm   Various controllers (including ones I've built) follow this practice.  This means that the PreenFM2 can't be used with these controllers.  If a synth handles NRPN, it also has to handle RPN at least enough to deal with RPN NULL.

There are some other problems with the NRPN handler: it's not properly parsing NRPN messages.  The PreenFM2 requires you to submit 38 (LSB) even though that's not an NRPN requirement.  Also you can't send a 38 first, then a 6 (which is also valid).   And it doesn't handle running status, so you can't send a stream of 6 or 38.  And I think you can interrupt an NRPN message with another random CC and it doesn't invalidate the NRPN message.

The Preen is presently doing this:

Code: [Select]

If I received a 99 then update paramMSB
Else if I received a 98 then update paramLSB
Else if I received a 6 then update MSB
Else if I received a 38 then
    update LSB
    Do(current param, current value)
Else if I receive anything else then
    process it (but don't clear out the paramMSB, paramLSB, MSB, or LSB!)

The problem is that this is not stateful, so it has some problems.  My C++ foo is not good but I might try to submit a patch which could fix things (it probably won't compile but it would give you an idea of what to do).  I probably won't be able to enable running status or allowing just a 6 or 38, because in order to do that you have to change the preen's parameters every time a 6 *or* 38 is received, not wait for both of them.

In general, if you don't want your device to have to pause to wait for an (optional) additional 6 or 38, then NRPN works as follows.

Code: [Select]

If I received a 99 then
    If I received a 98 then
        MSB = 0, LSB = 0
        Loop
            If I received a 6 then
                Update MSB
                Do(param, MSB, LSB)
            Else if I received a 38 then
                Update LSB
                Do(param, MSB, LSB)
            Else if I received anything else then
                push back into the CC stream
                break from loop
    Else if I received anything else then
        push back into the CC stream

This pseudocode is easy but requires you to do partial updates with just the MSB or just the LSB (unfortunately, NRPN permits this).  Your other option is to wait for a little bit to see if the LSB is coming next and then update, but this could slow down NRPN parsing slightly.

Some controllers send both LSB and MSB, but others (like the BeatStep series) only send LSB or only send MSB.  And the order is not guaranteed.  I think what should be done is to expect both MSB and LSB, but then continue to update single LSB or MSB updates on running status.  And if you want to do NRPN, I think you really HAVE to reserve 100 and 101 for RPN or it won't work with certain controllers.  This means moving or deleting arpeggiator direction and pattern.

This still doesn't solve the mystery of why sending CC 101 or CC 100 causes the PreenFM2 to slow down (a lot -- like 1 MS).

[Xavier]

From what i understand with NRPN, the device chose for each of its parameter whether CC38 is required.
Once you know for one parameter whether it must be sent (or not), you have to send it (or not).

The preenfm2 always expect CC38 to be sent. That's simple

I don't know what you send to CC 100 and 101, so it's difficult to guess why that makes your preenfm2 slow down.

Any patch are welcome, i don't guarantee i'll put it in the official firmware though

[feijai]

Well, NRPN has always been very, very vague in the spec.  But here's the rules regarding receiving machines (see MIDI 1.0 Detailed Specification 4.2, page 17).

"4. The receiver should be able to respond accordingly if the transmitter sends only an LSB or MSB to change the parameter number.  However, since the transmitter can't know when reception was enabled on the receiver which will be waiting for both the LSB and MSB (at least initially), it is recommended that the LSB and MSB be sent each time a new parameter number is selected."

and

"5. Once a new Parameter Number is chosen, that parameter retains its old value until a new Data Entry, Data Increment, or Data Decrement is received"  [I think this is interpreted as also permitting NRPN running status -- that is, a stream of data entry values -- though it's vague]

I think that this means that receivers are asked to be able to handle single LSBs, single MSBs, and LSB+MSB pairs (it doesn't specify the order).

Given that the PreenFM2's *primary mechanism* for control is NRPN, I'd STRONGLY recommend revisiting this to get it right.  The big issue is what happens if only a CC6 (or stream of CC6) is sent -- this is a common case and makes perfect sense.  For example, if I have a BeatStep and I want to control the Preen's Operator Modulation Indices, the BeatStep can *only* send either CC6 *or* CC38.  CC6 lets me change the indexes by a significant amount, CC38 does not.  So that means I can't really use a BeatStep with the PreenFM2 over NRPN.

Also, the inability to close properly with an RPN 127/127 is going to cause problems with a number of utility devices which were written to assume this.

I'll try to suggest a protocol that might work.  But it will almost certainly involve freeing up CC100 and CC101.