VS1053b - reducing general midi latency?

[seb]

Hi!

I'm working on a product using the General Midi capabilities of the VS1053b chip and I've run in to some latency issues.
I've designed a hardware module that takes both 5V gates from a modular synth system and hardware midi input to trigger the VS1053b's built in general midi synth. I've designed my own PCB and the VS1053b is set up in Real-Time MIDI mode. Everything works very well, however I have more latency than I expected.

I'm transmitting midi to the VS1053b via the hardware serial port of an Arduino Nano Every. I've measured the latency from the digital input on the arduino receiving a gate input, to the midi message output on the serial port with my oscilloscope and it's around 2-3 milliseconds. However, the time from a gate input into the Arduino until I get the first audio transient (using the midi ch. 10 drum sounds for instance) is around 13-15 milliseconds. So it seems the VS1053b is adding 10 or more milliseconds of latency. I measured the delay from the first flank of the midi message to sound output and that is 12-13 ms, so it checks out, it's the VS1053b that is adding all this latency. In normal circumstances using the chip as a standalone midi playback device, this latency does not matter. But in my application I'm triggering the sounds together with other modules in a modular synthesizer, and the difference in latency becomes noticable.

So, my question is if there is any way of reducing this latency in the chip? At this point I don't have any other means of programming the VS1053b implemented in my hardware, only the midi communication.

The VS1053b is clocked with a 12.288 MHz crystal. And I assume Earspeaker is turned off as I have put pull down resistors on GPIO 2 and 3.

Any help on this would be much appreciated.

[pasi]

The Audio output FIFO is 2048 stereo samples, 46.4ms at 44100Hz samplerate.

The vs1053b realtime MIDI mode generates 64 new samples whenever the fill state is < 512 samples (8*64), i.e. 11.6ms at 44100Hz. So, this matches your observations.

The threshold could be reduced by replacement code/patch. However, the note drop algorithm becomes more aggressive when it runs with less "tolerance". If your polyphony is low, this won't be an issue.

1. Do you have access to the vs1053b's SCI bus from your controller? (You could upload a replacement realtime MIDI code through SCI.)
2. Alternatively, you could upload the replacement code through UART using the ROM monitor. You need 9600bps to do this, and switch to 32500bps afterwards.
I assume that you can change the GPIO pull-ups to start the vs1053b in normal decoder mode.

[seb]

Hi Pasi!

Thanks for your detailed reply!

As my design is right now I haven't implemented connections to the SCI bus and the GPIO is configured with resistors on board. I treat the VS1053 as a pure MIDI controlled sound generator. So I'd have to make a new PCB revision to implement these changes. I guess if it's possible to upload the code via UART and I can store the code in available memory in my microcontroller, it would be managable to implement this as UART is already used for MIDI.

How much improvement in latency could be obtained? Say that I'd be happy with 16 polyphony, would that mean that the fill state could be set to 128 and I'd get 2.9 ms latency?

I do have the Adafruit board with the vs1053 at hand also, so I could do some tests on that before committing to a hardware change.

[pasi]

Take a look at VS1003b/VS1033c/VS1053b Real-Time MIDI Input Application from https://www.vlsi.fi/en/support/software ... tions.html . The vs1053b version contains a fixed handclap sound and a few others.

When you get that working with the Adafruit board, I can compile versions with different latencies for you to test with.

Loading code through UART is only a little bit more complex. The method we have used is to create a boot image and store it in memory, then parse it and use the ROM monitor protocol to upload the initialized data and code, the start it.

For do-once and no need to update, it's also possible to store just the UART bytes to be sent. After initializing the monitor connection (send 0xef, then wait for response), sending and executing the rtmidi program doesn't need any responses from the vs1053b.

30 minutes later: See below.

So, when powered up in normal mode (so that UART is not taken over by the RTMIDI mode), configure uart for 9600bps, send 0xef, should receive 0xa5, then send the data bytes from this file:

rtmidi1053b-uart.txt

uart bytes to send to upload rtmidi code for vs1053

(37.7 KiB) Downloaded 147 times

This sets 4.0x clock, no earspeaker, and latency 2.9ms. The comments tell how to set other value for latency. Then change UART to 31250bps for MIDI.
Disclaimer: I have not tested that this actually responds to RT MIDI.

[seb]

Thanks again for the help!

I've tried to get this working on macOS running VSIDE in a virtual machine, but ran into trouble getting serial ports to be recognized. So I think I'll have to wait until I can access a windows computer to actually test this.

However, are there any commands I can transmit to the stock firmware on vs1053b through a serial terminal to see that it responds at all? I haven't been able to get any response from it yet. I also have an oscilloscope with UART decoding and I get nothing on the TX pin of the vs1053 what ever I send it.

[Hannu]

Right click solution (not project) in solution browser and select properties. In debugging tab there is autodetect. Can it find the chip?

On OS X you can try to use the serial port. After all it is kind of BSD.

If you are comfortable on command line, you could try something like:
#set speed to 9600
stty -f /dev/THE_SERIAL_PORT 9600
#send 0xef
echo -en '\xef' > /dev/THE_SERIAL_PORT
#Not needed if tseting interactively
sleep 1
#read the answer
cat -v < /dev/THE_SERIAL_PORT

And actually if you make the program to binary, you can cat that too to serial port.

It has been a while since, I've used os x and I don't know if it has stty or the /dev for devices.

[pasi]

However, are there any commands I can transmit to the stock firmware on vs1053b through a serial terminal to see that it responds at all? I haven't been able to get any response from it yet.

The RT-Midi mode takes over the UART, but if you power up in normal mode (GPIO0 and GPIO1 low, or both high -- the chip tries SPI boot but fails), sending 0xef at 9600bps should produce the response of 0xa5 as an indicator that the chip has entered the ROM monitor.

You need to have power, xRESET high, and the crystal running, of course.