VS1005 4-channel HiRes Recorder

[Henrik]

Hello!

Something I forgot from the previous message: here is an example config.txt file for setting up the system for the 4-channel HiRes Recorder:

Code: Select all

[0]
# Default configuration, no buttons pressed during boot.
# Sets up audio drivers and clock, then starts HiRes Recorder..
#
# Set clock to a higher value so that even the more difficult codecs can play.
RUN SETCLOCK -l100 80
# Start SD card as device D
SDSD23 D
# Start UART in/out driver
UARTIN
# New 2015 audio DAC out driver
AUODAC s
# Enough buffer for HiRes Recorder to work
RUN AUOUTPUT -s4096
# New 2015 audio ADC in driver
AUIADC s 48000 line1_1 line1_3
# Enough buffer for HiRes Recorder to work
RUN AUINPUT -s4096
# S/PDIF automatic out, parameter can be either 48000 or 96000 (default)
AUOSPDA 96000
# 1.77" LCD driver
LCD177
# HiRes Recorder
RUN HIRESREC
# Start the shell if HiRes Recorder is exited
S:SHELL.AP3

Kind regards,
- Henrik

[Henrik]

Hello!

I am extremely pleased to announce that the HiRes Recorder now also supports 4-channel 96 kHz 24-bit PCM recording. Now it is possible to create a true professional quality multichannel recorder using the VS1005! Documentation has also been updated accordingly.

4-channel recording can be tested for example using the VS1005 Developer Board expanded with VS1005 Dev Board Extension 1.
http://wwwi.vlsi.fi/en/support/evaluati ... sion1.html

lcd4ch.png (9.34 KiB) Viewed 12136 times

4-CHANNEL 96 kHz RECORDING CONSIDERATIONS

DESIGNING A DEVICE THAT RECORDS AT 96 kHz

The VS1005 Dev Board Extension 1 board contains AsahiKASEI's AK5720 analog to digital converter.

As is the case with many other ADCs, AK5720 requires a 24.576 MHz MCLK to be able to properly sample 96 kHz. VS1005 is capable only capable of outputting 12.288 MHz through its MCLK pin. While still being fully able to communicate with VS1005 through 96 kHz 32-bit I2S, AK5720 will only sample at 48 kHz, outputting each sample two times. So, while communication and recording is technically 96 kHz, for the two channels coming from AK5720, the quality will only be 48 kHz equivalent.

To design a device that would have full-quality 96 kHz input, there are two options:

Option 1:
The external ADC must be able to sample 96 kHz at 128fs (12.288 MHz clock), or...

Option 2:
The device must have a 24.576 MHz crystal which is fed directly to the ADC's MCLK pin. This clock is also divided by two, and the resulting 12.288 MHz frequency is fed to VS1005. Both clocks MUST be from the same source so as to guarantee that they stay in sync. Then, when the ADC is configured for 256fs operation it can digitize at 96 kHz. If 48 kHz is needed as an option, it must be possible to set the ADC to 512fs mode.

MEMORY CARD SPEEDS

When recording at full quality (4 channels, 96 kHz, 24 bits), the resulting data bandwidth is approximately 9.2 Mbit/s, or 1125 KiB/s. While SD and microSD cards guarantee an average write speed way faster than this, every few seconds they pause briefly to do internal bookkeeping. These write freezes that can last up to several hundred milliseconds are a fundamental feature of modern SD and microSD cards, and they are a real issue when trying to record real-time data.

To survive these bookkeeping pauses, the VS1005 Dev Board Extension 1 board contains a 4 Mbit S-RAM buffer chip VS23S040. This memory is used as a temporary buffer (by the SDSD23.DL3 driver) when data cannot immediately be written to the SD card. This ensures an uninterrupted data flow as long as no more buffer space is needed than what is offered by VS23S040.

However, there are some memory cards that pause for so long that uninterrupted recording is not possible. In this case the current software shows an error message on the LCD. Also an "n samples lost" report is sent to the UART every second. If this error message occurs, you should try another type of SD card, preferably from a well-known high-quality vendor.

Of the SD and microSD cards tested by VLSI, Transcends, Lexars and Sandisks in the 8 - 256 GB range have consistently performed well. Some Kingston cards have also worked well, while some have had speed issues at maximum recording parameters. There have also been some issues with brandless cards. All cards VLSI have tested have worked well if data flow is halved from maximum, i.e. recording is done either at 2 channels 96 kHz 24 bits, or 4 channels 48 kHz 24 bits. VLSI cannot, of course, guarantee the suitability or speed of any SD card.

To see how close you are to the recording limits, you can follow the UART output of VS1005 while recording. Every second you will see a following report (the report below is from recording that has been going on for over 20 hours at 4 channels 96 kHz 24 bits to a high-quality 256 GB SDXC card:

Code: Select all

D:AUDIO/AUD02195.WAV: 78485s,    0 samples lost,  11/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78486s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78487s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78488s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78489s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78490s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78491s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78492s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78493s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78494s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78495s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78496s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78497s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78498s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78499s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78500s,    0 samples lost,  41/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78501s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78502s,    0 samples lost,   8/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78503s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78504s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78505s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78506s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78507s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78508s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78509s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78510s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78511s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78512s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78513s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78514s,    0 samples lost,  13/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78515s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78516s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78517s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78518s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78519s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78520s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78521s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78522s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78523s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78524s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78525s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78526s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78527s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78528s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78529s,    0 samples lost,  42/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78530s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78531s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78532s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78533s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78534s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78535s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78536s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78537s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78538s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78539s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78540s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78541s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78542s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78543s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78544s,    0 samples lost,  13/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78545s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78546s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02195.WAV: 78547s,    0 samples lost,   7/512 KiB buffer used

Because this is a very good card, the maximum amount of buffer space used during this example is 42 KiB of 512 available. If you look closely, you will notice that the freezes happen at regular intervals: approximately every 14-15 seconds, which translates to once for every 16 MiB of data written.

Usually (but not always) the biggest delays happen at the beginning of a new recording. Note how different the beginning of the next recording looks from the recording that had been going on for a long while:

Code: Select all

Continuous space 16138s (4:28:58) at 4 ch, 96000 Hz, 24 bits...
freeFrag.blocks 22a1480, reserved 0x3d00, miscData.spaceInBlocks 229d780, 241
D:AUDIO/AUD02197.WAV:    0s,    0 samples lost,   0/512 KiB buffer used
D:AUDIO/AUD02197.WAV:    1s,    0 samples lost,  89/512 KiB buffer used
D:AUDIO/AUD02197.WAV:    2s,    0 samples lost, 102/512 KiB buffer used
D:AUDIO/AUD02197.WAV:    3s,    0 samples lost, 100/512 KiB buffer used
D:AUDIO/AUD02197.WAV:    4s,    0 samples lost, 104/512 KiB buffer used
D:AUDIO/AUD02197.WAV:    5s,    0 samples lost, 100/512 KiB buffer used
D:AUDIO/AUD02197.WAV:    6s,    0 samples lost, 107/512 KiB buffer used
D:AUDIO/AUD02197.WAV:    7s,    0 samples lost,  93/512 KiB buffer used
D:AUDIO/AUD02197.WAV:    8s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02197.WAV:    9s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02197.WAV:   10s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02197.WAV:   11s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02197.WAV:   12s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02197.WAV:   13s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02197.WAV:   14s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02197.WAV:   15s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02197.WAV:   16s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02197.WAV:   17s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02197.WAV:   18s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02197.WAV:   19s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02197.WAV:   20s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02197.WAV:   21s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02197.WAV:   22s,    0 samples lost,  11/512 KiB buffer used
D:AUDIO/AUD02197.WAV:   23s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02197.WAV:   24s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02197.WAV:   25s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02197.WAV:   26s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02197.WAV:   27s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02197.WAV:   28s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02197.WAV:   29s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02197.WAV:   30s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02197.WAV:   31s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02197.WAV:   32s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02197.WAV:   33s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02197.WAV:   34s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02197.WAV:   35s,    0 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02197.WAV:   36s,    0 samples lost,  45/512 KiB buffer used

As you can see, for the first few seconds of this new recording, over 100 KiB of buffer space was needed while the card was doing internal bookkeeping. With a lower quality card, this could be much more, potentially even disastrous.

Below is an example recording with a non-disclosed 32 GB card that is too slow to record at full quality:

Code: Select all

Continuous space 26736s (7:25:36) at 4 ch, 96000 Hz, 24 bits...
freeFrag.blocks 395ee80, reserved 0x7c0, miscData.spaceInBlocks 395e6c0, 28
D:AUDIO/AUD02201.WAV:    0s,    0 samples lost,   0/512 KiB buffer used
D:AUDIO/AUD02201.WAV:    0s,    0 samples lost,   0/512 KiB buffer used
D:AUDIO/AUD02201.WAV:    1s, 56469 samples lost, 512/512 KiB buffer used
D:AUDIO/AUD02201.WAV:    2s, 56469 samples lost, 492/512 KiB buffer used
D:AUDIO/AUD02201.WAV:    3s, 56469 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02201.WAV:    4s, 56469 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02201.WAV:    5s, 56469 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02201.WAV:    6s, 56469 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02201.WAV:    7s, 56469 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02201.WAV:    8s, 56469 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02201.WAV:    9s, 56469 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02201.WAV:   10s, 56469 samples lost,  14/512 KiB buffer used
D:AUDIO/AUD02201.WAV:   11s, 56469 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02201.WAV:   12s, 56469 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02201.WAV:   13s, 56469 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02201.WAV:   14s, 56469 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02201.WAV:   15s, 56469 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02201.WAV:   16s, 56469 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02201.WAV:   17s, 56469 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02201.WAV:   18s, 56469 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02201.WAV:   19s, 56469 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02201.WAV:   20s, 56469 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02201.WAV:   21s, 56469 samples lost,  13/512 KiB buffer used
D:AUDIO/AUD02201.WAV:   22s, 56469 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02201.WAV:   23s, 56469 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02201.WAV:   24s, 56469 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02201.WAV:   25s, 56469 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02201.WAV:   26s, 56469 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02201.WAV:   27s, 56469 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02201.WAV:   28s, 56469 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02201.WAV:   29s, 56469 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02201.WAV:   30s, 56469 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02201.WAV:   31s, 56469 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02201.WAV:   32s, 56469 samples lost,  11/512 KiB buffer used
[...]
D:AUDIO/AUD02203.WAV: 4246s, 56469 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02203.WAV: 4247s, 56469 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02203.WAV: 4248s, 91378 samples lost, 512/512 KiB buffer used
D:AUDIO/AUD02203.WAV: 4249s, 91378 samples lost, 487/512 KiB buffer used
D:AUDIO/AUD02203.WAV: 4250s, 91378 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02203.WAV: 4251s, 91378 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02203.WAV: 4252s, 91378 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02203.WAV: 4253s, 91378 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02203.WAV: 4254s, 91378 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02203.WAV: 4255s, 91378 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02203.WAV: 4256s, 91378 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02203.WAV: 4257s, 91378 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02203.WAV: 4258s, 91378 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02203.WAV: 4259s, 91378 samples lost,  12/512 KiB buffer used
D:AUDIO/AUD02203.WAV: 4260s, 91378 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02203.WAV: 4261s, 91378 samples lost,   7/512 KiB buffer used
[...]
D:AUDIO/AUD02204.WAV: 7065s, 91378 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02204.WAV: 7066s, 126062 samples lost, 512/512 KiB buffer used
D:AUDIO/AUD02204.WAV: 7067s, 126062 samples lost, 512/512 KiB buffer used
D:AUDIO/AUD02204.WAV: 7068s, 126062 samples lost,   7/512 KiB buffer used
D:AUDIO/AUD02204.WAV: 7069s, 126062 samples lost,   7/512 KiB buffer used

This log shows very typical behaviour of a slow card: right after recording starts, the card takes way too long to respond, and as a result 56469 samples, or in other words, 0.59 seconds of audio is left unrecorded. In the case of this particular card, recording then continues without further issues for the next 71 minutes, after which it again locks up for long enough that another 0.36 seconds of audio is lost. Exactly the same happens again close to the 118 minute mark: another 0.36 seconds of audio is lost.

Kind regards,
- Henrik

PS. Whee, my 1111th message!

[Henrik]

Hi!

Here is version 0.71 of the 4-Channel HiRes Recorder. This is a minor update. The main thing is that a UART conflict has been resolved: one of the two functions of 'r' was changed, making the command unambiguous.

The HiRes Recorder can be run on one of the developer boards:

• http://www.vlsi.fi/en/support/evaluatio ... sion1.html - with LCD, UI, and additional ADC
• http://www.vlsi.fi/en/support/evaluatio ... order.html - in 2-channel mode through UART

Kind regards,
- Henrik

[EDIT: Seems the date on the documentation is incorrect by exactly one month. It should read 2019-04-05 instead of 2019-03-05]

[Passionate Jaguar]

Hello Henrik,

We could really use this uncompressed 96K recording technology on our existing VS1005g custom board design. We are already using a Spansion S34ML04G100TF100 Nand Flash with your existing SLC Nand driver k9f4g, controlled with the same hardware configuration as the Developer Board (with unmodified VSOS 3.55). The objective is to minimize SD read/write interference with realtime DSP processes and to free up application memory space by moving as much buffering to hardware as practical. We only use the SPI Flash for OS and application development and the SD for audio file storage.

What are the implications of using the SLC Nand Flash as a buffer verses your S-RAM on the new DevBoardExt1? Could the Nand Flash driver partition out a fixed contiguous memory section used only for buffering? Could the whole chip be used as a ring buffer (or multiple ring buffers)? What are the performance issues as compared to your S-RAM configuration?

We would perhaps like to extend this idea by applying the WavToMp3 symbol control methods to modulate any compression (mp3/Oog Vorbis) or direct WAV SD writes. This would give us absolute priority to Rec and other DSP real time processes. Periodically we need to send out data for a full spectrum audio display - which is easy to do by interrupting the compression with symbols and letting it then catch up.

To make this hopefully more clear, seems we can record uncompressed to a Nand Flash ring buffer, do any DSP/display processes from this buffer, then in our spare time (!) handle compression or WAV uncompressed SD writes - at least as long as the ring buffering lasts. This would give us one solution to handle both compression and SD irregularities that can disrupt realtime DSP.

Any other ideas also welcome. Without exception, all of VLSI's 1005 solutions have directly or indirectly added and refined our own application work. Much appreciated.

Scott

[Henrik]

Hello Scott!

Nice to hear from you after a long pause! You have put out a proper salvo of good questions! Let's see if I can answer them properly. This is going to be a long and involved message; I hope I can make my answers clear.

First, let me explain how the HiRes Recorder manages to buffer its data through VS23S040.

The driver currently used for combining SD card and VS23S040 SRAM memory operation is the pair SDSD23.dl3 and SDSDX23.dl3 which are functionally equivalent to SDSD.dl3 and SDSDX.dl3, except that they use a VS23S010 or VS23S040 as a ring buffer for contiguous writes. The driver does this by starting a new high priority task that clears the ring buffer to the SD card whenever it can.

Note that I said "contiguous writes". This limitation is actually more restricting than one might think. When normally writing to a file using fopen() / fwrite() methods, the FAT file system occasionally reads FAT tables looking for free sectors. This would cause the SDSD23 driver to flush its write buffer, then do a potentially slow read operation from the SD card, then resume writing, potentially making using the buffering driver completely useless.

The HiRes recorder goes to extra steps to prevent this buffering issue. Instead of writing through the file system, it starts by looking for the longest contiguous free area on the SD card - this is why starting it takes a couple of seconds, after which you get a print-out like this:

Code: Select all

---- Clusters 0x49a261, fatSecPerCl 0x40, blocks 12689840
Continuous space 411792s (114:23:12) at 4 ch, 48000 Hz, 16 bits...
freeFrag.blocks 12689840, reserved 0x3d00, miscData.spaceInBlocks 12685b40, 241
D:AUDIO/AUD02658.WAV:    0s,    0 samples lost,   0/512 KiB buffer used

When you start recording, the HiRes recorder doesn't go through the file system: it writes directly to the data area it found, with no book-keeping data. Only when you stop recording does the recorder build a FAT file index and does the necessary space allocation. That's why it has a print-out like this when you stop recording:

Code: Select all

Finalizing file entries.
MAKE SURE THE DEVICE IS NOT POWERED OFF WHILE
FILE ENTRIES ARE BEING BUILT!!!
Creating file entry for D:AUDIO/AUD02658.WAV, length  110.86 seconds
Fix length to RIFF WAV headers, cluster 3098911, block 198395712, aSC 3100211 -> 3100211
Create file handle, name D:AUDIO/AUD02658.WAV, firstCluster 3098911, bytes 42569216
File size should be 42569216 bytes
Fixing directory entry pointer
Closing file
File 2659, entries 1
Add BWF data for D:AUDIO/AUD02658.WAV
Finalizing took 0.4 seconds.

All of this means that the HiRes Recorder is FAT32-specific: it directly uses FAT32 structures so it couldn't work on a different file system without a major rewrite. Then again, FAT32 is quite adequate for the task, and its big achilles heel - 32-bit file size limitation - is also being circumvented by breaking files in slightly-below-2-gibibyte units.

An advantage of doing the bookkeeping by hand is one of my favourite features of the HiRes Recorder: every second it writes some bookkeeping data to VS1005's battery backed up RAM. If you have a battery connected to it like we have on the VS1005 DevBoard, you can recover the file system 100% even if you lose main power in midst of the recording. Think of a recorder that runs out of battery while recording, or even worse, is dropped to the floor and batteries get displaced, or something. All you need to do it to repower the unit, and the HiRes Recorder will automatically recognize there are things to clean up, and consequently it will fix the file system. All you lose is 2-5 last seconds of your recording!

Example below:

Code: Select all

D:>HiResRec

HiRes Recorder v0.71
VLSI Solution 2019

~0206=80
---- Clusters 0x499d4d, fatSecPerCl 0x40, blocks 12675340
Continuous space 411681s (114:21:21) at 4 ch, 48000 Hz, 16 bits...
freeFrag.blocks 12675340, reserved 0x3d00, miscData.spaceInBlocks 12671640, 241
D:AUDIO/AUD02659.WAV:    0s,    0 samples lost,   0/512 KiB buffer used
D:AUDIO/AUD02659.WAV:    0s,    0 samples lost,   0/512 KiB buffer used
D:AUDIO/AUD02659.WAV:    1s,    0 samples lost,  20/512 KiB buffer used
D:AUDIO/AUD02659.WAV:    2s,    0 samples lost,  23/512 KiB buffer used
[...]
D:AUDIO/AUD02659.WAV:   22s,    0 samples lost,   2/512 KiB buffer used

At this point I violently removed the power from the board. Now, let's power the board back on and see what happens when I start the HiRes Recorder:

Code: Select all

S:>HiResRec
HiRes Recorder v0.71
VLSI Solution 2019
~0206=80
WARNING! Found unfinished file entries from previous run!
Activating RESCUE MODE!
MAKE SURE THE DEVICE IS NOT POWERED OFF WHILE
FILE ENTRIES ARE BEING BUILT!!!
Creating file entry for D:AUDIO/AUD02659.WAV, length   21.99 seconds
Fix length to RIFF WAV headers, cluster 3100211, block 198478912, aSC 3100469 -> 3100469
Create file handle, name D:AUDIO/AUD02659.WAV, firstCluster 3100211, bytes 8445440
File size should be 8445440 bytes
Fixing directory entry pointer
Closing file
File 2660, entries 1
Add BWF data for D:AUDIO/AUD02659.WAV
Finalizing took 0.5 seconds.

We could really use this uncompressed 96K recording technology on our existing VS1005g custom board design. We are already using a Spansion S34ML04G100TF100 Nand Flash with your existing SLC Nand driver k9f4g, controlled with the same hardware configuration as the Developer Board (with unmodified VSOS 3.55). The objective is to minimize SD read/write interference with realtime DSP processes and to free up application memory space by moving as much buffering to hardware as practical. We only use the SPI Flash for OS and application development and the SD for audio file storage.

What are the implications of using the SLC Nand Flash as a buffer verses your S-RAM on the new DevBoardExt1?

Hmmm... Well, first of all, a new driver would need to be written, and its timings would need to be tuned to make sure it would fit the purpose. The timing of the SDSD23 driver task has been finely tuned along with the HiRes Recorder to make sure the recorder would not lose samples. For example, if there is lots of data in the buffer and the SD starts working fast again, the buffer is cleared only gradually to make sure the SDSD23 task doesn't eat so much processor time that the HiRes Recorder task couldn't receive new samples.

Could the Nand Flash driver partition out a fixed contiguous memory section used only for buffering?

I don't remember offhand whether the driver supports that to begin with, but I guess that could be done with minimal changes.

Could the whole chip be used as a ring buffer (or multiple ring buffers)?

I guess so. In that case its buffering power would actually be really good. It would also cycle so slowly that the typical 50,000 writes or so that the memory can take before breaking wouldn't be an issue.

What are the performance issues as compared to your S-RAM configuration?

Well, that is the million dollar question.

The interface for both the Nand Flash and VS23S040 is the 8-bit Nand Flash bus, so in that sense they are similar. They both need to share the bus with the LCD (if LCD support is compiled in).

I think that when using Nand Flash the most critical (=slowest) part would be erasing an erase block. If this takes more time than what can fit in VS1005's RAM, the project is dead to begin with. Writing and reading single data blocks I think might be fast enough - but not as fast as writing to S-RAM where there are no write or read delays to speak of.

All in all, using Nand Flash would be slower than S-RAM. The HiRes Recorder is highly tuned so that it just works at 4-channel 24-bit recording at 96 kHz (sustained data rate of 1125 KiB / second), and I can almost guarantee that this highest rate could not be made to work with Nand Flash however much the system is tweaked. But, honestly, because of the erase delay, it may be that even stereo 16-bit recording at 48 kHz would [EDIT 190417: NOT] be possible. Very difficult to say.

We would perhaps like to extend this idea by applying the WavToMp3 symbol control methods to modulate any compression (mp3/Oog Vorbis) or direct WAV SD writes. This would give us absolute priority to Rec and other DSP real time processes. Periodically we need to send out data for a full spectrum audio display - which is easy to do by interrupting the compression with symbols and letting it then catch up.

To make this hopefully more clear, seems we can record uncompressed to a Nand Flash ring buffer, do any DSP/display processes from this buffer, then in our spare time (!) handle compression or WAV uncompressed SD writes - at least as long as the ring buffering lasts. This would give us one solution to handle both compression and SD irregularities that can disrupt realtime DSP.

If you are compressing audio to MP3 or Ogg Vorbis, then I think that the old buffering solution which uses the VS1005's own RAMs should be enough for practically all if not all SD cards. By this I mean the SimpleMp3Encoder example project that uses the FILEBUF.dl3 buffering library. See SimpleMp3Encoder/SimplRec/main.c for details on the encoder.

Any other ideas also welcome. Without exception, all of VLSI's 1005 solutions have directly or indirectly added and refined our own application work. Much appreciated.

Thank you for your kind words! I hope this message didn't make things muddier!

Kind regards,
- Henrik

[Passionate Jaguar]

Hello Henrik,

This is going to be a long and involved message; I hope I can make my answers clear.

Well explained with sufficient detail. Especially significant are the details on bypassing the file system, using the battery backup ram (which we are using - now to schedule turning the device on and selecting FM station, SD audio file playback, etc), and how you overcame FAT32 while writing files. The FM part nearing completion, now we are upgrading the player/recorder from the VSOS 3.20 technology to include Panu's playlist type functions with your 24-bit/96kHz 2 channel record/playback. And your explanation inspired some new ideas.

And, a little more detail on what we are looking at.

Now that you are in professional recording quality range, we are dredging up an old idea of record verification - like on tape decks of old, you could monitor on a second head what is actually recorded on tape from the first head. So, that's what we want to attempt with Nand Flash. Maybe with some fast forward/rewind in the monitoring side. Crazy, but so was 24/96 4-track recording a year ago!!

They both need to share the bus with the LCD (if LCD support is compiled in).

We are using a UART interfaced intelligent display via single byte commands in both directions - much like your UART command, so no sharing and very minimal display loading on VS1005.

I think that when using Nand Flash the most critical (=slowest) part would be erasing an erase block. If this takes more time than what can fit in VS1005's RAM, the project is dead to begin with

Yes. The Spansion chip does erase 2 blocks at a time, but "twice as fast" remains to be verified against the k9f4g originally on my DevBoards 1.8. Using the whole Flash chip as a ring buffer would also eliminate battery ram updates (so I can use bat-RAM to save the application state (FM-Station/SD Play/SD/Record/Mic Rec/Memos/Schedules) at crash time, instead - you inspired us.)

It might also work to use a scheme with FILEBUF.dl3 to buffer the block erases. To test this out at 562.5 KiB/second for 2 channels, as you pointed out, its the fine tuning that matters. Understood. We made great effort to make the FM play so tuning or scanning never interrupted the VU display. Even Mute just let the VU settle gracefully back to 0. You made sure samples were not lost writing to SD. We need to make a tool to verify that no samples are lost writing to Nand Flash, correct? The only thing that comes to mind now is to check how early we arrive at fread(stdaudioin) before the buffer is filled. If we sent that byte to the intelligent display VU's, maybe we could see in real time what is going on. Any ideas?

And it might work better to keep it simple and just redesign our board and add some S-RAM.

Thanks for all the considerations,
Scott

[Passionate Jaguar]

Hello Henrik,

Forgot to mention in the last post that we would also need to know for using Nand Flash as a ring buffer:

• What boot code and/or boot helper rescue image code might need to remain on Nand Flash and where is it located?

• Where are the Bad Block Tables located? Obviously, we will still need to read/update these.

• Where are the FAT32 directories located?

• If we move our user presets and profile files from Nand Flash to Int or Ext Boot SPI Flash or SD, can we use the FAT32 areas on the Nand Flash, or is the OS expecting to find something there? Could we then eliminate FAT32 references from the to-be-modified k9f4g driver? And is this driver a good starting point, or is there a simpler approach? Looks like most of the existing k9f4g code would not be used anyway. And as this code does not seem to be updated lately, would the SDSD23 driver code interface better with VSOS 3.55 and thus make a more effective starting point?

• Is there anything else we have missed that when blasting through address spaces might brick the Flash (or everything else)?

And because we want to stay up to date with current VSOS releases, whatever we do, we would prefer not to customize the kernel; custom .dl3's are fine.

Thanks for your assistance,
Scott

[Henrik]

Hello Scott!

Well explained with sufficient detail. Especially significant are the details on bypassing the file system, using the battery backup ram (which we are using - now to schedule turning the device on and selecting FM station, SD audio file playback, etc), and how you overcame FAT32 while writing files. The FM part nearing completion, now we are upgrading the player/recorder from the VSOS 3.20 technology to include Panu's playlist type functions with your 24-bit/96kHz 2 channel record/playback. And your explanation inspired some new ideas.

Hmmh, speaking of FM... Are you talking about an FM receiver? And if that, perhaps even a stereo FM receiver? You know that we have something really neat for enhancing FM stereo reception? Namely the FM Stereo Noise Killer, a DSP algorithm that makes FM stereo sound as noiseless as FM mono without much affecting the stereo image? The noise killer is not a part of the VSOS 3.57 Root and Libraries package, but there is also a separate thread with examples that - at least to me - are impressive:
viewtopic.php?f=13&t=2273
NOTE! If you listen to the samples, do it with stereo headphones or speakers. Any mono output device like a mobile phone, pad or some laptops would cancel the whole effect and leave you confused what on earth I am babbling about.

Apart from that, we can go around in circles and then think whether something is possible or not... But there is one thing that is sure to work:

And it might work better to keep it simple and just redesign our board and add some S-RAM.

This would actually be my recommendation. Then you could just use what already exists. The fine tuning I had to do to make the SDSD23.dl3 driver timings reliably work with the HiRes Recorder 100% of the time is something I don't want someone else to have to go through. Even if it's doable (and we don't even know that), it will take time and may always be a bit... wobbly, for the lack of a better word.

If you want me to address the second of your posts, I can do that, but to me this would be the most straightforward implementation.

Oh, and by the way, I'm happy you got good ideas for using the battery backed up RAM. I've found it very useful myself!

Kind regards,
- Henrik

[Panu]

Dear All,

Here's the firmware image of the VS1005 Break-Out Board Mk2 running Hi-Res recorder. This is what we write to the internal flash of the VS1005 in production.

To reflash the breakout board, connect a USB UART cable to the breakout board RX, TX, GND and VHIGH pins, or use the programmer board described earlier. Extract the contents of the VS1005g_BoB2_VSOS357_HiresRec071_20190411_Win.zip file to a folder in your PC. Open a command prompt and navigate to that folder, and then give the command "runintflash" to run the prommer. Or you can just click the "runintflash.bat" file to run it, but when errors occur, it may be difficult to see the error messages before the window closes. The .bat file tries COM ports 1, 2, 3, 4, 5 ,6, 7 and 8.

Alternatively, you can use the VSPROM9 prommer from here: viewtopic.php?f=13&t=681&start=10#p11097 to write the VS1005g_BoB2_VSOS357_HiresRec071_20190411.img file to the internal flash.

-Panu

[Henrik]

Hello!

Here is version 0.72 of the 4-Channel HiRes Recorder. This is a minor regression update. There was an issue that if you removed the SD card while not recording, then inserted the same or another card, the recorder would not recognize the new card was FAT32, and would always suggest reformatting the drive. This was caused by an incorrect fopen("tStr", "rb") command that should have read fopen(tStr, "rb"); . So, you can either use this new version, or if you have branched a version of your own, just make this one correction to your source code.

Kind regards,
- Henrik