Tuesday Bluesday

I'm having a lot of back pain, night-time headaches, difficulty getting up in the morning.  Sigh.  I don't know how I am going to survive the next year.

• It's possible the re-entrant versions of the stdio routines actually aren't appropriate to use when MicroC/OS-II (the embedded real-time operating system) isn't running.  So, maybe they are just deadlocking when the ISR tries to print while the main thread is printing, whereas with a uC/OS2 based design, there would be a timer-induced preemptive context-switch between threads that would break the deadlock.  Anyway, I could try using the RTOS features, but I don't know whether there is enough memory on the board for it... I should try the "Hello World" example.  Anyway, I ordered a book on the OS so I can learn more about it.  (Altera's documentation on it is pretty sparse.)
  
• Another approach, short of installing a whole RTOS, would be to serialize the output myself using my own line-buffered output stream combinator.  However, this would take more memory.  Also, when this buffer fills up, how can a warning message about this situation be printed?  And, what is it that is really hanging, output to the stdout console via JTAG, or output to the serial UART?  (I suspect it must be one of these.)
  
• I should run the firmware under the JTAG debugger, and, when it hangs, stop and see where it is.  This might tell me something.

Also, need to back up the current design, and move it back to the shared drive.

I backed up the contents of R:\ (C:\LOCAL\FEDM_code\q91) on Dropbox in "FEDM_design\FEDM_code v2", and also copied it to the Q:\ network drive (C:\SHARED\FEDM_code\q91).  The old contents of the Q:\ drive were moved to C:\SHARED\FEDM_code_v3.

I am now working in Q:\ (shared) again, instead of in R:\ (not shared).

I am recompiling the Quartus design with the latest firmware integrated in.

Had a weird problem where the debugger crashed... Hmm.

I just looked at the disassembly in FEDM_ctrl_fw.objdump, and it appears that our attempt at semaphores by doing ++ and -- on global variables isn't really working, because those operations aren't atomic; the adds are done in registers, rather than directly on memory locations (RISC architecture).

Added a sequence number on output pulses, and noticed something interesting - some of the output is apparently getting lost somewhere in transit, because there is a jump in the sequence numbers of the lines arriving at the server.

I wonder if the writes to the serial port are being done in nonblocking mode, in such a way that if the serial output buffer is full, data just gets lost?

Anyway, the problem (or at least, the most immediate problem) is NOT in the pulse buffer.

The problem (hanging of the serial output) seems to only happen for pulse rates over about 30 Hz or so.

Here's an idea:  Let's send less data for each pulse, and see if it decreases the problem.  This might be further evidence that it is tied to serial output buffering.

Another idea:  Increase the serial data rate.

Decreasing the length of the lines seemed to help.  Let's now try upping the baud rate to 115,200 baud.  Changed it in SOPC builder; now regenerating...  And rebuilding firmware...  Also had to change the Wi-Fi script, reloading that...  Also recompiling the Quartus design...

Oops, just noticed that the IDE is still referencing the R:\ drive... Had to spend a while fixing that.

Still having problems at 40 Hz pulse rate, even with the faster baud rate.  And now, mysteriously, serial input isn't working.

While testing output from the Wi-Fi board, I noticed the Python server isn't treating Control-M (CR) like end-of-line; it is waiting for the next character first.  This is suboptimal.  However, it is difficult to fix since I am using the existing .readline() method.  I'd have to modify how that facility works to fix it.  I think I actually noticed this problem before.  Oh well, it doesn't matter too much since Control-J (LF) still works.

Can't figure out why the UART is ignoring serial input at this baud rate.  Looks like we will have to revert to the slower baud rate. 

I think we should cut our losses WRT trying to solve this problem.  We'll just have to be satisfied with an event rate of no more than about 30 pulses per second (sustained average), and occasional dropped pulses when this rate is exceeded.  I was hoping we could do better than that... But if the stdio library is too crappy... Although to be fair, the problem could be with limitations in the Wi-Fi board's bridge facility instead, like inadequate buffer sizes...  Since there's no flow control, if the Wi-Fi network isn't keeping up with the data rate, there's really nothing it can do, if it runs out of places to put serial data...

This brings up another question:  Can we use CTS/RTS flow control?  Check on this tomorrow.

Another thought:  We should perhaps try the (reportedly faster) flyover mode of the Wi-Fi board.  This will require some extensive changes to that script though.

Well, I took the baud rate back down to 57,600 and serial input is still not working!  Connected directly to the PC serial port - it works from there!  So I don't know WTF is up with that.  In positive news, though, when streaming directly to the PC serial port, we were able to sustain pulse rates around 70 Hz (theoretical maximum).  100 Hz caused the buffer to overflow, as expected.

I should perhaps try re-upping the baud rate to 115,200 and checking again to see what sustained data rate I can get without the Wi-Fi in the loop.  I should be able to get about 140 pulses per second, at least.

However, I eventually still need to fix the Wi-Fi input & data rate problems.

Note added later:  Tuesday stands for Tiwaz' day.