Wed., Nov. 30th

I'm hoping that Michael Dean & Juan Calderin will come in to the lab today and tell me that they are ready to try integrating their new timing edge-capture datapath into the system.  Emailed them to see.

Another thing I can work on today is preparing some slides with an overview of the current software design of the Python server, so the students can build their own visualization modules on top of it.

My overnight run of the GPS app is still running, currently at 75,900 seconds into the run.  I need to find my old Scilab analysis scripts (not to mention the Java file format-conversion app) and see what this data looks like.

Maybe instead of going through all that, I can just take some shortcuts to get a quick idea of what the time data is looking like.  In Cygwin, in the "/cygdrive/c/SHARED/Server Code" folder, I typed this command to extract just the PPSCNTR lines:

$ grep PPSCNTR node0.uart.trnscr > PPSCNTR.csv

Now, let's drag that file into an OpenOffice Calc window.  Ah, it truncated the import at 64K rows, but that's OK.  Oops, the last field, which is the main one of interest, still contains the NMEA checksums:

Tue Nov 29 17:55:33 2011 + 209 ms: < $PPSCNTR,0,0*58
Tue Nov 29 17:55:33 2011 + 212 ms: < $PPSCNTR,1,9999939*5A
Tue Nov 29 17:55:33 2011 + 214 ms: < $PPSCNTR,2,19999953*64
Tue Nov 29 17:55:33 2011 + 217 ms: < $PPSCNTR,3,29999966*60
...

So, let's use a Unix cut(1) command to get rid of those:

$ cut -d\* -f1 PPSCNTR.csv > PPSCNTR-trim.csv

That's better.  Now, we can add a spreadsheet column to compute the deltas.  Boy, that takes a while... OpenOffice is slow!  Had to kill the process.  Let's trim off a few lines from the end to avoid problems.  Now the valid data indices (PPS seconds) runs from 2 through 65529.  OK, based on the data, the average frequency over that interval is 10,000,012.8768.  Now we can plot the cumulative time wander relative to the average frequency as a baseline, as we've done previously:

Note we still have the usual large but brief phase glitches (off the chart), but other than these there is mostly just a pretty gradual drift, and no large divergences from this trajectory other than one noticeable discontinuity.  I don't know if we actually lost time lock at any point.  We can figure that out by looking at the PDMETRAIM figures.

Here is the self-reported accuracy from the TRAIM algorithm:

It looks like we actually lost time lock between about seconds 7,000 to 23,000.  At 7,000 is when the wander started drifting sharply upwards, then at 23,000 is a slight downwards dip to back in line with the background wander.  In the interim, was the POSHOLD mode controlling the time readings?  Bizarrely, looking in the middle of this range, it looks like there were plenty of satellites, but they were all being excluded!  Is the TRAIM algorithm broken?  Or, perhaps it's just that the signals are all getting reflected off of the nearby building and so the times are all way off?  Hard to say.

Number of satellites being received over the course of the run.

Perhaps we should set the limit of TRAIM to a larger value so that fewer satellites will be excluded?  Right now the limit is 100 ns.  Should we make it 200 ns?  Let's try that and then start another overnight run.

Looking at this graph, we can see that an unusually large number of satellites (perhaps all of them?) were eliminated by the TRAIM algorithm during the period where the self-assessed accuracy was 0.

TRAIM algorithm data.  Blue line: Self-assessed inaccuracy in seconds (left axis).
Red dots: Number of satellites eliminated by traim algorithm (right axis).

Note also that the more satellites are eliminated, generally the worse the accuracy is.  However, it's unclear in what direction the causality relation goes here.  When a lot of satellites are giving wild time readings, more of them are eliminated and so the accuracy is worse?  Or, is it that when more satellites are dropped from the calculations, there is less data to work from, and so the result of the calculation is more inaccurate?  Unclear.

Anyway, started a new run with the <traim_alarm> parameter set to 200 ns instead of 100 to see how this affects things.  At first glance, it seems like the accuracy is worse (perhaps b/c more satellites with wild readings, reflecting off Ware-Rhaney maybe, are being averaged into the calculations).  But anyway I'll leave it running overnight (or over the weekend, even) and analyze the data next time I'm in.

Tue., Nov. 29th

We went up on the roof above the lab window to see if we could get more accurate GPS coordinates there, using the MotionX-GPS app.

My iPhone says:

• Lat.:   30* 25' 42" (N)
• Lon.:  84* 17' 06" (W)
• Alt.:    97'
• Acc.:  56'

On Google Maps, this looks to be on the centerline of our wing of the building, about 30' away from the edge where our windowsill is.

Ray's iPhone says:

• Lat.:    30.428250
• Lon.:   84.284997
• Alt.:    131'
• Acc.:   56'

On Google Maps, this looks to be almost exactly in the right place.

Unfortunately, the iPhone is unable to acquire a fix at all in the window, so we are unable to compare it with the DeLorme to try to figure out whether the position discrepancy is due to the location or to something about the DeLorme.  But at least we've ruled out the problem being an inaccuracy in Google Maps / Google Earth (since Ray's phone, at least, agrees exactly with Google, within a couple of feet, as to where our window is, and my phone agrees almost exactly, within about 30' or so).

Aha, just got a fix in the windowsill from my iPhone, it says:

• Lat.   30* 25' 40" N
• Lon.  84* 17' 07" W
• Alt.    238 ft
• Acc.  156 ft

Cool, this is also over near Ware-Rhaney, so this confirms (more or less) that it's probably the windowsill location (and not the DeLorme unit itself) that is causing the positional inaccuracy problem.

[/] Need to make sure the POSHOLD location is as accurate as possible, WRT our real location.  OK, updated that code now with the slightly more accurate coordinates 30*25.694,N,084*17.1,W (on top of our window, instead of at corner of the wing).  Rebuilt firmware, recompiling gelware in Quartus...  Also need to rebuild the EEPROM config file... (Rev_A.jic)  Done & burned it.

Pascal dropped in and showed me the blueprints he got from the architecture firm of the I-beams, sprinklers, lighting, and vents (4 separate sheets).  He is going to work this information into the next presentation.  He also said that Reid Lambdin OK'd the idea to put the detectors on shelves instead of hanging them from I-beams.

Aarmondas reminded me that we had agreed to reschedule the presentation date/time to Friday of next week.  It will have to be 4 pm since Drifters is at 3 pm.  I emailed an updated announcement to the BB group.

I asked Aarmondas also to schedule a date/time either next week, during finals week, or in the 1st two weeks of classes in January for me to present to the ECE students the current design of the Python server software, so that the students will be familiar with the capabilities of what's already there and can build upon it for the visualization etc.

I'm starting a new overnight run to see if the time accuracy seems any different than it did in my experiments last Winter/Spring.  I think our POSHOLD position reference is more accurate now (due to both improvements in aerial photography on Google Maps / Google Earth - closer to overhead view - as well as to Ray's phone, which I think is an iPhone 4, apparently having a more accurate GPS than my 3GS does).  Tomorrow I'll do a plot showing the cumulative phase wander of the GPS's PPS signal vs. the OCXO clock, and look at the frequency and magnitude of any significant excursions that happen when time lock is temporarily lost.

Mon., Nov. 28th

David was out sick last week, but today he texted that he is coming in this afternoon after his class.

My plan today is to work some more on the GPS startup.  I am wondering if the problem might be that the position data in the POSHOLD command is not sufficiently accurate?  A better procedure (suggested in the manual) might be to average position data over a period of time before initiating POSHOLD mode.  This would also be helpful when installing the system at a new site location (or in general when moving the GPS antenna).

Also, I can still try the strategy of querying the current NTP time from the server to initialize the time to ~10 ms resolution.  However, even just 10 ms of time uncertainty equates to 3,000 km of position uncertainty, so I'm not sure how useful it will be to set the time more accurately.

Anyway, let's try to get this thing up and running again.  I'm going to carefully log all my steps to facilitate a postmortem next time the GPS fails to connect.

• (2:31 pm) My immediate plan is to first connect through UwTerminal, then select the default configuration, access the kit using the demo software, and then wait 15 min. or so to re-acquire satellites.
• (2:32 pm) The current configuration is:  Batteries not installed.  USB not plugged in.  AC adapter not plugged in. NMEA port connected to COM1 via gender bender.  NMEA port switch in rightwards ("DCE") position.  Power switch in leftwards ("on") position.
• (2:33 pm) Starting UwTerminal.  Setting COM=1, baudrate=57,600, handshaking=none.  Click OK.  No data of course because the kit isn't powered on yet.
• (2:35 pm) Now I'll plug in the AC adapter for power.  The unit confirms:

$PDMEHEADER1: DeLORME GPS2058_HW_1.0.1

$PDMEHEADER2: DeLORME GPS2058_FW_2.0.1

$GPTXT,COSMICi Custom_Config_0.0.3

$GPRMC,214717.165,V,3025.694,N,08417.100,W,0.0,0.0,221111,4.0,W*7B

$GPGGA,214717.165,3025.69416,N,08417.10000,W,0,00,99.0,069.66,M,-29.7,M,,*66

$PDMETRAIM,2,0,0.000000000,0,0,0,0,0,0,0,0,0,0,0,0,0*43

$PDMEPOSHOLD,0,0000.000,N,00000.000,E,000.00*4A

• (2:38 pm) Unplugged unit a few minutes ago to stop text from scrolling so I could note the above.  Now plugging it in again to see if it acquires anything.  I don't think it will.
• (2:40 pm) Going to send an NMEA-off command to make it easier to send other commands.  Download-->Stream File Out-->NMEA-off.txt.  No response.  I think the last time this happened, I decided it was a grounding problem.  Going to switch temporarily to the batteries for power supply.
• (2:41 pm) Inserted batteries (two AAs).
• (2:42 pm) Unplugging AC adapter.  Power light goes off (this is normal for battery mode), but status light still flashing red.  Now trying to send the file again.  Still no response!  Wait, maybe in battery mode the level shifter is disabled?  Doesn't look like it from a quick glance at schematic... Anyway, let's try the USB power supply now.
• (2:44 pm) Plugging in USB cord, unplugging battery.  Now the NMEA-off command works.
• (2:45 pm) Wondering if, before I revert to the demo app, I should first try the POSHOLD and TRAIM commands, with the correct time value in the POSHOLD.  Or actually, first, try the $PDME,9 command to just initialize the position and time.  "Must be issued immediately after a reset or startup to be most effective."  OK, so first, power off the unit.  Unplugging USB.
• (2:47 pm) Starting SciTE, opening "C:\Users\Mike\Documents\My Dropbox\FAMU\COSMICi\GPS FPGA app\NMEA command files\init-pos.txt", preparing to write new contents.  Old contents:
  • $PDME,9,30.428236,-84.285,40,2011,11,22,21,47,30
• (2:49 pm) All I have to do, I think, is update the time value to something semi-accurate.  Let's go to time.gov and make sure we know the correct UTC(GMT) time.  It is 19:50.  So, allowing a minute or so to start, let's write:
• $PDME,9,30.428236,-84.285,40,2011,11,28,19,52,00
• (2:52 pm) OK, sent that line right after plugging in the unit's power.  Looks like I was about 15 seconds early (compared to what the real time was), but at least it's close enough to help it find the satellites, hopefully.  Next step:  Just wait 15 minutes or so to acquire satellites, hopefully.
• (2:55 pm) An observation: The $PDMETRAIM line is reporting condition 2 (unknown), and the LED D9 is flashing green once every 2 seconds.  I don't know what this means!  (That signal TRACKLED connected to the green is supposed to mean "3D fix" but I don't know what it means for it to flash once every 2 seconds - undocumented behavior?)
• (2:58 pm) Checking the scope; there is no PPS edge currently (consistent with no time lock).  Still no satellites showing.
• (3:01 pm) OK, I'm still getting no sign of any satellites.  I think I'd better abort this test for now.  Sending NMEA-off.txt.
• (3:02 pm) Now sending select-default.txt.
• (3:03 pm) Now sending hot-start.txt.  Switching UwTerminal baud rate to 4800 to match.  I wonder if just doing this (switching back to default config) will help acquisition at all?  Maybe wait a little while to see.
• (3:06 pm) Still no satellites.  Wait a few more minutes and then try the demo app.

DAMMIT, I lost like two hours' worth of notes because Blogger fucked up.  Summary of status: Got it working thru the demo app, switched over to the custom config and finally got things working with power-up thru DE3 and with battery backup for the GPS.  However, the setup is pretty unreliable/touchy; the Wi-Fi tends to power-cycle and the DE3 app tends to crash.  But it's been running smoothly for a while.  I am afraid to breathe on it though.  Anyway, here's the pic of the current setup:

I should test again to see if it works without the USB blaster (JTAG) cable.  Yep, it does.

Here is a pic of the setup showing no cables but power and GPS antenna cable:

Here is a screenshot of the server windows:

And here is a closeup of the main output window:

Guess I'll have to recreate all those setup notes some other time...

Here is a screenshot showing both the output from the the CTU (DE3 board GPS app) and the FEDM simultaneously:

What next?  Maybe, try putting the GPS in standby when on battery power?  I think I'll do that when I leave today.

One more thing:  I averaged the coordinates from the GPGGA lines from the latest run, about an hour's worth (4:55 - 5:55 pm).  Here is the averaged data:

• Latitude:      30 degrees, 25.65445 minutes, sample standard deviation 0.01596 minutes (29.56 m).
• Longitude:   84 degrees (W), 17.11387 min., sample std. dev. 0.00881 minutes (~14 m).
• Altitude:      62.81 m, sample std. dev. 21.50 m

This puts us over in the Ware-Rhaney building, according to GPS Visualizer and Google Maps (http://www.gpsvisualizer.com/map?format=google&lat=30.42757416667&lon=-84.28523116667).
Also on Google Earth.  Is the signal reflecting off the building, or what?  Don't know...

OK, 6:22 pm - STOP, exit on FEDM; MUTE, STOP, exit on DE3; DE3 power off; switching GPS switch to OFF/STANDBY position.  Tomorrow we'll see how hard it is to re-acquire the signal...

Tue., Nov. 22nd

Was thinking of doing some work today on improving time resolution of CTU.

First, an issue list with the current CTU setup:

• [ ] For some reason, the Wi-Fi board failed to connect when I powered up the system today.  But then, after some fiddling, it started working again.  Loose connection somewhere?  Watch out for this to happen again, try to diagnose.
• [ ] When the CTU firmware receives a blank line on its input, at present it generates an "UNK_CMD" (unknown command) error message.  Instead it might make more sense to simply ignore these lines.
• [ ] After being powered off for a while (like overnight), the GPS module seems to have difficulty acquiring satellites again.  This is perhaps because the time-of-day is wrong.  (At the moment, the GPS thinks it's 19:50 GMT, when actually it is 20:17; so the GPS is running almost half an hour slow.)  One way to address this:  Have the CTU query the server to find out the current time, then send a command ("$PDME,9,...") to the GPS to set the time (as well as setting the position)
• [ ] When restarting, sometimes there are server errors on the 2nd (or later) connection attempts.  These need to be tracked down and fixed at some point.

Samad is here and he brought the new 10-pin connector with him.  I showed him the proto board I picked up the other day at Fouraker's and the extra breakaway headers and jumpers I ordered from Adafruit, so that he can work on soldering up our power distribution board.  We discussed the connections needed.  He is going to write it all up and include it in the presentation #3 (SLDR), as well as the next report (DDR).

I scanned the bare proto board for him, so if he runs out of time to finish the soldering today, he can at least still make a nice sketch of the wiring for the presentation:

Bare proto board, to be used for soldering
prototype of main power distribution board.

Back on the GPS - I inserted two fresh AA batteries into the GPS kit as a battery backup, so that once we get a fix we don't lose it again.  Although, this conflicts somewhat with the use of the $PDME,9 command, because the docs for that say it needs to be done right after startup to be most effective.

Based on aerial photos from Google Earth (latest imagery), the coordinates of the windowsill where the GPS antenna is currently located are as follows:

• Latitude:     30°25'41.65"N
• Longitude:  84°17'6.00"W

This is a little west of where it looked like the window was in earlier images, but I think the current image is more direct (from overhead), so let's go with that.

Using the coordinate converter at GPSVisualizer.com, in terms of just degrees and minutes (what we get back from the GPS) this is:

• Latitude:     N30°25.694167
• Longitude:  W084°17.1

and in terms of just degrees (which we supply on the $PDME,9 line), it is:

• Latitude:     30.42823611111
• Longitude:  -84.285

Altitude, guesstimating from Google Earth, is about 50 m (164 ft).  So, the actual $PDME,9 command line would be (right now, 2011 Nov. 22nd, 21:12:00 GMT):

• $PDME,9,30.428236,-84.285,50,2011,11,22,21,12,00

Now, the only problem is, getting this information to the GPS will be somewhat involved, requiring changes to the CTU firmware, the Wi-Fi script, and the server.  I think perhaps as a short-term solution, I will get the GPS set up manually, and just rely on the batteries for a while to keep the fix active.  Remember, though, to later go back and solve this issue more permanently!

Muting GPS echo and stopping timer.  Exiting from Wi-Fi.  Powering off CTU.  Yes, GPS still has power (from batteries).  Unplugging level shifter.  Plugging in COM1 cable.  Setting UwTerminal to 57600, no flow control.  Sending commands from UwTerminal:  NMEA-off.txt, hot-start.txt, init-pos.txt (with the new location/time as above).  Still no satellites!  Maybe I need to re-run the eval app.  Have to revisit this on Monday.  Happy Thanksgiving all!

Mon., Nov. 21st

This is a short week due to Thanksgiving.  Some things to tackle:

1. [/] Get GPS module acquiring satellites again.
2. [/] Disable CTU debugging output under control of a slider switch.
3. [/] Burn working CTU design to DE3 board.
4. [ ] Maybe experiment with faster clock for GPS timing edge capture in CTU app.
5. [ ] Start writing up some notes on Python server code for use by Cp.E. students.

Working now on satellite acquisition.  Using UwTerminal, I streamed command files to select the default configuration.  Then I hooked up the kit to the PC via USB and attempted to initialize using the approximate site coordinates in minutes & seconds (30*25' N, 84*17' W) and the current system time (14:50 EST).  I briefly saw a satellite flicker from gray into red once or twice, but not enough to get any coordinates or even download an updated almanac?  Although I'm not sure about the latter b/c the satellites are moving slowly.

Aha, finally got a fix after letting it sit for a while.  Now, let's see if I can switch into my custom config and start the GPS app...  Here are the steps for that:

1. Turn off debug stream with "$PDME,13,5,0" command.
2. Exit GPS kit eval app.
3. Plug in PC's COM1 cable with null modem back into NMEA port.  (With left switch in left position.)
4. UwTerminal at 4800, no flow control.
5. Plug in 6V adapter, unplug USB (it interferes with serial).
6. Stream C:\Users\Mike\Documents\My Dropbox\FAMU\COSMICi\GPS FPGA app\NMEA command files\NMEA-off.txt
7. Next, show-backup.txt.  Yes, custom config is still in flash.
8. Next, select-backup.txt.
9. Next, hot-start.txt, & switch baud rate to 115,200.
10. Unplug COM1 cable.
11. Plug in pin 24 of J5 from DE3, power on DE3, unplug AC adapter.  That works!
12. At this point you can burn the Quartus .sof file to start the design running.

At this point, I added the capability to switch off the debug output.  I added a new PIO from the slider switches and added macros PRINTF() and PRINTF_R() to check that slider switch 1 is up before sending any output.  This worked.

Next we (Darryl is here) tried burning the design to the board (to the EPCS device) so it would run automatically on power-up.  This worked, but 20 seconds was (in this test) not quite long enough for the Wi-Fi to finish starting up.  Changed it to 30 seconds (for now) and added a countdown display on the 7-segment digits so that the user doesn't get too impatient.

We also cleaned up the formatting a bit on the NMEA-formatted output.  There is a minor existing weirdness where it doesn't finish receiving & processing the WIFI_READY message (from just before the script main loop is entered) until a subsequent command line is typed.  Maybe I didn't terminate that line properly?  Anyway, go back and figure that out someday.  Everything else works fine.

Fri., Nov. 18th

Plan for today:

1. Finish testing CTU with new connections.
2. Maybe add a PLL to the CTU for improved resolution of GPS time measurements.

Tweaking pin assignments some more:

• GPIO 0 (Wi-Fi) - Inner header
• Pin 11 = +5V power supply to Wi-Fi module.
• Pin 9 (GPIO0_D6) <-- Serial CTS input from Wi-Fi kit
• Pin 10 (GPIO0_D7) --> Serial RTS output to Wi-Fi kit
• Pin 12 = GND supply to Wi-Fi module.
• Pin 13 (GPIO0_D8) <-- Serial RxD input from Wi-Fi kit
• Pin 14 (GPIO0_D9) --> Serial TxD output to Wi-Fi kit
• GPIO 1 (GPS) - Outer header
• Pin 27 (GPIO1_D20) <-- PPS input from GPS kit, J4 pin 15.
• Pin 29 = +3.3V power output to GPS kit.
• Pin 30 = GND supply to GPS kit.
• Pin 31 (GPIO1_D22) <-- Serial RxD input from GPS kit, J5 pin 15 (UART0TX).
• Pin 32 (GPIO1_D23) --> Serial TxD output to GPS kit, J5 pin 16 (UART0RX).

...Then I spent the whole afternoon doing a lot of tinkering and fiddling which I'm not going to go into all the details of.  Suffice it to say, I discovered the following:

For some reason, pin 31 of GPIO 1 on the DE3 board seems to be "broken," in the sense that it pulls to 0 even when it's only being used as an input.  Thus, I'm now using pins 39+40 for Rx/Tx to GPS instead of pins 31+32.

It doesn't seem to work to communicate with the GPS kit directly though pins 15+16 of the kit's J5, perhaps because its MAX3222 can't be disabled; so instead, I have plugged the SparkFun level-shifter onto J6 (powered with +3.3V from DE3's GPIO1) and meanwhile I am also supplying +5V power to the GPS module from GPIO1 through pins 4+24 of J5.

Oh, and BTW right now the OCXO board (which I soldered the temporary SMA connector onto) is being supplied +3.3V from GPIO0, while meanwhile the Wi-Fi board is powered by +5V from GPIO0.  So we're using all 4 of the power outputs on the GPIO boards!

Here is a photo of the fully assembled and running CTU electronics:

Assembled CTU electronics.  The GPS module, level-shifter and
OCXO board are all drawing their power through the DE3 board.

Some next steps:

1. Add a compile-time (or better, DIP-switchable) option to disable debugging output.
2. Burn design to EEPROM.
3. Test power-up sequence.

Just for fun, I did a test with both the CTU and the FEDM powered up and talking to the server at the same time.  That worked fine.  This works because currently CTU is node #0 and FEDM is node #1, based on the nodeid.txt files loaded into their respective Wi-Fi boards (#3 and #1).

One more thing: The GPS seems to be still having trouble acquiring satellites; I may need to go into the DeLorme app and tinker around with it until it is healthy again.

Thu., Nov. 17th

Had a little time between meetings, so decided to come into the lab to test the connections on the newly-soldered 40-pin header on Wi-Fi board #3.  Actually, I realized I can't test the connections to J9 because that is on the other side of the 0-ohm resistor jumpers R7-R14, which are not presently populated (their pads are underneath the level-shifter chip U3).  So instead, I'll test by running the GPS app on the DE3 board.

Pins of J2 that we need for the serial are:

• Pin 10 (yellow wire)   - (From DE3) Autorun (DE9 pin 4) -> M_DSR (To WiFi)        - Optional.
• Pin 11 (black wire)    - GND                          (DE9 pin 5)
• Pin 19 (green wire)    - (From DE3) RTS       (DE9 pin 7) -> M_CTS  (To WiFi)        - Handshaking in (optional)
• Pin 21 (red wire)       - (To DE3)     RX         (DE9 pin 2) <- M_TX    (From WiFi)
• Pin 23 (tan wire)        - (To DE3)     CTS      (DE9 pin 8) <- M_RTS  (From WiFi)    - Handshaking out (optional)
• Pin 25 (blue wire)      - (From DE3) TX        (DE9 pin 3) -> M_RX    (To WiFi)
• Pin 27 (purple wire)   - (To J10 pin 2 /EN)                      <- +3V out  (From WiFi)    - Disables level-shifted input.-

To start out, I'm just going to connect pins 11 (GNS), 21 (M_TX), 25 (M_RX), and 27 (/EN).

Starting the server to prepare for the run.  (First I moved the last round of log files to a subfolder with today's date.)

I have to open the GPS app project in Quartus.  Did that (version in C:\f).  Plugged in USB.  Powered up board.  It's running the builtin demo.  Starting programmer.  Selecting USB blaster.  Burning image.  Wait, forgot to start Wi-Fi first.  Powering DE3 off.

On Wi-Fi, connecting pins 1+2 of JP2 to enable USB power.  Now it's powering up.

Looking at the top-level file (DE3_GPSapp.v), it looks like I am going to have to add CTS/RTS after all, because the design is expecting them.  OK, added those two wires.

Now, powering up DE3 again, and re-programming design.  Nuthin!  (Just "--" on 7-seg display)

I have to go over to ECE now.  Things to try tomorrow:

• Hook up GPS module
• Check baud rate
• Run design in debugger (in legacy Nios II IDE)

*     *     *

Came back in later in the evening to work some more on the DE3 board connections.  I've now greatly simplified the design of the connections.  I've eliminated the ribbon cable and serial cable and all of the wire-wrap.  I did some reassignments of the DE3's GPIO header pins to group related pins closer together.  I also figured out that I can eliminate the level shifter (I think) by connecting directly to the GPS kit's internal logic-level Tx/Rx nodes instead of going through its level-shifter and DE9 connector.  And I figured out how to power the GPS kit directly from the DE3 board using +3.3V out.

New pin assignments:

• GPIO 0 (Wi-Fi) - Inner header
• Pin 11 = +5V power supply to Wi-Fi module.
• Pin 27 (GPIO0_D20) <-- Serial CTS input from Wi-Fi kit
• Pin 28 (GPIO0_D21) --> Serial RTS output to Wi-Fi kit
• Pin 30 = GND supply to Wi-Fi module.
• Pin 31 (GPIO0_D22) <-- Serial RxD input from Wi-Fi kit
• Pin 32 (GPIO0_D23) --> Serial TxD output to Wi-Fi kit
  
• GPIO 1 (GPS) - Outer header
• Pin 27 (GPIO1_D20) <-- PPS input from GPS kit, J4 pin 15.
• Pin 29 = +3.3V power output to GPS kit.
• Pin 30 = GND supply to GPS kit.
• Pin 31 (GPIO1_D22) <-- Serial RxD input from GPS kit, J5 pin 15 (UART0TX).
• Pin 32 (GPIO1_D23) --> Serial TxD output to GPS kit, J5 pin 16 (UART0RX).

The Adafruit jumper wires currently being used are not colored optimally, since I was running low on colors, but I ordered some more of them and they should arrive soon.  I can replace the temporary ones I have now with more sensibly-colored ones when they get in.

I think the new GPS communication setup is working because I got a "Yo" on the 7-segment display, indicating a line was received.  Still need to test communication in the other direction though.  (Sending config commands from the DE3 to the GPS.)

Tomorrow I'll hook up the new Wi-Fi board (it's drying at the moment; I had to solder a header pin for +5V power input) and do a complete test of this new CTU setup.

Wed., Nov. 16th

Michael Dean came by my office yesterday and I answered some of his questions on the timing system.

Accidentally turned in the wrong timesheet yesterday (for the wrong pay period), so had to spend some time today running around getting the correct one (don't know why I didn't already have it), filling it out, getting it signed & turning it in.  The system for getting paid is so inefficient!

Today I want to try compiling with different versions of the Nios core and see how much space it saves.

As a baseline, the present design (with the Nios II/f) is as follows:

• Entire design (COSMICi_FEDM_top12):
• Comb. ALUTs:  10,017 / 27,104 (37%)
• Ded. log. regs.:   25,678 / 27,104 (95%)
• M512s:                   188 / 202       (93%)
• M4Ks:                    144 / 144       (100%)
• M-RAMs:                   1 / 1           (100%)
  
• SOPC system (FEDM_NiosSys):
• Comb. ALUTs:   2,324  (8.6%)
• Ded. log. regs.:    1,901  (7%)
• M512s:                  150  (74%)
• M4Ks:                   144  (100%)
• M-RAMs:                  1  (100%)
  
• Nios CPU (the_cpu_0):
• Comb. ALUTs:   1,396  (5%)
• Ded. log. regs.:    1,486  (5%)
• M512s:                      4  (2%)
• M4Ks:                     16  (11%)
• M-RAMs:                  0  (0%)

So at best, by changing the CPU to a much smaller one, we could bump up the available logic registers a bit (less than double), increase the available M512s from 14 to (at most!) 18, and increase the available M4Ks from 0 to (at most!) 16 (11% of the total number).  So it's not a huge difference, but it does potentially make some difference.  Need to keep this in mind if we run out of space again (or if we need to loosen things up for improved performance).  Later on, we can do some test compiles to see exactly how much space we can safe by moving to the Nios II/s or e.  (Note: It could however increase firmware size, due to the need for software multiply/divide instructions.  It could also reduce performance and lead to more dropped pulses.)

I had a thought earlier in the day, which is that we could add a PLL in the GPS app, to more precisely measure the PPS edge times relative to the OCXO clock.  The OCXO clock is 10 MHz, but it would be pretty easy now (given the code already written for the FEDM) to step this up to a much faster speed.  50 MHz?  100 MHz?  500 MHz?  It would be fun to try.

Samad came by and we tried to solder the other 40-pin DIP header onto the other Wi-Fi board (#3), but had some problems, due to my hand shakiness; solder got stuck in some thru-holes, blocking them.  I tried several methods for removal, but no luck yet.  We may need to try again with another board.  Or, if I can find a small enough drill bit, we could re-drill the holes.  (I looked through all my Dremel tool attachments, but didn't find anything suitable.)

Finally managed to work the solder out of the holes by hand using a tiny eyeglass screwdriver, then soldered the second breakaway header strip of 20 pins.  After all the work with the solder tools there is a lot of charring of the board surface.  I cleaned the board as best I could with the electronics cleaner spray, then rinsed in water, shook the drops off, and placed it to try overnight on the exhaust vent of the soldering fan.  Test tomorrow to make sure it's working OK.

Not sure there's anything else I can really do at the moment.  Tomorrow I will test connectivity on Wi-Fi board #3 between pins of J2 (with the new header) and corresponding thru-holes of J9.  Once that's working, I will do a new serial connection between Wi-Fi board #3 and the DE3 board's GPIO1 header using a bundle of Adafruit jumper wires, like I did for the FEDM.  Then I'll test that Wi-Fi connection by running the existing GPS app.

Spent a little time playing around with using some of the Adafruit extra-long M-M pin headers and F-F jumper wires to supply the +3.3V, +5V, and +12V power on a small breadboard.  The quality of the power supply connection is poor though - it keeps going out, and you have to hold it in just the right way to maintain steady power.  So, getting the right connector from Digi-Key for mounting on a real proto board will be key here.

Temporary hand-made connector/wire bundle for distributing power supply voltages on breadboard.
Color code: Black/white = ground; Orange/gray = +3.3V; Green = /ON; Red = +5V; Yellow = +12V.

I am a little concerned about distributing +5V power (which will account for the biggest part of power consumption, I'm guessing roughly 4A = 20W worth, since it will drive the FEDM, both Wi-Fi boards, the GPS module, and the fan) via the relatively narrow-gauge jumper wire shown here.  This could potentially represent a fire hazard if this wire overheats.  It would be safer to use a slightly thicker-gauge wire such as the one used in the actual power-supply connector.  Alternatively, we could do an analysis to show that there is no hazard even with this narrow wire.

Tue., Nov. 15th

Still need to requisition the batteries/charger for Samad.  First, check to see if the vendor is in the vendor list.  It's not.  Emailed him asking him to either re-source it or get the vendor added.

Pascal talked to the construction firm, and they referred him to the architecture firm for the more detailed engineering drawings of the room.  Michael, Juan, David and Darryl were here and worked on the changes for the new input capture datapath for a bit, then once all the Senior Design students were here, they all broke off for their internal team meeting (which they're now holding in the library conference room), and meanwhile Darryl and David worked with Ray on the paper.  Meanwhile, while all this was going on, I continued my testing of the new FEDM setup with the (now) 3 input-capture datapaths.

Re-testing the board with 100 Hz pulse source on SMA#1.

Current configuration ("Configuration #1") is:

• SMA#1: Tektronix function generator (100 pps, -2.5V pulses).
• SMA #2: Case #2 (near window).  -->   ~86 pps
• SMA #3: Case #1 (near hallway).   -->   ~196 pps

In this configuration (spreadsheet "C:\SHARED\Server Code\Pulse-Counts.ods"), the pulse rates (averaged over a period of about a minute) seem to be as shown above.

But, mega-weird!  If I switch cables on SMA#2 and SMA#3 ("Configuration #2"), I get:

• SMA#1: Tektronix function generator (100 pps, -2.5V pulses).
• SMA #2: Case #1 (near hallway).   -->   ~72 pps
• SMA #3: Case #2 (near window).  -->   ~303 pps

This data makes it look like, Case #2 is producing more and/or stronger pulses, but in configuration #1, this is overwhelmed by the fact that SMA #3 is responding so much more strongly.

Suppose we put the pulse generator on the apparently-weakest input (#2), what do we get?
(Configuration #3)

• SMA #1:  Case #1 (near hallway).   -->   ~172 pps
• SMA #2: Tektronix function generator (100 pps, -2.5V pulses).
• SMA #3: Case #2 (near window).  -->   ~287 pps

So, at least we seem to have some consistency in the sense that when a given case (#2) is on a given input, and the reference is moved between inputs, the measured pulse rate from that case stays the same.

Let's switch case #1 & #2 again.  (Configuration #4.)  Now we get:

• SMA #1:  Case #2 (near window).  -->   ~241 pps
• SMA #2: Tektronix function generator (100 pps, -2.5V pulses).
• SMA #3: Case #1 (near hallway).   -->   ~196 pps

Again, a particular assignment (Case#1 on SMA#3) gives a consistent rate (196 Hz) no matter which of the other two inputs the Tektronix is on.

Might as well finish up by doing the other two arrangements.
Configuration #5:

• SMA #1:  Case #1 (near hallway).   -->   ~168 pps.
• SMA #2:  Case #2 (near window).  -->   ~81 pps.
• SMA #3: Tektronix function generator (100 pps, -2.5V pulses).

Interestingly, the above is the only arrangement (so far) where I don't get FIFO_FULL errors - perhaps because the pulse rates are lowest.

Case #1 on SMA #1 is still about 170 Hz.

Case #2 on SMA #2 is still pretty close to 83-84 Hz.

Last arrangement (Configuration #6):

• SMA #1:  Case #2 (near window).  -->   ~238 pps.
• SMA #2:  Case #1 (near hallway).   -->   ~69 pps.
• SMA #3: Tektronix function generator (100 pps, -2.5V pulses).

Case #2 on SMA #1 is still about 240 pps.

Case #1 on SMA #2 is still about 70 pps.

Here is a summary of all the data (all figures are averaged pps):

	Case #1	Case #2
SMA#1	170	240
SMA#2	70	83
SMA#3	196	296

Some comparisons:

• On average, Case #2 generates ~36% more pulses than Case #1 (fairly wide range though, 18-51% more, depending on which input pathway we're talking about).
  
• On average, SMA#1 detects ~2.6x (2.4-2.9x) more pulses than SMA#2.
• On average, SMA#3 detects ~3.2x (2.8-3.6x) more pulses than SMA#2.

Note: It seems like on a RESET, the pulse buffer isn't getting cleared out properly - I get some junk when I start again.  Need to fix that at some point.

In terms of a close ratio between the two detectors, the best configuration is #4; the ratio of pulse rates there is only 1.2x (241 vs. 196 pps). 

The team swarmed in after their team meeting, and I clarified for them again what connects to what in the overall system design (CTU + detector electronics), and they took more photos in preparation for figuring out the exact arrangement of boards within the enclosure.

Samad and I found an appropriate connector on Digi-Key to connect to the cable from the power supply and mount on a breadboard or proto board for splitting out the power connections to the various destinations.  He is going to order one out-of-pocket, so he can begin playing around with the power supply setup.


Addendum:  It occurred to me on the drive home that if the students have difficulty getting the speed of the FEDM design back up to 500 MHz, one possible workaround (for Phase I) might be to jumper the comparator outputs over to the DE3 board (which has a larger and faster FPGA) and do the time-to-digital conversion over there.  This would also simplify the Phase I system design, since all of the firmware could just run on that board, and we could eliminate one of the Wi-Fi modules.  The high-speed clock produced by the PLL could be directly slaved to the OCXO input, which would eliminate the need for the extra timing sync capture datapath, and save some coding time associated with that (in the firmware & on the server).

Some downsides:

• We would need to add 15 jumper wires arching from the FEDM over to the DE3.  (5 DAC levels x 3 PMT inputs).
  
• The rise times on these signals would probably be relatively large (several ns maybe), and there could be nonuniform signal delays, leading to additional sources of error in the time measurements, errors which would be difficult to quantify since they would depend sensitively on the exact physical arrangement of the wires.  (We could do some simple experiments pretty easily though with a scope to get a rough idea of how bad these problems are.)
  
• There would be some additional design time involved in merging the two firmware applications (but on the other hand, we have yet to see how much design time will be required to get the FEDM performance back up to 500 MHz in the current setup - could be less, could be more).
  
• This alternative approach would less directly lead into the Phase II design (with the optical sync), since for Phase II we would effectively have to undo the merging of the two applications, and figure out again how to get the FEDM back up to the desired speed.  So in a way it just postpones solving that problem.

I'll ask Ray for some feedback on this idea, see if he thinks we should let the students explore it.

Ray says he'd prefer to continue pushing the current approach as far as we can first...

Emailed Sachin to see if he knows if we could access the EEPROM as a ROM.

Another idea:  Just checked the SOPC design, and currently we are using a Nios II/f, which takes 1,400-1,800 logic elements, plus 3 M4K blocks and then some more M4Ks for the memory cache.   If necessary, we could back off to a smaller Nios II/s (or maybe even Nios II/e) and save some of these resources.

Mon., Nov. 14th

Still need engineering schematics/blueprints for both halves of the classroom at the Challenger Center - ping Pascal (or whoever I see today) about that.  Juan emailed me that he is planning to come by today.

Samad sent me quotes for the batteries / charger the other day, for demonstrating a wireless detector node.  Need to run them by Ray, and see if we should go ahead and requisition them.

Still need to solder the 40-pin breakaway header onto the other Wi-Fi board (the board labeled "#3").

Still need to work on the timing of the startup sequence for the FEDM+WiFi board combination... What I started doing last time was adding a delay to the FEDM startup sequence, to give the WiFi board more time to finish starting up before the FEDM started sending a large amount of serial data.

Changed gelware and firmware back to the original configuration for a 64KB working_memory module/linker section in M-RAM, except that the firmware now has the 20-second startup delay compiled in - I need to test that part, at least.  Rebuilt firmware; now recompiling gelware.

Plugged in the detectors (on SMA #1&2) and the function generator (on SMA #3) and the +5V power connector for the FEDM+WiFi unit and the cooling fan.

Moved existing server log files to "C:\SHARED\Server Code\logs 2011-11-14" and started the server.

To really test the startup sequence, I will need to re-burn a new image to the EEPROM via Active Serial, so that the newest gelware/firmware will load+run automatically on startup.  OK, it is burned.

OK, the new 20-second delay seems to be working, and seems to be adequate (just barely) to allow the present Wi-Fi script to finish starting up first.  (This is assuming that there aren't any networking issues.)

Weird, for some reason today I am getting a bunch of FIFO_FULL errors on SMA#1 (currently connected to Case #2), and the event counts seem higher than usual there relative to the others.

I switched the input connections (#1 and #2) and that seemed to fix the problem - which shouldn't happen!  Although to be fair, I don't know if perhaps the problem was just that the PMT in case #2 was still warming up, I didn't plug it in as soon as I plugged in #1.  Guess I could try switching them back...

Hitting STOP and HSC_STOP.  Switched cables back (Case #2 -> SMA #1, Case #1 -> SMA #2).

Whoa, that is weird!  Whichever way the cables are, the pulse counts are highest on SMA #1!  In the current configuration, the ratio is larger between SMA #1 and SMA #2&3.  OK, switched back to (Case #1 -> SMA #1, Case #2 -> SMA #2).  That has a more evenly-balanced ratio of pulse rates, but we still have a faster rate on SMA#1.

Theory: The parasitics on SMA #1 (impedance of on-board trace) could be smaller than those on SMA #2, even though these are (I thought) all supposed to be controlled-impedance traces.

I probably should also double-check all the pin assignments and the wiring from the DACs to make sure that the same threshold levels are being compared in all cases.  (Particularly the 1st threshold which controls detection.)

Let's see, looking at the OrCAD schematic, PMT_1 and VTH1 come to the pin pair TDCIN[0]/TDCIN[0](N), which are M21 and M20.  PMT_2 and VTH1 come to the pin pair TDCIN[6]/TDCIN[6](N), which are U22 and U21.  Let's check the layout.  M21 [/], M20 [/], U22 [/], U21 [/].  Yep, those connections all check out.  Now let's check the Quartus design.  TDCIN0[0]/TDCIN0[0](N) are M21 and M20, while TDCIN1[0]/TDCIN1[0](N) are U22 and U21.  TDCIN0-2 are all I/O standard LVDS.  Port TDCIN0[0] goes to signal pf0[0] and TDCIN1[0[ goes to pf1[0], and these go to thresh_pulse[1] of the corresponding datapaths.  So, no problems there.

To try tomorrow: Move the input from the pulse generator to SMA#1 and see if the weirdness persists (or if there is some other kind of weirdness).

Wed., Nov. 9th

Ray and I are planning to go over to the Challenger center at about 4:00 pm, to meet with Reed to finalize the issue of the detector locations.

Samad is coming by soon to talk about the power supply some more.  I emphasized to him the importance of building proper connectors to connect the power supply (and batteries) to other components.

Samad needs to get me quotes for the batteries, battery charger, and any connectors so that I can issue a requisition for them.

Also, the mechanical students need to design structures to hold the power supply INSIDE (not outside) the main electronics box, for neatness, and there should also be a little battery-holder box installed on one of the detectors (for a battery-powered detector demonstration).  Mechanicals need to coordinate with Samad on the requirements for this.

Cooling of the ambient interior temperature of the main electronics box should not be a problem, even with the power supply inside, as long as there is airflow with vents/fan.  Total system power dissipation is not all that large (20-30 W, laptop range).  Main requirement for cooling is a good heat pipe from the FPGA to the Peltier plate, to get the FPGA as close as possible to the cool end of its 0-85 C operating range.  And positioning the plate so any drips go into a tray and not onto the electronics.  Mechanicals should look into sealing all cold exposed metal parts inside a polymer/resin skin, spray-on perhaps, to reduce condensation.

Reed had to leave CLC before we arrived, but his assistant was there.  It looks like hanging the detectors below the ceiling is OK as long as they do not dangle down more than about 1' or so - ceiling height is 9'2".  Main concern is that people don't hit their heads.  Also, there are sprinkler clearances and location of above-ceiling girders to consider.  Requested blueprint for other half of room - Brian said Pascal had one half already, but it needs to be scanned and uploaded to the group blog so everyone can see it.

Brian is thinking now of arranging all boards on one plane, instead of using the blade approach.  This is probably a good idea since it will reduce the vertical profile of the enclosure.   Height of enclosure should probably be no more than 14".

Also, I discussed with Brian that the enclosure should NOT have RF shielding, since the Wi-Fi communication needs to pass through.  I don't think we're too worried about interference.  Also for educational purposes it would be nice if the electronics was visible.  We agreed that probably best is if 5 out of the 6 faces of the enclosure were transparent plexiglass (except where the exhaust fan and Peltier cooler are installed, obviously), and the other face can be metal that all the boards are bolted to (everything mounted upside-down under it).  That way, students can see all the electronics, and any indicator LEDs on the boards.

Tue., Nov. 8th

Ray and I are planning to go over to CLC at about 4 pm on Wednesday to talk with Reed about positioning of the detectors.  I created a Google Calendar event and sent them both an invite.

Ray points out that the highest-energy showers have a narrower area at ground level, so placing the detectors as far apart within the building as possible might not be the best approach.  He is going to ask Helio to help us find information about how the shower area varies as a function of primary particle energy.

Some things I could work on today:

• [/] Requisition for Tektronix BNC cables - submitted
• [,] Shrink existing design to make room for timing sync capture datapath (reduce code size, use more of M-RAM)
• [ ] Solder headers to 2nd Wi-Fi board, wire up its serial connection
• [ ] Reduce startup time of WiFi script / Increase startup delay of FEDM

I guess the quickest way to get the startup sequence working is just to add a delay to the FEDM firmware - we can always go back and optimize things later to reduce the delays on both sides (WiFi & FEDM).

Added a 20-second startup delay to the firmware in Q:\software_v4\FEDM_ctrl_fw; this firmware version is 0.9.

Incidentally, the linker space usage summary is as follows:

Info: (FEDM_ctrl_fw.elf) 69 KBytes program size (code + initialized data).
Info:                    62 KBytes free for stack + heap.

This might be a good time to try shrinking the code size.  Let's compile with space optimization.  In Nios II Design Tools for Eclipse, right-click fedm_ctrl_fw_bsp, select Nios II --> BSP Editor.  No, that's not it.  Select Properties instead from the pop-up menu.  Click "Nios II BSP Properties".  Select optimization level: Size.  Hit OK.  Select Nios II --> Generate BSP.  Select Project --> Build All.  Hm, that didn't seem to work - not enough files were recompiled.  Let's click "Generate" in BSP editor, and rebuild.  Still nothing.  Let's do a Project -> Clean, and clean all projects.  That'll force the issue.  OK, that does it - all files are getting recompiled with -Os.  Now we get in the linker summary:

Info: (FEDM_ctrl_fw.elf) 68 KBytes program size (code + initialized data).
Info:                    62 KBytes free for stack + heap.

Geez, we only saved 1 KB?  That's hardly worth it.  I think I'll change it back to the way it was.  (-O3 not -Os).  OK, that's changed back.

Next thing to try:  See if we can utilize the extra 8K of M-RAM (parity bits, not presently used).  Opening Q:\COSMICi_FEDM project, New_with_Nios_trim revision, SOPC builder.  Changing size of working_memory module from 64K to 72K.  This might let us get rid of extra_RAM.  System generation worked.   Extra_RAM is currently 9KB, and it uses M512 blocks, so getting rid of it might free up some space for other functionality.

OK, I went into the BSP editor, linker script tab, and changed the size of the working_memory linker region to 73,728 bytes (72 KB), and regenerated the BSP files.  After rebuilding, new linker stats are as follows:

Info: (FEDM_ctrl_fw.elf) 69 KBytes program size (code + initialized data).
Info:                    70 KBytes free for stack + heap.
So that worked as intended; we have an extra 8 KB now available in the stack + heap working memory.
So, the stuff that's currently going in extra_RAM could be shoved into there, maybe - except the problem with the M-RAM is that it can't be initialized!  So I'm not sure it really can be used for anything but stack & heap, unless we somehow work around the no-initialization restriction.  I guess .BSS sections can go in there too, but that's about it.  Damn, I just checked the linker script settings, and .BSS is already in there!

Darn, when I tried the Quartus compile, the fitter says it needs another M-RAM!  Maybe it can't allocate the extra 8K of parity bits as normal memory?  Let's see if it helps to change the word size to 16 bits (from 32 bits).  Nope, still no dice.  Sounds like I am going to have to stick with using 64KB of M-RAM, unless we find some other way to use the rest of it.  Also, when we try to generate with smaller word sizes, we get the following scary warning:

Warning: cpu_0.data_master/working_memory.s1: working_memory.s1 does not have byteenables. Narrow (less than 32-bit) writes from cpu_0.data_master will result in spurious writes to working_memory.s1

In other news, Juan and Michael Dean were here today, and I had them start work with Darryl & David on the new datapath for the timing sync capture.  Juan & Darryl are working on the new PIOs and the firmware changes, while Michael & David are working on the Quartus changes.  I copied the latest code (gelware & firmware) to the FEDM_design\FEDM_code\q91 folder so they could be working with the latest versions of everything.  Everyone got off to a good start today, but there is a lot left to do.  Juan & Darryl will give specs to Michael & David as to the PIO interface.

Aarmondas did not show up as promised.  Going to send him a strongly worded e-mail.

Also, Ray finished his data collection run that he started last night, in which the scintillators were overlapping at their centers.  It didn't change the location of the histogram peak much, for coincidence pulses due to showers propagating down through both detectors, so we are still thinking that 200 mV is the right level for the thresholds.

Speaking of which, currently the furthest threshold (DAC #6) is -1000 mV (off of a +2.5V DC bias) or in other words +1.5V absolute.  Thus, DAC #6 would be appropriate to use as the rising-edge threshold for the timing-sync input, since that pulse is going to be at (I believe, if I recall) the +3.3V logic level of the DE3 board.

Mon., Nov. 7th

Dr. O'Neal is out today because of a family medical situation.  Unfortunately, timesheets are due today because of the Veteran's Day holiday this Friday.  I don't know if Dr. O'Neal will be back soon enough that Sonja can still process timesheets for this pay period.  Darryl and I left ours in a folder on his office door so that he will see them as soon as he gets back to campus.  UPDATE: He texted he's on his way in; but, if he's not here by about 4:00, I'll run the timesheets over to Sonja unsigned so that she has time to enter the data.

George C. and Pascal D. (two of the ME Senior Design students) will be in the lab later this afternoon, around 3:30 or 4 or so.  I guess we'll discuss any issues relating to the enclosure/supports, and maybe they can do some temperature measurements.

I finished reading and marking up the Senior Design students' Project Proposal report over the weekend, and will give it back to them with my comments on it, along with my evaluation sheet, on Thursday.

I'm going to email Reed about the above/below ceiling issue, and the issue about him limiting us to only use half the classroom .  OK, did that; CC'd Ray and Michelle.

Some priorities to work on this week, from my perspective:

1. [ ] Start experimenting with LogicLock, for purposes of getting the speed back up to 500 MHz.  I should maybe also consider reverting to the original design with the double-edge-triggered carry-save counters, if that can be done while preserving high frequency.  However, I perhaps need to be careful here not to duplicate work of the Senior Design students too much.  However, it's difficult for them to do all of this work by themselves outside the lab, since they need the Stratix II license to be able to do the timing analysis.  Not quite sure yet re: the exact division of labor here.
   
2. [ ] Aarmondas is supposed to come by this week (Tuesday, with Juan and Michael D.?) so that we can start going over the design of the new input-capture datapath for the timing sync edges.  The interns (Darryl & David) can hopefully help w.r.t. bringing the new students up to speed on this.
   
3. [ ] We need to assemble the 3rd detector apparatus (gun case), and do some test runs with 3 detectors.  One thing we're missing is a coaxial cable to go inside the gun case, of the exact same length and type as the existing cables.  Their part number is 012125600 from Tektronix.  Sent them an RFQ thru their website.
   
4. [ ] The serial connection from the FEDM board is in pretty good shape now, but at some point I need to start working on similarly cleaning up the serial connections from the DE3 board (GPS app board) to its Wi-Fi board and GPS module.  
5. [ ] Also, we might want to remove the GPS kit's board from its protective (yellow and black plastic) enclosure since that will be redundant once it is inside our overall electronics enclosure.  The rubber feet on the enclosure can be removed to reveal the screws that need to be unscrewed to take apart the enclosure.  Sometime, we also need to address how the GPS kit will be powered exactly (barrel plug vs. USB connector, vs...)

I used the label maker to make some nicer-looking labels for gun cases #1 & #2.

George came by and took temperature measurements of the FPGA and the voltage regulators while the current FEDM firmware was running (without the cooling fan).  Of course, without the fan, the FPGA kept overheating and resetting itself, but at least this gives us some idea what the peak temperature would be currently without a cooling system.  The FPGA got up to about 50 C (122 F).

Also, George asked for mechanical details for all the boards.  I had given this information to Pascal earlier (I think I emailed him the docs), but for some reason Pascal apparently never passed this information along to George, because George didn't know about it!  (Tsk, tsk!!)  So this time, I just posted the key documents on the group blog so that everyone will have it.  The students really need to do a better job of sharing all pertinent project information with each other!  Everything relevant needs to go on the group blog, so everyone will have it!

George also didn't have access yet to the FEDM_design Dropbox folder, so I added him to the share list.

Pascal couldn't be here today because of a doctor's appointment.

George said he would come by again later in the week.

Changed nodeid.txt of EZURiO board #1 from 0 to 1, in preparation for running it simultaneously with EZURiO board for the CTU which will have node ID 0.  This was somewhat involved:

• Remove blue wire (M_RX) from pin #25 of J2 header of WiFi board to disable input from FEDM.
• Remove purple wire (+3V out) from pin #27 of J2 and connect it to pin #1 of J10 (GND) instead.  This re-enables external serial input from PC.
• Remove yellow wire (M_DSR in from FEDM) to disable script autorun
• Connect serial cable to COM1, start up UwTerminal, with DTR checked (no autorun).
• Power up FEDM (supplying +5V power to WiFi board).
• Use UwTerminal to Add File with the new version of nodeid.txt.

After this, oops, the AUXIO and UART bridge connections are no longer successfully opened, although the MAIN connection is still opened!  This may be a problem with the port numbers.  Let's examine C:\SHARED\Server Code\mainserver.py.  Oh, that's right, it dispatches command handling to commands.py.  The POWERED_ON message goes to the handle_node_on() method of CommandHandler.  That in turn dispatches to the sensor-net model, in model.py.  turnOn() is called on the newly-created node data structure, and this calls _setupBridgeServers() which adds the node number to the base port number.

Checked the server logs and it seems the port number is getting incremented twice for some reason.  Aha, bridge.py increments it again, and we are passing in the already-incremented number.  OK, fixed that problem.  Restored the normal connections between FEDM & WiFi board.

I had a task from last week to decrease the startup time for the WiFi board, so that the FEDM doesn't get hung up trying to communicate with it before it's ready.  Some ideas for this:

• Turn off info-level logging to the server in the WiFi script.
• Turn off the default help menu display (user can always type "help" to see it anyway, if needed).
• We can probably also somewhat decrease the 1-second delay where we wait for the server to open listeners on the bridge ports allocated for the node after receiving the node's powerup message.  Half a second may well be enough if the server isn't heavily loaded.  Anyway, the connection attempt should automatically retry for a while, so even if it doesn't work the first time, no big deal.

Also, we can add a delay to the FEDM, and/or add a protocol whereby the FEDM actually waits to receive the startup message from the WiFi module, and/or bounces a ping off the server.

Also, note to self: After the node number that's active on a given MAIN server connection is identified, the title of that connection window should really be updated to reflect this. (Add "(Node #1)" or something like this to the window title.)  This should be a pretty easy change to the server code.

Sat., Nov. 5th

I am wondering whether it might be best to locate the detectors farther apart in the building at CLC, to improve our angular resolution.

I got a building outline from the Leon County GIS.  It is not very detailed, but it gives you some idea of the building size and layout.  Here are a couple of views (click to zoom):

Building plat with image overlay from
Property Appraiser GIS.  This one has
dimensions of various line elements.

Building plat with image overlay from
Base Map GIS.  This one has terrain contours.

I have overlaid on these a triangle showing one (potentially) possible arrangement of detectors for maximum spacing.  The long side of the triangle is about 185' and the shorter sides are about 150'.  Altitude of the building is about 190' above sea level.

Here is another, slightly higher-resolution image from Google Earth:

Another aerial building image, from Google Earth.

For future reference, the location of the push-pin is 30°26'25.32"N,  84°17'0.22"W.  Google agrees that the altitude there (where the ground used to be, I assume) is about 190' above sea level.

Finally, here's another, even higher-resolution, but skewed, image from Google Maps:

Aerial or satellite image from Google Maps.

According to Google Maps, the coordinates of the green arrow are 30.440406, -84.283404.  Converted using http://www.gpsvisualizer.com/calculators, that is N30°26'25.462", W84°17'0.254", i.e., within less than 1 second of arc of the Google Earth coordinates from earlier.  A second of latitude is 101 feet and a second of longitude at this latitude is about 87'.  Based on the differences of 0.142" and 0.034", we're within ~14' on the N-S axis and about 3' on the E-W axis.  The larger N-S difference can be accounted for by the fact that the aerial photos were taken at different angles, so it was hard to eyeball where the green arrow should be.

After we install our GPS unit, we should take long-term averages of the position readings to get a more accurate idea of the position of the antenna.  (We could include code for this in the server, and test it here in the lab.)

Fri., Nov. 4th

The Senior Design students gave their Project Proposal presentation today.

Samad (EE guy) & the ME students proposed to distribute wired +12V power from the DE3 supply to the detectors via conduits above the ceiling tiles.  That sounds fine.  The coax signal cables (for phase 1) can also follow the conduits the other way, back to the central electronics unit.

There seems to be a disconnect with CLC regarding whether the detectors will be visible to the public - Reed wants to put them above the ceiling panels - but the whole point of locating them in the classroom to begin with is so that they would be visible to students - Ray and I need to talk to Reed and straighten this out.  Reed may not be aware of what the original agreement with CLC was.  Also, Reed wanted the detectors in only half of the classroom, but we need them to be spread out as far as possible to improve our angular resolution.  If they can't hang below the ceiling in one room, then they might as well be in different rooms from each other, or even farther apart, on opposite sides of the building, say!  

Brian Kirkland is coming by the lab on Monday to discuss the cooling system design (which needs some work), and the computer engineering students (including Aarmondas) are coming by on Tuesday to discuss next steps on the gelware, such as the new timing datapath and the performance optimizations.

Also on Monday, Ray and I may work on putting together the 3rd detector.  We have another long BNC-to-SMA 50-ohm coax, but we need another short 50-ohm BNC-to-BNC coax to go inside the gun case (same length as existing ones).  A new cable of the right length may still need to be ordered.

Turns out, according to Reed, there is an easy solution for the GPS antenna (run it along the same path as the connections to the roof solar panels), but we probably still need to order a new antenna with a longer antenna cable.  Need to contact DeLorme about this to see if we can order the new antenna cable through them, or if they can recommend a supplier of these.

I spoke with Michelle (director of CLC) this morning at the COE faculty meeting, and thanked her for her support, and apologized for missing Reed's replies to my earlier emails (which happened when my FAMU email was failing to be pulled by my gmail due to an outdated password).

Wed., Nov. 2nd

Plan for today is as follows (or at least, as far as we can get on this):

1. [/] Recompile latest firmware into gelware image (.sof/.pof files).
2. [/] Burn new .pof file to EEPROM (set as baseline code to revert to on power-up).
3. [/] Import files from Juan & Mike's new 56-bit version of datapath into master project folder.
4. [/] Do a test compile; fix any problems.
5. [/] Inspect space remaining (all types) after fitting.
6. [/] Check maximum PLL clock speed in timing analyzer.  Increase PLL clock speed if possible.
7. [/] Determine whether further compression of datapath will be necessary to add 3rd datapath.
8. [/] If not, add 3rd pulse-capture datapath to design
9. [/] Wire up LVDS inputs for "PMT #4" input to 3rd datapath (if not already).
10. [/] Recompile design with 3rd datapath, check space remaining & maximum clock speed again.
11. [/] Use Dremel tool to file down a 3rd SMA connector, populate it on "PMT #4" footprint.
12. [ ] Work with Ray to set up 3rd paddle/guncase assembly, connect it to board.
13. [/] Adjust firmware to look for data on all 3 input capture datapaths (currently it just checks 2).
14. [,] Do a first run gathering data from all 3 detectors!  This will be the acid test of the new 56-datapaths, to make sure that none of those design changes screwed up the functionality.

Here we go:

Task 1.  Starting Quartus license server.  Starting Quartus on Q:/COSMICi_FEDM.qpf project, New_with_Nios_trim revision.  Starting full compile with the latest .hex files (hexadecimal memory initialization files containing the latest compiled firmware).  This will take a while...

It occurs to me that it might be nice, at some point, to introduce a version identifier for the gelware too; this could be implemented as (for example) an LPM_CONSTANT megafunction that feeds a PIO input to the Nios core, which the C code can read and then report in the FEDM_STARTING message.

Compile successful (of course; nothing changed but the .hex files).  Some stats:

• Slow corner fmax for high-speed counter:  211.28 MHz
• Logic utilization:  24,314/27,104 (90%) =  2,790 remaining
• Dedicated logic registers used:  21,878/27,104 (81%) = 5,226 remaining
• M512 blocks: 193/202 (96%) = 9 remaining
• M4K blocks: 144/144 (100%) = 0 remaining
• M-RAM blocks: 1/1 (100%) = 0 remaining (but we may be able to use a bit more of the space)

Task 2.  Starting C:\SHARED\Server Code\COSMICi_server.py; it pops up its windows.  Powering up Mastech supply at ~4.5V driving fan (on metal block on FPGA), and then powering up Agilent supply at 5V driving FEDM + (via FEDM) WiFi board.  Total current draw: 2.11A.  Actually it just occurred to me I could power the fan from the same supply.  Now the total current draw is about 2.21A peak (fluctuating due to the EMF feedback from the fan motor).  Hope the voltage level isn't too unstable for the application.  It seems to be fine.  A new set of windows popped up of course when I power-cycled (b/c the Wi-Fi drops the old connections).  OK, let's burn the new firmware in there.  Switched burner to Active Serial mode, selected New_with_Nios_trim.pof file, added ByteBlaster hardware, doing program/configure + verify.  Finished programming; new firmware (0.7) is now installed and running.

Task 3.  I think I put Juan & Mike's new code into the C:\LOCAL\q91_56\ folder.  I need to copy all the _56 design files from there to Q:\, and then merge their changes to the toplevel schematic into a new version of my toplevel schematic, and then set it as the new toplevel file.

Let's do this in Cygwin.  cd to /cygdrive/c/LOCAL/q91_56 and type "ls *_56.*", and get:

COSMICi_FEDM_top9_56.bdf
FIFO_reader2_56.bsf
FIFO_writer2_56.bsf
cs_combine_56.bsf
cs_combine_56.vhd
cs_combine_56.vhd.bak
fifo_reader2_56.vhd
fifo_reader2_56.vhd.bak
fifo_writer2_56.vhd
fifo_writer2_56.vhd.bak
hspeed_counter_56.bsf
hspeed_counter_56.v
hspeed_counter_56.v.bak
our_fifo2_56.bdf
our_fifo2_56.bsf
our_fifo_v3_56.bdf
our_fifo_v3_56.bsf
pmt_ic_datapath2_56.bdf
pmt_ic_datapath2_56.bsf
pmt_ic_datapath_v3_56.bdf
pmt_ic_datapath_v3_56.bsf
pulse_cap_56.vhd
pulse_cap_56.vhd.bak
pulse_combine_56.bsf
pulse_combine_56.vhd
pulse_combine_56.vhd.bak
pulse_prep_56.bdf
pulse_prep_56.bsf
pulseform_FIFO_v2_56.bsf
pulseform_FIFO_v2_56.qip
pulseform_FIFO_v2_56.vhd
pulseform_cap_56.bdf
pulseform_cap_56.bsf
se_pulse_cap_56.bsf
se_pulse_cap_56.vhd
se_pulse_cap_56.vhd.bak
se_reg_en_56.vhd
se_reg_re_56.vhd
se_reg_re_56.vhd.bak
stream_pulse_data_56.bsf
stream_pulse_data_56.vhd
stream_pulse_data_56.vhd.bak

Great.  Now doing "cp *_56.* /cygdrive/q".  That is done.  Now in Quartus, opening for inspection "COSMICi_FEDM_top9_56.bdf", which is their new top-level file.  Basically, the only thing that should be changed in it is the new datapath implementations, pmt_ic_datapath2_56 (M512 fifo block type version), and pmt_ic_datapath_v3_56 (Auto fifo block type version).  

Oops, they forgot to create pulseform_FIFO_56 using the M512 blocks, use pulseform_FIFO_56 in their our_fifo2_56, and use pulseform_FIFO_v2_56 in their our_fifo_v3_56.  Did that for them.

Created new _top11 version of top-level schematic, added appropriate version comment, replaced the two ic datapath blocks with their new versions. Also replaced hspeed_counter with hspeed_counter_56.

Task 4.  Doing test compile...  Oops, forgot to replace the high-speed counter with new version.  Fixed that.  Forgot to add some .vhd components (not automatically included) to project.  Fixed that.  This is going to take a while, so I'm going to get a soda...

Compilation successful!

Task 5.  Let's look at the stats:

• Slow corner fmax for high-speed counter:  213.13 MHz (only slightly better)
• Logic utilization:  22,007/27,104 (81%) =  5,097 remaining (almost 2x as much left)
• Dedicated logic registers used:  19,463/27,104 (67%) = 7,641 remaining (about 25% better)
• M512 blocks: 188/202 (93%) = 24 remaining (more than 2x better)
• M4K blocks: 144/144 (100%) = 0 remaining (same as before)
• M-RAM blocks: 1/1 (100%) = 0 remaining (same as before)

This is, I think, about as I had expected. 

Task 6.   It's not yet possible to increase the PLL speed.  Will probably need to use LogicLock first.

Task 7.   Let's look at the resource usage of each ICDP now:

pmt_ic_datapath2_56 (M512 version, 4 pulses):

• Block memory bits:   2,700
• Combo ALUTs:          2,442
• Logic registers:          5,598
• M512 blocks:             38

pmt_ic_datapath_v3_56 (Auto version, 8 pulses):

• Block memory bits:    0
  
• Combo ALUTs:           2,410
  
• Logic registers:           11,576
  
• M512 blocks:              0

I don't think further compressions of the datapath will be needed.  The actual FIFO in the latter one is 6,083 logic registers, so if we cut it in half, we might save about 3,000 logic registers, I think at this point we might be able to fit two copies of it in the design.  Let's try.

Task 8.  Added 3rd input capture datapath on channel 2 of type pmt_ic_datapath_v3_56.

Task 9.  Wired up its LVDS inputs, skipping the one for DAC #2, as with the other datapaths.

Task 10.  Starting Quartus compile.  Compile successful!  Let's look at the stats:

• Slow corner fmax for high-speed counter:  212.49 MHz (only slightly better)
  
• Combinational ALUTs:  10,017 / 27,104 (37%)
• Dedicated logic registers used:  25,678/27,104 (95%) = 1,426 remaining
  
• Logic utilization:  26,664/27,104 (98%) =  440 remaining
  
• M512 blocks: 188/202 (93%) = 24 remaining (same as with 2 datapaths)
  
• M4K blocks: 144/144 (100%) = 0 remaining (same)
  
• M-RAM blocks: 1/1 (100%) = 0 remaining (same)

We probably will still need to free up some space before we can add the extra timing-sync edge-capture datapath.  We might conceivably be able to do that by getting rid of extra_RAM, and using more of the M-RAM (extra 8K in there, not currently allocated to anything).  If not, then we will have to shrink down the main input capture datapaths (and/or the firmware code size) further.  We could recompile with optimization for minimum code size turned on, and see if that lets us shrink the size of our RAM blocks.

Task 11.  While compile was running, Dremeled down the pins of another SMA connector to fit the thru-hole footprint.


Task 12.  Ray is out swimming.  Texted him that we need the 3rd guncase soon.


Task 13.  Back now in the Nios II design tools for Eclipse IDE, changed the value of NCHANS_TOBUF from 2 to NCHANNELS (i.e., 3) in pulsebuf.c.  This should cause the coincidence detector to now expect to receive pulses on all 3 input channels, and look for coincidences between any pair of them.  (Further filtering for 3-way coincidences can be done on the server side.)  I'll wait to recompile with this option until the Quartus compile is working.  OK, that's done now; recompiling the project.


Task 14.  Testing will have to wait until we get the 3rd detector hooked up.  I want to talk to Ray about that before putting it together to make sure it's OK with him to use a given paddle.  Well, we could hook the board up and test it with just two detectors, but that will confuse the coincidence-detection algorithm.  In the meantime, while waiting for the 3rd detector, I could perhaps start working on adding stuff to the gelware to allow us to positively verify the absence of a pulse in a given time interval.  Actually, it might be a good idea to first just do a test with the 56-bit version using just the first two channels.  Or, I could synthesize fake pulses on the new channel!  That last idea is probably best.

Doing another Quartus compile to pull in the new .hex files (to support the triple datapath).

Just remembered there are a couple more places in the firmware where I assumed there are only 2 input channels.  Going back & fixing those now...  In the NONCOIN_PULSES message...  Was that it?  I think so... Recompiling firmware & Quartus design again.  In the last compile, Quartus hung in the assembler (as it sometimes does).  Killing it and restarting.

Meanwhile, I hooked up the function generator to the "PMT#4" input port (which we're using as the new channel 2).  It is now outputing a 100 Hz, inverted, +2.5V pulse with 100 ns pulse width, 20 ns rise time, and 130 ns fall time, which makes it an asymmetric triangle (sawtooth) pulse.  Checked the pulse profile on the scope to make sure it looked OK.

Just remembered, I forgot to add the DC bias jumper for the new input port.  Checking the OrCAD schematic to confirm which one it was.  It is J63.  Adding an orange Adafruit F-F jumper there (I am out of bridges). Now I am ready to test, as soon as the Quartus compile finishes.  There, that's done.  JTAG programming new design... It's running.  Seems to be working as expected.  Here's an output sample:

Wed Nov 02 18:00:16 2011 + 636 ms: <
Wed Nov 02 18:00:16 2011 + 637 ms: < FEDM_STARTING,v0.8
Wed Nov 02 18:00:16 2011 + 842 ms: <
Wed Nov 02 18:00:16 2011 + 842 ms: < DAC_LEVELS,-0.200,-2.500,-0.299,-0.447,-0.669,-1.000
Wed Nov 02 18:00:16 2011 + 843 ms: <
Wed Nov 02 18:00:16 2011 + 843 ms: < FEDM_READY
Wed Nov 02 18:00:17 2011 + 915 ms: < NC_PULSES,245837961,201,100,123
Wed Nov 02 18:00:19 2011 +  37 ms: < NC_PULSES,469189411,203,90,112
Wed Nov 02 18:00:20 2011 + 223 ms: < NC_PULSES,706924408,201,90,119
Wed Nov 02 18:00:21 2011 + 456 ms: < NC_PULSES,954722914,205,94,123
Wed Nov 02 18:00:22 2011 + 670 ms: < NC_PULSES,1196743610,203,107,122
Wed Nov 02 18:00:23 2011 + 793 ms: < NC_PULSES,1422506353,202,98,112
Wed Nov 02 18:00:24 2011 + 873 ms: < NC_PULSES,1638620374,200,96,108
Wed Nov 02 18:00:26 2011 +  66 ms: < NC_PULSES,1876744706,200,94,120
Wed Nov 02 18:00:27 2011 + 371 ms: < NC_PULSES,2136745128,202,121,130
Wed Nov 02 18:00:28 2011 + 756 ms: < NC_PULSES,2414745580,203,104,139
Wed Nov 02 18:00:30 2011 +  53 ms: < NC_PULSES,2674182865,204,108,129
Wed Nov 02 18:00:31 2011 + 226 ms: < NC_PULSES,2908746387,203,91,118
Wed Nov 02 18:00:32 2011 + 354 ms: < NC_PULSES,3134746760,204,81,113
Wed Nov 02 18:00:32 2011 + 849 ms: < PULSE,2,1,3233112444,3,(0,(1,(1,3),4),2)
Wed Nov 02 18:00:32 2011 + 850 ms: < PULSE,1,1,3233112445,2,(0,(3,1),6)
Wed Nov 02 18:00:33 2011 + 493 ms: < NC_PULSES,3361671587,200,92,113
Wed Nov 02 18:00:34 2011 + 836 ms: < NC_PULSES,3630432436,201,116,134

Explanation:  The verbose timestamps at left (before the ":") are just when the server logged the data; they are not particularly accurate as event times (at best, to the 10s of milliseconds).  The "<" means this is input from the board (as opposed to output to the board on the return connection).  The FEDM_STARTING message says that the board has powered up, and gives the firmware version number (0.8 is the new version for 3 datapaths).  DAC_LEVELS message is there as a diagnostic to tell us what are all the voltage thresholds we set (relative to the +2.5V input bias).  This is a logarithmic ramp from -200 to -1000 mV, except that the 2nd value is null (-2.5) because that DAC is broken anyway.  FEDM_READY just tells us the firmware is entering the main loop.  NC_PULSES tells us how many non-coincidence pulses we have tallied up from each input channel as of a given (fairly arbitrary) point in time; this time is marked approximately by the arrival time (first threshold edge-crossing time) of the most-recently-arriving pulse in the tallied set, expressed in cycles of the high-speed counter.  PULSE expresses actual data about a pulse that is potentially a member of a shower, i.e., it arrived within +/- 50 ns of a pulse on another channel.  The fields are: Input port number (1, 2, 3); sequence number (out of coincidence pulses reported on this port), first edge-crossing time in HS clock cycles, number of thresholds crossed (1-5), time deltas (in HS cycles) between subsequent edges.  The ones after "(" are rising edges; the ones before ")" are falling edges.

A little later in the dataset (8 minutes later), we see an accidental coincidence between one of our synthesized pulses on port 3, and a real pulse on port 1.  This is evidence that coincidence detection is working between any pair of ports (not just ports 1&2) and that as expected the synthesized pulse crosses all 5 thresholds.  Its total measured width is 38*5 = 190 ms, which is about right considering its FWHM is synthesized at 100 ns and that we miss a little of the base (before it gets to -200 mV).

Wed Nov 02 18:08:21 2011 + 129 ms: < PULSE,1,17,96886924398,2,(0,(2,2),7)
Wed Nov 02 18:08:21 2011 + 130 ms: < PULSE,3,1,96886924408,5,(0,(1,(0,(0,(1,25),4),4),2),1)

One final note:  On power-up, the Wi-Fi script currently does not start up quite quickly enough to catch all the initial output from the FEDM board - so the earliest output (including the startup message and DAC levels) gets lost.   To prevent this, the FEDM firmware needs to be changed to actually wait to receive the WIFI_READY message before it turns on the input capture datapaths and starts streaming data.  However, this could create a problem in cases where the WIFI_READY message has already been sent earlier (like when restarting the FEDM via the JTAG).  There are a couple of ways to handle this:  (1) With a timeout, or (2) if WIFI_READY isn't seen within an expected interval, have the FEDM send a "PING" message to the server, which responds with a "PONG" that gets relayed via the Wi-Fi script to the FEDM, thereby confirming to the FEDM that the wireless link is up and running.  I can also work to reduce the Wi-Fi script's startup time (a lot of it right now is just unnecessary diagnostics), or (perhaps simplest) add a fixed delay in the FEDM firmware to give the Wi-Fi board time to start up.  Think about the various possible combinations of options here, & start working on them another day.

In other news, I started looking at the Senior Design students' report, & writing comments on it.  Emailed it to Ray and he also read it.  Ray (and I) noticed some lingering confusion about the project requirements in a number of places, and a general lack of specifics about the proposed cooling system design.  The students' presentation is scheduled for Friday, so we can give them our verbal feedback then.  I am hoping to get my written feedback done by tomorrow, so we can return the graded reports in class.