Friday, December 30, 2011

Fri., Dec. 30th

Felt like doing a little work today (it was more appealing than cleaning up my desk, I guess).

Samad emailed me a couple of days ago asking me to re-send my email with my instructions re: the power supply.  Did that.

Also got another email over the break from Michael Sprouse; replied to it.

Then I spent several hours working on my presentation on the COSMICi server structure for the workshop.  Added several slides illustrating the communication between components, and several more discussing some of the new modules/classes that need to be added.  

Thinking about rearranging the presentation to move the detailed run-through of the existing modules to later in the presentation.

Monday, December 19, 2011

Mon., Dec. 19th

Today all I did was come in and stop the data-collection run.  Copied the raw transcript file (17.4 MB) to Dropbox (in folder "C:\Users\Mike\Documents\My Dropbox\Server Code\logs 2011-12-19") so we can analyze it.

Friday, December 16, 2011

Fri., Dec. 16th

I have to leave a little earlier than usual today (5 pm) to pick up Colin, but here are some things I'm going to try to work on in the meantime.

  1. [ ] Do a test with all 3 detectors.  I finished assembling the 3rd detector and placed it over on the side of the room, about 10-15 feet away from each of the other two detectors.  (Really we'd like a somewhat larger separation, but this is OK for an initial test.)  I need to re-hookup the FEDM (which was washed on Wednesday) to do that test.
  2. [/] I also need to re-test the resistance between the +2.5VCC node and the PMT_3 (SyncPulse) internal node on which we are trying to receive our timing signal.  On Wednesday it was 10 ohms but it may be better now after the cleaning.  (It's not.)
  3. [ ] I need to check the voltage distribution on Samad's board.  First, double-check to make sure there are no shorts between different voltages.  After that's checked and the voltages are all verified, I can try powering the whole system through it.
  4. [ ] It might be a good idea to also re-test the bad DAC (#5) since it might conceivably have been healed by the cleaning.
  5. [ ] I can continue working on my slides for the workshop on the Python server.
Let's start with #2 since that is the easiest to set up.  JP14 pin 1 (top, +2.5VCC) <--> J59 pin 1 (top, PMT_3) = 9.54 ohms resistance.  Nope, still bad!

Continuity-tested Samad's board.  One of the grounds was disconnected from the others - bridged that with a solder blob. The +12V was shorted to ground through a loose fleck of metal - fixed that.  After those changes all connections were good.

Next, plugged in the power supply and tested voltage levels on the output pin headers.  The connector is still not very reliable - had to push down on it to make a good connection on all pins (like I've been having to do on the DE3 board itself).  After that, all voltages were correct.  What to do about bad reliability of connector?  Buy a new power cable?  Not sure.

Used 8" Adafruit F-F jumper wires (4 black = GND, 4 orange = +3.3V, 1 red = +5V, 1 green = PWR_N) to connect power from Samad's board to the DE3.  The DE3's power switch will thus turn on the power supply for the entire system.

Now let's try powering up the CTU through Samad's board.  No dice.  Maybe I need the +12V supply?  No, looks like it's just that the power connection to Samad's board (the mating between the connector and the jack) is EXTREMELY delicate.  It's very difficult to get it to make a solid connection without holding it in place.  (It's kind of just like the problem I have with my stupid phone when I try to plug it into the car charger.)  Not sure how we can fix this, except by buying a new power supply with a whole different kind of connector.  Or else, run jumper wires between the connector and the jack?  (It's kludgey, but it might work.)

Tried powering both CTU and FEDM through Samad's board simultaneously.  The Wi-Fi modules failed to establish a connection.  Thought at first they might be interfering with each other; tried turning off the FEDM's WiFi module but the CTU's still failed to connect.  Checked the +5V node and found its voltage was sagging significantly; it was about 4.4V instead.  I infer that the particular +5V wire coming out of the power supply that we're using is probably not itself rated to supply the full current we're drawing.  It looks like we are going to have to identify and procure a new power supply that is specifically rated to provide the full current we need at each of the voltage levels we use.

I next tried a test with the new detector (case #3, not yet labeled) connected to the SMA#3 input (the one labeled PMT#4), but the board failed to detect any pulses on it.  We checked its output on the scope, and sure enough it is producing pulses, several hundred mV big.  So there is something wrong here.

Aha, after failing to detect pulses from the Tektronix also, I realized that I had simply forgotten to reattach the +2.5V DC bias jumper at J64 (PMT_4 internal node, PMT4 footprint, input we're calling "SMA#3").  With it reattached, everything is working as expected.  Not all of the 2-way coincidences are also 3-way coincidences, but maybe 1 out of every 5 or 10 of them are.  Perhaps I'll leave this running over the weekend so we can collect some serious data on this.

Wednesday, December 14, 2011

Wed., Dec. 14th

The ECE department approved a $750 budget for students to buy needed parts & materials for the project.  I spoke to Donna to confirm that this was FSU money, and emailed the students that they could begin contacting her to process purchases and reimbursements.

Sonja emailed me that the Tektronix cables have arrived.  This will let us finish putting together the 3rd (and last, for now) detector apparatus.  I will get it this afternoon (and turn in my timesheet and sign my Spring paperwork at the same time).

Whenever Ray is ready, we can try removing that unwanted surface-mount capacitor from the FEDM board using the techniques we reviewed yesterday on YouTube.  This will then allow us to finish putting together the timing-sync input channel and then we can test it.  I think Ray is going to buy some more tools first.

Meanwhile, I will continue working on my slides for January's workshop on the Python server code.

LabView sent us a quote for 1-year renewal of the development suite license, for some reason.  I don't think we're going to renew since we haven't been using it anyway.

We picked up the Tektronix cable.  There was only 1 cable instead of 2 as I expected - was I wrong before about how many came in the package?  Still, that's enough to assemble our 3rd detector.  Ray said that's the only other part we need.  Went ahead and did receiving on the cable.

Trying to get through to Altera on the phone to see if they can tell us why the CLK_OUT output is only producing a 1.5V pulse (instead of 3.3V as we expected).  They were in a meeting, said try again after 5 pm.

I measured the resistance between the PMT_3 node and the other side of the J60 jumper.  It is 1kohm, which is the resistance on the pullup resistor.  This kind of suggests that the PMT_3 node is somewhere shorted to the +2.5V supply directly or through a small resistance.  (Maybe that's why this supply voltage is only +2.4V as measured, because the voltage is sagging due to a short?)  Also, across JP14 which directly measures the +2.5V supply, there is only 1 kohm resistance there as well.  There is also about 2 kohm resistance (a bit more) across J59, which is consistent with a short between PMT_3 and +2.5V, plus a 1kohm between +2.5V and ground, plus the 1kohm pulldown R102.

Aha, I directly measured the resistance between PMT_3 and pin 1 of JP14 which is the +2.5V supply, and it is only 10 ohms!  (9.63 ohms, to be precise.)  This accounts for the problematic DC offset.

I'm wondering if cleaning off the board with an electronics cleaning spray might help.  Ray says try it.  I sprayed the board down pretty good in the sink, and am letting it dry overnight.

I ordered some "shorting jumpers" (what I call "jumper bridges") from Digi-Key.  Pack of 100, to be delivered to my home address.  This will cut down on the number of jumper wires we need.

Going back to the presentation now.  For reference, here is the module hierarchy I sketched yesterday in OpenOffice Impress (based on my handwritten notes from a couple of years ago).

First draft of COSMICi server module hierarchy diagram.
Now I'll start describing the new components.  I'll go bottom-up.  I'll start with "flag" (at the lower-right corner of the hierarchy).

Finished a couple of slides on flag.py and the Flag class.  Next up: timestamp.

Tuesday, December 13, 2011

Tue., Dec. 13th

Some to-dos for today:

  • [/] Re-solder SMA connector to OCXO board.
  • [/] Re-test DC signal level (and received signal strength) on the PMT_3 (a.k.a. SyncPulse) signal node, to see if I've successfully moved the DC level closer to 0V (GND) by assigning all 12 of the input pins fed by that node to be LVDS inputs (which should have no pullup resistors on those ports).  This may not affect the signal strength though (that will probably require removing the extra capacitor, C148).
I should probably test the received signal strength on the actual PMT3 input to see if it differs from the TimingSig input (it shouldn't, but I should probably check to make sure).

Sometime soon, I also need to start working on my slides about the Python server architecture.

After re-testing the DC signal level of the PMT_3 node, there is no improvement since the board was re-burnt with the new pin assignments.

Emailed Sachin to see if he has any clue where the +2.5V DC bias might be coming from.

Talked with Ray about removing the capacitor C148.  My suggestion was to just use pliers to yank it off, or else try to push it aside with the tip of a hot soldering iron.  There are no other components that we need nearby it or on the other side of the board.  However, Ray wants to be extra careful since this is the only one of these boards that we have.  He is ordering a kit called "Chip Quik" ($135) that is supposed to be for safely removing surface-mount components.  Some of the tools in that kit might be helpful, although we already have the main things I think we would need... A soldering iron and a pick.  Anyway, no big hurry I guess.

In the meantime, maybe I'll work on my slides about the Python server to present to the students at the workshop in early January (which we still need to schedule).

Started working on the presentation slides.  Got as far as finishing the rough module hierarchy based on my notes.  Saved it in the Server Code folder on Dropbox.  Still need to go through the modules one by one explaining their main features.

Monday, December 12, 2011

Mon., Dec. 12th

Plan for today: Finish testing the new timing-sync datapath.

Juan says he can come by Friday morning to help with testing.

[.] Ask ECE if they can contribute to the project. --> Emailed Simon; follow up later.

Worked with Darryl and David for a bit to try to diagnose the timing signal problems.  It looks like the received signal is roughly 10x weaker than expected (instead of only the 4x weaker that we would expect from the replicated resistor & capacitor).  We checked some resistor values on the board (50 ohm and 1kohm) to make sure that these were not wrong, and they were correct.  We also checked the voltages on the outside of the GND and +2.5V pullup resistors, and they were correct.  We tried moving the input to PMT1, and there the signal was still about 2x weaker than expected.  We then tried hooking the scope directly to the DE3 output (with 50 ohm cable and 50 ohm input impedance) and this factor of 2x was still there.  For some reason the DE3 board is only outputting a roughly +1.5V amplitude pulse instead of +3.3V as we were expecting.  Perhaps it is expecting a 100 ohm cable?   Anyway, this output is a problem even if we remove the extra resistor/capacitor because our lowest threshold currently is only +1.5V.  However, we could move it a little lower.

Also, there is another problem which we didn't figure out yet, which is that the baseline logic level for the timing signal input seemed to be stuck at +2.5V, even when tied via the 1Kohm pulldown resistor to GND.  I have a theory that maybe there are pullup resistors by default on the 12 FPGA input pins that this node fans out to.  I'm going to try adding the pin assignments for the other pins to make sure this isn't the case.

The pins in question are (from the OrCAD schematic):

  • TimingSignal (if it were separate from PMT_3):
           Legacy port      Pin pair             Current input port name
    • TDCIN[12]:  K20/K19    --> TDCIN3[0]
    • TDCIN[13]:  J21/J20       --> TDCIN3[1]
    • TDCIN[14]:  H22/H21     --> TDCIN3[2]
    • TDCIN[15]:  H20/H19    --> TDCIN3[3]
    • TDCIN[16]:  G22/G21     --> TDCIN3[4]
    • TDCIN[17]:  F22/F21      --> TDCIN3[5]  (only one assigned at present)
  • PMT_3:
    • TDCIN[22]:  W1/W2       --> TDCIN4[0]
    • TDCIN[32]:  K1/K2        --> TDCIN4[1]
    • TDCIN[33]:  J2/J3           --> TDCIN4[2]
    • TDCIN[34]:  H3/H4         --> TDCIN4[3]
    • TDCIN[35]:  H1/H2         --> TDCIN4[4]
    • TDCIN[36]:  G1/G2         --> TDCIN4[5]
OK, added all those assignments to the New_with_Nios_trim revision of the Q:\COSMICi_FEDM project. Also added the corresponding input ports TDCIN3[5..0] and TDCIN4[5..0] to the top-level schematic COSMICi_FEDM_top13.bdf, although of course the only one of these that is actually used at present is TDCIN3[5] --> pf3[5].  This one tells us when the TimingSignal input crosses the lowest threshold (DAC#6).  TDCIN4[] is just a copy of TDCIN3[], due to the bug in the schematic, and is not useful.

The compile is running now.  It will take a while.  OK, that's finally done.

Burned new design into FEDM's EEPROM.  Hooked up timing sync cable and scope to redo the test of the received timing-sync signal.  However, I just noticed that the SMA connector that I had hand-soldered to the OCXO board came disconnected at some point in the last few minutes of fiddling.  Really, that board needs to be completely redesigned!  So I need to re-solder that connection before I continue.  No time today, so I'll do that tomorrow.

Wed., Dec. 7th

Ray and I went over to the Tech Transfer office to talk about the Cosmic Cube disclosure.  They are going to file for a provisional.  They also are going to pay for us to take the course offered by the Entrepreneurial Excellence Program run by the Tallahassee/Leon County Economic Development Council.

The GPS app is still running from last night, with the 25 ns setting of <traim_alarm>.  At some point today I'll stop it and look at the data.  I think it might have excluded all satellites throughout the run, but we'll see.

Today I want to Dremel down the fourth SMA connector, mount it on the TIMINGSIG pad, and hook up the boards and look at the output from the output-test stub for the new timing-sync datapath.

Dremeling done; boards are now connected together with one of the 6" SMA cables.

I think we should monitor the received signal because there is a bit of a capacitive voltage divider effect going on between TIMINGSIG and PMT3 since they are both AC-coupled but their back ends are wired together (due to the board design bug).  Also due to the bug, there is an effective 25 ohm termination resistance for AC signals, instead of 50 ohm, since both TIMINGSIG and PMT3 have terminating resistors.  Both of these could cause problems.  One fix would be to remove the terminating resistor or the coupling capacitor on PMT3.  Ray would prefer not to irreversibly change the board though, so instead, I'll look at the signal.

Aha, the DE3 isn't driving CLK_OUT presently (that line was commented out of the top-level Verilog file).  Fixing that.

Still not getting a signal.  Trying to send the clock also out to a GPIO1 header pin, GPIO1_CLKOUTp1.  Still nothing; I think I had problems trying to use the CLKOUT pins before, so let's try a normal pin, GPIO1_D[11] (which is pin #16 on header GPIO 1).  Still nothing.

Tried sending EXT_CLK to GPIO1_D[11] to make sure the pin works.  It does.

Now I'm examining the PULSE control signal.  Upon inspection of the code, it should become 1 after the first PPS edge is received.  It does.  So why no pulses?

It had occurred to me earlier in the day that there might be a timing error (although unlikely with the slow 10 MHz speed of the OCXO clock), but TimeQuest shows a very unconstraining fmax (371 MHz) for EXT_CLK, so it doesn't look like that's the problem.

Getting desperate, trying adding an intermediate wire named "pulse_out_100ns" inside the GPSapp.v Verilog module to see if that helps any.

D'oh, it looks like the trigger point was just off the scope display horizontally!  Boy, do I hate when that happens.  Don't know why I didn't catch that earlier.  Slapping myself.

Tuesday, December 6, 2011

Tue., Dec. 6th

Aarmondas and David finished up the timing sync datapath, we hooked it up to the appropriate input and to stream_pulse_out_test for testing, and are presently recompiling to see if the design still fits.

Meanwhile, I'm recompiling the GPS app from 50 to 25 ns (value of the <traim_alarm> parameter) so I can do another overnight run.  (Last night's run stopped at some point but still got a good 37 MB of data at 50 ns that I can analyze.)  Recompiled Quartus, rebuilt EEPROM programming file (Rev_A.jic), now burning it into EEPROM.

The fitter succeeded.  Some notes:

  • Combinational ALUTs:     10,089 / 27,104 (37%)
  • Dedicated logic registers:  25,747 / 27,104 (95%)
  • M512s:                            188 / 202 (93%)
  • M4Ks:                             144 / 144 (100%)
  • The new module tsedge_datapath_v1_56 takes:
    • Combinational ALUTs:   25
    • Dedicated logic regs:       19
    • Memory blocks:              none
It looks like the whole thing isn't getting synthesized, perhaps because we are only showing the low 8 bits of each word on output?  Need to look at size again after it is hooked up to new SOPC system.  Anyway, it is looking like it will probably be small.

I could hook it up and do a test, but I really should Dremel down a new SMA connector first so that I don't have to mess with any the existing ones.  Got the connector out, I'll dremel it tomorrow.



Monday, December 5, 2011

Mon., Dec. 5th

Spoke with Dr. O'Neal about the possibility of saving my Spring salary money for summer.  This would also have the advantage of freeing up my computer in the lab for someone else to use.  I need to check with ECE first to see if they still want me to teach Digital Logic in the Spring.

Need to check whether the new 50 ohm cables arrived from Tektronix.  The PO was dispatched.  I called Tektronix and it looks like they haven't received it yet.  [ ] Need to call Purchasing tomorrow and make sure it was sent out, find out on what date, and maybe ask them if we can get a PDF of the PO.

The CTU is still running from this run that I started last week.  That's good that it didn't crash!  I should take a look at the data.  This was a run with <traim_alarm>=200e-9.  (Next I should try 50e-9, and maybe 25e-9.)

Copied the current log files (200 MB) to log folder, "C:\SHARED\Server Code\logs 2011-12-05" & started a new run.

Wednesday, November 30, 2011

Wed., Nov. 30th

I'm hoping that Michael Dean & Juan Calderin will come in to the lab today and tell me that they are ready to try integrating their new timing edge-capture datapath into the system.  Emailed them to see.

Another thing I can work on today is preparing some slides with an overview of the current software design of the Python server, so the students can build their own visualization modules on top of it.

My overnight run of the GPS app is still running, currently at 75,900 seconds into the run.  I need to find my old Scilab analysis scripts (not to mention the Java file format-conversion app) and see what this data looks like.

Maybe instead of going through all that, I can just take some shortcuts to get a quick idea of what the time data is looking like.  In Cygwin, in the "/cygdrive/c/SHARED/Server Code" folder, I typed this command to extract just the PPSCNTR lines:
$ grep PPSCNTR node0.uart.trnscr > PPSCNTR.csv
Now, let's drag that file into an OpenOffice Calc window.  Ah, it truncated the import at 64K rows, but that's OK.  Oops, the last field, which is the main one of interest, still contains the NMEA checksums:

Tue Nov 29 17:55:33 2011 + 209 ms: < $PPSCNTR,0,0*58
Tue Nov 29 17:55:33 2011 + 212 ms: < $PPSCNTR,1,9999939*5A
Tue Nov 29 17:55:33 2011 + 214 ms: < $PPSCNTR,2,19999953*64
Tue Nov 29 17:55:33 2011 + 217 ms: < $PPSCNTR,3,29999966*60
...
So, let's use a Unix cut(1) command to get rid of those:
$ cut -d\* -f1 PPSCNTR.csv > PPSCNTR-trim.csv
That's better.  Now, we can add a spreadsheet column to compute the deltas.  Boy, that takes a while... OpenOffice is slow!  Had to kill the process.  Let's trim off a few lines from the end to avoid problems.  Now the valid data indices (PPS seconds) runs from 2 through 65529.  OK, based on the data, the average frequency over that interval is 10,000,012.8768.  Now we can plot the cumulative time wander relative to the average frequency as a baseline, as we've done previously:


Note we still have the usual large but brief phase glitches (off the chart), but other than these there is mostly just a pretty gradual drift, and no large divergences from this trajectory other than one noticeable discontinuity.  I don't know if we actually lost time lock at any point.  We can figure that out by looking at the PDMETRAIM figures.

Here is the self-reported accuracy from the TRAIM algorithm:


It looks like we actually lost time lock between about seconds 7,000 to 23,000.  At 7,000 is when the wander started drifting sharply upwards, then at 23,000 is a slight downwards dip to back in line with the background wander.  In the interim, was the POSHOLD mode controlling the time readings?  Bizarrely, looking in the middle of this range, it looks like there were plenty of satellites, but they were all being excluded!  Is the TRAIM algorithm broken?  Or, perhaps it's just that the signals are all getting reflected off of the nearby building and so the times are all way off?  Hard to say.
Number of satellites being received over the course of the run.
Perhaps we should set the limit of TRAIM to a larger value so that fewer satellites will be excluded?  Right now the limit is 100 ns.  Should we make it 200 ns?  Let's try that and then start another overnight run.

Looking at this graph, we can see that an unusually large number of satellites (perhaps all of them?) were eliminated by the TRAIM algorithm during the period where the self-assessed accuracy was 0.

TRAIM algorithm data.  Blue line: Self-assessed inaccuracy in seconds (left axis).
Red dots: Number of satellites eliminated by traim algorithm (right axis).
Note also that the more satellites are eliminated, generally the worse the accuracy is.  However, it's unclear in what direction the causality relation goes here.  When a lot of satellites are giving wild time readings, more of them are eliminated and so the accuracy is worse?  Or, is it that when more satellites are dropped from the calculations, there is less data to work from, and so the result of the calculation is more inaccurate?  Unclear.

Anyway, started a new run with the <traim_alarm> parameter set to 200 ns instead of 100 to see how this affects things.  At first glance, it seems like the accuracy is worse (perhaps b/c more satellites with wild readings, reflecting off Ware-Rhaney maybe, are being averaged into the calculations).  But anyway I'll leave it running overnight (or over the weekend, even) and analyze the data next time I'm in.

Tuesday, November 29, 2011

Tue., Nov. 29th

We went up on the roof above the lab window to see if we could get more accurate GPS coordinates there, using the MotionX-GPS app.

My iPhone says:

  • Lat.:   30* 25' 42" (N)
  • Lon.:  84* 17' 06" (W)
  • Alt.:    97'
  • Acc.:  56'
On Google Maps, this looks to be on the centerline of our wing of the building, about 30' away from the edge where our windowsill is.

Ray's iPhone says:

  • Lat.:    30.428250
  • Lon.:   84.284997
  • Alt.:    131'
  • Acc.:   56'
On Google Maps, this looks to be almost exactly in the right place.

Unfortunately, the iPhone is unable to acquire a fix at all in the window, so we are unable to compare it with the DeLorme to try to figure out whether the position discrepancy is due to the location or to something about the DeLorme.  But at least we've ruled out the problem being an inaccuracy in Google Maps / Google Earth (since Ray's phone, at least, agrees exactly with Google, within a couple of feet, as to where our window is, and my phone agrees almost exactly, within about 30' or so).

Aha, just got a fix in the windowsill from my iPhone, it says:

  • Lat.   30* 25' 40" N
  • Lon.  84* 17' 07" W
  • Alt.    238 ft
  • Acc.  156 ft
Cool, this is also over near Ware-Rhaney, so this confirms (more or less) that it's probably the windowsill location (and not the DeLorme unit itself) that is causing the positional inaccuracy problem.

[/] Need to make sure the POSHOLD location is as accurate as possible, WRT our real location.  OK, updated that code now with the slightly more accurate coordinates 30*25.694,N,084*17.1,W (on top of our window, instead of at corner of the wing).  Rebuilt firmware, recompiling gelware in Quartus...  Also need to rebuild the EEPROM config file... (Rev_A.jic)  Done & burned it.

Pascal dropped in and showed me the blueprints he got from the architecture firm of the I-beams, sprinklers, lighting, and vents (4 separate sheets).  He is going to work this information into the next presentation.  He also said that Reid Lambdin OK'd the idea to put the detectors on shelves instead of hanging them from I-beams.

Aarmondas reminded me that we had agreed to reschedule the presentation date/time to Friday of next week.  It will have to be 4 pm since Drifters is at 3 pm.  I emailed an updated announcement to the BB group.

I asked Aarmondas also to schedule a date/time either next week, during finals week, or in the 1st two weeks of classes in January for me to present to the ECE students the current design of the Python server software, so that the students will be familiar with the capabilities of what's already there and can build upon it for the visualization etc.

I'm starting a new overnight run to see if the time accuracy seems any different than it did in my experiments last Winter/Spring.  I think our POSHOLD position reference is more accurate now (due to both improvements in aerial photography on Google Maps / Google Earth - closer to overhead view - as well as to Ray's phone, which I think is an iPhone 4, apparently having a more accurate GPS than my 3GS does).  Tomorrow I'll do a plot showing the cumulative phase wander of the GPS's PPS signal vs. the OCXO clock, and look at the frequency and magnitude of any significant excursions that happen when time lock is temporarily lost.

Monday, November 28, 2011

Mon., Nov. 28th

David was out sick last week, but today he texted that he is coming in this afternoon after his class.

My plan today is to work some more on the GPS startup.  I am wondering if the problem might be that the position data in the POSHOLD command is not sufficiently accurate?  A better procedure (suggested in the manual) might be to average position data over a period of time before initiating POSHOLD mode.  This would also be helpful when installing the system at a new site location (or in general when moving the GPS antenna).

Also, I can still try the strategy of querying the current NTP time from the server to initialize the time to ~10 ms resolution.  However, even just 10 ms of time uncertainty equates to 3,000 km of position uncertainty, so I'm not sure how useful it will be to set the time more accurately.

Anyway, let's try to get this thing up and running again.  I'm going to carefully log all my steps to facilitate a postmortem next time the GPS fails to connect.

  • (2:31 pm) My immediate plan is to first connect through UwTerminal, then select the default configuration, access the kit using the demo software, and then wait 15 min. or so to re-acquire satellites.
  • (2:32 pm) The current configuration is:  Batteries not installed.  USB not plugged in.  AC adapter not plugged in. NMEA port connected to COM1 via gender bender.  NMEA port switch in rightwards ("DCE") position.  Power switch in leftwards ("on") position.
  • (2:33 pm) Starting UwTerminal.  Setting COM=1, baudrate=57,600, handshaking=none.  Click OK.  No data of course because the kit isn't powered on yet.
  • (2:35 pm) Now I'll plug in the AC adapter for power.  The unit confirms:
$PDMEHEADER1: DeLORME GPS2058_HW_1.0.1
    $PDMEHEADER2: DeLORME GPS2058_FW_2.0.1
      $GPTXT,COSMICi Custom_Config_0.0.3
        $GPRMC,214717.165,V,3025.694,N,08417.100,W,0.0,0.0,221111,4.0,W*7B
          $GPGGA,214717.165,3025.69416,N,08417.10000,W,0,00,99.0,069.66,M,-29.7,M,,*66
            $PDMETRAIM,2,0,0.000000000,0,0,0,0,0,0,0,0,0,0,0,0,0*43
              $PDMEPOSHOLD,0,0000.000,N,00000.000,E,000.00*4A

              • (2:38 pm) Unplugged unit a few minutes ago to stop text from scrolling so I could note the above.  Now plugging it in again to see if it acquires anything.  I don't think it will.
              • (2:40 pm) Going to send an NMEA-off command to make it easier to send other commands.  Download-->Stream File Out-->NMEA-off.txt.  No response.  I think the last time this happened, I decided it was a grounding problem.  Going to switch temporarily to the batteries for power supply.
              • (2:41 pm) Inserted batteries (two AAs).
              • (2:42 pm) Unplugging AC adapter.  Power light goes off (this is normal for battery mode), but status light still flashing red.  Now trying to send the file again.  Still no response!  Wait, maybe in battery mode the level shifter is disabled?  Doesn't look like it from a quick glance at schematic... Anyway, let's try the USB power supply now.
              • (2:44 pm) Plugging in USB cord, unplugging battery.  Now the NMEA-off command works.
              • (2:45 pm) Wondering if, before I revert to the demo app, I should first try the POSHOLD and TRAIM commands, with the correct time value in the POSHOLD.  Or actually, first, try the $PDME,9 command to just initialize the position and time.  "Must be issued immediately after a reset or startup to be most effective."  OK, so first, power off the unit.  Unplugging USB.
              • (2:47 pm) Starting SciTE, opening "C:\Users\Mike\Documents\My Dropbox\FAMU\COSMICi\GPS FPGA app\NMEA command files\init-pos.txt", preparing to write new contents.  Old contents:
                • $PDME,9,30.428236,-84.285,40,2011,11,22,21,47,30
              • (2:49 pm) All I have to do, I think, is update the time value to something semi-accurate.  Let's go to time.gov and make sure we know the correct UTC(GMT) time.  It is 19:50.  So, allowing a minute or so to start, let's write:
                • $PDME,9,30.428236,-84.285,40,2011,11,28,19,52,00
              • (2:52 pm) OK, sent that line right after plugging in the unit's power.  Looks like I was about 15 seconds early (compared to what the real time was), but at least it's close enough to help it find the satellites, hopefully.  Next step:  Just wait 15 minutes or so to acquire satellites, hopefully.
              • (2:55 pm) An observation: The $PDMETRAIM line is reporting condition 2 (unknown), and the LED D9 is flashing green once every 2 seconds.  I don't know what this means!  (That signal TRACKLED connected to the green is supposed to mean "3D fix" but I don't know what it means for it to flash once every 2 seconds - undocumented behavior?)
              • (2:58 pm) Checking the scope; there is no PPS edge currently (consistent with no time lock).  Still no satellites showing.
              • (3:01 pm) OK, I'm still getting no sign of any satellites.  I think I'd better abort this test for now.  Sending NMEA-off.txt.
              • (3:02 pm) Now sending select-default.txt.
              • (3:03 pm) Now sending hot-start.txt.  Switching UwTerminal baud rate to 4800 to match.  I wonder if just doing this (switching back to default config) will help acquisition at all?  Maybe wait a little while to see.
              • (3:06 pm) Still no satellites.  Wait a few more minutes and then try the demo app.
              DAMMIT, I lost like two hours' worth of notes because Blogger fucked up.  Summary of status: Got it working thru the demo app, switched over to the custom config and finally got things working with power-up thru DE3 and with battery backup for the GPS.  However, the setup is pretty unreliable/touchy; the Wi-Fi tends to power-cycle and the DE3 app tends to crash.  But it's been running smoothly for a while.  I am afraid to breathe on it though.  Anyway, here's the pic of the current setup:

                I should test again to see if it works without the USB blaster (JTAG) cable.  Yep, it does.

                Here is a pic of the setup showing no cables but power and GPS antenna cable:

                Here is a screenshot of the server windows:


                And here is a closeup of the main output window:

                Guess I'll have to recreate all those setup notes some other time...

                Here is a screenshot showing both the output from the the CTU (DE3 board GPS app) and the FEDM simultaneously:


                What next?  Maybe, try putting the GPS in standby when on battery power?  I think I'll do that when I leave today.

                One more thing:  I averaged the coordinates from the GPGGA lines from the latest run, about an hour's worth (4:55 - 5:55 pm).  Here is the averaged data:

                • Latitude:      30 degrees, 25.65445 minutes, sample standard deviation 0.01596 minutes (29.56 m).
                • Longitude:   84 degrees (W), 17.11387 min., sample std. dev. 0.00881 minutes (~14 m).
                • Altitude:      62.81 m, sample std. dev. 21.50 m
                This puts us over in the Ware-Rhaney building, according to GPS Visualizer and Google Maps (http://www.gpsvisualizer.com/map?format=google&lat=30.42757416667&lon=-84.28523116667).
                Also on Google Earth.  Is the signal reflecting off the building, or what?  Don't know...

                OK, 6:22 pm - STOP, exit on FEDM; MUTE, STOP, exit on DE3; DE3 power off; switching GPS switch to OFF/STANDBY position.  Tomorrow we'll see how hard it is to re-acquire the signal...

                Tuesday, November 22, 2011

                Tue., Nov. 22nd

                Was thinking of doing some work today on improving time resolution of CTU.

                First, an issue list with the current CTU setup:

                • [ ] For some reason, the Wi-Fi board failed to connect when I powered up the system today.  But then, after some fiddling, it started working again.  Loose connection somewhere?  Watch out for this to happen again, try to diagnose.
                • [ ] When the CTU firmware receives a blank line on its input, at present it generates an "UNK_CMD" (unknown command) error message.  Instead it might make more sense to simply ignore these lines.
                • [ ] After being powered off for a while (like overnight), the GPS module seems to have difficulty acquiring satellites again.  This is perhaps because the time-of-day is wrong.  (At the moment, the GPS thinks it's 19:50 GMT, when actually it is 20:17; so the GPS is running almost half an hour slow.)  One way to address this:  Have the CTU query the server to find out the current time, then send a command ("$PDME,9,...") to the GPS to set the time (as well as setting the position)
                • [ ] When restarting, sometimes there are server errors on the 2nd (or later) connection attempts.  These need to be tracked down and fixed at some point.
                Samad is here and he brought the new 10-pin connector with him.  I showed him the proto board I picked up the other day at Fouraker's and the extra breakaway headers and jumpers I ordered from Adafruit, so that he can work on soldering up our power distribution board.  We discussed the connections needed.  He is going to write it all up and include it in the presentation #3 (SLDR), as well as the next report (DDR).

                I scanned the bare proto board for him, so if he runs out of time to finish the soldering today, he can at least still make a nice sketch of the wiring for the presentation:

                Bare proto board, to be used for soldering
                prototype of main power distribution board.
                Back on the GPS - I inserted two fresh AA batteries into the GPS kit as a battery backup, so that once we get a fix we don't lose it again.  Although, this conflicts somewhat with the use of the $PDME,9 command, because the docs for that say it needs to be done right after startup to be most effective.

                Based on aerial photos from Google Earth (latest imagery), the coordinates of the windowsill where the GPS antenna is currently located are as follows:

                • Latitude:     30°25'41.65"N
                • Longitude:  84°17'6.00"W
                This is a little west of where it looked like the window was in earlier images, but I think the current image is more direct (from overhead), so let's go with that.

                Using the coordinate converter at GPSVisualizer.com, in terms of just degrees and minutes (what we get back from the GPS) this is:

                • Latitude:     N30°25.694167
                • Longitude:  W084°17.1
                and in terms of just degrees (which we supply on the $PDME,9 line), it is:
                • Latitude:     30.42823611111
                • Longitude:  -84.285
                Altitude, guesstimating from Google Earth, is about 50 m (164 ft).  So, the actual $PDME,9 command line would be (right now, 2011 Nov. 22nd, 21:12:00 GMT):
                • $PDME,9,30.428236,-84.285,50,2011,11,22,21,12,00
                Now, the only problem is, getting this information to the GPS will be somewhat involved, requiring changes to the CTU firmware, the Wi-Fi script, and the server.  I think perhaps as a short-term solution, I will get the GPS set up manually, and just rely on the batteries for a while to keep the fix active.  Remember, though, to later go back and solve this issue more permanently!

                Muting GPS echo and stopping timer.  Exiting from Wi-Fi.  Powering off CTU.  Yes, GPS still has power (from batteries).  Unplugging level shifter.  Plugging in COM1 cable.  Setting UwTerminal to 57600, no flow control.  Sending commands from UwTerminal:  NMEA-off.txt, hot-start.txt, init-pos.txt (with the new location/time as above).  Still no satellites!  Maybe I need to re-run the eval app.  Have to revisit this on Monday.  Happy Thanksgiving all!

                Monday, November 21, 2011

                Mon., Nov. 21st

                This is a short week due to Thanksgiving.  Some things to tackle:
                1. [/] Get GPS module acquiring satellites again.
                2. [/] Disable CTU debugging output under control of a slider switch.
                3. [/] Burn working CTU design to DE3 board.
                4. [ ] Maybe experiment with faster clock for GPS timing edge capture in CTU app.
                5. [ ] Start writing up some notes on Python server code for use by Cp.E. students.
                Working now on satellite acquisition.  Using UwTerminal, I streamed command files to select the default configuration.  Then I hooked up the kit to the PC via USB and attempted to initialize using the approximate site coordinates in minutes & seconds (30*25' N, 84*17' W) and the current system time (14:50 EST).  I briefly saw a satellite flicker from gray into red once or twice, but not enough to get any coordinates or even download an updated almanac?  Although I'm not sure about the latter b/c the satellites are moving slowly.

                Aha, finally got a fix after letting it sit for a while.  Now, let's see if I can switch into my custom config and start the GPS app...  Here are the steps for that:

                1. Turn off debug stream with "$PDME,13,5,0" command.
                2. Exit GPS kit eval app.
                3. Plug in PC's COM1 cable with null modem back into NMEA port.  (With left switch in left position.)
                4. UwTerminal at 4800, no flow control.
                5. Plug in 6V adapter, unplug USB (it interferes with serial).
                6. Stream C:\Users\Mike\Documents\My Dropbox\FAMU\COSMICi\GPS FPGA app\NMEA command files\NMEA-off.txt
                7. Next, show-backup.txt.  Yes, custom config is still in flash.
                8. Next, select-backup.txt.
                9. Next, hot-start.txt, & switch baud rate to 115,200.
                10. Unplug COM1 cable.
                11. Plug in pin 24 of J5 from DE3, power on DE3, unplug AC adapter.  That works!
                12. At this point you can burn the Quartus .sof file to start the design running.
                At this point, I added the capability to switch off the debug output.  I added a new PIO from the slider switches and added macros PRINTF() and PRINTF_R() to check that slider switch 1 is up before sending any output.  This worked.

                Next we (Darryl is here) tried burning the design to the board (to the EPCS device) so it would run automatically on power-up.  This worked, but 20 seconds was (in this test) not quite long enough for the Wi-Fi to finish starting up.  Changed it to 30 seconds (for now) and added a countdown display on the 7-segment digits so that the user doesn't get too impatient.

                We also cleaned up the formatting a bit on the NMEA-formatted output.  There is a minor existing weirdness where it doesn't finish receiving & processing the WIFI_READY message (from just before the script main loop is entered) until a subsequent command line is typed.  Maybe I didn't terminate that line properly?  Anyway, go back and figure that out someday.  Everything else works fine.

                  Friday, November 18, 2011

                  Fri., Nov. 18th

                  Plan for today:

                  1. Finish testing CTU with new connections.
                  2. Maybe add a PLL to the CTU for improved resolution of GPS time measurements.
                  Tweaking pin assignments some more:


                  • GPIO 0 (Wi-Fi) - Inner header
                    • Pin 11 = +5V power supply to Wi-Fi module.
                    • Pin 9 (GPIO0_D6) <-- Serial CTS input from Wi-Fi kit
                    • Pin 10 (GPIO0_D7) --> Serial RTS output to Wi-Fi kit
                    • Pin 12 = GND supply to Wi-Fi module.
                    • Pin 13 (GPIO0_D8) <-- Serial RxD input from Wi-Fi kit
                    • Pin 14 (GPIO0_D9) --> Serial TxD output to Wi-Fi kit
                  • GPIO 1 (GPS) - Outer header
                    • Pin 27 (GPIO1_D20) <-- PPS input from GPS kit, J4 pin 15.
                    • Pin 29 = +3.3V power output to GPS kit.
                    • Pin 30 = GND supply to GPS kit.
                    • Pin 31 (GPIO1_D22) <-- Serial RxD input from GPS kit, J5 pin 15 (UART0TX).
                    • Pin 32 (GPIO1_D23) --> Serial TxD output to GPS kit, J5 pin 16 (UART0RX).


                  ...Then I spent the whole afternoon doing a lot of tinkering and fiddling which I'm not going to go into all the details of.  Suffice it to say, I discovered the following:

                  For some reason, pin 31 of GPIO 1 on the DE3 board seems to be "broken," in the sense that it pulls to 0 even when it's only being used as an input.  Thus, I'm now using pins 39+40 for Rx/Tx to GPS instead of pins 31+32.

                  It doesn't seem to work to communicate with the GPS kit directly though pins 15+16 of the kit's J5, perhaps because its MAX3222 can't be disabled; so instead, I have plugged the SparkFun level-shifter onto J6 (powered with +3.3V from DE3's GPIO1) and meanwhile I am also supplying +5V power to the GPS module from GPIO1 through pins 4+24 of J5.

                  Oh, and BTW right now the OCXO board (which I soldered the temporary SMA connector onto) is being supplied +3.3V from GPIO0, while meanwhile the Wi-Fi board is powered by +5V from GPIO0.  So we're using all 4 of the power outputs on the GPIO boards!

                  Here is a photo of the fully assembled and running CTU electronics:
                  Assembled CTU electronics.  The GPS module, level-shifter and
                  OCXO board are all drawing their power through the DE3 board.

                  Some next steps:
                  1. Add a compile-time (or better, DIP-switchable) option to disable debugging output.
                  2. Burn design to EEPROM.
                  3. Test power-up sequence.
                  Just for fun, I did a test with both the CTU and the FEDM powered up and talking to the server at the same time.  That worked fine.  This works because currently CTU is node #0 and FEDM is node #1, based on the nodeid.txt files loaded into their respective Wi-Fi boards (#3 and #1).

                  One more thing: The GPS seems to be still having trouble acquiring satellites; I may need to go into the DeLorme app and tinker around with it until it is healthy again.

                    Thursday, November 17, 2011

                    Thu., Nov. 17th

                    Had a little time between meetings, so decided to come into the lab to test the connections on the newly-soldered 40-pin header on Wi-Fi board #3.  Actually, I realized I can't test the connections to J9 because that is on the other side of the 0-ohm resistor jumpers R7-R14, which are not presently populated (their pads are underneath the level-shifter chip U3).  So instead, I'll test by running the GPS app on the DE3 board.

                    Pins of J2 that we need for the serial are:
                    • Pin 10 (yellow wire)   - (From DE3) Autorun (DE9 pin 4) -> M_DSR (To WiFi)        - Optional.
                    • Pin 11 (black wire)    - GND                          (DE9 pin 5)
                    • Pin 19 (green wire)    - (From DE3) RTS       (DE9 pin 7) -> M_CTS  (To WiFi)        - Handshaking in (optional)
                    • Pin 21 (red wire)       - (To DE3)     RX         (DE9 pin 2) <- M_TX    (From WiFi)
                    • Pin 23 (tan wire)        - (To DE3)     CTS      (DE9 pin 8) <- M_RTS  (From WiFi)    - Handshaking out (optional)
                    • Pin 25 (blue wire)      - (From DE3) TX        (DE9 pin 3) -> M_RX    (To WiFi)
                    • Pin 27 (purple wire)   - (To J10 pin 2 /EN)                      <- +3V out  (From WiFi)    - Disables level-shifted input.-
                    To start out, I'm just going to connect pins 11 (GNS), 21 (M_TX), 25 (M_RX), and 27 (/EN).

                    Starting the server to prepare for the run.  (First I moved the last round of log files to a subfolder with today's date.)

                    I have to open the GPS app project in Quartus.  Did that (version in C:\f).  Plugged in USB.  Powered up board.  It's running the builtin demo.  Starting programmer.  Selecting USB blaster.  Burning image.  Wait, forgot to start Wi-Fi first.  Powering DE3 off.

                    On Wi-Fi, connecting pins 1+2 of JP2 to enable USB power.  Now it's powering up.

                    Looking at the top-level file (DE3_GPSapp.v), it looks like I am going to have to add CTS/RTS after all, because the design is expecting them.  OK, added those two wires.

                    Now, powering up DE3 again, and re-programming design.  Nuthin!  (Just "--" on 7-seg display)

                    I have to go over to ECE now.  Things to try tomorrow:
                    • Hook up GPS module
                    • Check baud rate
                    • Run design in debugger (in legacy Nios II IDE)
                    *     *     *
                      Came back in later in the evening to work some more on the DE3 board connections.  I've now greatly simplified the design of the connections.  I've eliminated the ribbon cable and serial cable and all of the wire-wrap.  I did some reassignments of the DE3's GPIO header pins to group related pins closer together.  I also figured out that I can eliminate the level shifter (I think) by connecting directly to the GPS kit's internal logic-level Tx/Rx nodes instead of going through its level-shifter and DE9 connector.  And I figured out how to power the GPS kit directly from the DE3 board using +3.3V out.

                      New pin assignments:
                      • GPIO 0 (Wi-Fi) - Inner header
                        • Pin 11 = +5V power supply to Wi-Fi module.
                        • Pin 27 (GPIO0_D20) <-- Serial CTS input from Wi-Fi kit
                        • Pin 28 (GPIO0_D21) --> Serial RTS output to Wi-Fi kit
                        • Pin 30 = GND supply to Wi-Fi module.
                        • Pin 31 (GPIO0_D22) <-- Serial RxD input from Wi-Fi kit
                        • Pin 32 (GPIO0_D23) --> Serial TxD output to Wi-Fi kit
                      • GPIO 1 (GPS) - Outer header
                        • Pin 27 (GPIO1_D20) <-- PPS input from GPS kit, J4 pin 15.
                        • Pin 29 = +3.3V power output to GPS kit.
                        • Pin 30 = GND supply to GPS kit.
                        • Pin 31 (GPIO1_D22) <-- Serial RxD input from GPS kit, J5 pin 15 (UART0TX).
                        • Pin 32 (GPIO1_D23) --> Serial TxD output to GPS kit, J5 pin 16 (UART0RX).
                      The Adafruit jumper wires currently being used are not colored optimally, since I was running low on colors, but I ordered some more of them and they should arrive soon.  I can replace the temporary ones I have now with more sensibly-colored ones when they get in.

                      I think the new GPS communication setup is working because I got a "Yo" on the 7-segment display, indicating a line was received.  Still need to test communication in the other direction though.  (Sending config commands from the DE3 to the GPS.)

                      Tomorrow I'll hook up the new Wi-Fi board (it's drying at the moment; I had to solder a header pin for +5V power input) and do a complete test of this new CTU setup.

                      Wednesday, November 16, 2011

                      Wed., Nov. 16th

                      Michael Dean came by my office yesterday and I answered some of his questions on the timing system.

                      Accidentally turned in the wrong timesheet yesterday (for the wrong pay period), so had to spend some time today running around getting the correct one (don't know why I didn't already have it), filling it out, getting it signed & turning it in.  The system for getting paid is so inefficient!

                      Today I want to try compiling with different versions of the Nios core and see how much space it saves.

                      As a baseline, the present design (with the Nios II/f) is as follows:
                      • Entire design (COSMICi_FEDM_top12):
                        • Comb. ALUTs:  10,017 / 27,104 (37%)
                        • Ded. log. regs.:   25,678 / 27,104 (95%)
                        • M512s:                   188 / 202       (93%)
                        • M4Ks:                    144 / 144       (100%)
                        • M-RAMs:                   1 / 1           (100%)
                      • SOPC system (FEDM_NiosSys):
                        • Comb. ALUTs:   2,324  (8.6%)
                        • Ded. log. regs.:    1,901  (7%)
                        • M512s:                  150  (74%)
                        • M4Ks:                   144  (100%)
                        • M-RAMs:                  1  (100%)
                      • Nios CPU (the_cpu_0):
                        • Comb. ALUTs:   1,396  (5%)
                        • Ded. log. regs.:    1,486  (5%)
                        • M512s:                      4  (2%)
                        • M4Ks:                     16  (11%)
                        • M-RAMs:                  0  (0%)
                      So at best, by changing the CPU to a much smaller one, we could bump up the available logic registers a bit (less than double), increase the available M512s from 14 to (at most!) 18, and increase the available M4Ks from 0 to (at most!) 16 (11% of the total number).  So it's not a huge difference, but it does potentially make some difference.  Need to keep this in mind if we run out of space again (or if we need to loosen things up for improved performance).  Later on, we can do some test compiles to see exactly how much space we can safe by moving to the Nios II/s or e.  (Note: It could however increase firmware size, due to the need for software multiply/divide instructions.  It could also reduce performance and lead to more dropped pulses.)

                      I had a thought earlier in the day, which is that we could add a PLL in the GPS app, to more precisely measure the PPS edge times relative to the OCXO clock.  The OCXO clock is 10 MHz, but it would be pretty easy now (given the code already written for the FEDM) to step this up to a much faster speed.  50 MHz?  100 MHz?  500 MHz?  It would be fun to try.

                      Samad came by and we tried to solder the other 40-pin DIP header onto the other Wi-Fi board (#3), but had some problems, due to my hand shakiness; solder got stuck in some thru-holes, blocking them.  I tried several methods for removal, but no luck yet.  We may need to try again with another board.  Or, if I can find a small enough drill bit, we could re-drill the holes.  (I looked through all my Dremel tool attachments, but didn't find anything suitable.)

                      Finally managed to work the solder out of the holes by hand using a tiny eyeglass screwdriver, then soldered the second breakaway header strip of 20 pins.  After all the work with the solder tools there is a lot of charring of the board surface.  I cleaned the board as best I could with the electronics cleaner spray, then rinsed in water, shook the drops off, and placed it to try overnight on the exhaust vent of the soldering fan.  Test tomorrow to make sure it's working OK.

                      Not sure there's anything else I can really do at the moment.  Tomorrow I will test connectivity on Wi-Fi board #3 between pins of J2 (with the new header) and corresponding thru-holes of J9.  Once that's working, I will do a new serial connection between Wi-Fi board #3 and the DE3 board's GPIO1 header using a bundle of Adafruit jumper wires, like I did for the FEDM.  Then I'll test that Wi-Fi connection by running the existing GPS app.

                      Spent a little time playing around with using some of the Adafruit extra-long M-M pin headers and F-F jumper wires to supply the +3.3V, +5V, and +12V power on a small breadboard.  The quality of the power supply connection is poor though - it keeps going out, and you have to hold it in just the right way to maintain steady power.  So, getting the right connector from Digi-Key for mounting on a real proto board will be key here.

                      Temporary hand-made connector/wire bundle for distributing power supply voltages on breadboard.
                      Color code: Black/white = ground; Orange/gray = +3.3V; Green = /ON; Red = +5V; Yellow = +12V.
                      I am a little concerned about distributing +5V power (which will account for the biggest part of power consumption, I'm guessing roughly 4A = 20W worth, since it will drive the FEDM, both Wi-Fi boards, the GPS module, and the fan) via the relatively narrow-gauge jumper wire shown here.  This could potentially represent a fire hazard if this wire overheats.  It would be safer to use a slightly thicker-gauge wire such as the one used in the actual power-supply connector.  Alternatively, we could do an analysis to show that there is no hazard even with this narrow wire.

                      Tuesday, November 15, 2011

                      Tue., Nov. 15th

                      Still need to requisition the batteries/charger for Samad.  First, check to see if the vendor is in the vendor list.  It's not.  Emailed him asking him to either re-source it or get the vendor added.

                      Pascal talked to the construction firm, and they referred him to the architecture firm for the more detailed engineering drawings of the room.  Michael, Juan, David and Darryl were here and worked on the changes for the new input capture datapath for a bit, then once all the Senior Design students were here, they all broke off for their internal team meeting (which they're now holding in the library conference room), and meanwhile Darryl and David worked with Ray on the paper.  Meanwhile, while all this was going on, I continued my testing of the new FEDM setup with the (now) 3 input-capture datapaths.

                      Re-testing the board with 100 Hz pulse source on SMA#1.

                      Current configuration ("Configuration #1") is:
                      • SMA#1: Tektronix function generator (100 pps, -2.5V pulses).
                      • SMA #2: Case #2 (near window).  -->   ~86 pps
                      • SMA #3: Case #1 (near hallway).   -->   ~196 pps
                      In this configuration (spreadsheet "C:\SHARED\Server Code\Pulse-Counts.ods"), the pulse rates (averaged over a period of about a minute) seem to be as shown above.

                      But, mega-weird!  If I switch cables on SMA#2 and SMA#3 ("Configuration #2"), I get:
                      • SMA#1: Tektronix function generator (100 pps, -2.5V pulses).
                      • SMA #2: Case #1 (near hallway).   -->   ~72 pps
                      • SMA #3: Case #2 (near window).  -->   ~303 pps
                      This data makes it look like, Case #2 is producing more and/or stronger pulses, but in configuration #1, this is overwhelmed by the fact that SMA #3 is responding so much more strongly.

                      Suppose we put the pulse generator on the apparently-weakest input (#2), what do we get?
                      (Configuration #3)
                      • SMA #1:  Case #1 (near hallway).   -->   ~172 pps
                      • SMA #2: Tektronix function generator (100 pps, -2.5V pulses).
                      • SMA #3: Case #2 (near window).  -->   ~287 pps
                      So, at least we seem to have some consistency in the sense that when a given case (#2) is on a given input, and the reference is moved between inputs, the measured pulse rate from that case stays the same.

                      Let's switch case #1 & #2 again.  (Configuration #4.)  Now we get:
                      • SMA #1:  Case #2 (near window).  -->   ~241 pps
                      • SMA #2: Tektronix function generator (100 pps, -2.5V pulses).
                      • SMA #3: Case #1 (near hallway).   -->   ~196 pps
                      Again, a particular assignment (Case#1 on SMA#3) gives a consistent rate (196 Hz) no matter which of the other two inputs the Tektronix is on.

                      Might as well finish up by doing the other two arrangements.
                      Configuration #5:
                      • SMA #1:  Case #1 (near hallway).   -->   ~168 pps.
                      • SMA #2:  Case #2 (near window).  -->   ~81 pps.
                      • SMA #3: Tektronix function generator (100 pps, -2.5V pulses).
                      Interestingly, the above is the only arrangement (so far) where I don't get FIFO_FULL errors - perhaps because the pulse rates are lowest.

                      Case #1 on SMA #1 is still about 170 Hz.
                      Case #2 on SMA #2 is still pretty close to 83-84 Hz.

                      Last arrangement (Configuration #6):
                      • SMA #1:  Case #2 (near window).  -->   ~238 pps.
                      • SMA #2:  Case #1 (near hallway).   -->   ~69 pps.
                      • SMA #3: Tektronix function generator (100 pps, -2.5V pulses).
                      Case #2 on SMA #1 is still about 240 pps.
                      Case #1 on SMA #2 is still about 70 pps.

                      Here is a summary of all the data (all figures are averaged pps):


                      Case #1 Case #2
                      SMA#1 170 240
                      SMA#2 70 83
                      SMA#3 196 296

                      Some comparisons:
                      • On average, Case #2 generates ~36% more pulses than Case #1 (fairly wide range though, 18-51% more, depending on which input pathway we're talking about).
                      • On average, SMA#1 detects ~2.6x (2.4-2.9x) more pulses than SMA#2.
                      • On average, SMA#3 detects ~3.2x (2.8-3.6x) more pulses than SMA#2.
                      Note: It seems like on a RESET, the pulse buffer isn't getting cleared out properly - I get some junk when I start again.  Need to fix that at some point.

                      In terms of a close ratio between the two detectors, the best configuration is #4; the ratio of pulse rates there is only 1.2x (241 vs. 196 pps). 

                      The team swarmed in after their team meeting, and I clarified for them again what connects to what in the overall system design (CTU + detector electronics), and they took more photos in preparation for figuring out the exact arrangement of boards within the enclosure.

                      Samad and I found an appropriate connector on Digi-Key to connect to the cable from the power supply and mount on a breadboard or proto board for splitting out the power connections to the various destinations.  He is going to order one out-of-pocket, so he can begin playing around with the power supply setup.


                      Addendum:  It occurred to me on the drive home that if the students have difficulty getting the speed of the FEDM design back up to 500 MHz, one possible workaround (for Phase I) might be to jumper the comparator outputs over to the DE3 board (which has a larger and faster FPGA) and do the time-to-digital conversion over there.  This would also simplify the Phase I system design, since all of the firmware could just run on that board, and we could eliminate one of the Wi-Fi modules.  The high-speed clock produced by the PLL could be directly slaved to the OCXO input, which would eliminate the need for the extra timing sync capture datapath, and save some coding time associated with that (in the firmware & on the server).


                      Some downsides:
                      • We would need to add 15 jumper wires arching from the FEDM over to the DE3.  (5 DAC levels x 3 PMT inputs).
                      • The rise times on these signals would probably be relatively large (several ns maybe), and there could be nonuniform signal delays, leading to additional sources of error in the time measurements, errors which would be difficult to quantify since they would depend sensitively on the exact physical arrangement of the wires.  (We could do some simple experiments pretty easily though with a scope to get a rough idea of how bad these problems are.)
                      • There would be some additional design time involved in merging the two firmware applications (but on the other hand, we have yet to see how much design time will be required to get the FEDM performance back up to 500 MHz in the current setup - could be less, could be more).
                      • This alternative approach would less directly lead into the Phase II design (with the optical sync), since for Phase II we would effectively have to undo the merging of the two applications, and figure out again how to get the FEDM back up to the desired speed.  So in a way it just postpones solving that problem.
                      I'll ask Ray for some feedback on this idea, see if he thinks we should let the students explore it.

                      Ray says he'd prefer to continue pushing the current approach as far as we can first...

                      Emailed Sachin to see if he knows if we could access the EEPROM as a ROM.

                      Another idea:  Just checked the SOPC design, and currently we are using a Nios II/f, which takes 1,400-1,800 logic elements, plus 3 M4K blocks and then some more M4Ks for the memory cache.   If necessary, we could back off to a smaller Nios II/s (or maybe even Nios II/e) and save some of these resources.

                      Monday, November 14, 2011

                      Mon., Nov. 14th

                      Still need engineering schematics/blueprints for both halves of the classroom at the Challenger Center - ping Pascal (or whoever I see today) about that.  Juan emailed me that he is planning to come by today.

                      Samad sent me quotes for the batteries / charger the other day, for demonstrating a wireless detector node.  Need to run them by Ray, and see if we should go ahead and requisition them.

                      Still need to solder the 40-pin breakaway header onto the other Wi-Fi board (the board labeled "#3").

                      Still need to work on the timing of the startup sequence for the FEDM+WiFi board combination... What I started doing last time was adding a delay to the FEDM startup sequence, to give the WiFi board more time to finish starting up before the FEDM started sending a large amount of serial data.

                      Changed gelware and firmware back to the original configuration for a 64KB working_memory module/linker section in M-RAM, except that the firmware now has the 20-second startup delay compiled in - I need to test that part, at least.  Rebuilt firmware; now recompiling gelware.

                      Plugged in the detectors (on SMA #1&2) and the function generator (on SMA #3) and the +5V power connector for the FEDM+WiFi unit and the cooling fan.

                      Moved existing server log files to "C:\SHARED\Server Code\logs 2011-11-14" and started the server.

                      To really test the startup sequence, I will need to re-burn a new image to the EEPROM via Active Serial, so that the newest gelware/firmware will load+run automatically on startup.  OK, it is burned.

                      OK, the new 20-second delay seems to be working, and seems to be adequate (just barely) to allow the present Wi-Fi script to finish starting up first.  (This is assuming that there aren't any networking issues.)

                      Weird, for some reason today I am getting a bunch of FIFO_FULL errors on SMA#1 (currently connected to Case #2), and the event counts seem higher than usual there relative to the others.

                      I switched the input connections (#1 and #2) and that seemed to fix the problem - which shouldn't happen!  Although to be fair, I don't know if perhaps the problem was just that the PMT in case #2 was still warming up, I didn't plug it in as soon as I plugged in #1.  Guess I could try switching them back...

                      Hitting STOP and HSC_STOP.  Switched cables back (Case #2 -> SMA #1, Case #1 -> SMA #2).

                      Whoa, that is weird!  Whichever way the cables are, the pulse counts are highest on SMA #1!  In the current configuration, the ratio is larger between SMA #1 and SMA #2&3.  OK, switched back to (Case #1 -> SMA #1, Case #2 -> SMA #2).  That has a more evenly-balanced ratio of pulse rates, but we still have a faster rate on SMA#1.

                      Theory: The parasitics on SMA #1 (impedance of on-board trace) could be smaller than those on SMA #2, even though these are (I thought) all supposed to be controlled-impedance traces.

                      I probably should also double-check all the pin assignments and the wiring from the DACs to make sure that the same threshold levels are being compared in all cases.  (Particularly the 1st threshold which controls detection.)

                      Let's see, looking at the OrCAD schematic, PMT_1 and VTH1 come to the pin pair TDCIN[0]/TDCIN[0](N), which are M21 and M20.  PMT_2 and VTH1 come to the pin pair TDCIN[6]/TDCIN[6](N), which are U22 and U21.  Let's check the layout.  M21 [/], M20 [/], U22 [/], U21 [/].  Yep, those connections all check out.  Now let's check the Quartus design.  TDCIN0[0]/TDCIN0[0](N) are M21 and M20, while TDCIN1[0]/TDCIN1[0](N) are U22 and U21.  TDCIN0-2 are all I/O standard LVDS.  Port TDCIN0[0] goes to signal pf0[0] and TDCIN1[0[ goes to pf1[0], and these go to thresh_pulse[1] of the corresponding datapaths.  So, no problems there.

                      To try tomorrow: Move the input from the pulse generator to SMA#1 and see if the weirdness persists (or if there is some other kind of weirdness).

                      Wednesday, November 9, 2011

                      Wed., Nov. 9th

                      Ray and I are planning to go over to the Challenger center at about 4:00 pm, to meet with Reed to finalize the issue of the detector locations.

                      Samad is coming by soon to talk about the power supply some more.  I emphasized to him the importance of building proper connectors to connect the power supply (and batteries) to other components.

                      Samad needs to get me quotes for the batteries, battery charger, and any connectors so that I can issue a requisition for them.

                      Also, the mechanical students need to design structures to hold the power supply INSIDE (not outside) the main electronics box, for neatness, and there should also be a little battery-holder box installed on one of the detectors (for a battery-powered detector demonstration).  Mechanicals need to coordinate with Samad on the requirements for this.

                      Cooling of the ambient interior temperature of the main electronics box should not be a problem, even with the power supply inside, as long as there is airflow with vents/fan.  Total system power dissipation is not all that large (20-30 W, laptop range).  Main requirement for cooling is a good heat pipe from the FPGA to the Peltier plate, to get the FPGA as close as possible to the cool end of its 0-85 C operating range.  And positioning the plate so any drips go into a tray and not onto the electronics.  Mechanicals should look into sealing all cold exposed metal parts inside a polymer/resin skin, spray-on perhaps, to reduce condensation.

                      Reed had to leave CLC before we arrived, but his assistant was there.  It looks like hanging the detectors below the ceiling is OK as long as they do not dangle down more than about 1' or so - ceiling height is 9'2".  Main concern is that people don't hit their heads.  Also, there are sprinkler clearances and location of above-ceiling girders to consider.  Requested blueprint for other half of room - Brian said Pascal had one half already, but it needs to be scanned and uploaded to the group blog so everyone can see it.

                      Brian is thinking now of arranging all boards on one plane, instead of using the blade approach.  This is probably a good idea since it will reduce the vertical profile of the enclosure.   Height of enclosure should probably be no more than 14".

                      Also, I discussed with Brian that the enclosure should NOT have RF shielding, since the Wi-Fi communication needs to pass through.  I don't think we're too worried about interference.  Also for educational purposes it would be nice if the electronics was visible.  We agreed that probably best is if 5 out of the 6 faces of the enclosure were transparent plexiglass (except where the exhaust fan and Peltier cooler are installed, obviously), and the other face can be metal that all the boards are bolted to (everything mounted upside-down under it).  That way, students can see all the electronics, and any indicator LEDs on the boards.

                      Tuesday, November 8, 2011

                      Tue., Nov. 8th

                      Ray and I are planning to go over to CLC at about 4 pm on Wednesday to talk with Reed about positioning of the detectors.  I created a Google Calendar event and sent them both an invite.

                      Ray points out that the highest-energy showers have a narrower area at ground level, so placing the detectors as far apart within the building as possible might not be the best approach.  He is going to ask Helio to help us find information about how the shower area varies as a function of primary particle energy.

                      Some things I could work on today:
                      • [/] Requisition for Tektronix BNC cables - submitted
                      • [,] Shrink existing design to make room for timing sync capture datapath (reduce code size, use more of M-RAM)
                      • [ ] Solder headers to 2nd Wi-Fi board, wire up its serial connection
                      • [ ] Reduce startup time of WiFi script / Increase startup delay of FEDM
                      I guess the quickest way to get the startup sequence working is just to add a delay to the FEDM firmware - we can always go back and optimize things later to reduce the delays on both sides (WiFi & FEDM).

                      Added a 20-second startup delay to the firmware in Q:\software_v4\FEDM_ctrl_fw; this firmware version is 0.9.

                      Incidentally, the linker space usage summary is as follows:

                      Info: (FEDM_ctrl_fw.elf) 69 KBytes program size (code + initialized data).
                      Info:                    62 KBytes free for stack + heap.

                      This might be a good time to try shrinking the code size.  Let's compile with space optimization.  In Nios II Design Tools for Eclipse, right-click fedm_ctrl_fw_bsp, select Nios II --> BSP Editor.  No, that's not it.  Select Properties instead from the pop-up menu.  Click "Nios II BSP Properties".  Select optimization level: Size.  Hit OK.  Select Nios II --> Generate BSP.  Select Project --> Build All.  Hm, that didn't seem to work - not enough files were recompiled.  Let's click "Generate" in BSP editor, and rebuild.  Still nothing.  Let's do a Project -> Clean, and clean all projects.  That'll force the issue.  OK, that does it - all files are getting recompiled with -Os.  Now we get in the linker summary:

                      Info: (FEDM_ctrl_fw.elf) 68 KBytes program size (code + initialized data).
                      Info:                    62 KBytes free for stack + heap.

                      Geez, we only saved 1 KB?  That's hardly worth it.  I think I'll change it back to the way it was.  (-O3 not -Os).  OK, that's changed back.

                      Next thing to try:  See if we can utilize the extra 8K of M-RAM (parity bits, not presently used).  Opening Q:\COSMICi_FEDM project, New_with_Nios_trim revision, SOPC builder.  Changing size of working_memory module from 64K to 72K.  This might let us get rid of extra_RAM.  System generation worked.   Extra_RAM is currently 9KB, and it uses M512 blocks, so getting rid of it might free up some space for other functionality.

                      OK, I went into the BSP editor, linker script tab, and changed the size of the working_memory linker region to 73,728 bytes (72 KB), and regenerated the BSP files.  After rebuilding, new linker stats are as follows:

                      Info: (FEDM_ctrl_fw.elf) 69 KBytes program size (code + initialized data).
                      Info:                    70 KBytes free for stack + heap.

                      So that worked as intended; we have an extra 8 KB now available in the stack + heap working memory.
                      So, the stuff that's currently going in extra_RAM could be shoved into there, maybe - except the problem with the M-RAM is that it can't be initialized!  So I'm not sure it really can be used for anything but stack & heap, unless we somehow work around the no-initialization restriction.  I guess .BSS sections can go in there too, but that's about it.  Damn, I just checked the linker script settings, and .BSS is already in there!

                      Darn, when I tried the Quartus compile, the fitter says it needs another M-RAM!  Maybe it can't allocate the extra 8K of parity bits as normal memory?  Let's see if it helps to change the word size to 16 bits (from 32 bits).  Nope, still no dice.  Sounds like I am going to have to stick with using 64KB of M-RAM, unless we find some other way to use the rest of it.  Also, when we try to generate with smaller word sizes, we get the following scary warning:

                      Warning: cpu_0.data_master/working_memory.s1: working_memory.s1 does not have byteenables. Narrow (less than 32-bit) writes from cpu_0.data_master will result in spurious writes to working_memory.s1

                      In other news, Juan and Michael Dean were here today, and I had them start work with Darryl & David on the new datapath for the timing sync capture.  Juan & Darryl are working on the new PIOs and the firmware changes, while Michael & David are working on the Quartus changes.  I copied the latest code (gelware & firmware) to the FEDM_design\FEDM_code\q91 folder so they could be working with the latest versions of everything.  Everyone got off to a good start today, but there is a lot left to do.  Juan & Darryl will give specs to Michael & David as to the PIO interface.

                      Aarmondas did not show up as promised.  Going to send him a strongly worded e-mail.

                      Also, Ray finished his data collection run that he started last night, in which the scintillators were overlapping at their centers.  It didn't change the location of the histogram peak much, for coincidence pulses due to showers propagating down through both detectors, so we are still thinking that 200 mV is the right level for the thresholds.

                      Speaking of which, currently the furthest threshold (DAC #6) is -1000 mV (off of a +2.5V DC bias) or in other words +1.5V absolute.  Thus, DAC #6 would be appropriate to use as the rising-edge threshold for the timing-sync input, since that pulse is going to be at (I believe, if I recall) the +3.3V logic level of the DE3 board.