Skip to main content

7 replies

Need Some Suggestion to Independently Verify the Accuracy of a Transponder System?

M millisecondsMember
We at Milliseconds have been using transponder timing systems for quite a while now and feel transponders are a great tool for timing many types of races. I have been curious for a long time about the claims of various manufacturers make over the accuracy of their systems. For example, AMB, our preferred vendor claims one thousandth of a second for both the ChipX and Activ systems. While Winning Time is claiming one hundredth of a second for their WTActive system. Most passive systems claim between a tenth and a hundredth of a second. I would like to design a test to independently determine if these claims are true.

I have three reasons for doing this: The first and most important reason is better know the hardware I am using and to use the right tool for the job. Second, while the clocks used by manufacturers may have an accuracy of what they claim, I would like to determine if the transponder read actually occurs precisely over the finish line every time like they claim. The third is to, hopefully, have some data to back us up when we say transponder timing, when used correctly, is a viable alternative to other forms of timing.

The methodology used for collecting and analyzing the data should be easily understood by someone who is familiar with timing issues, but not necessarily an electrical or RFID engineer or professional statistician. Also the equipment used should be off the shelf timing equipment and any other things used in the testing should be easily built by anyone wishing to reproduce the test. The idea being two fold: Someone looking at a report of what we did can say "While they may have a stake in the outcome of this experiment, they did a reasonable job in collecting and analyzing the data and their conclusions are sound." And, second, it is easily reproducible.

To that end, here is what I was thinking about how to gather the data. I have available to me an ALGE Timy, and Comet that I can sync with GPS. A couple of ALGE RLS1n photo cells. And AMB ChipX and Activ systems.

1) The Timy and Comet hooked up to the photo cells would serve as controls.

2) All timing equipment would be GPS synched.

3) The finish line would set up per the manufacturer's specifications, measuring antenna widths and photo cell and reflector heights, instead of using the calibrated eyeball.

4) There would be a way of consistently passing a transponder over the finish line and tripping photo cells at the same time. Perhaps a 2 by 2 vertically attached to a platform on wheels. I expect there would also have to be a way to ensure the speed of the platform is consistent.

5) I would pass the transponder over the finish line in various places (center, left side, right side etc), at various heights and speeds to make sure there are not variations.

6) I would use a few different transponders to see if there is any difference.

I do have a few questions:

1) Is there anything else I need to account for in my testing?

2) How many passings should I do to have a good set for statistical analysis?

3) Is this test too simple to be credible? If so, what do I have to do to make it more credible?


Thanks in advance

Mac
Milliseconds Sports Timing
Original Post

Replies sorted oldest to newest

G geosflsMember
I think you might be on the right track Mac but I would use FinishLynx too. I have used Lynx side by side with AMB Activ on both Mountain Bike, road cycling criterium and ski/snowboard cross. The AMB claims were shown to be quite accurate. I didn't use statistical measurements but direct comparison of transponder reading and Lynx results for hundreds of passings. The vendors in Europe involved with the Tour of Italy and France have had side by each Lynx and AMB systems going for several years now and I think the results have been positive.
themightyskunkthemightyskunkMember
Very simple methodology, easy to understand and dead-accurate:

Duct-tape a dozen chips 2-3 cm apart on the top tube of a bike (or the deck of a DH skateboard) and have somebody ride the bike/skateboard through a Lynx or OptiC2 photofinish camera. Do it one time to synch the two devices to each other. Repeat over and over, at different speeds, comparing the decoder times AND ORDER OF FINISH to the camera times / order of finish. Put the two sets of results into a spreadsheet, calculating the standard deviation and quantifying the errors in the order of finish.

The photofinish is the benchmark, as we know it is accurate and in fact admissible as prima facie evidence both in a court of law and in CAS.

A "statistically significant" sample space in a standard normal poisson distribution starts at 30 data points, but the more the better.

Let us know what you find.

Yes, I have done this myself.
M mosMember
Hi, all the methods suggested so far have merit. We designed a similar test for a few transponder systems. We used a car (in order to get reasonable speeds, with an indication of the speed, although a bike or similar would do fine).

We taped 10 of each transponder to a reasonably flat rear bumper of a car, and drove around for several laps or about 1 minute (although the lap time dosn't really matter), while using a photofinish to record each finish time and position. The photofinish gives a precise time gap between each transponder to account for any curve in the bumper.

By repeating a circuit, we recorded the precision of the finish times. We also got the relative times- using a circuit, each transponder starts and ends each loop in succession, which additionally shows their precision relative to their own triggering, as well as relative to the photofinish. In theory, I believe every transponder should have an identical lap time (except first and last laps) if the car went across the line each time straight.

For our second experiment, we taped 10 of each transponder along the side of the car at intervals as precise as we could make them: to test the consistent gap between as well as the sequence of the reads.

There are various ways- this is only one, hope it helps someone out.

We've sent some questions off as a result of the testing, which we're discussing (and prefer not name anyone until they've had the chance to respond). Once that's done I'd be happy to share our results, and interested to hear other results also.

+M
fkpfkpMember
Look forward to the results of the testing. We will go into our archives and find the data that we collected on the older AMB Cyclip system in 2003. We did a study using this active systems as a test for determining the feasibility of using them for UCI team pursuit events. We used human timers pressing buttons, Lynx cameras, and the Cyclip transponders to collect the data.

The biggest issue we discovered as I recall was getting the mechanics to install the transponders on the bikes in a consistent manner. The reading of each unit could vary by quite a margin depending on how the unit was mounted. Even a slight angle forward would result in the bike being read quite a distance from the actual line.

Now of course a great deal has been learned since 2003 and the earlier test may not be useful other than the fact that allowing the riders/mechanics to install the units is not a fair system.

The contractors such as Matsport that use these systems in conjunction with their photofinish systems take a great deal of care to make sure that each unit installed on a riders bike, spare bike on the team car, neutral support bike, etc is logged, checked, etc. This requires having a technician that travels on tour with the express task of making sure that the units are installed properly prior to every start. They have a gig that each bike is fitted into and a pointer shows the precise location. This takes into account the fork rake, tire size, wheel, etc. the technician then emails any issues to the finish line so that the finish crew can adjust the data base.

All of this seems to be more critical for this type of system. Just like registration requirements. You have to make sure that you do more work on the front end to make the back end of actually doing the results works.
themightyskunkthemightyskunkMember
Another issue here, which Milliseconds touched on briefly, is that nobody has established the accuracy of GPS synch. GPS is not very accurate at the resolutions we work (or at least the resolutions at which I work). In terms of geographic placement, 20 meters is as close as most GPS manufacturers will claim, so any data collected with GPS in the process is already suspect before the test even starts. 20m - when the vehicles are traveling at NASCAR speeds - is already 2/10ths of a second of slop introduced into the test.

I have tried synching several brands of decoders, and they do not allow a synch in the FIS-accepted or FEI-accepted sense. You can open a TCP/IP session and the decoder will accept a synch pulse to establish a baseline TOD, but it doesn't actually change the time base, it simply establishes an offset to the time base for that particular connection , which then vanishes when the TCP/IP session is closed. Obviously this can't be used for actual race timing, as if the session closed accidently in the middle of the race, the data collected in that session would be incompatible with the data collected from that point forward.

If GPS is going to be involved, then it must be disconnected once the baseline offset between the decoder time bases and the other time bases has been measured and documented, so that the time bases can all drift on their own. Otherwise, the drift pattern graph will look like a mobius strip (due to the constant, possibly erroneous GPS correction) instead of linear or asymptotical, and all your data will be crap before the analysis even begins.

Anybody who leaves the GPS synch going during a race is exposing themselves to this mobius-shaped correction error. Again, not a big deal if your vehicles are human long-distance runners, but for cars or even bicycles at pursuit/sprint speed, the error is significant enough to potentially invalidate any results.
G gregblaskoMember
James you bring up some valid points with regards to GPS Sync and the approach that is taken to actually sync a transponder hardware device.

Geographic accuracy of GPS devices seems to be far less accurate when compared to the actual time precision a GPS can provide with the proper type of GPS receiver. USNO claims an accuracy of +/- 340 nanoseconds.

Reference:
http://www.ntp-time-server.com.../gps-time-server.htm

http://tycho.usno.navy.mil/gpstt.html


Of course this sounds great but the accuracy will be limited to what type of GPS receiver you may have and the type of output that you can receive from it. For example AMB equipment uses either the Garmin 35 GPS (http://bit.ly/8pk0D) or the Garmin GPS 18 (http://bit.ly/8bx8h). I don't recall the exact configuration of these devices into an AMB system but the Garmin GPS 18 claims a +/- one microsecond at rising edge of the pulse.

What's nice about AMB is that there is already a provided interface into the FinishLynx software so syncing the devices to establish an equal time base is quite easy. The same should work for IPICO equipment as there is an interface within FinishLynx for this hardware too.

The real kicker here is taking this approach to other transponder timing systems. How could you get an equal time base with a Lynx camera and say a Championchip system or any other transponder system? Some type of sync interface would probably have to be written.


quote:
I have tried synching several brands of decoders, and they do not allow a synch in the FIS-accepted or FEI-accepted sense. You can open a TCP/IP session and the decoder will accept a synch pulse to establish a baseline TOD, but it doesn't actually change the time base, it simply establishes an offset to the time base for that particular connection , which then vanishes when the TCP/IP session is closed. Obviously this can't be used for actual race timing, as if the session closed accidently in the middle of the race, the data collected in that session would be incompatible with the data collected from that point forward.


With regards to having a consistent time base between connections, for AMB at least if a decoder is synced via GPS and even if GPS is then removed the raw time reference the AMB decoder spits out is seconds since January 1, 1970. If you know your offset time to TOD and that offset is used for all connections to the decoder to establish a TOD reference wouldn't this be valid? All recorded times could all be shown with reference to seconds since Jan 1, 1970 for auditing purposes by connecting a printer to the serial port on the decoder... I wonder if this could somehow get approval from FIS or FEI?

If you want to perform a far less accurate sync to US Naval Observatory time I've found this service useful for hand syncing TOD race clocks at starting lines for running events.

http://www.usno.navy.mil/USNO/time/telephone-time
themightyskunkthemightyskunkMember
Good points, Greg. I like the shape this conversation is taking.

Up until now, you've had three schools of thought: (A) Those who maintain transponder / decoder data is gospel, to whatever resolution the manufacturer claims and however it is deployed, (B) Those who think transponder data is semi-useless crap, because nobody knows WHAT is going on under the hood from manufacturer to manufacturer, and no standards exist to verify the various manufacturers' accuracy claims, and (C) those who really don't care, as long as nobody questions or complains.

The fact remains that, due to laboratory homologation standards such as FIS and FEI, you can set up timers and photocells from a dozen different manufacturers (Heuer, Alge, Longines, SEIKO, etc etc) on a table, synch them all using proven and clearly documented methodology, and you will get a very predictable result out of all of them, to >= their advertised resolution. You can even mix & match, (i.e. Alge cells with a Heuer time base) and get the same results.

There are no such standards, no such documented procedures, and no such demonstrable reproducible results, for chip timing products. Nada. Zip. Bupkis. In one of the few clearly documented tests of chip timing equipment, the AMB Cyclip product flunked a UCI test, and was banned from cycling events.

From the test data I have seen (and produced myself) I am firmly in camp (B). When I run the exact same test with chip timing equipment 10 times and get a different order-of-finish every time, it doesn't exactly leave me full of confidence in the technology. For those of us who work exclusively in a live TV environment, where results can be visually proven false or at least questionable (Phelps vs Kavic 100m Olympic butterfly, Schladming slalom 2005, Indycars @ ChicagoLand 2008), this lack of consistency is deadly.

The guys in the FIS Timing Working Group who came up with FIS's standards & procedures have been at times maligned, slandered, complained about, and attacked personally, but they did the industry a huge favour, a gift that keeps on giving. If you or I want to design and manufacture a photocell, the standards it must meet are crystal clear. If you or I go to buy a photocell or time base which has been certified to the FIS standard, we know exactly the minimum performance we are buying, regardless of the manufacturer's name or reputation. When we go to set the stuff up on the hill, the line between legal and illegal setup methodologies is clearly defined.

If somebody or group can come up with similar standards for chip timing devices and prove to the marketplace which technologies work as advertised (if any), then as far as I'm concerned chip timing becomes a whole different ballgame. I could be made into a believer.

Add Reply

Post

Membership Required

We're sorry. You must be signed in to continue. Sign In or Register
×
Are you sure you want to remove from your Block List?
×
Loading...
×
When you block a person, they can no longer invite you to a private message or post to your profile wall. Replies and comments they make will be collapsed/hidden by default. Finally, you'll never receive email notifications about content they create or likes they designate for your content.
Note: if you proceed, you will no longer be following .
×
Link copied to your clipboard.
×