High speed object detection (sport)

A

Anton Mamyko

"you need to use a filter on the lens to filter out most of the ambient light, and a strong strobe."

This is a great idea for isolating the image from ambient light. However I am concerned about price of a high-powered strobe lighting. However the double-exposure per frame-giving velocity is a good one.. The requirement is that the player be able to chose at which distance, from him, he wants the accuracy measured; 75,100,125,150 m. So the choice between:

1- A single, high fps camera with highly powered and synched stobes (giving velocity) being movable by a gantry-crane of some sort (which sounds like a project of its own)

or

2- A commercial-type camera over Each of the required sensing distances.

will be decided, like everything, on their respective costs. IF standard cameras can be implemented then the 2nd choice would be the cheapest of course.

"Running continuously is probably not a good mode as you need to synchronize with the target and not the power line."

I was under the impression that security cameras run continuously, at their FPS. Hence a capture command will always arrive asynchronously with the camera's scanning/exposure.

(source: "High speed, Real-Time Machine Vision by Perry C West for CyberOptics. pg.10)

So would I have to settle for the fact that I cannot use a standard camera and (at the very least) would have to get a specialised camera with the ability to start their exposure on command?

" NTSC and I think, PAL will try to sync with the line for reasons beyond the scope of this discussion but if you run the camera on DC and trigger a single frame you have your best chance of catching the event" It is definitely beyond my knowledge of electronics but does this have to do with the image being interlaced? Could you explain the "run the camera on DC" please?

"Yes, you don't care about the blur in the direction of motion. Whether you get a sharp circular image or a column of blur, given a symmetrical circle of confusion, the edges and center should be reasonably accurate. Strobe lighting would be costly and not necessary as the blur doesn't affect the information you need to extract."

But would the resulting image of the blur of the bowl be bright enough to isolate an image out of it? At 30 fps, the column could be over a meter long, 60 fps-half that etc.. and from what I understand- the light intensity of that would be register by each pixel will be "dilute"?

I am trying to get an idea of how to substitute strobing....say I use normal, diffuse light over the FOV, which is painted white to contrast with the steel bowl).

1 - The bowl ,going at full steam, trips the through-beam (neglecting latency to trigger)

2 - The light turns on, the camera start exposing for 16.6ms (say it is a 60 FPS, asynchronous camera-starts exposing on command)

3 - The bowl creates a dark contrast in the FOV, but as it travels the length of 0.5m the pixels in its path take on a bright value of the background (am I correct in saying this?)

This makes the resulting image hard to threshold due to the lack of contrast, correct?

If this is so then would my solution be a camera with high FPS or high-speed strobing to attempt to reduce the "smear" of the target and hence get a better-contrasting image right?...The challenge would then be to figure out how to balance the two.

The reason I keep talking about vision software packages is not because not because I am ignoring your advice but because I do not have much experience at programming and the though of writing an application scares me :)

Very thankful,
Anton.
 
I wasn't talking about a high speed (fps) video camera. I was talking about a still camera with manual control. For example, I have a camera where I can set the aperture, and also send commands to open and close the shutter (the shutter can stay open up to 30 minutes). The process would be:

1) Detect the approach of the bowl by some other means (i.e. an optical sensor).
2) Open the shutter.
3) Flash the strobe once.
4) Wait a specified time.
5) Flash the strobe again.
6) Close the shutter.

If everything is set right, you will get one image in which the bowl appears twice. It's the same effect as produces blurring in a still image, except the intermediate positions of the "blur" are not visible while the strobe is off. This is a single image, not a sequence of frames. You want a still camera for this, not a video camera.

As another idea altogether, you might take Steve Myres' idea of a sensor grid and modify it a bit as follows:

1) Two parallel sensors (A and B) cross the lane at a 45 degrees.

2) Two more sensors (C and D) also cross the lane, but at 90 degrees to A and B.

3) The time between A and B being tripped gives the velocity along the axis perpendicular to A and B.

4) The time between C and D being tripped gives the velocity along the axis perpendicular to C and D.

5) Combining the two velocities will give you the actual velocity and vector.

6) The time between A and D or C and B gives the distance from the centre of the lane.

7) The sign of the difference between the A-D time and the C-B time tells you which side of the lane the bowl is on.

This looks very simple on paper, but I expect the mechanical problems are non-trivial. You have to get everything lined up as precisely as possible, plus provide an accurate means of calibration to take the remaining errors into account. You would also have to take beam spread into account (lasers may help here). You also have the error introduced by the bowl slowing slightly while in the measuring box.

On the other hand, once it's fabricated properly it is probably easier to install and maintain. It's basically going to be a long aluminum extrusion beam with four sensors bolted to it.

As for software, there's a significant software component in this project no matter how you do it. The second idea above sounds simple in theory, but there will still be software involved and also some custom electronics. You will need to do the timing in hardware to get any worthwhile accuracy. You could probably prototype it using a counter-timer card in a PC, but a practical commercial product would need to do it with a custom micro-controller board. On the other hand, the micro-controller board would eliminate the PC from the system altogether and just leave a box with a read-out display.
 
S
That is VERY slick, Michael! You'd need very precise timing and interrupts, but if you have that you're good to go.
 
C

curt wuollet

>> "you need to use a filter on the lens to filter out most of the ambient light, and a strong strobe." <<

> This is a great idea for isolating the image from ambient light. However I am concerned about price of a high-powered strobe lighting. However the double-exposure per frame-giving velocity is a good one.. The requirement is that the player be able to chose at which distance, from him, he wants the accuracy measured; 75,100,125,150 m. So the choice between:

> 1- A single, high fps camera with highly powered and synched stobes (giving velocity) being movable by a gantry-crane of some sort (which sounds like a project of its own)

> or

> 2- A commercial-type camera over Each of the required sensing distances.

> will be decided, like everything, on their respective costs. IF standard cameras can be implemented then the 2nd choice would be the cheapest of course.

>> "Running continuously is probably not a good mode as you need to synchronize with the target and not the power line." <<

> I was under the impression that security cameras run continuously, at their FPS. Hence a capture command will always arrive asynchronously with the camera's scanning/exposure.

> (source: "High speed, Real-Time Machine Vision by Perry C West for CyberOptics. pg.10) <

True enough, but the uncertainty isn't all that bad, depending on the speed of the object. The frame grabber will grab a frame, which will likely be in progress, so you get it on completion. The aspect ratio is 4:3 so the window will be 3/4 of your lane width. You can do the math to determine if you will catch the target. Now, we don't know exactly what the camera is doing, but it should take the entire frame in one exposure, then read it out and send it as interlaced video. If the object can travel more than 3/4 of the lane width in 1/60th of a second you may be out of luck. But you could expand your field of view to the area needed and still have good resolution.

> So would I have to settle for the fact that I cannot use a standard camera and (at the very least) would have to get a specialised camera with the ability to start their exposure on command? <

>> " NTSC and I think, PAL will try to sync with the line for reasons beyond the scope of this discussion but if you run the camera on DC and trigger a single frame you have your best chance of catching the event" It is definitely beyond my knowledge of electronics but does this have to do with the image being interlaced? Could you explain the "run the camera on DC" please? " <<

This is just to have the camera free running. Some will try to sync to the line if it can read the ripple. With the camera free running you avoid some problems with lighting that is line dependent like fluorescent and metal halide lamps.

>> "Yes, you don't care about the blur in the direction of motion. Whether you get a sharp circular image or a column of blur, given a symmetrical circle of confusion, the edges and center should be reasonably accurate. Strobe lighting would be costly and not necessary as the blur doesn't affect the information you need to extract." <<

> But would the resulting image of the blur of the bowl be bright enough to isolate an image out of it? At 30 fps, the column could be over a meter long, 60 fps-half that etc.. and from what I understand- the light intensity of that would be register by each pixel will be "dilute"? <

The camera has it's own shutter that is controlled for the purposes of exposure. The CCDs are very sensitive, so with good lighting the shutter speed will be much, much less than 1/60th of a second. Some can do 1/10,000th in strong light, so the blur isn't as much of a problem as it might seem. And that complicates the idea of strobing.

> I am trying to get an idea of how to substitute strobing....say I use normal, diffuse light over the FOV, which is painted white to contrast with the steel bowl).

> 1 - The bowl ,going at full steam, trips the through-beam (neglecting latency to trigger)

> 2 - The light turns on, the camera start exposing for 16.6ms (say it is a 60 FPS, asynchronous camera-starts exposing on command)

> 3 - The bowl creates a dark contrast in the FOV, but as it travels the length of 0.5m the pixels in its path take on a bright value of the background (am I correct in saying this?)

> This makes the resulting image hard to threshold due to the lack of contrast, correct? <

No, for the reason above. With good lighting you should either get a strong image or miss it completely, depending on the latency. Since many of the details are up to the implementation in the card, the OS and the trigger device, probably the best thing to do is to set up a camera and try it. If you set your camera up to "see" the max distance the target should cover in a frame. you should catch it. You should be able to tune the trigger distance to catch it 100% of the time. At some field of view and trigger distance this should be true. Then you determine if the pixels in your active area are enough to give you the required accuracy, which they should be. If this is a little confusing, I am translating my intuition, and I think very strangely at times :^). I think the odds are good enough that I would proceed to proof of concept. With a board camera, a capture card, and a trigger sensor, you can save much agonizing over things you can't really know any other way. If you can consistently get an image, the processing should be straightforward. All my vision projects have worked so far, but of course, YMMV.

> If this is so then would my solution be a camera with high FPS or high-speed strobing to attempt to reduce the "smear" of the target and hence get a better-contrasting image right?...The challenge would then be to figure out how to balance the two. <

These are probably not practical due to cost, check it out.

> The reason I keep talking about vision software packages is not because not because I am ignoring your advice but because I do not have much experience at programming and the though of writing an application scares me :) <

I understood that, but, I can't imagine the packages I've seen as having a bowling scoring option. And I've written a lot of these type of hacks and the thought of doing an application scares me too:^) I don't generally write code because I am a programmer. I write code because it's the only way to make the computer do what I want it to do. Once I had the code returning the numbers, then I'd worry about what to do with them.

> Very thankful,
> Anton.

No problem, if I suddenly become unresponsive, it's not lack of interest, it's that I didn't figure out a way to pay my ISP.

Regards
cww
 
S
"The aspect ratio is 4:3 so the window will be 3/4 of your lane width."

Really? For some reason I thought it would give 1.33 lane widths in the DOT. Hmmm. ;-)
 
C

curt wuollet

NTSC video at 640x480 as an example.

You could orient the camera so that you have the long dimension down the lane. But that would limit your resolution in the axis of interest. By rotating the camera 90 degrees, you get the higher resolution in the axis of interest and the window down the lane is 3/4 of the lane width. As I said, I was translating my intuition and I assumed the camera would be rotated so that the higher resolution was across the lane.

Regards
cww
 
A
Hi guys, my apologies for abandoning this thread for so long without a word of explanation.

I would like to thank you all for guiding me through my lit review- for which I got 70% in semester 1. I know it means nothing but I Have all of you mentioned in the acknowledgments.

I took Mr. Griffin's innovative idea on to be developed in semester 2. The idea was well received in last week's annual engineering exhibition - I took no credit for the idea.

Admittedly, I have made little progress but this thread explains where I am at the moment:

http://forums.ni.com/t5/Counter-Timer/Timestamping-4-DI-Sequential-logic-PCI-6221/td-p/1526978

Thanks again and all the best with everything. I would like to believe that, once I reach a certain professional level, I will too help the up and coming engineers- selflessly.

Kind Regards,
Anton Mamyko
 
It's good to hear about your project. When you are trying to figure out the NI counter-timers however, I would suggest trying to figure out what the actual counter chips are doing, in terms of gating, register pre-loads etc. If you just look at them as a "black box" through the NI VIs they can be hard to understand. When you see that the output of counter "A" goes into the gate of counter "B", then it can be clearer.

The NI VIs are oriented towards people who want them to emulate traditional separate lab instruments that they already understand. That can actually make them harder to understand if you are trying to do something that doesn't work that way.
 
A
Amen to that, it took some amount of time to even begin understanding how some of the DAQmx labview VI's work.
However I got this rough thing out:
http://img851.imageshack.us/i/antonmamykod.png/

I am detecting changes on 4 lines and using those change events to trigger a buffer readout of the counter.
Both the events and counter output are written in the sequence of occurrence to a text file.
Then I got my housemate to put together a program in Java that assigns those values to the lasers A B C D and performs speed.position.accuracy calculations.

As rough as it is - it does what I set out to do - measure accuracy across the width of the track!
Testing has not happened yet because a 2 weeks delivery promise on 4 photo cells has now been broken by over 3 weeks.....

In my report I will explain the proof of concept on the one line and try to waffle on about how I would go about implementing this across the full length of the track etc...

SO here is my update. Cheers guys.
 
Top