Date: Tue, 25 Apr 2006 01:39:43 -0500 (CDT)
Subject: need adaptive vision
X-UID: 168

The robot was driven Sunday morning a few minutes after dawn. It was much 
smoother as now shell scripts are used for all major steps. The stress 
level is much lower when setting up and driving.

http://golem5.org/robot1/video/drive01.mpg

Software appears to be functional and stable. Most important, when the 
joystick controlling the drive motors is released, the robot stops. If the 
pause button is depressed, the robot stops. Remote kill is the one 
function that must work. The crash last week made this obvious.

http://golem5.org/robot1/images/img2888.jpg

The steering was sticky and prone to snowplowing. I think that tension 
from the elastic cords has dimished over time. The bungee has stretched 
out a bit. I've also moved the cords around which reduced the tension 
pulling the wheels into alignment.

I'm kind of amazed at how well the robot has handled in the past. It was 
very sluggish this last Sunday. It's amazing how software changes can 
completely alter the feel of driving. It's the exact same hardware but the 
handling is very different.

The movie above is the first time that sensor readings and commands have 
been correlated to the video stream from the network camera. I wrote the 
filter that extracts images out of the motion-JPEG stream (it is trivial). 
So I know there is absolutely no buffering in it.

Unfortunately, it appears that the one second delay is inside the Sony 
SNC-M1 network camera itself. There's no way around this. It always shows 
you what it saw about one second ago. If you watch the gyro readings, you 
can see that they lead the video by about one second. This is true of all 
overlaid robot state.

This is not completely hopeless as there is the concept of feedforward 
control. A controller must act to control plant state in the future rather 
than in the present. But again, things are more complicated than they 
should be. Combine this with the broken DMA on the single board computers 
which limits the USB webcams to perhaps one frame per second and the 1.5 
meter range of the Sharp IR rangefinders - sensing the environment is not 
trivial at all.

It's obvious that adaptive vision is necessary. In the old video,

http://golem5.org/robot1/video/oil2826.mpg

the image processing seemed to work quite well despite being very simple 
color quantization. The reason is the ambient lighting was good. It was 
overcast with bright and diffuse lighting. This accentuated the light 
colored parking lot surface and minimized shadows or specular reflection.

During the drive this last Sunday, there was much less light as the time 
of day was just after sunrise. Specular effects of light were significant. 
That's why color quantization didn't really work.

Adaptive vision is a requirement for a robot that has any sort of real 
world stability. That means there is cross-correlation and fusion between 
different sensor systems. In my case, there are three remote sensing 
systems.

1. monocular network camera
2. stereoscopic webcams
3. IR distance sensors

Each is very different. Ultimately, the IR distance sensors are the most 
reliable although they have short range. If the optical cameras are 
ineffective under current conditions, then the robot must slow down and 
rely on the distance sensors to feel it's way around. If the lighting is 
optimal, as it can be on an overcast day, then the optical cameras start 
working well again. There's no way to have one sensor that works well 
under all conditions in all situations. That's just as true of the human 
eye. If we walk into a dark room, then we require a flashlight and so 
switch from a passive to an active vision system (emitting energy).

I need to get in some better test runs driving the robot around obstacles 
to get an idea of what kind of image processing algorithms can work. The 
plan is to first drive and record all sensor, command, and video data. 
Then experimentation is done offline in postprocessing on the data sets to 
get an idea of what works and what doesn't. The movie with overlaid sensor 
data is the result of one such experiment.