Overview
--------

This a Robotics Vision API for lejos.

It provides motion detection, color detection, light detection and interfacing
with robots that use the Lego Mindstorms RCX brick, to allow Mindstorms robots
to respond to what they see.

It supports most of the functions of the Lego Vision Command software, but in 
an open extensible way that will allow much more sophisticated functionality 
to be added.

Although it is designed for use for the Lego Cam that comes with the Lego 
Mindstorms Vision Command product, it will work with most PC Web cameras. 
It has been with the X10 XCam2 wireless camera. This camera can be mounted 
on a Lego Robot to avoid the trailing USB wire, which is a problem with the 
Lego Cam.

Acknowedgements
---------------

Some of the ideas come from the Robotics SDK produced by the UK Sun Technology 
Evangelist, Simon Ritter - see 
http://www.sun.com/developers/evangcentral/totallytech/jmf.html - 
Image Capture From Webcams using the Java[tm] Media Framework API, 
and http://www.sun.com/developers/evangcentral/totallytech/robo-sdk.html -
Java[tm] Technology Robotics Developers Kit.

Some of the code and ideas come from the projects of Konrad Rzeszutek - 
see http://darnok.dhs.org/~konrad/download.html - Konrad Rzeszutek's Projects.

Functions supported
-------------------

Functions supported include:

 - A moving video display on your PC of what the robot sees
 - Definition of regions in the field of view
 - Flipping the image horizontally to fit the type of example best
 - Detecting motion
 - Detection of specific colors
 - Light detection
 - Taking snapshots
 - Recording videos, including sound
 - Playing sound files
 - Sending commands to the RCX
 - GUI control of color and motion sensitivity, region display, etc.
 - Extensibility of the GUI to allow remote control of the robot, etc.
 - The remote control class can also support speech control of the robot

Regions
-------

All recognition occurs within a region. Currently only rectangular regions 
are supported. Regions are numbered from 1, and are their outlines and numbers 
are overlayed on the moving video display.

Listeners
---------

Three types of listeners are currently supported:

 - Motion Listeners - detect motion in a region
 - Color Listeners - detect a specific color in a region
 - Light lsteners - detect bright light in a region

There are three interfaces corresponding to these: 
   
   MotionListener, ColorListener, and LightListener.
      
Responding to what is seen
--------------------------

The various types of listener can respond to what they see in specific regions.

Examples of possible responses, supported by the API, are:

 - Taking a snapshot, e.g. of an Intruder
 - Recoding a video of what is seen
 - Playing sound effects or musical instruments on the PC
 - Sending commands to the RCX to move the camera, turn towards light, avoid 
   motion etc.

Remote Command Execution
------------------------

Sending remote commands to the PC is done by the josx.rcxcomm remote method 
execution classes. Client stubs and a server implementation of functions 
needed for camera and robot movement are provided as part of the vision API.

Example
-------

An example program that detects motion, sounds an alarm and takes a snapshot of
an Intruder:

import josx.vision.*;

public class Motion implements MotionListener {
  private static int image = 1;
  long lastPlay = 0;

  public static void main(String [] args) {
    (new Motion()).run();
  }

  private void run() {
    Vision.setImageSize(320, 240);
    Vision.addRectRegion(1, 0, 0, 320, 240);
    Vision.addMotionListener(1, this);
    Vision.startViewer("Intruder Detector");
  }

  public void motionDetected(int region) {
    if ((System.currentTimeMillis() - lastPlay) > 1000) {
      lastPlay = System.currentTimeMillis();
      Vision.snapshot("Intruder" + image++ + ".jpg");
      Vision.playSound("../../Effects/Alarm.wav");
    }   
  }
}

Prerequisites
-------------

You need the Java Media framework on your PC to run the lejos Vision System.

Windows:

This is available from http://java.sun.com/products/java-media/jmf/

It is best that your camera is plugged in when you install JMF, as this will 
ensure that the capture devices for it are put in the JMF registry.

Linux:

Use a suitable driver for the camera. If you are using the Lego Camera, a driver can be got from:

http://qce-ga.sourceforge.net

This is for the Quickcam which is similar to the Lego Camera. Note that you need to get the source 
from CVS and checkout the qc-usb module.

The JMF from Sun does not pick up the camera correctly and so you need to use the JMF from
Blackdown.org. However, at the time of this writing, there was an issue with licensing and Blackdown
had stopped the downloads but you can still get it from the mirror below:

You can get this from:

http://www.opennms.org/files/mirrors/blackdown-java/JMF/2.1.1/i386/fcs/

You may have problems with the XLibRenderer plugin. If so, you can use the AWTRenderer
instead.

Building on Linux and Cygwin:

Make sure that JMFHOME is defined and pointing at your JMF installation directory.


Configuration
-------------

Ensure that JMF is on the CLASSPATH by following the JMF installation 
instructions. 

On Linux, make sure that the $JMFHOME/lib directory is added to LD_LIBRARY_PATH using
the command:

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$JMFHOME/lib

Run the JMStudio application to determine what video and sound capture 
devices are available on your system.
Look at properties to determine the video and sound capture devices.

If the camera was not on when JMF was installed try doing 
"Detect capture devices". 

If this does not work, try calling jmfinit from the JMF bin directory.

Edit the video.properties files in c:\vision\examples\motion etc. 
There is currently a version in each example directory and you need to edit 
each one to run the example. You need to set video-device-name and 
sound-device-name to the exact text in the JMF registry. If Direct Sound 
capture is available on your windows machine, use that, as Java Sound capture 
does not seem to work on some PCs.

For some of the examples, you need to copy the Effects directory from the 
Lego Vision Command VCSData and Piano and any other instrument
directory you want from the VCSData\Instruments directories.
For a default installation of the Lego Vision Command software

The full path for VCSData is 

c:\Program Files\LEGO MINDSTORMS\Vision Command\data\hdData\VCSData.

They should be copied to subdirectories of c:\vision, e.g. c:\vision\Efects 
and c:\vision\Piano.

Running the Examples
--------------------

Motion example:

Change directory to vision\examples\motion and do:
    
javac Motion.java
java Motion

An alarm is sounded and a file Intruder<n>.jpg is produced whenever motion 
is detected.

Alarm example:

Change directory to vision\examples\alarm and do:
      
javac Alarm.java
java Alarm

This is a security alarm that can be turned off by the correct color being
detected in region 3. You will probably need to edit the source to change 
the color to one that is detected by your camera. The current average RGB 
value is what is being detected in region 3 is displayed on System.out.

Music example:
 
Change directory to vision\examples\music and do:
      
javac Music.java
java Music Piano

You can use other instruments that are available with the Lego Vision Command 
software, e.g. "Guitar2"

Light Seeking example:

This example uses the RCX remote control class to respond to what the camera 
sees.

It is a very simple light seeking example. When it sees light in the central 
region it moves forward. When it sees light in the left or right regions, it 
turns towards the light.

It needs the Rover example program running on the RCX. 

Change directory to vision\examples\light\rcx and do:

lejosc Rover.java
lejos Rover

and press the Run button on the RCX.

You then need to compile and run the PC program. 

To do this, change directory to c:\vision\examples\light\pc and do:

javac Light.java
java Light

Shine a bright light at the camera to move the robot about.

Remote control example:

This example uses the RCX remote control class to control the robot and let 
the robot respond to what it sees.

It needs the Rover example program running on the RCX. 

Change directory to vision\examples\light\rcx and do:

lejosc Rover.java
lejos Rover
      
and press the Run button on the RCX.

You then need to compile and run the PC program. 

To do this, change directory to vision\examples\remote\pc and do:

javac *.java
java RemoteControl

You should see a display with the image viewer in the center, remote controls 
for the robot on the left, and camera and color controls of the right. 

Move the robot about with the remote controls. Flip the camera up and down,
as necessary, to see what you need to see. When the robot gets near a red 
object, it will move towards it and attempt to push it forwards. Click on 
the little Media Properties control at the bottom right of the ImageViewer 
and choose the Plug-In Settings, to control color sensitivity and the 
proportion of the region that needs to be filled with the specified color. 
Edit the RemoteControl.java program and change the color if you need to.

Speech control examples:

See the Readme file in the visin\examples\speech directory.

