Mindsensors NXTCam and Lejos

[wayneage]

Hi,
Has anyone tried to use the mindsensors nxt cam with lejos? I'm just starting my final year project and the idea is to play a game of pong with an NXT, using the camera. Does anyone know the response time of the camera also?

Thanks

[bbagnall]

I haven't tried it yet, but we would need to develop some code to expose the camera functionality to leJOS NXJ. I assume the camera is an I2C sensor? If that's the case, it's pretty easy to make new classes by extending the I2CSensor class.

If you get a camera and play around with this, please send us your code and we can incorporate it into leJOS NXJ.

[esmetaman]

Hi, I am developing a class to manage NXTCam, but I need support from mindsensors and I have not received new news.

read this thread:
viewtopic.php?t=559

[bbagnall]

Has anyone tried to use the mindsensors nxt cam with lejos? I'm just starting my final year project and the idea is to play a game of pong with an NXT, using the camera. Does anyone know the response time of the camera also?

Lawrie added a NXTCam class to the lejos.nxt package. It hasn't really been tested yet but theoretically it should work. We'll be awaiting on some feedback after the next release from anyone who owns this sensor.

[lawrie]

I have tested the NXTCam class. There is a NXTCamTest sample in SVN.

It works but the camera seems to be much more sensitive to red than other colors, and you have to be very careful with lighting, background color and object color to get useful tracking of objects from the camera.

The NXTCam class probably needs more methods adding to - I will look forward to feedback when the new relesase is out.

[wayneage]

Hey guys,
Sorry it's been so long since I replyed, I haven't really gotten into the project too much, because
1) I had my exams and
2) The backlog in cameras means the college only recieved the order yesterday.... so anyway I have the camera now and will be testing and implementing it over the next few weeks so I will continue to post.

Wayne

[wayneage]

Ok I've been working away with the camera. The class there now works fine, don't see what extra methods could be added really?
Except for maybe one that could to withing some degree decide that a bunch of close objects is actually just the one object, but then if you had two objects of same colour you'd have problems.

Anyway I've rewritten the cam_display class to display the rectangles seen on the nxt display:

Code: Select all



import java.awt.Rectangle;

import javax.microedition.lcdui.Graphics;

import lejos.nxt.Button;
import lejos.nxt.LCD;
import lejos.nxt.NXTCam;
import lejos.nxt.SensorPort;

/**
 * Class to display the rectangles that the camera picks up on the NXT screen
 * @author wayneage
 *
 */
public class Cam_Display
{
   
   // NXT Display is 100 x 64
   // Camera view is 176 x 144
   
   private static int INTERVAL=200;
   private static int CAM_WIDTH=176;
   private static int CAM_HEIGHT=144;
   
   private static int xscale(int x)
   {
      return Math.round((x/(float)CAM_WIDTH)*LCD.SCREEN_WIDTH);
   }


   private static int yscale(int y)
   {
      return  Math.round((y/(float)CAM_HEIGHT)*LCD.SCREEN_HEIGHT);
   }
   
   public static void main(String [] args)
   {
      try{
         NXTCam camera = new NXTCam(SensorPort.S1);
         
         camera.sendCommand('A');
         camera.sendCommand('E');
         int objectCount;
         
         Graphics g = new Graphics();
         
         while(!Button.ESCAPE.isPressed()){
            g.clear();
            
            objectCount=camera.getNumberOfObjects();
             if(objectCount>0 && objectCount<8){
                for(int i =0; i<objectCount; i++){
                   Rectangle rect = camera.getRectangle(i);

                   g.drawRect(xscale(rect.x),yscale(rect.y),
                            xscale(rect.width),yscale(rect.height));                  
                  
                }
             }
             Thread.sleep(INTERVAL);
         }
      }catch(Exception ex){
         System.exit(0);
      }
   }
}




[lawrie]

Good to hear you are using the NXTCam class successfully. I developed it, but did not spend much time on it, so I thought it might need more methods adding when people started to use it.

I have not tried to do any significant projects with the NXTCam, but on the small amount of experimentation that I did, I did not think the object recognition was very good. The distinguishing of colors did not seem very accurate - nearly everything seemed to be a shade of red.

Have you managed to do object tracking with it? It looked as if you would need to control the lighting and background and the color of the object being tracked quite carefully to get useful results.

[wayneage]

Yes I was, infact I have a simple class working that gets the robot to follow a ball as it moves from left to right.

I have noticed the red shade thing once or twice, I assume you have installed NXTCamView? I don't know if we can do much about the shading/lighting code wise, unless we actually modify the cameras firmware, which is entirely another project itself! I saw something about illumination as one of the commands to the camera, but haven't tested it.
I've noticed the shades of red when there is a light directly in the background, but for the most part the colours can be distinct enough, if you don't have a major source of light shining into the cameras view.
Every time I use the camera in a different environment and NXT I have to reconfigure the colour map through NXTCamView, due to different lighting.

It is indeed easier to track red it would seem, but the reason for that from what I see is when you try to track blue/green, shadowy bits of the image from the background can be picked up as dark blue/green.

[lawrie]

It is good to hear you can track objects effectively. Yes, I did use NxtCamView and itr showed most things as shades of red.

Would a method to set the color map be useful of do you need to use NxtCamView to do that effectively?

[wayneage]

Sorry about the delay, been a crazy week in college!

Ya I was considering that, but I can't think how you might use it to adjust to the colours to match what you want though, the only thing I can think is you could have somepreset colour maps stored in a program and then change them at will with the NXT, so say if you wanted to track some shade of blue that you recorded earlier.
Other than that I don't know would it be possible to upload a useful colour map?

[esmetaman]

It is a good opinion. It would be very important to know how NXTCamView send data to NXTCam to update colormaps.

I think that a good way is study source code of NXTCamview