NXTCam image capture

[gloomyandy]

Hi.
Sorry to bother you again but why can NxtCamView capture images within 4 seconds?
How can it read all the lines so fast?

Well for starters it is using USB not i2c to transfer the image. Then assuming your timings come from the standard firmware it is not running in the same resolution. The default image resolution is 176x144, I use the highest resolution available from the camera which is 352*288 so there is 4 times as much data to transfer. Finally the protocol for USB is much cleaner. The generic pic firmware supplied by Mindsensors which you have to use when using custom commands requires a lot more handshaking when using i2c (you basically have to poll the pic using i2c commands, and clear the buffer when you have finished the transfer).

why can't you use 75 bytes at a time instead of 60 bytes? (then you only have 5 sections instead of 6 ones)

I actually transfer 64 bytes of data at a time (there are 4 bytes of wrapper information around the actual frame data), which fits exactly into two 32byte i2c transactions (the maximum i2c transaction supported by leJOS). So to transfer a line requires 12 i2c transactions. If I used the full 75 bytes that would require 3 i2c transactions (two of 323bytes and one of 11 bytes) which would give a total of 15 i2c operations. So it is a swings and roundabouts thing really. I tested both and the 60 byte option was faster.

Do you mean changing the transfer order of the lines with "interleaving"?
Why is it so tricky?

Well let me know when you have it working....

Things are actually much more complex than your description. Firstly there is a longer gap between frames than between lines (the vsync period), then you have the various delays in the transfer process, and given that we have 3 processors talking asynchronously to each other it is hard to get the timing right. Finally if you get it wrong then the cost is quite high as you end up having to wait for an entire frame to get the requested line. It can be made to work but the results are not as good as your calculations would indicate. Also remember that the lines have to be processed, there is not really enough memory in the nxt to store the data, so you typically have to process them as you read them (even if you are simply sending the data on to a PC), and guess what, this also takes time...

I'm sure it can be made faster, but I've not really had the need to do this. ...

But feel free to try it for yourself, and let me know how you get on...

Andy

[skoehler]

Hi.
Sorry to bother you again but why can NxtCamView capture images within 4 seconds?
How can it read all the lines so fast?

so each line has to be read in sections (in my case I use 60bytes at a time), thus the 352byte line requires 6 transactions.

why can't you use 75 bytes at a time instead of 60 bytes? (then you only have 5 sections instead of 6 ones)

I believe the internal firmware buffer used for I2C transactions is only 60 bytes.

[gloomyandy]

Hi Sven,
I'm not sure which buffer you are referring to here? Can you clarify which firmware (AVR, PIC or AT91!) you are referring to?

Andy

[skoehler]

Hi Sven,
I'm not sure which buffer you are referring to here? Can you clarify which firmware (AVR, PIC or AT91!) you are referring to?

The one in the leJOS firmware. I think, we currently limit I2C transactions to 60 bytes, don't we?

[gloomyandy]

Hi Michael,
I'm not sure if you had seen my reply before you edited my post. But as I tried to explain above, coming up with an interleave algorithm is not the issue. The problem is that the various delays are not consistent particularly as you will probably need to process the data on the NXT on a line by line basis. I'm not sure what it is you intend to use the NXTCam for, but to be honest I think the transfer time is likely to be the least of the problems. Good luck with your project though, let me know how you get on...

Andy

[gloomyandy]

Hi Sven,
The current leJOS firmware limit is 32 bytes...

Andy

[tevanox]

Hi, I'm kinda noob in general with the minstorm and I have a few problems.
I have followed the tutorials that mindsensors.com had and I can't change my nxtcam avr firmware. The problem is that i cant find this files:
nxtcam-settings.rxe
nxtcam_gen.hex
nxtcam_def.hex

Can someone provide me the files?

[gloomyandy]

You should contact mindsensors and ask them for the files. That way you can be sure they match the version of the NXTCam that you have...

[tevanox]

Thanks Andy!

One more question. If I want to transfer the Image, do I have to modify the i2c classes to do it? or its possible with some methods?

[gloomyandy]

To do things like image capture you will need to use the generic version of the Mindsensors i2c interface which runs and must be downloaded to the PIC processor that is part of the NXTCam. This has a different interface to that used by the standard NxtCam interface and so will need a new leJOS sensor driver. This driver is written in Java like all of the other leJOS sensor drivers and will use the standard i2c interface methods. If you are not comfortable with any of this then I would not recommend you proceed. Replacing the firmware in the NXTCam PIC processor is a pretty advanced thing to be doing and if you get any of this wrong you may end up with a none functional NXTCam.

Good luck

Andy

[urobot]

Hi Andy,
need some help for a project for a thesis.
The project is a RoboChecker, a robot that plays checkers.
We did get the maincode to determine the move via MiniMax function.
Now we are wondering on image processing to get the coordinates of the pieces on the board to pass to the maincode.
Is a similar task to that of your Lego Mindstorms NXT Playing Connect 4 Using Image Processing.
A help configuring and programming the system will be greatly appreciated.
If of interest we may start a dedicated post to RoboChecker.
Saluti da Firenze,
Ugo

[gloomyandy]

Hi,
I think you will struggle to do this with an NXTCam. At best you may be able to identify which squares on the board have black pieces and which have white, but even then this will probably require that you have a board which has say red and blue squares rather than a standard one.

As I have said before, reprogramming the firmware on the NXTCam is very tricky, have you any experience of working in this environment? If not I suggest that you follow the tutorials on the Mindsensor site to become familiar with how you can work with the NXTCam.

Andy

[urobot]

Actually, I do not think the colour of the squares would make any difference.
Supposing the squares are red and blue, do you think nxtcam may be programmed so that it can recognize black&white pieces on the squares (either all red or all blue) and build a matrix with the coordinates of the black&white pieces (12 black +12 white) to pass to the minimax solver?
At the moment, I cannot find an answer in the tutorial I saw on Mindsensor site .
The alternative is to use a fixed cam and a PC with image processing to build the matrix of the position of the pieces to be passed to the main on the nxt brick, which would be connected to the PC.
Ugo

[gloomyandy]

The color of the squares will make a huge difference trying to identify if a black square is occupied by a black chess piece will be very hard to do! The mindsensors site will not tell you how to do exactly what it is you want to do, that will be down to you. It will give you a starting point on how to re-program the firmware of the NXTCam. But as I've said before this is a pretty complex thing to do especially if you re not familiar with programming embedded devices.

[robotnik]

Hi Andy,

I have been checking your NXTCam code, could you please also share the code of the program to receive the image in the computer?

You are supoused to receive this when requiring snapshots, right?

Byte 0: 0x0B – indicating the start of raw color data line
Byte 1: line number in hex, indicating which line number is being sent
Byte 2-178: 176 hex pixel stacks, where a pixel stack is structured as follows:
Each pixel stack consists of a one-byte value, where the high nibble represents one color value (green), and the low nibble represents another color value (either red or blue). The pixels on the camera are arranged in Bayer format.
High nibble: Green pixel value of the even rows
Low nibble: Red or Blue pixel value of the odd rows
Byte 179: 0x0F – indicating an end of line

Thanks in advance