USB stream performance

[matejdro]

I have been playing around with usb connection and I want to make to make program that will stream image to NXT display. This is what I came up with:

PC side:

Code: Select all

NXTConnector connector = new NXTConnector();
      connector.connectTo("usb://NXT");      
      
      DataOutputStream output = new DataOutputStream(connector.getOutputStream());
      
      Boolean black = true;
      
      while (true)
      {
         
         try {               
            
            for (int x = 0; x < 100; x++)
            {
               for (int y = 0; y < 64; y++)
               {
                  
                  output.writeByte(x);
                  output.writeByte(y);
                  output.writeBoolean(black);
                  output.flush();

               }
            }

            
            black = !black;

            

         } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
         }

      }

NXT Side:

Code: Select all

USBConnection connection = USB.waitForConnection();
         
         DataInputStream input = connection.openDataInputStream();

         while (true)
         {
            
            try {
               LCD.setPixel(input.readByte(), input.readByte(), input.readBoolean() ? 1 : 0);
               
            } catch (IOException e) {
               return;
            }
                     
            
         }

It's supposed to first set all pixels black, then all pixels white, then again all pixels black etc.

Ideally screen would be flickering. But everything is happening very slowly. Screen needs around 6 seconds to fully refresh (from left to right).

So there are 6400 pixels in display, each pixel needs 3 bytes (I know this could be done much more efficiently, but I feel it should still work much faster even with 3 bytes per pixel) and it needs around 6 seconds to update whole screen. So 6400 * 3 / 6 = 3200Bps.

Is that really maximum speed NXT can handle? It feels very slow for USB connection.

[skoehler]

What you are experiencing is the lack of buffering. You would experience the same, if you would write to a file using FileOutputStream, or similar.

On the PC-side use:
DataOutputStream output = new DataOutputStream(new BufferedOutputStream(connector.getOutputStream(), 1024));

And on the NXT-side use
DataInputStream input = new DataInputStream(new BufferedInputStream(connection.openInputStream(), 1024));

That should improve things.

[matejdro]

Adding buffering and moving flush() outside for loops nearly tripled the performance.

But that still means 2 seconds per frame or 9600Bps, which still sounds very slow for USB.

[gloomyandy]

With all performance problems you first need to identify where the bottleneck is. It could be...
1. The USB connection
2. The way you are doing I/O
3. The way you are updating the screen.
So try and test each item individually. First I would try just using local data to update the screen... How long does that take?
Next try writing the data as an array of bytes using the lowest level I/O routines you can, on the NXT use the NXTConnection read and write routines. Change just one side at a time, start with the NXT.

[skoehler]

some rules:
- flush as few as possible
- read/write as large chunks of data as possible (BufferedInput/OutputStream does that for you, unless you flush it too often)

Also, as gloomyandy noticed, modifying the LCD output inside an I/O loop will slow things down.

[matejdro]

By reading/writing large chunks of data you mean this between flushing?

I have timed LCD output without USB and it takes around 300ms, while with USB it takes around 2sec. Will test I/O today.

[skoehler]

By reading/writing large chunks of data you mean this between flushing?

Yes, of course.

If you flush after writing 3 bytes, you will essentially send a 3 byte chunk over the USB connection.
If you flush after writing 1066 bytes, you will send a chunk of 1024 bytes (buffered output stream runs full and flushes automatically after 1024 bytes) plus a chunk of 42 bytes.
You can also increase the buffer size of the BufferedOutputStream.

[gloomyandy]

Hi,
a few other things to think about. With your code at the moment the inner loop contains a call to setPixel, 2 calls to readByte and one call to readBoolean, each call to readByte and readBoolean makes a call to readByte0, which then makes a call to inputStream read. so this means that for each iteration of your loop you will have...
1xsetPixel
2xreadByte
1xreadBoolean
3xreadBytes0
3xread
which gives 10 method calls per loop or 64000 method calls in total.

I don't have any very recent figures but the last time I checked leJOS could execute approx 63500 method calls per second. So in this case over 1 second of the time may well be spent simply executing method calls, you will then have to add in the actual code within those method calls, but I can see how a substantial part of the time may be nothing to do with the USB speed....

Personally if I wanted to send full screen images to the nxt I would consider using the native image format which packs 8 pixels per byte. This gives a total of only 6400/8 = 800 bytes instead of the 19200 you are currently sending. These bytes can be read into an array using a relatively small number of read operations and then displayed on the screen using a single bitBlt operation. I think you will find this much faster, at least for full screen updates...

Andy

[matthias]

Using the Lejos Image format will not slow down the sending of data.
Even, when in one image-byte only one pixel changes its still a "byte" that is send.
so sending out chunks of image-bytes might be the better option, than sending single pixels (even, if you dont need it)

Also dont forget, that you can easily ddos your NXT to "program crash" with your pc.

[matejdro]

This is what I came up for sending images in lejos format:

Code: Select all

      BufferedImage image = new BufferedImage(100, 64, BufferedImage.TYPE_BYTE_BINARY );
      
      Graphics g = image.getGraphics();
      g.setColor(Color.BLACK);
      g.fillRect(0, 0, 100, 64);
      
      byte[] byti = convertImage(image);
      
      NXTConnector connector = new NXTConnector();
      if (!connector.connectTo("usb://NXT"))
      {
         System.out.println("Conn Error!");
         return;      
      }

      DataOutputStream output = new DataOutputStream(new BufferedOutputStream(connector.getOutputStream(), 1024));
      
      try {               
         System.out.println("Writing...");

         output.write(byti);
         System.out.println("Flushing...");

         output.flush();
         System.out.println("End!");

      } catch (IOException e) {
         // TODO Auto-generated catch block
         e.printStackTrace();
      }

(PC Side, convertImage is copied from image converter source)

But it will freeze at output.flush() line

[gloomyandy]

If it is blocking on a call to flush then this probably means you are not reading all of the data on the NXT. What does your NXT code look like?

Andy

[matejdro]

This is NXT side

Code: Select all

          System.out.println("Waiting...");

       USBConnection connection = USB.waitForConnection();
       
        DataInputStream input = new DataInputStream(new BufferedInputStream(connection.openInputStream(), 1024));

       
       byte[] byti = null;
       try {
          
       System.out.println("Connected!");
          
       while (input.available() < 800)
          Thread.yield(); 
       System.out.println("Reading!");

      input.read(byti, 0, 800);
      
      LCD.bitBlt(byti, 100, 64, 0, 0, 0, 0, 100, 64, LCD.ROP_COPY);


[skoehler]

The loop, where you check available() and call Thread.yield() is wasting precious CPU time. Just starting reading data. Also, I'm not sure if that loop ever exits. You simply assume, that the USB buffer on the NXT side can hold up to 800 bytes. But what if that is not the case? Yes, the buffer if the BufferedInputStream is 1024 bytes large. But that doesn't mean that available() can reach values up to 1024. Actually, available() is not guaranteed to return anything other than zero. If it returns something larger than zero, you're just lucky.

Also, please study the documentation of InputStream.read(). Because, like in many languages, a read call _tries_ to read as much as you tell it to read, but it is allowed to read less than you specify. That's why InputStream.read(...) returns the amout of data it has actually read.
You may use readFully() instead of read(). readFully() has the behaviour you desire.

Code: Select all

          System.out.println("Waiting...");
       USBConnection connection = USB.waitForConnection();
        DataInputStream input = new DataInputStream(new BufferedInputStream(connection.openInputStream(), 1024));

       byte[] byti = new byte[800];
      input.readFully(byti, 0, 800);
      LCD.bitBlt(byti, 100, 64, 0, 0, 0, 0, 100, 64, LCD.ROP_COPY);

[gloomyandy]

Well a few things you should check/change...
1. Have you checked how big the output is from the convert call? Are you sure it is just the data for the screen image with no headers etc?
2. It is not a good idea to have that while loop testing available. The input buffers on the NXT will almost certainly not be large enough to hold that many bytes. You should probably remove it and use the readFully method instead to read all of the data bytes...
3. You will need to allocate the array to read the bytes into, the read call will not create the array for you...

[matejdro]

Thanks, it worked!

There is little documentation about readFully(), so I had no idea what it does and how is different from read(). I think it would be a good idea to add that to javadocs.