Relative performance of leJOS?

[genom2]

Hello,
As far as I understand leJOS doesn't use JIT or HotSpot techniques. So the leJOS VM is a pure interpreter implementation of the JAVA VM. So far I don't have a NXT. But I'm curious to know how fast leJOS is compared to a native program, for example coded in C (here in contrast to NQC). In this forum I found something like this about performance:

Code: Select all

 200000 int add:                         648 ms     308641 ops/sec
 200000 int sub:                         662 ms     302114 ops/sec
 200000 int mul:                         680 ms     294117 ops/sec
 200000 int div:                         941 ms     212539 ops/sec
 200000 float add:                       871 ms     229621 ops/sec
 200000 float sub:                       879 ms     227531 ops/sec
 200000 float mul:                      1028 ms     194552 ops/sec
 200000 float div:                      1868 ms     107066 ops/sec
 200000 double add:                     1723 ms     116076 ops/sec
 200000 double sub:                     1790 ms     111731 ops/sec
 200000 double mul:                     1678 ms     119189 ops/sec
 200000 double div:                     6090 ms      32840 ops/sec


What would be the results if programmed in C? Will C be 10, 50 or 100 times faster? Any idea?

Or, if I put my question into another simple program (which uses 'invokevirtual' only rarely) like:

Code: Select all

public class Fibonacci32Bit {
  static int fib(int n) {
    if(n < 2)
      return 1;
    else
      return fib(n-2) + fib(n-1);
  }

  public static void main(String[] args) {
    long t0 = System.currentTimeMillis();
    for(int i=0;i<=30;i++)
      System.out.println("Fibonacci of "+i+" is "+fib(i)+" in "+(System.currentTimeMillis()-t0)+" ms");
  }
}

What will be the results (the elapsed times) for leJOS and results for C on the NXT?

Thanks!

[gloomyandy]

Hi,
Benchmarking is a very tricky subject. C (compiled using something like gcc for the ARM chip) will be considerably faster than Java/leJOS (after all that is what the leJOS interpreter is written in), however NQC (which uses an interpreter) is probably slower (not compared things recently). For simple operations like integer arithmetic the gains will be considerable, however for floating point operations the difference will be considerably less (the NXT does not have a floating point unit), for more complex operations like method calls things will be different again (remember that there are many different types of method calls in Java, some considerably more complex than the seemingly similar C equivalent).

But raw performance is only one item to take into account. Things like program size, ease of use, debugging, richness of the run time library etc. are all important factors. In particular on the NXT having garbage collection and a relatively safe mechanism for handling memory allocation and manipulation is likely to make life much easier than doing all of that using pointers/malloc etc. in C (remember that the ARM chip used in the NXT has no memory management, and a stray pointer/memory corruption can be very hard to debug). When you get access to an NXT why not spend some time comparing the two (or three if you include both C and NXC), they all of very different programming environments and performance. See which one you like best and which will give you the results you need in the time you have available...

Andy

[genom2]

Thank you gloomyandy for your fast answer.

I don't have access to NXT now. This is why I asked.

Things like program size, ease of use, debugging, richness of the run time library etc. are all important factors. In particular on the NXT having garbage collection and a relatively safe mechanism for handling memory allocation and manipulation is likely to make life much easier than doing all of that using pointers/malloc etc. in C (remember that the ARM chip used in the NXT has no memory management, and a stray pointer/memory corruption can be very hard to debug).

This is why I love JAVA and will love leJOS. Is anybody else is able to answer my question (in numbers)? Or, at least give me a first impression/estimate?

Genom2

[gloomyandy]

What alternative to Java do you want to compare against? Native C using gcc, RobotC which is interpreted, or NXC which is also interpreted. I don't think NXC will handle the recursion used by your test program... Out of interest what is it that you plan to do? Is there a reason why speed is so important?

Andy

[bullestock]

This page compares different languages for the NXT. In May I sent some data for LeJOS to the maintainer; he has not yet updated the page, but what I sent back then was

The loop counts are:

With display 54177
Without display 70543

I think you can safely put a "Yes" in the "Write 'fast' programs" column.

As for size, the compiled file is 6256 bytes.

Note that this was with 0.85.

[gloomyandy]

Hi,
That page is very out of date for all of the languages and the test program is a little odd and open to dispute (turns out that there are many interpretations of what reading a sensor means, do you wait for a new reading or not...).

Anyway it turns out that RobotC like NXC does not support recursion so the fib test is not looking like a good test case. But I ran it anyway in Java the total time taken for up to and including fib 30 is 197566, the same code in gcc based C takes 3432ms, so the native C code in this instance is 57 times faster.

Since I'd installed the RobotC compiler I thought I should probably put together a quick test program that did not use recursion, but which was primarily a test of method calls (like the fib test). Here is what I came up with:

Code: Select all

import lejos.nxt.Button;


public class Test
{

    static int test(int n1, int n2, int n3, int n4) {
        if(n1 < 50000)
          return n1*n2 + n3*n4;
        else
          return n1/n2 + n3/n4;
      }

     public static void main(String[] args) {
        long t0 = System.currentTimeMillis();
        int res = 0;
        for(int i=0;i<=100000;i++)
            res = test(i, i/2, res, res/2);
        System.out.println("result "+res+" in "+(System.currentTimeMillis()-t0)+" ms");
        Button.waitForAnyPress();
      }
}


It turns out that RobotC ints are 16bit not 32 so the RobotC version of the above uses long rather than int. The above program takes 4512ms with leJOS 13221ms with RobotC and with gcc C it takes 288ms. So in this test leJOS is 3 times faster than RobotC, and gcc C is 16 times faster than leJOS...

Make what you will of the above results I don't think they are very surprising, as I said benchmarking is a tricky thing... Oh and neither the fib test nor my test is a very good benchmark!

Andy

[genom2]

Thank you bullestock for your answer. But your numbers are really confusing. (Ok, gloomyandy is right when he says 'Benchmarking is a very tricky subject'.) For instance, I see no row 'loop counts' in your given page. Do you mean row 'Speed (loops/min)'? If so, does this mean leJOS has 54177/70543 (no display/display) and gcc has 2695 ? So 2695 is surprisingly slow for gcc? Looks to me like a mistake.

@gloomyandy
Yes, that is what I'm looking for. Thank you. (BTW a factor of 50 is what I expected. But I agree, it's a naive benchmark.)

OT: Because you asked. I plan to start with a simple line follower. And leJOS will be - without any doubt - a good tool for this. Later I will try to enhance the line follower by doing something else in parallel while following the line. This, by using JAVA/leJOS threading.

Genom2

[genom2]

Until now I'm using an Arduino Duemilanove for my robot projects. It's an ATmega328P running with 16Mhz having 32K flash and 2k RAM. I program it with WinAVR which is a gcc for AVRs.

The following might be of interest for you:
- The fib 30 code in gcc based C takes 19494ms with compiler option -Os (space). (And 16825ms with option -O3 (time).)
- Your test delivers 'result 4 in 8469 ms' for option -Os. (Which is over proportional slow.)

BTW, in WinAVR an int is 16Bit, so I had to use a long which is 32Bit.

Genom2

[bullestock]

Thank you bullestock for your answer. But your numbers are really confusing. (Ok, gloomyandy is right when he says 'Benchmarking is a very tricky subject'.) For instance, I see no row 'loop counts' in your given page. Do you mean row 'Speed (loops/min)'? If so, does this mean leJOS has 54177/70543 (no display/display) and gcc has 2695 ? So 2695 is surprisingly slow for gcc? Looks to me like a mistake.

Yes, that is what I meant. I have no idea where the 2695 comes from, but note that the 'optimized' value for LEJOS OSEK is 334k/18144k - the highest value of any of the listed languages.

[sioux]

I don't know if there is any native c api for the nxt. I think they all are interpreted anyway.
I haven't benchmarked lejOs but i run fuzzy controllers and neural networks that have neurons on the nxt
and need communication between the pc and nxt and the performance is higher than I had expected.

I can confirm to you that I can run these programs with quite a high sampling rate of the sensors. Depends on what
you want to do though. Me, I can't do my work without object oriented programs so lejos for me is ideal.
Keep in mind though that RobotC and NXC are not native, as mentioned earlier, but interpreted.

Overall, having tried 3-4 api's out there I beleive lejos, apart from being object oriented, is generally
more functional in basic tasks with quite good performance. I regard it to be a good all-around api.
in your java programs. But again it all depends on the complexity of your work. Manipulating matrices
and running algorithms can be a pain in the neck with c and you have to waste much time to do too little stuff.
I am not sure if native C on a robot will be any use at all to be honest so from that aspect, its performance
is relative. You can use JNI to hard code custom hardware if you need to.