jRuby on NXT Lejos

[pEhrlich]

Hi

In the past few years, I've spent a lot of time learning Ruby (and Rails), and have a lot to like about the language. Recently I've been thinking over what it would be like to get this running on the NXT brick. There is a ruby-nxt project, but it hasn't been touched in years, and is missing the fantastic & mature tools you guys have been building!

This would be exciting for a few reasons-- Ruby has a nice, non-verbose, user-friendly syntax, great for beginners and those new to programming. It has gained immense popularity in the web world recently with Rails, and there is a lot of talent out there. And most finally, that community has a strong culture of code with an emphasis on speed, developer happiness, and documentation.

jRuby is a Java implementation of Ruby. It can be compiled just-in-time or ahead-of-time in to java bytecode. It can call java methods directly, as well as the reverse. The jruby jar file is required in order for java to run the class files (http://kenai.com/projects/jruby/pages/JRubyCompiler)

Ruby is a highly meta language, making large use of method names to enable dynamic programming. (For example, a `method_missing` method is available, which receives the requested method's name). As per this thread: http://lejos.sourceforge.net/forum/viewtopic.php?f=7&t=225&start=45#p13581, it appears that some of these features may not be easily made available.

What do you guys think? Interesting? Doable? Easy?

for more info, the jRuby wiki isin't bad: http://en.wikipedia.org/wiki/JRuby. Also, the ruby site: http://www.ruby-lang.org/en/

[skoehler]

Use JRuby to create *.class files. Then try to use the linker to link those classfiles into an NXJ file. It will probably fail saying that this or that method is not available. Depending on the nature of the methods, we will or will not be able to implement them. leJOS does not support reflection or dynamic loading of bytecode. So if such code is referenced even though you're compiling ruby to bytecode ahead of time, then there's no way to use it with leJOS.

[pEhrlich]

Simple hello world script, taken from the tutorial:

Code: Select all

java_import 'lejos.nxt.Button'

class Index
  def main(args)
    System.out.println "Hello World from Ruby!"
     Button.waitForAnyPress()
  end
end


Error on linking:

Code: Select all

$ ruby -v
jruby 1.6.7.2 (ruby-1.9.2-p312) (2012-05-01 26e08ba) (Java HotSpot(TM) 64-Bit Server VM 1.6.0_35) [darwin-x86_64-java]

$ jrubyc Index.rb
$ nxj Index
leJOS NXJ> Linking...
js.tinyvm.TinyVMException: Class org.jruby.ast.executable.AbstractScript (file org/jruby/ast/executable/AbstractScript.class) not found in CLASSPATH /Users/peter/Downloads/leJOS_NXJ_0.9.1beta-3/bin/../lib/nxt/classes.jar:.
   at js.tinyvm.ClassRecord.getClassRecord(ClassRecord.java:855)
   at js.tinyvm.ClassRecord.storeReferredClasses(ClassRecord.java:394)
   at js.tinyvm.Binary.processClasses(Binary.java:381)
   at js.tinyvm.Binary.createFromClosureOf(Binary.java:309)
   at js.tinyvm.TinyVMTool.link(TinyVMTool.java:97)
   at js.tinyvm.TinyVMTool.link(TinyVMTool.java:48)
   at lejos.pc.tools.NXJLink.start(NXJLink.java:134)
   at lejos.pc.tools.NXJLinkAndUpload.run(NXJLinkAndUpload.java:108)
   at lejos.pc.tools.NXJLinkAndUpload.start(NXJLinkAndUpload.java:49)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at lejos.pc.tools.ToolStarter.startTool(ToolStarter.java:31)
   at lejos.pc.tools.NXJLinkAndUpload.main(NXJLinkAndUpload.java:42)


I'm not really sure what AbstractScript is for, or what the important bits are. Here's the doc and source, if they're any help:
http://jruby.org/apidocs/org/jruby/ast/ ... cript.html
https://github.com/jruby/jruby/blob/mas ... cript.java


To make sure I fully understand why this error is coming up: Linking is when lower level dependencies for running a program are included in to its executable (in this case, an .nxj file). This problem could then be approached from a couple of different angles:

- Consider AbstractScript, and other potential problem files in the future, missing from TinyVM (which I understand has been developed for LeJos). This would mean implementing these classes, presumably in locations such as jtools/src/js/tinyvm/AbstractScript.java. As these would only be required by jRuby software, these would not affect other programs. Alternatively, the sources paths could be specified manually while linking. In this way, any JRuby-specific features could be implemented.

- Consider AbstractScript (and future files) unnecessary. Perhaps they're trying to do something in a roundabout way, or something that is irrelevant from the perspective of a Lego brick. This I know less about-- neither what TinyVM has and avoids, or JRuby requires. Any thoughts/ideas in this groove would be exciting.

Thoughts on all this?

Thanks!

--Peter

[skoehler]

I'm sorry, but this is a very basic problem. Every JVM (also leJOS) needs to know where to find class files. When Oracle JVM would tell you that it cannot find the class AbstractScript if you would not tell it where to find it.
The folders and JAR files that are searched by the JVM are usually called "the classpath". I assume, that you need to to add some jruby specific JAR file to the classpath when you call nxj or nxjlink. That's what nxj's and nxjlink's -classpath parameter is for. Use it.

The classpath you're currently using is part of the error message you posted. It is "/Users/peter/Downloads/leJOS_NXJ_0.9.1beta-3/bin/../lib/nxt/classes.jar:." and clearly only included classes.jar and the folder ".". Obviously, AbstractScript cannot be found in that classpath.

To make sure I fully understand why this error is coming up: Linking is when lower level dependencies for running a program are included in to its executable (in this case, an .nxj file). This problem could then be approached from a couple of different angles:

That is actually correct. The nxj file includes fragments of all the class files that are needed to run the program, i.e. it included all dependencies.

- Consider AbstractScript, and other potential problem files in the future, missing from TinyVM (which I understand has been developed for LeJos). This would mean implementing these classes, presumably in locations such as jtools/src/js/tinyvm/AbstractScript.java. As these would only be required by jRuby software, these would not affect other programs. Alternatively, the sources paths could be specified manually while linking. In this way, any JRuby-specific features could be implemented.

Reimplementing parts of the JRuby runtime might be possible - or it might not be possible, since any implementation would have to be based on Java reflection which is not available on the NXT. However, the location you suggest it wrong. The code must not go into the jtools project, but into the classes project or even a completely seperate project.
Also, the question is how stable the API is.

- Consider AbstractScript (and future files) unnecessary. Perhaps they're trying to do something in a roundabout way, or something that is irrelevant from the perspective of a Lego brick. This I know less about-- neither what TinyVM has and avoids, or JRuby requires. Any thoughts/ideas in this groove would be exciting.

It sound like AbstractScript is a dependency of the bytecode that JRuby generates. I believe, you'd have to modify the jruby compiler in order to get rid of that dependency.

[skoehler]

After adding jruby.jar to the classpath, the linker fails with:

Code: Select all

$ nxjlink -o test.nxj -cp .:../lib/jruby.jar  test
js.tinyvm.TinyVMException: Class java.lang.ClassLoader (file java/lang/ClassLoader.class) not found in CLASSPATH /opt/lejos_nxt/bin/../lib/nxt/classes.jar:.:../lib/jruby.jar
   at js.tinyvm.ClassRecord.getClassRecord(ClassRecord.java:855)
   at js.tinyvm.ClassRecord.storeReferredClasses(ClassRecord.java:394)
   at js.tinyvm.Binary.processClasses(Binary.java:381)
   at js.tinyvm.Binary.createFromClosureOf(Binary.java:309)
   at js.tinyvm.TinyVMTool.link(TinyVMTool.java:97)
   at js.tinyvm.TinyVMTool.link(TinyVMTool.java:48)
   at lejos.pc.tools.NXJLink.start(NXJLink.java:134)
   at lejos.pc.tools.NXJLink.run(NXJLink.java:101)
   at lejos.pc.tools.NXJLink.start(NXJLink.java:33)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:601)
   at lejos.pc.tools.ToolStarter.startTool(ToolStarter.java:31)
   at lejos.pc.tools.NXJLink.main(NXJLink.java:28)

As you can see, it now cannot find the class ClassLoader. This might point towards a dependency on Java relection. As I said above, Java reflection is not available on the NXT.

[skoehler]

Let's take a look at the main method generated by jrubyc:

Code: Select all

  public static void main(java.lang.String[]);
    Code:
       0: new           #2                  // class test
       3: dup           
       4: invokespecial #182                // Method "<init>":()V
       7: dup           
       8: ldc           #2                  // class test
      10: invokevirtual #188                // Method java/lang/Class.getClassLoader:()Ljava/lang/ClassLoader;
      13: ldc           #190                // String test.class
      15: invokevirtual #196                // Method java/lang/ClassLoader.getResource:(Ljava/lang/String;)Ljava/net/URL;
      18: invokevirtual #202                // Method java/lang/Object.toString:()Ljava/lang/String;
      21: astore_1     
      22: aload_1       
      23: invokevirtual #206                // Method org/jruby/ast/executable/AbstractScript.setFilename:(Ljava/lang/String;)V
      26: new           #208                // class org/jruby/RubyInstanceConfig
      29: dup           
      30: invokespecial #209                // Method org/jruby/RubyInstanceConfig."<init>":()V
      33: dup           
      34: aload_0       
      35: invokevirtual #212                // Method org/jruby/RubyInstanceConfig.setArgv:([Ljava/lang/String;)V
      38: dup           
      39: aload_1       
      40: invokevirtual #215                // Method org/jruby/RubyInstanceConfig.setScriptFileName:(Ljava/lang/String;)V
      43: invokestatic  #221                // Method org/jruby/Ruby.newInstance:(Lorg/jruby/RubyInstanceConfig;)Lorg/jruby/Ruby;
      46: dup           
      47: invokevirtual #225                // Method org/jruby/Ruby.getCurrentContext:()Lorg/jruby/runtime/ThreadContext;
      50: swap         
      51: invokevirtual #229                // Method org/jruby/Ruby.getTopSelf:()Lorg/jruby/runtime/builtin/IRubyObject;
      54: ldc           #112                // int 0
      56: invokevirtual #231                // Method load:(Lorg/jruby/runtime/ThreadContext;Lorg/jruby/runtime/builtin/IRubyObject;Z)Lorg/jruby/runtime/builtin/IRubyObject;
      59: return

As you can see, first the URL of test.class is determined and then passed on to AbstractScript.setFilename. I bet, that JRuby then loads test.class via reflection, invokes methods via reflections etc.

JRuby is definitely not leJOS compatible. I'm not sure how far the use of reflection can be eliminated. But you will certainly have to mess with JRuby's runtime and compiler before it is leJOS compatible.

[skoehler]

Here's some code from the JRuby wiki:

Code: Select all

  require 'java'
  Sys = java.lang.System
  version = Sys.getProperties["java.runtime.version"]

I consider this proof, that JRuby uses reflection for method invocations. In the example, "getProperties" is invoked via reflection as which class is stored inside Sys might just as well be the result of a computation. JRuby won't work with leJOS. (unless it is used for PC-side programs)

[pEhrlich]

Alright, I've done a bit more research from the JRuby end of things. It looks like this challenge shares some similarity to bringing JRuby to J2ME (java micro edition): for example, both omit reflection. As luck would have it, some work has been done here, but unfortunately just enough to show what would need to be done.

Apparently there are two possible java specs, CDC and CLDC. CLDC is the more basic, with essential File io and not a whole lot else, whereas CDC is the more useful. In 2007, Roy Ben Hayun did some work on discovering what would be required to put this together. This is based on very old versions on JRuby (0.8.2, as opposed to the current: 1.7)

http://www.slideshare.net/pehrlich/intr ... -to-jrubme
https://github.com/jruby/jruby/wiki/JRu ... croEdition
http://www.coderanch.com/t/272622/java/ ... Reflection


In 2009, this project was revitalized in a blog post by Charles Nutter, one of the JRuby core team members. This is off a fork of JRuby 0.9.8.

CDC is probably about the smallest ME profile JRuby could reasonably run in, since anything lower (like CLDC, the Connected Limited Device Configuration) and you're reduced to the most basic collections, IO, and utility APIs.

http://blog.headius.com/2009/02/jruby-on-java-me.html
https://github.com/headius/jruby-cdc
https://github.com/headius/jruby-cdc/commits/master

Seeing this, my question is how close LeJos is to the Java ME-CDC spec (http://en.wikipedia.org/wiki/Connected_ ... figuration). If it is close, I might ask some of the JRuby team members about the status of the project, and if there's any lingering interest or updates. Perhaps a more universal solution would be applicable in other situations, and get demand & support accordingly.



There was some other interesting reading as well:

http://kenai.com/projects/jruby/pages/JRubyCompiler

Direct Invocation, Binding Into the MOP, and Reflection

Only class bodies, rescue/ensure bodies, and chained top-level script methods get directly invoked during script execution. The others are bound into the MetaObject Protocol (MOP) at runtime.

Binding occurs in one of two ways:
- By generating a small stub class that implements DynamicMethod and invokes the target method on the target script directly
- By doing the same with reflection

Finally, how did you get the printout of java bytecode in your previous post? Here's what I got:

Code: Select all

$ javap -c Index.rb
ERROR:Could not find Index.rb

I did find an online class decompiler, which resulted in this code. (Which more or less matches the description given in the JRubyCompiler link above)

Code: Select all

// Mobilefish.com Online Java Class Decompiler
// Author: Robert Lie
// Version: 0.1 (pre-alpha)
// http://www.mobilefish.com/services/java_decompiler/java_decompiler.php
// JVM version: 50.0
// File: Index.rb

import java.net.URL;
import org.jcodings.Encoding;
import org.jruby.Ruby;
import org.jruby.RubyClass;
import org.jruby.RubyInstanceConfig;
import org.jruby.RubyModule;
import org.jruby.RubyString;
import org.jruby.ast.executable.AbstractScript;
import org.jruby.ast.executable.RuntimeCache;
import org.jruby.internal.runtime.methods.CallConfiguration;
import org.jruby.javasupport.util.RuntimeHelpers;
import org.jruby.parser.StaticScope;
import org.jruby.runtime.Arity;
import org.jruby.runtime.Block;
import org.jruby.runtime.CallSite;
import org.jruby.runtime.ThreadContext;
import org.jruby.runtime.builtin.IRubyObject;

public  class Index extends AbstractScript {


    private static void setPosition(ThreadContext, int)
    public Index ()
    public static IRubyObject __file__(Index, ThreadContext, IRubyObject, IRubyObject[], Block)
    public IRubyObject __file__(ThreadContext, IRubyObject, IRubyObject[], Block)
    public static IRubyObject class_0$RUBY$Index(Index, ThreadContext, IRubyObject, Block)
    public static IRubyObject method__1$RUBY$main(Index, ThreadContext, IRubyObject, IRubyObject, Block)
    public static IRubyObject method__1$RUBY$main(Index, ThreadContext, IRubyObject, IRubyObject[], Block)
    public static IRubyObject class_0$RUBY$Index(Index, ThreadContext, IRubyObject, IRubyObject[], Block)
    public IRubyObject load(ThreadContext, IRubyObject, boolean)
    public static void main(String[])

}

I guess its not valid Java though, because I tried then compiling this with nxjc, and got about 20 errors like this:

Code: Select all

index.java:29: <identifier> expected
    private static void setPosition(ThreadContext, int)

Thoughts on this?

Cheers!
--Peter

[skoehler]

Code: Select all

$ javap -c Index.rb
ERROR:Could not find Index.rb

It should work like this:

Code: Select all

jrubyc Index.rb
javap -c -private Index

I still believe, that JRuby can't work without reflection. I'm really not sure how they were planning to make it work in J2ME. But let's start with a simple question for you:
In your example it says: java_import 'lejos.nxt.Button'

Is java_import a declaration or a statement? I.e. is it possible to implement the following pseudo-code in ruby?

Code: Select all

seconds = seconds of current time.
if (seconds is odd) java_import 'packageA.TestClass'
else java_import 'packageB.TestClass'

TestClass.println 'HelloWorld';


[pEhrlich]

Just a quick update here:

@pepijndevos has suggested a piece of software called Mirah, which allows ruby-esque syntax on the NXT with Lejos.

https://twitter.com/pepijndevos/status/ ... 1378678786
http://studl.es/2012/05/mirah-on-nxt/

Will be giving this a go when time is available!

[skoehler]

I assume, that mirah has some dependencies on the Java language runtime - e.g. for emulating associative arrays (if the language supports it).
After reading a little about mirah, I believe the main work of supporting mirah will be to extend the leJOS Java runtime (i.e. classes.jar) with all that is needed by mirah. You should ask the mirah developers about a list of classes that they expect to exist in the Java runtime.

I'm happy to confirm that "mirahc --bootclasspath /path/to/lejos/lib/nxt/classes.jar" is the correct command to compile mirah code for leJOS.