Q-learning Framework and leJOS

[losmani]

I just downloaded q-learning framework which is a Java implementation of the Q-Con model presented by Lin in 1993. I was trying to modify it and adapt it for leJOS NXJ 0.5.

Does anyone know on how to correctly modify it so that it works with leJOS NXJ 0.5 and how to add it in a project so that the the nxjc can compile it? If anyone knows, please let me know. Also, if anyone has already modified the thing and used it with leJOS, could you give me an example on implementing let's say an obstacle avoidance behavior by using two sensors and two motors?

Thanks in advance.

This framework can be downloaded from www.elsy.gdan.pl

[esmetaman]

I have ported the code.

You can download QCON for leJOS here:

http://www.juanantonio.info/jab_cms.php?id=216

[Dirtreverse]

After toying with the Connectionist code (http://elsy.gdan.pl/) I thought it would be great fun to create a physical robot and observe it learning some tasks.
As a professional Java programmer and Lego aficionado, finding out about Lejos was like

So here I am, I bought the Mindstorms 2.0 set, some extra sensors and holonomic wheels. I figured doing the math for driving the holonomic wheels wasn't my cup of tea so I ended up with a simple differential-drive bot. (later I discovered there is a Pilot for holonomic bots in the Lejos library that does the math... but I'm not rebuilding AGAIN so soon now )

Anyway, I've tried running this q-learning 'port' yesterday... but it didn't work.

Firstly, the Brain class has a constructor with too many parameters.
As I wasn't planning using that one, I've deleted it. (the constructor allows for extensive tweaking of the algorithm, now the constructors use the available parameter defaults)

Secondly, and this is a showstopper, my program will compile and upload, but upon running there's an out of memory exception. I didn't check out where exactly this happens, I still have to learn how to debug , but I did find out that exception 3 is supposed to be OutOfMemoryException.

So my question is, has anyone actually succesfully run this Q learning code on a NXT brick? Finding a non-compiling Brain class in the library classes I started to wonder about this.
I did see some Q learning Mindstorm robots on youtube. (maybe remotely controlled, that could be it)

I didn't add too much of my own code; just 5 actions to perform (albeit all actions have their own small class), a simple implementation of Perception...
Not doing anything wild. 22 classes in total.
The only thing I "feel" which might be the memory problem are the 3 dimensional arrays with doubles in them, in the imported Brain class.

Does anyone have any thoughts on this?

[gloomyandy]

Hi,
sorry can't help directly with your question, but I would like to point out that the previous posts on this thread are 4 years old, so it may be that any port is very out of date... Certainly over those four years leJOS has changed a lot...

Andy

[Dirtreverse]

Hi, yeah, I know, old thread, but I read the forum rules and it was adviced to use the search function before asking... I figured why start a new thread if there's already one on the subject

[gloomyandy]

No problem, just wasn't sure if you had seen the date. Good luck with your project....

[Dirtreverse]

Thanks, Andy!

I guess nobody has succesfully used this port then

Hmmm. This is a huge impediment.

[TechnoX]

Try to debug your code more.

If you get a exception you get the exception code (3 in this case), but also more number whom indicate where the exception where thrown.

If you compile your code with the verbose option all methods will get a unique number so you can box the problem!
Because you're a professional java programmer I think you soon will discover the error and make a new Q-learning API!