Data Logging with leJOS on the NXT

[PantsOnToast]

One thing that I haven't see so far (although it may exist already) in leJOS is the ability to log data and store it in a file. Today I started a project to do just that. As of now my program will write data to a file .txt file on the NXT (placed there with the nxjbrowser because it doesn't create one by default). It uses a TimerListener and can collect data from pretty much any sensor, kind of annoying to change sensor types at the moment, but I'll work on that. You can also vary the increment of time in which data is logged. The time in which it was logged is also logged.

I hope to eventually be able to take this data and be able to easily put it into an excel graph. It's one feature, the only feature, I miss about Robolab because it is great for finding a problem with the robot's behavior, and can be used in an educational setting.

If anyone is interested in this, or if you have any questions, or concerns please let me know, I'll be working on this for the next few days.

[imaqine]

Sounds great! Whenever a release is available it would be nice if you can post a link on here to download.

Also have you thought about having the text file be a CSV file instead? It would allow easy parsing into Excel. Or of course it could be a text file with comma seperated values.

Is there an ability to just write to the log using a function? The reason I say this is because it can easily be used as a debugging tool where when an error happens it writes to the .log file that can later be pulled up on the computer and analyzed. I think this would be awesome and a step in the direction of remote debugging. Errors could exist when a user calls the function to write something to the .log or when an actually error has occured in NXJ. Just an idea.

Another use of writing to a text file is to pass parameters from one program to another one the NXJ. I suppose this could be useful when calling programs from bluetooth? Or instead of sending each command/parameter back to a computer via bluetooth one at a time, a program would want to write all the parameters found to a text file then send all the parameters at once via the text file to the computer.

You could even use the Navigator class to pinpoint x,y of the land the robot has traveled then once transversing the land is complete, send the full set of coordinates back to the computer to be analyzed. You should be able to create a chart to visualize where the robot has been. This information can also be used to remember a maze setup and transverse it faster,etc.

I suppose the ideas are endless.

[PantsOnToast]

How I’ve set up the class is that there are different constructors that deal with the different data collection devices, so basically the user creates an instance according to what data device they want to use.

It might be a little early for this, but I was wondering if it would be at all possible to include some of my code within the motor and sensor port classes in the next release so that you can simply say Motor.A.startDataLogging(); or something like that. I am not sure how that would work exactly because my class imports from util, nxt, navigation, and io packages. I don’t know what the criteria for including such a thing into the existing classes is, so maybe someone can comment on that, if it is at all possible. This is something I’m interested in learning about.

I was able to get a lot done today. It now records motor and sensor data which is separated by a comma. It is easily imported into excel for very impressive looking graphs. I also added support for the navigation classes. It records the x and y value, but unfortunately the current version of leJOS only updates the coordinates when the robot is stopped. I did make the NXT drive in a square and stop at the corners and it worked, graphing the corners of a square. I have also added options for greater user control of when data starts and stops being collected.

As far as using this sort of technique for error logging, it would work; it can output text as well. I am just not 100% sure how it would be integrated easily; it would be rather clumsy right now as a separate class. It also makes sense that this would work as a tool for two programs to share data, I have not tried reading data, but I see no reason why it would not work. It can easily be changed so that there is no time limit, the user must terminate it themselves.

I’m going to work some more on it tomorrow and check again for any errors I missed and see what else I can add. I’ll post the source either late in the workday tomorrow or next Monday.

The possibilities really are endless!

[PantsOnToast]

Due to some non-lejos related issues, I am not quite ready to post my code for the data logger, should be ready tomorrow or wednesday!

[imaqine]

Nice! - A couple of days isn't much : )

[PantsOnToast]

I quickly got some free file hosting website to take care of this while I get my own hosting space set up, so here is the link (should work!).

http://www.mediafire.com/download.php?1nvimmy0i1u

There are a lot of comments about how to use it, and each method's general purpose is explained. I didn't comment on the inner-workings of the methods yet, if you have any questions just ask me.

I pretty much redesigned the whole thing today, instead of writting data as it comes in, I stored it in an array, then at the end write to the file. This regulated the writing of data and allows multiple sensors, motors, or navigators to be logged because you cannot use two output streams at once.

Bascially you create an instance of the class, as parameters give a filename, how often you want data collected in milliseconds, how much data you want (use a number less than 1 if you don't want to limit the amount), and whatever the data is comming from (sensorPort, motor etc.). Then call the startDataCollection method, do whatever you want, then call the writeDataToFile method to end and record data.

If you have any questions or concerns please let me know, there is a lot of included information. If you're a developer reading this, I would like to know if this, or any adaptation of this would be able to be useful in the next release, I would like to develop this further, and become a developer for this.


Thanks!

[imaqine]

I took a look at the comments and code and so far it looks very good! I didn't get a chance to load it onto the NXT because I do not have my NXT at the moment. I actually do not think I have any suggestions at the moment. I like how it is written to an array instead of directly to a file. A demo.java would be nice to show how it is used. Do you have anymore plans to this project?

Wait a couple of days for the developers to take a look at it and hopefully it will be added soon into a release.

[PantsOnToast]

Here is a little demo that will ues a light sensor and log data every 250 milliseconds for ten seconds (40 data points). You can change the 40 to a zero to use a non-timed mode, just remember the while loop will never exit in that case and you'll have to use a different approach.

Code: Select all

//-----------------------------------------------------------------------------------

import java.io.IOException;
import lejos.nxt.*;

public class Main {

   public static void main(String[] args) throws IOException {
      LightSensor ls = new LightSensor(SensorPort.S2);

      DataLoggerNXT lightLogger = new DataLoggerNXT("lightData.txt",250,40,SensorPort.S2);
      lightLogger.startDataCollection();

      while (lightLogger.isCollectingData()) {
//in reality this loop would be whatever behavior the robot is doing, all the loop does here is keep the program running until it's ready to quit
      }

      lightLogger.writeDataToFile();
      }
   
}
//-----------------------------------------------------------------------------------


As far as some other ideas for this project, I implemented the compass sensor and the other ones would be easy as well. I might add the ability to log battery voltage. It can also be made to log data on a button press rather than time, really anything that has a listener can be used.

Reading data shouldn't be too hard, you could specify coordinates in a text file and have the robot run a program that moves to those spots or something like that which might be a nice feature for users that aren't that technical.

The code really is very adaptable to other purposes if anyone has any specific ideas or requests.

[imaqine]

What happens if you record data from sensor port 1 and 2 both to sensors.txt and you write data to file for both of them. How would the data write? Would it write from port 1 first then write port 2 in the same file without erasing the original data? From what it sounds like it would erase whatever is written to sensors.txt first.

I won't have my NXT for awhile so I am just asking questions that I can think of might have issues.

[PantsOnToast]

You're right, as it was if you specified the name of a file that existed, the old file would be overwritten. I fixed that today so that if the file already exists you can simply write to that file so that all your data from one program can be neatly put into one file. There is also a clearFile method I added that deletes the file associated with the instance of the data logger it is called on, so if you're testing you can simply clear your test file each time you create a data logger. I also added some headings for the data that display what kind of data is being shown and how often the data was collected.


http://www.mediafire.com/download.php?4bhcwecczxy

Enjoy!

[inm8]

Thanks a lot for this I'm currently building a "lego explorer" which will be able to map and dimension regular sized rooms. Your work on data logging is going to really help me with co-ordinate logging.

The overall plan is to have a 2d map of the room generated as a TGA file, although I haven't decided whether I will have the map generated on the NXT or a local client yet.

[esmetaman]

Hi, I have tested your file and I would like to read this file. Do you have any tool to read txt files

[PantsOnToast]

Your idea sounds pretty cool inm8, but just remember that there are limitations with the coordinate navigator, it only collects data when the robot is stopped. That is just the way leJOS is implemented so far, not something I can change in my code. Good luck with that project, I hope it works out.

As far as esmetaman's question, a .txt file will open in notepad if you're using windows, if you have a Java design tool (ex. Eclipse) you can open the .java file with it as well. Should be pretty easy.

I haven't been, and won't be able to check the forum often, so if you have an urgent question just email me and I'll try to get back to you as soon as possible.

[PantsOnToast]

Hey, I'm back.

I am considering using the Data Logger as part of a paper. I was wondering if there was anyone who has used my software and could let me know what they think?

I am probably going to have to update it for the newest version on leJOS, and I would like to know if the developers would consider including it in a future release in some form?

I hope there is still some interest in the topic!

[venice]

hi there.
i did it another way to log information, since i didn't need to show anything on NXJ-display. the idea is to print anything, you want to log onto lcd. this "line-logger" is scrollable and lcd is only refreshed, if new lines are appended.
You can print text or objects (which results in printing object.toString). Given text is cutted into pieces of length 16 automatically to fit in lcd. As you can see, there is much space for optimization. You can scroll by calling Next() and Prev(). If you set alwaysShowLastEntry to true, then logger will always scroll to last line, if something is added (this last feature could be buggy - i can't remember if i fixed it!).
Feel free to use and yes feel free to change anything in code. i don't care
if there are questions feel free to post a message. i have been to lazy to comment code. sorry.

Code: Select all

package de.jot.teachin.client.logger;

import java.util.ArrayList;
import java.util.List;
import lejos.nxt.LCD;

/**
 * Clientside Logger. Prints log-text to LCD of NXJ. Log-text
 * can have any length. Following options are possible:
 * out()      : Prints text
 * out()              : Prints object by calling object.toString()
 * info()      : Prints text with "INFO:"-prefix
 * warning()   : Prints text with "WARN:"-prefix
 * error()      : Prints text with "ERR :"-prefix
 * seperator()   : Prints seperator "-------------"
 *
 * @author jot
 *
 */

public class LineLogger {
   
   private static LineLogger instance;
   private List<String> linesList;
   
   private int index;
   private boolean needsDrawUpdate;
   private boolean alwaysShowLastEntry;
   
   private final String SEPERATOR = "----------------";
   
   
   private LineLogger(){
      linesList = new ArrayList<String>();
      index = 0;
      needsDrawUpdate = true;
      alwaysShowLastEntry = false;
      out("LineLogger 0.1");
      out("by j.o.tepper");
      seperator();
   }
   
   public static LineLogger getInstance(){
      if (instance == null){
         instance = new LineLogger();
      }
      return instance;
   }
   
   public void out(String string){
      int stringIndex = 0;
      int stringLength = string.length();
      while(stringLength - stringIndex > 16){
         linesList.add(string.substring(stringIndex, stringIndex + 16));
         stringIndex += 16;
      }
      linesList.add(string.substring(stringIndex, string.length()));
      if (alwaysShowLastEntry){
         if (linesList.size() > 9){
            index = linesList.size() - 9; // 8 + 1 because index starts at '0'
         }
         needsDrawUpdate = true;
      }
   }

   public void out(Object object){
      out(object.toString());
   }
   
   public void info(String string){
      out("INFO: " + string);
   }
   
   public void error(String string){
      out("ERR : " + string);
   }
   
   public void warning(String string){
      out("WARN: " + string);
   }
   
   public void seperator(){
      out(SEPERATOR);
   }
   
   public void next(){
      if (index < linesList.size() - 8){
      index++;
      needsDrawUpdate = true;
      }
   }
   
   public void prev(){
      if (index > 0){
         index--;
         needsDrawUpdate = true;
      }
   }
   
   public void draw(){
      if (needsDrawUpdate){
         LCD.clear();
         for (int offset = index; offset < index + 8; offset++){
         System.out.println(getLine(offset));
         }
         LCD.refresh();
         needsDrawUpdate = false;
      }
   }
   
   private String getLine(int index){
      if (index >= linesList.size()){
         return "";
      }
      return linesList.get(index);
   }
}