Video Player

[matejdro]

This is little experiment, I was working on:

http://www.youtube.com/watch?v=_9IpoJ1llUQ

It's working pretty good except for annoying audio pops. I can eliminate it by lowering audio sending delay, but then audio is playing much faster and going out of sync with video.

Of course you can't see much of the video, except for titles, but I guess if someone would made monochrome movie, it would work well.

[TechnoX]

Really nice!

[gloomyandy]

Hi,
That is pretty cool. To fix the audio rather than have a delay in the audio, you may want to send fewer video frames (so drop video frames), this is the way that sync is normally performed for video playback, the audio runs in "real time" and the video is kept in sync, by dropping frames in need be.It is usually less unpleasant to have a dropped video frame than the pops and clicks you get with gaps in the audio...

All the best

Andy

[matejdro]

If I run audio in real time, it plays really fast, much faster than it should be.

I have converted audio to 8000Hz, so I need to send 8000 samples every second. So I send 255 samples every 8000 / 255 milliseconds.

[gloomyandy]

Hi,
That does not sound right. Can you post your NXT side audio playback code....

Andy

[matejdro]

Code: Select all

   byte[] audio = new byte[255];

       while (true)
       {
           byte mode = input.readByte();
           if (mode == 0)
           {
              input.readFully(byti, 0, 700);
              LCD.bitBlt(byti, 100, 56, 0, 0, 0, 0, 100, 56, LCD.ROP_COPY);
              LCD.refresh();


           }
           else if (mode == 1)
           {
              input.readFully(audio, 0, 255);
              
              Sound.playSample(audio, 0, 255, 8000, 50);
           }
           
       }

First, I send 1 then I send 255 bytes of audio data.

[gloomyandy]

Hi,
You need to check the return value from playSample it returns the number of bytes written to the playback queue if all of the bytes have not been written you may need to wait and try writing them again. Given that the playback speed is known you can easily estimate how long you need to wait for there to be sufficient space for the new write to succeed. You will be losing samples which is why the audio is playing faster than it should be...

Andy

[matejdro]

I was printing return value and it appears to always be 255.

[gloomyandy]

Have you tried it when sending the audio at full speed? I can't see any other way that the audio can play too fast other than you are dropping packets at some point and this seems the most likely place for that to be happening....

Andy

[gloomyandy]

Oh and one other thing how are you printing out the value? I'm not sure you will be able to see the none 255 values very easily as it will be printing them very fast. May be better to add code that prints out if the value is ever not 255, or better still count the number of times it isn't 255 and print out the counter...

[matejdro]

Yeah, I was thinking about that. But usually it at least flickers when value changes, but screen was "frozen" at 255s.

[gloomyandy]

Well up to you but I still think it is worth checking. I would be amazed if it always returns 255 as to keep things playing smoothly you really need to keep the buffer full which almost certainly means you will get none 255 values. Getting a 255 return basically means that the playback buffer was empty when you queued the new data, which to me says that it is likely that you are not writing the data fast enough to get continuous playback (hence all of the popping), if when writing data continuously and handling the none 255 returns correctly the playback speed is still wrong that that means that either there is a bug in the firmware, or the sample rates being used do not match the rate specified at playback time (I assume the sample is a mono sample)...

Andy

[mattallen37]

Very impressive!

How are you streaming the data? USB? Can you please share your code?

[matejdro]

That number actually goes under 255 rarely. However, it does not seem to be causing popping (value is less than 255 like once every 10 seconds, but it always pops). If I'm sending bytes just little faster, it will still pop, but it will start to play faster than normal. I need to speed it up quite considerably to remove popping.

This is how I determined frame rate on PC:

Code: Select all

audioS = AudioSystem.getAudioInputStream(fileName);
System.out.println(audioS.getFormat().getFrameRate());

Above returns 8000, so I use the same in sample rate on NXT:

Code: Select all

Sound.playSample(audio, 0, 255, 8000, 50);

Yes, all communication is through USB. I will release source code, but first I want to fix it or at least clean it up (currently it's full of commented experimental stuff).

[skoehler]

That number actually goes under 255 rarely. However, it does not seem to be causing popping (value is less than 255 like once every 10 seconds, but it always pops).

That's because 255 is actually a bad thing. If you get a lot of 255, then that means that you feed the samples just as fast as the samples are consumed. Or in other words: there are never more than 255 or 256 samples in the buffer. In fact, each time the chunk of 255 samples is lightly late, you will hear a pop.

Make sure, that you feed samples into the sample buffer at a rate faster than they are consumed. You will notice when that the buffer is full, when a value of less than 255 is returned.
Also, when you stop ignoring the return value of the playSamples method, then there will not be any loss of samples, which also causes a pop.

Regards,
Sven