Sunday, October 11, 2009

Video Streaming with the Android Phone

This page describes how to stream video to the Google Developer Phone. This topic is a very interesting, but also quite complex one, and it took me a while to figure all variables out in a proper way.

If you follow this walkthrough you should be able to install a video streaming server, adapt any of your video content and stream the result to your Android Phone. It also contains encoding settings that you might find useful to encode your videos to watch them (locally)on the G-Phone.
Hope this is useful to someone out there.

Table of Contents
  1. Video Streaming with the Google Phone
    1. Setting up a Streaming Media Server
      1. Installing Perl
      2. Installing Darwin Streaming Server
        1. Installation on Windows
        2. Installation on Ubuntu Linux
      3. Administrator Console
        1. Starting
        2. Stopping
      4. Administering Darwin Streaming Server
      5. Media Repository
      6. Firewalls and Darwin Streaming Server
    2. Creating Streaming Media for the Google Developer Phone
      1. Encoding a Movie
        1. Getting the Right Dimensions
      2. Hinting the Result
      3. Deploying the Media
      4. Testing the Streaming

Video Streaming with the Google Phone

Android comes with the ability to play back video and audio that is stored locally on the device. More important though, Android is also able to receive streaming media such as video and audio with its Media Player API. In this tutorial we show what you need to
  • set up a streaming media server
  • create streaming media compatible with the Google Phone's available codecs
  • Receive the streaming media on the phone

Setting up a Streaming Media Server

There are a variety of Streaming Media Servers out there, but the one that we think is the easiest to use is Apple's Darwin Streaming Server (DSS). Darwin Streaming Server is Open Source and its commercial brother is Apple's Quicktime Streaming Server (QSS). Darwin can be downloaded from here: http://dss.macosforge.org/. You can click on the previous releases link (http://dss.macosforge.org/post/previous-releases/) if you want to get a binary version and not go to the hassle of building Darwin Yourself. In this Tutorial we used DSS v5.5.
Installing Perl
Once you downloaded Darwin you need to install it on your machine. This requires that you have Perl installed. A working version of Perl (for Windows) can be found on the Strawberry Perl site: http://strawberryperl.com/. Download the Strawberry Perl and install it. Once you have it installed, open a command line and type the following command:
> perl -MCPAN -e shell

The CPAN shell opens and there you type:

CPAN> install Win32::Process

This will make Perl go to the Perl package network (the CPAN to be precise) and download the Win32::Process package needed by Darwin. If the downloading has succeeded, you can go on to installing Darwin Streaming Server.

Installing Darwin Streaming Server
Installation on Windows
Unpack the downloaded Darwin to a local folder and execute install.bat. If you are running the install.bat under Windows Vista, make sure you run it in administrator mode. You can do that by right-clicking on the file and select Start as Administrator from the context menu or you can start a command line shell in admin mode by doing the following:
  1. Press the Start button in your task bar
  2. type 'cmd' in the search bar
  3. press Shift+Ctrl+Enter
This will start the console in administrator mode. You can then browse to the folder containing install.bat and execute it from command line.
Installation on Ubuntu Linux
Installation on Ubuntu Linux is straight-forward. All you need to do is download the tar.gz file with the binaries (for this tutorial we used Darwin 5.5.5 with the binaries given here: http://dss.macosforge.org/downloads/DarwinStreamingSrvr5.5.5-Linux.tar.gz) to your Linux machine, unzip and untar it e.g. like this:

$> gunzip DarwinStreamingSrvr5.5.5-Linux.tar.gz
$> tar xf DarwinStreamingSrvr5.5.5-Linux.tar

The result is an untared folder, which contains several executable scripts, one of them being called Install. Before you can execute Install you need to modify the script a bit. Open the script (e.g. with vim) and search for the following part:

useradd -M qtss

Once you have found it, replace it by

useradd -m qtss

Note the lowercase -m option. Safe the file and then from the shell execute

$> ./Install

You might have to be root to be allowed to do that.
Administrator Console
If the installation of Darwin is successful, you will be prompted for entering a user name and a password. Those are used by the Administrator Console of Darwin and need to be remembered well.
Starting
The admin console is started right after installation, so if you close the installation window, it will be terminated. To start it again, open a command shell, browse to the installation folder of Darwin (on Windows this is C:\Program Files\Darwin Streaming Server) and type

C:\Program Files\Darwin Streaming Server> perl streamingadminserver.pl

This will start the admin console server and you can access it from your browser under http://localhost:1220.
Stopping
To stop the admin console simply terminate the Perl script. Note: Terminating the admin server console does not terminate the streaming server. Darwin is installed as a Windows service and as such can only be stopped via the Windows Services view (right-click on My Computer-->Manage-->Services and Applications-->Services).
Administering Darwin Streaming Server
Via the admin console server you can administer Darwin. The first time you will access the console, you will be asked to enter an MP3 administrator's password. You will be asked for that password whenever you want to upload a new MP3 to Darwin, so remember it well. The admin console is pretty self-explanatory, so take a look around and see what you can do.
Media Repository
Whatever media you want to stream with Darwin has to be placed inside the Movies folder of your Darwin installation. On Windows, that is C:\Program Files\Darwin Streaming Server\Movies. See the description on Media Creation below, to see what pre-requisites your media has to fulfil.
Firewalls and Darwin Streaming Server
Darwin is an RTSP server and by default uses the regular RTSP port 554 for RTSP both over TCP and UDP. That port may be blocked by the firewall you installed Darwin on. To make sure the ports are open, you need to open those ports in your firewall. If you don't know how to do it, ask your friendly administrator.

Important: If you fail to have the proper ports open, you will not receive any streaming on the phone.


Note: In the admin console server, you can choose to enable streaming over port 80, which means that RTSP requests will be tunnelled through port 80 which is usually open in firewalls. For RTSP requests from the phone / external client to your Darwin Streaming Server machine, this means that you have to use an RTSP URL including the port to use, e.g. rtsp://yourserver:80/moviename.mp4.

Creating Streaming Media for the Google Developer Phone

Creating streaming media depends on two things:
  1. the streaming server you are using
  2. the capabilities of the client, i.e. the phone
In our case, we have decided for the Darwin Streaming Server as streaming server solution and the Android Dev Phone 1 as the client.

Both the capabilities of the server and the phone are important as they decide which media formats will work and which wont.

Darwin Streaming Server supports a variety of containers, among them .3gp, .mp4 and .mov. It is more or less agnostic to the codecs used to encode the media but needs to be hinted how to stream a media file. The Dev Phone on the other hand has specifications that indicate which container types and codecs are supported for which media. A list of specifications can be found here http://developer.android.com/guide/appendix/media-formats.html.
Essentially this page explains that Android currently supports the following Video Formats:































CodecEncoderDecoderContainer Format
H.263yesyes3GPP (.3gp) and MPEG-4 (.mp4)
H.264 AVCnoyes3GPP (.3gp) and MPEG-4 (.mp4)
MPEG-4 SPnoyes3GPP (.3gp)


This limits our choice of codecs to H.263, H.264 and MPEG-4 Simple Profile. However, even if you know this it is difficult enough to create properly encoded videos that also show up in a suitable quality on the phone. In the next section we will give an overview of settings and tools that can be used to create suitable media.
Encoding a Movie
To make a movie suitable for playback and / or streaming on the phone, we need to encode it with the right codecs, and place the encoded movie into the right container format.
A tool that is invaluable for that task is SUPER which you can download from here: http://erightsoft.podzone.net/GetFile.php?SUPERsetup.exe

Once installed, the tool lets you add Media files to a list of files to encode and then specify the encoding settings for the resulting media file. To add a media file for encoding, right-click in the lower part of the SUPER window and select Add Multimedia File(s).
Browse to a folder with the movie you want to stream to the phone and select it. Now it is time to tune the settings to get the properly encoded result. A very good list of settings can be found here: http://androidcommunity.com/forums/vbglossar.php?do=showentry&catid=2&id=28

I copied the contents here, just in case the link will disappear one day:

























Codecs:MP4 H.264 AAC LC
Video:480:270 3:2 23.976 480kbps (yes, you set the video to 480x270)
Audio:44100 2 96kbps Default
Options:Hi Quality:OFF Top Quality:OFF
Pad:Top:24 Bottom:26 Left:0 Right:0
Crop:Top:0 Bottom:0 Left:Calc Right:Calc

While everything else stays the same from movie to movie, the Calc options above need to be calculated depending on the resolution of the video source.
I'll quickly give the calculation and then give an example.

The calculation is ( Width - ( Height * 16 / 9 ) ) / 2. Width and Height are of the source video and the result would be what you would select for the left and right crop. Here's a couple examples from DVD rips: * 886x480 source video: ( 886 - ( 480 * 16 / 9 ) ) / 2 = 16 (set the left and right crop to 16) * 1128x480 source video: ( 1128 - ( 480 * 16 / 9 ) ) / 2 = 137 (set the left crop to 136 and the right crop to 138) * 852x480 source video: ( 852 - ( 480 * 16 / 9 ) ) / 2 = -0.7 (this movie is already 16x9 so you can turn Crop OFF).
After you have made the settings, click the Encode (Active Files) button. This will create a video output encoded with the settings you specified. You can verify the quality, with VLC player for example.

Important Note:
The settings above may be too much for streaming to the phone. It could be noticed that there is considerable packet loss on the device, probably due to buffer overflows on the device's video player. The settings above are better for local playback on the phone. For streaming use the following settings for a movie in 16:9 format:





































Codecs:MP4 H.264 AAC LC
Video:256:144 3:2 23.976 480kbps
Audio:44100 2 96kbps Default
Options:Hi Quality:OFF Top Quality:OFF
Pad:Top:12 Bottom:14 Left:0 Right:0
Crop:off

As you can see the video size has been reduced to roughly half the size of the settings above. This reduces the bandwidth considerably. If you still experience packet loss you can further reduce the width and height. But pay attention since the dimensions (at least the width) have to be a multiple of 16, i.e. width modulo 16 must yield 0. Otherwise Super will fail to encode. This is not a bug, but a constraint of the codec / container format.
Getting the Right Dimensions
When you change the size of the video, the following things need to be considered:
  1. display size of the phone
  2. aspect ratio of the original video
  3. aspect ratio of the phone's display
  4. encoder constraints
The Android phone's display size is 480x320 which yields an aspect ratio of 3:2(= 480:320). So 3:2 is the aspect ratio we want the encoded video to be at finally (like this the phone's player does not have to do expensive rescaling).

Let's now assume we have a video of size 1024x576 which has an aspect ratio of 16:9 (= 1024:576). To get that movie on a 3:2 screen, we can either stretch / shrink it - adding unwanted distortion to the movie - or add padding to the movie. Let's further assume we want to end up with a movie that has a width of 256. For it to still be in 16:9, we need to set the height to 144, which is 256 / 16 * 9.

We will then end up with a movie of size 256x144. The problem is, that this is 16:9 and we want 3:2 so that the player on the phone does not have to do rescaling. If our movie width is 256 and the movie were in 3:2, then the height of the movie would have to be 170 (= 256 / 3 * 2).

However, it is only 144 since we don't want to distort the movie. So the solution is to pad the remaining height with black pixels (the black stripes you know from TV) so that we reach the 3:2 height. Thus we need to pad 26 (= 170 - 144) pixels, and we do that by padding 12 on the top and 14 on the bottom (12 + 14 = 26). The 2 pixel difference is not visible.

After encoding, we end up with a movie that is actually 3:2 with size 256x170 but "contains" the downsized original in 16:9 format. If you understood that you can reproduce it with any size and aspect ratio you like.

Note: keep in mind the restrictions of the codec. Width % 16 must be 0.
Hinting the Result
After having created the properly encoded video file, we are only one step away from streaming it to the phone. In order to be able to do so, Darwin Streaming Server needs to get hints how to stream media it is not able to decode.

As mentioned above, Darwin is agnostic to the codec used by the file that is streamed which makes it easier to support new codec types as they are being created. In order to stream the encoded media properly, however, Darwin needs to process hints which are a separate track inside the media file's container. Without those hints, Darwin will not be able to stream your media - whether properly encoded or not.

To create a hinted media file, you need a program called MP4Box, which you can download binary versions of from here:

http://www.tkn.tu-berlin.de/research/evalvid/

Windows: http://www.tkn.tu-berlin.de/research/evalvid/EvalVid/mp4box-w32.zip
Linux: http://www.tkn.tu-berlin.de/research/evalvid/EvalVid/mp4box-lin.tar.bz2
Mac: http://www.tkn.tu-berlin.de/research/evalvid/EvalVid/mp4box-mac.tar.bz2

Once downloaded you can add it to your system's PATH or simply put it into the same folder as the media file you want to add hints to. To hint the movie we have created in the encoding step, open a command line shell, browse to the folder containing the encoded video and the MP4Box executable then run

mp4box -hint <mediafile>.mp4

When the program terminates successfully, our encoded media file is hinted and ready for deployment to Darwin.
Deploying the Media
To make our video available for streaming, all we need to do is copy the file to Darwin's Movies folder, under Windows this is C:\Program Files\Darwin Streaming Server\Movies.
Testing the Streaming
To test if streaming of your new file is working, you can use VLC player. Start VLC player and select File-->Open Network Stream. In the dialog enter RTSP as the protocol and the url rtsp://localhost/<moviefile>.mp4 and press OK.
The video should start playing back.

Important: Depending on the Version of VLC you either have to type the full RTSP URL or only the URL omitting the protocol prefix, i.e. localhost/movie.mp4 instead of rtsp://localhost/movie.mp4.


If testing with VLC worked fine, you should be able to access the video from the Media Player on the Android Phone as well. The easiest way to do that is to create a Web page with a link to the RTSP URL of the movie you are streaming, i.e. somehting like this:

<a href=””>rtsp://yourserver/yourmediafile.mp4</a>

You can then access this page from your Android Phone’s browser and click on the link. This will start the internal Android Media Player and playback the video.

Of course, you can also create your own Media Player as shown in the Android Sample Applications or download one from the Android Market.

Happy streaming…

26 comments:

andy said...

Virtual Studio has all the required live media processing facilities to ensure successful completion of projects on time and within budgets. Its facilities includes: Talk shows hosting, using virtual sets; live & on-demand web-cast streaming studio; recording by voiceover artists; production facility for audio video presentations; option for live call outs or live phone-ins; radio shows; panel discussions; training-educational videos with software demos audio-video processing, optimization & encoding facility. All this with 100% power backup; multiple connectivity for live web-casts; online project management system.

psyher said...

Excellent article man! This has helped me tremendously. Keep up the good work mate.

john w said...

Hi,
Thanks for going into such detail on how to test!

Please pardon the plug, but I thought you might like to know you can request a free 10-stream Wowza Server 2 Developer edition license from us which now support both live and VOD streaming of h.264+aac content to the Android (the same file can be simultaneously streamed to a Flash player, an iPhone/iPad, to Microsoft Silverlight, and to other RTSP devices which play H.264+aac.

The installer package download and the free license request form are available here: http://www.wowzamedia.com/store.html

Yours,
John West
John [at] WowzaMedia [dot] com
+1-888-778-7997

Indicator Veritatis said...

This is a good article. I am particularly impressed by giving instructions for both Win32 and Linux. But I notice one step that seems to lack this: the one where you give only a link to a Win32 executable, i.e., http://erightsoft.podzone.net/GetFile.php?SUPERsetup.exe.

Is there really no Linux program we can use instead? It does not do a Linux user a whole lot of good to do all the other steps in Linux but then scramble to find a Win32 machine for this one step.

BTW: I cannot help but wonder how uptodate all this is. It was originally written last year, when some 1.x SDK was current. But we are now up to 2.2, and Google has a habit of making some pretty radical changes from version to version.

Sure, things have calmed down somewhat, the changes tend to not be as radical since 1.5, but still: a comforting word from the author concerning what is still up to date would be very welcome.

Android said...

I had the problem of being unable to install the Android plugin into eclipse. Whenever I clicked on "Next" it would give a mouse down look, but it wouldn't work. I ended up using the keyboard shortcuts - "Alt - N" and it worked fine.

swapnil said...

i have setup up everything and works fine in browser. but when i code and play that on android emulator it doesn't plays anything? is there anybody facing the same problem?

also when i play same video from sd card or by putting in the raw folder it gets played successfully.

Mistry Hardik (MSP 2010) said...

Thank you for the great tutorial for media streaming...

I am really thankful for this.

Regards,
Mistry Hardik

Anonymous said...

Nice Article mate. Perfectly written. I hope few critical things that i have been missing will be helped now. cheers.

Desifuntoosh

Anonymous said...

hi tis artical for very use full for me... and till now i did't get darwin streaming server can u give link
my mail id :convey2ragav@gmail.com

steven said...

any ideas on how to push live stream from android camera to an RTSP server like darwin or wowza?

using MediaRecorder to capture and record to target socket fd might be feasible but not sure if this is a good way. don't know of other ways other than to capture with android.hardware.Camera. This however will not be live but a final captured file to be later sent.

Tanmay said...

Hey thanks for the nicely explained article. Just wondering if you have looked into live streaming video from android to DSS ? I have been trying to find something in order to do this properly.. but havent found much.

Would be great if you could point me in the right direction.

Abs said...

Hello,

I did all the steps till where you said we need to create a webpage and a link to the video stating it this way -> rtsp://yourservername/yourfilename.mp4

Now lets say my server name is AABB and the name of my file is needs.mp4 and has been added to the playlist and i start it on the darwin server, can you tell me exactly the steps required and the stuff I have to do to get it on my phone?

I really need this urgently and your help will be invaluable as I have little to no experience in Android but require it as a part of another project.

Thanks a lot.

milton keynes said...

Hi , I am trying since few days, but there is an issue with my phone.
I third step after cmd search bar it gave me error.Why?

solar panel installers kent said...

Its hard to find some thing really healthy. Internet is a pool of websites and you have to drown in to it for good. Need tricks for search.Only then you will find what actually you want. Search engine has made searches easy but started a ranking wars. So always on top is not good
just

Android app developer said...

Thanks for providing the good information about android and i got a good knowledge to read your blog. we are based on android development company. so your information is very helpful to me.

Zakaria Chowdhury said...

If anyone is interested to build live radio for Android, then here is a complete app with source code:

http://codecanyon.net/item/android-live-radio/3697897?ref=zchowdhury

buzen said...

what is the difference between streaming video from a video server and streaming from an IP camera in terms of the protocol used and API.

hassaan naeem said...

Awesome! I love these delicate posts.Cheap Property For Sale

Tips Tricks Ideas said...

Long-touch over the selected text not working on my Galaxy S2 either. Verifiable rubbish I'm sad to say. Maybe one day Android will include the rudimentary basics.
i phones

Vishnu Kalsariya said...

hi tis artical for very use full for me... and till now i did't get darwin streaming server can u give full source cod.
my mail id :ahirvishnu@gmail.com

pranav said...
This comment has been removed by the author.
pranav said...
This comment has been removed by the author.
pranav said...
This comment has been removed by the author.
pranav said...

hey your article is suggesting how to stream to a android device. could you give in some information, where we could stream from android device to the computer?

i am trying to implement, video streaming for whatever you capture on your phone using its camera. (crime evidence). please do help me out.

here is my contact details,
pranav shah.
pranavrshah2006@gmail.com

Anonymous said...

i have an ERROR..
i did the windows way.it installed but when i open localhost:1220 in mozilla and IE...i login,but i dont see any buttons or anything,only this much in the whole screen

INDUANOOP
, 27. 2014 16:33:53
, 28. 2014 10:37:26
18 3 32

i dont see anything else.and i cant stream sample file from vlc..it doesnt work

Anonymous said...

Anonymous said...

same person here..its working in vlc player in pc...but not in my blackerry phone :error -server not responsive or unavailable

i have an ERROR..
i did the windows way.it installed but when i open localhost:1220 in mozilla and IE...i login,but i dont see any buttons or anything,only this much in the whole screen

INDUANOOP
, 27. 2014 16:33:53
, 28. 2014 10:37:26
18 3 32