Wednesday, March 24, 2010

X10 Home Automation using Java

This page describes the installation and usage of the Marmitek / ActiveHome X10 Home Automation System Components as well as the setup required to control those devices from Java.

Table of Contents

  1. History of X10
  2. How X10 Works
  3. Marmitek X10 Home Automation System
    1. CM15Pro PC Interface and Active Home Pro Software
      1. Setting Up Sender and Receiver
    2. Lamp Module LM12
    3. Appliance Module AM12
    4. Motion Sensor MS13
    5. Easy Touch Remote Control ET35
    6. Other Available Modules (not bought yet)
  4. The X10 SDK
    1. C++ Samples
    2. JavaScript Samples
  5. X10 and Java
    1. The Problem Statement
    2. The Solution
    3. Getting the Java Glue Code for Active Home
    4. Using the Generated Sources
  6. A Word On Performance
  7. A Word On The Future
  8. References

History of X10

X10 is a protocol standard for communication between home appliances over powerline that was developed more than 30 years ago in Scotland. The company General Electric, one of the biggest and most influential companies of the US heavily promoted this protocol which lead to a continuing success in North America.

Due to the different voltage system in America and Europe which makes the use of the original X10 protocol impossible in Europe, it has not become as much a success here. However, nowadays, appliances that were designed for X10 can be easily adapted to the European voltage systems and since then X10 - 30 years after its birth - gains more and more approval.

How X10 Works

X10 is a protocol that uses the powerline to transmit signals to home appliances. To address different devices, X10 divides a device address in

  1. a so-called House Code
  2. a Unit Code.

The House Code can assume 16 distinct values and ranges from A to P. The Unit Code can also assume 16 values ranging from 1 to 16. In total that allows X10 to address up to 256 independent devices.

This number, however, is slightly decreased by the fact that X10 powerline signals might have to be separated between different households by using different house codes. For example, if two households share the same powerline circuit, it is potentially possible that one neighbour controls the home of another one. In such a case the House Code of the devices in one of the two households would have to be changed to another value reducing the amount of available House Codes by one which is equal to a reduction of number of overall addressable devices by 16.

It is however also possible to apply specific hardware that is placed in the power circuit between the two homes and which will separate the power circuits in such a way that X10 signals will not traverse this boundary.

Marmitek X10 Home Automation System

Marmitek is a reseller for ActiveHome (www.activehome.com) and sell X10 devices for the European market. The devices have slightly different product codes than the original ones from America, but in principle they are exactly the same. Due to the difference in voltage systems, devices from America first need to be adapted to the European 250 Volts, and thus it usually takes a while until a newly introduced American model is available in Europe.

I purchased a starter kit for X10 Home Automation which consists of the following components.

CM15Pro PC Interface and Active Home Pro Software

The CM15Pro is the successor of the CM11 and provides a computer interface to control and monitor X10 powerline and radio devices. It comes with inbuilt memory for macros and a battery that keeps the memory alive.

The package comes with the Active Home Pro software that allows you to create virtual rooms and customise it with your available X10 appliances. It then lets you control and monitor those devices and create automated processing rules (macros) which you can execute from the PC or download to the CM15APro interface for later execution when your PC is switched off.

The CM15Pro is connected to powerline (at the same time driving the device) and to the computer via a USB cable. By default, the CM15APro is configured to listen and send to devices with a House Code of A, but of course that can be changed.

Setting Up Sender and Receiver

In order to control an X10 device, you always have to make sure that sender and receiver of X10 signals operate on the same House Code. For that to work, you need to adjust the address settings of the receiver appliance as well as that of the sender.

In case of simple appliance module (e.g. the lamp module or appliance module (see below)) this is done by turning the little dials for House and Unit Code to the appropriate settings. For example, turning the House Code dial to A, and Unit Code Dial to 1 gives that module the address of A1. In the CM15Pro, you set the House Code via the Active Home Pro Software that comes with it (Tools-->Hardware Configuration-->Monitored House Code).

Lamp Module LM12

The LM12 is an X10 module for lamps and appliances with little resistive and inductive loads. In other words, do not put your hairdryer or coffee machine into it. The LM12 can be switched on and off and can be dimmed also. Note however, that the lamp you plug into the module has to be dimmable - which is not the case for most of the low voltage halogen lamps that come with an inbuilt voltage transformer.

Appliance Module AM12

The AM12 is an appliance module, i.e. in contrast to the LM12 lamp module it can drive bigger appliances with more resistive and inductive loads (such as TVs, coffee machines or hairdryers). The AM12 can be switched on an off, but cannot be dimmed.

Motion Sensor MS13

The MS13 is a motion sensor for X10. It reacts both on motion as well as transitions from bright to dark and dark to bright (the latter can be disabled). When it detects motion it sends an "on" command to a predefined but configurable X10 address (default is A1). After a configurable time with no motion detected, it will send an "off" command to the same address. Likewise, if a transition from bright to dark is detected (and that feature is active), the MS13 will send an "on" command to the configured address + 1 (i.e. if it is configured to send motion detection to A1, it will send the brightness transition signal to A2). If the light level transitions back to bright, it sends an "off" command again. The MS13 is battery powered and sends X10 signals via a radio link. It cannot be controlled, it acts merely as a sensor.

Easy Touch Remote Control ET35

The ET35 is a universal remote control that can also control X10 appliances. It can send signals both over IR and radio frequency and comes with an inbuilt touch screen. It supports various device types (VCR, DVD, Satellite Receiver, etc.) and vendors (Sony, Philips, etc.). Furthermore it can be used to control X10 appliances remotely and send RF signals to the CM15Pro. The ET35 can be programmed to different devices using code tables and presets or it can learn keys from other remote controls.

Other Available Modules (not bought)

Marmitek offers a variety of other X10 appliances, among them security modules, locks and shutter systems. They also offer wall panels that look quite futuristic and can be used to remote control your X10 appliances from a single place. For a list of available products look at http://www.intellihome.be/english/products and make sure to poll it from time to time as still new products hit the market regularly.

The X10 SDK

X10 is fun the way it is. It becomes much greater fun, however, if you can write your own program for the control of the devices. Suddenly you are free to realise all those ideas that you had the first time you watched Star Trek and they are only a step away. In this chapter I will show what you have to do in order to connect your software to X10 appliances.

One of the best things about Marmitek's X10 devices is that an SDK is available. You can download it from here: http://www.activehomepro.com/sdk/sdk-info.html and it comes with (sparse) documentation and sample code.

The X10 SDK comprises a single DLL (ahscript.dll) with an ActiveXObject inside. The ActiveXObject provides an interface to send and receive X10 commands.

Marmitek provide sample code for

  1. C++
  2. Perl
  3. PHP
  4. JavaScript
  5. Visual Basic
  6. Visual Basic Script

No Java?! No Java! But don't worry, we will come to that soon. The code is rudimentary, as is the SDK itself, but with seemingly very little functionality you can actually do a great variety of really cool things.

The code works, and at least the JavaScript and C++ projects could be opened and edited out of the box.

C++ Samples

The C++ samples are provided with a Visual Studio 6 project, but it can easily be automatically imported into Visual Studio 2005 and works out of the box. I suggest you open AHCMD, which is a project for building a command line tool to send X10 commands.

The command line tool uses the following syntax:

> ahcmd [sendplc | sendrawplc | sendrf | queryplc] <device address> [on | off | dim] 
The commands that end on 'plc' are powerline commands, i.e. the signals are sent via powerline. Those that end on 'rf' are sending signals over radio frequency. The device address is a combination of House and Unit Code, e.g. A1, A2, P12 etc.

Note:
Whether a device can be dimmed or not depends on two factors basically: whether the actual device can be dimmed (e.g. a lamp) and whether the X10 module the device is connected to supports it. It is clear that a coffee machine can hardly be dimmed, and even if it could, the coffee would probably not be better. On the other hand a lamp that is connected to an appliance module like the AM12 will never be dimmed because the module itself does not support it.

There is another C++ sample called AHCVIEW, which is a GUI tool that monitors incoming X10 signals. Feel free to consult the code and learn how you can access X10 devices from a C++ program. The basic idea is pretty simple:
  1. You get the ActiveXObject
  2. You call the SendAction() method and pass in
    1. A String for the command (e.g. sendplc, sendrf, queryplc)
    2. The address of the device (e.g. A1)
    3. The command (e.g. on, off, dim 30)

And that is it.

JavaScript Samples

The JavaScript samples are a little bitch to say the least. The first thing you have to know about them is that they only work in Internet Explorer. The second thing you need to know is that they only work in Internet Explorer, if you allow ActiveX plugin execution. Usually Internet Explorer will pop up a tiny bar underneath its toolbar asking you if you want to allow the execution or not. Do yourself a favour and allow it.

The JavaScript sample you will most likely want to try out first is called "Dynamic Receive" and out of the box it does not seem to work. However, that is due the fact that the JavaScript code is trying to send a command via the radio frequency channel and not via powerline if you press the button. Unless you have a radio receiver module, you will notice that absolutely nothing is happening.

A simple way out of this problem is to open and edit dynamicreceive.html and search for the following lines of code:

    if (Command == 1) {
myForm.text2.value = "Command sent a1 ON";
x10.SendAction( "sendrf", "a1 on" );
} else if (Command == 2) {
myForm.text2.value = "Command sent a1 OFF";
x10.SendAction( "sendrf", "a1 off" );
}
Then you replace it with the following:

    if (Command == 1) {
myForm.text2.value = "Command sent a1 ON";
x10.SendAction( "sendplc", "a1 on" );
} else if (Command == 2) {
myForm.text2.value = "Command sent a1 OFF";
x10.SendAction( "sendplc", "a1 off" );
}
Notice that sendrf was replaced by sendplc.

Pressing the button now should switch your module on, provided that it is configured to be addressed as A1.

Have a look at the JavaScript code and learn how you access X10 devices from JavaScript. Note: Just because you are accessing your X10 devices via JavaScript and from a browser doesn't mean you are able to access them from remote or even the Web! I have seen posts in forums that asked "why can I only control my appliances from the browser on the PC that has Active Home installed?" Well, the answer is, because Internet Explorer loads the ActiveX object from the SDK's DLL directly and sends a command to the devices. No remoting, no Web involved. Also remember that JavaScript is executed on the client side always! If you are looking for ways to control X10 devices from remote or even the Web, you need to do it from PHP, or using Java, as described later.

X10 and Java

One of the most coveted solutions in the X10 world is that of being able to send and receive X10 signals using Java. Forums are full of questions like "How do I access my cool Active Home stuff from Java" and "Is there a Java SDK". Unfortunately, most of those entries lead nowhere and having searched for it myself, I usually ended up on pages that were providing solutions only for older, slightly outdated X10 interfaces (like the CM11) and relied on Java COMM - an API to access the serial port, unfortunately without SUN's support for Windows.

Well, good news: the solution I present here works fine with the new CM15Pro and it is likely to work for even newer devices.

The Problem Statement

The problem I was trying to solve is the following:
  1. I want to have the same functionality in Java as is provided by the SDK
  2. I cannot use the Java COMM approach, since
    1. the latest versions of Java COMM are only for Linux
    2. the CM15Pro is connected via USB and does not provide a serial port

  3. I don't want to re-implement X10 using javax.usb
  4. I want something that is (almost) future proof
I knew of a library called JNA (Java Native Access) which allows access to native libraries (e.g. a DLL) and access the exported functions directly from Java, no JNI involved. JNA is great however, it only works for native libraries that provide functions directly. The Active Home SDK however relies on Microsoft's programming paradigm called COM - Component Object Model.

Knowing that, the problem statement transforms into:
  1. How do I call a COM (ActiveX) object from Java
  2. Does it also support callbacks into my Java code
Well, that looks more like something you can type into Google, right?

The Solution

Knowing what I was looking for, I found a project called Com4J (https://com4j.dev.java.net/) which is simply spoken brilliant! Com4J provides a native wrapper for any COM library you can think of and the Java hooks to access the COM objects right from Java programmes. Com4J comes with a tool you can run over your DLL or OCX file and which creates Java sources for you to interact with whatever COM object is available from the DLL. You need to download and unpack the com4j archive to a local folder and then follow the steps which in the following I will outline in detail.

For the remainder of this section we assume that you have downloaded com4j and unpacked the archive to the folder C:\com4j. Furthermore we assume that you have installed the Active Home Scripting SDK in the folder C:\ahsdk


Getting the Java Glue Code for Active Home

The Active Home SDK uses only a single DLL - ahscript.dll. This is usually located in the bin folder of your SDK installation.

To get the Java glue code you execute the following:

C:\com4j> java -jar tlbimp.jar -o c:\ahsdk\java -p <your java package> -v C:\ahsdk\bin\ahscript.dll
This will inspect ahscript.dll and generate Java source code for accessing the COM objects in the output folder C:\ahsdk\java and in the package you specify.

That's it, really. There is no more to it than that.

For me, the generated sources look as follows (but that may vary depending on your version of ahscript.dll):

_IActiveHomeEvents.java:
package test  ;

import com4j.*;

/**
* _IActiveHomeEvents Interface
*/
@IID("{001000AF-3DEF-0912-10B6-DC5BA692C858}")
public interface _IActiveHomeEvents extends Com4jObject {
/**
* method RecvAction - Called when commands
* have been received
*/
@VTID(3)
@DefaultMethod
void recvAction(
@MarshalAs(NativeType.VARIANT)
java.lang.Object bszAction,
@MarshalAs(NativeType.VARIANT)
java.lang.Object bszParm1,
@MarshalAs(NativeType.VARIANT)
java.lang.Object bszParm2,
@MarshalAs(NativeType.VARIANT)
java.lang.Object bszParm3,
@MarshalAs(NativeType.VARIANT)
java.lang.Object bszParm4,
@MarshalAs(NativeType.VARIANT)
java.lang.Object bszParm5,
@MarshalAs(NativeType.VARIANT)
java.lang.Object bszReserved);

}
_DIActiveHomeEvents.java:
package test.events;

import com4j.*;

/**
* _DIActiveHomeEvents Interface
*/
@IID("{001000AF-3DEF-0911-10B6-DC5BA692C858}")
public abstract class _DIActiveHomeEvents {
/**
* method RecvAction - Called when
* commands have been received
*/
@DISPID(0)
@DefaultMethod
public void recvAction(
java.lang.Object bszAction,
java.lang.Object bszParm1,
java.lang.Object bszParm2,
java.lang.Object bszParm3,
java.lang.Object bszParm4,
java.lang.Object bszParm5,
java.lang.Object bszReserved) {
throw new UnsupportedOperationException();
}
}
IActiveHome.java:
package test  ;

import com4j.*;

/**
* IActiveHome Interface
*/
@IID("{001000AF-3DEF-0910-10B6-DC5BA692C858}")
public interface IActiveHome extends Com4jObject {
/**
* method SendAction - use to send commands
*/
@VTID(7)
@DefaultMethod
@ReturnValue(type=NativeType.VARIANT)
java.lang.Object sendAction(
@MarshalAs(NativeType.VARIANT)
java.lang.Object bszAction,
@MarshalAs(NativeType.VARIANT)
@DefaultValue("0")
java.lang.Object bstrParam,
@MarshalAs(NativeType.VARIANT)
@DefaultValue("0")
java.lang.Object vReserved1,
@MarshalAs(NativeType.VARIANT)
@DefaultValue("0")
java.lang.Object vReserved2);

@VTID(8)
void onRecvAction(
@MarshalAs(NativeType.Dispatch)
com4j.Com4jObject rhs);
}
ClassFactory.java:
package test  ;

import com4j.*;

/**
* Defines methods to create COM objects
*/
public abstract class ClassFactory {
private ClassFactory() {}

/**
* ActiveHome Class
*/
public static test.IActiveHome
createActiveHome() {
return COM4J.createInstance(
test.IActiveHome.class, "{001000AF-
2DEF-0208-10B6-DC5BA692C858}" );
}
}

Using the Generated Sources

Now that we have the sources generated, all we have to do is use them to access the X10 devices from our Java code. To that end, we create a new Java project in our favourite IDE and copy the generated sources over.

I created a simple test, which shows how
  1. you send an X10 signal
  2. you receive an X10 signal
Test.java:
package test;

import test.events._DIActiveHomeEvents;
import com4j.Com4jObject;
import com4j.EventCookie;

public class Test
{
public static void main(String[] args)
{
IActiveHome ah
= ClassFactory.createActiveHome();
ah.sendAction("sendplc", "A1 off", "", "");
EventCookie eventSubscription =
ah.advise(_DIActiveHomeEvents.class,
new EventReceiver());

while(true)
{
try
{
Thread.sleep(15000);
}
catch(Exception e)
{
e.printStackTrace();
}
}

//not necessary here, but usually you should
//not forget to release the event listener:
//eventSubscription.close();

}
}

class EventReceiver extends _DIActiveHomeEvents
{
public void recvAction(
java.lang.Object
bszAction,
java.lang.Object bszParm1,
java.lang.Object bszParm2,
java.lang.Object bszParm3,
java.lang.Object bszParm4,
java.lang.Object bszParm5,
java.lang.Object
bszReserved)
{
System.out.println("Called");
String receptionType = "<null>";
String x10Address = "<null>";
String plcCommand = "<null>";
String additionalData = "<null>";
String rfCommand = "<null>";
String keyData = "<null>";
String timestamp = "<null>";

if(bszAction != null)
System.out.println("action: " +
bszAction.getClass().getName());
if(bszParm1 != null)
System.out.println("1: " +

bszParm1.getClass().getName());
if(bszParm2 != null)
System.out.println("2: " +
bszParm2.getClass().getName());
if(bszParm3 != null)
System.out.println("3: " +
bszParm3.getClass().getName());
if(bszParm4 != null)
System.out.println("4: " +
bszParm4.getClass().getName());
if(bszParm5 != null)
System.out.println("5: " +
bszParm5.getClass().getName());

// if action is recvplc, then the param
// types are as follows:
//1: String -- X10 Address
//2: String -- X10 command
//3: String -- X10 additional data
//4: String -- not used ?
//5: String -- not used ?

// if action is recvrf, then the param types
// are as follows:
//1: java.lang.String -- X10 Address
//2: java.lang.String -- RF Command
//3: java.lang.Integer -- key data
//4: java.util.Date -- timestamp
//5: java.lang.String -- not used ?
}
}
As you can see, sending a command is done by calling
ah.sendAction("sendplc", "A1 off", "", "");
Receiving X10 commands is not much harder. All you have to do is register an event listener. For that purpose you first let your event listener extend the generated class _DIActiveHomeEvents and then override the recvAction method with your own code. You then register this listener by calling
EventCookie eventSubscription = ah.advise(_DIActiveHomeEvents.class, new EventReceiver());
This makes your listener available for callbacks from the native COM object. The EventCookie you get returned serves as a handle to the native event loop if you like, so once you are done listening for events, you simply call
eventSubscription.close()
to free resources.

The recvAction method is called by the SDK whenever an X10 event is received and depending on the type of event, the parameters that are passed along have different types and meanings. Those can be looked up in the SDK's compiled help file ahscript.chm under the topic of "Using the Scripting Interface".

A Word On Performance

Certainly calling from Java into some native wrapper library that then calls into the actual DLL will be possible at the cost of performance. However, given the latency of X10 powerline commands and the fact that we are not trying to do any real-time stuff here, the proposed solution has no performance problems at all.

A Word On The Future

What I presented here allows you to send and receive X10 commands both via powerline and radio frequency using the CM15Pro computer interface and X10 appliances both from ActiveHome and Marmitek. For example, you could write a Java application that receives commands from your X10 remote control, or you could write a Servlet that exposes X10 functionality on a Web server that is remotely accessible with a browser or your mobile phone. The options are endless and only limited by your fantasy. I believe we are going to see much more of the kind in the future, so go ahead and create the future. Today.

The beauty and elegance of the proposed solution is that it should be pretty future proof. I don't know how many future versions of the Active Home SDK are due to come out still, but given the fact that X10 has been around for more than 30 years now, the probability for future updates is quite high. As long as the Active Home SDKs keep on using COM objects and DLLs like the current one, this solution will always work. Given the tools to generate Java source code for existing DLLs, updating will not even take much time.

I hope this is useful to someone, and if you feel it is, let me know, spread the word and if you like drop me a line saying what cool things you implemented. Have fun.

References

http://www.intellihome.be/english/products
https://com4j.dev.java.net/
http://en.wikipedia.org/wiki/X10_%28industry_standard%29
http://www.x10.com/activehomepro/sdk/index.html

Sunday, October 11, 2009

Video Streaming with the Android Phone

This page describes how to stream video to the Google Developer Phone. This topic is a very interesting, but also quite complex one, and it took me a while to figure all variables out in a proper way.

If you follow this walkthrough you should be able to install a video streaming server, adapt any of your video content and stream the result to your Android Phone. It also contains encoding settings that you might find useful to encode your videos to watch them (locally)on the G-Phone.
Hope this is useful to someone out there.

Table of Contents
  1. Video Streaming with the Google Phone
    1. Setting up a Streaming Media Server
      1. Installing Perl
      2. Installing Darwin Streaming Server
        1. Installation on Windows
        2. Installation on Ubuntu Linux
      3. Administrator Console
        1. Starting
        2. Stopping
      4. Administering Darwin Streaming Server
      5. Media Repository
      6. Firewalls and Darwin Streaming Server
    2. Creating Streaming Media for the Google Developer Phone
      1. Encoding a Movie
        1. Getting the Right Dimensions
      2. Hinting the Result
      3. Deploying the Media
      4. Testing the Streaming

Video Streaming with the Google Phone

Android comes with the ability to play back video and audio that is stored locally on the device. More important though, Android is also able to receive streaming media such as video and audio with its Media Player API. In this tutorial we show what you need to
  • set up a streaming media server
  • create streaming media compatible with the Google Phone's available codecs
  • Receive the streaming media on the phone

Setting up a Streaming Media Server

There are a variety of Streaming Media Servers out there, but the one that we think is the easiest to use is Apple's Darwin Streaming Server (DSS). Darwin Streaming Server is Open Source and its commercial brother is Apple's Quicktime Streaming Server (QSS). Darwin can be downloaded from here: http://dss.macosforge.org/. You can click on the previous releases link (http://dss.macosforge.org/post/previous-releases/) if you want to get a binary version and not go to the hassle of building Darwin Yourself. In this Tutorial we used DSS v5.5.
Installing Perl
Once you downloaded Darwin you need to install it on your machine. This requires that you have Perl installed. A working version of Perl (for Windows) can be found on the Strawberry Perl site: http://strawberryperl.com/. Download the Strawberry Perl and install it. Once you have it installed, open a command line and type the following command:
> perl -MCPAN -e shell

The CPAN shell opens and there you type:

CPAN> install Win32::Process

This will make Perl go to the Perl package network (the CPAN to be precise) and download the Win32::Process package needed by Darwin. If the downloading has succeeded, you can go on to installing Darwin Streaming Server.

Installing Darwin Streaming Server
Installation on Windows
Unpack the downloaded Darwin to a local folder and execute install.bat. If you are running the install.bat under Windows Vista, make sure you run it in administrator mode. You can do that by right-clicking on the file and select Start as Administrator from the context menu or you can start a command line shell in admin mode by doing the following:
  1. Press the Start button in your task bar
  2. type 'cmd' in the search bar
  3. press Shift+Ctrl+Enter
This will start the console in administrator mode. You can then browse to the folder containing install.bat and execute it from command line.
Installation on Ubuntu Linux
Installation on Ubuntu Linux is straight-forward. All you need to do is download the tar.gz file with the binaries (for this tutorial we used Darwin 5.5.5 with the binaries given here: http://dss.macosforge.org/downloads/DarwinStreamingSrvr5.5.5-Linux.tar.gz) to your Linux machine, unzip and untar it e.g. like this:

$> gunzip DarwinStreamingSrvr5.5.5-Linux.tar.gz
$> tar xf DarwinStreamingSrvr5.5.5-Linux.tar

The result is an untared folder, which contains several executable scripts, one of them being called Install. Before you can execute Install you need to modify the script a bit. Open the script (e.g. with vim) and search for the following part:

useradd -M qtss

Once you have found it, replace it by

useradd -m qtss

Note the lowercase -m option. Safe the file and then from the shell execute

$> ./Install

You might have to be root to be allowed to do that.
Administrator Console
If the installation of Darwin is successful, you will be prompted for entering a user name and a password. Those are used by the Administrator Console of Darwin and need to be remembered well.
Starting
The admin console is started right after installation, so if you close the installation window, it will be terminated. To start it again, open a command shell, browse to the installation folder of Darwin (on Windows this is C:\Program Files\Darwin Streaming Server) and type

C:\Program Files\Darwin Streaming Server> perl streamingadminserver.pl

This will start the admin console server and you can access it from your browser under http://localhost:1220.
Stopping
To stop the admin console simply terminate the Perl script. Note: Terminating the admin server console does not terminate the streaming server. Darwin is installed as a Windows service and as such can only be stopped via the Windows Services view (right-click on My Computer-->Manage-->Services and Applications-->Services).
Administering Darwin Streaming Server
Via the admin console server you can administer Darwin. The first time you will access the console, you will be asked to enter an MP3 administrator's password. You will be asked for that password whenever you want to upload a new MP3 to Darwin, so remember it well. The admin console is pretty self-explanatory, so take a look around and see what you can do.
Media Repository
Whatever media you want to stream with Darwin has to be placed inside the Movies folder of your Darwin installation. On Windows, that is C:\Program Files\Darwin Streaming Server\Movies. See the description on Media Creation below, to see what pre-requisites your media has to fulfil.
Firewalls and Darwin Streaming Server
Darwin is an RTSP server and by default uses the regular RTSP port 554 for RTSP both over TCP and UDP. That port may be blocked by the firewall you installed Darwin on. To make sure the ports are open, you need to open those ports in your firewall. If you don't know how to do it, ask your friendly administrator.

Important: If you fail to have the proper ports open, you will not receive any streaming on the phone.


Note: In the admin console server, you can choose to enable streaming over port 80, which means that RTSP requests will be tunnelled through port 80 which is usually open in firewalls. For RTSP requests from the phone / external client to your Darwin Streaming Server machine, this means that you have to use an RTSP URL including the port to use, e.g. rtsp://yourserver:80/moviename.mp4.

Creating Streaming Media for the Google Developer Phone

Creating streaming media depends on two things:
  1. the streaming server you are using
  2. the capabilities of the client, i.e. the phone
In our case, we have decided for the Darwin Streaming Server as streaming server solution and the Android Dev Phone 1 as the client.

Both the capabilities of the server and the phone are important as they decide which media formats will work and which wont.

Darwin Streaming Server supports a variety of containers, among them .3gp, .mp4 and .mov. It is more or less agnostic to the codecs used to encode the media but needs to be hinted how to stream a media file. The Dev Phone on the other hand has specifications that indicate which container types and codecs are supported for which media. A list of specifications can be found here http://developer.android.com/guide/appendix/media-formats.html.
Essentially this page explains that Android currently supports the following Video Formats:































CodecEncoderDecoderContainer Format
H.263yesyes3GPP (.3gp) and MPEG-4 (.mp4)
H.264 AVCnoyes3GPP (.3gp) and MPEG-4 (.mp4)
MPEG-4 SPnoyes3GPP (.3gp)


This limits our choice of codecs to H.263, H.264 and MPEG-4 Simple Profile. However, even if you know this it is difficult enough to create properly encoded videos that also show up in a suitable quality on the phone. In the next section we will give an overview of settings and tools that can be used to create suitable media.
Encoding a Movie
To make a movie suitable for playback and / or streaming on the phone, we need to encode it with the right codecs, and place the encoded movie into the right container format.
A tool that is invaluable for that task is SUPER which you can download from here: http://erightsoft.podzone.net/GetFile.php?SUPERsetup.exe

Once installed, the tool lets you add Media files to a list of files to encode and then specify the encoding settings for the resulting media file. To add a media file for encoding, right-click in the lower part of the SUPER window and select Add Multimedia File(s).
Browse to a folder with the movie you want to stream to the phone and select it. Now it is time to tune the settings to get the properly encoded result. A very good list of settings can be found here: http://androidcommunity.com/forums/vbglossar.php?do=showentry&catid=2&id=28

I copied the contents here, just in case the link will disappear one day:

























Codecs:MP4 H.264 AAC LC
Video:480:270 3:2 23.976 480kbps (yes, you set the video to 480x270)
Audio:44100 2 96kbps Default
Options:Hi Quality:OFF Top Quality:OFF
Pad:Top:24 Bottom:26 Left:0 Right:0
Crop:Top:0 Bottom:0 Left:Calc Right:Calc

While everything else stays the same from movie to movie, the Calc options above need to be calculated depending on the resolution of the video source.
I'll quickly give the calculation and then give an example.

The calculation is ( Width - ( Height * 16 / 9 ) ) / 2. Width and Height are of the source video and the result would be what you would select for the left and right crop. Here's a couple examples from DVD rips: * 886x480 source video: ( 886 - ( 480 * 16 / 9 ) ) / 2 = 16 (set the left and right crop to 16) * 1128x480 source video: ( 1128 - ( 480 * 16 / 9 ) ) / 2 = 137 (set the left crop to 136 and the right crop to 138) * 852x480 source video: ( 852 - ( 480 * 16 / 9 ) ) / 2 = -0.7 (this movie is already 16x9 so you can turn Crop OFF).
After you have made the settings, click the Encode (Active Files) button. This will create a video output encoded with the settings you specified. You can verify the quality, with VLC player for example.

Important Note:
The settings above may be too much for streaming to the phone. It could be noticed that there is considerable packet loss on the device, probably due to buffer overflows on the device's video player. The settings above are better for local playback on the phone. For streaming use the following settings for a movie in 16:9 format:





































Codecs:MP4 H.264 AAC LC
Video:256:144 3:2 23.976 480kbps
Audio:44100 2 96kbps Default
Options:Hi Quality:OFF Top Quality:OFF
Pad:Top:12 Bottom:14 Left:0 Right:0
Crop:off

As you can see the video size has been reduced to roughly half the size of the settings above. This reduces the bandwidth considerably. If you still experience packet loss you can further reduce the width and height. But pay attention since the dimensions (at least the width) have to be a multiple of 16, i.e. width modulo 16 must yield 0. Otherwise Super will fail to encode. This is not a bug, but a constraint of the codec / container format.
Getting the Right Dimensions
When you change the size of the video, the following things need to be considered:
  1. display size of the phone
  2. aspect ratio of the original video
  3. aspect ratio of the phone's display
  4. encoder constraints
The Android phone's display size is 480x320 which yields an aspect ratio of 3:2(= 480:320). So 3:2 is the aspect ratio we want the encoded video to be at finally (like this the phone's player does not have to do expensive rescaling).

Let's now assume we have a video of size 1024x576 which has an aspect ratio of 16:9 (= 1024:576). To get that movie on a 3:2 screen, we can either stretch / shrink it - adding unwanted distortion to the movie - or add padding to the movie. Let's further assume we want to end up with a movie that has a width of 256. For it to still be in 16:9, we need to set the height to 144, which is 256 / 16 * 9.

We will then end up with a movie of size 256x144. The problem is, that this is 16:9 and we want 3:2 so that the player on the phone does not have to do rescaling. If our movie width is 256 and the movie were in 3:2, then the height of the movie would have to be 170 (= 256 / 3 * 2).

However, it is only 144 since we don't want to distort the movie. So the solution is to pad the remaining height with black pixels (the black stripes you know from TV) so that we reach the 3:2 height. Thus we need to pad 26 (= 170 - 144) pixels, and we do that by padding 12 on the top and 14 on the bottom (12 + 14 = 26). The 2 pixel difference is not visible.

After encoding, we end up with a movie that is actually 3:2 with size 256x170 but "contains" the downsized original in 16:9 format. If you understood that you can reproduce it with any size and aspect ratio you like.

Note: keep in mind the restrictions of the codec. Width % 16 must be 0.
Hinting the Result
After having created the properly encoded video file, we are only one step away from streaming it to the phone. In order to be able to do so, Darwin Streaming Server needs to get hints how to stream media it is not able to decode.

As mentioned above, Darwin is agnostic to the codec used by the file that is streamed which makes it easier to support new codec types as they are being created. In order to stream the encoded media properly, however, Darwin needs to process hints which are a separate track inside the media file's container. Without those hints, Darwin will not be able to stream your media - whether properly encoded or not.

To create a hinted media file, you need a program called MP4Box, which you can download binary versions of from here:

http://www.tkn.tu-berlin.de/research/evalvid/

Windows: http://www.tkn.tu-berlin.de/research/evalvid/EvalVid/mp4box-w32.zip
Linux: http://www.tkn.tu-berlin.de/research/evalvid/EvalVid/mp4box-lin.tar.bz2
Mac: http://www.tkn.tu-berlin.de/research/evalvid/EvalVid/mp4box-mac.tar.bz2

Once downloaded you can add it to your system's PATH or simply put it into the same folder as the media file you want to add hints to. To hint the movie we have created in the encoding step, open a command line shell, browse to the folder containing the encoded video and the MP4Box executable then run

mp4box -hint <mediafile>.mp4

When the program terminates successfully, our encoded media file is hinted and ready for deployment to Darwin.
Deploying the Media
To make our video available for streaming, all we need to do is copy the file to Darwin's Movies folder, under Windows this is C:\Program Files\Darwin Streaming Server\Movies.
Testing the Streaming
To test if streaming of your new file is working, you can use VLC player. Start VLC player and select File-->Open Network Stream. In the dialog enter RTSP as the protocol and the url rtsp://localhost/<moviefile>.mp4 and press OK.
The video should start playing back.

Important: Depending on the Version of VLC you either have to type the full RTSP URL or only the URL omitting the protocol prefix, i.e. localhost/movie.mp4 instead of rtsp://localhost/movie.mp4.


If testing with VLC worked fine, you should be able to access the video from the Media Player on the Android Phone as well. The easiest way to do that is to create a Web page with a link to the RTSP URL of the movie you are streaming, i.e. somehting like this:

<a href=””>rtsp://yourserver/yourmediafile.mp4</a>

You can then access this page from your Android Phone’s browser and click on the link. This will start the internal Android Media Player and playback the video.

Of course, you can also create your own Media Player as shown in the Android Sample Applications or download one from the Android Market.

Happy streaming…

Tuesday, July 7, 2009

Android DevPhone Essentials

This page compiles some useful information for setting up the Android Developer Phone for development.

Table of Contents

  1. Dev Phone Support Page
  2. Changing the Default Google Account
  3. Android SDK and Developer's Guide
  4. Mobile Provider Settings for the DevPhone
    1. E-Plus / Simyo
    2. Other Operators
  5. Sending SMS with the DevPhone
    1. SMSC Numbers for E-Plus / Simyo
  6. Hard Resetting the DevPhone

Dev Phone Support Page

Android is Open Source and as such work in progress. Occasionally there will be updates of the platform and APIs available, which have to be deployed to the phone. Since Android includes the phone's operating system (which may need to be enhanced with the proper phone hardware drivers), any updates available will have to be reviewed by the phones' manufacturers and enhanced with the proper drivers. If you like, the updates have to be customised for each individual phone model.

That means you should download Android updates only from your phone's manufacturer site. For our DevPhone, which is HTC, the following page is the one where you can find updates.

Note: By the time of writing this, the latest Android version is 1.5-r1

Page Link Description
HTC Support http://www.htc.com/www/support/android/adp.html HTC's support page with new and previous Android system images to flash the phone with. Make sure you follow the directions closely.

 

Changing the Default Google Account

You need a Google account to properly use the DevPhone. The first time you switch on the phone, you will be presented with a setup wizzard, that lets you create or set your Google account. To change that account later, the default procedure is to rest the phone, which removes all changes you have made. Fortunately, I have found another way, as described here:

http://androidcommunity.com/forums/f10/change-default-google-account-solved-10584/

The procedure is as follows: "The easiest way to change accounts is to go to Settings->Applications->Manage Applications->Google Mail and clear all data. I also did it for Google Talk and all other Google related apps . This created a notification that I needed to redo the setup wizard with a link in the notification to start it. This fixed everything and didn't have to worry about a hard-reset."

 

Android SDK and Developer's Guide

Whatever you do with Android, it is only half the fun if you do it without the SDK. Google provide the SDK and a Developer's Guide, including the API docs on the following pages: 

Page Link Description

Android SDK Download

http://developer.android.com/sdk/1.5_r1/index.html

Download the SDK from here.
Android Dev Guide http://developer.android.com/guide/index.html Developer's Guide. You can't do it without it.
Android API References http://developer.android.com/reference/packages.html Not always well documented but better than nothing.

Note: Reade the Developer Guide completely (especially the tools section), since it contains a lot of information that is not straight forward. By investing - say a week - in reading that guide, you spare yourself a lot of frustration and trouble getting started. Oh, and you will be frustrated... :)

 

Mobile Provider Settings for the DevPhone

Depending on your mobile provider, you will have to set up the DevPhone's access points to be able to access the internet and messaging services of your provider.

You set up the access points by pressing Menu-->Settings-->Wireless Controls-->Mobile Networks-->Access Point Names. Since in our lab we usually use E-Plus as mobile operator, here come the settings for E-Plus first. Below you will find (untested) setups for other operators, too.

Important Note: For some operator (like E-Plus) you have to create several access points on the phone. You then select one that is being used. Your selection depends on what service you want to use. E.g. in the case of E-Plus an APN for Internet and one for MMS have to be configured. To send MMS the MMS APN has to be selected and active, to access the Internet, you will have to switch access points in the Menu-->Settings-->Wireless Controls-->Mobile Networks-->Access Point Names menu to the Internet APN.

E-Plus / Simyo

Internet Access  
Name: E-Plus Internet
APN: internet.eplus.de
Proxy: <not set>
Port: <not set>
Username: eplus
Password: internet
Server: <not set>
MMSC: <not set>
MMS Proxy:

<not set>

MMS Port: <not set>
MCC: 262
MNC: 03
APN Type: <not set>

 

MMS Service  
Name: E-Plus MMS
APN: mms.eplus.de
Proxy: <not set>
Port: <not set>
Username: mms
Password: eplus
Server: <not set>
MMSC: http://mms/eplus
MMS Proxy: 212.23.97.153
MMS Port: 8080
MCC: 262
MNC: 03
APN Type: <not set>

Note: To send MMS the MMS APN has to be selected and active. To access the Internet, you will have to switch access points in the Menu-->Settings-->Wireless Controls-->Mobile Networks-->Access Point Names menu to the Internet APN (select the radio button behind the respective access point).

Other Operators

Settings for Operators other than E-Plus can be found here:

http://www.android-hilfe.de/t-mobile-g1-forum/56-g1-apn-einstellungen-netzbetreiber-vodafone-t-mobile-e-plus-o2.html

 

Sending SMS with the DevPhone

In order to send SMS from a phone you need to set up a so-called SMSC, which is short for Short Message Service Centre and specifies the SMS proxy / server at your operator. It is where you send outgoing SMS to and which then takes care of delivering it to the right recipient. The SMSC is identified by a simple telephone number, and each operator has its own.

Unfortunately there is no way to set the SMSC number on the DevPhone, since - for what reason ever - there is no input field available for it.

Fortunately though, there is an application available on the Android Market which is free and allows you to make those settings after all. The application is called AnyCut and is primarily used to make one-click shortcuts to popular functionalities of the phone.

Note: You can receive SMS also without this configuration. This is only to enable sending.

Here is what you need to do to change / add the SMSC address to your DevPhone:

  1. On your phone go to the Android Market and search for 'AnyCut'

  2. Download and Install the application to your phone
  3. With AnyCut, create a new shortcut like this:

    1. Press on New Short Cut

    2. Press on Activity

    3. Scroll way down to Phone Info

    4. Press Ok to create a short cut on your home view

  4. Press the Home button on your phone to go to the home view
  5. Select the Phone Info shortcut
  6. Scroll down to the end and enter the appropriate SMSC address / telephone number into the labeled field
  7. Press Update

This will set the SMSC number for your phone and you are ready to send SMS afterwards.

SMSC Numbers for E-Plus / Simyo

The following SMSC number is used by the E-Plus / Simyo operator: +49 1770610000.

For a list of (untested) SMSC numbers of other operators, see

http://www.umtslink.at/index.php?pageid=zeige_webseite&url_par=/sms/smsc_rufnummern.htm

 

Hard Resetting the DevPhone

/!\ Warning: If you follow these steps, the phone will be reset and all your settings will be lost!

Note: This does not reset the system image of the phone, so if e.g you want to switch from an Android 1.5 system image back to Android 1.0, you will have to install the previous image as is described on the HTC support page. Resetting the phone only removes all settings on the phone.

To do a hard reset of your DevPhone proceed as follows:

  1. Power off the device
  2. Press and hold the home key and the power key at the same time until you see a /!\ sign on the screen

  3. Once you see the triangle with exclamation mark open the keyboard and hit Alt+W

This will reboot the device and reset all settings.

Sunday, April 20, 2008

Building XULRunner and WinEmbed using Visual Studio 2005

Lately I have been searching around for ways to properly embed a Mozilla browser into my software and noticed that there is a huge community out there trying to do exactly the same. Unfortunately this is a tricky issue and documentation about it really hard to find and often ambiguous.
I created this brand new blog, to post as a first contribution a description of how to build XULRunner and a nice little test project called WinEmbed under windows and using Visual Studio 2005.
Please note, that none of the source code was written by me. It all comes from the Mozilla guys. I have merely gone to the hassle of compiling information from a million diverse sources, making things build and blogging all that in one location. So let's get started.

1. Some considerations before we start
There are several ways to embed a mozilla browser into your own software, some are better, some are worse but all of them greatly depend on what you actually want to achieve. As for me, I wanted to have a browser window embedded into my own GUI. Somethting that nicely renders web pages, can handle flash and javascript and is extensible and free.
In the beginning I tried embedding Firefox, meaning that I checked out the source code, built it and tried to figure out how to change it so that it would fit my needs. Not a very good choice!
But there is a better one: XULRunner. XULRunner is a kind of runtime environment that allows you to run XUL applications. These XUL applications are packaged pieces of software with descriptors that tell the XULRunner runtime, where to find resources and how to display them.
More than that, XULRunner can also be used for embedding the Mozilla browser engine (Gecko) into your own software. WinEmbed - a test project from Mozilla, and part of the CVS checkout - demonstrates how that is done.
The basic idea behind it is, that you write your embedding code in such a way that at startup of your software, the embedded browser components are loaded dynamically from a XULRunner installation on your system.
This means that when you want to deploy your software you will have to deploy a XULRunner installation along side with it. One of the many benefits of this solution is that you can use clean interfaces to the XULRunner components and don't have to change any browser code to fit it your needs.

2. Getting the Source Code
As posted on http://www.nathanm.com/building-xulrunner/, you have to get XULRunner and build it. You can either get a packaged version of the sources or you ceckout the sources from Mozilla's CVS.
In this walkthrough we will checkout the sources from CVS.

2.1 Checkout Mozilla Sources from CVS
You need a CVS client (e.g. WinCVS or Tortoise CVS) and you need to access it with the following parameters:
Protocol: pserver
Server: "cvs-mirror.mozilla.org"
Port: default (2401 or leave blank)
Repository folder: "/cvsroot"
User name: "anonymous"
Your client will have to use a CVSROOT of :pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot
This is specified also on the Mozilla web site.
If your CVS client supports it, select that you want to checkout using UNIX-like line endings. Otherwise you will have to run a python script over the checked out sources to convert line endings to UNIX format.

Checkout the module "mozilla" to your local disk.

2.2 Getting the Mozilla Build Tools
To build XULRunner you will need
  1. Visual Studio 2008 installed
  2. The Mozilla Build Tools installed and configured
See Mozilla's Build Prerequisites for the build tools and their setup.

2.3 Preparing the XULRunner build

Note: For the rest of this walkthrough, we assume that you have checked out the mozilla sources to MOZILLA_HOME

To build XULRunner, you need to create a .mozconfig.mk file in your MOZILLA_HOME folder. The file should have the following contents:
mk_add_options MOZ_CO_PROJECT=xulrunner
mk_add_options MOZ_OBJDIR=@topsrcdir@/obj-xulrunner
ac_add_options --enable-application=xulrunner
#to avoid problems on a non-vista system:
ac_add_options --disable-parental-controls
#Uncomment the following line if you
#don't want to build JavaXPCOM:

#ac_add_options --disable-javaxpcom
It may be best if you replace @topsrcdir@ with your MOZILLA_HOME directory path, since otherwise the built binaries may end up in a different place than you expect.
The .mozconfig.mk from above will make sure that the built binaries are placed in a folder called obj-xulrunner.

2.4 Building XULRunner
To build XULRunner, you will have to run "start-msvc8.bat" from the mozilla build tools. This will open a UNIX-like shell in which you can change directory to your MOZILLA_HOME. E.g. if your MOZILLA_HOME is E:\mozillacheckout, then you cd to: cd /e/mozillacheckout/mozilla
If you checked out without the UNIX-like line endings option, you have to convert the files in the source tree first by entering
python build/win32/mozilla-dos2unix.py
This will dos2unix the source tree.

After that you can start the build process by typing
make -f client.mk build
This can take a while and it will build XULRunner, the SDK, all the libraries and tests including WinEmbed.
The following important locations will be created by the build process:
  1. XULRunner: MOZILLA_HOME/mozilla/obj-xulrunner/dist/bin
  2. XULRunner Headers : MOZILLA_HOME/mozilla/obj-xulrunner/dist/include
  3. XULRunner Libraries: MOZILLA_HOME/mozilla/obj-xulrunner/dist/lib
  4. WinEmbed:MOZILLA_HOME/mozilla/obj-xulrunner/embedding/
    tests/winEmbed
2.5 Testing the XULRunner build with WinEmbed
After XULRunner was successfully built, you can test it with WinEmbed. For that you first have to register XULRunner to your system. For that
  1. open a windows command line shell
  2. cd to your MOZILLA_HOME\mozilla\obj-xulrunner\dist\bin folder
  3. type xulrunner --register-global
After that XULRunner will be registered to your system and the dynamically loaded libraries will be found at startup of WinEmbed.

To start WinEmbed, do as follows:
  1. open a windows command line shell
  2. cd to your MOZILLA_HOME\mozilla\obj-xulrunner\embedding\
    tests\winEmbed folder
  3. execute winEmbed.exe
You should see some output like "You are embedded man" and a window should open up which displays the mozilla home page in the embedded browser.

Note: If you did not register XULRunner properly, the window with the embedded browser will not show up.

3. Visual Studio 2005 Project for WinEmbed
I created a Visual Studio 2005 Solution and Project files that allow you to build and debug WinEmbed. All you have to do is download the first file below and unzip the contents to your MOZILLA_HOME\embedding\tests\winEmbed directory. Then you can open WinEmbedVS2005.sln. Make sure you study the included Readme.txt in detail.

Download Visual Studio 2005 Project Files
Download Visual Studio 2005 Project Files and Sources of WinEmbed


Resources
  • http://www.mozilla.org/projects/embedding/
  • http://developer.mozilla.org/en/docs/VC8_Build_Instructions
  • http://lxr.mozilla.org/seamonkey/
  • http://developer.mozilla.org/en/docs/XPCOM
  • http://www.mozilla.org/projects/embedding/embedapiref/embedapiTOC.html
  • http://www.mozilla.org/projects/xpcom/glue/Component_Reuse.html
  • http://developer.mozilla.org/en/docs/XPCOM_Glue#Using_Mozilla_internal_linkage
  • http://starkravingfinkle.org/blog/2006/10/mozilla-platform-xpcom-in-c/