Wednesday, March 24, 2010

X10 Home Automation using Java

This page describes the installation and usage of the Marmitek / ActiveHome X10 Home Automation System Components as well as the setup required to control those devices from Java.

Table of Contents

  1. History of X10
  2. How X10 Works
  3. Marmitek X10 Home Automation System
    1. CM15Pro PC Interface and Active Home Pro Software
      1. Setting Up Sender and Receiver
    2. Lamp Module LM12
    3. Appliance Module AM12
    4. Motion Sensor MS13
    5. Easy Touch Remote Control ET35
    6. Other Available Modules (not bought yet)
  4. The X10 SDK
    1. C++ Samples
    2. JavaScript Samples
  5. X10 and Java
    1. The Problem Statement
    2. The Solution
    3. Getting the Java Glue Code for Active Home
    4. Using the Generated Sources
  6. A Word On Performance
  7. A Word On The Future
  8. References

History of X10

X10 is a protocol standard for communication between home appliances over powerline that was developed more than 30 years ago in Scotland. The company General Electric, one of the biggest and most influential companies of the US heavily promoted this protocol which lead to a continuing success in North America.

Due to the different voltage system in America and Europe which makes the use of the original X10 protocol impossible in Europe, it has not become as much a success here. However, nowadays, appliances that were designed for X10 can be easily adapted to the European voltage systems and since then X10 - 30 years after its birth - gains more and more approval.

How X10 Works

X10 is a protocol that uses the powerline to transmit signals to home appliances. To address different devices, X10 divides a device address in

  1. a so-called House Code
  2. a Unit Code.

The House Code can assume 16 distinct values and ranges from A to P. The Unit Code can also assume 16 values ranging from 1 to 16. In total that allows X10 to address up to 256 independent devices.

This number, however, is slightly decreased by the fact that X10 powerline signals might have to be separated between different households by using different house codes. For example, if two households share the same powerline circuit, it is potentially possible that one neighbour controls the home of another one. In such a case the House Code of the devices in one of the two households would have to be changed to another value reducing the amount of available House Codes by one which is equal to a reduction of number of overall addressable devices by 16.

It is however also possible to apply specific hardware that is placed in the power circuit between the two homes and which will separate the power circuits in such a way that X10 signals will not traverse this boundary.

Marmitek X10 Home Automation System

Marmitek is a reseller for ActiveHome (www.activehome.com) and sell X10 devices for the European market. The devices have slightly different product codes than the original ones from America, but in principle they are exactly the same. Due to the difference in voltage systems, devices from America first need to be adapted to the European 250 Volts, and thus it usually takes a while until a newly introduced American model is available in Europe.

I purchased a starter kit for X10 Home Automation which consists of the following components.

CM15Pro PC Interface and Active Home Pro Software

The CM15Pro is the successor of the CM11 and provides a computer interface to control and monitor X10 powerline and radio devices. It comes with inbuilt memory for macros and a battery that keeps the memory alive.

The package comes with the Active Home Pro software that allows you to create virtual rooms and customise it with your available X10 appliances. It then lets you control and monitor those devices and create automated processing rules (macros) which you can execute from the PC or download to the CM15APro interface for later execution when your PC is switched off.

The CM15Pro is connected to powerline (at the same time driving the device) and to the computer via a USB cable. By default, the CM15APro is configured to listen and send to devices with a House Code of A, but of course that can be changed.

Setting Up Sender and Receiver

In order to control an X10 device, you always have to make sure that sender and receiver of X10 signals operate on the same House Code. For that to work, you need to adjust the address settings of the receiver appliance as well as that of the sender.

In case of simple appliance module (e.g. the lamp module or appliance module (see below)) this is done by turning the little dials for House and Unit Code to the appropriate settings. For example, turning the House Code dial to A, and Unit Code Dial to 1 gives that module the address of A1. In the CM15Pro, you set the House Code via the Active Home Pro Software that comes with it (Tools-->Hardware Configuration-->Monitored House Code).

Lamp Module LM12

The LM12 is an X10 module for lamps and appliances with little resistive and inductive loads. In other words, do not put your hairdryer or coffee machine into it. The LM12 can be switched on and off and can be dimmed also. Note however, that the lamp you plug into the module has to be dimmable - which is not the case for most of the low voltage halogen lamps that come with an inbuilt voltage transformer.

Appliance Module AM12

The AM12 is an appliance module, i.e. in contrast to the LM12 lamp module it can drive bigger appliances with more resistive and inductive loads (such as TVs, coffee machines or hairdryers). The AM12 can be switched on an off, but cannot be dimmed.

Motion Sensor MS13

The MS13 is a motion sensor for X10. It reacts both on motion as well as transitions from bright to dark and dark to bright (the latter can be disabled). When it detects motion it sends an "on" command to a predefined but configurable X10 address (default is A1). After a configurable time with no motion detected, it will send an "off" command to the same address. Likewise, if a transition from bright to dark is detected (and that feature is active), the MS13 will send an "on" command to the configured address + 1 (i.e. if it is configured to send motion detection to A1, it will send the brightness transition signal to A2). If the light level transitions back to bright, it sends an "off" command again. The MS13 is battery powered and sends X10 signals via a radio link. It cannot be controlled, it acts merely as a sensor.

Easy Touch Remote Control ET35

The ET35 is a universal remote control that can also control X10 appliances. It can send signals both over IR and radio frequency and comes with an inbuilt touch screen. It supports various device types (VCR, DVD, Satellite Receiver, etc.) and vendors (Sony, Philips, etc.). Furthermore it can be used to control X10 appliances remotely and send RF signals to the CM15Pro. The ET35 can be programmed to different devices using code tables and presets or it can learn keys from other remote controls.

Other Available Modules (not bought)

Marmitek offers a variety of other X10 appliances, among them security modules, locks and shutter systems. They also offer wall panels that look quite futuristic and can be used to remote control your X10 appliances from a single place. For a list of available products look at http://www.intellihome.be/english/products and make sure to poll it from time to time as still new products hit the market regularly.

The X10 SDK

X10 is fun the way it is. It becomes much greater fun, however, if you can write your own program for the control of the devices. Suddenly you are free to realise all those ideas that you had the first time you watched Star Trek and they are only a step away. In this chapter I will show what you have to do in order to connect your software to X10 appliances.

One of the best things about Marmitek's X10 devices is that an SDK is available. You can download it from here: http://www.activehomepro.com/sdk/sdk-info.html and it comes with (sparse) documentation and sample code.

The X10 SDK comprises a single DLL (ahscript.dll) with an ActiveXObject inside. The ActiveXObject provides an interface to send and receive X10 commands.

Marmitek provide sample code for

  1. C++
  2. Perl
  3. PHP
  4. JavaScript
  5. Visual Basic
  6. Visual Basic Script

No Java?! No Java! But don't worry, we will come to that soon. The code is rudimentary, as is the SDK itself, but with seemingly very little functionality you can actually do a great variety of really cool things.

The code works, and at least the JavaScript and C++ projects could be opened and edited out of the box.

C++ Samples

The C++ samples are provided with a Visual Studio 6 project, but it can easily be automatically imported into Visual Studio 2005 and works out of the box. I suggest you open AHCMD, which is a project for building a command line tool to send X10 commands.

The command line tool uses the following syntax:

> ahcmd [sendplc | sendrawplc | sendrf | queryplc] <device address> [on | off | dim] 
The commands that end on 'plc' are powerline commands, i.e. the signals are sent via powerline. Those that end on 'rf' are sending signals over radio frequency. The device address is a combination of House and Unit Code, e.g. A1, A2, P12 etc.

Note:
Whether a device can be dimmed or not depends on two factors basically: whether the actual device can be dimmed (e.g. a lamp) and whether the X10 module the device is connected to supports it. It is clear that a coffee machine can hardly be dimmed, and even if it could, the coffee would probably not be better. On the other hand a lamp that is connected to an appliance module like the AM12 will never be dimmed because the module itself does not support it.

There is another C++ sample called AHCVIEW, which is a GUI tool that monitors incoming X10 signals. Feel free to consult the code and learn how you can access X10 devices from a C++ program. The basic idea is pretty simple:
  1. You get the ActiveXObject
  2. You call the SendAction() method and pass in
    1. A String for the command (e.g. sendplc, sendrf, queryplc)
    2. The address of the device (e.g. A1)
    3. The command (e.g. on, off, dim 30)

And that is it.

JavaScript Samples

The JavaScript samples are a little bitch to say the least. The first thing you have to know about them is that they only work in Internet Explorer. The second thing you need to know is that they only work in Internet Explorer, if you allow ActiveX plugin execution. Usually Internet Explorer will pop up a tiny bar underneath its toolbar asking you if you want to allow the execution or not. Do yourself a favour and allow it.

The JavaScript sample you will most likely want to try out first is called "Dynamic Receive" and out of the box it does not seem to work. However, that is due the fact that the JavaScript code is trying to send a command via the radio frequency channel and not via powerline if you press the button. Unless you have a radio receiver module, you will notice that absolutely nothing is happening.

A simple way out of this problem is to open and edit dynamicreceive.html and search for the following lines of code:

    if (Command == 1) {
myForm.text2.value = "Command sent a1 ON";
x10.SendAction( "sendrf", "a1 on" );
} else if (Command == 2) {
myForm.text2.value = "Command sent a1 OFF";
x10.SendAction( "sendrf", "a1 off" );
}
Then you replace it with the following:

    if (Command == 1) {
myForm.text2.value = "Command sent a1 ON";
x10.SendAction( "sendplc", "a1 on" );
} else if (Command == 2) {
myForm.text2.value = "Command sent a1 OFF";
x10.SendAction( "sendplc", "a1 off" );
}
Notice that sendrf was replaced by sendplc.

Pressing the button now should switch your module on, provided that it is configured to be addressed as A1.

Have a look at the JavaScript code and learn how you access X10 devices from JavaScript. Note: Just because you are accessing your X10 devices via JavaScript and from a browser doesn't mean you are able to access them from remote or even the Web! I have seen posts in forums that asked "why can I only control my appliances from the browser on the PC that has Active Home installed?" Well, the answer is, because Internet Explorer loads the ActiveX object from the SDK's DLL directly and sends a command to the devices. No remoting, no Web involved. Also remember that JavaScript is executed on the client side always! If you are looking for ways to control X10 devices from remote or even the Web, you need to do it from PHP, or using Java, as described later.

X10 and Java

One of the most coveted solutions in the X10 world is that of being able to send and receive X10 signals using Java. Forums are full of questions like "How do I access my cool Active Home stuff from Java" and "Is there a Java SDK". Unfortunately, most of those entries lead nowhere and having searched for it myself, I usually ended up on pages that were providing solutions only for older, slightly outdated X10 interfaces (like the CM11) and relied on Java COMM - an API to access the serial port, unfortunately without SUN's support for Windows.

Well, good news: the solution I present here works fine with the new CM15Pro and it is likely to work for even newer devices.

The Problem Statement

The problem I was trying to solve is the following:
  1. I want to have the same functionality in Java as is provided by the SDK
  2. I cannot use the Java COMM approach, since
    1. the latest versions of Java COMM are only for Linux
    2. the CM15Pro is connected via USB and does not provide a serial port

  3. I don't want to re-implement X10 using javax.usb
  4. I want something that is (almost) future proof
I knew of a library called JNA (Java Native Access) which allows access to native libraries (e.g. a DLL) and access the exported functions directly from Java, no JNI involved. JNA is great however, it only works for native libraries that provide functions directly. The Active Home SDK however relies on Microsoft's programming paradigm called COM - Component Object Model.

Knowing that, the problem statement transforms into:
  1. How do I call a COM (ActiveX) object from Java
  2. Does it also support callbacks into my Java code
Well, that looks more like something you can type into Google, right?

The Solution

Knowing what I was looking for, I found a project called Com4J (https://com4j.dev.java.net/) which is simply spoken brilliant! Com4J provides a native wrapper for any COM library you can think of and the Java hooks to access the COM objects right from Java programmes. Com4J comes with a tool you can run over your DLL or OCX file and which creates Java sources for you to interact with whatever COM object is available from the DLL. You need to download and unpack the com4j archive to a local folder and then follow the steps which in the following I will outline in detail.

For the remainder of this section we assume that you have downloaded com4j and unpacked the archive to the folder C:\com4j. Furthermore we assume that you have installed the Active Home Scripting SDK in the folder C:\ahsdk


Getting the Java Glue Code for Active Home

The Active Home SDK uses only a single DLL - ahscript.dll. This is usually located in the bin folder of your SDK installation.

To get the Java glue code you execute the following:

C:\com4j> java -jar tlbimp.jar -o c:\ahsdk\java -p <your java package> -v C:\ahsdk\bin\ahscript.dll
This will inspect ahscript.dll and generate Java source code for accessing the COM objects in the output folder C:\ahsdk\java and in the package you specify.

That's it, really. There is no more to it than that.

For me, the generated sources look as follows (but that may vary depending on your version of ahscript.dll):

_IActiveHomeEvents.java:
package test  ;

import com4j.*;

/**
* _IActiveHomeEvents Interface
*/
@IID("{001000AF-3DEF-0912-10B6-DC5BA692C858}")
public interface _IActiveHomeEvents extends Com4jObject {
/**
* method RecvAction - Called when commands
* have been received
*/
@VTID(3)
@DefaultMethod
void recvAction(
@MarshalAs(NativeType.VARIANT)
java.lang.Object bszAction,
@MarshalAs(NativeType.VARIANT)
java.lang.Object bszParm1,
@MarshalAs(NativeType.VARIANT)
java.lang.Object bszParm2,
@MarshalAs(NativeType.VARIANT)
java.lang.Object bszParm3,
@MarshalAs(NativeType.VARIANT)
java.lang.Object bszParm4,
@MarshalAs(NativeType.VARIANT)
java.lang.Object bszParm5,
@MarshalAs(NativeType.VARIANT)
java.lang.Object bszReserved);

}
_DIActiveHomeEvents.java:
package test.events;

import com4j.*;

/**
* _DIActiveHomeEvents Interface
*/
@IID("{001000AF-3DEF-0911-10B6-DC5BA692C858}")
public abstract class _DIActiveHomeEvents {
/**
* method RecvAction - Called when
* commands have been received
*/
@DISPID(0)
@DefaultMethod
public void recvAction(
java.lang.Object bszAction,
java.lang.Object bszParm1,
java.lang.Object bszParm2,
java.lang.Object bszParm3,
java.lang.Object bszParm4,
java.lang.Object bszParm5,
java.lang.Object bszReserved) {
throw new UnsupportedOperationException();
}
}
IActiveHome.java:
package test  ;

import com4j.*;

/**
* IActiveHome Interface
*/
@IID("{001000AF-3DEF-0910-10B6-DC5BA692C858}")
public interface IActiveHome extends Com4jObject {
/**
* method SendAction - use to send commands
*/
@VTID(7)
@DefaultMethod
@ReturnValue(type=NativeType.VARIANT)
java.lang.Object sendAction(
@MarshalAs(NativeType.VARIANT)
java.lang.Object bszAction,
@MarshalAs(NativeType.VARIANT)
@DefaultValue("0")
java.lang.Object bstrParam,
@MarshalAs(NativeType.VARIANT)
@DefaultValue("0")
java.lang.Object vReserved1,
@MarshalAs(NativeType.VARIANT)
@DefaultValue("0")
java.lang.Object vReserved2);

@VTID(8)
void onRecvAction(
@MarshalAs(NativeType.Dispatch)
com4j.Com4jObject rhs);
}
ClassFactory.java:
package test  ;

import com4j.*;

/**
* Defines methods to create COM objects
*/
public abstract class ClassFactory {
private ClassFactory() {}

/**
* ActiveHome Class
*/
public static test.IActiveHome
createActiveHome() {
return COM4J.createInstance(
test.IActiveHome.class, "{001000AF-
2DEF-0208-10B6-DC5BA692C858}" );
}
}

Using the Generated Sources

Now that we have the sources generated, all we have to do is use them to access the X10 devices from our Java code. To that end, we create a new Java project in our favourite IDE and copy the generated sources over.

I created a simple test, which shows how
  1. you send an X10 signal
  2. you receive an X10 signal
Test.java:
package test;

import test.events._DIActiveHomeEvents;
import com4j.Com4jObject;
import com4j.EventCookie;

public class Test
{
public static void main(String[] args)
{
IActiveHome ah
= ClassFactory.createActiveHome();
ah.sendAction("sendplc", "A1 off", "", "");
EventCookie eventSubscription =
ah.advise(_DIActiveHomeEvents.class,
new EventReceiver());

while(true)
{
try
{
Thread.sleep(15000);
}
catch(Exception e)
{
e.printStackTrace();
}
}

//not necessary here, but usually you should
//not forget to release the event listener:
//eventSubscription.close();

}
}

class EventReceiver extends _DIActiveHomeEvents
{
public void recvAction(
java.lang.Object
bszAction,
java.lang.Object bszParm1,
java.lang.Object bszParm2,
java.lang.Object bszParm3,
java.lang.Object bszParm4,
java.lang.Object bszParm5,
java.lang.Object
bszReserved)
{
System.out.println("Called");
String receptionType = "<null>";
String x10Address = "<null>";
String plcCommand = "<null>";
String additionalData = "<null>";
String rfCommand = "<null>";
String keyData = "<null>";
String timestamp = "<null>";

if(bszAction != null)
System.out.println("action: " +
bszAction.getClass().getName());
if(bszParm1 != null)
System.out.println("1: " +

bszParm1.getClass().getName());
if(bszParm2 != null)
System.out.println("2: " +
bszParm2.getClass().getName());
if(bszParm3 != null)
System.out.println("3: " +
bszParm3.getClass().getName());
if(bszParm4 != null)
System.out.println("4: " +
bszParm4.getClass().getName());
if(bszParm5 != null)
System.out.println("5: " +
bszParm5.getClass().getName());

// if action is recvplc, then the param
// types are as follows:
//1: String -- X10 Address
//2: String -- X10 command
//3: String -- X10 additional data
//4: String -- not used ?
//5: String -- not used ?

// if action is recvrf, then the param types
// are as follows:
//1: java.lang.String -- X10 Address
//2: java.lang.String -- RF Command
//3: java.lang.Integer -- key data
//4: java.util.Date -- timestamp
//5: java.lang.String -- not used ?
}
}
As you can see, sending a command is done by calling
ah.sendAction("sendplc", "A1 off", "", "");
Receiving X10 commands is not much harder. All you have to do is register an event listener. For that purpose you first let your event listener extend the generated class _DIActiveHomeEvents and then override the recvAction method with your own code. You then register this listener by calling
EventCookie eventSubscription = ah.advise(_DIActiveHomeEvents.class, new EventReceiver());
This makes your listener available for callbacks from the native COM object. The EventCookie you get returned serves as a handle to the native event loop if you like, so once you are done listening for events, you simply call
eventSubscription.close()
to free resources.

The recvAction method is called by the SDK whenever an X10 event is received and depending on the type of event, the parameters that are passed along have different types and meanings. Those can be looked up in the SDK's compiled help file ahscript.chm under the topic of "Using the Scripting Interface".

A Word On Performance

Certainly calling from Java into some native wrapper library that then calls into the actual DLL will be possible at the cost of performance. However, given the latency of X10 powerline commands and the fact that we are not trying to do any real-time stuff here, the proposed solution has no performance problems at all.

A Word On The Future

What I presented here allows you to send and receive X10 commands both via powerline and radio frequency using the CM15Pro computer interface and X10 appliances both from ActiveHome and Marmitek. For example, you could write a Java application that receives commands from your X10 remote control, or you could write a Servlet that exposes X10 functionality on a Web server that is remotely accessible with a browser or your mobile phone. The options are endless and only limited by your fantasy. I believe we are going to see much more of the kind in the future, so go ahead and create the future. Today.

The beauty and elegance of the proposed solution is that it should be pretty future proof. I don't know how many future versions of the Active Home SDK are due to come out still, but given the fact that X10 has been around for more than 30 years now, the probability for future updates is quite high. As long as the Active Home SDKs keep on using COM objects and DLLs like the current one, this solution will always work. Given the tools to generate Java source code for existing DLLs, updating will not even take much time.

I hope this is useful to someone, and if you feel it is, let me know, spread the word and if you like drop me a line saying what cool things you implemented. Have fun.

References

http://www.intellihome.be/english/products
https://com4j.dev.java.net/
http://en.wikipedia.org/wiki/X10_%28industry_standard%29
http://www.x10.com/activehomepro/sdk/index.html

14 comments:

Jeff said...

I have visited this site.it give me information about work from home. i like to know more about this site.
This site is very helpful for work from home.


work from home

evision said...

I have gone through this blog.I found it very interesting and helpful.Now a days I am doing my work from my home only.
And this blog is really doing great for me.And got extra income from my home.

work from home

Alex said...

winkler,

Thanks very much. Just got this running with my CM15 without a problem.

One glaringly obvious thing I initially missed when I ran your code was the first command was to turn the light off when I already had the light off.

I thought that your code was broken, when in fact if the command was "A1 on", it would have worked first time.

Would it be much more difficult to expose this service with a UPnP wrapper?

Daniel said...

Hi there, I found your article most interesting, I'm been looking at X10 before but always given up on the fact that the Java support seems to be very limited.

After reading this I'm actually starting to turn the other way and I'd be looking at creating a sort of home automation web app.

Thanks for a great article.

Peter said...

Thanks a lot for the instructions and sample code. I am developing inside Eclipse and deploying to a Geronimo server. My application consists of a couple of JSP files and also some servlets to do the work. My goal is to be able to control my X10 equipment from the web, mainly my iPhone.
My questions are quite simple: Do you need to place the com4j and ahscript dlls somewhere special?
Do the DLLs need to be registered with windows?
I know I need to put the com4j jar files in the server's classpath.
I am getting calssdefnotfound errors at runtime.
Thanks a lot!
...Peter

Simon said...

It worked for me on Windows, but how to setup this working on Unix?

Dallas home automation said...

An automated home is about the convenience of saving your time and effort by having your home automatically do routine functions such as watering your grass (but only if it has not rained recently), or turning off all lights, setting the thermostat to economy mode and arming the security system when you retire for the night.

Anonymous said...

cpan> install win32::process
Warning: Cannot install win32::process, don't know what it is.
Try the command

i /win32::process/

to find objects with matching identifiers.
i tis is ragav... i will get this type of error on run perl

ragav said...

Hi tis is ragav... i will get this type of error on run perl


cpan> install win32::process
Warning: Cannot install win32::process, don't know what it is.
Try the command

i /win32::process/

to find objects with matching identifiers.

Turbidity Meter said...

This is working well. I have implemented it in my new module.

Sherief Mowafey said...

I would like to ask a question..
i had gone through your great article and i have done all the steps to control Lamp Module LM12 using Java..
all steps are done.. the project is working fine.. but there is a slight problem whenever i try to send dimming commands for 2 LM12 module like ah.sendAction("sendplc", "A1 dim 20", "", "");
ah.sendAction("sendplc", "A2 dim 20", "", "");

it dims only the 1st module and ignore the 2nd one..

Do you have any idea why this is happeing??

Anonymous said...

Do we need to install X10 drivers(X10_Drivers.exe)?

Android app developer said...

This is one of the suitable post.I like your blog quality.This is one of the unique post.Thanks for your support.

Tony Walker said...

I recently hired the team at Pure Audio Video to install a home automation system for my elderly parents living in Pompano Beach. Unlike many companies I’ve met, they are licensed and insured. They took the time to understand my parent’s lifestyle and what they needed before recommending the appropriate system, and their installation team was courteous and professional. Now I have peace of mind knowing that my loved ones are comfortable and safe. Contact them today at www.pureaudiovideo.com or (954)-581-4434.