Homey - Home Automation

Home automation project using X10 devices and Raspberry PI

Home automation, or, simply said, controlling your home appliances automatically, or via web is a very trendy topic nowadays. Smart bulbs, WIFI-enabled washing machines are just few of the gadgets that are being connected to the World Wide Web. They call it IOT, internet of things.

What only a few people know is that this is not something that appeared nowadays. Controlling your devices remotely was already possible years ago, and maybe it was implemented better. Why? We will see that later.

Also one note here, this tutorial assumes that you have at least some basic Linux knowledge. This means you know how to connect via SSH, how to copy, move and manipulate files generally, change the ownership or permissions.

Also you need to know how to install software packages. If you do not have this knowledge, you can still try to go through this tutorial, however, you will have to search Google for additional information.

Although nowadays getting more and more deprecated, way before wifi-enabled devices became main-stream, there was a well-defined protocol meant to be used for home appliance control: X10

The main benefit of this protocol, compared to the currently existing ones was that the transmission of the information was done through the power line wiring, not via radio signals (at least not necessarily).

This added a great flexibility and reliability to the whole system, while keeping the installation costs at minimum.

There were various devices made that supported the so called x10 protocol, from simple relays (that could turn on or off devices) to lamp control modules (supported dimming of incandescent bulbs), alarms etc.

All devices were controlled from a base station that was connected to a PC (via Serial port , see CM11A devices from Marmitek, lately rebranded to Hailbrain) or later on via USB (Cm15Pro, also from the same company).

http://www.haibrain.com/domotica-haibrain-voorheen...

The protocol itself is quite simple, and on Wikipedia you can find a good description:

https://en.wikipedia.org/wiki/X10_(industry_standa...

Citing from Wikipedia:

"

Household electrical wiring which powers lights and appliances is used to send digital data between X10 devices. This data is encoded onto a 120 kHz carrier which is transmitted as bursts during the relatively quiet zero crossings of the 50 or 60 Hz AC alternating current waveform. One bit is transmitted at each zero crossing.[3]
The digital data consists of an address and a command sent from a controller to a controlled device. More advanced controllers can also query equally advanced devices to respond with their status. This status may be as simple as "off" or "on", or the current dimming level, or even the temperature or other sensor reading. Devices usually plug into the wall where a lamp, television, or other household appliance plugs in; however some built-in controllers are also available for wall switches and ceiling fixtures.

The relatively high-frequency carrier wave carrying the signal cannot pass through a power transformer or across the phases of a multiphase system. For split phase systems, the signal can be passively coupled from phase-to-phase using a passive capacitor, but for three phase systems or where the capacitor provides insufficient coupling, an active X10 repeater can be used. To allow signals to be coupled across phases and still match each phase's zero crossing point, each bit is transmitted three times in each half cycle, offset by 1/6 cycle.

It may also be desirable to block X10 signals from leaving the local area so, for example, the X10 controls in one house do not interfere with the X10 controls in a neighboring house. In this situation, inductive filters can be used to attenuate the X10 signals coming into or going out of the local area.

"

The addressing of the devices consists of setting 2 switches: one of the house code ('A' - 'P') and a unit code ('1' - '16').

The unique address is determined by both 'parameters'.

The reason of being a house and the unit code separately is that you can group your devices on separate areas ,and the protocol supports batch command (all units on, all units off) that will let you turn on or off all of your devices with one command that belong to the same house code.

Of course, you can address each device separately by specifying it's address (e.g. 'A1').

The protocol is quite simple, as already stated, and you can find it on wiki, but I'll also include it here, for convenience. See the attached table.

Homey, as this project was called was meant to allow remote control of home appliances via Web interface, but also locally.

The setup is the following:

There is a web server running on an Raspberry PI single board computer (instead of a PC that was used in the past when there were no such tiny devices) to which we connect a CM15Pro X10 device via USB. This will transmit the commands issued by the PI via the power line to the receivers, which on our case are AM12G X10 enabled relays. To these relays you can connect any home appliance that will be turned on or off, based on the commands issued by the PI.

• Raspberry PI Single board computer (any variant will do it, as long as it has Ethernet capability)

• Piface board (used as IO to allow in-location management) or Adafruit LCD shield (http://www.adafruit.com/products/714) ,
• Marmitek CM11A (or the newer one CM15Pro) X10 <-> serial/USB transceiver
• Marmitek AM12 remote controlled relay
• USB <-> Serial Adapter if CM11A is used instead of CM15Pro. The latter one has already built-in USB <-> Serial interface

The Raspberry PI doesn't require any presentation, it's the most widely used barebone computer nowadays. Has great community support, many addon-boards and features several Linux based distributions.

The PIface board is and addon-on daughter card that can be easily fitted on top of the PI via the 40-pin header. Features several relays, outputs that can drive small loads via transistors, 8 LED-s and some other features.

This board was meant to be used as a Human Interface to allow control of the X10 devices not only from a web browser. The reason why I chose this was that I already had it and was not used for anything else.

The buttons are used to select which of the devices (as there are only 8 LEDs, you could control only 8 devices with it) should be turned on or off and the 8 LEDs were used as status indicators.

Of course, a better board would be the Adafruit addon-on board, which features an 16x2 alphanumerical display, 5 buttons. With this board we can display much more information than with a PIface. Unfortunately, the SW was never written to it, due to lack of time, but all of the sources are published on Githuh (you will find the link later on), so feel free to update the SW to support his.

The CM11A serial-powerline adapter is a very old product, already deprected. This needs an USB-Serial connector supported by Linux (any with the PL2303 or with FT232RL will do it)

If you are buying, then the CM15Pro is a better choice, especially that it has also RF support , so you can use a remote control like the one in the picture .

The most popular one is Raspbian, which is a debian-based distro,
therefore you can use the APT package manager to install packages in the same manner as you would do it on an Ubuntu machine.

Installing Raspbian is quite easy, there are various tutorials on the internet regarding this. For more info, you can check the official website:

https://www.raspberrypi.org/documentation/installa...

As you won't need any Graphical user interface, I would recommend the minimal install.

If you are not familiar with Linux, connecting via SSH and controlling your device from there will be difficult at the beginning, so I would recommend to familiarize yourself with Linux before proceeding.

Having a raspbian up and running should be quite easy for those who already worked with Linux before.

As the control of the devices will be done also via web browser, a web server needs to be also installed.

The webpage is quite simple, uses html and php and mysql, calles bash scripts to communicate with the CM15Pro.

Therefore, the following SW packages must be installed:

• lighttpd with php support
• mysql server
• phpmyadmin, for better database management.

A good tutorial can be found on the following link:

http://www.penguintutor.com/linux/light-webserver

that explains also how to configure your router, to make your webserver accessible also from outside of your network.

If you are not into reading long explanations, the following commands will do the trick and install all of the necessary packages:

Installing lighttpd:

sudo apt-get install lighttpd

Install mysql database:

sudo apt-get install mysql-server

Install PHP support:

sudo apt-get install php5-common php5-cgi php5

sudo apt-get install php5-mysql

sudo lighty-enable-mod fastcgi-php

sudo systemctl restart lighttpd if your system uses systemd (most of the already migrated to it) or

sudo service lighttpd force-reload, if you are still using init daemon

Set permissions on the web directory /var/www/

Change the directory owner and group

sudo chown www-data:www-data /var/www

allow the group to write to the directory

sudo chmod 775 /var/www

Add the pi user to the www-data group

sudo usermod -a -G www-data pi

Note: You should run the last commands also after you copy the files to the /var/www/html folder containing the web gui for Homey.

Install phpmyadmin:

The link below contains a good turorial on how to configure phpmyadmin. Just make sure you select lighttpd on the blue panel when prompted instead of apache (as we are using this lightweight web server instead of the more resource hungry apache) and skip the step that tells about configuring apache. The install script will already configure it to be used with lighttpd:

https://www.stewright.me/2012/09/tutorial-install-...

In a nutshell, a

sudo apt-get install phpmyadmin

will install phpmyadmin for you.

This is a web gui that allows you to manage your mysql database server. It is widely used nowadays, so you will find a lot of tutorials regarding how to use this piece of SW.

Once you are ready with the web server installation, you can access it via

http://[IP_ADDRESS_OF_THE_PI]:80

Of course, replace it with your correct IP address.

Note: By default, as any web server does, it will listen on port 80. So if you need access from outside of your network, you will have to forward this port towards your internal IP address.

This configuration is beyond our purpose here, and it won't be described in detail, but you will have to do the following:

- Configure the PI to have always the same internal address allocated (this can be done from the router, if it supports static ip adresses based on MAC addresses)

OR

- Configure the PI to use a static address. (I would prefer the first variant, as then you can carry your device and connect it to another network without having to reconfigure it). But this means that your router must be capable of issue-ing static IP addresses based on the MAC addresses. If you are running one of the custom FWs on your router, like OpenWRT, DDWRT, or Tomato, then it will support it for sure.

- If you don't have a fix public IP address (you most probably don't have, as ISP don't do that unless you are a company and pay extra), you will have to use a Dynamic DNS service. Dyndns is a good choise, however is not quite free. This again depends on your router, which of the DDNS services are supported. If neither, you can always install inadyn on your Raspberry Pi and let it handle the IP address update. Then you could use http://freedns.afraig.org for example.

I won't go into much details on this topic, you can find various tutorials on the internet.

The Homey project can be found on Github under the following link:

https://github.com/trex2000/Homey

You can clone it to your local repository by issuing the following command:

sudo git clone https://github.com/trex2000/Homey

If you do not have git installed, then do it so prior to cloning the repo with the following command:

sudo apt-get install git

This will sync the whole project into the local folder. Make sure you change the directory first, as otherwise you will most probably be in /home/pi home-directory, if you have used a raspberrian distro.

The structure of the project is the following:

1. Linux App Source:This folder contains the source files (written in C) of the small daemon that controls the Piface addon board. It also syncronizes with the database, so if you modify the status of one of the relays, the LEDs on the Piface board will also light accordingly.
2. Webpage This folder contains the html and php files that implements the web page used to control your home appliances from a web browser.
3. heyu: this is a small utility used to control the CM11A device. The CM15Pro requires another tool, mochad, so install one of them only, based on what device you have. Please note that the git repo has been already updated shell scripts in the Webpage folder to use the mochad to communicate with the CM15Pro. Use an older version of the file if you still own an CM11A.
4. mochad: this is also a small utility, used to communicate with the CM15Pro. The difference between this one and heyu is that the former one is a command line utility, while the second one is a a Linux TCP gateway daemon. So in order to issue commands, you will need to send it via TCP, using nc for example. See the
   

   execheyu.sh script from Webpages folder, as an example.

5. root of the folder: During development, eclipse was used on a separate PC to write the C souce code. Source code was placed on the PI itself and using a samba share, it was also mounted into a windows machine as a shared network drive. The SW was built on the Raspberry itself and to automate this process, the
   

   build.bat batch file was created. This basically connects with the help of the plink utility (part of the putty package) to the raspberry via SSH, executes the commands and then returns the status into the window. Eclipse can be configured to run an external script file when pressing build, but this is not part of this totorial. Bsaically you can configure the IDE in such a way, that pressing the build button will launch the build remotely on the Raspberry and return the output right into the command window from eclipse. If you do not want to use a separate editor, then you can edit the source code directly on the PI and use the included make file to build it (a simple make will do it).

In order to communicate with the CM15Pro (we will assume you will be using this one, not the CM11A), you will have to install mochad utility.

This is well described on the following link:

http://x10linux.blogspot.ro/2012/08/installing-moc...

but bear in mind that you will most probably have to install at least the basic build package before, as you will need it anyway for building the C application

To do this, issue the following command

sudo apt-get install build-essential

If for some reason the link from above doesn't work, then here are the steps copy-pasted from the page from above:

sudo apt-get install libusb-1.0-0-dev

wget -O mochad.tgz http://sourceforge.net/projects/mochad/files/late...

tar xf mochad.tgz

Now build mochad:

cd mochad*

./configure

make

sudo make install

This will configure, build and install the mochad daemon and also place an UDEV rule, so that each time the CM15Pro is plugged in, it will start the daemon, if not already started.

However, it seems that at boot, this daemon is not started. Therefore, a service file was created (mochad.service, see the location of the file above)

that needs to be placed in the /lib/systemd/system/mochad.service location.

In oder to start the daemon automatically, issue the following command:

sudo systemctl enable mochad.service

If you've cloned the repository, inside the main folder you will find the Webpages folder. Copyt it's conntent to /var/www/html folder, then make sure it has the right user rights:

sudo chown -R www-data:www-data /var/www

Also grant exec right to the 2 .sh bash scripts:

sudo chmod -R 755 /var/www/html/*.sh

There is config file (config.inc.php) in which you will need to set the connection credentials to the database.

Make sure you modify this file to suit your needs.

The webpage used an SQL database (in our case we will use mysql), therefore the database needs to be created and the tables shall be populated before the webserver will work.

To do this, phpmyadmin will be used.

Connect to the database via phpmyadmin:

http://[YOUR_LOCAL_IP_OF_THE_PI]/phpmyadmin

and login with the credentials you've provided during installation.

Once connected, create a database (homey should be fine) and then import the content of the sql file located in the sql folder you've cloned from the GIT repo

The database will contain 2 tables: devices and logs.

The first one contains the configured available devices.

Here, for each of your X10 device, you can provide a name, unit and housecode. At each change, the last modification time and the last known state is also stored.

It's important to note here that as some of the X10 devices do not support status report, there is no way to know if the state of the X10 device is ON or OFF. Therefore this information is based on the last issued command by the Raspberry.
In case you use also an RF remote (if you have an CM15Pro) then it's possible to alter the state of the X10 device via RF, but then the known state will be different.

In case the X10 device supports reporting of it's state, then set the appropriate value to '1' when configuring the device.

The log folder is what it's name says, contains a journal of the issued commands (date/time of change and the state together with the device id).

Once all set up, you should be able to access the webpage:

http://[YOUR_LOCAL_IP_OF_THE_PI]

The page was written in php and it's quite simple. There is an index.php file that contains almost all of the implementation while the config.inc.php contains just the settings for the database connection.

There are 2 bash scripts ( execheyu.sh and heyustatus.sh) The webpage will call the first one with the respective parameters (house and unit code of the device and the newly requested status). This script will connect via netcat to mochad daemon and transmit the needed information.

heyustatus.sh is called to return the status of the X10 devices that support status report. This was not tested, as such a device was not available when this project was created.

The php script will querry the database for available devices and it's last state and will display them (as seen in the screenshot). Once you modify the status, the php script will update the database to reflect the latest status (but first it will check if at least 5s has elapsed since the last request, to allow fast toggling of the relay - this can have a negative impact on the devices that are connected to the X10 relays).

Password protecting the webpage

By default, anyone who has the link can access the page and modify the state of the X10 devices freely. This is not desired, therefore some kind of authentication is needed.

The most simple is basic auth, provided by the http server. A good explanation can be found here:

http://www.cyberciti.biz/tips/lighttpd-setup-a-password-protected-directory-directories.html

Again, in case the link becomes obsolete, here's the extract from the webpage above:

Step #1: Open /etc/lighttpd/lighttpd.conf file

Add the folllowing line, but check if it's not yet available.:

server.modules += ( "mod_auth" )

Now add following three directives:

auth.debug = 2
auth.backend = "plain"

auth.backend.plain.userfile = "/etc/lighttpd/.htpasswd"

Next, you need specify which directory you want to password protect. We want to protect the whole domain, so this means that we need to specify that everything inside the /var/www/ shall be protected:

auth.require = ( "/" =>

(

"method" => "basic",
"realm" => "Password protected area",

"require" => "user=vivek"

)

)

Step # 2: Create a password file

vi /etc/lighttpd/.htpasswd

and add the user and password in the following format:

vivek:mysecretepassword

There is a resident application written in ANSI C that will handle the Human Interface part: status display with the 8 led's from the Piface and the 5 buttons.

The app will also update the database if the status is changed via the buttons, or update the status of the LED's if the state of the X10 devices are changed via the Web interface (it will be notified via shared memory mechanism).

To be able to build the application, some additional SW packages needs to be installed first:

WiringPI

The source code can be downloaded from the following link:

http://wiringpi.com/download-and-install/

Citing from the webpage:

"WiringPi is a GPIO access library written in C for the BCM2835 used in the Raspberry Pi. "

The complete installation steps are provided on the link from above, but in case it becomes inaccessible, in a nutshell the following steps are needed:

sudo apt-get install git-core

git clone git://git.drogon.net/wiringPi

(this will clone the source to the current folder. Make sure to change it to suit your needs).

cd wiringPi

./build

Normally no additional packages are needed at this step and the sw shall be build without errors.

Mysql devel libraries

To install the necessary header files needed by the C app, the following packages shall be also installed:

sudo apt-get install libmysqlclient-dev libmysqld-dev

Enable SPI for PIface board

The Piface board uses SPI interface for communication with the PI. In oder to use it, first the SPI kernel modules shall be loaded.

There are different methods available based on the distribution you are using, but for Raspbian the following commands shall be fine:

sudo apt-get install raspi-config

(if not already available)

Now start raspi-config by typing
sudo raspi-config

To view the list of advanced configuration options, select Option 8 Advanced Options.

Choose the A5 SPI option. Set this to "Yes"

Select "Ok" then "Finish"

More information can be found here:

http://www.piface.org.uk/guides/Install_PiFace_Software/Enabling_SPI/

Building Homey

In order to build the application, go into the

Linux App Source

folder where you've cloned the Homey repository

and issue the make command.

If you have installed everything correctly, then there shall be a homey binary inside the folder with execution rights.

The application can be started with ./homey.

In case you need automatic start, based on the linux distrubution that is being used, a .service file or an init script shall be created and placed in the correct folder.

There are various tutorials based on whether you are using systemd or still init , therefore this won't be discussed in this tutorial.

In case the SW is working, a LED connected to GPIO23 should begin to pulse slowly

This chapter is not necessary if you just want to use the provided application. In case you are a develper, and you're interested in further developing this application, perhaps change the PiFace board to and alphanumerical LCD (like the proposed Adafruit board) then the informations provided here might come in handy.

The application is split in several C files based on the functionalities they contain:

• main
• homey

• out
• shmem
• db

The main file contains a simple 'task scheduler' that will call some functions cyclically every X milliseconds.

Of course, Linux is not a realtime operating system, therefore a very precise cyclicity cannot be guaranteed, but is should provide a good time base for our needs.

The main function also calls some init functions (InitBoard - to initialize the HW peripherials, InitSW - to initialize the shared memory, database connection etc.) It will also call a cleanup function once the application receives a SIGINT signal from the OS. This is needed to properly clear any dynamically allocated memory, close database connections, destroy the shared memory, etc, etc, before exiting the application.

The db file contains the functions responsible with manipulating the MySQL database. It is always executed in a separate thread to avoid blocking the main thread for long time in case of slow database connection. It's always better to use threads in case of a TCP/IP communication, as you can never rely on the response time of the server. And your application will be more responsive this way.

The out file contains the functions that manages the I/O system of the Piface. It reads the buttons, debounces them, sets the LEDs based on the X10 device statuses and also provides blinking via the SW PWM driver of the WIring PI library (to mark the currently selected device when navigating with the buttons)

The shmem file contains the functions that create the shared memory that will be used to communicate with the webserver. When an X10 device status is updated via the webpage, the webserver will notify through the shared memory the application that there has been an update.

Afterwards, the application will querry the database and update the status of the 8 LEDs accordingly.

Another implementation could have been to periodically query the DB for changes, but this solutions was found to be a better one at the time of the design.

The homey file contains just one function, could be removed and the function moved to main.c

For each C file there is a [FILENAME].h and an [FILENAME]-EXT.h header file created. The first one is used for the prototypes and variables that are used locally in the module only, while the -EXT.h header files shall contain the exported functions and variables.

As the SW used multiple threads, semaphores were also used to solve the resource racing problem.

The project created here is working successfully for a while.

The X10 protocol is a slow, but reliable protocol and can be successfully used also nowadays for home automation purposes.

The X10 relays are quite expensive (20$-30$) as well, while the CM15Pro costs around 50$... therefore a cheap, Chinese-made wifi-enabled smart switch could be a better solution. But in case radio communication is not an option, then using the power line as communication medium is a good alternative.

A drawback of the AM12G relays that were used in this project is that the do not provide status. This means, you cannot tell if the device was activated (or deactivated) successfully.

The provided application uses the Piface board. This board was not meant to be used as an Human Interface Device, therefore an LCD would be a better approach and could be implemented with minimal effort.

There is no need to implement the whole communication.

The lcdproc daemon already supports many LCD add-on boards available today and provides an easy way to display information on alphanumerical displays.

https://sourceforge.net/projects/lcdproc/

There are various tutorials about building, installing lcdproc. (There are even binaries built for the PI)

This is the end of the tutorial. I hope it will be useful for somebody, as it was also useful for me.

By managing your household appliances while you are away from home, you can not only spare money by turning on your devices remotely only when needed, but also provides you an easy way to 'reset' one of the 'frozen' devices, be that a computer, satellite receiver, media center, server, etc.

I would like to say special thanks to the Raspberry foundation for making such a great board, to Gordon, for writing such a great library (WiringPi) and also for the rest of the people involved in the libraries and sw packages used in this project.