Voice Controlled Smart Home

Hi everyone, The Internet of Voice has come and now you can monitor and control all various devices in your home by using your voice, Amazing!!! Amazon Alexa and Raspberry Pi made the thing supper easy and now every hobbyist can enter in the era of hands-free home control using Alexa Voice Service.

Project Demo

Getting started with a voice controlled smart home is easier than you might think, and it won’t cost you much money either. You just need a Raspberry Pi with Alexa Voice Service. In this project I will give you complete instruction to convert the dream to reality. I made the project to control 4 appliances and I used a Raspberry Pi to do this. Single Raspberry Pi is used for dual purpose. It is serving as a Alexa Voice Client and Paho MQTT client at the same time. You may use Amazon ECHO device for Alexa Voice Service. If you do so then you can escape first two steps (STEP 1 & STEP 2). For voice interface you will be required Alexa custom voice skill. Don't worried, making Alexa custom voice skill is very easy.

I hope you already watched demo video of my project. If you like it let's start making it. For easy understanding I divided full project documentation into several steps. Follow step by step, I think you will enjoy it. If you have Amazon Echo device and want to use that ignore first two steps.

All necessary files & source code for the project are available here.

Task involved in this project:

1. To run the system you will be required an Alexa Voice Enable device like Amazon Echo or Echo Dot. But good news is that your Raspberry Pi can be used as Amazon Echo! In first two steps I will show you how to setup Raspberry Pi as Alexa Voice Client (Amazon Echo).

2. You will be required a custom voice skill which can understand your own voice command to do a specific task. In step 7 I will explain how to prepare your own Alexa custom voice skill. To make a custom voice skill Lambda is required. Lambda is a platform which can store and run your own code from server. You just need to upload your source code and dependencies. The source code run by lambda is called Lambda function. A Lambda function can be trigger from different source like Alexa Voice Service in our case. In step 6 & 8 I will show you how you can make a Lambda function.

3. To control any device from remote location IoT cloud service is required. I used AWS IoT for my project and in step 5 I will show you how to prepare AWS IoT. Any other IoT cloud service support MQTT with SSL may be used for the project. According to your voice command defined in your skill Lambda function will be executed and send a specific message into IoT cloud.

4. You need to receive message containing control information for your device from the IoT cloud. AWS IoT support MQTT protocol and I used this for my project. To communicate with IoT cloud using MQTT protocol a MQTT client is required. Cloud works as MQTT broker. In step 9 I will show you how you can set up an MQTT client program into your Raspberry Pi so that Pi can communicate and receive MQTT message from AWS IoT server.

You can get a clear understanding how the system works from the following figure.

5. Your hardware and devices must be connected to this Pi installed with MQTT client program. When any new MQTT message is arrived, it is received by the client Pi and control the devices according to the receive message and a python program developed for the system.

1. Amazon Echo or Echo Dot (Amazon.com)

2. Raspberry Pi 2 or 3 with Raspbian OS (Seeedstudio)

3. Raspberry Pi compatible WiFi module (adafruit.com)

3. Relay Module (Seeedstudio)

4. Light & Fan

5. Wires

1. Amazon Alexa Alexa Skills Kit

2. Amazon Alexa Voice Service

3. Amazon Web Services AWS IoT

4. Amazon Web Services AWS Lambda

5. Node.js & Python

6. MQTT

What is the Alexa Skills Kit?

Alexa, the voice service that powers Amazon Echo, provides capabilities, or skills, that enable customers to interact with devices in a more intuitive way using voice. Examples of skills include the ability to play music, answer general questions, set an alarm or timer, and more.
The Alexa Skills Kit is a collection of self-service APIs, tools, documentation and code samples that make it fast and easy for you to add skills to Alexa. You can also use the Smart Home Skill API, a new addition to the Alexa Skills Kit, to easily teach Alexa how to control your cloud-controlled lighting and thermostat devices. All of the code runs in the cloud and nothing is on any user device.

Alexa Voice Service

he Alexa Voice Service (AVS) allows you to integrate Alexa’s built-in voice capabilities into your connected products. With AVS, your users will have the ability to play music, request the weather forecast and local news, get updates on traffic conditions, ask general knowledge questions, set timers and alarms, query Wikipedia and much more, the same way they would with an Amazon Echo. You also have access to third-party skills developed using the Alexa Skills Kit (ASK). Using AVS and ASK together, you can build sophisticated interactions to extend the capabilities of your Alexa-enabled product.

Use the Alexa Voice Service (AVS) to add intelligent voice control to any connected product that has a microphone and speaker. Your customers will be able to ask Alexa to play music, answer questions, get news and local information, control smart home products, and more on their voice-enabled products.

Getting Started with the Alexa Voice Service

Amazon Web Services AWS IoT

AWS IoT (Amazon internet of things) is an Amazon Web Services platform that collects and analyzes data from internet-connected devices and sensors and connects that data to AWS cloud applications. AWS IoT can collect data from billions of devices and connect them to endpoints for other AWS tools and services, allowing a developer to tie that data into an application.

An AWS user accesses AWS IoT with the AWS Management Console, software development kits (SDKs) or the AWS Command Line Interface. An application accesses the service through AWS SDKs. AWS IoT APIs are divided into the control plane, which includes service configuration, deviceREGISTRATION and logging; and the data plane, which includes data ingestion.

The IoT service includes a Rules Engine feature that enables an AWS customer to continuously ingest, filter, process and route data that is streamed from connected devices. A developer can configure rules in a syntax that's similar to SQL to transform and organize data. The feature also allows the user to configure how data interacts with other big data and automation services, such as AWS Lambda, Amazon Kinesis, Amazon Machine Learning, Amazon DynamoDB and Amazon Elasticsearch Service. Each rule consists of an SQL statement and an action list that defines and executes the rule using an editable JSON-based schema.

AWS IoT supports HTTP, MQTT and WebSockets communication protocols between connected devices and cloud apps through the Device Gateway, which provides secure two-way communication while limiting latency. The Device Gateway scales automatically, removing the need for an enterprise to provision and manage servers for a pub/sub messaging system, which allows clients to publish and receive messages from one another.

What Is AWS Lambda?

AWS Lambda is a compute service that lets you run code without provisioning or managing servers. AWS Lambda executes your code only when needed and scales automatically, from a few requests per day to thousands per second. You pay only for the compute time you consume - there is no charge when your code is not running. With AWS Lambda, you can run code for virtually any type of application or backend service - all with zero administration. AWS Lambda runs your code on a high-availability compute infrastructure and performs all of the administration of the compute resources, including server and operating system maintenance, capacity provisioning and automatic scaling, code monitoring and logging. All you need to do is supply your code in one of the languages that AWS Lambda supports (currently Node.js, Java, C# and Python).

You can use AWS Lambda to run your code in response to events, such as changes to data in an Amazon S3 bucket or an Amazon DynamoDB table; to run your code in response to HTTP requests using Amazon API Gateway; or invoke your code using API calls made using AWS SDKs. With these capabilities, you can use Lambda to easily build data processing triggers for AWS services like Amazon S3 and Amazon DynamoDB process streaming data stored in Amazon Kinesis, or create your own back end that operates at AWS scale, performance, and security.

What is Node.js?

Node.js is a software platform for scalable server-side and networking applications. Node.js applications are written in JavaScript and can be run within the Node.js runtime on Mac OS X, Windows, and Linux without changes.
Node.js applications are designed to maximize throughput and efficiency, using non-blocking I/O and asynchronous events. Node.js applications run single-threaded, although Node.js uses multiple threads for file and network events. Node.js is commonly used for real-time applications due to its asynchronous nature. Node.js internally uses the Google V8 JavaScript engine to execute code; a large percentage of the basic modules are written in JavaScript. Node.js contains a built-in, asynchronous I/O library for file, socket, and HTTP communication. The HTTP and socket support allows Node.js to act as a web server without additional software such as Apache.

What is MQTT?

MQTT stands for MQ Telemetry Transport. It is a publish/subscribe, extremely simple and lightweight messaging protocol, designed for constrained devices and low-bandwidth, high-latency or unreliable networks. The design principles are to minimise network bandwidth and device resource requirements whilst also attempting to ensure reliability and some degree of assurance of delivery. These principles also turn out to make the protocol ideal of the emerging “machine-to-machine” (M2M) or “Internet of Things” world of connected devices, and for mobile applications where bandwidth and battery power are at a premium.

MQTT allows devices to send (publish) information about a given topic to a server that functions as an MQTT message broker. The broker then pushes the information out to those clients that have previously subscribed to the client's topic. To a human, a topic looks like a hierarchial file path. Clients can subscribe to a specific level of a topic's hierarchy or use a wildcard character to subscribe to multiple levels.

MQTT is a good choice for wireless networks that experience varying levels of latency due to occasional bandwidth constraints or unreliable connections. Should the connection from a subscribing client to the broker get broken, the broker will buffer messages and push them out to the subscriber when it is back online. Should the connection from the publishing client to the broker be disconnected without notice, the broker can close the connection and send subscribers a cached message with instructions from the publisher.

To know more about MQTT take a look to:

http://www.hivemq.com/ & http://internetofthingsagenda.techtarget.com/defin...

If you are not already enrolled, setup an Amazon Developer Account which will allow the development of Alexa Voice Service and many more. Follow the link: https://developer.amazon.com/home.html

1. Log in to Amazon Developer Console and get started with Alexa Voice Service.

2. Click to Register a Product Type and select Device.

3. Give a Device Type ID and Display Name and then click to Next.

4. Select Create a new profile from the drop down.

5. Choose a name for the Security profile and give a description and then click to Next.

6. Your Security profile is ready. Click to Next.

7. Click Web Setting tab and configure it like the picture below.

8. Click to Add Another and add the IP address of your Raspberry Pi.

9. Put necessary information and click to Next.

10. Select No and Click to Submit.

10. You are now successfully registered to Alexa Voice Service.

Full instructions to make your Pi into an Alexa Client is available in AVS official Github page (https://github.com/amzn/alexa-avs-raspberry-pi) but I found it more easy to Sam Machin Github page (https://github.com/sammachin/AlexaPi).

1. Clone the Github repo to your Pi typeing the following command into terminal:

sudo git clone https://github.com/sammachin/AlexaPi.git

2. Go to AlexaPi directory and Run the setup script:

 ./setup.sh

3. Wait for a while to ask your security profile information. Enter all information (product ID, Security profile ID, Client ID etc.) from the security profile of AVS when asked.

4. Open your browser and type your Pi's IP address with port 5000 e.g. (http://192.168.1.104:5000). Just replace IP with your own.
You'll be taken to a Login with Amazon web page. Enter your Amazon credentials.

5. You'll be taken to a Dev Authorization page, confirming that you’d like your device to access the Security Profile created earlier.
Click Okay. You will see following response into browser

and like following to the terminal.

Congratulation!!! Your Raspberry Pi is now ready to work as Amazon ECO. You only need to connect a USB microphone to the USB port of Raspberry Pi and a button to GPIO 18 to talk to it.

All services in AWS are delivered via a rich set of REST APIs. You can use a service programmatically through the APIs by leveraging one of the many SDKs. You can use AWS cli. Alternatively, you can invoke services through the web-based AWS Management Console. To build my AWS environment, we will invoke AWS Management console. But keep in mind that every step we perform today can be done completely programmatically, which is one of the many reasons why the AWS cloud is so powerful. Follow the steps:

1. Create an AWS account from the link: http://aws.amazon.com/

2. Sign in to AWS Management console and from the Services select AWS IoT (Fig-1).

3. Click on Get Started (Fig-2)

4. Click on Create a thing

5. Type a thing name (e.g. raspberry) & click to Create

6. A thing named as raspberry will be created (Fig-5)

7. Now, click to Create a certificate & then click 1-Click certificate create (Fig-6)

8. Download all the keys & certificates, it will be required for connection with AWS IoT (Fig-7)

9. To activate already created certificate, select particular certificate and then Activate it from the Actions menu (Fig-8).

10. Now, you need to attach the thing to the certificate. To do it select the certificate and then Attach a thing from the Actions menu (Fig-9).

11. Type the thing name and click attach (Fig-10).

12. You can see details of thing on the right side as in screenshot by clicking on the thing name.

AWS IoT is now ready for our project. Now, it is the time to connect Raspberry Pi to AWS IoT.

1. Log in to AWS Management Console and from the Service select Lambda.

2. Click to Create a Lambda function.

3. From the bottom right corner click to Skip.

4. Click the rectangle box to choose trigger source and select Alexa Skills Kit.

5. Then click to next.

6. Give a name and select Node.js 4.3 as Run time. Description is optional. From Lambda function code section select Upload a .ZIP file for Code entry type dropdown.

7. Upload a dummy zip file to successfully crate the Lambda function. We will upload our final code later.

8. From Lambda function handler and role section choose lambda_basic_execution from Existing role.

9. Then click Next.

10. Finally click to Create function.

Note the ARN, it will be required later during the creation of Alexa Skill.

1. From Amazon Developer Console click to Get Started from Alexa Skills Kit.

2. From top-right corner click to Add a New Skill.

3. Select Custom Interaction Model and give a Name and Invocation Name and than click Next.

4. From the code section copy the JSON file from Intent Schema and pest it to Intent Schema section.

5. Do the same for the Sample Utterances and click to Next.

6. Select Lambda ARN as Endpoint and paste the ARN code you got at time of Lambda function creation to the textbox.

You can test your skill from Test tab but as we did not upload lambda code so we will be able to do this after next step.

After successfully testing your skill you may submit it for certification. To do that click to Publishing Information and follow the instructions.

Previously we just created a lambda function but we didn't upload or write any code for the lambda. In this step, I will show how to prepare your code and dependencies and upload it to lambda. For preparing code I used Raspberry Pi.

1. Install node.js to your Raspberry Pi (lots of tutorials are available to Internet).

2. Install npm to your raspberry pi (lots of tutorials are available to Internet).

3. Make a directory to your Pi:

mkdir awssdk

4. Install aws-iot-device-sdk module to the directory using the following command

npm install --prefix ./awssdk aws-iot-device-sdk

After completing go to the directory and use ls command to test either it successfully installed or not. You will see two directory named etc & node_modules .

If you found this then you successfully did the job. Transfer the directory to your PC using FTP client. This module has to be uploaded with lambda code. You may also upload it directly from raspberry pi using VNC.

Make your own lambda function or download index.js from the code section.

5. Make a zip folder and integrate all certificate files you got from AWS IoT thing creation, node modules you just prepared and the lambda function named index.js.

6. Go to lambda console of AWS Services and upload the zip file as Lambda function package.

Note: You can also download completed node_modules & Lambda function from my Github repository.

To subscribe the topics publish in AWS IoT we need to install MQTT client to raspberry pi. I used open source MQTT client Paho developed by Eclipse. To install it to your raspberry pi go to terminal and write the following command.

sudo pip install paho-mqtt

After successfully installing paho client go to the code section and download the code named as Raspberry Pi. Save it as paho_client.py or use another name as you like and transfer it to your raspberry pi. To run the program go the the directory where you saved the program and write following command to terminal.

sudo python paho_client.py

To make the script auto-run at startup from the terminal window type:

sudo nano /etc/profile

Scroll to the bottom and add the following line :

sudo python /home/pi/paho_client.py

where “/home/pi/paho_client.py” is the path to your script.

Type “Ctrl+X” to exit, then “Y” to save followed by “Enter” twice.

If you are totally new in raspberry pi the Ultimate Raspberry Pi Configuration Guide may help you.

If you give a look to the block diagram you will get a complete overview how the system is working. A Raspberry Pi is working as a Alexa voice service (AVS) client. A custom voice skill (Pi Home Master) is designed for controlling the appliances. When a user starts the Alexa custom skill (Pi Home Master) it triggers the AWS Lambda to run and execute the Lambda function related to the skill (Lambda function is written in Node.js). Then, Lambda function generate and sends an appropriate request to AWS IoT.

Another (or same) Raspberry Pi is connected to the appliances to be controlled. Paho MQTT client is installed to the raspberry pi and connected to AWS IoT thing. When Lambda function sends any request to AWS IoT, then AWS IoT sends a message to a specific topic subscribed by the client installed in the Pi. The pi gets direction from the MQTT message and turn on or off the corresponding thing connected to it.

Voice User Interface (VUI) diagram

The VUI diagram includes a few sample voice commands to control the appliances. To see all the available voice command for the skill Pi Home Master I designed for the project, please see the Sample Utterances file from the github repo. Sample Utterances are the phrases may be used by a user to control any thing.

Circuit Diagram

Circuit diagram shows the connection between raspberry Pi and the load (light & fan). I used only four appliances (three light and one fan) for demonstration. You can increase the number as your requirement with the modification of the software. For controlling AC load using Raspberry Pi relay is required. I used a relay module with four relay for my project. Using relay module will reduce the wiring and connection in a great extend. For driving the relay a relay driver IC (ULN2003) is used because i/o pin of Raspberry Pi is not capable to control the relay directly. If you connect relay directly to Raspberry Pi you may damaged you Pi. The button is required if the same Pi works as AVS client. If you have a Echo device then ignore the button.

Connection of the Lamp holder is very simple. Connect three lamp holder with three relay. Motive of using relay is clear and explained in previous step. As I made it for demo and for that I attached all the holders into a acrylic box. I add one socket with a relay for connecting a fan.

In this step I will show the connection between relay module and raspberry pi. Relay module I used has four relays with integrated relay driver. For the reason I did not used ULN2003 but must be required if you used standalone relay instead of using module. Connect the signal pins of the relay module to Raspberry pi I/O pins shown in schematic. For powering the relay circuit you must use separate power source. Never connect 5V or 3.3V pin of raspberry pi to power the relay. This may damaged your raspberry pi.

For the demonstration I used a acrylic box to set all the appliances (three light and one fan). I attached some photos of my complete setup. I have no echo device. I am using same raspberry pi as Alexa voice client here. You may use Echo device or may use virtual Echo device Echosim from echosim.io.

Github link of all necessary files and code.

In the directory lambda you will get all the necessary files for AWS Lambda. Just replace certificate and private key files with you own files and make a .ZIP file. Upload the .ZIP file to AWS Lambda. Don't forget to replace app id in the index.js file with your own Alexa skill app id.