Amazon Echo and Alexa

Begonnen von andrejs, 03 Oktober 2015, 23:32:23

Vorheriges Thema - Nächstes Thema

andrejs

I have recently bought Amazon Echo also with intention to use it for voice control of home automation devices in FHEM. I managed to build the necessary applications and in order to make it work you will need:
1.   Buy Amazon Echo gadget
2.   Install/copy 77_ECHO.pm FHEM module and define device in FHEM (define <name of device> ECHO). This application will enable you to create Intent Schema, Sample Utterances and create API in json format for script in Amazon Lambda service (see point 4). 

To create Intent scheme
http://fhem-server-ip:port/fhem?cmd=get%20nameofECHOdevice%20intentscema&XHR=1
or
get <name of echo device> intentscema

To create Sample Utterances
http://fhem-server-ip:port/fhem?cmd=get%20nameofECHOdevice%20uterances&XHR=1
or
get <name of echo device> uterances

API data in json format
http://fhem-server-ip:port/fhem?cmd=get%20nameofECHOdevice%20alexa%20nameofFHEMdevice&XHR=1
or
get <name of echo device> alexa <name of fhem device>

3.   Build the Alexa skill - declare Intent Schema and Sample Utterances

First you need to sign in Amazon developer console and find apps&services Alexa (https://developer.amazon.com/edw/home.html#/) and select Alexa Skills Kit.
Here you will build the Alexa Skills - declare Intent Schema and Sample Utterances. You will also define the inovacation name – for example Smarty.
Example how the intent scheme and sample uterance is defined for FHEM

intent scheme
Zitat3 intents are defined for FHEM:
SRset – for set the state of FHEM device
SRstater  - get the value and time of change for specific reading of FHEM device
Srgeneral – ask specific question and get answer

Sample Utterances
ZitatSRset {cabinet room blinds|litdev} {up|litrea}
SRset set {brightness|litrea} of {living room light|litdev} to {one hundred|numval}
SRstater {outside|litdev} {temperature|litrea}
SRgeneral {what time is it|litrea}
Each sample uterance should have defined the name of intent (SRset), the name of device (living room light), the name of reading (brightness), the value of reading (one hundred).   
After the  | the the type of slot should be written (for example llitdev)

For more information about Intent Schema and Sample Utterances  see here https://developer.amazon.com/public/solutions/alexa/alexa-skills-kit/docs/defining-the-voice-interface

4.   Create the script in order to get information from FHEM server or to set the state of FHEM devices. In my case I used Amazon's AWS Lambda service and node.js. if you have access to a secure webserver yourself, you can set it up on your server, in your preferred programming environment, but as I said I have not done this,

To use Amazon AWS Lambda service you need to sign and create Lambda function.  When you create the function you will need to define:
-   Tab Code – select that the code will be uploaded in zip file (archive zip consists index.js, AlexaSkill.js). In index.js you will need to define var APP_ID (application ID od Alexa skill – can be seen at the first page of Alexa skill), devecho (the name of ECHO FHEM device) and urlPrefix (ip of FHEM server, port.! The script only covers http connection and not https!
-   Tab Configuration – select Node.js, index handler, basic execution role, Advanced settings timeout should be 5 s
-   Tab Event sources – select Alexa Skills Kit

More information is available here https://developer.amazon.com/public/solutions/alexa/alexa-skills-kit/docs/developing-an-alexa-skill-as-a-lambda-function


5.   for device which you want to control over Amazon echo you need to define specific attribute (SRset SRstater SRword) for FHEM device.  This attributes should be defined in FHEM as attr global userattr.

SRword – english name of device.
SRset – for set the state of FHEM device


Syntax: request1, request2,.....

Each request is divided by comma and in the request the following elements should be defined:
-   *1 – it is mandatory and represents the english name of FHEM device
-   the event/reading should be enclosed in round brackets with following syntax (english name of event:fhem name of event:english name of unit). This element is mandatory
-   the value of event/reading should be enclosed in square brackets with following syntax [english name of value:fhem value]. This element is optional
-   the specific answer – the default answer  from Alexa is »I have set [device] to [state].« and if you want to have your specific answer you should use this element enclosed in curly brackets. This element is optional. In order to create the response the following parameters are available:
o   *1 – english name of FHEM device
o   *2 – english name of event/reading
o   *3 – english name of value of event/reading
o   *4 – date and time of last change of event/reading   
Example for specific answer: {I have changed *2 of *1 to value *3. The change was made *4}
Spoken: {I have changed brightness of room light  to value hundred. The change was made 2015-10-03 20:04:03}   

Example for SRset attribute for LIFX led light (SRword: room light):
switch *1 (on:on),switch *1 (off:off),set (brightness:bri) of *1 to [one hundred:100],set (color:rgb) of *1 to [red:FF4766]

In the example are 4 requests:
-   first and second are very simple for switching off and switch on the light. You will need to say »switch room light on »
-   third request is for control the brightness. You need to say »set brightness of room light to one hundred. Fhem request will be made using the fhem name of parameters (bri,100, fhem name of device)
-   last request set the color of light. You need to say »set color of room light to red«.


SRstater – to get information about the state of specific reading of FHEM device.

Syntax: request1, request2,.....

Each request is divided by comma and in the request the following elements should be defined:
-   *1 – it is mandatory and represents the english name of FHEM device
-   the name of reading should be enclosed in round brackets with following syntax (english name of reading:fhem name of reading:english name of unit). This element is mandatory
-   the specific answer – the default answer  from Alexa is »[english name of device] [english name of reading] is [state of reading] [english name of unit]. Last change was made [date and time of reading change].« and if you want to have your specific response you should use this element enclosed in curly brackets. This element is optional. In order to create the answer the following parameters are available:
o   *1 – english name of FHEM device
o   *2 – english name of event/reading
o   *3 – value of reading
o   *4 – date and time of last change of event/reading   

Example for specific answer: {In the garden *1 *2  is *3 degrees Celsius. The measurement was made *4}
Spoken: In the garden outside temperature is twenty degrees Celsius. The measurement was made 2015-10-03 20:04:03}   

Example for SRstater attribute for weather station (SRword: outside):

*1 (temperature:temperature:degrees Celsius),*1 (humidity:humidity:percent)
In the example are 2 requests:
-   first to get the temperature and second to get humidity. You will need to say »Outside temperature». The response will be: »the outside temperature is twenty degrees celsius. The last change was made  2015-10-03 20:04:03«

SRgeneral – to ask general question and get some answers. This attribute is defined only for ECHO device!

Syntax: request1, request2,.....

Each request is divided by comma. In the request two elements should be present question and answer divided by :. Two parameters can be used in element answer:
-   *10 – get current date
-   *11 – get current time

Examples for Srgeneral

thank you:you are welcome,go to sleep:good night and sleep well,what time is it:Time is *11,what day is it:Today is *10

In the example are 4 requests:
-   first and second are without parameters and if you say »thank you« the Alexa will answer »you are welcome«.
-   For third request you should ask »what time is it » and the answer will be the current time in format »Time is 20:20«
-   For fourth request you should ask »what day is it » and the answer will be the current dat ein format »Today is October 3rd«


When you sucesffuly »passed« all described five points then you will be able to control FHEM devices with amazon echo. To start you have two possibilities:

-   Say: Alexa start [inovacation name]. Alexa will ask you »How would you like to control your devices«. This answer can be changed in index.js var lanunchtext. After that you can put/say your request
-   If you want directly to access you need to say »Alexa tell [inovacation name] to ....« or »ask [inovacation name] for....«

This should be all for now and have fun.

Andrej 


rudolfkoenig

Andrej, as the prefix 99 is special (the module will always be loaded at startup), could you please rename your module into something like 77_ECHO.pm? It would make a later integration into the standard FHEM distribution easier.

The Echo is not available in Germany yet, please be patient until you get some response here.

andrejs


breezybadger

#3
Hey Andrejs,

thanks a lot! I ordered a Echo just because of your work. You bring us one step close to sky net and Iron Man. Thanks!

Arrived today, I will report

@Update1:

I was able to set it up and now startet to configure it within FHEM. Sadly it is not so easy. I register at the AMAZON Services for Alexa Skills and the Lambda Plattform.

When i try to add your utterances in the Webapp I got the response:

Error: There was a problem with your request: The slot 'litdev' has been referenced with two different slot types: LITERAL and LITERAL|NUMBER. A slot name can only be of a single type.

@Update2

When i say "alexa ask smarty ... " -> "There is problem with the requested skill respond"

I uploaded you index (edited) and alexa skill file in AWS Lambda File and put the settings in there you described.
In the Apps&Service Development I make a new skill with the following settings :
Intent Schema*

{
  "intents": [
  {
     "intent":"SRset",
     "slots": [
       {
         "name": "litdev",
         "type": "LITERAL"
       },
       {
         "name": "litval",
         "type": "NUMBER"
       },
       {
         "name": "litrea",
         "type": "LITERAL"
       }
     ]
  },
  {
     "intent":"SRstates",
     "slots": [
       {
         "name": "litdev",
         "type": "LITERAL"
       },
       {
         "name": "litval",
         "type": "NUMBER"
       },
       {
         "name": "litrea",
         "type": "LITERAL"
       }
     ]
  },
  {
     "intent":"SRgeneral",
     "slots": [
       {
         "name": "litdev",
         "type": "LITERAL"
       },
       {
         "name": "litval",
         "type": "NUMBER"
       },
       {
         "name": "litrea",
         "type": "LITERAL"
       }
     ]
  }
]
}


Sample Utterances*
SRset set {light|litdev} {on|litrea}
SRset set {light|litdev} {off|litrea}


In FHEM I have for my Homatic Light Switch for testing:

define HM_37764C CUL_HM 37764C
attr HM_37764C SRword light
attr HM_37764C SRset set *1 (on:on),set *1 (off:off)


My Alex App Answers:

The SpeechletResponse must not be null

Thanks in advance for your help and awesome work!

andrejs

Hi Dani,
just my quick observation. Regarding the Intentscema if you use ECHO module in FHEM (http://fhem-server-ip:port/fhem?cmd=get%20nameofECHOdevice%20intentscema&XHR=1) to create intent schema you should get the following:

Zitat{
  "intents": [
  {
     "intent":"SRset",
     "slots": [
       {
         "name": "litdev",
         "type": "LITERAL"
       },
       {
         "name": "litval",
         "type": "LITERAL"
       },
       {
         "name": "litrea",
         "type": "LITERAL"
       },
       {
         "name": "numdev",
         "type": "NUMBER"
       },
       {
         "name": "numval",
         "type": "NUMBER"
       },
       {
         "name": "numrea",
         "type": "NUMBER"
       }
     ]
  },
  {
     "intent":"SRstater",
     "slots": [
       {
         "name": "litdev",
         "type": "LITERAL"
       },
       {
         "name": "litval",
         "type": "LITERAL"
       },
       {
         "name": "litrea",
         "type": "LITERAL"
       }
     ]
  },
  {
     "intent":"SRgeneral",
     "slots": [
       {
         "name": "litdev",
         "type": "LITERAL"
       },
       {
         "name": "litval",
         "type": "LITERAL"
       },
       {
         "name": "litrea",
         "type": "LITERAL"
       }
     ]
  }
]
}

This works me (see attached print screen) in amazon developer console and you can see the interaction module is checked.Have you passed all controls on Amazon developer console (ALEXA SKILL) and your Intentscema is checked? 
I also noticed that in intentschema you should replace the name of intent "SRstates" with "SRstater" but this I think it so not the cause of your problem. Regarding the "uterances" I do not see any errors. 

Andrej


andrejs

#5
Dani,
it is still old version of ECHO module attached in the first post. Check attribute SRintents in echo device it should have value
ZitatSRset:LITERAL#NUMBER,SRstater:LITERAL,SRgeneral:LITERAL
Use attached echo module to this post and create again the intentscema (it will be the same as it was shown in the previous post).

Andrej

breezybadger

#6
Dear Andrej,

now I got the same output and it is saved in the Apps and Services. But in the Alexa Skill app I got the following output in the attached pic.

I put the index.js and the AlexaSkill.js in a zip Archive and uploaded it to AWS Lamda function - I can't edit it because it was in a archive. Of course I choose the Alexa Skill Kit and the index handler, when I run "Test" I got "Unable to invoke the function because invoke parameters were not valid JSON." back.

Did I may have done something wrong in there?

The Alexa App Still say:
ZitatThe SpeechletResponse must not be null

Dan

andrejs

#7
Dani,

I have never tested in Alexa skill KIt but I tested in AWS Lambda. I also got the same response as you (response is invalid) did but it still works when I ask Alexa to do something. First I would like to ask you have you changed in index.js (before you make a zip with AlexaSkill.js file and upload in AWS) the following lines:
var APP_ID = "?"; //replace with your amazon app ID it should be something like amzn1.echo-sdk-ams.app.xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
var devecho = "?";//replace with the name of your echo device in FHEM
var urlPrefix = "http://myname:mypassword/myhome:8090/fhem?cmd=
if your fhemweb page is not password protected then you can skip the username and password but of course it should be accessible outside your home network.   
How I test in AWS Lamda?
In AWS Lambda function I select Actions and Configure test event. After that the "input test event" window will be opened and in this window I copy the following code (change applicationId):
Zitat{
  "session": {
    "new": false,
    "sessionId": "session1234",
    "attributes": {},
    "user": {
      "userId": null
    },
    "application": {
      "applicationId": "amzn1.echo-sdk-ams.app.xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
    }
  },
  "version": "1.0",
  "request": {
    "intent": {
      "slots": {
        "litdev": {
          "name": "litdev",
          "value": "light"
        },
        "litrea": {
          "name": "litrea",
          "value": "on"
        }
      },
      "name": "SRset"
    },
    "type": "IntentRequest",
    "requestId": "request5678"
  }
}

Then click on the button "Save and test" and you will see the execution result and log output (see example in attached picture).

Andrej

breezybadger

Hey,

the first lines of my uploaded file looks like :

Zitatvar APP_ID = "amzn1.echo-sdk-ams.app.2a79ae4f-2492-xxxx-a0b6-xxxxx"; //replace with your amazon app ID <- which i got from the APP ID I created in APPS and Services Application Id :The ID for this skill
/**************************************\
                Globals
\**************************************/
var https = require('https');
var http = require('http');
var AlexaSkill = require('./AlexaSkill');
var devecho = "echo";
var urlPrefix = "[url="http://192.168.1.13:8083/fhem?cmd=";"]http://192.168.1.13:8083/fhem?cmd=";[/url]
var urlPrefixget = urlPrefix+"get%20"+devecho+"%20alexa%20";
var launchText = "How would you like to control your devices?";


My FHEM Config looks like:
Zitatdefine echo ECHO
attr echo SRfile 0
attr echo SRintdefault SRword
attr echo SRintents SRset:LITERAL#NUMBER,SRstater:LITERAL,SRgeneral:LITERAL
attr echo SRword echo

The Result of the test in Lamda was:
ZitatExecution result: failed (logs) The area below shows the result returned by your function execution using the context methods. Learn more about returning results from your function.
{
  "errorMessage": "Unexpected token ILLEGAL",
  "errorType": "SyntaxError",
  "stackTrace": [
    "Module._compile (module.js:439:25)",
    "Object.Module._extensions..js (module.js:474:10)",
    "Module.load (module.js:356:32)",
    "Function.Module._load (module.js:312:12)",
    "Module.require (module.js:364:17)",
    "require (module.js:380:17)"
  ]
}

Of course I replaced the ID with the one from my Apps&Services

Any Ideas what I am doing wrong?

andrejs

Dani,

the issue is var urlPrefix = "http://192.168.1.13:8083/fhem?cmd=" because in the script index.js the Amazon server can not access to your internal network. How you access to fhemweb if you are not connected to your network?

Andrej

breezybadger

Hey,
i only access it from my Homenetwork ( the same Alexa is into)

So I have to setup a public network access ?

Dan

andrejs

Yes, otherwise it will not work.

breezybadger

Since my DS-Lite connection and a fritz.box that does not like me, it will not work for me :(

But thanks again a lot for your help!

andrejs

Dani,
I do not have a lot of knowledge but your router should be configured to use dynamic DNS and port forwarding or if you have static IP (in my case I got static IP from internet provider) you just set up port forwarding on your router?

Andrej

breezybadger

Hi,

I did, but it does not work. I already send an Email to the AVM Support, to find out - why it does not work. I let you know, as soon as I have found a solution.

Dan