FHEM Forum

FHEM => English Corner => Thema gestartet von: AlxDmr am 03 November 2014, 19:58:17

Titel: Communication with Fhem via websockets ?
Beitrag von: AlxDmr am 03 November 2014, 19:58:17
Hello,
is there a way to establish communication with fhem via websocket (or a classical TCP) so that it is possible to receive events from devices as well as sending command (e.g. turn off a light). I know that it is possible to establish a telnet connection but it seems to me not suitable to retrieve and send such data. Maybe I am wrong as I don't know this protocol very well...?
Actually, I am trying to integrate enOcean devices using Fhem for a personnal smart Home project. I still developped some code using NodeJS and would like to interoperate with Fhem.

    Alex.
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: ntruchsess am 03 November 2014, 21:11:37
you may checkout integrating NodeJS with FHEM using websockets via mqtt-protocoll using latest mosquitto (compiled with websocket-support). As far as I know there are some existing solutions to access mqtt from nodeJS. Use fhem-modul MQTT (http://fhem.de/commandref.html#MQTT) and MQTT_BRIDGE (http://fhem.de/commandref.html#MQTT_BRIDGE) to translate your existing fhem enocean-devices to mqtt-topics (both in- and outbound).

- Norbert
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: rudolfkoenig am 03 November 2014, 23:12:48
Alternatives:
- plain TCP/IP (known in FHEM telnet), with or without SSL
- HTTP/HTTPS via longpoll (used by the default WEB frontend).
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: AlxDmr am 06 November 2014, 21:57:15
The MQTT option sounds quite good to me :)
However I updated Fhem but I have no MQTT module.
I'm brand new in the world of FHEM so maybe I missed something?
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: fruit am 07 November 2014, 09:23:56
I know nothing of MQTT but update should have brought the MQTT modules in, they should be in /opt/fhem/FHEM/ as 00_MQTT.pm, 10_MQTT_BRIDGE.pm and 10_MQTT_DEVICE.pm

If they are not there try update force
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: AlxDmr am 07 November 2014, 22:18:41
It was may fault, I was using Unix apt-get update ...
after using telnet : update  I got everything I need :)
Thanks, now I can start to play :)
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: ntruchsess am 07 November 2014, 23:31:24
just to let you know: I started implementing websocket-support for fhem. It will allow to execute the quivalent commands as telnet and get notified for all events. Not ready yet (I have to build a websocket-testclient first), but maybe worth a first look: 98_websocket.pm (https://github.com/ntruchsess/fhem-mirror/blob/websockets/fhem/FHEM/98_websocket.pm)
The main difference to using the mqtt-bridge and accessing mqtt via websockets is that mqtt will only publish and subscribe selected fhem-devices while direct websocket access will allow to control fhem completely. The advantage of the mqtt-way is that it is standard and easy to understand when you have used mqtt before. The websocket-interface will be more lowlevel and fhem-like.

- Norbert
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: rudolfkoenig am 08 November 2014, 11:56:05
@Norbert: iOS devices have problems with communicating to FHEMWEB vial longpoll: as long a connection with longpoll is open, no second one will be opened by the iOS device, meaning that you cannot load images from FHEMWEB with activated longpoll. The current workaround is to use SVG icons, as they are transmitted "in-line". I hope that by using websockets, which is (are?) supported by all current browsers, we can avoid this problem.
It would be nice if FHEMWEB could somehow use your module, so that it is not implemented twice in FHEM.
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: AlxDmr am 09 November 2014, 20:15:18
Sorry again for the stupid question but...
now I get the required file, however when I try to add :
    define localMQTT  MQTT  192.168.1.11:1883
in fhem.cfg, but I get the error :
    ERROR:

    Cannot load module MQTT

I searched for specific loading module command but did not found anything. Where did I went wrong ?

Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: ntruchsess am 09 November 2014, 21:32:16
you need to install Net-MQTT from CPAN (http://search.cpan.org/~beanz/Net-MQTT-1.142010/) first.

(as an alternative you currently might check out the files from fhem-svn (http://sourceforge.net/p/fhem/code/HEAD/tree/trunk/fhem/FHEM/lib/) as being available on cpan they are not part of fhem-update and are likely to be removed from fhem-svn in the future).
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: ntruchsess am 10 November 2014, 18:52:48
Zitat von: rudolfkoenig am 08 November 2014, 11:56:05
It would be nice if FHEMWEB could somehow use your module, so that it is not implemented twice in FHEM.
@rudolfkoenig: sure, as any other webapp FHEMWEB will be able to use websockets. It replaces longpoll entirely.

here is first working version. Does execute all fhem-commands (like telnet):
98_websocket.pm (https://github.com/ntruchsess/fhem-mirror/raw/websockets/fhem/FHEM/98_websocket.pm)
simple testclient:
websocket.html (https://github.com/ntruchsess/fhem-mirror/raw/websockets/fhem/websocket.html)

required modules from cpan:JSON and Protocol::WebSocket (and Data::Dumper ...)

have fun!

- Norbert
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: rudolfkoenig am 10 November 2014, 21:46:22
And how can FHEMWEB rewrite the events, supplying its own data, e.g. the state as icon or inline SVG?
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: ntruchsess am 10 November 2014, 22:43:56
as this is just the starting point getting the communication running this is open for discussion. From my point of view webbsocket acces is primary for pushing pure data, not design, but I'm open fur suggestions. Did you check whether iOS does limit open websockets as it does with longpolling requests?
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: herrmannj am 10 November 2014, 23:02:45
it is not a big deal to simply replace the existing longpoll with websockets. It can be entirely done within fhemweb.js - provided that the websocket delivers the same data. (simple hook within fhemweb.pm). That way the direction browser fhem (the way back) would be the usual xhr way.

but I would suggest to do a broader rewrite and give it the ability to transfer more structured data.

Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: justme1968 am 11 November 2014, 08:05:12
this (structured JSON? data) would make the parsing of the message in fhemweb.js mich simpler and should also avoid the current duplication Insider the messages.
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: ntruchsess am 11 November 2014, 08:41:40
im also in favor to implement a pure json-based protocol (in fact that is what the current implementation does response-wise). But as websocket-handshake is designed it's not limited to this - when connecting the client submits an URI-string and a list of supported protocols. It would be possible to support both: new pure json rpc protocol and existing 'newline-separated' fhemweb protocol by checking those on connection-establishment and choose the requested mode of operation.
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: rudolfkoenig am 11 November 2014, 10:15:57
My problem is not the newline vs. JSON protocol, but the fact, that the current fhemweb.js needs more than pure FHEM events. It relies on the FHEMWEB enriched data like status .png.
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: justme1968 am 11 November 2014, 10:51:43
why is this a problem?

the enriched data can be send by websockets and also 'pure' fhem events can contain images. see here for example: http://forum.fhem.de/index.php/topic,28866.msg216874.html#msg216874 (http://forum.fhem.de/index.php/topic,28866.msg216874.html#msg216874)
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: ntruchsess am 11 November 2014, 10:57:20
sure, I understand that. For that reason I'd like to have two protocols, one that is optimized for fhemweb.js and one that is going to access the 'raw' data. The second option is more efficient when we talk about high-cpu-power smartphones accessing a fritzbox ;-)

If you like to get all enriched data for fhemweb.js in json-format to, then everything may go into a single protocol (that has special commands for fhemweb.js). If you are talking about binary data (streaming e.g png through websockets) than this is should not be encoded as json but binary-messages. I have to check whether it's possible to enrich binary-messages with metadata.
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: rudolfkoenig am 11 November 2014, 18:39:51
@ntruchsess: I dont need binary data (FHEMWEB only sends the .png URL), but I need a hook to enrich/replace events with the corresponding image-url, SVG, etc. But I think that defining/deleting a second device and opening a new port for each FHEMWEB instance is too complicated, so I think I will just use your module as an example -> Don't bother with FHEMWEB or me :)

@justme1968: I am not convinced, that putting images in events is a good idea: for some users these images will land in a FileLog or DbLog.
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: justme1968 am 11 November 2014, 19:38:23
if there is a general way to add events/messages/data to the longpoll mechanism i will be happy to use it.

if a user is logging everything then a few images should be no problem.
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: ntruchsess am 11 November 2014, 23:43:29
images in events is good for nothing else than a specific frontend (and maybe a specific device and/or resolution) only. My assumption is that the main reason this got introduced is the lack of appropriate bidirectional remote-protocoll for web-browser in fhemweb that allows to receive both raw events and query for design-specific metadata through the very same (longpoll) channel. As websockets are both message-oriented and bidirectional by desing I'm pretty sure that they are a good basis to implement such.
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: justme1968 am 12 November 2014, 09:57:16
readingsGroup is very fhemweb specific and the messages are device specific and only intended for the readingsGroup display. the same will be the case for some new widgets that can be used for heating control and shutter/blind control.

readings group even tries to detect if it is currently displayed and will not send any events if it is not visible in a browser window.

so yes. a fronted/device specific subscription method would be perfect for this.

would you suggest using a separate web sockets for each purpose? or a general one with a subscribe and dispatch mechanism similar to the current fhemweb 'widget' idea?

the last could allow for different channels (tags/message types) on the same socket and for example fhemweb would use a longpoll channel and the readingsGroup frontend would request a readingsGroup channel. the backend would just create both message types and send them to the same websocket. a message and device specific filter like the current inform filter could be integrated in the backend so unneeded messages would not be send. in addition for modules like readingsGroup a hasSubscribers method could query if there are subscribers for a specific device and stop generating the events altogether.
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: ntruchsess am 12 November 2014, 11:19:13
Best practice is to use a single websocket per browers-session, and use a subscribe-mechanism to dispatch the messages coming from server. One can easily implement multiple communication-channels through a single websocket using structured (json) data. All you need is some 'type' or 'channel'-attribute on the outermost level of the message-object.

Having multiple websockets per browser-session is not advisable as it may conflict with limits imposed by the client-device.

On the server-side Fhem currently doesn't have an appropriate subscribe-mechanism for its events as they are not type-save but string-data only (so you can 'subscribe' on device-level (NOTIFYDEV) and filter CHANGED by regexp and you may mark an event within it's content as e.g. 'widget-only' but as the event-data is unstructured this eventually will conflict with other events).

Therefore I suggest to leave the fhem-event-sytem unchanged and use it for what it is designed for (distributing reading-changes and lifecycle-events within fhem), filter the events a widget is interested in on the server side and send 'enriched' messages (enriched in respect to e.g. visual representation specific data) through the appropriate websocket-channel only if there's a subscrition for it on the client-side.

On the server-side there should be an api to allow for multiple frontendspecific modules to mediate fhem-events to widget-type specific messages.

(read 'suggest' and 'should' as: 'This is what i'm currently implementing...')

- Norbert
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: ntruchsess am 13 November 2014, 15:17:50
update: subscribe-mechanism for the client and for fhem-modules implemented:

98_websocket.pm (https://github.com/ntruchsess/fhem-mirror/raw/websockets/fhem/FHEM/98_websocket.pm)
websocket.html (https://github.com/ntruchsess/fhem-mirror/raw/websockets/fhem/websocket.html)

on the webbrowser-side:
use json as protocol when opening the connection:

var con = new WebSocket('ws://localhost:8080/fhem', ['json']);


if protocoll json is not specified all text-messages sent to the websocket are interpreted as fhem-commands, the messages sent back will contain the response of those commands.

if protocoll is 'json' than the following message-structure is used:


{"type":"...", "payload":{...}}


'payload' is message-'type' specific. Types defined so far are: 'command', 'commandreply' and 'event'

commands defined so far are 'subscribe', 'unsubscribe', 'list' and the other fhem-commands.

fhem-modules may subscribe to receive messages of (custom) types from websocket using websocket::subscribeMsgType() (->see below)

e.g.

{"type":"command","payload":{"command":"subscribe","type":"","name":"Licht_.*zimmer","changed":"on|off","arg":"widget1"}}

will register for fhem-events in json-format (equivalent to a call to websocket::subscribeEvent() (->see below))

The Messages of type 'event' that are send back to the browser have the following structure:

{"type":"event","payload":{"name":"Licht_Wohnzimmer","type":"OWSwitch","arg":"switch1","changed":{"state":"on"}}}



{"type":"command","payload":{"command":"list","arg":"room=raum"}}

does call jsonlist2 and wrapps the output into a json-response message type 'commandreply':

{"payload":{"command":"list","reply":{"Arg":"dummy","Results":[{"Internals":{"NAME":"dummy","NR":"15","TYPE":"dummy","STATE":"test: off"},"Readings":{"state":{"Time":"2014-11-12 22:27:23","Value":"test: off"}},"PossibleAttrs":"verbose:0,1,2,3,4,5 room group comment alias eventMap userReadings setList event-on-change-reading event-on-update-reading event-min-interval stateFormat devStateIcon devStateStyle icon sortby webCmd widgetOverride userattr","Attributes":{"room":"raum"},"Name":"dummy","PossibleSets":" "}],"totalResultsReturned":1}},"type":"commandreply"}


on the serverside:
an fhem-module may use the following methods to subscribe for lifecycle-events

websocket::subscribeOpen($hash,$fn,$arg)
websocket::unsubscribeOpen($hash,$arg)
websocket::subscribeClose($hash,$fn,$arg)
websocket::unsubscribeClose($hash,$arg)


$hash is the master websocket device, $fn is a code-reference, $arg is an (optional) argument that is both used as key (to unsubscribe) and passed as argument to the code-reference (equivalent to the argument of InternalTimer)

the signature of code-referece is:

sub <name>($cl,$arg)

$cl being the hash of the connected websocket-child-device, $arg is the argument that was passed to the subscribe-method

an fhem-module may subscribe to messaged being received by a connected websocket instance:

websocket::subscribeMsgType($cl,$type,$fn,$arg)
websocket::unsubscribeMsgType($cl,$type,$arg)

signature of $fn is:

sub <name>($cl,$payload,$arg);

$cl again is the hash of the connected websocket-child-device, $type is the type of json-message to subscribe to, $fn is the code-referece to be called and $arg the (optional) argument (that again can be used as key to unsubscribe), $payload is the payload value of the message received via websocket.

additionally an fhem-module may register subscriptions to events is such way that for each event that matches the regexp a message type 'event' is sent to the connected websocket:

websocket::subscribeEvent($cl,name => $name, type => $type, changed => $changed, arg => $arg)
websocket::unsubscribeEvent($cl,$arg)


$cl is the hash of the connected websocket-child-device, $name is a regular-expression matching the NAME of the device emmitting the event (optional), $type a regexp matching the TYPE (also optional), $changed a regexp to filter contents of the fhem-event, $arg is a value (string, array or hash) that is passed with the messages payload as value of key 'arg'. That can be used on the browser-side to identify a widget that should receive the event-message. The 'CHANGED'-array of the fhem-event is filtered by the regexp and all matching entries are send as value of key 'changed' in the payload of the message.

Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: AlxDmr am 22 November 2014, 14:42:27
Halo,
I tried to connect to Fhem using websocket (after updating Fhem). But I get the following errors when doing:
con = new WebSocket('ws://192.168.1.14:8080/fhem', ['json']);
the error code is :
WebSocket connection to 'ws://192.168.1.14:8080/fhem' failed: Error in connection establishment: net::ERR_CONNECTION_REFUSED

I also tried on port 8083, wich is the port used for HTTP server :
con = new WebSocket('ws://192.168.1.14:8083/fhem', ['json']);
the error code is :
WebSocket connection to 'ws://192.168.1.14:8083/fhem' failed: Error during WebSocket handshake: Unexpected response code: 200

I guess I did not updated code correctly or that I do not target the right port... how could I check?

   Alex.
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: ntruchsess am 22 November 2014, 16:19:22
98_websocket.pm is (not yet) includet in update as the api is not settled yet. You have to install manually (use download-links you find a few posts above) or checkout from my fhem-mirror on github branch websocket (https://github.com/ntruchsess/fhem-mirror/tree/websockets).
If you have any suggestions about the javascript-api (see file fhem.js (https://github.com/ntruchsess/fhem-mirror/tree/websockets/fhem/www), your feedback is highly aprechiated.

regards,

Norbert
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: AlxDmr am 22 November 2014, 16:41:10
Halo,
thanks for the advice, for sure I will test that :)
However, I downloaded 98_websocket.pm in /opt/fhem/FHEM and I restarted everything (the raspberry itself). I still get the very same error messages.
Should I modify something else so that the module is taken into account?
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: ntruchsess am 23 November 2014, 06:07:28
You have to define it first. ('define websocket websocket 8080 global' - same semantics as telnet-device) and you might have to install the dependencies...Protocol::WebSocket (https://metacpan.org/pod/Protocol::WebSocket))
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: AlxDmr am 23 November 2014, 18:06:05
Thanks,
I added the command in fem.cfg and indeed it requires some dependencies.
I went to the website you indicated, downloaded and unpacked the file but I have no idea how I should install that, there are no information about that on the website. It's probably a Linux thing but I mainly use windows so it's not easy for me. I tried some make and make install but it did not worked...
I know it's a bit out of context for this forum but could you indicate me how to install the dependency once files are on the Raspberry please ?
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: ntruchsess am 23 November 2014, 21:53:11
here you find some information on how to install perl modules from cpan (http://www.lmgtfy.com/?q=cpan+windows).

An alternative is to manually place the unzipped library files to fhem/FHEM/lib, 98_websocket.pm is able to load dependencies from there to. To do so locate Directory 'Protocol' in 'Protocol-WebSocket-0.18/lib' (from the unzipped archive) and move that (and all its subdirectories) to the target location.

regards,

Norbert
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: AlxDmr am 27 November 2014, 15:22:16
Hallo,
I finally succeeded to install dependencies (actually Protocol::Websocket but also JSON) so that fhem seems to be able to intitialize your websocket module.
However, when doing in the browser :
con = new WebSocket('ws://192.168.1.14:8080/fhem', ['json']);
I get the following error :
WebSocket connection to 'ws://192.168.1.14:8080/fhem' failed: Error during WebSocket handshake: Sent non-empty 'Sec-WebSocket-Protocol' header but no response was received

Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: AlxDmr am 27 November 2014, 15:23:17
The fhem logs are the followings :
2014.11.27 14:14:29 3: websocket websocket:192.168.1.24:62032 disconnect due to timeout
2014.11.27 14:14:29 1: PERL WARNING: Use of uninitialized value in ref-to-glob cast at ./FHEM/98_websocket.pm line 323.
2014.11.27 14:14:29 1: PERL WARNING: syswrite() on unopened filehandle at ./FHEM/98_websocket.pm line 323.
2014.11.27 14:14:59 3: websocket websocket:192.168.1.24:62032 disconnect due to timeout
2014.11.27 14:15:29 3: websocket websocket:192.168.1.24:62032 disconnect due to timeout
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: ntruchsess am 27 November 2014, 16:58:43
what platform does this run on?
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: AlxDmr am 27 November 2014, 18:52:25
It's a Raspberry Pi model B+. I installed the Raspberrian distribution on it.
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: ntruchsess am 28 November 2014, 09:06:03
and the javascript-code runs in which broser? Or is this on node.js?
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: ntruchsess am 28 November 2014, 09:53:03
I just added a fix for the missing Sec-WebSocket-Protocol header (https://github.com/ntruchsess/fhem-mirror/commit/176a9395fb4ce7941f72ebfb7cd6b922d2c609da) (plus a fix that avoids the warnings when connection is closed (https://github.com/ntruchsess/fhem-mirror/commit/fc4e75978cdcd50a48d7ba2aba91c378b782496f)).

Please upgrade 98_websocket.pm (https://github.com/ntruchsess/fhem-mirror/raw/websockets/fhem/FHEM/98_websocket.pm)

- Norbert
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: AlxDmr am 28 November 2014, 10:35:26
Thanks for the fix, I will test that as soon as possible.
I did the test from a browser (Opera, last version based on webkit) although the real usage I will have next will be to integrate the client socket inside NodeJS.

Best regards,
    Alex.
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: AlxDmr am 28 November 2014, 20:43:09
It works :)
well at least the connection is established, but now I'm struggling to understand the protocol you set up. I would like to subscribe to any event comming from Fhem, how should I do that ?

I am also trying to subscribe to a specific device (an EnOcean switch) with :

con.send( JSON.stringify({ "type" : "command"
            , "payload" : { "command" : "subscribe"
, "type" : ""
, "name" : "EnO_switch_002842C2"
, "changed" : "*"
, "arg" : "widget1"
}
  }) );


but I got no result when using the switch. I guess that payload object is not properly initialized but I do not know what is expected for attribute changed, type and arg.
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: ntruchsess am 01 Dezember 2014, 09:59:31
The parameters 'type', 'name' and 'changed' are being used (https://github.com/ntruchsess/fhem-mirror/blob/websockets/fhem/FHEM/98_websocket.pm#L274) as regular-expressions to filter the events on fhem-side. All three parameters are optional (if not specified it matches any 'TYPE', 'NAME' or 'CHANGED' internal of the event). '*' taken as a regular expression doesn't match anything ('' or '.*' would).

I suggest you use fhem.js (https://github.com/ntruchsess/fhem-mirror/raw/websockets/fhem/www/fhem.js) to access the websocket-interface. fhem.js is able to process the messages returned by 'list'-command. It keeps a list of all devices in 'fhem.devices' and does update the devices readings with evey event received via event-message. You may register additional event-handlers with fhem.js. See websocket.html (https://github.com/ntruchsess/fhem-mirror/blob/websockets/fhem/www/websocket.html) as example how to use the lib.

regards,

Norbert
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: ntruchsess am 02 Dezember 2014, 00:25:50
I did some refactoring to the websocket-module:

split 98_websocket.pm into 00_websocket.pm, 10_websocket_json.pm and 10_websocket_fhem.pm (old 98_websocket.pm is removed)

00_websocket.pm (https://github.com/ntruchsess/fhem-mirror/raw/websockets/fhem/FHEM/00_websocket.pm) is pure websocket-protocol-handler and acts as IODev for 10_websocket_xxx.pm
10_websocket_json.pm (https://github.com/ntruchsess/fhem-mirror/raw/websockets/fhem/FHEM/10_websocket_json.pm) does handle the json-protocol (that was integrated into 98_websocket.pm before).
10_websocket_fhem.pm (https://github.com/ntruchsess/fhem-mirror/raw/websockets/fhem/FHEM/10_websocket_fhem.pm) does handle plain-text fhem command protocol (similar to 98_telnet.pm, but over websocket).

As the core 00_websocket.pm module does handle the websocket-handshake only delegating all message-handling to it's clients, this refactoring enables the websocket-stack to be used by other frontend-specific modules more easily.

- Norbert

Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: ntruchsess am 02 Dezember 2014, 12:10:34
- just added support for SSL (https://github.com/ntruchsess/fhem-mirror/commit/d259a9dfde7c18e4de55e6ab232206ad4345538b) (just specify attribute 'SSL' on websocket-device). Then connect using wss:// as protocol of websocket. For ssl to work a certificate has to be installed in fhem 'cert' directory (http://fhem.de/commandref.html#HTTPS). If you are using a self-signed certificate make sure your browser does trust this certificate. To import the certificate into the browser you may try to open the websocket-port in browser (by using a https://<hostname>:<port> type url and then accepting the security warning).

- js-library fhem.js has been renamed to fhem_websocket.js (https://github.com/ntruchsess/fhem-mirror/raw/websockets/fhem/www/fhem_websocket.js)

- when using fhem_websocket.js you may enable ssl by setting 'secure'-property of fhem-object (https://github.com/ntruchsess/fhem-mirror/blob/websockets/fhem/www/websocket.html#L38).

- support for a 'resource-path' in websocket-url for module websocket_json is implemented. Use 'define <name> websocket_json /path_to_resource' in definition and then ws://<ip:port>/path_to_resource when opening the websocket. Defaults to '/'. A resource-path may be used to share a single websocket-port for different websocket_client-modules serving different purposes.

regards,
Norbert
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: svenson08 am 11 Dezember 2014, 06:29:24
hello norbert,

I have websocket and websocket_json defined. In your sample html relevant info from FHEM come back. The sending of commands (set Office on) but does not seem to be working. from the logfile

Zitat
2014.12.11 06:17:52 5: websocket text {"type":"command","payload":{"command":"set Office off"}}
2014.12.11 06:17:52 5: websocket jsonmessage: $VAR1 = {
          'payload' => {
                         'command' => 'set Office off'
                       },
          'type' => 'command'
        };

2014.12.11 06:17:52 4: websocket_json: Undefined subroutine &websocket_json::AnalyzeCommandChain called

websocket_fhem also needs to be defined? in my case i have following errors in the log.

Zitat2014.12.11 06:24:04 5: Loading ./FHEM/10_websocket_fhem.pm
2014.12.11 06:24:04 1: reload: Error:Modul 10_websocket_fhem deactivated:
Global symbol "%inform" requires explicit package name at ./FHEM/10_websocket_fhem.pm line 96.
Global symbol "%defs" requires explicit package name at ./FHEM/10_websocket_fhem.pm line 97.
Global symbol "%inform" requires explicit package name at ./FHEM/10_websocket_fhem.pm line 98.

the last thing. some garbage data seen in the log, like this

Zitat2014.12.11 06:17:00 5: Š€ðÃÔ'
2014.12.11 06:17:00 5: websocket pong
2014.12.11 06:17:30 5: $VAR1 = {
          'type' => 'ping'
        };

2014.12.11 06:17:30 5: Š€þGz·
2014.12.11 06:17:30 5: websocket pong
2014.12.11 06:17:52 5: ¹X¨i #ŠÙ(ÍKšzËÍ5ÉÄz,,KÐ9ÑÏ9ÌKš#Š

svenson
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: AlxDmr am 21 Dezember 2014, 16:49:42
Hello Norbert,
I downloaded the latest files from your git and there is a handshake problem :
in the javascript console of Chrome (latest version, windows 8 ) :
WebSocket connection to 'ws://192.168.1.12:8080/' failed: Error during WebSocket handshake: Sent non-empty 'Sec-WebSocket-Protocol' header but no response was received
I think you fixed the problem but there might have been a regression when you refactored the code.

Regards,
    Alex.
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: CaliHo am 23 Dezember 2014, 00:04:00
Hello Norbert,
I loaded the three pm files 00_websocket, 10_websocket_fhem and 10_websocket_json into my Fhem server and installed the necessary perl modules (protocol::websocket..) via cpan. I then defined "websocket websocket 8080 global" in the fhem.cfg and uploaded the websocket.html and the fhem_websocket.js onto the server. The button "open" seems to work: I get a debug output:  "Connection opened to fhem server!" But entering a command like "set Rolladen1 up" or "list Rolladen1"  does not work. When reloading the 10_websocket_fhem I get the same messages as svenson08 in his post "...Global symbol "%inform" requires explicit package name at ./FHEM/10_websocket_fhem.pm line 96...". Do I have to define anything else? Is ist wrong to load all three pm files. If i want to replace a telnet communication I suppose the 10_websocket_fhem ist what I need to get working? I hope you can help me. Thank you for your work so far.

regards,
Carsten
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: svenson08 am 24 Dezember 2014, 15:24:06
Hello,

i have only defined
define wsPort websocket 8080 global
define wsPort_fhem websocket_json


and set verbose to 5

when i send an command i found in the fhemlogfile
websocket_json: Undefined subroutine &websocket_json::AnalyzeCommandChain called

i fix this bug in the 10_websocket_json.pm, restart fhem and the send out of the websocket.html now works.
i attach the fixed version of the 10_websocket_json.pm.

In my case send and receive over websocket works. now i can make further tests.

svenson

Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: ntruchsess am 07 Januar 2015, 17:24:14
Hi Svenson,

thank you for reporting (and fixing) this bug. Will include your patch soon (didn't see your post more early as I was away from my computer during winter-hollidays...)

- Norbert
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: svenson08 am 07 Januar 2015, 19:27:08
Hi Norbert,

That's OK. It's running for me yet (successfully) as an test with my front end. And on the other side, it is not as if I can not manage myself.

I still have some ideas for improvement that I wanted to watch in the modules-code before, but have not gotten to it.

for example:

Then I noticed if you sent over the websocket a "invalid" command is sent an erroranswer ("Error" feedback from FHEM) to the client, but there I have not "see" on the client side. I wanted to see detail. I think with such a "fault" should somehow be returned as an error.

svenson
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: AlxDmr am 17 Januar 2015, 16:19:51
Hello Norbert,
I get a handshake error with your new code :
WebSocket connection to 'ws://192.168.1.12:8080/' failed: Error during WebSocket handshake: Sent non-empty 'Sec-WebSocket-Protocol' header but no response was received

Using Chrome under with windows 8.1, latest versions.

Regards,
    Alex.
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: AKNOT am 18 Januar 2015, 20:26:51
Hey there,

I tried to use your websocket module, but I don't get it to work. I installed JSON and Websocket from metacpan.org into /etc/perl/ (Im using a Raspberry with wheezy).
After restart the fhem log tells me:


2015.01.18 20:20:51 1: PERL WARNING: Use of uninitialized value in ref-to-glob cast at ./FHEM/00_websocket.pm line 310.
2015.01.18 20:20:51 1: PERL WARNING: syswrite() on unopened filehandle at ./FHEM/00_websocket.pm line 310.


As client I'm using a nodeJs Server. The connection is established successfuly (also the counter in the fhem ws device counts up), but when I do


fhem.sendCommand({
command: 'set KuecheLED off',
});


in the Connection Callback nothing happens.

What could be wrong?

Thanks in advance!
AKNOT
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: AlxDmr am 09 März 2015, 10:23:48
Hello Norbert,
sorry to bother you again about that but I still encounter the same problem when establishing the websocket connection using the JSON subprotocol. I use your code for connection :
https://raw.githubusercontent.com/ntruchsess/fhem-mirror/websockets/fhem/www/fhem_websocket.js

However there is an error, here is what happen :
The HTTP request from client to server is well formatted, it indicates that the client wnat to change protocole to websocket and use the subprotocol json.
The server handle the request and send back a HTTP answer specifying that it is OK to switch protocol to websocket BUT it does not specify that subprotocol is now json. As a consequence the web-browser throw an error saying that webserver does not properly handle the communication protocol.
WebSocket connection to 'ws://192.168.1.14:8080/' failed: Error during WebSocket handshake: Sent non-empty 'Sec-WebSocket-Protocol' header but no response was received
GET ws://192.168.1.14:8080/ HTTP/1.1
Host: 192.168.1.14:8080
Connection: Upgrade
Pragma: no-cache
Cache-Control: no-cache
Upgrade: websocket
Origin: http://www.clubic.com
Sec-WebSocket-Version: 13
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36 OPR/27.0.1689.76
Accept-Encoding: gzip, deflate, lzma, sdch
Accept-Language: fr-FR,fr;q=0.8,en-US;q=0.6,en;q=0.4
Sec-WebSocket-Key: J9gkEgJR38c7Ks4NFADLHA==
Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits
Sec-WebSocket-Protocol: json

HTTP/1.1 101 WebSocket Protocol Handshake
Upgrade: WebSocket
Connection: Upgrade
Sec-WebSocket-Accept: ZgItETgg6fXcea/Yzl/I/4AtvQ8=


I identified the code responsible for setting the subprotocol in 00_websocket.pm :
    if ($hash = $main::defs{$sname}) {
      foreach my $arg (keys %{$hash->{onopen}}) {
        my $protocol = eval {
          return &{$hash->{onopen}->{$arg}->{fn}}($cl,$hash->{onopen}->{$arg}->{arg},$cl->{resource},@{$cl->{protocols}});
        };
        Log3 ($sname,4,"websocket: ".GP_Catch($@)) if $@;
        if (defined $protocol) {
          Log3 ($sname,4,"websocket: protocol chosen '$protocol'");
          $cl->{hs}->res->subprotocol($protocol);
          $cl->{protocol} = $protocol;
          last;
        }
      }
    }


Unfortuntly I am unable to program in PERL language so I cannot even add some traces (using Log3 properly). could anyone help?
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: AlxDmr am 09 März 2015, 20:55:56
I added HTTP request and answer to my previous post, hope it can help.
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: AlxDmr am 15 März 2015, 10:52:37
The struggle continue...
I managed to add some traces using Log3 function, it appears that %{$hash->{onopen}} is "empty", meaning that no one subscribed to onopen.
So I think that 10_websocket_json.pm is not loaded. Looking back at the post of svenson08 I added the very same line of code in fhem?cfg :
define wsPort websocket 8080 global
define wsPort_fhem websocket_json


But when I save the file, it displays an error 0 Cannot load module websocket_json.

So... I think I am progressing although it is quite difficult for a newbies like me! What should I configure in order to load this module properly ? Of course I checked that the file 10_websocket_json.pm is present in /opt/fhem/FHEM

Regards,
    Alex.
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: AlxDmr am 15 März 2015, 20:22:36
Here I am!
I "just" forgot to install the JSON library...
so, in case it could be useful for someone, if you want to test the great work done by Norbert :
1) Download the files from his repository (00_websocket.pm, 10_websocket_json.pm, ...) as mentionned in a previous post of Norbert
2) Install library using cpan
sudo cpan App::cpanminus
sudo cpanm Protocol::WebSocket
sudo cpanm JSON
3) in fhem.cfg, add :
define wsPort websocket 8080 global
define wsPort_json websocket_json


then reboot and it should be done!
I got an "error" when loading fhem.cfg :
ERROR:

0 0

but modules are loaded and I am able to communicate

Für Norbert :Entschuldigung, ich bin ein bisschen dumm... Danke schöne für ihren werk, sehr nützlich !
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: AlxDmr am 31 März 2015, 18:45:48
Hello,
I get all the events now (plus the list of devices) but I cannot apply a "get DEVICE ..." command.
Well in fact it does not returns anything not does it trace something in the logs...
I checked via telnet that the command is working :
get EnO_UTE_008785CE measurement input energy
it emits an events to an update in the readings
However, when I try to apply the same command via websocket (using the websocket.html), I get nothing. I tried several syntax :
{"command":"get EnO_UTE_008785CE measurement input energy"}
{"command":"get", "device":"EnO_UTE_008785CE", "property":"measurement input energy"}
or directly in the javascript console :
fhem.sendCommand( {command:"get EnO_UTE_008785CE measurement input energy"} );
fhem.sendCommand( {command:"get", device:"EnO_UTE_008785CE", property:"measurement input energy"} );
fhem.sendCommand( {command:"get", arg:"EnO_UTE_008785CE measurement input energy"} );

What is the right way to send a "get" command ? looking at the 00_websocket.pm file I would have suspect at least one of them to work fine :(

     Alex.
Titel: Antw:Communication with Fhem via websockets ?
Beitrag von: AlxDmr am 31 März 2015, 21:10:01
I red the code of the file 10_websocket_json.pm (https://raw.githubusercontent.com/ntruchsess/fhem-mirror/websockets/fhem/FHEM/10_websocket_json.pm (https://raw.githubusercontent.com/ntruchsess/fhem-mirror/websockets/fhem/FHEM/10_websocket_json.pm)), there is one typo when processing the websocket command message :
AnylyzeCommand as to be AnalyzeCommand  (well, I guess)

Anyway, the code just after also fails for an unknown reason for me :
my $ret = AnalyzeCommandChain($cl, $command);
I added a Log3 just before and just after and the second one is never displayed in the logs. Every command seems to fails when processed by AnalyzeCommandChain, I tried to change AnalyzeCommandChain into AnalyzeCommand but there is no improvment.

I looked at 98_telnet.pm and this function is also used (AnalyzeCommandChain), maybe the first argument is not the right one?

Don't know what to do now :(