[SOLVED] MQTT and MQTT2_DEVICE

[Beta-User]

Thanks for your positive feedback, glad to hear you making good progress .

Please note:
- As Pump seems to be kind of general on/off, you should try to get this (in both directions) into a "special" reading called "state". This e.g. gives the option to work with so called SetExtensions including especially "on-for-timer" commands.
(To get that to work properly, we will need on/off as lower case setters (seperate for both, that part is easy) and feedback interpretation (that's a little more advanced)).
(Btw.: Perhaps with your growing background knowledge it's now a good idea to have a look in the mentioned attrTemplate-file ).

- Wrt. to the look and feel, you'd better provide exact links

If you provide more info on the whole thing (and some RAW codes from FHEM as text), it would be easier to explain the necessary changes.

- Thx for the last post in the other thread, please additionally use the "close" button (locatet somewhere below the first post).

[bmentink]

Hmmm,
For some reason my buttons are not sending ON/OFF events anymore, here is the device text.

define ESP8266 MQTT2_DEVICE dc_4f_22_18_e3_f5_72
attr ESP8266 IODev mqtt2_server
attr ESP8266 readingList dc_4f_22_18_e3_f5_72:/Garden/Aeroponics/SolarIntensity:.* SolarIntensity\
dc_4f_22_18_e3_f5_72:/Garden/Aeroponics/SolarAccumulation:.* SolarAccumulation\
dc_4f_22_18_e3_f5_72:/Garden/Aeroponics/Conductivity:.* Conductivity\
dc_4f_22_18_e3_f5_72:/Garden/Aeroponics/Pump_Status:.* Pump_Status\
dc_4f_22_18_e3_f5_72:/Garden/Aeroponics/Manual_Auto:.* Manual_Auto\
dc_4f_22_18_e3_f5_72:/Garden/Aeroponics/Threshold:.* Threshold
attr ESP8266 room Aeroponics
attr ESP8266 setList Pump:ON,OFF /Garden/Aeroponics/Pump $EVTPART1\
Manual_Auto:ON,OFF /Garden/Aeroponics/Manual_Auto $EVTPART1\
Threshold:colorpicker,BRI,0,1,100 /Garden/Aeroponics/Threshold $EVTPART1
attr ESP8266 stateFormat Solar Intensity: SolarIntensity
attr ESP8266 webCmd Pump:Manual_Auto:Threshold

setstate ESP8266 Solar Intensity: 14
setstate ESP8266 2019-09-05 10:23:37 Conductivity 1297
setstate ESP8266 2019-09-05 09:18:26 Manual_Auto 0
setstate ESP8266 2019-09-05 08:46:10 Pump_Status 0
setstate ESP8266 2019-09-05 09:18:26 SolarAccumulation 293
setstate ESP8266 2019-09-05 09:18:25 SolarIntensity 14
setstate ESP8266 2019-09-05 10:19:43 Threshold 27
setstate ESP8266 2019-09-05 10:19:43 state Threshold

From event window: Note Pump has no value sent ...

2019-09-05 10:40:53 MQTT2_DEVICE ESP8266 Pump
2019-09-05 10:40:54 MQTT2_DEVICE ESP8266 Conductivity: 1299
2019-09-05 10:40:54 MQTT2_DEVICE ESP8266 SolarIntensity: 50
2019-09-05 10:40:54 MQTT2_DEVICE ESP8266 SolarAccumulation: 400

EDIT: My bad, my device expects 1:0 not ON:OFF as I indicated earlier ... all working again.
Even though the topic sends 0:1, is there a way for the Button to display ON:OFF and display the name of the button?
(I am using uzsuSelectRadio  buttons now)

EDIT2: I did do a "eventMap 1:ON 0:OFF" so my events get displayed as ON/OFF, but it did not help with the button values.

Thanks

[bmentink]

- Thx for the last post in the other thread, please additionally use the "close" button (locatet somewhere below the first post).

I only see a "Remove" button, and like I said, It won't let me ..

Wrt. to the look and feel, you'd better provide exact links

https://fhem.de/TabletUI_1.png

[Beta-User]

OK, thanks for the RAW code (no need to change "defmod", see "help defmod", code tags instead of quote is better (# icon)).

I'd suggest the following changes, but unfortunately couldn't fully test it, especially the handling of the Pump_Status topic is just a first guess...

Code Auswählen Erweitern

attr ESP8266 readingList /Garden/Aeroponics/SolarIntensity:.* SolarIntensity\
/Garden/Aeroponics/SolarAccumulation:.* SolarAccumulation\
/Garden/Aeroponics/Conductivity:.* Conductivity\
/Garden/Aeroponics/Pump_Status:.* { $EVENT ? {"state"=>"on"} : {"state"=>"off"} }\
/Garden/Aeroponics/Threshold:.* Threshold
attr ESP8266 setList on:noArg /Garden/Aeroponics/Pump 1\
off:noArg /Garden/Aeroponics/Pump 0\
Manual_Auto:uzsuToggle,ON,OFF /Garden/Aeroponics/Manual_Auto $EVTPART1\
Threshold:colorpicker,BRI,0,1,100 /Garden/Aeroponics/Threshold $EVTPART1
attr ESP8266 setStateList on off
attr ESP8266 stateFormat state\
Solar Intensity: SolarIntensity
attr ESP8266 webCmd Manual_Auto:Threshold
To see if it really works you should do two tests:
- try to switch the thing from outside FHEM (local switch or web interface) => state should change, and
- unpower it, switch from within FHEM => you should see the state remaining set_on or set_off which leads to a lamp with a exclamation mark to be dispayed in the device overview.

Wrt. to the deletion of "dc_4f_22_18_e3_f5_72:/Garden/Aeroponics/Manual_Auto:.* Manual_Auto\" from the readingList: Under any circumstance avoid using the same topic for sending commands/info _to_ the target hardware and receive feedback _from_ it. This may lead to infinite loops....

In case state feedback does not work properly, I'd need the MQTT traffic going back and forth. To get that, use the "rawEvents" attribute of the MQTT2_SERVER, in our case ".*" will be ok.
Then open a second browser window and do some switch procedures from within FHEM, then some locals in the ESP's web interface (assuming, it has one...).
Btw.: This is a good instrument for debugging MQTT issues .


Wrt. to the "closing": There should be two additional buttons (in my case: on the left below the ones you cited, but I may have different settings for forum use). Afaik, you are allowed to close all the threads you opened.

Wrt. to the UI thing:

The linked picture shows a different UI called FTUI. This needs further customisation, so I'd recommend to have a look at that as soon as you are familiar with the basic functionality of the internal data and "timer" and "event" handling FHEM offers.

My personal focus is more on "automation", not too much on shiny UI design, so I'm only familiar with FHEMWEB, especially it's "f18" style. There's a second, rather new and not yet officially distributed Java-based style for FHEMWEB called "flex" you may have a look at. It may offer a more shiny UI without the need to "learn the FTUI language" in parallel to FHEM mechanisms.

If you want absolutely to start with UI customisation, you may search the forum for the keyword "FUIP". This is a tool for easy creation of FTUI (-alike(?)) pages, the tool's author (!) also provides some video material on that. (Please search the forum for it, imo this should not be part of this thread).

[bmentink]

Great,  I will have a look at that tomorrow ... it is my bed time ..

Quick question: We seem to have lost the "Pump" button in your above code, I don't see it in webCmd.

[Beta-User]

It's not "lost" - your pump schould be switchable via the "state" icon which now is part of the stateFormat . So no additional graphical element is needed any longer.

Wrt. to the look (and perhaps functionality) we might get back to that later, in addition a devStateIcon attribute might be usefull. As soon as state works bidirectional, you may have a second look on "Device Overview anpassen".

See you "tomorrow"  .

[bmentink]

@Beta-User
Hi, I have tried your changes, but have seemed to go backwards now to a broke system again ..

I end up with a Toggle button, but it has the words "set 0" or "set 1" on it, and when you press it the MQTT topic
sent is "set 0/1" I also end up with a light bulb that does nothing (and I have no idea what change added it )

You do realize that the Topic "Pump"  with payload 0/1 is what is sent to turn the pump on/off and "Pump_Status"
with payload 0/1 is what is replied from the device in confirmation?

I presume your changes where intended to light the bulb when I get a "1" in Pump_status?

What you may not be aware off, is that the Manual_Auto topic (payload 1/0) needs to be in Manual mode (0)
to operate the "Pump" switch .. I had to change that line to:

Code Auswählen Erweitern

Manual_Auto:uzsuToggle,1,0 /Garden/Aeroponics/Manual_Auto $EVTPART1

Look forward to further changes ..

[Beta-User]

OK, so this attempt was too eager, the devStateIcon seems not to automatically work.

As a first step, please just add on/off commands to the webCmd and change the auto/manual thing; now should be as follows:

Code Auswählen Erweitern


attr ESP8266 readingList /Garden/Aeroponics/SolarIntensity:.* SolarIntensity\
/Garden/Aeroponics/SolarAccumulation:.* SolarAccumulation\
/Garden/Aeroponics/Conductivity:.* Conductivity\
/Garden/Aeroponics/Pump_Status:.* { $EVENT ? {"state"=>"on"} : {"state"=>"off"} }\
/Garden/Aeroponics/Threshold:.* Threshold
attr ESP8266 setList on:noArg /Garden/Aeroponics/Pump 1\
  off:noArg /Garden/Aeroponics/Pump 0\
  Manual_Auto:uzsuToggle,0,1 /Garden/Aeroponics/Manual_Auto $EVTPART1\
  Threshold:colorpicker,BRI,0,1,100 /Garden/Aeroponics/Threshold $EVTPART1
attr ESP8266 setStateList on off
attr ESP8266 stateFormat state\
Solar Intensity: SolarIntensity
attr ESP8266 webCmd on:off:Manual_Auto:Threshold

The most important point will be to get the ESP's feedback on on/off commands in "state" reading as text on/off. If the switching itself works with the seperate icons from webCmd, I'll need the info what's going on on the MQTT traffic side as already described.

(Now, it's up to me to go to bed...)

[bmentink]

You have a good night.

When you wake, please look at the attached image (sorry, I know you don't like pictures, but it is the only way to describe what is going on)

I can click on the on/off words and the light bulb changes state with the "!", that is because I can not change the state of Manual_Auto, it still has the "set 0/1" thing going on .. if I keep pressing SET  in the device window, it keeps adding "set" so I can get "set set set 0" sent ..

On the MQTT message front, the message that gets sent when pressing the Pump "on" button is "set_on" and for the "off" button "set_off"
So my device is not going to respond to those, as it is expecting 1 or 0 ..
The "state" variable has "set_on/off" as well ..

Very odd ..

The way I see this working, is:

1. The light bulb should "ONLY" respond to Pump_Status topics of "1" or "0", it should NOT be tied to a switch.
2. The "Pump" switch should indicate on/off to the GUI, but send topics "1" or "0"
3. For "Manual_Auto" switch, it should as 2. above, but should indicate a status from the subscribed topic of the same name,
(I know that is not ideal, but my device takes care of any "loops")

[Beta-User]

When you wake, please look at the attached image (sorry, I know you don't like pictures, but it is the only way to describe what is going on)

Good Evening/Morning, I took a look .

But still, I'd get a better "picture" of what's really going on, if you'd added two things:
- The output of RAW definition as text (in code-tags) => This allows me (and others) to get the (besides some skinning identical) picture you attached in my own test system and includes some additional info that may have been usefull
- The log of the MQTT traffic you are referring to (the "set_on" is the reading value as long as there is no feedback from the ESP, but the payload sent out via MQTT should be different). If you want to use other than "rawEvent .*" output of MQTT2_SERVER in the Event-Monitor, use tools like mosquito_sub or MQTT.fx to listen to the MQTT2_SERVER from outside FHEM.

So please provide both of the above.

The way I see this working, is:

1. The light bulb should "ONLY" respond to Pump_Status topics of "1" or "0", it should NOT be tied to a switch.
2. The "Pump" switch should indicate on/off to the GUI, but send topics "1" or "0"

Imo, point 2 _should_ already work pretty close to that as a result of e.g.

Code Auswählen Erweitern

attr ESP8266 setList on:noArg /Garden/Aeroponics/Pump 1\This should lead to payload "1" when setting the device on (but one can _only_ see this to work properly in the MQTT traffic, _not_ (really) in the FHEM-device-Overview or readings). As this seem to be kind of main switch, I'd recommend to use the "state" reading for this info - this is what you probably find to be somehow irritating.
If it's not kind of main switch, we'll have to get back to a different notation, but for the moment, I'd ask for some patience on that issue. You may decide _after_ it's working the "main switch way" if you insist in  expectation as described in your point 1. .

(If you even want to get more irritated, delete the setStateList attribute and do some switching of different elements like also Threshold ).

3. For "Manual_Auto" switch, it should as 2. above, but should indicate a status from the subscribed topic of the same name,
(I know that is not ideal, but my device takes care of any "loops")

Imo, there are more arguments to keep in mind. First one:
You finally want to have a shiny UI. There are some more options than only FTUI/FUIP. One is to use MQTT based other HA software like OpenHAB . So one can combine the total flexibility within FHEM with a shiny UI... But: How to get OpenHAB to distinguish between FHEM commands and info coming from the device itself? They are on the same topic, but different to within FHEM, the origin (CID) most likely is no longer visible.

Second:
I'm not sure, but also MQTT2_SERVER _might_ filter out "double" info to avoid loops. So this might be the cause we don't get proper feedback on switching "Manual_Auto".

So I'd really recommend to use different topics for commands and feedback, shouldn't be very hard to implement in the firmware. (You may add the code and ask for help, there's quite a lot of guys hanging around here with "some" experience on firmware coding... Just make sure to delete "critical" info like WiFi passwords).

[bmentink]

HI there, here is the RAW output of the device:

Code Auswählen Erweitern

define ESP8266 MQTT2_DEVICE dc_4f_22_18_e3_f5_72
attr ESP8266 IODev mqtt2_server
attr ESP8266 readingList /Garden/Aeroponics/SolarIntensity:.* SolarIntensity\
/Garden/Aeroponics/SolarAccumulation:.* SolarAccumulation\
/Garden/Aeroponics/Conductivity:.* Conductivity\
/Garden/Aeroponics/Pump_Status:.* { $EVENT ? {"state"=>"on"} : {"state"=>"off"} }\
/Garden/Aeroponics/Manual_Auto:.* Manual_Auto\
/Garden/Aeroponics/Threshold:.* Threshold
attr ESP8266 room Aeroponics
attr ESP8266 setList on:noArg /Garden/Aeroponics/Pump 1 \
off:noArg /Garden/Aeroponics/Pump 0\
Manual_Auto:uzsuToggle,1,0 /Garden/Aeroponics/Manual_Auto $EVTPART1\\
Threshold:colorpicker,BRI,0,1,100 /Garden/Aeroponics/Threshold $EVTPART1
attr ESP8266 setStateList on off
attr ESP8266 stateFormat Solution_Conductivity: Conductivity\
state\

attr ESP8266 webCmd on:off:Manual_Auto:Threshold

setstate ESP8266 Solution_Conductivity: 1285\
off\

setstate ESP8266 2019-09-06 21:06:33 Conductivity 1285
setstate ESP8266 2019-09-06 16:13:08 Manual_Auto 1
setstate ESP8266 2019-09-06 07:58:38 Pump_Status 0
setstate ESP8266 2019-09-06 21:06:34 SolarAccumulation 1005
setstate ESP8266 2019-09-06 21:06:34 SolarIntensity 5
setstate ESP8266 2019-09-06 16:13:08 Threshold 26
setstate ESP8266 2019-09-06 21:03:22 state off

Please let me know if I have done anything wrong ..

Cheers

[Beta-User]

OK, (I indeed have been a little frustrated about having to repeat some earlier writings, but for now, everything's ok, beside the fact, the MQTT traffic still is missing )

For the moment, the most important info seems to be this:

Code Auswählen Erweitern

setstate ESP8266 2019-09-06 21:03:22 state off

So I'm quite sure, there is feedback on the pump-on command and this feedback is handled correctly as on/off in state.

Now the bulb icon should be shown without the red exclamation mark (might take some short time to change) and act as kind of toggle button. Can you please confirm on both? (And on the fact, the red ! will stay, when the ESP is not reachable, eg. by unpowering it).

If yes: why not only use this as grafical element to switch the pump and reduce webCmd to "Manual_Auto:Threshold"?
To avoid future confusion on that delete the Pump_Status reading, as this will not be updated any longer:

Code Auswählen Erweitern

deletereading ESP8266 Pump_Status
For the Auto/Manual thing:
We could try to "translate" 1/0 to better readable format, but this would require some more work better to be done as soon as set topic and feedback topic are different. Then we could use some Perl, pretty similar to what is done with "Pump_Status". Are you working on the topic issue?

[bmentink]

I am sorry you have been frustrated, so am I. You must remember you are a long time developer, I have just been on this code a few days ..

The line:

Code Auswählen Erweitern

setstate ESP8266 2019-09-06 21:03:22 state off  rarely happens ....

Most of the time it looks like this:

Code Auswählen Erweitern

setstate ESP8266 2019-09-07 08:23:32 state set_on

I havn't given you MQQT traffic yet because I am not sure where to get it from, if you mean the event monitor,
here is a sample:

When I activate the pump(on,off):

Code Auswählen Erweitern

2019-09-07 08:29:16 MQTT2_DEVICE ESP8266 Conductivity: 1251
2019-09-07 08:29:16 MQTT2_DEVICE ESP8266 set_on
2019-09-07 08:29:17 MQTT2_DEVICE ESP8266 Conductivity: 1250
2019-09-07 08:29:18 MQTT2_DEVICE ESP8266 Conductivity: 1250
2019-09-07 08:29:18 MQTT2_DEVICE ESP8266 set_off
2019-09-07 08:29:19 MQTT2_DEVICE ESP8266 Conductivity: 1251
2019-09-07 08:29:20 MQTT2_DEVICE ESP8266 Conductivity: 1251
2019-09-07 08:29:21 MQTT2_DEVICE ESP8266 Conductivity: 1252

When I hit The Manual_Auto sw (Twice):

Code Auswählen Erweitern

2019-09-07 08:31:07 MQTT2_DEVICE ESP8266 Conductivity: 1250
2019-09-07 08:31:07 MQTT2_DEVICE ESP8266 Manual_Auto: set 0
2019-09-07 08:31:08 MQTT2_DEVICE ESP8266 Conductivity: 1250
2019-09-07 08:31:09 MQTT2_DEVICE ESP8266 Manual_Auto: set set 0
2019-09-07 08:31:09 MQTT2_DEVICE ESP8266 Conductivity: 1251
2019-09-07 08:31:10 MQTT2_DEVICE ESP8266 Conductivity: 1251
2019-09-07 08:31:11 MQTT2_DEVICE ESP8266 Conductivity: 1251

Now you see what I am getting? The messages are all wrong!
Also, The bulb always has a question mark.

[Beta-User]

Hmm, to see also the MQTT traffic in the EventMonitor, you have to set an attribute on the IO (TYPE=MQTT2_SERVER).
In your case:

Code Auswählen Erweitern

attr mqtt2_server rawEvents .*

If you get "sometimes" the right result, I'd assume there's a communication problem on the WiFi side. Please have a look at this and make sure, we have a reliable connection. Otherwise we would have to use extended MQTT functionality (QoS ?); that would also be new to me.

Wrt to knowledge level: "developer" is somehow ridiculous, and mostly due to my activities in the attrTemplate file for MQTT2_DEVICE. So I know quite some things wrt to "skinning" and (functional) "design" of MQTT2_DEVICE for FHEMWEB, but there the story ends very quickly ... Additionally, I'm not too familiar in explaining things in English .

[bmentink]

No, it is nothing to do with the WiFi, my original code works fine, both pump & Manual_Mode do the right thing, and I get all the MQTT traffic messages.

With your modifications, I don't.
See:

Code Auswählen Erweitern

2019-09-07 19:04:24 MQTT2_DEVICE ESP8266 Conductivity: 1327
2019-09-07 19:04:24 MQTT2_SERVER mqtt2_server /Garden/Aeroponics/Conductivity:1327
2019-09-07 19:04:24 MQTT2_DEVICE ESP8266 set_off
2019-09-07 19:04:25 MQTT2_DEVICE ESP8266 Conductivity: 1326
2019-09-07 19:04:25 MQTT2_SERVER mqtt2_server /Garden/Aeroponics/Conductivity:1326
2019-09-07 19:04:26 MQTT2_DEVICE ESP8266 Conductivity: 1327
2019-09-07 19:04:26 MQTT2_SERVER mqtt2_server /Garden/Aeroponics/Conductivity:1327
2019-09-07 19:04:27 MQTT2_DEVICE ESP8266 set_on
2019-09-07 19:04:27 MQTT2_DEVICE ESP8266 Conductivity: 1327
2019-09-07 19:04:27 MQTT2_SERVER mqtt2_server /Garden/Aeroponics/Conductivity:1327
2019-09-07 19:04:28 MQTT2_DEVICE ESP8266 Conductivity: 1327
2019-09-07 19:04:28 MQTT2_SERVER mqtt2_server /Garden/Aeroponics/Conductivity:1327
2019-09-07 19:04:29 MQTT2_DEVICE ESP8266 Conductivity: 1327
2019-09-07 19:04:29 MQTT2_SERVER mqtt2_server /Garden/

Now here I set the pump off and on, I see the messages "set_off and "set_on, as above, but there are no MQTT traffic as a result, so something is wrong, the messages are not being sent. Anyway, "1" and "0" should be sent, as I said many times.

You have not explained why I am seeing the messages "set_off" etc, where does the "set" come from? That got introduced with your changes.

I think I will go pack to my code that worked (I saved the .cfg file) and then look further at the UI improvment/frontends etc as you mentioned.

At least with my original code I have a "working system". But many thanks for the learning experience.

Cheers,