[SOLVED] MQTT and MQTT2_DEVICE

[bmentink]

Hi All,

I am very new to Fhem and have a question about setting up MQTT.

I have a mosquito server and have defined the MQTT device ok,  as the event window says it is connected, so far so good.
I then would like a client to listen to the server, I presume this is the MQTT2_DEVICE, if so I am having trouble linking it to my mqtt device.

With MQTT my device name is "mqtt" and with MQTT2_DEVICE the device name is "mqtt_dev", it is very confusing on how all the mqtt things are linked together ..

I also try to use MQTT2_CLIENT but I get this error: Unknown module MQTT2_CLIENT

I am running on a freeNAS server .. and have installed all the perl modules I could find for MQTT

Please help
Cheers

[Beta-User]

Welcome to our community.

First of all: did you update FHEM to the most recent version (note: updates in FHEM are initiated inside FHEM, not via OS mechanisms like apt-get)? MQTT2_CLIENT beeing missed indicates a rather old version (as all MQTT2-Modules had been under heavy development the last 12 Months).

Then you may not combine a TYPE=MQTT interface module with MQTT2_DEVICE, this is only for use with MQTT_DEVICE, MQTT_BRIDGE or MQTT_GENERIC_BRIDGE type of client definitions.

So first update (and restart) FHEM, then either use MQTTDEVICE as client device or define a MQTT2_CLIENT to support MQTT2_DEVICE type of clients (which imo is easier to configure than MQTT_DEVICE).

If you are rather new to the MQTT world and do not need MQTT data in a larger setup, you might also consider using MQTT2_SERVER instead of mosquitto+MQTT or mosquitto+MQTT2_CLIENT. This in most cases is the easiest way to handle MQTT protocol in FHEM.

[bmentink]

Many thanks for your reply & welcome.

I have downloaded the latest version of fhem, but I am running on a FreeNAS server (based on FreeBSD) which may not be as so up to date as Linux. So I guess I am stuck with standard MQTT for now. (It runs Perl5)

Yes, I have created my device with MQTT and have an instance of MQTT_DEVICE that I am trying to get working.
Can you offer an example of a "slider" control that uses the MQTT_DEVICE to publish a topic? I am struggling with the documentation ..

EDIT: Oops, I forgot the update, I now have the MQTT2 classes .. thanks. Could you give the above example based on MQTT2. Thanks.
I have managed to get all the Readings automatically from the MQTT2_DEVICE, but I am struggling to get a user gui that connects to the Readings. For example, How do I create a user Switch that sends publish on/off events .. I did the dummy switch, but that is as far as I got ..


Regards,
Bernie

[bmentink]

Hi again,

I notice that if I have autocreate turned on, my MQTT2_DEVICE gets loaded with Readings which is good and a log file auto associated.

So I turn off autocreate, rename the MQTT2 device that has a name that includes the CID to something readable. That works and log file gets renamed.

However if I turn off the power to the mqqt device that is publishing events, and turn back on again. THat device that got created does not update it's Readings.

I have to turn on autocreate again, and it creates a NEW device with a different CID that does update Readings.

How do I get a consistent device if the CID keeps changing on power cycle .. that just won't work for me.

PS: I am feeling a bit like a second class citizen, as there is no manual (Heimautomatisierung-mit-FHEM.pdf ) in English for me to read, so all I have is the on-line help.
Also, all youtube videos are only in German .

[Beta-User]

OK, the CID issue is rather hard to explain - even in German... (So I try to avoid that at least for now).

First: Do you really need en external MQTT server like mosquitto? The issue you described just doesn't exist when using MQTT2_SERVER instead.

Then: When using MQTT2_CLIENT as IO, I'd recommend to configure a "special" MQTT2_DEVICE instance to try to sort out corresponding information. This uses the structure of the topic tree. As a starting point, FHEM provides a so called attrTemplate named "A_00_MQTT2_CLIENT_general_bridge" which is able to deal with "typical" topic trees of some popular devices like shelly and tasmota.

For more templates (including several to easily configure tasmota-based switches), just use "set <yourMQTT2-device> attrTemplate ?" to get more info (in English... .).

To get help to configure your device(s) (if still needed after a short look at the attrTemplate feature), it would be helpful to provide more info, "list" or "RAW Definition" (I prefer the later in MQTT2 context) is needed (use code tags/ # button to make them easier to read). Based on that it would also be easier to explain the CID thing (if still necessary).


Wrt. your critics on the docu:
- The most important and up to date part is the commandref (what you see using help within FHEM). For official modules, it's mandatory for module authors to provide at least an english version. So this is the best authorized info _everyone_ gets. It may be very short, but in most cases, the most important keywords are explained to an extend allowing to get things up and running. There are a few exeptions, though... So just avoid modules not providing sufficient info - (imo explicitly the very popular DOIF module provides a really poor English version).

- Despite there's no 1:1 translation of the pdf (btw: the German version is also not up-to-date...), the wiki contains at least english versions of the most important basic instructions, see especially https://wiki.fhem.de/wiki/Quick-Start/en and https://wiki.fhem.de/wiki/First_steps_in_FHEM. You'll find also some more links to english versions of other important stuff there.

- Wrt. to "the youtube videos": There's especially one individual providing a serie of videos on FHEM. These videos - for good reason - are not part of the official docu; my personal view on the ones I had a look on: despite them containing often good ideas, they very often suffer from logical or technical mistakes and misleading information; additionally, they are not beeing updated. So just use them as kind of inspiration and better think twice before copy/paste these "instructions".

In case you have suggestions on how to enhance the English part of the official documentation on FHEM feel free to contribute...

[rob]

Hello bmentink.

Your goal seems to be getting a human readable friendlyname for your device. Am I right? Beta-User already gave you a deeper support. I wasn´t that fast 

What about setting an alias alternatively?

You could just put

Code Auswählen Erweitern

attr yourdevicename alias yourfriendlyname into the global command field on top of the page and hit enter.

I´m not sure if it helps in every case, but it may be worth giving it a try

Kind regards
rob

PS: Please don't feel this way. Imho 99% of all manuals, descriptions and stuff in the internet are not in German. Nobody translates it for me 
Beta-User gave you support in English. If I would ask vice versa a question in German in a English board, I surely won´t get any help    How should I feel then?

[bmentink]

Hi,

Thanks for the replies ..

Regarding the CID issue, that was the fault of the MQTT device, it was changing the CID every boot to stay unique, I have reprogrammed that to not do that, and all is well ..

The MQTT device I am trying to talk to is a custom ESP8266 module that is set up as an Aeroponics controller.

Regarding the MQTT client, I am using MQTT2_SERVER and MQTT2_DEVICE ..

I just need some help with setting up attributes to MQTT2_DEVICE to enable me to connect switches and set/read topics.
I have a list of readings on MQTT2_DEVICE so the server etc is working, and I can log some data ..
It is not clear to me how to publish/subscribe to the server using MQTT2_DEVICE, or should I be using a BRIDGE?
(I asked about how to set up a slider control earlier, still in the dark how to relate it to MQTT2_DEVICE)

Just an example of a switch publishing on/off, and a slider publishing number 1..100, my device is called MQTT2_AERO

I have attached photo of where I have got so far ..

Thanks again.

PS: Guys, I am not having ago about the documentation, just frustrated at not being able to put my hands on the right stuff to get started.I Apologize if my reply was harsh, it was not meant to be   

[Beta-User]

It is not clear to me how to publish/subscribe to the server using MQTT2_DEVICE, or should I be using a BRIDGE?

It should be possible to configure all that stuff within the one MQTT2_DEVICE for the Aeroponics controller. To send out any MQTT data/commands, a setList is needed. You may find a lot of ideas in the file behind the attrTemplate stuff: https://svn.fhem.de/trac/browser/trunk/fhem/FHEM/lib/AttrTemplate/mqtt2.template. To give more or detailed support, I'd need more info on your setup and especially the exact topic-payload-combinations to send out (just what mosquito_pub would issue; to make tests, you can use the publish command at your MQTT2_SERVER device).

For configuring other, less special devices (tasmota-based, e.g.), I'd recommend using the attrTemplate feature - then you may have a look at the completely configured devices allowing easier transfer to the Aeroponics thing?

PS: Guys, I am not having ago about the documentation, just frustrated at not being able to put my hands on the right stuff to get started.I Apologize if my reply was harsh, it was not meant to be   

Nevermind. Imo you just sounded frustrated, not harsh. For the future: just provide clear feedback in case you have constructive suggestions, this might be helpfull to others in the same situation . (Btw.: I myself had also been very frustrated about the partly outdated german and english info provided in the past - the only way out is to continuously try to improve these and keep up with the speed of developent... - For this reason there also exists a german wiki page trying to get a more or less complete introduction in MQTT2_DEVICE: https://wiki.fhem.de/wiki/MQTT2-Module_-_Praxisbeispiele. Perhaps you could use a automated translator to get at least some benefit from it.)

Please note: you are by far not the only guy lost in the variety of options, the MQTT protocol offers (or the client devices may require).

[bmentink]

Hi,

Please understand I am NOT using the mosquitto, I am using MQTT2_SERVER!

The two links you give are just as confusing for me ... too complicated.

I have had a look at the setList & getList attributes of MQTT2_DEVICE and they don't make sense either, they both talk about publishing topics, when you want to react to a topic from a sensor, why do you need to publish? (I am talking about sensors that send topics at a regular rate .. so I just need to display the info)

I appreciate the help but you have not answered my questions.

1. How do I set up a simple ON:OFF switch that publishes a topic, when the switch changes state  (i.e how do I connect a switch to a topic, I.E. dc_4f_22_18_e3_f5_72:/Garden/Aeroponics/Pump)
2. An example of a "slider" control sending a topic to the mqtt server when it changes value. (I.E. I want to send changes to dc_4f_22_18_e3_f5_72:/Garden/Aeroponics/Threshold)
3. I want to see in my Aeroponics ROOM the value of dc_4f_22_18_e3_f5_72:/Garden/Aeroponics/SolarIntensity,  it has the readingList item SolarIntensity)

I have attached a picture of my mqqt device and it's current readings ..

I don't know how I can be clearer ..

[Beta-User]

Try the following, please:

Code Auswählen Erweitern

attr MQTT2_AERO setList Pump_Status:ON,OFF /Garden/Aeroponics/Pump $EVTPART1\
Threshold:slider,0,0.5,10 /Garden/Aeroponics/Threshold $EVTPART1
attr MQTT2_AERO webCmd Pump_Status:Threshold
attr MQTT2_AERO stateFormat SolarIntensity
(Code snipplet is ment to be entered in a multiline command field; to get one, just klick on the "+" icon left to the command field or use the RAW-Option, see pictures included here: https://wiki.fhem.de/wiki/Import_von_Code_Snippets).
If this leads in the right direction, I'd suggest to add a link in the other post in the english corner and close it to avoid double postings. So other non-German speaking users may be forwarded to this thread?

Wrt. to the other points you mentioned:
- mosquitto_pub (and mosquitto_pub) are commandline tools that may be used with any MQTT server. So if one is used to these tools, it's no problem using them to communicate to a MQTT2_SERVER instance as well.
- To get single informations displayed in the room there are at least 4 options: the above "stateFormat SolarIntensity" beeing one of them (you may add more readings and text also).

For the future, please do not use pictures but RAW definitions instead (see above) - so anyone could easily transfer the device e.g. to a test system .

[bmentink]

Hi,

Thanks. Would be good if explained how the lines of code work for my education.

What I see is not very GUI friendly though (have a look at attached image) the slide has no border so di not see it initially, it is the box with 22. It does work though setting the threshold.

The pump control is the empty box, if I press on it I see a drop-down list ON/OFF if I select ON, the pump does not turn on ...
(I will check that it expects ON/OFF, it may be 0/1 )

The other number 5 is the Intensity, so that works  ..

How do I nice them up, there needs to be labels, borders, I thought these would be there ..

I am sorry, I am used to OpenHAB which does everything for you. BUt I do like the way Fhen is totally configurable ..

NOTE: I have renamed the device to ESP8266 .. just to be confusing

[rob]

Hi.

Could you please throw

Code Auswählen Erweitern

list ESP8266
into the command box on top and paste the result here?

Kind regards
rob

[bmentink]

Sure, sorry still a picture as I havn't worked out RAW yet ..

[Beta-User]

Please add a final post in the other thread referrring to the discussion to be continued here and close it (this is an exeptional procedure, due to the double posting), so others may find the following remarks also:

The setList (and very similar also getList) works as follows:

Code Auswählen Erweitern

Pump_Status:ON,OFF /Garden/Aeroponics/Pump $EVTPART1
- Pump_Status refers to the reading name to be set. As you already got the status (for local switching, I guess) of the pump sent by the Aeroponics-hardware (seems to be a ESP8266 based self-made thing?), I've chose (at first instance, more to be explained later...) this as a starting point.
- the ":" seperates the reading name from a value selection (or a widget in case of the threshold, see below).
- The rest is a topic / payload pair, $EVTPART1 there represents the selected value returned by the widget we use in part 1. (Note: if needed, you may use a perl function returning topic and payload here (in {}, as usual).

Every line represents one reading (exception: state, see the code from supernova1963 in the other thread; I'll come back to that later).

Wrt. to the "look" of the whole thing: It's completely customizable, you may use all the widgets you find here: https://wiki.fhem.de/wiki/FHEMWEB/Widgets, they also work in an MQTT2_DEVICE context.
So eg. this might be alternatives (note: for testing purpose with 0/1 as pump payload):

Code Auswählen Erweitern

attr MQTT2_AERO setList Pump_Status:uzsuToggle,1,0 /Garden/Aeroponics/Pump $EVTPART1\
Threshold:colorpicker,BRI,0,0.5,10 /Garden/Aeroponics/Threshold $EVTPART1

I'll make some annotations to the differences wrt. the on/off thing later in the other thread, as this may be valueable for others also.
[...]
So is there any description on the MQTT interface of your Aeroponics system? Especially: you provided (some) info about the topic structure, but not really about the payload (seemed to be ON and OFF wrt. to the pump) nor the range of the threshold.

So my guess with ON/OFF seems to be wrong. Please provide info about what to send (or at least a link to eg. a github repo with the code on the ESP8266), if above 0/1 doesn't work.

Wrt. to labeling: The look in FHEMWEB can be influenced in several ways, also using additional modules (represent one or more reading values in additional FHEM-devices) is possible. E.g. for commands within the device the webCmdLabel attribute is available (see wiki link above or https://wiki.fhem.de/wiki/DeviceOverview_anpassen). to some extent stateFormat can also help. As there may be some more changes to come, I'd suggest to have a look at this in a later phase. For now, please just make a short test with

Code Auswählen Erweitern

attr MQTT2_AERO stateFormat Solar: SolarIntensityPlease: just copy text, do not add pictures.

see also "list -r ESP8266" or "list -r MQTT2_AERO"(in case both are there, decide, which one to use for the future and delete the other one. Otherwise, we will risk non consistent results from the autocreate feature).

[bmentink]

Hi @Beta-User,

Wow, thanks, that really helps a lot, I can now start to link pieces together ..

I have the pump switch working, I did not make it clear to you that "Pump" was the topic to send, "Pump_Status" is what is returned from the ESP8266 as feedback that the pump really did turn on. I modified the code example you sent to reflect that, and it worked.
(ON/OFF was correct)

Regarding the GUI. I will read up on the Widgets, I tried the colorpicker instead of slider and that is better but number hard to read, but it will do for now .. what I would really like is a GUI that looks like "example1" given on your home page, that would be awesome, but I have no idea how to get from here to there at the moment ..
(The label on Solar Intensity works ..)
I could not get uzsuToggle to work correctly. It looks ok in the device window, but when I go to my Room it shows up weird with an initial label of "Intensity", then when you click on it, it says "Pump" .... in the device setup it correctly has ON/OFF when you toggle ..
Not sure what goes on there ..

But I am now making good progress, thanks again for the help ..

PS: I tried closing the other thread, but it would not let me ..
PPS: I only have ESP8266 device, I had renamed MQTT2_AERO to ESP8266, there is not two devices ..

Cheers