Wireless M-Bus für CUL

Begonnen von tostmann, 12 Juni 2014, 17:34:32

Vorheriges Thema - Nächstes Thema

kaihs

Maybe the data you received is indeed not from your meter but from some unknown gas meter.
But the high RSSI of -38 indicates that the sender must be be quite near to your receiver.
Banana Pi, Add-On Board mit 1.8" TFT LCD und IR-Sender, CULFW V1.61, div. Homematic Komponenten, Pollin Funksteckdosen, Selbstbau CUL433 MHz, Jeelink Clone, EC3000
Selbstbau CUL868MHz für Wireless M-Bus, SIGNALduino mit Logilink Temp.-sensoren und Auriol Wetterstation

bilbolodz

Zitat von: kaihs am 28 Dezember 2017, 12:59:46
But the high RSSI of -38 indicates that the sender must be be quite near to your receiver.
Distance between meter and culfw antenta is about 1,5m open space so probably it's my meter.

Ingram

Zitat von: kaihs am 28 Dezember 2017, 12:42:18
Are the different prefixes for the different modes necessary? The payload should be the same for all modes, shouldn't it?
I suggest you send the data just as before, i.e. bXXXX, independent of the mode used.
This is already the case for mode T and S and the current WMBUS module isn't be able to handle the new prefixes.

The 't' and 'c' prefix are unnecessary, however we need to distinguish between format A and format B frames. T- and S-mode use only format A as that was defined originally, with C-mode they also added format B, which introduced a byte before payload for C-mode which indicates whether the upcoming frame is format A or format B. The payload itself does not contain any information about the frame type that I know of (other than trying to parse it as format A and format B, see which one passes, which one fails - what happens when both pass?). Formats are described well in ST's application note.

kaihs

Zitat von: Ingram am 28 Dezember 2017, 13:44:24
The 't' and 'c' prefix are unnecessary, however we need to distinguish between format A and format B frames. T- and S-mode use only format A as that was defined originally, with C-mode they also added format B, which introduced a byte before payload for C-mode which indicates whether the upcoming frame is format A or format B. The payload itself does not contain any information about the frame type that I know of (other than trying to parse it as format A and format B, see which one passes, which one fails - what happens when both pass?). Formats are described well in ST's application note.

Interesting. The OMS Spec 4.1.2 doesn't specify format B.
From section 5.2.1
Zitat
The Data Link Layer with Frame Format A as described in [EN 13757-4:2013] shall be used
for wireless communication.

I don't have access to the EN standard documents, maybe it's described there.
And your meter is using format B?
Banana Pi, Add-On Board mit 1.8" TFT LCD und IR-Sender, CULFW V1.61, div. Homematic Komponenten, Pollin Funksteckdosen, Selbstbau CUL433 MHz, Jeelink Clone, EC3000
Selbstbau CUL868MHz für Wireless M-Bus, SIGNALduino mit Logilink Temp.-sensoren und Auriol Wetterstation

Ingram

Zitat von: kaihs am 28 Dezember 2017, 19:56:45
I don't have access to the EN standard documents, maybe it's described there.
And your meter is using format B?

The TI application report on receiving both C and T mode frames simultaneously also shows the existence of format B. If the last byte of the actual syncword is 0xCD, then format A is used and if it is 0x3D, then format B is used. This is how I could tell. Also because of the length byte and CRC matching the description of format B in the ST application note, it has to be format B.

I don't have access to the standard either and this is the most frustrating part about it. The application notes by chipmakers were helpful enough to get it working though, each one revealing a bit of the details about the standard but useless alone.

bilbolodz

Zitat von: bilbolodz am 28 Dezember 2017, 12:57:19
I've wrote an message to the vendor. I'm NOT expecting any replay but maybe......
I've got an answer from vendor. As it was expected they don't wont to provide any specifications of protocol. The most bizarre is 'the reason": they are using some "law tricks" about license term and similar stuff. If someone is going to buy some equipment from Apator I strongly NOT recommend buying it.

kaihs

Zitat von: Ingram am 28 Dezember 2017, 13:44:24
The 't' and 'c' prefix are unnecessary, however we need to distinguish between format A and format B frames. T- and S-mode use only format A as that was defined originally, with C-mode they also added format B, which introduced a byte before payload for C-mode which indicates whether the upcoming frame is format A or format B. The payload itself does not contain any information about the frame type that I know of (other than trying to parse it as format A and format B, see which one passes, which one fails - what happens when both pass?). Formats are described well in ST's application note.

To be compatible with culfw versions not supporting C Mode could you use a prefix that is not a valid hexadecimal digit?
E. g. something like
bxa for format A
bxb for format B?
That way it will be possible to distinguish the new data format from the old one that starts with the payload directly after the 'b'.
Banana Pi, Add-On Board mit 1.8" TFT LCD und IR-Sender, CULFW V1.61, div. Homematic Komponenten, Pollin Funksteckdosen, Selbstbau CUL433 MHz, Jeelink Clone, EC3000
Selbstbau CUL868MHz für Wireless M-Bus, SIGNALduino mit Logilink Temp.-sensoren und Auriol Wetterstation

Ingram

Zitat von: kaihs am 29 Dezember 2017, 17:23:09
To be compatible with culfw versions not supporting C Mode could you use a prefix that is not a valid hexadecimal digit?
E. g. something like
bxa for format A
bxb for format B?
That way it will be possible to distinguish the new data format from the old one that starts with the payload directly after the 'b'.

S and T modes via `brs` and `brt` preserve the old output. The prefix is currently only added in C-mode `brc`, which you have to knowingly configure before receiving frames. But sure, it can be changed so that it would be easier to parse. You can probably tell from experience better how the output should be formatted if you want to easily parse it in FHEM or elsewhere.

bilbolodz

Zitat von: kaihs am 28 Dezember 2017, 12:17:36Another possibility is reverse engineering it yourself as has been done for Techem heatcostmeters.
I've made some research and I've "partial results". I've "raw data" (raw means messages from log):
2018-01-01_08:18:02 WMBUS_APT_00036030_3_3 Unsupported CI Field a0, remaining payload is 31fb32000c40002b01080401011234dc4e0a0306ff0c394c2a00c7902a009dec2a007
1aa2b00a29c2d00c7722f001d4930001aaf31006e083200ac583200c8a03200f8fa3200030000000037c5081f000000002000000000000023000000002400000000000027000000280000000000002
c00200301050c2f000000000000
from over 3 weeks

It looks that:
1) Messages are not encrypted (great)
2) Messages are always 128 bytes long

I've made an assumption that data is correctly aligned into 8 bits chunks (looks fine for me but it could be some bit stuffing into messages). During 3 weeks my logs changes at "9 places", (bytes in message indexed from 0 to 127):

Index - meaning
76 - day number - changes +1 every day
11 - day of month (for 24.12 it's 24)
10 - day of week (strange because Friday is 1 ... Sunday is 3 .... Monday is 4 ... Thursday is 7)
9 - hour
8 - minute
7 - seconds


it looks that internal clock of meter is slight "of of sync": e.x. 15:19:26 (time of receive by fhem) is 15:03:08 according meter

2,1,0 are water volume ([2]*256*256 + [1] * 256 + [0])


I don't know yet how to convert these number to "square meter" but I will try to make some photos of my real meter and do some moths.

So decoded above record looks like:

2018-01-01 (Mon) 08:18:02 Day number: 197 Day of week: 4 Day of month: 01 time: 08:01:43 Counter: 3341105

Now a few question:

1) Is there any "standard" factor for converting "meter counter" into square meter?
2)  What is the  usually size of  meter counter number ("number of bytes")? There is a some "free space" between index 2 and 7 but I don't know which bytes belongs to volume counter but which could be "other data".
3) How to patch (in elegant way) FHEM to support my meter?

Few more examples:

2017-12-24_08:29:10 WMBUS_APT_00036030_3_3 Unsupported CI Field a0, remaining payload is a1e532000c4000370c0803180c1134dc4e0a0306ff0b394c2a00c7902a009dec2a007
1aa2b00a29c2d00c7722f001d4930001aaf31006e083200ac583200c8a032006c092a00030000000037bd081f000000002000000000000023000000002400000000000027000000280000000000002
c00200301050c2f000000000000
2017-12-24_08:30:58 WMBUS_APT_00036030_3_3 Unsupported CI Field a0, remaining payload is a1e532000c40002a0e0803180c1134dc4e0a0306ff0b394c2a00c7902a009dec2a007
1aa2b00a29c2d00c7722f001d4930001aaf31006e083200ac583200c8a032006c092a00030000000037bd081f000000002000000000000023000000002400000000000027000000280000000000002
c00200301050c2f000000000000
2017-12-24_08:33:46 WMBUS_APT_00036030_3_3 Unsupported CI Field a0, remaining payload is a1e532000c40001f110803180c1134dc4e0a0306ff0b394c2a00c7902a009dec2a007
1aa2b00a29c2d00c7722f001d4930001aaf31006e083200ac583200c8a032006c092a00030000000037bd081f000000002000000000000023000000002400000000000027000000280000000000002
c00200301050c2f000000000000
2017-12-24_08:34:23 WMBUS_APT_00036030_3_3 Unsupported CI Field a0, remaining payload is a1e532000c400007120803180c1134dc4e0a0306ff0b394c2a00c7902a009dec2a007
1aa2b00a29c2d00c7722f001d4930001aaf31006e083200ac583200c8a032006c092a00030000000037bd081f000000002000000000000023000000002400000000000027000000280000000000002
c00200301050c2f000000000000

2017-12-24 (Sun) 08:29:10 Day number: 189 Day of week: 3 Day of month: 24 time: 08:12:55 Counter: 3335585
2017-12-24 (Sun) 08:30:58 Day number: 189 Day of week: 3 Day of month: 24 time: 08:14:42 Counter: 3335585
2017-12-24 (Sun) 08:33:46 Day number: 189 Day of week: 3 Day of month: 24 time: 08:17:31 Counter: 3335585
2017-12-24 (Sun) 08:34:23 Day number: 189 Day of week: 3 Day of month: 24 time: 08:18:07 Counter: 3335585

2018-01-04_16:03:38 WMBUS_APT_00036030_3_3 Unsupported CI Field a0, remaining payload is f60333000c4000122f0f0704011234dc4e0a0306ff0c394c2a00c7902a009dec2a0071aa2b00a29c2d00c7722f001d4930001aaf31006e083200ac583200c8a03200f8fa3200030000000037c8081f0000000020000000000023000000002400000000000027000000280000000000002c00200301050c2f000000000000
2018-01-04_16:06:23 WMBUS_APT_00036030_3_3 Unsupported CI Field a0, remaining payload is ff0333000c400002320f0704011234dc4e0a0306ff0c394c2a00c7902a009dec2a0071aa2b00a29c2d00c7722f001d4930001aaf31006e083200ac583200c8a03200f8fa3200030000000037c8081f000000002000000000000023000000002400000000000027000000280000000000002c00200301050c2f000000000000
2018-01-04_16:07:52 WMBUS_APT_00036030_3_3 Unsupported CI Field a0, remaining payload is ff0333000c40001f330f0704011234dc4e0a0306ff0c394c2a00c7902a009dec2a0071aa2b00a29c2d00c7722f001d4930001aaf31006e083200ac583200c8a03200f8fa3200030000000037c8081f000000002000000000000023000000002400000000000027000000280000000000002c00200301050c2f000000000000
2018-01-04_16:09:49 WMBUS_APT_00036030_3_3 Unsupported CI Field a0, remaining payload is 000433000c40001c350f0704011234dc4e0a0306ff0c394c2a00c7902a009dec2a0071aa2b00a29c2d00c7722f001d4930001aaf31006e083200ac583200c8a03200f8fa3200030000000037c8081f000000002000000000000023000000002400000000000027000000280000000000002c00200301050c2f000000000000
2018-01-04_16:10:25 WMBUS_APT_00036030_3_3 Unsupported CI Field a0, remaining payload is 000433000c400004360f0704011234dc4e0a0306ff0c394c2a00c7902a009dec2a0071aa2b00a29c2d00c7722f001d4930001aaf31006e083200ac583200c8a03200f8fa3200030000000037c8081f000000002000000000000023000000002400000000000027000000280000000000002c00200301050c2f000000000000
2018-01-04_16:13:19 WMBUS_APT_00036030_3_3 Unsupported CI Field a0, remaining payload is 000433000c40003a380f0704011234dc4e0a0306ff0c394c2a00c7902a009dec2a0071aa2b00a29c2d00c7722f001d4930001aaf31006e083200ac583200c8a03200f8fa3200030000000037c8081f000000002000000000000023000000002400000000000027000000280000000000002c00200301050c2f000000000000
2018-01-04_16:14:26 WMBUS_APT_00036030_3_3 Unsupported CI Field a0, remaining payload is 000433000c4000053a0f0704011234dc4e0a0306ff0c394c2a00c7902a009dec2a0071aa2b00a29c2d00c7722f001d4930001aaf31006e083200ac583200c8a03200f8fa3200030000000037c8081f000000002000000000000023000000002400000000000027000000280000000000002c00200301050c2f000000000000

2018-01-04 (Thu) 16:03:38 Day number: 200 Day of week: 7 Day of month: 04 time: 15:47:18 Counter: 3343350
2018-01-04 (Thu) 16:06:23 Day number: 200 Day of week: 7 Day of month: 04 time: 15:50:02 Counter: 3343359
2018-01-04 (Thu) 16:07:52 Day number: 200 Day of week: 7 Day of month: 04 time: 15:51:31 Counter: 3343359
2018-01-04 (Thu) 16:09:49 Day number: 200 Day of week: 7 Day of month: 04 time: 15:53:28 Counter: 3343360
2018-01-04 (Thu) 16:10:25 Day number: 200 Day of week: 7 Day of month: 04 time: 15:54:04 Counter: 3343360
2018-01-04 (Thu) 16:13:19 Day number: 200 Day of week: 7 Day of month: 04 time: 15:56:58 Counter: 3343360
2018-01-04 (Thu) 16:14:26 Day number: 200 Day of week: 7 Day of month: 04 time: 15:58:05 Counter: 3343360


kaihs

Zitat von: bilbolodz am 04 Januar 2018, 21:39:36
1) Is there any "standard" factor for converting "meter counter" into square meter?

I assume you mean cubic meter (m³) and not square meter (m²), right?

Well, WMBUS doesn't use an abstract "meter counter" but transmits the volume.
But there are different ways to encode it, e.g. as an 8 byte BCD value.

But as your meter doesn't use WMBUS standard appliation layer the values could be encoded in any possible way.

Zitat
2)  What is the  usually size of  meter counter number ("number of bytes")? There is a some "free space" between index 2 and 7 but I don't know which bytes belongs to volume counter but which could be "other data".

You might want to have a look at the WMBUS spec to could an idea what kind of different encodings are possible.
Unfortunately the specification isn't available for download anymore. I could send it to you by private mail if you like.

Zitat
3) How to patch (in elegant way) FHEM to support my meter?

Have a look at the TechemHKV module for a way to achieve this.
It basically intercepts the messages received from CUL before they are send to the WMBUS module.
If a message is identified to have a special format (Techem or APT in your case) it is processed. All other messages are left for further processing by the WMBUS module.

Good luck.
Banana Pi, Add-On Board mit 1.8" TFT LCD und IR-Sender, CULFW V1.61, div. Homematic Komponenten, Pollin Funksteckdosen, Selbstbau CUL433 MHz, Jeelink Clone, EC3000
Selbstbau CUL868MHz für Wireless M-Bus, SIGNALduino mit Logilink Temp.-sensoren und Auriol Wetterstation

bilbolodz

Zitat von: kaihs am 05 Januar 2018, 18:07:00
I assume you mean cubic meter (m³) and not square meter (m²), right?
Yes of course "cubic". My fault.

Zitat von: kaihs am 05 Januar 2018, 18:07:00
Well, WMBUS doesn't use an abstract "meter counter" but transmits the volume.
But there are different ways to encode it, e.g. as an 8 byte BCD value.
But as your meter doesn't use WMBUS standard appliation layer the values could be encoded in any possible way.
In my meter is rather transmuted as unsigned integer (or longer) but I've to check how long is volume variable and how to convert in (multiply factor) to cubic meters.

Zitat von: kaihs am 05 Januar 2018, 18:07:00
You might want to have a look at the WMBUS spec to could an idea what kind of different encodings are possible.
Unfortunately the specification isn't available for download anymore. I could send it to you by private mail if you like.
Done. Please check PM.

Zitat von: kaihs am 05 Januar 2018, 18:07:00
Have a look at the TechemHKV module for a way to achieve this.
It basically intercepts the messages received from CUL before they are send to the WMBUS module.
If a message is identified to have a special format (Techem or APT in your case) it is processed. All other messages are left for further processing by the WMBUS module.
I will try but I'm not a python programmer. I've have to learn (a little) another programing language.

bilbolodz

Zitat von: bilbolodz am 09 Januar 2018, 13:42:54
I will try but I'm not a python programmer. I've have to learn (a little) another programing language.
Could someone point me to some documentation about writing own modules to FHEM?

kaihs

Have a look at
https://wiki.fhem.de/wiki/DevelopmentModuleIntro

There is also a whole section of development information at the start page of the wiki.

Unfortunately for you it is in German.

As already pointed out, you can probably reuse a lot of the TechemHKV module.
If you have questions about it you could probably contact hermannj, the author.
Banana Pi, Add-On Board mit 1.8" TFT LCD und IR-Sender, CULFW V1.61, div. Homematic Komponenten, Pollin Funksteckdosen, Selbstbau CUL433 MHz, Jeelink Clone, EC3000
Selbstbau CUL868MHz für Wireless M-Bus, SIGNALduino mit Logilink Temp.-sensoren und Auriol Wetterstation

bilbolodz

I'm reading these page (not finished yet) but right now one question:

Is there a way to "simulate" receiving message from CUL (reply saved real signal during development). My meter sends telegrams between 8AM to 4 PM - it's very inconvenient time (for me) to play with module.....

bilbolodz

#524
Zitat von: kaihs am 28 Dezember 2017, 12:50:59
Is the S/N 00036030 printed somewhere on your meter?
I've catch raw message. Begging of message looks like:

b89441486D0F9020003034CCEA0

and after these string there is "Unsupported CI Field a0" rest of telegram.

Label (bar code) printed on my meter is: 195024 which in HEX is 0x2F9D0 so

(b)8944 1486 D0F90200 03 03 4CCE A0

lfield[0]: 0x89 - message lenght
cfield[1]: 0x44 ??
Manufacturer identuification[23]: 0x1486 = decoded ASCII  APT
Serial[4567]: 0xD0F90200 = 195024
Version[8]: 03
Type[9]: 03
CRC[1011] 0x4CCE
CI field 0xA0 ??