Hi,
I have the CM160_1301 electricity meter, which generated the following events.
2013-07-31_16:58:43 electricity energy_current: 482 W
2013-07-31_16:58:43 electricity energy_total: 962.666 kWh
2013-07-31_16:58:43 electricity battery: ok
2013-07-31_16:58:43 electricity ECUR: 482 ESUM: 962.666 BAT: ok
I can plot the current energy use quite nicely into a graph but what I'd really like is to plot daily, weekly and months usage in kWh as a histogram, and this I really have no idea how to do. I've tried several things but failed. so any help would be appreciated. My guess is that current energy total should be subtracted from the previous and then this amount added to variables for year,month,week and day.....
Thanks
Andrew
- you can zoom out from the default plot to see weekly/monthly/yearly usage.
- probably you'd like to have the daily/weekly/etc totals: you can put this into the title of the plot by setting the label attribute to "....$data{sumX}..." (or avgX/etc, see the label documentation) where X is the number of the plot-line holding the desired value. You can make calculations here, as the label is always evaluated as a perl expression, even without the usual {}
- as the above method is CPU intensive for large data sets, you may want to try the average module, and create your plots with the average data computed. It is important to set up a separate FileLog for average data only, to keep the data set small.
- another alternative is to use the Charting Frontend, which needs the data logged in a database.
Hi,
The problem is that the energy total event is energy total for the device over its lifetime so if I plot it its a straight line. It doesnt show daily, monthly totals. Somehow I have to subtract the previous reading to get the amount used in between readings then I could plot those or use the average function. However I've failed to get user readings to work at all for this. Power is not too much of a problem as I've upgraded from a rPi to a core i3 for the web interface.. I had DbLog working but it caused e rpi to crash a lot so I've turned it off... I may try again though.
I'm curious how you get the average values to go into a seperate log. My logs all have the regexp devicename so they log everything, including the average functions, I then have to try an exclude them from graphs. I've made a seperate post as I'm really struggling to get specific regexp to work when they are not very simple, could you maybe give a few examples for me.
Link (http://forum.fhem.de/index.php?topic=14097.0)
Thanks for the help, its impossible for me to search the forum effectively as its all in German so I'm sure your replies will help all the English folk!
Andrew
> the energy total event is energy total for the device over its lifetime
Did you tried the delta-h / delta-d functions? These can be entered in the FHEMWEB .gplot Editor function column (see below) or in the .gplot file directly.
> I'm curious how you get the average values to go into a seperate log.
My (current) recommended method is:
Define an eventTypes instance first (http://fhem.de/commandref.html#eventTypes (//fhem.de/commandref.html#eventTypes)), and wait a while for all types of events to occur. For events of the average module this may take a while due to the day/month change.
Create a FileLog, go into the Details page, and add the needed regexp parts (i.e. events) from the dropdown.
eventTypes is needed for this dropdown. After that delete (the initial) regexp parts of the FileLog not needed. Save it. You are ready with the FileLog part, the next step is to define the Plot.
Wait a while (again) until the FileLog is filled with data. This again is somehow unlucky with the day/month average events. Go to the FileLog detail page and "Create a new SVG plot". Here you can customize the plot. See also Link (http://forum.fhem.de/index.php?topic=11275.msg75594#msg75594)
> I'm really struggling to get specific regexp to work
The FileLog (and notify/watchdog/average/etc) module regexp checks first for the devicename only, then for the device:event combination, adding an implicit ^$ (== from-start-to-end) to your regexp. E.g to log the average for all devices to a FileLog you would define:
define avgLog FileLog log/average-%Y.log .*_avg_(day|month).*
Note: this regexp-form is not compatible with the FileLog "Regexp-Editor", there you have to add each device:event separately, which looks like this
define avgLog FileLog log/average-%Y.log dev1:temp_avg_day.*|dev1:temp_avg_month.*|dev2:temp_avg_day.*
There are a lot of internet sites explaining regexps, the FileLog "Regexp-Editor" is meant to be a help for beginners only, and cannot be used with all possible regexp-variants.
You can also watch events in the Event Monitor or in telnet (via inform timer).
> its impossible for me to search the forum effectively as its all in German
Until the english part of the forum fills up, you have to stick with the commandref.html, where I tried to document everything first. Some German fhem-users in turn have problem with this document, as they dont speak english well enough to understand it.
For creating plots there is also a fhemwiki entry in english, but it was written before the eventTypes/.gplot-Editor was created, so it should be much more easy to create/modify a plot now.
Hi,
Thanks again for the quick reply. So the latest version of FHEM with the built in gplot created has made things amazingly better. You were right the delta-h/d functions instantly show hourly usage in the graphs very quickly so that is fantastic.
I have also used the userReadings option to successfully create a new field where the difference is calculated from the total_energy field using
(1) energy_diff difference { ReadingsVal("electricity","energy_total",0); }
I then used the average function to create averages for these values, intending to get the daily cumulative and monthly cumulative which should give me the data that i want. However this is where things get a bit strange. the average function is generated using
(2) define energy.average average electricity:energy_diff.*
now the values generated from the code in (1) give very small numbers 0.00699999999994816 every 5 seconds or so and these are KwH if I add up a whole days worth i get 2-5KwH and this is using excel. However the cumulative feature of the average function generated numbers towards 2000 and on the day i started all this with an empty log the cum_month and cum_day were completely different. They also do not appear to generate events in the log file (only the _avg appear). The command_ref seems to also support this. Are the cumulative events not logged?
Thanks for the help with the regexp, I've got them to work fine with these types of events where the event follows the device name.
How would you write the regexp to generate a separate events for these two events
2013-08-04_10:16:52 boilerswitch valve 0
2013-08-04_10:16:52 boilerswitch valve 100
boilerswitch:valve.* (both alert)
boilerswitch:valve.*100.* (gives an event for 100)
boilerswitch:valve.*0.* (gives an event for both)
*******
This issue is solved in another post. Device:Event.value works!
*******
I agree until the forum fills up with some English the command ref will have to be used, or I have to learn German! I really do appreciate the help though!
Thanks again
Andrew
I've solved the problem in my own way and thought I would share the solution.
1) Easy
If all you require is the energy use per hour and per day in a graph then you can add the delta-h or delta-d options to the energy_total field in the plot editor. The result will be this in the plot file.
#FileLog 4:electricity.energy_total\x3a::delta-h
This is however not exactly what i want, for two reason. 1) I wanted to have actual numbers for my hour + day usage, 2) I wanted monthly totals as well....
2) Complicated way...
To solve this I created 3 dummy variables for each time point (cum_* will hold the cumulative number, raw_* will hold the value of energy_total that the sensor generates at the beginning of the time period, and total_* is written to once per period, and has associated log files that will hold the data for graphing. Once per time period the raw is updated with a new energy_total reading and the cum value will reset to 0, but before it is its value is set to the total_ variable..... This is done by extracting from the timestamp the hour, day, month and year and comparing it to now... if they are the same then a cumulative action is performed, if they are different.. ie its the next hour, day, month or year then the current cumulative value is written to the total value and generates an event, and the raw value is reset to the current energy total to reset the cum_ value...
define cum_hour dummy
define cum_day dummy
define cum_month dummy
define cum_year dummy
define raw_hour dummy
define raw_day dummy
define raw_month dummy
define raw_year dummy
define total_hour dummy
define total_day dummy
define total_month dummy
define total_year dummy
All of the automation is then handled by an at command that causes a trigger of the following.... so define the following notify....
define electricitycalc notify electricitycalc {
my $newcum = 0;
my $kwhnow = ReadingsVal("electricity","energy_total",0);
$kwhnow =~s/[^\d.]//g;
my $oldhour = substr(OldTimestamp("raw_hour"),11,2);
my $oldday = substr(OldTimestamp("raw_day"),8,2);
my $oldmonth = substr(OldTimestamp("raw_month"),5,2);
my $oldyear = substr(OldTimestamp("raw_year"),0,4);
if($oldhour == $hour) {
$newcum = $kwhnow - Value("raw_hour");
fhem("set cum_hour $newcum");
} else {
$newcum = Value("cum_hour");
fhem("set total_hour $newcum");
fhem("set raw_hour $kwhnow");
};
if($oldday == $mday) {
$newcum = $kwhnow - Value("raw_day");
fhem("set cum_day $newcum");
} else {
$newcum = Value("cum_day");
fhem("set total_day $newcum");
fhem("set raw_day $kwhnow");
};
if($oldmonth == $month) {
$newcum = $kwhnow - Value("raw_month");
fhem("set cum_month $newcum");
} else {
$newcum = Value("cum_month");
fhem("set total_month $newcum");
fhem("set raw_month $kwhnow");
};
if($oldyear == $year) {
$newcum = $kwhnow - Value("raw_year");
fhem("set cum_year $newcum");
} else {
$newcum = Value("cum_year");
fhem("set total_year $newcum");
fhem("set raw_year $kwhnow");
};
}
then issue this
define blah at XX:XX:XX {fhem("define electricityautocalc at +*00:00:10 trigger electricitycalc"); }
where XX:XX:XX is the next whole minute.. this ensures that the following at command starts at +00,10,20.... seconds and not 11 seconds etc...
hopefully with this code you then have every 10 seconds the cumulative values being updated..and every hour, day, month, year entries appear in the total_*.. so all that is left is to make filelogs for those and generate the graphs... This appears to work for me....
If anyone has any idea how to round to 4 decimal places it would be much appreciated... I have tried...
my $blah = sprintf("%.4d",$var)
This seems to work sometimes and then not others.. and give me a field with "%.4d" in it instead of the rounded number... however, initially it worked.... it seems to be erratic...
I am open to any ideas of how to make this code more efficient, or better... it is just how i solved a problem I had.. but i am not experienced in perl at all...
Thanks
Andrew
> If anyone has any idea how to round to 4 decimal places it would be much appreciated...
my $blah = sprintf("%.4f",$var);
f as float. d (as decimal) has no decimal places :)
Btw. some of your job might be done by the average FHEM Module.
Hi,
that is exactly what i've been trying and it seems not to work. What I have that currently works is this:
$newcum = sprintf '%%.4f', $newcum;
and in the StateFormat boxes this is what works
{ sprintf("%4d",ReadingsVal("cum_year","state",0))." kWh" }
I tried many times to get the average module to work, however, it never logged events under the cumulative numbers, and they were also wrong! I've attached a sample where I used a userReadings to generate the difference between the current and the last energy_total
energy_diff difference { ReadingsVal("electricity","energy_total",0); }
(siehe Anhang / see attachement)
This would generate small values 0.00123432 kWh the sum of which would be the total energy usage per day.... however as you can see the energy_diff_cum_day
reading give a number in the hundreds... if I cut and past all the _diff readings into excel and add them myself I get the correct value of 5-10 kWh, so I don't know what is going on there. So the average module was not suitable, as it doesn't generate per hour information, the cumulative values it does generate are wrong, and it doesn't seem to generate events/log them so it is impossible to use the information.
The method I've got generates the correct values that can be put into a log file and then plotted.... as shown in the second picture
(siehe Anhang / see attachement)