Originally posted by: <email address deleted>
Hi,
a quick question: I started interfacing my WS3600 weather station to FHEM, but
instead of hacking the Perl scripts which deal with it's little sister WS2300,
I want to rely on the binaries from open2300/open3600 projects (available for
Linux and Windows). Thus, basically I will call
open($FH, "/path/to/fetch3600-binary |"); while(<$FH>) {}; close ($FH);
about every minute; on my current system, "fetch3600" takes ~6 seconds of real
time to complete (reading all 87 data points the WS3600 has to offer). (As for
"every minute": sensor data is updated every 128 seconds normally, except for
wind speed and direction about 10 km/h, interval then is 32 seconds -- refer
to http://www.wetterstationsforum.de/ws3600_master-touch.php for details. It
might be worth to make the interval 64 instead of 60 seconds though.)
I modelled this around 70_SCIVT.pm which shedules it's internal call-back via
InternalTimer(); commandref states: "Defining an SCIVT device will schedule an
internal task". Question is: will doing it this way hold processing of other
events for the duration of reading the data from WS3600 (or, more precisely,
from the pipe to /path/to/fetch3600-binary)? If so, what's the preferred way
of dealing with "slow to read" sensors/devices?
Regards,
kai
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "FHEM users" group.
To post to this group, send email to fhem-users@googlegroups.com
To unsubscribe from this group, send email to fhem-users+unsubscribe@googlegroups.com
For more options, visit this group at http://groups.google.com/group/fhem-users?hl=en
-~----------~----~----~----~------~----~------~--~-
> Question is: will doing it this way hold processing of other events for the
> duration of reading the data from WS3600 (or, more precisely, from the pipe
> to /path/to/fetch3600-binary)?
Yes.
> If so, what's the preferred way of dealing with "slow to read"
> sensors/devices?
Setting $hash->{FD} and inserting $hash in $selectlist. See 00_CUL.pm,
00_FHZ.pm, 00_CM11.pm, 00_LIRC.pm and 87_WS2000.pm
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "FHEM users" group.
To post to this group, send email to fhem-users@googlegroups.com
To unsubscribe from this group, send email to fhem-users+unsubscribe@googlegroups.com
For more options, visit this group at http://groups.google.com/group/fhem-users?hl=en
-~----------~----~----~----~------~----~------~--~-
Originally posted by: <email address deleted>
Rudolf Koenig wrote:
>> If so, what's the preferred way of dealing with "slow to read"
>> sensors/devices?
>
> Setting $hash->{FD} and inserting $hash in $selectlist. See 00_CUL.pm,
> 00_FHZ.pm, 00_CM11.pm, 00_LIRC.pm and 87_WS2000.pm
Err, all those are doing more or less the following, if I'm not
overlooking something:
_Define() {
$whatever=OpenSerialOrNetwork();
...
$hash->{FD} = $whatever->FILENO;
...
}
_Undef() {
...
close_what_was_opened($hash->{FD});
...
}
SOMEWHERE() {
...
read_from($hash->{FD});
...
}
I don't have long standing open file descriptors. I start at a pre-
defined interval a read on an Perl file handle from open($FH, "cmd |").
That is Perl starts a command in the background and I read that cmds
output from within the FHEM process. On EOF, file handle is close()d
and gone for good.
I don't interface on serial level and there's no daemon to provide
the current data on time to a caller. I interface on stdin, running
a commandline query tool any time I want fresh data. I don't see yet
how to adapt this to the mentioned method?
Thanks,
kai
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "FHEM users" group.
To post to this group, send email to fhem-users@googlegroups.com
To unsubscribe from this group, send email to fhem-users+unsubscribe@googlegroups.com
For more options, visit this group at http://groups.google.com/group/fhem-users?hl=en
-~----------~----~----~----~------~----~------~--~-
Originally posted by: <email address deleted>
> I don't interface on serial level and there's no daemon to provide
> the current data on time to a caller. I interface on stdin, running
> a commandline query tool any time I want fresh data. I don't see yet
> how to adapt this to the mentioned method?
To make myself clear: Basically, I do the same as 88_IPWE.pm, just
without network overhead.
kai
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "FHEM users" group.
To post to this group, send email to fhem-users@googlegroups.com
To unsubscribe from this group, send email to fhem-users+unsubscribe@googlegroups.com
For more options, visit this group at http://groups.google.com/group/fhem-users?hl=en
-~----------~----~----~----~------~----~------~--~-
> I don't have long standing open file descriptors. I start at a pre-
> defined interval a read on an Perl file handle from open($FH, "cmd |").
I am fully aware of your situation.
At the beginning of the operation you define $hash->{FD} and insert $hash into
the %selectlist. Then your ReadFn function is called whenever there is data,
and you are only allowed a single read to avoid blocking, so you have to
remember your context. If you're done you remove your $hash from the
%selectlist.
As an alternative you fork a separate process, read the data in a loop, write
it to a temporary file, and then notify the parent e.g. by sending him a "get
data ". Or something similar.
None of the alternatives is nice, but this is the downside of having a
single-threaded model. On the other side there are a lot of advantages which
are more important in my opinion.
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "FHEM users" group.
To post to this group, send email to fhem-users@googlegroups.com
To unsubscribe from this group, send email to fhem-users+unsubscribe@googlegroups.com
For more options, visit this group at http://groups.google.com/group/fhem-users?hl=en
-~----------~----~----~----~------~----~------~--~-
Originally posted by: <email address deleted>
Rudolf Koenig wrote:
>> I don't have long standing open file descriptors. I start at a pre-
>> defined interval a read on an Perl file handle from open($FH, "cmd |").
>
> I am fully aware of your situation.
;)
> At the beginning of the operation you define $hash->{FD} and insert $hash into
> the %selectlist. Then your ReadFn function is called whenever there is data,
> and you are only allowed a single read to avoid blocking, so you have to
> remember your context. If you're done you remove your $hash from the
> %selectlist.
Ah. Now I understand; it's a shame no module implemented this before so
I could just reuse that code ;) While that would be a more elegant (and
more portable) solution, I most likely will go the other way:
> As an alternative you fork a separate process, read the data in a loop, write
> it to a temporary file, and then notify the parent e.g. by sending him a "get
> data ". Or something similar.
This basically requires a cron job outside of FHEM and the move of the
read-loop to my _Get() subroutine. Is reading 80+ lines from a file in
one go ok regarding the overall timings within FHEM? (Sure, doing this
with %selectlist would be nicer ;), but is that *neccessary* to prevent
data loss from other sources?)
> None of the alternatives is nice, but this is the downside of having a
> single-threaded model. On the other side there are a lot of advantages which
> are more important in my opinion.
I agree.
kai
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "FHEM users" group.
To post to this group, send email to fhem-users@googlegroups.com
To unsubscribe from this group, send email to fhem-users+unsubscribe@googlegroups.com
For more options, visit this group at http://groups.google.com/group/fhem-users?hl=en
-~----------~----~----~----~------~----~------~--~-
> This basically requires a cron job outside of FHEM and the move of the
> read-loop to my _Get() subroutine.
You should use the InternalTimer instead of a cron job to make the setup easier
for the user.
> Is reading 80+ lines from a file in one go ok regarding the overall timings
> within FHEM?
As the other process just finished writing and the data is small, it is
probably completely in the OS cache, so reading it should be <1ms.
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "FHEM users" group.
To post to this group, send email to fhem-users@googlegroups.com
To unsubscribe from this group, send email to fhem-users+unsubscribe@googlegroups.com
For more options, visit this group at http://groups.google.com/group/fhem-users?hl=en
-~----------~----~----~----~------~----~------~--~-
Originally posted by: <email address deleted>
Rewrote the code, hit a problem ;)
Rudolf Koenig wrote:
> At the beginning of the operation you define $hash->{FD} and insert $hash into
> the %selectlist. Then your ReadFn function is called whenever there is data,
> and you are only allowed a single read to avoid blocking, so you have to
> remember your context. If you're done you remove your $hash from the
> %selectlist.
sub WS3600_Initialize($) {
...
$hash->{DefFn} = "WS3600_Define";
$hash->{ReadFn} = "WS3600_Read";
$hash->{UndefFn} = "WS3600_Undef";
...
}
sub WS3600_Define($$) {
...
InternalTimer(gettimeofday()+ $hash->{Timer}, "WS3600_GetStatus", $hash, 1);
$hash->{STATE} = "initialized";
return undef;
}
sub WS3600_GetStatus($) {
...
Log 3, "WS3600 contacting station";
open($FH, $dev);
if(!$FH) {
return "WS3600 Can't start $dev: $!";
}
$hash->{FD}=$FH;
$selectlist{"$name.pipe"} = $hash;
Log 3, "WS3600 pipe opened";
$hash->{STATE} = "running";
return $hash->{STATE};
}
sub WS3600_Read($) {
...
if(!defined($hash->{FD})) {
return undef;
}
$FH = $hash->{FD};
...
if( $inputline = <$FH>) {
$inputline =~ s/\s+$//;
my ($reading, $val)=split(/ /, $inputline);
if(defined($TranslatedCodes{$reading})) {
Log 3, "WS3600 read $reading:$val";
...
$hash->{STATE} = "reading";
} else {
close($FH);
Log 3, "WS3600 done reading pipe";
DoTrigger($name, undef);
delete $hash->{FD};
undef($selectlist{"$name.pipe"});
$hash->{STATE} = "read";
InternalTimer(gettimeofday()+ $hash->{Timer}, "WS3600_GetStatus", $hash, 1);
}
return $hash->{STATE};
}
Well, it works, but far beyond expectation:
2009.11.13 00:08:26 3: WS3600 contacting station
2009.11.13 00:08:26 3: WS3600 pipe opened
2009.11.13 00:08:30 3: WS3600 read Date:13-Nov-2009
2009.11.13 00:08:33 3: WS3600 read Time:00:08:30
2009.11.13 00:08:34 3: WS3600 read Ti:22.2
2009.11.13 00:08:36 3: WS3600 read Timin:20.8
2009.11.13 00:08:55 3: WS3600 read Timax:27.9
2009.11.13 00:08:55 3: WS3600 read TTimin:10:27
2009.11.13 00:08:59 3: WS3600 read DTimin:15-10-2009
2009.11.13 00:09:00 3: WS3600 read TTimax:23:31
2009.11.13 00:09:12 3: WS3600 read DTimax:20-08-2009
2009.11.13 00:09:13 3: WS3600 read To:7.2
2009.11.13 00:09:14 3: WS3600 read Tomin:-0.4
2009.11.13 00:09:19 3: WS3600 read Tomax:35.6
2009.11.13 00:09:19 3: WS3600 read TTomin:07:03
2009.11.13 00:09:20 3: WS3600 read DTomin:15-10-2009
2009.11.13 00:09:31 3: WS3600 read TTomax:16:52
2009.11.13 00:09:31 3: WS3600 read DTomax:20-08-2009
2009.11.13 00:09:32 3: WS3600 read DP:5.8
[...]
2009.11.13 00:16:27 3: WS3600 read DRPmax:11-09-2009
2009.11.13 00:16:34 3: WS3600 read Tendency:Rising
2009.11.13 00:16:38 3: WS3600 read Forecast:Sunny
2009.11.13 00:16:44 3: WS3600 done reading pipe
In other words: it takes AGES for my ReadFn to get called again. In
between invocation, I could have read in anything :( It's totally use-
less this way; what am i doing wrong?
And, as it rains ...
2009.11.13 00:17:49 0: Strange call for nonexistent : ReadFn
Use of uninitialized value in vec at /usr/local/bin/fhem.pl line 296.
Use of uninitialized value in vec at /usr/local/bin/fhem.pl line 321.
Use of uninitialized value $d in hash element at /usr/local/bin/fhem.pl line 1907.
Use of uninitialized value $d in concatenation (.) or string at /usr/local/bin/fhem.pl line 1908.
after "done reading pipe", I think my version of "remove your
$hash from the %selectlist", i. e. undef($selectlist{"$name.pipe"}), does
not work as expected ;)
Help, anyone?
kai
--
You received this message because you are subscribed to the Google Groups "FHEM users" group.
To post to this group, send email to fhem-users@googlegroups.com.
To unsubscribe from this group, send email to fhem-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/fhem-users?hl=.
> if( $inputline = <$FH>) {
This may block longer than expected as it will read (blocking) until a \n is
encountered. The function to use is sysread. But I don't think this is the
cause of the described problem, it "only" makes the use of select superfluous.
> In other words: it takes AGES for my ReadFn to get called again. In
> between invocation, I could have read in anything :( It's totally use-
> less this way; what am i doing wrong?
I suspect that one of the other ReadFn/ReadyFn functions is blocking. Put some
debug lines in fhem.pl around the ReadFn and ReadyFn calls.
> 2009.11.13 00:17:49 0: Strange call for nonexistent : ReadFn
I think only more debugging would help here, as I do not see the problem yet.
--
You received this message because you are subscribed to the Google Groups "FHEM users" group.
To post to this group, send email to fhem-users@googlegroups.com.
To unsubscribe from this group, send email to fhem-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/fhem-users?hl=.
Originally posted by: <email address deleted>
Rudolf Koenig wrote:
>> if( $inputline = <$FH>) {
>
> This may block longer than expected as it will read (blocking) until a \n is
> encountered. The function to use is sysread. But I don't think this is the
> cause of the described problem, it "only" makes the use of select superfluous.
No, not the cause. I rewrote the stuff to open the pipe, shedule my ReadFn-
function to be called in 6 seconds, reading pipe's whole stdout in one go,
close pipe and shedule opening function to be called in 60 seconds ...
Works so far:
2009.11.13 14:37:24 3: WS3600 contacting station
2009.11.13 14:37:24 3: WS3600 pipe opened
2009.11.13 14:37:30 3: WS3600 Read entered
2009.11.13 14:37:30 3: WS3600 start reading
2009.11.13 14:37:30 3: WS3600 done reading pipe
2009.11.13 14:37:30 5: Triggering WS3600 (4 changes)
But it still could block, unfortunately :(
>> In other words: it takes AGES for my ReadFn to get called again. In
>> between invocation, I could have read in anything :( It's totally use-
>> less this way; what am i doing wrong?
>
> I suspect that one of the other ReadFn/ReadyFn functions is blocking. Put some
> debug lines in fhem.pl around the ReadFn and ReadyFn calls.
Yepp, I will do that after the next update I install from the CVS, i. e. one of
these days.
>> 2009.11.13 00:17:49 0: Strange call for nonexistent : ReadFn
>
> I think only more debugging would help here, as I do not see the problem yet.
That was simply caused by my Perl ignorance, I suspect. My implementation
of "remove your handle from %selectlist" was obviously gargabe, with
»delete $selectlist{"$name.pipe"};« I haven't seen these messages yet.
kai
--
You received this message because you are subscribed to the Google Groups "FHEM users" group.
To post to this group, send email to fhem-users@googlegroups.com.
To unsubscribe from this group, send email to fhem-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/fhem-users?hl=.
Originally posted by: <email address deleted>
Salve,
quoting myself:
> Rudolf Koenig wrote:
>>> if( $inputline = <$FH>) {
>> This may block longer than expected as it will read (blocking) until a \n is
>> encountered. The function to use is sysread. But I don't think this is the
>> cause of the described problem, it "only" makes the use of select superfluous.
>
> No, not the cause. I rewrote the stuff to open the pipe, shedule my ReadFn-
> function to be called in 6 seconds, reading pipe's whole stdout in one go,
> close pipe and shedule opening function to be called in 60 seconds ...
>
> Works so far:
>
> 2009.11.13 14:37:24 3: WS3600 contacting station
> 2009.11.13 14:37:24 3: WS3600 pipe opened
> 2009.11.13 14:37:30 3: WS3600 Read entered
> 2009.11.13 14:37:30 3: WS3600 start reading
> 2009.11.13 14:37:30 3: WS3600 done reading pipe
> 2009.11.13 14:37:30 5: Triggering WS3600 (4 changes)
>
> But it still could block, unfortunately :(
>
>>> In other words: it takes AGES for my ReadFn to get called again. In
>>> between invocation, I could have read in anything :( It's totally use-
>>> less this way; what am i doing wrong?
>> I suspect that one of the other ReadFn/ReadyFn functions is blocking. Put some
>> debug lines in fhem.pl around the ReadFn and ReadyFn calls.
>
> Yepp, I will do that after the next update I install from the CVS, i. e. one of
> these days.
I reworked the code again, now with %selectlist again but a blocking-
aware ReadFn. The timing issues I noticed earlier seem to be caused by
a problem with Perl/FHEM noticing that my pipe is ready to be read from
(or fhem.pl spending time elsewhere; I put Log() around the calls to
ReadFn & ReadyFn in fhem.pl, the delays therefore seem to be coming from
outside of these functions).
Example (ommisions only where [...] is):
2009.11.14 11:18:31 3: WS3600 contacting station
2009.11.14 11:18:31 3: WS3600 pipe opened
2009.11.14 11:18:56 5: Calling ReadFn of WS3600
2009.11.14 11:18:56 3: WS3600 Read entered
2009.11.14 11:18:56 3: WS3600 start reading
2009.11.14 11:18:56 3: WS3600 ended reading (eof=0)
2009.11.14 11:18:56 3: WS3600 read Date 14-Nov-2009:Date:14-Nov-2009
2009.11.14 11:18:56 3: WS3600 read Time 11:18:34:Time:11:18:34
[...]
2009.11.14 11:18:56 3: WS3600 read Forecast Cloudy:Forecast:Cloudy
2009.11.14 11:18:56 3: WS3600 reading would block
2009.11.14 11:18:56 5: ReadFn returned
2009.11.14 11:18:57 5: exec at command at_get_CUN_version
2009.11.14 11:18:57 5: Cmd: >{ \
fhem("get CUL2 version"); \
}<
2009.11.14 11:18:57 5: Cmd: >get CUL2 version<
2009.11.14 11:18:57 5: CUL/RAW: V 1.31.4 CUN
2009.11.14 11:18:57 3: CUL2 version => V 1.31.4 CUN
2009.11.14 11:18:57 5: redefine at command at_get_CUN_version as +*00:01 { \
fhem("get CUL2 version"); \
}
2009.11.14 11:18:57 5: Calling ReadFn of CUL2
2009.11.14 11:18:57 5: CUL/RAW: /T320242690D32
2009.11.14 11:18:57 3: CUL2: T320242690D -49
2009.11.14 11:18:57 5: CUL2 dispatch 810c04xx0909a00132024200690d
2009.11.14 11:18:57 5: ReadFn returned
2009.11.14 11:18:57 5: Calling ReadFn of WS3600
2009.11.14 11:18:57 3: WS3600 Read entered
2009.11.14 11:18:57 3: WS3600 start reading
2009.11.14 11:18:57 3: WS3600 ended reading (eof=1)
Use of uninitialized value $rawreading in concatenation (.) or string at /usr/local/lib/FHEM/70_WS3600.pm line 263.
Use of uninitialized value $val in concatenation (.) or string at /usr/local/lib/FHEM/70_WS3600.pm line 263.
2009.11.14 11:18:57 3: WS3600 read ::
Use of uninitialized value $rawreading in hash element at /usr/local/lib/FHEM/70_WS3600.pm line 264.
2009.11.14 11:18:57 3: WS3600 done reading pipe
2009.11.14 11:18:57 5: Triggering WS3600 (1 changes)
2009.11.14 11:18:57 5: WS3600 trigger: Checking FMS1Log for notify
2009.11.14 11:18:57 5: WS3600 trigger: Checking HGS1Log for notify
2009.11.14 11:18:57 5: WS3600 trigger: Checking Logfile for notify
2009.11.14 11:18:57 5: WS3600 trigger: Checking RRDNotify for notify
2009.11.14 11:18:57 5: Cmd: >"/usr/local/bin/fhem2rrd.pl "WS3600" "WS3600" "Status: updated""<
2009.11.14 11:18:57 5: WS3600 trigger: Checking TestNotify for notify
2009.11.14 11:18:57 5: Cmd: >"/usr/bin/logger "WS3600 - Status: updated (WS3600)""<
2009.11.14 11:18:57 5: WS3600 trigger: Checking at_FMS1_Off for notify
2009.11.14 11:18:57 5: WS3600 trigger: Checking at_FMS1_On for notify
2009.11.14 11:18:57 5: ReadFn returned
I doubt that it took 25 seconds for the command to become ready to
be read from, the (DCF77-synchronized) reading "Time 11:18:34" fur-
ther suggests that the pipe opened at 11:18:31 was ready at 11:18:34.
(The "uninitialized value" errors are from crude debugging code; as
the pipe seems to block just before delivering EOF, the loop gets
called with empty $line ("containing" only EOF), which then gets
split() on " " ... Sue me ;))
Any ideas on where to look further?
kai
--
You received this message because you are subscribed to the Google Groups "FHEM users" group.
To post to this group, send email to fhem-users@googlegroups.com.
To unsubscribe from this group, send email to fhem-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/fhem-users?hl=.