(Solved) weather error empty answer received

[HarryT]

Hi

I updated fhem today and installed the json files on my fritzbox 7490. Now I get the error:

2016.04.15 11:48:47 3: myWeather: https://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20weather.forecast%20where%20woeid=24549334%20and%20u=%27c%27&format=json&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys: empty answer received

every ten minutes. (I use the interval 600)

I searched the internet and the forum, but didn't find any clue why I get this.  Has anybody an idea what is wrong? (In german is fine too)

[jschmitt]

Hi Harry,

I had the same Problem. Look here:

http://mkleine.de/blog/2014/01/19/raspberry-pi-zugriff-auf-fhem-per-https-sichern/

After that it worked for me.

Viele Grüße,

Johannes

[Dr. Boris Neubert]

Dear Harry,

does the problem persist? This could be due to the Yahoo Weather API being quite fragile at the moment.

Do you get some json code back when you enter the URL in your browser's address line?

Kind regards,
Boris

[HarryT]

Dear Harry,

does the problem persist? This could be due to the Yahoo Weather API being quite fragile at the moment.

Yes.  Now I get the error every hour.

Do you get some json code back when you enter the URL in your browser's address line?

Yes, if I enter the code in my browser I get correct information back.  Also twilights seems to have the correct info and as far as I know it also uses Yahoo.

Kind regards,
Boris

I will have a look in the code, but if you have already some info which I could test or how to get more info from the module that could help.

BTW, maybe important, the state is reported as ¨Initialized".

{HT}

[Dr. Boris Neubert]

Just realized that my latest version did not make it to the repo yet. There is now a new version of 59_Weather.pm and YahooWeatherAPI.pm in the SVN repo. You either wait until tomorrow morning and make an update or just retrieve the aforementioned files from Sourceforge. In either case the most recent version should be our common base for debugging.

Boris

[HarryT]

Just realized that my latest version did not make it to the repo yet. There is now a new version of 59_Weather.pm and YahooWeatherAPI.pm in the SVN repo. You either wait until tomorrow morning and make an update or just retrieve the aforementioned files from Sourceforge. In either case the most recent version should be our common base for debugging.

Boris

Ok

As I go to the theator tonight I will wait till tomorrow.

My findings, the problem exists within httputil.pm

I see twilight calling httputils and get a correct answer and weather gets an empty answer.  I was tracing this code:

    $data = $hdr.(defined($data) ? $data:"");
    $hash->{directWriteFn} = sub($) { # Nonblocking write
      my $ret = syswrite $hash->{conn}, $data;
      if($ret <= 0) {
        my $err = $!;
        RemoveInternalTimer(\%timerHash);
        HttpUtils_Close($hash);
        Debug "httputil connect2 1b";
        return $hash->{callback}($hash, "write error: $err", undef)
      }
      $data = substr($data,$ret);
      if(length($data) == 0) {
        shutdown($hash->{conn}, 1) if($s);
        delete($hash->{directWriteFn});
        RemoveInternalTimer(\%timerHash);
        InternalTimer(gettimeofday()+$hash->{timeout},
                      "HttpUtils_ReadErr", \%timerHash, 0);
        Debug "httputil connect2 1c";
      }
    };
    $selectlist{$hash} = $hash;
    InternalTimer(gettimeofday()+$hash->{timeout},
                  "HttpUtils_WriteErr", \%timerHash, 0);
    Debug "httputil connect2 1e";
    return undef;

And see that  Debug "httputil connect2 1c"; is executed which seems not so good to me. 

BTW I set the timeout in yaohooweather_api.pm to 150. So that shouldn't be a problem.

Maybe this gives already a clue.

PHT}




.

[HarryT]

Hi

I just updated till the current version. The error stil exists,.

What can I do?

{HT}

[Dr. Boris Neubert]

I compared the mechanisms in Twilight and Weather and both modules do it the same. So no idea why Twilight should work and Weather not (apart from Twilight trying harder/more often to retrieve the data).

Do you never receive any data or only from time to time? Would you mind having a look at the pubDate... and validity readings?

Boris

[HarryT]

I compared the mechanisms in Twilight and Weather and both modules do it the same. So no idea why Twilight should work and Weather not (apart from Twilight trying harder/more often to retrieve the data).

Do you never receive any data or only from time to time?

No, at the moment it seems weather tries every minute. I inserted  a debug line in httputils

HttpUtils_NonblockingGet($)
{
  my ($hash) = @_;
  my ($url)= @_;
  $hash->{hu_blocking} = 0;
  my ($isFile, $fErr, $fContent) = HttpUtils_File($hash);
  return $hash->{callback}($hash, $fErr, $fContent) if($isFile);
  my $err = HttpUtils_Connect($hash);
  Debug "HttpUtils_NonblockingGet Err: $err  Url: $url $hash->{displayurl}";
  $hash->{callback}($hash, $err, "") if($err);
}

and this gives me a message every minute.

Would you mind having a look at the pubDate... and validity readings?

I am afraid you over estimate my knowledge. I can read a bit of perl but for sure not a guru. Please give me so directions.

Boris

[HarryT]

Hi

Weather_ReturnWithError there is a rearm every 60 seconds. Was this intentionally?
    my $next= 600; # $next= $hash->{INTERVAL};
or should the 60 seconds just be temporarily?

If 60 seconds is your intention I would suggest introducing a counter which steps this after 20 tries or so.

{HT}

[Dr. Boris Neubert]

Would you mind having a look at the pubDate... and validity readings?


I am afraid you over estimate my knowledge. I can read a bit of perl but for sure not a guru. Please give me so directions.

What I meant is: just have a look at the readings of your device myWeather, preferably through FHEMWEB or through list myWeather in the FHEM console.

The update every minute is intended. It does so as long as no valid answer is received. I consider your suggestion to increment the retrial period after several unsuccessful tries.

Kind regards
Boris

[HarryT]

Ok

It seems I can't attach a screendump here. So I send it in private mail. I don't see anything strange in the readingsl. And the url which is reported back in my logging works fine if I use it from my browser.

At the moment I am trying to log the url's in httputils originated from weather and twilight. Maybe that gives a clue.

During the night several calls from twilight where succesfull and as far as I can see all calls from weather failed.   (I use different location codes to see the difference)

{HT}

[Dr. Boris Neubert]

Dear Harry,

thanks for the image. This is unambiguous: you never get a reply other than the empty answer.

Please let it run for some more time, anyway. The logs will be useful as well.

Kind regards
Boris

[HarryT]

I now have this in my logging:

called by twilight:
http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20weather.forecast%20where%20woeid=24549334
%20and%20u=%27c%27&format=json&env=store%3A%2%2Fdatatables.org%2Falltableswithkeys

called by weather:
https://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20weather.forecast%20where%20woeid=12596696
%20and%20u=%27c%27&format=json&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys

The first works, the second not. The big difference is the https call instead of http I guess.

Is https necessary?

And has anybody described how to get the https calls working from a fritzbox?

{HT}

[HarryT]

Changing https in http in YahooWeather.pm indeed is a workaround.

Is it possible to let HttpUtils give an error when this is a problem? I guess I won't be the only one who gets this problem?

{HT}