[gelöst] [HTTPMOD] Login mit ASP.NET und zusätzlichen login parametern?

roman1528 · 17 Juli 2020, 10:46:10

Ergebnis: https://forum.fhem.de/index.php/topic,112942.msg1074500.html#msg1074500

Moin.
Um eine XML abzurufen muss ich mich einloggen. Bin den diversen Beispielen hier im Forum gefolgt. Die Login-Seite wird geholt, Cookies werden gelesen... ABER

es hat eine weile gedauert bis ich dahinter gekommen bin:

Im POST-Request eines Browsers (normaler Login) stehen zusätzliche versteckte Daten, die in den Request müssen (hidden input).

Wie kann ich diese auslesen und dem "sid02Data" voranstellen?

__VIEWSTATE
__VIEWSTATEGENERATOR
__EVENTVALIDATION

Login-Seite:

Code Auswählen


<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head><title>
	smart-me  - Smart-Energy Lösungen
</title><meta name="description" content="smart-me ist eine einfache Cloud-basierte Energiemonitoring Lösung." /><meta charset="utf-8" /><meta http-equiv="X-UA-Compatible" content="IE=edge" /><meta name="viewport" content="width=device-width, initial-scale=1" />

<link rel="stylesheet" href="Content/css/bootstrap/bootstrap.min.css" type="text/css" />
<link rel="stylesheet" href="Content/css/bootstrap/font-awesome.min.css" type="text/css" />
<link rel="stylesheet" href="Content/css/bootstrap/animate.min.css" type="text/css" />
<link rel="stylesheet" href="Content/css/bootstrap/prettyPhoto.css" type="text/css" />
<link rel="stylesheet" href="Content/css/bootstrap/main3.css" type="text/css" />
<link rel="stylesheet" href="Content/css/bootstrap/responsive.css" type="text/css" />
<!--[if lt IE 9]>
    <script src="Scripts/bootstrap/html5shiv.js" type="text/javascript"></script>
    <script src="Scripts/bootstrap/respond.min.js" type="text/javascript"></script>
    <![endif]-->
<link rel="shortcut icon" href="favicon.ico" />
<link rel="apple-touch-icon-precomposed" sizes="144x144" href="Images/icons/apple-touch-icon-144-precomposed.png" />
<link rel="apple-touch-icon-precomposed" sizes="114x114" href="Images/icons/apple-touch-icon-114-precomposed.png" />
<link rel="apple-touch-icon-precomposed" sizes="72x72" href="Images/icons/apple-touch-icon-72-precomposed.png" />
<link rel="apple-touch-icon-precomposed" href="Images/icons/apple-touch-icon-57-precomposed.png" />
<script src="Scripts/jquery-1.7.1.min.js" type="text/javascript"></script>
<script src="Scripts/bootstrap/bootstrap.min.js" type="text/javascript"></script>
<script src="Scripts/bootstrap/jquery.prettyPhoto.js" type="text/javascript"></script>
<script src="Scripts/bootstrap/jquery.isotope.min.js" type="text/javascript"></script>
<script src="Scripts/bootstrap/main.js" type="text/javascript"></script>
<script src="Scripts/bootstrap/wow.min.js" type="text/javascript"></script>
<script src="Content/Scripts/jquery.cookiefy.min.js" type="text/javascript"></script>
<link href="/Content/css/bootstrap/SignIn.css" type="text/css" rel="stylesheet" />
</head>
<body class="homepage">
<form method="post" action="./SignIn.aspx?ReturnUrl=%2fDetails%2fGetDetailsNew.ashx%3fid%3d<<ZÄHLER-ID>>&amp;id=<<ZÄHLER-ID>>" id="form1">
<div class="aspNetHidden">
<input type="hidden" name="__VIEWSTATE" id="__VIEWSTATE" value="Y8MGO3Kwlq2AM3PiUyWqM7BqoccESSf9cx+ePc1cRLAYQ0mXsLFzpYsd9R3QDNzEbqauznt5qY8p+XIUC1JUJiABohY=" />
</div>
<div class="aspNetHidden">
<input type="hidden" name="__VIEWSTATEGENERATOR" id="__VIEWSTATEGENERATOR" value="ECDA716A" />
<input type="hidden" name="__EVENTVALIDATION" id="__EVENTVALIDATION" value="706pTDWfdZ83uEXxAjlDhn7/ivkrcB+IU4WG99MMkKCMkuSbT+kY0Tpp19OvWSpXtDGuz2CkcTcgMfs4KW30yrYCCWNTkRLuX0UjK4bndR47kZniUezq2Rwv+GoWnfysDsBUjFcnONP3ja/FaIn1euvTtUk=" />
</div>

<header id="header">
<nav class="navbar navbar-inverse" role="banner">
<div class="container">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-collapse">
<span class="sr-only">Toggle navigation</span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="default.aspx">
<img src="Images/bootstrap/logo.png" alt="logo" style="width: 140px;" /></a>
</div>
<div class="collapse navbar-collapse navbar-right">
<ul class="nav navbar-nav">
<li><a href="Details/V2/All.aspx">Login</a></li>
<li><a href="Description/HowItWorks.aspx">Wie es funktioniert</a></li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Anwendungen <i class="fa fa-angle-down"></i></a>
<ul class="dropdown-menu">
<li><a href="Description/Solutions.aspx?slide=0">Liegenschaften</a></li>
<li><a href="Description/Solutions.aspx?slide=1">Industrie</a></li>
<li><a href="Description/Solutions.aspx?slide=2">E-Mobilität</a></li>
</ul>
</li>
<li><a href="Description/Products.aspx">Produkte</a></li>
<li><a href="Description/Pricing.aspx">Preise</a></li>
<li><a href="buy/">Kaufen</a></li>
<li><a href="Description/Demo/">Live demo</a></li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Support <i class="fa fa-angle-down"></i></a>
<ul class="dropdown-menu">
<li><a href="installation/">Installation</a></li>
<li><a href="Description/Tutorials/">Tutorials</a></li>
<li><a href="Description/Tutorials/OnlineTraining.aspx">Online Schulungen</a></li>
<li><a href="http://wiki.smart-me.com" target="_blank">Wiki</a></li>
<li><a href="http://wiki.smart-me.com/index.php/Downloads" target="_blank">Downloads</a></li>
</ul>
</li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Mehr <i class="fa fa-angle-down"></i></a>
<ul class="dropdown-menu">
<li><a href="Description/App.aspx">Apps</a></li>
<li><a href="Description/billing/">smart-me Billing</a></li>
<li><a href="Developer/">Entwickler</a></li>
<li><a href="About/AboutUs.aspx">Über uns</a></li>
<li><a href="About/ContactUs.aspx">Kontakt</a></li>
</ul>
</li>
</ul>
</div>
</div>

</nav>

</header>

<div class="container">
<div class="row">
<div class="col-sm-6 col-md-4 col-md-offset-4">
<div class="account-wall">
<div id="my-tab-content" class="tab-content">
<div class="tab-pane active" id="login">
<img class="profile-img" src="Images/logoBlueSmall.png" alt="" style="width:140px" />
<table id="ContentPlaceHolder1_Login1" cellspacing="0" cellpadding="0" style="border-collapse:collapse;">
<tr>
<td>
<p class="validation-summary-errors">
</p>
<input name="ctl00$ContentPlaceHolder1$Login1$UserName" type="text" id="ContentPlaceHolder1_Login1_UserName" class="form-control" autofocus="" placeholder="Benutzername" />
<input name="ctl00$ContentPlaceHolder1$Login1$Password" type="password" id="ContentPlaceHolder1_Login1_Password" class="form-control" placeholder="Passwort" />
<input type="submit" name="ctl00$ContentPlaceHolder1$Login1$ctl01" value="Login" class="btn btn-lg btn-default btn-block" />
</td>
</tr>
</table>
<br />
<div id="tabs" data-tabs="tabs">
<p class="text-center"><a href="Users/SignUp.aspx">Als neuer Benutzer registieren</a></p>
<p class="text-center"><a href="Users/RequestPasswordReset.aspx">Passwort vergessen?</a></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br />

<footer id="footer" class="midnight-blue">
<div class="container">
<div class="row">
<div class="col-sm-6">
&copy; 2018 <a target="_blank" href="https://smart-me.com" title="smart-me smart-energy">smart-me</a>
</div>
<div class="col-sm-6">
<ul class="pull-right">
<li><a href="About/Privacy.aspx">Datenschutzrichtlinie</a></li>
<li><a href="About/TermsOfService.aspx">Nutzungsbedingungen</a></li>
<li><a href="About/ContactUs.aspx">Kontakt</a></li>
<li><a href="About/LegalNotice.aspx">Impressum</a></li>
</ul>
</div>
</div>
</div>
</footer>

</form>
<script type="text/javascript">

        $('body').cookiefy({
            displayedHtml: 'Um unsere Webseite für Sie optimal zu gestalten und fortlaufend verbessern zu können, verwenden wir Cookies. Durch die weitere Nutzung der Webseite stimmen Sie der Verwendung von Cookies zu. Weitere Informationen zu Cookies erhalten Sie in unserer Datenschutzrichtline'
        });

        var _gaq = _gaq || [];
        _gaq.push(['_setAccount', 'UA-39019091-1']);
        _gaq.push(['_trackPageview']);

        (function () {
            var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
            ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
            var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
        })();

    </script>
</body>
</html>

Hier noch ein list vom Device:

Code Auswählen

Internals:
   BUSY       0
   CFGFN      
   DEF        https://www.smart-me.com/Details/GetDetailsNew.ashx?id=<<ZÄHLER-ID>> 0
   FUUID      5f10990f-f33f-8842-5e72-8a8146beea1a6cc2
   Interval   0
   LASTSEND   1594974367.67264
   LastAuthTry 2020-07-17 10:26:04
   MainURL    https://www.smart-me.com/Details/GetDetailsNew.ashx?id=<<ZÄHLER-ID>>
   ModuleVersion 3.5.22 - 7.2.2020
   NAME       solar_xml_test
   NOTIFYDEV  global
   NR         2166
   NTFY_ORDER 50-solar_xml_test
   STATE      ???
   TRIGGERTIME 0
   TRIGGERTIME_FMT 
   TYPE       HTTPMOD
   addr       https://www.smart-me.com:443
   auth       0
   buf        
   code       200
   compress   1
   conn       
   data       
   displayurl https://www.smart-me.com:443/SignIn.aspx?ReturnUrl=%2fDetails%2fGetDetailsNew.ashx%3fid%3d<<ZÄHLER-ID>>&id=<<ZÄHLER-ID>>
   header     Cookie: $sid
Cookie: ASP.NET_SessionId=255kfaefkck2kidiii4isqwu; __cfduid=de727087768a4b83ca49b7bbb4b307b351594974025
   host       www.smart-me.com
   httpbody   
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head><title>
	smart-me  - Smart energy solutions
</title><meta name="description" content="smart-me is a simple Cloud-based Energy Monitoring Solution." /><meta charset="utf-8" /><meta http-equiv="X-UA-Compatible" content="IE=edge" /><meta name="viewport" content="width=device-width, initial-scale=1" />

<link rel="stylesheet" href="Content/css/bootstrap/bootstrap.min.css" type="text/css" />
<link rel="stylesheet" href="Content/css/bootstrap/font-awesome.min.css" type="text/css" />
<link rel="stylesheet" href="Content/css/bootstrap/animate.min.css" type="text/css" />
<link rel="stylesheet" href="Content/css/bootstrap/prettyPhoto.css" type="text/css" />
<link rel="stylesheet" href="Content/css/bootstrap/main3.css" type="text/css" />
<link rel="stylesheet" href="Content/css/bootstrap/responsive.css" type="text/css" />
<!--[if lt IE 9]>
    <script src="Scripts/bootstrap/html5shiv.js" type="text/javascript"></script>
    <script src="Scripts/bootstrap/respond.min.js" type="text/javascript"></script>
    <![endif]-->
<link rel="shortcut icon" href="favicon.ico" />
<link rel="apple-touch-icon-precomposed" sizes="144x144" href="Images/icons/apple-touch-icon-144-precomposed.png" />
<link rel="apple-touch-icon-precomposed" sizes="114x114" href="Images/icons/apple-touch-icon-114-precomposed.png" />
<link rel="apple-touch-icon-precomposed" sizes="72x72" href="Images/icons/apple-touch-icon-72-precomposed.png" />
<link rel="apple-touch-icon-precomposed" href="Images/icons/apple-touch-icon-57-precomposed.png" />
<script src="Scripts/jquery-1.7.1.min.js" type="text/javascript"></script>
<script src="Scripts/bootstrap/bootstrap.min.js" type="text/javascript"></script>
<script src="Scripts/bootstrap/jquery.prettyPhoto.js" type="text/javascript"></script>
<script src="Scripts/bootstrap/jquery.isotope.min.js" type="text/javascript"></script>
<script src="Scripts/bootstrap/main.js" type="text/javascript"></script>
<script src="Scripts/bootstrap/wow.min.js" type="text/javascript"></script>
<script src="Content/Scripts/jquery.cookiefy.min.js" type="text/javascript"></script>
<link href="/Content/css/bootstrap/SignIn.css" type="text/css" rel="stylesheet" />
</head>
<body class="homepage">
<form method="post" action="./SignIn.aspx?ReturnUrl=%2fDetails%2fGetDetailsNew.ashx%3fid%3d<<ZÄHLER-ID>>&amp;id=<<ZÄHLER-ID>>" id="form1">
<input type="hidden" name="__VIEWSTATE" id="__VIEWSTATE" value="sV4MotHGv2vAU/SPhMTmTYUwseJMR8vf0JBeJC0UVqwmrhUHGwYI8jwengzuIjU5Akov4dqDCTDdsOrgPVmwqoP8rrQ=" />
<input type="hidden" name="__VIEWSTATEGENERATOR" id="__VIEWSTATEGENERATOR" value="ECDA716A" />
<input type="hidden" name="__EVENTVALIDATION" id="__EVENTVALIDATION" value="3MQwDHa/1WsFzaTAtxRoVZanYmQIhxi2BFSB9vG0WNXz3RR1o2dkyNzGuz+YumRmJJWkw+6NNGNekcBCMnGe2gbMX2b6xyt3R4v6RFLPrkhL4CXlNnSHA7IZENlpG3HC7ibp/R+PoLmPKvWygqLNoQLPhH8=" />

<header id="header">
<nav class="navbar navbar-inverse" role="banner">
<div class="container">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-collapse">
<span class="sr-only">Toggle navigation</span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="default.aspx">
<img src="Images/bootstrap/logo.png" alt="logo" style="width: 140px;" /></a>
</div>
<div class="collapse navbar-collapse navbar-right">
<ul class="nav navbar-nav">
 <li><a href="Details/V2/All.aspx">Login</a></li>
<li><a href="Description/HowItWorks.aspx">How it works</a></li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Solutions <i class="fa fa-angle-down"></i></a>
<ul class="dropdown-menu">
<li><a href="Description/Solutions.aspx?slide=0">Properties</a></li>
<li><a href="Description/Solutions.aspx?slide=1">Industrial</a></li>
<li><a href="Description/Solutions.aspx?slide=2">E-Mobility</a></li>
</ul>
</li>
<li><a href="Description/Products.aspx">Products</a></li>
<li><a href="Description/Pricing.aspx">Pricing</a></li>
<li><a href="buy/">Buy</a></li>
<li><a href="Description/Demo/">Live Demo</a></li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Support <i class="fa fa-angle-down"></i></a>
<ul class="dropdown-menu">
<li><a href="installation/">Installation</a></li>
<li><a href="Description/Tutorials/">Tutorials</a></li>
<li><a href="Description/Tutorials/OnlineTraining.aspx">Online training</a></li>
<li><a href="http://en.wiki.smart-me.com" target="_blank">Wiki</a></li>
<li><a href="http://en.wiki.smart-me.com/index.php/Downloads" target="_blank">Downloads</a></li>
</ul>
</li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">More <i class="fa fa-angle-down"></i></a>
<ul class="dropdown-menu">
<li><a href="Description/App.aspx">Apps</a></li>
<li><a href="Description/billing/">smart-me Billing</a></li>
<li><a href="Developer/">Developer</a></li>
<li><a href="About/AboutUs.aspx">About us</a></li>
<li><a href="About/ContactUs.aspx">Contact</a></li>
</ul>
</li>
</ul>
</div>
</div>

</nav>

</header>

<div class="container">
<div class="row">
<div class="col-sm-6 col-md-4 col-md-offset-4">
<div class="account-wall">
<div id="my-tab-content" class="tab-content">
<div class="tab-pane active" id="login">
<img class="profile-img" src="Images/logoBlueSmall.png" alt="" style="width:140px" />
<table id="ContentPlaceHolder1_Login1" cellspacing="0" cellpadding="0">
<tr>
<td>
<p class="validation-summary-errors">
</p>
<input name="ctl00$ContentPlaceHolder1$Login1$UserName" type="text" id="ContentPlaceHolder1_Login1_UserName" class="form-control" autofocus="" placeholder="User name" />
 <input name="ctl00$ContentPlaceHolder1$Login1$Password" type="password" id="ContentPlaceHolder1_Login1_Password" class="form-control" placeholder="Password" />
<input type="submit" name="ctl00$ContentPlaceHolder1$Login1$ctl01" value="Sign In" class="btn btn-lg btn-default btn-block" />
</td>
</tr>
</table>
<br />
<div id="tabs" data-tabs="tabs">
<p class="text-center"><a href="Users/SignUp.aspx">Register as new user</a></p>
<p class="text-center"><a href="Users/RequestPasswordReset.aspx">Forgot password?</a></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br />

<footer id="footer" class="midnight-blue">
<div class="container">
<div class="row">
<div class="col-sm-6">
&copy; 2018 <a target="_blank" href="https://smart-me.com" title="smart-me smart-energy">smart-me</a>
</div>
<div class="col-sm-6">
<ul class="pull-right">
<li><a href="About/Privacy.aspx">Data protection guidelines</a></li>
<li><a href="About/TermsOfService.aspx">Terms of service</a></li>
<li><a href="About/ContactUs.aspx">Contact</a></li>
<li><a href="About/LegalNotice.aspx">Legal details</a></li>
</ul>
</div>
</div>
</div>
</footer>

</form>
<script type="text/javascript">

        $('body').cookiefy({
            displayedHtml: 'We use cookies to ensure that we give you the best experience on our website. If you continue, you agree with &lt;strong&gt;Data protection guidelines&lt;/strong&gt;.'
        });

        var _gaq = _gaq || [];
        _gaq.push(['_setAccount', 'UA-39019091-1']);
        _gaq.push(['_trackPageview']);

        (function () {
            var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
            ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
            var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
        })();

    </script>
</body>
</html>

   httpheader HTTP/1.1 200 OK
Date: Fri, 17 Jul 2020 08:26:07 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: close
Cache-Control: private
Vary: Accept-Encoding
Set-Cookie: ASP.NET_SessionId=sce5mub55v02btjepgctwhy4; path=/; HttpOnly; SameSite=Lax
X-Powered-By: ASP.NET
P3P: CP="CAO PSA OUR"
CF-Cache-Status: DYNAMIC
cf-request-id: 03fd7a8813000072c3c40f7200000001
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Server: cloudflare
CF-RAY: 5b4293868bc272c3-AMS
Content-Encoding: gzip
   httpversion 1.1
   hu_blocking 0
   hu_filecount 5
   hu_port    443
   hu_portSfx 
   ignoreredirects 1
   loglevel   4
   path       /SignIn.aspx?ReturnUrl=%2fDetails%2fGetDetailsNew.ashx%3fid%3d<<ZÄHLER-ID>>&id=<<ZÄHLER-ID>>
   protocol   https
   redirects  0
   sid        
   timeout    2
   url        https://www.smart-me.com:443/SignIn.aspx?ReturnUrl=%2fDetails%2fGetDetailsNew.ashx%3fid%3d<<ZÄHLER-ID>>&id=<<ZÄHLER-ID>>
   value      0
   CompiledRegexes:
   HTTPCookieHash:
     ASP.NET_SessionId;/:
       Name       ASP.NET_SessionId
       Options    path=/; HttpOnly; SameSite=Lax
       Path       /
       Value      sce5mub55v02btjepgctwhy4
     __cfduid;/:
       Name       __cfduid
       Options    expires=Sun, 16-Aug-20 08:20:25 GMT; path=/; domain=.smart-me.com; HttpOnly; SameSite=Lax
       Path       /
       Value      de727087768a4b83ca49b7bbb4b307b351594974025
   QUEUE:
   READINGS:
   REQUEST:
     data       
     header     Cookie: $sid
     ignoreredirects 0
     retryCount 1
     type       update
     url        https://www.smart-me.com:443/SignIn.aspx?ReturnUrl=%2fDetails%2fGetDetailsNew.ashx%3fid%3d<<ZÄHLER-ID>>&id=<<ZÄHLER-ID>
     value      0
   sslargs:
Attributes:
   clearSIdBeforeAuth 1
   enableCookies 1
   handleRedirects 1
   httpVersion 1.1
   reAuthRegex /SignIn.aspx.*
   requestHeader1 Cookie: $sid
   showBody   1
   sid01IdRegex (ASP.NET_SessionId[^;;\n]+)
   sid02Data  ctl00$ContentPlaceHolder1$Login1$UserName=<<E-MAIL>>&ctl00$ContentPlaceHolder1$Login1$Password=<<PASSWORT>>&ctl00%24ContentPlaceHolder1%24Login1%24ctl01=Login
   sid02Header1 Cookie: $sid
   sid02IdRegex Cookie: ([^\n]+)
   sidURL     https://www.smart-me.com/SignIn.aspx?ReturnUrl=%2fDetails%2fGetDetailsNew.ashx%3fid%3d<<ZÄHLER-ID>>
   userattr   requestHeader1 sid01Data sid01Header1 sid01IdRegex sid02Data sid02Header1 sid02IdRegex sidURL

Vielen Dank im Voraus
Grüße^^

amenomade · 17 Juli 2020, 17:22:44

smart-me hat eine REST API, die sogar BasicAuth unterstützt. Da musst Du nur ein Header "Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==" mitschicken, wo die kryptische Zeichen einfach nur user:passwort base64 kodiert sind,

https://smart-me.com/swagger/ui/index

roman1528 · 17 Juli 2020, 17:59:40

Zitat von: amenomade am 17 Juli 2020, 17:22:44
smart-me hat eine REST API, die sogar BasicAuth unterstützt. Da musst Du nur ein Header "Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==" mitschicken, wo die kryptische Zeichen einfach nur user:passwort base64 kodiert sind,

https://smart-me.com/swagger/ui/index

Moin.

Danke, die ist bekannt... liefert aber nur alle 60 Sekunden neue Daten wenn man kein Premium kauft. Ja... LifeTime 149€ okayyyy... aber wieso wenn es anders geht. [Man könnte es ja auch einfach ohne Cloud realisieren... bei dem Kaufpreis]
Die XML-Daten die ich ziehen möchte aktualisieren öfters... teilweise in Sekundenschritten... scheinbar ein Poll oder so. Diese XML liefert auch die LiveDaten für das Frontend von smart-me.

Ist es irgendwie möglich "name" und "value" (3 mal) aus der html zu lesen und in den neuen request zu verpacken? also innerhalb HTTPMOD, oder wird das bloß brutal aufwändig? mehrere Umwege möchte ich nämlich ungern gehen.

Grüße^^

amenomade · 17 Juli 2020, 21:53:08

Also, theoretisch ist alles machbar. Einfach vielleicht aber nicht. Deswegen gibt es eine API...

Sekündlich eine Seite abfragen kann aber dazu führen, dass die Webseite irgendein "ban" Mechanismus einsetzt...

Zur Frage: Das ganze div aspNetHidden in sid holen
Dieses dann mit eine zusätzlich parse Funktion in myUtils und die "replacement" Funktionen von HTTPMOD benutzen

amenomade · 17 Juli 2020, 23:22:13

So... hab es probiert. Nach ein paar falsche Requests hat mich smart-me.com gebannt. Versuche morgen wieder.

amenomade · 18 Juli 2020, 01:40:50

So... jetzt haben wir "morgen"

Folgendes in myUtils:

Code Auswählen

sub urlencode {
    my $s = shift;
    $s =~ s/ /+/g;
    $s =~ s/([^A-Za-z0-9\+-])/sprintf("%%%02X", ord($1))/seg;
    return $s;
}
sub getVIEWSTATE
{ 	
	my ($sid) = shift;
	$sid=~ /__VIEWSTATE" value="(.*?)"/gs;
	return urlencode($1);
}

sub getVIEWSTATEGENERATOR
{
	my ($sid) = shift;
	$sid=~ m/__VIEWSTATEGENERATOR" value="(.*?)"/gs;
	return urlencode($1);
}

sub getEVENTVALIDATION
{
	my ($sid) = shift;
	$sid=~ /__EVENTVALIDATION" value="(.*?)"/gs;
	return urlencode($1);
}

und folgende Definition:

Code Auswählen

defmod smartme HTTPMOD https://www.smart-me.com/Details/V2/All.aspx 0
attr smartme reAuthRegex SignUp.aspx
attr smartme replacement01Mode expression
attr smartme replacement01Regex %%VIEWSTATE%%
attr smartme replacement01Value getVIEWSTATE($defs{smartme}{sid})
attr smartme replacement02Mode expression
attr smartme replacement02Regex %%VIEWSTATEGENERATOR%%
attr smartme replacement02Value getVIEWSTATEGENERATOR($defs{smartme}{sid})
attr smartme replacement03Mode expression
attr smartme replacement03Regex %%EVENTVALIDATION%%
attr smartme replacement03Value getEVENTVALIDATION($defs{smartme}{sid})
attr smartme sid01IdRegex (?s)(__VIEWSTATE.*?__EVENTVALIDATION.*?)\/>
attr smartme sid02Data __VIEWSTATE=%%VIEWSTATE%%&__VIEWSTATEGENERATOR=%%VIEWSTATEGENERATOR%%&__EVENTVALIDATION=%%EVENTVALIDATION%%&ctl00$ContentPlaceHolder1$Login1$UserName=testusername&ctl00$ContentPlaceHolder1$Login1$Password=testpassword&ctl00$ContentPlaceHolder1$Login1$ctl01=Sign+In
attr smartme sid02URL https://www.smart-me.com/SignIn.aspx?ReturnUrl=/Details/V2/All.aspx
attr smartme sidHeader01 Origin: https://www.smart-me.com
attr smartme sidHeader02 Referer: https://www.smart-me.com/SignIn.aspx?ReturnUrl=/Details/V2/All.aspx
attr smartme sidHeader03 User-Agent: Mozilla/5.0 (Windows NT 10.0;; Win64;; x64;; rv:78.0) Gecko/20100101 Firefox/78.0

("testusername" und "testpassword" in sid02Data natürlich ersetzen)
Vorsicht in replacementXXValue wenn das Device anders genannt wird : $defs{smartme}{sid} muss mit dem Name stimmen

Damit bin ich angemeldet und komme zu Hauptseite https://www.smart-me.com/Details/V2/All.aspx
(da ich aber kein Device sondern nur ein Testkonto habe, kann ich nicht weiter testen)

Man kann das sicher noch optimieren, aber es ist spät

: Z.B. nur eine Funktion für alle 3 Parameter, oder Passwort mit replacementMode key verstecken, usw. Man könnte wahrscheinlich auch mit readings und sid01ParseResponse statt mit Funktionen in myUtils arbeiten. Hab ich nicht getestet.

roman1528 · 18 Juli 2020, 09:32:27

Moin.

Du bist ein Schatz

nur leider läuft es nicht

list:

Code Auswählen

Internals:
   BUSY       0
   DEF        https://www.smart-me.com/Details/GetDetailsNew.ashx?id=m40b79723-0f3c-468c-8853-90bba06917fd 0
   FUUID      5f10990f-f33f-8842-5e72-8a8146beea1a6cc2
   Interval   0
   LASTSEND   1595056847.84038
   LastAuthTry 2020-07-18 09:19:54
   MainURL    https://www.smart-me.com/Details/GetDetailsNew.ashx?id=m40b79723-0f3c-468c-8853-90bba06917fd
   ModuleVersion 3.5.22 - 7.2.2020
   NAME       dr_solar_smartme
   NOTIFYDEV  global
   NR         440
   NTFY_ORDER 50-solar_xml_test
   ReplacementEnabled 1
   STATE      ???
   TRIGGERTIME 0
   TRIGGERTIME_FMT 
   TYPE       HTTPMOD
   addr       https://www.smart-me.com:443
   auth       0
   buf        

   code       302
   data       
   displayurl https://www.smart-me.com:443/SignIn.aspx?ReturnUrl=%2fDetails%2fGetDetailsNew.ashx%3fid%3dm40b79723-0f3c-468c-8853-90bba06917fd&id=m40b79723-0f3c-468c-8853-90bba06917fd
   header     Cookie: $sid
Cookie: __cfduid=d30423cb63f47b1d7273f19bf2bebf42a1595056847
   host       www.smart-me.com
   httpbody   
   httpdata   
   httpdatalen -1
   httpheader HTTP/1.1 200 OK
Date: Sat, 18 Jul 2020 07:20:48 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: close
Cache-Control: private
Vary: Accept-Encoding
Set-Cookie: ASP.NET_SessionId=sizg4nyavyhay3p0w2nljzie; path=/; HttpOnly; SameSite=Lax
X-Powered-By: smart-me
P3P: CP="CAO PSA OUR"
CF-Cache-Status: DYNAMIC
cf-request-id: 04026514610000f3ebd7836200000001
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Server: cloudflare
CF-RAY: 5b4a7133c83ef3eb-LHR
Content-Encoding: gzip
   httpversion 1.1
   ignoreredirects 1
   loglevel   4
   path       /SignIn.aspx?ReturnUrl=%2fDetails%2fGetDetailsNew.ashx%3fid%3dm40b79723-0f3c-468c-8853-90bba06917fd&id=m40b79723-0f3c-468c-8853-90bba06917fd
   protocol   https
   redirects  0
   sid        
   timeout    2
   url        https://www.smart-me.com:443/SignIn.aspx?ReturnUrl=%2fDetails%2fGetDetailsNew.ashx%3fid%3dm40b79723-0f3c-468c-8853-90bba06917fd&id=m40b79723-0f3c-468c-8853-90bba06917fd
   value      0
   CompiledRegexes:
   HTTPCookieHash:
     ASP.NET_SessionId;/:
       Name       ASP.NET_SessionId
       Options    path=/; HttpOnly; SameSite=Lax
       Path       /
       Value      sizg4nyavyhay3p0w2nljzie
     __cfduid;/:
       Name       __cfduid
       Options    expires=Mon, 17-Aug-20 07:20:47 GMT; path=/; domain=.smart-me.com; HttpOnly; SameSite=Lax
       Path       /
       Value      d30423cb63f47b1d7273f19bf2bebf42a1595056847
   QUEUE:
   READINGS:
   REQUEST:
     data       
     header     Cookie: $sid
     ignoreredirects 0
     retryCount 0
     type       update
     url        https://www.smart-me.com:443/SignIn.aspx?ReturnUrl=%2fDetails%2fGetDetailsNew.ashx%3fid%3dm40b79723-0f3c-468c-8853-90bba06917fd&id=m40b79723-0f3c-468c-8853-90bba06917fd
     value      0
   sslargs:
Attributes:
   bodyDecode utf-8
   clearSIdBeforeAuth 1
   enableCookies 1
   handleRedirects 1
   httpVersion 1.1
   reAuthRegex /SignIn.aspx.*
   replacement01Mode expression
   replacement01Regex %%VIEWSTATE%%
   replacement01Value getVIEWSTATE($defs{dr_solar_smartme}{views})
   replacement02Mode expression
   replacement02Regex %%VIEWSTATEGENERATOR%%
   replacement02Value getVIEWSTATEGENERATOR($defs{dr_solar_smartme}{viewg})
   replacement03Mode expression
   replacement03Regex %%EVENTVALIDATION%%
   replacement03Value getEVENTVALIDATION($defs{dr_solar_smartme}{eventv})
   requestHeader1 Cookie: $sid
   showBody   1
   sid01IdRegex (ASP.NET_SessionId[^;;\n]+)
   sid02Data  __VIEWSTATE=%%VIEWSTATE%%&__VIEWSTATEGENERATOR=%%VIEWSTATEGENERATOR%%&__EVENTVALIDATION=%%EVENTVALIDATION%%&ctl00$ContentPlaceHolder1$Login1$UserName=E-MAIL&ctl00$ContentPlaceHolder1$Login1$Password=PASSWORT&ctl00%24ContentPlaceHolder1%24Login1%24ctl01=Login
   sid02Header1 Cookie: $sid
   sid02IdRegex Cookie: ([^\n]+)
   sid02URL   https://www.smart-me.com/SignIn.aspx?ReturnUrl=%2fDetails%2fGetDetailsNew.ashx%3fid%3dm40b79723-0f3c-468c-8853-90bba06917fd
   sidHeader1 Origin: https://www.smart-me.com
   sidHeader2 Referer: https://www.smart-me.com/SignIn.aspx?ReturnUrl=%2fDetails%2fGetDetailsNew.ashx%3fid%3dm40b79723-0f3c-468c-8853-90bba06917fd&id=m40b79723-0f3c-468c-8853-90bba06917fd
   sidHeader3 User-Agent: Mozilla/5.0 (Windows NT 10.0;; Win64;; x64;; rv:78.0) Gecko/20100101 Firefox/78.0
   userattr   replacement01Mode:reading,internal,text,expression,key replacement01Regex replacement01Value replacement02Mode:reading,internal,text,expression,key replacement02Regex replacement02Value replacement03Mode:reading,internal,text,expression,key replacement03Regex replacement03Value requestHeader1 sid01Data sid01Header1 sid01IdRegex sid02Data sid02Header1 sid02IdRegex sid02URL sidHeader1 sidHeader2 sidHeader3
   verbose    5

htmlbody musste ich rausnehmen... da standen nur komische Zeichen und das Forum hat gemeckert.

Log:

Code Auswählen

2020.07.18 09:20:45 5:  dr_solar_smartme: set called with clearCookies 
2020.07.18 09:20:47 5:  dr_solar_smartme: set called with reread 
2020.07.18 09:20:47 5:  dr_solar_smartme: GetUpdate called (reread)
2020.07.18 09:20:47 5:  dr_solar_smartme: AddToQueue adds type update to URL https://www.smart-me.com/Details/GetDetailsNew.ashx?id=m40b79723-0f3c-468c-8853-90bba06917fd, no data, header Cookie: $sid, retry 0, initial queue len: 0
2020.07.18 09:20:47 5:  dr_solar_smartme: HandleSendQueue called, qlen = 1
2020.07.18 09:20:47 5:  dr_solar_smartme: GetRegex found precompiled Regex for replacement01 as (?^:%%VIEWSTATE%%)
2020.07.18 09:20:47 5:  dr_solar_smartme: Replace called for type update, regex (?^:%%VIEWSTATE%%), mode expression, value getVIEWSTATE($defs{dr_solar_smartme}{views}) input: Cookie: $sid
2020.07.18 09:20:47 5:  dr_solar_smartme: GetRegex found precompiled Regex for replacement02 as (?^:%%VIEWSTATEGENERATOR%%)
2020.07.18 09:20:47 5:  dr_solar_smartme: Replace called for type update, regex (?^:%%VIEWSTATEGENERATOR%%), mode expression, value getVIEWSTATEGENERATOR($defs{dr_solar_smartme}{viewg}) input: Cookie: $sid
2020.07.18 09:20:47 5:  dr_solar_smartme: GetRegex found precompiled Regex for replacement03 as (?^:%%EVENTVALIDATION%%)
2020.07.18 09:20:47 5:  dr_solar_smartme: Replace called for type update, regex (?^:%%EVENTVALIDATION%%), mode expression, value getEVENTVALIDATION($defs{dr_solar_smartme}{eventv}) input: Cookie: $sid
2020.07.18 09:20:47 5:  dr_solar_smartme: GetRegex found precompiled Regex for replacement01 as (?^:%%VIEWSTATE%%)
2020.07.18 09:20:47 5:  dr_solar_smartme: Replace called for type update, regex (?^:%%VIEWSTATE%%), mode expression, value getVIEWSTATE($defs{dr_solar_smartme}{views}) input: 
2020.07.18 09:20:47 5:  dr_solar_smartme: GetRegex found precompiled Regex for replacement02 as (?^:%%VIEWSTATEGENERATOR%%)
2020.07.18 09:20:47 5:  dr_solar_smartme: Replace called for type update, regex (?^:%%VIEWSTATEGENERATOR%%), mode expression, value getVIEWSTATEGENERATOR($defs{dr_solar_smartme}{viewg}) input: 
2020.07.18 09:20:47 5:  dr_solar_smartme: GetRegex found precompiled Regex for replacement03 as (?^:%%EVENTVALIDATION%%)
2020.07.18 09:20:47 5:  dr_solar_smartme: Replace called for type update, regex (?^:%%EVENTVALIDATION%%), mode expression, value getEVENTVALIDATION($defs{dr_solar_smartme}{eventv}) input: 
2020.07.18 09:20:47 5:  dr_solar_smartme: GetRegex found precompiled Regex for replacement01 as (?^:%%VIEWSTATE%%)
2020.07.18 09:20:47 5:  dr_solar_smartme: Replace called for type update, regex (?^:%%VIEWSTATE%%), mode expression, value getVIEWSTATE($defs{dr_solar_smartme}{views}) input: https://www.smart-me.com/Details/GetDetailsNew.ashx?id=m40b79723-0f3c-468c-8853-90bba06917fd
2020.07.18 09:20:47 5:  dr_solar_smartme: GetRegex found precompiled Regex for replacement02 as (?^:%%VIEWSTATEGENERATOR%%)
2020.07.18 09:20:47 5:  dr_solar_smartme: Replace called for type update, regex (?^:%%VIEWSTATEGENERATOR%%), mode expression, value getVIEWSTATEGENERATOR($defs{dr_solar_smartme}{viewg}) input: https://www.smart-me.com/Details/GetDetailsNew.ashx?id=m40b79723-0f3c-468c-8853-90bba06917fd
2020.07.18 09:20:47 5:  dr_solar_smartme: GetRegex found precompiled Regex for replacement03 as (?^:%%EVENTVALIDATION%%)
2020.07.18 09:20:47 5:  dr_solar_smartme: Replace called for type update, regex (?^:%%EVENTVALIDATION%%), mode expression, value getEVENTVALIDATION($defs{dr_solar_smartme}{eventv}) input: https://www.smart-me.com/Details/GetDetailsNew.ashx?id=m40b79723-0f3c-468c-8853-90bba06917fd
2020.07.18 09:20:47 4:  dr_solar_smartme: HandleSendQueue sends update with timeout 2 to https://www.smart-me.com/Details/GetDetailsNew.ashx?id=m40b79723-0f3c-468c-8853-90bba06917fd, No Data, 
header: Cookie: $sid
2020.07.18 09:20:47 5:  HttpUtils url=https://www.smart-me.com/Details/GetDetailsNew.ashx?id=m40b79723-0f3c-468c-8853-90bba06917fd
2020.07.18 09:20:47 4:  IP: www.smart-me.com -> 104.18.42.92
2020.07.18 09:20:47 5:  HttpUtils request header:
GET /Details/GetDetailsNew.ashx?id=m40b79723-0f3c-468c-8853-90bba06917fd HTTP/1.1
Host: www.smart-me.com
User-Agent: fhem
Accept-Encoding: gzip,deflate
Connection: Close
Cookie: $sid

2020.07.18 09:20:47 4:  https://www.smart-me.com/Details/GetDetailsNew.ashx?id=m40b79723-0f3c-468c-8853-90bba06917fd: HTTP response code 302
2020.07.18 09:20:47 5:  HttpUtils https://www.smart-me.com/Details/GetDetailsNew.ashx?id=m40b79723-0f3c-468c-8853-90bba06917fd: Got data, length: 258
2020.07.18 09:20:47 5:  HttpUtils response header:
HTTP/1.1 302 Found
Date: Sat, 18 Jul 2020 07:20:47 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: close
Set-Cookie: __cfduid=d30423cb63f47b1d7273f19bf2bebf42a1595056847; expires=Mon, 17-Aug-20 07:20:47 GMT; path=/; domain=.smart-me.com; HttpOnly; SameSite=Lax
Cache-Control: private
Location: /SignIn.aspx?ReturnUrl=%2fDetails%2fGetDetailsNew.ashx%3fid%3dm40b79723-0f3c-468c-8853-90bba06917fd&id=m40b79723-0f3c-468c-8853-90bba06917fd
X-Powered-By: smart-me
P3P: CP="CAO PSA OUR"
CF-Cache-Status: DYNAMIC
cf-request-id: 040265135b00000712ae0ef200000001
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Server: cloudflare
CF-RAY: 5b4a71322d290712-LHR
2020.07.18 09:20:47 4:  dr_solar_smartme: Read callback: request type was update retry 0,
header: HTTP/1.1 302 Found
Date: Sat, 18 Jul 2020 07:20:47 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: close
Set-Cookie: __cfduid=d30423cb63f47b1d7273f19bf2bebf42a1595056847; expires=Mon, 17-Aug-20 07:20:47 GMT; path=/; domain=.smart-me.com; HttpOnly; SameSite=Lax
Cache-Control: private
Location: /SignIn.aspx?ReturnUrl=%2fDetails%2fGetDetailsNew.ashx%3fid%3dm40b79723-0f3c-468c-8853-90bba06917fd&id=m40b79723-0f3c-468c-8853-90bba06917fd
X-Powered-By: smart-me
P3P: CP="CAO PSA OUR"
CF-Cache-Status: DYNAMIC
cf-request-id: 040265135b00000712ae0ef200000001
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Server: cloudflare
CF-RAY: 5b4a71322d290712-LHR, body length 258
2020.07.18 09:20:47 5:  dr_solar_smartme: Read callback: body

Object moved to here.



2020.07.18 09:20:47 4:  dr_solar_smartme: Read is decoding the buffer as utf-8 
2020.07.18 09:20:47 5:  dr_solar_smartme: GetCookies is looking for Cookies
2020.07.18 09:20:47 4:  dr_solar_smartme: GetCookies parsed Cookie: __cfduid Wert d30423cb63f47b1d7273f19bf2bebf42a1595056847 Rest expires=Mon, 17-Aug-20 07:20:47 GMT; path=/; domain=.smart-me.com; HttpOnly; SameSite=Lax
2020.07.18 09:20:47 5:  dr_solar_smartme: ExtractSid called, context reading, num 
2020.07.18 09:20:47 4:  dr_solar_smartme: checking for redirects, code=302, ignore=0
2020.07.18 09:20:47 4:  dr_solar_smartme: https://www.smart-me.com/Details/GetDetailsNew.ashx?id=m40b79723-0f3c-468c-8853-90bba06917fd: Redirect (1) to https://www.smart-me.com:443/SignIn.aspx?ReturnUrl=%2fDetails%2fGetDetailsNew.ashx%3fid%3dm40b79723-0f3c-468c-8853-90bba06917fd&id=m40b79723-0f3c-468c-8853-90bba06917fd
2020.07.18 09:20:47 5:  dr_solar_smartme: AddToQueue prepends type update to URL https://www.smart-me.com:443/SignIn.aspx?ReturnUrl=%2fDetails%2fGetDetailsNew.ashx%3fid%3dm40b79723-0f3c-468c-8853-90bba06917fd&id=m40b79723-0f3c-468c-8853-90bba06917fd, no data, header Cookie: $sid, retry 0, initial queue len: 0
2020.07.18 09:20:47 5:  dr_solar_smartme: HandleSendQueue called, qlen = 1
2020.07.18 09:20:47 5:  dr_solar_smartme: GetRegex found precompiled Regex for replacement01 as (?^:%%VIEWSTATE%%)
2020.07.18 09:20:47 5:  dr_solar_smartme: Replace called for type update, regex (?^:%%VIEWSTATE%%), mode expression, value getVIEWSTATE($defs{dr_solar_smartme}{views}) input: Cookie: $sid
2020.07.18 09:20:47 5:  dr_solar_smartme: GetRegex found precompiled Regex for replacement02 as (?^:%%VIEWSTATEGENERATOR%%)
2020.07.18 09:20:47 5:  dr_solar_smartme: Replace called for type update, regex (?^:%%VIEWSTATEGENERATOR%%), mode expression, value getVIEWSTATEGENERATOR($defs{dr_solar_smartme}{viewg}) input: Cookie: $sid
2020.07.18 09:20:47 5:  dr_solar_smartme: GetRegex found precompiled Regex for replacement03 as (?^:%%EVENTVALIDATION%%)
2020.07.18 09:20:47 5:  dr_solar_smartme: Replace called for type update, regex (?^:%%EVENTVALIDATION%%), mode expression, value getEVENTVALIDATION($defs{dr_solar_smartme}{eventv}) input: Cookie: $sid
2020.07.18 09:20:47 5:  dr_solar_smartme: GetRegex found precompiled Regex for replacement01 as (?^:%%VIEWSTATE%%)
2020.07.18 09:20:47 5:  dr_solar_smartme: Replace called for type update, regex (?^:%%VIEWSTATE%%), mode expression, value getVIEWSTATE($defs{dr_solar_smartme}{views}) input: 
2020.07.18 09:20:47 5:  dr_solar_smartme: GetRegex found precompiled Regex for replacement02 as (?^:%%VIEWSTATEGENERATOR%%)
2020.07.18 09:20:47 5:  dr_solar_smartme: Replace called for type update, regex (?^:%%VIEWSTATEGENERATOR%%), mode expression, value getVIEWSTATEGENERATOR($defs{dr_solar_smartme}{viewg}) input: 
2020.07.18 09:20:47 5:  dr_solar_smartme: GetRegex found precompiled Regex for replacement03 as (?^:%%EVENTVALIDATION%%)
2020.07.18 09:20:47 5:  dr_solar_smartme: Replace called for type update, regex (?^:%%EVENTVALIDATION%%), mode expression, value getEVENTVALIDATION($defs{dr_solar_smartme}{eventv}) input: 
2020.07.18 09:20:47 5:  dr_solar_smartme: GetRegex found precompiled Regex for replacement01 as (?^:%%VIEWSTATE%%)
2020.07.18 09:20:47 5:  dr_solar_smartme: Replace called for type update, regex (?^:%%VIEWSTATE%%), mode expression, value getVIEWSTATE($defs{dr_solar_smartme}{views}) input: https://www.smart-me.com:443/SignIn.aspx?ReturnUrl=%2fDetails%2fGetDetailsNew.ashx%3fid%3dm40b79723-0f3c-468c-8853-90bba06917fd&id=m40b79723-0f3c-468c-8853-90bba06917fd
2020.07.18 09:20:47 5:  dr_solar_smartme: GetRegex found precompiled Regex for replacement02 as (?^:%%VIEWSTATEGENERATOR%%)
2020.07.18 09:20:47 5:  dr_solar_smartme: Replace called for type update, regex (?^:%%VIEWSTATEGENERATOR%%), mode expression, value getVIEWSTATEGENERATOR($defs{dr_solar_smartme}{viewg}) input: https://www.smart-me.com:443/SignIn.aspx?ReturnUrl=%2fDetails%2fGetDetailsNew.ashx%3fid%3dm40b79723-0f3c-468c-8853-90bba06917fd&id=m40b79723-0f3c-468c-8853-90bba06917fd
2020.07.18 09:20:47 5:  dr_solar_smartme: GetRegex found precompiled Regex for replacement03 as (?^:%%EVENTVALIDATION%%)
2020.07.18 09:20:47 5:  dr_solar_smartme: Replace called for type update, regex (?^:%%EVENTVALIDATION%%), mode expression, value getEVENTVALIDATION($defs{dr_solar_smartme}{eventv}) input: https://www.smart-me.com:443/SignIn.aspx?ReturnUrl=%2fDetails%2fGetDetailsNew.ashx%3fid%3dm40b79723-0f3c-468c-8853-90bba06917fd&id=m40b79723-0f3c-468c-8853-90bba06917fd
2020.07.18 09:20:47 5:  dr_solar_smartme: HandleSendQueue is using Cookie __cfduid with path / and Value d30423cb63f47b1d7273f19bf2bebf42a1595056847 (key __cfduid;/, destination path is /SignIn.aspx?ReturnUrl=%2fDetails%2fGetDetailsNew.ashx%3fid%3dm40b79723-0f3c-468c-8853-90bba06917fd&id=m40b79723-0f3c-468c-8853-90bba06917fd)
2020.07.18 09:20:47 5:  dr_solar_smartme: HandleSendQueue is adding Cookie header: __cfduid=d30423cb63f47b1d7273f19bf2bebf42a1595056847
2020.07.18 09:20:47 4:  dr_solar_smartme: HandleSendQueue sends update with timeout 2 to https://www.smart-me.com:443/SignIn.aspx?ReturnUrl=%2fDetails%2fGetDetailsNew.ashx%3fid%3dm40b79723-0f3c-468c-8853-90bba06917fd&id=m40b79723-0f3c-468c-8853-90bba06917fd, No Data, 
header: Cookie: $sid
Cookie: __cfduid=d30423cb63f47b1d7273f19bf2bebf42a1595056847
2020.07.18 09:20:47 5:  HttpUtils url=https://www.smart-me.com:443/SignIn.aspx?ReturnUrl=%2fDetails%2fGetDetailsNew.ashx%3fid%3dm40b79723-0f3c-468c-8853-90bba06917fd&id=m40b79723-0f3c-468c-8853-90bba06917fd
2020.07.18 09:20:47 4:  IP: www.smart-me.com -> 104.18.42.92
2020.07.18 09:20:47 5:  HttpUtils request header:
GET /SignIn.aspx?ReturnUrl=%2fDetails%2fGetDetailsNew.ashx%3fid%3dm40b79723-0f3c-468c-8853-90bba06917fd&id=m40b79723-0f3c-468c-8853-90bba06917fd HTTP/1.1
Host: www.smart-me.com
User-Agent: fhem
Accept-Encoding: gzip,deflate
Connection: Close
Cookie: $sid
Cookie: __cfduid=d30423cb63f47b1d7273f19bf2bebf42a1595056847

2020.07.18 09:20:49 3:  dr_solar_smartme: Read callback: Error: read from https://www.smart-me.com:443 timed out
2020.07.18 09:20:49 4:  dr_solar_smartme: Read callback: request type was update retry 0,
header: HTTP/1.1 200 OK
Date: Sat, 18 Jul 2020 07:20:48 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: close
Cache-Control: private
Vary: Accept-Encoding
Set-Cookie: ASP.NET_SessionId=sizg4nyavyhay3p0w2nljzie; path=/; HttpOnly; SameSite=Lax
X-Powered-By: smart-me
P3P: CP="CAO PSA OUR"
CF-Cache-Status: DYNAMIC
cf-request-id: 04026514610000f3ebd7836200000001
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Server: cloudflare
CF-RAY: 5b4a7133c83ef3eb-LHR
Content-Encoding: gzip, no body
2020.07.18 09:20:49 5:  dr_solar_smartme: Read callback: body empty
2020.07.18 09:20:49 4:  dr_solar_smartme: Read is decoding the buffer as utf-8 
2020.07.18 09:20:49 5:  dr_solar_smartme: GetCookies is looking for Cookies
2020.07.18 09:20:49 4:  dr_solar_smartme: GetCookies parsed Cookie: ASP.NET_SessionId Wert sizg4nyavyhay3p0w2nljzie Rest path=/; HttpOnly; SameSite=Lax
2020.07.18 09:20:49 5:  dr_solar_smartme: ExtractSid called, context reading, num 
2020.07.18 09:20:49 4:  dr_solar_smartme: checking for redirects, code=200, ignore=0
2020.07.18 09:20:49 4:  dr_solar_smartme: no redirects to handle
2020.07.18 09:20:49 5:  dr_solar_smartme: GetRegex found precompiled reAuthRegex for  as (?^:/SignIn.aspx.*)
2020.07.18 09:20:49 5:  dr_solar_smartme: CheckAuth is checking buffer with ReAuthRegex (?^:/SignIn.aspx.*)
2020.07.18 09:20:49 5:  dr_solar_smartme: CheckAuth decided no authentication required

Zudem bekomm ich häufig so ein Müll angezeigt den keiner lesen kann... (list)

Aber wir sind schon einen Schritt weiter... ich bekomme ab und an Klartext mit einem Error wegen der Origin.

Code Auswählen

<!DOCTYPE html>
<html>
<head>
<title>Invalid postback or callback argument. &nbsp;Event validation is enabled using &lt;pages enableEventValidation=&quot;true&quot;/&gt; in configuration or &lt;%@ Page EnableEventValidation=&quot;true&quot; %&gt; in a page. &nbsp;For security purposes, this feature verifies that arguments to postback or callback events originate from the server control that originally rendered them. &nbsp;If the data is valid and expected, use the ClientScriptManager.RegisterForEventValidation method in order to register the postback or callback data for validation.</title>
<meta name="viewport" content="width=device-width" />
<style>
         body {font-family:"Verdana";font-weight:normal;font-size: .7em;color:black;} 
         p {font-family:"Verdana";font-weight:normal;color:black;margin-top: -5px}
         b {font-family:"Verdana";font-weight:bold;color:black;margin-top: -5px}
         H1 { font-family:"Verdana";font-weight:normal;font-size:18pt;color:red }
         H2 { font-family:"Verdana";font-weight:normal;font-size:14pt;color:maroon }
         pre {font-family:"Consolas","Lucida Console",Monospace;font-size:11pt;margin:0;padding:0.5em;line-height:14pt}
         .marker {font-weight: bold; color: black;text-decoration: none;}
         .version {color: gray;}
         .error {margin-bottom: 10px;}
         .expandable { text-decoration:underline; font-weight:bold; color:navy; cursor:hand; }
         @media screen and (max-width: 639px) {
          pre { width: 440px; overflow: auto; white-space: pre-wrap; word-wrap: break-word; }
         }
         @media screen and (max-width: 479px) {
          pre { width: 280px; }
         }
        </style>
</head>
<body bgcolor="white">
<span><H1>Server Error in '/' Application.<hr width=100% size=1 color=silver></H1>
<h2> <i>Invalid postback or callback argument. &nbsp;Event validation is enabled using &lt;pages enableEventValidation=&quot;true&quot;/&gt; in configuration or &lt;%@ Page EnableEventValidation=&quot;true&quot; %&gt; in a page. &nbsp;For security purposes, this feature verifies that arguments to postback or callback events originate from the server control that originally rendered them. &nbsp;If the data is valid and expected, use the ClientScriptManager.RegisterForEventValidation method in order to register the postback or callback data for validation.</i> </h2></span>
<font face="Arial, Helvetica, Geneva, SunSans-Regular, sans-serif ">
<b> Description: </b>An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.
<br><br>
<b> Exception Details: </b>System.ArgumentException: Invalid postback or callback argument. &nbsp;Event validation is enabled using &lt;pages enableEventValidation=&quot;true&quot;/&gt; in configuration or &lt;%@ Page EnableEventValidation=&quot;true&quot; %&gt; in a page. &nbsp;For security purposes, this feature verifies that arguments to postback or callback events originate from the server control that originally rendered them. &nbsp;If the data is valid and expected, use the ClientScriptManager.RegisterForEventValidation method in order to register the postback or callback data for validation.<br><br>
<b>Source Error:</b> <br><br>
<table width=100% bgcolor="#ffffcc">
<tr>
<td>
<code>
An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.</code>
</td>
</tr>
</table>
<br>
<b>Stack Trace:</b> <br><br>
<table width=100% bgcolor="#ffffcc">
<tr>
<td>
<code><pre>
[ArgumentException: Invalid postback or callback argument.  Event validation is enabled using &lt;pages enableEventValidation=&quot;true&quot;/&gt; in configuration or &lt;%@ Page EnableEventValidation=&quot;true&quot; %&gt; in a page.  For security purposes, this feature verifies that arguments to postback or callback events originate from the server control that originally rendered them.  If the data is valid and expected, use the ClientScriptManager.RegisterForEventValidation method in order to register the postback or callback data for validation.]
   System.Web.UI.ClientScriptManager.ValidateEvent(String uniqueId, String argument) +11822878
   System.Web.UI.Control.ValidateEvent(String uniqueID, String eventArgument) +143
   System.Web.UI.WebControls.TextBox.LoadPostData(String postDataKey, NameValueCollection postCollection) +38
   System.Web.UI.Page.ProcessPostData(NameValueCollection postData, Boolean fBeforeLoad) +558
   System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) +2927
</pre></code>
</td>
</tr>
</table>
<br>
<hr width=100% size=1 color=silver>
<b>Version Information:</b>&nbsp;Microsoft .NET Framework Version:4.0.30319; ASP.NET Version:4.7.3535.0
</font>
</body>
</html>

Ich hab so ein bissl das gefühl, dass was mit meinen sid01 / sid02 und den Cookies nicht passt. Laut Log scheint er auch irgendwie nicht diese Hash's VIEWSTATE und co. richtig zu lesen.
Könnest du da nochmal drüber schauen? Bitte nicht wundern, ich habe die Variablen der replacements geändert, weil ich das Gefühl hatte, dass er da was falsches ersetzt (ja auch in der myUtils).

Grüße

EDIT: Mir ist da noch was in den Sinn gekommen... <form method="post"> hat das Relevanz? Im Log lese ich immer nur GET.

amenomade · 18 Juli 2020, 23:31:11

Zitat2020.07.18 09:20:49 3: dr_solar_smartme: Read callback: Error: read from https://www.smart-me.com:443 timed out

Das ist dein Problem. Der war noch nicht so weit, dass er das Login Verfahren ausführen konnte. Schon beim sid01 Step hat er ein Timeout bekommen
Also, das Timeout erhöhen.

roman1528 · 19 Juli 2020, 07:32:13

Moin.

Timeout 10 und deine Attribute und wir sind wieder einen Schritt weiter...

Code Auswählen

An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.</code>
</td>
</tr>
</table>
<br>
<b>Stack Trace:</b> <br><br>
<table width=100% bgcolor="#ffffcc">
<tr>
<td>
<code><pre>
[FormatException: Invalid length for a Base-64 char array or string.]
   System.Convert.FromBase64_Decode(Char* startInputPtr, Int32 inputLength, Byte* startDestPtr, Int32 destLength) +456
   System.Convert.FromBase64CharPtr(Char* inputPtr, Int32 inputLength) +128
   System.Convert.FromBase64String(String s) +49
   System.Web.UI.ObjectStateFormatter.Deserialize(String inputString, Purpose purpose) +91
   System.Web.UI.Util.DeserializeWithAssert(IStateFormatter2 formatter, String serializedState, Purpose purpose) +61
   System.Web.UI.HiddenFieldPageStatePersister.Load() +177

[ViewStateException: Invalid viewstate. 
	Client IP: 162.158.159.109
	Port: 29184
	Referer: https://www.smart-me.com/SignIn.aspx?ReturnUrl=%2fDetails%2fGetDetailsNew.ashx%3fid%3dm40b79723-0f3c-468c-8853-90bba06917fd&amp;id=m40b79723-0f3c-468c-8853-90bba06917fd
	Path: /SignIn.aspx
	User-Agent: Mozilla/5.0 (Windows NT 10.0;; Win64;; x64;; rv:78.0) Gecko/20100101 Firefox/78.0
	ViewState: VzE2gG NBZyvN/pFEqTqW4eTgfetFdcHjLVRlCt9vFr83LA8MZxVdqZrurYoDcOge9qpPgQca2SNc8xnmrJK5Va5Te8=]

[HttpException (0x80004005): The state information is invalid for this page and might be corrupted.]
   System.Web.UI.ViewStateException.ThrowError(Exception inner, String persistedState, String errorPageMessage, Boolean macValidationError) +153
   System.Web.UI.HiddenFieldPageStatePersister.Load() +317
   System.Web.UI.Page.LoadPageStateFromPersistenceMedium() +367
   System.Web.UI.Page.LoadAllState() +46
   System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) +9458
   System.Web.UI.Page.ProcessRequest(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) +345
   System.Web.UI.Page.ProcessRequest() +75
   System.Web.UI.Page.ProcessRequest(HttpContext context) +70
   System.Web.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() +790
   System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step) +195
   System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean&amp; completedSynchronously) +88
</pre></code>

Ich glaube da geht irgendein URLENCODE in die Hose

Grüße^^

roman1528 · 19 Juli 2020, 09:27:58

Moin nochmal...

Ich hab's... die Login-Daten sowie der Rest der POST-data sollten natürlich auch URL-encoded sein... Mein Fehler

Ich hab die XML im httpbody!

Vielen vielen Dank für deine Unterstützung. Das Cookie für die session wird auch übernommen.

Grüße^^

amenomade · 19 Juli 2020, 11:20:23

Schön

Da bitte noch ein [gelöst] im Titel des ersten Posts,
Und für die Nachwelt noch ein "list" des Devices + ggf Code aus myUtils, wie es jetzt aussieht (das ganze natürlich anonymisiert

)

roman1528 · 19 Juli 2020, 11:25:18

Ja mach ich.. sobald das stabil läuft...

ich bekomme nämlich immer noch diesen Blödsinn den man nicht lesen kann, woraus httpmod natürlich auch nichts lesen kann...

ich habe allerdings das gefühl, dass ich gebannt wurde, weil ich auch bei 30 Sekunden ein TimeOut bekomme.

An meinem Rechner (gleiche öffentliche IP) bekomme ich allerdings noch zugriff.
Ich glaube httpmod liest nicht beim ersten zugriff auf SignIn.aspx die Daten korrekt aus... sondern erst beim 2. oder 3. mal. in der zwischenzeit sind so viele anfragen eingegangen, dass smart-me dicht macht. 500 unhandled exception, validation error.......

Grüße^^

amenomade · 19 Juli 2020, 11:59:57

Ja so war es auch bei mir nach ein Paar HTTP 500.

Ich habe mehrere Ideen, wie man es optimieren könnte. Im Moment (zumindest in meiner Version) läuft es so:

- Aufruf Zielseite. Da nicht angemeldet, wird man nach Loginseite (schon) automatisch weitergeleitet
- reAuthRegex merkt, dass eine Authentifizierung nötig ist => sid01 = 2. Aufruf der Zielseite, mit Weiterleitung nach Loginseite
- hier liest man die VIEWSTATE Infos
- Aufruf POST Loginseite, und nach erfolgreicher Anmeldung, automatische Weiterleitung auf Zielseite
- HTTPMOD macht wiederum eine Aufruf der Zielseite

Also insgesamt viele unnötige Requests, davon einige ignoriert werden.
Man kann wahrscheinlich:
- schon beim ersten Aufruf und folgenden automatischen Weiterleitung die VIEWSTATE Daten holen
- Nach dem POST und automatische Weiterleitung direkt das Ergebnis parsen (dontRequeueAfterAuth + sid02ParseResponse)

Und dann bleibt noch die Frage: braucht die Seite systematisch bei jedem Aufruf aktuelle VIEWSTATE Daten von der letzten Antwort, damit er eingeloggt bleibt?

Das habe ich noch nicht ganz analysieren können. Ich komme hoffentlich erst heute Abend drauf.

roman1528 · 19 Juli 2020, 12:06:39

Zitat von: amenomade am 19 Juli 2020, 11:59:57Und dann bleibt noch die Frage: braucht die Seite systematisch bei jedem Aufruf aktuelle VIEWSTATE Daten von der letzten Antwort, damit er eingeloggt bleibt?

So wie ich das sehe braucht es dann nicht direkt wieder eine Auth. Schau dir das Cookie an welches du nach dem GET der XML bekommst (im Browser). Da ist ein fetter .ASPXAUTH-Token und 2 id's drin. Wenn ich die XML im Browser nach einer Weile aktualisiere will er ja auch nicht wieder einen LogIn.

Grüße^^

amenomade · 19 Juli 2020, 15:18:11

Zitat von: roman1528 am 19 Juli 2020, 12:06:39
Wenn ich die XML im Browser nach einer Weile aktualisiere will er ja auch nicht wieder einen LogIn.

Die Frage ist eher, ob der Browser wieder die aktuellste VIEWSTATE Daten schickt. Aber Du hast Recht, wenn schon ein .ASPXAUTH-TOken da ist, sollte es reichen.