[gelöst] DbLogRep Probleme mit Timeout

Begonnen von andies, 12 Juni 2020, 22:57:18

Vorheriges Thema - Nächstes Thema

andies

Ich benötige mal Hilfe. Ich habe ein DbLogRep Device, bei dem alte Einträge gelöscht werden sollen:
nals:
   DATABASE   fhem
   DEF        DbLog
   FUUID      5e244bd9-f33f-1115-8e73-a46a39d31299868e
   FVERSION   93_DbRep.pm:v8.40.1-s21965/2020-05-18
   LASTCMD     
   MODEL      Client
   NAME       DbLogRep
   NOTIFYDEV  global,DbLogRep
   NR         33
   NTFY_ORDER 50-DbLogRep
   ROLE       Client
   STATE      connected
   TYPE       DbRep
   UTF8       0
   HELPER:
     DBLOGDEVICE DbLog
     GRANTS     ALL PRIVILEGES
     IDRETRIES  3
     MINTS      2020-04-19 06:28:59
     PACKAGE    main
     SQLHIST   
     VERSION    8.40.1
     DBREPCOL:
       COLSET     1
       DEVICE     64
       EVENT      512
       READING    64
       TYPE       64
       UNIT       32
       VALUE      128
   READINGS:
     2020-06-12 22:53:56   index_state     Index Report_Idx doesn't exist. Please create the index by "set DbLogRep index recreate_Report_Idx" command !
     2020-06-12 04:18:49   number_rows_deleted 96486
     2020-06-12 22:53:56   state           connected
Attributes:
   allowDeletion 1
   group      intern
   timeOlderThan d:50

Statt dessen sehe ich im Log
2020.06.11 02:01:00 1: DbRep DbLogRep -> BlockingCall  pid: Timeout: process terminated
Weiss jemand, was ich da falsch mache?
FHEM 6.1 auf RaspPi3 (Raspbian:  6.1.21-v8+; Perl: v5.32.1)
SIGNALduino (433 MHz) und HM-UART (868 MHz), Sonoff, Blitzwolf, Somfy RTS, CAME-Gartentor, Volkszähler, Keyence-Sensor, Homematic-Sensoren und -thermostat, Ferraris-Zähler für Wasseruhr, Openlink-Nachbau Viessmann

DS_Starter

Es gibt ein timeout Attribut welches du hochsetzen kannst.  Du solltest auch den index anlegen wie angegeben um die Performance zu verbessern.

Lg,
Heiko
ESXi@NUC+Debian+MariaDB, PV: SMA, Victron MPII+Pylontech+CerboGX
Maintainer: SSCam, SSChatBot, SSCal, SSFile, DbLog/DbRep, Log2Syslog, SolarForecast,Watches, Dashboard, PylonLowVoltage
Kaffeekasse: https://www.paypal.me/HMaaz
Contrib: https://svn.fhem.de/trac/browser/trunk/fhem/contrib/DS_Starter

andies

Danke, nach dem Anlegen des Index ist die Meldung weg.
FHEM 6.1 auf RaspPi3 (Raspbian:  6.1.21-v8+; Perl: v5.32.1)
SIGNALduino (433 MHz) und HM-UART (868 MHz), Sonoff, Blitzwolf, Somfy RTS, CAME-Gartentor, Volkszähler, Keyence-Sensor, Homematic-Sensoren und -thermostat, Ferraris-Zähler für Wasseruhr, Openlink-Nachbau Viessmann

andies

#3
Leider kam die Meldung zwei Tage später wieder. Also habe ich mal ein verbose 5 gemacht und musste mich dann durch 17MB Logfiles quälen. Und dann stand da folgendes
2020.06.20 02:01:00 4: DbRep DbLogRep - -------- New selection ---------
2020.06.20 02:01:00 4: DbRep DbLogRep - Command: delEntries
2020.06.20 02:01:00 4: DbRep DbLogRep - timeOlderThan - year: , day: 50, hour: , min: , sec:
2020.06.20 02:01:00 4: DbRep DbLogRep - Year 2020 is leap year
2020.06.20 02:01:00 4: DbRep DbLogRep - startMonth: 3 endMonth: 4 lastleapyear:  baseYear: 2020 diffdaylight:1 isdaylight:1
2020.06.20 02:01:00 4: DbRep DbLogRep - FullDay option: 0
2020.06.20 02:01:00 5: DbRep DbLogRep - Timestamp begin epocheseconds: 1587270539
2020.06.20 02:01:00 4: DbRep DbLogRep - Timestamp begin human readable: 2020-04-19 06:28:59
2020.06.20 02:01:00 4: DbRep DbLogRep - Time difference to current time for calculating Timestamp end: 4320001 sec
2020.06.20 02:01:00 5: DbRep DbLogRep - Timestamp end epocheseconds: 1588291259
2020.06.20 02:01:00 4: DbRep DbLogRep - Timestamp end human readable: 2020-05-01 02:00:59
2020.06.20 02:01:00 5: DbRep DbLogRep - weekday start for selection: So  ->  wdadd: 86400
2020.06.20 02:01:00 3: DbRep DbLogRep - WARNING - old process 11924 will be killed now to start a new BlockingCall
2020.06.20 02:01:00 1: DbRep DbLogRep -> BlockingCall  pid: Timeout: process terminated
2020.06.20 02:01:00 2: DbRep DbLogRep - Database command aborted due to "Timeout: process terminated"
2020.06.20 02:01:00 4: DbRep DbLogRep - -------- New selection ---------
2020.06.20 02:01:00 4: DbRep DbLogRep - Command: delSeqDoublets delete
2020.06.20 02:01:00 4: DbRep DbLogRep - timeOlderThan - year: , day: 50, hour: , min: , sec:
2020.06.20 02:01:00 4: DbRep DbLogRep - Year 2020 is leap year
2020.06.20 02:01:00 4: DbRep DbLogRep - startMonth: 3 endMonth: 4 lastleapyear:  baseYear: 2020 diffdaylight:1 isdaylight:1
2020.06.20 02:01:00 4: DbRep DbLogRep - FullDay option: 0
2020.06.20 02:01:00 5: DbRep DbLogRep - Timestamp begin epocheseconds: 1587270539
2020.06.20 02:01:00 4: DbRep DbLogRep - Timestamp begin human readable: 2020-04-19 06:28:59
2020.06.20 02:01:00 4: DbRep DbLogRep - Time difference to current time for calculating Timestamp end: 4320001 sec
2020.06.20 02:01:00 5: DbRep DbLogRep - Timestamp end epocheseconds: 1588291259
2020.06.20 02:01:00 4: DbRep DbLogRep - Timestamp end human readable: 2020-05-01 02:00:59
2020.06.20 02:01:00 5: DbRep DbLogRep - weekday start for selection: So  ->  wdadd: 86400
2020.06.20 02:01:00 5: DbRep DbLogRep - Daylight savings changed: 0 (on Mon Apr 20 06:28:59 2020)
2020.06.20 02:01:00 5: DbRep DbLogRep - runtime_string: 2020-04-19, runtime_string_first: 2020-04-19 06:28:59, runtime_string_next: 2020-04-20
2020.06.20 02:01:00 5: DbRep DbLogRep - Daylight savings changed: 0 (on Tue Apr 21 06:28:59 2020)
2020.06.20 02:01:00 5: DbRep DbLogRep - runtime_string: 2020-04-20, runtime_string_first: 2020-04-20, runtime_string_next: 2020-04-21
2020.06.20 02:01:00 5: DbRep DbLogRep - Daylight savings changed: 0 (on Wed Apr 22 06:28:59 2020)
2020.06.20 02:01:00 5: DbRep DbLogRep - runtime_string: 2020-04-21, runtime_string_first: 2020-04-21, runtime_string_next: 2020-04-22
2020.06.20 02:01:00 5: DbRep DbLogRep - Daylight savings changed: 0 (on Thu Apr 23 06:28:59 2020)
2020.06.20 02:01:00 5: DbRep DbLogRep - runtime_string: 2020-04-22, runtime_string_first: 2020-04-22, runtime_string_next: 2020-04-23
2020.06.20 02:01:00 5: DbRep DbLogRep - Daylight savings changed: 0 (on Fri Apr 24 06:28:59 2020)
2020.06.20 02:01:00 5: DbRep DbLogRep - runtime_string: 2020-04-23, runtime_string_first: 2020-04-23, runtime_string_next: 2020-04-24
2020.06.20 02:01:00 5: DbRep DbLogRep - Daylight savings changed: 0 (on Sat Apr 25 06:28:59 2020)
2020.06.20 02:01:00 5: DbRep DbLogRep - runtime_string: 2020-04-24, runtime_string_first: 2020-04-24, runtime_string_next: 2020-04-25
2020.06.20 02:01:00 5: DbRep DbLogRep - Daylight savings changed: 0 (on Sun Apr 26 06:28:59 2020)
2020.06.20 02:01:00 5: DbRep DbLogRep - runtime_string: 2020-04-25, runtime_string_first: 2020-04-25, runtime_string_next: 2020-04-26
2020.06.20 02:01:00 5: DbRep DbLogRep - Daylight savings changed: 0 (on Mon Apr 27 06:28:59 2020)
2020.06.20 02:01:00 5: DbRep DbLogRep - runtime_string: 2020-04-26, runtime_string_first: 2020-04-26, runtime_string_next: 2020-04-27
2020.06.20 02:01:00 5: DbRep DbLogRep - Daylight savings changed: 0 (on Tue Apr 28 06:28:59 2020)
2020.06.20 02:01:00 5: DbRep DbLogRep - runtime_string: 2020-04-27, runtime_string_first: 2020-04-27, runtime_string_next: 2020-04-28
2020.06.20 02:01:00 5: DbRep DbLogRep - Daylight savings changed: 0 (on Wed Apr 29 06:28:59 2020)
2020.06.20 02:01:00 5: DbRep DbLogRep - runtime_string: 2020-04-28, runtime_string_first: 2020-04-28, runtime_string_next: 2020-04-29
2020.06.20 02:01:00 5: DbRep DbLogRep - Daylight savings changed: 0 (on Thu Apr 30 06:28:59 2020)
2020.06.20 02:01:00 5: DbRep DbLogRep - runtime_string: 2020-04-29, runtime_string_first: 2020-04-29, runtime_string_next: 2020-04-30
2020.06.20 02:01:00 5: DbRep DbLogRep - Daylight savings changed: 0 (on Fri May  1 06:28:59 2020)
2020.06.20 02:01:00 5: DbRep DbLogRep - runtime_string: 2020-04-30, runtime_string_first: 2020-04-30, runtime_string_next: 2020-05-01
2020.06.20 02:01:00 5: DbRep DbLogRep - Daylight savings changed: 0 (on Sat May  2 06:28:59 2020)
2020.06.20 02:01:00 5: DbRep DbLogRep - runtime_string: 2020-05-01, runtime_string_first: 2020-05-01, runtime_string_next: 2020-05-01 02:00:59
2020.06.20 02:01:00 4: DbRep DbLogRep - Aggregation: day
2020.06.20 02:01:00 4: DbRep DbLogRep - delSeqDoublets params -> positive variance: , negative variance: , EDGE: balanced
2020.06.20 02:01:00 5: DbRep DbLogRep - Timestamp-Array:
2020-04-19#2020-04-19 06:28:59#2020-04-20 2020-04-20#2020-04-20#2020-04-21 2020-04-21#2020-04-21#2020-04-22 2020-04-22#2020-04-22#2020-04-23 2020-04-23#2020-04-23#2020-04-24 2020-04-24#2020-04-24#2020-04-25 2020-04-25#2020-04-25#2020-04-26 2020-04-26#2020-04-26#2020-04-27 2020-04-27#2020-04-27#2020-04-28 2020-04-28#2020-04-28#2020-04-29 2020-04-29#2020-04-29#2020-04-30 2020-04-30#2020-04-30#2020-05-01 2020-05-01#2020-05-01#2020-05-01 02:00:59
2020.06.20 02:01:00 5: DbRep DbLogRep - Devices for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2020.06.20 02:01:00 5: DbRep DbLogRep - Readings for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2020.06.20 02:01:00 5: DbRep DbLogRep - Devices for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2020.06.20 02:01:00 5: DbRep DbLogRep - Readings for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2020.06.20 02:01:00 4: DbRep DbLogRep - SQL execute: SELECT DEVICE,READING,TIMESTAMP,VALUE FROM history where TIMESTAMP >= '2020-04-19 06:28:59' AND TIMESTAMP < '2020-04-20' ;
2020.06.20 02:01:02 5: DbRep DbLogRep - Devices for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2020.06.20 02:01:02 5: DbRep DbLogRep - Readings for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2020.06.20 02:01:02 4: DbRep DbLogRep - SQL execute: SELECT DEVICE,READING,TIMESTAMP,VALUE FROM history where TIMESTAMP >= '2020-04-20' AND TIMESTAMP < '2020-04-21' ;
2020.06.20 02:01:05 5: DbRep DbLogRep - Devices for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2020.06.20 02:01:05 5: DbRep DbLogRep - Readings for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2020.06.20 02:01:05 4: DbRep DbLogRep - SQL execute: SELECT DEVICE,READING,TIMESTAMP,VALUE FROM history where TIMESTAMP >= '2020-04-21' AND TIMESTAMP < '2020-04-22' ;
2020.06.20 02:01:07 5: DbRep DbLogRep - Devices for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2020.06.20 02:01:07 5: DbRep DbLogRep - Readings for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2020.06.20 02:01:07 4: DbRep DbLogRep - SQL execute: SELECT DEVICE,READING,TIMESTAMP,VALUE FROM history where TIMESTAMP >= '2020-04-22' AND TIMESTAMP < '2020-04-23' ;
2020.06.20 02:01:10 5: DbRep DbLogRep - Devices for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2020.06.20 02:01:10 5: DbRep DbLogRep - Readings for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2020.06.20 02:01:10 4: DbRep DbLogRep - SQL execute: SELECT DEVICE,READING,TIMESTAMP,VALUE FROM history where TIMESTAMP >= '2020-04-23' AND TIMESTAMP < '2020-04-24' ;
2020.06.20 02:01:12 5: DbRep DbLogRep - Devices for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2020.06.20 02:01:12 5: DbRep DbLogRep - Readings for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2020.06.20 02:01:12 4: DbRep DbLogRep - SQL execute: SELECT DEVICE,READING,TIMESTAMP,VALUE FROM history where TIMESTAMP >= '2020-04-24' AND TIMESTAMP < '2020-04-25' ;
2020.06.20 02:01:20 5: DbRep DbLogRep - Devices for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2020.06.20 02:01:20 5: DbRep DbLogRep - Readings for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2020.06.20 02:01:20 4: DbRep DbLogRep - SQL execute: SELECT DEVICE,READING,TIMESTAMP,VALUE FROM history where TIMESTAMP >= '2020-04-25' AND TIMESTAMP < '2020-04-26' ;
2020.06.20 02:01:23 5: DbRep DbLogRep - Devices for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2020.06.20 02:01:23 5: DbRep DbLogRep - Readings for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2020.06.20 02:01:23 4: DbRep DbLogRep - SQL execute: SELECT DEVICE,READING,TIMESTAMP,VALUE FROM history where TIMESTAMP >= '2020-04-26' AND TIMESTAMP < '2020-04-27' ;
2020.06.20 02:01:25 5: DbRep DbLogRep - Devices for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2020.06.20 02:01:25 5: DbRep DbLogRep - Readings for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2020.06.20 02:01:25 4: DbRep DbLogRep - SQL execute: SELECT DEVICE,READING,TIMESTAMP,VALUE FROM history where TIMESTAMP >= '2020-04-27' AND TIMESTAMP < '2020-04-28' ;
2020.06.20 02:01:28 5: DbRep DbLogRep - Devices for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2020.06.20 02:01:28 5: DbRep DbLogRep - Readings for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2020.06.20 02:01:28 4: DbRep DbLogRep - SQL execute: SELECT DEVICE,READING,TIMESTAMP,VALUE FROM history where TIMESTAMP >= '2020-04-28' AND TIMESTAMP < '2020-04-29' ;
2020.06.20 02:01:32 5: DbRep DbLogRep - Devices for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2020.06.20 02:01:32 5: DbRep DbLogRep - Readings for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2020.06.20 02:01:32 4: DbRep DbLogRep - SQL execute: SELECT DEVICE,READING,TIMESTAMP,VALUE FROM history where TIMESTAMP >= '2020-04-29' AND TIMESTAMP < '2020-04-30' ;
2020.06.20 02:01:34 5: DbRep DbLogRep - Devices for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2020.06.20 02:01:34 5: DbRep DbLogRep - Readings for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2020.06.20 02:01:34 4: DbRep DbLogRep - SQL execute: SELECT DEVICE,READING,TIMESTAMP,VALUE FROM history where TIMESTAMP >= '2020-04-30' AND TIMESTAMP < '2020-05-01' ;
2020.06.20 02:03:00 4: DbRep DbLogRep - SQL execute: delete FROM history where TIMESTAMP = '2020-04-30 01:36:34' AND DEVICE = 'Badezimmerfenster' AND READING = 'rssi_at_WLAN_HmUART' AND VALUE = '-57';
2020.06.20 02:03:00 4: DbRep DbLogRep - SQL execute: delete FROM history where TIMESTAMP = '2020-04-30 17:40:36' AND DEVICE = 'Badezimmerfenster' AND READING = 'rssi_at_WLAN_HmUART' AND VALUE = '-56';
2020.06.20 02:03:00 4: DbRep DbLogRep - SQL execute: delete FROM history where TIMESTAMP = '2020-04-30 22:24:39' AND DEVICE = 'Badezimmerfenster' AND READING = 'rssi_at_WLAN_HmUART' AND VALUE = '-57';[
... ca 95.000 Zeilen...
2020.06.20 02:23:42 4: DbRep DbLogRep - SQL execute: delete FROM history where TIMESTAMP = '2020-05-01 00:50:36' AND DEVICE = 'Windmesser' AND READING = 'temperature' AND VALUE = '13.0';
2020.06.20 02:23:42 4: DbRep DbLogRep - SQL execute: delete FROM history where TIMESTAMP = '2020-05-01 01:06:06' AND DEVICE = 'Windmesser' AND READING = 'temperature' AND VALUE = '13.1';
2020.06.20 02:23:42 3: DbRep DbLogRep - rows deleted by "delSeqDoublets delete": 98232
2020.06.20 02:23:42 5: DbRep DbLogRep - row result list:

2020.06.20 07:51:19 1: RMDIR: ./restoreDir/save/2020-06-17

Ich lösche ja alte Daten. Dabei fallen mir zwei Dinge auf. Anscheinend werden alte Daten nicht durch einen SQL-Befehl gelöscht, sondern durch soviel Befehle, wie es Einträge gibt? Und dann gibt es zu Beginn des Logfiles eine Anmerkung, dass er den alten Löschprozess killt, weil der noch aktiv ist.

Ich glaube, ich brauche da mal Hilfe, wie man das verbessert.

<edit>Ein direkter Eingriff ergab
DELETE FROM history where TIMESTAMP < '2020-05-01'
Abfrage ausgeführt, 321 832 Datensätze betroffen. (794.444 s)
FHEM 6.1 auf RaspPi3 (Raspbian:  6.1.21-v8+; Perl: v5.32.1)
SIGNALduino (433 MHz) und HM-UART (868 MHz), Sonoff, Blitzwolf, Somfy RTS, CAME-Gartentor, Volkszähler, Keyence-Sensor, Homematic-Sensoren und -thermostat, Ferraris-Zähler für Wasseruhr, Openlink-Nachbau Viessmann

DS_Starter

Guten Morgen,

anhand deiner logs erkennt man, dass du das gleiche DbRep für unterschiedliche Aufgaben benutzt (delEntries und delSeqDoublets). Das kann man machen, keine Frage. Aber die dürfen sich zeitlich nicht überschneiden, sonst schießt der neue Prozess den vorher noch laufenden / gestarteten Prozess ab was sich auch durch den timeout-Fehler äußert.
Deswegen ist es besser für wiederkehrende Aufgaben separate Devices zu definieren. Die Zeiten können sich dann auch überschneiden.

Zitat
Dabei fallen mir zwei Dinge auf. Anscheinend werden alte Daten nicht durch einen SQL-Befehl gelöscht, sondern durch soviel Befehle, wie es Einträge gibt?
Nein, das täuscht. Die Daten werden mit einem Befehl gelöscht.

Was dich vielleicht wundert, sind diese Einträge bei delSeqDoublets delete


2020.06.20 02:01:00 5: DbRep DbLogRep - runtime_string: 2020-04-19, runtime_string_first: 2020-04-19 06:28:59, runtime_string_next: 2020-04-20
2020.06.20 02:01:00 5: DbRep DbLogRep - Daylight savings changed: 0 (on Tue Apr 21 06:28:59 2020)
2020.06.20 02:01:00 5: DbRep DbLogRep - runtime_string: 2020-04-20, runtime_string_first: 2020-04-20, runtime_string_next: 2020-04-21
2020.06.20 02:01:00 5: DbRep DbLogRep - Daylight savings changed: 0 (on Wed Apr 22 06:28:59 2020)
2020.06.20 02:01:00 5: DbRep DbLogRep - runtime_string: 2020-04-21, runtime_string_first: 2020-04-21, runtime_string_next: 2020-04-22
2020.06.20 02:01:00 5: DbRep DbLogRep - Daylight savings changed: 0 (on Thu Apr 23 06:28:59 2020)
2020.06.20 02:01:00 5: DbRep DbLogRep - runtime_string: 2020-04-22, runtime_string_first: 2020-04-22, runtime_string_next: 2020-04-23
2020.06.20 02:01:00 5: DbRep DbLogRep - Daylight savings changed: 0 (on Fri Apr 24 06:28:59 2020)
2020.06.20 02:01:00 5: DbRep DbLogRep - runtime_string: 2020-04-23, runtime_string_first: 2020-04-23, runtime_string_next: 2020-04-24
2020.06.20 02:01:00 5: DbRep DbLogRep - Daylight savings changed: 0 (on Sat Apr 25 06:28:59 2020)
2020.06.20 02:01:00 5: DbRep DbLogRep - runtime_string: 2020-04-24, runtime_string_first: 2020-04-24, runtime_string_next: 2020-04-25
2020.06.20 02:01:00 5: DbRep DbLogRep - Daylight savings changed: 0 (on Sun Apr 26 06:28:59 2020)
2020.06.20 02:01:00 5: DbRep DbLogRep - runtime_string: 2020-04-25, runtime_string_first: 2020-04-25, runtime_string_next: 2020-04-26
2020.06.20 02:01:00 5: DbRep DbLogRep - Daylight savings changed: 0 (on Mon Apr 27 06:28:59 2020)
2020.06.20 02:01:00 5: DbRep DbLogRep - runtime_string: 2020-04-26, runtime_string_first: 2020-04-26, runtime_string_next: 2020-04-27
2020.06.20 02:01:00 5: DbRep DbLogRep - Daylight savings changed: 0 (on Tue Apr 28 06:28:59 2020)
2020.06.20 02:01:00 5: DbRep DbLogRep - runtime_string: 2020-04-27, runtime_string_first: 2020-04-27, runtime_string_next: 2020-04-28
2020.06.20 02:01:00 5: DbRep DbLogRep - Daylight savings changed: 0 (on Wed Apr 29 06:28:59 2020)
2020.06.20 02:01:00 5: DbRep DbLogRep - runtime_string: 2020-04-28, runtime_string_first: 2020-04-28, runtime_string_next: 2020-04-29
2020.06.20 02:01:00 5: DbRep DbLogRep - Daylight savings changed: 0 (on Thu Apr 30 06:28:59 2020)
2020.06.20 02:01:00 5: DbRep DbLogRep - runtime_string: 2020-04-29, runtime_string_first: 2020-04-29, runtime_string_next: 2020-04-30
2020.06.20 02:01:00 5: DbRep DbLogRep - Daylight savings changed: 0 (on Fri May  1 06:28:59 2020)
2020.06.20 02:01:00 5: DbRep DbLogRep - runtime_string: 2020-04-30, runtime_string_first: 2020-04-30, runtime_string_next: 2020-05-01
2020.06.20 02:01:00 5: DbRep DbLogRep - Daylight savings changed: 0 (on Sat May  2 06:28:59 2020)
2020.06.20 02:01:00 5: DbRep DbLogRep - runtime_string: 2020-05-01, runtime_string_first: 2020-05-01, runtime_string_next: 2020-05-01 02:00:59


Das sind die Zeitscheiben für die Doublettenbehandlung entsprechend der Einstellung Aggregation: day.

Um dein Problem zu lösen solltest du also dein DbRep kopieren und eins für delEntries und das andere für delSeqDoublets benutzen oder die Startzeiten voneinander abändern.

LG,
Heiko
ESXi@NUC+Debian+MariaDB, PV: SMA, Victron MPII+Pylontech+CerboGX
Maintainer: SSCam, SSChatBot, SSCal, SSFile, DbLog/DbRep, Log2Syslog, SolarForecast,Watches, Dashboard, PylonLowVoltage
Kaffeekasse: https://www.paypal.me/HMaaz
Contrib: https://svn.fhem.de/trac/browser/trunk/fhem/contrib/DS_Starter

DS_Starter

#5
Ach jetzt sehe ich erst dass du dich vermutlich hierauf beziehst:


2020.06.20 02:01:34 4: DbRep DbLogRep - SQL execute: SELECT DEVICE,READING,TIMESTAMP,VALUE FROM history where TIMESTAMP >= '2020-04-30' AND TIMESTAMP < '2020-05-01' ;
2020.06.20 02:03:00 4: DbRep DbLogRep - SQL execute: delete FROM history where TIMESTAMP = '2020-04-30 01:36:34' AND DEVICE = 'Badezimmerfenster' AND READING = 'rssi_at_WLAN_HmUART' AND VALUE = '-57';
2020.06.20 02:03:00 4: DbRep DbLogRep - SQL execute: delete FROM history where TIMESTAMP = '2020-04-30 17:40:36' AND DEVICE = 'Badezimmerfenster' AND READING = 'rssi_at_WLAN_HmUART' AND VALUE = '-56';
2020.06.20 02:03:00 4: DbRep DbLogRep - SQL execute: delete FROM history where TIMESTAMP = '2020-04-30 22:24:39' AND DEVICE = 'Badezimmerfenster' AND READING = 'rssi_at_WLAN_HmUART' AND VALUE = '-57';[
... ca 95.000 Zeilen...
2020.06.20 02:23:42 4: DbRep DbLogRep - SQL execute: delete FROM history where TIMESTAMP = '2020-05-01 00:50:36' AND DEVICE = 'Windmesser' AND READING = 'temperature' AND VALUE = '13.0';
2020.06.20 02:23:42 4: DbRep DbLogRep - SQL execute: delete FROM history where TIMESTAMP = '2020-05-01 01:06:06' AND DEVICE = 'Windmesser' AND READING = 'temperature' AND VALUE = '13.1';
2020.06.20 02:23:42 3: DbRep DbLogRep - rows deleted by "delSeqDoublets delete": 98232


Ja, das sind Löschungen die aus delSeqDoublets delete herrühren. Hier wird nur der jeweilige Datensatz ermittelt und selektiv gelöscht.
ESXi@NUC+Debian+MariaDB, PV: SMA, Victron MPII+Pylontech+CerboGX
Maintainer: SSCam, SSChatBot, SSCal, SSFile, DbLog/DbRep, Log2Syslog, SolarForecast,Watches, Dashboard, PylonLowVoltage
Kaffeekasse: https://www.paypal.me/HMaaz
Contrib: https://svn.fhem.de/trac/browser/trunk/fhem/contrib/DS_Starter

andies

Alles klar, danke. Dann werde ich zwei devices einrichten und das mal beobachten. Mir war die Funktionsweise nicht klar.
FHEM 6.1 auf RaspPi3 (Raspbian:  6.1.21-v8+; Perl: v5.32.1)
SIGNALduino (433 MHz) und HM-UART (868 MHz), Sonoff, Blitzwolf, Somfy RTS, CAME-Gartentor, Volkszähler, Keyence-Sensor, Homematic-Sensoren und -thermostat, Ferraris-Zähler für Wasseruhr, Openlink-Nachbau Viessmann

andies

Heute morgen gab es ein Problem, aber das könnte mit YAAHM zusammenhängen
2020.06.21 02:01:00 1: ERROR evaluating {YAAHM_time('Profil','aftermidnight',1)}: Month '-1' out of range 0..11 at ./FHEM/93_DbRep.pm line 2046.

2020.06.21 06:00:00 1: [YAAHM_today] on device Profil called for this day

Der zweite Befehl kommt normalerweise um 00:01.
FHEM 6.1 auf RaspPi3 (Raspbian:  6.1.21-v8+; Perl: v5.32.1)
SIGNALduino (433 MHz) und HM-UART (868 MHz), Sonoff, Blitzwolf, Somfy RTS, CAME-Gartentor, Volkszähler, Keyence-Sensor, Homematic-Sensoren und -thermostat, Ferraris-Zähler für Wasseruhr, Openlink-Nachbau Viessmann

DS_Starter

Ja, da ist die Angabe für den Monat >Month '-1'< nicht im gültigen Bereich 0..11.
Hat etwas mit der Angabe in DbRep -> timestamp_begin etc. zu tun.
ESXi@NUC+Debian+MariaDB, PV: SMA, Victron MPII+Pylontech+CerboGX
Maintainer: SSCam, SSChatBot, SSCal, SSFile, DbLog/DbRep, Log2Syslog, SolarForecast,Watches, Dashboard, PylonLowVoltage
Kaffeekasse: https://www.paypal.me/HMaaz
Contrib: https://svn.fhem.de/trac/browser/trunk/fhem/contrib/DS_Starter

andies

#9
Merkwürdig, der Fehler ist immer noch da
2020.06.22 00:00:34 1:  [YAAHM_updater] on device Profil called for this day
2020.06.22 02:01:01 1:  ERROR evaluating {YAAHM_time('Profil','aftermidnight',1)}: Month '-1' out of range 0..11 at ./FHEM/93_DbRep.pm line 2046.

Hmm, wo suche ich am besten weiter?

<edit>Ich habe YAAHM neu initialisiert und danach war der Fehler weg. Ist also ein YAAHM-Problem.
FHEM 6.1 auf RaspPi3 (Raspbian:  6.1.21-v8+; Perl: v5.32.1)
SIGNALduino (433 MHz) und HM-UART (868 MHz), Sonoff, Blitzwolf, Somfy RTS, CAME-Gartentor, Volkszähler, Keyence-Sensor, Homematic-Sensoren und -thermostat, Ferraris-Zähler für Wasseruhr, Openlink-Nachbau Viessmann

andies

ich muss hier nochmal nachhaken, obwohl das anscheinend ein neues Problem ist. Denn die Fehlermeldung
2020.06.25 06:43:30 1: ERROR evaluating {YAAHM_time('Profil','aftermidnight',1)}: Month '-1' out of range 0..11 at ./FHEM/93_DbRep.pm line 2046.
tauchte wieder auf. Was mich wundert: DbLogRep greift bei mir gar nicht auf YAAHM zu?! Es müsste in dem Reading korrekterweise s_aftermidnight heissen.
FHEM 6.1 auf RaspPi3 (Raspbian:  6.1.21-v8+; Perl: v5.32.1)
SIGNALduino (433 MHz) und HM-UART (868 MHz), Sonoff, Blitzwolf, Somfy RTS, CAME-Gartentor, Volkszähler, Keyence-Sensor, Homematic-Sensoren und -thermostat, Ferraris-Zähler für Wasseruhr, Openlink-Nachbau Viessmann

DS_Starter

In der Zeile 2046 steht lediglich:


my $nthants = timelocal($sec1, $min1, $hh1, $dd1, $mm1-1, $yyyy1-1900);


Mehr nicht. Die Zeitvariablen kommen aus den time* Attributen welche du verwendest.
DbRep greift auch nicht auf andere YAHHM zu. Ich kenne das Modul auch nur dem Namen nach, weiß dass es das gibt, nutze es aber nicht.
Setzt du über das YAHHM vielleicht die time* Attribute im DBRep ?
Vllt. hilft ein stacktrace ?
ESXi@NUC+Debian+MariaDB, PV: SMA, Victron MPII+Pylontech+CerboGX
Maintainer: SSCam, SSChatBot, SSCal, SSFile, DbLog/DbRep, Log2Syslog, SolarForecast,Watches, Dashboard, PylonLowVoltage
Kaffeekasse: https://www.paypal.me/HMaaz
Contrib: https://svn.fhem.de/trac/browser/trunk/fhem/contrib/DS_Starter

andies

war doch mein Fehler. Ich hatte im Profil die beiden devices verwechselt und dann den Selektionsbereich falsch ausgewiesen. Hat sich also erledigt.
FHEM 6.1 auf RaspPi3 (Raspbian:  6.1.21-v8+; Perl: v5.32.1)
SIGNALduino (433 MHz) und HM-UART (868 MHz), Sonoff, Blitzwolf, Somfy RTS, CAME-Gartentor, Volkszähler, Keyence-Sensor, Homematic-Sensoren und -thermostat, Ferraris-Zähler für Wasseruhr, Openlink-Nachbau Viessmann