angeblich keine Daten geloggt

Begonnen von rabehd, 08 Januar 2024, 15:43:51

Vorheriges Thema - Nächstes Thema

rabehd

Ich habe mein System neu aufgesetzt.
In der logdb werden per count 68 Tausend Datensätze in histroy gezählt.
DbRep zählt 0.
Was könnte da falsch sein?

Internals:
   COLUMNS    field length used for Device: 64, Type: 64, Event: 512, Reading: 64, Value: 128, Unit: 32
   CONFIGURATION ./db.conf
   DEF        ./db.conf .*:.*
   FD         5
   FUUID      65943443-f33f-44e9-29a5-6e334aa1924cc8aa
   FVERSION   93_DbLog.pm:v5.9.5-s28345/2024-01-05
   MODE       asynchronous
   MODEL      MYSQL
   NAME       logdb
   NR         2
   NTFY_ORDER 50-logdb
   PID        905
   REGEXP     .*:.*
   SBP_PID    950
   SBP_STATE  running
   STATE      connected
   TYPE       DbLog
   UTF8       1
   dbconn     mysql:database=fhem;host=localhost;port=3306
   dbuser     fhemuser
   eventCount 2226
   HELPER:
     COLSET     1
     DEVICECOL  64
     EVENTCOL   512
     OLDSTATE   connected
     PACKAGE    main
     READINGCOL 64
     TC         current
     TH         history
     TYPECOL    64
     UNITCOL    32
     VALUECOL   128
     VERSION    5.9.5
   OLDREADINGS:
   READINGS:
     2024-01-08 15:41:11   CacheOverflowLastNum 0
     2024-01-03 11:58:07   CacheOverflowLastState normal
     2024-01-08 15:41:11   CacheUsage      1
     2024-01-08 15:41:11   NextSync        2024-01-08 15:41:41 or when CacheUsage 500 is reached
     2024-01-08 15:38:01   countCurrent    0
     2024-01-08 15:38:01   countHistory    68880
     2024-01-08 15:41:11   state           connected
Attributes:
   DbLogSelectionMode Include
   DbLogType  SampleFill/History
   asyncMode  1
   insertMode 1
   room       Maschinenraum->Logs
   verbose    2

Internals:
   DATABASE   fhem
   DEF        logdb
   FUUID      659563ce-f33f-44e9-f532-554f333c01a60c4d
   FVERSION   93_DbRep.pm:v8.52.15-s28267/2023-12-08
   LASTCMD    countEntries history
   MODEL      Client
   NAME       logdb.Serv
   NOTIFYDEV  global,logdb.Serv
   NR         214
   NTFY_ORDER 50-logdb.Serv
   ROLE       Client
   STATE      done
   TYPE       DbRep
   UTF8       1
   eventCount 27
   HELPER:
     DBLOGDEVICE logdb
     GRANTS     INSERT,SELECT,UPDATE,USAGE,DELETE
     IDRETRIES  2
     MINTS      2024-01-02 17:20:01
     PACKAGE    main
     VERSION    8.52.15
     CV:
       aggregation no
       aggsec     1
       destr      2024-01-01
       dsstr      2023-07-12
       epoch_seconds_end 1704149999
       mestr      01
       msstr      07
       testr      23:59:59
       tsstr      00:00:00
       wdadd     
       yestr      2024
       ysstr      2023
     DBREPCOL:
       COLSET     1
       DEVICE     64
       EVENT      512
       READING    64
       TYPE       64
       UNIT       32
       VALUE      128
     REDUCELOG:
       logdb.Serv
       reduceLog
       7:180
       average=day
       INCLUDE=%:humidity
   OLDREADINGS:
   READINGS:
     2024-01-08 15:38:40   2023-07-12__ALLREADINGS__COUNT_history__no_aggregation -
     2024-01-08 15:38:40   state           done
Attributes:
   allowDeletion 1
   countEntriesDetail 1
Auch funktionierende Lösungen kann man hinterfragen.

DS_Starter

DbRep hat als ältesten Datensatz den

  MINTS      2024-01-02 17:20:01


ermittelt.
Das wird üblicherweise bei einem Neustart automatisch ausgeführt.
Führe mal ein "get ... minTimestamp" aus und poste das Ergbnis bzw. danach ein countEntries.
Proxmox+Debian+MariaDB, PV: SMA, Victron MPII+Pylontech+CerboGX
Maintainer: SSCam, SSChatBot, SSCal, SSFile, DbLog/DbRep, Log2Syslog, SolarForecast,Watches, Dashboard, PylonLowVoltage
Kaffeekasse: https://www.paypal.me/HMaaz
Contrib: https://svn.fhem.de/trac/browser/trunk/fhem/contrib/DS_Starter

rabehd

Ich sehe durch "get ... minTimestamp" keine Veränderung.

Internals:
   DATABASE   fhem
   DEF        logdb
   FUUID      659563ce-f33f-44e9-f532-554f333c01a60c4d
   FVERSION   93_DbRep.pm:v8.52.15-s28267/2023-12-08
   LASTCMD    minTimestamp
   MODEL      Client
   NAME       logdb.Serv
   NOTIFYDEV  global,logdb.Serv
   NR         214
   NTFY_ORDER 50-logdb.Serv
   ROLE       Client
   STATE      done
   TYPE       DbRep
   UTF8       1
   eventCount 30
   HELPER:
     DBLOGDEVICE logdb
     GRANTS     INSERT,UPDATE,SELECT,USAGE,DELETE
     IDRETRIES  3
     MINTS      2024-01-02 17:20:01
     PACKAGE    main
     VERSION    8.52.15
     CV:
       aggregation no
       aggsec     1
       destr      2024-01-01
       dsstr      2023-07-12
       epoch_seconds_end 1704149999
       mestr      01
       msstr      07
       testr      23:59:59
       tsstr      00:00:00
       wdadd     
       yestr      2024
       ysstr      2023
     DBREPCOL:
       COLSET     1
       DEVICE     64
       EVENT      512
       READING    64
       TYPE       64
       UNIT       32
       VALUE      128
     REDUCELOG:
       logdb.Serv
       reduceLog
       7:180
       average=day
       INCLUDE=%:humidity
   OLDREADINGS:
   READINGS:
     2024-01-08 16:48:16   state           done
     2024-01-08 16:48:16   timestamp_oldest_dataset 2024-01-02 17:20:01
Attributes:
   allowDeletion 1
   countEntriesDetail 1
   room       Maschinenraum->Logs

auch nicht nach countEntries.
Internals:
   DATABASE   fhem
   DEF        logdb
   FUUID      659563ce-f33f-44e9-f532-554f333c01a60c4d
   FVERSION   93_DbRep.pm:v8.52.15-s28267/2023-12-08
   LASTCMD    countEntries history
   MODEL      Client
   NAME       logdb.Serv
   NOTIFYDEV  global,logdb.Serv
   NR         214
   NTFY_ORDER 50-logdb.Serv
   ROLE       Client
   STATE      done
   TYPE       DbRep
   UTF8       1
   eventCount 33
   HELPER:
     DBLOGDEVICE logdb
     GRANTS     INSERT,UPDATE,SELECT,USAGE,DELETE
     IDRETRIES  3
     MINTS      2024-01-02 17:20:01
     PACKAGE    main
     VERSION    8.52.15
     CV:
       aggregation no
       aggsec     1
       destr      2024-01-01
       dsstr      2023-07-12
       epoch_seconds_end 1704149999
       mestr      01
       msstr      07
       testr      23:59:59
       tsstr      00:00:00
       wdadd     
       yestr      2024
       ysstr      2023
     DBREPCOL:
       COLSET     1
       DEVICE     64
       EVENT      512
       READING    64
       TYPE       64
       UNIT       32
       VALUE      128
     REDUCELOG:
       logdb.Serv
       reduceLog
       7:180
       average=day
       INCLUDE=%:humidity
   OLDREADINGS:
   READINGS:
     2024-01-08 16:49:26   2023-07-12__ALLREADINGS__COUNT_history__no_aggregation -
     2024-01-08 16:49:26   state           done
Attributes:
   allowDeletion 1
   countEntriesDetail 1
   room       Maschinenraum->Logs

Auch "tableCurrentFillup" läst die current-Tabelle leer.
Auch funktionierende Lösungen kann man hinterfragen.

DS_Starter

#3
Ja, es wird tatsächlich

   READINGS:
     2024-01-08 16:48:16   state           done
     2024-01-08 16:48:16   timestamp_oldest_dataset 2024-01-02 17:20:01

ermittelt.
Führe mal bitte das SQL

   SELECT min(TIMESTAMP) FROM history;

im sqlCmd aus bzw. in einem SQL-Editor deiner Wahl völlig unabhängig von FHEM. Natürlich für die DB logdb.

EDIT: Kann das Datum 2024-01-02 17:20:01 als ältesten Datensatz stimmen?
Proxmox+Debian+MariaDB, PV: SMA, Victron MPII+Pylontech+CerboGX
Maintainer: SSCam, SSChatBot, SSCal, SSFile, DbLog/DbRep, Log2Syslog, SolarForecast,Watches, Dashboard, PylonLowVoltage
Kaffeekasse: https://www.paypal.me/HMaaz
Contrib: https://svn.fhem.de/trac/browser/trunk/fhem/contrib/DS_Starter

rabehd

MariaDB [(none)]> use fhem
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
MariaDB [fhem]> SELECT min(TIMESTAMP) FROM history;
+---------------------+
| min(TIMESTAMP)      |
+---------------------+
| 2024-01-02 17:20:01 |
+---------------------+
1 row in set (0.032 sec)

das Datum passt zum Neuaufsatz des Raspberry und der MariaDB.
Auch funktionierende Lösungen kann man hinterfragen.

DS_Starter

Ok. Dann setze bitte verbose 4 im Device und führe coutEntries aus.
Dann schauen wir mal den Logauszug an.
Proxmox+Debian+MariaDB, PV: SMA, Victron MPII+Pylontech+CerboGX
Maintainer: SSCam, SSChatBot, SSCal, SSFile, DbLog/DbRep, Log2Syslog, SolarForecast,Watches, Dashboard, PylonLowVoltage
Kaffeekasse: https://www.paypal.me/HMaaz
Contrib: https://svn.fhem.de/trac/browser/trunk/fhem/contrib/DS_Starter

rabehd

2024.01.08 20:25:32 4: DbRep logdb.Serv - -------- New selection ---------
2024.01.08 20:25:32 4: DbRep logdb.Serv - Command: countEntries history
2024.01.08 20:25:32 4: DbRep logdb.Serv - timeDiffToNow - year: , day: 180, hour: , min: , sec:
2024.01.08 20:25:32 4: DbRep logdb.Serv - Year 2024 is leap year
2024.01.08 20:25:32 4: DbRep logdb.Serv - startMonth: 6 endMonth: 0 lastleapyear: 2024 baseYear: 2023 diffdaylight:1 isdaylight:0
2024.01.08 20:25:32 4: DbRep logdb.Serv - timeOlderThan - year: 0, day: 7, hour: 0, min: 0, sec: 0
2024.01.08 20:25:32 4: DbRep logdb.Serv - Year 2024 is leap year
2024.01.08 20:25:32 4: DbRep logdb.Serv - startMonth: 0 endMonth: 0 lastleapyear: 2024 baseYear: 2024 diffdaylight:0 isdaylight:0
2024.01.08 20:25:32 4: DbRep logdb.Serv - FullDay option: 1
2024.01.08 20:25:32 4: DbRep logdb.Serv - Time difference to current time for calculating Timestamp begin: 15555601 sec
2024.01.08 20:25:32 4: DbRep logdb.Serv - Timestamp begin human readable: 2023-07-12 00:00:00
2024.01.08 20:25:32 4: DbRep logdb.Serv - Timestamp end human readable: 2024-01-01 23:59:59
2024.01.08 20:25:32 4: DbRep logdb.Serv - Aggregation: no
2024.01.08 20:25:32 4: DbRep logdb.Serv - Database connect - user: fhemuser, UTF-8 option set: yes
2024.01.08 20:25:32 4: DbRep logdb.Serv - SQL execute: SHOW VARIABLES LIKE 'collation_database'
2024.01.08 20:25:32 4: DbRep logdb.Serv - Database Character set is >utf8mb4_bin<
2024.01.08 20:25:32 4: DbRep logdb.Serv - simple do statement: set names "utf8mb4" collate "utf8mb4_bin"
2024.01.08 20:25:32 4: DbRep logdb.Serv - SQL execute: SELECT READING, COUNT(*) FROM history where TIMESTAMP >= '2023-07-12 00:00:00' AND TIMESTAMP <= '2024-01-01 23:59:59' group by READING;

Hat er noch Reste vom nächtlichen "(set logdb.Serv reduceLog 7:180 average=day INCLUDE=%:humidity)" drin.
Auf den alten System ging die Kombi.
Auch funktionierende Lösungen kann man hinterfragen.

DS_Starter

ZitatHat er noch Reste vom nächtlichen "(set logdb.Serv reduceLog 7:180 average=day INCLUDE=%:humidity)" drin.
Ja. Das ist natürlich nicht ok.
Ich suche die Ursache fixe es sobald ich sie finde.
Ein Neustart sollte in diesem Fall zunächst helfen.
Proxmox+Debian+MariaDB, PV: SMA, Victron MPII+Pylontech+CerboGX
Maintainer: SSCam, SSChatBot, SSCal, SSFile, DbLog/DbRep, Log2Syslog, SolarForecast,Watches, Dashboard, PylonLowVoltage
Kaffeekasse: https://www.paypal.me/HMaaz
Contrib: https://svn.fhem.de/trac/browser/trunk/fhem/contrib/DS_Starter

DS_Starter

In meinem contrib (Fußtext) liegt eine DbRep Version die einen Fix für dieses Problem enthält.
Du kannst die V gern testen (Restart nach download nicht vergessen). Ansonsten wird der Fix mit dem nächsten Repo-CheckIn von DbRep im normalen Update ausgerollt.
Proxmox+Debian+MariaDB, PV: SMA, Victron MPII+Pylontech+CerboGX
Maintainer: SSCam, SSChatBot, SSCal, SSFile, DbLog/DbRep, Log2Syslog, SolarForecast,Watches, Dashboard, PylonLowVoltage
Kaffeekasse: https://www.paypal.me/HMaaz
Contrib: https://svn.fhem.de/trac/browser/trunk/fhem/contrib/DS_Starter

rabehd

Ein Neustart hat erstmal geholfen.

DANKE

Ich nehme aber auch noch den Fix und schaue morgen mal.
Auch funktionierende Lösungen kann man hinterfragen.

rabehd

schlechte Nachrichten.
Nach einem Neustart geht es.
Wenn ich jetzt mein Löschselekt ausführe, dann wird danach wieder 0 gezählt.
 
Zitat(set logdb.Serv reduceLog 7:180 max=day INCLUDE=%:ENERGY_Power)

2024.01.08 21:55:10 3: DB ENERGY_To gelöscht
2024.01.08 21:56:03 4: DbRep logdb.Serv - -------- New selection ---------
2024.01.08 21:56:03 4: DbRep logdb.Serv - Command: countEntries history
2024.01.08 21:56:03 4: DbRep logdb.Serv - Timestamp begin human readable: not set
2024.01.08 21:56:03 4: DbRep logdb.Serv - Timestamp end human readable: not set
2024.01.08 21:56:03 4: DbRep logdb.Serv - Aggregation: no
2024.01.08 21:56:03 5: DbRep logdb.Serv - BlockingCall with PID "15674" started
2024.01.08 21:56:03 4: DbRep logdb.Serv - Database connect - user: fhemuser, UTF-8 option set: yes
2024.01.08 21:56:03 4: DbRep logdb.Serv - SQL execute: SHOW VARIABLES LIKE 'collation_database'
2024.01.08 21:56:03 4: DbRep logdb.Serv - Database Character set is >utf8mb4_bin<
2024.01.08 21:56:03 4: DbRep logdb.Serv - simple do statement: set names "utf8mb4" collate "utf8mb4_bin"
2024.01.08 21:56:03 5: DbRep logdb.Serv - IsTimeSet: 0, IsAggrSet: 0
2024.01.08 21:56:03 5: DbRep logdb.Serv - Timestamp-Array:
no_aggregation
2024.01.08 21:56:03 5: DbRep logdb.Serv - Devices for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2024.01.08 21:56:03 5: DbRep logdb.Serv - Readings for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2024.01.08 21:56:03 4: DbRep logdb.Serv - SQL execute: SELECT READING, COUNT(*) FROM history where 1 group by READING;
2024.01.08 21:56:03 5: DbRep logdb.Serv - SQL result: Aussenlichtstaerke 916
2024.01.08 21:56:03 5: DbRep logdb.Serv - SQL result: Aussenluftfeuchte 916
2024.01.08 21:56:03 5: DbRep logdb.Serv - SQL result: Aussentaupunkt 916
2024.01.08 21:56:03 5: DbRep logdb.Serv - SQL result: Aussentemperatur 916
2024.01.08 21:56:03 5: DbRep logdb.Serv - SQL result: ENERGY_Power 7514
2024.01.08 21:56:03 5: DbRep logdb.Serv - SQL result: ENERGY_Today 9664
2024.01.08 21:56:03 5: DbRep logdb.Serv - SQL result: ENERGY_Total 8642
2024.01.08 21:56:03 5: DbRep logdb.Serv - SQL result: ENERGY_Yesterday 47
2024.01.08 21:56:03 5: DbRep logdb.Serv - SQL result: Ertrag_gesamt 68
2024.01.08 21:56:03 5: DbRep logdb.Serv - SQL result: Ertrag_gestern 12
2024.01.08 21:56:03 5: DbRep logdb.Serv - SQL result: Ertrag_heute 74
2024.01.08 21:56:03 5: DbRep logdb.Serv - SQL result: Kosten_gesamt 6510
2024.01.08 21:56:03 5: DbRep logdb.Serv - SQL result: Kosten_gestern 5598
2024.01.08 21:56:03 5: DbRep logdb.Serv - SQL result: Kosten_heute 5957
2024.01.08 21:56:03 5: DbRep logdb.Serv - SQL result: Kosten_proJahr 35
2024.01.08 21:56:03 5: DbRep logdb.Serv - SQL result: Kosten_proMonat 4
2024.01.08 21:56:03 5: DbRep logdb.Serv - SQL result: Kosten_proTag 1
2024.01.08 21:56:03 5: DbRep logdb.Serv - SQL result: Kosten_proWoche 2
2024.01.08 21:56:03 5: DbRep logdb.Serv - SQL result: Luftdruck 916
2024.01.08 21:56:03 5: DbRep logdb.Serv - SQL result: Regenmenge 2055
2024.01.08 21:56:03 5: DbRep logdb.Serv - SQL result: RegenmengeYesterday 4
2024.01.08 21:56:03 5: DbRep logdb.Serv - SQL result: SunAlt 2901
2024.01.08 21:56:03 5: DbRep logdb.Serv - SQL result: SunAz 3110
2024.01.08 21:56:03 5: DbRep logdb.Serv - SQL result: VorhersageHeute 30
2024.01.08 21:56:03 5: DbRep logdb.Serv - SQL result: VorhersageMorgen 30
2024.01.08 21:56:03 5: DbRep logdb.Serv - SQL result: h2o_temperature 28
2024.01.08 21:56:03 5: DbRep logdb.Serv - SQL result: power 1
2024.01.08 21:56:03 5: DbRep logdb.Serv - SQL result: state 5496
2024.01.08 21:56:03 5: DbRep logdb.Serv - SQL result: tC 10918
2024.01.08 21:56:03 5: DbRep logdb.Serv - BlockingCall PID "15674" finished
2024.01.08 21:56:03 5: DbRep logdb.Serv - SQL result decoded: no_aggregation#Aussenlichtstaerke#916#|no_aggregation#Aussenluftfeuchte#916#|no_aggregation#Aussentaupunkt#916#|no_aggregation#Aussentemperatur#916#|no_aggregation#ENERGY_Power#7514#|no_aggregation#ENERGY_Today#9664#|no_aggregation#ENERGY_Total#8642#|no_aggregation#ENERGY_Yesterday#47#|no_aggregation#Ertrag_gesamt#68#|no_aggregation#Ertrag_gestern#12#|no_aggregation#Ertrag_heute#74#|no_aggregation#Kosten_gesamt#6510#|no_aggregation#Kosten_gestern#5598#|no_aggregation#Kosten_heute#5957#|no_aggregation#Kosten_proJahr#35#|no_aggregation#Kosten_proMonat#4#|no_aggregation#Kosten_proTag#1#|no_aggregation#Kosten_proWoche#2#|no_aggregation#Luftdruck#916#|no_aggregation#Regenmenge#2055#|no_aggregation#RegenmengeYesterday#4#|no_aggregation#SunAlt#2901#|no_aggregation#SunAz#3110#|no_aggregation#VorhersageHeute#30#|no_aggregation#VorhersageMorgen#30#|no_aggregation#h2o_temperature#28#|no_aggregation#power#1#|no_aggregation#state#5496#|no_aggregation#tC#10918#|no_aggregation#ALLREADINGS#73281#|
2024.01.08 21:56:25 4: DbRep logdb.Serv - -------- New selection ---------
2024.01.08 21:56:25 4: DbRep logdb.Serv - Command: countEntries history
2024.01.08 21:56:25 4: DbRep logdb.Serv - Timestamp begin human readable: not set
2024.01.08 21:56:25 4: DbRep logdb.Serv - Timestamp end human readable: not set
2024.01.08 21:56:25 4: DbRep logdb.Serv - Aggregation: no
2024.01.08 21:56:25 5: DbRep logdb.Serv - BlockingCall with PID "15676" started
2024.01.08 21:56:25 4: DbRep logdb.Serv - Database connect - user: fhemuser, UTF-8 option set: yes
2024.01.08 21:56:25 4: DbRep logdb.Serv - SQL execute: SHOW VARIABLES LIKE 'collation_database'
2024.01.08 21:56:25 4: DbRep logdb.Serv - Database Character set is >utf8mb4_bin<
2024.01.08 21:56:25 4: DbRep logdb.Serv - simple do statement: set names "utf8mb4" collate "utf8mb4_bin"
2024.01.08 21:56:25 5: DbRep logdb.Serv - IsTimeSet: 0, IsAggrSet: 0
2024.01.08 21:56:25 5: DbRep logdb.Serv - Timestamp-Array:
no_aggregation
2024.01.08 21:56:25 5: DbRep logdb.Serv - Devices for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2024.01.08 21:56:25 5: DbRep logdb.Serv - Readings for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2024.01.08 21:56:25 4: DbRep logdb.Serv - SQL execute: SELECT READING, COUNT(*) FROM history where 1 group by READING;
2024.01.08 21:56:26 5: DbRep logdb.Serv - SQL result: Aussenlichtstaerke 916
2024.01.08 21:56:26 5: DbRep logdb.Serv - SQL result: Aussenluftfeuchte 916
2024.01.08 21:56:26 5: DbRep logdb.Serv - SQL result: Aussentaupunkt 916
2024.01.08 21:56:26 5: DbRep logdb.Serv - SQL result: Aussentemperatur 916
2024.01.08 21:56:26 5: DbRep logdb.Serv - SQL result: ENERGY_Power 7514
2024.01.08 21:56:26 5: DbRep logdb.Serv - SQL result: ENERGY_Today 9664
2024.01.08 21:56:26 5: DbRep logdb.Serv - SQL result: ENERGY_Total 8642
2024.01.08 21:56:26 5: DbRep logdb.Serv - SQL result: ENERGY_Yesterday 47
2024.01.08 21:56:26 5: DbRep logdb.Serv - SQL result: Ertrag_gesamt 68
2024.01.08 21:56:26 5: DbRep logdb.Serv - SQL result: Ertrag_gestern 12
2024.01.08 21:56:26 5: DbRep logdb.Serv - SQL result: Ertrag_heute 74
2024.01.08 21:56:26 5: DbRep logdb.Serv - SQL result: Kosten_gesamt 6510
2024.01.08 21:56:26 5: DbRep logdb.Serv - SQL result: Kosten_gestern 5598
2024.01.08 21:56:26 5: DbRep logdb.Serv - SQL result: Kosten_heute 5957
2024.01.08 21:56:26 5: DbRep logdb.Serv - SQL result: Kosten_proJahr 35
2024.01.08 21:56:26 5: DbRep logdb.Serv - SQL result: Kosten_proMonat 4
2024.01.08 21:56:26 5: DbRep logdb.Serv - SQL result: Kosten_proTag 1
2024.01.08 21:56:26 5: DbRep logdb.Serv - SQL result: Kosten_proWoche 2
2024.01.08 21:56:26 5: DbRep logdb.Serv - SQL result: Luftdruck 916
2024.01.08 21:56:26 5: DbRep logdb.Serv - SQL result: Regenmenge 2055
2024.01.08 21:56:26 5: DbRep logdb.Serv - SQL result: RegenmengeYesterday 4
2024.01.08 21:56:26 5: DbRep logdb.Serv - SQL result: SunAlt 2901
2024.01.08 21:56:26 5: DbRep logdb.Serv - SQL result: SunAz 3110
2024.01.08 21:56:26 5: DbRep logdb.Serv - SQL result: VorhersageHeute 30
2024.01.08 21:56:26 5: DbRep logdb.Serv - SQL result: VorhersageMorgen 30
2024.01.08 21:56:26 5: DbRep logdb.Serv - SQL result: h2o_temperature 28
2024.01.08 21:56:26 5: DbRep logdb.Serv - SQL result: power 1
2024.01.08 21:56:26 5: DbRep logdb.Serv - SQL result: state 5496
2024.01.08 21:56:26 5: DbRep logdb.Serv - SQL result: tC 10918
2024.01.08 21:56:26 5: DbRep logdb.Serv - BlockingCall PID "15676" finished
2024.01.08 21:56:26 5: DbRep logdb.Serv - SQL result decoded: no_aggregation#Aussenlichtstaerke#916#|no_aggregation#Aussenluftfeuchte#916#|no_aggregation#Aussentaupunkt#916#|no_aggregation#Aussentemperatur#916#|no_aggregation#ENERGY_Power#7514#|no_aggregation#ENERGY_Today#9664#|no_aggregation#ENERGY_Total#8642#|no_aggregation#ENERGY_Yesterday#47#|no_aggregation#Ertrag_gesamt#68#|no_aggregation#Ertrag_gestern#12#|no_aggregation#Ertrag_heute#74#|no_aggregation#Kosten_gesamt#6510#|no_aggregation#Kosten_gestern#5598#|no_aggregation#Kosten_heute#5957#|no_aggregation#Kosten_proJahr#35#|no_aggregation#Kosten_proMonat#4#|no_aggregation#Kosten_proTag#1#|no_aggregation#Kosten_proWoche#2#|no_aggregation#Luftdruck#916#|no_aggregation#Regenmenge#2055#|no_aggregation#RegenmengeYesterday#4#|no_aggregation#SunAlt#2901#|no_aggregation#SunAz#3110#|no_aggregation#VorhersageHeute#30#|no_aggregation#VorhersageMorgen#30#|no_aggregation#h2o_temperature#28#|no_aggregation#power#1#|no_aggregation#state#5496#|no_aggregation#tC#10918#|no_aggregation#ALLREADINGS#73281#|
2024.01.08 21:57:00 3: CUL_HM set AQ_HeizungH2O off noArg
2024.01.08 21:57:03 3: DbRep logdb.Serv - ################################################################
2024.01.08 21:57:03 3: DbRep logdb.Serv - ###                    new reduceLog run                     ###
2024.01.08 21:57:03 3: DbRep logdb.Serv - ################################################################
2024.01.08 21:57:03 4: DbRep logdb.Serv - -------- New selection ---------
2024.01.08 21:57:03 4: DbRep logdb.Serv - Command: reduceLog 7:180
2024.01.08 21:57:03 4: DbRep logdb.Serv - timeDiffToNow - year: , day: 180, hour: , min: , sec:
2024.01.08 21:57:03 4: DbRep logdb.Serv - Year 2024 is leap year
2024.01.08 21:57:03 4: DbRep logdb.Serv - startMonth: 6 endMonth: 0 lastleapyear: 2024 baseYear: 2023 diffdaylight:1 isdaylight:0
2024.01.08 21:57:03 4: DbRep logdb.Serv - timeOlderThan - year: 0, day: 7, hour: 0, min: 0, sec: 0
2024.01.08 21:57:03 4: DbRep logdb.Serv - Year 2024 is leap year
2024.01.08 21:57:03 4: DbRep logdb.Serv - startMonth: 0 endMonth: 0 lastleapyear: 2024 baseYear: 2024 diffdaylight:0 isdaylight:0
2024.01.08 21:57:03 4: DbRep logdb.Serv - FullDay option: 1
2024.01.08 21:57:03 4: DbRep logdb.Serv - Time difference to current time for calculating Timestamp begin: 15555601 sec
2024.01.08 21:57:03 5: DbRep logdb.Serv - Timestamp begin epocheseconds: 1689112800
2024.01.08 21:57:03 4: DbRep logdb.Serv - Timestamp begin human readable: 2023-07-12 00:00:00
2024.01.08 21:57:03 5: DbRep logdb.Serv - Timestamp end epocheseconds: 1704149999
2024.01.08 21:57:03 4: DbRep logdb.Serv - Timestamp end human readable: 2024-01-01 23:59:59
2024.01.08 21:57:03 3: DB ENERGY_To gelöscht
2024.01.08 21:57:04 3: CUL_HM set LD_blau_2.Nacht statusRequest noArg
2024.01.08 21:57:14 4: DbRep logdb.Serv - -------- New selection ---------
2024.01.08 21:57:14 4: DbRep logdb.Serv - Command: countEntries history
2024.01.08 21:57:14 4: DbRep logdb.Serv - timeDiffToNow - year: , day: 180, hour: , min: , sec:
2024.01.08 21:57:14 4: DbRep logdb.Serv - Year 2024 is leap year
2024.01.08 21:57:14 4: DbRep logdb.Serv - startMonth: 6 endMonth: 0 lastleapyear: 2024 baseYear: 2023 diffdaylight:1 isdaylight:0
2024.01.08 21:57:14 4: DbRep logdb.Serv - timeOlderThan - year: 0, day: 7, hour: 0, min: 0, sec: 0
2024.01.08 21:57:14 4: DbRep logdb.Serv - Year 2024 is leap year
2024.01.08 21:57:14 4: DbRep logdb.Serv - startMonth: 0 endMonth: 0 lastleapyear: 2024 baseYear: 2024 diffdaylight:0 isdaylight:0
2024.01.08 21:57:14 4: DbRep logdb.Serv - FullDay option: 1
2024.01.08 21:57:14 4: DbRep logdb.Serv - Time difference to current time for calculating Timestamp begin: 15555601 sec
2024.01.08 21:57:14 5: DbRep logdb.Serv - Timestamp begin epocheseconds: 1689112800
2024.01.08 21:57:14 4: DbRep logdb.Serv - Timestamp begin human readable: 2023-07-12 00:00:00
2024.01.08 21:57:14 5: DbRep logdb.Serv - Timestamp end epocheseconds: 1704149999
2024.01.08 21:57:14 4: DbRep logdb.Serv - Timestamp end human readable: 2024-01-01 23:59:59
2024.01.08 21:57:14 4: DbRep logdb.Serv - Aggregation: no
2024.01.08 21:57:14 5: DbRep logdb.Serv - BlockingCall with PID "15679" started
2024.01.08 21:57:14 4: DbRep logdb.Serv - Database connect - user: fhemuser, UTF-8 option set: yes
2024.01.08 21:57:14 4: DbRep logdb.Serv - SQL execute: SHOW VARIABLES LIKE 'collation_database'
2024.01.08 21:57:14 4: DbRep logdb.Serv - Database Character set is >utf8mb4_bin<
2024.01.08 21:57:14 4: DbRep logdb.Serv - simple do statement: set names "utf8mb4" collate "utf8mb4_bin"
2024.01.08 21:57:14 5: DbRep logdb.Serv - IsTimeSet: 1, IsAggrSet: 0
2024.01.08 21:57:14 5: DbRep logdb.Serv - Timestamp-Array:
no_aggregation#2023-07-12 00:00:00#2024-01-01 23:59:59
2024.01.08 21:57:14 5: DbRep logdb.Serv - Devices for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2024.01.08 21:57:14 5: DbRep logdb.Serv - Readings for operation -
included (1): %
included with wildcard: 
excluded (0): 
excluded with wildcard:
2024.01.08 21:57:14 4: DbRep logdb.Serv - SQL execute: SELECT READING, COUNT(*) FROM history where TIMESTAMP >= '2023-07-12 00:00:00' AND TIMESTAMP <= '2024-01-01 23:59:59' group by READING;
2024.01.08 21:57:14 5: DbRep logdb.Serv - BlockingCall PID "15679" finished
2024.01.08 21:57:14 5: DbRep logdb.Serv - SQL result decoded: no_aggregation#ALLREADINGS#0#2023-07-12|
Auch funktionierende Lösungen kann man hinterfragen.

DS_Starter

#11
Hmm, Problem ist die noch vorhandene Time-Eingrenzung nach reduceLog:

2024.01.08 21:57:14 4: DbRep logdb.Serv - Timestamp begin human readable: 2023-07-12 00:00:00
...
2024.01.08 21:57:14 4: DbRep logdb.Serv - Timestamp end human readable: 2024-01-01 23:59:59

Allerdings verstehe ich das momentan nicht wieso es so ist. Bei meinen Tests war es ok.
Gibt es an deinem System irgendweine Besonderheit? (Kann ich mir allerdings nicht vorstellen was es sein könnte).

EDIT: Hast du den Test jetzt mit der V 8.53.0 aus meinem contrib gemacht? Wenn nicht, ist das Verhalten klar.
Proxmox+Debian+MariaDB, PV: SMA, Victron MPII+Pylontech+CerboGX
Maintainer: SSCam, SSChatBot, SSCal, SSFile, DbLog/DbRep, Log2Syslog, SolarForecast,Watches, Dashboard, PylonLowVoltage
Kaffeekasse: https://www.paypal.me/HMaaz
Contrib: https://svn.fhem.de/trac/browser/trunk/fhem/contrib/DS_Starter

rabehd

Ich denke es ist die Version aus Deinem contrib.
ZitatFVERSION 93_DbRep.pm:v8.53.0-s28267/2023-12-08

Ich habe die Devices aus dem alten FHEM kopiert (RAW). Eine Besonderheit im Gegenzug dazu gibt es nicht. Ich glaube es war vorher eine MySQL und ist jetzt eine MariaDB.
Ich führe jede Nacht verschiedene reduceLog überein DOIF aus. Gerade das ist aus dem alten Systen übernommen.
Auch funktionierende Lösungen kann man hinterfragen.

DS_Starter

#13
Ja die V passt.
Wenn ich den Test bei mir ausführe zum Vergleich:

Zitat2024.01.08 22:22:28.245 3: DbRep Rep.Test - ################################################################
2024.01.08 22:22:28.246 3: DbRep Rep.Test - ###                    new reduceLog run                     ###
2024.01.08 22:22:28.247 3: DbRep Rep.Test - ################################################################
2024.01.08 22:22:28.262 4: DbRep Rep.Test - -------- New selection ---------
2024.01.08 22:22:28.263 4: DbRep Rep.Test - Command: reduceLog 7:180
2024.01.08 22:22:28.264 4: DbRep Rep.Test - timeDiffToNow - year: , day: 180, hour: , min: , sec:
2024.01.08 22:22:28.264 4: DbRep Rep.Test - Year 2024 is leap year
2024.01.08 22:22:28.265 4: DbRep Rep.Test - startMonth: 6 endMonth: 0 lastleapyear: 2024 baseYear: 2023 diffdaylight:1 isdaylight:0
2024.01.08 22:22:28.265 4: DbRep Rep.Test - timeOlderThan - year: 0, day: 7, hour: 0, min: 0, sec: 0
2024.01.08 22:22:28.266 4: DbRep Rep.Test - Year 2024 is leap year
2024.01.08 22:22:28.266 4: DbRep Rep.Test - startMonth: 11 endMonth: 0 lastleapyear: 2024 baseYear: 2024 diffdaylight:0 isdaylight:0
2024.01.08 22:22:28.267 4: DbRep Rep.Test - FullDay option: 1
2024.01.08 22:22:28.267 4: DbRep Rep.Test - Time difference to current time for calculating Timestamp begin: 15555601 sec
2024.01.08 22:22:28.268 4: DbRep Rep.Test - Timestamp begin human readable: 2023-07-12 00:00:00
2024.01.08 22:22:28.268 4: DbRep Rep.Test - Timestamp end human readable: 2024-01-01 23:59:59

2024.01.08 22:22:28.333 4: DbRep Rep.Test - Database connect - user: no, UTF-8 option set: yes
2024.01.08 22:22:28.339 3: DbRep Rep.Test - reduce data older than: 2024-01-01 23:59:59, newer than: 2023-07-12 00:00:00
2024.01.08 22:22:28.340 3: DbRep Rep.Test - reduceLog requested with options:
max=day
INCLUDE -> %:ENERGY_Power
2024.01.08 22:22:28.340 4: DbRep Rep.Test - SQL prepare: DELETE FROM history WHERE (DEVICE=?) AND (READING=?) AND (TIMESTAMP=?) AND (VALUE=?)
2024.01.08 22:22:28.342 4: DbRep Rep.Test - SQL prepare: DELETE FROM history WHERE (DEVICE=?) AND (READING=?) AND (TIMESTAMP=?) AND VALUE IS NULL
2024.01.08 22:22:28.342 4: DbRep Rep.Test - SQL prepare: UPDATE history SET TIMESTAMP=?, EVENT=?, VALUE=? WHERE (DEVICE=?) AND (READING=?) AND (TIMESTAMP=?) AND (VALUE=?)
2024.01.08 22:22:28.343 4: DbRep Rep.Test - SQL prepare: DELETE FROM history WHERE (DEVICE=?) AND (READING=?) AND (TIMESTAMP=?)
2024.01.08 22:22:28.344 4: DbRep Rep.Test - SQL prepare: UPDATE history SET TIMESTAMP=?, EVENT=?, VALUE=? WHERE (DEVICE=?) AND (READING=?) AND (TIMESTAMP=?)
2024.01.08 22:22:28.344 4: DbRep Rep.Test - SQL execute: SELECT TIMESTAMP,DEVICE,'',READING,VALUE FROM history WHERE DEVICE like '%' AND READING like 'ENERGY_Power' AND TIMESTAMP <= '2024-01-01 23:59:59' AND TIMESTAMP >= '2023-07-12 00:00:00' ORDER BY TIMESTAMP ASC
2024.01.08 22:22:28.671 3: DbRep Rep.Test - reduceLog finished. Rows processed: 0, deleted: 0, updated: 0
2024.01.08 22:22:58.130 4: DbRep Rep.Test - -------- New selection ---------
2024.01.08 22:22:58.130 4: DbRep Rep.Test - Command: countEntries history
2024.01.08 22:22:58.131 4: DbRep Rep.Test - Timestamp begin human readable: not set
2024.01.08 22:22:58.132 4: DbRep Rep.Test - Timestamp end human readable: not set

2024.01.08 22:22:58.132 4: DbRep Rep.Test - Aggregation: no
2024.01.08 22:22:58.173 4: DbRep Rep.Test - Database connect - user: no, UTF-8 option set: yes
2024.01.08 22:22:58.179 4: DbRep Rep.Test - SQL execute: SELECT COUNT(*) FROM history where ( DEVICE = 'SMA_Energymeter' ) AND 1 ;

Nur mal zum Vergleich die Time-Eingrenzungen bei mir und dir. Das hilft zwar jetzt nicht, aber verdeutlicht den Unterschied.
Das macht mir grad etwas Kopfzerbrechen...

EDIT: Hast du nach dem Download der V auch restartet?
Proxmox+Debian+MariaDB, PV: SMA, Victron MPII+Pylontech+CerboGX
Maintainer: SSCam, SSChatBot, SSCal, SSFile, DbLog/DbRep, Log2Syslog, SolarForecast,Watches, Dashboard, PylonLowVoltage
Kaffeekasse: https://www.paypal.me/HMaaz
Contrib: https://svn.fhem.de/trac/browser/trunk/fhem/contrib/DS_Starter

DS_Starter

Wenn du mit dieser V das reducelog ausgeführt hast, poste mal bitte ein list des Devices direkt nach Abschluß des reducelog.
Proxmox+Debian+MariaDB, PV: SMA, Victron MPII+Pylontech+CerboGX
Maintainer: SSCam, SSChatBot, SSCal, SSFile, DbLog/DbRep, Log2Syslog, SolarForecast,Watches, Dashboard, PylonLowVoltage
Kaffeekasse: https://www.paypal.me/HMaaz
Contrib: https://svn.fhem.de/trac/browser/trunk/fhem/contrib/DS_Starter