Backup doppelt ausgeführt

cotecmania · 01 November 2014, 10:39:29

Hallo,

wenn ich einmal den Befehl "backup" ausführe wird dieser zweimal ausgeführt :

2014.11.01 10:08:37 2: Backup with command: tar -cf - fhem.cfg ./log/fhem.save ./CHANGED ./configDB.pm ./contrib ./culfw ./demolog ./docs ./FHEM ./fhem.cfg ./fhem.cfg.demo ./fhem.pl ./ipcam ./log ./README_DEMO.txt ./restoreDir ./unused ./www |gzip > ./backup/FHEM-20141101_100837.tar.gz
2014.11.01 10:13:36 1: backup done: FHEM-20141101_100837.tar.gz (42091464 Bytes)
2014.11.01 10:13:37 1: 192.168.1.151:1000 disconnected, waiting to reappear (HMLAN1)
2014.11.01 10:13:37 1: HMLAN_Parse: HMLAN1 new condition disconnected
2014.11.01 10:13:37 1: 192.168.1.151:1000 reappeared (HMLAN1)
2014.11.01 10:13:37 1: HMLAN_Parse: HMLAN1 new condition init
2014.11.01 10:13:37 1: HMLAN_Parse: HMLAN1 new condition ok
2014.11.01 10:13:40 2: Backup with command: tar -cf - fhem.cfg ./log/fhem.save ./CHANGED ./configDB.pm ./contrib ./culfw ./demolog ./docs ./FHEM ./fhem.cfg ./fhem.cfg.demo ./fhem.pl ./ipcam ./log ./README_DEMO.txt ./restoreDir ./unused ./www |gzip > ./backup/FHEM-20141101_101340.tar.gz
2014.11.01 10:18:33 1: backup done: FHEM-20141101_101340.tar.gz (42091711 Bytes)

Jemand ne Idee ?

Ralli · 01 November 2014, 10:43:22

Das Backup dauert lange.

Hat mit dem Timeout bzw. Reload der Webseite zu tun. Nach Eingabe des Befehls die Seite schließen, dann tritt das Problem nicht mehr auf.

rudolfkoenig · 01 November 2014, 11:28:58

Ich habe vor ca 6 Wochen ein Bug ausgebaut, der mehrere (3?) backups gestartet hat, wenn update im Hintergrund lief:

Zitatr6752 | rudolfkoenig | 2014-10-12 14:25:44 +0200 (Sun, 12 Oct 2014) | 2 lines
98_update: fix multiple backup if running in the background

oracess · 09 Dezember 2014, 04:23:51

Habe auch die doppelte Ausführung von Backup mit Stand 9.12.14 !! Der Bug war aber zwischenzeitlich schon weg und tritt jetzt wieder auf. Bedeutet dies wieder auf die Version von 12.10.14 zurück gehen ??

rudolfkoenig · 09 Dezember 2014, 19:48:19

Vermutlich, solange ich nicht eine detaillierte Fehlermeldung bzw. eine Anleitung zum Nachstellen bekomme.

oracess · 12 Dezember 2014, 14:20:56

Wie kann ich helfen ?
Wenn man den Browser nach Eingabe von "backup" schließt, dann nur einmal Backup sonst 2x !!
Bei backup before update komischerweise nur ein Backup.
Lustiges noch am Rande, bei Feueralarm notify bekomme ich 2 Mails ist aber besser als gar keines !

lg oracess

rudolfkoenig · 12 Dezember 2014, 14:38:50

Am besten "attr global loglevel 5" setzen, und das Protokoll hier anhaengen.

forum-merlin · 18 Juni 2015, 00:27:30

Hallo Rudi,

Ich habe heute auch noch das Problem, dass das Backup doppelt erzeugt wird.
Ich habe eben mal global auf verbose 5 gesetzt, und dann ein FHEMCMD>backup im Webinterface angestoßen.
Ich habe einen Pi2 und eine extrem schnelle SDHC Karte drinnen. Daher braucht ein backup bei mir nun nur noch 3 Minuten, aber eben zweimal.

Hier ein eingekürztes Log via
cmd>tail -f fhem-2015-06.log | grep -i backup >>fhem-only_backup_messages-2015-06-17.log

Code Auswählen


2015.06.17 23:47:04 4: HTTP FHEMWEB:192.168.178.97:52690 GET /fhem&room=hidden&cmd=backup
2015.06.17 23:47:04 5: Cmd: >backup<
2015.06.17 23:47:04 5: Loading /opt/fhem/FHEM/98_backup.pm
2015.06.17 23:47:04 4: backup include: 'fhem.cfg'
2015.06.17 23:47:04 4: backup include: './log/fhem.save'
2015.06.17 23:47:04 4: backup exclude: '/opt/fhem/backup'
2015.06.17 23:47:04 4: backup exclude: '/opt/fhem/backup'
2015.06.17 23:47:04 4: backup include: '/opt/fhem/certs'
2015.06.17 23:47:04 4: backup include: '/opt/fhem/CHANGED'
2015.06.17 23:47:04 4: backup include: '/opt/fhem/config.example'
2015.06.17 23:47:04 4: backup include: '/opt/fhem/configDB.pm'
2015.06.17 23:47:04 4: backup include: '/opt/fhem/contrib'
2015.06.17 23:47:04 4: backup include: '/opt/fhem/demolog'
2015.06.17 23:47:04 4: backup include: '/opt/fhem/docs'
2015.06.17 23:47:04 4: backup include: '/opt/fhem/FHEM'
2015.06.17 23:47:04 4: backup include: '/opt/fhem/fhem.cfg'
2015.06.17 23:47:04 4: backup include: '/opt/fhem/fhem.cfg.demo'
2015.06.17 23:47:04 4: backup include: '/opt/fhem/fhem.pl'
2015.06.17 23:47:04 4: backup include: '/opt/fhem/log'
2015.06.17 23:47:04 4: backup include: '/opt/fhem/restoreDir'
2015.06.17 23:47:04 4: backup include: '/opt/fhem/unused'
2015.06.17 23:47:04 4: backup include: '/opt/fhem/www'
2015.06.17 23:47:04 4: backup include: '/opt/fhem/Z_FHEMDEMO'
2015.06.17 23:47:04 4: backup include: '/opt/fhem/Z_NOTES'
2015.06.17 23:47:04 4: backup include: '/opt/fhem/Z_OLD.WhatsAppStuff'
2015.06.17 23:47:04 2: Backup with command: tar -cf - fhem.cfg ./log/fhem.save /opt/fhem/certs /opt/fhem/CHANGED /opt/fhem/config.example /opt/fhem/configDB.pm /opt/fhem/contrib /opt/fhem/demolog /opt/fhem/docs /opt/fhem/FHEM /opt/fhem/fhem.cfg /opt/fhem/fhem.cfg.demo /opt/fhem/fhem.pl /opt/fhem/log /opt/fhem/restoreDir /opt/fhem/unused /opt/fhem/www /opt/fhem/Z_FHEMDEMO /opt/fhem/Z_NOTES /opt/fhem/Z_OLD.WhatsAppStuff |gzip > /opt/fhem/backup/FHEM-20150617_234704.tar.gz
2015.06.17 23:47:04 2: Backup with command: tar -cf - fhem.cfg ./log/fhem.save /opt/fhem/certs /opt/fhem/CHANGED /opt/fhem/config.example /opt/fhem/configDB.pm /opt/fhem/contrib /opt/fhem/demolog /opt/fhem/docs /opt/fhem/FHEM /opt/fhem/fhem.cfg /opt/fhem/fhem.cfg.demo /opt/fhem/fhem.pl /opt/fhem/log /opt/fhem/restoreDir /opt/fhem/unused /opt/fhem/www /opt/fhem/Z_FHEMDEMO /opt/fhem/Z_NOTES /opt/fhem/Z_OLD.WhatsAppStuff |gzip > /opt/fhem/backup/FHEM-20150617_234704.tar.gz
2015.06.17 23:49:49 1: backup tar: Removing leading `/' from member names
2015.06.17 23:49:49 1: backup done: FHEM-20150617_234704.tar.gz (78375342 Bytes)
2015.06.17 23:49:49 4: 2410:FHEMWEB:192.168.178.97:52690: /fhem&room=hidden&cmd=backup / RL:1696 / text/html; charset=UTF-8 / Content-Encoding: gzip

2015.06.17 23:50:03 4: HTTP FHEMWEB:192.168.178.97:52709 GET /fhem&room=hidden&cmd=backup
2015.06.17 23:50:03 5: Cmd: >backup<
2015.06.17 23:50:03 4: backup include: 'fhem.cfg'
2015.06.17 23:50:03 4: backup include: './log/fhem.save'
2015.06.17 23:50:03 4: backup exclude: '/opt/fhem/backup'
2015.06.17 23:50:03 4: backup exclude: '/opt/fhem/backup'
2015.06.17 23:50:03 4: backup include: '/opt/fhem/certs'
2015.06.17 23:50:03 4: backup include: '/opt/fhem/CHANGED'
2015.06.17 23:50:03 4: backup include: '/opt/fhem/config.example'
2015.06.17 23:50:03 4: backup include: '/opt/fhem/configDB.pm'
2015.06.17 23:50:03 4: backup include: '/opt/fhem/contrib'
2015.06.17 23:50:03 4: backup include: '/opt/fhem/demolog'
2015.06.17 23:50:03 4: backup include: '/opt/fhem/docs'
2015.06.17 23:50:03 4: backup include: '/opt/fhem/FHEM'
2015.06.17 23:50:03 4: backup include: '/opt/fhem/fhem.cfg'
2015.06.17 23:50:03 4: backup include: '/opt/fhem/fhem.cfg.demo'
2015.06.17 23:50:03 4: backup include: '/opt/fhem/fhem.pl'
2015.06.17 23:50:03 4: backup include: '/opt/fhem/log'
2015.06.17 23:50:03 4: backup include: '/opt/fhem/restoreDir'
2015.06.17 23:50:03 4: backup include: '/opt/fhem/unused'
2015.06.17 23:50:03 4: backup include: '/opt/fhem/www'
2015.06.17 23:50:03 4: backup include: '/opt/fhem/Z_FHEMDEMO'
2015.06.17 23:50:03 4: backup include: '/opt/fhem/Z_NOTES'
2015.06.17 23:50:03 4: backup include: '/opt/fhem/Z_OLD.WhatsAppStuff'
2015.06.17 23:50:03 2: Backup with command: tar -cf - fhem.cfg ./log/fhem.save /opt/fhem/certs /opt/fhem/CHANGED /opt/fhem/config.example /opt/fhem/configDB.pm /opt/fhem/contrib /opt/fhem/demolog /opt/fhem/docs /opt/fhem/FHEM /opt/fhem/fhem.cfg /opt/fhem/fhem.cfg.demo /opt/fhem/fhem.pl /opt/fhem/log /opt/fhem/restoreDir /opt/fhem/unused /opt/fhem/www /opt/fhem/Z_FHEMDEMO /opt/fhem/Z_NOTES /opt/fhem/Z_OLD.WhatsAppStuff |gzip > /opt/fhem/backup/FHEM-20150617_235003.tar.gz
2015.06.17 23:50:03 2: Backup with command: tar -cf - fhem.cfg ./log/fhem.save /opt/fhem/certs /opt/fhem/CHANGED /opt/fhem/config.example /opt/fhem/configDB.pm /opt/fhem/contrib /opt/fhem/demolog /opt/fhem/docs /opt/fhem/FHEM /opt/fhem/fhem.cfg /opt/fhem/fhem.cfg.demo /opt/fhem/fhem.pl /opt/fhem/log /opt/fhem/restoreDir /opt/fhem/unused /opt/fhem/www /opt/fhem/Z_FHEMDEMO /opt/fhem/Z_NOTES /opt/fhem/Z_OLD.WhatsAppStuff |gzip > /opt/fhem/backup/FHEM-20150617_235003.tar.gz
2015.06.17 23:52:49 1: backup tar: Removing leading `/' from member names
2015.06.17 23:52:49 1: backup done: FHEM-20150617_235003.tar.gz (78371769 Bytes)
2015.06.17 23:52:49 4: 2410:FHEMWEB:192.168.178.97:52709: /fhem&room=hidden&cmd=backup / RL:1694 / text/html; charset=UTF-8 / Content-Encoding: gzip

Hier noch ein paar generelle infos

Code Auswählen


Fhem info: 
  Release  : 5.6 
  OS       : linux 
  Arch     : arm-linux-gnueabihf-thread-multi-64int 
  Perl     : v5.14.2 
  uniqueID : 0ec966ae6xxxxxxxxxxx82ce99923311357b 
  upTime   : 00:39:35 

Defined modules: 
  CUL            : 1 
  CUL_HM         : 64 
  Calendar       : 1 
  DOIF           : 5 
  Dashboard      : 1 
  ENIGMA2        : 2 
  FB_CALLMONITOR : 1 
  FHEMWEB        : 3 
  FLOORPLAN      : 5 
  FileLog        : 57 
  HMLAN          : 1 
  HTTPSRV        : 1 
  JeeLink        : 1 
  LaCrosse       : 5 
  ONKYO_AVR      : 1 
  PRESENCE       : 15 
  PROPLANTA      : 1 
  SVG            : 14 
  TRX            : 1 
  TRX_LIGHT      : 35 
  Twilight       : 1 
  WOL            : 1 
  Weather        : 2 
  at             : 5 
  autocreate     : 1 
  dummy          : 34 
  harmony        : 10 
  notify         : 51 
  speedtest      : 1 
  telnet         : 1 
  watchdog       : 6 
  weblink        : 5 
  yowsup         : 1 

Defined models per module: 
  CUL_HM         : ActionDetector,HM-CC-RT-DN,HM-ES-PMSw1-Pl,HM-LC-BL1-FM,HM-LC-Dim1L-Pl-3,HM-LC-SW4-DR,HM-SEC-SCo,HM-SEC-WDS-2 
  ENIGMA2        : dm500hd 
  JeeLink        : [LaCrosseITPlusReader.10.1e @AutoToggle 30 Seconds / 868280 kHz] 
  ONKYO_AVR      : TX-NR636 

Transmitting this information during an update: no 
You can change this via the global attribute sendStatistics

update check

Code Auswählen


Got remote controlfile with 1537 entries. 
Got local controlfile with 1537 entries. 
nothing to do...

cmd> cat /proc/version

Code Auswählen


Linux version 3.18.16-v7+ (dc4@dc4-XPS13-9333) (gcc version 4.8.3 20140303 (prerelease) (crosstool-NG linaro-1.13.1+bzr2650 - Linaro GCC 2014.03) ) #795 SMP PREEMPT Wed Jun 17 13:24:01 BST 2015

98_backup.pm

Code Auswählen


################################################################
# $Id: 98_backup.pm 5893 2014-05-19 09:45:08Z rudolfkoenig $
# vim: ts=2:et
#
#  (c) 2012 Copyright: Martin Fischer (m_fischer at gmx dot de)
#  All rights reserved
#
#  This script free software; you can redistribute it and/or modify
#  it under the terms of the GNU General Public License as published by
#  the Free Software Foundation; either version 2 of the License, or
#  (at your option) any later version.
#
#  The GNU General Public License can be found at
#  http://www.gnu.org/copyleft/gpl.html.
#  A copy is found in the textfile GPL.txt and important notices to the license
#  from the author is found in LICENSE.txt distributed with these scripts.
#
#  This script is distributed in the hope that it will be useful,
#  but WITHOUT ANY WARRANTY; without even the implied warranty of
#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#  GNU General Public License for more details.
#
################################################################

package main;
use strict;
use warnings;

sub CommandBackup($$);
sub parseConfig($);
sub readModpath($$);
sub createArchiv($);

my @pathname;

#####################################
sub
backup_Initialize($$)
{
  my %hash = (  Fn => "CommandBackup",
               Hlp => ",create a backup of fhem configuration, state and modpath" );
  $cmds{backup} = \%hash;
}

#####################################
sub
CommandBackup($$)
{
  my ($cl, $param) = @_;

  my $modpath = $attr{global}{modpath};
  my $configfile = AttrVal("global", "configfile", undef);
  my $statefile  = AttrVal("global", "statefile", undef);
  my $msg;
  my $ret;

  Log 1, "NOTE: make sure you have a database backup!" if(configDBUsed());

  # set backupdir
  my $backupdir;
  if (!defined($attr{global}{backupdir})) {
    $backupdir = "$modpath/backup";
  } else {
    if ($attr{global}{backupdir} =~ m/^\/.*/) {
      $backupdir = $attr{global}{backupdir};
    } elsif ($attr{global}{backupdir} =~ m/^\.+\/.*/) {
      $backupdir = "$modpath/$attr{global}{backupdir}";
    } else {
      $backupdir = "$modpath/$attr{global}{backupdir}";
    }
  }

  # create backupdir if not exists
  if (!-d $backupdir) {
    Log 4, "backup create backupdir: '$backupdir'";
    $ret = `(mkdir -p $backupdir) 2>&1`;
    if ($ret) {
      chomp $ret;
      $msg = "backup: $ret";
      return $msg;
    }
  }

  if(configDBUsed()) {
    # add configDB configuration file
    push @pathname, 'configDB.conf';
    Log 4, "backup include: 'configDB.conf'";
  } else {
    # get pathnames to archiv
    push @pathname, $configfile;
    Log 4, "backup include: '$configfile'";
    $ret = parseConfig($configfile);
    push @pathname, $statefile;
    Log 4, "backup include: '$statefile'";
  }
  $ret = readModpath($modpath,$backupdir);

  # create archiv
  $ret = createArchiv($backupdir);

  @pathname = [];
  undef @pathname;

  return $ret;
}

sub
parseConfig($)
{
  my $configfile = shift;
  my $fh;
  my $msg;
  my $ret;

  if (!open($fh,$configfile)) {
    $msg = "Can't open $configfile: $!";
    Log 1, "backup $msg";
    return $msg;
  }

  while (my $l = <$fh>) {
    $l =~ s/[\r\n]//g;
    if ($l =~ m/^\s*include\s+(\S+)\s*.*$/) {
      if (-e $1) {
        push @pathname, $1;
        Log 4, "backup include: '$1'";
        $ret = parseConfig($1);
      } else {
        Log 1, "backup configfile: '$1' does not exists! File not included."
      }
    }
  }
  close $fh;
  return $ret;
}

sub
readModpath($$)
{
  my ($modpath,$backupdir) = @_;
  my $msg;
  my $ret;

  if (!opendir(DH, $modpath)) {
    $msg = "Can't open $modpath: $!";
    Log 1, "backup $msg";
    return $msg;
  }
  my @files = <$modpath/*>;
  foreach my $file (@files) {
    if ($file eq $backupdir && (-d $file || -l $file)) {
      Log 4, "backup exclude: '$file'";
    } else {
      Log 4, "backup include: '$file'";
      push @pathname, $file;
    }
  }
  return $ret;
}

sub
createArchiv($)
{
  my $backupdir = shift;
  my $backupcmd = (!defined($attr{global}{backupcmd}) ? undef : $attr{global}{backupcmd});
  my $symlink = (!defined($attr{global}{backupsymlink}) ? "no" : $attr{global}{backupsymlink});
  my $tarOpts;
  my $msg;
  my $ret;

  my $dateTime = TimeNow();
  $dateTime =~ s/ /_/g;
  $dateTime =~ s/(:|-)//g;

  my $cmd="";
  if (!defined($backupcmd)) {
    if (lc($symlink) eq "no") {
      $tarOpts = "cf";
    } else {
      $tarOpts = "chf";
    }

    # prevents tar's output of "Removing leading /" and return total bytes of archive
    $cmd = "tar -$tarOpts - @pathname |gzip > $backupdir/FHEM-$dateTime.tar.gz";

  } else {
    $cmd = "$backupcmd \"@pathname\"";

  }
  Log 2, "Backup with command: $cmd";
  $ret = `($cmd) 2>&1`;

  if($ret) {
    chomp $ret;
    Log 1, "backup $ret";
  }
  if (!defined($backupcmd) && -e "$backupdir/FHEM-$dateTime.tar.gz") {
    my $size = -s "$backupdir/FHEM-$dateTime.tar.gz";
    $msg = "backup done: FHEM-$dateTime.tar.gz ($size Bytes)";
    Log 1, $msg;
    $ret .= "\n".$msg;
  }
  return $ret;
}

1;

=pod
=begin html

<a name="backup"></a>
<h3>backup</h3>
<ul>
  <code>backup</code><br>
  <br>
  The complete FHEM directory (containing the modules), the WebInterface
  pgm2 (if installed) and the config-file will be saved into a .tar.gz
  file by default. The file is stored with a timestamp in the
  <a href="#modpath">modpath</a>/backup directory or to a directory
  specified by the global attribute <a href="#backupdir">backupdir</a>.<br>
  Note: tar and gzip must be installed to use this feature.
  <br>
  <br>
  If you need to call tar with support for symlinks, you could set the
  global attribute <a href="#backupsymlink">backupsymlink</a> to everything
  else as "no".
  <br>
  <br>
  You could pass the backup to your own command / script by using the
  global attribute <a href="#backupcmd">backupcmd</a>.
  <br>
  <br>
</ul>


=end html
=cut

Um ehrlich zu sein kenne ich mich da nicht aus da ich kein perl (oder sonst irgendwie programmieren) kann, aber ich denke irgtendwo hier ist der Aufruf doppelt oder?

Code Auswählen


  my $cmd="";
  if (!defined($backupcmd)) {
    if (lc($symlink) eq "no") {
      $tarOpts = "cf";
    } else {
      $tarOpts = "chf";
    }

    # prevents tar's output of "Removing leading /" and return total bytes of archive
    $cmd = "tar -$tarOpts - @pathname |gzip > $backupdir/FHEM-$dateTime.tar.gz";

  } else {
    $cmd = "$backupcmd \"@pathname\"";

  }
  Log 2, "Backup with command: $cmd";
  $ret = `($cmd) 2>&1`;

Hoffe es hilft.

Nutzt Du selbst nicht auch diese Backup Funktion?
Is das bei Dir nicht auch doppelt angelegt, wenn Du es ausführst?

Gruß

Holger

rudolfkoenig · 23 Juni 2015, 20:46:13

ZitatIch habe einen Pi2 und eine extrem schnelle SDHC Karte drinnen. Daher braucht ein backup bei mir nun nur noch 3 Minuten, aber eben zweimal.

"nur" ist relativ, dein Browser wurde ungeduldig, und hat die Seite nochmal bei FHEM angefragt, und damit das backup erneut ausgefuehrt. Ich habe 98_backup.pm angepasst, dass es bei einem Aufruf aus FHEMWEB backup das Befehl im Hintergrund ausfuehrt, und die Ergebnisse ins logfile schreibt.

Das backup laeuft auch dann im Hintergrund, wenn es vom update mit updateInBackground und backup_before_update gestartet wurde, und damit sollte es auch keine Probleme geben.

ZitatNutzt Du selbst nicht auch diese Backup Funktion?

Natuerlich nicht, ich mache komplett-Backups des Rechners mit "cp -al & rsync" (das sind Stichworte, und nicht so auszufuehren).
Ich war angenehm ueberrascht, wie ich gesehen habe, dass das TimeMachine backup im wesentlichen auch aus cp -al & rsync besteht.