RE: Networker/NDMP backup problems

From: Blake, Delroy (delroy.blake@gs.com)
Date: Thu Aug 14 2003 - 13:40:34 EDT

  • Next message: Jonathan: "Firmware corrupt"

    I would check media and drives.
    NDMP backups have one nasty/fatal issue and that is if you get a scsi reset
    while backups are going on you your drive will re-wind and overwrite your
    label. (NULL)
    Networker only writing a small amount of data on a tape is normally when he
    has had bad media or drive needs some TLC.

    Both combined I would look at data path and the drive while checking if
    /var/adm/messages complains about SCSI transport errors or the like.

    -----Original Message-----
    From: Matt Musgrove [mailto:matt.musgrove@thales-is.com]
    Sent: Thursday, August 14, 2003 7:27 AM
    To: Toasters
    Subject: Networker/NDMP backup problems

    Hello Toasters,

    I hope someone can help with this; I am having some annoying backup
    problems. My backup server is prematurely marking tapes as full sometimes
    with as little as 10 MB written. Also, some tapes have nothing written to
    them and are rejected by Networker. Errors are reported in
    /nsr/logs/daemon.log and are reproduced below. Usually the backups succeed
    after several failed attempts.

    Environment
    Networker Version: 6.1.3 Build.428
    Backup Server OS: Solaris 8
    NDMP Client: Network Appliance F820 OnTap 6.4.1
    Library: Qualstar TLS-6430 library with 3 * SDLT320 connected via LVD/SCSI

    Actions Taken
    Confirmed installation of Solaris Patch 108725-12 st driver patch Checked on
    SunSolve that no st.conf update is required for SDLT320 Checked
    /var/adm/messages - nothing reported. Sent message to Networker discussion
    group. On advice given set parallelism to 2 (to equal number of NDMP
    drives) and increased unload sleep from 5 to 180 and load sleep from 50 to
    180. This had no affect. Checked NetApp's NOW website - nothing helpful
    found, submitted case

    Typical daemon.log demonstrating error tape rejection problem

    08/09/03 21:00:36 nsrd: media waiting event: Waiting for 1 writable volumes
    to backup pool 'NetApp2 Full' tape(s) on ttsnas02-153 08/09/03 21:01:59
    nsrd: Jukebox 'qs6430' failed: expected volume 'NetApp2Full.003' got 'NULL'.
    08/09/03 21:02:07 nsrd: media info: suggest mounting NetApp2Full.005
    (E0011) on ttsnas02-153 for writing to pool 'NetApp2 Full' 08/09/03
    21:02:07 nsrd: media info: loading volume NetApp2Full.005 into
    rd=ttsnas02-153:nrst0a

    Typical daemon.log demonstrating backup errors

    08/06/03 20:30:00 nsrd: savegroup info: starting NetApp-2 (with 1
    client(s))
         application information: HIST=y, UPDATE=y, DIRECT=y, EXTRACT_ACL=T;
                      auth index: ttsnas02-153;
           auth index name space: backup, 1;
                      auth level: 5;
                       auth mode: save;
                     auth server: perky;
                     auth ssname: /vol/EMAIL1/data;
                auth ssname long: /vol/EMAIL1/data;
                     auth sstime: 1060198204;
              auth sstime 64-bit: 1060198204;
                       client id: \
    6627d4a5-00000004-3e817cda-3e817cd9-00010000-9901040a;
                          groups: NetApp-2;
              hard session limit: 1;
                        hostname: perky;
                          locale: C;
                            ndmp: Yes;
                        password: ********;
                     remote user: root;
                       save sets: /vol/EMAIL1/data;
             store index entries: Yes;
                     volume pool: NetApp2 NonFull;
    08/06/03 20:30:04 nsrd: media info: suggest mounting NetApp2NonFull.001
    (E0006) on ttsnas02-153 for writing to pool 'NetApp2 NonFull' 08/06/03
    20:30:04 nsrd: media waiting event: Waiting for 1 writable volumes to backup
    pool 'NetApp2 NonFull' tape(s) on ttsnas02-153 08/06/03 20:30:05 nsrd: media
    info: loading volume NetApp2NonFull.001 into rd=ttsnas02-153:nrst0a 08/06/03
    20:31:10 nsrd: rd=ttsnas02-153:nrst0a Verify label operation in progress
    08/06/03 20:31:11 nsrmmd #3: ndmp tape bsf failed 08/06/03 20:31:13 nsrd:
    rd=ttsnas02-153:nrst0a Mount operation in progress 08/06/03 20:31:14 nsrmmd
    #2: ndmp tape bsf failed 08/06/03 20:31:53 nsrmmd #2: ndmp tape fsf failed
    08/06/03 20:33:32 nsrmmd #2: ndmp tape fsf failed 08/06/03 20:35:10 nsrmmd
    #2: ndmp tape fsf failed 08/06/03 20:35:10 nsrd: media info: ndmp fsf failed
    08/06/03 20:35:10 nsrd: write completion notice: Writing to volume
    NetApp2NonFull.001 complete 08/06/03 20:35:21 nsrd: media info: suggest
    mounting NetApp2NonFull.002 (E0013) on ttsnas02-153 for writing to pool
    'NetApp2 NonFull' 08/06/03 20:35:21 nsrd: rd=ttsnas02-153:nrst0a Eject
    operation in progress 08/06/03 20:36:43 nsrd: media info: loading volume
    NetApp2NonFull.002 into rd=ttsnas02-153:nrst0a 08/06/03 20:37:48 nsrd:
    rd=ttsnas02-153:nrst0a Verify label operation in progress 08/06/03 20:37:49
    nsrmmd #3: ndmp tape bsf failed 08/06/03 20:37:51 nsrd:
    rd=ttsnas02-153:nrst0a is now write enabled 08/06/03 20:37:51 nsrd:
    rd=ttsnas02-153:nrst0a Mount operation in progress 08/06/03 20:37:52 nsrmmd
    #2: ndmp tape bsf failed 08/06/03 20:40:31 nsrd: media event cleared:
    Waiting for 1 writable volumes to backup pool 'NetApp2 NonFull' tape(s) on
    ttsnas02-153 08/06/03 20:40:31 nsrd: ttsnas02-153:/vol/EMAIL1/data saving to
    pool 'NetApp2 NonFull' (NetApp2NonFull.002)
    * ttsnas02-153:/vol/EMAIL1/data NDMP Service Log: DUMP: creating
    "/vol/EMAIL1/../snapshot_for_backup.16" snapshot.
    * ttsnas02-153:/vol/EMAIL1/data
    * ttsnas02-153:/vol/EMAIL1/data NDMP Service Log: DUMP: Using subtree dump
    * ttsnas02-153:/vol/EMAIL1/data
    * ttsnas02-153:/vol/EMAIL1/data NDMP Service Log: DUMP: Date of this level 5
    dump: Wed Aug 6 20:38:33 2003.
    * ttsnas02-153:/vol/EMAIL1/data
    * ttsnas02-153:/vol/EMAIL1/data NDMP Service Log: DUMP: Date of last level 4
    dump: Tue Aug 5 20:58:50 2003.
    * ttsnas02-153:/vol/EMAIL1/data
    * ttsnas02-153:/vol/EMAIL1/data NDMP Service Log: DUMP: Dumping
    /vol/EMAIL1/data to NDMP connection
    * ttsnas02-153:/vol/EMAIL1/data
    * ttsnas02-153:/vol/EMAIL1/data NDMP Service Log: DUMP: mapping (Pass
    I)[regular files]
    * ttsnas02-153:/vol/EMAIL1/data
    * ttsnas02-153:/vol/EMAIL1/data NDMP Service Log: DUMP: mapping (Pass
    II)[directories]
    * ttsnas02-153:/vol/EMAIL1/data
    * ttsnas02-153:/vol/EMAIL1/data NDMP Service Log: DUMP: estimated 782668 KB.
    * ttsnas02-153:/vol/EMAIL1/data
    * ttsnas02-153:/vol/EMAIL1/data NDMP Service Log: DUMP: dumping (Pass III)
    [directories]
    * ttsnas02-153:/vol/EMAIL1/data
    * ttsnas02-153:/vol/EMAIL1/data NDMP Service Log: DUMP: dumping (Pass IV)
    [regular files]
    * ttsnas02-153:/vol/EMAIL1/data
    * ttsnas02-153:/vol/EMAIL1/data NDMP Service Log: DUMP: Tape write failed.
    * ttsnas02-153:/vol/EMAIL1/data
    * ttsnas02-153:/vol/EMAIL1/data NDMP Service Log: DUMP: DUMP IS ABORTED
    * ttsnas02-153:/vol/EMAIL1/data
    * ttsnas02-153:/vol/EMAIL1/data NDMP Service Log: DUMP: Deleting
    "/vol/EMAIL1/../snapshot_for_backup.16" snapshot.
    * ttsnas02-153:/vol/EMAIL1/data
    * ttsnas02-153:/vol/EMAIL1/data
    ******************************************************************
    * ttsnas02-153:/vol/EMAIL1/data ------ E R R O R -------
    * ttsnas02-153:/vol/EMAIL1/data Data server halted: Dump aborted.
    * ttsnas02-153:/vol/EMAIL1/data
    ******************************************************************
    * ttsnas02-153:/vol/EMAIL1/data
    ******************************************************************
    * ttsnas02-153:/vol/EMAIL1/data ------ E R R O R -------
    * ttsnas02-153:/vol/EMAIL1/data Error running backup. status=3
    * ttsnas02-153:/vol/EMAIL1/data
    ******************************************************************
    * ttsnas02-153:/vol/EMAIL1/data Error during NDMP backup 08/06/03 20:46:57
    savegrp: ttsnas02-153:/vol/EMAIL1/data will retry 5 more time(s)

    regards

    Matt Musgrove, Thales Information Systems



    This archive was generated by hypermail 2b29 : Thu Aug 14 2003 - 13:45:34 EDT