List of NBU Utilities

Problem

This document lists utilities which are available for download for use with Symantec NetBackup.

Solution

The following utilities are available for download for use with Symantec NetBackup:

NetBackup License Capacity:

Symantec NetBackup Traditional and Capacity License Deployment Utility
http://www.symantec.com/docs/TECH148678

Network Validation Utility: DOCUMENTATION and DOWNLOAD: How to use Symantec's NetBackup Domain Network Analyzer (NBDNA) version 2.1 and BPTESTNETCONN.
http://www.symantec.com/docs/TECH125454

Log Upload Utility:Symantec NetBackup nbcplogs simplifies the tedious, time consuming, and error prone task of copying the correct and necessary customer NBU logs to Support
http://www.symantec.com/docs/TECH126223

Support Utility:Symantec NetBackup Support Utility (NBSU) is a Symantec utility used to gather diagnostic information about the system on which the utility is run
http://www.symantec.com/docs/TECH160944

Catalog Consistency:Symantec's Veritas NetBackup Catalog Consistency Utility (NBCC) is used to confirm the integrity of portions of the NetBackup catalog
http://www.symantec.com/docs/TECH156730

Symantec's Veritas NetBackup Catalog Repair Utility (NBCCR) is used to apply catalog repairs to the NetBackup catalog
http://www.symantec.com/docs/TECH156731

Media Server Decommissioning/Removal:

Note:  This utility is included in NetBackup 6.5.6 and 7.x Releases:

How to use Symantec's NetBackup Media Server Decommissioning Tool for NetBackup 6.5.5.
http://www.symantec.com/docs/TECH125456 

How to use Symantec's NetBackup Media Server Decommissioning Tool for NetBackup 6.5.4.
http://www.symantec.com/docs/TECH125193

How to use Symantec's NetBackup Media Server Decommissioning Tool for NetBackup 6.5.3.
http://www.symantec.com/docs/TECH125192

Drives go into AVR mode after nbrbutil -suspend / resume command is run

Problem

BUG REPORT: Drives go into Automatic Volume Recognition (AVR) mode after nbrbutil -suspend / resume command is run

Solution

Bug:  1179773 nbrbutil -release/-resume is run by customer on pending request, shortly after it is run customer sees drives go AVR

Symptom(s):

After running the /usr/openv/netbackup/bin/nbrbutil -suspend and/usr/openv/netbackup/bin/nbrbutil -resume commands, "opposite" or non-robotic control host media severs' drives go into Automatic Volume Recognition (AVR) mode. In order to get the drives out of AVR control, it is necessary to stop and restart ltid on both the robotic and non-robotic control hosts (media servers).

Example:

ServerA has robotic control over the "even numbered" libraries.

ServerB has robotic control over the "odd numbered" libraries.

ServerA drives will go AVR for odd numbered robots (controlled by ServerB).

ServerB drives will go AVR for even numbered robots (controlled by ServerA).

The drives on media servers that do not have any robotic control are fine. It is only the drives associated with servers that have robotic control that go AVR. And in more detail, only the drives that are controlled by the opposite robotic control host go AVR.

NOTE: Drive control may be observed in the NetBackup Administration Console under the Media and Device Management -> Device Monitor section.

Workaround:
In order to get the drives out of AVR control, it is necessary to stop and restart ltid on both the robotic and non-robotic control hosts (media servers).

ETA of Fix:
Symantec Corporation has acknowledged that the above-mentioned issue is present in the current version(s) of the product(s) mentioned at the end of this article. Symantec Corporation is committed to product quality and satisfied customers.

This issue is currently being considered by Symantec Corporation to be addressed in a forthcoming Maintenance Pack, Release Update, or version of the product.  The fix for this issue is expected to be released in the first quarter of  2008. Please note that Symantec Corporation reserves the right to remove any fix from the targeted release if it does not pass quality assurance tests or introduces new risks to overall code stability.  Symantec's plans are subject to change and any action taken by you based on the above information or your reliance upon the above information is made at your own risk.  Please refer to the maintenance pack readme or contact NetBackup Enterprise Support to confirm this issue (ET1179773) was included in the maintenance pack.

As future maintenance packs are released, please visit the following link for download and readme information:
 http://www.symantec.com/enterprise/support/downloads.jsp?pid=15143 

Correctly decommission a NBU Media Server and remove it from Env

Problem

How to correctly decommission a NetBackup Media Server and remove it from the NetBackup environment

Solution:

Before removing a media server from the NetBackup environment, several activities must be completed, or restores will require the import of tapes, which is a time consuming process.

These activities include:

• Determining which media need to be related with a new media server
• Updating the internal NetBackup database references to any tapes with active images that are moved to a new media server
• Removing the media server from the NetBackup environment
• Updating all references involving storage units, policies, and volume groups and pools
• Changing the configuration so NetBackup does not continue to recognize the old system as a valid media server

Perform the following steps to decommission a media server:

1. On the media server being decommissioned, determine which tapes have images that are not expired. This is done by running the bpmedialist command with the following options (Note: Use -l to get one line of output per tape):

    bpmedialist -mlist -l -h

2. Select another media server (or the master server, if it is also a media server) to inherit the tapes from the decommissioned media server.  Then the NetBackup internal databases (the mediaDB and the images database) need to be updated to reflect the new media server assignment. Since the mediaDB databases are binary, they can only be updated by the use of the bpmedia command. To update these databases, execute the bpmedia command with the following options for each tape identified in step 1:

    bpmedia -movedb -allvolumes -oldserver -newserver

           

Below command can be used in 7.x if there are many tapes in the media server that needs to be decommissiond and want to move all tapes 

        bpmedia  -movedb  -allvolumes   -oldserver     -newserver  

3. Move all tapes in all robots attached to the decommissioned media server to non-robotic status (standalone). This is done through the NetBackup Administrative Console.  Under Media and Device management, select Media.

 

Next, select the media to be moved, right-click on it, and select Move.

 

In the resulting dialog box, specify Standalone.

 

Multiple media can be selected using the or keys.

4. After moving all tapes in all robots associated with the decommissioned media server to standalone, use the Administrative Console to delete the tape drives and then the robots from the decommissioned media server

 

5. Once the tape drives and robots are deleted, delete all storage units associated with all robots on decommissioned media server through the Administrative Console

6. If any robots from the decommissioned media server are being moved to other media servers, power down the affected servers and make any cabling changes required, and physically attach the robots to the new media servers. Once the robots are recognized by the operating systems on the new media servers, add the robot and tape drives to those media servers through the Administrative Console.  Then, in the NetBackup Administrative Console, create the appropriate storage units. Next, physically load the tapes from the decommissioned media server into the correct robot. Finally, inventory all new robots, or robots with new tapes. This updates NetBackup with thew new location of all tapes in those robots.

7. Modify any policy that explicitly specifies any of the storage units on the decommissioned media server. These policies must point to any other defined storage unit in the NetBackup configuration or to Any Available, as appropriate.

8. Update the bp.conf and vm.conf files on the master server and all media servers (or the servers list on a Windows master or media server) in the NetBackup environment to remove any reference to the decommissioned media server. Also update the server list on all clients to no longer include a reference to the decommissioned media server. Cycle the NetBackup daemons/services on any system where these files are modified.

 

How to create a Catalog Archiving (catarc) policy

Problem

DOCUMENTATION: How to create a Catalog Archiving (catarc) policy

Solution

The following steps can be taken to create a catalog archiving policy. There are several features that are specific to a catalog archiving policy. 

For example:

• The policy must be named catarc.  
• The policy must also be inactive.  
• The catalog archiving commands must be run manually and the administrator must specify a list of images to archive using the bpcatlist command. As a result, there isn't a way to run scheduled catalog archiving backups.

To create the policy in the GUI, do the following:
1. Go to the Policies tab in the GUI and create a new policy
2. For the Policy name: enter catarc
3. For the Policy type: enter Standard or Windows
4. Select the storage unit and volume pool you want to use for catalog archiving.  It is often best to use a pool that is not used for other backups.
5. Clear the Active. Go into effect at: check box. The policy is not run by the scheduler. Rather thebpcatarc command activates this special policy to perform a catalog backup job, then deactivates the policy after the job is done.
6. Accept the defaults for all the other options in the Attributes tab of the policy

Next, create a schedule for the catarc policy.
7. Go to the Schedules tab of the policy
8. Create a new schedule. You can enter any name for the schedule. Only the policy name must becatarc,  but the schedule could also be named catarc.
9. Set the Type of backup: to be User Backup
10. Select a retention level for the catalog archive. This must be at least as long as the longest retention period being archived.
Note: Failure to set the retention level of the catalog archive for a time at least as long as the longest retention period of the backups being archived can result in the loss of catalog data.
11. Accept the defaults for all the other options in the Schedule tab of the policy
12. Go to the Start Window tab in the schedule. Enter a backup window for when the catalog archiving commands can be run by the administrator.

Finally, complete the Backup Selections and Clients tab entries.
13. Go to the Files/Backup Selections tab of the policy
14. Go to the Clients tab of the policy
15. Add the master server to the client list and select the correct hardware/OS type

Save the new policy and exit the GUI. The catarc policy is now available for catalog archiving.
 

How to import disk images into NetBackup 6.x

Problem

DOCUMENTATION: How to import disk images into VERITAS NetBackup (tm) 6.0

Solution

Manual:  VERITAS NetBackup (tm) 6.0 System Administrators Guide for UNIX and Linux, Volume I

Page(s):  266 - 269

Modification Type:  Addition


Modification:
Starting with NetBackup 6.0, disk based images can be imported into NetBackup. The process for importing disk images is essentially the same as importing images from tape.  The NetBackup Administration Console will have additional entries during the "phase one import" to allow selecting a disk path instead of a Media ID.

In order to import disk based images, the images that were written to a disk storage unit must be present on the server.  Copy the disk images to a directory on the server, or restore from a backup that contains the images from a disk storage unit.  Once the files reside in a directory on the server, the NetBackup Administration Console or command line can be used to import the disk images.

Importing NetBackup disk images using the NetBackup Administration Console
First, run the "phase 1 import".
1. Start the NetBackup Administration Console and go to the Catalog section.
2. From the menu select Actions > Initiate Import and a dialog box will appear.
3. Check the "The images to be imported are on disk" radio button and enter a disk path to import images from.
4. Check the Activity Monitor for progress of the phase one import.

Figure 1 shows the new Initialize Import screen.
Figure 1
  

Once the "phase 1 import" completes, run the "phase 2 import".
1. Start the NetBackup Administration Console and go to the Catalog section.
2. Under Pathname:, enter the disk path used for the phase 2 import.
3. Select a date range for the images to import and click the Search button.
4. Select Backup IDs to import from the search results.
5. From the menu select Actions > Import to begin the phase 2 import.
6. Check the Activity Monitor for progress of the phase 2 import.

Once the phase 2 import completes successfully, the disk images will be available for restores.

Importing NetBackup disk images using the command line

The command line utility bpimport can also be used to import images from a disk storage unit.  This will use the "-id" option that was previously used to pass the media ID to use for the import.  However, the "-id" option now supports passing either a disk path or a media ID.

To start a phase 1 import from the command line run the command:

# cd /usr/openv/netbackup/bin/admincmd

# ./bpimport -create_db_info -id -L /usr/tmp/phase1.log

Enter the disk path to use for the import.  Then, monitor the /usr/tmp/phase1.log file to monitor the progress of the phase 1 import.

To start a phase 2 import from the command line run the command:

# cd /usr/openv/netbackup/bin/admincmd

# ./bpimport -id -s -e -L /usr/tmp/phase2.log

Enter the disk path to use for the import as well as the date range for the images to import.  The format to use for the start and end dates is 'MM/DD/YYYY hh:mm:ss'.  Then, monitor the /usr/tmp/phase2.log file to monitor the progress of the phase 2 import.

Once the phase 2 import completes successfully the disk images will be available for restores.

Notes

It will not be possible to import disk images from a NetBackup 5.x server into NetBackup 6.0.  This is due to a difference in the way disk images are handled in NetBackup 6.0.  A phase 1 import of a directory that contains NetBackup 5.x disk images will fail with the message "INF - No importable NetBackup disk images found in path ".  The activity monitor will show a Status 0 for the phase 1 import, even though no disk images are processed.

Imports from a read only filesystem are not supported.  Attempts to import from a read only filesystem will result in an "invalid disk header file" error.

Update NBU after replacing failed drive

When a tape drive is replaced NetBackup configuration will need to be updated to reflect the changed drive:

Eight step procedure needs to be followed:

To swap a shared serialized drive or to update drive firmware on a shared drive:

1) Down the drive. In the Device Monitor, select the drive to swap or update. From the Actions menu, select Down Drive.

2) Replace the drive or physically update the firmware for the drive. If you replace the drive, specify the same SCSI ID for the new drive as the old drive.

3) To produce a list of new and missing hardware, run tpautoconf -report_disc on one of the reconfigured servers. This command scans for  new hardware and produce a report that shows the new and the replaced hardware.

4) Ensure that all servers that share the new hardware are up and that all NetBackup services are active.

5) Run tpautoconf with the -replace_drive -path options or -replace_robot -path options. The tpautoconf command reads the serial number from the new hardware device and then updates the EMM database.

6) If the new device is an unserialized drive, run the device configuration wizard on all servers that share the drive. If the new device is a robot, run the device configuration wizard on the server that is the robot control host.

7) Up the drive. In the Device Monitor, select the new drive. From the Actions menu, select Up Drive.

Here is an example but make sure the drive is "down" prior to running the tpautoconf -replace_drive. If it is not the info could actually revert back to the old drive information:

Once the drive is replaced, run the following command to report the discrepancy:

/usr/openv/volmgr/bin/tpautoconf -report_disc

This produces information similar to the following:

======================= New Device (Tape) ============
Inquiry = "QUANTUM DLT7000         245F"
Serial Number = PXA51S3232
Drive Path = /dev/rmt/21cbn
Found as TLD(0), Drive = 1
===================== Missing Device (Drive) =========
Drive Name = QUANTUMDLT70001
Drive Path = /dev/rmt/11cbn
Inquiry = "QUANTUM DLT7000         245F"
Serial Number = PXA51S3587
TLD(0) definition, Drive = 1
Hosts configured for this device:
Host = HOSTA
Host = HOSTB

This reports the discrepancy between the device database and the new device found. Take note of the new Drive Path for the device as this will be needed for the tpautoconf command.  To resolve this, run:

# cd /usr/openv/volmgr/bin
#./tpautoconf -replace_drive QUANTUMDLT70001 -path /dev/rmt/21cbn

Found a matching device in global DB, QUANTUMDLT70001 on host HOSTA
update of local DB on host HOSTA completed
globalDB update for host HOSTA completed

Found a matching device in global DB, QUANTUMDLT70001 on host HOSTB
update of local DB on host HOSTB completed
globalDB update for host HOSTB completed 

This will update the global and local database to reflect the new device being replaced.

Up the drive at this point.

Below is an example for Windows:

...\Veritas\Volmgr\bin>tpautoconf -report_disc
======================= Missing Device (Drive) =======================
Drive Name = QUANTUM.SDLT320.000
Drive Path = {4,0,2,0}
Inquiry = "QUANTUM SDLT320         5555"
Serial Number = RBF37Y6236
======================= New Device (Drive) =======================
Inquiry = "QUANTUM SDLT320         5555"
Serial Number = RBF37Y6282
Drive Path = Tape0


The new device path syntax for Windows is not "Tape0" as suggested in the output above, but is actually the SCSI coordinates {port, bus, target, lun}, which can be acquired as per the following command:

...\Veritas\Volmgr\bin>tpautoconf -t
TPAC60 QUANTUM SDLT320         5555 RBF37Y6282 4 0 5 0 Tape0 - -

EXAMPLE using the needed device path syntax and observed success statement:

...\Veritas\Volmgr\bin>tpautoconf -replace_drive QUANTUM.SDLT320.000 -path {4,0,5,0}
Found a matching device in global DB, QUANTUM.SDLT320.000 on host nbumedia

8) Restart media manager via command line for the tape drive(s).

UNIX
/usr/openv/volmgr/bin/stoplid            *** Stop media manager
/usr/openv/volmgr/bin/vmps              *** Ensure ltid is stopped
/usr/openv/volmgr/bin/ltid                 *** Start media manager
/usr/openv/volmgr/bin/vmps              *** Ensure ltid is started also with library drive should have tldd process running as well

Windows
...\Veritas\Volmgr\bin>stopltid         *** Stop media manager
...\Veritas\Volmgr\bin>ltid                *** Start media manager     sometimes may require to run "bpdown -v"  and "bpup -v"
...\Veritas\NetBackup\bin>bpps       *** Ensure ltid is started also with library drive should have tldd process running as well


Optionally could stop/start NetBackup on each of the SSO media servers that shares the tape drive or just the media server that has the tape drive if not shared would also work.

 

Hot online catalog backup jobs

Problem

DOCUMENTATION: In NetBackup 6.5, a hot online catalog backup normally consists of one parent job and three child jobs.

Solution

When an online catalog backup is run, it normally generates four jobs on NetBackup 6.5: a parent job, a child job for copying database files to a staging directory, a child job for NetBackup relational database tables, and a child job for catalog images and configuration data.

Here is an example of how a catalog backup might look in the Activity Monitor (Figure 1):

 

In this example:

• JOBID 1 is the parent job.
• JOBID 2 is a child job that makes a temporary copy of database files to a staging directory (/usr/openv/db/staging for UNIX and Linux, \NetBackupDB\staging for Windows)
• JOBID 3 is a child job for NetBackup relational database(s)

Configuration files (server.conf, database.conf, vxdbms.conf)

Database files:

NBDB.db

NBDB.log

EMM_DATA.db

EMM_INDEX.db

If BMR has been installed:

BMRDB.db

BMRDB.log

BMR_DATA.db

BMR_INDEX.db

• JOBID 4 is a child job for image catalog backup.

Note: Transaction logs are truncated after a successful full or incremental backup.
If the transaction logs are manually changed or deleted, there could be a hole in the recovery.

Overview of Catalog Archiving process

Overview of Catalog Archiving process:

The following is an overview of the catalog archiving process. Catalog archiving is meant to be used to reclaim disk space used by the files file of images that backed up many thousands of files and that also have infinite or multi-year retention periods.  This allows the large files files for these images to be removed from disk while the image header files remain in the image catalog. It should not be used as a method to reclaim disk space when a catalog filesystem fills up. For those cases, investigate catalog compression or adding disk space and growing the filesystem.

Warning: Catalog archiving modifies existing catalog images. As a result, it should never be run when the catalog filesystem is 100% full. Entries are added to either the header files in/usr/openv/netbackup/db/images (pre-NB 7.5) or into the image records in the Sybase NBDB (post-NB 7.5).  If the filesystem is at 100%, it is impossible to predict what would happen.

Note: There is no simple method to determine what tape the catalog has been archived to. Thebpcatlist -offline command is the only administrative command to determine what images have been archived. This command does not list what tape was used for the archive. As a result, caution should be exercised to ensure the tapes used for catalog archiving are available for restoring the archived catalog images. Either create a seperate volume pool to use exclusively for catalog archives or find a method to label the tape as a catalog archive tape.

Step 1: Use bpcatlist to determine what image files will be archived.

Before attempting to run bpcatarc or bpcatrm use the bpcatlist command to display what images are available for archiving. Running bpcatlist alone will not modify any catalog images. Only when thebpcatlist output is piped to bpcatarc will the files files be backed up, and only when the output is piped to bpcatrm will the the files files be deleted from disk.

To determine what images have files files on disk that can be archived, run the following command.  The catarcid column will indicate whether the files file is not currently backed up (0) or the catarcid of the backup of that image.

# /usr/openv/netbackup/bin/admincmd/bpcatlist -online

To determine what images have been previously archived and removed from disk, run the following command.

# /usr/openv/netbackup/bin/admincmd/bpcatlist -offline

Note: This should return "no entity was found" if catalog archiving has not been previously run.  For more information on what the fields in the bpcatlist output indicate, refer to TECH36412 in the Related Articles section.

To display all images for a specific client prior to January 1st, 2000 run this command.

# bpcatlist -client -before Jan 1 2000

To display the help for the bpcatlist command run this command.

# bpcatlist -help

Once the bpcatlist output correctly lists all the images that are to be archived or deleted, then those commands can be added.

Step 2: Running the catalog archive.

Before running the catalog archive, create a catarc policy. This is required in order for the bpcatarccommand to successfully process images. Refer to TECH36434 in the Related Articles section for more details on creating a catarc policy.

To run the catalog archive, run the bpcatlist command with the same options used in step 1 to display images. Then just pipe the output through bpcatarc and typically bpcatrm.

# bpcatlist -client all -before Jan 1 2000 | bpcatarc | bpcatrm

A new job will appear in the Activity Monitor. The command will wait until the backup completes before returning the prompt. It will report an error only if the catalog archive fails. Otherwise the commands will simply return to the prompt. The File List: section of the Job Details in the Activity Monitor will show a list of image files that have been processed. When the job completes with a status 0, the bpcatrm will remove the corresponding .f files. If the job fails, then no catalog .f files will be removed.

If bpcatlist is piped to bpcatarc but the results is not piped to bpcatrm, then the backup will occur but the files files will not be removed from disk.  The same bpcatlist command can then be rerun and piped tobpcatrm to remove the files files.

Step 3: Restoring the Catalog archive

To restore the catalog archive, you must first use the bpcatlist command to list the files that need to be restored. Once bpcatlist displays the proper files to restore, then the bpcatres can be run to restore the actual files.

To restore all the archived files from Step 2 above, run the command:
# bpcatlist -client all -before Jan 1 2000 | bpcatres

This will restore all the catalog archive files prior to Jan 1, 2000.

Some miscellaneous notes about catalog archiving:

In the /usr/openv/netbackup/db/images// directory, a header and files file will be created for the catalog archive job.
The header file will be named: catarc__UBAK 
The files file will be named: catarc__UBAK.f

Do not attempt to archive the catarc__UBAK image files. These are not archived by default. Attempting to archive catalog archives will make it nearly impossible to determine what files are needed for restoring catalog entries. The catarc__UBAK image files contain a listing of what catalog images were archived and need to be present and intact on the master in order to do a catalog restore.

Note: Starting with NetBackup 7.5 the image header files are stored in the Sybase NBDB and are no longer present on disk.  Because the files files can be very large, they still exist on disk unless archived.

The catalog archive images will also appear in the Backup, Archive and Restore GUI. The catarcpolicy is of type Standard so it will display the catalog archive backups along with the regular filesystem backups. However, this is not the correct method to restore archived files. Catalog archive files should be restored using the bpcatlist | bpcatres commands.

Warning: Running bpcatlist | bpcatarc | bpcatrm without any options will archive the entire NetBackup catalog. To recover from this, run bpcatlist | bpcatres to restore all archived images. Then work with the bpcatlist command to determine what options are needed to archive only the desired images.

Some recommendations for catalog archiving:
Perform catalog archiving operations only on images that are not currently be operated upon by other NetBackup operations such as duplications, vaulting, or Storage LifeCycle Policies.  Use appropriate options on the bpcatlist command to select only appropriate images..

To ensure that catalog backup images are not on the same tapes as user backups, create a separate media pool for catalog archives.

You may find it useful to set up, then designate, a special retention level for catalog archive images. To specify retention levels, go to Host Properties | Master Server | Retention Periods.

Drive Status Information

The following NetBackup Drive statuses can appear in command line output or in the Device Monitor in the NetBackup Administration Console:

Column Description Note for the Administration Console:
Drive Name - Drive name assigned to the drive during configuration.
Control -  Control mode for the drive can be any of the following: robot_designation. For example, TLD.
The robotic daemon managing the drive has connected to LTID (the device daemon and Device Manager service) and is running. The drive is in the usable state. AVR is assumed to be active for the drive, as all robotic drives must be in automatic volume recognition (AVR) mode (not OPR mode).
Applies only to robotic drives.

DOWN- 

For example, DOWN-TLD.

The drive is in an usable state because it was downed by an operator or by NetBackup; or when the drive was configured, it was added as a down drive. Applies only to robotic drives.

DOWN

In this mode, the drive is not available to Media Manager. 

Applies only to standalone drives.

A drive can be in a DOWN mode because of problems or because it was set to that mode usingActions | Down Drive.

PEND-

For example, PEND-TLD. 

The drive is in a pending status.  Applies only to robotic drives.

PEND

The drive is in a pending status.  Applies only to standalone drives. 

If the drive reports a SCSI RESERVATION CONFLICT status, this column will show PEND. This status means that the drive is reserved when it should not be reserved. Some server operating systems (Windows, Tru64, and HP-UX) may report PEND if the drive reports Busy when opened. You can use the AVRD_PEND_DELAY entry in the Media Manager configuration file to filter out these reports.

AVR

The drive is running with automatic volume recognition enabled.

The drive is in a usable state with automatic volume recognition enabled, but the robotic daemon managing the drive is not connected or is not  working. Automated media mounts do not occur with a drive in this state (unless the media is in a drive on the system, or, this is a standalone tape drive), but the operator can physically mount a tape in the drive or use robtest to cause a tape mount as needed.

OPR

The drive is running in a secure mode where operators must manually assign mount requests to the drive. AVRD is not scanning

this drive when in this mode. This mode gives operators control over which mount requests can be satisfied.

Applies only to standalone drives.

NO-SCAN

A drive is configured for shared storage option (SSO), but has no available scan host (to be considered available, a host must register with a SSO_SCAN_ABILITY factor of non-zero and have the drive in the UP state). NO-SCAN may be caused if all available scan hosts have the drive in the DOWN state. Other hosts (that are not scan hosts) may want to use the drive, but they registered with a scan factor of zero. The drive is unusable by NetBackup until a scan host can be assigned.

Mixed

The control mode for a shared drive may not be the same on all hosts sharing the drive. For shared drives, each host can have a

different status for the drive. If the control modes are all the same, that mode is displayed.

RESTART

The control mode for a shared drive may not be the same on all hosts sharing the drive. This status indicates that ltid needs to be restarted. To determine what server need to be restarted, right-click the drive in the device monitor and select up. This will tell you what servers that ltid needs to be restarted.

Steps to rebuild the /dev/sg/* and /dev/rmt/* devices on a Solaris Master or Media server

Steps to rebuild the /dev/sg/* and /dev/rmt/* devices on a Solaris server:

1.  Create a backup copy of the current st.conf file:
# cp /kernel/drv/st.conf /kernel/drv/st.conf.`date +%m%d%y_%H%M%S`

2.  Move the existing sg.conf to a backup (this must be a move, otherwise a later step will fail):
# mv /kernel/drv/sg.conf /kernel/drv/sg.conf.`date +%m%d%y_%H%M%S`

3.  Create a backup copy of the current devlink.tab file:
# cp /etc/devlink.tab /etc/devlink.tab.`date +%m%d%y_%H%M%S`

4.  Delete SCSI targets/LUNs from the /kernel/drv/st.conf file:
name="st" class="scsi"
       target=0 lun=0;

All of these entries should be removed, otherwise duplicates will be added later.

5.  Delete SCSI targets/LUNs from /etc/devlink.tab.  This is typically the section near the end of the file and the entries are typically of the form:

# begin SCSA Generic devlinks file - creates nodes in /dev/sg
type=ddi_pseudo;name=sg;addr=0,0;       sg/c\N0t0l0
type=ddi_pseudo;name=sg;addr=1,0;       sg/c\N0t1l0
type=ddi_pseudo;name=sg;addr=2,0;       sg/c\N0t2l0
type=ddi_pseudo;name=sg;addr=3,0;       sg/c\N0t3l0
type=ddi_pseudo;name=sg;addr=4,0;       sg/c\N0t4l0
type=ddi_pseudo;name=sg;addr=5,0;       sg/c\N0t5l0
type=ddi_pseudo;name=sg;addr=6,0;       sg/c\N0t6l0
type=ddi_pseudo;name=sg;addr=0,1;       sg/c\N0t0l1
type=ddi_pseudo;name=sg;addr=1,1;       sg/c\N0t1l1
type=ddi_pseudo;name=sg;addr=2,1;       sg/c\N0t2l1
type=ddi_pseudo;name=sg;addr=3,1;       sg/c\N0t3l1
type=ddi_pseudo;name=sg;addr=4,1;       sg/c\N0t4l1
type=ddi_pseudo;name=sg;addr=5,1;       sg/c\N0t5l1
type=ddi_pseudo;name=sg;addr=6,1;       sg/c\N0t6l1
# end SCSA devlinks

Everything in this section should be removed, inclusive of the beginning and ending lines.

6.  Change to the appropriate directory to run commands:
# cd /usr/openv/volmgr/bin/driver

7.  Generate the configuration files (st.conf, sg.conf and sg.links):
../sg.build all -mt -ml

Note: You will need to know what the max_target and max_lun values will need to be (this is the maximum SCSI Target and LUN value). For example, to create entries for lun-id 0 through 4, use "-ml4".

8.  Append the generated st.conf entries to the OS configuration file:
# cat st.conf >> /kernel/drv/st.conf

9.  Unload the sg driver:
# rem_drv sg

Check that this command has been successful. If the following message is received then Step 10 will not successful

"device busy cannot unload module sg will be unloaded upon reboot"

running the command

rem_drv sg

a second time has been known to succeed and you can proceed to step 10.

Note: Unloading the sg driver may impact NetBackup performance.

10. Use the provided script to re-create the /kernel/drv/sg.conf file, append the SCSA entries to/etc/devlink.tab and reload the sg driver:
# ./sg.install

11.  Reboot the server

12. Now sgscan should see the appropriate devices:
# /usr/openv/volmgr/bin/sgscan all conf -v

Configure NetBackup Resource Broker (nbrb) to improve resource allocation performance

In certain environments, the time it takes the NetBackup Resource Broker (nbrb) to execute through its queue of jobs waiting for resources can limit overall system performance. This may occur in situations where many jobs are queued for resources and the jobs are completing faster than nbrb's ability to re-use the released resources for the queued jobs.

Beginning in the NetBackup 6.5.2 Release Update, the nbrb.conf can be utilized to improve the performance of nbrb.  nbrb.conf is located in:
UNIX: /usr/openv/var/global/nbrb.conf
Windows: \Veritas\NetBackup\var\global\nbrb.conf

This file offers the following parameters:

• SECONDS_FOR_EVAL_LOOP_RELEASE
• RESPECT_REQUEST_PRIORITY
• DO_INTERMITTENT_UNLOADS

Note: By default all values are 0.

Example format for parameters in nbrb.conf:
SECONDS_FOR_EVAL_LOOP_RELEASE = 180
RESPECT_REQUEST_PRIORITY = 0
DO_INTERMITTENT_UNLOADS = 1

Explanation of nbrb.conf parameters:

SECONDS_FOR_EVAL_LOOP_RELEASE

Default value is 0 if not present in nbrb.conf.  If the value is 0, nbrb reverts to normal default behavior, evaluating all queued job requests before releasing any drives that have been released by completing jobs.

When set to a nonzero value, SECONDS_FOR_EVAL_LOOP_RELEASE controls the time interval after which nbrb will break into its evaluation cycle and release drives that have been given up by completed jobs, making them available for use by other jobs.

RESPECT_REQUEST_PRIORITY

Default value is 0 if not present in nbrb.conf; only has effect ifSECONDS_FOR_EVAL_LOOP_RELEASE is set to a nonzero value.

When set to a nonzero value, nbrb will restart its evaluation queue at the top of the prioritized job queue after resources have been released due to the SECONDS_FOR_EVAL_LOOP_RELEASEtime interval having passed during the nbrb evaluation cycle.

If set to 0, causes NBRB to continue evaluating jobs in the prioritized job queue at the point where the evaluation cycle was interrupted for drive releases due toSECONDS_FOR_EVAL_LOOP_RELEASE interval.

This mode of operation may make it more likely for a job to reuse a drive more quickly after the drive has been released, at the expense of allowing some lower priority jobs to get drives before higher priority jobs.

DO_INTERMITTENT_UNLOADS

Default value is 0 if not present in nbrb.conf; only has effect ifSECONDS_FOR_EVAL_LOOP_RELEASE is set to a nonzero value.

When set to a nonzero value, nbrb will initiate unloads of drives that have exceeded the media unload delay when resources are released due to the SECONDS_FOR_EVAL_LOOP_RELEASEtime interval having passed during the nbrb evaluation cycle.

This will make drives available more quickly to jobs that require different media servers or different media than the job that last used the drive.

This is achieved at the expense of not having the loaded media/drive pair available for jobs further down in the prioritized evaluation queue that could use the drive/media without unload.

Notes:
-  If nbrb.conf does not exist, or if parameters are not set in the file, then those parameters take on their default values (0), and nbrb will behave as it normally does without these configuration parameters.
-  The addition or modification of the nbrb.conf file does not require a recycle of the NetBackup processes.  The process will read this file at the start of every evaluation cycle, and changes of any type will be implemented at that time.

Recommended cache settings for the Sybase ASA database within NetBackup 6.0

Recommended cache settings for Sybase ASA:

A good general rule is to keep the initial and minimum ASA database cache equal to or more than at least 10% of the total database size.

If a message such as the following is seen in server.log, then the cache (25M by default) is less than 10% of the total database file size.  Note that this figure includes the transaction logs:
I. 09/25 12:11:10. Note: Server cache size is too small for database "NBDB"

Therefore, if this message appears, it may serve as an indication that the database cache settings need to be changed.  Catalog backups should be run regularly to truncate the transaction logs.  However, cache settings should be based upon the size of the actual database, not including the transaction logs.

Server.log is in the following location:
UNIX: /usr/openv/db/log/server.log
Windows: \VERITAS\NetBackupDB\log\server.log

Determining the size of the database
By default, the database files are in the following location:

UNIX: /usr/openv/db/data
Windows: InstallPath\VERITAS\NetBackupDB\data

alfred% cd /usr/openv/db/data
alfred% ls
EMM_DATA.db   EMM_INDEX.db  NBDB.db       NBDB.log      vxdbms.conf
alfred% ls -al
total 108294
drwxr-xr-x   2 root     bin          512 Oct 23 14:59 .
drwxr-xr-x  11 root     bin          512 Oct 18 17:58 ..
-r--------   1 root     other    26226688 Oct 23 16:07 EMM_DATA.db
-r--------   1 root     other    26226688 Oct 23 16:07 EMM_INDEX.db
-r--r--r--   1 root     other    2453504 Oct 23 16:07 NBDB.db
-r--------   1 root     other     458752 Oct 23 16:07 NBDB.log
-rw-------   1 root     bin          362 Oct 23 15:14 vxdbms.conf

Add up the total byte count for the following files:

EMM_DATA.db

EMM_INDEX.db

NBDB.db

If Bare Metal Restore (BMR) is also installed, include the following files in the calculation:

BMR_DATA.db 

BMR_INDEX.db

BMRDB.db

Once the files have been totalled, then allocate at least 10% of this amount to the initial and minimumcache (do not include .log files in this calculation).

If the database files are not in the default location, look in vxdbms.conf to find them.

alfred # more v*.conf

VXDBMS_NB_SERVER = VERITAS_NB_alfred.min.veritas.com

VXDBMS_NB_PORT = 13785

VXDBMS_NB_DATABASE = NBDB

VXDBMS_NB_DATA = /usr/openv/db/data

VXDBMS_NB_INDEX = /usr/openv/db/data

VXDBMS_NB_TLOG = /usr/openv/db/data

VXDBMS_NB_PASSWORD = 2371d950c797cb6cc83064e619715c908381bc9fc814d836

VXDBMS_NB_FULL_KEYWORD = NBDB:49:1161632832:F

VXDBMS_NB_INCREMENTAL = NBDB.log.2

The location of vxdbms.conf itself is given in the bp.conf file in UNIX:

alfred # more /bp/bp.conf

SERVER = alfred.min.veritas.com

CLIENT_NAME = alfred.min.veritas.com

EMMSERVER = alfred.min.veritas.com

VXDBMS_NB_DATA = /usr/openv/db/data

VERBOSE = 5

In Windows, the location of vxdbms.conf is given in the registry at the following location: 

HKEY_LOCAL_MACHINE\SOFTWARE\VERITAS\NetBackup\CurrentVersion\Config\VXDBMS_NB_DATA

Configuring the Database cache settings
Cache settings can be changed by editing the server.conf file, kept in the following location:

UNIX: /usr/openv/var/global/server.conf
Windows: \VERITAS\NetBackupDB\conf\server.conf 

The System Administrator's Guide notes that the customer should only change this file with the help of Support.

There are three parameters related to the amount of memory in megabytes. The default settings are as follows:

-c 25M

Indicates the initial memory reserved for caching database pages and other server information.  This should be adjusted in accordance with the calculation mentioned above.

-cl 25M

Indicates minimum cache size, as a limit to automatic cache resizing.  This should be adjusted in accordance with the calculation mentioned above.

-ch 500M

Indicates the maximum cache size, as a limit to automatic cache growth.

Once the server.conf is edited, NetBackup Services should be restarted on the EMM server.  Restart these services only when no jobs are active.

Backups staying queued for hours before going ACTIVE

Overview:

Jobs remain queued while resources are available due to length of time it takes nbrb to complete an evaluation cycle.

Troubleshooting/Log Files:
Determine if there is an excessive amount of time taken between nbjm requesting a resource andnbrb granting the resource.

For example:
1. From the Activity Monitor within the NetBackup GUI, view the job details.  In this particular instance, it took 10 minutes to grant a resource:
8/21/2007 2:58:44 PM - requesting resource figrin.NBU_POLICY.MAXJOBS.sybase-arvel1-SDS_SY48-sybsecurity-DB 
8/21/2007 3:08:31 PM - granted resource figrin.NBU_CLIENT.MAXJOBS.arvel1 

2. In the nbrb log, look for the amount of time between "evaluation cycle is in progress" and "reserved resource" for a particular jobid :
08/21/07 20:58:13.512 [Debug] NB 51216 nbrb 118 PID:16608 TID:3 File ID:118 [jobid=9180586] 2 [ResBroker_i::evaluateNow] evaluation cycle is in progress
08/21/07 21:08:30.706 [Debug] NB 51216 nbrb 118 PID:16608 TID:9 File ID:118 [jobid=9180586] 3 [CountedProvider::allocate] reserved resource figrin.NBU_CLIENT.MAXJOBS.arvel1 (current_ref_count=2, max_ref_count=10)

To generate an nbrb log, use this command:
/usr/openv/netbackup/bin/vxlogview -p 51216 -o 118 -d all

3. Search the raw nbrb log (/usr/openv/logs/*118*) for the "msec" string:
For example:
51216-118-1067800690-070912-0000000011.log:0,51216,118,118,34289634,1189573603242,17599,9,0:,83:Resource evaluation completed. Evaluated 779 requests, evaluation time: 481417 msec,25:ResBroker_i::doEvaluation,2 
51216-118-1067800690-070912-0000000024.log:0,51216,118,118,34308513,1189574183188,17599,10,0:,83:Resource evaluation completed. Evaluated 763 requests, evaluation time: 579944 msec,25:ResBroker_i::doEvaluation,2 
51216-118-1067800690-070912-0000000040.log:0,51216,118,118,34327216,1189574666005,17599,9,0:,83:Resource evaluation completed. Evaluated 916 requests, evaluation time: 482815 msec,25:ResBroker_i::doEvaluation,2 

Resolution:
There are a number of steps that were taken to increase the time taken to grant resources:

Note: All configuration files listed below can be found in the following subdirectory:
UNIX: /usr/openv/var/global
Windows: \Veritas\NetBackup\var\global\

1. If the master server has greater than ten CPUs, it is necessary to increase the number of CPUs available to Sybase.  See TechNote 285579, linked below.

2. Enable additional database connections via emm.conf changes.

For example, the following lines could be added or edited:

NUM_DB_BROWSE_CONNECTIONS=10

NUM_DB_CONNECTIONS=11

NUM_ORB_THREADS=21

Note: In NetBackup 6.0, the server.conf file must also be changed to allow greater than 10 connections to the ASA database.  This can be accomplished by editing the server.conf file and changing the value of the -gn parameter to 20.

In NetBackup 6.5, the -gn parameter does not exist.

3.  Starting in NetBackup 6.0 MP6 and 6.5.2, nbrb can be tuned via the nbrb.conf file.

This file offers the following parameters:

• SECONDS_FOR_EVAL_LOOP_RELEASE
• RESPECT_REQUEST_PRIORITY
• DO_INTERMITTENT_UNLOADS

Note: By default all values are 0.

Example format for parameters in nbrb.conf:
SECONDS_FOR_EVAL_LOOP_RELEASE = 180
RESPECT_REQUEST_PRIORITY = 0
DO_INTERMITTENT_UNLOADS = 1

An explanation of nbrb.conf parameters can be found in TechNote 300442, linked below.

4. VERY IMPORTANT for nbrb caching: do not mix SSO and dedicated drives within a library.  Have all drives SSO or all drives dedicated.

5. Do not exceed the total number of drives within a storage unit (max concurrent drives).
For example: if the library has 8 drives, the largest value that total max concurrent drives can be is 8.

6. Ensure drives do not need cleaning. (CLEANING REQUIRED flag flipped). Clean drives. Runtpclean -M on all drives to clear flag if still set.

7. If using custom scripts or Aptare, ensure that scripts are not accessing media servers for drive/tape information. All information is now stored on the EMM server.

8. Change HPUX kernel by increasing dbc_max_pct to 50 and dbc_min_pct to 10  to improve file caching.

9. Move the database transaction log and database to different disks:
For example:
If 2 additional drives (/u01 and /u02) are available:
# mkdir /u01/nbu 
# mkdir /u02/nbu 
# /usr/openv/netbackup/bin/goodies/netbackup stop
# /usr/openv/db/bin/nbdbms_start_server 
# /usr/openv/db/bin/nbdb_move -data /u01/nbu -index /u01/nbu -tlog /u02/nbu 
# /usr/openv/netbackup/bin/goodies/netbackup start

The NBDB database transaction log (NBDB.log) may become too large or corrupt for proper NetBackup operation.

Problem

Problem occurs when NetBackup Relational Database Manager service failed to start. When running following command shows Database unavailable.

usr/openv/db/bin/nbdb_ping
Database [NBDB] is not available.

Error

There may be errors reported by the ASA database engine in the /usr/openv/db/log/server.log file as well. These errors would look similar to the following:Error: Cannot open transaction log file -- Can't use log file "/usr/openv/db/data/NBDB.log" since it is shorter than expected

Cause

Problem occurs when transaction log for the ASA database that runs under NetBackup to become corrupt or too large for proper operation. For example, in a file system that limits file size to 2GB, the transaction log will be truncated and corrupt once it reaches the 2Gb limit. The transaction log is truncated during online catalog backup, but it might be possible that catalog backups did not take place to prevent the log from growing too large.

The ASA database failing due to transaction log problems can include these errors:

- Catalog backups and regular backups can hang.

- Operator is unable to cancel queued jobs.

- The EMM database does not come back up after restarting NetBackup

Solution

Rebuild the transaction log via the following steps:-

1. Stop NetBackup if it is not already stopped:/usr/openv/netbackup/bin/goodies/netbackup stop

 2. Remove NBDB and BMRDB from auto_start option

/usr/openv/db/bin/nbdb_admin -auto_start NONE

3. Start the ASA database engine/usr/openv/db/bin/nbdbms_start_server

4. To add the necessary environment variables to your shell, run the command: /usr/openv/db/vxdbms_env.sh

Note 1: please make note of the period at the beginning of the line.
 

Note 2: if using 'csh' for a shell, the command is 'source /usr/openv/db/vxdbms_env.csh' without the quotes.

Note 3: depending on the root user's shell environment, this command could fail with a "Paremeter not set" or "unbound variable" error.  If one of these errors occurs, run the "set +u" command and then reattempt the ". /usr/openv/db/vxdbms_env.sh" command.
 

5. Change directories to where the NBDB resides:
 cd /usr/openv/db/data 

  

6. Remove or rename the bad transaction log:
 mv NBDB.log NBDB.log.bad

If BMR is used, also rename the BMR transaction log:

 mv BMRDB.log BMRDB.log.bad

 7. To force database recovery, while still in the /usr/openv/db/data directory, run the command (please note, the NBDB.log will not be created until step 10):

/usr/openv/db/bin/dbeng9 -f /usr/openv/db/data/NBDB.db

If BMR is used , force recovery of the BMR database:
/usr/openv/db/bin/dbeng9 -f /usr/openv/db/data/BMRDB.db

(Note: for NetBackup 7.0.1,  it is running on ASA11.  This command is :  /usr/openv/db/bin/dbeng11 -f NBDB  or  /usr/openv/db/bin/dbeng11 -f BMRDB) 

NOTE: If the dbeng11 -f NBDB should fail with the error "A database server with that name has already started", please check /usr/openv/tmp and move the sqlany directory to different name and all the sqlany files to different names and restart the database server on step 3 above and the dbeng11 -f NBDB should work.

At this point of recovery, no transaction log will be created just yet until the database is opened.

8. Stop the database by running the command: /usr/openv/db/bin/nbdbms_start_server -stop

9. Add NBDB and BMRDB to auto_start option (Add BMRDB only if using BMR)

/usr/openv/db/bin/nbdb_admin -auto_start NBDB

/usr/openv/db/bin/nbdb_admin -auto_start BMRDB

10. To start the database, run the command: 

/usr/openv/db/bin/nbdbms_start_server
 

11. To test if the database is operational, run the command:

/usr/openv/db/bin/nbdb_ping

which should return something similar to the following:
 

Database [NBDB] is alive and well on server [NB_master1].