Friday, May 6, 2011

How to Collect Solaris OS coredump for analysis

There will be situation when you might need to collect OS kernel core dumps for Sun support analysis. The procedure is below.


1. login to the Service Processor (root or admin)

2 send a break signal to the console

ILOM syntax: -> set /HOST send_break_action=dumpcore

ALOM syntax: sc> break -D

NOTE: the above commands will send a break signal and create
a coredump. Make sure you have enough disk space available
in /var or whatever directory you have dumpadm configured to
capture a corefile

3. connect to the system console after executing above "break"
command, and if Solaris OE boots, check for corefile and upload
to Oracle for analysis by a Kernel TSE

ILOM syntax: -> start /SP/console

ALOM syntax: sc> console -f

Friday, April 29, 2011

How to clear and delete "waiting for retry" status 50 job?



Are you ever wondered by seeing jobs in Netbackup java console which has state "Waiting for Retry" and status 50? If you try to delete it... ooops its not deleting :) What we will do???


This situation will come if the client process aborted. The solution is to stop the netbackup master services. And run:

/usr/openv/netbackup/bin/bpjobd -r


Once you cleared all the jobids start the netbackup services. You are back to buisiness...








Wednesday, March 16, 2011

Replacing a Failed Tape drive in netbackup

If you replace a tape drive attached to solaris media server some time it wont recognise the new drive and will endup in misconfiguration in media device databsase. This procedure is helpfull if you replacing failed Tape drive which attached to Solaris media server.

1. Remove the faulty tape drive from library
2. Login to the media server as root and identify which controller the removed tape drive was attached using cfgadm, you can identify using the condition failing

# cfgadm -al
Ap_Id Type Receptacle Occupant Condition
c0 scsi-bus connected configured unknown
c0::dsk/c0t0d0 disk connected configured unknown
c0::dsk/c0t1d0 disk connected configured unknown
c0::sg/c0t0l0 unknown connected configured unknown
c0::sg/c0t1l0 unknown connected configured unknown
c1 scsi-bus connected configured unknown
c1::dsk/c1t0d0 CD-ROM connected configured unknown
c2 fc connected unconfigured unknown
c3 fc-private connected configured unknown
c3::50060e801029f140 disk connected configured unknown
c5 fc connected unconfigured unknown
c6 fc-fabric connected configured unknown
c6::500104f0008e3031 med-changer connected configured unknown
c6::500104f0008e3041 tape connected configured unknown
c7 fc-fabric connected configured unknown
c7::500104f0008e3044 tape connected configured unknown
c7::500104f0008e3047 tape connected configured unknown
c8 fc-fabric connected configured unknown
c8::500104f0008e304a tape connected configured unknown
c8::500104f0008e304d tape connected configured unknown
c9 fc-fabric connected configured unknown
c9::500104f0008e3050 tape connected configured unknown
c9::500104f0008e3053 unavailable connected configured failing
c10 fc-private connected unconfigured unknown
usb0/1 unknown empty unconfigured ok
usb0/2 unknown empty unconfigured ok
usb1/1.1 unknown empty unconfigured ok
usb1/1.2 unknown empty unconfigured ok
usb1/1.3 unknown empty unconfigured ok
usb1/1.4 unknown empty unconfigured ok
usb1/2 unknown empty unconfigured ok

3. Shake controller which has the failed tape drive
#/usr/sbin/cfgadm -c configure c9

4. Do a cleanup of the device files (This will leanup the device files of removed tape drive.
# devfsadm -C -c tape -v
devfsadm[9600]: verbose: removing file: /dev/rmt/4
devfsadm[9600]: verbose: removing file: /dev/rmt/4n
devfsadm[9600]: verbose: removing file: /dev/rmt/4b
devfsadm[9600]: verbose: removing file: /dev/rmt/4bn
devfsadm[9600]: verbose: removing file: /dev/rmt/4l
devfsadm[9600]: verbose: removing file: /dev/rmt/4m
devfsadm[9600]: verbose: removing file: /dev/rmt/4h
devfsadm[9600]: verbose: removing file: /dev/rmt/4c
devfsadm[9600]: verbose: removing file: /dev/rmt/4u
devfsadm[9600]: verbose: removing file: /dev/rmt/4ln
devfsadm[9600]: verbose: removing file: /dev/rmt/4mn
devfsadm[9600]: verbose: removing file: /dev/rmt/4hn
devfsadm[9600]: verbose: removing file: /dev/rmt/4cn
devfsadm[9600]: verbose: removing file: /dev/rmt/4un
devfsadm[9600]: verbose: removing file: /dev/rmt/4lb
devfsadm[9600]: verbose: removing file: /dev/rmt/4mb
devfsadm[9600]: verbose: removing file: /dev/rmt/4hb
devfsadm[9600]: verbose: removing file: /dev/rmt/4cb
devfsadm[9600]: verbose: removing file: /dev/rmt/4ub
devfsadm[9600]: verbose: removing file: /dev/rmt/4lbn
devfsadm[9600]: verbose: removing file: /dev/rmt/4mbn
devfsadm[9600]: verbose: removing file: /dev/rmt/4hbn
devfsadm[9600]: verbose: removing file: /dev/rmt/4cbn
devfsadm[9600]: verbose: removing file: /dev/rmt/4ubn
devfsadm[9600]: verbose: removing file: /dev/rmt/5
devfsadm[9600]: verbose: removing file: /dev/rmt/5n
devfsadm[9600]: verbose: removing file: /dev/rmt/5b
devfsadm[9600]: verbose: removing file: /dev/rmt/5bn
devfsadm[9600]: verbose: removing file: /dev/rmt/5l
devfsadm[9600]: verbose: removing file: /dev/rmt/5m
devfsadm[9600]: verbose: removing file: /dev/rmt/5h
devfsadm[9600]: verbose: removing file: /dev/rmt/5c
devfsadm[9600]: verbose: removing file: /dev/rmt/5u
devfsadm[9600]: verbose: removing file: /dev/rmt/5ln
devfsadm[9600]: verbose: removing file: /dev/rmt/5mn
devfsadm[9600]: verbose: removing file: /dev/rmt/5hn
devfsadm[9600]: verbose: removing file: /dev/rmt/5cn
devfsadm[9600]: verbose: removing file: /dev/rmt/5un
devfsadm[9600]: verbose: removing file: /dev/rmt/5lb
devfsadm[9600]: verbose: removing file: /dev/rmt/5mb
devfsadm[9600]: verbose: removing file: /dev/rmt/5hb
devfsadm[9600]: verbose: removing file: /dev/rmt/5cb
devfsadm[9600]: verbose: removing file: /dev/rmt/5ub
devfsadm[9600]: verbose: removing file: /dev/rmt/5lbn
devfsadm[9600]: verbose: removing file: /dev/rmt/5mbn
devfsadm[9600]: verbose: removing file: /dev/rmt/5hbn
devfsadm[9600]: verbose: removing file: /dev/rmt/5cbn
devfsadm[9600]: verbose: removing file: /dev/rmt/5ubn
devfsadm[9600]: verbose: removing file: /dev/rmt/7
devfsadm[9600]: verbose: removing file: /dev/rmt/7n
devfsadm[9600]: verbose: removing file: /dev/rmt/7b
devfsadm[9600]: verbose: removing file: /dev/rmt/7bn
devfsadm[9600]: verbose: removing file: /dev/rmt/7l
devfsadm[9600]: verbose: removing file: /dev/rmt/7m
devfsadm[9600]: verbose: removing file: /dev/rmt/7h
devfsadm[9600]: verbose: removing file: /dev/rmt/7c
devfsadm[9600]: verbose: removing file: /dev/rmt/7u
devfsadm[9600]: verbose: removing file: /dev/rmt/7ln
devfsadm[9600]: verbose: removing file: /dev/rmt/7mn
devfsadm[9600]: verbose: removing file: /dev/rmt/7hn
devfsadm[9600]: verbose: removing file: /dev/rmt/7cn
devfsadm[9600]: verbose: removing file: /dev/rmt/7un
devfsadm[9600]: verbose: removing file: /dev/rmt/7lb
devfsadm[9600]: verbose: removing file: /dev/rmt/7mb
devfsadm[9600]: verbose: removing file: /dev/rmt/7hb
devfsadm[9600]: verbose: removing file: /dev/rmt/7cb
devfsadm[9600]: verbose: removing file: /dev/rmt/7ub
devfsadm[9600]: verbose: removing file: /dev/rmt/7lbn
devfsadm[9600]: verbose: removing file: /dev/rmt/7mbn
devfsadm[9600]: verbose: removing file: /dev/rmt/7hbn
devfsadm[9600]: verbose: removing file: /dev/rmt/7cbn
devfsadm[9600]: verbose: removing file: /dev/rmt/7ubn

5. Fix the drive and shake the controller:

#/usr/sbin/cfgadm -c configure c9

6. Check device files created for new drives, if not run a devfsadm and check

#ls -lrt /dev/rmt


7. if its created UP the drive in console and verify the backup