Apg40 C4 Alexserv

OPERATIONAL INSTRUCTION

36/154 31-ANZ 222 03 Uen F

© Ericsson AB 2006 - all rights reserved

AP, System Disaster Recovery APG40C/4 Windows Server 2003 Contents 1 1.1

Introduction Scope

2 2.1 2.2

Procedure Prerequisites Actions

3

Additional Information

4

Glossary

5 5.1 5.2

References Operational Instructions User Guides

1 Introduction 1.1 Scope This Operational Instruction describes the procedure to restore the system disks on one or both nodes in an APG40 Adjunct Processor (AP) that is not possible to boot. Hardware related errors are not covered in this Operational Instruction, If you cannot determine if the problem with the system disk is Hardware related or software then please contact next level of maintenance support before proceeding with the Procedure. All steps in the procedure must be performed on site. Note: WARNING! AP, System Disaster Recovery is an extreme procedure and should only be used in extreme situations.

2 Procedure 2.1 Prerequisites 2.1.1

Conditions

The following conditions must apply before this procedure can be completed on the APG40. When the system disk on both nodes need to be restored: • A valid backup of the nodes available on the Local Craft Terminal (LCT). • A valid backup of the node to be restored available on the APG. • A technician is available on site. For more information about the LCT, see How to Access an AXE with APG40. When at least one system disk is functional and the other node needs to be restored from the working node: • A valid backup of the node to be restored available on the APG. • A technician is available on site. 2.1.2

Data

The following data should be known before starting this procedure. When the system disk on each node needs to be restored: • The user name and password valid for a user with administrative rights on the LCT if the backup to be restored is located on the LCT. When at least one system disk is functional and the other node needs to be restored from the working node: • The user name and password valid for a user with administrative rights on the active node. 2.1.3 • • • • • •

Special Aids A VGA monitor A keyboard with PS/2 or USB connector. A mouse with PS/2 or USB connector. An adapter to connect the keyboard and monitor to the same cable. An LCT, if the backup to be restored is located on the LCT. A cross connected Ethernet cable if the LCT is to be used.

2.2 Actions Identify Problem 1. How many nodes are not working properly or not at all? One. Go to Step 68 Two. Go to Step 2 Enter BIOS on the Right Node (Node B) 2. Connect to the right node (node b) by inserting the adapter to the mouse/keyboard input on the AP. Connect the keyboard and the mouse to the adapter and connect the monitor to the monitor input. 3. Reboot the node by pressing the reset button. 4. When the node reboots, press to enter the CPU Bios. Note: It might happen that a node reboots before the operator has time to press . If so, reset the node by pressing the reset button at the lower end of the CPU board and repeat this step. Connect to the Left Node (Node A) 5. Connect to the left node (node a) by inserting the adapter to the mouse/keyboard input on the AP. Connect the keyboard and the mouse to the adapter and connect the monitor to the monitor input. Enable Flash Disk 6. Reboot the node by pressing the reset button. 7. When the node reboots, press to enter the CPU Bios. Note: It might happen that a node reboots before the operator has time to press . If so, reset the node by pressing the reset button at the lower end of the CPU board and repeat this step. 8. Check the time and date on the first screen that appears. The date format is MM/DD/YY. In case it is not the current date, use <->/<+> to change to correct values. The time format is HH:MM:SS. In case it is not the current time, use <->/<+> to change to correct values. 9. Select menu "Advanced" by using Arrow keys. 10. Select item "Compact Flash" 11. Press <SPACE> to enable the flash disk. 12. Select the menu "Boot" by using Arrow Keys 13. Select "Boot Device Priority" and press <ENTER> 14. Under "Exclude from boot order", select "IDE 0"

15. Press <X> Note: "IDE 0" will now be visible under "Boot priority order" 16. Select item "IDE 0" under "Boot priority order" 17. Press <+> to make "IDE 0" the first item under "Boot priority order" 18. Press to save changes. 19. Confirm changes (save configuration) and exit CPU Bios. 20. Wait until the system has completed the boot process and the "Factory Preinstallation" window has disappeared. Note: The screen might "flash" multiple times during boot. Restore Partition Layout 21.

Warning! System Disaster Recovery is an extreme procedure that leads to complete loss of all the data on the system disks! Enter the following command: quickprep.cmd Note: Answer Y (Yes) when prompted to format the drives. 22. Did quickprep.cmd succeded in formatting all drives? Yes. Go to Step 23 No. Reboot the Node by typing EXIT at the command prompt and go to Step 21 Setup Access to the Backup File 23. On which media do you have a valid backup? The Data Disk M: Go to Step 34. The LCT Go to Step 24. Prepare LCT 24. Logon to the LCT as a user with administrative rights. 25. Specify 192.168.200.2 as the IP Address of the LCT and 255.255.255.0 as the Subnet Mask. Remove any defined Default Gateway.

Note: After a change of IP Address there might appear a message saying that the LCT must be rebooted. Reboot the LCT if such a message appears by clicking the OK button. 26. Connect the LCT to the node by connecting the extra cross-connected Ethernet cable to Ethernet port one, labeled ETH1. Note: If a cable already is connected to this port, disconnect it. 27. On the LCT, share the directory in which the backup file is located. Enter the following command: net share lctimages="" Note: is the full path to the directory in which the backup file is located. Restore the Node 28. On the node to be restored, setup access to the shared directory on the LCT. Enter the command: net use A: \\192.168.200.2\lctimages Note: When prompted, logon with a user with administrative rights. 29. Enter the following command: burrestore -a A:\.zip -d C:\ Note: The backup should be valid for the node to be restored. Note: Select either a or b in (a/b) depending on the last letter of the restored AP node's hostname. The prompt is not returned until the backup is restored. 30. Copy the Windows Server 2003 boot files to ntboot partition. Enter command: bootcopy C: E: 31. Disconnect the LCT share. Enter command: net use A: /DELETE 32. Disconnect the cross-connected Ethernet cable between Ethernet port one, labeled ETH1, and the LCT. Note: If a cable was disconnected in Step 26 connect that cable again. 33. Go to Step 45 Unpack Backup to System Disk 34. Was Step 21 performed on the Left Node (Node A)? Yes. Go to Step 35

No. Go to Step 40 35. Enter the following command: listvolumes.cmd Note: Make a note of the "Disk M" partition drive letter (see the "Ltr" column) if available. 36. Is Data Disk M: available? Yes. Go to Step 38 No. Go to Step 37 37. The Data Disk M: is not available to fetch the backup from. Please try to restore from LCT instead. Goto Step 24 38. Enter the following command: burrestore -a< disk m partition drive letter>:\Images\node(a/b) \.zip -d C:\ Note: The backup should be valid for the node to be restored. Note: Select either a or b in (a/b) depending on the last letter of the restored AP node's hostname. Note: The prompt is not returned until the backup is restored. 39. Goto Step 44 40. Try to set up a share towards the other node. Enter following command: net use A: \\192.168.201.1\images\nodeb Note: When prompted, logon with a user with administrative rights. 41. Was it possible to set up the share? Yes Goto Step 43 No Step 42 42. Try to set up a share towards the other node on the second heartbeat net. Enter following command: net use A: \\192.168.202.1\images\nodeb Note: When prompted, logon with a user with administrative rights. 43. Enter the following command: burrestore -a A:\.zip -d C:\

Note: The backup should be valid for the node to be restored. Note: The prompt is not returned until the backup is restored. 44. Copy the Windows Server 2003 boot files to ntboot partition. Enter command: bootcopy.cmd C: E: Disable the Flash Disk 45. Reboot by entering the command "exit" followed by <ENTER> 46. When the node reboots, press to enter the CPU Bios. Note: It might happen that a node reboots before the operator has time to press . If so, reset the node by pressing the reset button at the lower end of the CPU board and repeat this step. 47. Select menu "Advanced" by using Arrow keys. 48. Select item "Compact Flash" 49. Press <SPACE> to disable the flash disk. 50. Select the menu "Boot" by using Arrow Keys 51. Select "Boot Device Priority" and press <ENTER> 52. Under "Boot priority order", select "IDE 0" 53. Press <X> Note: "IDE 0" will disappear from "Boot priority order" 54. Press to save changes. 55. Confirm changes (save configuration) and exit CPU Bios. 56. Was the previous step performed on the Left Node (Node A)? Yes. Go to Step 57. No. Go to Step 59. Repeat Steps on the Right Node (Node B) 57. Connect to the node by inserting the adapter to the mouse/keyboard input on the AP. Connect the keyboard and the mouse to the adapter and connect the monitor to the monitor input. 58. Go to Step 8 Audit Restored Cluster

59. Connect to the Left node (Node A) by inserting the adapter to the mouse/keyboard input on the AP. Connect the keyboard and the mouse to the adapter and connect the monitor to the monitor input. Logon as a user with administrative rights. 60. Check that the node has status 'active'. Enter the following command in a Command Prompt window: prcstate Note: Error messages can popup if the cluster is not started. Retry this step a number / f times up to 3 minutes. 61. Verify that the SYSVOL share is available by entering the following command: net share SYSVOL Note: It may take up to 10 minutes until the share is available. Repeat this step until the command succeeds. 62. Check that the alarm(s) is ceased. Enter the following command: alist Note: Following alarms can be considered normal, and should be ceased with command acease if they appear:Alarm Id: 8799Class: A2Alarm Text: AP FAULT PROBLEM GENERAL ERRORProblem Data: SAM EVENTLOG_ERROR_TYPEAlarm Id: 8799Class: A2Alarm Text: AP FAULT PROBLEM GENERAL ERRORProblem Data: Userenv EVENTLOG_ERROR_TYPE dom.int Local Error Note: Take appropriate actions to solve any remaining alarm(s). 63. Connect to the Right node (Node B) by inserting the adapter to the mouse/keyboard input on the AP. Connect the keyboard and the mouse to the adapter and connect the monitor to the monitor input. Logon as a user with administrative rights. 64. Check that the node has status 'pa3sive'. Enter the following command in a Command Prompt window: prcstate Note: Error messages can popup if the cluster is not started. Retry this step a number of times up to 3 minutes. 65. Verify that the SYSVOL share is available by entering the following command: net share SYSVOL Note: It may take up to 10 minutes until the share is available. Repeat this step until the command succeeds. 66. Check that the alarm(s) is ceased. Enter the following command: alist Note: Following alarms can be considered normal, and should be ceased with command acease if they appear:Alarm Id: 8799Class: A2Alarm Text: AP FAULT PROBLEM GENERAL ERRORProblem Data: SAM EVENTLOG_ERROR_TYPEAlarm Id:

8799Class: A2Alarm Text: AP FAULT PROBLEM GENERAL ERRORProblem Data: Userenv EVENTLOG_ERROR_TYPE dom.int Local Error Note: Take appropriate actions to solve any remaining alarm(s). 67. Is the restored cluster working normally? Yes. Go to Step 133. No. Go to Step 121. Single Node Restore 68. Connect to the faulty node by inserting the adapter to the mouse/keyboard input on the AP. Connect the keyboard and the mouse to the adapter and connect the monitor to the monitor input. Enable Flash Disk 69. Reboot the node by pressing the reset button. 70. When the node reboots, press to enter the CPU Bios. Note: It might happen that a node reboots before the operator has time to press . If so, reset the node by pressing the reset button at the lower end of the CPU board and repeat this step. 71. Check the time and date on the first screen that appears in the CPU Bios setup page, the date format is MM/DD/YY. In case it is not the current time and date, use the <+> or <-> to change values and use Arrow keys to select Time or Date. Use to change between hour, minute and second and press <ESC> when the values have been corrected. 72. Select menu "Advanced" by using Arrow keys. 73. Select item "Compact Flash" 74. Press <SPACE> to enable the flash disk. 75. Select the menu "Boot" by using Arrow Keys 76. Select "Boot Device Priority" and press <ENTER> 77. Under "Exclude from boot order", select "IDE 0" 78. Press <X> Note: "IDE 0" will now be visible under "Boot priority order" 79. Select item "IDE 0" under "Boot priority order" 80. Press <+> to make "IDE 0" the first item under "Boot priority order" 81. Press to save changes. 82. Confirm changes (save configuration) and exit CPU Bios.

83. Wait until the system has completed the boot process and the "Factory Preinstallation" window has disappeared. Note: The screen might "flash" multiple times during boot. Restore Partition Layout 84.

Warning! System Disaster Recovery is an extreme procedure that leads to complete loss of all the data on the system disks! Enter the following command: quickprep.cmd Note: Answer 'Y' when prompted to format the drives. Unpack Backup to System Disk 85. On which media do you have a valid backup? The Data Disk M: Go to Step 94 DAT Tape Go to Step 86 FTP site Go to Step 93 Restore from DAT 86. In the active node, insert a tape containing the backup. 87. Copy the backup file to be restored from the tape containing the backup file to M:\Images. Enter the following command in a Command Prompt window: emfcopy -m FROMTAPE .zip M:\Images\node(a/b) Note: The backup should be valid for the node to be restored. Note: Select either a or b in (a/b) depending on the last letter of the restored AP node's hostname. 88. Enter the following command in a Command Prompt window: emfinfo -n Note: is the name of the node on which the copy operation was initiated. 89. Is the copy operation finished?

YES Go to Step 90. NO Go to Step 88. 90. When copying is finished, eject the tape. 91. Connect to the faulty node by inserting the adapter to the mouse/keyboard input on the AP. Connect the keyboard and the mouse to the adapter and connect the monitor to the monitor input. 92. Go to Step 94. Restore from FTP Site 93. The backup should be transferred to the directory M:\Images\node(a/b). The backup should be valid for the node to be restored. Select either a or b in (a/b) depending on the last letter of the restored AP node's hostname. Use FTP to transfer the compressed backup. See Operational Instruction AP, File, Transfer and then return to Step 94 in this Operational Instruction. Set up share to Data Disk M: 94. Is the backup stored on the Left Node (A-node) or on the Right Node (B-node)? Left Node (A-node) Goto Step 95 Right Node (B-Node) Goto Step 99 Restore From Left Node (A-node) 95. Try to set up the share on the heartbeat net. Enter the following command: net use M: \\192.168.201.1\images Note: When prompted, logon with a user with administrative rights. 96. Was it possible to set up the share? Yes GotoStep 102 No Goto Step 97 97. Try to set up the share on the second heartbeat net. Enter the following command: net use M: \\192.168.202.1\images Note: When prompted, logon with a user with administrative rights. 98. Goto Step 102 Restore From Right Node (B-node) 99. Try to set up the share on the heartbeat net. Enter the following command:

net use M: \\192.168.201.2\images Note: When prompted, logon with a user with administrative rights. 100. Was it possible to set up the share? Yes GotoStep 102 No Goto Step 101 101. Try to set up the share on the second heartbeat net. Enter the following command: net use M: \\192.168.202.2\images Note: When prompted, logon with a user with administrative rights. Restore from Data Disk M: 102. Enter the following command: burrestore -a M:\node(a/b)\.zip -d C:\ Note: The backup should be valid for the node to be restored. Note: Select either a or b in (a/b) depending on the last letter of the restored AP node's hostname. Note: The prompt is not returned until the backup is restored. 103. Copy the Windows Server 2003 boot files to ntboot partition. Enter command: bootcopy.cmd C: E: Disable Flashdisk 104. Reboot by entering the command "exit" followed by <ENTER> 105. When the node reboots, press to enter the CPU Bios. Note: It might happen that a node reboots before the operator has time to press . If so, reset the node by pressing the reset button at the lower end of the CPU board and repeat this step. 106. Select menu "Advanced" by using Arrow keys. 107. Select item "Compact Flash" 108. Press <SPACE> to disable the flash disk. 109. Select the menu "Boot" by using Arrow Keys 110. Select "Boot Device Priority" and press <ENTER> 111. Under "Boot priority order", select "IDE 0"

112. Press <X> Note: "IDE 0" will disappear from "Boot priority order" 113. Press to save changes. 114. Confirm changes (save configuration) and exit CPU Bios. Audit Restored Node 115. Connect to the restored node by inserting the adapter to the mouse/keyboard input on the AP. Connect the keyboard and the mouse to the adapter and connect the monitor to the monitor input. Logon as a user with administrative rights. 116. Logon as a user with administrative rights. 117. Check that the restored node has status 'passive'. Enter the following command in a Command Prompt window: prcstate Note: Error messages can popup if the cluster is not started. Retry this step a number of times up to 3 minutes. 118. Verify that the SYSVOL share is available by entering the following command: net share SYSVOL Note: It may take up to 10 minutes until the share is available. Repeat this step until the command succeeds. 119. Check that the alarm(s) is ceased. Enter the following command: alist Note: Following alarms can be considered normal, and should be ceased with the command acease if they appear:Alarm Id: 8799Class: A2Alarm Text: AP FAULT PROBLEM GENERAL ERRORProblem Data: SAM EVENTLOG_ERROR_TYPEAlarm Id: 8799Class: A2Alarm Text: AP FAULT PROBLEM GENERAL ERRORProblem Data: Userenv EVENTLOG_ERROR_TYPE dom.int Local Error Note: Take appropriate actions to solve any remaining alarm(s). 120. Is the restored node working properly? Yes Go to Step 133. No Go to Step 121. System Quorum Log Recreate 121. Go to Operational Instruction AP, System Quorum Log Recreate. Perform the steps in that Operational Instruction, and return to Step 122 in this Operational Instruction.

122. Did the previous step solve the problems? Yes. Go to Step 133. No. Go to Step 123. System Data Disk Restore 123.

Warning! System Data Disk Restore is an extreme procedure that leads to complete loss of all the data on all data disks, including any charging data! Go to Operational Instruction AP, System Data Disk Restore Perform the steps in that Operational Instruction, and return to Step 124 in this Operational Instruction. 124. Did the previous step solve the problems? Yes. Go to Step 125. No. Go to Step 138. Small CP Restart 125. Go to Operational Instruction System Start/Restart, Manually Initiate . Perform the steps in that Operational Instruction to perform a system restart without reload using the IO system (syrei:rank=small;), and return to Step 126 in this Operational Instruction. Reconfigure Audit Log 126. Audit Log has lost the configuration data on the data disk and must be reconfigured. Find out how Audit Log was configured before the Disaster Recovery procedure. 127. Shall the Audit Log be active? Yes Go to Step 130 No Go to Step 128 128. Deactivate Audit Log according to Operational Instruction AP, Audit Log, Deactivate and return to Step 132. 129. Go to Step 132 130. Configure the attribute for Audit Log. Go to Operational Instruction AP, Audit Log, Attributes, Set. Perform the steps in that Operational Instruction and return to Step 131.

131. Configure the excluding command for Audit Log. Go to Operational Instruction AP, Audit Log, Item in Exclusion List, Define. Perform the steps in that Operational Instruction and return to Step 132 in this Operational Instruction. Reconfigure Applications 132. All applications that in one way or another have configuration data stored on the data disks, must be reconfigured before continuing with Step 133. Restore Reparse Points 133. Enter the following command in a Command Prompt window: cd c:\Program Files\AP\ACS\BURREMRECBIN\serversw\bur 134. Enter the following command: pkzipc -extract \\\m$\images\node(a/b) \.zip ReparsePoints.bur Note: The backup should be valid for the restored node. Note: Select either a or b in (a/b) depending on the last letter of the restored AP node's hostname. 135. To restore the reparse points, enter the following command: RPRestore -s ReparsePoints.bur -d C:\ Backup System 136. Go to Operational Instruction AP, System Backup and Verify, Initiate . Perform the steps in that Operational Instruction, and return to Step 137 in this Operational Instruction. 137. Go to Step 139. Consult Next Level of Maintenance Support 138. Consult the next level of maintenance support. Further action is outside the scope of this Operational Instruction. Job Completed 139. Make a report. See Operational Instruction Report of Finished Work. 140. The job is completed.

3 Additional Information No additional information is applicable to this document.

4 Glossary This document contains no glossary entries.

5 References 5.1 Operational Instructions AP, Audit Log, Attributes, Set AP, Audit Log, Deactivate AP, Audit Log, Item in Exclusion List, Define AP, System Backup and Verify, Initiate AP, System Data Disk Restore AP, System Quorum Log Recreate Report of Finished Work System Start/Restart, Manually Initiate

5.2 Manual Pages alist(1m) acease(1m) burrestore(1m) emfcopy(1m) emfinfo(1m) prcboot(1m) prcstate(1m)

5.3 User Guides How to Access an AXE with APG40