Complete Guide to Cisco Switch and Router Password Recovery & System Recovery Procedures

Understanding Cisco Device Recovery

Password recovery and system recovery are critical skills for any network administrator managing Cisco devices. Whether you've inherited equipment with unknown passwords, experienced configuration corruption, or encountered system failures, knowing the proper recovery procedures can save valuable time and prevent network downtime.

This comprehensive guide covers password recovery procedures for Cisco switches and routers, IOS recovery methods, configuration restoration techniques, and troubleshooting common boot failures. We'll explore both traditional and modern recovery methods, security considerations, and best practices for maintaining recoverability in production environments.

Cisco Recovery Overview

Recovery Scenarios Overview

Common Recovery Situations

  • Password Recovery: Forgotten or unknown enable password, console password, or VTY passwords
  • Configuration Recovery: Corrupted or deleted configuration files
  • IOS Recovery: Missing, corrupted, or incompatible IOS image
  • ROMMON Recovery: Device stuck in ROMMON mode
  • Boot Failure: Device unable to complete boot process
  • Flash Memory Issues: Corrupted or full flash storage

Prerequisites for Recovery

  • Physical Access: Console cable and direct physical access to device
  • Console Connection: Terminal emulator software (PuTTY, SecureCRT, Tera Term)
  • Recovery Files: Valid IOS images and backup configurations (for IOS recovery)
  • TFTP/FTP Server: For transferring files during recovery
  • Documentation: Device model, current IOS version information
  • Time Window: Planned downtime for production devices

Important Security Considerations

Warning: Password recovery procedures typically require physical access to the device and will bypass security mechanisms. This highlights why physical security of network infrastructure is critical. Always follow organizational security policies and document all recovery procedures.

Cisco Router Password Recovery

Overview of Router Password Recovery

Router password recovery involves interrupting the normal boot process, bypassing the startup configuration, and resetting passwords. The exact procedure varies slightly by platform, but the general methodology remains consistent across Cisco router models.

Password Recovery for Cisco ISR Routers (1900, 2900, 3900, 4000 Series)

Step 1: Connect to Console Port

Connect your console cable to the router and establish a terminal session with these settings:

  • Baud Rate: 9600
  • Data Bits: 8
  • Parity: None
  • Stop Bits: 1
  • Flow Control: None

Step 2: Power Cycle and Send Break

Power off the router, then power it back on. Within the first 60 seconds of boot, send a break sequence to interrupt the boot process and enter ROMMON mode.

Break Sequence by Terminal Emulator:

  • PuTTY: Ctrl + Break (or right-click > Special Command > Break)
  • SecureCRT: Alt + B or Send Break from menu
  • Tera Term: Alt + B or Control > Send Break
  • HyperTerminal: Ctrl + Break
  • macOS Terminal: Ctrl + Shift + 6, then X

Step 3: Change Configuration Register

Once in ROMMON mode, you'll see a prompt like rommon 1 >. Change the configuration register to bypass the startup configuration:

rommon 1 > confreg 0x2142
rommon 2 > reset
  

What is 0x2142? This configuration register value tells the router to bypass the startup-config during boot. The default value is 0x2102, which loads the startup-config normally.

Step 4: Enter Privileged EXEC Mode

The router will boot without loading the startup configuration. Press Enter to enter user mode, then type enable to enter privileged mode (no password required):

Router> enable
Router#
  

Step 5: Copy Startup Config to Running Config

Load the existing configuration into running memory to preserve your settings:

Router# copy startup-config running-config
Destination filename [running-config]? 
  

Important: Press Enter when prompted for the filename. Do NOT use copy running-config startup-config at this stage, as it would overwrite your existing configuration.

Step 6: Reset the Password

Enter configuration mode and set a new enable password or enable secret:

Router# configure terminal
Router(config)# enable secret NewPassword123!
Router(config)# no enable password
Router(config)# exit
  

Step 7: Restore Configuration Register

Change the configuration register back to the default value so the router loads the startup-config on the next boot:

Router# configure terminal
Router(config)# config-register 0x2102
Router(config)# exit
  

Step 8: Save Configuration

Router# write memory
Building configuration...
[OK]

! Or alternatively
Router# copy running-config startup-config
  

Step 9: Verify and Reload

Router# show version | include Configuration register
Configuration register is 0x2142 (will be 0x2102 at next reload)

Router# reload
Proceed with reload? [confirm]
  

Verification After Recovery

After the router reboots, verify you can log in with the new password:

Router> enable
Password: NewPassword123!
Router# show running-config | include enable secret
enable secret 5 $1$mERr$hx5rVt7rPNoS4wqbXKX7m0
  

Cisco Switch Password Recovery

Password Recovery for Catalyst 2960/2960-X/2960-XR Switches

Step 1: Connect Console and Prepare

Connect to the switch console port using the same terminal settings as routers (9600 baud, 8-N-1).

Step 2: Power Cycle and Interrupt Boot

Disconnect power, reconnect, and immediately hold the Mode button on the front of the switch. Keep holding until the SYST LED blinks amber and then solid green (approximately 10-15 seconds), then release.

Alternative Method: Some switches allow sending a break sequence during boot instead of using the Mode button. Check your specific model documentation.

Step 3: Initialize Flash and Load Helper Files

Once in switch boot loader mode (switch:), initialize the flash file system:

switch: flash_init
Initializing Flash...
flashfs[0]: 4 files, 1 directories
flashfs[0]: 0 orphaned files, 0 orphaned directories
flashfs[0]: Total bytes: 32514048
flashfs[0]: Bytes used: 12345678
flashfs[0]: Bytes available: 20168370
flashfs[0]: flashfs fsck took 5 seconds.
...done Initializing Flash.

switch: load_helper
  

Step 4: Rename Configuration File

Rename the configuration file to prevent it from loading during boot:

switch: dir flash:
Directory of flash:/
    2  -rwx  12345678  <date>  c2960-lanbasek9-mz.150-2.SE11.bin
    3  -rwx  2654      <date>  config.text
    4  -rwx  5         <date>  private-config.text

switch: rename flash:config.text flash:config.old
switch: rename flash:private-config.text flash:private-config.old
  

Step 5: Boot the Switch

switch: boot

Loading "flash:c2960-lanbasek9-mz.150-2.SE11.bin"...
  

Step 6: Skip Initial Configuration

When prompted for initial configuration dialog, enter no:

--- System Configuration Dialog ---

Would you like to enter the initial configuration dialog? [yes/no]: no

Press RETURN to get started!
  

Step 7: Enter Privileged Mode and Restore Config

Switch> enable
Switch# rename flash:config.old flash:config.text
Switch# rename flash:private-config.old flash:private-config.text
Switch# copy flash:config.text system:running-config
Destination filename [running-config]? 
  

Step 8: Change Password

Switch# configure terminal
Switch(config)# enable secret NewSecurePassword123!
Switch(config)# no enable password
Switch(config)# exit
  

Step 9: Save and Reload

Switch# write memory
Building configuration...
[OK]
Switch# reload
  

Catalyst 3560/3750/3850 Switch Password Recovery

Recovery Procedure for 3560/3750 Series

Step 1: Enter Boot Loader Mode

Power cycle the switch while holding the Mode button until the LED turns briefly amber, then solid green (about 2 seconds after release).

Step 2: Initialize Flash

switch: flash_init
Initializing Flash...
...done Initializing Flash.
  

Step 3: Set BOOT Environment Variable

Check the current BOOT variable and set it if needed:

switch: set

! Look for BOOT variable, if not set or incorrect:
switch: BOOT=flash:c3560-ipservicesk9-mz.122-55.SE12.bin
switch: set
  

Step 4: Bypass Startup Configuration

switch: dir flash:

! Rename the config files
switch: rename flash:config.text flash:config.old
switch: rename flash:private-config.text flash:private-config.old
switch: boot
  

Step 5: Recover Configuration and Set New Password

Switch> enable
Switch# rename flash:config.old flash:config.text
Switch# copy flash:config.text running-config
Destination filename [running-config]? 

Switch# configure terminal
Switch(config)# enable secret MyNewPassword456!
Switch(config)# exit
Switch# write memory
Switch# reload
  

Recovery Procedure for 3850/9300 Series (Modern Switches)

Differences from Older Models

Catalyst 3850 and 9300 series switches have a different boot process and may require slightly different commands:

Step 1: Interrupt Boot Process

Press the Mode button within 2 seconds of powering on until you see:

The system has been interrupted prior to initializing the
flash file system. The following commands will initialize
the flash file system, and finish loading the operating
system software:

    flash_init
    load_helper
    boot
  

Step 2: Initialize and Bypass Configuration

switch: flash_init
switch: load_helper
switch: dir flash:

! Note the config file name (usually config.text)
switch: rename flash:config.text flash:config.backup
  

Step 3: Boot Without Configuration

switch: boot

! When prompted:
Would you like to enter the initial configuration dialog? [yes/no]: no
  

Step 4: Restore and Modify Configuration

Switch> enable
Switch# rename flash:config.backup flash:config.text
Switch# copy flash:config.text running-config

Switch# configure terminal
Switch(config)# enable secret SecurePass789!
Switch(config)# username admin privilege 15 secret AdminPass789!
Switch(config)# end

Switch# write memory
Switch# reload
  

Nexus Switch Password Recovery

Password Recovery for Cisco Nexus 5000/7000/9000 Series

Nexus switches use a different recovery procedure compared to Catalyst switches. The process varies between Nexus families.

Nexus 9000 Series Password Recovery

Step 1: Interrupt Boot and Enter Loader Mode

Power cycle the switch and press Ctrl-C when you see the following message:

Abort Auto Provisioning and continue with normal setup ?(yes/no)[n]: 
  

Or press Ctrl-C during the boot process to enter the loader prompt:

loader>
  

Step 2: Boot in Single-User Mode

loader> cmdline recoverymode=1
loader> boot
  

Step 3: Load System and Reset Password

The system will boot into bash shell. Load the NX-OS environment:

switch(boot)# config t
switch(boot-config)# admin-password NewNexusPassword123!
switch(boot-config)# exit
switch(boot)# load-nxos
  

Step 4: Verify and Save

switch# configure terminal
switch(config)# username admin password NewNexusPassword123! role network-admin
switch(config)# exit
switch# copy running-config startup-config
  

Nexus 5000/7000 Series Password Recovery

Method 1: Using Loader Prompt

! Power cycle and press Ctrl-] to enter loader
loader> cmdline recoverymode=1
loader> boot nxos.7.3.1.N1.1.bin

! In recovery mode
switch(boot)# config t
switch(boot-config)# admin-password YourNewPassword
switch(boot-config)# exit
switch(boot)# load-nxos
  

Method 2: Using Init Command (Alternative)

loader> init
loader> boot

! During boot, you'll see option to skip configuration
! Answer 'no' to initial setup and proceed with password reset
  

Important for Nexus: Unlike IOS devices, Nexus switches running NX-OS have built-in password recovery features that don't require renaming configuration files. The admin password can be reset directly from the loader.

IOS Recovery and ROMMON Procedures

Understanding ROMMON Mode

ROM Monitor (ROMMON) is a bootstrap program that initializes the hardware and boots the Cisco IOS. When a device cannot find a valid IOS image or encounters boot failures, it enters ROMMON mode.

Common ROMMON Scenarios

  • Missing or corrupted IOS image in flash
  • Incorrect boot system commands
  • Flash memory corruption or failure
  • Boot configuration register issues
  • Manual entry for troubleshooting

ROMMON Mode Recovery - Router

Method 1: TFTP Boot (Network Recovery)

Prerequisites:

  • TFTP server accessible from the router
  • Valid IOS image on TFTP server
  • Direct Ethernet connection to TFTP server (preferred)

Step 1: Enter ROMMON and Set IP Configuration

rommon 1 > IP_ADDRESS=192.168.1.10
rommon 2 > IP_SUBNET_MASK=255.255.255.0
rommon 3 > DEFAULT_GATEWAY=192.168.1.1
rommon 4 > TFTP_SERVER=192.168.1.100
rommon 5 > TFTP_FILE=c2900-universalk9-mz.SPA.157-3.M5.bin
  

Step 2: Verify Settings

rommon 6 > set

IP_ADDRESS=192.168.1.10
IP_SUBNET_MASK=255.255.255.0
DEFAULT_GATEWAY=192.168.1.1
TFTP_SERVER=192.168.1.100
TFTP_FILE=c2900-universalk9-mz.SPA.157-3.M5.bin
  

Step 3: Test Connectivity

rommon 7 > ping 192.168.1.100

Sending 5, 100-byte ICMP Echos to 192.168.1.100, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5)
  

Step 4: Boot from TFTP

rommon 8 > tftpdnld

IP_ADDRESS: 192.168.1.10
IP_SUBNET_MASK: 255.255.255.0
DEFAULT_GATEWAY: 192.168.1.1
TFTP_SERVER: 192.168.1.100
TFTP_FILE: c2900-universalk9-mz.SPA.157-3.M5.bin

Receiving c2900-universalk9-mz.SPA.157-3.M5.bin from 192.168.1.100
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[output omitted]

File reception completed.
Launching tftp image...

! Router will boot from TFTP image
  

Step 5: Copy IOS to Flash

Once booted, copy the IOS to flash for permanent storage:

Router# copy tftp: flash:
Address or name of remote host []? 192.168.1.100
Source filename []? c2900-universalk9-mz.SPA.157-3.M5.bin
Destination filename [c2900-universalk9-mz.SPA.157-3.M5.bin]? 

Accessing tftp://192.168.1.100/c2900-universalk9-mz.SPA.157-3.M5.bin...
Loading c2900-universalk9-mz.SPA.157-3.M5.bin from 192.168.1.100 (via GigabitEthernet0/0): 
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[OK - 123456789 bytes]

123456789 bytes copied in 234.567 secs (526234 bytes/sec)
  

Step 6: Configure Boot System

Router# configure terminal
Router(config)# boot system flash:c2900-universalk9-mz.SPA.157-3.M5.bin
Router(config)# exit
Router# write memory
Router# reload
  

Method 2: XMODEM Boot (Console Recovery)

Use XMODEM when network connectivity is unavailable. This method is slower but only requires console access.

Step 1: Prepare Terminal Emulator

Ensure your terminal emulator supports XMODEM file transfer (PuTTY, SecureCRT, Tera Term).

Step 2: Enter ROMMON and Initialize

rommon 1 > dir flash:
! Verify flash is accessible

rommon 2 > xmodem -c c2900-universalk9-mz.SPA.157-3.M5.bin
Do not start the sending program yet...

File size           Checksum    File name
123456789 bytes     0x1234      c2900-universalk9-mz.SPA.157-3.M5.bin

WARNING: All existing data in bootflash will be lost!
Invoke this application only for disaster recovery.
Do you wish to continue? y/n  [n]: y

Ready to receive file c2900-universalk9-mz.SPA.157-3.M5.bin ...
  

Step 3: Send File via XMODEM

  • PuTTY: Right-click > File Transfer > XMODEM > Select file
  • SecureCRT: Transfer > Send Xmodem
  • Tera Term: File > Transfer > XMODEM > Send

Note: XMODEM transfer is very slow (typically 3-5 KB/s). Transferring a 100MB IOS image can take 6-8 hours. Use this method only as a last resort.

Step 4: Boot After Transfer

rommon 3 > boot flash:c2900-universalk9-mz.SPA.157-3.M5.bin
  

Method 3: USB Recovery (ISR G2 Routers)

Modern ISR routers support booting from USB, which is faster than XMODEM:

Step 1: Prepare USB Drive

  • Format USB drive as FAT16 or FAT32
  • Copy IOS image to root of USB drive
  • Insert USB into router's USB port

Step 2: Boot from USB in ROMMON

rommon 1 > dir usbflash0:
! Verify USB is detected and contains IOS

rommon 2 > boot usbflash0:c2900-universalk9-mz.SPA.157-3.M5.bin

Loading usbflash0:c2900-universalk9-mz.SPA.157-3.M5.bin
  

Step 3: Copy IOS to Flash

Router# copy usbflash0:c2900-universalk9-mz.SPA.157-3.M5.bin flash:
Destination filename [c2900-universalk9-mz.SPA.157-3.M5.bin]? 

Copy in progress...
  

Switch IOS Recovery Procedures

Catalyst Switch TFTP Recovery

Step 1: Enter Boot Loader Mode

Power cycle the switch and hold the Mode button, or send break signal during boot.

Step 2: Configure IP and TFTP Settings

switch: set IP_ADDR=192.168.1.10
switch: set NETMASK=255.255.255.0
switch: set DEFAULT_GATEWAY=192.168.1.1
switch: set TFTP_SERVER=192.168.1.100
switch: set TFTP_FILE=c2960-lanbasek9-mz.150-2.SE11.bin
  

Step 3: Initialize Flash

switch: flash_init
Initializing Flash...
...done Initializing Flash.

switch: dir flash:
! Verify flash is accessible
  

Step 4: Download IOS via TFTP

switch: tftp

! IOS will download from TFTP server
IP_ADDRESS: 192.168.1.10
NETMASK: 255.255.255.0
DEFAULT_GATEWAY: 192.168.1.1
TFTP_SERVER: 192.168.1.100
TFTP_FILE: c2960-lanbasek9-mz.150-2.SE11.bin

Receiving file...
!!!!!!!!!!!!!!!!!!!!!!!!!!!
File received successfully
  

Step 5: Set Boot Variable and Boot

switch: set BOOT=flash:c2960-lanbasek9-mz.150-2.SE11.bin
switch: boot

Loading "flash:c2960-lanbasek9-mz.150-2.SE11.bin"...
  

Step 6: Verify and Save Boot Configuration

Switch> enable
Switch# show boot
BOOT path-list      : flash:c2960-lanbasek9-mz.150-2.SE11.bin

Switch# configure terminal
Switch(config)# boot system flash:c2960-lanbasek9-mz.150-2.SE11.bin
Switch(config)# exit
Switch# write memory
  

Switch USB Recovery

Many newer Catalyst switches support USB recovery:

Step 1: Prepare USB Drive

  • Format as FAT32
  • Copy IOS image to root directory
  • Insert into switch USB port

Step 2: Boot from USB

switch: dir usbflash0:
switch: boot usbflash0:c2960x-universalk9-mz.152-4.E8.bin
  

Step 3: Copy to Flash

Switch# copy usbflash0:c2960x-universalk9-mz.152-4.E8.bin flash:
Switch# configure terminal
Switch(config)# boot system flash:c2960x-universalk9-mz.152-4.E8.bin
Switch(config)# exit
Switch# write memory
  

Configuration File Recovery

Recovering from Configuration Loss

Scenario 1: Startup Configuration Deleted

If startup-config is accidentally deleted but running-config is intact:

Router# copy running-config startup-config
Destination filename [startup-config]? 
Building configuration...
[OK]
  

Scenario 2: Both Configurations Lost

If both configurations are lost but you have a backup file on TFTP server:

Router# copy tftp: startup-config
Address or name of remote host []? 192.168.1.100
Source filename []? router-backup-config.txt
Destination filename [startup-config]? 

Accessing tftp://192.168.1.100/router-backup-config.txt...
Loading router-backup-config.txt from 192.168.1.100: !
[OK - 3456 bytes]

3456 bytes copied in 2.123 secs (1628 bytes/sec)

Router# reload
  

Scenario 3: Recover from Flash

If configuration was previously archived to flash:

Router# dir flash:
Directory of flash:/
    1  -rw-   123456789  Jan 15 2026 c2900-universalk9-mz.bin
    2  -rw-        3456  Jan 20 2026 backup-config-jan20.cfg
    3  -rw-        5678  Jan 25 2026 startup-config.bak

Router# copy flash:startup-config.bak startup-config
Destination filename [startup-config]? 

3456 bytes copied in 0.456 secs
Router# reload
  

Using Configuration Archive Feature

Configure Automatic Configuration Archiving

Router(config)# archive
Router(config-archive)# path flash:config-backup-$t
Router(config-archive)# maximum 10
Router(config-archive)# time-period 1440
Router(config-archive)# write-memory
Router(config-archive)# exit
  

View Archived Configurations

Router# show archive
The maximum archive configurations allowed is 10.
The next archive file will be named flash:config-backup-26
 Archive #  Name
    1       flash:config-backup-16
    2       flash:config-backup-17
    3       flash:config-backup-18
    4       flash:config-backup-19
    5       flash:config-backup-20
  

Restore from Archive

Router# configure replace flash:config-backup-20

This will apply all necessary additions and deletions
to replace the current running configuration with the
contents of the specified configuration file, which is
assumed to be a complete configuration, not a partial
configuration. Enter Y if you are sure you want to proceed. ? [no]: yes

Total number of passes: 1
Rollback Done
  

Configuration Rollback

Configure Configuration Rollback

Router# configure terminal
Router(config)# archive
Router(config-archive)# path flash:rollback-config
Router(config-archive)# maximum 14
  

Save Configuration Checkpoint

Router# archive config
  

Rollback to Previous Configuration

! View available configurations
Router# show archive

! Rollback to specific configuration
Router# configure replace flash:rollback-config-5

! Or rollback to most recent
Router# configure revert now
  

Flash Memory Recovery

Diagnosing Flash Memory Issues

Check Flash Status

Router# show flash:
-#- --length-- -----date/time------ path
1   123456789  Jan 25 2026 10:15:30 c2900-universalk9-mz.bin
2        3456  Jan 25 2026 09:30:00 startup-config

32514048 bytes total (20168370 bytes free)

! Check for errors
Router# show file systems
File Systems:

       Size(b)       Free(b)      Type  Flags  Prefixes
*    32514048      20168370     flash     rw   flash:
  

Verify Flash Integrity

Router# verify /md5 flash:c2900-universalk9-mz.bin

.................................Done!
verify /md5 (flash:c2900-universalk9-mz.bin) = 1234abcd5678efgh9012ijkl3456mnop
  

Recovering Corrupted Flash

Format Flash Memory

Warning: Formatting flash will erase ALL data including IOS and configurations. Only do this if flash is corrupted and unusable. Ensure you have IOS available via TFTP or USB before formatting.

! From ROMMON mode
rommon 1 > format flash:

All sectors will be erased, proceed? y/n [n]: y
Format operation may take a while. Continue? y/n [n]: y
Format operation will destroy all data in "flash:". Continue? y/n [n]: y

Format: All system sectors written. OK...
Format: Total sectors in formatted partition: 63488
Format: Total bytes in formatted partition: 32514048
Format: Operation completed successfully.

Format of flash: complete

! After formatting, download IOS via TFTP
rommon 2 > IP_ADDRESS=192.168.1.10
rommon 3 > TFTP_SERVER=192.168.1.100
rommon 4 > TFTP_FILE=c2900-universalk9-mz.bin
rommon 5 > tftpdnld
  

Recover Files from Corrupted Flash

If flash is partially corrupted but some files are readable:

Router# dir flash:
! If directory is accessible, copy critical files to TFTP

Router# copy flash:startup-config tftp:
Address or name of remote host []? 192.168.1.100
Destination filename [startup-config]? router-backup.cfg

! Copy IOS image if possible
Router# copy flash:c2900-universalk9-mz.bin tftp:
  

Flash Space Management

Delete Unnecessary Files

Router# dir flash:
Directory of flash:/
    1  -rw-   123456789  c2900-universalk9-mz.SPA.157-3.M5.bin
    2  -rw-   100000000  c2900-universalk9-mz.SPA.155-3.M4a.bin (old)
    3  drw-           0  .installer

! Delete old IOS image
Router# delete flash:c2900-universalk9-mz.SPA.155-3.M4a.bin
Delete filename [c2900-universalk9-mz.SPA.155-3.M4a.bin]? 
Delete flash:c2900-universalk9-mz.SPA.155-3.M4a.bin? [confirm]

! Permanently remove deleted files
Router# squeeze flash:
Squeeze flash: All deleted files will be removed. Continue? [confirm]
Squeeze operation may take a while. Continue? [confirm]
Squeeze of flash complete
  

Clean Up Installer Files

Router# dir flash:/.installer/
! Check for installation temp files

Router# request platform software package clean
  

Troubleshooting Boot Failures

Common Boot Failure Symptoms

  • Device stuck in ROMMON mode
  • Boot loop (continuous reloading)
  • Partial boot (stops at certain percentage)
  • Error messages during boot
  • Cannot find IOS image

Diagnostic Steps

Check Boot System Commands

Router# show boot
BOOT path-list:      flash:c2900-universalk9-mz.bin
Config file:         flash:startup-config
Private Config file: flash:private-config.text
Enable Break:        no
Manual Boot:         no
HELPER path-list:
Auto upgrade:        yes
NVRAM/Config file
      buffer size:   524288
  

Verify Configuration Register

Router# show version | include Configuration register
Configuration register is 0x2102

! Common configuration register values:
! 0x2102 - Normal boot, load startup-config
! 0x2142 - Bypass startup-config (password recovery)
! 0x2100 - Boot to ROMMON
! 0x2101 - Boot from ROM (not flash)
  

Check for Boot Errors

Router# show logging | include boot
Jan 25 10:15:30.123: %SYS-5-RELOAD: Reload requested by admin on console
Jan 25 10:15:45.456: %SYS-5-CONFIG_I: Configured from memory by console
Jan 25 10:16:00.789: Boot image file is "flash:c2900-universalk9-mz.bin"
  

Fixing Boot Sequence Issues

Reset Boot System Variables

Router# configure terminal
Router(config)# no boot system
Router(config)# boot system flash:c2900-universalk9-mz.bin
Router(config)# exit
Router# write memory
Router# reload
  

Fix Configuration Register

! If stuck with wrong config register
Router# configure terminal
Router(config)# config-register 0x2102
Router(config)# exit
Router# reload
  

Boot Manually from ROMMON

rommon 1 > dir flash:
! Identify available IOS images

rommon 2 > boot flash:c2900-universalk9-mz.bin

! Or set BOOT variable
rommon 3 > BOOT=flash:c2900-universalk9-mz.bin
rommon 4 > set
rommon 5 > boot
  

Hardware-Related Boot Issues

Power Supply Problems

Router# show environment power
Environmental monitor, Version V0.98
...
Power supply 1 is NORMAL
...

! Check for power-related errors
Router# show logging | include power
  

Memory Issues

Router# show version | include memory
Cisco 2911 (revision 1.0) with 487424K/36864K bytes of memory

! Check for memory errors
Router# show memory summary

                Head    Total(b)     Used(b)     Free(b)   Lowest(b)  Largest(b)
Processor    654B2B10   492359540   122368972   369990568   367990568  367890568
      I/O    400000000    33554432    11854348    21700084    21700084   21600084
  

Flash Card Issues

! Check for flash errors in boot messages
Router# show logging | include flash

! Test flash card
Router# test flash:

! In ROMMON, check flash initialization
rommon 1 > flash_init
rommon 2 > dir flash:
  

Advanced Recovery Techniques

FTP-Based IOS Recovery

FTP can be faster than TFTP for large IOS images:

Configure FTP Parameters in ROMMON

rommon 1 > IP_ADDRESS=192.168.1.10
rommon 2 > IP_SUBNET_MASK=255.255.255.0
rommon 3 > DEFAULT_GATEWAY=192.168.1.1
rommon 4 > FTP_SERVER=192.168.1.100
rommon 5 > FTP_USER=cisco
rommon 6 > FTP_PASSWORD=cisco123
rommon 7 > FTP_FILE=c2900-universalk9-mz.bin
rommon 8 > set

! Download via FTP
rommon 9 > dev-ftpdnld

! Boot the downloaded image
rommon 10 > boot flash:c2900-universalk9-mz.bin
  

RCP Recovery (Remote Copy Protocol)

! From IOS (if partially booted)
Router# copy rcp: flash:
Address or name of remote host []? 192.168.1.100
Source username [router]? cisco
Source filename []? c2900-universalk9-mz.bin
Destination filename [c2900-universalk9-mz.bin]? 
  

HTTP/HTTPS Recovery

Some newer platforms support HTTP for file transfers:

Router# copy http://192.168.1.100/ios/c2900-universalk9-mz.bin flash:
Destination filename [c2900-universalk9-mz.bin]? 

Accessing http://192.168.1.100/ios/c2900-universalk9-mz.bin...
Loading http://192.168.1.100/ios/c2900-universalk9-mz.bin
!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[OK - 123456789 bytes]
  

SCP Recovery (Secure Copy)

! Enable SCP server on router first
Router# configure terminal
Router(config)# ip scp server enable
Router(config)# exit

! From Linux/Mac terminal
$ scp c2900-universalk9-mz.bin admin@192.168.1.10:flash:

! Or copy FROM router
Router# copy flash:config-backup.cfg scp:
Address or name of remote host []? 192.168.1.100
Destination username [router]? backup-user
Destination filename [config-backup.cfg]? 
  

Recovery Best Practices

Preventive Measures

1. Regular Configuration Backups

! Automated backup using Kron (IOS scheduler)
Router(config)# kron occurrence DAILY-BACKUP at 2:00 recurring
Router(config-kron-occurrence)# policy-list BACKUP-CONFIG

Router(config)# kron policy-list BACKUP-CONFIG
Router(config-kron-policy)# cli write memory
Router(config-kron-policy)# cli copy running-config tftp://192.168.1.100/backups/$h-$t.cfg

! Or use archive feature
Router(config)# archive
Router(config-archive)# path tftp://192.168.1.100/backups/$h-$t
Router(config-archive)# write-memory
Router(config-archive)# time-period 1440
  

2. Maintain IOS Image Repository

  • Keep copies of IOS images on TFTP/FTP server
  • Document MD5 checksums for verification
  • Maintain compatible IOS versions for each platform
  • Keep USB drives with IOS images for emergency recovery

3. Document Device Configurations

  • Maintain configuration templates
  • Document IP addressing schemes
  • Keep inventory of device models and IOS versions
  • Record configuration register settings
  • Document boot system commands

4. Implement Change Control

  • Test configuration changes in lab before production
  • Create configuration checkpoints before major changes
  • Use configuration rollback features
  • Maintain change logs
  • Schedule maintenance windows for risky changes

Recovery Procedure Best Practices

Before Recovery

  • Verify Physical Access: Ensure console access is available
  • Check Equipment: Have console cables, terminal software ready
  • Gather Information: Collect device model, current IOS version, configuration backups
  • Prepare Files: Have IOS images and configurations on TFTP/USB
  • Plan Downtime: Schedule recovery during maintenance window
  • Document Steps: Have recovery procedures printed/accessible offline

During Recovery

  • Log Everything: Capture console output for documentation
  • Take Screenshots: Document error messages and settings
  • Verify Each Step: Confirm success before proceeding
  • Don't Rush: Take time to understand what's happening
  • Keep Backups: Don't overwrite old configs until new one is verified

After Recovery

  • Verify Functionality: Test all critical functions
  • Check Connectivity: Verify all interfaces and routing
  • Review Logs: Check for errors or warnings
  • Update Documentation: Record what was done
  • Create Fresh Backup: Save known-good configuration
  • Monitor Stability: Watch device for issues in following days

Security Considerations

Physical Security

  • Restrict physical access to network equipment rooms
  • Implement badge access or locks on equipment racks
  • Use port security on console ports in shared facilities
  • Consider disabling console access when not needed

Password Recovery Security

  • Log All Recovery Procedures: Maintain audit trail
  • Require Approval: Get authorization before password recovery
  • Two-Person Rule: Have witness for recovery procedures
  • Change Passwords Immediately: Set new secure passwords after recovery
  • Review Access: Check who had access during recovery

Configuration Register Security

! Disable break during boot to prevent unauthorized recovery
Router(config)# no service password-recovery

! This will show warning - device cannot be recovered without RMA
WARNING: Executing this command will disable password recovery.
The only recourse to regain access to the router will be to return the
router to the factory to have the password reset. Do NOT execute this
command without another plan for password recovery.
Are you sure you want to continue? [yes/no]: yes
  

Important: Only use no service password-recovery in high-security environments where physical security cannot be guaranteed. This makes password recovery impossible without returning device to factory.

Recovery Tools and Utilities

Essential Software Tools

Terminal Emulators

  • PuTTY: Free, Windows - supports XMODEM, session logging
  • SecureCRT: Commercial, multi-platform - best for professionals
  • Tera Term: Free, Windows - excellent XMODEM support
  • Screen/Minicom: Linux/Unix - command-line console access
  • ZTerm: macOS - good serial communication tool

File Transfer Servers

  • Tftpd64: Free TFTP/DHCP server for Windows
  • SolarWinds TFTP Server: Free, feature-rich TFTP server
  • FileZilla Server: Free FTP/FTPS server
  • Linux tftpd: Built-in TFTP daemon for Linux

Cisco Tools

  • Cisco Feature Navigator: Find compatible IOS versions
  • Cisco IOS Upgrade Planner: Plan IOS upgrades
  • TAC Case Collection Tool: Gather diagnostic information

Hardware Tools

Console Cables

  • RJ45-to-DB9: Traditional blue Cisco console cable
  • USB-to-Console: Modern USB-A or USB-C adapters
  • USB-to-Serial + RolloverCable: Alternative USB solution

Storage Media

  • USB Flash Drives: For USB-capable devices (FAT32 formatted)
  • Compact Flash Cards: For older routers and switches
  • SD Cards: For some Nexus and wireless devices

Setting Up a Recovery Laptop

Recommended Configuration

Recovery Laptop Setup:

1. Install Terminal Emulator
   - PuTTY or SecureCRT
   - Configure for 9600-8-N-1

2. Install TFTP Server
   - Tftpd64 or SolarWinds TFTP
   - Configure root directory for IOS images

3. Create IOS Image Library
   - Organize by platform
   - Include MD5 checksums
   - Keep multiple versions

4. Prepare Configuration Templates
   - Basic configs for each device type
   - Emergency recovery procedures
   - IP addressing schemes

5. Configure Static IP
   - Set laptop to 192.168.1.100/24
   - Disable firewall for TFTP temporarily

6. USB Drive Preparation
   - Format as FAT32
   - Copy common IOS images
   - Include basic configs
  

Common Recovery Scenarios and Solutions

Scenario 1: "Cannot Find IOS Image"

Symptoms:

boot: cannot determine first file name on device "flash:"
  

Solution:

! Enter ROMMON
rommon 1 > dir flash:
! If flash is empty or corrupted, use TFTP recovery

rommon 2 > IP_ADDRESS=192.168.1.10
rommon 3 > TFTP_SERVER=192.168.1.100
rommon 4 > TFTP_FILE=c2900-universalk9-mz.bin
rommon 5 > tftpdnld

! Or boot from USB
rommon 6 > dir usbflash0:
rommon 7 > boot usbflash0:c2900-universalk9-mz.bin
  

Scenario 2: Continuous Reload Loop

Symptoms:

  • Device keeps rebooting
  • Never reaches full boot
  • May show crash information

Solution:

! Send break during boot to enter ROMMON
rommon 1 > confreg 0x2142
rommon 2 > reset

! After boot without config
Router> enable
Router# configure terminal
Router(config)# config-register 0x2102

! Check for problematic configuration
Router# show startup-config
! Fix or remove problematic commands

Router# write memory
Router# reload
  

Scenario 3: Corrupted Configuration

Symptoms:

%Error opening flash:/startup-config (Bad file number)
  

Solution:

! Boot without configuration
rommon 1 > confreg 0x2142
rommon 2 > reset

! After boot
Router> enable
Router# delete flash:startup-config
Router# delete flash:private-config.text

! Restore from backup
Router# copy tftp://192.168.1.100/router-backup.cfg startup-config
Router# configure terminal
Router(config)# config-register 0x2102
Router(config)# exit
Router# reload
  

Scenario 4: Flash Memory Full

Symptoms:

%Error copying: Flash device is full
  

Solution:

Router# dir flash:
! Identify large files to delete

Router# delete flash:old-ios-image.bin
Router# delete /recursive flash:old-directory

! Permanently remove deleted files
Router# squeeze flash:

! Clean installer files
Router# request platform software package clean
  

Scenario 5: Wrong IOS Version

Symptoms:

%Error: IOS version mismatch
  

Solution:

! Boot old IOS from ROMMON
rommon 1 > dir flash:
rommon 2 > boot flash:old-compatible-ios.bin

! Or download correct version
Router# copy tftp://192.168.1.100/correct-ios.bin flash:
Router# configure terminal
Router(config)# boot system flash:correct-ios.bin
Router(config)# exit
Router# reload
  

Verification and Testing After Recovery

Post-Recovery Checklist

1. Verify Boot Process

Router# show version
Cisco IOS Software, C2900 Software (C2900-UNIVERSALK9-M), Version 15.7(3)M5
...
System returned to ROM by reload
System image file is "flash:c2900-universalk9-mz.SPA.157-3.M5.bin"
...
Configuration register is 0x2102

! Check boot variables
Router# show boot
BOOT path-list:      flash:c2900-universalk9-mz.SPA.157-3.M5.bin
Config file:         flash:startup-config
...
  

2. Verify Configurations

! Compare running and startup
Router# show running-config | redirect flash:running-temp.txt
Router# show startup-config | redirect flash:startup-temp.txt

! Visual check
Router# show running-config
Router# show startup-config

! Verify critical settings
Router# show running-config | include hostname
Router# show running-config | include enable secret
Router# show running-config | begin interface
  

3. Test Network Connectivity

! Check interface status
Router# show ip interface brief
Interface              IP-Address      OK? Method Status                Protocol
GigabitEthernet0/0     192.168.1.1     YES manual up                    up
GigabitEthernet0/1     10.0.0.1        YES manual up                    up

! Test routing
Router# show ip route

! Ping critical destinations
Router# ping 8.8.8.8
Router# ping 192.168.2.1
  

4. Verify Services

! Check routing protocols
Router# show ip protocols
Router# show ip ospf neighbor
Router# show ip bgp summary

! Verify NAT/ACLs
Router# show ip nat translations
Router# show access-lists

! Check security
Router# show users
Router# show privilege
  

5. Review Logs

Router# show logging
! Look for errors or warnings

Router# show logging | include Error
Router# show logging | include Failed
  

Performance Verification

Check Resource Utilization

Router# show processes cpu sorted
CPU utilization for five seconds: 5%/2%; one minute: 4%; five minutes: 3%

Router# show memory summary
                Head    Total(b)     Used(b)     Free(b)   Lowest(b)
Processor    654B2B10   492359540   122368972   369990568   367990568

Router# show interfaces stats
  

Baseline Performance

! Establish new baselines after recovery
Router# show processes cpu history

Router# show interfaces counters
Router# show environment all
  

Creating Recovery Documentation

Recovery Runbook Template

Create a standardized recovery runbook for your organization:

DEVICE RECOVERY RUNBOOK
======================

Device Information:
-------------------
Hostname: CORE-RTR-01
Model: Cisco 2911
Serial Number: FTX1234ABCD
Location: Main Data Center - Rack 42
Management IP: 192.168.100.10

Current Configuration:
---------------------
IOS Version: 15.7(3)M5
IOS File: c2900-universalk9-mz.SPA.157-3.M5.bin
Flash Size: 256 MB
RAM: 512 MB
Config Register: 0x2102

Backup Locations:
-----------------
Configuration Backup: tftp://192.168.1.100/backups/CORE-RTR-01/
IOS Images: tftp://192.168.1.100/ios-images/2900-series/
Archive Location: flash:/archive/

Recovery Contact Information:
----------------------------
Primary Admin: John Doe (john.doe@company.com, +1-555-0100)
Secondary Admin: Jane Smith (jane.smith@company.com, +1-555-0101)
TAC Case Number: (if applicable)

Password Recovery Procedure:
---------------------------
1. Connect console cable
2. Power cycle device
3. Send break within 60 seconds (Ctrl+Break in PuTTY)
4. At rommon prompt: confreg 0x2142
5. At rommon prompt: reset
6. Wait for boot (no config loaded)
7. Type: enable
8. Type: copy startup-config running-config
9. Type: configure terminal
10. Type: enable secret NewPassword123!
11. Type: config-register 0x2102
12. Type: exit
13. Type: write memory
14. Type: reload

IOS Recovery Procedure:
----------------------
TFTP Server IP: 192.168.1.100
Recovery Laptop IP: 192.168.1.10/24
IOS Filename: c2900-universalk9-mz.SPA.157-3.M5.bin

1. Enter ROMMON mode (break during boot)
2. rommon> IP_ADDRESS=192.168.1.10
3. rommon> IP_SUBNET_MASK=255.255.255.0
4. rommon> DEFAULT_GATEWAY=192.168.1.1
5. rommon> TFTP_SERVER=192.168.1.100
6. rommon> TFTP_FILE=c2900-universalk9-mz.SPA.157-3.M5.bin
7. rommon> tftpdnld
8. Wait for download to complete
9. rommon> boot flash:c2900-universalk9-mz.SPA.157-3.M5.bin

Configuration Recovery:
----------------------
1. Router# copy tftp://192.168.1.100/backups/CORE-RTR-01-latest.cfg startup-config
2. Router# reload
3. Verify configuration after boot

Critical Configuration Settings:
-------------------------------
Enable Secret: [Encrypted - stored securely]
VTY Password: [Encrypted - stored securely]
Console Password: [Encrypted - stored securely]

Management VLAN: 100
Management Interface: GigabitEthernet0/0
Default Gateway: 192.168.100.1

Testing After Recovery:
----------------------
1. Verify boot: show version
2. Check interfaces: show ip interface brief
3. Test routing: show ip route
4. Ping gateway: ping 192.168.100.1
5. Check services: show ip protocols
6. Verify NAT: show ip nat translations
7. Test remote access: ssh from management station

Rollback Procedure:
------------------
If recovery fails:
1. Boot old IOS from flash: rommon> boot flash:old-ios.bin
2. Restore previous config: copy flash:config-backup.old startup-config
3. Reload device

Last Updated: 2026-01-25
Updated By: Network Admin Team
  

Device Inventory Spreadsheet

Maintain a spreadsheet with recovery information for all devices:

Hostname Model Serial IOS Version Flash Size Config Backup Last Updated
CORE-RTR-01 Cisco 2911 FTX1234ABCD 15.7(3)M5 256MB tftp://backup/CORE-RTR-01/ 2026-01-25
DIST-SW-01 Catalyst 3850 FCW1234ABCD 16.12.4 512MB tftp://backup/DIST-SW-01/ 2026-01-20

Automation and Scripting

Automated Backup Script (Python)

#!/usr/bin/env python3
"""
Automated Cisco Device Configuration Backup
Connects to devices via SSH and saves configurations to TFTP
"""

import paramiko
import time
from datetime import datetime

def backup_device(hostname, username, password, tftp_server):
    """
    Backup configuration from Cisco device to TFTP server
    """
    try:
        # Connect via SSH
        ssh = paramiko.SSHClient()
        ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
        ssh.connect(hostname, username=username, password=password)
        
        # Get device hostname
        stdin, stdout, stderr = ssh.exec_command("show run | include hostname")
        device_name = stdout.read().decode().split()[1]
        
        # Create timestamp
        timestamp = datetime.now().strftime("%Y%m%d-%H%M%S")
        backup_file = f"{device_name}-{timestamp}.cfg"
        
        # Execute backup command
        command = f"copy running-config tftp://{tftp_server}/{backup_file}\n\n\n"
        stdin, stdout, stderr = ssh.exec_command(command)
        
        # Wait for completion
        time.sleep(5)
        
        print(f"✓ Backup completed: {hostname} -> {backup_file}")
        ssh.close()
        return True
        
    except Exception as e:
        print(f"✗ Backup failed for {hostname}: {str(e)}")
        return False

# Device list
devices = [
    {'hostname': '192.168.1.1', 'username': 'admin', 'password': 'password'},
    {'hostname': '192.168.1.2', 'username': 'admin', 'password': 'password'},
    {'hostname': '192.168.1.3', 'username': 'admin', 'password': 'password'},
]

# TFTP server
tftp_server = "192.168.1.100"

# Backup all devices
print("Starting automated backup...")
for device in devices:
    backup_device(device['hostname'], device['username'], 
                  device['password'], tftp_server)

print("Backup process completed.")
  

Configuration Verification Script (Bash)

#!/bin/bash
# Cisco Configuration Verification Script
# Checks critical configuration parameters after recovery

DEVICE_IP="192.168.1.1"
USERNAME="admin"
PASSWORD="password"

echo "=== Cisco Device Verification Script ==="
echo "Device: $DEVICE_IP"
echo "Time: $(date)"
echo ""

# Function to execute command via SSH
exec_command() {
    sshpass -p "$PASSWORD" ssh -o StrictHostKeyChecking=no \
    ${USERNAME}@${DEVICE_IP} "$1"
}

# Check device is reachable
echo "[1] Testing connectivity..."
if ping -c 3 $DEVICE_IP > /dev/null 2>&1; then
    echo "✓ Device is reachable"
else
    echo "✗ Device is NOT reachable"
    exit 1
fi

# Check IOS version
echo ""
echo "[2] Checking IOS version..."
exec_command "show version | include IOS"

# Check configuration register
echo ""
echo "[3] Checking configuration register..."
exec_command "show version | include Configuration register"

# Check boot system
echo ""
echo "[4] Checking boot configuration..."
exec_command "show boot"

# Check interfaces
echo ""
echo "[5] Checking interface status..."
exec_command "show ip interface brief"

# Check routing
echo ""
echo "[6] Checking routing table..."
exec_command "show ip route | begin Gateway"

# Check for errors
echo ""
echo "[7] Checking for errors in logs..."
exec_command "show logging | include Error"

echo ""
echo "=== Verification Complete ==="
  

EEM Script for Automatic Config Backup

Configure Embedded Event Manager to automatically backup configuration on changes:

! EEM script to backup config on every save
Router(config)# event manager applet CONFIG-BACKUP
Router(config-applet)# event syslog pattern "%SYS-5-CONFIG_I"
Router(config-applet)# action 1.0 cli command "enable"
Router(config-applet)# action 2.0 cli command "copy running-config tftp://192.168.1.100/$h-$t.cfg" pattern "Address"
Router(config-applet)# action 2.1 cli command "192.168.1.100" pattern "filename"
Router(config-applet)# action 2.2 cli command "$h-$t.cfg" pattern "confirm"
Router(config-applet)# action 2.3 cli command ""
Router(config-applet)# action 3.0 syslog msg "Configuration backed up to TFTP server"

! Archive on write-memory
Router(config)# archive
Router(config-archive)# log config
Router(config-archive-log-cfg)# logging enable
Router(config-archive-log-cfg)# notify syslog
Router(config-archive)# path tftp://192.168.1.100/archive/$h-$t
Router(config-archive)# write-memory
  

Emergency Recovery Kit

Physical Recovery Kit Components

Prepare an emergency recovery kit for quick response to device failures:

Hardware Components

  • Console Cables:
    • 2x RJ45-to-DB9 (blue Cisco cables)
    • 2x USB-to-Console adapters
    • 1x USB-to-Serial adapter with rollover cable
  • Network Cables:
    • 3x CAT6 ethernet cables (various lengths)
    • 1x Crossover cable (for direct connections)
  • Storage Media:
    • 2x USB flash drives (32GB, FAT32) with common IOS images
    • 1x Compact Flash card with reader
    • 1x External hard drive with complete IOS library
  • Tools:
    • Small screwdriver set
    • Label maker or labels
    • Flashlight
    • Cable tester

Software Components

  • Recovery Laptop:
    • Laptop with console port or USB ports
    • Windows/Linux with terminal software installed
    • TFTP/FTP server software configured
    • Static IP configured (192.168.1.100/24)
  • Software Installed:
    • PuTTY or SecureCRT
    • Tftpd64 or SolarWinds TFTP Server
    • FileZilla FTP Server
    • Wireshark (for troubleshooting)
    • SSH client (PuTTY/OpenSSH)

Documentation

  • Printed recovery procedures for each device type
  • Network diagrams and IP addressing schemes
  • Device inventory with serial numbers
  • Configuration templates
  • TAC support contact information
  • Escalation procedures

Recovery USB Drive Structure

RECOVERY-USB/
├── IOS-Images/
│   ├── Router-1900-2900/
│   │   ├── c2900-universalk9-mz.SPA.157-3.M5.bin
│   │   └── MD5-checksums.txt
│   ├── Router-4000/
│   │   ├── isr4300-universalk9.16.12.04.SPA.bin
│   │   └── MD5-checksums.txt
│   ├── Switch-2960/
│   │   ├── c2960-lanbasek9-mz.150-2.SE11.bin
│   │   └── MD5-checksums.txt
│   └── Switch-3850/
│       ├── cat3k_caa-universalk9.16.12.04.SPA.bin
│       └── MD5-checksums.txt
├── Configurations/
│   ├── Templates/
│   │   ├── router-basic-template.txt
│   │   ├── switch-access-template.txt
│   │   └── switch-distribution-template.txt
│   └── Backups/
│       ├── CORE-RTR-01-latest.cfg
│       ├── DIST-SW-01-latest.cfg
│       └── ACCESS-SW-01-latest.cfg
├── Documentation/
│   ├── Recovery-Procedures.pdf
│   ├── Network-Diagram.pdf
│   ├── IP-Address-Plan.xlsx
│   └── Device-Inventory.xlsx
└── Tools/
    ├── putty.exe
    ├── tftpd64.exe
    └── md5sum.exe
  

Troubleshooting Recovery Failures

TFTP Transfer Failures

Problem: TFTP Timeout

%Error opening tftp://192.168.1.100/ios.bin (Timed out)
  

Solutions:

  • Check connectivity:
    rommon 1 > ping 192.168.1.100
  • Verify TFTP server is running: Check TFTP service on server
  • Check firewall: Temporarily disable firewall or allow UDP port 69
  • Verify IP configuration:
    rommon 2 > set
    ! Check IP_ADDRESS, IP_SUBNET_MASK, TFTP_SERVER
  • Use direct connection: Connect device directly to TFTP server

Problem: File Not Found

%Error opening tftp://192.168.1.100/ios.bin (No such file)
  

Solutions:

  • Verify exact filename (case-sensitive on Linux servers)
  • Check TFTP server root directory
  • Verify file permissions on TFTP server
  • Use shorter filename if too long

ROMMON Issues

Problem: Cannot Enter ROMMON

Solutions:

  • Try different break sequences:
    • Ctrl + Break (Windows)
    • Ctrl + Shift + 6, then X (macOS)
    • Alt + B (some terminal programs)
  • Check terminal settings: Verify baud rate is 9600
  • Hold Mode button: For switches, use Mode button method
  • Timing: Send break within first 60 seconds of boot

Problem: ROMMON Commands Not Working

rommon 1 > tftpdnld
Invalid command
  

Solutions:

  • Command may not be available on all platforms
  • Try alternative: tftp instead of tftpdnld
  • Use ? to see available commands
  • Check platform-specific documentation

Boot Failures After Recovery

Problem: IOS Loads But Crashes

Solutions:

  • Check memory: IOS may require more RAM
    Router# show version | include memory
  • Verify IOS compatibility: Ensure IOS matches platform
  • Check MD5 checksum: File may be corrupted
    Router# verify /md5 flash:ios-image.bin
  • Download IOS again: Re-transfer from trusted source

Problem: Configuration Not Loading

Solutions:

  • Verify config register:
    Router# show version | include Configuration register
    ! Should be 0x2102, not 0x2142
  • Check boot system:
    Router# show boot
  • Manually load config:
    Router# copy flash:startup-config running-config

When to Contact Cisco TAC

Situations Requiring TAC Support

  • Hardware Failures:
    • Flash memory completely failed
    • Power supply issues
    • Boot ROM corruption
    • Physical damage to device
  • Software Issues:
    • IOS bugs causing boot failures
    • Licensing issues preventing boot
    • Persistent crashes after recovery
  • Recovery Failures:
    • All recovery methods exhausted
    • ROMMON not accessible
    • Device completely unresponsive
  • Critical Production Issues:
    • Urgent recovery needed
    • Uncertainty about procedures
    • Risk of data loss

Preparing for TAC Case

Information to Gather

TAC Case Information Checklist:

Device Details:
□ Device model and part number
□ Serial number
□ Current IOS version (if accessible)
□ Hardware revision
□ Memory and flash specifications

Problem Description:
□ Detailed symptoms
□ When problem started
□ What changed recently
□ Error messages (exact text)
□ Steps already attempted

Network Information:
□ Device role and location
□ Network topology
□ Connected devices
□ Business impact

Documentation:
□ Console output logs
□ Configuration files
□ Show tech-support output
□ Photos of error messages
  

Useful TAC Commands

! Comprehensive diagnostic output
Router# show tech-support

! Save output to file
Router# show tech-support | redirect flash:show-tech.txt

! Copy to TFTP for TAC
Router# copy flash:show-tech.txt tftp://192.168.1.100/

! Show logging
Router# show logging

! Environment status
Router# show environment all

! Hardware inventory
Router# show inventory
  

TAC Contact Information

  • Web: https://www.cisco.com/c/en/us/support/index.html
  • Phone (Worldwide): Available on Cisco Support page
  • Case Priority Levels:
    • P1 (Critical): Production network down
    • P2 (High): Significant degradation
    • P3 (Medium): Minor feature not working
    • P4 (Low): General questions

Conclusion

Recovery procedures are essential skills for network administrators managing Cisco infrastructure. Whether dealing with forgotten passwords, corrupted IOS images, or complete system failures, understanding the proper recovery techniques can minimize downtime and prevent data loss.

Key Takeaways

  • Prevention is Better Than Recovery: Regular backups and proper change management prevent most recovery scenarios
  • Physical Access Required: Most recovery procedures require console access and physical presence
  • Document Everything: Maintain detailed recovery procedures and device inventory
  • Test Recovery Procedures: Practice recovery in lab environments before emergencies
  • Prepare Recovery Tools: Keep emergency recovery kit with necessary cables, software, and documentation
  • Security Matters: Password recovery bypasses security - ensure physical security of devices
  • Know Your Limits: Contact Cisco TAC when facing hardware failures or exhausted options

Final Recommendations

  • Implement automated configuration backup systems
  • Maintain current IOS image repository
  • Create and test disaster recovery plans
  • Train team members on recovery procedures
  • Keep recovery documentation updated
  • Perform regular recovery drills
  • Monitor devices for early warning signs
  • Maintain valid Cisco support contracts

With proper preparation, documentation, and practice, you can confidently handle any Cisco device recovery scenario. Remember that recovery procedures are your safety net - invest time in preparing them properly, and they'll serve you well when you need them most.