My DPM2010 System State protection for another DPM 2010 Server started failing all of a sudden. Windows server backup also fails on the protected server when I run it manually. I know that if the wbadmin doesn't work then DPM can't work either. This problem appears to be an issue on the protected server with wbadmin and numerous writer errors. I'm including the details I've collected at this point. Thank you for reading this thread...any helpful ideas would be greatly appreciated.
Windows Server Backup error message:
C:\Windows\system32>wbadmin start systemstatebackup -backuptarget:m:
wbadmin 1.0 - Backup command-line tool
(C) Copyright 2004 Microsoft Corp.
Starting System State Backup [3/25/2014 7:05 AM]
Retrieving volume information...
This would backup the system state from volume(s) Local Disk(C:),DPM Vol(D:) to m:.
Do you want to start the backup operation?
[Y] Yes [N] No Y
Creating the shadow copy of volumes requested for backup.
Creating the shadow copy of volumes requested for backup.
Creating the shadow copy of volumes requested for backup.
Creating the shadow copy of volumes requested for backup.
Creating the shadow copy of volumes requested for backup.
Creating the shadow copy of volumes requested for backup.
Summary of backup:
------------------
Backup of system state failed [3/25/2014 7:11 AM]
Log of files successfully backed up
'C:\Windows\Logs\WindowsServerBackup\SystemStateBackup 25-03-2014 07-10-05.log'
Log of files for which backup failed
'C:\Windows\Logs\WindowsServerBackup\SystemStateBackup_Error 25-03-2014 07-10-05.log'
Failure in a Volume Shadow Copy Service operation.
ERROR - Volume Shadow Copy Service operation error (0x800423f3)
The writer experienced a transient error. If the backup process is retried,
the error may not reoccur.
DPM protection error message:
Affected area: Computer\System Protection
Occurred since: 3/24/2014 7:04:24 AM
Description: The replica of System Protection Computer\System Protection on [Server Name] is inconsistent with the protected data source. All protection activities for data source will fail until the replica is synchronized with consistency check. You can recover data from existing recovery points, but new recovery points cannot be created until the replica is consistent.
For SharePoint farm, recovery points will continue getting created with the databases that are consistent. To backup inconsistent databases, run a consistency check on the farm. (ID 3106)
DPM cannot create a backup because Windows Server Backup (WSB) on the protected computer encountered an error (WSB Event ID: 521, WSB Error Code: 0x807800A1). (ID 30229 Details: Internal error code: 0x809909FB)
More information
Recommended action: For resolution actions and more information on the WSB error, go to
http://technet.microsoft.com/en-us/library/cc734488(WS.10).aspx.
Synchronize with consistency check.
Run a synchronization job with consistency check...
Resolution: To dismiss the alert, click below
Inactivate alert
Windows Event Viewer log error messages:
Log Name: Application
Source: VSS
Date: 3/25/2014 7:10:59 AM
Event ID: 8204
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: [Server Name]
Description:
Volume Shadow Copy Service error: An invalid XML document was returned from writer {d156f92e-363a-471e-a70f-34b4536e86b4}.
Operation:
Writer Modifying Modifying Backup Document
Context:
Execution Context: Requestor
Writer Instance ID: {D156F92E-363A-471E-A70F-34B4536E86B4}
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="VSS" />
<EventID Qualifiers="0">8204</EventID>
<Level>2</Level>
<Task>0</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2014-03-25T14:10:59.000Z" />
<EventRecordID>233016</EventRecordID>
<Channel>Application</Channel>
<Computer> [Server Name] </Computer>
<Security />
</System>
<EventData>
<Data>{d156f92e-363a-471e-a70f-34b4536e86b4}</Data>
<Data>
Operation:
Writer Modifying Modifying Backup Document
Context:
Execution Context: Requestor
Writer Instance ID: {D156F92E-363A-471E-A70F-34B4536E86B4}</Data>
<Binary>2D20436F64653A2042554543584D4C4330303030303232332D2043616C6C3A2042554543584D4C4330303030303139302D205049443A202030303030303436382D205449443A202030303030303432342D20434D443A202022433A5C57696E646F77735C73797374656D33325C7762656E67696E652E657865222020202020202D20557365723A204E5420415554484F524954595C53595354454D20202020202D205369643A2020532D312D352D3138</Binary>
</EventData>
</Event>
______________________________________________________________________________
Log Name: Application
Source: Microsoft-Windows-Backup
Date: 3/25/2014 7:11:05 AM
Event ID: 521
Task Category: None
Level: Error
Keywords:
User: SYSTEM
Computer: [Server Name]
Description:
Backup started at '3/25/2014 2:10:05 PM' failed as Volume Shadow copy operation failed for backup volumes with following error code '2155348129'. Please rerun backup once issue is resolved.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Microsoft-Windows-Backup" Guid="{1db28f2e-8f80-4027-8c5a-a11f7f10f62d}" />
<EventID>521</EventID>
<Version>0</Version>
<Level>2</Level>
<Task>0</Task>
<Opcode>0</Opcode>
<Keywords>0x8000000000000000</Keywords>
<TimeCreated SystemTime="2014-03-25T14:11:05.032Z" />
<EventRecordID>233017</EventRecordID>
<Correlation />
<Execution ProcessID="468" ThreadID="6540" />
<Channel>Application</Channel>
<Computer> [Server Name] </Computer>
<Security UserID="S-1-5-18" />
</System>
<EventData>
<Data Name="BackupTime">2014-03-25T14:10:05.862Z</Data>
<Data Name="ErrorCode">2155348129</Data>
<Data Name="ErrorMessage">%%2155348129</Data>
</EventData>
</Event>
Writer status before backup attemp:
Writer status after backup attempt:
C:\Windows\system32>vssadmin.exe list writers
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.
Writer name: 'DPM Writer'
Writer Id: {0074a271-4f24-43e0-9ec7-28a44189cb80}
Writer Instance Id: {f63d8406-777c-4ad5-a000-c1fc509673fc}
State: [1] Stable
Last error: No error
Writer name: 'System Writer'
Writer Id: {e8132975-6f93-4464-a53e-1050253ae220}
Writer Instance Id: {bcc605de-b52e-4ebe-8712-341e63c1c4c2}
State: [7] Failed
Last error: No error
Writer name: 'ASR Writer'
Writer Id: {be000cbe-11fe-4426-9c58-531aa6355fc4}
Writer Instance Id: {d156f92e-363a-471e-a70f-34b4536e86b4}
State: [7] Failed
Last error: Retryable error
Writer name: 'SqlServerWriter'
Writer Id: {a65faa63-5ea8-4ebc-9dbd-a0c4db26912a}
Writer Instance Id: {a21298a8-352d-4d27-acc8-531eb9dcd5d8}
State: [7] Failed
Last error: No error
Writer name: 'Shadow Copy Optimization Writer'
Writer Id: {4dc3bdd4-ab48-4d07-adb0-3bee2926fd7f}
Writer Instance Id: {37c43d93-ed3a-40f7-828e-b3b370d828e7}
State: [7] Failed
Last error: No error
Writer name: 'WMI Writer'
Writer Id: {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}
Writer Instance Id: {e1718057-885e-4008-b1f2-2c741e5ce2bc}
State: [7] Failed
Last error: No error
Writer name: 'Registry Writer'
Writer Id: {afbab4a2-367d-4d15-a586-71dbb18f8485}
Writer Instance Id: {e2a833f4-fcb5-4705-aedd-28e7e63a0a1d}
State: [7] Failed
Last error: No error
Writer name: 'IIS Config Writer'
Writer Id: {2a40fd15-dfca-4aa8-a654-1f8c654603f6}
Writer Instance Id: {35c45908-d4a0-45fa-bd4d-147a95dd3977}
State: [7] Failed
Last error: No error
Writer name: 'COM+ REGDB Writer'
Writer Id: {542da469-d3e1-473c-9f4f-7847f01fc64f}
Writer Instance Id: {546d77e5-ea29-4e14-846a-394cc2f3009a}
State: [7] Failed
Last error: No error
Writer name: 'IIS Metabase Writer'
Writer Id: {59b1f0cf-90ef-465f-9609-6ca8b2938366}
Writer Instance Id: {a536a87b-1693-403a-9580-6e537e1624a5}
State: [7] Failed
Last error: No error
C:\Windows\system32>vssadmin.exe list writers
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.
Writer name: 'System Writer'
Writer Id: {e8132975-6f93-4464-a53e-1050253ae220}
Writer Instance Id: {bcc605de-b52e-4ebe-8712-341e63c1c4c2}
State: [1] Stable
Last error: No error
Writer name: 'DPM Writer'
Writer Id: {0074a271-4f24-43e0-9ec7-28a44189cb80}
Writer Instance Id: {f63d8406-777c-4ad5-a000-c1fc509673fc}
State: [1] Stable
Last error: No error
Writer name: 'ASR Writer'
Writer Id: {be000cbe-11fe-4426-9c58-531aa6355fc4}
Writer Instance Id: {d156f92e-363a-471e-a70f-34b4536e86b4}
State: [1] Stable
Last error: No error
Writer name: 'SqlServerWriter'
Writer Id: {a65faa63-5ea8-4ebc-9dbd-a0c4db26912a}
Writer Instance Id: {a21298a8-352d-4d27-acc8-531eb9dcd5d8}
State: [1] Stable
Last error: No error
Writer name: 'IIS Config Writer'
Writer Id: {2a40fd15-dfca-4aa8-a654-1f8c654603f6}
Writer Instance Id: {35c45908-d4a0-45fa-bd4d-147a95dd3977}
State: [1] Stable
Last error: No error
Writer name: 'Registry Writer'
Writer Id: {afbab4a2-367d-4d15-a586-71dbb18f8485}
Writer Instance Id: {e2a833f4-fcb5-4705-aedd-28e7e63a0a1d}
State: [1] Stable
Last error: No error
Writer name: 'Shadow Copy Optimization Writer'
Writer Id: {4dc3bdd4-ab48-4d07-adb0-3bee2926fd7f}
Writer Instance Id: {37c43d93-ed3a-40f7-828e-b3b370d828e7}
State: [1] Stable
Last error: No error
Writer name: 'WMI Writer'
Writer Id: {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}
Writer Instance Id: {e1718057-885e-4008-b1f2-2c741e5ce2bc}
State: [1] Stable
Last error: No error
Writer name: 'COM+ REGDB Writer'
Writer Id: {542da469-d3e1-473c-9f4f-7847f01fc64f}
Writer Instance Id: {546d77e5-ea29-4e14-846a-394cc2f3009a}
State: [1] Stable
Last error: No error
Writer name: 'IIS Metabase Writer'
Writer Id: {59b1f0cf-90ef-465f-9609-6ca8b2938366}
Writer Instance Id: {a536a87b-1693-403a-9580-6e537e1624a5}
State: [1] Stable
Last error: No error