I have a 2 node Hyper V Cluster backed up via Veeam Off-Host Proxy. I've already logged, without answer, a thread about the Off-Host Proxy server randomly (I guess) creating new duplicate iSCSI connections.
Today I have a single VM that won't backup. On inspection through the hosting servers Application Log I see a high amount of VSS errors being logged which I believe is possibly related to the iSCSI issue being logged by the SAN.
I'm not sure how to "show" the issue without a giant wall of the events but there is like half a dozen different events being logged when whatever this is is happening. For instance as well last night 60/61 VMs backed up find, but these errors
were logged constantly while the backup ran...
The question as well is is this a Windows/VSS issue, a Veeam issue or a Equallogic issue?
Also how do I find what '\\?\Volume{06fba49e-9519-11e4-80cc-000af75dc050}\'. actually is?
In order of how they come I guess:
Log Name: Application
Source: VSS
Date: 6/01/2015 7:28:19 AM
Event ID: 8229
Task Category: None
Level: Warning
Keywords: Classic
User: N/A
Computer: HOST02.domain.private
Description:
A VSS writer has rejected an event with error 0x800423f3, The writer experienced a transient error. If the backup process is retried,
the error may not reoccur.
. Changes that the writer made to the writer components while handling the event will not be available to the requester. Check the event log for related events from the application hosting the VSS writer.
Operation:
PrepareForSnapshot Event
Context:
Execution Context: Writer
Writer Class Id: {66841cd4-6ded-4f4b-8f17-fd23f8ddc3de}
Writer Name: Microsoft Hyper-V VSS Writer
Writer Instance ID: {881f5207-f769-4b40-986a-c6bd56d8aa1e}
Command Line: C:\Windows\system32\vmms.exe
Process ID: 3268
Log Name: Application
Source: VSS
Date: 6/01/2015 7:29:31 AM
Event ID: 8193
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: HOST02.domain.private
Description:
Volume Shadow Copy Service error: Unexpected error calling routine Error calling CreateFile on volume '\\?\Volume{06fba49e-9519-11e4-80cc-000af75dc050}\'. hr = 0x80070037, The specified network resource or device is no longer available.
.
Operation:
Check If Volume Is Supported by Provider
Context:
Execution Context: Coordinator
Provider ID: {d4689bdf-7b60-4f6e-9afb-2d13c01b12ea}
Volume Name: \\?\Volume{06fba49e-9519-11e4-80cc-000af75dc050}\
Log Name: Application
Source: EqualLogic
Date: 6/01/2015 7:29:43 AM
Event ID: 4001
Task Category: VSS
Level: Error
Keywords: Classic
User: N/A
Computer: HOST02.domain.private
Description:
iSCSI logout error 0xEFFF0040 from target NULL.
Log Name: Application
Source: VSS
Date: 6/01/2015 7:29:43 AM
Event ID: 12293
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: HOST02.domain.private
Description:
Volume Shadow Copy Service error: Error calling a routine on a Shadow Copy Provider {d4689bdf-7b60-4f6e-9afb-2d13c01b12ea}. Routine details OnLunStateChange(\\?\mpio#disk&ven_eqlogic&prod_100e-00&rev_7.0_#1&7f6ac24&0&363846433631363644434545463233334146344141353632363038303136#{53f56307-b6bf-11d0-94f2-00a0c91efb8b})
failed with error 0xefff0040 [hr = 0xefff0040].
Operation:
Notifying hardware provider to free a drive
Break with LUN mask
Delete Shadow Copies
Processing PostFinalCommitSnapshots
Executing Asynchronous Operation
Context:
Volume Name: \\?\mpio#disk&ven_eqlogic&prod_100e-00&rev_7.0_#1&7f6ac24&0&363846433631363644434545463233334146344141353632363038303136#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}
Volume Name: \\?\Volume{06fba49e-9519-11e4-80cc-000af75dc050}
Snapshot ID: {3091377b-2e26-43ed-a11b-f31adbde0b1f}
Execution Context: Provider
Provider Name: Dell EqualLogic VSS HW Provider
Provider Version: 4.7.1
Provider ID: {d4689bdf-7b60-4f6e-9afb-2d13c01b12ea}
Snapshot Context: 4194336
Provider Name: Dell EqualLogic VSS HW Provider
Provider Version: 4.7.1
Provider ID: {d4689bdf-7b60-4f6e-9afb-2d13c01b12ea}
Current State: DoSnapshotSet
Log Name: Application
Source: VSS
Date: 6/01/2015 7:29:43 AM
Event ID: 12293
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: HOST02.domain.private
Description:
Volume Shadow Copy Service error: Error calling a routine on a Shadow Copy Provider {d4689bdf-7b60-4f6e-9afb-2d13c01b12ea}. Routine details could not free LUN [hr = 0x8004230f, The shadow copy provider had an unexpected error while trying to process the
specified operation.
].
Operation:
Break with LUN mask
Delete Shadow Copies
Processing PostFinalCommitSnapshots
Executing Asynchronous Operation
Context:
Volume Name: \\?\Volume{06fba49e-9519-11e4-80cc-000af75dc050}
Snapshot ID: {3091377b-2e26-43ed-a11b-f31adbde0b1f}
Execution Context: Provider
Provider Name: Dell EqualLogic VSS HW Provider
Provider Version: 4.7.1
Provider ID: {d4689bdf-7b60-4f6e-9afb-2d13c01b12ea}
Snapshot Context: 4194336
Provider Name: Dell EqualLogic VSS HW Provider
Provider Version: 4.7.1
Provider ID: {d4689bdf-7b60-4f6e-9afb-2d13c01b12ea}
Current State: DoSnapshotSet
(Below logged 3 times)
Log Name: Application
Source: VSS
Date: 6/01/2015 7:29:49 AM
Event ID: 8193
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: HOST02.domain.private
Description:
Volume Shadow Copy Service error: Unexpected error calling routine Error calling CreateFile on volume '\\?\Volume{06fba49e-9519-11e4-80cc-000af75dc050}\'. hr = 0x80070037, The specified network resource or device is no longer available.
.
Operation:
Check If Volume Is Supported by Provider
Context:
Execution Context: Coordinator
Provider ID: {d4689bdf-7b60-4f6e-9afb-2d13c01b12ea}
Volume Name: \\?\Volume{06fba49e-9519-11e4-80cc-000af75dc050}\