Download Click: Veeam 9 HyperV 2012 R2 and public Lan Fault during backup

Thursday, October 27, 2016

Veeam 9 HyperV 2012 R2 and public Lan Fault during backup

During last months we faced a strange problem on Cluster HyperV 2012 R2 VM backup.

Infact, randomly, with Veeam 9 we got a single hyperV public lan to be unreachable causing backup failure (Heartbit lan was working fine)

Here they are actions taken trying to solve issue:

1. When a single host was unreachable with public lan we connected to freeze host using Heartbeat lan via RDP.

We stopped any Veeam processes and services to finish veeam backup (VM was always in backing up mode in HyperV console)

2. There was not any solution to put public lan to working again other than reboot host or disabling and renabling lan connection.

3. Backup faults happened again utilizing Veeam Throttling feature and using windows backup tools indeed than Veeam itself.

4. Finally we suspected that problem was related to lan driver itself.

Searching on internet Lan device name and server model we found that:

HP DL380 Gen8 servers as Windows 2012 R2 Hyper-V hosts and HP Ethernet 1Gb 4-port 331FLR / Broadcom NetXtreme

http://serverfault.com/questions/742520/any-known-fixes-for-hyper-v-host-networking-randomly-stops-responding

What really helped us in our case, was not only to disable the VMQ in all affected Network adapters properties as suggested by Microsoft, but also to disable it in the Virtual machines properties (to uncheck ?Enable Virtual Machine Queue? under each of the virtual machine settings). By default is it enabled.

From Miscrosoft Article:

https://support.microsoft.com/en-us/kb/2986895

Resolution

This issue is resolved in Broadcom driver b57nd60a.sys version 16.8 and newer. In March 2015, Broadcom published driver version 17.0 for download. In April 2015, HP published version 16.8 of the driver for their affected network adapters, here is one of their download locations. Please contact your server OEM if you need a driver that is specific to your server.

More information

If you are unable to update your network adapter driver to resolve the issue, you can work around the issue by disabling VMQ on each affected Broadcom network adapter by using the Set-NetAdapterVmq Windows PowerShell command. For example, if you have a dual-port network adapter, and if the ports are named NIC 1 and NIC 2 in Windows, you would disable VMQ on each adapter by using the following commands:

Set-NetAdapterVmq -Name "NIC 1" -Enabled $False
Set-NetAdapterVmq -Name "NIC 2" -Enabled $False
You can confirm that VMQ is disabled on the correct network adapters by using the Get-NetAdapterVmq Windows PowerShell command.

Note By default, VMQ is disabled on the Hyper-V virtual switch for virtual machines that are using 1-gigabit network adapters. VMQ is enabled on a Hyper-V virtual switch only when the system is using 10-gigabit or faster network adapters. This means that by disabling VMQ on the Broadcom network adapter, you are not losing network performance or any other benefits because this is the default. However, you need to do this to work around the driver issue.

Get-NetAdapterVmqQueue shows the virtual machine queues (VMQs) that are allocated on network adapters. You will not see any virtual machine queues that are allocated to 1-gigabit network adapters by default.

Available link for download