We've seen this in our environment for a while. Tried moving backup times around, that helped some but not in every case. TCP Chimney Offload didn't seem to be our issue either, though it has been for others.
Got this nugget courtesy of kgerde, which seems to be a near-magic-bullet for us.
Create a New DWORD value with the name: SessTimeout
set the value: 360 keep it Hexadecimal
(This value might not work for your backup but it was high enough for mine. If this doesn't work increase the value and try again.)