While setting up my gigabit ethernet network, during simulataneous Windows 7 file transfers to the HTPC+B setup, I've received a number of disconnects, virtually every time while transferring my files.  At the point, I not only lost connectivity with the HTPC+B but also my web access most of the time.  Here are the culprits and here's how we went about resolving them:

WINDOWS:
An error occurred while reconnecting Y: to \\192.168.0.14\HTPCFirstEntertainment Microsoft Windows Network: The local device name is already in use.  This connection has not been restored.

WireShark:
Header checksum: 0x0000 [incorrect, should be 0xd7e7 (may be caused by “IP checksum offload”?)]
TCP Dup ACK microsoft-ds (port 445)

WINDOWS:
There is a problem accessing Z:\

Make sure you are connected to the network and try again.
HTPCFileStorage (\\192.168.0.14)
 

The first thing we noticed from WireShark is that there was alot of checksum errors on the line.  So we went to check where this setting is enabled to see if toggling it would help:

Here's how to disable CSUM on the NIC in Windows 7:

1. Click the Start menu.
2. Next select Control Panel
3. Select Network and Internet
4. Select Network and Sharing Center
5. Click on your interface (ie Local Area Connection is a typical name)
6. On the panel that appears, click Properties button.
7. Click Configure button.
8. Click the Advanced tab.
9. Look for properties that have the word Checksum in it.  Disable as necessary recording original value.

List of properties on my system:

IPV4 Checksum Offload
TCP CHecksum Offload (IPv4)
TCP CHecksum Offload (IPv6)

UDP CHecksum Offload (IPv4)
UDP CHecksum Offload (IPv6)

However, the rather surprising and unexpected effect of this was that the network speed slowed down to roughly 1MB/s at most (From it's 1GB/s max theoretical speed of 125MB/s).  (Interestingly, we will see why that has happened further down but it is not because of turning off Checksum Offload.)

So I've reverted everything back to the way it was in the two Windows 7 workstations to see if the speed will revert back to the quicker speeds earlier but it remained at 1MB/s.  One interesting thing to note here is that the disconnects only happen when two or more workstations are transferring files.  But not a single one.

At this point I checked the errors / messages in /var/log/samba/*.logs to see if resolving these will bring joy to this issue.

The first error we notice is from the samba logs is:

getpeername failed. Error was Transport endpoint is not connected

For this error, accordingly, we check what global settings are available and / or set to:

# cat /etc/samba/smb.conf|grep -Eiv ";|#"|sed "/^[ \n\t]*$/d"
[global]
        workgroup = WORKGROUP
        server string = Samba Server Version %v
        log file = /var/log/samba/log.%m
        max log size = 50
        security = user
        passdb backend = tdbsam
        load printers = yes
        cups options = raw

netsh interface tcp show global
netsh interface tcp set global autotuning=disabled
netsh interface tcp show global
netsh interface tcp set global autotuning=normal

After tinkering around with the SMB config file, nothing appeared to change the windows behaviour above.  Looking a bit deeper and into the network itself, I notice the speed was renagotiated to 10Mbits/s.  (This explains the horrible download and upload times):

# mii-tool bond0
bond0: 10 Mbit, half duplex, link ok
[root@mbpc samba]# mii-tool eth0
eth0: negotiated 10baseT-FD flow-control, link ok
#

And even deeper, I see that the advertised modes are not the fastest possible for the GBE NIC:

# ethtool eth0
Settings for eth0:
        Supported ports: [ TP MII ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Half 1000baseT/Full
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Half 1000baseT/Full
        Advertised pause frame use: No
        Advertised auto-negotiation: Yes
        Link partner advertised link modes:  10baseT/Half 10baseT/Full
        Link partner advertised pause frame use: No
        Link partner advertised auto-negotiation: Yes
        Speed: 10Mb/s
        Duplex: Full
        Port: MII
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: on
        Supports Wake-on: pumbg
        Wake-on: g
        Current message level: 0x00000033 (51)
        Link detected: yes
# ethtool bond0
Settings for bond0:
        Link detected: yes
# mii-tool bond0
bond0: 10 Mbit, half duplex, link ok
#

 

Any alternate CAT NET cable plugged into the system and then directly to the gigabit router results in the same thing: 10 Mbit, half duplex, link ok.

So this is leading us to check the drivers itself (before we actually go and buy a new NIC card).  For this we'll check if there isn't a latest kernel version available that can be used here, potentially including an updated driver.  (Let's try the easy stuff first.)

yum update kernel

This should show you if there is any new kernel available.  If there is, try the upgrade:

# rpm -aq|grep kernel
dracut-kernel-004-53.el6.noarch
kernel-2.6.32-220.4.1.el6.x86_64
kernel-headers-2.6.32-220.4.1.el6.x86_64
kernel-doc-2.6.32-220.4.1.el6.noarch
kernel-firmware-2.6.32-220.4.1.el6.noarch
kernel-debug-2.6.32-220.4.1.el6.x86_64
kernel-2.6.32-131.0.15.el6.x86_64
kernel-debug-devel-2.6.32-220.4.1.el6.x86_64
#

 

===================================================================================================
 Package                                 Arch                           Version                                    Repository                           Size
===================================================================================================
Installing:
 kernel                                  x86_64                         2.6.32-279.5.1.el6                         sl-security                          25 M
Updating for dependencies:
 kernel-firmware                         noarch                         2.6.32-279.5.1.el6                         sl-security                         8.7 M

Transaction Summary
===================================================================================================
Install       1 Package(s)
Upgrade       1 Package(s)

Total download size: 34 M
Is this ok [y/N]:

NOTE: If you are on a mission critical host, you may wish to check if the new kernel will break any other functionality first.

After all, we can see from kernel.org that there's been quite a few changes to the r8169 driver so one of these might be addressing the issue we are facing.  This isn't always the case however, even the updated surrounding code not directly related to r8169 could be the culprit:

 

* Wed Jun 20 2012 Frantisek Hrbata <fhrbata@redhat.com> [2.6.32-279.1.1.el6]
  - [netdrv] r8169: fix typo in firmware filenames (Ivan Vecera) [832359 829211]



* Tue Jan 17 2012 Aristeu Rozanski <arozansk@redhat.com> [2.6.32-227.el6]
  - [netdrv] r8169: fix Config2 MSIEnable bit setting (Ivan Vecera) [772565]
  - [netdrv] r8169: fix Rx index race between FIFO overflow recovery and NAPI handler (Ivan Vecera) [772565]
  - [netdrv] r8169: Rx FIFO overflow fixes (Ivan Vecera) [772565]
  - [netdrv] r8169: corral some wayward N/A fw_version dust bunnies (Ivan Vecera) [772565]
  - [netdrv] r8169: Convert MAC_ADDR_LEN uses to ETH_ALEN (Ivan Vecera) [772565]
  - [netdrv] r8169: sweep the floors and convert some .get_drvinfo routines to strlcpy (Ivan Vecera) [772565]
  - [netdrv] r8169: jumbo fixes (Ivan Vecera) [772565]
  - [netdrv] r8169: expand received packet length indication (Ivan Vecera) [772565]
  - [netdrv] r8169: support new chips of RTL8111F (Ivan Vecera) [772565]
  - [netdrv] r8169: define the early size for 8111evl (Ivan Vecera) [772565]
  - [netdrv] r8169: remove use of ndo_set_multicast_list in drivers (Ivan Vecera) [772565]
  - [netdrv] r8169: MAC address change fix for the 8168e-vl (Ivan Vecera) [772565]
  - [netdrv] r8169: Add support for D-Link 530T rev C1 (Ivan Vecera) [772565]
  - [netdrv] r8169: use pci_dev->subsystem_{vendor|device} (Ivan Vecera) [772565]
  - [netdrv] r8169: fix sticky accepts packet bits in RxConfig (Ivan Vecera) [772565]
  - [netdrv] r8169: adjust the RxConfig settings (Ivan Vecera) [772565]
  - [netdrv] r8169: don't enable rx when shutdown (Ivan Vecera) [772565]
  - [netdrv] r8169: fix wake on lan setting for non-8111E (Ivan Vecera) [772565]
  - [netdrv] r8169: support RTL8111E-VL (Ivan Vecera) [772565]
  - [netdrv] r8169: add ERI functions (Ivan Vecera) [772565]
  - [netdrv] r8169: modify the flow of the hw reset (Ivan Vecera) [772565]
  - [netdrv] r8169: adjust some registers (Ivan Vecera) [772565]
  - [netdrv] r8169: remove unnecessary read of PCI_CAP_ID_EXP (Ivan Vecera) [772565]
  - [netdrv] r8169: fix wrong register use (Ivan Vecera) [772565]
  - [netdrv] r8169: check firmware content sooner (Ivan Vecera) [772565]
  - [netdrv] r8169: support new firmware format (Ivan Vecera) [772565]
  - [netdrv] r8169: explicit firmware format check (Ivan Vecera) [772565]
  - [netdrv] r8169: move the firmware down into the device private data (Ivan Vecera) [772565]
  - [netdrv] r8169: fix static initializers (Ivan Vecera) [772565]
  - [netdrv] r8169: avoid late chip identifier initialisation (Ivan Vecera) [772565]
  - [netdrv] r8169: merge firmware information into the chipset description data (Ivan Vecera) [772565]
  - [netdrv] r8169: provide some firmware information via ethtool (Ivan Vecera) [772565]
  - [netdrv] r8169: remove non-NAPI context invocation of rtl8169_rx_interrupt (Ivan Vecera) [772565]
  - [netdrv] r8169: link speed selection timer rework (Ivan Vecera) [772565]
  - [netdrv] r8169: rtl8169_set_speed_xmii cleanup (Ivan Vecera) [772565]
  - [netdrv] r8169: remove some code duplication (Ivan Vecera) [772565]
  - [netdrv] r8169: style cleanups (Ivan Vecera) [772565]
  - [netdrv] r8169: Use ethtool ethtool_cmd_speed API (Ivan Vecera) [772565]
  - [netdrv] r8169: Use full 32 bit speed range in ethtool's set_settings (Ivan Vecera) [772565]
  - [netdrv] r8169: fix merge conflict fix (Ivan Vecera) [772565]
  - [netdrv] r8169: don't request firmware when there's no userspace (Ivan Vecera) [772565]
  - [netdrv] r8169: TSO fixes (Ivan Vecera) [772565]
  - [netdrv] r8169: convert to hw_features (Ivan Vecera) [772565]
  - [netdrv] r8169: support RTL8168E (Ivan Vecera) [772565]
  - [netdrv] r8169: add a new chip for RTL8168DP (Ivan Vecera) [772565]
  - [netdrv] r8169: add a new chip for RTL8105 (Ivan Vecera) [772565]
  - [netdrv] r8169: fix a bug in rtl8169_init_phy() (Ivan Vecera) [772565]
  - [netdrv] r8169: support control of advertising (Ivan Vecera) [772565]
  - [netdrv] r8169: support the new chips for RTL8105E (Ivan Vecera) [772565]
  - [netdrv] r8169: disable ASPM (Ivan Vecera) [772565]
  - [netdrv] r8169: adjust rtl8169_set_speed_xmii function (Ivan Vecera) [772565]
  - [netdrv] r8169: fix RTL8168DP power off issue (Ivan Vecera) [772565]
  - [netdrv] r8169: correct settings of rtl8102e (Ivan Vecera) [772565]
  - [netdrv] r8169: fix incorrect args to oob notify (Ivan Vecera) [772565]
  - [netdrv] r8169: Call netif_carrier_off at the end of the probe (Ivan Vecera) [772565]
  - [netdrv] r8169: prevent RxFIFO induced loops in the irq handler (Ivan Vecera) [772565]
  - [netdrv] r8169: keep firmware in memory (Ivan Vecera) [772565]
  - [netdrv] r8169: Update the function of parsing firmware (Ivan Vecera) [772565]
  - [netdrv] r8169: delay phy init until device opens (Ivan Vecera) [772565]
  - [netdrv] r8169: more 8168dp support (Ivan Vecera) [772565]
  - [netdrv] r8169: rtl_csi_access_enable rename (Ivan Vecera) [772565]
  - [netdrv] r8169: magic (Ivan Vecera) [772565]
  - [netdrv] r8169: phy power ops (Ivan Vecera) [772565]
  - [netdrv] r8169: 8168DP specific MII registers access methods (Ivan Vecera) [772565]
  - [netdrv] r8169: use device dependent methods to access the MII registers (Ivan Vecera) [772565]
  - [netdrv] r8169: identify different registers (Ivan Vecera) [772565]
  - [netdrv] r8169: remove the firmware of RTL8111D (Ivan Vecera) [772565]
  - [netdrv] r8169: Use static const (Ivan Vecera) [772565]
  - [netdrv] r8169: move PHY regs tables to .rodata (Ivan Vecera) [772565]
  - [netdrv] r8169: don't use flush_scheduled_work() (Ivan Vecera) [772565]
  - [netdrv] r8169: fix sleeping while holding spinlock (Ivan Vecera) [772565]
  - [netdrv] r8169: print errors when dma mapping fail (Ivan Vecera) [772565]
  - [netdrv] r8169: (re)init phy on resume (Ivan Vecera) [772565]
  - [netdrv] r8169: changing mtu clean up (Ivan Vecera) [772565]
  - [netdrv] r8169: do not account fragments as packets (Ivan Vecera) [772565]
  - [netdrv] r8169: use pointer to struct device as local variable (Ivan Vecera) [772565]
  - [netdrv] r8169: replace PCI_DMA_{TO, FROM}DEVICE to DMA_{TO, FROM}_DEVICE (Ivan Vecera) [772565]
  - [netdrv] r8169: init rx ring cleanup (Ivan Vecera) [772565]
  - [netdrv] r8169: check dma mapping failures (Ivan Vecera) [772565]
  - [netdrv] r8169: Don't check for vlan group before vlan_tx_tag_present (Ivan Vecera) [772565]
  - [netdrv] r8169: use 50 less ram for RX ring (Ivan Vecera) [772565]
  - [netdrv] r8169: use device model DMA API (Ivan Vecera) [772565]
  - [netdrv] r8169: allocate with GFP_KERNEL flag when able to sleep (Ivan Vecera) [772565]
  - [netdrv] r8169: add gro support (Ivan Vecera) [772565]
  - [netdrv] r8169: avoid some skb->ip_summed initializations (Ivan Vecera) [772565]
  - [netdrv] r8169: incorrect identifier for a 8168dp (Ivan Vecera) [772565]
  - [netdrv] r8169: fix mdio_read and update mdio_write according to hw specs (Ivan Vecera) [772565]
  - [netdrv] r8169: fix random mdio_write failures (Ivan Vecera) [772565]
  - [netdrv] r8169: remove unnecessary cast of readl()'s return value (Ivan Vecera) [772565]
  - [netdrv] r8169: failure to enable mwi should not be fatal (Ivan Vecera) [772565]
  - [netdrv] r8169: Fix rtl8169_rx_interrupt() (Ivan Vecera) [772565]
  - [netdrv] r8169: convert multiple drivers to use netdev_for_each_mc_addr (Ivan Vecera) [772565]
  - [netdrv] r8169: Use netif_printk macros (Ivan Vecera) [772565]
  - [netdrv] r8169: use netdev_mc_count and netdev_mc_empty when appropriate (Ivan Vecera) [772565]
  - [netdrv] r8169: use DEFINE_PCI_DEVICE_TABLE() (Ivan Vecera) [772565]
  - [netdrv] r8169: use pM to shown MAC address (Ivan Vecera) [772565]
  - [netdrv] r8169: Move && and || to end of previous line (Ivan Vecera) [772565]

* Thu Jul 21 2011 Kyle McMartin <kmcmarti@redhat.com> [2.6.32-171.el6]
  - [netdrv] r8169: fix Rx checksum offloading bugs (Ivan Vecera) [635596]

Unfortunately the latest kernel didn't fix this issue and the system was still at 10MBits/s with virtually none of the changed flags having any effect:

# lspci -s 03:00.0 -vvvvvvvvvvvvvvvvvvvv
03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 03)
        Subsystem: Giga-byte Technology GA-EP45-DS5 Motherboard
        Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Interrupt: pin A routed to IRQ 18
        Region 0: I/O ports at ee00 [size=256]
        Region 2: Memory at fdfff000 (64-bit, prefetchable) [size=4K]
        Region 4: Memory at fdff8000 (64-bit, prefetchable) [size=16K]
        [virtual] Expansion ROM at fdf00000 [disabled] [size=128K]
        Capabilities: [40] Power Management version 3
                Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=375mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
                Address: 00000000fee0100c  Data: 4199
        Capabilities: [70] Express (v2) Endpoint, MSI 01
                DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop-
                        MaxPayload 128 bytes, MaxReadReq 4096 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
                LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0 <512ns, L1 <64us
                        ClockPM+ Surprise- LLActRep- BwNot-
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Not Supported, TimeoutDis+
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
                LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -6dB
        Capabilities: [ac] MSI-X: Enable- Count=4 Masked-
                Vector table: BAR=4 offset=00000000
                PBA: BAR=4 offset=00000800
        Capabilities: [cc] Vital Product Data
                Unknown small resource type 00, will not decode more.
        Capabilities: [100] Advanced Error Reporting
                UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
                CESta:  RxErr+ BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
                CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
        Capabilities: [140] Virtual Channel <?>
        Capabilities: [160] Device Serial Number 03-00-00-00-68-4c-e0-00
        Kernel modules: r8169

#

 

# uname -r
2.6.32-279.5.1.el6.x86_64
#

# ethtool eth0
Settings for eth0:
        Supported ports: [ TP MII ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Half 1000baseT/Full
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Half 1000baseT/Full
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: Yes
        Link partner advertised link modes:  10baseT/Half 10baseT/Full
        Link partner advertised pause frame use: Symmetric Receive-only
        Link partner advertised auto-negotiation: Yes
        Speed: 10Mb/s
        Duplex: Full
        Port: MII
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: on
        Supports Wake-on: pumbg
        Wake-on: g
        Current message level: 0x00000033 (51)
        Link detected: no
#

Trying to set speed to 1000 only results in 10Mb/s anyway:

# ethtool -s eth0 speed 1000 duplex full autoneg on
# ethtool eth0
Settings for eth0:
        Supported ports: [ TP MII ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Half 1000baseT/Full
        Supports auto-negotiation: Yes
        Advertised link modes:  1000baseT/Full
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: Yes
        Speed: 10Mb/s
        Duplex: Half
        Port: MII
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: on
        Supports Wake-on: pumbg
        Wake-on: g
        Current message level: 0x00000033 (51)
        Link detected: no
#

 

However, trying to set to 100Mb/s results in 100Mb/s, which is now better.   If it's able to set to 100Mb/s, this suggests the cable again because at least the NIC is trying.  Well, at least some of the time we can set it to 100Mb/s:

# ethtool -s eth0 speed 1000 duplex full autoneg on
# ethtool eth0
Settings for eth0:
        Supported ports: [ TP MII ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Half 1000baseT/Full
        Supports auto-negotiation: Yes
        Advertised link modes:  1000baseT/Full
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: Yes
        Speed: 10Mb/s
        Duplex: Half
        Port: MII
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: on
        Supports Wake-on: pumbg
        Wake-on: g
        Current message level: 0x00000033 (51)
        Link detected: no
#

 

And when the speed is set to 100Mb/s or higher, mii-tool shows no linkup:

# mii-tool eth0
eth0: no link
#

Time to try the drivers off the realtek site.  Another way to get it off the site directly is to issue the below command from a Linux command line like this:

 wget ftp://WebUser:AxPL9s3k@209.222.7.36/cn/nic/r8168-8.032.00.tar.bz2

Extract the files then go into the folders so we can compile it according to the README file:

[root@mbpc r8168-8.032.00]# ls
autorun.sh  Makefile  README  src
[root@mbpc r8168-8.032.00]# cd src
[root@mbpc src]# ls -al
total 612
drwx——. 2 root root   4096 Jul 19 04:31 .
drwx——. 3 root root   4096 Sep  4 21:04 ..
-rw-r–r–. 1 root root   2436 Jul 19 04:24 Makefile
-rw-r–r–. 1 root root   2517 Jul 19 04:24 Makefile_linux24x
-rw-r–r–. 1 root root  13670 Jul 19 04:24 r8168_asf.c
-rw-r–r–. 1 root root   7795 Jul 19 04:24 r8168_asf.h
-rw-r–r–. 1 root root  37622 Jul 19 04:24 r8168.h
-rw-r–r–. 1 root root 513138 Jul 19 04:24 r8168_n.c
-rw-r–r–. 1 root root   6260 Jul 19 04:24 rtl_eeprom.c
-rw-r–r–. 1 root root   2036 Jul 19 04:24 rtl_eeprom.h
-rw-r–r–. 1 root root   9190 Jul 19 04:24 rtltool.c
-rw-r–r–. 1 root root    801 Jul 19 04:24 rtltool.h
[root@mbpc src]#

So we try the recommended compile / install script:

[root@mbpc r8168-8.032.00]# ./autorun.sh

Check old driver and unload it.
Build the module and install
make: *** /lib/modules/2.6.32-279.5.1.el6.x86_64/build: No such file or directory.  Stop.
make[1]: *** [clean] Error 2
make: *** [clean] Error 2
[root@mbpc r8168-8.032.00]#

But we need the kernel sources so we run: yum install kernel-devel.x86_64

After that, we rerun autorun.sh again:

[root@mbpc r8168-8.032.00]# ./autorun.sh

Check old driver and unload it.
Build the module and install
Backup r8169.ko
rename r8169.ko to r8169.bak
DEPMOD 2.6.32-279.5.1.el6.x86_64
load module r8168
Completed.
[root@mbpc r8168-8.032.00]# pwd
/root/r8168/r8168-8.032.00
[root@mbpc r8168-8.032.00]#

/var/log/messages
Sep  5 01:38:22 mbpc kernel: r8168 Gigabit Ethernet driver 8.032.00-NAPI loaded
Sep  5 01:38:22 mbpc kernel: r8168 0000:03:00.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
Sep  5 01:38:22 mbpc kernel: eth%d: RTL8168D/8111D at 0xffffc90000666000, 1c:6f:65:3f:fc:14, IRQ 29
Sep  5 01:38:22 mbpc kernel: r8168: This product is covered by one or more of the following patents: US5,307,459, US5,434,872, US5,732,094, US6,570,884, US6,115,776, and US6,327,625.
Sep  5 01:38:22 mbpc kernel: r8168  Copyright (C) 2012  Realtek NIC software team <nicfae@realtek.com>
Sep  5 01:38:22 mbpc kernel: This program comes with ABSOLUTELY NO WARRANTY; for details, please see <http://www.gnu.org/licenses/>.
Sep  5 01:38:22 mbpc kernel: This is free software, and you are welcome to redistribute it under certain conditions; see <http://www.gnu.org/licenses/>.

And check the speed:

[root@mbpc r8168-8.032.00]# ethtool eth0
Settings for eth0:
        Supported ports: [ TP ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: Yes
        Speed: 10Mb/s
        Duplex: Full
        Port: Twisted Pair
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: on
        MDI-X: Unknown
        Supports Wake-on: pumbg
        Wake-on: g
        Current message level: 0x00000033 (51)
        Link detected: yes
[root@mbpc r8168-8.032.00]#

Damn.  Still connecting at 10Mb/s.  Time to recheck the cables and plug into the router directly with the shortest CAT6e cable I have this time:

[root@mbpc r8168-8.032.00]# ethtool -s eth0 speed 1000 duplex full autoneg off
[root@mbpc r8168-8.032.00]#
[root@mbpc r8168-8.032.00]# mii-tool eth0
eth0: no link
[root@mbpc r8168-8.032.00]#

Notice, this time no error about setting speed to 1000 once the RealTek driver is installed and after a while:

[root@mbpc r8168-8.032.00]# mii-tool eth0
eth0: negotiated 10baseT-FD flow-control, link ok
[root@mbpc r8168-8.032.00]#

But no luck.  Trying to disable the TCP checksum had no effect either:

[root@mbpc r8168]# ethtool -K eth0 rx off tx off
[root@mbpc r8168]#

[root@mbpc r8168]# mii-tool eth0
eth0: negotiated 10baseT-FD flow-control, link ok
[root@mbpc r8168]#

Next we'll try to unplug the device from the system using the command line like this:

# echo -n 0000:03:00.0 > /sys/bus/pci/drivers/r8168/unbind

/var/log/messages
Sep  6 00:38:44 mbpc kernel: bonding: bond0: releasing active interface eth0
Sep  6 00:38:44 mbpc kernel: r8168 0000:03:00.0: PCI INT A disabled

# echo 1 > /sys/bus/pci/devices/0000\:03\:00.0/remove
# lspci -s 03:00.0
#

At this point all the lights went out on the NIC card so it looks like it's off.  Next we rescan the BUS to readd it back in:

# lspci|grep -i ether
# echo 1 > /sys/bus/pci/rescan
# lspci|grep -i ether
03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 03)
#
 

Lights came on now.  But ethtool still showed 10MBits/s.  Lastly, we try a simple cold reboot (Allow voltage to drain by unplugging the power cord after system is shut down or press the on/off switch at the back of the power supply) When the system came back, we get 100Mbits/s:
 

# ethtool eth0
Settings for eth0:
    Supported ports: [ TP ]
    Supported link modes:   10baseT/Half 10baseT/Full
                            100baseT/Half 100baseT/Full
                            1000baseT/Full
    Supports auto-negotiation: Yes
    Advertised link modes:  10baseT/Half 10baseT/Full
                            100baseT/Half 100baseT/Full
                            1000baseT/Full
    Advertised pause frame use: Symmetric Receive-only
    Advertised auto-negotiation: Yes
    Speed: 100Mb/s
    Duplex: Full
    Port: Twisted Pair
    PHYAD: 0
    Transceiver: internal
    Auto-negotiation: on
    MDI-X: Unknown
    Supports Wake-on: pumbg
    Wake-on: g
    Current message level: 0x00000033 (51)
    Link detected: yes
#

And after plugging in another cable, this time we get 1000MBits/s and our transfers from the Windows computers above are in the 60MB/s range (each).

Now the question is, why did the NIC keep the settings on the card so long, even after reboots, it affected both the r8169 and r8168 driver.  One possibility, as hinted here, might be the retention of gain settings.  Further reading is available about such settings being retained.

Sep 12 2012
So we now set the Jumbo Frames to 8K to improve speed performance and turn all Checksum flags (Both on client and server) back to on though this part, leaving Checksum off should be fine as well:

# ifconfig eth0 mtu 8192
# ifconfig bond0 mtu 8192

And the network speed is affected 3X:

# ifconfig
bond0     Link encap:Ethernet  HWaddr 1C:6F:65:3F:FC:14
          inet addr:192.168.0.14  Bcast:192.168.1.255  Mask:255.255.254.0
          inet6 addr: fe80::1e6f:65ff:fe3f:fc14/64 Scope:Link
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:8192  Metric:1
          RX packets:16666288 errors:0 dropped:0 overruns:0 frame:0
          TX packets:9083665 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:24601265292 (22.9 GiB)  TX bytes:645010340 (615.1 MiB)

eth0      Link encap:Ethernet  HWaddr 1C:6F:65:3F:FC:14
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:8192  Metric:1
          RX packets:16568050 errors:0 dropped:0 overruns:0 frame:0
          TX packets:9070634 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:24459219089 (22.7 GiB)  TX bytes:644077764 (614.2 MiB)
          Interrupt:29 Base address:0x2000

 

Cheers,
TK