Device eth0 has different MAC address than expected, ignoring.
So once our upgrade to FC19 was done, we notice we haven't got a network connection anymore. This is because we try to use yum and get a bunch of error messages. Here's how to solve it on the new system through a debugging session into the issue.
QUICK SOLUTION
Turned out that not much has changed in the NetworkManager software from the early days of Fedora
# service NetworkManager stop
Followed by
# service network restart
did the trick.
LONG INVESTIGATION
[root@localhost F13toF18]# yum update all
Error: Cannot retrieve metalink for repository: fedora/19/i386. Please verify its path and try again
[root@localhost F13toF18]# yum update
Error: Cannot retrieve metalink for repository: fedora/19/i386. Please verify its path and try again
[root@localhost F13toF18]# ps -ef|grep -i sshd
root 1183 1 0 16:40 ? 00:00:00 /usr/sbin/sshd -D
root 2600 2091 0 16:54 pts/0 00:00:00 grep –color=auto -i sshd
[root@localhost F13toF18]# service sshd restart
Redirecting to /bin/systemctl restart sshd.service
[root@localhost F13toF18]#
[root@localhost F13toF18]# ifconfig
eth0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether 00:0d:88:38:0a:34 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 0 (Local Loopback)
RX packets 81 bytes 11963 (11.6 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 81 bytes 11963 (11.6 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@localhost F13toF18]# service network restart
Restarting network (via systemctl): Job for network.service failed. See 'systemctl status network.service' and 'journalctl -xn' for details.
[FAILED]
and we try again using systemctl:
[root@localhost F13toF18]# systemctl status network.service
network.service – LSB: Bring up/down networking
Loaded: loaded (/etc/rc.d/init.d/network)
Active: failed (Result: exit-code) since Sat 2013-08-10 16:54:19 EDT; 22s ago
Process: 2636 ExecStart=/etc/rc.d/init.d/network start (code=exited, status=1/FAILURE)
Aug 10 16:54:19 localhost network[2636]: RTNETLINK answers: File exists
Aug 10 16:54:19 localhost network[2636]: RTNETLINK answers: File exists
Aug 10 16:54:19 localhost network[2636]: RTNETLINK answers: File exists
Aug 10 16:54:19 localhost network[2636]: RTNETLINK answers: File exists
Aug 10 16:54:19 localhost network[2636]: RTNETLINK answers: File exists
Aug 10 16:54:19 localhost network[2636]: RTNETLINK answers: File exists
Aug 10 16:54:19 localhost systemd-sysctl[2870]: Duplicate assignment of kernel/sysrq in file '/usr/lib/sysctl.d/50-default.conf', ignoring.
Aug 10 16:54:19 localhost systemd[1]: network.service: control process exited, code=exited status=1
Aug 10 16:54:19 localhost systemd[1]: Failed to start LSB: Bring up/down networking.
Aug 10 16:54:19 localhost systemd[1]: Unit network.service entered failed state.
[root@localhost F13toF18]#
[root@localhost F13toF18]# service network restart
Restarting network (via systemctl): Job for network.service failed. See 'systemctl status network.service' and 'journalctl -xn' for details.
[FAILED]
[root@localhost F13toF18]#
[root@localhost F13toF18]#
[root@localhost F13toF18]#
[root@localhost F13toF18]# journalctl -xn
— Logs begin at Sat 2013-08-10 11:08:22 EDT, end at Sat 2013-08-10 16:55:12 EDT. —
Aug 10 16:55:12 localhost network[2893]: RTNETLINK answers: File exists
Aug 10 16:55:12 localhost network[2893]: RTNETLINK answers: File exists
Aug 10 16:55:12 localhost network[2893]: RTNETLINK answers: File exists
Aug 10 16:55:12 localhost network[2893]: RTNETLINK answers: File exists
Aug 10 16:55:12 localhost network[2893]: RTNETLINK answers: File exists
Aug 10 16:55:12 localhost network[2893]: RTNETLINK answers: File exists
Aug 10 16:55:12 localhost systemd-sysctl[3127]: Duplicate assignment of kernel/sysrq in file '/usr/lib/sysctl.d/50-default.conf', ignoring.
Aug 10 16:55:12 localhost systemd[1]: network.service: control process exited, code=exited status=1
Aug 10 16:55:12 localhost systemd[1]: Failed to start LSB: Bring up/down networking.
— Subject: Unit network.service has failed
— Defined-By: systemd
— Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
— Documentation: http://www.freedesktop.org/wiki/Software/systemd/catalog/be02cf6855d2428ba40df7e9d022f03d
—
— Unit network.service has failed.
—
— The result is failed.
Aug 10 16:55:12 localhost systemd[1]: Unit network.service entered failed state.
[root@localhost F13toF18]# service start lsb
Redirecting to /bin/systemctl lsb start.service
Unknown operation 'lsb'.
[root@localhost F13toF18]#
[root@localhost network-scripts]# ifup eth0
ERROR : [/etc/sysconfig/network-scripts/ifup-eth] Device eth0 has different MAC address than expected, ignoring.
[root@localhost network-scripts]#
so our friend ifup still does check the network scripts folder, however new this FC19 distribution is:
[root@localhost ~]# strace ifup eth0 2>&1|grep network
stat64("/etc/sysconfig/network-scripts", {st_mode=S_IFDIR|0755, st_size=4096, …}) = 0
chdir("/etc/sysconfig/network-scripts") = 0
open("./network-functions", O_RDONLY|O_LARGEFILE) = 3
stat64("../network", {st_mode=S_IFREG|0644, st_size=15, …}) = 0
open("../network", O_RDONLY|O_LARGEFILE) = 3
open("/etc/sysconfig/network-scripts/ifcfg-eth0", O_RDONLY|O_LARGEFILE) = 3
faccessat(AT_FDCWD, "/etc/sysconfig/network-scripts/ifup-eth", X_OK) = 0
faccessat(AT_FDCWD, "/etc/sysconfig/network-scripts/ifup-eth", X_OK) = 0
execve("/etc/sysconfig/network-scripts/ifup-eth", [“/etc/sysconfig/network-scripts/i”…, “ifcfg-eth0”], [/* 71 vars */]) = 0
stat64("/etc/sysconfig/network-scripts", {st_mode=S_IFDIR|0755, st_size=4096, …}) = 0
open("/etc/sysconfig/network-scripts/ifup-eth", O_RDONLY|O_LARGEFILE) = 3
stat64("/etc/sysconfig/network-scripts", {st_mode=S_IFDIR|0755, st_size=4096, …}) = 0
chdir("/etc/sysconfig/network-scripts") = 0
open("./network-functions", O_RDONLY|O_LARGEFILE) = 3
stat64("../network", {st_mode=S_IFREG|0644, st_size=15, …}) = 0
open("../network", O_RDONLY|O_LARGEFILE) = 3
open("/etc/sysconfig/network-scripts/ifcfg-eth0", O_RDONLY|O_LARGEFILE) = 3
write(1, "ERROR : [/etc/sysconfig/netwo”…, 116ERROR : [/etc/sysconfig/network-scripts/ifup-eth] Device eth0 has different MAC address than expected, ignoring.
[root@localhost ~]# ifup eth0
ERROR : [/etc/sysconfig/network-scripts/ifup-eth] Device eth0 has different MAC address than expected, ignoring.
[root@localhost ~]#
so let's check the udev rules. Maybe things got scrambled up in a manner of polite speaking:
[root@localhost ~]# ifup eth0
ERROR : [/etc/sysconfig/network-scripts/ifup-eth] Device eth0 has different MAC address than expected, ignoring.
[root@localhost ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
ONBOOT=yes
# BOOTPROTO=dhcp
BOOTPROTO=none
# IPADDR=192.168.1.2
IPADDR=192.168.0.13
NETMASK=255.255.255.0
GATEWAY=192.168.0.1
# GATEWAY=192.168.1.1
HWADDR=94:de:80:73:b1:78
USERCTL=no
IPV6INIT=yes
DNS1=192.168.0.1
# DNS1=192.168.1.1
NM_CONTROLLED=no
TYPE=Ethernet
[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]# cat /etc/udev/rules.d/70-persistent-net.rules
# This file was automatically generated by the /lib/udev/write_net_rules
# program run by the persistent-net-generator.rules rules file.
#
# You can modify it, as long as you keep each rule on a single line.
# PCI device 0x1106:0x3065 (via-rhine)
# TOAST DUE TO THUNDERSTORM: SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:13:d4:3d:3b:6a", ATTR{type}=="1", NAME="eth0"
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="94:de:80:73:b1:78", ATTR{type}=="1", NAME="eth0"
# PCI device 0x100b:0x0020 (natsemi)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:0d:88:38:0a:34", ATTR{type}=="1", NAME="eth1"
# USB device 0x:0x (rt73usb)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:23:cd:bc:6e:22", ATTR{type}=="1", KERNEL=="wlan*", NAME="wlan0"
# PCI device 0x1106:0x3065 (via-rhine)
# SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:13:d4:3d:3b:6a", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth2"
# PCI device 0x10ec:0x8168 (r8169)
# SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="94:de:80:73:b1:78", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth3"
[root@localhost ~]#
they match so let's check what's on the disk in /sys and in /proc and with ifconfig -a:
[root@localhost net]# ifconfig -a
eth0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether 00:0d:88:38:0a:34 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth1: flags=4098<BROADCAST,MULTICAST> mtu 1500
ether 94:de:80:73:b1:78 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 0 (Local Loopback)
RX packets 81 bytes 11963 (11.6 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 81 bytes 11963 (11.6 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@localhost net]#
And we quickly see that our cards got reversed on reboot. So we need to update our persistent net rules:
[root@localhost net]# ls -altri
total 0
10 drwxr-xr-x 46 root root 0 Aug 10 16:39 ..
17222 lrwxrwxrwx 1 root root 0 Aug 10 16:40 eth0 -> ../../devices/pci0000:00/0000:00:14.4/0000:05:06.0/net/eth0
3226 drwxr-xr-x 2 root root 0 Aug 10 16:40 .
17425 lrwxrwxrwx 1 root root 0 Aug 10 16:40 eth1 -> ../../devices/pci0000:00/0000:00:06.0/0000:03:00.0/net/eth1
3231 lrwxrwxrwx 1 root root 0 Aug 10 16:40 lo -> ../../devices/virtual/net/lo
[root@localhost net]# cd eth0
[root@localhost eth0]# pwd
/sys/class/net/eth0
[root@localhost eth0]# ls
addr_assign_type addr_len carrier dev_id duplex ifalias iflink mtu operstate queues statistics tx_queue_len uevent
address broadcast device dormant flags ifindex link_mode netdev_group power speed subsystem type
[root@localhost eth0]# cat address
00:0d:88:38:0a:34
[root@localhost eth0]# pwd
/sys/class/net/eth0
[root@localhost eth0]# cd ../eth1
[root@localhost eth1]# cat address
94:de:80:73:b1:78
[root@localhost eth1]#
But on close inspection, our udev rules are fine and should be working. This means our udev system isn't working then. So we work on that instead:
[root@localhost eth1]# udevadm trigger
[root@localhost eth1]#
And when running
service network restart
we get a bunch of new errors:
/var/log/messages
Aug 11 00:14:58 localhost systemd[1]: Cannot add dependency job for unit sys-kernel-security.mount, ignoring: Unit sys-kernel-security.mount failed to load: Invalid argument. See system logs and 'systemctl status sys-kernel-security.mount' for details.
Aug 11 00:14:58 localhost systemd[1]: Starting LSB: Bring up/down networking…
Aug 11 00:14:58 localhost systemd-sysctl[4548]: Duplicate assignment of kernel/sysrq in file '/usr/lib/sysctl.d/50-default.conf', ignoring.
Aug 11 00:14:58 localhost network[4522]: Bringing up loopback interface: [ OK ]
Aug 11 00:14:58 localhost network[4522]: Bringing up interface Auto_BELL537: [ OK ]
Aug 11 00:14:59 localhost network[4522]: Bringing up interface Auto_BELLPerna1: [ OK ]
Aug 11 00:14:59 localhost network[4522]: Bringing up interface Auto_Boragina: [ OK ]
Aug 11 00:14:59 localhost NetworkManager[710]: <warn> connection /org/freedesktop/NetworkManager/Settings/2 failed to activate: (2) Device not managed by NetworkManager or unavailable
Aug 11 00:14:59 localhost network[4522]: Bringing up interface Auto_Ethernet: Error: Connection activation failed: Device not managed by NetworkManager or unavailable
Aug 11 00:14:59 localhost network[4522]: [FAILED]
Aug 11 00:14:59 localhost network[4522]: Bringing up interface Auto_VWgolf: [ OK ]
Aug 11 00:14:59 localhost network[4522]: Bringing up interface Auto_covello: [ OK ]
Aug 11 00:14:59 localhost network[4522]: Bringing up interface ChiefSheriff: [ OK ]
Aug 11 00:14:59 localhost network[4522]: Bringing up interface eth0: ERROR : [/etc/sysconfig/network-scripts/ifup-eth] Device eth0 has different MAC address than expected, ignoring.
Aug 11 00:14:59 localhost /etc/sysconfig/network-scripts/ifup-eth: Device eth0 has different MAC address than expected, ignoring.
Aug 11 00:14:59 localhost network[4522]: [FAILED]
Aug 11 00:14:59 localhost network[4522]: Bringing up interface eth1: ERROR : [/etc/sysconfig/network-scripts/ifup-eth] Device eth1 has different MAC address than expected, ignoring.
Aug 11 00:14:59 localhost /etc/sysconfig/network-scripts/ifup-eth: Device eth1 has different MAC address than expected, ignoring.
Aug 11 00:14:59 localhost network[4522]: [FAILED]
Aug 11 00:14:59 localhost network[4522]: RTNETLINK answers: File exists
Aug 11 00:14:59 localhost network[4522]: RTNETLINK answers: File exists
Aug 11 00:14:59 localhost network[4522]: RTNETLINK answers: File exists
Aug 11 00:14:59 localhost network[4522]: RTNETLINK answers: File exists
Aug 11 00:14:59 localhost network[4522]: RTNETLINK answers: File exists
Aug 11 00:14:59 localhost network[4522]: RTNETLINK answers: File exists
Aug 11 00:14:59 localhost network[4522]: RTNETLINK answers: File exists
Aug 11 00:14:59 localhost network[4522]: RTNETLINK answers: File exists
Aug 11 00:14:59 localhost network[4522]: RTNETLINK answers: File exists
Aug 11 00:14:59 localhost systemd-sysctl[4756]: Duplicate assignment of kernel/sysrq in file '/usr/lib/sysctl.d/50-default.conf', ignoring.
Aug 11 00:14:59 localhost systemd[1]: network.service: control process exited, code=exited status=1
Aug 11 00:14:59 localhost systemd[1]: Failed to start LSB: Bring up/down networking.
Aug 11 00:14:59 localhost systemd[1]: Unit network.service entered failed state.
We try to see if we are up to date with udev but get this instead:
[root@localhost eth1]# yum search udev
Error: Cannot retrieve metalink for repository: fedora/19/i386. Please verify its path and try again
[root@localhost eth1]#
and we quickly notice when running the above yum search udev command through strace that we have some old folders in the yum cache directory from our upgrade:
stat64("/var/cache/yum/updates/packages/filesystem/etc/opt", {st_mode=S_IFDIR|0755, st_size=4096, …}) = 0
openat(AT_FDCWD, "/var/cache/yum/updates/packages/filesystem/etc/opt", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY|O_CLOEXEC) = 5
getdents64(5, /* 2 entries */, 32768) = 48
getdents64(5, /* 0 entries */, 32768) = 0
close(5) = 0
stat64("/var/cache/yum/updates/packages/filesystem/run", {st_mode=S_IFDIR|0755, st_size=4096, …}) = 0
openat(AT_FDCWD, "/var/cache/yum/updates/packages/filesystem/run", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY|O_CLOEXEC) = 5
getdents64(5, /* 2 entries */, 32768) = 48
getdents64(5, /* 0 entries */, 32768) = 0
close(5) = 0
stat64("/var/cache/yum/updates/packages/filesystem/srv", {st_mode=S_IFDIR|0755, st_size=4096, …}) = 0
openat(AT_FDCWD, "/var/cache/yum/updates/packages/filesystem/srv", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY|O_CLOEXEC) = 5
getdents64(5, /* 2 entries */, 32768) = 48
getdents64(5, /* 0 entries */, 32768) = 0
close(5) = 0
stat64("/var/cache/yum/updates/packages/filesystem/opt", {st_mode=S_IFDIR|0755, st_size=4096, …}) = 0
openat(AT_FDCWD, "/var/cache/yum/updates/packages/filesystem/opt", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY|O_CLOEXEC) = 5
getdents64(5, /* 2 entries */, 32768) = 48
getdents64(5, /* 0 entries */, 32768) = 0
close(5) = 0
gettimeofday({1376194705, 953066}, NULL) = 0
write(2, "Error: Cannot retrieve metalink "…, 101Error: Cannot retrieve metalink for repository: fedora/19/i386. Please verify its path and try again
) = 101
unlink("/var/run/yum.pid") = 0
close(4) = 0
munmap(0xb704d000, 4096) = 0
close(3) = 0
rt_sigaction(SIGINT, {SIG_DFL, [], 0}, {0xb765e890, [], 0}, 8) = 0
rt_sigaction(SIGQUIT, {SIG_DFL, [], 0}, {0xb765e890, [], 0}, 8) = 0
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
exit_group(1) = ?
+++ exited with 1 +++
[root@localhost eth1]#
[root@localhost eth1]#
[root@localhost eth1]# ls -ald var/cache/yum/updates/packages/filesystem/
ls: cannot access var/cache/yum/updates/packages/filesystem/: No such file or directory
[root@localhost eth1]# ls -ald /var/cache/yum/updates/packages/filesystem/
drwxr-xr-x 17 root root 4096 Aug 10 16:43 /var/cache/yum/updates/packages/filesystem/
[root@localhost eth1]# ls -al var/cache/yum/updates/packages/filesystem/
ls: cannot access var/cache/yum/updates/packages/filesystem/: No such file or directory
[root@localhost eth1]# ls -al /var/cache/yum/updates/packages/filesystem/
total 264
drwxr-xr-x 17 root root 4096 Aug 10 16:43 .
drwxr-xr-x 3 root root 200704 Aug 10 16:43 ..
lrwxrwxrwx 1 root root 7 Aug 10 10:21 bin -> usr/bin
dr-xr-xr-x 2 root root 4096 Jul 8 04:56 boot
drwxr-xr-x 2 root root 4096 Jul 8 04:56 dev
drwxr-xr-x 11 root root 4096 Aug 10 10:21 etc
drwxr-xr-x 2 root root 4096 Jul 8 04:56 home
lrwxrwxrwx 1 root root 7 Aug 10 10:21 lib -> usr/lib
drwxr-xr-x 2 root root 4096 Jul 8 04:56 media
drwxr-xr-x 2 root root 4096 Jul 8 04:56 mnt
drwxr-xr-x 2 root root 4096 Jul 8 04:56 opt
dr-xr-xr-x 2 root root 4096 Jul 8 04:56 proc
dr-xr-x— 2 root root 4096 Jul 8 04:56 root
drwxr-xr-x 2 root root 4096 Jul 8 04:56 run
lrwxrwxrwx 1 root root 8 Aug 10 10:21 sbin -> usr/sbin
drwxr-xr-x 2 root root 4096 Jul 8 04:56 srv
drwxr-xr-x 2 root root 4096 Jul 8 04:56 sys
drwxrwxrwt 2 root root 4096 Jul 8 04:56 tmp
drwxr-xr-x 12 root root 4096 Aug 10 10:21 usr
drwxr-xr-x 17 root root 4096 Aug 10 10:21 var
[root@localhost eth1]#
And we see more details on why yum is complaining:
[root@localhost packages]# strace yum search udev 2>&1|grep -i error
read(6, "\0\0\0t\3\0\0\0intt\n\0\0\0ValueErrort\23\0\0\0_"…, 4096) = 995
read(7, "\0\0ImportErrort\v\0\0\0__func_namet\7\0"…, 4096) = 181
stat64("/usr/lib/python2.7/site-packages/yum/Errors", 0xbfef28b0) = -1 ENOENT (No such file or directory)
open("/usr/lib/python2.7/site-packages/yum/Errors.so", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
open("/usr/lib/python2.7/site-packages/yum/Errorsmodule.so", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
open("/usr/lib/python2.7/site-packages/yum/Errors.py", O_RDONLY|O_LARGEFILE) = 6
open("/usr/lib/python2.7/site-packages/yum/Errors.pyc", O_RDONLY|O_LARGEFILE) = 7
open("/lib/libgpg-error.so.0", O_RDONLY|O_CLOEXEC) = 8
write(2, "Error: Cannot retrieve metalink "…, 101Error: Cannot retrieve metalink for repository: fedora/19/i386. Please verify its path and try again
[root@localhost packages]#
[root@localhost eth1]# ls -al /var/cache/yum/fedora/metalink.xml
ls: cannot access /var/cache/yum/fedora/metalink.xml: No such file or directory
[root@localhost eth1]#
This essentially means yum cannot get on the network to do anything:
Error: Cannot retrieve metalink for repository: fedora/19/i386. Please verify its path and try again
So we need to look at another option to see why the persistent naming rules do not work. And udevadm trigger doesn't even process these rules:
[root@localhost fedora]# strace udevadm trigger -vvvv 2>&1 | grep -i rules.d
[root@localhost fedora]# strace udevadm trigger -vvvv 2>&1 | grep -i persistent
[root@localhost fedora]#
hence the problem. So we temporarily change the rules in /etc/sysconfig/network-scripts/ifcfg-eth0/1 to the expected MAC to get some connectivity and we startup the network:
[root@localhost network-scripts]# vi ifcfg-eth0
[root@localhost network-scripts]# vi ifcfg-eth1
[root@localhost network-scripts]# service network restart
Restarting network (via systemctl): Job for network.service failed. See 'systemctl status network.service' and 'journalctl -xn' for details.
[FAILED]
[root@localhost network-scripts]# systemctl status network.service
network.service – LSB: Bring up/down networking
Loaded: loaded (/etc/rc.d/init.d/network)
Active: failed (Result: exit-code) since Sun 2013-08-11 00:38:36 EDT; 7s ago
Process: 5290 ExecStart=/etc/rc.d/init.d/network start (code=exited, status=1/FAILURE)
Aug 11 00:38:36 moto.localhost network[5290]: RTNETLINK answers: File exists
Aug 11 00:38:36 moto.localhost network[5290]: RTNETLINK answers: File exists
Aug 11 00:38:36 moto.localhost network[5290]: RTNETLINK answers: File exists
Aug 11 00:38:36 moto.localhost network[5290]: RTNETLINK answers: File exists
Aug 11 00:38:36 moto.localhost network[5290]: RTNETLINK answers: File exists
Aug 11 00:38:36 moto.localhost network[5290]: RTNETLINK answers: File exists
Aug 11 00:38:36 moto.localhost systemd-sysctl[5647]: Duplicate assignment of kernel/sysrq in file '/usr/lib/sysctl.d/50-default.conf', ignoring.
Aug 11 00:38:36 moto.localhost systemd[1]: network.service: control process exited, code=exited status=1
Aug 11 00:38:36 moto.localhost systemd[1]: Failed to start LSB: Bring up/down networking.
Aug 11 00:38:36 moto.localhost systemd[1]: Unit network.service entered failed state.
[root@localhost network-scripts]# ifup eth0
RTNETLINK answers: File exists
[root@localhost network-scripts]# ifconfig -a
eth0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet 192.168.0.13 netmask 255.255.255.0 broadcast 192.168.0.255
ether 00:0d:88:38:0a:34 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.1 netmask 255.255.0.0 broadcast 192.168.255.255
inet6 fe80::96de:80ff:fe73:b178 prefixlen 64 scopeid 0x20<link>
ether 94:de:80:73:b1:78 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 25 bytes 4363 (4.2 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 0 (Local Loopback)
RX packets 81 bytes 11963 (11.6 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 81 bytes 11963 (11.6 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@localhost network-scripts]# ping google.com
^C
[root@localhost network-scripts]# service iptables stop
Redirecting to /bin/systemctl stop iptables.service
[root@localhost network-scripts]# systemctl stop iptables.service
[root@localhost network-scripts]# systemctl start iptables.service
[root@localhost network-scripts]# iptables -nL|head
Chain INPUT (policy ACCEPT)
target prot opt source destination
DROP all — 218.189.179.82 0.0.0.0/0
ACCEPT tcp — 127.0.0.1 0.0.0.0/0 tcp dpt:3306
ACCEPT all — 127.0.0.0/8 127.0.0.0/8
ACCEPT tcp — 192.168.0.0/16 192.168.0.0/16 tcp dpt:21
ACCEPT tcp — 192.168.0.0/16 192.168.0.0/16 tcp spt:21
ACCEPT tcp — 192.168.0.0/16 192.168.0.0/16 tcp dpt:20
ACCEPT tcp — 192.168.0.0/16 192.168.0.0/16 tcp spt:20
LOG tcp — 192.168.1.0/24 192.168.1.0/24 tcp LOG flags 0 level 4
[root@localhost network-scripts]# systemctl stop iptables.service
[root@localhost network-scripts]# iptables -nL|head
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
[root@localhost network-scripts]# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
^C
[root@localhost network-scripts]#
[root@localhost network-scripts]# ping 192.168.0.1
PING 192.168.0.1 (192.168.0.1) 56(84) bytes of data.
From 192.168.0.13 icmp_seq=1 Destination Host Unreachable
From 192.168.0.13 icmp_seq=2 Destination Host Unreachable
From 192.168.0.13 icmp_seq=3 Destination Host Unreachable
From 192.168.0.13 icmp_seq=4 Destination Host Unreachable
^C
— 192.168.0.1 ping statistics —
5 packets transmitted, 0 received, +4 errors, 100% packet loss, time 4001ms
pipe 4
[root@localhost network-scripts]#
And routing seems fine:
[root@localhost network-scripts]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.0.1 0.0.0.0 UG 0 0 0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 1003 0 0 eth1
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1
[root@localhost network-scripts]#
We comment this line out in:
[root@localhost network-scripts]# cat /etc/sysctl.conf|grep sysrq
# kernel.sysrq = 0
[root@localhost network-scripts]#
to avoid this warning:
Duplicate assignment of kernel/sysrq in file '/usr/lib/sysctl.d/50-default.conf', ignoring.
Then we try something from the error messages above:
[root@localhost network-scripts]# etc/rc.d/init.d/network static.ifcfg-wlan0
2ISNOW0.ifcfg-eth2 ifcfg-Auto_covello _ifcfg-eth1 ifdown-ipv6 ifup-bnep ifup-post network-functions-ipv6
back.ifcfg-eth0 ifcfg-Auto_Ethernet _ifcfg-eth1_ ifdown-isdn ifup-eth ifup-ppp old.ifcfg-eth1
back.ifcfg-eth1 ifcfg-Auto_linksys ifcfg-eth1 ifdown-post ifup-ippp ifup-routes old.ifcfg-eth2
Dec.19.2011.ifcfg-eth2 ifcfg-Auto_VWgolf ifcfg-lo ifdown-ppp ifup-ipsec ifup-sit static.ifcfg-wlan0
dhcp.ifcfg-wlan0 ifcfg-ChiefSheriff ifdown ifdown-routes ifup-ipv6 ifup-tunnel strace-ifup.txt
FRIED.ifcfg-eth0 __ifcfg-eth0 ifdown-bnep ifdown-sit ifup-ipx ifup-wireless
ifcfg-Auto_BELL537 _ifcfg-eth0 ifdown-eth ifdown-tunnel ifup-isdn init.ipv6-global
ifcfg-Auto_BELLPerna1 _ifcfg-eth0_ ifdown-ippp ifup ifup-plip lshal-list.txt
ifcfg-Auto_Boragina ifcfg-eth0 ifdown-ipsec ifup-aliases ifup-plusb network-functions
[root@localhost network-scripts]# etc/rc.d/init.d/network static.ifcfg-wlan0
Pinging the network interface works fine:
[root@localhost network-scripts]# ifconfig -a eth0
eth0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet 192.168.0.13 netmask 255.255.255.0 broadcast 192.168.0.255
ether 00:0d:88:38:0a:34 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@localhost network-scripts]# ping 192.168.0.13
PING 192.168.0.13 (192.168.0.13) 56(84) bytes of data.
64 bytes from 192.168.0.13: icmp_seq=1 ttl=64 time=0.026 ms
64 bytes from 192.168.0.13: icmp_seq=2 ttl=64 time=0.035 ms
^C
— 192.168.0.13 ping statistics —
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.026/0.030/0.035/0.007 ms
[root@localhost network-scripts]#
When checking further we see that:
[root@localhost network-scripts]# ping 192.168.0.1
PING 192.168.0.1 (192.168.0.1) 56(84) bytes of data.
From 192.168.0.13 icmp_seq=1 Destination Host Unreachable
From 192.168.0.13 icmp_seq=2 Destination Host Unreachable
From 192.168.0.13 icmp_seq=3 Destination Host Unreachable
From 192.168.0.13 icmp_seq=4 Destination Host Unreachable
^C
— 192.168.0.1 ping statistics —
5 packets transmitted, 0 received, +4 errors, 100% packet loss, time 4000ms
pipe 4
[root@localhost network-scripts]#
And still no luck. Checking the ARP table:
[root@localhost network-scripts]# arp -an
? (192.168.0.1) at <incomplete> on eth0
[root@localhost network-scripts]#
[root@localhost network-scripts]#
[root@localhost network-scripts]#
[root@localhost network-scripts]# ethtool eth0
Settings for eth0:
Supported ports: [ TP MII ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: 10Mb/s
Duplex: Half
Port: MII
PHYAD: 32
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: pumbg
Wake-on: d
Current message level: 0x00000007 (7)
drv probe link
Link detected: no
[root@localhost network-scripts]#
And when using our recent post looks like because of the network card swap and MAC swap, our cable has been in the wrong port all along. Swapping the cable to the working NIC worked like a charm!
[root@localhost network-scripts]# ethtool eth0
Settings for eth0:
Supported ports: [ TP MII ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Link partner advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
Link partner advertised pause frame use: Symmetric
Link partner advertised auto-negotiation: Yes
Speed: 100Mb/s
Duplex: Full
Port: MII
PHYAD: 32
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: pumbg
Wake-on: d
Current message level: 0x00000007 (7)
drv probe link
Link detected: yes
[root@localhost network-scripts]#
And now to check the udev daemon and why it's not renaming things properly or using our rules!
[root@localhost network-scripts]# ifdown eth0
[root@localhost network-scripts]# ifup eth0
[root@localhost network-scripts]# ifconfig -a eth0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.13 netmask 255.255.255.0 broadcast 192.168.0.255
inet6 fe80::20d:88ff:fe38:a34 prefixlen 64 scopeid 0x20<link>
ether 00:0d:88:38:0a:34 txqueuelen 1000 (Ethernet)
RX packets 5 bytes 322 (322.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 62 bytes 13593 (13.2 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@localhost network-scripts]# ping 192.168.0.1
PING 192.168.0.1 (192.168.0.1) 56(84) bytes of data.
64 bytes from 192.168.0.1: icmp_seq=1 ttl=64 time=16.3 ms
64 bytes from 192.168.0.1: icmp_seq=2 ttl=64 time=16.2 ms
^C
— 192.168.0.1 ping statistics —
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 16.268/16.327/16.387/0.140 ms
[root@localhost network-scripts]#
But systemctl still fails:
[root@localhost packages]# service network restart
Restarting network (via systemctl): Job for network.service failed. See 'systemctl status network.service' and 'journalctl -xn' for details.
[FAILED]
[root@localhost packages]#
NetworkManager was also enabled which was disconnecting us. To disable, we ran this:
[root@localhost network-scripts]#
[root@localhost network-scripts]# ps -ef|grep -i 710
root 710 1 0 Aug10 ? 00:00:01 /usr/sbin/NetworkManager –no-daemon
root 9575 710 0 01:06 ? 00:00:00 /sbin/dhclient -d -sf /usr/libexec/nm-dhcp-client.action -pf /var/run/dhclient-eth0.pid -lf /var/lib/NetworkManager/dhclient-d20ee4a8-6a2e-422b-bd2c-994ec166f400-eth0.lease -cf /var/lib/NetworkManager/dhclient-eth0.conf eth0
root 9591 3920 0 01:07 pts/0 00:00:00 grep –color=auto -i 710
[root@localhost network-scripts]# service NetworkManager stop
Redirecting to /bin/systemctl stop NetworkManager.service
[root@localhost network-scripts]# ps -ef|grep -i 710
root 9607 3920 0 01:07 pts/0 00:00:00 grep –color=auto -i 710
[root@localhost network-scripts]# ps -ef|grep -i NetworkManager
root 9609 3920 0 01:07 pts/0 00:00:00 grep –color=auto -i NetworkManager
[root@localhost network-scripts]#
This is the same problem with NetworkManager we had in the early days of Fedora. Looks like not much has changed since then. Once turned off, networking restarts just fine:
[root@localhost network-scripts]# ifconfig
eth1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet 192.168.1.1 netmask 255.255.0.0 broadcast 192.168.255.255
inet6 fe80::96de:80ff:fe73:b178 prefixlen 64 scopeid 0x20<link>
ether 94:de:80:73:b1:78 txqueuelen 1000 (Ethernet)
RX packets 189 bytes 16226 (15.8 KiB)
RX errors 0 dropped 6 overruns 0 frame 0
TX packets 37 bytes 5083 (4.9 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 0 (Local Loopback)
RX packets 240 bytes 28636 (27.9 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 240 bytes 28636 (27.9 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@localhost network-scripts]# service network restart
Restarting network (via systemctl): Job for network.service failed. See 'systemctl status network.service' and 'journalctl -xn' for details.
[FAILED]
[root@localhost network-scripts]#
[root@localhost network-scripts]#
[root@localhost network-scripts]# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.13 netmask 255.255.255.0 broadcast 192.168.0.255
inet6 fe80::20d:88ff:fe38:a34 prefixlen 64 scopeid 0x20<link>
ether 00:0d:88:38:0a:34 txqueuelen 1000 (Ethernet)
RX packets 6797 bytes 7048610 (6.7 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 5519 bytes 770855 (752.7 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet 192.168.1.1 netmask 255.255.0.0 broadcast 192.168.255.255
inet6 fe80::96de:80ff:fe73:b178 prefixlen 64 scopeid 0x20<link>
ether 94:de:80:73:b1:78 txqueuelen 1000 (Ethernet)
RX packets 189 bytes 16226 (15.8 KiB)
RX errors 0 dropped 6 overruns 0 frame 0
TX packets 37 bytes 5083 (4.9 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 0 (Local Loopback)
RX packets 240 bytes 28636 (27.9 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 240 bytes 28636 (27.9 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@localhost network-scripts]# ping google.com
PING google.com (173.194.43.110) 56(84) bytes of data.
64 bytes from yyz08s10-in-f14.1e100.net (173.194.43.110): icmp_seq=1 ttl=56 time=23.2 ms
64 bytes from yyz08s10-in-f14.1e100.net (173.194.43.110): icmp_seq=2 ttl=56 time=22.1 ms
^C
— google.com ping statistics —
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 22.111/22.662/23.214/0.571 ms
[root@localhost network-scripts]#
And that's that.
Cheers,
TK
[…] « Device eth0 has different MAC address than expected, ignoring. […]