Skip to content
Reticux Engineering by Reticux Engineering

Lab 001: Corporate Network: Dual-Firewall OSPF | An MTU change silently moves all traffic to the backup firewall

Reticux Engineering —

RETICUX Multi-Vendor Labs · Lab 001 · Routing Foundations

An MTU change silently moves all traffic to the backup firewall. Vendors: Juniper, Fortinet, Cisco. Fault: Interface MTU mismatch on the R1–FW1 link.

Objectives

  • Build a corporate network: Juniper edge router, two FortiGate firewalls, Cisco and Juniper core switches, Cisco access switches.
  • Run VLAN 10 and VLAN 20 with MSTP, a 4-link LACP bundle and VRRP gateways shared by Cisco and Juniper.
  • Run OSPF area 0 with costs that make FW1 the primary path and FW2 the backup.
  • Inject an MTU mismatch on the R1–FW1 link and watch traffic move silently to FW2.
  • Troubleshoot it step by step, roll it back and prove the primary path is back.

Platforms

DevicePlatformSoftware
ISPCisco IOSv (any router works)IOS 15.9
R1Juniper vJunos-routerJunos OS 24.x
FW1, FW2FortiGate-VMFortiOS 7.4
SW-CORE1Cisco IOSvL2IOS 15.2
SW-CORE2Juniper vJunos-switchJunos OS 24.x
SW-ACCESS1, SW-ACCESS2Cisco IOSvL2IOS 15.2
PC1 to PC4VPCSGNS3 / EVE-NG built-in

Addressing

Point-to-point links

LinkSide ASide BOSPF cost
InternetR1 ge-0/0/0 203.0.113.2/30ISP Gi0/0 203.0.113.1/30static default
R1–FW1R1 ge-0/0/1 10.10.1.1/30FW1 port1 10.10.1.2/3010
R1–FW2R1 ge-0/0/2 10.10.1.5/30FW2 port1 10.10.1.6/30100
FW1–CORE1FW1 port2 10.10.2.1/30SW-CORE1 Gi1/1 10.10.2.2/3010
FW2–CORE2FW2 port2 10.10.2.5/30SW-CORE2 ge-0/0/6 10.10.2.6/30110
FW1–CORE2FW1 port4 10.10.2.9/30SW-CORE2 ge-0/0/7 10.10.2.10/3020
FW2–CORE1FW2 port4 10.10.2.13/30SW-CORE1 Gi1/2 10.10.2.14/30100
FGSP syncFW1 port3 10.10.9.1/30FW2 port3 10.10.9.2/30not in OSPF

VLANs, gateways and router IDs

DeviceInterfaceAddressRole
SW-CORE1Vlan10192.168.10.2/24VRRP group 10 master, priority 110
SW-CORE2irb.10192.168.10.3/24VRRP group 10 backup, priority 100
SW-CORE1Vlan20172.16.10.2/24VRRP group 20 master, priority 110
SW-CORE2irb.20172.16.10.3/24VRRP group 20 backup, priority 100
VRRPvirtual IPs192.168.10.1 · 172.16.10.1User default gateways
PC1, PC2eth0192.168.10.11 / .12VLAN 10
PC3, PC4eth0172.16.10.11 / .12VLAN 20
ISPLoopback0198.51.100.10/32Internet test address
Router IDsR1 .1 · CORE1 .11 · CORE2 .1210.255.0.x (loopbacks)
Router IDsFW1 .21 · FW2 .2210.255.0.x

Layer 2 links

LinkSide ASide BMode
Core LAGSW-CORE1 Po1: Gi0/2, Gi0/3, Gi1/0, Gi1/3SW-CORE2 ae0: ge-0/0/2 to ge-0/0/5LACP trunk, VLAN 10 and 20
ACCESS1–CORE1SW-ACCESS1 Gi0/1SW-CORE1 Gi0/1Trunk, root port (forwarding)
ACCESS1–CORE2SW-ACCESS1 Gi0/0SW-CORE2 ge-0/0/0Trunk, alternate port on ACCESS1 (blocked)
ACCESS2–CORE2SW-ACCESS2 Gi0/1SW-CORE2 ge-0/0/1Trunk, alternate port on ACCESS2 (blocked)
ACCESS2–CORE1SW-ACCESS2 Gi0/0SW-CORE1 Gi0/0Trunk, root port (forwarding)
UsersSW-ACCESS1 Gi0/2-3: PC1, PC2SW-ACCESS2 Gi0/2-3: PC3, PC4Access VLAN 10 / VLAN 20

Normal behavior

  • All six OSPF adjacencies are Full: R1 to both firewalls, and each firewall to both core switches.
  • SW-CORE1 is the MST root and the VRRP master for VLAN 10 and VLAN 20. SW-CORE2 is the backup.
  • Each access switch forwards on its link to SW-CORE1. Its link to SW-CORE2 is an MST alternate (blocked) port.
  • Lower OSPF costs on the FW1 links give one path in both directions: users → SW-CORE1 → FW1 → R1 → Internet.
  • FW1 and FW2 synchronize sessions over port3 (FGSP), so a move to FW2 keeps existing sessions.

Build the lab

ISP (IOS 15.9)

hostname ISP
!
interface Loopback0
 description Internet test address
 ip address 198.51.100.10 255.255.255.255
!
interface GigabitEthernet0/0
 description to R1 ge-0/0/0
 ip address 203.0.113.1 255.255.255.252
 no shutdown
!
ip route 192.168.10.0 255.255.255.0 203.0.113.2
ip route 172.16.10.0 255.255.255.0 203.0.113.2
!
end

The ISP router stands in for the Internet. It routes the two user subnets back to R1, so this lab needs no NAT.

R1 (Junos OS 24.x)

set system host-name R1
set interfaces ge-0/0/0 description "to ISP Gi0/0"
set interfaces ge-0/0/0 unit 0 family inet address 203.0.113.2/30
set interfaces ge-0/0/1 description "to FW1 port1"
set interfaces ge-0/0/1 unit 0 family inet address 10.10.1.1/30
set interfaces ge-0/0/2 description "to FW2 port1"
set interfaces ge-0/0/2 unit 0 family inet address 10.10.1.5/30
set interfaces lo0 unit 0 family inet address 10.255.0.1/32
set routing-options router-id 10.255.0.1
set routing-options static route 0.0.0.0/0 next-hop 203.0.113.1
set policy-options policy-statement DEFAULT-TO-OSPF term 1 from protocol static
set policy-options policy-statement DEFAULT-TO-OSPF term 1 from route-filter 0.0.0.0/0 exact
set policy-options policy-statement DEFAULT-TO-OSPF term 1 then accept
set protocols ospf export DEFAULT-TO-OSPF
set protocols ospf area 0.0.0.0 interface ge-0/0/1.0 interface-type p2p
set protocols ospf area 0.0.0.0 interface ge-0/0/1.0 metric 10
set protocols ospf area 0.0.0.0 interface ge-0/0/2.0 interface-type p2p
set protocols ospf area 0.0.0.0 interface ge-0/0/2.0 metric 100
set protocols ospf area 0.0.0.0 interface lo0.0 passive
commit and-quit

FW1 (FortiOS 7.4 · Interfaces)

config system interface
    edit "port1"
        set ip 10.10.1.2 255.255.255.252
        set allowaccess ping
        set alias "to-R1"
    next
    edit "port2"
        set ip 10.10.2.1 255.255.255.252
        set allowaccess ping
        set alias "to-SW-CORE1"
    next
    edit "port4"
        set ip 10.10.2.9 255.255.255.252
        set allowaccess ping
        set alias "to-SW-CORE2"
    next
    edit "port3"
        set ip 10.10.9.1 255.255.255.252
        set allowaccess ping
        set alias "FGSP-SYNC"
    next
end

If you manage the FortiGate-VM through port1, move management to a spare port (for example port10) before you apply this.

FW1 (FortiOS 7.4 · OSPF)

config router ospf
    set router-id 10.255.0.21
    config area
        edit 0.0.0.0
        next
    end
    config ospf-interface
        edit "to-R1"
            set interface "port1"
            set network-type point-to-point
            set cost 10
        next
        edit "to-core-a"
            set interface "port2"
            set network-type point-to-point
            set cost 10
        next
        edit "to-core-b"
            set interface "port4"
            set network-type point-to-point
            set cost 20
        next
    end
    config network
        edit 1
            set prefix 10.10.1.0 255.255.255.252
            set area 0.0.0.0
        next
        edit 2
            set prefix 10.10.2.0 255.255.255.252
            set area 0.0.0.0
        next
        edit 3
            set prefix 10.10.2.8 255.255.255.252
            set area 0.0.0.0
        next
    end
end

FW1 (FortiOS 7.4 · Session sync (FGSP))

config system standalone-cluster
    set standalone-group-id 1
    set group-member-id 1
    set layer2-connection available
    set session-sync-dev "port3"
    config cluster-peer
        edit 1
            set peerip 10.10.9.2
            set syncvd "root"
        next
    end
end
config system ha
    set session-pickup enable
    set session-pickup-connectionless enable
end

FW1 (FortiOS 7.4 · Firewall policy)

config firewall address
    edit "VLAN10-USERS"
        set subnet 192.168.10.0 255.255.255.0
    next
    edit "VLAN20-USERS"
        set subnet 172.16.10.0 255.255.255.0
    next
end
config firewall addrgrp
    edit "CORP-USERS"
        set member "VLAN10-USERS" "VLAN20-USERS"
    next
end
config firewall policy
    edit 1
        set name "USERS-TO-INTERNET"
        set srcintf "port2" "port4"
        set dstintf "port1"
        set action accept
        set srcaddr "CORP-USERS"
        set dstaddr "all"
        set schedule "always"
        set service "ALL"
        set logtraffic all
    next
end

FW2 (FortiOS 7.4 · Interfaces)

config system interface
    edit "port1"
        set ip 10.10.1.6 255.255.255.252
        set allowaccess ping
        set alias "to-R1"
    next
    edit "port2"
        set ip 10.10.2.5 255.255.255.252
        set allowaccess ping
        set alias "to-SW-CORE2"
    next
    edit "port4"
        set ip 10.10.2.13 255.255.255.252
        set allowaccess ping
        set alias "to-SW-CORE1"
    next
    edit "port3"
        set ip 10.10.9.2 255.255.255.252
        set allowaccess ping
        set alias "FGSP-SYNC"
    next
end

FW2 (FortiOS 7.4 · OSPF)

config router ospf
    set router-id 10.255.0.22
    config area
        edit 0.0.0.0
        next
    end
    config ospf-interface
        edit "to-R1"
            set interface "port1"
            set network-type point-to-point
            set cost 100
        next
        edit "to-core-a"
            set interface "port2"
            set network-type point-to-point
            set cost 110
        next
        edit "to-core-b"
            set interface "port4"
            set network-type point-to-point
            set cost 100
        next
    end
    config network
        edit 1
            set prefix 10.10.1.4 255.255.255.252
            set area 0.0.0.0
        next
        edit 2
            set prefix 10.10.2.4 255.255.255.252
            set area 0.0.0.0
        next
        edit 3
            set prefix 10.10.2.12 255.255.255.252
            set area 0.0.0.0
        next
    end
end

FW2 (FortiOS 7.4 · Session sync (FGSP))

config system standalone-cluster
    set standalone-group-id 1
    set group-member-id 2
    set layer2-connection available
    set session-sync-dev "port3"
    config cluster-peer
        edit 1
            set peerip 10.10.9.1
            set syncvd "root"
        next
    end
end
config system ha
    set session-pickup enable
    set session-pickup-connectionless enable
end

With layer2-connection available, session updates travel over port3 at layer 2 (EtherType 0x8892). The /30 on port3 is only for peer discovery and the bulk session sync, and it stays out of OSPF.

FW2 (FortiOS 7.4 · Firewall policy)

config firewall address
    edit "VLAN10-USERS"
        set subnet 192.168.10.0 255.255.255.0
    next
    edit "VLAN20-USERS"
        set subnet 172.16.10.0 255.255.255.0
    next
end
config firewall addrgrp
    edit "CORP-USERS"
        set member "VLAN10-USERS" "VLAN20-USERS"
    next
end
config firewall policy
    edit 1
        set name "USERS-TO-INTERNET"
        set srcintf "port2" "port4"
        set dstintf "port1"
        set action accept
        set srcaddr "CORP-USERS"
        set dstaddr "all"
        set schedule "always"
        set service "ALL"
        set logtraffic all
    next
end

FGSP does not copy configuration between the firewalls, so keep the address objects and policy identical on FW1 and FW2.

SW-CORE1 (IOS 15.2 (IOSvL2) · VLANs, MSTP, LACP and trunks)

hostname SW-CORE1
!
vlan 10
 name USERS-LEFT
vlan 20
 name USERS-RIGHT
!
spanning-tree mode mst
spanning-tree mst configuration
 name RETICUX
 revision 1
 instance 1 vlan 10, 20
 exit
spanning-tree mst 0-1 priority 4096
!
interface range GigabitEthernet0/2 - 3 , GigabitEthernet1/0 , GigabitEthernet1/3
 description LAG to SW-CORE2 ae0
 switchport trunk encapsulation dot1q
 switchport mode trunk
 switchport trunk allowed vlan 10,20
 channel-group 1 mode active
 no shutdown
!
interface Port-channel1
 description LAG to SW-CORE2 ae0
 switchport trunk encapsulation dot1q
 switchport mode trunk
 switchport trunk allowed vlan 10,20
!
interface GigabitEthernet0/1
 description to SW-ACCESS1 Gi0/1
 switchport trunk encapsulation dot1q
 switchport mode trunk
 switchport trunk allowed vlan 10,20
 no shutdown
!
interface GigabitEthernet0/0
 description to SW-ACCESS2 Gi0/0
 switchport trunk encapsulation dot1q
 switchport mode trunk
 switchport trunk allowed vlan 10,20
 no shutdown

SW-CORE1 (IOS 15.2 (IOSvL2) · Routing, VRRP and OSPF)

ip routing
!
interface Loopback0
 ip address 10.255.0.11 255.255.255.255
!
interface GigabitEthernet1/1
 description to FW1 port2
 no switchport
 ip address 10.10.2.2 255.255.255.252
 ip ospf network point-to-point
 ip ospf cost 10
 no shutdown
!
interface GigabitEthernet1/2
 description to FW2 port4
 no switchport
 ip address 10.10.2.14 255.255.255.252
 ip ospf network point-to-point
 ip ospf cost 100
 no shutdown
!
interface Vlan10
 ip address 192.168.10.2 255.255.255.0
 vrrp 10 ip 192.168.10.1
 vrrp 10 priority 110
 no shutdown
!
interface Vlan20
 ip address 172.16.10.2 255.255.255.0
 vrrp 20 ip 172.16.10.1
 vrrp 20 priority 110
 no shutdown
!
router ospf 1
 router-id 10.255.0.11
 passive-interface Vlan10
 passive-interface Vlan20
 passive-interface Loopback0
 network 10.10.2.0 0.0.0.3 area 0
 network 10.10.2.12 0.0.0.3 area 0
 network 192.168.10.0 0.0.0.255 area 0
 network 172.16.10.0 0.0.0.255 area 0
 network 10.255.0.11 0.0.0.0 area 0
!
end

This is classic IOS VRRP (version 2), which matches the Junos default. On Catalyst switches running IOS XE 16 or later, VRRP is configured with fhrp version vrrp v3 and vrrp 10 address-family ipv4 instead.

SW-CORE2 (Junos OS 24.x (vJunos-switch) · VLANs, MSTP, LACP and trunks)

set system host-name SW-CORE2
delete protocols rstp
delete interfaces ge-0/0/2
delete interfaces ge-0/0/3
delete interfaces ge-0/0/4
delete interfaces ge-0/0/5
delete interfaces ge-0/0/6
delete interfaces ge-0/0/7
set chassis aggregated-devices ethernet device-count 1
set vlans V10 vlan-id 10
set vlans V10 l3-interface irb.10
set vlans V20 vlan-id 20
set vlans V20 l3-interface irb.20
set interfaces ge-0/0/2 ether-options 802.3ad ae0
set interfaces ge-0/0/3 ether-options 802.3ad ae0
set interfaces ge-0/0/4 ether-options 802.3ad ae0
set interfaces ge-0/0/5 ether-options 802.3ad ae0
set interfaces ae0 description "LAG to SW-CORE1 Po1"
set interfaces ae0 aggregated-ether-options lacp active
set interfaces ae0 unit 0 family ethernet-switching interface-mode trunk
set interfaces ae0 unit 0 family ethernet-switching vlan members [ V10 V20 ]
set interfaces ge-0/0/0 description "to SW-ACCESS1 Gi0/0"
set interfaces ge-0/0/0 unit 0 family ethernet-switching interface-mode trunk
set interfaces ge-0/0/0 unit 0 family ethernet-switching vlan members [ V10 V20 ]
set interfaces ge-0/0/1 description "to SW-ACCESS2 Gi0/1"
set interfaces ge-0/0/1 unit 0 family ethernet-switching interface-mode trunk
set interfaces ge-0/0/1 unit 0 family ethernet-switching vlan members [ V10 V20 ]
set protocols mstp configuration-name RETICUX
set protocols mstp revision-level 1
set protocols mstp bridge-priority 8k
set protocols mstp msti 1 bridge-priority 8k
set protocols mstp msti 1 vlan [ 10 20 ]
set protocols mstp interface ae0
set protocols mstp interface ge-0/0/0
set protocols mstp interface ge-0/0/1

The delete lines clear any factory-default settings on those ports. Junos only prints a warning if there is nothing to delete. Commit once, at the end of the next block.

SW-CORE2 (Junos OS 24.x (vJunos-switch) · Routing, VRRP and OSPF)

set interfaces ge-0/0/6 description "to FW2 port2"
set interfaces ge-0/0/6 unit 0 family inet address 10.10.2.6/30
set interfaces ge-0/0/7 description "to FW1 port4"
set interfaces ge-0/0/7 unit 0 family inet address 10.10.2.10/30
set interfaces lo0 unit 0 family inet address 10.255.0.12/32
set interfaces irb unit 10 family inet address 192.168.10.3/24 vrrp-group 10 virtual-address 192.168.10.1
set interfaces irb unit 10 family inet address 192.168.10.3/24 vrrp-group 10 priority 100
set interfaces irb unit 10 family inet address 192.168.10.3/24 vrrp-group 10 accept-data
set interfaces irb unit 20 family inet address 172.16.10.3/24 vrrp-group 20 virtual-address 172.16.10.1
set interfaces irb unit 20 family inet address 172.16.10.3/24 vrrp-group 20 priority 100
set interfaces irb unit 20 family inet address 172.16.10.3/24 vrrp-group 20 accept-data
set routing-options router-id 10.255.0.12
set protocols ospf area 0.0.0.0 interface ge-0/0/6.0 interface-type p2p
set protocols ospf area 0.0.0.0 interface ge-0/0/6.0 metric 110
set protocols ospf area 0.0.0.0 interface ge-0/0/7.0 interface-type p2p
set protocols ospf area 0.0.0.0 interface ge-0/0/7.0 metric 20
set protocols ospf area 0.0.0.0 interface irb.10 passive
set protocols ospf area 0.0.0.0 interface irb.20 passive
set protocols ospf area 0.0.0.0 interface lo0.0 passive
commit and-quit

SW-ACCESS1 (IOS 15.2 (IOSvL2))

hostname SW-ACCESS1
!
vlan 10
 name USERS-LEFT
vlan 20
 name USERS-RIGHT
!
spanning-tree mode mst
spanning-tree mst configuration
 name RETICUX
 revision 1
 instance 1 vlan 10, 20
 exit
!
interface GigabitEthernet0/1
 description to SW-CORE1 Gi0/1
 switchport trunk encapsulation dot1q
 switchport mode trunk
 switchport trunk allowed vlan 10,20
 no shutdown
!
interface GigabitEthernet0/0
 description to SW-CORE2 ge-0/0/0
 switchport trunk encapsulation dot1q
 switchport mode trunk
 switchport trunk allowed vlan 10,20
 no shutdown
!
interface range GigabitEthernet0/2 - 3
 description users VLAN 10
 switchport mode access
 switchport access vlan 10
 spanning-tree portfast
 no shutdown
!
end

SW-ACCESS2 (IOS 15.2 (IOSvL2))

hostname SW-ACCESS2
!
vlan 10
 name USERS-LEFT
vlan 20
 name USERS-RIGHT
!
spanning-tree mode mst
spanning-tree mst configuration
 name RETICUX
 revision 1
 instance 1 vlan 10, 20
 exit
!
interface GigabitEthernet0/1
 description to SW-CORE2 ge-0/0/1
 switchport trunk encapsulation dot1q
 switchport mode trunk
 switchport trunk allowed vlan 10,20
 no shutdown
!
interface GigabitEthernet0/0
 description to SW-CORE1 Gi0/0
 switchport trunk encapsulation dot1q
 switchport mode trunk
 switchport trunk allowed vlan 10,20
 no shutdown
!
interface range GigabitEthernet0/2 - 3
 description users VLAN 20
 switchport mode access
 switchport access vlan 20
 spanning-tree portfast
 no shutdown
!
end

All four switches use the same MST region (name RETICUX, revision 1, instance 1 = VLAN 10 and 20). A different name, revision or VLAN mapping on any switch splits the region.

PC1 to PC4 (VPCS)

PC1> ip 192.168.10.11/24 192.168.10.1
PC2> ip 192.168.10.12/24 192.168.10.1
PC3> ip 172.16.10.11/24 172.16.10.1
PC4> ip 172.16.10.12/24 172.16.10.1

Baseline verification

SW-CORE1

show etherchannel summary
show spanning-tree mst 1
show vrrp brief
show ip ospf neighbor
show ip route 0.0.0.0

Expect: Po1 is (SU) with four (P) ports. "This switch is the root" for MST 1. Vlan10 and Vlan20 are Master. Neighbors 10.255.0.21 and 10.255.0.22 are FULL. Default route via 10.10.2.1 (FW1).

SW-CORE2

show lacp interfaces ae0
show vrrp summary
show ospf neighbor

Expect: All four ae0 members Collecting distributing. Both VRRP groups in backup. FW1 and FW2 neighbors Full.

R1

show ospf neighbor
show route 192.168.10.0/24

Expect: 10.10.1.2 (FW1) and 10.10.1.6 (FW2) Full. 192.168.10.0/24 via 10.10.1.2 on ge-0/0/1.0.

FW1

get router info ospf neighbor
get router info routing-table all
diagnose sys session list | grep "SN\|total"

Expect: R1 and both cores Full. Default route via 10.10.1.1 on port1. Run the session command on FW2 too: the totals match when FGSP is in sync.

PC1

ping 198.51.100.10
trace 198.51.100.10 -P 1

Expect: Replies. Hops: 192.168.10.2 (SW-CORE1) → 10.10.2.1 (FW1) → 10.10.1.1 (R1) → 198.51.100.10.

PC3

trace 198.51.100.10 -P 1

Expect: Hops: 172.16.10.2 (SW-CORE1) → 10.10.2.1 (FW1) → 10.10.1.1 (R1) → 198.51.100.10.

Inject the fault

R1

configure
set interfaces ge-0/0/1 unit 0 family inet mtu 1400
deactivate protocols ospf area 0.0.0.0 interface ge-0/0/1.0
commit
activate protocols ospf area 0.0.0.0 interface ge-0/0/1.0
commit and-quit

OSPF compares MTUs only during the Database Description exchange, so an adjacency that is already Full survives the MTU change. Deactivating and reactivating OSPF on ge-0/0/1.0 forces a new exchange, which is exactly what the next link flap or reboot would do.

Behavior after the fault

DeviceOSPF stateRouting tableTraffic
R1FW1 (10.255.0.21) stuck in ExStart; FW2 FullUser subnets now via 10.10.1.6 (FW2)Sends all return traffic to FW2
FW1R1 stuck in ExStart (some releases show Exchange); both cores FullDefault route now via 10.10.2.2 (SW-CORE1), not R1Carries no user traffic
FW2R1 and both cores FullDefault via 10.10.1.5 (R1)Carries all user traffic, both directions
SW-CORE1FW1 and FW2 FullDefault now via 10.10.2.13 (FW2)Still VRRP master and MST root
UsersNo changeNo changePing still works; traceroute hop 2 changes to 10.10.2.13

Troubleshoot step by step

Step 1: Notice the path change (PC1)

trace 198.51.100.10 -P 1

The ping works, but hop 2 is now 10.10.2.13 (FW2) instead of 10.10.2.1 (FW1). Nothing is down, yet the primary firewall is out of the path.

Step 2: Check the core's default route (SW-CORE1)

show ip route 0.0.0.0
show ip ospf neighbor

Default via 10.10.2.13 on Gi1/2. Both firewall neighbors are FULL, so the core is healthy: FW1 simply no longer offers the better default route.

Step 3: Look at FW1's routing table (FW1)

get router info routing-table all

FW1's default route points to 10.10.2.2 (SW-CORE1) instead of 10.10.1.1 (R1). FW1 has lost its own path to R1.

Step 4: Check FW1's OSPF neighbors (FW1)

get router info ospf neighbor

R1 (10.255.0.1) on port1 never leaves ExStart while both cores are Full. The problem is the R1–FW1 adjacency.

Step 5: Confirm on R1 and test the link (R1)

show ospf neighbor
ping 10.10.1.2 count 3

10.10.1.2 is stuck in ExStart on ge-0/0/1.0, yet the ping succeeds. Layers 1 to 3 work, so this is not cabling or addressing.

Step 6: Rule out duplicate router IDs (R1 and FW1)

R1>  show ospf overview
FW1#  get router info ospf status

Router IDs are 10.255.0.1 and 10.255.0.21: unique. The next common cause of a neighbor stuck in ExStart or Exchange is an MTU mismatch.

Step 7: Compare the MTU on both ends (R1 and FW1)

R1>  show interfaces ge-0/0/1.0
FW1#  get router info ospf interface port1

R1 shows Protocol inet, MTU: 1400. FW1 port1 uses the default 1500. The two ends disagree.

Step 8: Find the change (R1)

show configuration interfaces ge-0/0/1
show system commit

family inet mtu 1400 is present, and the commit history shows when it was added. Decision: roll it back.

Root cause

  • OSPF writes the sending interface's IP MTU into every Database Description (DBD) packet.
  • RFC 2328 section 10.6: a router rejects a DBD packet whose MTU is larger than it can receive on that interface without fragmentation.
  • R1 (1400) rejects every DBD from FW1 (1500), so the database exchange never finishes and the R1–FW1 adjacency never reaches Full. FW1 has the higher router ID and expects to lead the exchange, which is why both sides usually stay in ExStart.
  • With no Full adjacency the link drops out of the SPF calculation. OSPF picks the next best path through FW2, and FW1 even learns its default route through SW-CORE1.
  • Nothing goes down, so there is no outage and usually no alarm. The redundancy is gone: if FW2 fails now, users lose the Internet.

Why not mtu-ignore? Do not force the adjacency up with mtu-ignore (set mtu-ignore enable on the FortiGate OSPF interface). It only skips the check: IP packets between 1401 and 1500 bytes from FW1 to R1 still fail, and large OSPF updates can be lost. Fix the MTU itself.

Roll back

R1

configure
delete interfaces ge-0/0/1 unit 0 family inet mtu
commit and-quit

R1 accepts FW1's next DBD retransmission and the adjacency reaches Full within seconds. OSPF costs then move traffic back to FW1. To return to the last good commit instead, use show | compare rollback 1 to check it, then rollback 1 and commit.

Verify the rollback

R1

show ospf neighbor
show route 192.168.10.0/24
ping 10.10.1.2 size 1472 do-not-fragment count 3

Pass: FW1 and FW2 both Full. 192.168.10.0/24 back via 10.10.1.2. A 1500-byte packet (1472 + 28 bytes of headers) with DF set crosses the link.

FW1

get router info routing-table all

Pass: Default route back via 10.10.1.1 on port1.

SW-CORE1

show ip route 0.0.0.0

Pass: Default route back via 10.10.2.1 (FW1).

PC1 and PC3

trace 198.51.100.10 -P 1

Pass: Hop 2 is 10.10.2.1 (FW1) again.

Key takeaways

  1. A neighbor stuck in ExStart is not always an outage. With redundant paths, OSPF reroutes and hides it.
  2. Watch for path changes: a new traceroute hop or a default route through the wrong neighbor is a clue.
  3. Neighbor never leaves ExStart or Exchange? Check MTU first, then router IDs.
  4. OSPF checks MTU only during the DBD exchange, so the fault appears at the next flap, not at the change.
  5. Fix the MTU, not the symptom, and prove it with a 1500-byte ping with DF set.

References

  1. RFC 2328, OSPF Version 2 (section 10.6, Receiving Database Description packets)
  2. RFC 3768, Virtual Router Redundancy Protocol (VRRP)
  3. Cisco: Troubleshoot OSPF Neighbors Stuck in Exstart/Exchange State
  4. Cisco: Understand Multiple Spanning Tree Protocol (802.1s)
  5. Cisco: Configuring VRRP (IOS First Hop Redundancy Protocols guide)
  6. Juniper: Setting the Protocol MTU
  7. Juniper: Configuring OSPF Interfaces
  8. Juniper: Example, Configuring a Conditional Default Route Policy
  9. Juniper: Example, Configuring VRRP for IPv4
  10. Juniper: VRRPv3 support in Junos (version 2 is the default)
  11. Juniper: Routed VLAN interfaces (IRB)
  12. Juniper: Configuring aggregated Ethernet links (EX Series)
  13. Juniper: mstp (ELS) configuration statement
  14. Fortinet CLI reference: config router ospf
  15. Fortinet: Basic OSPF configuration and MTU settings in OSPF
  16. Fortinet: OSPF neighbour stuck in EXSTART/EXCHANGE state
  17. Fortinet: FGSP configuration changes as of FortiOS 7.2 and 7.4
  18. Fortinet: Suggested parameters for an FGSP cluster

Download the full lab PDF and watch the 30-second traffic-flow video on the RETICUX Youtube.

Written by Reticux Engineering

Practical, lab-tested network engineering: Cisco, Juniper and Fortinet configurations, troubleshooting steps and exam-focused practice.

No comments:

Learn BGP, Cisco routing and network security with daily lab-driven lessons, from CCNA basics to CCNP level policy design.