RETICUX Multi-Vendor Labs · Lab 001 · Routing Foundations
An MTU change silently moves all traffic to the backup firewall. Vendors: Juniper, Fortinet, Cisco. Fault: Interface MTU mismatch on the R1–FW1 link.
Objectives
- Build a corporate network: Juniper edge router, two FortiGate firewalls, Cisco and Juniper core switches, Cisco access switches.
- Run VLAN 10 and VLAN 20 with MSTP, a 4-link LACP bundle and VRRP gateways shared by Cisco and Juniper.
- Run OSPF area 0 with costs that make FW1 the primary path and FW2 the backup.
- Inject an MTU mismatch on the R1–FW1 link and watch traffic move silently to FW2.
- Troubleshoot it step by step, roll it back and prove the primary path is back.
Platforms
| Device | Platform | Software |
|---|---|---|
| ISP | Cisco IOSv (any router works) | IOS 15.9 |
| R1 | Juniper vJunos-router | Junos OS 24.x |
| FW1, FW2 | FortiGate-VM | FortiOS 7.4 |
| SW-CORE1 | Cisco IOSvL2 | IOS 15.2 |
| SW-CORE2 | Juniper vJunos-switch | Junos OS 24.x |
| SW-ACCESS1, SW-ACCESS2 | Cisco IOSvL2 | IOS 15.2 |
| PC1 to PC4 | VPCS | GNS3 / EVE-NG built-in |
Addressing
Point-to-point links
| Link | Side A | Side B | OSPF cost |
|---|---|---|---|
| Internet | R1 ge-0/0/0 203.0.113.2/30 | ISP Gi0/0 203.0.113.1/30 | static default |
| R1–FW1 | R1 ge-0/0/1 10.10.1.1/30 | FW1 port1 10.10.1.2/30 | 10 |
| R1–FW2 | R1 ge-0/0/2 10.10.1.5/30 | FW2 port1 10.10.1.6/30 | 100 |
| FW1–CORE1 | FW1 port2 10.10.2.1/30 | SW-CORE1 Gi1/1 10.10.2.2/30 | 10 |
| FW2–CORE2 | FW2 port2 10.10.2.5/30 | SW-CORE2 ge-0/0/6 10.10.2.6/30 | 110 |
| FW1–CORE2 | FW1 port4 10.10.2.9/30 | SW-CORE2 ge-0/0/7 10.10.2.10/30 | 20 |
| FW2–CORE1 | FW2 port4 10.10.2.13/30 | SW-CORE1 Gi1/2 10.10.2.14/30 | 100 |
| FGSP sync | FW1 port3 10.10.9.1/30 | FW2 port3 10.10.9.2/30 | not in OSPF |
VLANs, gateways and router IDs
| Device | Interface | Address | Role |
|---|---|---|---|
| SW-CORE1 | Vlan10 | 192.168.10.2/24 | VRRP group 10 master, priority 110 |
| SW-CORE2 | irb.10 | 192.168.10.3/24 | VRRP group 10 backup, priority 100 |
| SW-CORE1 | Vlan20 | 172.16.10.2/24 | VRRP group 20 master, priority 110 |
| SW-CORE2 | irb.20 | 172.16.10.3/24 | VRRP group 20 backup, priority 100 |
| VRRP | virtual IPs | 192.168.10.1 · 172.16.10.1 | User default gateways |
| PC1, PC2 | eth0 | 192.168.10.11 / .12 | VLAN 10 |
| PC3, PC4 | eth0 | 172.16.10.11 / .12 | VLAN 20 |
| ISP | Loopback0 | 198.51.100.10/32 | Internet test address |
| Router IDs | R1 .1 · CORE1 .11 · CORE2 .12 | 10.255.0.x (loopbacks) | |
| Router IDs | FW1 .21 · FW2 .22 | 10.255.0.x |
Layer 2 links
| Link | Side A | Side B | Mode |
|---|---|---|---|
| Core LAG | SW-CORE1 Po1: Gi0/2, Gi0/3, Gi1/0, Gi1/3 | SW-CORE2 ae0: ge-0/0/2 to ge-0/0/5 | LACP trunk, VLAN 10 and 20 |
| ACCESS1–CORE1 | SW-ACCESS1 Gi0/1 | SW-CORE1 Gi0/1 | Trunk, root port (forwarding) |
| ACCESS1–CORE2 | SW-ACCESS1 Gi0/0 | SW-CORE2 ge-0/0/0 | Trunk, alternate port on ACCESS1 (blocked) |
| ACCESS2–CORE2 | SW-ACCESS2 Gi0/1 | SW-CORE2 ge-0/0/1 | Trunk, alternate port on ACCESS2 (blocked) |
| ACCESS2–CORE1 | SW-ACCESS2 Gi0/0 | SW-CORE1 Gi0/0 | Trunk, root port (forwarding) |
| Users | SW-ACCESS1 Gi0/2-3: PC1, PC2 | SW-ACCESS2 Gi0/2-3: PC3, PC4 | Access VLAN 10 / VLAN 20 |
Normal behavior
- All six OSPF adjacencies are Full: R1 to both firewalls, and each firewall to both core switches.
- SW-CORE1 is the MST root and the VRRP master for VLAN 10 and VLAN 20. SW-CORE2 is the backup.
- Each access switch forwards on its link to SW-CORE1. Its link to SW-CORE2 is an MST alternate (blocked) port.
- Lower OSPF costs on the FW1 links give one path in both directions: users → SW-CORE1 → FW1 → R1 → Internet.
- FW1 and FW2 synchronize sessions over port3 (FGSP), so a move to FW2 keeps existing sessions.
Build the lab
ISP (IOS 15.9)
hostname ISP
!
interface Loopback0
description Internet test address
ip address 198.51.100.10 255.255.255.255
!
interface GigabitEthernet0/0
description to R1 ge-0/0/0
ip address 203.0.113.1 255.255.255.252
no shutdown
!
ip route 192.168.10.0 255.255.255.0 203.0.113.2
ip route 172.16.10.0 255.255.255.0 203.0.113.2
!
end
The ISP router stands in for the Internet. It routes the two user subnets back to R1, so this lab needs no NAT.
R1 (Junos OS 24.x)
set system host-name R1
set interfaces ge-0/0/0 description "to ISP Gi0/0"
set interfaces ge-0/0/0 unit 0 family inet address 203.0.113.2/30
set interfaces ge-0/0/1 description "to FW1 port1"
set interfaces ge-0/0/1 unit 0 family inet address 10.10.1.1/30
set interfaces ge-0/0/2 description "to FW2 port1"
set interfaces ge-0/0/2 unit 0 family inet address 10.10.1.5/30
set interfaces lo0 unit 0 family inet address 10.255.0.1/32
set routing-options router-id 10.255.0.1
set routing-options static route 0.0.0.0/0 next-hop 203.0.113.1
set policy-options policy-statement DEFAULT-TO-OSPF term 1 from protocol static
set policy-options policy-statement DEFAULT-TO-OSPF term 1 from route-filter 0.0.0.0/0 exact
set policy-options policy-statement DEFAULT-TO-OSPF term 1 then accept
set protocols ospf export DEFAULT-TO-OSPF
set protocols ospf area 0.0.0.0 interface ge-0/0/1.0 interface-type p2p
set protocols ospf area 0.0.0.0 interface ge-0/0/1.0 metric 10
set protocols ospf area 0.0.0.0 interface ge-0/0/2.0 interface-type p2p
set protocols ospf area 0.0.0.0 interface ge-0/0/2.0 metric 100
set protocols ospf area 0.0.0.0 interface lo0.0 passive
commit and-quit
FW1 (FortiOS 7.4 · Interfaces)
config system interface
edit "port1"
set ip 10.10.1.2 255.255.255.252
set allowaccess ping
set alias "to-R1"
next
edit "port2"
set ip 10.10.2.1 255.255.255.252
set allowaccess ping
set alias "to-SW-CORE1"
next
edit "port4"
set ip 10.10.2.9 255.255.255.252
set allowaccess ping
set alias "to-SW-CORE2"
next
edit "port3"
set ip 10.10.9.1 255.255.255.252
set allowaccess ping
set alias "FGSP-SYNC"
next
end
If you manage the FortiGate-VM through port1, move management to a spare port (for example port10) before you apply this.
FW1 (FortiOS 7.4 · OSPF)
config router ospf
set router-id 10.255.0.21
config area
edit 0.0.0.0
next
end
config ospf-interface
edit "to-R1"
set interface "port1"
set network-type point-to-point
set cost 10
next
edit "to-core-a"
set interface "port2"
set network-type point-to-point
set cost 10
next
edit "to-core-b"
set interface "port4"
set network-type point-to-point
set cost 20
next
end
config network
edit 1
set prefix 10.10.1.0 255.255.255.252
set area 0.0.0.0
next
edit 2
set prefix 10.10.2.0 255.255.255.252
set area 0.0.0.0
next
edit 3
set prefix 10.10.2.8 255.255.255.252
set area 0.0.0.0
next
end
end
FW1 (FortiOS 7.4 · Session sync (FGSP))
config system standalone-cluster
set standalone-group-id 1
set group-member-id 1
set layer2-connection available
set session-sync-dev "port3"
config cluster-peer
edit 1
set peerip 10.10.9.2
set syncvd "root"
next
end
end
config system ha
set session-pickup enable
set session-pickup-connectionless enable
end
FW1 (FortiOS 7.4 · Firewall policy)
config firewall address
edit "VLAN10-USERS"
set subnet 192.168.10.0 255.255.255.0
next
edit "VLAN20-USERS"
set subnet 172.16.10.0 255.255.255.0
next
end
config firewall addrgrp
edit "CORP-USERS"
set member "VLAN10-USERS" "VLAN20-USERS"
next
end
config firewall policy
edit 1
set name "USERS-TO-INTERNET"
set srcintf "port2" "port4"
set dstintf "port1"
set action accept
set srcaddr "CORP-USERS"
set dstaddr "all"
set schedule "always"
set service "ALL"
set logtraffic all
next
end
FW2 (FortiOS 7.4 · Interfaces)
config system interface
edit "port1"
set ip 10.10.1.6 255.255.255.252
set allowaccess ping
set alias "to-R1"
next
edit "port2"
set ip 10.10.2.5 255.255.255.252
set allowaccess ping
set alias "to-SW-CORE2"
next
edit "port4"
set ip 10.10.2.13 255.255.255.252
set allowaccess ping
set alias "to-SW-CORE1"
next
edit "port3"
set ip 10.10.9.2 255.255.255.252
set allowaccess ping
set alias "FGSP-SYNC"
next
end
FW2 (FortiOS 7.4 · OSPF)
config router ospf
set router-id 10.255.0.22
config area
edit 0.0.0.0
next
end
config ospf-interface
edit "to-R1"
set interface "port1"
set network-type point-to-point
set cost 100
next
edit "to-core-a"
set interface "port2"
set network-type point-to-point
set cost 110
next
edit "to-core-b"
set interface "port4"
set network-type point-to-point
set cost 100
next
end
config network
edit 1
set prefix 10.10.1.4 255.255.255.252
set area 0.0.0.0
next
edit 2
set prefix 10.10.2.4 255.255.255.252
set area 0.0.0.0
next
edit 3
set prefix 10.10.2.12 255.255.255.252
set area 0.0.0.0
next
end
end
FW2 (FortiOS 7.4 · Session sync (FGSP))
config system standalone-cluster
set standalone-group-id 1
set group-member-id 2
set layer2-connection available
set session-sync-dev "port3"
config cluster-peer
edit 1
set peerip 10.10.9.1
set syncvd "root"
next
end
end
config system ha
set session-pickup enable
set session-pickup-connectionless enable
end
With layer2-connection available, session updates travel over port3 at layer 2 (EtherType 0x8892). The /30 on port3 is only for peer discovery and the bulk session sync, and it stays out of OSPF.
FW2 (FortiOS 7.4 · Firewall policy)
config firewall address
edit "VLAN10-USERS"
set subnet 192.168.10.0 255.255.255.0
next
edit "VLAN20-USERS"
set subnet 172.16.10.0 255.255.255.0
next
end
config firewall addrgrp
edit "CORP-USERS"
set member "VLAN10-USERS" "VLAN20-USERS"
next
end
config firewall policy
edit 1
set name "USERS-TO-INTERNET"
set srcintf "port2" "port4"
set dstintf "port1"
set action accept
set srcaddr "CORP-USERS"
set dstaddr "all"
set schedule "always"
set service "ALL"
set logtraffic all
next
end
FGSP does not copy configuration between the firewalls, so keep the address objects and policy identical on FW1 and FW2.
SW-CORE1 (IOS 15.2 (IOSvL2) · VLANs, MSTP, LACP and trunks)
hostname SW-CORE1
!
vlan 10
name USERS-LEFT
vlan 20
name USERS-RIGHT
!
spanning-tree mode mst
spanning-tree mst configuration
name RETICUX
revision 1
instance 1 vlan 10, 20
exit
spanning-tree mst 0-1 priority 4096
!
interface range GigabitEthernet0/2 - 3 , GigabitEthernet1/0 , GigabitEthernet1/3
description LAG to SW-CORE2 ae0
switchport trunk encapsulation dot1q
switchport mode trunk
switchport trunk allowed vlan 10,20
channel-group 1 mode active
no shutdown
!
interface Port-channel1
description LAG to SW-CORE2 ae0
switchport trunk encapsulation dot1q
switchport mode trunk
switchport trunk allowed vlan 10,20
!
interface GigabitEthernet0/1
description to SW-ACCESS1 Gi0/1
switchport trunk encapsulation dot1q
switchport mode trunk
switchport trunk allowed vlan 10,20
no shutdown
!
interface GigabitEthernet0/0
description to SW-ACCESS2 Gi0/0
switchport trunk encapsulation dot1q
switchport mode trunk
switchport trunk allowed vlan 10,20
no shutdown
SW-CORE1 (IOS 15.2 (IOSvL2) · Routing, VRRP and OSPF)
ip routing
!
interface Loopback0
ip address 10.255.0.11 255.255.255.255
!
interface GigabitEthernet1/1
description to FW1 port2
no switchport
ip address 10.10.2.2 255.255.255.252
ip ospf network point-to-point
ip ospf cost 10
no shutdown
!
interface GigabitEthernet1/2
description to FW2 port4
no switchport
ip address 10.10.2.14 255.255.255.252
ip ospf network point-to-point
ip ospf cost 100
no shutdown
!
interface Vlan10
ip address 192.168.10.2 255.255.255.0
vrrp 10 ip 192.168.10.1
vrrp 10 priority 110
no shutdown
!
interface Vlan20
ip address 172.16.10.2 255.255.255.0
vrrp 20 ip 172.16.10.1
vrrp 20 priority 110
no shutdown
!
router ospf 1
router-id 10.255.0.11
passive-interface Vlan10
passive-interface Vlan20
passive-interface Loopback0
network 10.10.2.0 0.0.0.3 area 0
network 10.10.2.12 0.0.0.3 area 0
network 192.168.10.0 0.0.0.255 area 0
network 172.16.10.0 0.0.0.255 area 0
network 10.255.0.11 0.0.0.0 area 0
!
end
This is classic IOS VRRP (version 2), which matches the Junos default. On Catalyst switches running IOS XE 16 or later, VRRP is configured with fhrp version vrrp v3 and vrrp 10 address-family ipv4 instead.
SW-CORE2 (Junos OS 24.x (vJunos-switch) · VLANs, MSTP, LACP and trunks)
set system host-name SW-CORE2
delete protocols rstp
delete interfaces ge-0/0/2
delete interfaces ge-0/0/3
delete interfaces ge-0/0/4
delete interfaces ge-0/0/5
delete interfaces ge-0/0/6
delete interfaces ge-0/0/7
set chassis aggregated-devices ethernet device-count 1
set vlans V10 vlan-id 10
set vlans V10 l3-interface irb.10
set vlans V20 vlan-id 20
set vlans V20 l3-interface irb.20
set interfaces ge-0/0/2 ether-options 802.3ad ae0
set interfaces ge-0/0/3 ether-options 802.3ad ae0
set interfaces ge-0/0/4 ether-options 802.3ad ae0
set interfaces ge-0/0/5 ether-options 802.3ad ae0
set interfaces ae0 description "LAG to SW-CORE1 Po1"
set interfaces ae0 aggregated-ether-options lacp active
set interfaces ae0 unit 0 family ethernet-switching interface-mode trunk
set interfaces ae0 unit 0 family ethernet-switching vlan members [ V10 V20 ]
set interfaces ge-0/0/0 description "to SW-ACCESS1 Gi0/0"
set interfaces ge-0/0/0 unit 0 family ethernet-switching interface-mode trunk
set interfaces ge-0/0/0 unit 0 family ethernet-switching vlan members [ V10 V20 ]
set interfaces ge-0/0/1 description "to SW-ACCESS2 Gi0/1"
set interfaces ge-0/0/1 unit 0 family ethernet-switching interface-mode trunk
set interfaces ge-0/0/1 unit 0 family ethernet-switching vlan members [ V10 V20 ]
set protocols mstp configuration-name RETICUX
set protocols mstp revision-level 1
set protocols mstp bridge-priority 8k
set protocols mstp msti 1 bridge-priority 8k
set protocols mstp msti 1 vlan [ 10 20 ]
set protocols mstp interface ae0
set protocols mstp interface ge-0/0/0
set protocols mstp interface ge-0/0/1
The delete lines clear any factory-default settings on those ports. Junos only prints a warning if there is nothing to delete. Commit once, at the end of the next block.
SW-CORE2 (Junos OS 24.x (vJunos-switch) · Routing, VRRP and OSPF)
set interfaces ge-0/0/6 description "to FW2 port2"
set interfaces ge-0/0/6 unit 0 family inet address 10.10.2.6/30
set interfaces ge-0/0/7 description "to FW1 port4"
set interfaces ge-0/0/7 unit 0 family inet address 10.10.2.10/30
set interfaces lo0 unit 0 family inet address 10.255.0.12/32
set interfaces irb unit 10 family inet address 192.168.10.3/24 vrrp-group 10 virtual-address 192.168.10.1
set interfaces irb unit 10 family inet address 192.168.10.3/24 vrrp-group 10 priority 100
set interfaces irb unit 10 family inet address 192.168.10.3/24 vrrp-group 10 accept-data
set interfaces irb unit 20 family inet address 172.16.10.3/24 vrrp-group 20 virtual-address 172.16.10.1
set interfaces irb unit 20 family inet address 172.16.10.3/24 vrrp-group 20 priority 100
set interfaces irb unit 20 family inet address 172.16.10.3/24 vrrp-group 20 accept-data
set routing-options router-id 10.255.0.12
set protocols ospf area 0.0.0.0 interface ge-0/0/6.0 interface-type p2p
set protocols ospf area 0.0.0.0 interface ge-0/0/6.0 metric 110
set protocols ospf area 0.0.0.0 interface ge-0/0/7.0 interface-type p2p
set protocols ospf area 0.0.0.0 interface ge-0/0/7.0 metric 20
set protocols ospf area 0.0.0.0 interface irb.10 passive
set protocols ospf area 0.0.0.0 interface irb.20 passive
set protocols ospf area 0.0.0.0 interface lo0.0 passive
commit and-quit
SW-ACCESS1 (IOS 15.2 (IOSvL2))
hostname SW-ACCESS1
!
vlan 10
name USERS-LEFT
vlan 20
name USERS-RIGHT
!
spanning-tree mode mst
spanning-tree mst configuration
name RETICUX
revision 1
instance 1 vlan 10, 20
exit
!
interface GigabitEthernet0/1
description to SW-CORE1 Gi0/1
switchport trunk encapsulation dot1q
switchport mode trunk
switchport trunk allowed vlan 10,20
no shutdown
!
interface GigabitEthernet0/0
description to SW-CORE2 ge-0/0/0
switchport trunk encapsulation dot1q
switchport mode trunk
switchport trunk allowed vlan 10,20
no shutdown
!
interface range GigabitEthernet0/2 - 3
description users VLAN 10
switchport mode access
switchport access vlan 10
spanning-tree portfast
no shutdown
!
end
SW-ACCESS2 (IOS 15.2 (IOSvL2))
hostname SW-ACCESS2
!
vlan 10
name USERS-LEFT
vlan 20
name USERS-RIGHT
!
spanning-tree mode mst
spanning-tree mst configuration
name RETICUX
revision 1
instance 1 vlan 10, 20
exit
!
interface GigabitEthernet0/1
description to SW-CORE2 ge-0/0/1
switchport trunk encapsulation dot1q
switchport mode trunk
switchport trunk allowed vlan 10,20
no shutdown
!
interface GigabitEthernet0/0
description to SW-CORE1 Gi0/0
switchport trunk encapsulation dot1q
switchport mode trunk
switchport trunk allowed vlan 10,20
no shutdown
!
interface range GigabitEthernet0/2 - 3
description users VLAN 20
switchport mode access
switchport access vlan 20
spanning-tree portfast
no shutdown
!
end
All four switches use the same MST region (name RETICUX, revision 1, instance 1 = VLAN 10 and 20). A different name, revision or VLAN mapping on any switch splits the region.
PC1 to PC4 (VPCS)
PC1> ip 192.168.10.11/24 192.168.10.1
PC2> ip 192.168.10.12/24 192.168.10.1
PC3> ip 172.16.10.11/24 172.16.10.1
PC4> ip 172.16.10.12/24 172.16.10.1
Baseline verification
SW-CORE1
show etherchannel summary
show spanning-tree mst 1
show vrrp brief
show ip ospf neighbor
show ip route 0.0.0.0Expect: Po1 is (SU) with four (P) ports. "This switch is the root" for MST 1. Vlan10 and Vlan20 are Master. Neighbors 10.255.0.21 and 10.255.0.22 are FULL. Default route via 10.10.2.1 (FW1).
SW-CORE2
show lacp interfaces ae0
show vrrp summary
show ospf neighborExpect: All four ae0 members Collecting distributing. Both VRRP groups in backup. FW1 and FW2 neighbors Full.
R1
show ospf neighbor
show route 192.168.10.0/24Expect: 10.10.1.2 (FW1) and 10.10.1.6 (FW2) Full. 192.168.10.0/24 via 10.10.1.2 on ge-0/0/1.0.
FW1
get router info ospf neighbor
get router info routing-table all
diagnose sys session list | grep "SN\|total"Expect: R1 and both cores Full. Default route via 10.10.1.1 on port1. Run the session command on FW2 too: the totals match when FGSP is in sync.
PC1
ping 198.51.100.10
trace 198.51.100.10 -P 1Expect: Replies. Hops: 192.168.10.2 (SW-CORE1) → 10.10.2.1 (FW1) → 10.10.1.1 (R1) → 198.51.100.10.
PC3
trace 198.51.100.10 -P 1Expect: Hops: 172.16.10.2 (SW-CORE1) → 10.10.2.1 (FW1) → 10.10.1.1 (R1) → 198.51.100.10.
Inject the fault
R1
configure
set interfaces ge-0/0/1 unit 0 family inet mtu 1400
deactivate protocols ospf area 0.0.0.0 interface ge-0/0/1.0
commit
activate protocols ospf area 0.0.0.0 interface ge-0/0/1.0
commit and-quitOSPF compares MTUs only during the Database Description exchange, so an adjacency that is already Full survives the MTU change. Deactivating and reactivating OSPF on ge-0/0/1.0 forces a new exchange, which is exactly what the next link flap or reboot would do.
Behavior after the fault
| Device | OSPF state | Routing table | Traffic |
|---|---|---|---|
| R1 | FW1 (10.255.0.21) stuck in ExStart; FW2 Full | User subnets now via 10.10.1.6 (FW2) | Sends all return traffic to FW2 |
| FW1 | R1 stuck in ExStart (some releases show Exchange); both cores Full | Default route now via 10.10.2.2 (SW-CORE1), not R1 | Carries no user traffic |
| FW2 | R1 and both cores Full | Default via 10.10.1.5 (R1) | Carries all user traffic, both directions |
| SW-CORE1 | FW1 and FW2 Full | Default now via 10.10.2.13 (FW2) | Still VRRP master and MST root |
| Users | No change | No change | Ping still works; traceroute hop 2 changes to 10.10.2.13 |
Troubleshoot step by step
Step 1: Notice the path change (PC1)
trace 198.51.100.10 -P 1The ping works, but hop 2 is now 10.10.2.13 (FW2) instead of 10.10.2.1 (FW1). Nothing is down, yet the primary firewall is out of the path.
Step 2: Check the core's default route (SW-CORE1)
show ip route 0.0.0.0
show ip ospf neighborDefault via 10.10.2.13 on Gi1/2. Both firewall neighbors are FULL, so the core is healthy: FW1 simply no longer offers the better default route.
Step 3: Look at FW1's routing table (FW1)
get router info routing-table allFW1's default route points to 10.10.2.2 (SW-CORE1) instead of 10.10.1.1 (R1). FW1 has lost its own path to R1.
Step 4: Check FW1's OSPF neighbors (FW1)
get router info ospf neighborR1 (10.255.0.1) on port1 never leaves ExStart while both cores are Full. The problem is the R1–FW1 adjacency.
Step 5: Confirm on R1 and test the link (R1)
show ospf neighbor
ping 10.10.1.2 count 310.10.1.2 is stuck in ExStart on ge-0/0/1.0, yet the ping succeeds. Layers 1 to 3 work, so this is not cabling or addressing.
Step 6: Rule out duplicate router IDs (R1 and FW1)
R1> show ospf overview
FW1# get router info ospf statusRouter IDs are 10.255.0.1 and 10.255.0.21: unique. The next common cause of a neighbor stuck in ExStart or Exchange is an MTU mismatch.
Step 7: Compare the MTU on both ends (R1 and FW1)
R1> show interfaces ge-0/0/1.0
FW1# get router info ospf interface port1R1 shows Protocol inet, MTU: 1400. FW1 port1 uses the default 1500. The two ends disagree.
Step 8: Find the change (R1)
show configuration interfaces ge-0/0/1
show system commitfamily inet mtu 1400 is present, and the commit history shows when it was added. Decision: roll it back.
Root cause
- OSPF writes the sending interface's IP MTU into every Database Description (DBD) packet.
- RFC 2328 section 10.6: a router rejects a DBD packet whose MTU is larger than it can receive on that interface without fragmentation.
- R1 (1400) rejects every DBD from FW1 (1500), so the database exchange never finishes and the R1–FW1 adjacency never reaches Full. FW1 has the higher router ID and expects to lead the exchange, which is why both sides usually stay in ExStart.
- With no Full adjacency the link drops out of the SPF calculation. OSPF picks the next best path through FW2, and FW1 even learns its default route through SW-CORE1.
- Nothing goes down, so there is no outage and usually no alarm. The redundancy is gone: if FW2 fails now, users lose the Internet.
Why not mtu-ignore? Do not force the adjacency up with mtu-ignore (set mtu-ignore enable on the FortiGate OSPF interface). It only skips the check: IP packets between 1401 and 1500 bytes from FW1 to R1 still fail, and large OSPF updates can be lost. Fix the MTU itself.
Roll back
R1
configure
delete interfaces ge-0/0/1 unit 0 family inet mtu
commit and-quitR1 accepts FW1's next DBD retransmission and the adjacency reaches Full within seconds. OSPF costs then move traffic back to FW1. To return to the last good commit instead, use show | compare rollback 1 to check it, then rollback 1 and commit.
Verify the rollback
R1
show ospf neighbor
show route 192.168.10.0/24
ping 10.10.1.2 size 1472 do-not-fragment count 3Pass: FW1 and FW2 both Full. 192.168.10.0/24 back via 10.10.1.2. A 1500-byte packet (1472 + 28 bytes of headers) with DF set crosses the link.
FW1
get router info routing-table allPass: Default route back via 10.10.1.1 on port1.
SW-CORE1
show ip route 0.0.0.0Pass: Default route back via 10.10.2.1 (FW1).
PC1 and PC3
trace 198.51.100.10 -P 1Pass: Hop 2 is 10.10.2.1 (FW1) again.
Key takeaways
- A neighbor stuck in ExStart is not always an outage. With redundant paths, OSPF reroutes and hides it.
- Watch for path changes: a new traceroute hop or a default route through the wrong neighbor is a clue.
- Neighbor never leaves ExStart or Exchange? Check MTU first, then router IDs.
- OSPF checks MTU only during the DBD exchange, so the fault appears at the next flap, not at the change.
- Fix the MTU, not the symptom, and prove it with a 1500-byte ping with DF set.
References
- RFC 2328, OSPF Version 2 (section 10.6, Receiving Database Description packets)
- RFC 3768, Virtual Router Redundancy Protocol (VRRP)
- Cisco: Troubleshoot OSPF Neighbors Stuck in Exstart/Exchange State
- Cisco: Understand Multiple Spanning Tree Protocol (802.1s)
- Cisco: Configuring VRRP (IOS First Hop Redundancy Protocols guide)
- Juniper: Setting the Protocol MTU
- Juniper: Configuring OSPF Interfaces
- Juniper: Example, Configuring a Conditional Default Route Policy
- Juniper: Example, Configuring VRRP for IPv4
- Juniper: VRRPv3 support in Junos (version 2 is the default)
- Juniper: Routed VLAN interfaces (IRB)
- Juniper: Configuring aggregated Ethernet links (EX Series)
- Juniper: mstp (ELS) configuration statement
- Fortinet CLI reference: config router ospf
- Fortinet: Basic OSPF configuration and MTU settings in OSPF
- Fortinet: OSPF neighbour stuck in EXSTART/EXCHANGE state
- Fortinet: FGSP configuration changes as of FortiOS 7.2 and 7.4
- Fortinet: Suggested parameters for an FGSP cluster
Download the full lab PDF and watch the 30-second traffic-flow video on the RETICUX Youtube.

No comments:
Post a Comment