ACI knobs: APIC Connectivity Preferences

One thing that bugs me lately, that there are configuration knobs and the vendor does not explain what happens under the hood when chaning these knobs. One example is the ACI “APIC Connectivity Preferences” setting.

GUI: System > APIC Connectivity Preferences

CLI:

mgmt-connectivity-pref { ooband | inband }

A litte background: The APIC controller has two 1G interfaces of out-of-band management (oob). The two 10G fabric facing interfaces may be used for management purposes as well (using a VLAN).

So we have a box connected to two networks with a default route pointing to each network. Network devices typically solve this by using a dedicated management VRF for the out-of-band management interface.

The APIC solves this differently as verified in the APIC controller (version 3.2x) bash shell:

Preference set to inband (interface bond0.<VLAN-ID>)

admin@apic1:~> route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.0.2.254     0.0.0.0         UG    8      0        0 bond0.<VLAN-ID>
0.0.0.0         198.51.100.254  0.0.0.0         UG    16     0        0 oobmgmt

Preference set to out-of-band (interface oobmgmt)

admin@apic1:~> route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         198.51.100.254  0.0.0.0         UG    16     0        0 oobmgmt
0.0.0.0         192.0.2.254     0.0.0.0         UG    32     0        0 bond0.<VLAN-ID>

The metric for the default route is changing depending on the setting. The lower route metric “wins”.

Note: If you test this at home, it takes a few seconds after the change of the setting (approx 60) until the routing table is updated on the APIC.