Network & Security VMUG Community

Expand all | Collapse all

ARP timeout configuration in T1 gateways

  • 1.  ARP timeout configuration in T1 gateways

    Posted 06-26-2020 12:05 PM
    Hi all there,

    I just wanted to try and see if anyone out there has gone through a similar case and how it has overcome it (by the way, my environment runs in NSX-T 2.5.0)

    My main question is, does anybody knows if there is a way to change the default ARP timer of a T1 gateway? I mean, if I check the ARP table of a T1 gateway, this is what I see:

    rkalvbvedgediarnd01(vrf)> get neighbor
    Logical Router
    UUID        : e98fd954-3a03-435a-a74f-2f4e21493f3e
    VRF         : 18
    LR-ID       : 12
    Name        : DR-T1-SISWFP-1
    Type        : DISTRIBUTED_ROUTER_TIER1
    Neighbor
        Interface   : 1894a777-fc06-4d91-b63c-0b49b06b4bc2
        IP          : fe80::50:56ff:fe56:4452
        MAC         : 02:50:56:56:44:52
        State       : perm
    
        Interface   : 1894a777-fc06-4d91-b63c-0b49b06b4bc2
        IP          : fc06:817a:33a4:6001::1
        MAC         : 02:50:56:56:44:52
        State       : perm
    
        Interface   : 0550b587-43ee-4640-a29b-971a2b8aa1af
        IP          : 10.145.148.55
        MAC         : 00:50:56:8a:fb:fd
        State       : reach
        Timeout     : 1117
    
        Interface   : 1894a777-fc06-4d91-b63c-0b49b06b4bc2
        IP          : 100.64.192.2
        MAC         : 02:50:56:56:44:52
        State       : perm
    
    rkalvbvedgediarnd01(vrf)>
    ​

    As you can see, there is an ARP entry for a VM in the segment with IP 10.145.148.55, and a timer running at 1117 seconds

    I can't find any possible way of changing that value. I know how to change the IP discovery values related to Logical segments, but nothing for T1 gateways.

    The thing is that my customer needs to be capable to create and destroy VMs (they use Terraform) quicly. They use static IPs, and an Ubuntu cloud-init image which changes the MAC address every time they re-deploy the VM, so most of the times they need to way in the range of 10-20 minutes for the external connectivity to get restored... basically they need the ARP timer to expire to capture the new ARP map.

    I know that this is the way it works ARP, but I wonder that if it is possible to re-configure IP discovery for a Logical Segment... should it be also for the T1 gateway shouldn't be?... otherwise what is the point? (actually, my initial expectation was that VMtools notification to NSX-T would automatically update all tables, not only the Logical Segments ones)

    I can't find any command or API call or document reference, so just trying here in case anyone can shed some light.

    Huge thanks in advance


    ------------------------------
    Ruben Tripiana
    Network Engineer
    Hoffman-La Roche, Ltd
    Madrid
    ------------------------------


  • 2.  RE: ARP timeout configuration in T1 gateways

    Posted 06-29-2020 12:42 PM
    Are you trying to modify the T0-T1 timeouts? Those run BFD so there shouldn't be a whole lot to worry about there...

    IP Discovery is not globally configured on the T1, including ARP. This should be done on the logical segment.

    TBH, it sounds like there's something wrong with your T1s - those should update with VMTools as soon as the new device comes online.

    Since it's per segment, I'd really recommend trying to disable ARP and DHCP snooping first. If that resolves the issue, you can re-enable and set "ARP Binding Limit" to 2 or more. It may just be the ARP inspection timeout that's giving you troubles.

    If not, I'd recommend raising that to support - that is not normal behavior.

    ------------------------------
    Nicholas Schmidt
    Engineer
    AK
    ------------------------------



  • 3.  RE: ARP timeout configuration in T1 gateways

    Posted 06-29-2020 02:24 PM
    Thanks for the hints Nicholas!

    I already tried to disable ARP Snooping on the Segment's profiles, but no joy. I didn't try with DHCPSnooping, but because in my scenario there is no DHCP involved.

    What I'm looking for is not to modify all timers for all gateways, but the ARP timer specifically, and for specific T1. As you said, the behavior looks weird to me, but it is true that all references for IP discovery point to the segment, and not to the T1, which has its own tables.... as you can see from the "get neighbor" command (which is another big mystery, as this command is barely detailed in the documentation as far as I could find).

    I'll probably do what you say and open a support ticket to try sort this out

    thanks!
    Ruben T

    ------------------------------
    Ruben Tripiana
    Engineer
    Hoffman-La Roche, Ltd
    Madrid
    ------------------------------