ACI Data Plane Policing (DPP): A Deep Dive on L3Out Bandwidth Control

1. Overview and Core Concepts

Data Plane Policing (DPP) is a crucial ACI feature used to manage and restrict bandwidth consumption on specific fabric access interfaces, ensuring efficient and controlled use of network resources. This blog uses an ACI fabric running 5.2(7f) and focuses on Data Plane Policing of l3Out interfaces.

Action on Excess Traffic:

  • Dropping: Packets exceeding the configured rate are discarded (policing).
  • Marking: QoS fields (DSCP/CoS) in packets are flagged as “exceeding,” allowing downstream devices to prioritize dropping them only if congestion occurs (traffic shaping).

Traffic Applicability:

  • Direction: Both ingress (incoming) and egress (outgoing) traffic can be policed.
  • Layer: Works on L2 (data link) and L3 (network) interfaces.

Configuration Options:

  • Fabric Access Policies: Policies applied to physical or virtual interfaces connecting devices to the ACI fabric.
  • Tenant Policies: Policies applied to Endpoint Groups (EPGs), controlling traffic for groups of servers or applications and boarder leaf access ports (l3extOut or l2extOut).

Policer Sharing Modes (Granularity vs. Efficiency)

ModeGranularityResource UseBest Practice
Shared ModeLow. A single hardware policer enforces the policy for all entities (interfaces or EPG members) referencing it.Efficient. Saves hardware resources, ideal for large deployments with similar, low-bandwidth needs.Use when all members have the same general traffic requirement and a single burst won’t hurt the collective.
Dedicated ModeHigh. Each entity (e.g., L3Out interface) gets its own dedicated policer.Less efficient. Consumes a separate hardware policer per member.Use when strict, independent bandwidth control for specific interfaces is required, or when a member’s burst needs to be isolated.

Choosing the Right Mode:

The choice between shared and dedicated mode depends on your specific needs:

  • Use shared mode for deployments with many low-bandwidth policers and similar traffic patterns.
  • Use dedicated mode for scenarios requiring strict control over individual member bandwidth or when specific members might burst and impact others.

https://www.cisco.com/c/en/us/td/docs/dcn/aci/apic/5x/security-configuration/cisco-apic-security-configuration-guide-release-52x/data-plane-policing-52x.html

2. Configuration

Figure 1 – L3out Topology for DPP Demo

Figure 2 – DPP Policy

Choose or input the following information on GUI:

  • Name: The name for policy. Use the object naming standard of your organization.
  • Administrative Status: Enabled
  • Domain Policer mode and type: Bit policier vs. Packet Policier & 1 Rate 2 Color vs. 2 Rate 3 Color

    Policer Types: 1R2C vs. 2R3C

    DPP uses color-based policing to classify traffic:

    • 1 Rate 2 Color (1R2C): Uses Committed Information Rate (CIR) to classify traffic into two colors: Conforming (Green) or Violating (Red).
    • 2 Rate 3 Color (2R3C): Uses both CIR and Peak Information Rate (PIR). Traffic can be Conforming (Green), Exceeding (Yellow) (between CIR and PIR), or Violating (Red) (above PIR).
    • Burst size: Maximum allowed traffic exceeding the rate limit for a short duration.
    • Rate: Committed Information Rate (CIR) – the sustained bandwidth allowed.
    • Peak Rate (optional): Peak information Rate (PIR) is allowed bandwidth for short bursts.
    • Conform Action: Specify whether to drop or mark or transmit packets conforming the policy.
    • Violate Action: Specify whether to drop or mark packets exceeding the limit.
    • Sharing Mode: Dedicated Policier vs. Shared Policier

    3. Real-World Use Case: L3Out Rate Limiting

    Configure DPP on an L3Out interface to limit external traffic destined for a specific external service.

    Objective: Apply a dedicated DPP policy on the L3Out Logical Interface Profile to restrict ingress (incoming) and egress (outgoing) traffic to a specific maximum bandwidth (example: 500 Mbps).

    • Scenario: You need to enforce a Service Level Agreement (SLA) with a customer or control the bandwidth consumed by a backup process connecting externally.

    4. Configuration Steps (APIC GUI)

    Pre-Requisites: The Tenant, VRF, and external L3Out are already configured.

    Step 4.1: Create Data Plane Policing Policy

    Navigate to Tenant -> Policies -> Protocol -> Data Plane Policing.

    FieldValueNotes / Best Practice
    NameL3out1-Core-DPPUse a descriptive naming convention for operational clarity.
    Administrative StatusEnabled
    DomainPolicer Mode: Bit PolicerBit Policer for bandwidth control; Packet Policer for packet-per-second control.
    Type1 Rate 2 ColorStandard rate limiting using only the CIR.
    Rate (CIR)500 MbpsThe sustained bandwidth limit.
    Burst Size25 MB (approx. 40ms at 500Mbps)This is the allowed burst allowance. A good rule of thumb is 10ms-50ms of traffic at CIR.
    Sharing ModeDedicated PolicerEnsures this policy is enforced independently for the L3Out without affecting other interfaces.
    Conform ActionTransmitAllows conforming traffic to pass.
    Violate ActionDropViolating traffic is discarded. (Use Mark if you prefer preferential drop instead of hard drop).

    Step 4.2: Assign DPP Policy to L3Out Interface

    The policy must be applied to the relevant L3 interface profile that peers externally.

    Navigate to Tenant -> Networking -> L3Outs -> L3Out Name -> Logical Node Profiles -> Logical Node Profile Name -> Logical Interfaces Profiles.

    Attach the newly created L3out1-Core-DPP policy to the desired L3 Interface Profile.

    Figure 3 – Assign DPP policy to L3out Logical Interface Profile

    5. Verification and Troubleshooting (NX-OS CLI)

    Verification is crucial to confirm the policy is correctly installed in hardware and is actively processing traffic.

    CommandPurposeKey Output to Check
    show policy-map type data-planeView the policy settings on the APIC/fabric-wide.Confirm Rate (CIR), Burst, Sharing Mode, and the Violate Action are correct.
    show dpp policyView the operational status and parameters on the Leaf switch itself.Confirm the Interface name, Direction (ingress/egress), and Oper State (enabled).
    vsh_lc -c "show sys int aclqos dpp"View the hardware counters on the line card (LC).Check the RedPkts and RedBytes counters for both ingress and egress directions. A non-zero value here means the policy is actively dropping traffic.

    1. show running-config

    APIC1# sh running-config tenant B-SRMPLS policy-map type data-plane L3out1-Core-DPP  
    # Command: show running-config tenant B-SRMPLS policy-map type data-plane L3out1-Core-DPP
    # Time: Mon Mar  4 14:30:33 2024
      tenant B-SRMPLS
        policy-map type data-plane L3out1-Core-DPP
          set burst 750 mega
          set cir 500 mega
          set sharing-mode shared
          no shutdown
          exit
        exit
    

    2. show policy-map type data-plane

    APIC1# show policy-map type data-plane 
    Type data-plane policy-maps
    ====================
    Policy in Tenant: B-SRMPLS
    policy-map type data-plane L3out1-Core-DPP
        set burst 750 mega
        set conform-cos-transmit unspecified
        set conform-dscp-transmit unspecified
        set conform transmit
        set excessive-burst unspecified
        set exceed-cos-transmit unspecified
        set exceed-dscp-transmit unspecified
        set exceed drop
        set mode bit
        set pir 0
        set cir 500 mega
        set type 1R2C
        set violate-cos-transmit unspecified
        set violate-dscp-transmit unspecified
        set violate drop
    

    3. show dpp policy

    LEAF-101# show dpp policy
    Data Plane Policers
    
    Policer Name    : B-SRMPLS:L3out1-Core-DPP
    Interface       : vlan69              
    Direction       : egress              
    Layer           : Layer3              
    Oper State      : enabled             
    Mode            : bit                 
    Type            : 1R2C                
    Rate            : 500 mega            
    Burst           : 750 mega            
    SharingMode     : shared              
    Conform action  : transmit            
    Violate action  : drop                
    
    Policer Name    : B-SRMPLS:L3out1-Core-DPP
    Interface       : vlan69              
    Direction       : ingress             
    Layer           : Layer3              
    Oper State      : enabled             
    Mode            : bit                 
    Type            : 1R2C                
    Rate            : 500 mega            
    Burst           : 750 mega            
    SharingMode     : shared              
    Conform action  : transmit            
    Violate action  : drop               
    

    4. show sys int aclqos dpp

    LEAF-101# vsh_lc -c "show sys int aclqos dpp"
    -----------------------------------------------------------
                 DPP: INGRESS POLICER ENTRIES                  
    -----------------------------------------------------------
    Name            Interface       ACL-Id      Pol-SwId Pol-HwId Label       Mask        Rate        Burst       GreenPkts   GreenBytes  RedPkts     RedBytes            
    B-SRMPLS:L3out1-Core-DPP Vlan69  1112        47       513      0x10000     0x38000  524288000   786432000   0    34270        0           0          
    
    -----------------------------------------------------------
                 DPP: EGRESS  POLICER ENTRIES                  
    -----------------------------------------------------------
    Name            Interface       ACL-Id      Pol-SwId Pol-HwId Label       Mask        Rate        Burst       GreenPkts   GreenBytes  RedPkts     RedBytes          
    B-SRMPLS:L3out1-Core-DPP Vlan69  1110        5        4        0x4         0xe      524288000   786432000   0   136916       0           0          
    
    

    Leave a Comment

    Your email address will not be published. Required fields are marked *