Understanding MACsec Replay Protection: Window Sizing for Real-World Deployments

MACsec (IEEE 802.1AE) is a Layer 2 hop-by-hop encryption standard that protects Ethernet frames traveling between two directly connected or bridged MACsec-capable devices, providing confidentiality, integrity, data-origin authenticity, and replay protection. Every frame protected by MACsec carries a Packet Number (PN) in its SecTAG header, a value that increments by one for each frame transmitted under a given Secure Association Key (SAK). On the receiving side, MACsec does not just decrypt and verify integrity — it also tracks these packet numbers to detect and drop frames that appear to be duplicates or replays of previously seen traffic.[1][2][3][4][^5]

Replay protection works through a sliding "replay window." The receiver maintains a lowest acceptable PN, calculated as the highest PN it has successfully validated so far, minus the configured replay window size. Any incoming frame with a PN below that threshold is discarded and counted as a late/replayed packet, while frames with a PN inside the window are accepted, even if they arrive out of strict sequence. If the window is set to zero MACsec enforces strict, in-order reception — the receiver accepts only PNs greater than the last one seen, which is the tightest possible replay protection but also the least tolerant of any legitimate reordering.[2][4][6][5]

MACsec's threat model explicitly assumes an active, on-path or off-path attacker who can copy, modify, remove, or inject frames onto the wire. Without replay protection, an attacker who intercepts a single valid encrypted MACsec frame could capture it and retransmit it into the network repeatedly. Because the frame is still cryptographically valid (correct integrity check value and encryption), a receiver that only checked confidentiality and integrity — but not sequence — would accept the duplicate as legitimate traffic every time. Depending on what that frame represents, this could be used to flood the receiving network, desynchronize higher-layer protocols that assume each message is delivered once, or otherwise manipulate protocol behavior through unintended repeated delivery. Replay protection closes that specific gap: it ensures that even a perfectly captured, byte-for-byte valid frame cannot be endlessly or arbitrarily reinjected once its packet number falls outside the acceptable window.[7][8][3][5][^2]

In real deployments, the replay window is rarely left at its most restrictive setting unless the link is a simple, single, well-ordered point-to-point connection. Metro Ethernet and provider networks frequently reorder frames because of QoS-based priority queuing, ECMP or LAG load balancing across multiple parallel physical links, and per-hop queuing variance — none of which are malicious, but all of which can cause legitimately transmitted frames to arrive out of packet-number order. If replay protection were strictly enforced with a zero-size window over such a network, MACsec would start discarding valid, non-replayed frames purely because of ordinary jitter, which breaks the data flow.[4][9]

To accommodate this, vendors expose a configurable replay-window size, typically ranging from 0 up to 2^32−1 packets for standard 32-bit packet numbering ciphers (GCM-AES-128/256), with some implementations supporting even larger ranges. Network operators size this window based on how much legitimate reordering their specific link or provider network is expected to introduce — large enough to avoid false-positive drops of valid traffic, but no larger than necessary, since every additional packet number of tolerance is also additional room for a genuine replay to slip through undetected.

The IEEE 802.1AEbw amendment later added Extended Packet Numbering (XPN) ciphers, GCM-AES-XPN-128/256, which extend the packet number from 32 to 64 bits specifically to support very high-speed links (100 Gbps and above), because at those speeds a 32-bit counter exhausts quickly enough to force disruptively frequent key (SAK) rollovers.[10][8][5][9][^11]

This is the crux of the security trade-off, and it is easy to misjudge if you look only at the raw packet-count size of the replay window rather than the real time it represents. The replay window is not a fixed span of time — it's a fixed count of packet numbers. How much real-world time that count corresponds to depends entirely on how fast packets are actually flowing across the wire, which is a function of line rate and average frame size.[1][10]

The relevant math is straightforward. Packets-per-second (pps) is determined by line rate divided by total on-wire frame size (payload plus Ethernet header, FCS, any VLAN tags, MACsec SecTAG/ICV overhead, and preamble/interpacket gap). The replay window's real-time "exposure" — how long an attacker's captured frame remains theoretically replayable before its packet number falls below the lowest acceptable PN — is simply:

As line rate increases, pps increases proportionally, which means the identical packet-number window represents progressively less real time. A replay window of 65,536 packets might represent nearly a full second of traffic on a lightly loaded 1 Gbps link, but only a few hundred microseconds of traffic on a heavily loaded 400 Gbps link. The window size configured on the device hasn't changed — but the actual opportunity an attacker has to capture a frame, decide to reuse it, and successfully re-inject it back onto the wire before that packet number ages out of the window has shrunk dramatically.[11][10]

The table below illustrates how dramatically the same replay window's real-time exposure shrinks as line rate increases, shown for both small (64-byte) and large (1518-byte) frames, which represent the low and high ends of typical Ethernet frame sizes:

These figures are derived using standard MACsec/Ethernet framing overhead (14-byte Ethernet header, 4-byte FCS, MACsec SecTAG and ICV, and preamble/interpacket gap) applied consistently across each line rate.[11][1]

Feasibility comes down to whether the exposure window is long enough for a realistic attacker workflow — intercept the frame off the wire, buffer it, decide it's worth replaying, and physically re-inject it — to complete before the packet number falls out of range.

1 Gbps: This is the slowest and most forgiving speed for an attacker. With larger frames and a generously sized replay window, exposure can stretch into the hundreds of milliseconds or even multiple seconds, which is comfortably within the reach of commodity packet-capture and replay tooling. A replay attempt is technically much more feasible here than at higher speeds, though it still requires physical or logical access to the link and the ability to intercept and re-inject frames without disruption.[3][2]

10 Gbps: Exposure windows shrink by roughly an order of magnitude compared to 1 Gbps for the same packet-count window, generally into the tens of milliseconds range for large windows. This is still plausibly actionable with purpose-built high-speed capture/replay hardware, though the margin for error narrows considerably compared to 1 Gbps.[9][10]

100 Gbps: At this tier, exposure for a well-sized replay window typically falls into the single-digit-milliseconds range or less, even for fairly large window configurations. This is why the IEEE mandated XPN ciphers for 100+ Gbps MACsec deployments — not because of the replay window directly, but because standard 32-bit packet numbering exhausts too quickly at this pps to keep the SAK stable, which itself is a related but distinct high-speed challenge. A successful replay at this rate would require extremely specialized, high-throughput interception and injection capability operating with sub-millisecond precision.[10][11]

400 Gbps: This represents the narrowest practical attack surface by a wide margin. Even a comparatively large replay window can correspond to only a few milliseconds — or fractions of a millisecond for smaller windows — of real exposure time. Successfully capturing, evaluating, and re-injecting a frame within that margin using real-world hardware (which introduces its own processing, buffering, and transmission latency) is at or beyond the edge of what is practically achievable with current off-the-shelf equipment, making a genuine replay attack at 400 Gbps a largely theoretical concern under normal operating conditions.[10][11]

The replay window is a deliberate trade-off between operational tolerance and security exposure: too small, and normal network jitter or failover events cause false-positive drops of legitimate traffic; too large, and the real-time opportunity for a captured frame to be successfully replayed grows. Because packets-per-second scales directly with line rate, the same configured window size becomes progressively less exploitable as networks move from 1 Gbps toward 400 Gbps — which is exactly why replay window sizing should always be evaluated in terms of real-time exposure at the actual physical line rate, not just as an abstract packet count.[8][5][4][9][1][10]

The following link will take you to a calculator where you can derive a recommended replay window size based on various factors such as link speeds, frame sizes & headers, RTT, Jitter, and RTT variance between two paths.

MACsec Replay Window Sizing Calculator

This link will take you to a guide that provides in-depth explanation of the various fields and options of the calculator. A “Help & Usage Guide” so to speak.

MACsec Calculator User Guide

References

1. 802.1AE: MAC Security (MACsec) - This standard specifies provision of connectionless user data confidentiality, data integrity, data ...
2. [PDF] MACsec Feature Overview and Configuration Guide - Allied Telesis
3. MACsec 802.1ae - Media Access Control Security (IEEE 802.1ae ...
4. MACsec Encryption
5. 802.1AE-2006 - MAC security (MACsec) provides connectionless user data confidentiality, frame data integrity, and d...
6. df-fedyk-ieee802-dot1ae-0719-v00.yang Page 1
7. [PDF] Preemption and MACsec replay protection - IEEE 802 LMSC - MACsec assumes that an attacker can copy, modify, remove, and add frames at will. 3. MACsec goals. I...
8. MACsec_preemption_proposal
9. MACSEC and MKA Configuration Guide, Cisco IOS XE Release 3S - MACSEC and MKA Configuration Guide, Cisco IOS XE Release 3S
10. Securing High Speed Private Link to Public Cloud Providers
11. 802.1AEbw-2013: MAC Security (MACsec)–Extended Packet Numbering - IEEE 802.1AEbw-2013