Azure Expressroute Q-in-Q Translation

Preface

Azure Expressroute uses 2 VLAN tags - S-Tag and C-Tag. S-Tag is allocated by Azure, and C-Tag is managed by Expressroute Provider. Usually S-Tag is unique between Expressroute Circuits.

But if provider have multiple NNIs, there are chance of duplication because S-Tag allocation is managed for each NNIs. For example, if circuit A got S-Tag 10 through NNI-A, circuit B on NNI-B have a chance of getting S-Tag 10 too.

Because of this, Azure requires L2 virtualization features such as VDC on Cisco Nexus to accept S-Tag duplication without problem. And if your infrastructure does not support it, you have to get another pair of aggregation switches for each NNI pair. This is cumbersome and expensive.

VLAN mapping

To mitigate this, you can utilize VLAN mapping feature. I have done this with Cisco Catalyst and many of you might use same, therefore I am going to explain based on Cisco Catalyst.

Cisco Catalyst switch supports "VLAN Mapping" feature, but not all models provides every subfeature of it. For example, C9500-40X only supports 1:1 mapping while C9500-48Y4C supports all mapping featue although both are all in same Catalyst 9500 family.

1:1 Mapping

1:1 mapping is simple as it's name. It translates one VLAN number into another and vice versa. But you should keep in mind that it only changes outermost tag only. For example, if your packet is triple tagged, only 3rd tag will be affected by 1:1 mapping.

Q-in-Q

This is your traditional double tagging. This will add or pop outer tag with your single tagged packet. But there's a catch. Q-in-Q setting doesn't care about inner tag. That means, all packets comming through trunk will get same outer tag no matter what inner tag is.

Selective Q-in-Q

Selective Q-in-Q is just like Q-in-Q but selective. You can instruct switch to add or pop certain outer tag for choosen inner tag. Therefore you can maintain multiple outer tag through single trunk port. Plus, you can use normal Q-in-Q setting along with Selective Q-in-Q for default behaviour.

Then how to?

Now let's get back to our initial problem. When there's multiple NNIs, S-Tag(outer tag) might be get duplicated. And our L2 switch will get confused with same S-Tag is allowed in more than one trunk.

But here's the catch.

VLAN mapping is processed when packet is arrived or leaving interface. That means if mapping is configured on the port, packet's VLAN that recognized by switching processor is already translated one. Therefore, if we set VLAN mapping that translates duplicated S-Tag into discrete translated VLAN, our aggregation switch won't be confused.

This can be aplied to return packets too. Since two packets are different VLAN for the our aggregation switch, it will push packets through corresponding trunks. But when it leaves interface, VLAN ID is swapped with it's original duplicated S-Tag. However, since now they are travling through different trunks and processed by different Azure edge switch, duplicated S-Tag can be processed properly via Azure network.

Deatiled instruction

Now let's get to the real work. In this section, I am going to use topology depicted below as a example and it might be similar as your network.

On the interface of aggregation switch towards Azure side (ER-L2 eth0, eth1), make it trunk port and config 1:1 mapping. VLAN statement order should be Original VLAN then Translated VLAN

                
                ER-L2(config)# vlan 1010
                ER-L2(config)# vlan 2010
                ER-L2(config)# interface eth0
                ER-L2(config)# switchport mode trunk
                ER-L2(config)# switchport vlan mapping 10 1010
                ER-L2(config)# interface eth1
                ER-L2(config)# switchport mode trunk
                ER-L2(config)# switchport vlan mapping 10 2010
                
            

Now interface towards provider edge(PE) side(Eth2~3). Usually you have to configure .1Q tunnel to add S-Tag on the packet tagged with C-Tag from provider edge. But in this case, you have to add translated S-Tag not the original one on top of C-Tag.

                
                ER-L2(config)# interface eth2
                ER-L2(config)# switchport access vlan 1010
                ER-L2(config)# switchport mode dot1q-tunnel
                ER-L2(config)# interface eth3
                ER-L2(config)# switchport access vlan 2010
                ER-L2(config)# switchport mode dot1q-tunnel
                
            

If you're using selective Q-in-Q to aggregate PE side links, you have to use translated S-Tag in the mapping command.

                
                ER-L2(config)# interface eth2
                ER-L2(config)# switchport mode trunk
                ER-L2(config)# switchport vlan mapping 2000 dot1q-tunnel 1010
                ER-L2(config)# switchport vlan mapping 3000 dot1q-tunnel 2010