Pseudowire Headend Termination (PWHT) For Juniper MX

I’ve been doing quite a lot of MX BNG stuff this year, so I thought I’d run through another quite flexible way of terminating broadband subscribers onto a Juniper MX router.

The feature is called Psuedowire headend termination, “PWHT” or simply Psuedowire head-end “PWHE” depending on whether you work for Cisco or Juniper 😉 but it essentially solves a relatively simple problem.

In traditional broadband designs – especially in DSL “FTTC” or Fibre Ethernet “FTTP” we’re used to seeing large numbers of subscribers, connecting into the ISP edge at layer-2 with PPPoE or plain Ethernet. This is normally performed with VLANS, either via an MSAN (DSL/FTTC) or as is the case with Ethernet FTTP subscribers – a plain switched infrastructure or some form of passive-optical (PON/GPON) presentation:

 

Capture

These subscribers then terminate on a BNG node on the edge of the network, which would historically have been a Cisco 7200, GSR10k, Juniper ERX or Redback router, which essentially bridges the gap between the access network and the internet.

For very large service providers with millions of subscribers this sort of approach normally works well, because their customer base is so large; it makes sense for them to provision a full-size BNG node in every town in the country and so subscribers terminate directly at the edge of the network.

However – modern BNG can be expensive. In order to provide the required throughput and features, (IPv4 / IPv6 / VPN / Quad play / QoS) it requires a significant investment in router chassis, fancy line cards and expensive licenses, at every point in the network where BNG is to be performed. For smaller ISPs this can be a deal breaker – especially if they have small chunks of subscribers dotted around.

One way of getting around this problem is to provision a centralised BNG deployment, where that function is performed somewhere centrally inside the service-provider network. Edge connectivity (PPPoE / Ethernet VLAN) is tunnelled directly from the access network, through an intermediate edge-router (U-PE) and onto a centralised BNG node where it terminates, allowing for the ISP to service a large number of subscribers from many different remote areas – using a single BNG function:

Capture2

Essentially, in the above topology – the “U-PE” or access facing PE is running a standard EoMPLS LDP signalled “martini tunnel” back towards the centralised BNG router, buried deep inside the core somewhere.

The U-PE itself can be a cheaper, standard edge router, so long as it supports MPLS and LDP signalled EoMPLS tunnels – these can be provisioned anywhere on the network edge, whilst providing direct connectivity back to the BNG node at layer-2 – all this is done using PWHT (Pseudowire headend termination)

On Juniper – PWHT as a feature came into existence on, or around JUNOS version 13.1, before then there was a relatively simple “hack” that had to be performed, in order to provide the functionality. It basically involved the good ole trick of using physical loopback cables on the same device, in order to “make it work” as shown below:

Capture3

This is a pretty heavy handed approach and also quite expensive – as it involves burning up expensive ports on the router, simply to bridge the gap between the access network and the subscriber termination interface.

With the PWHT feature a new type of interface is defined, known as the psuedowire service interface “PS” this is bound to a tunnel-services PIC, which essentially performs the heavy lifting.

Looking at this in a lab, I have the following topology setup on an MX480 containing a MPC2E-Q and a 4x10GE MIC:

Capture4

If we look at the configuration, there’s a few things we need to do – lets check out the PS interface and the l2circuit configuration:

  1. tim@MX480-3> show configuration chassis
  2. pseudowire-service {
  3.     device-count 2048;
  4. }
  5. fpc 1 {
  6.     pic 0 {
  7.         tunnel-services {
  8.             bandwidth 10g;
  9.         }
  10.     }
  11. }
  12. network-services enhanced-ip;
  13. tim@MX480-3>

 

The command “pseudowire-service” basically enables the PWHT feature, and an MX chassis supports a total of 2048 pseudowire-service interfaces – each interface is bound to a l2circuit that points back to a “U-PE” edge device, that provides more than enough for most deployments,

It’s also necessary to enable tunnel-services, then when we take look at the “PS” interface, it’s easy to see how this fits together:

  1. ps0 {
  2.     anchor-point {
  3.         lt-1/0/0;
  4.     }
  5.     flexible-vlan-tagging;
  6.     auto-configure {
  7.         stacked-vlan-ranges {
  8.             dynamic-profile vlan-prof-0 {
  9.                 accept [ inet pppoe ];
  10.                 ranges {
  11.                     10-100,100-4000;
  12.                 }
  13.                 access-profile aaa-profile;
  14.             }
  15.         }
  16.         remove-when-no-subscribers;
  17.     }
  18.     mtu 1530;
  19.     unit 0 {
  20.         encapsulation ethernet-ccc;
  21.     }
  22. }

 

The anchor-point statement basically binds the logical-tunnel interface directly to the PS interface, so that the “heavy lifting” can be done by the MIC, unit 0 binds directly to the l2circuit configuration – which creates the EoMPLS connectivity to the U-PE:

  1. tim@MX480-3> show configuration protocols l2circuit
  2. neighbor 1.1.1.2 {
  3.     interface ps0.0 {
  4.         virtual-circuit-id 10;
  5.         no-vlan-id-validate;
  6.     }
  7. }

 

Essentially we have a standard l2circuit configuration pointing at the U-PE (the U-PE simply has a reciprocal configuration bound to it’s physical access-facing interface. Because this psuedowire will be carrying multiple VLANs (S-VLAN and C-VLAN) we don’t want to consider that information when the psuedowire is signalled, so “no-vlan-id-validate” command takes care of this.

Lets take a look at the wider BNG configuration for completeness:

  1. dynamic-profiles {
  2.     vlan-prof-0 {
  3.         interfaces {
  4.             “$junos-interface-ifd-name” {
  5.                 unit “$junos-interface-unit” {
  6.                     no-traps;
  7.                     vlan-tags outer “$junos-stacked-vlan-id” inner “$junos-vlan-id”;
  8.                     family inet {
  9.                         unnumbered-address lo0.0;
  10.                     }
  11.                     family pppoe {
  12.                         dynamic-profile pppoe-client-profile;
  13.                     }
  14.                 }
  15.             }
  16.         }
  17.     }
  18.     pppoe-client-profile {
  19.         interfaces {
  20.             pp0 {
  21.                 unit “$junos-interface-unit” {
  22.                     no-traps;
  23.                     ppp-options {
  24.                         chap;
  25.                     }
  26.                     pppoe-options {
  27.                         underlying-interface “$junos-underlying-interface”;
  28.                         server;
  29.                     }
  30.                     keepalives interval 30;
  31.                     family inet {
  32.                         unnumbered-address lo0.0;
  33.                     }
  34.                 }
  35.             }
  36.         }
  37.     }
  38. }
  39. access {
  40.     radius-server {
  41.         192.168.3.158 {
  42.             port 1812;
  43.             accounting-port 1813;
  44.             secret “xxx”; ## SECRET-DATA
  45.             timeout 10;
  46.             retry 10;
  47.             source-address 192.168.3.54;
  48.         }
  49.     }
  50.     profile aaa-profile {
  51.         authentication-order radius;
  52.         radius {
  53.             authentication-server 192.168.3.158;
  54.             accounting-server 192.168.3.158;
  55.             options {
  56.                 interface-description-format {
  57.                     exclude-sub-interface;
  58.                 }
  59.                 nas-identifier mx5-1;
  60.                 accounting-session-id-format decimal;
  61.                 vlan-nas-port-stacked-format;
  62.             }
  63.         }
  64.         radius-server {
  65.             192.168.3.158 {
  66.                 port 1812;
  67.                 accounting-port 1813;
  68.                 secret “xxx”; ## SECRET-DATA
  69.                 timeout 10;
  70.                 retry 10;
  71.                 source-address 192.168.3.54;
  72.             }
  73.         }
  74.         accounting {
  75.             order radius;
  76.             accounting-stop-on-failure;
  77.             accounting-stop-on-access-deny;
  78.             immediate-update;
  79.             coa-immediate-update;
  80.             update-interval 60;
  81.             statistics volume-time;
  82.         }
  83.     }
  84.     profile no-radius-auth {
  85.         authentication-order none;
  86.     }
  87.     address-assignment {
  88.         pool Subscriber-pool {
  89.             family inet {
  90.                 network 130.0.0.0/8;
  91.                 range Sub-range-0 {
  92.                     low 130.16.0.1;
  93.                     high 130.31.255.255;
  94.                 }
  95.                 dhcp-attributes {
  96.                     maximum-lease-time 25200;
  97.                 }
  98.             }
  99.         }
  100.     }
  101. }

 

That’s the basic configuration, lets fire up some subscribers and see what it looks like – I’m using IXIA to generate PPPoE simulated clients, we’ll start with a single double-tagged subscriber, (The S-VLAN normally represents the MSAN, the C-VLAN normally represents the subscriber’s own VLAN)

Capture

Lets check the outputs from the MX BNG:

  1. tim@MX480-3> show subscribers
  2. Interface           IP Address/VLAN ID                      User Name                      LS:RI
  3. ps0.1073761693      10 111                                                            default:default
  4. pp0.1073761694      130.16.0.3                              user1@users.com           default:default
  5. tim@MX480-3> show subscribers detail
  6. Type: VLAN
  7. Logical System: default
  8. Routing Instance: default
  9. Interface: ps0.1073761693
  10. Interface type: Dynamic
  11. Underlying Interface: ps0
  12. Dynamic Profile Name: vlan-prof-0
  13. State: Active
  14. Session ID: 19870
  15. Stacked VLAN Id: 10
  16. VLAN Id: 111
  17. Login Time: 2016-07-10 12:34:46 UTC
  18. Type: PPPoE
  19. User Name: user1@users.com
  20. IP Address: 130.16.0.3
  21. IP Netmask: 255.255.255.255
  22. Logical System: default
  23. Routing Instance: default
  24. Interface: pp0.1073761694
  25. Interface type: Dynamic
  26. Underlying Interface: ps0.1073761693
  27. Dynamic Profile Name: pppoe-client-profile
  28. MAC Address: 00:11:01:00:00:01
  29. State: Active
  30. Radius Accounting ID: 19871
  31. Session ID: 19871
  32. Stacked VLAN Id: 10
  33. VLAN Id: 111
  34. Login Time: 2016-07-10 12:34:57 UTC
  35. tim@MX480-3>

 

So we can see the subscriber coming in, with an S-VLAN of 10 and a C-VLAN of 111, with an address handed out from the subscriber pool. Readers familiar with MX BNG will be used to using a “demux” interface, for the layer-2 side of the service, when PWHT is used – demux is replaced with the PS interface as shown in line 3.

Everything else remains the same, the subscriber layer-3 virtual interface is a “pp0” interface with an attached IP address placed into the inet routing table, this can be inserted into a routing-instance or logical-system if needed, by altering the BNG configuration and Radius config – for radius I’m using Freeradius with a basic configuration.

If we send some traffic – we should see it function end to end, and also see it on the PS0 interface:

Traffic works as expected:

Capture

Outputs from the “PS0” interface and attached subscriber units:

  1. tim@MX480-3> show interfaces ps0
  2. Physical interface: ps0, Enabled, Physical link is Up
  3.   Interface index: 154, SNMP ifIndex: 599
  4.   Type: Software-Pseudo, Link-level type: 90, MTU: 1530, Clocking: 1, Speed: 10000mbps
  5.   Device flags   : Present Running
  6.   Interface flags: Point-To-Point Internal: 0x4000
  7.   Current address: dc:38:e1:fc:85:4a, Hardware address: dc:38:e1:fc:85:4a
  8.   Last flapped   : Never
  9.   Input rate     : 979688 bps (89 pps)
  10.   Output rate    : 989712 bps (89 pps)
  11.   Logical interface ps0.0 (Index 336) (SNMP ifIndex 601)
  12.     Flags: Up Point-To-Point 0x4000 Encapsulation: Ethernet-CCC
  13.     Input packets : 5061878827
  14.     Output packets: 5068397825
  15.     Protocol ccc, MTU: 1514
  16.       Flags: Is-Primary
  17.   Logical interface ps0.32767 (Index 337) (SNMP ifIndex 600)
  18.     Flags: Up 0x4000 VLAN-Tag [ 0x0000.0 ]  Encapsulation: ENET2
  19.     Input packets : 9950
  20.     Output packets: 0
  21.   Logical interface ps0.1073761693 (Index 325) (SNMP ifIndex 527)
  22.     Flags: Up 0x4000 VLAN-Tag [ 0x8100.10 0x8100.111 ]  Encapsulation: ENET2
  23.     Input packets : 77850
  24.     Output packets: 6605
  25.     Protocol inet, MTU: 1508
  26.       Flags: Sendbcast-pkt-to-re, Unnumbered
  27.       Donor interface: lo0.0 (Index 322)
  28.       Addresses, Flags: Is-Default Is-Primary
  29.         Local: 1.1.1.1
  30.     Protocol pppoe
  31.       Dynamic Profile: pppoe-client-profile,
  32.       Service Name Table: None,
  33.       Max Sessions: 32000, Max Sessions VSA Ignore: Off,
  34.       Duplicate Protection: Off, Short Cycle Protection: Off,
  35.       Direct Connect: Off,
  36.       AC Name: MX480-3
  37. tim@MX480-3>

 

That’s about it! PWHT is a pretty cool feature for tunnelling subscriber connectivity into a centralised BNG environment, it’s also possible to design resilient active/standby or active/active solutions by using multiple l2circuits.

It’s also worth pointing out, that provided you have the standard subscriber management licenses, no additional licenses are required to enable PWHT.