SD-WAN – VMware SD-WAN by VeloCloud – Part 2


VeloCloud is a leading SD-WAN product which was procured by VMware 2 years ago and has since been rebranded VMware SD-WAN by VeloCloud. VeloCloud along with Viptela were the first pure players which really kick started the SD-WAN revolution so its no surprise really that both were scooped up by VMware and Cisco respectively.

Note. For background info on VeloCloud please read my Part 1 blog.

Cisco’s purchase of Viptela was a no-brainer with Cisco’s I-WAN effort not really making the cut but VMware’s purchase of VeloCloud seemed odd to me at the time. What is the virtualization giant going to do with a WAN product? Turns out it was a great purchase. VeloCloud comes as a hardware or virtual appliance and is supported on Azure, AWS and GCP. VMware have now partnered with all three of these public cloud providers enabling VMware customers to move their virtual machine workloads from on-premise to the cloud and also between cloud providers, enabling customers to optimize workloads. And what can you use as the transit mechanism for this? Yep, you’ve guessed it, VMware SD-WAN by VeloCloud.

The key advantages of using SD-WAN and the Internet as transit when you are moving workloads to the cloud are:

  • Single WAN platform – Using SD-WAN you have one platform to manage as opposed to using a combination of products, VPNs and circuits (e.g. expressroute and direct connect) which often don’t interoperate. This also creates a routing nightmare with a mix of dynamic and static routing and often manual intervention. Using SD-WAN means you have any-to-any connectivity (Mesh VPN) between all your sites and cloud environments, simplifying your WAN.
  • Time to Deliver – One benefit I have seen first hand of using SD-WAN and the Internet as transit is that when I have introduced a new cloud region, for example in Azure, I have deployed a virtual VeloCloud Edge in the new VNET and within minutes the VNET is part of my VPN overlay and routable from any site and cloud environment. It finally feels like you can deploy a network as easily as you have been able to deploy virtual machines for all these years.
  • Multi Cloud Strategy – If you are planning on a multi-cloud strategy and by this i mean multiple public cloud providers and potentially private cloud in the mix then it really doesn’t make sense to use expressroute or direct connect circuits. The Internet is the next corporate network. Wedding yourself to your MPLS network/provider by installing expressroute or direct connect circuits shouldn’t even be entertained in most instances (there are always exceptions).
  • Internet is the next corporate network  – Now if you have already started using Software as a Service such as Office 365 then you have already started to adopt this premise. If you are using a mixture of on-premise, SaaS and IaaS then using the Internet as your corporate network starts to make sense as the balance starts to shift as you migrate applications from your on-premise data centres to the cloud. SD-WAN is an enabler to use the Internet as your corporate network by securing your sites and enabling connectivity between them and your cloud environments. Want proof? You don’t see Service Providers investing in MPLS anymore they are partnering with VeloCloud and Viptela, among others, to provide managed SD-WAN solutions.

Using VMware SD-WAN by VeloCloud has enabled me to design solutions which are optimal for hybrid and mutli cloud deployments and deliver real business and technical benefits. For more information about why SD-WAN and vendor selection please see my other blogs.


SD-WAN – A New Hope

SD-WAN – Vendor/Platform Selection

SD-WAN – VMware SD-WAN by VeloCloud – Part 1

I’ve been using VMware NSX SD-WAN by VeloCloud (and that mouthful for the remainder of the post will be referred to as VeloCloud), for the last 4 months. I thought it would be useful to provide an unbiased technical evaluation of the product with a lot of network architects and engineers beginning to dip their toe into the SD-WAN market either this year or next hardware refresh.

I’ve covered why SD-WAN in another blog but in short the key driver for SD-WAN is Cloud service adoption. The shift of traffic patterns from on-premise data centres to the Internet (SaaS/PaaS/IaaS) and SD-WAN’s ability to provide simple deployment, VPN and secure local Internet breakout either by itself, Virtual Network Function (such as a Palo Alto) or service chaining to a cloud security service such as Zscaler.

Product Features

VeloCloud stood out to me when compared to rivals for a few reasons. I’ve covered SD-WAN product selection in another blog so these are the features having now productionised an SD-WAN environment that I think VeloCloud excels at.

Deployment – Once you have designed and piloted your SD-WAN environment deploying branch VCEs is simple. It takes the stress out of a site migration for an engineer and time to deliver is much quicker. In one instance I arrived at small branch site, migrated, tested and left in less an hour. This isn’t always the case as with larger or more complex branch sites requiring more testing but the VeloCloud deployment was never the issue and in most cases I could deploy the VCE in parallel of the existing WAN so that the downtime was minimal when cutting across.

QoE (Quality of Experience) – If you intend to use multiple WAN links for all or some of your branch sites, the QoE monitoring and VeloCloud’s Dynamic MultiPath Optimization (DMPO) protocol is seriously impressive. DMPO enables per packet steering across the best WAN link and QoE reports on why it moved the traffic whether it detected packet loss, increased latency or jitter. It also has the ability to buffer, duplicate packets or perform error correction on the fly. Sounds good right? I’m currently running Voice over IP over the Internet, granted my bandwidth and latency are good but this is typically a reason why enterprises are looking to deploy SD-WAN in hybrid mode (Internet and MPLS), my experience so far is that even with contention on the WAN links there has yet to be any call degradation.

QoS (Quality of Service) – Referred to as the business policy, this has application recognition and the ability for an administrator to move applications to different queues to fit the customer’s traffic pattern. VeloCloud kindly provide a starter policy which to be fair doesn’t need that much altering to get you started. The VOIP traffic as mentioned above is categorized as Real Time Protocol and put in the realtime queue which gives it outbound and inbound priority over other traffic, so even if you only have a single WAN link the VCE will still perform error correction. buffering, duplication and QoS.

Monitoring – Having used CA Spectrum and SolarWinds for years network engineers are used to fairly basic monitoring systems. The VeloCloud monitoring is mind blowing. You get all your typical statistics on bandwidth, latency, jitter etc however you also get appflow data to see what applications are being used and how much bandwidth they are using so you can see how much bandwidth a SQL query takes up one hour to the extent that you can see how much bandwidth Windows Updates have used over a year. That’s right almost instantly you can go back up to a year to that level of granularity. And then you can do something about it, for example I could see Windows updates continually using a significant amount of Internet bandwidth so instead of letting it impact users I created a business policy rule for Windows updates and put it in the low priority queue, I did this at the profile level which meant once I finished, the rule got pushed out by the Orchestrator to all branch VCEs. This visibility into the WAN and simplicity to implement application layer QoS is awesome and will benefit many enterprises.

Product Integration

VeloCloud was bought by VMware just over a year ago. In terms of product integration, there doesn’t seem to have been many announcements. I Instantly think of Viptela’s integration with Cisco and how it is already available as a container on the Cisco ISR router. But is this like-for-like? Yes and no, VMware doesn’t do routers so that was never going to be their market, so I can only surmise that VeloCloud will integrate with their existing NSX portfolio to improve their workload management capabilities between on-premise, private and public cloud providers.

While VeloCloud’s integration into VMware appears a bit slow, its integration with Microsoft and their Azure Virtual WAN has been quick. Azure Virtual WAN offers an alternative to spinning up a VeloCloud Edge virtual appliance in Azure which if you are already using Virtual WAN or a have a mixed bag of WAN solutions maybe worth investigating. Myself, I prefer a full stack, having VeloCloud throughput provides end-to-end QoS (within the VPN) using the Business Policy and excellent traffic visibility in a single dashboard. However if you have many Azure locations or subscriptions this might not be cost effective in which case Azure Virtual WAN should be strongly considered.

If I had to ask for an integration feature I would like the ability to use an external Identity Provider (IdP) such as Azure AD or Okta to control management access to the VeloCloud Orchestrator, leveraging an existing identity source and conditional based access policies.

Product Development

VeloCloud is still a developing product, like all SD-WAN products out there. I am currently using the latest version as it allows me to have direct IPsec tunnels from a branch site VeloCloud Edge (VCE) to a Cloud Security Service (Zscaler). Before this release the tunnel to Zscaler was from the cloud hosted VeloCloud Gateways (VCG). The idea of a single, resilient egress point from your VPN overlay to your cloud security service seems like a good design however, remember that IPsec tunnels to Zscaler are limited to 200Mbps so depending on the size of your organisation this may not be suitable. To be fair this isn’t a VeloCloud issue but it is something to be mindful of if you are integrating with a Cloud Security Service. Direct Tunnels from a VCE to Zscaler allow me to pick and choose which sites should use Zscaler and whether it should be used as the default route (using the Zscaler Cloud Firewall) or just for web traffic. The profile functionality of VeloCloud is very good for this and enables you to configure at scale.

One area that still needs improvement is the tunnel failover, which is planned to use IP SLA soon (IP SLA is used if using the Cloud Gateway method today).

Having used earlier versions of the code it does seem to have stabilized which is good with more emphasis on stability as opposed to rushing new features into a release, which shows a maturing product. Getting hold of release notes however was not easy, it would be good if these were made available publicly or in the VeloCloud Orchestrator.

Product Documentation

When down selecting VeloCloud it was very difficult to find any technical documentation, just the sales and solution briefs on the website which made it hard to compare the product to its rivals; although Cisco made a good mess of their website with a mix of I-WAN and Viptela (now resolved). The technical documentation is located in the VeloCloud Orchestrator which is great once you have access but trying to ascertain whether the product met a customer’s technical requirements prior to an evaluation was a bit painful and may put off some potential customers if they can’t find the information.

Now the help documentation is a mixed bag. I hold the Zscaler help documentation to be one of the better public help document repositories as it is easy to navigate and the instructions, whether is be design, implement or troubleshoot are clear and inline with the current version at that time. The VeloCloud technical documentation needs some work.

Product Support

VeloCloud Support are very responsive and the engineers I have dealt with so far are very knowledgable about the product. In some scenarios I wish I was able to SSH into the VCEs like the support engineers do to troubleshoot but then I remember this isn’t a Cisco router, its a cloud managed service. It would be good to have a cloud health status page and I believe that service providers have this kind of visibility but for enterprise customers it would be good to know if there are live issues with the underlying cloud service.


The above is my opinion based on my experience of the VeloCloud ecosystem. I look forward to see how it evolves and i’m sure my minor gripes will be resolved at some point. If you would like to discuss SD-WAN or VeloCloud please reach out to me.


SD-WAN – A New Hope

The latest trend in the network world is Software Defined WAN (SD-WAN). By 2023 Gartner predicts 80% of organisations will have moved to SD-WAN so it’s worth taking note. So why SD-WAN? What are the drivers to bin your trusty Cisco routers and even your MPLS network.

Key Drivers

  • Applications have moved to the cloud. Traffic patterns have changed, Apps and data are moving from your on-premise data centre to the cloud, whether that be Infrastructure, Platform or Software as a Service (IaaS/PaaS/SaaS). The biggest game-changer here is Office 365.
  • Office 365 adoption is a global phenomenon. Every business is either thinking, designing or implementing their Office 365 strategy. Office 365 provides a variety of services but the key ones from a network perspective are Exchange Online, Skype for Business, Sharepoint Online and Azure AD. The problem is that most businesses neglect the network and assume it will just work with the current network and security infrastructure that they have. Typical problems include, but are not limited to:
    • Forward Proxy – Exchange Online uses multiple connections per user, typically two per mailbox and calendar. Average user probably requires 6 TCP connections. Doesn’t sound like much but times that by the number of concurrent users, then how many shared mailboxes and calendars and this can be a vast amount. Problem is your firewalls and proxies can only do between 25k to 50k NAT translations per IP so often you get port exhaustion. Note. I know the theoretical limit is 64k but check what your appliances can actually handle, its probably a lot lower when you take into account reservations. This problem is compounded as half of these connections are persistent and remain open for large periods of time not like typical web browsing behaviour. Add your general web browsing on top of this traffic and you have got between 20-30 connections per user.
    • Bandwidth – Exchange Online is hungry. On average I’ve seen 1000 users require 50Mbps of Internet bandwidth just for exchange online. Times this by the number of concurrent users in your organisation and that could be a significant amount of bandwidth and potential uplift in network and security infrastructure.
    • Microsoft best practice – Microsoft say not to proxy Office 365 traffic, no authentication, no SSL inspection as it adds latency which Exchange Online in particular does not like. If Outlook performance is poor and you raise a call with Microsoft and you have traffic traversing proxies they instantly blame your infrastructure. The only approved forward proxy is Zscaler and their one-click configuration. Microsoft also recommend 1 public IP address per 2000 users, this can also be an issue for some organisations and i’m not sure if this is still relevant now that Outlook 2013 SP1 and above uses MAPI-over-HTTPS as opposed to that piece of crap protocol RPC-over-HTTPS, so do your diligence on this point.


A new hope… We have been building data centres and centralising IT infrastructure for the last 20 years. The next IT cycle is a form of network decentralisation. This is in the form of local branch Internet breakout. For years we have backhauled Internet traffic to data centres, have it traverse expensive and complex network and security infrastructure (load balancers, forward proxies, DLP, AV, SSL inspection, NGFW) before the traffic even hit the Internet. Local Internet breakout and SD-WAN offer a decentralised solution to meet the business demand for utilising cloud services, reduce cost and reliance on on-premise data centres.


  • SD-WAN enables local Internet breakout at branch sites. Leveraging local Internet  improves latency, reduces data centre infrastructure costs and removes potential bandwidth bottlenecks attributed with backhauling.
  • The Internet is the new corporate network. MPLS circuits are typically very expensive and have a long lead time. If most of your apps are moving or are already in the cloud why should you buy an MPLS circuit? Organisations can look to buy commodity Internet or broadband services cheaper and quicker to provide access directly to cloud services. SD-WAN helps your journey from MPLS to Internet by using a hybrid deployment (MPLS and Internet) or Internet only deployment., if you need to keep your MPLS circuits for certain traffic types such as on-premise VOIP or certain business applications you could run in hybrid mode until you completely transition to an Internet-only deployment. Most organisations see this as a phased approach but it really depends on your business requirements and existing traffic flows. If you can, move to Internet-only, it’s a lot simpler from a routing perspective and you can present a good saving in the move.
  • SD-WAN can (in some Vendors) prioritise traffic. Vendors like VeloCloud, Silverpeak and Riverbed can prioritise traffic using application recognition so you can prioritise Office 365 traffic over productivity-loss applications like youtube. VeloCloud is particularly good at prioritising realtime traffic, measuring jitter and latency it will duplicate packets to fill the gaps and make the voice or video call smoother.
  • Service Chaining. Most SD-WAN vendors do not bake a Next Gen Firewall into their product. Why would they when so many are on the market. VeloCloud for example allows a Palo Alto Networks NGFW as a Virtual Network Function (VNF) on an edge device so you could have a local NGFW firewall. This doesn’t really do it for me though, imagine you had 100+ sites, that’s 100+ NGFWs to manage. What is cool, is integration with cloud security services like Zscaler. You establish a tunnel (GRE or IPSEC depending on vendor) to the security service and all your Internet traffic is tunnelled via their security service. One policy to manage, no VNF to manage and maintain, now that’s cool.
  • Bring QoS back from the dead. Yes, I know QoS is a key requirement for VOIP and Video but when we started moving to web apps and encrypting web traffic using SSL, WAN Optimization and QoS started to lose their value because all traffic looked the same, so how can you prioritise it at the network level? You can’t. SD-WAN, however can as it does it at layer-7 using application recognition and its also very easy to do using a business policy and profile as opposed to QoS which is just overly complex.
  • Simplified management. Most SD-WAN vendors provide a cloud management platform which make it very easy to manage and deploy a new WAN. The key seller here is zero-touch deployment. This makes me cringe a little bit because its not like the DHL van man can rock up at your branch site and install it. Someone need to migrate your existing LAN/WLAN equipment to the SD-WAN appliance and inevitably some backend routing/firewall changes would be required so its not really zero touch its just perhaps you don’t need a CCNP certified engineer to attend and install at each site.
    • Profiles. From a VeloCloud perspective, profiles allow you to configure a profile with a lot of standard settings that can be applied to all or a subset of edge devices so that there is minimal effort to get a new edge device up and running. Only unique config is required such as static WAN IP and VLANs.
    • Integrated PKI. Having designed and implemented Cisco PKI’s in the past, I have a sigh of relief that your SD-WAN’s PKI can be handled automatically by the SD-WAN Orchestrator. Certificates are generated on activation, can be revoked and update much more frequently without having to worry about maintaining a PKI.
    • Crypto. I reckon I’ve configured over 500 VPN tunnels over my career and the amount of time I’ve spent troublehooting mis-matched crypto settings with 3rd parties makes me sad. Not an issue with SD-WAN your crypto is set at the policy/system level so all the edge devices use the same IKE version, encryption, hashing and SA timeouts.
    • In-sourcing. Another driver for simplified management is in-sourcing. A lot of organisations are moving away from big outsourced contracts and using their in-house IT department. Now, this team might not be Cisco DMVPN experts, so SD-WAN can really help them effectively manage the WAN with very little network skills. It does help if you have some though.
    • Netflow. I love Netflow it gives some really good analytics into bandwidth usage. This is baked into the SD-WAN platform so you can see top talkers, applications used and then create your business policy to reflect conditions such as prioritising business collaboration applications and lowering file sharing applications to the bulk queue.
    • Packet Capture. On VeloCloud Edges you can perform packet captures up to 120 seconds and download them from the Orchestrator and open the pcap in Wireshark to troubleshoot any issues. That’s handy.


Yes there are some negative to SD-WAN.

  • Product Maturity. The code is riddled with bugs. The amount of code issues I have had on VeloCloud and Riverbed platforms makes me pine for my trusty Cisco router. Vendors need to focus on stability rather than more and more features to how do the competition. If it ain’t stable, i’m not putting into production. The only saviour here is that most code issues I have identified are fixed in days not months like traditional network vendors so they are a lot more agile.
  • Expensive. Most SD-WAN vendors try and sell their product based on you reducing or removing your MPLS circuits for cheap Internet circuits. Fact is, you are unlikely to go and get broadband at all your sites its likely to be decent Internet circuits which aren’t that cheaper than MPLS circuits. SD-WAN appliances aren’t as cheap as a Cisco router. They typically have an OPEX model so you will typically buy a 3 or 5 year service as opposed to a one-off purchase of a router. Most license based on bandwidth so it really depends on the size and amount of  circuits your organisation has.
  • Service Chaining limits. Zscaler has a soft limit of 200Mbps for each IPsec tunnel and 1Gbps for each GRE tunnel. Bear in mind when choosing an SD-WAN vendor that most of them only do IPsec tunnels (Cisco Viptela, VeloCloud and Riverbed). Zscaler’s solution is to load balance tunnels if you require greater bandwidth. Now this may not be an issue for you, but some of my customers have 2000+ users in a campus so this is going to be an issue.
  • Couldn’t I just do split-tunnel DMVPN with my Cisco routers? Yes you could, but as per the benefits I’ve mentioned, SD-WAN is not just split tunnelling. I’ve not even gone into the detail about bonding multiple Internet links, 4G resilience etc. You could change your DMVPN to some kind of IWAN type solution but even Cisco is ditching this so why design a solution that even the vendor is moving away from. Cisco’s proposition is Viptela for Service Provider and Large enterprise and Meraki for small to mid enterprise. If your business has no cash and you want to be a hero with an A-Team type solution then by all means “do up the van” 🙂
  • Not everyone will be onboard. Believe it or not, not everyone is in love with the cloud. You really need key stakeholders on board for SD-WAN as it is a drastic change to typical enterprise network design. Some people want to hold onto their castle and moat network designs. This is no longer fit for purpose. Why protect the castle if your apps are outside of it?


It is very likely that your next WAN upgrade will involve SD-WAN, whether that be in hybrid mode or Internet only. Network design needs to evolve to user centric and not data centre centric. When making your product decision, make sure you gather your business and technical requirements. Not all SD-WAN vendors are the same, make sure they can do decent crypto, PKI, integration with routing protocols etc. Don’t assume anything, these device are not Cisco routers.