Zscaler – Disaster Recovery using ZPA

zpa-casestudy

Over the years we have designed and implemented a variety of Disaster Recovery (DR) environments and processes for customers to meet their requirements. These DR projects have ranged from Active/Active data centres or Primary/DR data centres to application specific DR.

The latest DR project we were involved in was for a government department who host their applications in Microsoft Azure. The requirement was that in the event of a disaster at their primary Azure Region to be able to spin up their applications and infrastructure in another Azure region within their DR Recovery Time Objective (RTO).

For the customer’s external facing applications we used native Azure front-door functionality such as Azure Traffic Manager to failover DNS between regions so this was relatively straight forward to redirect traffic once the application had been spun-up. However for the internal applications we needed another method to fail traffic over because these applications were not exposed to the Internet.

Enter Zscaler Private Access (ZPA). ZPA is a cloud service from Zscaler which provides Zero-Trust Network Access (ZTNA) to internal applications hosted on-premise or in the public cloud. We had already deployed ZPA for the customer 18 months ago to provide users access to internal applications in Azure. So to provide DR for internal applications we leveraged the customer’s existing investment in ZPA.

One of the key customer requirements was for zero infrastructure costs in the DR Azure region so for DR testing or for an actual DR event we would spin up the infrastructure on-demand. Another requirement was that the RTO for their applications must be within 3 hours.

We were able to stand up a configured Zscaler connector within 30 minutes and fail all traffic that currently traversed the Zscaler connectors in Azure region A to the DR Zscaler connector in Azure region B.

Using ZPA for disaster recovery of the customer’s internal applications allowed us to achieve their key DR requirements for cost and RTO whilst also providing an easy set of steps for DR testing or an actual DR event. Another requirement we were also able to achieve was per-application DR, which was desirable but not a mandatory requirement as the customer did not think we would be able to achieve because of duplicate IP addressing. The fact that ZPA is not a network based VPN enabled us to achieve per-application DR.

If you’d like to talk Zscaler Private Access get in touch.

Alex

 

 

SD-WAN – A New Hope

The latest trend in the network world is Software Defined WAN (SD-WAN). By 2023 Gartner predicts 80% of organisations will have moved to SD-WAN so it’s worth taking note. So why SD-WAN? What are the drivers to bin your trusty Cisco routers and even your MPLS network.

Key Drivers

  • Applications have moved to the cloud. Traffic patterns have changed, Apps and data are moving from your on-premise data centre to the cloud, whether that be Infrastructure, Platform or Software as a Service (IaaS/PaaS/SaaS). The biggest game-changer here is Office 365.
  • Office 365 adoption is a global phenomenon. Every business is either thinking, designing or implementing their Office 365 strategy. Office 365 provides a variety of services but the key ones from a network perspective are Exchange Online, Skype for Business, Sharepoint Online and Azure AD. The problem is that most businesses neglect the network and assume it will just work with the current network and security infrastructure that they have. Typical problems include, but are not limited to:
    • Forward Proxy – Exchange Online uses multiple connections per user, typically two per mailbox and calendar. Average user probably requires 6 TCP connections. Doesn’t sound like much but times that by the number of concurrent users, then how many shared mailboxes and calendars and this can be a vast amount. Problem is your firewalls and proxies can only do between 25k to 50k NAT translations per IP so often you get port exhaustion. Note. I know the theoretical limit is 64k but check what your appliances can actually handle, its probably a lot lower when you take into account reservations. This problem is compounded as half of these connections are persistent and remain open for large periods of time not like typical web browsing behaviour. Add your general web browsing on top of this traffic and you have got between 20-30 connections per user.
    • Bandwidth – Exchange Online is hungry. On average I’ve seen 1000 users require 50Mbps of Internet bandwidth just for exchange online. Times this by the number of concurrent users in your organisation and that could be a significant amount of bandwidth and potential uplift in network and security infrastructure.
    • Microsoft best practice – Microsoft say not to proxy Office 365 traffic, no authentication, no SSL inspection as it adds latency which Exchange Online in particular does not like. If Outlook performance is poor and you raise a call with Microsoft and you have traffic traversing proxies they instantly blame your infrastructure. The only approved forward proxy is Zscaler and their one-click configuration. Microsoft also recommend 1 public IP address per 2000 users, this can also be an issue for some organisations and i’m not sure if this is still relevant now that Outlook 2013 SP1 and above uses MAPI-over-HTTPS as opposed to that piece of crap protocol RPC-over-HTTPS, so do your diligence on this point.

SD-WAN

A new hope… We have been building data centres and centralising IT infrastructure for the last 20 years. The next IT cycle is a form of network decentralisation. This is in the form of local branch Internet breakout. For years we have backhauled Internet traffic to data centres, have it traverse expensive and complex network and security infrastructure (load balancers, forward proxies, DLP, AV, SSL inspection, NGFW) before the traffic even hit the Internet. Local Internet breakout and SD-WAN offer a decentralised solution to meet the business demand for utilising cloud services, reduce cost and reliance on on-premise data centres.

Benefits

  • SD-WAN enables local Internet breakout at branch sites. Leveraging local Internet  improves latency, reduces data centre infrastructure costs and removes potential bandwidth bottlenecks attributed with backhauling.
  • The Internet is the new corporate network. MPLS circuits are typically very expensive and have a long lead time. If most of your apps are moving or are already in the cloud why should you buy an MPLS circuit? Organisations can look to buy commodity Internet or broadband services cheaper and quicker to provide access directly to cloud services. SD-WAN helps your journey from MPLS to Internet by using a hybrid deployment (MPLS and Internet) or Internet only deployment., if you need to keep your MPLS circuits for certain traffic types such as on-premise VOIP or certain business applications you could run in hybrid mode until you completely transition to an Internet-only deployment. Most organisations see this as a phased approach but it really depends on your business requirements and existing traffic flows. If you can, move to Internet-only, it’s a lot simpler from a routing perspective and you can present a good saving in the move.
  • SD-WAN can (in some Vendors) prioritise traffic. Vendors like VeloCloud, Silverpeak and Riverbed can prioritise traffic using application recognition so you can prioritise Office 365 traffic over productivity-loss applications like youtube. VeloCloud is particularly good at prioritising realtime traffic, measuring jitter and latency it will duplicate packets to fill the gaps and make the voice or video call smoother.
  • Service Chaining. Most SD-WAN vendors do not bake a Next Gen Firewall into their product. Why would they when so many are on the market. VeloCloud for example allows a Palo Alto Networks NGFW as a Virtual Network Function (VNF) on an edge device so you could have a local NGFW firewall. This doesn’t really do it for me though, imagine you had 100+ sites, that’s 100+ NGFWs to manage. What is cool, is integration with cloud security services like Zscaler. You establish a tunnel (GRE or IPSEC depending on vendor) to the security service and all your Internet traffic is tunnelled via their security service. One policy to manage, no VNF to manage and maintain, now that’s cool.
  • Bring QoS back from the dead. Yes, I know QoS is a key requirement for VOIP and Video but when we started moving to web apps and encrypting web traffic using SSL, WAN Optimization and QoS started to lose their value because all traffic looked the same, so how can you prioritise it at the network level? You can’t. SD-WAN, however can as it does it at layer-7 using application recognition and its also very easy to do using a business policy and profile as opposed to QoS which is just overly complex.
  • Simplified management. Most SD-WAN vendors provide a cloud management platform which make it very easy to manage and deploy a new WAN. The key seller here is zero-touch deployment. This makes me cringe a little bit because its not like the DHL van man can rock up at your branch site and install it. Someone need to migrate your existing LAN/WLAN equipment to the SD-WAN appliance and inevitably some backend routing/firewall changes would be required so its not really zero touch its just perhaps you don’t need a CCNP certified engineer to attend and install at each site.
    • Profiles. From a VeloCloud perspective, profiles allow you to configure a profile with a lot of standard settings that can be applied to all or a subset of edge devices so that there is minimal effort to get a new edge device up and running. Only unique config is required such as static WAN IP and VLANs.
    • Integrated PKI. Having designed and implemented Cisco PKI’s in the past, I have a sigh of relief that your SD-WAN’s PKI can be handled automatically by the SD-WAN Orchestrator. Certificates are generated on activation, can be revoked and update much more frequently without having to worry about maintaining a PKI.
    • Crypto. I reckon I’ve configured over 500 VPN tunnels over my career and the amount of time I’ve spent troublehooting mis-matched crypto settings with 3rd parties makes me sad. Not an issue with SD-WAN your crypto is set at the policy/system level so all the edge devices use the same IKE version, encryption, hashing and SA timeouts.
    • In-sourcing. Another driver for simplified management is in-sourcing. A lot of organisations are moving away from big outsourced contracts and using their in-house IT department. Now, this team might not be Cisco DMVPN experts, so SD-WAN can really help them effectively manage the WAN with very little network skills. It does help if you have some though.
    • Netflow. I love Netflow it gives some really good analytics into bandwidth usage. This is baked into the SD-WAN platform so you can see top talkers, applications used and then create your business policy to reflect conditions such as prioritising business collaboration applications and lowering file sharing applications to the bulk queue.
    • Packet Capture. On VeloCloud Edges you can perform packet captures up to 120 seconds and download them from the Orchestrator and open the pcap in Wireshark to troubleshoot any issues. That’s handy.

Negatives

Yes there are some negative to SD-WAN.

  • Product Maturity. The code is riddled with bugs. The amount of code issues I have had on VeloCloud and Riverbed platforms makes me pine for my trusty Cisco router. Vendors need to focus on stability rather than more and more features to how do the competition. If it ain’t stable, i’m not putting into production. The only saviour here is that most code issues I have identified are fixed in days not months like traditional network vendors so they are a lot more agile.
  • Expensive. Most SD-WAN vendors try and sell their product based on you reducing or removing your MPLS circuits for cheap Internet circuits. Fact is, you are unlikely to go and get broadband at all your sites its likely to be decent Internet circuits which aren’t that cheaper than MPLS circuits. SD-WAN appliances aren’t as cheap as a Cisco router. They typically have an OPEX model so you will typically buy a 3 or 5 year service as opposed to a one-off purchase of a router. Most license based on bandwidth so it really depends on the size and amount of  circuits your organisation has.
  • Service Chaining limits. Zscaler has a soft limit of 200Mbps for each IPsec tunnel and 1Gbps for each GRE tunnel. Bear in mind when choosing an SD-WAN vendor that most of them only do IPsec tunnels (Cisco Viptela, VeloCloud and Riverbed). Zscaler’s solution is to load balance tunnels if you require greater bandwidth. Now this may not be an issue for you, but some of my customers have 2000+ users in a campus so this is going to be an issue.
  • Couldn’t I just do split-tunnel DMVPN with my Cisco routers? Yes you could, but as per the benefits I’ve mentioned, SD-WAN is not just split tunnelling. I’ve not even gone into the detail about bonding multiple Internet links, 4G resilience etc. You could change your DMVPN to some kind of IWAN type solution but even Cisco is ditching this so why design a solution that even the vendor is moving away from. Cisco’s proposition is Viptela for Service Provider and Large enterprise and Meraki for small to mid enterprise. If your business has no cash and you want to be a hero with an A-Team type solution then by all means “do up the van” 🙂
  • Not everyone will be onboard. Believe it or not, not everyone is in love with the cloud. You really need key stakeholders on board for SD-WAN as it is a drastic change to typical enterprise network design. Some people want to hold onto their castle and moat network designs. This is no longer fit for purpose. Why protect the castle if your apps are outside of it?

Summary

It is very likely that your next WAN upgrade will involve SD-WAN, whether that be in hybrid mode or Internet only. Network design needs to evolve to user centric and not data centre centric. When making your product decision, make sure you gather your business and technical requirements. Not all SD-WAN vendors are the same, make sure they can do decent crypto, PKI, integration with routing protocols etc. Don’t assume anything, these device are not Cisco routers.

Zscaler – Zenith Live 2018 – Roadmap / Best Practices

Throughout the event I will be speaking to employees and product managers from Zscaler to get information on the next big thing, hopefully some best practices and also push for some feature requests.

Keynote from Jay Chaudhry, CEO of Zscaler

Jay talked about the cloud journey that we are all on but also the move from network centric architectures to user centric and I think that is key for businesses and engineers to understand to succeed in cloud transformation.

Zapp

Spoke to David Creedy, product manager for Zapp about current forwarding issues and roadmap for Zapp.

Roadmap – Zapp will be changing its tunnel mode from TCP to DTLS which will make it much more lightweight and improve user performance. No date on this yet.

Roadmap – Zapp user interface is being revamped and will provide the user more information. This looks more like a mobile app and shows ZIA and ZPA status in parallel which is really nice.

IMG_7953

Roadmap – Currently if you are using Zapp on-premise with no GRE/IPSec tunnelling you are NATing all clients behind a single public IP address so all clients will hit the same ZEN node, no load balancing. Coming soon you will be able to modify the PAC file to distribute the load in this scenario to multiple ZENs.

IMG_7948

Roadmap – Zapp will be able to forward all ports and protocols to the Zscaler cloud in Q1 2019 as opposed to currently where only HTTP and HTTPS traffic is forwarded. This is pretty cool for a split tunnel solution as we can now protect/restrict all ports from a device from any location. Would I want to retrospectively implement this? I’m not sure, it would take a lot of UAT to ensure we don’t break any existing connectivity. Think about it your localised SMB traffic gets thrown to the cloud, might cause some issues. Look forward to testing this though.

IMG_7951

David talked about forwarding modes for Zapp. Zscaler seem to have changed their minds on best practice from ‘tunnel with local proxy mode’ to ‘tunnel using the packet based filter’ which is a new addition in version 1.3.1 of the Zapp. The key advantage is that all system context traffic is captured and only for HTTP and HTTPS so is more optimal. Where as ‘tunnel using routing mode’ captures all traffic and can cause issues, as I have experienced in the past. This best practice also depends on the customer’s environment and if they are using an existing VPN RAS solution like Cisco AnyConnect or PAN Global Protect when you would need to use Tunnel with Local Proxy mode for traffic when off-network or on VPN. Forwarding decision definitely take some testing.

Best practice tip – Forwarding PAC – By-pass URLs in the forwarding PAC as opposed to the App Profile PAC so the browser deals with the PAC and not the Zapp as browsers are better at handling this.

Best practice tip – Zscaler also saying you should disable Zapp on trusted network and let it route over your tunnel to the Zscaler cloud. This seems debatable as how would the authentication work without the Zapp? You would have to open a browser first before Outlook for example as Outlook is not cookie aware so would appear as unauthenticated traffic which in some environments may not be allowed. Can’t see users liking this experience. I’m still in favour of tunnelling within a tunnel, so Zapp on and via a tunnel, ok there is some additional overhead but the user experience is much better.

Feature Request – Zapp Pre-Login Gina. I asked David whether it was on the roadmap for Zapp to start pre-login. Zscaler are looking to develop this which is awesome as it makes ZPA a real contender against PAN Global Protect if users can reset password with AD, GPO updates etc via pre-login. They are looking to start development on this so don’t expect it anytime soon as they need to work on a machine context for Zapp as everything is currently in the user context.

Feature Request – Zapp for Citrix Session Based Compute. Currently I use PAC files on Citrix SBC which works ok but has limitations, such as websites using connect method strips the authentication cookie. Zscaler are also looking into Zapp for session based compute so multiple users on the same server have their own Zapp context/tunnel.

Zscaler Internet Access

I visited the ZIA user interface booth and gave a load of feedback for UI improvements such as referencing CVEs in their threat library, exclusion searches and more Office 365 statistics, hopefully some of these will make it into the next release. Some cool new reports and dashboards being rolled out across the Zscaler clouds, for example a quarterly business powerpoint which is cool for senior stakeholders and you can edit it so really nice feature.

Attended the keynote from Microsoft which was really interesting as most of my customers use Azure AD and Office 365. Microsoft demo’d the CASB integration with Zscaler which is a really nice feature whereby using the Microsoft App Security portal you can effectively block access to cloud apps like dropbox by just clicking on it and this updates your Zscaler policy via an API call. This is really useful for Information and Security officers who perhaps are not hands on but want to instantly block user traffic to file sharing apps or other.

Roadmap – I have issues with a customer who uses 3rd parties which permit access via Source IP, sigh. Using Zscaler, obviously your IP comes from a pool of shared IP addresses so often the 3rd party won’t allow you to use this. As a work around, if you are luck, you can do a proxy chain in the PAC file back to your existing on-premise Proxy, but if you’ve got rid of that then you’ve got a problem. Now, really you should talk to the business and the partner about changing this control to something a bit more modern and user based like SAML but if that’s not going to happen Microsoft are baking a new feature into Azure AD Conditional Access which will allow you expose your location IP address as opposed to the Zscaler IP address, this is going to really help in these scenarios.

IMG_7954

What was also cool to see was the Microsoft network map and their peerings with Zscaler. Zscaler is peered with Microsoft in London, Paris and Amsterdam which is three data centres which I use the most which means very low latency.

IMG_7955

Attended a session where the company SFK presented their Zscaler solution using regional Internet breakouts as opposed to local Internet breakouts, using key locations where Zscaler, Akamaii and Microsoft were located such as London and Paris in Europe. SFK also used Zapp when users are On-Network so they were doing tunnel (Zapp) within a tunnel (GRE), which is a solution i’m looking to deploy, so good to know its works and performance is good.

IMG_7946

SFK also talked about routing out through the Great Chinese Firewall which was really interesting the issues you will have trying to egress out of china, especially around latency and packet loss as if the great firewall is overloaded, scary stuff.

IMG_7947

Roadmap – The Cloud Firewall if you have the advanced license does IPS for all ports. Coming soon is SNORT integration and a bit later the ability to add your own SNORT rules which is handy if you get advanced notification of threats from people like NCSC. In the meanwhile you can send Zscaler the SNORT rules and they will implement them for you.

Tech note – Skype traffic was brought up regarding latency as Zscaler can add up to 100ms latency. This is obviously unacceptable from a voice/video perspective so is not as inspected as web traffic to provide better latency to this type of UDP traffic.

Tech note. sub locations and countries. So one customer I have wanted to preface the data centres that could be used by clients. So in the PAC file you can change the $gateway setting to a specific data centre like lon3.sme.zscalerone.net. The reasoning is to ensure traffic is traversing a DC in a trusted country. However you don’t get the geo-location so users working in a different country would have a poorer experience because of the increased latency. Now you could introduce rules in the PAC file to combat this but if their travel is dynamic then this is not scalable. A solution for this is for Zscaler to allocate you a sub-cloud. This is purely a whitelist of ZEN nodes you can use, so you can raise a ticket and say I want London, Paris and Amsterdam only and change your PAC to customerdomain_$gateway. That’s pretty cool. Another alternative is to use the following $gateway_country. This parameter doesn’t chose the ZEN via geo location or lowest latency but by what country you are in which gives you more control with very little config.

I got the PAC file information from Yogi at Zscaler, that guy knows his stuff, if you ever get to meet him you can get a lot of answers and put your requests to him.

Network Transformation – Hub and Spoke to a Hybrid network

When we talk about hybrid networks we are talking about local Internet break-out at branch sites and SD-WAN and how goliaths like Siemens have moved to the model of Internet as the Corporate Network. Why are we changing from hub and spoke to SD-WAN? Because workloads are moving to the cloud. in the 90s/00s all the apps were in the data centre but now companies are migrating apps to SaaS, Public Cloud and Private cloud, the traffic pattern has shifted to the Internet and the network needs to adapt to reflect this.

Some interesting points made at the network transformation session around the types of Internet circuits being procured at branch sites. Circuits ranging from carrier grade to broadband, how do you procure these internationally, who makes the decision on the grade of circuit. These are business decisions, but it is rare to get a customer to make these decisions. This needs to be driven technically and justified to the business.

Siemens and Orange Business Services presented their network transformation journey using Zscaler and SD-WAN. The project plan below is really good and could be applied to most customer deployments.

img_7944.jpg

Zscaler – Zenith Live 2018 – SD-WAN

We are at Zenith Live at Heathrow this week to check out the latest Zscaler features and roadmap as well as to give them some grief on some of the issues we’ve had on customer implementations.

21/10/18 – SD-WAN Integration Training

Spending my Sunday configuring IPSec VPNs to Zscaler from Cisco Viptela and Riverbed SD-WAN appliances, now that’s dedication. I’m at the SD-WAN training session today to get a feel for other SD-WAN providers and see how they compare to VeloCloud.

Roadmap – Zscaler allows SD-WAN vendors to link in via API calls. Both Riverbed and VeloCloud should have this integration in Q1 2019. What does this give you? Well it means the VeloCloud Orchestrator can automatically configure the VPN Credentials and Location in the Zscaler admin portal using an API call. Another provisioning step automated.

Riverbed Lab

13:04 Brandon Carroll, Senior Technical Evangelist from Riverbed is up next. I’m excited to hear the visionary imaginearing he’s been doing in his role 😀 To be fair Riverbed are pushing SD-WAN hard, and so they should or they will fold with the WAN Optimisation market dying, if not already dead. Companies who have already invested in Riverbed will probably be looking at their platform but I can’t see why you would otherwise.

Initial thoughts, the Riverbed Steelhead Manager, which is the Riverbed Orchestrator, looks pretty good. Their network map shows all the sites status and the VPN tunnels between the sites, which is really neat when compared to the VeloCloud network map which is really basic and lacking info. The map might get messy on a large deployment but does look cool and handy to see established tunnels. The Zscaler configuration is very simple and you get a lot of information such as events and latency to the chosen Zen nodes. There is a balance here, the manager is rather cluttered and overwhelming with tabs and options, there is a lot going on but also a lot of useful information so it really depends on what you want. It looks very complicated in comparison to VeloCloud however I do wish VeloCloud had more information and stats.

The Riverbed lab was pretty good to get some hands on the Riverbed SD-WAN. Like VeloCloud it is very easy to connect to Zscaler. The dashboard is pretty slick to be fair (screenshot below).

river-1.png

Ok speaking to the evangelist, its starting to fall apart. The Riverbed solution is full mesh only or to the hub only. Now this means in full mesh mode the smallest site would need to connect to every site. So the hardware spec of your smallest site depends on how many sites you have so you may need a large appliance at a small site because you have so many sites/tunnels to support. That doesn’t seem very scalable for a large deployment. How much branch-to-branch traffic do you really have? Its only voice these days really so why not make it dynamic tunnels to reduce the overhead.

Cisco Viptela Lab

15:02 No representative from Cisco to push Viptela which is a shame but nonetheless we deep dive into Viptela configuration. If you want an SD-WAN solution which is pretty much the same as configuring a Cisco ISR router then this is for you. Personally I don’t see the point. One of the main goals of moving to SD-WAN is to simplify branch site administration and yes I know you can use templates but there are a lot of templates. If you need this type of granularity then maybe this is for you. Viptela is really aimed at Service Providers / Large Enterprises and it has the price tag to match. It is cool to see exactly what code is being pushed to the devices instead of just hitting a button and just hoping the magic happens as with VeloCloud and Riverbed and the Cisco geek in me loves to reminisce over DMVPN rollouts and CLI but I just don’t see it for SMBs or small to mid enterprises.viptela-1

The integration with Zscaler was fairly easy but a lot more convoluted then VeloCloud and Riverbed. I had to create two feature templates and modify my routing feature template then bind to my device template. When you have more templates than devices the dynamic is wrong.

 

 

 

Zscaler – Forwarding Choices

I have been using Zscaler Internet Access (ZIA) for nearly two years and the product still impresses me with its simplicity yet feature full platform. I am currently working on my third ZIA design and whilst a lot can be replicated each customer environment makes the design different. One of the main decisions is how you forward the user traffic to the Zscaler cloud.

Zscaler has a range of forwarding options but the decision really depends on your customer’s requirements, current environment, end user devices and user behaviour. Fortunately you can use combinations of options to try and suit the customer’s needs. Now I always opt for simplicity in my designs as while complex is cool to design its not cool to support.

Zscaler App (Zapp)

My preferred method of forwarding is the Zscaler App (Zapp). Whilst I am a big fan of clientless methods of forwarding like PAC files, the Zapp is really good. Its a lightweight app that is supported on Windows, Mac, Android and Apple which if your customer has a big mobile device presence makes this a no brainer. The Zapp is managed via the admin portal so you can push policy updates and software updates centrally. The zap client checks in every 20 seconds so updates are quick. The software update process is particularly slick and you get a report of what devices are running which version so you can gauge the process over time with no additional management tooling like Microsoft SCCM. The Zapp handles the authentication which I have used both Microsoft ADFS and Azure AD successfully. The app installs a virtual adapter which forwards all (unless specified not to) web traffic to the Zscaler cloud. This means that all user and system traffic is captured and forwarded which is a lot better than a pure PAC file solution.

PAC File

The simplest way of forwarding is using a PAC file which is pushed out via group policy. If you have a large static device estate or want an easy migration then this maybe the easiest option and also there is no real minimum operating system spec, so again it depends on the customer. I deploy a PAC file forwarding method to a Citrix environment because it is session based computing the Zapp wouldn’t work correctly with authenticating the correct user so PAC had to be used. Whilst this works fine and integrates easily with Microsoft ADFS for authentication there are limitations. Often the user credentials get stripped because they are carried in a cookie e.g. a CONNECT strips this out so we get logs showing unauthenticated user which is not ideal. Zscaler recommend using PAC in combination with GRE and IPSec tunnels to capture all the traffic. Again you don’t have to and in some environments I haven’t, for example Zscaler doesn’t play well with Checkpoint firewalls which meant an IPSec tunnel couldn’t be used. Also using a tunnel means implementing policy based routing or having a default route to the Zscaler cloud, two options not to be taken lightly, this is major change territory so think carefully considering this in complex customer networks.

GRE / IPSec Tunnelling

If you need to tunnel all your traffic, perhaps a Guest WIFI environment then GRE or IPSec tunnelling is a good option when you can’t control your clients. However Zscaler isn’t cheap so I wouldn’t imagine you would buy it just for Guest Wifi so it is likely you would use a combination of forwarding methods.

SD-WAN

At the time of writing I am currently working on a VeloCloud SD-WAN design. The customer already uses Zscaler and VeloCloud is a Zscaler partner so integrates nicely. I am deploying the Zapp to all client devices but also having IPSec tunnels to the nearest Zscaler cloud to ensure all branch web traffic is captured e.g. client and non client devices, printers, Guest WiFi etc. SD-WAN and Security as a Service providers like Zscaler compliment each other nicely and I can see more customers going down this route as more services shift to the cloud and they want to manage less data centre infrastructure.

 

Zscaler – Authentication Choices

Zscaler is the market leading cloud proxy service. I have implemented Zscaler for a number of customers in the Transport and Government sector over the last 3 years and its adoption is rapidly increasing as customers look to migrate services to the cloud, increased workforce mobility, a move to OPEX based services and away from traditional on-premise CAPEX heavy forward proxy solutions like Bluecoat. Note. Bluecoat do offer a cloud service but it is doesn’t have the maturity and cloud scale that Zscaler has built up as a pure cloud player over the last 10 years.

There are many ways to authenticate users of the Zscaler service, locally defined database, LDAP and SAML to name the main ones. However the problem comes in populating the user database. If you had a small organisation (sub 50 users) then a locally defined user database might work for you, however most organisation are way above this figure and want to leverage their existing Identity management system such as Active Directory or Azure AD. Now this is fine as we can integrate Zscaler into both of these IDPs using SAML. However populating the user database depends on your requirements. We can auto-populate (preferred) the Zscaler user database using the data from the SAML tokens, which is great as each time the user authenticates to Zscaler their information is updated! This works great when using Active Directory (ADFS) or Azure AD (see my other articles on how to integrate these). However when using AzureAD the user’s group membership cannot be added to the SAML token. You could define local groups in Zscaler if that meets your requirements however if like most organisations you are migrating from an on-premise forward proxy solution to Zscaler then chances are you will want to ease the pain by using the existing AD groups which are used to define your proxy policy. With ADFS we can insert the AD Groups that the user belongs to in the SAML token, it just puts a dependency on having an ADFS infrastructure in place or building one.

The other options are LDAP or Zscaler Authentication Bridge (ZAB). Where you could still carry out user authentication via SAML but populate the Zscaler user database using LDAP or the ZAB. I don’t like either of these solutions. Exposing your domain controllers to the Internet, albeit source IP restricted doesn’t feel right to me, it also puts a dependency on LDAPS being in place which a lot of organisations still don’t use. I guess you could use LDAP but i’m not even going to suggest exposing your IDP data unencrypted over the Internet. and the ZAB, what’s the point of a pure cloud proxy solution if you have to put a VM in your on-premise data centre to sync your user data. (Also Bluecoat’s and Cisco Umbrella’s solution) Also the ZAB is currently only supported on VMware so you couldn’t even host this is Azure or AWS!

I like the elegant solution of authentication and auto-populating the user database using SAML. I’ve spoken to Microsoft about being able to include Groups in the SAML token when using Azure AD as the IDP and it doesn’t seem to be supported for Zscaler. Note. You can modify app manifest files to populate User Groups in the SAML token for other apps so you maybe able to do this for Zscaler. Off topic, I don’t understand why Azure has App registrations and Enterprise Apps, this should be merged! Zscaler is deployed as an Enterprise App as such there is no Manifest file to modify, So you would have to manually register the app if this is even possible. I would hope, as this is not the only use case, that Groups will be possible using Azure AD at some point, Microsoft have got pretty good at listening to the community now that they have other options.

Links

https://support.zscaler.com/hc/en-us/articles/204455339-Choosing-Provisioning-and-Authentication-Methods