Skip navigation
duo labs

Test New macOS Versions Early to Protect Your Users

Out-of-date endpoints pose a major security risk for any organization. In a study from 2015 titled “‘ one can hack my mind’: Comparing Expert and Non-Expert Security Practices,” Google researchers compared the top five security habits of “regular” Internet users to those of computer security experts.

In it, 35% of security experts stated that installing updates for the operating system, computer firmware and third-party applications is one of their top three important habits to prevent security breaches, compared to only 2% of regular users. This made updating the number one habit of security experts in the survey.

In addition, 25% of experts stated that software updates are installed “immediately.” Unfortunately, deploying major macOS upgrades to an entire organization can be risky without having tested them with your organization’s applications prior to release day. If even one essential application is incompatible with a new macOS version, its deployment may have to be delayed by days, weeks or months, which can impact security.

Joining an Apple beta program and testing early in your organization can ensure your organization is better prepared for upcoming major macOS releases and can upgrade its users as quickly as possible.

About Apple Beta OS Releases

With the release of OS X 10.9 Mavericks, Apple adopted an annual release schedule for macOS. This faster schedule has made it more crucial than ever for organizations to take advantage of the availability of beta versions of Apple’s operating system. The beta OS program is most useful during the period from mid-June through mid-September when the next major macOS release is in development.

And while the term “beta” may sound like something to avoid, it is actually a unique opportunity to get ahead of any potential trouble. The reality is that important security fixes and improvements are only made available for the shipping macOS version and the one before it, while older versions are quietly dropped from Apple’s patching schedule.

Organizations that want to maintain secure macOS deployments must therefore keep up with Apple’s schedule while also making sure that its users have the applications they need to work. This can become a major issue if your users find out on release day that they can’t connect to the VPN, email or other resources.

How Can I Make Sure My Organization is Ready?

In a perfect world, third-party vendors would ensure that they have access to macOS beta versions and test their software for breaking changes. An example of such a breaking change is the gradual tightening of Apple’s Gatekeeper feature, which aims to block potentially malicious applications by either allowing all applications, only signed applications or only Mac App Store applications to run.

When initially introduced, Gatekeeper was an optional feature configured to its most permissive state which could be changed by the user to be more restrictive. In macOS 10.12, Apple removed the ability for the user to bypass Gatekeeper in an effort to further increase macOS security. While this is a positive change for overall macOS security, it will cause any application that is not signed by the vendor to be blocked from running.

This can impact users directly when it stops a crucial tool like a VPN application from running, preventing them from connecting to important network resources. Vendors are not always aware of changes like this quickly enough and are therefore unable to complete timely patching. This can lead to unexpected delays if your organization does not perform its own testing ahead of time. In addition, an organization usually relies on a diverse mix of software for its core operations, further complicating release day preparedness.

macOS Beta Options

Apple maintains several beta testing programs: a free, public program; a paid program as part of its developer services; and an invite-only program.

Apple Public Beta Program

The easiest and lowest-commitment program to be a part of is the Apple Public Beta program. The only requirement for joining this program is a valid Apple ID. After joining the program, the user is provided with an enrollment profile that grants them access to beta releases of macOS.

The Public Beta program includes a Feedback Assistant that can be used to report bugs to Apple. Bug reports through this channel are treated with the same priority as those reported through Apple’s public Feedback page. During a typical beta OS cycle, Apple releases 3-4 Public Betas.

Apple Developer Program

The Apple Developer program provides access to macOS betas as part of the program’s annual $99 fee. In fact, a Developer program member gets access to betas for all of Apple’s platforms: iOS, macOS, tvOS and watchOS. Membership in the Developer program also grants access to other tools and resources for developing applications for all of Apple’s platforms.

An important advantage for those just interested in access to OS betas is that the Developer macOS beta program offers a faster iteration at the rate of about 7-8 releases per cycle compared to the Public Beta’s 3-4. Bug reports reported through the Developer program are handled by Apple’s dedicated Developer Relations team instead of through Apple’s regular public feedback channel.

Apple Seed Beta Program

Finally, the Apple Seed beta program is run on an invite-only base at Apple’s discretion. Based on a customer request to AppleCare Enterprise support, the Apple Seed team identifies eligible customers to test specific macOS features. An invited customer will be expected to file regular bug and progress reports with the AppleSeed team, which takes care of routing the reports to the responsible development teams.

Public Beta Releases

During the most recent cycle leading up to macOS 10.12 Sierra, Apple officially announced the first beta at their annual Worldwide Developer Conference (WWDC) on June 13th, 2016 with the first Developer Preview (DP) made available that same day. The Public Beta program signup opened up the same day, with the first Public Beta (PB) made available on July 7th.

After this first PB release, Apple had regular DP releases followed a few days later by a PB release until their release dates combined for DP6/PB5 on August 15th. The final combined release was DP8/PB7 on August 29th. The Golden Master (GM) was made available to both Developer and Public beta members on September 8th. A GM release or RTM is what used to be actual physical master media sent to replication facilities for retail sale.

Because Apple has not shipped a physical OS installer since OS X 10.6 Snow Leopard, the “GM” designation is now mainly used to indicate a feature freeze milestone and to tag the build version that will be available from the Mac App Store on release day.

Who Uses macOS Betas?

Looking at our most recent data, we see steadily growing customer adoption of macOS 10.12 beta versions, as shown in Figure 1. In the graph, we measure unique endpoint check-ins to Duo’s two-factor authentication service during the macOS 10.12 Sierra beta period from June 13th through September 19th, 2016.

We can see that once the PB releases were available, Duo customer adoption started picking up. Endpoints running macOS 10.12 went from a handful mid-June to several hundred by mid-August. By the end of the beta period in mid-September, 3-4% of total endpoints were running macOS 10.12 Sierra.

macOS Sierra Adoption Rate

Why Should I Test? And What?

We are happy to see some of our customers running (and hopefully testing) beta versions of macOS and we saw adoption quickly ramp up after release day, reaching nearly 20% of total macOS endpoints by early October. Earlier, we briefly covered reasons for testing your crucial applications with beta versions of macOS, so let’s talk about details.

By testing an upcoming version of macOS running your organization’s essential applications and identifying issues you can:

  • Provide early warning to your internal IT team and formulate an adoption policy
  • Engage a vendor about an issue related to an upcoming OS version
  • Provide feedback to Apple about issues unique to your IT environment

How Should I Test?

A macOS beta testing environment can be as simple as using virtualization software like VMware Fusion or Parallels Desktop to install the beta macOS version in a virtual machine. Once the beta macOS version is installed, you can then perform standard installations of your organization’s crucial software and assess the outcome:

  • Does the software install at all?
  • Does it run at first launch?
  • Does it run but lack certain features?
  • Does it run and work as expected?

If your organization uses endpoint management tools it is important to test those as well:

  • Do required management agents install?
  • Does the management agent check in and retrieve configuration?
  • Do the configurations apply as expected?
  • Is the management agent able to install/update applications?

All of these outcomes will allow you to keep a running checklist during the beta period while you file bug reports with Apple or vendors. After initial testing, it is important to update the macOS beta version whenever Apple releases a new build. Next, repeat the testing again to check if bugs were fixed or regressions were introduced, always filing or updating bug reports as needed.

By knowing what parts of your crucial applications are incompatible and having filed support requests in advance, it will be much easier to have a realistic picture of when your organization’s users can apply the upcoming OS upgrade.

macOS Release Day - Now What?

If nothing is holding you back, great - Duo recommends upgrading everyone in your organization as soon as possible to ensure your users are protected and take advantage of platform security improvements. If there are any applications holding up an OS upgrade, you will at least already have opened issues and can apply pressure on the vendor now that the new macOS version is released.

It may also be possible to deploy the new macOS version to a subset of your users that are not impacted by a third-party application’s incompatibility. The most important part will be that you already know well ahead of time and are not blindsided and sent scrambling for solutions on release day.


No matter how much effort your organization will be able to put into early testing of an upcoming macOS release, one thing should be clear: update as soon as testing is done and ensure that your users get subsequent macOS point updates and security updates as soon as possible as well. Security experts (hey, that’s us!) recommend it.