Has Microsoft Office 365 Beat Phishing?

Roger_Grimes_400x400By Roger A. Grimes, KnowBe4's Data-Driven Defense Evangelist.

Microsoft recently announced a big update to their Microsoft Office 365 (O365) anti-phishing technical capabilities. According to Microsoft, their “miss phish catch rate” is down to near zero, beating all other O365 anti-phish competitors by orders of magnitude.

Has Microsoft Office 365 (0365) got phishing beat? Well, I wouldn’t get rid of your security awareness training just yet.

I think I have a interesting perspective. Until recently, I worked for Microsoft for over a decade and I still love the company, its people, and products. Microsoft security really is the best in the world. I’m also a long-term 0365 user for my private side work company. I’m now the data-driven defense evangelist for KnowBe4, the world’s leading security awareness training vendor.

I moved from Microsoft to KnowBe4 because I wanted to dedicate the remaining years of my computer security career to making the biggest impact in computer security possible. This isn’t hyperbole. Social engineering and phishing have been the number one way that malicious data breaches happen for over a decade. If you want to have the biggest impact minimizing computer security risk you might as well jump into the lion’s den. And today, that means fighting social engineering and phishing.

From within Microsoft, I saw how hard Microsoft tried to stop phishing emails for its 0365 customers. Microsoft didn’t like that a majority of its large 0365 customers felt the need to purchase additional email protection. Every third-party anti-phishing purchase was a sign that Microsoft, itself, wasn’t doing enough to stop phishing.

It took years, but if you trust Microsoft’s data (and I have no reason to distrust it right now), it looks like Microsoft 0365 has some pretty solid anti-phishing results. According to their own data, phishing emails that escape detection and prevention are near zero percent. In fact, it’s hard to tell if their graphed data is saying exactly 0 percent or just above 0 percent. It’s that close, graphically.

The question some observers might have is if security awareness training is still worth the cost if Microsoft has “beat” phishing?

The Answer

Microsoft’s anti-phishing detection rate is non-zero. It might look like zero or near-zero on a graphical chart, but the fact is that there are phishing emails that escape detection and prevention. As long as that is true, you will need security awareness training. Microsoft says they examine 400 billion emails each month. If the detection bypass rate is just .0005 (which seems to implied by the graph), that means 200 million phishing emails still getting by. That’s definitely not zero.

In my personal use of 0365, I get somewhere between 50-200 personal emails a day. On average, I still have 1 to 3 phishing emails getting to me each day. Not only are they still arriving, but their sophistication and targeting are up. I still get phishing attempts from banks and other companies I don’t do business with, but it seems the phishing attempts that seem to have advanced knowledge of the companies and services I do do business with seem to be increasing. I’ve just assumed they are getting clues from my social media postings and other information leaks.

Without the appropriate training, I would be more susceptible to responding incorrectly to a phishing email. How would your employees do?

Increasingly, phishers are compromising internal or third-party email accounts with whom you have much history and trust. They lurk, learning all the relationships they can take advantage of. And when the time is right, they strike, sending a phishing email which comes from a person you frequently correspond with, with a request that is just slightly different, or possibly no different, than what you are expecting. This particular class of spear phishing is becoming more common.

For example, people buying houses now have to be aware of compromised mortgage agents whose email has been taken over by a phisher, who then sends a bogus request for the closing payment to the buyer to wire the money to another bank. The house buyers were expecting the wire transfer request, and it appears from the email account of the person they were told to expect the wire transfer request from. It appears legitimate in every way, including the amount of the money they were told to expect to have to bring to closing, with the only changed details being the bank they are wiring the money to. If the unsuspecting home buyer wires the money to the wrong bank, they are often permanently out of the money (if not the house they were wanting to buy unless they can and want to pay the closing costs again).

Like mortgage closing payment fraud, all spear phishing is increasing in sophistication. It is coming from people and businesses you trust. Regardless of what any vendor tells you, their anti-phishing miss rate will never be zero. I’ve been in the computer security business for over three decades. And each year, I hear from some vendor how they finally have phishing beat. And each year, it seems to get worse. Despite every vendor’s best effort, it seems more phishes are hitting my inbox than ever before, across all email vendors, Microsoft and others. I’m definitely a skeptic when it comes to anyone saying that they will beat phishing.

What is your experience?

But let’s say that Microsoft, or another vendor, did significantly improve it’s phishing-blocking rate. Let’s even go so far as to say that they were successful enough to get to a zero miss rate. Could you skip security awareness training then?


Even if a vendor solved the email phishing problem at work, it doesn’t stop email phishes from getting to your employees. Most employees have personal email accounts, and if that service has a non-zero phishing rate, then you and your company still need to educate those employees with security awareness training. Just because you solve the problem at work doesn’t mean the problem is gone.

But let’s go even further and assume that your employee either only has your perfect email solution or has the same zero phishing protection on their personal accounts. Could you do away with security awareness training then?


This is because email phishing is just but one component of phishing and social engineering. Increasingly, phishing is moving to non-email methods, including social media, web-based, mobile devices, and voice phone calls, and those phishes are more likely to be successful than the traditional email methods. And today’s security awareness training is about far more than phishing alone. 

Security awareness training is absolutely necessary regardless of the success rate of traditional email phishing. Phishing detection may improve, but it will never be non-zero, and long as it is, you’ll need to prepare organization and its employees.

How many of your users would click on a convincing phishing link? Run a Free Phishing Security Test and find out!

pst30_V2 Here's how it works:
  • Immediately start your test for up to 100 users (no need to talk to anyone)
  • Customize the phishing test template based on your environment
  • Choose the landing page your users see after they click
  • Show users which red flags they missed, or a 404 page 
  • Get a PDF emailed to you in 24 hours with your Phish-prone % and charts to share with management
  • See how your organization compares to others in your industry 
Go Phishing Now!

Topics: Phishing

Subscribe To Our Blog

Free Phishing Security Resource Kit

Get the latest about social engineering

Subscribe to CyberheistNews