I’ve been reading the 2021 MSP Threat Report from Perch (a ConnectWise company). It has a great review of the evolving threats to MSPs and their customers from ransomware attackers this last year. What I like about this report that it puts a number of relevant factors into perspective. Why are MSPs a target? What do the attacks look like? Who are some of the groups that are behind these attacks? What do they want (doh)? How are MSPs responding, and how effective are these responses? And, of course, what should MSPs be doing to counter the ransomware threats.

You can find the report here:

https://www.connectwise.com/resources/ebook-2021-msp-threat-report

Here are a few of the take-aways that I found interesting:

MSPs represent a valuable target. Why is that? Well, it turns out that MSPs are the gateway to a lot of end customers. They call this the “Buffalo Jump”. If an attacker can compromise an MSP they can get downstream access to all of the MSP’s customers. Based on some industry averages Perch estimates that an MSP an its customers represent a $2 BILLION opportunity. Yeah, that’s Billion with a “B”. The attacker expects to collect a ransom payment from the MSP and from each of the MSP’s end customers. The financial incentives to attack and MSP are huge.

As we know from recent experience the MSPs who have been attacked were surprised by the event. In many cases the MSP systems were not compromised, but the software they used to manage their business became the path to the compromise. A so-called “supply chain” attack. However, the supply chain attack does not cover all of the MSPs who encountered problems – many experienced routine phishing attacks and credential compromises. But the multiplier effects of the supply chain attacks stretched the resources of many MSPs.

The characteristics of a ransomware attack are pretty well known now. The common sequence of events of a ransomware attack are:

• Infiltration – access to the MSP and their end customer.
• Planting malware on breached systems.
• Exfiltration – steal copies of the data to the attacker’s server
• Poisonous Encryption – deny you access to your data and systems using a secret key.
• Extort the ransom – usually through cryptocurrency payments.
• Release of the hostage – decryption of your hostage data (if you are lucky).

While theft of data is common in traditional data breaches, the Exfiltration step is relatively new in ransomware attacks, and this is where many ransomware defenses fail. The MSP and the end customer may be able to restore systems from backups, but that won’t stop the extortion attempt. The ransomware attacker now has your sensitive data and threatens to release publicly it if the ransom payment is not made. The release of sensitive information can be devastating to MSPs and to their end customers. The threat is real and substantial. You need a backup and restore strategy, but it won’t protect you from the threat of the release of sensitive data.

What can you do?

The Perch Threat Report does not discuss this, but you do have tools to protect against Exfiltration. You have the ability to encrypt your data before the attacker with your own secret key. And that is what I call “Defensive Encryption”. You must encrypt your sensitive data first. The attacker can’t use the Exfiltrated data against you if they can’t read it. This is where encryption becomes you friend. Defensive Encryption renders Exfiltration useless by denying the attacker the ability to extort the MSP and the end customer. You still have to restore from backup, but you are in a much stronger position to defeat the extortion attempt.

There is a lot to like about the 2021 Perch Threat Report. It is concise but at the same time covers a lot of ground. I think this is an excellent report to share with upper management in your company. If you are an MSP you can share this with your end customers to help get them motivated.

MSP Note:

If you want to move forward with Defensive Encryption we have a solution you are going to love. Proper encryption key management is crucial to an encryption defense, but MSPs can be put off by the cost of key management systems. We’ve solved that problem. More here:

https://info.townsendsecurity.com/msp

Patrick

Every now and then something completely unexpected happens that changes your life. No, I’m not talking about the COVID pandemic - that’s a completely different story. What happened for me is that in the course of my work in business development of our key management server (KMS), I met the CEOs of two different Managed Service Providers (MSPs) and they welcomed me into their world. With grace and patience, they helped me leave behind my preconceived notions about software sales and introduced me to how their world works. Neither of these two CEOs were obligated to mentor me and to give me their time, but I am so grateful that they did. It opened a new vision for me and our team here at Townsend Security.

If you work at an MSP firm, I hope you will read on. I will tell you how I turned my lessons into real benefits for the MSP.

Managed Service Providers are varied in what they do, but at the core of their business is the desire to provide IT expertise, hosting facilities, business continuity and disaster recovery, and lots of other IT services to small and large organizations. They do everything from fixing user PCs to deploying top-end servers, security, and cloud services. Expertise is at the core of the value they provide to organizations. During the COVID crisis, they are on the front lines of trying to help everyone migrate to work-from-home and they are trying to secure that environment.

They are just some of the quiet, hidden heroes who don masks and rush into data centers and offices to keep us all operational. They provide great value to organizations especially in the current crisis. These MSPs taught me about their business and about the difficulties they have with key management vendors. In a time when security is top of mind for their customers, they struggle with a KMS industry that is stuck in the past. We were definitely one of those. As we talked, the light came on for me. All of the problems they were having with KMS vendors were problems that we could solve! All it took was a commitment from us, and a change in our business practices.

Here are some things I learned from my MSP CEO mentors:

• Their businesses run on a usage-based model. For example, they might host a VMware environment for an end customer and charge them on the basis of the number of Virtual Machines (VMs) or vSAN storage they manage on a monthly basis. They provide immediate, on-going value to their customers and they prove their worth on a day-to-day basis.
• They deploy third-party software solutions to help them accomplish their mission. They prefer to use software solutions that match their business model. For example, some of the common backup solutions like Veeam can be deployed by MSPs on a per-month, per-VM basis. It’s great when an MSP can deploy these types of solutions on a usage basis. It is how they run their business and greatly reduces their risk. KMS vendors are not helping.
• MSPs live in a complex technical world, and they have special needs from their software vendors. They probably deal with more technical complexity than any other IT segment. Hardware, software, Windows, Linux, security, networking, cloud, smart phones – where does it end? This means they need software solutions that are easy to install, deploy, manage and report on.
• An MSP deals with a lot of software “vendors”. What they really need are software
  “partners”. A software vendor sees the MSP as a resource (money) extraction
  opportunity. A partner is someone who saddles up and goes into battle with you. With a partner, you will either win together or lose together. This is an incredibly important distinction to the MSP, and a really big challenge to the software vendor.
• The MSP needs more than a software solution from a partner. With all of the complexity of the services an MSP delivers, the MSP needs help from the software partner to sell the solution, to support the solution, and to be a trusted advisor. Can the software partner help with sales collateral? How about with joint sales calls? Can we do joint webinars and podcasts that help build confidence in customers and potential customers?

Here at Townsend Security we live in the world of data security. We have encryption and key management solutions to protect data at rest. We have a number of MSP customers. Before I had the conversation with our MSP mentors, we approached each of our MSP customers the way any legacy software company would. We offered the basic perpetual and subscription licenses. We have always been very price competitive, but it was basically a take-it-or-leave it approach. We charged for each key manager that we sold.
We were a perfect example of the “vendor” problem the MSP experiences. So, we set out on a journey to see if we could align our business with MSPs and become the “partner” they want and need. It meant changing a lot of our assumptions and business practices. You will know when you have a true partner when they lean in with their marketing and technical teams to make you successful. Our goal is to be that partner!
Here are some of the things we’ve done:

• Adopted a Pay-As-You-Go model for MSP partners. We now charge a very small monthly fee for each encrypted VM or database. Gone are the perpetual and annual subscription licenses. Scale up or scale down as you like. We get paid when you get paid. Full stop.
• Dropped all upfront fees or annual minimums. We are aiming for perfect cost and
  revenue predictability for your MSP business.
• Stopped counting the number of key management servers the MSP runs. The MSP
  deploys key servers in the way that makes sense. Multiple physical hosting sites, on-premise deployments, Disaster Recovery as a Service (DRaaS), encrypted storage? We don’t care, we are all in.
• We trust the MSP to deliver their services and expertise on their hosting or cloud
  platform, and on their customer’s premises. MSPs conduct their businesses in a variety of ways. If we achieve true partner status you will feel that we are fully behind you and support you and take the risks with you.
• We train the MSP on how to deploy our solution. We have video, on-line
  documentation, and one-on-one training to help you get up and running quickly. We don’t charge for training; we just lean in to help you get the job done.
• We support the MSP with 24/7/365 business interruption support program at no extra charge. Support is built right into the low monthly fee.
• Provide sales support by doing joint customer calls, answering security questions, and providing guidance on meeting compliance regulations. We don’t charge for helping you close a sale; we will win the deal together.
• Provide sales collateral that includes sell sheets, educational material, joint webinars and podcasts, and much more. We don’t charge for sales and marketing collateral.

I feel like I’ve been on a fast learning track and have gained some great new friends. They are sharing with us what they need, and we are leaning in to help them be successful. It is an immensely rewarding experience.

Here is what one of our MSP customers said:

“You said the magic words of MSP and Low cost, consumption based! We’ve struggled to find a KMS solution we can properly price and sell to our customers to do VM encryption. Solutions like XXXXXX are prohibitively expensive. Your low cost per encrypted VM per month is very reasonable. I’m glad those MSP’s helped you understand our market and that you were able to see the opportunity. You NEED to be marketing this. You’re solving a problem that MSP’s a) don’t think they can afford to fix, and b) are just ignoring the compliance of because it’s “too hard and too expensive.” I highly encourage you to get the word out through marketing to MSP’s. Thank you, Patrick. You made my day.”

If you are an MSP we would like to “make your day.” You can start your journey here. 

Evaluations of our Alliance Key Manager are available at no charge. We provide technical
support through the evaluation at no charge. Let’s do this together!

Patrick

In our practice here at Townsend Security we engage with a lot of startups and mature ISVs who are trying to grow their business and customer base, leverage their technologies into new opportunities, and grow or migrate to the cloud. We know how difficult it is to start and grow a company, and what a wide set of business challenges have to be overcome. Our hats are off to every entrepreneur who has created a successful company, and every ISV who has kept it going!

I want to share a few thoughts on some pitfalls that can damage your ability to grow your company with a focus on the encryption of sensitive data. Too many promising companies flounder because of poor security implementations, and failing to get encryption right can lead to lost opportunities - maybe even the loss of that breakout sale you need to land a global company. Some early thought and planning about data security can help you weather your migration up the food chain and avoid such losses.

Number 1: Failure to encrypt sensitive data

The single biggest failure of data security is not doing it at all. Even in this age of massive public data breaches, and the damage that they do to companies of all sizes, most startups and ISVs are not implementing encryption of sensitive data. When product managers and developers work on their next big idea, they focus on exciting features in their product and often ignore the work it takes to implement encryption. They instead rely on access control lists and other mechanisms to protect data. These are, of course, important things to do. But the failure to encrypt sensitive data leaves a big hole in your security strategy.

What can go wrong if you haven’t implemented encryption? LOTS !!!

• The publicity around a data breach can tarnish your reputation and kill opportunities.
• The lack of encryption may cause compliance regulation failures making it impossible to enter new markets.
• You may not be able to pass a security review of your software by that large global Enterprise.
• You may not be able to enter government channels where encryption is a mandate.
• If your customer experiences a data breach you may encounter substantial litigation costs that damage your financial resources and delay critical development.
• You may fail to secure that next round of funding when an investor discovers the security gaps in your product.

When these kinds of events damage your ability to grow your company, it can be hard to mitigate them in a timely fashion. And you often won’t know about these dangers until you get fairly far down the road with your business plan.

Number 2: Failure to get key management right

For startups and ISVs who DO understand the need for encryption of sensitive data, the next biggest pitfall is the failure to protect encryption keys properly. Almost every database that supports encryption also supports the ability to protect the database encryption keys with a key manager. But that doesn’t mean that good key management is the default! In most cases the default database key management option is to store the encryption keys on the same server as the sensitive data. Sometimes the database will even store the encryption key locally and in the clear! So getting encryption key management right is critical to your security strategy. It won’t help to have encryption of your data enabled, and then have a cybercriminal steal your data along with the encryption key.

Related to key management here are some things to look for when you consider databases for your application:

• Does your database have built-in encryption? Relying on third-party encryption solutions at the file/folder level will certainly cause deployment and scalability problems.
• Does your database support integration with third-party key managers? If there is no easy way to integrate proper key management into the database, this will also cause deployment and technology delays.
• Does your database support open standards for key management? For example, the Key Management Interoperability Protocol (KMIP) defines how applications like databases can easily integrate a key manager.
• Does your database support key management failover? Remember that protecting encryption keys with a key manager also brings along the question of high availability and failover.

HINT:

If you are a startup be sure to choose a database that supports built-in encryption and proper key management. You have lots of good choices in both commercial and open source solutions. So go with a database with native, built-in encryption and key management!

Number 3: Failure to get FIPS 140-2 right

There are important standards and certifications for key management solutions. The most important of these is the National Institute of Standards and Technology’s (NIST) FIPS 140-2 standard. In addition to being a published standard, there is also a validation process for key management systems. The standard, and the validation to that standard, are critically important to your data security strategy. All professional key management solutions have been validated to the FIPS 140-2 standard and you should be sure to deploy a validated key management solution. This will help you avoid failing a security audit by that important new customer!

In addition to ensuring that your key manager is validated to FIPS 140-2, be sure that the entire key management solution is validated. There are many cases where the encryption library alone is validated to FIPS 140-2, but the key management application is not. It is good to have validated encryption, but that is just the start! Encryption key management has its own validation points and you will need both.

Snake Oil Alert !!!

Unfortunately, there are some key management solutions that make unwarranted claims about FIPS 140-2 compliance and validation. Here are a few warning signs to look for when you evaluate a key management solution:

• A vendor makes compliance claims, but there is no validation. Some vendors claim to be “FIPS 140-2 compliant” but in fact have never completed a FIPS 140-2 validation. Security is hard, and unsubstantiated claims should be a red flag.
• A vendor claims FIPS 140-2 compliance, but the validation is “in process”, but not complete. A security product can be “in process” for many months or even years. A claim of FIPS 140-2 compliance without actual completion should also be a red flag.
• A vendor makes some claims of FIPS 140-2 validation, but research shows that the key management solution was not validated by that vendor.
• A vendor makes a claim of FIPS 140-2 compliance, but the solution is only compliant when backed by a third party validated key management solution. In this case the vendor solution itself is not validated, but relies on the validation of another solution. You may be fooled into thinking that the solution itself is compliant when it is not. Especially watch for this pitfall with open source solutions.

You can always check a vendor’s claims of FIPS 140-2 compliance. Ask for the NIST FIPS 140-2 certificate number, and then Google it. NIST makes the validation certificate available to the public on their website. Copy and paste this into Google search:

NIST FIPS 140-2 certificate number 1449

That was easy!

Number 4:  Failure to make encryption and key management easy and invisible

Now that you are on the road to getting encryption and key management right, it is important to also make it easy and invisible. Your customers have a lot on their agendas, and becoming a key management expert is probably not one of them. So even if you follow the above advice and implement encryption and key management, do your customers a favor and make key management easy. The best way to do this is to bundle a key management solution into your product, and make key management automatic. You can still enable the configuration of an external key management system (some customers will want this), but you can really make it easy for most of your customers if you automate the key management tasks.

Automating key management is a great competitive advantage! One of our partners in the archival and backup space implemented this strategy and make great competitive wins on this feature alone! Their message was simple:

“We have encryption and key management. It is FIPS 140-2 validated. It is completely automatic so you don’t have to spend time fiddling around with a complex key management system.”

This strategy won them a lot of competitive deals and it was easy to talk about - and it shortened the sales cycle.  Of course, be sure that your key management solution supports this type of integration and automation!

Number 5:  Failure to segment customer data

As you move to the cloud and create shared, multi-tenant SaaS solutions, be sure to plan for and architect data segmentation into your solution. You will encounter large customers who will not want to have their data in the same space as other customers. They will want the additional security of segmenting their data into a virtual private cloud. With planning, your technical team can meet this kind of requirement, and help you close that very large deal.

Of course, a data segmentation plan requires a key management segmentation plan. For the same reasons customers want to segment their data, they don’t want to share key management with other customers. And they want to maintain full control of the key management implementation. So be sure to plan for customer-specific deployments of encryption key management and failover key management servers. A properly implemented data and key management segmentation plan will even allow for on-premise deployments that are “cloud ready.”

Number 6:  Failure to develop new market opportunities

Think about Amazon (the company) for a moment. At one point in their history they were an online bookstore. Today the company is very different. Amazon first leveraged its technologies to sell all kinds of products, and then created Amazon Web Services (AWS) to enable all of us to benefit from cloud technologies.

Are you thinking like Amazon? If not, you might be missing some big opportunities. Now that you have secure applications, are there lateral opportunities or technology licensing opportunities available to you? When you approach new opportunities and partners, don’t be afraid to talk about security. Regardless of what you’ve heard:

SECURITY SELLS!