MSPs and Certified VMware Cloud Providers deliver a critical set of services and infrastructure to small and medium size organizations. They deploy, manage, secure, and backup the VMware environment for their end customers. That mix of expertise and cloud or hosting infrastructure positions them to fill an important role in the IT services industry. Customers are rightly concerned about the security of their sensitive information in VMware Virtual Machines (VMs) and Virtual SAN (vSAN) and want to be sure that this information is encrypted. The VMware platform provides the ability to encrypt VMs and vSAN, but needs the addition of an Enterprise Key Management System (KMS) to protect the encryption keys. The following sections describe the encryption implementation of VMware, the mechanism for protecting encryption keys, and the unique business challenges facing an MSP and VMware Cloud Providers.
The VMware story began in 1998 when five forward-thinking technologists launched an innovative virtualized computing solution. Shortly thereafter, it was the first commercially successful company to virtualize x86 architecture. Today VMware is the recognized leader in on-premise computing virtualization. VMware’s applications extend across management, networking, monitoring, administration and security. VMware’s enterprise software hypervisor for servers, VMware ESXi, is a bare-metal hypervisor that runs directly on server hardware without needing an additional underlying OS. Organizations have achieved immense cost, security and administrative benefits through the deployment of VMware for their IT infrastructure.
While the benefits of VMware are undeniable, the proper deployment and management of VMware requires specialized expertise. While reducing the overall cost of IT hardware through virtualization, there remains the need for hardware and data center investment. MSPs and VMware Cloud Providers are helping customers in both areas - providing expertise and hosted data center infrastructure.
VMware Cloud Providers
VMware Cloud Providers are specialized partners in the VMware partner ecosystem. Not only do they bring expertise to the deployment and management of VMware in hosted and cloud environments, but they maintain a special partner relationship with VMware that involves certification and the ability to cost effectively license VMware platforms. This special relationship as a VMware partner must be revalidated on a periodic basis and this helps build confidence by end customers in their services. You can find the Certified VMware Cloud Providers on the VMware website.
While the number of VMware partners is quite large, only about 180 VMware partners have achieved VMware Cloud Verified status. You will find both small, regional partners as well as global partners.
VMware vSphere encryption was first introduced in vSphere 6.5 and vSAN 6.6; enabling encryption of virtual machines (VMs) and disk storage (vSAN). It only requires the vCenter vSphere Server, a third-party Key Management Server (KMS), and ESXi hosts to work. It implements standards-based AES encryption, uses the open KMIP standard for the key management interface, is highly performant, and is easy to deploy.
In our increasingly insecure cyber world, VMware understands the critical nature of robust security solutions, including encryption capabilities. We need strong encryption and key management solutions that run natively in our virtual environments to meet compliance regulations and security best practices.
The implementation of VMware encryption and the key management interface now span several releases of VMware and have earned the trust of customers and partners. If you’d like to first learn the fundamentals of encryption and key management before diving in, please view The Definitive Guide to Encryption Key Management Fundamentals.
Encryption of Virtual Machines (VMs) and Best Practices for VMware Cloud Providers
With vSphere 6.5 and above, an MSP can encrypt their customers' VMs to help protect sensitive data-at-rest and to meet compliance regulations. vSphere encryption allows you to encrypt existing virtual machines as well as encrypt new VMs, right out of the box. Additionally, vSphere VM encryption not only protects your virtual machine, but can also encrypt your other associated files. Organizations typically have mission-critical information in VMs. This means that getting encryption and key management right the first time is paramount.
VMware provides excellent documentation on the configuration, deployment and best practices for encryption. Here are a few highlights of best practices as you deploy encryption for your customers:
- Do not encrypt any vCenter Server Appliance VMs. These are vital to the functioning of VMware and should never be encrypted.
- Do not edit VMX files or VMDK descriptor files as they contain the encryption bundle information. Any changes may make the VM unrecoverable.
- Always designate a high availability failover key manager in your KMS cluster. If your primary key server goes down with no failover key server in place, your encrypted VMs will be unable to be decrypted until the key server is recovered.
- Once you name your key management server (KMS) cluster, do not rename it. If you change the name of the KMS cluster the ESXi host will be unable to find the KMS and a VM that is encrypted with a key encryption key (KEK) from that KMS will be unable to be decrypted.
- Once you encrypt a virtual machine, you cannot relocate the VM to a host that does not have the key ID information. Only a ESXi host with the key ID information for that VM can properly locate the encryption key for decryption.
At Townsend Security, we provide additional technical support and guidance to our MSP and VMware Cloud Provider partners to ensure successful deployments of encrypted VMs.
Encryption of Virtual Storage (vSAN) and Best Practices for VMware Cloud Providers
VMware’s Virtual SAN (vSAN) is powerful hyper-converged infrastructure that offers you greater performance and high scalability. vSAN encryption is easy to deploy but does have a few best practices in order to avoid interruption of service. As an MSP, before you begin your vSAN encryption project, consider these VMware best practices:
- Do not deploy your KMS server on the same vSAN datastore that you are encrypting. This will encrypt your key managers and in some cases render them useless in recovery scenarios.
- Do not attempt to encrypt your witness host as they do not contain any sensitive data. They only contain metadata concerning other vSAN clusters and do not need to be encrypted.
- Encryption can be CPU intensive. For vSAN encryption on Intel hardware, make sure AES-NI is enabled in BIOS. It can significantly improve encryption performance.
- You should ensure that your Core dumps are encrypted. They can contain sensitive information such as encryption keys.
- When you decrypt a core dump, you should handle it as if it contains sensitive information. Core dumps may contain encryption keys either for the vSAN host and/or the data on it.
One way that our VMware Cloud Provider partners are helping their customers protect data is to deploy common commercial and open source databases on encrypted vSAN storage. PostgreSQL, MariaDB, MongoDB Community Edition, Oracle Database and many others can be secured at rest using encrypted vSAN and VMware provides excellent guidance on how to do this. Using encrypted vSAN for your databases can help your customers avoid expensive software upgrades.
Encryption Through Virtual Trusted Platform Module (vTPM)
Operating systems like Microsoft Windows and others have implemented support for the Trusted Platform Module (TPM). TPM provides additional security to the operating system encryption support by protecting the master encryption key in the underlying hardware. While TPM works well in traditional server settings it does not work well in a VMware virtualized environment. One of the benefits of VMware is independence from the underlying hardware, and the ability to move workloads across hardware servers, remote nodes and the cloud. VMware has solved the problem with Virtual TPM (vTPM). VMware customers can now deploy vTPM from VMware and get encryption key protection through the same vSphere KMS Cluster configuration used to protect VMs and vSAN.
Because support for encrypted VMs is easy and scalable, VMware Cloud Providers rarely need to deploy vTPM. However, if an end customer wants vTPM protection, it is available and fully supported through the vSphere KMS Cluster configuration.
Industry Standards for Encryption & Key Management
MSPs and VMware users are concerned with the implementation of encryption and key management.
One important standard is from National Institute of Standards and Technology (NIST): NIST FIPS-197 which defines and validates AES encryption. Why is verifying that your data is secured with AES encryption important? AES is an internationally recognized standard for encryption and VMware has validated its encryption to this standard. All major compliance regulations recognize AES encryption for protecting sensitive data.
Encryption Key Management
FIPS 140-2 certification ensures that the key management software has been tested by third parties to meet the highest standards in key management technology, so you can establish strong key management. For VMware customers, FIPS 140-2 compliant encryption and key management are a key defense for data security. Proper key management is required to implement VMware encryption.
Key Management Interoperability Protocol (KMIP)
VMware allows users to manage encryption keys using a third-party key management vendor through a standard key management protocol called Key Management Interoperability Protocol, or KMIP. All of VMware’s KMS Certification tests contained in KMS plug-ins verify that the vendor’s KMIP KMS works with the vSphere VM encryption feature and encrypted vSAN virtual disk. Testing consists of verifying the correct behavior of a KMS and ensuring that it does not introduce undesirable impacts on the operation of the system.
VMware Customers, Compliance REgulations, & Business Secrets
VMware partner and MSP end customers are concerned with protecting sensitive business secrets and meeting compliance regulations. Here are a few of those regulations that are of concern:
Health Insurance Portability and Accountability Act - HIPAA
The Health Insurance Portability and Accountability Act and Health Information Technology for Economic and Clinical Health Act (HIPAA/HITECH) outlines data security regulations for the healthcare industry. While the HIPAA/HITECH does not specifically require encryption of sensitive data, a backdoor “safe harbor” mandate states that if a healthcare organization or one of its Business Associates (BA) does experience a data breach, and Protected Health Information (PHI) is not obscured using encryption or some other method, then that organization will be heavily penalized.
Organizations can reduce the complexity and cost of HIPAA Security Rule compliance by replacing traditional non-integrated products with integrated solutions. To further address this gap, VMware, together with the VMware partner ecosystem delivers compliance-oriented integrated solutions, enabling compliance by automating the deployment, provisioning, and operation of regulated environments. In this way, VMware provides the solution reference architecture, HIPAA Security Rule specific guidance, and software solutions that businesses require to achieve continuous compliance, along with speed, efficiency, and agility for their applications.
California Consumer Privacy Act (CCPA)
If your end customers collect data on people or households who are in California, and meet the minimum criteria, and are not explicitly excluded, they must meet the requirements of the new law. Notice, this does not just apply to “California citizens”, but people who are in the state at the time of data collection. You are not exempt if your organization resides outside of California. If your customer collects data on people in California, they should assume they are covered by the law. Under the CCPA the only way to provide protection against class action lawsuits is to encryption your sensitive data and to use proper encryption key management.
Payment Card Industry Security Standard (PCI DSS)
VMware meets the standards of the PCI DSS, which was developed to encourage and enhance cardholder data security and facilitate the broad adoption of consistent data security measures globally. For VMware users who need to meet compliance, Alliance Key Manager has been validated for PCI DSS in VMware by Coalfire, a PCI-qualified QSA assessor and independent IT and audit firm. Additionally, Alliance Key Manager for VMware can also help businesses meet other compliance regulations such as CCPA, HIPAA, GLBA/FFIEC, FISMA, etc.
VMware and GDPR
In response to escalating external and internal threats and uncertainty, lawmakers and regulators around the world have been strengthening their data security compliance requirements, implementing new legal frameworks and levying higher noncompliance penalties. This places organizations at tremendous risk for compliance violations, along with the resulting fines and remediation costs. On May 25th, 2018, the European Union made securing citizens’ data an even bigger challenge for companies doing business that involves handling their citizens’ data. That was launch day for the new European Union General Data Protection Regulation (GDPR),
Encryption and key management can help meet GDPR’s privacy requirements, as well as citizens’ right of erasure (right to be forgotten). While the EU does not mandate that all organizations encrypt sensitive data, there is an exclusion for subject data breach notification and financial penalties for those organizations that use encryption and other security methods to protect the data. Thanks to VMware’s wide-ranging focus on security, implementing encryption and key management tools will help users meet requirements for GDPR.
One of the key values that an MSP and VMware Cloud Providers bring to their customers is a reliable and resilient infrastructure to protect their customer’s ongoing operations. VMware infrastructure is crucial to this effort and provides world-class business continuity. This includes support for encrypted VMs that may move across multiple vCenter nodes. MSP and VMware partners know that a reliable and resilient key management solution is also critical to this effort. A key management solution should be able to meet or exceed customer expectations for business continuity. MSPs and VMware Cloud Providers need to be able to deploy a key management solution that provides real-time failover and integrates with vSphere KMS Cluster configurations. It is important that key management systems fully support role-swap system recovery operations and this involves the dynamic change in roles between a primary and secondary key server. When a primary key server is unavailable a secondary key server automatically steps in to serve various encryption key functions. In this situation it is important that the secondary key server now becomes the primary key server for a period of time. New encryption keys may be created, the status of existing keys may change, and access policies may also change. A good key mirroring architecture will allow for these changes to migrate back to the original primary key server when it becomes available. This is the central feature of Active-Active mirroring implementations.
MSP and VMware Cloud Providers & Encryption Key Management
As the landscape for delivering VMware cloud services rapidly changes to a new usage model, MSPs and VMware partners find themselves struggling with traditional key management infrastructure. Legacy key management systems involve fixed up-front costs, annual maintenance and support contracts, complex and rigid deployment tasks, and inflexible infrastructure. This makes it difficult for MSPs and VMware Cloud Providers to manage costs across a growing number of customers and to scale their businesses effectively.
Further adding to the pain is the difficulty of managing complex customer relationships that span on-premise, hosted, and cloud environments. Trying to deploy a key management solution that bridges the customer on-premise VMware environment with the VMware partner’s cloud platform again involves complex and expensive licensing costs. The KMS industry has failed to keep up with this new paradigm for delivering VMware services.
Townsend Security's VMware Cloud Provider Partner Program
The Townsend Security MSP and VMware Cloud Provider partner program is designed to match the partner’s business model with usage-based key management and flexible deployment options that eliminate licensing headaches, and which empowers the MSP and VMware Cloud Provider to scale their business. There are no upfront costs or annual fees, no annual minimum license fees, and no restrictions on KMS deployment in the cloud or on customer premises. Townsend Security works with partners to design and implement a pricing plan that matches the way they do business. The result is a highly scalable and predictable deployment of VMware encryption and key management with no administrative overhead.
MSPs and VMware Cloud Providers also benefit from the ongoing certification of Townsend Security’s Alliance Key Manager with VMware to ensure customer confidence and easy integration. The MSP and VMware Cloud Provider benefits from the advanced security posture and certifications of Alliance Key Manager in a variety of compliance environments.
Townsend Security provides key management technology, no cost training, 24/7 technical support, and an easy partner program with no unpleasant surprises. It is easy to get started at this link.
Alliance Key Manager
“A very cost effective solution in terms of performance, manageability, security, and availability. As a result, my company
was quickly able to implement full database encryption leveraging the AKM as our key management solution in weeks. Comparable solutions could have taken months.”
The solution offers unparalleled security, flexibility and affordability for all users of VMware. With no client-side software to install, customers can deploy Alliance Key Manager and install the PKI certificates on the database server to easily begin retrieving encryption keys.
Alliance Key Manager is FIPS 140-2 compliant and in use by over 3,000 organizations worldwide. The solution is available as a hardware security module (HSM), VMware instance, and in the cloud (Amazon Web Services, Microsoft Azure, and VMware vCloud). Townsend Security offers a 30-day, fully-functional evaluation of Alliance Key Manager.