Townsend Security Data Privacy Blog

Fixing Encryption Key Management Audit Failures in Microsoft Windows C# Applications

Posted by Patrick Townsend on Dec 23, 2015 11:00:00 AM

It seems like every week I talk to a new Microsoft Windows customer who has just failed a security audit because they are not handling encryption keys correctly in their Microsoft applications. I hear an assortment of descriptions like this:

Encryption Key Management for Microsoft .NET We’re doing encryption, but the encryption key is stored in a table in the database.

We’re doing encryption, but the key is burned in our C# code.

We’re doing encryption, but the key is stored in a flat file protected with a password.

We’re doing encryption, but our encryption key is weak - it’s just a password

These Microsoft customers are trying to do the right thing when it comes to encrypting sensitive data, they just did not get the proper security controls in place for the encryption keys. And this is understandable, Microsoft does not provide very good guidance in this area, and it is probably the area where most organizations fail to get encryption right. When you do encryption, using strong encryption keys and protecting them properly is the hardest part.

You’ve made a big investment in your application built on the Microsoft C# .NET architecture, so what do you do now?

It turns out that it is not difficult to remediate this problem. You need a few things:

  • A good key manager built and certified to industry standards.
  • A Windows software library that is friendly to your C# application.
  • A developer who can implement some simple methods for key management.

Let’s look at how we help our customers solve these three challenges:

1) A key manager is easy. Our Alliance Key Manager solution is an easy-to-install and configure key management solution that runs as a Cloud instance (AWS, Azure), as a VMware virtual machine, as a network-attached hardware security module (HSM), or as a dedicated Cloud HSM. It configures in a few seconds and you have a fully functional, dedicated, FIPS 140-2 compliant key management solution. It even automatically creates unique encryption keys for you on the first boot.

2) The second thing you need is a good Windows .NET software library that makes retrieving the encryption key seamless. We provide that at no charge in the form of our Windows .NET Client. It installs on Windows in the usual way and is ready to add to your Visual Studio project. The Windows .NET Client software DLL handles all of the complexities for you. It makes the authenticated connection to the key server, audits all key retrieval activity, and caches the key for the best performance. Sample C# code shows you exactly how to do key retrieval from your code and is easy to implement.

3) The third thing you need is a good developer. With over 2 million Windows developers out there it is not hard to find one. Maybe this is you? Making the code change to incorporate key retrieval with Alliance Key Manager’s Windows .NET Client is really simple in C#. You just remove the logic of the burned in key or the logic to retrieve it from a file, add the Alliance Key Manager Windows .NET Client to your Visual Studio project, and put in code that looks something like this (see the sample code for the complete picture):

// Load the data to be encrypted
plaintext = Encoding.UTF8.GetBytes(data);
// Initialize for the client configuration file
SetClientConfiguration(keyService.ClientConfiguration);
// Retrieve the encryption key from the key server
key = keyService.GetSymmetricKey(keyName, instance);
// Perform encryption
using (var algorithm = new RijndaelManaged())
{
algorithm.Key = key.KeyBytes();
using (var encryptor = algorithm.CreateEncryptor())
{
ciphertext = encryptor.TransformFinalBlock(plaintext, 0,
plaintext.Length);
}
}
// Display the ciphertext (you would probably save it in a database)
Console.WriteLine("ciphertext: " + BitConverter.ToString(ciphertext));

The Alliance Key Manager Windows .NET Client code handles a lot of issues for you. Here are just a few:

Authentication
The Windows .NET Client performs the TLS connection to Alliance Key Manager and handles the full authentication sequence. You get a secure, mutually authenticated TLS connection without needing to handle that in your C# code.

High Availability Failover
You can define one or more Alliance Key Managers in a high availability failover group. If one key manager is not available due to a network failure or a key server failure, the client-side code will automatically use the next available key server. You can define a primary and multiple failover key servers, and you can mix and match cloud, virtual, and HSMs in your key management topology.

Performance
The Alliance Key Manager Windows .NET Client is designed for high performance. It will automatically securely cache encryption keys so that you don’t need to retrieve the keys more than one. If the key has been retrieved before in your current process it will be returned to you without a round-trip communications session with the key server. That gives you maximum performance.

PCI Compliance - Coalfire and PAG
Alliance Key Manager is FIPS 140-2 compliant (certificate 1449) and has been PCI validated by Coalfire, a top-tier QSA auditor. This means that you can achieve very rapid compliance with PCI and other compliance regulations without a lot of fuss. If you are under the gun to fix a compliance failure, you should start with a solution that provably meets the compliance regulation. Alliance Key Manager does exactly that. The PCI Product Applicability Guide is here.

Most C# applications that we see are well-modularized and the encryption logic is in one place, or a very small number of locations in the code. This makes the change to your code easy to make and easy to test. The complexity is handled for you in the Alliance Key Manager Windows .NET Client software.

You can find the Alliance Key Manager Windows .NET Client application on the supplemental download for the product. It is available at no charge and there are no client-side license fees. In addition to the installation package you will find full sample code and documentation.  For interested developers, we also offer a free developer tools.  As always, let us know if you have any questions.

Enjoy.

Patrick

Encryption Key Management for Microsoft .NET

Topics: C#, Microsoft, .NET, Encryption Key Management

Securing your IBM i (AS/400, iSeries) - Webinar Q&A

Posted by Michelle Larson on Dec 4, 2015 10:41:00 AM

Data security doesn’t need to be a challenge! 

Whether your data is stored within your database files or transmitted to your trading partners, using the right tools can mean the difference between meeting compliance and security best practices or suffering a data breach.  Webinar: Securing Data on Your IBM i

Townsend Security is well known for providing security solutions for a wide variety of platforms (IBM i (AS400, iSeries), IBM mainframe, VMware, Windows, Linux). Recently, our founder and CEO produced an excellent webinar on “Securing Your IBM i Using the Right Tools”. Many of our IBM i customers are under multiple compliance requirements including PCI, HIPAA, SOX, GLBA, and others which require securing data-at-rest and data-in-motion, real-time security event logging, file integrity monitoring, and two-factor authentication. The webinar covers encryption and key management as well as a number of our other auxiliary products specifically for the IBM i (AS400, iSeries) platform.

There were a number of excellent questions asked after the presentation, and the following is a brief recap of that Q&A session with links to additional information:

Q:  Can I use FieldProc to protect multiple fields in a file?

A:  The short answer is yes, encrypting multiple fields is a fully supported capability. FieldProc does allow, from both DB2 and from our solution, protecting multiple columns or fields in a DB2 file. You can define multiple fields and then enable FieldProc with one command on all those fields within our solution. People often ask if indexes can also be encrypted and yes, that is fully supported as well. There are a few limitations in legacy RPG applications, while with a native SQL application there are no limitations.

To learn more about Field Procedures on the IBM i, check out this blog:  5 Common FAQs About IBM i Encryption Using FIELDPROC 

Q: Can’t I just transfer my file from IBM i to Windows and then PGP encrypt it there?

A:  This is a great compliance question and comes up often with QSA Auditors. Yes, of course you can move a file from one machine to another using operations navigator. The problem is that it exposes data in the clear during the movement, either across the network if it is an unsecured connection or when the data lands on the other side of the transfer. Any of these exposures will likely result in an audit failure.

Good security practice, regardless of the platforms involved is:

  •   Encrypt at the Source
  •   Decrypt at the Destination

Make sure to securely move it in encrypted form and don’t let it get loose anywhere in between!

Learn more about the core components of a total encryption strategy in this blog: Secure Managed File Transfer and PGP Encryption

Q: How does key management work with older back-up copies of data that was secured with earlier keys?

A:  Any Enterprise key management solution should maintain encryption keys under policy for as long as they are needed. As you generate new keys, the older keys are retained and made available for decryption purposes, until you retire those keys. Our solution will maintain multiple versions, and doesn’t have a limit on how many keys or generations you can have of master or key encryption keys. If you have a loss or need to delete data that is out of your control, you can then delete the key. 

For more information on the full life cycle of keys, check out this blog: Why Key Management is So Critical in the Life Cycle of an Encryption Key

Q:  Your two-factor authentication product is supported in which countries?

A:  Since our 2FA solution is through TeleSign, a global company, it has a broad presence in more than 200 countries around the world and in 87 languages.  By leveraging an individual's mobile phone, a reliable means of authentication has become readily available for the IBM i platform. For example, instead of tokens, businesses can simply send an SMS or voice message that contains a one-time authentication code to the individual user’s phone. This means cyber criminals cannot log into the IBM i without physical control of the actual phone. 

This blog will outline Making a Case for Two Factor Authentication: Taking Security Beyond Usernames and Passwords

Webinar: Sec

Topics: Data Security, IBM i, Webinar, iSeries, AS/400, IBM Security Solutions

State of Encryption Key Management

Posted by Liz Townsend on Nov 24, 2015 9:32:00 AM

Looking into 2016, what is the role encryption key management will play in securing sensitive data?

Encryption and key management are the Fort Knoxes of security technologies for organizations wanting to protect sensitive data from hackers and data breaches. While commonly used by retail and financial institutions (and gaining even more traction after the onslaught of retail data breaches we saw in 2014), we still see major gaps and problems with implementation of these technologies across multiple industries. In 2015, with over 181 million records exposed in data breaches by mid November, we ask ourselves, what are the challenges of implementing encryption and key management, how widely are they used today, and what can we expect from encryption and key management vendors looking forward? eBook The Encryption Guide

While encryption has become an easily accessible technology, it remains a major point of struggle for most companies. Since organizations have multiple departments with siloed technical infrastructure, many different tools must be used to manage data across the enterprise. From HR to Accounting to stored customer data, many different platforms, operating systems, databases, and applications are used to store and process sensitive information. This makes locating this data extremely difficult as well as achieving consistent data encryption that can be managed from a single, central location.

Boards of directors and executives are becoming more aware that data security is not just a technical problem, but a governance, risk management, and compliance problem that deserves the same level of attention to risk as financial, legal, and corporate aspects of their business. However, employees at the IT level still hold the most buying influence over encryption and key management technologies.

These sorts of buying decisions have historically landed in the wheelhouse of IT Operations; however, the primary issue that arises in these decisions is that  complicated data security projects are often perceived as a threat to operational continuity. When an IT professional feels they must choose between security and functionality, they will always choose function to avoid the dreaded business-down scenario. Companies should not have to chose between security and continuity, and today, security professionals advocate that executives assign an IT security team to advocate for security solutions and work with IT Operations to implement these technologies.

According to the Ponemon Institute 2015 Global Encryption & Key Management Trends Study, meeting compliance requirements such as PCI-DSS remains the primary driver for encryption and key management implementation. PCI-DSS and federal and financial regulations such as FISMA and GLBA/FFIEC also continue to set the strictest data security regulations. However, despite compliance with industry regulations, organizations still experience breaches, often by a hacker accessing their network through a third party vendor or through employee mistakes. Sadly, often these breaches reveal that data was not encrypted, despite industry compliance.

This flagrant lack of encryption begs the question, will our data security ever get better, or will hackers continue to be one or even two steps ahead?

The answer to that question may come from the fact that in many large corporations, about 80% of resources allocated for data security apply towards network and anti-virus security. This includes firewalls, malware detection, and other intrusion-prevention software. The problem with relying mostly on network security is that hackers continually succeed in breaking through these barriers, often using social engineering and phishing scams to achieve enough authority to open a door and walk right in. Once inside, they discover sensitive data stored in the clear and steal it.

Network security is always an important part of a data security plan, but time after time we see encryption, which is also a critical part of that plan, implemented after-the-fact. This comes back to the issue of sensitive data being difficult to locate inside an enterprise, but the sheer amounts of unencrypted data that hackers are able to discover leads one to believe that some organizations simply do not implement encryption very well. This may be backed up by the discovery that only 37% of companies in the U.S. deploy encryption extensively (as opposed to partially) across their enterprise.

Diving deeper into the challenges surrounding encryption, one of the most painful parts of encrypting data is managing encryption keys. Even if a company encrypts a database of customer credit card numbers, if they do not protect the encryption key, a hacker could easily find the key and decrypt the data, rendering the encryption useless. Unfortunately, protecting and managing encryption keys away from encrypted data is still something organizations fail to do.

As organizations begin to move into the cloud and virtualized environments, as many already have, another stumbling block will be lack of availability of hybrid (cloud and in-house) encryption and key management solutions.

Looking into 2016 and beyond, the key management solutions that will excel will be the solutions that can manage encryption keys anywhere your sensitive data is located whether that be in the cloud, virtual platforms, or hardware. A majority of companies believe that hybrid deployment in both cloud and on-premise is the most important feature of an encryption solution. Without strong hybrid key management, encryption of data spread across an enterprise and the cloud will become even more difficult. Key management vendors that follow their customers into virtual environments will, in the long term, deliver more comprehensive data security.

It’s hard to imagine that data breaches will begin to diminish any time soon, but hopefully organizations will learn from others’ mistakes. It is clear from the evidence that deployment of encryption is nowhere near complete across most organizations, and lack of encryption key management continues to be a challenge, but working with the right encryption key management vendor can ease this pain.

When looking for a key management vendor that can help you manage encryption keys across your enterprise, including the cloud, look for a key management vendor that has:

  • No hidden or additional fees for nodes or client-side applications
  • Commitment to innovation and development
  • Commitment to legacy products
  • Excellent reputation for customer support

 

The Encryption Guide eBook

Topics: Data Security, Encryption, eBook, Encryption Key Management, Defense-in-Depth

Monitoring IBM i Logs with IBM QRadar - Improve Your Security

Posted by Patrick Townsend on Nov 2, 2015 10:25:00 PM

We all now know that active monitoring and rapid response is one of the critical security controls that really make a difference. That is why system log monitoring makes the Top Ten list of almost all cyber security controls. What is not so well known is how hard it can be to get active monitoring right. We have a lot of Security Information and Event Monitoring (SIEM) solutions to choose from, but very few of them are effective right out of the box. Why is this?

Podcast: Monitoring IBM i Security Logs with QRadar First, system generated logs are a mess. They are largely unformatted text messages without unique identifiers that make it hard for a SIEM solution to interpret. Add many different spoken languages and you have a major headache when it comes to interpreting log messages.

Second, other than some basic formatting guidelines, information in system logs is not normalized. While some log formatting standards like Common Event Format (CEF) and Log Event Extended Format (LEEF) attempt to provide this, very few devices actually format to these standards. The lack of system log standards contributes to the confusion when SIEM solutions attempt to interpret the log messages. It would make a database administrator shed tears.

Lastly, many SIEM solutions collect logs once or twice a day with some type of batch transfer, and events are not processed in real-time. Real-time analysis is core to effective SIEM monitoring of system logs. Without real time event collection it is difficult or impossible to do event correlation and the result is missed positives. All of that intelligence built into modern SIEM solutions can go to waste.

One thing I like about the IBM Security QRadar solution is that it comes with pre-defined definitions that out of the box know how to interpret logs from a wide variety of devices. IBM packages these definitions in a configuration object known as a Device Support Module, or DSM. IBM QRadar customers get access to all of these DSM definitions and they can be easily updated as new and revised configurations become available. This saves a security administrator a lot of time in configuring the SIEM to recognize events.

Another thing I like about IBM Security QRadar is that it understands that normalized data is important. The QRadar Log Event Extended Format, or LEEF, builds on IETF system log standards by adding well-defined data formats and field definitions. If all of your systems are reporting an IP address like this:

src=1.2.3.4

then you know that event correlation is going to work a lot better.

Our IBM i (AS/400, iSeries) solution for IBM Security QRadar integration is named Alliance LogAgent for IBM QRadar. It implements support for the QRadar LEEF data format for all IBM i security events, and transmits events in real time. IBM has now released an updated AS/400 DSM that includes recognition of the more than 200 security events transmitted by Alliance LogAgent for IBM QRadar. This means that customers deploying or updating their QRadar implementation get a much faster implementation and a much better security posture right out of the box. This new solution installs on an IBM i server very quickly and in minutes can be sending security events to IBM Security QRadar.

No one security control will make you safe. But actively monitoring your system and audit logs is crucial to a good security implementation.

For more information, visit our Alliance LogAgent for IBM QRadar or get started with a free evaluation.

Othwer Resources

Center for Internet Secuirity

SANS Top Ten (see CSC 6)

Monitoring IBM i Security Logs with IBM QRadar

Topics: Alliance LogAgent, logging

How Does LogAgent Send Security Information? Is Information Batched?

Posted by Patrick Townsend on Oct 23, 2015 4:34:00 PM

Q: How does LogAgent send security information to my SIEM or log collection server? Is information batched or real time?

System Logging Resource Kit The Townsend Security solution for system logging and SIEM integration is Alliance LogAgent. It works with a large number of SIEM solutions including IBM QRadar, LogRhythm, Dell SecureWorks, NTT/Solutionary, Splunk, Alert Logic, HP ArcSight, McAfee, and many others. It brings the IBM i (iSeries, AS/400) into an active monitoring strategy that is so important to good security. Since real-time security event collection is crucial to active monitoring, customers often ask us how Alliance LogAgent achieves this? Let’s take a deeper dive into how this is accomplished.

The IBM security audit journal is named QAUDJRN and it collects most of the critical security events on the IBM i platform. Unlike many IBM i system logging tools, Alliance LogAgent collects events from this journal in real time. Using IBM provided application program interfaces (APIs), events are collected from the security journal as they are written to the journal by the operating system. There is no batch-oriented extraction of events once or twice a day, and no batch transfer using unsecure FTP. Alliance LogAgent is able to grab the events as they become available. This provides the real-time view of security events that is so critical to active monitoring, correlation and alerting by SIEM solutions.

Once the event is extracted it has to be converted into a usable format. The security event information in QAUDJRN is in an internal IBM format and is stored in the EBCDIC character set which is largely unusable by SIEM solutions. Alliance LogAgent immediately converts the important information into a system log format (syslog, Common Event Format, or Log Event Extended Format), and translates it to the ASCII character set that is used by SIEM solutions. To make the information usable to SIEM solutions the event information is normalized into fields that are easy for SIEM solutions to understand. These normalized fields are in the keyword=value format (more on this is another blog). The formatting also happens in real time so that there are no delays imposed by the conversion process.

Once the security event is extracted and converted to a usable format, it must be communicated to the SIEM solution for processing. Alliance LogAgent implements a set of syslog communications modules that immediately send the security event to the SIEM server. Alliance LogAgent supports three different syslog communications options:

  • Internet UDP protocol
  • Internet TCP protocol
  • Internet TLS encrypted TCP protocol

By default these communications programs send security events to the standard syslog port of 514 on the SIEM server, but you can easily change the port number if needed. Not every SIEM solution supports encrypted transfer of log events, but Alliance LogAgent gives you this option along with non-encrypted options for log transfer.

Alliance LogAgent runs in a background batch process at a low priority so that it does not disrupt normal interactive response times. Using the optimized processes of Alliance LogAgent the IBM i customer achieves real-time processing of security events and gets the best results and maximum benefit from their SIEM solution. Security issues are identified immediately and the IBM i system administrator can react swiftly to potential security breaches.

Additionally, Alliance LogAgent takes a similar approach to monitoring other security event sources on the IBM i platform. The QHST system message facility is also monitored in near real-time as messages are logged to the QHST files. For messages sent to the system operator message queue QSYSOPR or QSYSMSG, Alliance LogAgent also monitors these message queues for events and sends the information to the SIEM server in real time. The same is also true of the Alliance LogAgent exit point monitoring applications.

Alliance LogAgent was built from the ground up to accommodate real-time security event collection and transmission to your SIEM solution. It is fast, efficient, and non-intrusive. Exactly what you need to collect and monitor security information on your IBM i platform.

Request your System Logging Resource Kit

Topics: System Logging, IBM i, Alliance LogAgent

How Much Disk Storage Does Alliance LogAgent Use?

Posted by Patrick Townsend on Oct 16, 2015 3:57:00 PM

When IBM i users deploy Alliance LogAgent to integrate their IBM i servers with their log collection and SIEM solutions they naturally ask about the storage requirements. This is probably because some IBM i logging solutions perform batch extractions of the security events and then use FTP or other file transfer mechanisms to transfer the data. And there can be a LOT of security event information to transfer thus expanding the need for storage on the IBM i platform.

System Logging Resource Kit Alliance LogAgent does not use an periodic batch extraction architecture for its implementation. Instead, Alliance LogAgent extracts security event information from the security audit journal QAUDJRN in real time and pushes the information directly to the log collection server or SIEM solution directly. So the answer to the storage question is easy:

Zero. Zilch. Zed. Nada.

There is no intermediate or temporary storage utilization when you deploy Alliance LogAgent. All events are extracted, converted to a standard system logging format, and transmitted directly without the need for intermediate files. This is true for all of the security log sources on the IBM i including the security audit journal QAUDJRN, the system message files QHST, the exit points, the message queues, and the Linux-style message files in the IFS file system.

Of course, the application itself including the programs and configuration files require some storage on the IBM server. A typical installation of Alliance LogAgent will require about 115 MB of disk storage. But this storage will not grow over time due to historical information or temporary storage.

That is good news for IBM i customers who are trying to control costs.

Securing our systems is a demanding task and we don’t need the added worry of additional system resource costs!

Request your System Logging Resource Kit

Topics: System Logging, IBM i, Alliance LogAgent

EU Data Privacy, Safe Harbour and Encryption

Posted by Patrick Townsend on Oct 7, 2015 12:19:00 PM

In a ruling that shocked Internet service providers and businesses in the US and abroad, the European Court of Justice ruled this week that current data Safe Harbour rules may not be adequate to protect the privacy of EU citizens and that individual countries may make their own rules about data privacy. Anyone who has lived in Europe and knows the historical context of governmental tracking and abuse of individual rights will certainly not be surprised by this ruling.

But why is this such a big deal?

Download the EU Data Privacy White Paper Have you noticed how good Google, Facebook, Microsoft, Amazon, Yahoo and others are at showing you advertising that reflects your interests? They are really good at this because they are the ultimate data aggregators. They use their vast network of global systems to bring data about you together and then perform sophisticated analytics. This means that most Internet service providers are moving data across country boundaries into the United States or areas controlled by the US where that data is subject to government inspection.

Beyond the obvious advertising aspects of Internet services, many backup and archival systems are built on Internet-based storage services. This means sensitive backup data moves over the Internet and may move to servers or networks outside of the host country. Internet service providers have been working hard to make their systems resilient and this often means integrating across borders.

In fairness, it is not just the US government that snoops on individual activity - many governments around the world do the same thing. And that is the concern of the European courts.

If data can’t leave a country, that will have a major impact on Internet service providers. And, of course, that will have a major impact on the small and large businesses that use these services. It’s potentially a very large problem!

In a Computer Weekly interview with Andy Hardy, Managing Director of Code42, he noted the importance of encryption and key management in meeting the new requirements. Andy said:

“It need not be the end of business as we know it in terms of data handling. What businesses need to do now is safeguard data,” he said.

According to Hardy, businesses must ensure they can keep company and customer data private, even when backed up into a public cloud.

“The right technology will ensure data it is encrypted before it leaves the endpoint device, so that it cannot be decrypted in the cloud and hence remains private. The best technologies will ensure that encryption keys are kept by our customers on-premise, so only they can decrypt the data and that no one else can access it unless with prior direct request. This is the only way to ensure privacy in the public cloud post-Safe Harbour,” he said.

I think Andy has this exactly right. When encryption is done right it makes the data unintelligible to anyone without the encryption keys. Using a key management solution that is resident in the EU, which is dedicated to the data holder, and which does not allow third party administrative access will be crucial to meeting the new EU privacy laws.

That’s exactly what we do with our encryption solutions that integrate with Alliance Key Manager and we are already helping EU customers protect their data with strong encryption. EU customers can locate Alliance Key Manager within their own data center, or in a country-specific hosting center, or even in a cloud service provider platform where there are adequate guarantees around in-country hosting.

EU Data Privacy Protections and Encryption

Topics: Compliance, EU Data Privacy Protection

Can I Encrypt Key Fields (Indexes) with IBM i FieldProc?

Posted by Patrick Townsend on Oct 6, 2015 1:50:00 PM

IBM introduced the DB2 Field Procedures (FieldProc) column level encryption interface in V7R1 of the operating system. It has been a great way for IBM i (iSeries, AS/400) customers to protect sensitive data in their DB2 for i files and tables, but customers often have questions about how this new capability works. One of the most common questions is “Can I encrypt index fields and will they work correctly?”

FIELDPROC Encryption The answer to the first part of the question is YES you can encrypt index fields, and the answer to the second part of the question is THAT DEPENDS. Let’s take a deeper look at encrypted indexes with FieldProc.

First, let’s look at DB2 FieldProc strictly from a SQL point of view. Remember that SQL is IBM’s preferred interface to the DB2 relational database. So let’s start there:

The first thing to understand is that FieldProc is fundamentally a SQL construct. That is, it is designed for and implemented as a SQL facility. You can specify a FieldProc program on the SQL CREATE TABLE or ALTER TABLE commands, but you can’t specify FieldProc on traditional DDS source descriptions. FieldProc works great on index fields in your SQL applications! Your SQL statements will work just as you would hope and you will have a great new facility for implementing automatic encryption. With very few limitations you will find that encrypted indexes work without any issues for your SQL applications. I’ve rarely found a customer who was unhappy with IBM’s implementation of FieldProc in native SQL applications. This includes SQLRPG applications that use native SQL for the database interface.

But, of course, most IBM i customers are running a lot of legacy RPG or COBOL applications that do not use SQL. And this is where there are some significant restrictions on encrypted indexes.

First, you CAN use FieldProc on traditional database files created with DDS. It is not necessary to convert the database files to SQL in order to use FieldProc. Of course, FieldProc application support is installed using SQL statements, but they will work on traditional DDS created files with some minor limitations. So this part is not complicated.

Next, you CAN encrypt indexes that are created with DDS. However, you do have some significant limitations when using FieldProc with DDS files. For example, some join logical files that join on encrypted index fields will not work. You simply won’t be able to create join logical files that link using fields encrypted under FieldProc.

A more fundamental problem is that legacy RPG and COBOL applications will not work correctly with most encrypted indexes. Since the legacy file interface is not SQL, the legacy applications will not work as expected in many cases. For example, it is very common to use the Set Lower Limits (SETLL) command with the Read (READ) command to read a range of values in a table. In these legacy applications the SETLL value will be converted to the encrypted value by FieldProc, and then the next record will be read using the encrypted key value. But encrypted values will not be in the same order as the original plaintext values. This will lead to empty subfiles and empty or incorrect information on reports.

For many IBM i customers the limitations on encrypted indexes are not a big problem and they live with them. For many others encrypted indexes with legacy RPG applications is a significant problem that will make the use of FieldProc impossible.

Is there a solution for this problem? Well, of course you can convert all of your legacy databases and applications to SQL databases and SQL RPG applications, or even to native SQL applications. But this represents a major investment by many customers. But there is an alternative provided in the Open Access for RPG (OAR) implementation by IBM.

The Open Access for RPG implementation allows you to define a handler for file operations using one F specification in your RPG program. With this implementation the legacy file operations are handled by your new handler application. And that can be a set of SQL functions! This means that a legacy RPG program can become enabled for true SQL operations with a simple change and re-compile of the application. Of course, you must have the SQL handler functions ready to take over the file operations. I won’t go into creating SQL handlers in this blog, but be aware that creating SQL handlers is not for the faint of heart. You need to have extensive experience with SQL and understand the OAR architecture. If you’ve not done this before the IBM Lab Services team can provide assistance.

In summary, FieldProc is a great new facility and it is already helping a lot of IBM i DB2 customers to protect data with strong encryption. It works great with native SQL applications, but you need to be aware of some limitations when used with legacy RPG and COBOL applications.

Our Alliance AES/400 solution provides everything you need to implement FieldProc. 

IBM i FIELDPROC Encryption

Topics: Alliance AES/400, IBM i, AES Encryption

How Does Data Masking Work with FieldProc Automatic Encryption

Posted by Patrick Townsend on Sep 29, 2015 8:24:00 AM

Many compliance regulations such as PCI-DSS and HIPAA require that data be completely hidden or partially masked for all users who are not authorized to see the data. When IBM i (AS/400, iSeries) customers implement DB2 Field Procedure (FieldProc) encryption, how do they implement data masking at the same time? In this blog I want to talk about general principles of data masking, how IBM DB2 FieldProc applications can implement data masking, and some limitations that you need to know.

Data Masking and the FieldProc Architecture
IBM i FIELDPROC Webinar There is no direct facility provided by IBM in the DB2 FieldProc architecture to implement data masking for you. This means that your FieldProc application must implement this for you. FieldProc applications are written by you or provided by your vendor (see below for information about the Townsend Security solution). Since data masking is designed to protect sensitive data in the clear, this means that data masking should be implemented in the decryption logic. When a row is read from a database table the FieldProc application is called by DB2 to perform decryption. At that point the determination must be made on whether the user is allowed to see the entire column value in the clear, whether it should be partially masked, or whether it should be fully masked. 

Data Masking Security Architecture 

One of the core security principles of data masking is that it should be based on user whitelists and not on user object authority. This is not an intuitive idea for most IBM i system administrators because so much of IBM i security is based on file object level authorities. So why would data masking not rely on object authority?

One of the main efforts of a cyber criminal upon gaining access to your system to escalate their privilege level in order to access sensitive data and achieve the ability to manipulate system configurations. On the IBM i platform the attacker will try to gain security administrator privilege or All Object (*ALLOBJ) authority. If your data masking strategy is based on object authority it is immediately defeated when the attacker gains greater privilege. This is why data masking should be based on a whitelist approach and not on native object authorities, and why we took this approach in our Alliance AES/400 FieldProc implementation. 

Users and Groups
Like many other operating systems the IBM i provides for user groups. On the IBM i platform user groups are defined by a Group Profile, and individual users can belong to this group or it can be included in their supplemental group. Group profiles are a convenient way for IBM i security administrators to define authority rights for anyone in the group and not have to define these rights for each individual user. If you are a user you inherit the rights to the groups you belong to. A data masking implementation should incorporate user groups into its implementation through a whitelist approach.

Data Masking Options
Data masking options should allow for masking of an entire field, or just a portion of the field. The PCI Data Security Standard (PCI DSS) is probably most clear on data masking requirements for compliance. Only authorized users (defined by PCI as “personnel with a legitimate business need can see the full PAN”) should see an entire field unmasked and all others should see only a portion of the field (The first six and/or last four digits) or the field should be fully masked. When establishing your whitelist of users you should be able to define the data masking options at the same time as you define the user or the group.

Data Masking for Numeric Fields
On the IBM i platform we have a challenge when it comes to masking numeric fields. Numeric fields can only contain a numeric digit and can’t contain an asterisk or other typical masking character. For numeric fields you must determine a masking pattern that can’t naturally occur in the data. Perhaps you could choose a masking option that filled a numeric field with all 9’s. This would work if the field could not naturally contain all 9’s. An example might be a salary field. If you have a 9.2 zoned numeric field it is unlikely that anyone would have a salary equal to $9,999,999.99 and masking with all 9’s would work. Just be aware that masking numeric fields requires some forethought and planning.

Limitations and Gotchas
Data masking with FieldProc encryption is generally very effective and easy to do. But there are a few limitations. Consider a program that reads data from table A and uses a column value to read a record from table B. And assume that the value read from table A is encrypted. If a user can only see masked data, the data will be masked when read from table A and the lookup on table B will fail. This is a subtle limitation but it has occurred in the real world!

Summary
Data masking is a powerful additional security control for your FieldProc applications. If you are aware of the limitations it is a great tool to help you get better security for your IBM i databases and applications.

IBM i FieldProc encryption by Townsend Security
Townsend Security provides a full IBM DB2 FieldProc solution for the IBM i server platform with Alliance AES/400. In addition to strong 256-bit AES encryption, it provides IBM i customers with flexible data masking options on decryption. It has full support for individual and group profiles, and the security administrator can specify a default masking rule to apply to any users not explicitly allowed to see unmasked or partially masked data.

FIELDPROC Encryption IBM i

Topics: Alliance AES/400, FIELDPROC

How Does IBM i FieldProc Encryption Affect Performance?

Posted by Patrick Townsend on Sep 14, 2015 8:49:00 AM

IBM i (AS/400, iSeries) customers have a great automatic encryption option with DB2 Field Procedures, or “FieldProc”. As with any encryption facility, users always have questions and concerns about performance. Performance impacts extend beyond just the impact of encryption itself, so let’s look at various aspects of performance when it comes to IBM i FieldProc.

IBM FieldProc Architecture
IBM i FIELDPROC Webinar One of the largest impacts on performance comes from the actual architecture of FieldProc itself. IBM DB2 FieldProc is basically implemented as an event-driven exit point at the column level. What this means is that any insert, read, or update operation will trigger a dynamic program call to the FieldProc application program to perform encryption or decryption. There is definitely a performance penalty for this architecture. An application program that reads a large database on a modern IBM i server may be able to process hundreds of thousands of records per second. With FieldProc, that may be reduced to tens of thousands of records per second as the FieldProc program is invoked for each row in the table. You can still get good performance with FieldProc enabled (read on), but there will be an impact.


FieldProc Program Performance and Optimization
A FieldProc program is just an application program that you create or that your encryption vendor provides to you, so it can have its own performance issues. How much file I/O does the FieldProc program perform for each encryption or decryption task? How optimized is the application code? How optimized is the compilation of the program? Does the program perform any caching of internal information to improve performance? Like any program on any platform or operating system, a FieldProc program may perform well or not.

Encryption Performance
Surprisingly, there can be really big differences in the performance of encryption libraries even when doing the same type of encryption. You might think that 256-bit AES would have the same performance regardless of the vendor. And you would be really wrong about that. On the IBM i server platform I’ve seen a difference of more than 100 times between two different 256-bit AES encryption libraries. To put this in a practical context, this is the difference between 10 hours of batch processing versus 5 minutes of batch processing. That’s pretty dramatic. Encryption libraries can be optimized and should be optimized for performance. That is not always the case.

Number of Columns Under Encryption Control
The number of columns in a table will affect the performance of your FieldProc implementation. If you have three columns in a table under FieldProc control you will definitely see an impact on performance compared to a single column. Each read of a row in the table will result in three separate calls to a FieldProc program to perform decryption. This is not a linear impact on performance. That is, you won’t see an impact on the order of three times the impact of one column under FieldProc control. But there is a gradual impact as you add columns in the table. By the way, FieldProc will be called for each column even if your application does not use the column.

Encryption Key Management
Using encryption means using encryption keys. Assuming that you are not using a poor security practice such as storing the key on the same server as the encrypted data, the interface to your key management server represents another potential performance impact. How keys are retrieved and prepared for use by the encryption software can represent a hidden drag on performance. While a single key retrieval from a key server may take just a few milliseconds, the performance impact can be dramatic when thousands or millions of key retrievals are needed from a key server.

Encryption Key Caching
Because encryption key retrieval can slow the overall encryption process, it is important that a FieldProc application use secure key caching logic to minimize the number of key retrieval operations. If your nightly processing retrieves 10 million records for reporting, you definitely don’t want to retrieve encryption keys 10 million times. A good FieldProc implementation should securely cache encryption keys. This means that keys should not be exposed in program dumps or debug mode of operation.

CPU Performance
IBM i servers vary a great deal in CPU performance and the number of processors that are available to applications. Entry level servers may have a single processor that is shared between multiple partitions. High end IBM i servers can have a large number of processors and rival Mainframes in raw processing power. This will definitely have an impact on encryption performance. The number of processors is less important than the power of each processor. It sometimes surprises IBM i customers that adding a processor to a system might have minimal impact on encryption performance. But upgrading to a faster processor can make a big difference. Also, more modern IBM i servers have very powerful POWER7 and POWER8 chips and these will help with encryption performance.

POWER8 On-Board Encryption
The new IBM POWER8 systems now have built-in support for AES encryption. This is similar to the Intel AES-NI implementation. While this does provide some improvement in encryption performance, it won’t be as much as you might expect. The built-in chip support for AES encryption seems to be optimized for encrypting very large chunks of data at one time. If you are encrypting a credit card number of social security number, you won’t see a really dramatic improvement in performance. IBM i customers using ASP encryption should really benefit from the built-in encryption. In some cases such as with Townsend Security's Alliance AES/400 encryption for IBM i, the software implementation provides big performance advantages over the on-chip POWER8 implementation.

Native SQL Applications
As most IBM i customers know, IBM has been on a tear to improve SQL in DB2 for some time. We’ve seen increasingly better performance of SQL applications over time. In the current release of the IBM i operating system and DB2 database the performance of SQL is impressive. Because SQL performs better, you will see better performance when implementing FieldProc in native SQL applications. Of course, you don’t need to convert your databases from DDS to DDL/SQL to use FieldProc, but if you do you will see better overall performance.

IBM i Navigator and SQL Plan Cache
When discussing database performance it is always important to mention the IBM i Navigator SQL Plan Cache function. This application comes with every IBM i server and is always available. It can show you how well your DB2 applications are performing, and can even recommend steps you can take to improve performance! When using FieldProc it can be a very helpful tool.

More about Townsend Security’s AES/400 FieldProc Solution for IBM i DB2
The Townsend Security solution for FieldProc encryption is called Alliance AES/400. It is the fastest performing FieldProc solution in the market and implements all of the FieldProc recommendations above. 

FIELDPROC Encryption IBM i

Topics: Encryption, Alliance AES/400, FIELDPROC