In this blog we’ll go through an overview of data retention, policies to implement a plan and why you should invest in ...
In this blog we’ll go through an overview of data retention, policies to implement a plan and why you should invest in a better alternative – intelligent retention.
Data retention management is the process of storing information for a specified period. It is relevant to all businesses that produce and retain data to service their customers and comply with government or industry regulations and legal requirements.
Without a data retention strategy, companies might store too much information or keep it longer than needed, which leads to operational inefficiencies, increased costs, and legal and security risks. Regulatory compliance is a must to avoid fines and streamline workflows.
To formalize the data retention best practices , every organization needs a data retention policy to codify the requirements and give clear guidance across the organization: what data must be retained and how it should be archived.
A data retention policy clarifies what data should be stored or archived, where that should happen and for how long. Essentially, it outlines data retention best practices your business should have in place.
It defines how the organization will store and manage data for compliance or regulatory reasons and the disposition once the data is no longer required.
Even a simple data retention policy should clarify how records and data should be formatted, how long they must be kept, and what storage system or devices are used to retain them.
Once a data set exceeds its retention time, the policy should specify whether the data can be deleted or moved to secondary (cold) data storage, depending on business requirements. These decisions will typically be based on local laws and regulations (GDPR) or rules of the industry’s regulatory body.
When you create a data retention policy, you will begin to explore data retention management software that can help achieve your goals.
While it is important to retain historical data for business use, most data retention best practices really exist to fulfill regulatory requirements.
Data retention policies for electronic communications are often the crux of data management. While one could argue that there is already a long tradition of filing paper documentation as a form of record retention, information created as large streams of electronic communications cannot as easily be stored or cataloged in traditional filing systems and does not contain business records.
Nearly 50% of all business communications is “noise”: anything from airline newsletters, legitimate advertising, IT notifications or daily updates from social networks such as LinkedIn.
But telling which half of the information is irrelevant has historically been difficult.
In the past, many records management experts therefore translated the legal or tax-focused retention requirements into blanket statements, such as
“Keep all emails in a ten-year retention category”.
Even worse, this was often done without explicitly requiring the corresponding expiry mechanisms to automatically prune those emails once they are past their due date.
Today, they struggle to enforce this deletion process remains for various reasons:
On the other hand, most organizations do not have the financial ability to retain all data forever, nor is that even a desirable goal given the emergence of harsher data privacy requirements (such as GDPR) which might require data to have a much shorter retention period.
For instance, personnel records and sensitive financial or medical records may all have different (shorter) retention periods. The complications with data retention lead many businesses to turn to data retention management services for assistance.
Due to the complex nature of the regulatory frameworks, organizations often use an industry-specific data retention management template that includes several different retention puzzle pieces:
Using a simple template is just a starting point, which should be refined to your organization’s needs.
After many years of discussion, the European Union has unified the data privacy approaches of its member states in a single, ambitious compliance framework: the General Data Protection Regulation – GDPR.
It grants data subjects various rights, which in turn means that organizations should not treat data from European customers as their own but rather as data under their stewardship. One way to deal with personal data from a Data Retention Management perspective is to create a separate retention policy just for personal data.
When considering a personal data retention policy, you must carefully audit all data collected to be sure your policy considers all personal data your organization stores. Data stored in databases, documents, email, financial data, images, production data, system state information, and videos might all be important for your personal data retention policy.
Think about this from a customer, as well as from an employee perspective.
The example above shows clearly that in a world of conflicting regulations and endless streams of irrelevant information, the classic “capture everything and store forever” approaches to data retention no longer work.
Fortunately, data retention management softwares are now available that will dramatically change how we look at Data Retention Management: Deep Learning, Natural Language processing and AI-based content recognition.
These transformative technologies pick up the age-old idea of filing the data on a “per record” basis but transfer the decision-making from the employee to an IT algorithm. Of course, this is not perfect, and a big chunk of the data will still be labeled as “miscellaneous”. Still, for many record types, the confidence levels of the AI are so trustworthy that it opens a glimpse into the future of Data Retention Management Services.
This is especially true as the data volumes have become unmanageable not only from a storage but, more importantly, from an eDiscovery or Data subject request perspective. It simply takes too long to sift manually through the irrelevant noise, especially for a highly paid legal expert.
In fact, it is now cheaper to use existing CPU capacity to classify 1,000 documents than having a manual reviewer read a single document to the end.
To outline the core principles of Intelligent Retention, we have created a list of 6 steps that will transform how organizations will retain, use and expire data in the future.
Not all created data is equal, which is why not all data has the same retention periods. Start by identifying what data your organization stores and then classify the data to determine which data needs to be archived and for how long.
If the data is not critical or a business record, it should be expired aggressively.
At the same time, there might be documents that the company wants to keep for a very long time: Contracts, design plans or anything that is patent related, which will likely include vast amounts of research data.
If any of that data also serves the current needs of the business, you might want to move it to the correct repository for your employees to find it, instead of keeping it hidden in an archived mailbox of someone who left the company years ago.
The right smart data retention management software can help your business effortlessly sift through data to determine if it’s critical or not.
Many organizations can reduce their communications archive by 25% by simply identifying and expiring newsletters and notifications older than two years.
Classifying your legacy data also allows for removing irrelevant and duplicated files. Aggressively expiring this data expedites searches, avoids confusion, and enhances the user experience.
One problem worth mentioning is “over-collecting” data during a legal investigation, which results in spiralingcosts for the review. How can that happen? Many newsletters and advertising emails contain disclaimers with several paragraphs of legal terminology. A search for “Luxemburg, contract, governing law, liability” produces thousands of Amazon newsletters.
Removing those “false positives” has an immediate positive effect on the legal team.
As you classify data, ensure your data retention policies align with compliance and legal restrictions. You may also have to consider pre-existing contractual needs shaping data retention schedules.
Understand your compliance verticals, whether it’s HIPAA, the Sarbanes-Oxley Act or GDPR. Know the law to determine what data must be kept and for how long. Likewise, keep data that you might need if legal action should arise. Keeping track of your data retention requirements helps with compliance in the long run.
By tagging health records as “HIPAA” or financial statements as “SOX”, a modern, intelligent retention platform helps deal with different and conflicting regulations.
Identifying personal data starts with the question:
What kinds of documents create the biggest risk in an archiving system?
Usually, those are documents that contain personal data by design, such as CVs, resumes or HR records. On top of that, health data is often protected and cannot be retained forever in most geographies. For a good reason: A sick note statingdepression or mental health problems after a divorce five years ago should play no role in promoting a top performer today.
The amount of personal data in question depends on the vertical industry and whether collecting information about the customer’s personal situation is part of the business process (insurance) or whether the health status needs to be documented regularly (healthcare).
You also need to consider the location of the data subject. In some cases, data located in different places may require unique data retention policies. A customer in California is now protected under the CCPA, while an employee in Europe has privacy rights that prevent the unsolicited recording or monitoring of certain communications.
Let us take a real-life example: There is hardly any document that contains more personal data than a CV or resume. GDPR has clarified that for rejected candidates, such a CV needs to be deleted no later than six months after the job application has been received.
How do you deal with informal job applications sent to the hiring manager via email?
Without an AI-based classification, your data retention policy will likely struggle to cater for these cases.
Identifying and capturing accounting records, customer correspondence, electronic communications financial data, sales data, and other mission-critical digital business data does more than ensure compliance. It also helps in re-introducing certain records into the actively used CRM or HR systems, such as Salesforce or Workday.
Emails that were sent during a patent application should be copied to the ECM system of choice, where the paper forms and laboratory records are stored, so that you do not accidentally lose this critical information after a standard 10-year retention period, given that patents tend to be granted for a minimum of 25 years.
An intelligent retention platform should allow you to re-insert data into other platforms if a corresponding tag has been confidently applied.
While this should be obvious, many businesses have second thoughts and hold on to data longer than required in hopes they might need it in the unforeseeable future.
Intelligent retention can make more storage available by saving space for new data and files while eliminating duplicates. The entire process saves time and money overall with lower storage costs and increased speed.
And you do not run into the danger of destroying important records or keeping personal data years longer than allowed.
Finally, it is much more likely to find an executive to sign off the deletion of content that has been confidently tagged as “non-relevant” than getting the buy-in to do a blanket deletion of data that might still contain important records.
The good news is that finally, the archiving and data retention community is waking up to this new challenge. While basic data classification has been available for a while, the trend is to provide ready-made Deep Learning capabilities within the Data Retention platform itself.
At Cloudficient, we have designed our Expireon solution as a true “Intelligent Retention” platform. It is designed with open interfaces so that different data classification plug-ins can be used, depending on the requirements of your organization.
The data retention management software can identify newsletters, system messages and many other content types. But the true power lies in how the learning algorithms can be trained to recognize industry-specific content and get more accurate over time. A document that might only score a confidence level of 50% will likely be significantly upgraded or downgraded when revisited after a few months.
While there is a trade-off between the required compute power versus the saved storage capacity (due to the cost of CPU calculations), Intelligent Retention saves a significant amount of storage cost and with further savings during the e-Discovery process.
Add the fact that Expireon uses industry-standard S3 storage and has a much lower subscription cost than competing products, and you end up with a cost-saving strategy that will fit even into tight budgets.
We recommend making Intelligent Retention capabilities a cornerstone of your future investments in the Data Retention Management space. Without the intelligence of an AI-powered solution, you will likely hit a brick wall when dealing with conflicting regulations, such as SOX and GDPR. By identifying personal data and telling them from financial records, Intelligent Retention will give you the agility to define your Data Retention policy in a future-proof way.
Now the good news: Expireon has migration capabilities built right into the platform so that onboarding data from legacy systems is a breeze. And there is no better time to classify your information than during a technology refresh or cloud migration.
If you want to learn more about Expireon and Intelligent Retention, click here to visit our website.
With unmatched next generation migration technology, Cloudficient is revolutionizing the way businesses retire legacy systems and transform their organization into the cloud. Our business constantly remains focused on client needs and creating product offerings that match them. We provide affordable services that are scalable, fast and seamless, including for data retention management.
If you would like to learn more about how to bring Cloudficiency to your migration or data retention project, visit our website, or contact us.
Let's talk about data retention, data protection, and the policies and strategies that can help your organization navigate them.
Explore email retention policy best practices for compliance, covering retention duration, policy development, and organization-wide implementation.
Find out more about Intelligent Retention and how it can help your organization in a data migration project.
