Email archiving’s heyday began in the early 2000s when a category of solutions arose to address the high costs of high-speed storage for email servers. The hard drives required for those servers were costly, and organizations struggled with users who were reluctant and/or unwilling to delete their emails.
Archiving platforms like Enterprise Vault, SourceOne, and Commvault, for example, managed the mail that was resident on actual email servers by leveraging a variety of rules and policies to move mail items from the users’ mailboxes to less expensive archive storage. Typically, the actual stored items were deduplicated and compressed to mitigate storage concerns further, while only a very small “shortcut” or “stub” was left in the original location.
As storage became less expensive - and eventually just plain cheap - over the years, email archiving evolved: “mailbox management” became less critical, while the archiving of journaled mail for eDiscovery and Compliance reasons became increasingly commonplace. In the latter 2010s and early 2020s, however, organizations shifted from on-premises mail servers to cloud email providers, and even journal archiving began to fade as organizations looked to cut costs and maximize cloud investments. In the mid-2020s, however, a number of factors have led to something of a renaissance – a “returnaling” to journaling.
Ironically, the same issues that on-premises archiving systems were designed to address for email ultimately doomed those archiving platforms. Like email servers, archiving servers required expensive infrastructure with high costs for software and hardware maintenance, services from third parties, limitations on storage and scalability and required substantial in-house expertise.
While it took time for organizations to warm to Microsoft Exchange Online, it eventually seemed like a magic bullet for many: cost savings, scalability, flexibility and reduced IT burden were significant factors in its success. Distrust of cloud solutions was everywhere – was it a fad? Would the risks be too great for business-critical systems?
Much of the initial reluctance to move email to Exchange Online, however, eventually disappeared after years of growing cloud acceptance across a wide range of workloads. At that point, Exchange Online became an inevitability for the overwhelming majority of organizations’ email.
The transition from on-premises email to cloud email was not the end of archiving, however, though it did effectively establish the prioritization of journal archiving. Before transitioning their email to the cloud, many organizations had already been working with platforms like Barracuda, Veritas EV.Cloud (now Arctera Alta Archive), Mimecast and Proofpoint who could already provide SaaS archiving.
The model for virtually all SaaS archives relied upon a journal stream of all an organization’s email to a third-party provider where it would be archived and indexed for search and eDiscovery. Individual end users typically were granted access to their own “slice” of the data as a function of data migrated into the SaaS platform and data that was subsequently journaled in.
In some cases, of course, organizations continued archiving simply because they had previously been convinced by a vendor or manufacturer that they needed to do so. Others looked to SaaS archiving providers because their industries (financial services, healthcare) provided clear mandates for retention of a pristine repository of all email. These and other highly regulated or heavily litigious entities had a clear understanding that journal archiving was critical to their eDiscovery and/or Compliance obligations. Further, those organizations often found Microsoft’s early eDiscovery and Compliance capabilities lacking.
As cloud adoption marched ever onwards, organizations that had simply continued archiving out of habit began to re-evaluate their needs for a dedicated archiving solution. As Microsoft improved its eDiscovery product, now known as Purview, many companies faced hard choices: Although Purview’s custodian-based model for eDiscovery (as opposed to item-level searching in journals) was different and at times uncomfortable, many customers realized they simply didn’t need to do much eDiscovery in the first place.
Exchange Online customers increasingly found Purview was either “good enough” or that duplicative archiving solutions were too expensive, particularly given the sunk (but high) cost of Microsoft licensing that included Purview as an alternative.
It’s important to understand, however, that a significant number of customers with legitimate reason to continue archiving eventually also decided that Microsoft’s “good enough” solution was sufficient for them to discontinue journal archiving and let their archived data simply expire instead of paying for new data to be archived or for the cost of extraction. That said, some of them later realized they were wrong.
While some organizations moved on, heavily regulated entities still found their SaaS archiving solutions that relied on journal archiving to be both useful and necessary. Their decisions to retain those platforms were often “aided” by exorbitant data ransom costs (the oft-hidden fees to extract data upon exit from a SaaS archiving platform can exceed $10k/TBT).
Unsurprisingly, technology is not immune to trends. Evolution is constant and change is cyclical. In recent years, Cloudficient has observed a resurgence of interest in journaling from customers. There are a number of contributing factors, some of which are highlighted below.
The first and most obvious are customers who simply never should have stopped journaling in the first place but were led astray by the promise of lower costs. Those customers face significant regulatory and legal pressure, whether from SEC, FINRA, HIPAA, GDPR, or an alphabet soup of other evolving regulations. Their regulators still expect immutable, complete, and contemporaneous copies of communications.
These organizations took a chance to maximize their Microsoft ROI, but ultimately were confronted with the reality that they cannot accept the risk of end-user tampering of email and the fact that, Microsoft Exchange Online can journal (relay) mail to a third party but it simply does not allow journaling of mail into its own repositories.
Also in that category: Customers whose eDiscovery obligations either always were or simply became more significant than Purview is able to satisfy. That might mean an eDiscovery team that needs a reliable, single source of truth, centralized storage separate from active mailboxes or perhaps easier defensible deletion and retention management.
We have also observed customers who increasingly consider journaling for several more novel use cases. The primary driver is the increasing prevalence of hyperlinked files, sometimes called “Modern Attachments.”
Historically, when an attachment was sent from a sender to a recipient, its contents and attachments were static. That meant that at the time an email was sent, the attachment was the most current version.
In the age of SaaS applications, however, this model is shifting. An email is now more likely to include a link to a Word document or Excel spreadsheet that resides in OneDrive or SharePoint than the file itself. Meanwhile, those files may change dozens or hundreds of times after the email is sent, with earlier versions potentially “rolling off” after a maximum number of stored versions is reached. Increasingly, legal professionals and regulators believe that organizations have an obligation to preserve a “contemporaneous version” – the most current version available at the time the email was sent.
The best way to capture hyperlinked files is with a solution built around them. Enter Cloudficient’s Expireon, developed as a repository for ingesting archive data so that it remains discoverable and expirable for the duration of its useful life without the high costs of data ransom associated with other SaaS archiving platforms. Like its competitors, it also allows customers to continue (or resume) journaling mail, but Expireon was uniquely designed to address the retention of contemporaneous versions of hyperlinked files by capturing them from cloud storage at the time the email arrives. By capturing each version at the time that it was emailed and retaining it for the given retention period, Expireon effectively eliminates the “roll-off” problem inherent to versioning.
Other customers, meanwhile, might need to retain important information about messages that are not captured in the user mailboxes. The “journal envelope” or “P1 Header,” for example, retains information about Distribution Lists and BCCs that are not captured when searching an email in a user’s mailbox.
Some organizations worry that journaling will generate an excessive amount of data for production in a matter, especially if keyword culling is not used before the review stage. To the contrary, journaling, with the right technology supporting it, can actually make downstream review much more efficient.
First, if a reviewer is looking exclusively at individual end user mailboxes for custodian data, they may encounter multiple copies of the same item, one for Recipient A, another for Recipient B, etc. In a Journal archive, however, only a single copy is stored with metadata identifying all potential recipients. This has been true of journal archives for many years.
A more recent development, however, is the ability of AI tools to use larger quantities of data to increase their own accuracy. For example, Expireon AI Studio applies machine learning models to classify emails by category and sensitivity. Trained on each customer’s own data, it continues to learn over and improve time as it encounters more data. AI Studio identifies privileged, confidential, or sensitive data and flags those items for human review, preventing improper disclosure. The model learns quickly, and to accelerate accuracy, AI Studio allows customers to review items based on a threshold of accuracy to improve the model’s decisions over time. AI Studio also provides a second layer of review for those items using an LLM to help speed up the human intervention required. Categorization tags can be exported from Expireon to prioritize document review.
AI accuracy has improved dramatically with models that learn what is specifically relevant to individual organizations and get better with time. AI Studio reduces manual review time, helps legal teams identify relevant and non-relevant data more quickly, and reduces the risk of errors.
The 25-plus-year history of email archiving has been at once turbulent and familiar to those who follow the technology space. “Mailbox management,” an archiving use case that was the byproduct of unanticipated user behavior and high infrastructure costs that, in hindsight, were destined to decrease, has largely vanished.
For a time, it seemed like journal archiving might suffer a similar fate, bleeding market share until its target audience shrank to just a few highly regulated sectors of industry. As often occurs, however, shifting marketplaces shift more than once. Current trends are creating new opportunities for a trusted old technology to re-emerge.
Customers are learning that their initial eagerness to utilize their Microsoft licenses to their fullest extent ignored the limitations of Microsoft’s own capabilities, whether for archiving, eDiscovery or both. In the meantime, email itself has evolved - the transition from static attachments to hyperlinked files is forcing changes in regulation that are best served by journaling contemporaneous files into an archive. Is everything old new again?
Cloudficient’s unrivaled, next generation technology is revolutionizing the way businesses retire legacy systems, transform their organizations into the cloud, and capture, retain and protect the data once it’s there. Guiding customers through every stage of the enterprise lifecycle, our comprehensive services include cloud migration, information governance, and custodian management.
Whether you're transitioning to the cloud, ensuring data compliance, or managing electronic data for legal purposes, our expert team provides the support and solutions you need to succeed.
Bring Cloudficiency to your enterprise lifecycle: visit our website or contact us directly.