False: Marketers Should Only Use Realtime Data
True: They must combine realtime with batch data to achieve their dreams
The “marketing best practices” discussion in the last few years has been marked by a shift to trigger-driven marketing centered around realtime data usage. This has led to a fallacy that marketers should only use realtime data. This post will show you why this shift will only break your customers’ hearts and what you need instead.
Last year, a survey by research firm Gartner found that 66% of marketers indicate that event-triggered multichannel marketing is their primary approach for delivering marketing messages to customers and prospects. However, when compared to the 87% that back in 2017 had projected their organization’s multichannel marketing would be primarily event-triggered by 2019, the number comes up short.
Why the difference?
As marketers attempted to make their event-driven dreams a reality, vendors focused on providing realtime data solutions. However, customers aren’t as simple as “an abandoned cart with female activewear in it.” They are complex, have a history with the brand, and expect to be recognized by the brand they are purchasing from.
Slowly but surely, marketers discovered that event-driven marketing does not equal realtime data-based marketing. And to fulfill marketer demands and meet customer expectations, technologies need to adopt a hybrid approach.
This is where customer data comes into play – it is similar to the human brain, which has two categories. Professor Daniel Kahneman called them System 1 and System 2, but I prefer Mark Manson’s “feeling brain” and “thinking brain.” While the thinking brain is slow but conscientious, accurate, and impartial, the feeling brain arrives at conclusions quickly, is impulsive, irrational, and inaccurate.
In CRM marketing, the thinking brain results from the transformation, processing, and unification of your historical customer data. While the feeling brain is the trigger set to react to realtime data streams. If you only rely on the latter, you are bound to be inaccurate. But if you combine both and strike the right balance, your marketing dreams could come true.
We are about to get a bit technical, so feel free to engage your thinking brain.
Putting the combined approach to the test
One of the most classic triggered campaigns in the eCommerce world is the post-purchase email that uses the purchase event to send customers immediate follow-up emails. Sometimes, these will include offers for complimentary products, other times a list of how to maximize the product’s usage. But how do you know whom to send what email to?
You can ensure triggered campaigns differentiate between customers by leveraging a basket value model in conjunction with the trigger. By combining these two data points, the historical basket value with the realtime trigger can easily segment customers into multiple tiers or group different clusters together into tiers—for example, low, medium, and high.
This will help you plan hyper-relevant triggered campaigns for each segment. You can then provide promotional and complimentary product information to the low tier and send more informative campaigns to the higher tier, for instance. By appending the data from the latest purchase to the historical data already available, you can ensure customers receive the correct campaign.
How Optimove combines batch and realtime data
Building an effective customer profile that is updated in realtime is a multi-step process at Optimove; first, a batch process analyzes all available data related to each customer. This historical data can go back years and includes data dimensions from multiple systems, fused and enriched by different algorithms such as clustering and ML.
One of the products of this deep analysis process is a customer profile containing hundreds of customizable attributes. When a fresh customer profile is created, it is encoded, timestamped, and streamed into Apache Kafka for integration with the realtime process.
An independent process is feeding customer-related events from multiple sources such as Optimove’s SDK or Segment.com into Apache Kafka. This raw event data is grouped by customer ID and fed into a very flexible aggregation framework driven by a graphical UI. It describes which events update which profile attributes, and in what way.
A simple example would be a “Total Purchase Amount” attribute that tracks “purchase” events, extracts the amount field from within it, and sums the amounts across events for the present day. This aggregation framework may update many such attributes, across all customers, and do it all in realtime.
Next, a smart combining mechanism receives both the batch-process produced profiles and the realtime attributes as a continuous stream of information to perfectly stitch the two data sources and calculate the exact correct values. It then matches the data based on customer ID and timestamp.
Finally, this data is made available as a live feed in Optimove that also feeds back into the Self-Optimizing Journey algorithm, allowing for a continuous reevaluation of decisions.
Final thoughts for the thinking brain
Using Optimove, marketers can easily leverage deep insights daily by looking into all available customer data. All while keeping an eye on live event data to maintain an up-to-date customer profile and improved orchestration decisions.
Effectively rolling out a contextually relevant, event-driven marketing plan with personalized offers requires a smooth integration of both the thinking and feeling brains. By combining realtime customer activity with historical data, marketers can ensure that each customer’s complexity is taken into account before any campaign is sent. And maybe next time they are surveyed, the gap between expectations and reality will be closed.