Real-Time Machine Learning is emerging as an increasingly important component for product and marketing teams. Use cases range from dynamic decisioning to targeting anonymous users (more important with the emerging depreciation of 3rd-party cookies) to in-session content recommendations.
The Value of Real-Time
Here are some examples of why real-time is valuable for a business –
Training vs. Inference
Any machine learning system can be divided into two distinct parts – a training portion and an inference portion. Training is when the machine learning pipeline is built. The training portion is sometimes referred to as modeling. Inference is when the pipeline is used to make future predictions, predict uplift, generate look-alikes, or create user recommendations. Typically “real-time” machine learning refers to the inference component happening very fast and with the latest information.
Why Real-Time, <100ms, Inference is Hard
There are many definitions of what qualifies for “real-time”, but, in this context, let’s consider that real-time means the ability to make an inference in <100ms, which would be required for most dynamic decisioning, contextual targeting, etc. experiences. From an engineering perspective it can be very difficult to reliably return results in <100ms. In addition, returning results in this amount of time limits the amount of feature engineering possible. In addition, it limits the complexity of the model and the machine learning pipeline. Ultimately, limiting the power of features and the model can severely limit the performance. It is the age old engineering tradeoff of “power vs. speed”.
Combining the Best of Both Worlds
One solution is to combine non-real-time features and models with real-time features and models. YouTube, for instance, takes this approach with their video recommendations. The non-real-time portion allows for sophisticated features and models. This can be combined with a real-time model which, while less powerful, allows one to integrate real-time signals. This combination of non-real-time signals with real-time signals provides a great balance and often results in the best overall performance.
For example, consider a user who has visited a commerce experience several times in the past and visits again today. The past behavior can be modeled with more sophisticated features and models while the current session can be modeled in real-time with, potentially, a less powerful combination of features and models. On the other hand, a new user can only be modeled based on the current session. Without real-time inference, you can’t draw any conclusion about the optimal experience to provide.
Vidora and Real-Time
We’ve already built out several real-time features for some of our pipelines and are actively working to expand those capabilities and improve the performance of real-time machine learning. We should be making some exciting announcements in this area shortly! If you are interested in learning more please email us at info@vidora.com
