It’s generally agreed upon that machine learning is really good at making predictions with extensive data sets. Humans just can’t sift through billions of data points and turn them into anything intelligible. But machines fall short with respect to running a business. You need a lot of creativity to apply machine learning predictions in a way that has a positive net effect on both your goal and the efficiency of your business operations. The Vidora team recently applied some of our creativity to one of our customer’s use cases. We were predicting the best time of day to send each of their users a newsletter. The steps we took are outlined below:
Step 1: Break the problem down
It requires a little thought and planning before getting your hands dirty with machine learning. In our customer’s case of predicting time of day to send newsletters, a decision needed to be made: use a regression model or take another approach? You could have a more complex regression model that predicted the best time down to the exact second, but after some thought, the simplest solution was going to be breaking time of day down into 4 time windows – morning, day, evening, and night.
Step 2: Add features to the pipelines
Vidora provides hooks to inject features directly into your fully automated ML pipelines, providing customers the power to improve performance with their own unique domain knowledge. Since we focused on predicting time of day using windows, we added new features that summed the total activity for each time window. One for the morning activity, one for the day, and so forth. Other features were added as well that summed activity for each day of the week. These features helped the pipelines get a more accurate signal when predicting the user’s likelihood to be active at specific times.
Step 3: Build the pipelines
Since there were 4 time windows, we built 4 pipelines. Each predicted how likely the user would be in each time window. Pretty straightforward when using Vidora – just log into Cortex, create each pipeline with a few clicks, and wait for them to finish. No need to worry about infrastructure, prepping the data, or trying to pick the best performing model. We automate all of these steps.
Step 4: Remove the overlaps
There’s a caveat with building 4 pipelines and getting 4 sets of predictions. What if a user is likely to be active in the morning and the evening? Fortunately, Vidora has made the process of comparing output quite simple by treating all your predictions as probabilities. Instead of getting a convoluted model score, your pipelines provide predictive probabilities that can be compared across other pipelines. If the end user is active in the morning and the evening, just choose the one with the higher probability. Removing the overlaps is an important step when using multiple pipelines. This is because you don’t want to send the same newsletter to a user in the morning and the evening!
Step 5: Uncover insights with Explainable ML
Understanding the output of your pipelines with Explainable ML and communicating it to others in the context of your business is paramount. You never know what you might find that could influence your deployment strategies. Here are a few interesting conclusions that the pipelines discovered in this case:
- The users most active in the night time tended to have higher feature importance scores for Friday and Saturday.
- Users more active during the evening (4 to 8pm) gravitated towards sports content. Meanwhile, the rest of the users read local news with a splash of entertainment.
- Users most active during the day were less active on their phones, compared to their computers and tablets.
Step 6: Deploy
With the users spread across 4 different time frames, the last and final step is to send those emails. If inclined to incorporate some of the insights from step 5, you can adjust your strategies a bit. In this example, maybe you want to tailor the evening newsletter with different content compared to other time windows. Or maybe you can adjust each newsletter depending on what device the user will likely use. In either case, there will always be plenty of things you can experiment with and test.
Machine learning can be burdensome for your business if not approached in the right way. But as you can see from the example above, it is easier and easier to turn what used to be very complicated problems into something more tractable. Whether you’re having issues with setting up machine learning infrastructure, engineering the right features for your models, or explaining the output, the Vidora platform provides ways to move forward with less headaches and great results.
