Marketing Tech in Transition: Artificial Intelligence
Nearly 30 years ago, when studying Artificial Intelligence (AI), we had just come out of an 80’s surge in the demand for reasoning and expert systems. We were shifting the field of study toward machine learning, neural networks and the like, with other university friends in splinter topics on computer vision and robotics. If you were a young man, like me, you liked to read and apply game theory and considered non-zero-sum games lunch conversation. I really thought we were on the precipice of something big happening. It had been 40 years since the establishment of the Turing Test and Ray Kurzweil was publishing the belief that we could build something far more intelligent than ourselves. I couldn’t wait to get started.
I ended up in marketing technology. I loved it. There was a ton of data and we were dealing with predictions and human behavior. There was a lot of linear and non-linear regression. There was cluster analysis, factor analysis, etc. It was slow, but good work and we made a difference in marketing. Budgets were allocated better. Our idea of segments or audiences improved.
AI, though, didn’t really take off like wildfire in marketing. Marketers liked having an a priori assumption about where to spend their marketing dollars and what customers like. They embraced rules and machines that automate those rules. Outsourcing marketing cognition to machine intelligence wasn’t yet trusted or embraced.
As stated in previous posts, though, things started changing rapidly with the flexibility and accessibility of cloud platforms and the scalability of Big Data architectures. With computational power at reasonable cost, the analytics community began sharing their libraries and techniques to apply on these platforms. Now analytic innovation could occur frugally and it fed experimentation from far corners. Some AI in marketing was necessary and invisible, like that used behind programmatic advertisement. Marketing started to let AI in.
Even in the past five years, there has been tremendous evolution. Streaming and analytics, for example, was limited to perhaps Twitter Storm. Now there are four or five different frameworks, like Spark Structured Streaming or Kafka Streams. This evolution creates diversity, buoyed by open source and contributions from large companies to AI dependent technologies, like IBM, Cloudera, Data Bricks, LinkedIn, Confluent, etc. Frameworks are maturing quickly and access layers are becoming more sophisticated, inferring schemas on the fly so you don’t have to load data to a large RDBMS. The frequency of update of open source allows us to both use and contribute to the frameworks. It all happens very close to developers as well, versus the “AI guru” which means iterations or adaptations happen faster.
A typical approach for us would start with a Data Scientist. They define the problem (the goal ballast), craft an approach to solve the problem and then the Data Scientist would translate the approach for an engineer to understand. Engineering will automate data transformation and preparation and then collaborate on machine learning iterations. We would use Java on Spark, for example. We’d often apply an ensemble approach, like combining unsupervised clustering with a value or attrition or potential model (or a combination of all three models) and then optimize for real-time streaming of data. We have the fortune of having vast amounts of data for the machine to learn from. Over time we can aggregate new events and pass them to a k-medoids clustering algorithm and determine if the individual flips a cluster before applying our models.
In layman’s terms, AI provides us the ability to determine customer audiences based on all their previous behaviors and their current (in the millisecond) behavior and immediately apply a prediction on their potential value or potential for attrition from a brand. It’s always sensing for new events and we’re always adapting our marketing. It’s happening in real-time. Marketing relevancy depends on recency of decisions, especially for new customers where there is a flourish of new events happening. The dynamic scoring explained above allows that to occur.
Of course, there’s more to come. We aren’t alone in quickly applying the increasingly available AI techniques or technologies. Perhaps we’re applying it all very narrowly at first, but the positive financial results will clearly mean more adoption and more comfort for marketers. Not only will AI help us with predictions and thus better promotions to customers, but it’s helping us with fraud, marketing investment decisions and real-time services of the customer. That’s the key. We’re focusing on the customer and their needs. That’s our goal ballast. If our machines can help us predict and solve for the customers’ needs or desires, I believe we will establish the right guard rails and we will continue to improve the customer experience with the brands we serve. Perhaps we aren’t on the precipice of a big general intelligence event, but I do think marketing is on the precipice of dramatic transformation fueled by AI.