Back in the 1950s, when computers were quite the novelty, some researchers were optimistic they could turn their glorified calculator into an intelligent being. Since then, artificial intelligence (AI) has built a remarkably scattered track record: they can outperform doctors at diagnosing cancers, yet can barely talk to us about the weather. The lessons are clear: tasks easy for humans are not necessarily so for machines. Understanding how and why is much more complicated but critical for applying data science and machine learning to business applications. We'll begin by looking at the relationship between intelligence and machines to explain the two main types of contributions data science can make: predictive intelligence and insights. We'll then discuss some common applications of each type, and the general limitations of machine learning.

Smart Programs vs Learning Programs

There are two categories of programs: smart programs and learning programs. While it may not be immediately evident how this relates to data science, the fundamental differences between the two lay the groundwork for understanding what machine learning can do.

"Smart programs" (a name I coined for the lack of an official one) are those that have their intelligence directly programmed by humans. My favorite example is the chess engine. These chess programs easily beat the best human grandmasters, exhibiting incredible intelligence by this standard. However, they achieve this feat by abusing the lightning-fast calculation ability of computers. They don't learn, adapt, or innovate beyond what their creators programmed them to do, and sometimes that's just fine! In fact, it wasn't until 2018 that a learning program finally ended the multi-decade reign of chess smart programs.

While chess engines are extremely complicated, many very simple systems also fit into the criteria of smart programs. Here's another example of a smart program that a banker might have written to determine whether she should give out a loan.

You may find the decision tree too simple to consider "intelligent." However, you'll find it difficult to distinguish even the simplest rules engine from a superhuman chess program except by their complexity. Both are merely instructions of human-programmed logic that make decisions. This begs the question: what level of complexity does a smart program need to attain "intelligence"? And to look at intelligence from another angle, what if this simple program has higher lending profitability than your local banker?

By this loose definition, smart programs are everywhere, and there's no clear way to categorically distinguish one from another by any metric of intelligence. So, while colloquially people will refer to the more capable of these as "AI", the experts generally consider smart programs to be just that, smart programs, not AI.

The opposite of smart programs is learning programs, those that are not explicitly programmed by their creators, but instead rely on data (data science) or experience (reinforcement learning) to come to their conclusions. Broadly, this is the domain we call machine learning. Not only do learning programs tend to outperform smart programs, but since humans aren't teaching machines, the machines can teach humans. Isn't that really cool?

Machine Learning Applications

This gives us two main advantages that learning programs have over smart programs. They are generally more capable than smart programs and are not limited to the knowledge of the programmer.

In fact, these two advantages translate directly to the two categories of machine learning applications:

• Predictive intelligence
• Discovering insights

Predictive Intelligence

Because learning programs exhibit very good and sometimes super-human performance in certain tasks, we can simply replace the human. They're already excelling in applications ranging from approving loans to diagnosing patients. As the science of machine learning grows, the universe of decisions we can delegate to the machine grows with it. Unsurprisingly, using machine learning for predictive intelligence is by far the most common and straightforward application of the technology.

Discovering Insights

Learning programs often exhibit behaviors and decision-making patterns that surprise even their programmer! While this sounds exciting, humans have generally struggled to learn from the opaque numerical mass that is most learning programs. One example is from the game of Go, in which a revolutionary learning program in 2017 defeated the best humans (smart programs never got close) and turned the time-tested strategies of the three-thousand-year-old game upside down. Experts describe the computer moves as often alien, foreign, and incomprehensible, acknowledging that the AI was perhaps too advanced for humans to understand. And while the field of "explainable AI" has made some progress in recent years, we're still far from consistently applying insights from learning programs.

What Machine Learning Isn't Good At

Machine learning has two primary disadvantages:

• Learning programs aren't always correct
• We have a hard time understanding them

The struggle for self-driving cars exemplifies the weaknesses of learning programs. Despite billions of dollars in investments, self-driving cars are still years away. While learning programs struggle on the task for many reasons, the underlying challenge is the incredibly high cost of failure. Learning programs will inevitably make mistakes and applications need to account for it. For example, Google Search can balance out its occasional mistake by making ten or a hundred great searches. Self-driving cars, in contrast, offer no opportunity for thousands of machine-quality decisions to compensate for one collision a human might have avoided and thus presents a much more difficult task for a learning program.

In addition to the pure engineering hurdle, self-driving poses a legal challenge as well. Who do we hold liable if a self-driving car kills someone? Do we fault the engineer, even if the engineer used a learning program that learned by itself? Again, we have a very tough time explaining the decisions from learning programs. As a result, we generally have a harder time using machine learning for applications requiring decision-making transparency.

AI Today and Tomorrow

Harnessing machine learning is less about math and more about the nature of computerized intelligence. Letting machines figure their own path towards truth grants us a double-edged sword offering computer-speed, quasi-human-quality decisions without any guarantee of understanding them. Whether or not you can apply the technology to your situation primarily comes down to whether the learning programs are good enough to emulate humans and if you can accept them making errors. Fortunately, the rapid developments in AI will ease both limitations, giving me great optimism that we'll see truly remarkable AIs lead the future.