Better Decisions, Explaining AI, and Overfitting
Lemmalytica, No. 1
In the inaugural issue of Lemmalytica, I thought we’d start off with what is arguably the most critical part of making good decisions: asking the right question. Let’s dive in.
Asking the Right Question
At its core, decision-making is about answering questions. Should we launch this product? Should I order take-out for dinner? Should I go on this date? These are conditional questions that reflect some number of possible futures. The decision occurs when you select one of the futures and take the action that it prescribes.
From a mechanical point of view, decision-making is straight-forward. Each time you come to a branch in the road, you select one of the options and proceed onwards. If you aren’t concerned with the ultimate destination, it’s a simple process. Of course, most of us are concerned with the destination, and it may be that not every branch in the road will get you to where you’re going.
Now, we start to see how complex decision-making really is. Knowing that there is a branch in the road isn’t enough--you need more information. Does the left branch lead in the direction you’re going? Does the right branch? Is one of the two routes longer? Safer? More scenic? In short order, your top-level question has devolved into a series of sub-questions.
If the top-level question is what defines your decision, then the sub-questions are what define whether you’ll make a good decision. It’s here where things become tricky. If you fail to ask the right sub-questions, then you may miss critical information. If you ask too many sub-questions, then you may never make a decision at all (which is often worse than making the wrong decision.)
Let’s look at an example. Imagine that you’re planning a vacation and you ask yourself, “should I go to Athens?” What information do you need to make a good decision? First, you might ask “what is the cost?” (and it’s various constituent questions--cost of hotel, airfare, average meals, souvenirs, etc.) Then, you might look at the weather for the time you plan to travel. Will it be rainy? Will it be hot? Relying on the advice of the people you trust, you might ask “do I know anyone who has been?” And more importantly, “did they have a good time?”
The list of questions could go on forever--and as we’ve established, at some point you have to stop asking questions and make a decision. That’s why it’s critical to take your priorities into account during the information-gathering phase. Here, we come to yet another question: “what information is going to have the most impact on the outcome of this decision?”
In the case of your vacation, a positive outcome is presumably your own personal enjoyment. If you hate museums, then perhaps you shouldn’t bother asking “does Athens have good museums?” This is a low-priority question, which is either a waste of time because you won’t go to any of them, or it’s a negative outcome because if you do visit the museums then you won’t enjoy yourself.
Rather than superfluous information-gathering that will complicate your decision-making, you want to focus on high-priority questions. For example, if you love being outside, then the question “is there good hiking near Athens?” probably ought to be one of the first things you ask. High-priority questions produce useful data that will inform your decision-making and lead to a positive outcome.
(For the record, Athens is awesome. The food alone is worth the trip.)
In the vacation example, judging your priorities is easy because you’re operating in a personal context. You know yourself best, so establishing a hierarchy of priorities is instinctual. The same is not true in most other contexts. This is why it’s critical that you have expertise in the domain associated with your decision.
In organizations, being a great decision-maker means knowing how to set priorities in the context of your organization. Unsurprisingly, this leads us to a whole host of other questions. What kind of organization are you operating in? What is its mission? What are its goals? What strategy is the organization using to reach those goals? Answering these questions isn’t always straightforward. It requires that you have a thorough understanding of all aspects of your organization--not just your particular function.
By building a picture of your organizational context, you’ll be able to more effectively evaluate priorities. In turn, these priorities help you gather the right information when you need to make a decision. And as we have seen, the information-gathering phase is critical to making a good decision, which will ultimately lead to an outcome that advances your organization’s goals.
In our discussion, we started with a top-level question (the decision you need to make), then discussed sub-questions (information gathering), and then reviewed priorities (context). In practice though, this is a bit backwards. Making a good decision, in a methodical fashion, requires starting in the right place and answering the questions in the right order. In general, it looks something like this:
Recognition. Recognize that you’ve come to a branch in the road and a decision is necessary. In order to move forward, you have to make some selection and then take the corresponding action.
Context. Use your domain expertise to evaluate the context of your decision. This could be organizational (“should we launch this product?”), or it could be something more fleeting (“should I order pizza for dinner?”). In either case, knowing the context will help you in the next step.
Priorities. Within the context of your decision, what priorities should you seek to maximize? It could be a question of cost, efficiency, personal enjoyment, reputation, or one of many other considerations. Your job is to identify the considerations that matter most, and therefore are most likely to lead to a positive outcome.
Information gathering. Given the selection that you have to make, break the decision down into sub-questions that address your priorities. The sub-questions don’t necessarily constitute actions themselves, rather, they provide you with information that you can use in selecting an action.
Maximize positive outcomes. Using the information you gathered, evaluate what it tells you about your possible futures. Is one of them clearly better than another? Does one maximize attainment of your desired outcomes?
Act. Now you’re ready to select a path forward. Move ahead decisively, because another decision is almost certainly right around the corner.
One can see how gathering information before thinking about priorities might lead to wasted time (or bad information.) Similarly, trying to maximize positive outcomes before asking sub-questions will inevitably fail. The order matters because each step flows naturally into the next.
Decisions + Life
Making decisions is a fundamental part of existence. As individuals, we make decisions all day long—life is just a series of decisions strung together, each of which opens up some new set of available paths. The same is true for organizations. In truth, this never-ending process can be exhausting. Thankfully, not every decision requires a methodical six-step process. But for those that do, following the process is more likely than not to produce a better outcome.
Note: This article originally appeared on Medium.
Paper of the Week
This week, I’ve been thinking a lot about artificial intelligence (AI) policy and how we can use AI to produce better outcomes for society. The good folks at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) are helping me refine that thinking. For Lemmalyitica No. 1’s paper of the week, I’ve chosen Explaining Explanations to Society (Gilpin, Leilani H., Testart, Cecilia, Fruchter, Nathaniel, Adebayo, Julius; 19 January 2019). It’s a great read and I highly recommend that you check it out. To whet your appetite, here is a quick summary, with a big thanks to the original authors, who deserve all the credit.
Summary: Explaining Explanations to Society
As AI programs become more prevalent, policy, law, and regulatory considerations will force AI developers to ensure that their systems are transparent. To this end, explanatory artificially intelligent (XAI) systems must expand the scope of their explanations to target the end-user and policy-maker, rather than merely the programmer and expert.
Existing XAI systems are overly-focused on inside explanations, that is, explanations aimed at technical experts. These explanations generally fail to address the critical why questions that are relevant to outside audiences. A proper explanation for outside audiences must be interpretable, meaning that it is understandable to humans, and complete, meaning that it is true to the model.
As non-experts who are nonetheless subject to the decisions made by AI systems, outside audiences have different explanatory needs. For example, in an automated decision-making process to approve or deny loan applicants, a subject may want to know why they were denied, rather than simply the result of the decision. Further, a sensitivity analysis may be relevant so that the subject knows what they could do to have the loan approved in the future. Similarly, in a self-driving car accident, outside audiences want to know what factors led to the accident, and how an accident can be prevented in the future. This is a distinct question from purely technical explanations regarding which system or sub-system failed.
XAI systems that do not satisfy outside audiences are likely to result in a trust gap with the end-user and policy-maker. General explanations are not enough to build trust in models. For example, a misalignment in expected data and the actual training data can lead to systems that focus on the wrong feature (ie. the Amazon recruiting algorithm that learned to favor male candidates.) A classification system may be accurate, but if we don't understand how it is making decisions, and whether it is making those decisions for the right reasons, then we can't trust the system.
In order to build trust, we should judge the behavior of an algorithm in a similar fashion that we would judge a person working under similar circumstances. We must be able to ask the AI system the same types of questions, and expect to get the same type of explanatory answers. In this fashion, we can build trust and confidence that we understand how an AI system is making decisions, and moreover, that we have recourse regarding those decisions.
Term of the Week
Whether you’re a technologist, a business-person, or just someone trying to get by in the modern world, understanding the lingo is critical.
Overfitting is a phenomenon in statistics and machine learning, wherein a model maximizes performance on training data to the extent that it is unsuitable for unseen data. Overfitting prevents the model from generalizing well because it corresponds too closely to a particular dataset, thus preventing it from accurately predicting future observations.
In overfitting, a model can be thought to have "memorized" the training data rather than "learned" the general patterns in the training data. Overfitting is a common problem in machine learning and can occur for a variety of reasons, including the presence of too many adjustable parameters. An optimal model may ultimately need less data and flexibility than an overfit model. For this reason, nonparametric and nonlinear functions can be more susceptible to overfitting than linear models.
Training data can be thought of as consisting of two categories: information that is relevant to the future; and, noise that is irrelevant. In overfitting, a model learns too much of the noise, which makes it less reliable on unseen data. Compare this to underfitting, where the model doesn't learn enough about its training data and thus performs poorly even on training data.
Several techniques are available to address overfitting, including cross-validation, pruning, regularization, Bayesian priors, and more. Validation data is particularly useful to identify model performance.
 Programming PyTorch for Deep Learning; Pointer, Ian; O’Reilly Media
 Overfitting; Wikipedia
What to Read
This week, my reading recommendation is The Information: A History, A Theory, A Flood, by James Gleick. Prior to reading this book, I knew almost nothing about information theory. Now, I’m kind of obsessed. The Information is an engrossing review of the meaning, evolution, and organization of information. Beginning with a definition of the term "information" itself, Gleick leads his readers on a tour through communication methodology, information theory, and the degree to which we control (or don't control) our own thoughts. The Information is vital reading for the modern age and will open your eyes to the ways in which information is the currency of our time.
Thats it for this week! I’m Severin and I hope you’ve enjoyed Lemmalytica, No. 1. For more of my writing, you can find me on Medium. And if you have questions, comments, thoughts, or just want to connect, feel free to hit me up on Twitter.