More Than a Game: Applying Analytics to Decision Making In Sports and Business (Part Two)
By Professor Eric T. Bradlow (@ebradlow) (GBK Collective and the Wharton School)
Last month, I had the pleasure of joining Wharton's panel discussion, “More Than a Game: How Analytics Gives Sports Teams a Competitive Advantage”, moderated by Wharton Dean Erika H. James with co-panelist FanDuel CEO and Wharton MBA alum Amy Howe.
In part two of this blog topic, I explore several more key takeaways from the panel including how to use analytics as a decision-making tool (even when you’re working with incomplete information), the difference between “data driven” and “data informed”, and why “gut instinct” and judgment still matter.
Better data, not big data
As I explored in my previous post, there are a number of areas where businesses can learn from the approaches used in sports analytics (and vice versa). One is the use of real-time data by sports teams to improve decision-making.
Another foundational step for any analytics project is asking the right questions: What problems do you need to solve? What data or information do you have available to apply to that problem? What other data or insights are discoverable? Last, but not least, what data or variables can you rule out?
Making successful predictions and solving problems also requires a “test and learn” mindset where brands are constantly identifying and applying better data to inform the choices they make across areas, and then running experiments to iterate and improve.
Taking a lean data approach
Another ongoing issue for brands today is ensuring data integrity and privacy. How do sports teams and brands collect and store data responsibly? I shared two perspectives on our panel.
One is that companies need to take a lean data approach, focusing only on collecting and storing the data they need to optimize the customer experience or solve other specific business problems. By default, this approach improves security, but it’s also one of the most challenging issues in computer science today. What level of aggregation or granularity is needed to ensure your data is valuable?
Photo credit: iStock
“It’s important to remember that not all data is created equal. In fact, collecting too much data can actually be detrimental to decision-making processes.”
Data security and accuracy are also pivotal issues for both sports teams and companies across industries. For example, let’s say you’re a pro sports team where every player wears biometric sensors. Well, does the team own the data? If athletes are training with sensors on, and the team decides to lower their salary or cut the player based on data they’ve gotten from sensors, that raises some privacy concerns.
In today's data-driven world, companies are collecting more data than ever before, but it is important to remember that not all data is created equal. In fact, collecting too much data can actually be detrimental to the decision-making processes.
As FanDuel CEO Amy Howe emphasized during our panel, "data is a key reason we've been successful and we continue to strengthen and protect it. At the same time, we also need to make sure we are only collecting the data that is necessary for our specific business needs and required by each state’s regulator respectively."
Solving problems with incomplete information
Another question we discussed on our panel is whether big data, AI, and analytics can truly solve business problems in areas we know data is insufficient. The reality is that nearly every situation or business decision that companies have to make is made with incomplete information. In fact, making empirically driven decisions while working with incomplete information is one of the core things we teach at the Wharton School.
As Howe shared, creating the right approach “starts with making sure that data that we are using is of the highest integrity available, and that we're applying the most accurate data signals to feed the model in real-time."
Every model has observable and unobservable factors, and the job of a data scientist is to find out where models are limited and try to posit ways of improving them by using measurable variables to proxy unobserved factors.
“Gut instincts" vs. statistics
Another interesting question the panel covered is whether “gut instinct” still matters in decision-making in light of the increasing dependency on analytics. As a statistician and economist, I believe that data and statistics act as a very good normative benchmark and that it’s vital for every organization to use them as a baseline in decision-making. But that’s not to say that experience or human intuition doesn’t matter.
As Amy Howe shared "I think one of the things that is really important to remember is analytics is not a crystal ball. It's not going to tell you everything, and it's not going to predict the future perfectly. And so, I think it's important to have a balance between using data and analytics, but also relying on your instincts and your experience."
Photo credit: iStock
“Analytics is not a crystal ball. It's not going to tell you everything or predict the future perfectly. And so it’s important to have a balance.” - FanDuel CEO Amy Howe
In other words, it's not one or the other, but a balance that can help you make better decisions. There is also a false dichotomy that algorithms can solve every problem. What can algorithms do? They can help solve prediction problems. But let's remember, there are many things that go into broad decision-making: Is it fair? Is it equitable? Is it inclusive?
When I worked with the Philadelphia Eagles, the way I always viewed the work that I did was when Mr. Lurie or Mr. Roseman needed to make a decision, I wanted them to look at the computer screen and say, here's what the model says, and use it as a decision support tool. It's a multi-attribute objective function. I can give models that have great predictions, but what does it mean for the organization more broadly? What does it mean for fairness? What does it mean for the risk of the organization? I'm all in on algorithms and using them as a decision-support tool to make judgments.
Data driven vs. data informed
Another great point that my colleague Wharton Dean Erika James made on the panel is the phrase “data-driven” we often hear oversimplifies the decision-making process leaders go through.
“I've always taken exception to the phrase ‘data driven’, because it assumes that data is always correct, and whatever the data says that's what you should do,” said James. “But that approach leaves out factors such as judgment, personnel information, history, culture, and relationships that aren't a part of an algorithm.”
“I've always taken exception to the phrase ‘data driven’… as it leaves out factors such as judgment, personnel information, history, culture, and relationships that aren't a part of an algorithm.” - Wharton Dean Erika James
I couldn't agree more. When I worked for the Eagles and people used to ask me what do you do? I would tell them: “let me describe it to you in a few sentences. When the Eagles have five minutes on the clock to make a draft pick and a scout uses their judgment and says we should pick this player. I want them to be able to look at their screen and say, this is what Eric Bradlow’s algorithm says the success of that player will be. You have 30 seconds to tell me why the data is saying one thing, and your professional judgment is saying something else.”
And that's what good managers do. We can't ignore data, we can't ignore algorithms, we can't ignore the predictions, but they are data and information and support tools to help the expert make those decisions. And that's why we often scream at our TVs when our favorite NFL team isn't going for it on 4th and 2 from the 37-yard line. We have both our intuition and data on the probability of success. Great managers use data as a support tool to make decisions, but also to consider other factors to judge the uncertainty and risk in predictions.
“As someone who runs a company every day, I couldn't agree more,” said Howe. “I love data and analytics. My bias is always to start with the data, but ultimately every decision we make as executives also requires our judgment which incorporates a range of factors beyond the data alone. Very few decisions are isolated, and we have to make trade-offs every day as a management team around how we allocate resources.”
“Depending on whether you’re considering personnel decisions, strategic priorities, or investment opportunities, it may actually lead you down different paths. I think it's important to contextualize how you're using the data and what decision you're ultimately making.”
“Ultimately every decision we make as executives requires our judgment which incorporates a range of factors beyond the data alone. Very few decisions are isolated, and we have to make trade-offs.” - FanDuel CEO Amy Howe