A Notebook and a Drone

from the human-edited captions published with the video. timestamps jump to that point in the video.

1. 0:33

   If you're like my children, you're probably full of questions, and it's probably only natural for us, humans, stuck here on this strange rock, to constantly ask why. In particular, you might be asking why I'm seated here at a table in the middle of the woods with a drone and a notebook. And you'd probably be right to ask. So I guess without further ado, I'd like to introduce you to the first recording of Wednesday in the Woods. Enjoy.

2. 1:12

   So we're here on a Wednesday in the woods, and today we're going to be talking about two really simple things that, yes, do actually correspond to a notebook and a pen and a drone. And these two things are arguably the most fundamental, but often most underlooked and underappreciated elements of developing any kind of strategy, whether it has to do with ESG or not. And those two things are taking a notebook, whether it's a literal notebook as an individual or as a team of stakeholders sitting down and going through a deliberative process, and agreeing on what your goals are. We often go about our "business as usual" lives, following processes and status quo, without actually clearly agreeing to or articulating across our teams, what our goals are.

3. 2:09

   When we pursue some activity, whether it's a machine learning model or any activity, we ought to be clear about what it is we're trying to accomplish. And when we do that, it's important that we actually formulate, to the best of our abilities in the context, an outcome, a goal that's at least somewhat correlated with a measurable concept, and that needs to be something that's, ideally, communicated across your team. In some circumstances, clearly not all goals and not all payoff functions ought to be communicated across the entire team, but in many contexts, the alignment of the team with the payoff function, both for the organization and for them, will result in better outcomes than when either no outcome or goal is set forth, or when that outcome is not clear and explicit with all of the decision-makers and actors in the organization. So again, first simple thing is to,

4. 3:12

   obviously agree first, but then to write down what it is we're trying to solve in a given context. Now, clearly there's more to goal setting than just writing down anything. So you might ask what is an example of a good goal and what is an example of a bad goal? Let's start with the easy one, which is a bad goal. Many organizations start some kind of journey towards a data strategy, data science, and they just say, "I want to have data science. I want to have machine learning," whether it's for purely marketing purposes or whether they're better, more complete intentions, but simply a lack of understanding. A goal like that is bound to start the process, but not necessarily produce a result that is a result that we want. So for example, "I want machine learning model. I want to have machine learning at my company." I need to hire a data scientist, I guess, a developer, what's this DevOps thing. You end up staffing

5. 4:06

   and investing in resources, and yes, you may develop a capability that does meet that label, but for what purpose, to what end, at what cost. And so you may have moved the needle and begun the journey in this process, but not exactly succeeded. Instead, a good goal might clearly articulate not just the capability from a qualitative sense that we hope to produce, but also what it is that would be an acceptable, let alone a great, outcome using that capability. So let's say that we want to predict the probability of failure for a part. That's a clear, stated goal, right? We want to increase our proactive maintenance and management of a manufacturing site. And to do that, yes, we would like to develop a data science or machine learning capability; arguably, these are traditional actuarial analyses that have a long history, but we could develop a

6. 5:12

   classification or regression model that suggests when in the next 30 days a part is likely to fail or which quality metrics are likely to be at what level corresponding to some proactive maintenance activities. That's an example of a good goal. It clearly articulates what it is we're trying to do, but it's still missing something. In particular, it's missing its correspondence with not just an efficiency metric, but also a financial metric. So yes, we could proactively maintain parts before they fail, impacting the actual manufacturing process, but how does that correspond to the financial metrics or, as we discussed in this week's post, other kinds of metrics such as environmental, social, and governance metrics. And so goals aren't just capabilities that we want to have or marketing press releases that we'd like to someday publish, they are clearly articulated

7. 6:17

   outcomes that, yes, do correspond to capabilities, but that ideally have some measurable threshold for acceptability or success that is either purely financial or, again, corresponds to perhaps some other goals the organization has that are truly measurable. So we know what a goal is and we know why we brought the notebook, but why is the drone here? Well, as we alluded to a moment ago, good goals have measurable outcomes and ideally measurable processes. And the question is often, how should we measure?

8. 6:57

   So should we implement employee-level metrics and begin with a bottom-up approach that results in so much data that we might not know what to do with it? In the absence of a model for this process or more understanding of the details of the actual people and processes on the ground, this data can sometimes be overwhelming or might not be utilized in a way that really warrants the cost of implementing such a measurement strategy.

9. 7:29

   Instead, we might start with a measurement strategy like aerial reconnaissance or like remote sensing, where from a top-down perspective, we look at the qualitative metrics that are best measured from a 10 or 40,000 foot level. And yes, there are details on the ground, sticks and leaves that we may not see when we're flying at 400 feet AGL or 40,000 feet or perhaps using Landsat 9 imagery, but it's a journey. I think that's the important thing that most organizations forget in the moment is in order to get to a maturity level that we are ultimately happy with, we have to take a journey and that journey often begins, from a measurement perspective, not in the weeds on the ground, looking through every leaf, but at some reasonable height.

10. 8:31

    On our next Wednesday in the Woods, I have two goals. The first goal is to walk through an actual example of a real data science context and machine learning task, of goal setting and measurement at different levels of organizational maturity. For the second goal, I plan to not get hit on the head with an acorn while taping this. So I hope you enjoyed, and I look forward to many more Wednesdays in the Woods with you as the seasons turn.