Wednesday in the Woods · episode 009
Zeitgeiß, not Zeitgeist
7:49youtube · Licensio, LLC
description
Some people call me The Jung Goat. No, not like a spritely version of fellow Wolverine, Mr. Brady. More like a curmudgeonly creature that relentlessly nibbles away at our collective consciousness.
Anyway, in that spirit, I’m here to energy-drain...I mean, discuss with you why we should be thinking Zeitgeiß, not Zeitgeist.
That’s right. For our Thanksgiving #WednesdayInTheWoods, we’re going to talk turkey about why your organization needs to think Goat - barnyard animal, not magical spirit.
See, when it comes to #modeling and #strategy, we often focus on the models and approaches in front of us. Lately, these models and strategies have taken on an occult power. Is #DigitalTransformation alchemy? Are #deeplearning terms meant to convey mystique?
Yes, #GPT3 may actually seem like a magical ghost spirit with the powers of AGI. but it’s also front and center because it’s captured our collective consciousness - and now, is writing it.
None of this is to say that we shouldn’t learn from research and best practices. But, the real world over time is like a goat. If you’re not prepared for it - and even when you are prepared - it will do goat things.
What are goat things, you ask?
First, goats like to jump fences. Like a pasture, you might think that your world and its data are bounded. Your #assumptions herd you in. But to the goat, your conceptual fence is simply decorative. For example, in many term structure models used in finance, interest rates cannot be negative. Oops!
Goats also love to kick things. Think of them like the original chaos monkeys. In our prior episodes about CNAFFM, we talked about how good production testing might include fuzzing with random or pathological test #data. You need at least one person on your team who is at least *thinking* about kicking everything. Just get them to write down what they’re visualizing instead of letting them actually do it.
When they’re not busy jumping and kicking things, they also love to taste everything. In this sense, goats exemplify the mantra of “know thy data.” Most normal species of animal, data scientists included, stick to normal, edible-looking things that their peers consume. Goats are unencumbered by this reasoning, driven by their curiosity to lick or chew literally anything they can get into their mouth. You need at least one team member with the same kind of insane curiosity to comb over every detail in your data and assumptions.
Finally, goats love to test hierarchies. You may have built a model and it may have been the best performer. But did you test it today? Did you test it by jumping on top of it or ramming your head into it over and over again? That’s what a goat would do. Similarly, your team should make sure that you have a similar process in place to test challengers, though perhaps slightly more polite.
So next time you put together your team, make sure to bring along a Goat, maybe two.
Happy Thanksgiving!
transcript
~972 words · showhide
from the human-edited captions published with the video. timestamps jump to that point in the video.
- 0:29
Hello and welcome to the Thanksgiving edition of Wednesday in the Woods. Today, I want to talk turkey with you about why your research team needs more barnyard animal, not more magical spirit. In other words, more zeitgeiß, not zeitgeist. So, what do I mean by this? Well, we seem to spend a lot of our time these days focusing on modeling with the data that we have, not the data that we could have or the data that we might have or the world that we might operate our model in.
- 1:04
And and I think this comes down increasingly to our focus on the mystique and the occult of some of the techniques, in particular related to deep learning, that we are increasingly interested in. Deep learning is the Zeitgeist of our time in many senses, both capturing our attention and increasingly creating that which our attention is directed at. However, I think we need more Goat. Yeah, you heard me right - goat - the german word Geiß, not geist.
- 1:39
What I mean when I say that is we need people who exemplify the behavior of goats in the barnyard. So, what do goats do? First of all, goats are something that Jill still does not let me get. Next spring, we'll have some additions to the farm here, and I hope to bring you actual goats as part of footage in the near future. But what do goats do? First of all, goats like to jump fences. What do I mean by that? Goats would literally take these two lines of fences and jump them.
- 2:16
What does this have to do with your research team? Well, in many contexts, we operate within four corners that we create or fence or bound in our problem space or in our data or modeling techniques. And when we reach the real world, those boundaries either were never real - defense posts might be rotten - or the actors or objects in the real world might jump them like a goat. Always think like a goat and try to jump the walls that you've created in your modeling and your hypothesis and your problem statement.
- 2:55
A good example would be in the context of the Cox-Ingersoll-Ross model, the CIR model in term structure modeling - there's an assumption that interest rates will always be non negative. The modern world is a goat that has clearly jumped the negative interest rate fence. The second thing that goats do is that they kick stuff. Goats are the original chaos monkeys - chaos monkeys, a fun name for a paradigm that's obviously been popularized by the FAANGS, but the original chaos monkey is the original goat.
- 3:31
And goats are always trying to break things. So when you build a model, part of our CNAFFM-MMM-PPP paradigm, you go through production testing - it's one of the P's, I'm sure you remember. So production testing, as we discussed, might include things like fuzzing or introducing pathological data to your model to see how the model performs. Minimally, you should be aware of when the model is doing something weird, but you really ought to be testing your model by kicking it just like a goat would before it goes into production.
- 4:09
And as a part of your continual monitoring in production, you should be kicking it. It's very similar to the chaos monkey ideas that you've probably heard about, but I think the goat is a much better analogy, because the goat is the original chaos monkey. The third thing that goats do is that they taste everything. They're insatiably curious, in a literal sense. And so the goat is the kind of data scientist that you want.
- 4:37
The goat never assumes that a column has a certain type of value. The goat goes and he chews on it, he licks it, he tastes it, and he checks to see if the data model actually has the things in it that you were told. Sometimes that might backfire on the goat, a goat might spend a lot of time tasting things that don't provide a lot of calories, or the goat might eat something that doesn't make it feel good. But the goat is out there, never assuming that its environment is what it's told.
- 5:11
And I think that's a very important aspect of a team. Everyone on your team can't be going around tasting everything or licking everything, for a wide variety of reasons, including human resources reasons, but somebody on your team needs to be out there, never taking the assumption for granted. The last thing that goats do here in this context today is that goats challenge hierarchies. Goats are never content with the established status quo.
- 5:41
And it's important that when you build a modeling practice that goes into production, you take that hierarchy, the dominant model and you test it. Maybe you don't want to mount the model, maybe you don't want to butt it in the head, but you want to be allowing a challenger model to to prove its alpha or dominant status in your modeling hierarchy and you don't want to do that the way goats do. Trust me - having seen goats and ducks and other barnyard animals test hierarchies, I strongly recommend you take a little bit more formal approach to testing, but nonetheless, you do want to regularly be checking that the model that you assumed was the alpha or the dominant model is still the dominant model over time.
- 6:29
So, with that, I hope everyone enjoys their Thanksgiving and, by the time Christmas rolls around, I hope you have a goat on your team - not to eat, but to collaborate with. We can work on the other part later. See you next time and enjoy.