AI Winters: Structural or cyclical?

from the human-edited captions published with the video. timestamps jump to that point in the video.

1. 0:07

   Good morning and Welcome to Wednesday in the Woods. Good morning and welcome to Wednesday in the Woods. If you're like me, you probably spend your time thinking about the most important questions we face today. Things like: is there an AI winter coming? And today, I want to talk to you about winter and AI winters. You see, many people refer to an AI winter - a period of colder reception to technology in general, a period where investment slows, a period where adoption slows.

2. 1:01

   And the question is, are these things cyclical? Does the word "winter," in its connotation of seasons, make sense? Or is this actually really more of a structural problem that we could change or create? And I think it's important to remember that these winters, these AI winters may be a function of a broader technological winter but have traditionally been discussed in the context of solely an AI winter. So, let's talk about this again, the structural versus cyclical aspect of an AI winter.

3. 1:40

   And You see, this is a part of a larger area of research around technology adoption and technology diffusion, and these model types - the structural and cyclical - exist in the broader discussion around technology adoption. Generally, in some areas, people look at technology as being adopted or being developed in waves. Those waves maybe actual adoption, or they may be the derivative or the rate of change in technology adoption.

4. 2:10

   In other cases, for example, adoption of certain technologies historically, like an agricultural situations, these diffusion or adoptions, may be much more binary, more structural, in the sense that there's a before and an after and the after is the after, not a wave or a cycle. And these models often don't talk, right. There are very few models if any, that are capable of producing both structural in the binary sense, and cyclical in the periodic sense, phenomenon when the models are run.

5. 2:47

   And all of this has to do in a more general sense with discussions of philosophy of science. So AI is a product, generally at the beginning of the pipeline, of research. And the question in science is - is this research a kind of a linear or a small step, almost an evolutionary mutation process of growth, as Popper, Karl Popper posits? Or is it more of a kind of a a multi-dimensional exploration, punctuated by periods of very large, seismic growth or change - often called paradigm shifts?

6. 3:27

   And in my opinion, both of these things are true, at different points in time, in different parts of science, and often simultaneously, there is both. This kind of micro evolution as Popper posits, as well as this macro or seismic evolution as Kuhn posits. And the the way that AI research obviously follows has mirrored this. So, neural networks, for example, were a technology that was a species, if you will, like mammals, that may have existed.

7. 3:58

   There was a long period of time where they were not dominant; for example, expert systems or other statistical approaches dominated, and then subsequently, following some exogenous change, say, the availability of GPUs or GPU computing approaches to fitting such models, we now see a world where deep learning models are dominant, the dominant kingdom or the dominant species, if you will, in the world. So just like mammals, once a kind of a second-order, second-fiddle species or type of organism, today, after some exogenous change, are much more in the driver's seat.

8. 4:35

   So to bring all this back, research is clearly where many of these ideas begin. And so what we do in research certainly must contribute structurally to what we subsequently observe through these cycles of technology adoption. Whether it's a hype cycle, like the Gartner hype cycle or not, is obviously a matter of opinion or fact to the extent that we observe evidence. But research is where these ideas begin. And so, it's important that we think as a as a group of researchers and as a society about what we're doing when we fund research, when we match faculty with graduate students or researchers in industry with researchers in academia, because if we, for example, don't have a robust research ecosystem, if we put too many eggs in one species basket and some exogenous event occurs - for example, privacy regulation that enforces a standards with respect to

9. 5:35

   the transparency of model types - then we may be left with very few healthy organisms left in such an environment. But research, research is where this begins - but it's not where it ends. As the Gartner hype cycle, for example, posits there's the pipeline from research begins to transition into adoption in industry. And this is where the hype cycle typically begins, where researchers may or may not exaggerate their claims and publication.

10. 6:08

    But once taken into the industrial or business context, it is often the case that such claims or applications rapidly lose touch with the reality of the research. So, when the research task is better aligned with the real world use cases, I think we're safer. When the data used in the research, whether it's data used for assessment or data used in the training portion of the task, the closer that is to the real world, the better. When we assess what our models are doing, the closer that assessment is to whatever the value proposition or propositions are in the industry, the better, the less likely there is to be, let's say, a hype cycle where we get out over our skis and may stumble.

11. 6:59

    And lastly, the more transparent, the more clear the publications are related to the research, the less likely I think we are to get out over our skis. When, for example, publications clearly articulate where not to use or where weaknesses are known to be in models or in techniques, then the simpler it is for others and industry to either clearly defend why something shouldn't be done or to understand originally that there are limitations to use cases.

12. 7:34

    But in industry there is agency, right? There's autonomy. And frequently, we see that many of the issues when it comes to so called "winters" arise, let's say, from too much capital chasing too few ideas or too many sales and marketing approaches that are not aligned with the reality. That, for example, overpromise and underdeliver when it comes to the actual sophistication or complexity or investment required in a project, or in the actual deliverable or values.

13. 8:10

    And these are all really structural reasons, right? These are things that we can do as researchers, things that we can do as an industry, ways that we can organize our organizations, ways that we can allocate the investment that's made available to us, so that we we have robust research and we emphasize and properly fund the other parts of the pipeline like communication and assistance with industry. But, to be fair, there are other structural reasons why things may or may not be adopted or why adoption may may not be so good. For example, in the case of the hype cycle and winters, there are reasons we don't have control over typically, right?

14. 8:50

    So for example, if you're bringing a technology to market in a highly concentrated monopolistic or oligopolist environment, then adoption is probably going to look different than a fragmented environment full of competition. For example, in environments where there's only two or three players, you really only need to get one or two of the market participants in the monopolist position, whether it's a monopolist buyer or monopolist seller, in some sense, to adopt or to desire the adoption of the technology.

15. 9:24

    And you might see rapid adoption, it might be easier to standardize, to enforce what's necessary. And for example, data exchange, to get the value that's promised out of the actual adoption processes. On the other hand, in highly competitive markets, it may be difficult to get the collaboration or coordination necessary for the market participants to experience the benefits that are promised. And so, that's a structural aspect of the markets that we actually try to address.

16. 9:56

    The customers that we try to work with that we can't really control as researchers or an industry when we try to commercialize ideas separately. And even more exogenously and out of our control is the fact that so much of this is driven by the macroeconomic cycles. So, to the extent that the macroeconomic world still experiences traditional cycles - which we'll see where we land and whether the cycles look the same or whether the peaks and troughs look a little different -but to the extent that there are cycles in the macroeconomic environment, they clearly drive access to capital capital, for research, for company formation, for investment within companies, and obviously to the extent that companies purchase something.

17. 10:40

    The actual buying function for the commercialization of AI or technology generally. We can't control that stuff, right? So, I guess to bring it back to the original question is an AI winter necessary? Is it is it fated? Is it really seasonal and cyclical or their structural elements? I'd have to say that, my opinion is that like so many things, it's both. There are cyclical elements, some of which are endogenous to AI and technology and research and commercialization, but there are also exogenous cyclical factors like the macroeconomic environment or social perception of things like privacy separately.

18. 11:24

    There are structural elements and what we do when we make choices as individual, companies, or as research communities from the structural perspectives - where we put funding or how we decide to go to market - do impact how quickly the hype cycle might occur. How big the difference between what we promised and what we delivered is obviously is correlated to the degree to which there is regret, remorse, negative PR, et cetera among buyers.

19. 11:56

    And so, we can control those things. There are both structural and cyclical elements and while we can't control most of the factors and some of them are truly outside of the control of our community, there are many things that we can do that will improve or alter the depth or duration of an AI winter or technology winter should it occur. And so, I think it's an interesting question. It's a perennial question. It's a question that's important for us not to forget as we're again, starting in research, allocating funding and working our way through, investing in ideas or bringing ideas to market in an operating company sense, or as a buyer, selecting or purchasing technologies.

20. 12:41

    But, at the end of the day, there will probably be an AI winter. It may or may not be a part of a broader technology winter. The drivers could be purely economic, they could be societal, they could be technical. And, over time, probably all three things will occur, right? We will have purely cyclical, purely structural, and mixed winters that will occur. And so, I hope we'll do what we can to minimize or avoid those winters where they may occur, but we should expect them, like the seasons, to simply occur at some point.

21. 13:18

    Hopefully, it'll be a mild one. This one isn't so far. Signing off. See you next week, and enjoy.