Closing the Gap Between Machine Learning and Business | by Stephanie Kirmer | Jan, 2024

Now that many of us are returning to the office and getting back into the swing after a winter break, I have been thinking a bit about the relationship between machine learning functions and the rest of the business. I have been getting settled in my new role at DataGrail since November, and it has reminded me how much it matters for machine learning roles to know what the business is actually doing and what they need.

My thoughts here are not necessarily relevant to all practitioners of machine learning — the pure research folks among us can probably move along. But for anyone whose role is machine learning in service of a business or organization, as opposed to just advancing machine learning for its own sake, I think it’s worth reflecting on how we interact with the organization we’re a part of.

By this, I mean to say, why did someone decide to hire your skillset here? Why was a new headcount called for? New hires aren’t cheap, especially when they’re technical roles like ours. Even if you are backfilling a role for someone who left, that isn’t guaranteed to happen these days, and there was probably a specific need. What was the case made to the purse-string-holder that someone with machine learning skills needed to be hired?

You can learn several useful things from looking into this question. For one, what are the ideal results people expect to see from having you around? They want some data science or machine learning productivity to happen, and it can be hard to meet those expectations if you don’t know what they are. You can also learn something about the company culture from this question. Once you know what they thought the value would be of bringing in a new ML headcount, is that thinking realistic about the contribution ML might make?

Besides these expectations you are walking into, you should create your own independent views about what machine learning can do in your organization. To do this, you need to take a look at the business and talk to lots of people in different functional areas. (This is in fact something I spend a lot of my time doing right now, as I’m answering this question in my own role.) What is the business trying to do? What’s the equation they believe will lead to success? Who is the customer, and what is the product?

Somewhat tangentially to this, you should also inquire about data. What data does the business have, where is it, how is it managed, etc. This is going to be really important for you to accurately assess what kind of initiatives you should focus your attention on, in this organization. We all know that you having data is a prerequisite in order to do data science, and if the data is disorganized or (god help you) absent entirely, then you need to be the one who speaks up to your stakeholders about what the reasonable expectations are for machine learning objectives in light of that. This is part of bridging the gap between business vision and machine learning reality, and is sometimes overlooked when everyone wants to be full steam ahead developing new projects.

Once you get a sense of these answers, you need to bring to the table perspectives on how elements of data science can help. Don’t assume everyone already knows what machine learning can do, because this is almost certainly not the case. Other roles have their own areas of expertise and it’s unfair to assume they will also know about the intricacies of machine learning. This can be a really fun part of the job, because you get to explore the creative possibilities! Is there the hint of a classification problem somewhere, or a forecasting task that would really help some department succeed? Is there a big pile of data sitting somewhere that probably has useful insight potential, but no one has had time to dig around in it? Maybe an NLP project is waiting in a bunch of documentation that hasn’t been kept tidy.

By understanding the goal of the business, and how people expect to achieve it, you will be able to make connections between machine learning and those goals. You don’t need to have a silver bullet solution that’s going to solve all the problems overnight, but you’ll have a lot more success integrating your work with the rest of the company if you can draw a line from what you want to do to the goal everyone is working towards.

This may seem like a left-field question, but in my experience, it matters a great deal.

If your work isn’t both aligned with the business AND understood by your colleagues, it’s going to be misused or ignored, and the value you could have contributed will be lost. If you read my column regularly, you’ll know that I am a huge booster for data science literacy and that I believe practitioners of DS/ML bear responsibility for improving it. Part of your job is helping people understand what you create and how it is going to help them. It is not the responsibility of Finance or Sales to understand machine learning without being given education (or ‘enablement’ as many say these days), it is your responsibility to bring the education.

This may be easier if you’re part of a relatively mature ML organization within the business — hopefully, this literacy has been attended to by others before you. However, it’s not a guarantee, and even large and expensive ML functions within companies can be siloed, isolated, and indecipherable to the rest of the business — a terrible situation.

What should you do about this? There are a number of options, and it depends a lot on the culture of your organization. Talk about your work at every opportunity, and make sure you speak at a lay-understandable level. Explain the definitions of technical terms not just once but many times, because these things are challenging and people will need time to learn. Write documentation so people can refer to it when they forget things, in whatever wiki or documenting system your company uses. Offer to answer questions and be sincerely open and friendly about it, even when questions seem simplistic or misguided; everyone has to start somewhere. If you have a base level of interest from colleagues, you can set up learning opportunities like lunch and learns or discussion groups about broader ML related topics than just your particular project of the moment.

In addition, it’s not enough to just explain all the cool things about machine learning. You also need to explain why your colleagues should care, and what this has to do with the success of the business as a whole and your peers individually. What is ML bringing to the table that’s going to make their job easier? You should have good answers for this question.

I’ve framed this in some ways as how to get started in a new organization, but even if you’ve been working on machine learning in your business for some time, it can still be useful to review these topics and take a look at how things are going. Making your role effective isn’t a one-and-done type deal, but takes ongoing care and maintenance. It gets easier if you keep at it, however, because your colleagues will learn that machine learning isn’t scary, that it can help them with their work and goals, and that your department is helpful and collegial instead of being obscure and siloed.

To recap:

• Find out why your company has hired for machine learning, and interrogate the expectations underneath that choice.
• Understanding what the business does and its goals are vital for you to do work that will contribute to the business (and keep you relevant).
• You need to help people understand what you’re doing and how it helps them, because they won’t magically understand it automatically.