Research vs. Application in AI
The Artificial Intelligence world consists of two schools of thought: research and application. This is an important distinction, where the implementation of AI will be the focus of this. When looking at the application side, we are looking at consulting firms and in-house data science teams; with their focus on leveraging AI to provide solutions that solve business-level problems.
The Overemphasis on Models
Something that is evident in the Artificial Intelligence sphere is that data scientists, and more specifically, AI companies, are placing too much emphasis on model development and hyper-parameter tuning. There are two other key cogs in the wheel that are needed for a successful adoption of AI in any business, and these are often overlooked.
When you think of AI, you think of revolutionary technologies that are changing the face of the world as we know it. What is the engine that is driving these quantum developments? Most people will immediately think that it’s the model! The neural network or the support vector machine, this is what must be driving it, right? Well, yes… partially. But this is not the end of the story. One could argue that the data is the one driving these developments; our society is generating data at an exponential rate and placing more and more emphasis on the curation of this data. Parallel to this, we are seeing these revolutionary technologies starting to emerge. The model and the data have a symbiotic relationship, where one cannot exist without the other; and yes these technologies cannot be developed with just data alone, but it is evident that the bias is often placed on the model by both the companies beginning their AI transformation and the teams providing the solution. The model and the data are the first two cogs in a successful implementation of AI within a business, with the third being the pipeline or solution-architecture.
One example that I always use with clients to illustrate the importance of data and deployment framework is the story of Google and its founders, Larry Page and Sergey Brin. I remember hearing an interview of which, for the life of me, I can’t find again or remember who was telling the story. But they spoke of talking to Page at a party in the early 2000s about his relatively new company, Google. As best as I can remember it, when Page is asked why he is venturing into the search engine space, he replies with something along the lines of, “We aren’t entering the search engine space, we are entering the AI space”.
Google's Data-First Approach
To validate this story, Page in 2000, when talking to Online Magazine, said that, “The ideal search engine is smart, it has to understand your query, and it has to understand all the documents, and that’s clearly AI.” All this goes to say that at its inception, Google was seen as an AI company; yet Page and Brin understood the importance of data (and what better way to collect data about human behaviour than a search engine tracking every searchable thought) and the importance of a vessel for delivery of this AI. Since Pichai took over as CEO in 2012 and set out to make Google an “AI first” company, we have seen their AI capability mature with the advent of in-house teams, like DeepMind, or technologies, like the predictive suggestions in Gmail. However, all of this wouldn’t have been possible without the data-mining and data-curation that had been happening since the inception of Google in 1998. This is why Google has exponentially crushed their opposition, Bing and Yahoo!; and, in turn, with more people shifting to Google, it had the ability to gather more and more data, which further improved the search engines’ capabilities to beyond what either competitor could compete with.
Research vs. Application in AI
The Artificial Intelligence world consists of two schools of thought: research and application. This is an important distinction, where the implementation of AI will be the focus of this. When looking at the application side, we are looking at consulting firms and in-house data science teams; with their focus on leveraging AI to provide solutions that solve business-level problems.
The Overemphasis on Models
Something that is evident in the Artificial Intelligence sphere is that data scientists, and more specifically, AI companies, are placing too much emphasis on model development and hyper-parameter tuning. There are two other key cogs in the wheel that are needed for a successful adoption of AI in any business, and these are often overlooked.
When you think of AI, you think of revolutionary technologies that are changing the face of the world as we know it. What is the engine that is driving these quantum developments? Most people will immediately think that it’s the model! The neural network or the support vector machine, this is what must be driving it, right? Well, yes… partially. But this is not the end of the story. One could argue that the data is the one driving these developments; our society is generating data at an exponential rate and placing more and more emphasis on the curation of this data. Parallel to this, we are seeing these revolutionary technologies starting to emerge. The model and the data have a symbiotic relationship, where one cannot exist without the other; and yes these technologies cannot be developed with just data alone, but it is evident that the bias is often placed on the model by both the companies beginning their AI transformation and the teams providing the solution. The model and the data are the first two cogs in a successful implementation of AI within a business, with the third being the pipeline or solution-architecture.
One example that I always use with clients to illustrate the importance of data and deployment framework is the story of Google and its founders, Larry Page and Sergey Brin. I remember hearing an interview of which, for the life of me, I can’t find again or remember who was telling the story. But they spoke of talking to Page at a party in the early 2000s about his relatively new company, Google. As best as I can remember it, when Page is asked why he is venturing into the search engine space, he replies with something along the lines of, “We aren’t entering the search engine space, we are entering the AI space”.
Google's Data-First Approach
To validate this story, Page in 2000, when talking to Online Magazine, said that, “The ideal search engine is smart, it has to understand your query, and it has to understand all the documents, and that’s clearly AI.” All this goes to say that at its inception, Google was seen as an AI company; yet Page and Brin understood the importance of data (and what better way to collect data about human behaviour than a search engine tracking every searchable thought) and the importance of a vessel for delivery of this AI. Since Pichai took over as CEO in 2012 and set out to make Google an “AI first” company, we have seen their AI capability mature with the advent of in-house teams, like DeepMind, or technologies, like the predictive suggestions in Gmail. However, all of this wouldn’t have been possible without the data-mining and data-curation that had been happening since the inception of Google in 1998. This is why Google has exponentially crushed their opposition, Bing and Yahoo!; and, in turn, with more people shifting to Google, it had the ability to gather more and more data, which further improved the search engines’ capabilities to beyond what either competitor could compete with.
In a roundabout way, this perfectly illustrates the importance of the other two drivers mentioned prior. Yes, Google was a pioneer by not using the traditional search ranking techniques of the time (ranking a page by how many times the searched words appeared on said page), and instead developing and leveraging what they initially called “PageRank”: A technique that determines a page’s relevance by the number of pages and the importance of those pages that linked back to the initial site. But they also placed equal, if not more, relevance on the data-curation. And finally, they had a vessel to deliver this to the world.
Why Data Curation Comes First
At the core of any artificial intelligence solution is data, and often something that is misunderstood by those that have no experience in the field. Therefore, when beginning a journey to AI capability, the first step should always be to understand and curate the data needed to solve the problem. Don’t waste resources on trying to ‘develop’ AI without first addressing this challenge. Understanding the data isn’t just knowing you need it: it is understanding the quantity needed to train an effective and efficient mode. It is understanding the pipeline that can be developed to continue this flow of data for inference and further training once implemented. It is understanding what data structure is needed for the solution. It is understanding how and where you will store and persist this data. You can’t build a house without sturdy foundations, and the same goes for building AI capability or developing a solution. If you can solve this challenge early in your journey, it will enable an efficient and effective development process down the line.
"If you have the foundations and the house ready to be built, and you don't have a block of land to build on, there is no point continuing."
Secondly, it is important to conceptualise early how you will implement the solution so the team can develop towards a goal. Knowing you want to use AI and that you have the data isn’t enough anymore. The developed solution might have a large volume of perfectly structured data and perform beyond expectation in testing; however, if you have not considered how you will apply this within the business then you will quickly see it fail. To simplify this, it is important to consider the upstream and downstream interactions: how the data will be accessed or served to the system. How the user will interact with the solution, or how it will notify of the results. The infrastructure capable of scaling to the demand, or provisioned for the life of the solution. Whether it will be cloud-based or on-premise deployment. These need to be considered in conjunction with anything you consider ‘doing AI’, as they can drastically change the nature of the solution as well as the allocated budget for a solution. To continue the analogy of the house; if you have the foundations and the house ready to be built, and you don’t have a block of land to build on, there is no point continuing.
The Role of AI Providers in Successfull Adoption
All this goes to say that for a successful integration of an AI solution into a business’s workflow there are more factors at play than just developing ‘the AI’, and it is the responsibility of the provider to educate the companies along their journey into AI (at whatever maturity point they may be at). This also goes to say that the AI company, or team, has a responsibility to focus on all aspects of the solution; where a large portion of this is focused around the curation and cleansing of data, as well as solution-architecture. Working closely with the client to understand and help them understand the requirements of the project is an essential step. It is easy to get caught up in wanting to develop cool and exciting models and ignore the necessary laborious tasks that go along with this, meaning we often are the reason for the failure of a project.
In a roundabout way, this perfectly illustrates the importance of the other two drivers mentioned prior. Yes, Google was a pioneer by not using the traditional search ranking techniques of the time (ranking a page by how many times the searched words appeared on said page), and instead developing and leveraging what they initially called “PageRank”: A technique that determines a page’s relevance by the number of pages and the importance of those pages that linked back to the initial site. But they also placed equal, if not more, relevance on the data-curation. And finally, they had a vessel to deliver this to the world.
Why Data Curation Comes First
At the core of any artificial intelligence solution is data, and often something that is misunderstood by those that have no experience in the field. Therefore, when beginning a journey to AI capability, the first step should always be to understand and curate the data needed to solve the problem. Don’t waste resources on trying to ‘develop’ AI without first addressing this challenge. Understanding the data isn’t just knowing you need it: it is understanding the quantity needed to train an effective and efficient mode. It is understanding the pipeline that can be developed to continue this flow of data for inference and further training once implemented. It is understanding what data structure is needed for the solution. It is understanding how and where you will store and persist this data. You can’t build a house without sturdy foundations, and the same goes for building AI capability or developing a solution. If you can solve this challenge early in your journey, it will enable an efficient and effective development process down the line.
"If you have the foundations and the house ready to be built, and you don't have a block of land to build on, there is no point continuing."
Secondly, it is important to conceptualise early how you will implement the solution so the team can develop towards a goal. Knowing you want to use AI and that you have the data isn’t enough anymore. The developed solution might have a large volume of perfectly structured data and perform beyond expectation in testing; however, if you have not considered how you will apply this within the business then you will quickly see it fail. To simplify this, it is important to consider the upstream and downstream interactions: how the data will be accessed or served to the system. How the user will interact with the solution, or how it will notify of the results. The infrastructure capable of scaling to the demand, or provisioned for the life of the solution. Whether it will be cloud-based or on-premise deployment. These need to be considered in conjunction with anything you consider ‘doing AI’, as they can drastically change the nature of the solution as well as the allocated budget for a solution. To continue the analogy of the house; if you have the foundations and the house ready to be built, and you don’t have a block of land to build on, there is no point continuing.
The Role of AI Providers in Successfull Adoption
All this goes to say that for a successful integration of an AI solution into a business’s workflow there are more factors at play than just developing ‘the AI’, and it is the responsibility of the provider to educate the companies along their journey into AI (at whatever maturity point they may be at). This also goes to say that the AI company, or team, has a responsibility to focus on all aspects of the solution; where a large portion of this is focused around the curation and cleansing of data, as well as solution-architecture. Working closely with the client to understand and help them understand the requirements of the project is an essential step. It is easy to get caught up in wanting to develop cool and exciting models and ignore the necessary laborious tasks that go along with this, meaning we often are the reason for the failure of a project.
