top of page

Why companies are struggling to deploy their AI projects?

We are in the data-driven economy. More and more companies are transitioning into data-driven businesses to compete well in this global market. Artificial Intelligence (AI) is an important technology for your data-driven business transformation journey. AI is not a buzzword technology anymore. AI is already here and it is here to stay. We are already using AI in our day to day lives and in many cases we may not even be aware of its presence. AI adoption in business is increasing and it has entered nearly all industries now. AI is changing how we build our products/services, how we sell, how our customers buy and how we compete in the global market.

International Data Corporation (IDC) forecasts strong 12.3% growth for AI market in 2020, amidst challenging circumstances due to the pandemic. According to IDC, the global spending on AI is forecast to double over the next four years, growing from $50 billion in 2020 to more than $110 billion in 2024. Also, according to Gartner, the AI Augmentation will create $2.9 trillion of business value in 2021.


In spite of increased adoption by companies/industries, major portion of AI projects are not reaching production deployments. According to NewVantage Partners, among firms that report ongoing interest and an active embrace of AI technologies and solutions, 91.5% of firms report ongoing investment in AI, however, only 14.6% of firms report that they have deployed AI capabilities into widespread production. This is alarming and this requires closer attention to find out the actual reasons for the failures and to come up with the suitable Enterprise AI Strategy for your company. The following things might help during your assessment of the situation and in developing a winning AI strategy for your company.


We believe that focusing on the following key areas might help to remove major roadblocks and move your AI projects in the right direction and into production deployments.


1. Business problem that really needs an AI solution

Before starting any AI project, work with your business team to understand in-depth the business problem that you are going to solve using AI. Find out if the business problem really needs an AI solution or could it be solved using other easier solutions. Who needs to drive these efforts actually depends on the project. If this project is in the IT organization then a Business Analyst can take ownership and if this is in the product organization then a Product Manager can take ownership of these efforts. They need to work with the executive team to understand the business problem in detail and agree on the metrics that will be used to determine the quality of the AI model that will be deployed. The meetings with the executive team should not be just an initial requirements gathering and a final model demo, rather it should be iterative so that you have the opportunity to get valuable feedbacks from the business team during the development. This ensures that both the business side and the technical side are on the same page and that the business team has their trust and confidence in the AI solution that is being developed to solve the business problem.


2. Data-driven enterprise culture

We are in the data-driven economy already. More and more companies are transitioning into data-driven businesses to compete well in this global market. Accelerated transition of digital businesses into data-driven businesses to shape the products, services and experiences they offer to their customers is required in this data-driven economy to sustain and grow business. This would be a major shift for many companies and this should be woven into the company culture. Company level data-driven culture is important to ensure successful adoption of AI and to see transformative business outcomes for the company.


3. AI initiatives driven from the top executive level

AI is a disruptive technology for business and it requires careful change management for successful adoption. AI initiatives should not be just for Business Unit (BU) level, rather should be for the company level. For better success rate, AI initiatives should be driven from the top executive level and this executive team should make sure that they have a company level AI strategy for successful implementation and deployment of AI projects. Also, when companies start their AI journey, the executive teams should encourage their AI teams to focus on the human-centric, Augmented AI solutions instead of trying to develop an AI solution that totally eliminates humans in the loop and takes years to come up with a working solution or even only a POC. This would help to reduce the risk and increase the success rate for your AI projects.


4. Enterprise level data integration

Data is the new currency for the data-driven economy. Enterprises have many different types of data and often these are in different systems or silos. To obtain good ROI on AI investments, it is very important to integrate data from different systems and silos into a single data management platform at the company level so that the Business Analysts and Data Scientists can see the complete view at any customer account, at any regional level, or of the entire operation itself. This is why the Data Engineers are equally important in the AI team. This fully integrated data and the level of details that it provides enables Data Scientists to develop an effective AI model to solve the business problem and deliver significant value to customers.


5. Relevant and adequate data

More data is always better, but it has to be relevant. You need to make sure that you have adequate and relevant data to train your AI models. Many AI projects fail because either they can’t get enough relevant-data or they have bad data. Training an AI model on this data does not create an AI model that can solve a real business problem, rather it might create more problems for the business. Companies need to have a clear Data Strategy for their AI projects. Many companies these days have a CDO (Chief Data Officer) role. The CDO owns and spearheads these company level Data Strategy initiatives. They need to make sure that the right data is collected, cleansed and integrated from all relevant areas and sources (from inside and also from outside in some cases) for the AI projects.


6. Data quality and governance

With the recent push for “Ethical AI”, “Explainable AI” and “AI For Good” industry level initiatives, data quality and governance is becoming more and more important. Poor data quality and governance is one of the main reasons why many AI projects never see any light. Company data is the valuable asset for the company as long as this data can be used to generate useful insights to transform the business to provide better customer experiences and value. However, this data comes with some responsibility. Companies need to make sure that this data is secure and protected. This data is collected from right sources and it is relevant, accurate, clean and has no bias. Due to these requirements or expectations, the CDO role is becoming more and more important in these companies that are becoming data-driven. As the stewards of data quality and governance, the CDO’s should ensure that the data used for the AI projects meets these requirements or expectations by creating and owning a company level Data Strategy and also by driving its successful implementation across different Business Units and functions.


7. Right AI team and structure

AI is a team sport. Looking for a super-hero, multi-disciplinary Data Scientist who can handle end-to-end for your AI project is a waste of time and money. The reason being they are in scarce supply, very expensive, and very hard to find. Even if you find one eventually, it might be very difficult to keep them enthusiastic about your AI project and to retain them for a good amount of time. Instead, you need to build a collaborative AI team for your AI project success. Right team structure is also key for successful implementation and deployment of your AI projects. These are the roles that your might have to consider while building your AI team.


Data Engineers: You need Data Engineers to collect, cleans and catalog data for the AI projects. Many AI projects fail due to bad data or inadequate amount of data. Data is the oil for the AI engine, so need to make sure that this oil is of premium quality for the smooth operation of this engine. So Data Engineers are an integral part of the AI team.


Business/Data Analysts: Business/Data Analysts work with the business teams to gather AI project requirements. They need to understand clearly the business problem that is being addressed by the AI project and then articulate that in a way that the AI team can understand easily and work on creating a solution to solve the problem. Also, once the data has been cataloged in the company wide data management systems by the Data Engineers, you need Business/Data Analysts to analyze and visualize this data and extract high level insights.


Data Scientists: You need Data Scientists to extract deep insights and create predictive and prescriptive AI models to transform your business for delivering significant value to your customers and to create sustainable competitive advantages. There are many AutoML solutions in the market now with the promise to replace expensive and difficult to find Data Scientists by utilizing Business/Data Analysts instead. This could be true for your lower hanging fruit AI endeavors, however for your ambitious AI initiatives you still need Data Scientists, even if you are using AutoML tools, to oversee and validate the models generated by these AutoML tools.


Applied Research Scientists: Many AI projects may not require an Applied Research Scientist. If your company wants to be on the cutting-edge of AI, then only you need Applied Research Scientists to develop ambitious cutting-edge AI approaches or solutions, either to leap-frog or to maintain leadership in your target markets.


AI Engineers: AI Engineers own the entire MLOps pipeline - AI model development, deployment, monitoring and infrastructure management. AI Engineers work very closely with the Data Scientists and Data Engineers. AI Engineers are software developers, so do not expect them to also do Data Science for your AI projects.


Project Managers: It is really good to have a project manager as part of your AI team. Project Managers are important for driving successful completion of AI projects on-time and on-budget. A good project manager also foresees risks and comes up with the mitigation plans to eliminate risks or to drastically reduce its effects. It would be also good if the project manager come from the agile software development background as AI projects are very much of iterative nature.


Product Managers: In a product organization, you need a Product Manager to gather requirements by working closely with the executive team, customers, partners, and through extensive market research. Product Managers are key for defining a winning product and for its successful implementation and delivery (often with the help from the project managers on the execution side). Because AI is complex and very technical it would be better if the Product Manager for the AI project comes from the technical background, if possible. The technical background might help the Product Manager to work easily and comfortably with the highly technical AI teams.


8. AI training for employees

AI is a new technology. Many people within your organization may not know fully how AI can really help your company. It requires company level culture change for successful completion and deployment of AI projects. As mentioned before, AI initiatives should be owned and driven at the executive level and this executive team should also ensure that AI training for employees is part of developing the new company culture to foster AI based innovations and business transformations.


9. Scalable AI architecture

It is a good practice to think about the deployable and scalable architecture for your AI project from the get-go. This is because many AI projects start as a POC (proof of concept) and companies spend months or even years to successfully complete the POC and then to realize that the POC is not deployable in production because it is not a scalable solution. As the saying goes, “Think big, but start small”, it is always a good idea to think about the final scalable architecture right from the beginning even though you implement only a smaller part of that architecture for the POC purpose. This way you are clear and confident from the beginning that the company is not investing on any throw away work and also you are making sure that the concept to deployment timeline is shorter and predictable.


10. Project key performance metrics

Key Performance Metrics are very important in AI projects, because AI is complex and you might run into many issues. They guide you in the right direction and help you in making the right decisions. Sometimes your may have to completely change the direction or even abandon your AI project based on what you see for these metrics. AI project success is not completely predictable right at the concept phase, so these tools help guide your AI-driven business transformation journey. It is also a good idea to create a project specific performance metrics dashboard to keep everyone on your project on the same page. This also helps the executive team to get a quick sense of how your AI project is progressing.

11. Iterative and agile development process

Never start your AI projects with a big and ambitious objectives right from get-go. These endeavors mostly result in disappointments and project failures. Instead adopt an iterative strategy. AI projects are iterative by nature due to its complexity, so it is very important to have this as part of your development process. Lean and Agile software development process is suitable here because you want to start the AI project small first (think big, but start small) and then make incremental updates to add additional functionality. This would help you to reduce the complexity and to make continuous progress in your AI projects.


12. MLOps

AI software development is different from traditional software development. In traditional software development, once your test and release the software, you will re-release the software only for bug-fixes, feature enhancements or to add new functionality to your software. Unlike traditional software, AI software might require an update whenever there is drift in the data that has been used to develop the AI model or if you have acquired new data that need to be used for training your AI model. This might happen any time and whenever it happens, the AI model needs to be re-trained and re-deployed. The traditional DevOps is not suitable for these kinds of AI model updates which has to happen very quickly (shorter cycle time). MLOps (Machine Learning Operations) is designed for quick AI iterations and for faster product deployments and it makes your AI deployments much easier. Many companies fail to deploy their AI POCs because they did not take into account MLOps in their AI journey from the beginning. So it is essential to consider MLOps right from the beginning to increase your AI project success rate and to reduce your AI deployment cycle time.


Summary

AI is a disruptive technology and it is permeating nearly every industry. Every software will have some form of AI technology infused into it in the next several years. AI becomes an integral part of your strategy to maintain your competitive advantages. Embrace AI to create new efficiencies and to make disruptive AI-driven transformations (ethical & explainable at the same time) of your business to remain relevant in the new data-driven economy.


Where are you in your data-driven business transformation journey? Questions? Need help? Contact us.


273 views0 comments

Comments


bottom of page