With the growing adoption of AI across various industry sectors the demand for data scientists is increasing.
Based on data from job site Indeed, job postings related to AI have seen a steady rise in the last two years, with a 46% increase between June 2018 and June 2019, and a 51% increase between June 2019 and June 2020. Due to this demand, there are many AI training organizations (within and outside colleges and also both onsite and online programs) that have sprung up across the country to provide training in data science to fill the talent gap. Even though these organizations help create more data scientists, many hiring managers still wonder why it takes so long to find a good data scientist.
To understand this we will look at the traditional business decision making process and then we will look at the new AI-based business decision making process and how data scientist’s work influences business decision making. This should give you some idea about why it is hard to find good data scientists and also why data scientists are an integral part of your AI team that enables AI-driven business transformations to deliver innovative and unique values to your customers.
Traditional data analytics based business decision making process
In this model the data engineers collect data from various, relevant data sources, clean them, transform them and store them in a centralized data management system. When the data is ready, the business/data analysts start exploring data, analyze them, generate actionable insights and discuss these insights with the business leaders and assist them to make the right business decisions. In this process, the decision making is still 100% done by the business leaders and the insights from the analytics team help these business leaders to make informed business decisions. This is completely a human-centric model where business leaders are in the loop 100%. So in this model, the business leaders feel totally comfortable using the insights from the analytics team in their business decision making process. In fact, these insights make them even more efficient.
New AI-based business decision making process
Now let us look at the new AI-based business decision making process. In this model, the data engineers role remains pretty much the same. They collect, clean, transform and store relevant data in a centralized data management platform. The data analysts/data scientists generate insights from this data and based on these insights the data scientists then develop AI models that automate some part of the business decision making process (% depends on the project and the company). These AI models would make some decisions for the business team. For example, the model might decide to send an email or a text message to a customer, might decide on whether to send a 10% or 25% discount coupon to a customer, instead of the business team making these decisions based on the insights as in the traditional decision making process.
When the data scientists create these AI models it was based on the requirements provided by the business analysts or the product managers to solve the business problems that business leaders want to solve using AI. The business analysts or product managers work closely with the business leaders to get their clear requirements and then discuss these requirements with the data scientists. Most companies have a well established process here to make sure that everyone is on the same page without any confusion.
Then where is the problem?
Data science is a multi-disciplinary role requiring expertise in math, statistics, data modeling, programming, and ML operations. It is hard to find talent that meet all the technical requirements. On top of this, the data scientists also need to have domain expertise and certain soft skills to work well with the business leaders. This combination is not easy to find and they are scarce in supply.
The main problem here is the trust and confidence that the business leaders will have on the data scientists and on the AI-models that are used in assisting business leaders in the decision making process. The expectations are very high for the data scientists and that is why it is becoming harder and harder to find good ones.
These AI models will have major impact on the business outcomes if designed properly. Finding a data scientist that your business team likes and trusts is hard. The AI hiring manager’s job is to find good data scientists who would never deploy a bad quality AI model that would make business leaders lose their sleep. Also, the AI model performance might deteriorate over time due to drift in data or due to availability of new data. In these scenarios, the data scientists need to retrain and re-deploy the models within a short cycle-time, sometimes within days and in some extreme cases even within a day, because there might be risk of these stale models impacting your business negatively if they are not taken care of soon enough. So the data scientist role has high responsibility and high visibility within the company and they are willing to pay big bucks for good data scientists (a scarce resource even in the silicon valley, US) who can work well with the business leaders and can help create positive business impact for the company using innovative and quality AI models.
These are also the reasons why it is difficult for data scientists with no experience or little experience to land a job in a good company. So, their best bet might be to enter as a data analyst or a data engineer or an ML engineer first and then move into a data scientist role after couple of years.
The new AI-based decision making process creates few challenges in companies that are trying to deploy AI and they need to be managed carefully to have successful AI deployments.
Trust and confidence: It is important that the data science team has full confidence and trust of their business team. Without this the AI will never thrive in the company. AI is a team sport, so build a team, with the right team structure, that your business team can rely on. Gain their trust and confidence by focusing on the human-centric AI designs that make business leaders feel more comfortable and also deliver real business values.
Internal power shift: AI might bring some power shift within the company. Business team now has to depend on the data science team for automated decision making. Also, data science team might feel more important within the company as their actions are directly impacting the company bottom-line in a positive way. This might create some new internal politics and it requires proper management to make AI a great success within your company.
Organization structure: The right organization structure is key for the AI success in your company. In some cases you might prefer to have a centralized data science team and in other cases you might prefer to absorb the data science team into the business team to avoid any conflicts or concerns. For example, data science as part of the sales and marketing analytics team and this team reporting to the sales and marketing executive. Likewise, the data science team as part of the HR analytics team and this team reporting to the HR executive. However, this may not be practical for every company, so you need to evaluate this based on your company situation.
Fear of automation: AI might make business teams bit uncomfortable due to its automation. Business teams might feel less in control of the decision making process and might also feel that they might be left out of the loop and out of sync in the business decision making process. This might create some fear or discomfort and as a result there might be some resistance and mis-alignment. This can be reduced if you focus more on the human-centric, augmented AI solutions that keep business leaders in the loop in the decision making process as this makes them feel more comfortable.
Resistance to change: AI is a new technology and many people may not know well about AI technology and how it might help their business. As a result there might be some skepticism and hence resistance to AI adoption within the company. AI should be woven into the company culture to make it more effective. Company level awareness and trainings about AI would definitely help here.
AI is a new and disruptive technology. Companies are trying to figure out the right team structure for AI success. AI also requires company level change management initiatives and a new company culture that allows AI to thrive in your company.
Build a multi-disciplinary AI team for success
Finding a super-hero data scientist is really hard. Also, they are very expensive too. Instead of trying to search for a data scientist who has all the required skills mentioned above, which is hard to find and might really take a long time, it is better to form a multi-disciplinary AI team that include data engineers, business analysts, data analysts, data scientists, machine learning engineers, and project managers, where data scientists can get help in areas such as data preparation, ML operations and interaction with the business teams. This also minimizes risk because data scientists may not stay for more than 2-3 years (avg.) in one company as they are in high demand.
Focus on human-centric, augmented AI designs
For better ROI and successful deployments, AI should be designed to help humans and make them more efficient in their work rather than focusing on ambitious automations to remove human in the loop. These ambitious AI undertakings often take years to complete and sometimes without any good ROI at the end. Also, focus on creating incremental value by starting out with smaller AI projects rather than starting out on a big project with the ambitious goals right from the beginning. These big projects often end up with disappointing results. As the saying goes, "Think big, but start small".
Make AI part of your company culture
For successful AI deployments, make AI part of your company culture and create AI awareness and trainings for the entire company. Have your executive management team own and drive company level AI initiatives for successful implementation and deployment of AI projects. Create a company culture that fosters AI innovations and its successful implementations.
Summary
AI is going to be everywhere whether we like it or not in the next several years. It is permeating into every industry sector now. To deliver disruptive value to your customers you need to build a solid data science team that is technically good and also can work very well with your business team. Building a great data science team takes time, so start building your data science team early and transform your company into AI-based data-driven company to deliver exceptional value to your customers and to remain competitive and relevant in this data-driven economy.
Have more questions? Contact us.
#datascience #dataanalytics #businessanalytics #dataengineering #ai #artificialintelligence #machinelearning #deeplearning #datascientists #dataanalysts #businessanalysts #dataengineers #mlengineers
コメント