Strategic thinking in Artificial Intelligence
Matt Armstrong-Barnes, Chief Technology Officer - Artificial Intelligence, HPE
Technology innovation is evolving at such a rapid pace that old school thinking has just run out of steam. Back in 2017, more data was created in a single year than the sum of all information recorded by humanity in the past 5,000. Since then, we have experienced an average growth rate of 29%. This age of hyper-connectivity has created a data explosion that doesn’t show any signs of slowing down. Our ability to intelligently process this constant tsunami needs a new set of tools but implementing them is fraught with peril for the unsuspecting traveller.
This exponential growth has pushed the people to process it and systems to derive value from it to breaking point. In short, Artificial Intelligence is the only answer. However, building AI systems suffers from the traditional software engineering challenges, such culture change, skills, use case selection and scope creep as well as the added complexity of handling lots of data and having the right infrastructure to run it on. Naive extrapolation which is the human habit of fastening onto some current trend without understanding the long-term impact, lack of understanding or even fear can result in AI projects failing to get moving.
Generative AI: promise vs reality
AI and more specifically Generative AI promise to radically transform many industries, but they also pose significant risks; many of which have yet to be full considered. Challenges with legal & regulatory compliance are still in their infancy with the world watching the evolution of the EU AI Act as it moves closer to becoming enforceable legislation. Generative AI works on a broad variety data and without fully understanding the implications it is possible to hand over sensitive / confidential data or even intellectual property. Before becoming exposed to these risks, it is imperative to understand how closed-source generative AI models handle data privacy and confidentiality. Avoiding these issues in a way that doesn’t result in significant model hallucination (yes, they make things up if they don’t know the answer) requires a strategy. Generative AI solutions needs designing, planning, and architecting in a way that addresses business problems.
AI is a journey not a destination
Every outing starts with a single step and AI is a journey not a destination. The first thing to think about is where are you going - your strategic vision. Just because you have a hammer with the words ‘Artificial Intelligence’ on it, do not see everything as a nail. AI is a powerful tool and should be in every organisation’s toolkit for strategic use. Once you have your destination defined start work on the use cases you want AI to address. In the data driven economy, start by understanding your data and what insight you want to gain, have a data strategy. This is an encapsulation of your long-term objectives across people, processes and technology in a way that shows how these valuable assets are managed and governed. Without strategic vision, vast amounts of low value data will be generated and stored. This is not good for the environment or organisational profitability because storing worthless data consumes space and power while generating heat all of which cost money to buy and maintain.
Set a clear strategy to understand your existing data
Having a clear strategy for your data means collecting and finding value in the significantly simplified. This will influence the decision-making process when it comes to selecting the right models and AI frameworks. Quality is still the determining factor. Just throwing data at AI means you will quickly encounter the ‘curse of dimensionality’; throwing more hay onto the haystack does not make finding the needle any easier.
Understanding and refining your data means you will be training your model on the right feature set while avoiding bias. Historic data is a great source for training models, however, can often be incomplete and have underrepresented classes. There are various kinds of discrimination, that come from not representing classes correctly like race, gender or age and these need to be guarded against in the input data set. Many statistical techniques can be employed to guard against this undue model influence.
Use your own organisational principles, compliance requirements, goals, and values as a starting point.
AI requires cross collaboration
The road to successful AI deployment may be fraught with dangers, but it is a well-worn path. There is a great phrase ‘if you want to go fast go alone, if you want to go farther go together’, the most successful implementations adopt a team AI approach. Team AI should be a virtual team, drawn from a variety of functions and including business users, external partners, and other stakeholders, working together to solve a set of well-defined goals. As a team sport, it is important to consider partners who already have both a track record of successful AI project delivery and extensive ecosystem of partners. AI is a multi-disciplinary cross collaboration that requires a broad spectrum of communities to be successful. Also, many AI problems have already been solved, so you don’t always need to build from scratch.
As a partner of Ultima, HPE has considerable depth of experience in both AI deployments and working with both open and closed source AI solutions. If you have already started on your AI journey or are thinking of starting; we have a broad and deep understanding of how to accelerate you on the road.