Stuart ReidCTO @ STA Consulting

Testing AI: A New Testing Specialism

Artificial intelligence (AI) is now a mainstream technology that touches the lives of most of us, with a global market predicted to reach US$500 billion within two years.

AI-based systems, and machine-learning (ML) systems in particular, are different from traditional IT systems, as the underlying code is not written by a programmer but is inferred from patterns in data.  This major change to classic software development has two significant knock-on effects.  It has created the attractive new role of AI specialist.  However, it has also created several new sources of specialised defects, such as the training data, the deliverable ML model and the ML development framework.

This talk proposes a new software testing specialism that is focused on addressing the new challenges introduced by AI and ML.  To be able to test a ML system, testers need to understand the underlying technology and how it can go wrong.  They also need to be able to address those system attributes, such as ethics and trust, which are so important with AI-based systems.

From now on, as AI-based systems grow in complexity and criticality, achieving acceptable levels of quality and trust will only be possible by employing independent, AI test specialists.  This talk will explain why they are needed, what specialist tasks they will do, and suggest how we should handle the transition to a future where the AI test specialist is a recognized role.

Stuart Reid is a software testing consultant with over 38 years’ experience in the IT industry, working in development, testing and education. Application areas range from safety-critical to financial and media.

Stuart supports the worldwide testing community in a number of roles.  He is convener of the ISO Software Testing Working Group, which has already published several software testing standards and is currently developing new standards in the areas of AI, performance and automotive testing.  He founded the International Software Testing Qualifications Board (ISTQB) to promote software testing qualifications globally.