<img height="1" width="1" style="display:none;" alt="" src="https://px.ads.linkedin.com/collect/?pid=4958233&amp;fmt=gif">
RSS Feed

QA | Gregory Solovey |
06 August 2019

I would like to reflect on three major test trends that were presented in the 2019 edition of the World Quality Report (WQR). According to responses from 1,700 executives from across 10 different sectors and 32 countries, automation of test generation and test execution continued to be ranked highest, and artificial intelligence and machine learning are transformers for software testing. But devil is in details.


One of the main findings relates to how low levels of basic test automation have become a critical bottleneck in the growth and adoption of Agile, and DevOps, and the spread of digital transformation for a significant number of the businesses surveyed. The report also points to regulatory and infrastructural challenges like handling test data, provisioning environments, and finding the right talent required for testing.

According to the survey, 99% of respondents said they were using Agile and DevOps in at least some part of their business. Despite this growth in adoption, organisations are still not able to tap the full benefits promised by these approaches, mainly due to low levels of automation and challenges with test data and test environments. The survey also clearly reveals that levels of basic test automation are still quite low (between 14%–18%).

Endava provides a strong response to these challenges. To unlock Agile and DevOps, we utilise a set of test accelerators that enable the rapid creation of test automation environments to help jump start the automated test processes. And rapid means just a few days for automating end-to-end tests and including them in the CI/ CD environment. These testing accelerators include: API testing accelerator, Java testing accelerator, .NET testing accelerator, multi-browser testing accelerator, mobile testing accelerator, security testing accelerator and a performance testing accelerator. This is a unique, Endava-specific approach, something that does not exist in the world test market today.


AI and ML are the force multipliers for QA and testing. There will be a transformation of the QA and testing function in the future to support emerging trends such as the Internet of Things (IoT), blockchain and the convergence of analytics, artificial intelligence (AI) and machine learning (ML). When it comes to AI, one must keep two things in mind: AI in testing can refer to the application of AI to quality assurance and testing, as well as to the testing of AI products.

2.1. Testing AI applications.

When asked about the challenges faced in implementing AI projects, 55% of respondents reported that they had “difficulties with identifying where business might actually apply AI.” Many of them first build up their knowledge and expertise in the AI/ ML technology and only then plan to build a business case for it.

From my point of view, there is no specificity in modelling AI compared to non-AI applications. The AI applications can be represented as a set of UML models, and, if so, the same test design methods can be successfully applied.

2.2. Using AI/ ML approaches in testing.

There is enthusiasm for and excitement about AI technologies and solutions, but their actual application in testing is still emerging. The purpose of applying AI to QA and testing is to create a testing architecture that adapts itself automatically to application changes: which tests to run, how many tests to run, assist in the test case creation.

The challenges to implement the above approaches arise from the quality and quantity of the data used: test case coverage, defect data, production data, code coverage, operational logs. Most organizations are still stuck at the level of data analytics rather than using AI technologies such as machine learning, neural networks, fuzzy logic, robotics, or deep learning.

Modern publications propose more approaches to applying AI/ ML to testing. For example:
Use image recognition to bring UI testing to the next level. Tools compare the UI elements’ appearance with recognised page images (regardless of their location and presentation).
Identify patterns and prevent future defects, via data mining historical log files.
Learn an application and automatically generate tests. The tool “explores” an application and, based on the discovered functionalities, automatically generates test cases.
Maintain the test code robustness based on historical data. Each UI element uses a few identifiers and even though some of them can be changed, the object will still be recognised by the test tools.

There are quite a few companies that provide AI/ ML applications for test / QA purposes: IBM IGNITE, Appvance IQ, Applitools, Sauce Labs, Testim, Test.AI, Mabl, ReTest, ReportPortal.io. However, the consensus around their efficiency is not very encouraging - they’re not quite there yet. For example, some use machine learning-like (AI-like) technology, or their tools are still in beta version, or would only be commercially available in a few years. Only time will tell if AI/ ML will eventually fulfil the promise of reliable, easy-to-maintain test automation for all.


Model-based testing is a new automation technique that 61% of respondents foresee using in the coming years. Today’s ever-changing applications, the increased complexity and number of new releases has led to a rise in the importance of model-based testing for automated test cases and automated script generation based on requirements.

Model-based testing is not a new concept — it has existed since the first software bugs. However, the reality of test design automation based on models is not as bright as World Quality Report respondents’ aspirations. The applications’ high complexity levels and the low quality of the existing systems’ structural and behavioural models today do not permit automatically generating tests for production scale systems, and I think this is unlikely happen in the near future.

The recent white paper “Model based test design and automation” presents an Endava approach that can be applied to all our engagements. It proposes to formalise the system description as a hierarchy of UML models, use known test design methods and a multi-level test hierarchy to separate the presentation, logic and data related tests. Thus, even if today's model-based testing does not provide the test design automation, it guarantees the completeness and elegance of a manually built test.

Gregory Solovey

Test Architect

Gregory spent a good part of the last 30 years adapting test design methods, that were implemented 40 years ago, and automating them. Gregory holds an unusual PhD in test design methods, speaks at conferences and writes on test related topics. When he is not architecting test automation and continuous integration solutions, he’s busy traveling, playing tennis and volleyball, and distorting reality in Photoshop.


From This Author



  • 13 November 2023

    Delving Deeper Into Generative AI: Unlocking Benefits and Opportunities

  • 07 November 2023

    Retrieval Augmented Generation: Combining LLMs, Task-chaining and Vector Databases

  • 19 September 2023

    The Rise of Vector Databases

  • 27 July 2023

    Large Language Models Automating the Enterprise – Part 2

  • 20 July 2023

    Large Language Models Automating the Enterprise – Part 1

  • 11 July 2023

    Boost Your Game’s Success with Tools – Part 2

  • 04 July 2023

    Boost Your Game’s Success with Tools – Part 1

  • 01 June 2023

    Challenges for Adopting AI Systems in Software Development

  • 07 March 2023

    Will AI Transform Even The Most Creative Professions?

  • 14 February 2023

    Generative AI: Technology of Tomorrow, Today

  • 25 January 2023

    The Joy and Challenge of being a Video Game Tester

  • 14 November 2022

    Can Software Really Be Green

  • 26 July 2022

    Is Data Mesh Going to Replace Centralised Repositories?

  • 09 June 2022

    A Spatial Analysis of the Covid-19 Infection and Its Determinants

  • 17 May 2022

    An R&D Project on AI in 3D Asset Creation for Games

  • 07 February 2022

    Using Two Cloud Vendors Side by Side – a Survey of Cost and Effort

  • 25 January 2022

    Scalable Microservices Architecture with .NET Made Easy – a Tutorial

  • 04 January 2022

    Create Production-Ready, Automated Deliverables Using a Build Pipeline for Games – Part 2

  • 23 November 2021

    How User Experience Design is Increasing ROI

  • 16 November 2021

    Create Production-Ready, Automated Deliverables Using a Build Pipeline for Games – Part 1

  • 19 October 2021

    A Basic Setup for Mass-Testing a Multiplayer Online Board Game

  • 24 August 2021

    EHR to HL7 FHIR Integration: The Software Developer’s Guide – Part 3

  • 20 July 2021

    EHR to HL7 FHIR Integration: The Software Developer’s Guide – Part 2

  • 29 June 2021

    EHR to HL7 FHIR Integration: The Software Developer’s Guide – Part 1

  • 08 June 2021

    Elasticsearch and Apache Lucene: Fundamentals Behind the Relevance Score

  • 27 May 2021

    Endava at NASA’s 2020 Space Apps Challenge

  • 27 January 2021

    Following the Patterns – The Rise of Neo4j and Graph Databases

  • 12 January 2021

    Data is Everything

  • 05 January 2021

    Distributed Agile – Closing the Gap Between the Product Owner and the Team – Part 3

  • 02 December 2020

    8 Tips for Sharing Technical Knowledge – Part 2

  • 12 November 2020

    8 Tips for Sharing Technical Knowledge – Part 1

  • 30 October 2020

    API Management

  • 22 September 2020

    Distributed Agile – Closing the Gap Between the Product Owner and the Team – Part 2

  • 25 August 2020

    Cloud Maturity Level: IaaS vs PaaS and SaaS – Part 2

  • 18 August 2020

    Cloud Maturity Level: IaaS vs PaaS and SaaS – Part 1

  • 08 July 2020

    A Virtual Hackathon Together with Microsoft

  • 30 June 2020

    Distributed safe PI planning

  • 09 June 2020

    The Twisted Concept of Securing Kubernetes Clusters – Part 2

  • 15 May 2020

    Performance and security testing shifting left

  • 30 April 2020

    AR & ML deployment in the wild – a story about friendly animals

  • 16 April 2020

    Cucumber: Automation Framework or Collaboration Tool?

  • 25 February 2020

    Challenges in creating relevant test data without using personally identifiable information

  • 04 January 2020

    Service Meshes – from Kubernetes service management to universal compute fabric

  • 10 December 2019

    AWS Serverless with Terraform – Best Practices

  • 05 November 2019

    The Twisted Concept of Securing Kubernetes Clusters

  • 01 October 2019

    Cognitive Computing Using Cloud-Based Resources II

  • 17 September 2019

    Cognitive Computing Using Cloud-Based Resources

  • 03 September 2019

    Creating A Visual Culture

  • 20 August 2019

    Extracting Data from Images in Presentations

  • 06 August 2019

    Evaluating the current testing trends

  • 23 July 2019

    11 Things I wish I knew before working with Terraform – part 2

  • 12 July 2019

    The Rising Cost of Poor Software Security

  • 09 July 2019

    Developing your Product Owner mindset

  • 25 June 2019

    11 Things I wish I knew before working with Terraform – part 1

  • 30 May 2019

    Microservices and Serverless Computing

  • 14 May 2019

    Edge Services

  • 30 April 2019

    Kubernetes Design Principles Part 1

  • 09 April 2019

    Keeping Up With The Norm In An Era Of Software Defined Everything

  • 25 February 2019

    Infrastructure as Code with Terraform

  • 11 February 2019

    Distributed Agile – Closing the Gap Between the Product Owner and the Team

  • 28 January 2019

    Internet Scale Architecture