Using artificial intelligence to find skin cancer earlier: the technology is not yet ready for use in primary care

Artificial Intelligence (AI) aimed at detecting skin cancer is not yet ready for use in primary care settings, due to a lack of evidence in settings where the prevalence of skin cancer is low, according to CanTest researchers.

The researchers emphasise that AI technology demonstrates promising results in detecting skin cancer in images from specialist clinical settings, where diagnostic accuracy is high.

Skin cancers are the most common cancer, with over 150,000 new cases every year in the UK. Most people diagnosed with skin cancer first present to primary care, where General Practitioners (GPs) face the difficult task of differentiating common benign skin lesions from rarer skin cancers.

Primary care is where AI could make a big difference. More accurate assessment of skin lesions could lead to earlier diagnosis of skin cancer, potentially improving outcomes for patients and boosting survival rates. The burden on specialist dermatology services could also be reduced.

The research team, led by CanTest researcher Dr Owain Jones from the University of Cambridge carried out the first systematic review of studies of machine learning-based AI techniques to see if they could be used to enable earlier diagnosis of skin cancer in primary care. The study is published in Lancet Digital Health.

They found 272 relevant studies, but only 2 studies used data from populations with low prevalence of skin cancer. Many studies didn’t provide detailed descriptions of the data that was used to develop the algorithms – lacking information on how images were obtained and processed, for example.

Few studies considered the process of implementation in clinical settings in developing and validating their algorithms. There was little evidence of validation in prospective or ‘real world’ clinical settings, and no research from a health economic or patient and clinician acceptability perspective.

The team also identified a handful of commercially available technologies that are supported by peer-reviewed, published evidence; although the majority of commercially available technologies identified lacked independent evaluation or academic detailing.

The researchers emphasised that only a few of the algorithms described in the published research were developed using data from populations with low prevalence of skin cancer, and images of skin lesions in black and brown skin were rarely used.

Because the algorithms may not perform well in populations outside those they were trained on, for example in primary care populations and in people with black and brown skin, widespread adoption into routine primary care clinical practice cannot yet be recommended.

“AI techniques have the potential to aid clinical decision making and support the early diagnosis of skin cancer in primary care settings. But these algorithms need to be evaluated carefully in appropriate populations before widespread adoption into routine primary care clinical practice can be recommended,” said Dr Owain Jones, clinical research fellow, Primary Care Unit, University of Cambridge and lead author.

To help developers overcome some of the challenges identified through this review, the research team have drawn up a checklist for the design, development, and evaluation of AI algorithms which aim to triage or detect skin cancers in primary care.

Owain explains: ‘We hope this checklist will address the issues with study design identified in this review, enable meaningful comparison between studies, increase the clinical relevance of AI/ML algorithms, and improve the likelihood of these promising technologies being successfully translated into clinical tools that can be of benefit to patients and primary care clinicians.’

The checklist is freely available in the research article.

The research team was funded by the NIHR’s Cancer Policy Research Unit and Cancer Research UK.

Jones OT, Matin RN, van der Schaar M, Prathivadi Bhayankaram K, Ranmuthu CKI, Islam MS, Behiyat S, Boscott R, Calanzani N, Emery J, Williams HC, Walter FM. “Are artificial intelligence/machine learning (AI/ML) algorithms ready for implementation in community and primary care settings to facilitate the early detection of skin cancer? A systematic review.” Lancet Digital Health. 24 May 2022.