The Data Lab

From Data to Diagnosis: How AI Is Revolutionising Healthcare

Community & Events, Healthcare, Thought Leadership

photo of doctor

Healthcare systems are facing unprecedented pressures, with demand outpacing resources and services struggling to keep up. In Scotland alone, Public Health Scotland estimates that the NHS could be managing around 1,300 additional unplanned hospital admissions every week, largely driven by an ageing population. Cancer treatment waiting times are now the worst on record, with a third of patients with suspected cancer waiting longer than the 62-day target to start treatment. Across the wider UK, workforce shortages are a major challenge, with 32,000 nursing vacancies reported last year in England and 16% fewer qualified GPs compared to similar-sized countries. 

At the same time, advances in data and artificial intelligence (AI) are creating powerful new opportunities to tackle these challenges. By analysing vast amounts of health data — from medical images and lab results to patient histories and genetics — AI is helping clinicians diagnose diseases earlier, tailor treatments to individuals, and manage services more efficiently. 

This blog explores how AI and data are driving progress in three key areas: personalised medicine, efficiency and productivity, and preventive healthcare — and the questions we must address to ensure these innovations are used safely and equitably. 

Personalised medicine

Personalised medicine represents a pivotal shift in healthcare — moving away from the traditional “one-size-fits-all” approach towards treatments that are tailored to each individual’s unique characteristics, such as their genetic profile, biomarkers, lifestyle, and environmental factors. 

AI plays a critical role in this transformation. By bringing together and analysing vast amounts of health data — from electronic health records to genomic and imaging data — AI helps clinicians identify patterns and make precise, data-driven decisions about diagnosis, treatment, and ongoing care. This integration is enabling a new era of truly patient-specific healthcare. 

Rare diseases affect over 300 million people worldwide, including 36 million in the EU. These conditions are often overlooked due to the small number of patients and the high costs associated with developing targeted treatments. One area where AI is making a real difference is in the diagnosis of inherited retinal diseases (IRDs), the leading cause of blindness in children and working-age adults. Globally, IRDs affect more than 10 million people, yet fewer than half of these conditions can currently be genetically diagnosed. In the UK, there are only around 50 consultants specialising in these diseases, creating a significant bottleneck for patients seeking diagnosis and treatment options. To address this challenge, Eye2Gene, a pioneering project at UCL and Moorfields Eye Hospital, is using AI to analyse retinal scans and predict which rare genetic condition a patient may have, down to the specific gene involved. Once fully validated, this tool could dramatically speed up diagnosis, support clinicians in planning care, and help researchers track disease progression. 

While rare diseases highlight the opportunities of AI in precision diagnostics, similar approaches are transforming care for common conditions such as heart disease. Heart and circulatory diseases are amongst the world’s biggest killers, causing nearly 1 in 3 deaths globally. In the UK, coronary heart disease remains one of the leading causes of death for men and women. Heartflow, a digital health company, is tackling this by turning heart disease into a condition that can be actively managed throughout a patient’s life. Their technology creates a personalised 3D model of a patient’s arteries using images from a standard, non-invasive CT scan. AI then analyses these images to assess blood flow and plaque build-up, giving doctors detailed insights into the patient’s unique condition. This allows them to diagnose problems earlier and make more informed decisions about treatment — whether that’s lifestyle changes, medication, or surgery. Since 2021, Heartflow has been rolled out across 56 NHS hospitals in England, helping over 24,300 patients. The technology has saved an estimated £9.5M and has reduced the need for invasive diagnostic procedures by 16%, improving both patient outcomes and healthcare efficiency. 

Efficiency and productivity in healthcare

Improving efficiency and productivity is essential for strengthening healthcare systems like the NHS. AI is helping to make this possible by automating time-consuming tasks, streamlining workflows, and providing faster, data-driven insights. This not only frees up clinicians to spend more time with patients but also helps hospitals reduce backlogs and make better use of limited resources. 

One example of this is the GRACE project, which introduced AI software to assess chest scan images across NHS Grampian. The software checks for over 120 potential issues, flagging any that may show signs of lung cancer for urgent review. Since being deployed across 13 NHS Grampian sites, this tool has led to 12% more treatable cancers being detected and helped 95% of patients access treatment 30 days sooner. With the number of urgent cancer referrals in Scotland increasing by 50% over the last five years, NHS Grampian is now collaborating with the Accelerated National Innovation Adoption Pathway to scale the system across Scotland. This work demonstrates how AI can both improve patient outcomes and ease the burden on overstretched services. 

In the UK, skin cancer generates more urgent referrals than any other cancer, with referrals growing by over 10% year on year. At the same time, one in four consultant dermatologist posts are unfilled, creating mounting waitlists and longer wait times for patients. Skin Analytics is aiming to tackle this challenge with its AI-driven technology, DERM, designed to triage suspicious skin lesions more efficiently. Using a smartphone with a dermoscopic lens attachment, healthcare staff can capture high-quality images of a patient’s skin lesion. These images are then uploaded for remote assessment and diagnosis, eliminating the need for an immediate in-person visit. DERM’s algorithm analyses each image, comparing it to a vast database of known skin conditions to identify visual patterns that may indicate cancer. If a suspicious lesion is detected, the patient is fast-tracked to a dermatologist for further investigation. If the lesion appears non-cancerous, the patient can be offered guidance without needing to attend hospital. Early evidence suggests that using DERM could halve the number of urgent skin cancer referrals compared to teledermatology alone, while still accurately distinguishing between cancerous and non-cancerous lesions. The NHS is now rolling out the technology for a three-year period to further evaluate its impact on patient outcomes and service capacity. 

Preventative healthcare

Preventive healthcare is about identifying health risks early and taking action before serious illness develops. By catching diseases in their earliest stages, healthcare systems can improve outcomes for patients while also reducing the long-term costs and pressures of treating advanced disease. 

AI and data-driven technologies are revolutionising this space. They enable faster, more accurate screening and monitoring, helping clinicians to spot subtle signs of disease that might otherwise go unnoticed. By analysing vast amounts of data — from medical images to patient histories — AI can prioritise high-risk cases, ensure timely follow-up, and even predict future health issues before symptoms appear. 

Bowel cancer is a prime example of where prevention can save lives. It is the second most common cause of cancer death in Scotland, with around 1,600 people dying of the disease each year. Traditionally, patients are screened using colonoscopy – an effective but invasive and uncomfortable procedure that also requires significant hospital resources. The AICE colon capsule project – a collaboration between the University of Strathclyde, NHS Highland and Islands, The Data Lab, and the Digital Health and Care Innovation Centre – is exploring a less invasive approach that will also make cancer screening faster. Instead of a traditional colonoscopy, patients swallow a ‘smart pill’ equipped with tiny cameras, which record images of the intestines as they pass through. These images are then analysed by AI to assist clinicians, flag any abnormalities, and identify areas that may require further investigation. This approach has the potential to detect cancers earlier and at less advanced stages, enabling faster treatment and improving survival rates. 

AI is also being applied to breast cancer screening, the most common cancer in the UK. Every year, around 55,000 women and 400 men are diagnosed, with one woman diagnosed every 10 minutes. To address this, the NHS has launched the EDITH trial (Early Detection using Information Technology in Health) — one of the largest AI breast screening trials in the world. Nearly 700,000 women will take part in the study across 30 sites in the UK, where five different AI technologies are being tested to assist radiologists in reviewing mammograms at routine appointments. Currently, every mammogram must be assessed by two specialists to ensure accuracy, but AI has the potential to safely reduce this to one specialist, speeding up the process and freeing valuable resources. By helping radiologists identify subtle changes in breast tissue that may indicate early cancer, this trial could improve early detection and ensure patients receive timely, potentially life-saving care. While results will take time to emerge, the trial represents a major step forward in modernising cancer screening and prevention. 

Challenges and questions we need to solve

As powerful as AI and data-driven technologies are, their use in healthcare raises several complex challenges that must be addressed to ensure safe and fair adoption: 

  • Privacy and security – Healthcare data is some of the most sensitive information there is. High-profile data breaches, such as the 2022 cyberattack that disrupted the NHS 111 service, have shown how vulnerable systems can be. As more health data is collected and shared, strong safeguards are essential to protect patient privacy and maintain public trust. 
  • Bias and fairness – AI systems are only as good as the data they are trained on. If certain groups are underrepresented — for example, skin tone diversity in dermatology datasets — the results can be inaccurate or even harmful. Research has shown that some skin cancer detection algorithms perform significantly worse on darker skin, risking missed diagnoses and widening health inequalities. 
  • Integration into real-world systems – Introducing new technology is rarely straightforward, and with healthcare systems already under immense pressure, poorly designed systems can add to workloads rather than reduce them. The success of projects like GRACE in NHS Grampian comes down not just to the technology itself, but to training, support, and integration with existing processes. 
  • Trust and transparency – Clinicians and patients need to understand how and why AI makes decisions. Clear, explainable systems are essential for building confidence and ensuring safe use. In the UK, regulators are starting to address this challenge. The Medicines and Healthcare products Regulatory Agency (MHRA) is developing and refining its regulatory framework for software and AI medical devices. New guidance and the ongoing “Software and AI as a Medical Device Change Programme” roadmap aim to improve clarity around risk management, performance, and features such as explainability and auditability. For example, in the EDITH breast screening trial, every AI recommendation is reviewed by a human specialist before action is taken, ensuring safety while trust in the technology builds. 
  • Equitable access – Without careful planning, AI could risk reinforcing existing inequalities. Some early UK AI healthcare initiatives are being piloted primarily in larger or more resourced trusts, which raises questions of access for smaller hospitals, rural areas, or under-served regions. Efforts must be made to ensure advances in AI and data benefit everyone, regardless of location or background. 

Addressing these challenges requires collaboration between technologists, clinicians, policymakers, and patients — as well as robust regulation and ethical oversight.

The road ahead

The future of healthcare will increasingly be shaped by data and AI. From diagnosing rare diseases to streamlining services and detecting cancers earlier, these technologies are already transforming care and saving lives. 

However, their potential will only be fully realised if we tackle the challenges around privacy, bias, and equitable access. By putting patients at the heart of innovation and working collaboratively across sectors, we can create a healthcare system that is more personalised, efficient, and preventive — ultimately improving outcomes for everyone.

Health month at The Data Lab Community

In October, we’re shining a spotlight on how data and AI are transforming healthcare. From tackling real-world challenges in predictive AI to exploring the future of healthcare systems and the role of AI in early cancer detection, our events bring together experts, innovators, and curious minds to share insights and drive meaningful change.

Here’s what we’ve got lined up:

Join a thriving network of 7000+ data & AI professionals, students, and enthusiasts. Take part in discussions, connect with peers, and be the first to hear about future events – join The Data Lab Community for free!