Inside the new AI model - trained on wearables - that could transform healthcare
Oxford University fellow explains the power of multimodal models for wearable data
Welcome to this week’s FREE edition of PULSE by Wareable. One day we’ll settle into a firm cadence of when you can expect the paid and free editions, but illness in the camp has put some strain on our planning.
But I also like writing about different topics as and when they arrive, and I hope these newsletters are as enjoyable to read and receive as they are to write. We’re having a great time thinking of and exploring topics for you.
As such, this week I wanted to focus on a new paper and foundation AI model created at the University of Oxford here in the UK that could have a transformative effect on wearables - especially in healthcare.
The paper, which has just been published in npj Digital Medicine, outlines the creation of AI models trained on hundreds of thousands of person-hours of wearables data. And it could transform the assumptions and analysis of wearables for a host of use cases.
I spoke to Hang Yuan, a research fellow at the university, who explained the foundation model and its potential for Parkinson’s disease patients, as well as health and wellness generally.
What’s the deal?
Researchers at the University of Oxford have built an open-source foundation model using the data from the accelerometers of wearable devices.
By training the model on over 700,000 person-days of data, which has been provided by the UK Biobank and gleaned from over 100,000 participants, the foundation model can analyze data more accurately than from traditional wearable algorithms.
“Our foundation models significantly outperform established benchmarks, with improvements ranging from 2.5% to 130.9% across populations, devices, activities, and placements,” said Hang Yuan in a LinkedIn post.
Capturing ‘hidden’ wearable data
Mostly, the foundation model will interest companies such as Johnson & Johnson, who are building the next generation of treatments and outcomes.
Those companies don’t get access to the data from general consumer wearable sensors, so, when research is done, it’s usually based on relatively small-scale studies. Having a foundation model trained on this wearable data at this scale can be transformative.
“I like to use the analogy coined by Andrej Karpathy, a co-founder of OpenAI and later worked on Tesla AI. He said you can view this kind of large language model like an operating system,” Hang Yuan told PULSE by Wareable.
“You can use a large language model as a basic operating system like Macintosh or Linux. But to enable large language models to work, you need to have foundation models. You need to have a foundation model for wearable signals, otherwise the large language model will not be able to interface with the modality you are interested in,” he said.
“In the medium to long term, the kind of foundation model that we have created will allow people to embed the wearable sensing data to go together with those large language models with clinical databases. And I think that's where we can truly capture all the information that has been hidden.”
A ‘gem of 21st-century biomedical research”
The data from this study has been taken from the UK Biobank, which Yuan called “one of the gems of 21st-century biomedical research.”
“It contains rich measurement of half a million individuals across the UK, across different demographic backgrounds, and followed up with health outcomes through the National Health Service,” he explained.
“The interesting thing about our model is that it will allow us to compare data with actual health outcomes, such as Parkinson's disease, cardiovascular disease or cancer.”
How could this model be used?
Right now, the model – available to be used as open-source code – is designed to demonstrate its potential.
“I think the short-term benefit is that we can demonstrate the ability to measure different kinds of activities, devices and populations.
“Even in the population with Parkinson's disease, who demonstrate very severe motor symptoms, we can improve the performance of measuring different activity sometimes even to about 100% with a limited amount of label data.
“We have shown with our foundation model, we can improve sleep staging and sleep classification as well,'“ Yuan said.
I asked if it was exciting that the model could unlock such huge potential from just the accelerometer, which is one of the most basic sensors in the wearable tech cannon.
“I think that's very much a fair statement to make. And what's very interesting about human sensing data with this foundation model integrated with a large language model, is that we can truly understand what the data is telling us,” he said.
The future is multimodel… but it’s hard
This foundation model - or ‘operating system’ - is only trained on accelerometer data. So that means heart rate data, or any other type of wearable sensor information, isn’t included. And making that a reality would be transformational, according to Yuan. But even the biggest companies out there are struggling with multimodal models.
“You have to create a foundation model for every modality you want. So that's why people are moving towards having multimodal foundation models – but that's harder and substantially more challenging and also much more resource intensive,” explained Hang Yuan.
“This foundation model was done two years back and we are thinking about creating those kinds of multimodal foundation models in the future,” he said.
“Apple recently released a foundation model of its own, looking at the heart rate ECG data, where they also cited our work. But even Apple is only looking at two modalities – but not more,” he said.
“That's where a lot of those kinds of integrations, with all the different modalities, will be really interesting – because it gives us the insight that we would never able to access by just looking at the data.”