Medical affairs and the real world

Medical affairs and the real world

By Christiane Truelove | [email protected]

It’s not just about drug development anymore. When it comes to the use of real-world evidence (RWE) by pharma companies, while most efforts have been focused on the clinical development side, judiciously deployed RWE can help medical affairs departments enhance their operations.

Medical affairs, focused on communicating scientific and clinical information to healthcare providers and responsible for activities that bridge the gap between the development and commercialization of drugs, engage with providers and other stakeholders to educate them around correct usage of drugs as well as answer their queries around safety, off-label usage, scientific publications, and more. But getting the right real-world data (RWD) to support RWE is crucial.

RWD and RWE’s growing acceptance

The importance of RWE and RWD continues to grow for the pharma industry as regulatory bodies accept the data generated by these types of studies in submissions for new drugs and biologics.

In fiscal year 2023, the U.S. Food and Drug Administration  approved 10 clinical study protocols using RWE, and four New Drug Applications/Biologics License Applications (NDAs/BLAs) had RWE in their submissions to the Center for Drug Evaluation and Research (CDER) that met reporting criteria. Two drugs, Genentech’s Actemra and UCB Inc.’s Vimpat, were approved, at least in part, due to RWE. The Actemra approval was for a new route of administration, and the Vimpat approval was for a new dosing regimen.

The challenge is trying to generate viable RWE is finding “fit for purpose”  real-world data. As FDA Commissioner Robert Califf blogged in 2023, “Data from routine-care settings are focused on patients at the bedside or in the clinic, requiring consideration of the data’s reliability and relevance to address regulatory questions.

“For example, information on clinical measures of disease severity can be lacking in healthcare insurance claims, and we need better ways of examining unstructured data in electronic health records. Another category of challenges involves safeguarding patient privacy while having suitable mechanisms for data access and inspection by regulatory agencies. These challenges will require continued collaboration among a range of stakeholders including regulators, industry, academia, and the clinical research community.”

Medical affairs and RWE

Writing for the Medical Affairs Professional Society, attorney Neil Belson, MAPS’ Cerise James, and Aetion’s Ashley Jaksa say, “As RWE becomes part of the health care mainstream, medical affairs professionals have an opportunity to guide their internal and external stakeholders in adopting RWE. MA professionals can take advantage of this opportunity by championing the use of RWE where applicable, working with stakeholders to ensure RWE studies meet high-quality methodological standards, collaborating with internal and external stakeholders on projects that fill known gaps in RWE methodology to push standards forward and making sure that their companies and organizations are aware of the benefits RWE can offer.”

According to Russ Corb at PurpleLab Healthcare Analytics, in his July 2024 blog “Pharmaceutical Medical Affairs and the Role of Real-World Evidence,” RWE can supplement clinical trial data, identify unmet needs, and inform key opinion leaders.  “RWE, such as identifying clinical practice trends, can complement findings from clinical trials by providing a broader picture of a drug’s effectiveness and safety in real-world clinical practice,” he says. “This can be particularly useful for understanding how a drug performs in different patient populations or identifying rare side effects not captured in trials.

Corb states that RWE can help medical affairs teams understand disease burden, treatment patterns, and patient outcomes in real-world settings, which can aid in identifying areas where existing treatments fall short and inform strategies for developing new drugs that address unmet medical needs.

And when it comes to talking with professionals and patients, “medical affairs teams can enter into conversations with KOLs and patients advocates in an informed manner, and can more readily address any queries they have.”

Additionally, RWE can be used to create targeted educational materials for healthcare providers that address real-
world challenges and considerations when using a drug; help medical affairs teams demonstrate the value proposition of a drug to HCPs and payers by providing evidence on real-world outcomes and cost-effectiveness compared to alternative treatments; and enhance scientific publications by providing a more comprehensive view of a drug’s impact.

RWE also allows medical affairs teams to continuously monitor the safety profile of their drugs after launch and reveal how physicians are prescribing drugs off label. “Overall, RWE empowers medical affairs teams by enhancing the credibility of their products among healthcare professionals and payers,” Corb says. “By leveraging robust data from real-world settings, medical affairs teams can present compelling evidence that supports the efficacy and safety of their company’s drugs. This in turn fosters greater trust and acceptance from the medical community.”

AI and RWE for medical affairs

Pharma continues to deal with a tidal wave of data. According to McKinsey and Company, the amount of healthcare data available is estimated to have reached 2 zettabytes, or 2 trillion gigabytes, in 2022. But these experts say the  “explosion” in data “provides a unique opportunity for better medical affairs decision making.”

“Specifically, it is now possible to close the loop by measuring deviations from optimal care in claims data, reorienting strategies, and quantifying the impact of medical affairs activities to reduce suboptimal care,” McKinsey says.

McKinsey says leading organizations  are focusing on three priorities to deliver the promise of integrated data and analytics in medical affairs, by establishing medical data and analytics capabilities; integrating digital and analytics into day-to-day decision making; and automating repetitive non-core activities.

The latter is where AI has been coming to the fore, allowing medical affairs teams to prioritize high-impact activities while deprioritizing but still delivering traditional activities, such as medical review and medical-information standard responses, according to McKinsey. “Automation, including recent breakthroughs in gen AI, could significantly increase efficiency across all of medical affairs,” these experts say. “For example, gen AI can scan a broad range of sources, including external interactions, social media, surveys, trial data, and real-world data, to instantaneously produce insights and recommendations for evidence generation plans, medical strategies, and other applications. An emerging gen AI capability could also observe and generate actions in the digital or physical world to optimize medical processes such as medical review.”

Voices from the field

Annie Yim is a data scientist working for  Boehringer Ingelheim’s central data science team. Although she is not involved in specific business functions such as medical affairs, she and her team work with other teams to achieve the ideas that could assist them with their day-to-day work.

“The projects that we do really vary a lot,” Yim told Med Ad News. “Yes, it could be from R&D, to the efficiency of machines in manufacturing, to marketing. We also have projects with our CDMO [contract design and manufacturing organization] that help our colleagues with designing global trials. It really varies a lot, and as a data scientist, we would try to always leverage AI, machine learning, and data science to assist our colleagues and their projects.”

The tools she and Boehringer Ingelheim’s data science team use range from classic machine learning to newer generative AI and large language models (LLMs). Projects that she has collaborated on include scientific communications. Doctors frequently have questions about products, and medical writers must provide information and support.

“Sometimes this could be repetitive, because usually the responses written by our writers to GPs have to be in a certain kind of standardized format,” Yim says. “But now with generative AI, those large language models could also help with composing these responses. It would make their lives much easier, and by providing the relevant references or scientific publications, our colleagues could shorten the time of composing these responses to the doctors.”

Another way AI can be deployed to assist the medical affairs team is when they are engaged in supporting clinical trials.

“We can perhaps develop models that predict patients’ outcome, or even make early diagnosis for some diseases, using claims data,” Yim says. “And this could help us more easily identify patients, for future clinical trials.”

Additionally, this kind of information can be shared with HCPs as educational resources to help them recognize the early signs of some diseases and make more informed treatment decisions.

In the post-marketing surveillance area, once a drug or product has been launched, AI can help sift through the data obtained by monitoring for safety or efficacy, giving medical affairs insights beyond what could be learned from standard, controlled clinical trials. “This can  be used to craft messages for doctors and help them understand the effectiveness of our drug in real world settings,” Yim says.

One of the biggest challenges of working with AI is data, “because we need data to train our algorithm and the quality of the data is very, very important,” Yim says. “And the challenge with real-world evidence or real-world data is that usually they are quite messy and sometimes incomplete.”

One problem is that electronic health records can be lacking critical information. “We might have missing data, we might have inaccurate data, inaccurate entries, or the format could be inconsistent, and this could be challenging for us to use as an input for AI algorithms.”

Another challenge is biases in the algorithms. “Because they are trained on data, they can be biased based on the data that we’re providing, if it is not representative enough,” Yim says. “For example, if we’re using trials but only from a certain country, from a certain region, then we might be training an algorithm that’s biased.

“Going back to this example that I mentioned about disease diagnosis, what if this disease is particularly prevalent in this region and the patients of this region, for some reason, show slightly different symptoms from other patients, from other regions. So if the data itself is not representative enough, and we use those data to train our algorithm, then it might cause biases in the outcome and makes it not really reliable.

Another challenge for medical affairs teams is the interpretability of AI algorithms, especially complex ones such as deep learning models that include generative AI and large language models, because of their complexity.

“The output of these models can be difficult to interpret or to explain, and this could be challenging for our medical affairs colleagues, when they have to communicate or explain the results to healthcare professionals and doctors,” Yim says. HCPs may then not trust the output of the model.

However, Yim says her medical affairs colleagues have had a good understanding of what the data science team is doing.

“We collaborate quite closely, and sometimes they reach out to us with an idea,” she states. “It’s not always feasible, because they might not necessarily understand what it means by using AI or machine learning. So before we start a project, we usually would have a quite extensive exchange to make sure that they know what we’re doing before we really start with the project. We would also investigate, for example, the data that we would have to use for this project, [and] make sure the quantitative is enough. It’s not like we have just 10 data points. And also, if the quality is good enough, if it’s representative enough.”

While pharma as a whole has been known to be conservative about employing new technologies, Yim says her colleagues outside of the data team, including medical affairs, are generally open to the use of new technology. To support learning, the data science team has produced educational material about AI and machine learning.

“So with even more colleagues having knowledge about AI and machine learning, I’m sure they will come up with even more ideas of how they can apply to technology, to their use cases or to their projects,” Yim says, adding that she is optimistic about the use of AI at least within Boehringer Ingelheim.

Technology vendors are eager to meet pharma’s needs with AI solutions, and see potential for medical affairs teams. For example, PicnicHealth primarily helps life sciences and pharmaceutical companies run their observational research by running studies, building registries, and getting data that is describing patients’ journeys in the real world.

“What I think we’re really able to do, kind of uniquely, is really simplify observational research on the basis of the technology platform that we’ve built,” according to Troy Astorino, one of PicnicHealth’s founders and chief technology officer. “Our technology platform lets us, for any patient that consents and authorizes us to act on their behalf, to be able to get all of their records from any site of care across the U.S, and then, using the LLM, the AI models that we’ve trained, transform those records into a unified view of patient care and the patient journey to really understand the care that patients received.”

A core part of how PicnicHealth operates is when patients sign up to participate in research to get their records, the company gives any of the records collected back to the patient through a portal that lets them see their care across all their providers. “We’ve been around for 10 years now, building this platform. We work with 12 of the top 20 or top 30 pharma companies, and we work with over 100,000 patients. And things have been really, really accelerating over the past couple of years,  especially being able to incorporate newer AI and machine learning techniques, we have better systems for reaching patients.”

While pharma has traditionally used data sets derived from claims data as well as de-identified patient information from electronic medical records, Astorino notes that the challenge with these generalized data sets is there are a lot of gaps in the EMRs.  “Depending on the story you’re trying to understand, the story you’re trying to tell,  those existing data sets might not have the endpoints, the outcomes, [or] the information that you need.”

Where PicnicHealth often works with medical affairs teams is when those off-the-shelf data sets are missing information because it appeared in notes or other parts of the medical record that aren’t available. “We can provide these deep longitudinal journeys that make sure we capture the information that our customers would need,” Astorino says. The company can survey patients and get patient-reported outcome measures and other descriptions of how the patient experiences treatment, “which I think more providers and payers do care about.”

Medical affairs teams using off-the-shelf claims data or EMR data may find themselves having to run a traditional site-based observational trial or site-based registry to address gaps in the data. “That is very, very expensive and inefficient,” Astorino says. “I think we provide a much more efficient, simplified option that can get you all the data that historically you’d have to run one of these site based studies” with a lower cost, and with more transparency and flexibility. “If you want need to adapt your study design, adapt the endpoints that you’re looking for, we can do that.”

One thing PicnicHealth can do is pick up imaging as part of data sets. One customer was trying to get images of a neurological disease, to see the progression of lesions that show up on MRIs. “We were able to capture all of the actual DICOM images, not just the reports, and then they were able to use those to better quantify the progression of the disease through how the size of those lesions changed over time,” Astorino says.

Using AI responsibly

According to Forrest Pascal, a former AI leader at Kaiser Permanente and now VP of healthcare and responsible AI at ModelOp, which operates AI governance software for enterprises, when looking at RWD, “we have to have responsible AI.”

“That responsible AI is going to look across  your data quality, the reliability of your data, the trust, misinterpretation, [and] overrepresentation – representing the results would be key,” Pascal says. “If we’re looking for recruitment selection, understanding the skewness of the data, that bivariant of the data, [or] the multivariant of the data, making sure it’s stratified and segmented right [is important]. There’s lots of places where it can go off the track.”

Making sure that AI does not skew the data and its results is why  humans need to be in the loop. “All of these types of areas are the same thing that we would have with any other type of data,” Pascal says. “From a medical affairs point of view, your education, your medical liaison folks, you want to make sure you’re getting the right message across to them and not overrepresenting [the data].”

At this point when it comes to using generative AI, the industry is “in a point of continuous change,” Pascal says. “So we have to just stay close to it like any other application. And we certainly have to line up what we’re doing with the regulatory compliance.”

Pascal says AI just gives pharma companies a new way to address the same challenges. “I think we still have the same kinds of problems, basically, we just have the computing power. And we can make those inferences better. We can look at the data better, we can look at the skewness better, we just have to have better trained humans to interpret what we get back. I think it’s the human in the loop. And many times it’s checking the human, because you have researcher bias and selection bias, and you’ve got to have compounding variables.

“At the end of the day, I don’t see where we’ve shifted anything other than that we’re able to hit a button versus playing the computer and doing check-by-check, line-by-line studies. The key thing is that the outcome has to be explainable, the results have to be explainable, and the population has to be explained with all the same things you would do in any type of research.”

Pascal points out that using AI also does not eliminate the possibility that medical affairs and promotion teams will cherry pick data, and certain algorithms may support cherry picking as well. “Not all algorithms behave the same way. Gradient boosts are going to behave much different than random forests, for example, or support vector machines. Most of them are geared for supervised learning. And then if you’re using deep learning and using neural nets, and, convolutional neural nets, explainability is the key. How did you get to that result? How did they get there?”

ModelOp comes with at least 40 out-of-the-box metrics that look at performance. “And that performance looks at the area under the curve, which is the receiver operating curve. It looks at accuracy, it looks at loss, and looks at mean scars and square error. It looks at confusing kinds of matrix with sensitivity and specificity, so that you can look at your ground truth,” Pascal told Med Ad News. “There’s a ton of metrics that come out of the box that will look at the data and give you an idea of its accuracy.

“But even on top of that, model ops allows those AI models that are processing real-world evidence, through real-world data that is pulled in from electronic records, [and lets you] set up checkpoints using a workflow, so that you can validate and have levels of validation. And that could be manual, it could be automated, or it could be both.

“And then lastly, out of the box, it comes with lots of bias models for group bias. Whatever it may be, it comes up with monitors that you can apply to the model that will look at equal odds.” It also has LLM standard tests that it can run to compare a client’s large language model process compared to ChatGPT, looking for the correctness and groundedness of factual data.

“Missingness” in medical data will also skew results. “If I have a mother in Oakland who is also going through an infusion program or some kind of cancer infusion treatment, and she has a choice to send her kids to school with lunch, or take the bus [so she can] make her appointment to get her care, if she chooses her child, she’s not represented in the data,” Pascal says. “Your clinical trials departments, your medical education departments, medical affairs, they have to be aware of this – you can’t over-represent an overreach, because we don’t really know what went on with the data. The models are only as good as the data.”

Another way to think of AI models is like a boat and data as the water it floats in. “If the data is stagnant or static, then the model will work pretty effectively on the data because that’s what it was trained for,” Pascal says. “But if you take that model, or if you enter new data, we’ve opened up the dam, and the pond has got new ‘water.’ We call that drifting. Now the model is not anchored to that data, it wasn’t trained on that new data. So it’s accuracy on being able to predict outcomes, no matter how good the evidence is, will diminish.”

Ultimately, “we don’t want to have an overreliance on RWE, and AI should be an augment to that,” Pascal says.

Chris Truelove is a contributing editor at Med Ad News.