Perspectives from the pandemic: Information gaps, hidden variables can stymie predictive models


Despite having some of the most advanced tools and technologies, our health care ecosystem lags when it comes to using and sharing data to improve outcomes and experience. Many factors are to blame, including policy and data collection challenges that become even more vexing when faced with major disruptions like the current COVID-19 pandemic.

In the first article in this series, Steve Griffiths, COO Enterprise Analytics for Optum, rightly pointed to a lack of necessary information at the beginning of the COVID-19 pandemic — a dearth that hindered our ability to anticipate the spread and impact of an entirely new virus. Srinivas Sridhara also weighed in on the importance of real-time data in our last blog. Deficiencies in these areas during the COVID-19 pandemic have reinforced the concept that information is the lifeblood of health care.

In this article, we examine the challenges these deficiencies and the variability of human behavior pose to disease modeling and surveillance efforts, which are essential to responding to crises and ongoing public health issues alike. How do we account for these factors and build better model forecasts that support better outcomes?

A modeling and surveillance evolution

From the outset of discovering a public health emergency, epidemiologists, public health officials and other planners turn to forecasting models, which apply new information to learnings from past experience to help inform their decisions. Predominant among these models is the SEIR framework, which tracks the number of people who are Susceptible, Exposed, Infected and Recovered (or died) from an illness.

When armed with good data on these variables, the SEIR framework is a valuable resource planning tool. For example, it delivers insights to help inform forecasts around hospital utilization and equipment needs (e.g., beds, ICUs, ventilators). However, early in a pandemic, SEIR models, like other forecasting resources, must make the best use of incomplete information. This introduces wide variability in forecasts because, as more data becomes available, rates of each factor in the SEIR model may change considerably.

The early COVID-19 experience was no exception. There was simply too much we didn’t know about the disease. That, coupled with the lack of testing and bottlenecks preventing efficient, standardized collection of information, left modelers to make assumptions or calculations that could not become more accurate until more complete data became available.

On this front, artificial intelligence (AI) may be proving to be a powerful ally to fill in information gaps. AI systems work by feeding computer models with leading- and lagging-indicator data. These data might be reflective of internet search patterns or population mobility (i.e., where people travel and how long they stay as recorded from their mobile phones or other sources) and integrated with related clinical, demographic, socioeconomic and health care-related data. We use these capabilities to identify early micro trends — assuming the data and analyst perspectives are sufficiently diverse and representative of the populations of interest. By filling in certain knowledge gaps, AI capabilities can help us understand more about the course of a disease and how people respond to it, and produce more reliable disease forecasts.

Accounting for variability in human behavior

Despite the advantages, even AI-powered modeling tools have limitations. Variations in human behavior, including, for example, the extent to which people will follow social distancing guidelines or how many will wear masks properly or consistently, are difficult to anticipate and may be observed only in hindsight. Recent social protests and other large events were also unanticipated yet may have impacted the course of COVID-19.

To fill in information gaps, promising developments in the availability and adoption of opt-in mobile apps and other tools currently help researchers collect anonymized information (e.g., location and proximity details) and other data. Observing and analyzing more individual-level disease surveillance indicators in close to real time — ahead of the appearance of symptoms — could power far more accurate forecasts than we can currently achieve, and provide insights that inform better public policy and hospital and supply chain preparedness.

The Optum COVID dashboard is an example of these ideas in action. The dashboard leverages AI, as well as robust, anonymized private and public data to identify patterns and make inferences and predictions about the timing and location of COVID-19 outbreaks, at the state and county level.

Building a more flexible, responsive health system

If understanding and managing COVID-19 is like running a marathon, we’ve just begun the race. We’ve learned a lot about this virus, but there is so much more we don’t know, especially about health problems that remain even after the virus becomes undetectable.

The good news is that in the response to the pandemic, our health care delivery system has taken an unforeseen step forward to a more digitally agile world. We’ve seen greater willingness among insurers, providers, government and other stakeholders to collaborate, and rapid innovation to expedite collection, analysis and sharing in near real time of data that had once taken months to process. And almost overnight, it seems, virtual care technologies such as telehealth and remote patient monitoring gained widespread acceptance.

These are important shifts that reinforce the enabling power of data and analytics in health care. Our next article in this series will discuss efforts to sustain momentum in ways that not only enhance how we respond to a public health emergency but empower a more flexible and responsive system for everyone.

Additional stories around the industry response to COVID-19 and our efforts to confront current challenges can be found in the Optum® News Room. You can also find more perspectives on enabling health care innovation on our data, analytics and technology blog.

About the authors:

Ron Ozminkowski, PhD, Senior Vice President, The Lewin Group

Dr. Ron Ozminkowski joined The Lewin Group in 2019 as senior vice president in the government business. He is responsible for executive review of analytics and methods and for driving the use the machine learning and artificial intelligence to benefit our state and federal clients.

Dr. Ozminkowski is an expert in health care analytics, machine learning and artificial intelligence. He has over 20 years of total management experience. His executive experience includes 12 years in senior leadership and chief scientific officer roles at IBM and Optum, leading large groups of analysts.

In his previous role at Optum, Ron and his teammates achieved double-digit improvements in operating discipline, managerial efficiency and employee development.

Dr. Ozminkowski has participated in over 200 in-depth collaborations and has provided senior thought leadership and guidance for over 1,200 other research and reporting projects. He also has over 120 peer-reviewed publications and more than 40 other professional magazine articles and book chapters. His work has been cited or quoted often in the media.

Ron Ozminkowski has a doctorate in health services organization and policy and master’s degrees in applied economics and health services administration, medical care organization, both from the University of Michigan. His bachelor’s degree is in health education, and he brings an integration of all of these perspectives to his work on behalf of our clients.

Jay Hazelrigs

Jay P. Hazelrigs, ASA, MAAA, Vice President and National Practice Lead, Provider Actuarial Services, Optum Advisory Services

Jay is a vice president and the practice lead for the Optum Provider Actuarial Services practice. He has worked as a health care actuary for over 25 years and consulted with multiple Fortune 500 companies throughout his career. His unique background includes consulting to health systems, ACOs, providers, employers, government and commercial payers across a multitude of business and actuarial issues.

Jay’s more recent consulting engagements include providing value-based care (VBC) expertise, developing new transformative payer-provider models, leading the Optum COVID-19 epidemiological forecasting and financial scenario modeling capabilities and developing strategic growth strategies for health systems. He assists organizations in understanding their changing market dynamics and helps his clients formulate market, product and benefit plan strategies.

Additionally, Jay has worked in the traditional actuarial capacity with providers evaluating risk-based and fee-for-service contracts and with payers performing actuarial activities such as rate certifications, annual reports, provider contracting, plan design impact analyses, reserve estimations and benchmarking analyses.

Jay is an Associate of the Society of Actuaries and a member of the American Academy of Actuaries. He graduated from Georgia State University with a Bachelor of Business Administration degree. Jay is a frequent speaker at actuarial and ACO events and the co-chair of the Value Based Care Strategic Initiative for the Society of Actuaries.


Danita Kiser, Director of Product Research, Advanced Technology Collaborative, Optum

Danita has over 20 years’ experience in advanced and emerging technology products, covering areas such as machine learning and artificial intelligence, big data technologies and analytics, gaming, media distribution, cloud services and mobile applications.

As Director of Research at Optum, she’s responsible for connecting research and emerging technology to real-life health care use cases in an effort to make the health care system better for everyone.

Prior to Optum, she worked at Turner Broadcasting, AOL, Cox Communications and multiple first-round start-ups.

Danita earned a Bachelor of Science degree in chemistry and mathematics from University of Virginia and an M.S. and Ph.D. in chemistry from Emory University.


Elya Papoyan, MPH, Actuarial Senior Consultant, Provider Actuarial Services, Optum

Elya is a senior consultant in the Provider Actuarial practice of Optum Advisory Services and based out of Los Angeles, California. When not in the midst of a pandemic, Elya works in health outcomes, economics and services research and analytics. Elya trained as an epidemiologist at the Columbia Mailman School of Public Health with a concentration in infectious diseases.

During her studies, she was also a Fulbright scholar to the Republic of Armenia where she studied the epidemiology of drug-resistant tuberculosis. For the last five months Elya has been working on Covid-19 pandemic-related forecasting tools and surveillance models aimed at consolidating and interpreting the constantly evolving data for actionable results for payers, providers and employers.

Leave a Reply