Note: Fall is back to school time and my now 13-year-old son is comfortably situated in seventh grade. Last year's initial shock of middle school is long past. My getting back into the nightly homework swing with my son reminded me of a blog post I wrote last fall on these pages that points out similarities between middle school math and data analytics for health center networks and providers. During my recent trip to the California Primary Care Association’s annual
Below you will find a “repost” of my article. I urge you to take the time to read and understand it if you are making decisions around reporting and analytics. If you have questions, don’t hesitate to contact us here at Azara.
Lessons in Healthcare Data Warehousing and Reporting from Sixth Grade Math
Hello everyone. Jeff Brandes here. My 13-year-old son Jacob started middle school last year, and helping him to complete his homework has forced me to unearth lessons and fundamentals buried deep in my gray matter for more than 30 years.
You might ask what that this has to do with healthcare and data warehousing. Well, the time spent reviewing some fundamental math concepts has led me to a better way to articulate some of the challenges facing Azara’s clients and prospects and the ways in which they must consider solving such challenges.
First, let me remind you of some basic math concepts we learned in middle school:
- You can’t add percentages! If one health center has 80 percent of its diabetics under control and another has only 20 percent under control, that doesn’t mean that in aggregate, they have 100 percent of their diabetics under control. In fact, in aggregate, with just percentages, you have no clue exactly how many diabetics are under control.
- You need a common denominator when adding fractions! This means that if you have two health centers measuring their diabetics under control using different measure definitions (or interpretations), you can’t add those two numbers and expect a meaningful result. In other analogies, you’re adding apples and oranges, taking you even further back -- all the way to first-grade math.
- Fractions are just numbers; they don’t tell the whole story! If you want to know what percentage of your diabetics are under control, you first need to know how many diabetic patients you have (denominator). Next, you need to know of those how many of them are controlled (numerator). If you put those two numbers into a fraction, you know the exact percentage of controlled diabetics in your patient population, but you don’t know anything else -- not the causes, not the disparities and not even if they have been into your office in the last year or if they have an appointment next week.
So why does this all matter, and why the heck is Azara writing about it?
Well, a potential client who runs a medium size network of health centers, with 15-20 or so participating members recently approached me and said, “Jeff, we might need you to work with some of our members who have another ‘data warehouse’ product so that we can see the performance of all of our members, both individually and in aggregate, in one place and run some new network wide programs. In fact, we are also thinking about forming an Independent Practice Association of health centers and we will need aggregated data for that as well.” I took some time to formulate a thoughtful answer, and while Azara always wants to be supportive of our clients and inclusive of member centers using alternative technologies to Azara DRVS, this one was hard.
I asked my potential client to tell me about this “other data warehouse” and he proceeded to describe a system where each individual center has a reporting tool that is connected to its EHR database. Within this reporting tool, the centers have a variety of functions and the ability to explore their data, create clinical quality measures and monitor performance. He also explained that this vendor has another product, or second system, that will aggregate measure data from the tool at the individual centers and create a “data warehouse” that will allow his network staff or individual center staff to compare the centers to each other and provide “global health analytics” across their network. However, I uncovered all sorts of interesting things as I asked more questions to determine if we could incorporate their data in Azara DRVS.
First, I asked who determines the measures and how can the network staff be sure they are the same at each health center. There was a pause, then a “well, we have a standard set of measures we have chosen, but each center does have the ability to modify them on their own. I am sure they will use the standard templates we all agreed upon, so there will be no issues.”
I said “OK” and moved on. I then asked what information is in the central data warehouse, and he stated confidently, “Oh, they take the measure results from each center, numerator, denominator and exceptions/exclusions and bring them to a server in the cloud so we can see comparative analytics.”
I pondered this for a moment and asked, “So how do you get at the detail behind each measure? Is there a way to produce identified or de-identified patient lists?”
A blank stare came over him, and he admitted, “I don’t think they do that. Why would we need to do that?” I gently followed up, “So this data warehouse you have doesn’t have patients?”
His expression grew more puzzled. At this point, I did not want to embarrass my potential client and politely ended this line of questioning, but what I wanted to know was if the supposed data warehouse had no patient data, exactly what did it have that was important and why it was referred to as a data warehouse.
I guess -- using a liberal definition -- the particular product might be a data warehouse, although WIKIPEDIA defines a data warehouse as:
“In computing, a data warehouse (DW, DWH), or an enterprise data warehouse (EDW), is a system used for reporting and data analysis. Integrating data from one or more disparate sources creates a central repository of data, a data warehouse (DW). Data warehouses store current and historical data and are used for creating trending reports for senior management reporting such as annual and quarterly comparisons.
The data stored in the warehouse is uploaded from the operational systems (such as marketing, sales, etc., shown in the figure to the right). The data may pass through an operational data store for additional operations before it is used in the DW for reporting.”
But when it comes to reporting on primary care clinical performance, an aggregation of numerators and denominators across a group of clinical quality measures DOES NOT constitute a functional or useful data warehouse. Sure, you can generate an overall performance metric across many sites and do benchmarking between centers and sites, but that is about it. You can’t even do simple things like validate the data behind the measures with your participating provider organizations. While this approach may be helpful for some specific state or network-wide quality improvement efforts, it’s quite limiting when it comes to running impactful, coordinated programs that improve the quality, coordination and delivery of care across a group of heterogeneous primary care or community health center practices. It’s certainly of no use in achieving clinical integration, a legal requirement for operating an IPA or building an ACO.
In fact, this type of data aggregation would not even meet the current grant reporting needs of many of Azara’s community health center clients because is doesn’t contain the patient level detail required by many of these programs. The recent CMS innovation grant application for the Transforming Clinical Practice Initiative (TCPI) and Practice Transformation Networks (PTN’s) contains an explicit requirement for: the applicant to demonstrate established data sharing capabilities with clinical providers that includes the ability to collect, hold and evaluate personally identifiable information (PII).
In addition, a data warehouse needs to be capable of assembling a variety of information types, not just clinical quality related elements. Your master data store should hold not just demographic and clinical data, but be able to integrate items like cost, risk, utilization and transitions of care, all data points that come from sources other than the primary care EHR setting. Beyond that, it’s impossible to gain deeper understanding behind specific results, pinpoint causal relationships or devise effective action plans without the patient detail, even de-identified detail, behind the numerators and denominators (ex: how can one explore the disparities within the results, whether by demographic factors, socio-economic or even clinical components without underlying data). Going back to my math class experience, you can’t figure out who to recruit for the math team if individual test scores are not available and all you know is that on average, Mrs. Smith’s math class does better than Mr. Jones’ math class.
Azara feels strongly that it’s absolutely necessary to build a data warehouse with the breadth and depth of data necessary to support CHCs and primary care provider networks as they prepare to respond to the challenges of healthcare reform. We take pride in what we deliver to the marketplace and the sophisticated reporting and analytics that we build upon a rich store of information.
As you contemplate the suite of tools and technologies that you need to remain competitive in a rapidly evolving healthcare marketplace, I urge you to dig deeper, ask the hard questions, look past both the fancy demos and the proposal’s price page. Understand what your network really needs and what your members need for themselves, as these are not always the same things. Closely consider what each proposed offering will provide (and not provide) now and in the future, and take the time to truly understand the implication of not having a centralized collection of patient data.
And when you are around the table making these evaluations, understand that you learned much of what you need to know for making this decision in your middle school math class.