The better clinical research is, the better medical care will be. It is so crucial to the future of best medical care that I have highlighted deficiencies of the present conduct of randomized trials (RTs) in previous articles to suggest ways to improve. A system of better research must accommodate studies on any intervention aimed to improve care, including interventions such as a change in practice, any quality or safety plan, or an economic principle such as fee-for-service versus capitation—not just studies of new drugs.
In my last article, I coined the term “Gallup Research Medicine” as a model to improve the generalizability of RT studies by
- using random samples of full populations of disease registry patients, and
- increasing the number of people in RTs with variable prognostic characteristics.
Here I propose a second model for better clinical research: a stock-market model.
Random Sampling Is Not Always Random
A criticism of random sampling of even full populations of people is that not all people sampled agree to participate in a study. Gallup, for example, uses a full population of phone numbers and randomly calls those numbers. However, Gallup cannot control who accepts their call (as few as 35 percent of those contacted). The problem is that the variable “accept a call” is not necessarily randomly distributed. For example, some area codes are inundated with spam calls, which may make people less likely to participate by accepting Gallup’s call. This introduces error, and Gallup accounts for this variable, and others, as best it can. However, as soon as accounting or adjusting is needed, randomization is compromised. (Of course, who does/does not accept entry into a study is a problem for any study design).
If random sampling is problematic due to non-random reasons to participate/not participate in a study, studying a full sample of patients would circumvent the problems. Full sample research presents its own challenges. But before throwing the idea out at first mention, let’s consider what situations might be compatible with this idea and what measurement concepts would need to be followed.
How a “Medical Stock Market” Model Could Benefit Research
The stock market serves as an example. First, the stock market is a full sample of selected companies (more later), and, second, it uses a standardized, uniform measure for the value of all companies.
The stock market is a compilation of stock indexes. Each index (e.g., Dow) reports the performance of all companies in the index. An index’s measure is the weighted average of each company’s stock price times the number of shares.
The stock market is also a transparent measurement system. We see daily reports of an index’s outcome value (dollars). The presentation of the measure is a run-time chart, the measure is the average, weighted index value over time. In summary, the stock market includes a full sample; a sensible, stable, uniform measure of value; and transparent presentations of data.
Not all companies are in stock indexes, but each index has a defined and full set of companies representing a type of business (e.g., technology). Clinical research might follow this model. Each “index” could be a defined entity such as a hospital, a group of hospitals, or an ACO. Alternatively, an index might be a disease registry. All patients in those entities would be included in the index.
The stable measures in clinical research would not be as simple as stock price times the number of shares. Clinical conditions have multiple outcome measures. Disease conditions, however, share standard measures used by researchers. For example, diabetes research may measure blood sugar, A1C values, or creatinine clearance. Breast cancer outcome measures include stage, treatments given, recurrence rates, life and death. In any research study, a limited set of measures are defined and followed. If a parsimonious set of measures were followed over time, a stock-market model would fit.
Measuring Hospital Safety with Medical Indexes for a Full Patient Sample
An example might help. When I was the safety officer for our department of medicine, we measured 20 variables on every patient admitted to our department. The index “fund” was all patients admitted to our medical wards.
We spent several months deciding measures for our “index fund”; some were disease specific, some measured utilization. For an example of a disease measure, we tracked blood sugar values on admitted diabetic patient and presented a run-time chart at our monthly department meetings. The chart showed the average blood sugar value and range of values. This means we knew the percent of patients with low or high blood sugar values as well. Another clinical measure was percent of patients with pain scores higher than 6 (out of 10). An example of a utilization measure was the percent of paired orders for amylase and lipase. Since they are redundant, we hoped to show we could reduce the tandem use of the tests.
These clinical and utilization stock-like measures showed the functioning of our hospital. With stable measures presented monthly, it was easy to note if any of the measures changed. For example, one month we noted a marked change in the percent of patients with pain scores over 6 and learned that nursing had altered rules for how the pain score was measured.
The observation of this change in both practice and pain scores suggested a research avenue. It was easy to show differences in the pain outcome measure as a consequence of the systematic change in nursing practice because the entire population of patients had been intervened on. Given the standard and stable measure, we noted the change in a single month’s report.
This observation, in fact, was clinical research in action; nurses intervened on all patients and outcome measures changed. We easily noted how much the percent of patients having a pain score over 6 changed with the intervention; comparison was obvious. The change was so dramatic that statistics were not needed to show a significant difference, but we did do statistical testing if needed.
Following the observation of change for full population interventions, we then intervened on the entire population of patients getting laboratory tests with amylase and lipase for presumed pancreatitis. After the intervention—an educational plan involving all ordering physicians— the percent of paired tests was reduced from over 90 percent to less than 10 percent, a statistically and clinically significant difference.
We conducted full population research studies for many of our outcome measures. We reduced the percent of patients with hypoglycemia by raising the target level for blood sugar during a patient’s hospital stay; we reduced the use of chronic medications for non-life threatening or altering conditions; we eliminated dangerously high potassium levels by imposing restrictions on the amount of potassium supplementation.
We were just one hospital, but the model is transferable to any other, and groups of any other hospitals. This model would not work, perhaps, for introducing new drugs, but for any quality/safety/policy intervention, intervening on full populations after establishing baseline measures for outcomes would allow for generalizable and locally responsive advances in care.
Other Applications of Full Population Sampling
To recap, random sampling of full populations is ideal, but difficult. Research on entire populations is equally valuable, but there are limits on interventions that may be tested. Stable, routine measures for outcomes is paramount. Our hospital chose what we wanted measured, but our government research agencies could encourage similar measures for all covered patients in our country. If organized, we could test multiple interventions on full samples of patients across the entire country to see what might be best. The key, again, is full samples, not non-random portions of full samples.
This model is also useful for learning how local environments of care may alter what interventions are possible or useful. The results of studies in this model are tangible and readily shared. The cost of doing research will be relatively low as this model makes research part of the day to day care of patients. We watched our “stock index hospital fund” monthly and changed actions when needed.
Research efforts can get better, and I propose two models. There are likely others. The stock market analogy is not perfect for clinical research, as medical care will be more systematic when asking questions and planning interventions. The stock market, on the other hand, is a sometimes capricious animal that reacts, rather than plans. Yet, the conceptual model of information management works as a platform for clinical research.
In the next blog I will address logistical considerations in hopes of spurring discussion about how to change the organization of clinical research.
Founded as ICLOPS in 2002, Roji Health Intelligence guides health care systems, providers and patients on the path to better health through Solutions that help providers improve their value and succeed in Risk. Roji Health Intelligence is a CMS Qualified Clinical Data Registry.
Image: Aditya Vyas
