Real World Testing Results 2023

 

General Information

Changes to Original Plan

Summary of Change Reason Impact
1.)For Measure 1, rather than testing monthly, Roji LLC tested the import of QRDA I once, at the end of the year. 2.)Enabling a user to timely create an export file(s) with all of a single patient’s electronic health information stored at the time of certification by the product, of which the Health IT Module is a part. 1.)Client was transitioning to a new EHR system and sent QRDA I data only at the end of the year, rather than monthly. 2.)2015 Edition Cures Update 170.315(b)(10) Electronic Health Information export 1.)No discernible impact. Roji Health Intelligence was able to generate QRDA I documents for 100% of cases, as expected. 2.)No discernible impact. Roji was able to generate EHI  documents for 100% of cases, as expected.

Summary of Testing Methods and Key Findings

The focus of Roji Health Intelligence’s Real World Testing measures was squarely on the import of data from clients and the capacity to create QRDA documents from that data. Measure 1 tests Roji’s ability to successfully import QRDA documents from clients. Measure 2 tests Roji’s ability to generate QRDA III documents from the data that a client has provided.
As expected, Roji Health Intelligence was able to successfully import QRDA I documents and generate QRDA III documents in 100% of cases in which this was attempted. This success demonstrates that Roji is a. compliant with the certification criteria; b. is exchanging electronic health information (EHI) in the care and practice settings for which it is marketed for use; and c. that EHI is received by and used in the certified health IT.

Measures used in overall  approach

RWT Measure 1
  • Description of the measurement/metric: Test a random subset of the patients per client every month. 
    • The numerator will be the number of distinct patients that valid QRDA I documents successfully imported.
    • The denominator will be the number of distinct patients this was attempted for.
  • Associated certification criteria: 
    • 45 CFR § 170.315(c)(2) Clinical quality measures (CQMs) – import and calculate
  • Justification for selected measurement/metric: Some clients can give us patient-level QRDA I documents that we normalize into our database. Other clients give us raw data that we normalize into our database. A successful import will allow us to perform calculations.
  • Care setting(s) that is addressed: A random subset should cover a variety of clinical variables.
  • Expected outcomes: We expect to be able to generate a QRDA I document for 100% of the cases.
RWT Results for Measure 1

Roji was able to generate QRDA I documents for 100% of cases.

RWT Measure 2
  • Description of the measurement/metric: Test a subset of the practice & provider combinations per client every month. 
    • The numerator will be the number of distinct client-practice-provider combos that a valid QRDA III document was generated.
    • The denominator will be the number of distinct client-practice-provider combos this was attempted for.
    • Report the particular CQMs applicable to the subset.
  • Associated certification criteria: 
    • 45 CFR § 170.315(c)(2) Clinical quality measures (CQMs) – import and calculate
    • 45 CFR § 170.315(c)(3) Clinical quality measures (CQMs) – report
  • Justification for selected measurement/metric: We should be able to generate valid QRDA III documents regardless of the CQMs.
  • Care setting(s) that is addressed: Clients, practices, and providers only report on applicable CQMs.
  • Expected outcomes: We expect to be able to generate a QRDA III document for 100% of the cases.
RWT Results for Measure 2

Roji was able to generate QRDA III documents for 100% of cases.

Standards Updates (Including Standards Version Advancement Process (SVAP) and United States Core Data for Interoperability (USCDI))

  • No, none of my products include these voluntary standards.

Update 1

  • Standard (and version): CMS Implementation Guide for Quality Reporting Document Architecture: Category I; Hospital Quality Reporting; Implementation Guide for 2020 (https://ecqi.healthit.gov/sites/default/files/QRDA-HQR-2020-CMS-IG-v1.1-508.pdf) 
  • Updated certification criteria and associated product:
    • Certification Criteria: 45 CFR § 170.315(c)(2) Clinical quality measures (CQMs) – import 
    • Product: Roji Clinical Data Registry
  • Health IT Module CHPL ID: 15.04.04.2951.Roji.16.00.0.170202
  • Method used for standard update: Update code in Roji QRDA Parser and CQM Engine.
  • Conformance measure: RWT Measure 2

Update 2

  • Standard (and version): CMS Implementation Guide for Quality Reporting Document Architecture: Category III; Eligible Clinicians and Eligible Professionals Programs; Implementation Guide for 2020 (https://ecqi.healthit.gov/sites/default/files/2020-CMS-QRDA-III-Eligible-Clinicians-and-EP-IG-v1.2.1-508.pdf) 
  • Updated certification criteria and associated product:
    • Certification Criteria: 45 CFR § 170.315(c)(3) Clinical quality measures (CQMs) – report
    • Product: Roji Clinical Data Registry
  • Health IT Module CHPL ID: 15.04.04.2951.Roji.16.00.0.170202
  • Method used for standard update: Update code in Roji QRDA Generator and CQM Engine.
  • Conformance measure: RWT Measure 2

Care Setting(s)

Measure 1 and Measure 2 in Roji Health Intelligence’s Real World Testing Plan are not specific to a care setting. The clinical settings of the data involved may include ambulatory, inpatient, outpatient, and emergency department.

Schedule of Key Milestones

Key Milestone Timeframe
Collect data for Roji RWT Measures 2023-06 to 2023-09
Conduct testing 2023-10 to 2023-12
Collation and finalization of results 2024-01

Attestation

These Real World Testing results are complete with all required elements, including measures that address all certification criteria and care settings. All information in this document is up to date and fully addresses the health IT developer’s Real World Testing requirements. 

  • Authorized Representative Name: Don Newman
  • Authorized Representative Email: [email protected]
  • Authorized Representative Phone: (312)258-8004 x712
  • Authorized Representative Signature:

  • Date: January 24, 2024