Roji Health Intelligence

Navigate The Path To Better Health Care.

Real World Testing Results 2024

Summary

This is the Real World Testing Results document for 2024 by Roji Health Intelligence LLC. The results are designed to meet certification requirements for Clinical Quality Measures (CQMs) and Electronic Health Information (EHI) export in compliance with 21st Century Cures Act: Interoperability, Information Blocking, and the ONC Health IT Certification Program final rule, 85 FR 25642 (https://www.govinfo.gov/content/pkg/FR-2020-05-01/pdf/2020-07419.pdf) (May 1, 2020) (ONC Cures Act Final Rule), Section VII.B.5 “Real World Testing”.

This Real World Testing Result is for the following Certification Criteria 

  • 45 CFR § 170.315(c)(2) Clinical quality measures (CQMs) – import and calculate
  • 45 CFR § 170.315(c)(3) Clinical quality measures (CQMs) – report
  • 45 CFR § 170.315(b)(10) Electronic Health Information (EHI) Export

Audience

  • Drummond Group LLC as an ONC Authorized Certification Body (ACB) and ONC Authorized Testing Lab (ACL).
  • The Office of the National Coordinator for Health Information Technology (ONC) and its ONC Health IT Certification Program for the purposes of Real World Testing and posting publicly on the ONC Certified Health IT Product List (CHPL). 

General Information

Summary of Testing Methods and Key Findings

The focus of Roji Health Intelligence’s Real World Testing measures was on the import of data from clients and the capacity to create QRDA documents from that data. Measure 1 tests Roji’s ability to successfully import QRDA documents from clients. Measure 2 tests Roji’s ability to generate QRDA III documents from the data clients have provided. Measure 3 tests Roji’s ability to generate single patient and patient population EHI files that are electronic and in computable format for a specific set of identified users.

As expected, Roji Health Intelligence was able to successfully import QRDA I documents and generate QRDA III documents in 100% of cases attempted. Additionally, Roji Health Intelligence was able to successfully create JSON files for single patient EHIs and patient population EHI in 100% of cases attempted. Our 100% success rate demonstrates that Roji is compliant with certification requirements, exchanges electronic health information (EHI) in the care and practice settings for its marketed use, and the EHI is received by and used in the certified health IT setting. 

RWT Measure 1

  • Description of the measurement/metric: Test a random subset of the patients per client that submits QRDA I documents.
    • The numerator will be the number of distinct patients that valid QRDA I documents successfully imported.
    • The denominator will be the number of distinct patients this was attempted for.
  • Associated certification criteria:
    • 45 CFR § 170.315(c)(2) Clinical quality measures (CQMs) – import and calculate
  • Justification for selected measurement/metric: Some clients can give us patient-level QRDA I documents that we normalize into our database. Other clients give us raw data that we normalize into our database. A successful import will allow us to perform calculations.
  • Care setting(s) that is addressed: A random subset should cover a variety of clinical variables.
  • Expected outcomes: We expect to be able to generate a QRDA I document for 100% of the cases.

RWT Results for Measure 1

Roji was able to successfully import and parse QRDA I documents for 100% of cases.

RWT Measure 2

  • Description of the measurement/metric: Test a subset of the practice & provider combinations per client that submits eCQMs.
    • The numerator will be the number of distinct client-practice-provider combos that a valid QRDA III document was generated.
    • The denominator will be the number of distinct client-practice-provider combos this was attempted for.
    • Report the particular CQMs applicable to the subset.
  • Associated certification criteria:
    • 45 CFR § 170.315(c)(2) Clinical quality measures (CQMs) – import and calculate
    • 45 CFR § 170.315(c)(3) Clinical quality measures (CQMs) – report
  • Justification for selected measurement/metric: We should be able to generate valid QRDA III documents regardless of the CQMs.
  • Care setting(s) that is addressed: Clients, practices, and providers only report on applicable CQMs.
  • Expected outcomes: We expect to be able to generate a QRDA III document for 100% of the cases.

RWT Results for Measure 2

Roji was able to generate QRDA III documents for 100% of cases.

RWT Measure 3

  • Description of measurement/metric: Test the measure that will assess functionality used to export EHI for a single patient and patient population.
    • Create an export file: Single patient data is exported to a JSON file. Patient population data is exported to a CSV, Excel, or table file. 
    • Execute at any time: Feature is available at all times
    • Limit ability of users who can create export: Only authorized users are allowed to export patient data
    • Electronic and computable format: Export file is in JSON format. The format for data can be viewed at: https://client.rojihealthintel.com/helpfiles/ehi-export/PatientExport.yml
  • Associated certification criteria:
    • § 170.315(b)(10) EHI export – Single patient EHI export
      • Justification: Exporting EHI associated with patients is a way to share information with external organizations. Export is typically used when users require a full patient record. This metric provides information on the type of data exported and the frequency of exports.
      • Test Methodology: System logs will be reviewed to determine the frequency of use. Log files are de-identified and used to determine proper functionality. This method is used to ensure compliance with this criteria
      • Expected Outcome: It is expected that authorized users will be able to share patient EHI using the export function. Successful and failed exports will be logged and tracked over time.
    • § 170.315(b)(10) EHI export – Patient population EHI export
      • Justification: Exporting health information associated with patient populations is another way to share health info to an external organization. Export of patient population is restricted to administrative and credential users. It is assumed that the data is used for research or quality purposes. This will provide a metric on the use of the export of EHI for patient population associated with Roji
      • Test Methodology: System logs will be reviewed to determine the frequency of use to ensure the export function is operating properly. Log files obtained are de-identified and used for analysis to validate proper operation. This method is used to ensure compliance with this criteria
      • Expected outcome: It is expected that authorized users will be able to share patient population data using the export function. Successful and failed exports will be logged and tracked over time.

RWT Results for Measure 3

Roji was able to generate JSON files for 100% of cases in a timely manner. Each export attempt was logged and tracked within our system.

Standards Updates (Including Standards Version Advancement Process (SVAP) and United States Core Data for Interoperability (USCDI))

uncheckedYes, I have products certified with voluntary SVAP or USCDI standards.

checkedNo, none of my products include these voluntary standards.

Update 1

  • Updated certification criteria and associated product:
    • Certification Criteria: 45 CFR § 170.315(c)(2) Clinical quality measures (CQMs) – import 
    • Product: Roji Clinical Data Registry
  • Health IT Module CHPL ID: 15.04.04.2951.Roji.16.00.0.170202
  • Method used for standard update: Update code in Roji QRDA Parser and CQM Engine.
  • Conformance measure: RWT Measure 2

Update 2

  • Updated certification criteria and associated product:
    • Certification Criteria: 45 CFR § 170.315(c)(3) Clinical quality measures (CQMs) – report
    • Product: Roji Clinical Data Registry
  • Health IT Module CHPL ID: 15.04.04.2951.Roji.16.00.0.170202
  • Method used for standard update: Update code in Roji QRDA Generator and CQM Engine.
  • Conformance measure: RWT Measure 2

Update 3

  • Standard (and version): All standards versions are those specified in USCDI v1
  • Updated certification criteria and associated product:
    • Certification Criteria: 45 CFR §170.315(b)(10) Electronic Health Information export
    • Product: Roji Clinical Data Registry
  • Health IT Module CHPL ID: 15.04.04.2951.Roji.16.00.0.170202
  • Method used for standard update: Update code in Roji Patient Export Generator
  • USCDI-updated criteria: The plan documents the support of all USCDI v1 data elements
  • Conformance measure: RWT Measure 3

Care Setting(s)

Measure 1, 2, and 3 in Roji Health Intelligence’s Real World Testing Plan are not specific to a care setting. The clinical settings of the data involved may include ambulatory, inpatient, outpatient, and emergency department.

Schedule of Key Milestones

Key Milestone Timeframe
Collect data for Roji RWT Measures 2024-06 to 2024-09
Conduct testing 2024-10 to 2024-12
Collation and finalization of results 2025-01

Attestation

These Real World Testing results are complete with all required elements, including measures that address all certification criteria and care settings. All information in this document is up to date and fully addresses the health IT developer’s Real World Testing requirements. 

  • Authorized Representative Name: Don Newman
  • Authorized Representative Email: [email protected]
  • Authorized Representative Phone: (312) 258-8004 x712
  • Authorized Representative Signature:
  • Date: Jan 9, 2025