Updated Visitor Policy as of December 29, 2023: Learn More  
×
Affiliated Organizations
  HN Physicians   School of Nursing   HNH Fitness   Villa Marie Claire   Simulation Learning   Haiti Health Promise
Medical Partners Offices
Cardiovascular Specialists Pulmonary Specialists Obstetrics & Gynecology North Jersey Heart North Jersey Surgical Surgical Specialistss Primary Care Specialty Assoc. Urologic Specialties Women's Health Care

Real World Testing

  201-833-3000    |      info@holyname.org

Real World Testing for 2023

Summary

Summary

This Real World Test (RWT) plan is intended to verify the adoption of WebHIS certified functionality.

The RWT plan will focus on certification criteria, represented as individual user stories for Ambulatory only care settings, Inpatient only care settings as well as user stories that are the same regardless of the care setting.

General Information

Developer Name: Holy Name Medical Center
Product Name:WebHIS
Version Number:1.2
Certified Health IT Edition:2015 Ed.
Product List (CHPL) ID:15.07.09.1632.WE01.01.02.1.230127
Withdrawn CHPL ID:15.07.07.1632.WE01.01.01.1.211116
Real World Testing Public URL:holyname.org/Policy/realworldtesting.aspx

Background

The following elements are addressed for each User Story (listed above). Inpatient is the care settings where:

  • Testing methodology:
    • demonstrate real-world interoperability and conformance to the criterion requirements
    • include scenario and use case-focused testing
  • Description:
    • of how the test is performed
    • of how conformance is demonstrated
  • Schedule:
    • of key Real World Testing milestones;
  • Expected Outcomes:
    • based on feature adoption in the current year
  • Measurement/ metric:
    • all measures used to validate criteria
  • Justification for the Health IT Developer's Real World Testing approach
    • description of how the measurements/metrics selected reflect the adoption rate of each required Real World Testing element

Introduction

The EHR analyzed in this Real World Test is WebHIS EHR designed to present medical information to healthcare providers in Ambulatory and Inpatient healthcare settings. The workflows in WebHIS help users with Transitions of Care, Electronic prescribing, public health initiatives and patient engagement.

The purpose of this testing is to validate the adoption of the current user interface and EHR capabilities and to provide evidence of usability within WebHIS. To this end, measures of real world utilization of interoperability features and functionality are captured during the testing.

Table of Content

Table of Content

  • § 170.315(b)(1) Transitions of care
  • § 170.315(b)(2) Clinical information reconciliation and incorporation
  • § 170.315(b)(6) Data export
Key Milestones Summary

Key Milestones Summary

Criteria Care Setting Measurement Period Date Date Key Milestones
§ 170.315(b)(1) Transitions of care - Send Inpatient 6/1/2023 - 6/30/2023 June, 2023
  • Confirm Ability to Send Clinical Documents to Trading Partner Brightree.
  • Confirm with Brightree that production data will be used, whether in an actual live environment or a copy of a live environment.
  • User confirms third party received documents.
June, 2023
  • TOC CCD generated and sent to Brightree by direct protocol.
  • User confirms successful delivery.
§ 170.315(b)(1) Transitions of care - Receive
§ 170.315(b)(2) Clinical information reconciliation and incorporation
Inpatient 6/1/2023 - 6/30/2023 June, 2023
  • HIM User navigates to Medical Records > CCDA Inbox.
  • User chooses the clinical document received from Trading Partner for a Patient.
  • CCD is validated to be conformant. Any errors found are available for review.
  • User confirms that the document has been matched and incorporated to the correct patient.
June, 2023
  • User determines if the patient's problems, medications, and medication allergies will be reconciled into WebHIS with no duplicates.
June, 2023
  • Calculate and compile metrics.
§ 170.315(b)(6) Data export Inpatient 6/1/2023 - 6/30/2023 June, 2023
  • Date and time ranges can be configurable via the UI.
  • Export Locations can be configurable via the UI.
  • Patients exported can be configurable via the UI.
June, 2023
  • Use the Edge Test Tool to check validity of output file.
June, 2023
  • Export summary was created and completed successfully.
June, 2023
  • Calculate and compile metrics.
170.315 (b)(1) (b)(2) Plan

170.315 (b)(1) (b)(2) Plan

Measure Description:

Send and receive Transition of Care (TOC) messages with other providers to close the referral loop. The patient's ePHI will be exchanged using a C-CDA 2.1 Care Referral or Referral Note and DIRECT secure messaging for data transport. Patient data from incoming TOCs will be reconciled with existing data in the EHR including, at minimum, the patient's problems, medications, and medication allergies.

Justification:

We chose to concentrate on the aspects of this criterion that would:

  1. showcase WebHIS's streamlined approach to provider-to-provider patient referrals and transitions of care with the ultimate goal being higher quality patient care.
  2. eliminate as much risk of data entry errors as possible by transmitting patient data securely and electronically rather than relying on manual data entry for referrals.
  3. reduce the overall time burden of manual data entry.
  4. ensure private and secure transmission of patients' PHI.
  5. result in increased interoperability between disparate HIT systems.

Metric Description:

  1. Over 80 percent of outbound TOC's successfully received by Third Party.
  2. Average C-CDA grade from scorecard for C-CDAs generated from WebHIS is a "C" or better.
  3. Over 75 percent of trading partner's TOC C-CDAs successfully received by WebHIS.
  4. Over 90 percent of patients' problems, medications, and medication allergies from incoming TOCs are reconciled into the EHR.

Standards Implemented: (SVAP)

N/A

Care Setting:

Inpatient

Developer Info:

Holy Name Medical Center
718 Teaneck Road
Teaneck, NJ 07666
(201) 833-3000

Product Info:

Product Name: WebHIS

Product Version: 1.2

CHPL ID

15.07.07.1632.WE01.01.01.1.211116

Methods Use to Demonstrate Interoperability:

HISP via Direct Protocol (SMTP)

Relied Upon Software:

DataMotion (Version 6.4)


Test Step Testing Procedure Expected Outcomes Key Milestone Date
1 Coordinate with Brightree for sending/receiving clinical documents using production data as described in this RWT plan.
  • Confirm Ability to Send Clinical Documents to Trading Partner Brightree.
  • Confirm with Brightree that production data will be used, whether in an actual live environment or a copy of a live environment.
  • User confirms third party received documents.
June, 2023
2 Patient has inpatient admission and discharge and data is captured in WebHIS.
  • CCDS data elements captured in EHR (system under test).
  • Care provider is able to create a C-CDA Release 2.1 Discharge Summary Document that also includes the discharge instructions.
3 User enters transfer disposition.
  • TOC CCD generated and sent to Brightree by direct protocol.
  • User confirms successful delivery.
June, 2023
* Next steps take place at trading partner.
4 Trading partner refers Patient B from TP EHR to WebHIS by generating C-CDA Clinical Document or Referral Note.
  • TP care provider selects WebHIS from directory of Direct addresses and initiates sending of Clinical Document.
5 In WebHIS, tester acknowledges receipt of valid Clinical Document.
  • HIM User navigates to Medical Records > CCDA Inbox.
  • User chooses the clinical document received from Trading Partner for a Patient.
  • CCD is validated to be conformant. Any errors found are available for review.
  • User confirms that the document has been matched and incorporated to the correct patient.
June, 2023
6 In WebHIS, the incoming data is incorporated via reconciliation into the Patient's existing medical record.
  • User determines if the patient's problems, medications, and medication allergies will be reconciled into WebHIS with no duplicates.
June, 2023
7 Calculate and compile metrics June, 2023

* Attestation

170.315 (b)(1) (b)(2) Results

170.315 (b)(1) (b)(2) Results

Measure Description:

Send and receive Transition of Care (TOC) messages with other providers to close the referral loop. The patient's ePHI will be exchanged using a C-CDA 2.1 Care Referral or Referral Note and DIRECT secure messaging for data transport. Patient data from incoming TOCs will be reconciled with existing data in the EHR including, at minimum, the patient's problems, medications, and medication allergies.

Justification:

We chose to concentrate on the aspects of this criterion that would:

  1. showcase WebHIS's streamlined approach to provider-to-provider patient referrals and transitions of care with the ultimate goal being higher quality patient care.
  2. eliminate as much risk of data entry errors as possible by transmitting patient data securely and electronically rather than relying on manual data entry for referrals.
  3. reduce the overall time burden of manual data entry.
  4. ensure private and secure transmission of patients' PHI.
  5. result in increased interoperability between disparate HIT systems.

Metric Description:

  1. Over 80 percent of outbound TOC's successfully received by Third Party.
  2. Average C-CDA grade from scorecard for C-CDAs generated from WebHIS is a "C" or better.
  3. Over 75 percent of trading partner's TOC C-CDAs successfully received by WebHIS.
  4. Over 90 percent of patients' problems, medications, and medication allergies from incoming TOCs are reconciled into the EHR.

Standards Implemented: (SVAP)

N/A

Care Setting:

Inpatient

Real World Test Results - 2023:

  • Measurement period moved to 09/1/23 - 09/30/23. All expected outcomes achieved.
  • WebHIS configured to generate and send a TOC CCD document for all inpatient discharges that were transitioned to trading partner.
  • WebHIS generated and sent 81 TOC CCDs to Brightree, Brightree received 81 CCDs.
  • A random selection (10) CCDs were tested using the Healthit.gov C-CDA Scorecard 2.0 tool, each scored a "C".
  • WebHIS configured to process all expected problems, meds and allergies were successfully reconciled.

Developer Info:

Holy Name Medical Center
718 Teaneck Road
Teaneck, NJ 07666
(201) 833-3000

Product Info:

Product Name: WebHIS

Product Version: 1.2

Product List (CHPL) ID:

15.07.09.1632.WE01.01.02.1.230127

Withdrawn CHPL ID:

15.07.07.1632.WE01.01.01.1.211116

Methods Use to Demonstrate Interoperability:

HISP via Direct Protocol (SMTP)

Relied Upon Software:

DataMotion (Version 6.4)


Test Step Testing Procedure Expected Outcomes Key Milestone Date
1 Coordinate with Brightree for sending/receiving clinical documents using production data as described in this RWT plan.
  • Confirm Ability to Send Clinical Documents to Trading Partner Brightree.
  • Confirm with Brightree that production data will be used, whether in an actual live environment or a copy of a live environment.
  • User confirms third party received documents.
September, 2023
2 Patient has inpatient admission and discharge and data is captured in WebHIS.
  • CCDS data elements captured in EHR (system under test).
  • Care provider is able to create a C-CDA Release 2.1 Discharge Summary Document that also includes the discharge instructions.
3 User enters transfer disposition.
  • TOC CCD generated and sent to Brightree by direct protocol.
  • User confirms successful delivery.
September, 2023
* Next steps take place at trading partner.
4 Trading partner refers Patient B from TP EHR to WebHIS by generating C-CDA Clinical Document or Referral Note.
  • TP care provider selects WebHIS from directory of Direct addresses and initiates sending of Clinical Document.
5 In WebHIS, tester acknowledges receipt of valid Clinical Document.
  • HIM User navigates to Medical Records > CCDA Inbox.
  • User chooses the clinical document received from Trading Partner for a Patient.
  • CCD is validated to be conformant. Any errors found are available for review.
  • User confirms that the document has been matched and incorporated to the correct patient.
September, 2023
6 In WebHIS, the incoming data is incorporated via reconciliation into the Patient's existing medical record.
  • User determines if the patient's problems, medications, and medication allergies will be reconciled into WebHIS with no duplicates.
September, 2023
7 Calculate and compile metrics November, 2023

* Attestation

170.315 (b)(6) Plan

170.315 (b)(6) Plan

Measure Description:

Export all available data elements from the Common Clinical Dataset (CCDS) for a population of patients for use in a different health information technology product or a third party system. This export can be used for many purposes, including data portability when a physician practice switches to a new EHR platform.

Justification:

We chose to concentrate on the aspects of this criterion that would:

  1. demonstrate WebHIS's ability to export batches of patient data in a straightforward fashion.
  2. facilitate interoperability by providing the exported data in the form of valid CCD files that conform to the HL7 standards as described in the HL7 Implementation Guide for CDA® Release 2: Consolidated CDA Templates for Clinical Notes (US Realm).

Metric Description:

  1. Over 90 Percent of Exports ran at the correct time.
  2. C-CDA count matches actual patient count for requested date range.
  3. Spot-check C-CDAs for conformance and errors.

Standards Implemented: (SVAP)

N/A

Care Setting:

Inpatient

Developer Info:

Holy Name Medical Center
718 Teaneck Road
Teaneck, NJ 07666
(201) 833-3000

Product Info:

Product Name: WebHIS

Product Version: 1.2

CHPL ID

15.07.07.1632.WE01.01.01.1.211116

Methods Use to Demonstrate Interoperability:

  1. Ensure endpoint receives correct number of CCDs

Test Step Testing Procedure Expected Outcomes Key Milestone Date
1 Using production data in an actual live environment, demonstrate the ability to configure data export configurations for Timeframe and Location.
  • Date and time ranges can be configurable via the UI.
  • Export Locations can be configurable via the UI.
  • Patients exported can be configurable via the UI.
June, 2023
2 User with appropriate permissions accesses the CCDA Exporter Module.
  • Logging in as an authorized user will allow access to the CCDA Exporter Module.
3 Create and validate an export for a single patient.
  • Use the Edge Test Tool to check validity of output file.
June, 2023
4 Create an export summary for discharges within a entered date and time range.
  • Data was available for the entered date and time range.
  • The export summary contained data for discharges only within that date and time range.
5 Create an export summary in real time.
  • Export summary was created and completed successfully.
June, 2023
6 Create an export summary based on a relative date and time (Ex. Every first of every month @ 7 AM).
  • The scheduled export summary would be display and be visually validated.
7 Create an export summary for a specific date/time (Ex. 07/16/2021 @ 3:30 PM).
  • The scheduled export summary was created successfully.
  • The specific date/time would be in the near future so the export could be confirmed.
8 Save the export summary to a preferred location at the time of export.
  • Saving to a preferred location is allowed.
  • Visually confirming the export after save is performed and successful.
9 Calculate and compile metrics. June, 2023
170.315 (b)(6) Results

170.315 (b)(6) Results

Measure Description:

Export all available data elements from the Common Clinical Dataset (CCDS) for a population of patients for use in a different health information technology product or a third party system. This export can be used for many purposes, including data portability when a physician practice switches to a new EHR platform.

Justification:

We chose to concentrate on the aspects of this criterion that would:

  1. Demonstrate WebHIS's ability to export batches of patient data in a straightforward fashion.
  2. Facilitate interoperability by providing the exported data in the form of valid CCD files that conform to the HL7 standards as described in the HL7 Implementation Guide for CDA® Release 2: Consolidated CDA Templates for Clinical Notes (US Realm).

Metric Description:

  1. Over 90 Percent of Exports ran at the correct time.
  2. C-CDA count matches actual patient count for requested date range.
  3. Spot-check C-CDAs for conformance and errors.

Standards Implemented: (SVAP)

N/A

Care Setting:

Inpatient

Real World Test Results - 2023:

  • All options are configurable via the UI as expected.
  • Output file confirmed valid by Edge Test Tool. The scored were "C".
  • Export summary confirmed.
  • 99% of exports ran at the configured time.
  • All patients were requested for export, and C-CDAs were generated for all.
  • Tested C-CDAs were found conformant.

Developer Info:

Holy Name Medical Center
718 Teaneck Road
Teaneck, NJ 07666
(201) 833-3000

Product Info:

Product Name: WebHIS

Product Version: 1.2

Product List (CHPL) ID:

15.07.09.1632.WE01.01.02.1.230127

Withdrawn CHPL ID:

15.07.07.1632.WE01.01.01.1.211116

Methods Use to Demonstrate Interoperability:

  1. Ensure endpoint receives correct number of CCDs

Test Step Testing Procedure Expected Outcomes Key Milestone Date
1 Using production data in an actual live environment, demonstrate the ability to configure data export configurations for Timeframe and Location.
  • Date and time ranges can be configurable via the UI.
  • Export Locations can be configurable via the UI.
  • Patients exported can be configurable via the UI.
October, 2023
2 User with appropriate permissions accesses the CCDA Exporter Module.
  • Logging in as an authorized user will allow access to the CCDA Exporter Module.
3 Create and validate an export for a single patient.
  • Use the Edge Test Tool to check validity of output file.
October, 2023
4 Create an export summary for discharges within a entered date and time range.
  • Data was available for the entered date and time range.
  • The export summary contained data for discharges only within that date and time range.
5 Create an export summary in real time.
  • Export summary was created and completed successfully.
October, 2023
6 Create an export summary based on a relative date and time (Ex. Every first of every month @ 7 AM).
  • The scheduled export summary would be display and be visually validated.
7 Create an export summary for a specific date/time (Ex. 07/16/2021 @ 3:30 PM).
  • The scheduled export summary was created successfully.
  • The specific date/time would be in the near future so the export could be confirmed.
8 Save the export summary to a preferred location at the time of export.
  • Saving to a preferred location is allowed.
  • Visually confirming the export after save is performed and successful.
9 Calculate and compile metrics. November, 2023

* Attestation:

This Real World Testing plan is complete with all required elements, including measures that address all certification criteria and care settings.
All information in this plan is up to date and fully addresses the Health IT Developer's Real World Testing requirements.

Real World Testing URL: https://www.holyname.org/Policy/realworldtesting.aspx
Authorized Representative Name:Eduardo Morales
Authorized Representative Email:emorales@holyname.org
Authorized Representative Phone:(201) 833-3160
Date:1/16/2024