Challenges in Testing of Health Information Exchange (HIE) Solution

Background

In July 2004 the United States Department of Health and Human Services released their vision of how America’s healthcare system could be rebuilt during the next decade. That vision is in the process of maturing and the lead in this effort is the Office of the National Coordinator for Health Information Technology. The position of National Coordinator was created in 2004, through an Executive Order, and legislatively mandated in the Health Information Technology for Economic and Clinical Health Act (HITECH Act) of 2009. With the passage of the HITECH Act, the Office of the National Coordinator for Health Information Technology (ONC) is charged with building an interoperable, private and secure nationwide health information system and supporting the widespread, meaningful use of health IT.

One of the missions of ONC is to establish governance for National Health Information Network (NHIN).

Health information exchange (HIE) is the transmission of healthcare-related data among facilities, health information organizations (HIO) and government agencies according to national standards. HIE is an integral component of the health information technology (HIT) infrastructure under development in the United States and the associated National Health Information Network (NHIN). To meet requirements, HIE technology must enable reliable and secure transfer of data among diverse systems and also facilitate access and retrieval data. The purpose of HIE development is to improve healthcare delivery and information gathering.

AFour Technologies has been providing functional, non-functional as well as compliance related testing services to HIE solution vendors for past few years. Each of those engagements presented challenges unique to HIE solutions. Challenges of testing HIE solutions are a superset of those faced in design, engineering of HIE solution and implementation of HIE solutions for the end customers.

This document describes few common challenges that HIE solution vendors face during testing of HIE solutions and covers solutions that the AFour team has devised in order to combat those challenges.

Baselining and Monitoring Performance of the HIE Systems

Typical design of HIE solution is based on integration of different subsystems with different architectures (e.g. HL7 outbound interface, HL7 inbound interface, Document Registry and Repository, Enterprise Master Patient Index (EMPI), Integration with external Public Health Record (PHR) systems, Services of Continuity of Care Document (CCD) generation, Secure Messaging, IHE actors). HIE solution vendors can also choose to assemble components from third-party vendors to form the HIE solution. In a typical HIE solution, component and system testing activities consume a significant portion of the software development life cycle (SDLC), leaving inadequate cycles for QA to do proper justice to performance requirements of the HIE system.

AFour QA team has done extensive performance testing of end to end HIE system as well as specific system components for the base data of 1 Million patient records and 20 Million documents. AFour has developed custom tools to generate rich patient demographic and clinical information and has developed many applications to verify correctness of data at various layers of HIE system. AFour has come up with a refined performance testing cycle for HIE systems. This includes end to end system testing scenarios, component specific stress tests and rich set of business metrics and platform metrics. AFour HIE QA team is capable of providing recommendations for common pain areas, bottlenecks related to HIE performance.

Non-UI Interfaces

Main purpose of HIE system is to exchange information across EHRs, PHRs. Most of the interfaces (e.g. HL7 outbound interface, CCD inbound interface, IHE actor interfaces (XDS.b, PIX, PDQ, XCA, XCPD, etc.)) are non UI interfaces. HIE system can be imagined as an application server which provides information exchange services to applications like EHRs, PHRs, etc. Due to such high portion of non UI interfaces, testing of HIE system needs development of EHR, PHR simulators. These simulators send and receive the information through HIE interfaces (e.g. HL7 custom socket interface or IHE web service interface).

Backward compatibility and interoperability of HIE interfaces is very important and hence locking and control of API and testing of those APIs carries a lot of weight in HIE testing. AFour team has developed many custom tools and has also built expertise in using MESA tools for testing Non UI interfaces of HIE solution.

Asynchronous and Synchronous Transactions

HIE systems have synchronous as well as asynchronous interfaces. Due to the heavy weight routing and multilayer processing of information by HIE for the turnover of a large number of messages , most of the HIE transactions are designed to be asynchronous in nature. HIE systems also have synchronous transactions. Examples of synchronous transactions are PDQ queries, PIX queries. One common example of asynchronous transaction is generation of Continuity of Care Document (CCD) based on ADT message trigger. Due to mix of asynchronous as well as synchronous transactions of HIE, one cannot use “off the shelf” tools for their testing. One needs to design and develop custom tools that can verify for both types of transactions.

AFour has designed test tools considering the requirements of test data generation and verification of such mixed type of transactions.

Compatibility with Different HL7 Versions

HL7 organization releases many publications for demographic and clinical information standards. HL7 V 2.3.1, HL7 V 2.5.1, HL7 V3, CDA 2.0, CCOW 1.6 are few key standards that are used by the healthcare industry (there are several other versions that HL7 has released). HL7 organization expects backward compatibility across old standards. Hence HIE vendor must ensure that the HIE system supports HL7 documents from different HL7 releases. Best way to test the interfaces for HL7 standards compliance is to use IHE compliance tools called MESA test tools. MESA test tools are developed by Mallinckrodt Institute of Radiology (MIR). These tools are also used for verification and validation of PreConnectathon tests.

Commercial tools like 7Edit can be used for verification of demographic or clinical messages against schemas of different HL7 versions. AFour team can further recommend correct set of tools or customize AFour tools for compatibility testing activities of HIE solution

Complex Standards for Implementation of Interoperability Compliance

Actors defined by IHE standards (also referred to as profiles – e.g. Document Registry, Document Repository, PIX Manger, PDQ Supplier, etc.) are typically implemented as web services. These web services use very rich metadata and usually contain deep hierarchical information (such as event codes, event code groups, etc.) In order to ensure correct implementation of such complex actors, IHE recommends use of MESA software tools.

AFour had to develop a sound understanding and expertise in the complex configuration of these tools and interpretation of the results.

Revision of Healthcare Standards

Certification Commission for Health Information Technology (CCHIT) certifications, IHE profiles (such as PIX, PDQ, XDS.b, ATNA, XCA, XCPD, etc.) undergo periodic revisions. It is important for the test team to be on top of the latest developments on this front and notify the same to the rest of the engineering team.

AFour has dedicated healthcare domain experts who keep themselves up to date with the latest industry standards by participating in various forums, groups, mailing lists and literature reference.

Testing for Data Loss

A software solution fails certification from the quality engineering team even when a single data loss issue is observed during testing. Information comes to HIE outbound interface in the form of HL7 messages. These are a mix of demographic (ADT) and clinical (MDM, PPR, ORU, ORM, CCD) messages.

Each message flows through multiple layers of HIE system (e.g. receiving interface layer, business layer, data layer). This message is processed or routed to different entities before it finally gets associated with a patient / person record.

Any exception at any layer (e.g. message validation failure according to HL7 standards, unavailability of processing web service, etc.) should be reported back to the message originating entity (EHR, PHR, etc.) along with the rejection of return of the underlying clinical message. Not doing so results in the loss of message (the piece of clinical or demographic information) and defeats the main objective of HIE solution for integrity of clinical information across EHRs.

AFour healthcare test team has designed several negative and boundary test scenarios to simulate such situations in the HIE system in order to identify such issues and get them fixed proactively.

Security and Privacy of Health Information (HIPAA Compliance)

Two HIPAA rules are dedicated to security and privacy of individual identifiable health information. HIE vendors typically comply with these requirements by robust implementation of security and privacy attributes in the solution.

Few features implemented by an HIE solution to comply with HIPAA security and privacy rules are: Information must not be shared without patient consent for using health information; allowing breaking of glass in case of emergencies; 128 bit encryption of information while in transit; handshake with strong authentication across two nodes of HIE system before sending or receiving information; no caching of information on end user machines / browsers; Presenting latest information to the end users; implementation of detailed access control list. In the end HIE should ensure that correct information should be shared with the correct users with correct privileges to perform correct actions.

AFour team had to come up with a comprehensive set of test cases that are focused on HIPAA security and privacy requirements.

EMPI Testing

EMPI (Enterprise Master Patient Index) is key for correct functionality of HIE system. This component is responsible of grouping of same patient records with different MRN numbers from different EHRs. Such grouping by EMPI enables HIE system to identify and collect clinical data of patient from different EHRs and use it for HIE activities like formation of clinical summary; sending documents against XDS.b calls; etc.

If desired patients are not grouped correctly the end users might see data loss for their system calls. If wrong patients are grouped it will create functional and legal havoc for HIE solution vendor. Other than providing basic service of grouping of patient from different EHRs, EMPI should also identify duplicate patient records from the same source and report to the administrators.

AFour team has designed and developed rich patient test data and truth tables to validate grouping scenarios of EMPI.