The goal of the Indiana IAIMS Initiative (I3) is to create an integrated, Indianapolis MSA (Metropolitan Statistical Area) wide clinical data and knowledge information management system that serves the needs of patient care, public health, education, policy generation and research.  Patients, health care providers, public health officials, educators, policy makers and researchers will use the information management system.

The key question then is how well we serve the needs of patient care, public health, education, policy generation and research  Based on needs assessments we have already completed, we know that more complete outpatient prescription data, additional allergy data and physician encounter data are the most important deficiencies in our existing data both for patient care and public health. We know that the system provides usable information for public health, education, policy generation and research since it is already used extensively for these purposes.

For each of the four projects, we describe the baseline data acquisition, if appropriate, and the assessment of change.

Context Sensitive Information/Access to Knowledge Resources

Our most important measure of success is how heavily clinicians use the system.  We log all system access now and will continue to do so. These access logs provide detailed information about which users are accessing the system from where and which system components they are using.  These logs provide detailed baseline data about access to resources such as InfoButtons and Thomson DiseaseDex (formerly PraxisMD). Baseline data for this activity will be garnered from use of e-resources by the individual health care provider.  This is frequently available via the user's unique ID.  If this information is not available then aggregate data will be used.  A question will also be included on the baseline survey to determine the relative weight of this information from the perspective of the health care providers.  Counts of information access will be made and validated against the results from the qualitative evaluation that will take place during Years 3 and 4.

Enhanced Results Reporting

The collection of baseline data beyond an indication of its desirabililty through the baseline survey is impossible because no results reporting enhancements exist at this time.  If the key stakeholders determine that these enhancements are desirable, this group will test the enhancements as part of a pilot test.  The increase in system use as well as the actual use of the enhancements will be measured electronically.  At the end of the pilot period, the test group will be surveyed to determine the relative value of the enhancements.  We will also look at use of the system following the implementation of the enhancements compared to the use of the system prior to the enhancement availability.

Public Health Alerts

A question will be included on the baseline survey to determine the relative weight of this information from the perspective of the health care providers.  If a public health event, either an emerging infectious disease or a bioterrorist attack, takes place, then use of the web-based digital archive will be monitored and during the qualitative evaluation taking place in years three and four, the efficacy of this activity will be discussed.  We will also interview the respective health departments to determine if there is a sense that this activity has lessened their work load.

Patient Portal/Personal Health Information

To date, we don't have any experience with personal health records and we will examine patient interactions with the system in a pilot mode.  Upon recommendations from participating providers, a group of patients will be identified and invited to participate in a needs assessment to determine the desirability of a Patient Portal to personalized health information.  A Likert-type survey instrument will be used.  Following analysis of the questionnaires, the system will be implemented with as much customizability as appropriate.  The patients participating in the needs assessment will be invited to participate in the pilot test of the system.  A similar survey will be used to determine acceptability by patients participating in the pilot test.  Participating physicians will be queried to determine the impact on provider-patient communication of patient access to personal health records. In addition, we will monitor the level of usage.

As described, we will measure and track operational activities with numbers of edits, amount of data added, interface "uptime" and similar measures.

Enhance Global Doctor File   

We current know the number of physicians in the Global Doctor File.  Two process will take place; the updating of records and the additions and deletions.  These activities will be counted separately and tracked annually.

Expand Outpatient Pharmacy Data

Baseline data regarding the feelings of the provides about the importance of this activity will be included on the baseline questionnaire administered to all new participants, both health care providers and pharmacists.  We will count the numbers of e-prescriptions generated to show increase in use of the utility over time; a stratified random sample of providers and pharmacists will be interviewed to determine their level of satisfaction with this component and this data will be compared to the aggregate level of importance of this utility by the two different groups; researchers will be asked to participate in the development of the research-centric system and will be expected to provide feedback on usefulness and usability.

Data from Provider Practices

Assessment of change following the implementation of this component will be handled in two ways, quantitatively and qualitatively.  Prior to the implementation of the system in the individual provider's offices, an pre-test/post-test instrument in the form of a desired components Likert-type questionnaire will be administered to enable the measurement of success factors from the provider's perspective.  As mentioned above, the system has the ability to count the numbers of accesses at the attribute level by each provider.  This will generate data that can be tracked over time to determine whether use of the system increases, decreases, or remains the same.  The administration of the post-test, and comparison of the results to a randomized sample of participants, should validate whether meeting the specific needs of the providers results in system use.  The post-test will be used to determine changes in expectations.  This activity will also be discussed during the focus group sessions with specific emphasis being paid to the impact of the system attributes on improving health care quality.

Development of Funding Model

System costs will provide the baseline data on which the funding model will be developed.  Evaluation of the workability of the funding model will be determined through participant feedback as well as willingness to continue paying for the system features.

Promotion and Training

Evaluation of this activity will focus on the educational outcomes.  Participants will be given a pre-test on content prior to the educational activity and a post-test following the content delivery.  This is true for either the web-based tutorials or the face-to-face didactic lecture or experiential learning activity.  The perceptions about the quality of the class and its applicability to the learning needs of the participants will also be assessed.

Summative Evaluation of the IAIMS project

This will begin during year three and address a number of global questions including, but not limited to the following:

• What are the critical success factors in implementing an IAIMS initiative across unrelated or marginally related health care institutions?
• Is there an optimum organizational structure to facilitate cross institutional health care improvement through access to information?
• What is the minimum amount of participation [both percentage of possible participants and level of participation] to insure perceived or real quality improvement?

There will be a number of specific questions that arise during the formative evaluation stage that cannot be anticipated and these, too, will be addressed.  As part of the evaluation aim, there is the specific goal of looking at generalizability and change processes on internal organizational activities that have been impacted by the IAIMS construct.