nce the implementation of your project is complete, you are ready to review the results of your interventions. This module addresses the fifth step in the EBP process: evaluating the outcomes. Evaluation is the comparison of the data collected pre- and post- implementation. The comparison will tell you whether your project has had an impact on the measures that you were looking to change. 

Determining data and data collection

To evaluate the impact of your project, you need to have quantifiable measures that relate to the goals of your project and a plan of how you will obtain the data. In other words: you need to know what you are going to measure and determine how you are going to get the data.  

Let’s start with the first key component: what are you going to measure to determine the impact of your project? Recall what it was that you were attempting to impact when you first started your project. Your PICOT questions will outline this clearly. 

The second component is knowing where or how you can get the data. Start by ensuring that you have access to the data you need and how difficult it might be to collect the data. For example: 

• Is it in Epic or another database? 

• Who can help you retrieve that information? 

• Will you have to manually collect it and, if so, who can assist you in data collection? 

Then, consider the time it will take to collect the data if it is a manual process. The more manual the data collection, the more work it will take to collect. It may be worth switching to another similar measure that provides adequate measurement of your project.

Measuring data

When selecting the measures, ensure that they address what you intended them to measure and that they are related to your PICOT question. Perhaps there are existing measurement tools that were identified in the literature you reviewed that your project would benefit from. Using a pre-existing measurement tool, such as a validated survey, will make the evaluation easier.

Institute for Healthcare Improvement's Don Berwick unpacks measures in under 3 minutes.

Consider the ease of measuring the data. While it shouldn’t be a limiting factor in your project, it will help you determine the resources that you might need to document the project impact. Evaluation is made easier by having metrics that you can quantify and that are readily available. The most helpful metrics are numeric. Numerical data can be collected via Epic or other electronic data sources. If these are not available, you can always manually collect the data.

Also consider how accurate or reliable your data is. Electronic data that is automatically captured is usually the most reliable and consistent, though it may have some accuracy issues. Manual collection from staff can be accurate, but due to human factors during data collection, is not likely to be as reliable. You may have to take the manually collected data and transfer it into an electronic format for analysis, which takes time and could lead to data entry mistakes.

Evaluation: before you begin

Prior to evaluation, you’ll want to gather the following: 

1. First, you’ll need the data you used initially to determine if there was a need for improvement. This is also known as baseline data.

2. Next, you will need the data you collected during the implementation phase, the project data. You’ll use this data to compile a summary of how it compares to the baseline data. 

3. You should also reflect on whether you need additional data measures to make sure you are not having a negative impact on other areas that are not part of your project. These are known as balance measures. Here’s an example: Let’s say a team is trying to reduce the length of patient stay, but then, readmissions increase. A way to track this is to evaluate length of stay as well as readmission rates. Other examples for balance measures in this situation would be increased costs, decreased patient experience, or increased infections.

Developing an evaluation plan

Once you have arrived at the measure you want to use and determined how you are going to collect it, you can create an evaluation plan for your project. The following are suggestions to consider as you work through your project and your evaluation. Use these to add or expand to your plan to fit your needs.

• Baseline data: Be sure you collected baseline data prior to implementing your project. You’ll need to make a few decisions regarding how much data is enough and what data is relevant to your project. In other words: What is the time range for data retrieval? One quarter prior to implementation…one year? Consider how old the data is and if it is still relevant. Finally, be sure that you can collect the same type of data after implementing your project so that you can compare. 

• Displaying data: How do you plan to display the data so that it can be understood and operationalized by you and your team? Will a simple writeup be easy to understand or would graphs and tables be helpful? 

• Impact: Is the measure related to quality, cost, service or a balance measure? Which way do you want to see the measure moving, if at all?

• Frequency of data collection: How frequently do you plan on collecting the data after implementation? If you plan to continue to make adjustments to the implementation of the project, consider doing more frequent evaluations such as daily, weekly, or monthly. If you have an uncomplicated rollout of your project, you may decide to go from a monthly to quarterly review.

• Duration of data collection: How much post-implementation data do you need to evaluate the impact of your project? If you are looking for a long-term change in practice, you may consider monitoring for at least one year to reduce the likelihood of the process reverting back to the baseline state.

• Summary of goal: The final piece of an evaluation plan is to determine whether you have met the desired outcome of the project. What was the goal you had in the project? Did you meet it? You may need to continue to make changes to the process to see if you can meet the desired outcome. Also, how will you evaluate the process? Will your project be successful if you reach your goal? Do you need to do statistical analysis to see if there was a meaningful change?  

As you go through your plan, know that you don’t have to figure this all out on your own. Contact the Evidenced Basic Practice Committee for guidance or to ask about speaking with an EBP mentor.

Summary

Evaluation is the collection of the current practice’s baseline data compared to your new intervention. Evaluation will let you know whether you made an impact with your project and will provide you with additional evidence to continue to make practice changes in your area and around the hospital. To complete your evaluation step in the EBP process, follow the guidelines below:

1. Decide what you are going to measure and how you are going to measure it.  

2. Make sure you are measuring what you intended.

3. Collect data prior to the implementation of the project. If you don’t have baseline data or data prior to implementation, you will not know if your project had an impact.

4. Consider using measurement tools found in the literature you reviewed.

5. Determine how to communicate and display the data and the impact of your interventions.

6. Determine how often and for how long you will collect data. Be prepared to collect data for six months, and up to one year, after your implementation to ensure that the project has been integrated into the work process and that it has not returned to the status quo.

Now that you have made a practice change, how can you compile your work so that you can communicate it to others in the organization? Our final module will review the dissemination of your project outcomes.

Originally published April 2021.