Optimal evaluations:
- Define measurable, validated outcomes across clinical, social, and operational domains
- Collect quantitative and qualitative data in real time
- Center patient and partner experiences and equity in data collection and interpretation
- Use results to inform iterative changes and program improvement
- Align findings with organizational mission, quality improvement, and population health strategies
There are many ways to integrate evaluation and learning for FIM programs within a healthcare context. Best and promising practices for integrating evaluation and learning into healthcare-based FIM programs come from implementation science, public health evaluation frameworks, and health system transformation work. These include CMS Innovation Center pilots, the National FIM Task Force, NIH HEAL, and CDC initiatives. The evaluation approach should be aligned to the program’s maturity, budget, and the decisions that the findings will inform.
In designing FIM program evaluations, programs should consider:
What are the goals or expected targets of the program?
Health systems should select priority measures that are feasible to collect and support assessment program goals.
What does the organization have the resources and abilities to collect?
Assess the operational bandwidth and technical infrastructure of both clinical staff and community partners to ensure that data collection requirements are realistic, sustainable, and do not impede core service delivery.
What specific process and outcomes measures do you want to understand?
Consider the measures that have identified as best practice (e.g., food security, nutrition security, and program engagement) while also measuring metrics aligned to the specific program goals.
Pre/Post Studies
Overview
-
Look at changes in participant outcomes before starting a FIM program and after receiving services.
-
No comparison groups.
Pros
-
Provides a clear, direct measurement of change with the participant group.
-
Provides insights within a faster, lower cost generation of findings.
-
Provides insights on patient experience and operational value to support planning and scaling.
Cons
-
Easiest to implement but can be biased as changes in outcomes may occur for reasons outside the program.
-
Studies will add little value to the existing FIM research base but may be useful for early program monitoring and preliminary data collection.
Quasi-Experimental Studies
Overview
-
Compare changes in outcomes between program participants and a non-randomized comparison group of similar patients who did receive FIM services.
Pros
-
Offer stronger evidence and statistical techniques, accounting for differences between groups, to isolate the effect FIM.
-
Provide more generalizable evidence, with findings relevant to larger populations.
-
Support large-scale policy analyses.
Cons
-
Limited value if there are variables impacting the treatment and comparison groups differently that are unmeasured.
-
Unmeasured variables cannot be accounted for and may incorrectly estimate the true program impact.
Interviews and Focus Groups
Overview
-
Generates qualitative data through individual or group discussions with individuals with experience in FIM.
-
Complement quantitative studies that assess changes in measurable, quantitative outcomes by learning about direct FIM experiences.
Pros
-
Valuable for new programs focused on improving the patient experience.
-
Qualitative data can contextualize findings from quantitative analyses.
-
Provide insight into why changes in health outcomes did/did not occur.
Cons
-
Small, non-representative samples of the broader enrolled population that may not be representative.
-
High possibility of bias in the information shared.
-
Resource intensive to identify, engage, and quantify insights.
Randomized Trials
Overview
-
Randomly allocate receipt of FIM services, with treatment and comparison groups being similar on both measured and unmeasured variables.
Pros
-
Provide the strongest evidence and the gold standard in medical research.
-
Often required to create widely adapted standards of clinical care.
Cons
-
Expensive and challenging to implement in regular course of care.
-
Focus on specific patient populations, making findings less generalizable.
-
Requires academic partners and funding.