Better Decisions Leveraging Existing Data: LMI’s Approach to Evidence Act Evaluations

The “Evidence Act” (Evidence-Based Policymaking Act of 2018) calls for federal agencies to use the best available evidence for decision-making. Every Chief Financial Officers Act (CFO Act) agency is required to assign an evaluation officer, statistical officials, and a chief data officer (CDO) to ensure they collect and leverage the best available data, including financial and budget information, while constantly seeking to improve mission execution.

That’s great, but what does this mean? 

Every year, federal agencies evaluate how they use their resources. This information helps them understand how they execute their mission initiatives and how to improve. 

In 2021, LMI established and implemented tools and approaches to assist evaluations officers in scoring and selecting evaluation topics, creating annual evaluation plans, and executing discrete evaluations. LMI’s full lifecycle evaluations capability applies repeatable processes across four evaluation stages:

1. Collecting, vetting, and adjudicating topics: Before kicking off any evaluation, LMI Evidence Act teams use standardized selection criteria, scoring methods, and topic value assessments to rate a proposed topic’s maturity and whether an evaluation will yield results for decision-makers.
2. Creating an evaluation logic model: Our evaluation logic model comprises the essential elements that need to be either proven or disproven to successfully complete the evaluation. To establish the logic model, our teams perform a deep dive to understand the resources and activities relating to how the chosen evaluation topic contributes to our client’s mission. By mapping the relationships between these elements and outcomes, we can demonstrate the topic’s effectiveness and areas for improvement. 
3. Developing evaluation planning: Our teams uncover available data, what still needs to be collected and analyzed, and the stakeholders and operators to interview. A key part of this stage is developing sampling strategies to set how much data and how many interviews are required for statistically significant findings. 
4. Executing evaluations, detailing effective practices, and reporting findings: We progressively interview stakeholders, collect, and analyze data, and compile results and findings. Our teams meet regularly to discuss and review preliminary findings, request follow-up data, and seek additional feedback from interview subjects. We close this stage with our final evaluation report, containing findings, recommendations.  Our evaluation reports include appendices that encompass collected data, how sampling sizes were determined, interview notes, and any other artifacts that contributed to the evaluation to be fully transparent as to how we came to our findings and observations.

Throughout each stage, our teams constantly focus on developing valuable evaluation results to assist decision-makers justify expanding or closing programs, acquiring new or additional resources, or adjusting agency data sets.

In 2021, LMI established and implemented tools and approaches to assist evaluations officers in scoring and selecting evaluation topics, creating annual evaluation plans, and executing discrete evaluations.

Have a question or concern related to your organization’s Evidence Act evaluations capability? Contact LMI’s financial operations and accountability practice area lead, Brad Hallman.