Utility

Utility, at its core, deals with whether or not the data that practitioners collect from an assessment, tool, or survey will be useful to make reasonable decisions about the activity, intervention, strategy, or program being implemented. Data about children’s learning and development needs to have a practical goal that allows practitioners and policy makers to learn about how their program affects children and what changes they can make to programming to improve children and adolescents’ development. In this module, practitioners will learn how to balance the time, resources, and commitment needed to rigorously measure learning and development outcomes, and use the data they collect.

Additional Resources

Measurement budget template

Nikhit D'Sa

This template guides teams through the different budget line items they should consider when attempting to collect data about learning or development outcomes of children or adolescents. This template and information are provided for reference only.

Checklist for developing and evaluating evaluation budgets

Jerry Horn

“This checklist is designed to assist evaluators and others thinking through the many issues that should be considered when developing an evaluation budget. This instrument is divided into several categories of typical cost items for conducting evaluations, and questions or statements are used as prompts for users to consider. In some cases, an item on the checklist may not be applicable.”

Download Resource Handout

Explore another module

What is practical measurement?

Practical measurement is measurement of children and adolescents’ learning and development that is driven by and focused on the needs of practitioners working with limited time and resources in low-resource, conflict-affected, and fragile contexts. Practical measurement is not a specific tool but rather a conceptual framework that can be used to develop a simple, yet rigorous plan to measure holistic learning and development outcomes related to physical health, mental health, psychosocial wellbeing, academic learning (e.g., literacy, numeracy), social and emotional learning, or values, morals, and spirituality.

Why this course?

When searching for online resources on measuring children and adolescents’ learning and development, there are two common forms of resources: those aimed at monitoring outputs of programs (e.g., number of training sessions implemented or number of beneficiaries engaged) and those aimed at rigorous program or impact evaluations such as randomized controlled trials. However, while these resources are valuable, they do not address many practitioners’ immediate purpose to go beyond monitoring outputs and to measure actual changes in children and adolescents. Recognizing the need for something that is driven by and focused on the needs of practitioners working with limited time and resources in low-resource and fragile contexts, the Practical Measurement course focuses on equipping practitioners with the knowledge and resources to develop rigorous yet feasible measurement initiatives.

Who is this course designed for?

Maybe you are an education technical expert with years of experience working in low-resource or fragile settings on programs for children. Or perhaps you are a monitoring and evaluation officer in the early stages of your career. Wherever you find yourself in your career, the Practical Measurement course is designed to support practitioners like you in understanding how to develop an effective approach to measuring holistic learning and development outcomes, especially when faced with limited time and resources. For this reason, you will find limited research and statistical jargon throughout the course. Instead, you will find clear and concise descriptions, tips, tools, and resources to support you on your journey of measuring the learning and development outcomes of children or adolescents.

What will I learn?

This course is focused on four key parameters that are important to consider when attempting to measure how children learn and develop through programs in low-resource and fragile contexts: appropriateness, utility, feasibility, and rigor. The materials in this course are not comprehensive nor are they meant to prepare you to take on a large research initiative. Instead, you can anticipate gaining a deeper understanding of the why and how behind measuring children and adolescents’ holistic learning and development outcomes. You can also expect to leave this course with a foundational understanding of how to start down the road of choosing a measure, adapting it, administering it with children, and ensuring that the data is useful. Throughout the course, you will find a curated list of resources to help develop a strategy to measure learning and development whether you are interested in tracking (formative assessments), screening, situation analyses, monitoring, or evaluations.

Return to the course homepage

Connect with Us

Introduction-Practical Measurement

Introduction

Module 1

Course Homepage

Descarga las diapositivas traducidas en español

Télécharger les présentations en français

Introduction

This brief introduction discusses the measurement focus of this course and frames the landscape of education and development research with children and adolescents. The module concludes by presenting the conceptual framework for practical measurement used in the rest of the course.

Explore another module

What is practical measurement?

Why this course?

Who is this course designed for?

What will I learn?

Return to the course homepage

Connect with Us

Feasibility Tool Selection-Practical Measurement

Feasibility: Tool Selection

Module 4

Course Homepage

Descarga las diapositivas traducidas en español

Télécharger les présentations en français

Feasibility: Tool Selection

Feasibility deals with the actual process of practitioners measuring learning or development outcomes for children and adolescents in low-resource, crisis-affected, and fragile contexts and asks whether the process is doable given the logistical, operational, and systematic limitations that are in place. In this module, practitioners will learn about the tradeoffs between adopting an existing tool, contextualizing or adapting a measure, or developing a survey from scratch. We also discuss the steps we recommend that practitioners take if they decide to contextualize or adapt an existing tool.

Additional Resources

Measure guidance: Choosing and contextualizing assessment measures in educational contexts

Silvia Diazgranados and Jeongmin Lee

“In this document, we focus on the assessment of children’s holistic learning and development (CHILD): specifically, the knowledge, skills and attitudes that may both promote and/or prevent children from thriving, now and in the future. We provide a step-by-step decision-making guide for researchers and practitioners interested in using CHILD measures in emergency contexts. The decision-making tree, tools and resources presented in this document will help these users make informed decisions about how to choose, contextualize and implement reliable and valid measures and understand the needs, challenges and critical support faced by the communities they serve.”

Cognitive interviewing: A “how to” guide

Gordon B. Willis

“This document describes the cognitive interviewing techniques appropriate for questionnaire development and testing.”

Cognitive interviewing with young children

Global Center for the Development of the Whole Child

The cognitive interviewing process has been primarily focused on youth and adults from high-resource countries. In order to ensure a more reliable and comprehensive study in low-resource or fragile settings, cognitive interviewing must be adapted to account for the age, expressive vocabulary, and capacities of children in the local community. This guide describes the experience of using cognitive interviewing in a low-resource setting when practitioners adapted the ISELA for Haitian school children. This guide includes discussion of the changes made and what was learned through the process, focusing on recommendations for researchers and practitioners who want to conduct similar measure adaptation with children in low-resource and fragile contexts.

Selection of learning and development measures and measurement libraries (sorted alphabetically). Please follow the copyright information provided with each tool.

Caregiver Reported Early Development Instruments (CREDI)

“CREDI were designed to serve as a population-level measure of early childhood development (ECD) for children from birth to age three. As the name suggests, the CREDI exclusively relies on caregiver reports, and thus primarily focuses on milestones and behaviors that are easy for caregivers to understand, observe, and describe.”

Children, Youth, and Families At-Risk (CYFAR) Evaluation Library

An initiative in the United States of America “that provides funding for Sustainable Community Projects (SCP). These projects are developed to meet locally identified needs, are research based, and provide quality programs for vulnerable, at-risk, low-income, and low-resource children, youth, and families to promote positive life outcomes.” Library includes approximately 300 different measures of child and adolescent health and development outcomes.

Collaborative for Academic, Social, and Emotional Learning (CASEL) Assessment Guide

“The Assessment Guide provides several resources for practitioners to select and use measures of student SEL, including guidance on how to select an assessment and use student SEL data, a catalog of SEL assessments equipped with filters and bookmarking, and real-world accounts of how practitioners are using SEL assessments.”

International Development and Early Learning Assessment (IDELA)

“IDELA, is an easy-to-use, rigorous global tool that measures children’s early learning and development and provides ECCD programs, donors, and government partners with clear evidence on the status of children from 3.5 to 6 years.”

Inter-agency Network for Education in Emergencies (INEE) Measurement Library

“The Measurement Library is a collection of measurement tools to assess children’s learning and holistic development and service provider quality in crisis contexts. This includes measures that have been vetted and tested by members of the Evidence to Action: Education in Emergencies (3EA) MENAT Consortium, along with technical working papers on the validity and reliability of the measures, guidance materials and training materials.”

National Institutes of Health (NIH) Toolbox

“NIH Toolbox includes over 100 stand-alone measures, also available in 30-minute batteries to assess Cognition, Emotion, Motor, and Sensation…NIH Toolbox is appropriate to use in the general population, in individuals with chronic conditions, and across the lifespan. While many measures can assess function from early childhood, others target specific age bands. All scores for a given domain are on a common scale and can be used for longitudinal measurement.”

People’s Action for Learning (PAL) Network Assessment Library

“The People’s Action for Learning (PAL) Network is a south-south partnership of organisations working across three continents. Member organisations conduct citizen-led assessments and/or citizen-led actions aimed at improving learning outcomes.” Includes primarily literacy/reading and numeracy assessments.

RAND Education Assessment Finder

“A web-based tool that provides information about assessments of K-12 students’ interpersonal, intrapersonal, and higher-order cognitive competencies. Practitioners, researchers, and policymakers can use it to explore what assessments are available, what they are designed to measure, how they are administered, what demands they place on students and teachers, and what kinds of uses their scores support.”

Download Resource Handout

Explore another module

What is practical measurement?

Why this course?

Who is this course designed for?

What will I learn?

Return to the course homepage

Connect with Us

Feasibility Design-Practical Measurement

Feasibility: Design

Module 5

Course Homepage

Descarga las diapositivas traducidas en español

Télécharger les présentations en français

Feasibility: Design

Feasibility deals with the actual process of practitioners measuring learning or development outcomes for children and adolescents in low-resource, crisis-affected, and fragile contexts and asks whether the process is doable given the logistical, operational, and systematic limitations that are in place. The feasibility of a measurement and evaluation plan must also address how simply and easily a practitioner can implement a plan and what level of guidance is needed for them to do so effectively. In this module on feasibility, practitioners will learn how to feasibly plan for and collect data from children and adolescents.

Additional Resources

Enumerator training

The World Bank

A detailed, step-by-step guide for how to train groups of assessors or enumerators to collect data.

Six rules of thumb for determining sample size and statistical power

J-PAL

“The ability of an evaluation to detect a meaningful impact of a program is determined by the evaluation’s sample size and statistical power. This is a tool for policymakers and practitioners that describes some of the factors that affect statistical power and sample size.”

Sample size calculator

Qualtrics

Online sample size calculator that can help you determine your ideal survey sample size.

PowerUp!: A Tool for Calculating Minimum Detectable Effect Sizes and Minimum Required Sample Sizes

Rebecca A. Maynar and Nianbo Dong

“This workbook is primarily designed to aide in the a priori power analysis calculation of minimum detectable effect size (MDES) and sample size required for a given research design and analysis. There are multiple worksheets included in this workbook, each specific to a certain design and analysis.”