Practical Measurement

Homepage

Course Introduction

The current discourse around measurement and evaluation of children and adolescents' learning and development has moved the sector away from reliance on anecdotal information to a focus on capturing the actual change in outcomes for children and adolescents in specific programs or interventions. While there has been significant attention on the rigor of measurement, limited focus has been placed on ensuring that measurement of children and adolescents' learning and development is appropriate and feasible for practitioners in low-income and fragile contexts. Through the programs supported by the Global Center for the Development of the Whole Child (GC-DWC), we focus on practical measurement: measurement that is driven-by and focused on the needs of practitioners working with limited time and resources in low-resource and fragile contexts.

Designed with this in mind, the Practical Measurement course focuses on four key parameters —appropriateness, utility, feasibility, and rigor— that practitioners should consider when attempting to measure children and adolescents’ learning and development. Pulling on over a decade of experience working on measurement and evaluation in low-resource, conflict-affected, and fragile contexts, Dr. Nikhit D’Sa guides users through the self-paced modules, sharing practical measurement and evaluation guidance, insights, tips, and resources. Explore the modules in any order you wish, collect Dr. D’Sa’s recommendations on additional measurement and evaluation resources from global organizations, and begin to develop a plan to simply and effectively measure and evaluate children and adolescents’ outcomes in your context.

Explore the course

Course contents

Introduction

This brief introduction discusses the measurement focus of this course and frames the landscape of education and development research with children and adolescents. The module concludes by presenting the conceptual framework for practical measurement used in the rest of the course.

Appropriateness

Appropriateness deals with what is actually being measured and how it is defined. Within child and adolescent development, there are numerous competency domains practitioners may want to measure and evaluate: physical wellbeing, mental health, cognitive functioning, academic skills (e.g., literacy and numeracy), social and emotional learning, and values, morals, and spirituality. In determining a measure’s appropriateness, practitioners need to consider the following questions: 1) what do I want to measure? 2) why do I want to measure it? 3) how will I measure it? and 4) who will use the results? We discuss the nuances of these questions when using formative or summative measures for tracking, screening, situation analyses, monitoring, or evaluation.

Utility

Utility, at its core, deals with whether or not the data that practitioners collect from an assessment, tool, or survey will be useful to make reasonable decisions about the activity, intervention, strategy, or program being implemented. Data about children’s learning and development needs to have a practical goal that allows practitioners and policy makers to learn about how their program affects children and what changes they can make to programming to improve children and adolescents’ development. In this module, practitioners will learn how to balance the time, resources, and commitment needed to rigorously measure learning and development outcomes, and use the data they collect.

Feasibility: Tool Selection

Feasibility deals with the actual process of practitioners measuring learning or development outcomes for children and adolescents in low-resource, crisis-affected, and fragile contexts and asks whether the process is doable given the logistical, operational, and systematic limitations that are in place. In this module, practitioners will learn about the tradeoffs between adopting an existing tool, contextualizing or adapting a measure, or developing a survey from scratch. We also discuss the steps we recommend that practitioners take if they decide to contextualize or adapt an existing tool.

Feasibility: Design

Feasibility deals with the actual process of practitioners measuring learning or development outcomes for children and adolescents in low-resource, crisis-affected, and fragile contexts and asks whether the process is doable given the logistical, operational, and systematic limitations that are in place. The feasibility of a measurement and evaluation plan must also address how simply and easily a practitioner can implement a plan and what level of guidance is needed for them to do so effectively. In this module on feasibility, practitioners will learn how to feasibly plan for and collect data from children and adolescents.

Rigor

Rigor, in the context of practical measurement, deals with not only the level to which a measure, survey, assessment, or tool provides information that is both valid and reliable but also whether the thresholds of rigorous measurement and research are actually viable for practitioners and researchers working in low-resource, crisis-affected, and fragile contexts. In this module, we discuss not only the different types of validity and reliability tests/statistics, but also which ones we recommend using when measuring outcomes for tracking, screening, situation analyses, monitoring, or evaluation.

Common questions

What is Practical Measurement?

Practical measurement is measurement of children and adolescents’ learning and development that is driven by and focused on the needs of practitioners working with limited time and resources in low-resource, conflict-affected, and fragile contexts. Practical measurement is not a specific tool but rather a conceptual framework that can be used to develop a simple, yet rigorous plan to measure holistic learning and development outcomes related to physical health, mental health, psychosocial wellbeing, academic learning (e.g., literacy, numeracy), social and emotional learning, or values, morals, and spirituality.

Why this course?

When searching for online resources on measuring children and adolescents’ learning and development, there are two common forms of resources: those aimed at monitoring outputs of programs (e.g., number of training sessions implemented or number of beneficiaries engaged) and those aimed at rigorous program or impact evaluations such as randomized controlled trials. However, while these resources are valuable, they do not address many practitioners’ immediate purpose to go beyond monitoring outputs and to measure actual changes in children and adolescents. Recognizing the need for something that is driven by and focused on the needs of practitioners working with limited time and resources in low-resource and fragile contexts, the Practical Measurement course focuses on equipping practitioners with the knowledge and resources to develop rigorous yet feasible measurement initiatives.

Who is this course designed for?

Maybe you are an education technical expert with years of experience working in low-resource or fragile settings on programs for children. Or perhaps you are a monitoring and evaluation officer in the early stages of your career. Wherever you find yourself in your career, the Practical Measurement course is designed to support practitioners like you in understanding how to develop an effective approach to measuring holistic learning and development outcomes, especially when faced with limited time and resources. For this reason, you will find limited research and statistical jargon throughout the course. Instead, you will find clear and concise descriptions, tips, tools, and resources to support you on your journey of measuring the learning and development outcomes of children or adolescents.

What will I learn?

This course is focused on four key parameters that are important to consider when attempting to measure how children learn and develop through programs in low-resource and fragile contexts: appropriateness, utility, feasibility, and rigor. The materials in this course are not comprehensive nor are they meant to prepare you to take on a large research initiative. Instead, you can anticipate gaining a deeper understanding of the why and how behind measuring children and adolescents’ holistic learning and development outcomes. You can also expect to leave this course with a foundational understanding of how to start down the road of choosing a measure, adapting it, administering it with children, and ensuring that the data is useful. Throughout the course, you will find a curated list of resources to help develop a strategy to measure learning and development whether you are interested in tracking (formative assessments), screening, situation analyses, monitoring, or evaluations.

How long will this course take?

The course consists of six modules that each have a video and accompanying resources. The videos range from six to eleven minutes in length, and you can take them in any order you like as is convenient with your schedule. There is no registration required, and you can come back to a module as frequently as you like. Additionally, within each module, you can choose to delve deeper into a topic by exploring Dr. D’Sa’s curated list of resources.

Meet your instructor

Dr. Nikhit D’Sa is a developmental psychologist with a decade of experience as an education technical advisor and applied researcher in low-resource, crisis, and conflict settings. He has expertise developing and managing cross-sectoral learning frameworks, leading impact evaluations, conducting qualitative and mixed-methods studies, designing and validating assessment tools, and working with practitioners to make evidence-based changes to interventions. Dr. D’Sa has worked as a trainer, counselor, and consultant on child and youth development projects around the world with multiple international and civil society organizations. Before joining the University of Notre Dame, Dr. D’Sa served as the Director of Research, Evaluation, and Learning at Save the Children. In this role, he helped establish and grow the Education in Emergencies research team, while also designing and validating the International Social and Emotional Learning Assessment (ISELA) and the Holistic Assessment of Learning and Development Outcomes (HALDO). Dr. D’Sa holds a Master of Education in International Education Policy and a Doctor of Education in Human Development from the Harvard Graduate School of Education.