Below are answers to some of the common questions about the Measurement Library. For other questions and information, contact email@example.com
- What is the Measurement Library?
- Why is a resource like this Measurement Library necessary?
- What is available in Measurement Library 1.0, available in November 2019?
- Who is this Measurement Library intended for? What types of professionals might find it useful?
- Who helped design the Measurement Library?
- How were the organizations involved in putting together content for the first phase of the Measurement Library selected for this project?
- How did the consortium determine which tools to include and analyze in this Measurement Library?
- I’d like to select a measure to use. How do I get started?
- What if the measures in the Library don’t capture what I/my organization want to assess?
- What do we mean when we say, “high-quality data”?
- How can I determine whether data that I collect is high-quality?
- What if I want to adapt the measures and guidance materials?
- What can I/my organization do to support the Measurement Library and grow the evidence base on the validity and reliability of measurement tools? How can I/my organization support a resource like this Measurement Library?
- What’s needed to improve the quality of data? The Measurement Library includes the measures themselves plus a range of other tools. Why?
- Why aren’t more measures in the Measurement Library “ready for purpose”?
- Why is it important that measures meet a high standard of evidence of validity and reliability?
- Why are measures that are “proceed for purpose with caution” and “under development” listed in the MENAT Measurement Library?
- There are measures out there that say they can be used in any context and produce reliable, valid data. Why not just use one of those?
- Why do you require my name and email address to download a measure?
What is the Measurement Library?
The Measurement Library provides stakeholders with what they need to collect data that will help them strengthen education and protection of children and youth in emergency contexts. In its current phase, the Library contains tools that can be used to collect data on the quality of service provision and children's holistic development outcomes within specific education and psychosocial programs. These tools include measures and information on their psychometric properties, training and guidance materials.
The vision for the Measurement Library is that it will grow to become a centerpiece of a robust community of practice supporting the measurement of child development and learning in crisis contexts around the world. In this more advanced phase, the Measurement Library has the potential to include measures developed and tested for use in different global regions and different purposes, including: providing formative feedback to a child and gauging the quality of learning at national levels.
Why is a resource like this Measurement Library necessary?
A longstanding challenge for those working to educate children and youth in crisis contexts has been a lack of data and evidence to guide decision-making. This knowledge gap includes data and evidence about whether children are learning, what skills individual children have or have not mastered, the need for targeted educational and mental health supports and whether programs are working for all intended beneficiaries.
Without this knowledge, researchers, practitioners and funders are often relying on intuition and instinct rather than facts when deciding how to design and fund programs meant to serve this children affected by conflict and crisis. The Library will help those working with children to collect and interpret data that can inform design, policy and funding decisions.
What is available in Measurement Library 1.0, available in November 2019?
In this Measurement Library 1.0 you will find information on measures in one of two formats. Consortium measures include a comprehensive set of materials that have been vetted and reviewed. The other format is for Inventory measures: We included these measures because we know they have been used in the MENAT context, and might be informative or interesting to users. Inventory measures do not have an accompanying set of materials and have not undergone the same review process as Consortium measures.
For more information on the content of the Library, check out the recording and presentation from a webinar held in November 2019.
Measures or assessments of children’s holistic learning and development and quality of service provision that have been tested for use in crisis contexts in MENAT by members of the Evidence to Action: Education in Emergencies (3EA) MENAT Consortium. Individual measure landing pages may include:
The measure itself in the language of administration;
A technical working paper that details evidence of the extent to which the measure provides consistent (reliable) and meaningful (valid) data when used for the specified purpose and in the specified context;
Guidance materials that can be used to make decisions about how to adapt the measure and how to analyze the resulting data;
Training materials developed to support data collectors to administer the measure.
Measures or assessments of children’s learning and holistic development that have been used in MENAT prior to the development of this Library. This Excel sheet includes:
A high-level overview of where and with whom measures have been used in MENAT;
Citations for journal articles that document the measure development and/or testing process that can be searched independently.
Who is this Measurement Library intended for? What types of professionals might find it useful?
At each stage, the Measurement Library is and will be intended for anyone working on or interested in the education, protection, health, safety and well-being of children and youth in emergency situations. During its first phase, the Library is most useful for education researchers and practitioners. Practitioners will find it helpful as they select, adapt and administer measures to be used as they monitor and evaluate their programs, provide feedback to children and service providers and identify children in need of specific services. For researchers working in MENAT, the first phase of the Library will help them better determine the quality of the data that results from the use of the measures and understand what measures are available or relevant as they pursue further data collection.
More broadly, practitioners, policymakers and researchers working in other regions may use the Library to see the types and breadths of measures currently available and help them identify which measures could be adapted for use in their own contexts. For these stakeholders, later iterations of the Library will in turn include information about measures that are reliable and valid beyond the MENAT region. Donors engaged or interested in engaging with education in crisis contexts may use the library to see what measures can and cannot be used for and get a deeper sense of what collecting and analyzing high-quality data entails. For donors, the Library will inform their decision-making about investments in measurement, data analysis and data use.
Who helped design the Measurement Library?
The Library itself was designed by the Inter-agency Network for Education in Emergencies (INEE), an international network of NGOs, governments, UN agencies, donors, teachers and other stakeholders working to safeguard the right of all people to a safe, high-quality education in crisis and conflict settings.
How were the organizations involved in putting together content for the first phase of the Measurement Library selected for this project?
The Library’s content in this initial phase has been selected and provided by the Evidence to Action: Education in Emergencies (3EA) MENAT Consortium. The Consortium is comprised of eight research-practice-policy partnerships who are working to promote the holistic learning and development of children and youth in crisis contexts.
How did the consortium determine which tools to include and analyze in this Measurement Library?
The process to determine which tools to include or analyze in the Measurement Library was led by the 3EA MENAT Consortium, convened by NYU Global TIES. The web design of the Library was the result of a collaboration between the INEE, IRC and NYU. An organizing principle guiding the process to decide what to include in the Library was the desire to assemble, with rigor, the most thorough and useful collection of measures currently available to the child protection and education in emergencies space.
I’d like to select a measure to use. How do I get started?
Before selecting what other tools to use in the Library, we strongly encourage all users to consult the Measure Guidance. This resource will not only help you determine which tools are best suited to the assessment you are pursuing; it may also help you better understand the other questions you ought to be asking yourself when it comes to program design and evaluation, screening and diagnostic processes and curriculum development.
What if the measures in the Library don’t capture what I/my organization want to assess?
There is a chance that the Measurement Library won’t contain the precise tools you need. If this occurs, we encourage you to reach out to the Library’s designers at firstname.lastname@example.org directly and let us know about what information you’re seeking and the ways the Library could better suit your organization’s needs. While we cannot guarantee that we will then be able to adequately address your concerns, hearing from other individuals and organizations working in this space is always helpful and holds the potential to improve future iterations of the Measurement Library.
What do we mean when we say, “high-quality data”?
“High-quality” data refers to data that is complete, consistent, valid, timely, verified and accurate.
How can I determine whether data that I collect is high-quality?
Before using the Measurement Library’s tools, be sure to review the Library’s evidence on the validity and reliability of each tool, the information about the testing context and purpose on each measure’s landing page, and the training materials. Following the guidance provided will in turn help ensure that the data you collect is reliable and valid, which is a key component of data that is “high-quality.” High-quality data will also contain a minimal amount of missing values and will be presented in a clear, coherent fashion.
What if I want to adapt the measures and guidance materials?
The Measurement Library’s materials have been carefully selected and vetted to ensure they are as useful and effective as possible. While we can only vouch for the quality of the tools as they are presented, we appreciate that you may also benefit from adapting them for your own purposes. However, any use of a measure included in the Library that deviates from the use clearly specified in the measure’s landing page requires the express permission of the measure’s developer.
For adapting measures and guidance materials, please consult the Measure Guidance.
What can I/my organization do to support the Measurement Library and grow the evidence base on the validity and reliability of measurement tools? How can I/my organization support a resource like this Measurement Library?
The most important and helpful way that you and your organization can support the Measurement Library is to share the results and data you gather while using the Library. Hearing directly from users is a powerful way to make sure that the resources contained in the Library are helpful and relevant and presented in a way that shows respect for the user’s time and attention. Specifically as we move into the second version of the Library, the feedback from practitioners and researchers about what they need and how useful they’ve found the Library to be will be essential.
What’s needed to improve the quality of data? The Measurement Library includes the measures themselves plus a range of other tools. Why?
It is not enough to make tools for assessing child learning and development available; we must also ensure that these practitioners, policymakers, and researchers know which tools to use and how to use them. As a start, the Measurement Library 1.0 contains a Measure Guidance document to assist practitioners and researchers as they select, adapt, administer, and analyze data resulting from assessments. The Measurement Library 1.0 also contains other training materials to help ensure that the measures are being used properly and effectively. High-quality data can only be achieved through a precise, rigorous process and the Measurement Library contains resources that help make that sort of process possible.
Over time, knowledge gained from the repeated use of the different measures in the Measurement Library will provide a more tested, thorough idea of which measures can and should be used more broadly. This repeated use of the measures over long periods of time and in distinct contexts (regional, population size, assessment focus, etc.) will lead to an accumulation of evidence about how reliable and valid that measure is generally. To build a Measurement Library of the highest quality requires the creation of a large, dispersed community of users who are empowered and committed to using the measures over time and report back, in great detail, about all they’ve learned.
Why aren’t more measures in the Measurement Library “ready for purpose”?
The Measurement Consortium’s testing of measures produced varied results with having sufficient evidence of validity and reliability to be designated as “ready for purpose”. There are two main reasons for this. First, strong measures are not instantly created. They are developed and refined over time as evidence accumulates with new trials, in different contexts or for distinct purposes. Identifying stronger measures is like trying to identify what programs work best for children. It's hard to draw broad conclusions from one or two evaluations of programs in different contexts; dozens of trials are needed to have confidence that the program is really working and achieving what is intended.
The same is true for measures. Those in the Measurement Library that are “ready for purpose” were further along in the process of testing and iteration, having already gone through extensive rounds of revisions prior. Others were at a more nascent stage when our group convened.
In addition, we have set a high bar for designating measures as “ready for purpose”: to meet this standard a measure must have good to excellent evidence of multiple types of reliability and validity when tested in a given context. Moreover, we have to have confidence in the stability of the results or that the results could be replicated if tested with the same sample again.
Read more about the Measurement Library Measure Review Criteria.
Why is it important that measures meet a high standard of evidence of validity and reliability?
Just as a broken thermometer would not accurately tell you whether or not you are sick and need to stay home from work, a measure yielding inaccurate results on children’s learning and development cannot be trusted and used for good decision-making. Program managers and policymakers working in the MENAT educational context operate with an excess of challenges and a shortage of reliable, actionable information. Faced with limited resources, they need measures that they know can provide information that will direct their efforts in the most productive way possible.
Why are measures that are “proceed for purpose with caution” and “under development” listed in the MENAT Measurement Library?
It is crucial that information generated by measures in the Library meets a high standard of reliability and validity. It is also equally important to be candid and transparent about measures that thus far have mixed or little evidence of reliability and validity for the situation in which it was tested. Ultimately, even when the results are not as straightforward as we had hoped, we can learn important things about the measure we are testing and how to revise it for the next iteration. We see this gain in knowledge as an opportunity, not a loss.
At the same time, we recognize that there is tremendous risk in using measures that provide inaccurate information. This risk underscores the importance of transparency; if we refuse to over-sell our progress or wave away inconvenient facts learned along the way, we ultimately protect the credibility of this initiative and help ensure that time and resources are not wasted going down any unproductive roads.
There are measures out there that say they can be used in any context and produce reliable, valid data. Why not just use one of those?
While caregivers, teachers and policymakers across contexts may broadly agree on the skills and competencies critical for children’s long-term success, how such skills are named, defined, manifested, operationalized and/or prioritized differs according to the context. For that reason, we recommend that measures are always at minimum adapted for use in a new context. The measure should also be retested to evaluate the evidence for use in a new situation and context, and the results of the adaptation shared back to promote shared learning.
Why do you require my name and email address to download a measure?
In order for the measures in the Measurement Library to be accessible to the public, we require users to review and agree to the terms listed on the measure landing page. This ensures all measures in the Measurement Library will be used carefully and for the appropriate purpose.
If you have any questions about the Library, its resources, or these terms, please contact email@example.com.
The Measurement Library is a collection of measurement tools to assess children’s learning and holistic development and service provider quality in crisis contexts.