PROFESSOR JO-ANNE BAIRD
Keynote Title: Challenges for the Future of Assessment
I outline five challenges that need to be recognised and addressed if public examinations are to continue to be trusted by societies. These are: 1) crises of knowledge, 2) spiraling reform cycles, 3) globalisation, 4) performativity and 5) grade inflation. Assessment has assumed even more power than it had in the past because it provides data with which we can monitor people and organisations. Targets are set for teachers and schools, as well as for individual pupils, using assessment data. As such, assessment data are an instrument of governance and management.
Concurrently, information technology is changing our lived experiences rapidly. In part due to technological advances, in education there has generally been a shift away from valuing memorisation of facts towards a requirement for higher order thinking skills. This is the essence of the first challenge, ‘crisis of knowledge’, because there are debates about what the curriculum and examinations should be assessing. In many countries, government regime change signals a new wave of assessment reforms, as the new government tries to ensure that the assessment system reflects its educational values. This is the second challenge, of ‘spiraling educational reform’. Educational assessment has largely been the province of national policy, but this century has seen the rise of international testing and this is having an effect upon educational assessment. The third challenge, of ‘globalisation’, looks at the effects of these tests upon national systems. Ultimately, there is broad consensus that education should have deep and lasting effects upon learning. However, the fourth challenge, ‘performativity’, makes us question whether that is always the case. Use of assessment data for accountability regimes has encouraged outcomes-based approaches that can treat the educational goals superficially. This leads to the fifth challenge, of ‘grade inflation’. If undesirable educational processes are pushing up educational outcomes, the outcomes become devalued and ‘inflated’, which in turn undermines public trust.
The implications of these issues for curriculum and pedagogy at multiple levels of the education system are explored. Tensions run through our education systems to the point that there are power struggles over what constitutes knowledge, who can be trusted to impart knowledge and by what criteria we credit knowledge with a qualification.
Baird, J. and Hopfenbeck, T. (forthcoming) Curriculum in the 21st century & the future of examinations. In Wyse, D., Hayward, L. and Pandya, J. (Editors) The Sage Handbook of Curriculum, Pegagogy and Assessment. Chapter 59.
Spotlight Title: Assessment and Learning
Teaching, learning and assessment have long been recognised as necessarily being connected. However, in this presentation, I argue that the relationship between assessment and learning theories and practices have not been as closely connected as they ought to be. I trace the trajectory of learning theories and their connections with assessment theory and practices. Psychometrics and assessment more broadly will be distinguished and the consequences of adopting on of these approaches will be discussed. Psychometrics is the basis of modern test theory; it was built upon statistical modeling to address mental testing issues such as what it means to be intelligent and how personality can be gauged.
Underlying assessments is a notion that there is a ‘construct’ to be assessed. The construct depicts the subject area and what it means to make progress in that subject. In this presentation, I distinguish features of educational assessment constructs that make them different from psychological constructs. I raise questions such as whether assessment has moved learning theory forward, whose role it should be to provide substantive theories of learning and what the current state of the field is.
Two significant trends in assessment this century have been the ascent of international tests such as the Programme for International Student Assessment (PISA) and the rise in Assessment for Learning (AfL) programmes. These assessment models are based upon very different philosophies and views of learning, have distinctly different purposes and are formalized in distinct ways. It is interesting to consider what international tests and AfL teach us about learning and what effects they have upon it at different levels of education systems.
Overall, it is difficult not to conclude that there is a disjuncture between learning theory and assessment theory. I discuss why this has occurred, whether it is desirable and what could be done to bring the two closer.
Baird, J., Hopfenbeck, T.N., Newton, P., Stobart, G. and Steen-Utheim, A.T. (2014) State of the field review. Assessment and Learning. Norwegian Knowledge Centre for Education Report Case Number 13/4697. OUCEA/14/2. http://goo.gl/bd7Bzd
PROFESSOR RAYMOND PECHEONE
Keynote Title: Reimagining Teaching and Learning: Bridging the Curriculum and Assessment Divide
Many educators believe the new standards are best acquired and assessed in novel, real world learning environments relevant to students’ personal interests and career aspirations. This belief is shared by teachers across the world inside and outside of the U.S. who employ interdisciplinary project- and problem-based practices as a way to engage and assess deeper, more complex, and more rigorous student thinking and problem solving.
To support deeper learning students must persevere in a range of “curriculum-embedded tasks and assessment” activities (including drop-in and extended performance assessments, projects, portfolios and exhibitions of learning) at the same time as they develop the skills necessary to acquire and demonstrate the levels of cognition required to meet the demands of 21st Century learning. Schools will have to build capacity and expertise in both interdisciplinary and disciplinary concepts, curriculum content, teaching practices, and assessments. The heart of the presentation will focus on problem of practice that focuses on the development curriculum embedded performance assessments as a catalyst for transforming teaching and learning in the US and worldwide. The presentation will focus on:
How can we collaborate with educators to develop a network of schools that builds skills simultaneously in disciplinary and interdisciplinary teaching, so students can apply the content and practices required by the new 21st Century standards to engage in challenging and authentic cross-disciplinary authentic real world problems in preparation for the jobs of the future?
This problem is both disciplinary and interdisciplinary because problems in the real world and within non-academic careers don’t emerge in single-subject area. Instead, authentic problems require cross-cutting approaches, broad schema, independence and innovation. The teaching and assessment methods that support interdisciplinary learning are therefore generalizable across contexts and include non-traditional subject and skill sets (for example cross-cutting and cognitively challenging tasks, growth mindsets, metacognition, design thinking, group dynamics). Much of what educators call “21st century learning”.
This problem is also disciplinary and embedded in college and career success because texts, tasks and problems exist within disciplinary and job related contexts. The materials and types of reasoning that students need to access and use to make meaning and demonstrate their understanding have been produced by disciplinary and industry experts. Academic and career contexts differ, sometimes dramatically, for example, in the types of evidence that they use, in how they use evidence, and in how they structure claims and arguments in evidence-based reasoning.
Spotlight Title: Using Teacher Student Assignments to Evaluate Teacher Effectiveness
In the U.S. student performance results have become a central feature of transforming teacher effectiveness systems to take into account student learning. In addition, measures of teacher practice (e.g., classroom observations) are important measures of teacher practice, but classroom observations of teacher practice is only one part of judging effective teaching –what is missing is evidence of student learning. Incorporating the results of Measures of Student Learning (MSL) into teacher effectiveness determinations, is technically challenging on many fronts, from the quality of the assessments to the type of analytic methods employed to transform those data into indicators of teacher effectiveness that take into account differences in classroom practice and do not resort to evaluating teachers on a single standardized tests of student learning (e.g., state level multiple choice tests). The Common Assignment Study (CAS) sponsored by the Bill and Melinda Gates Foundation and featured in this presentation holds promise for helping to improve the quality of MSL for use in the design of a high quality teacher effectiveness system.
The Common Assignment Study (CAS) is a research and development project in which teachers collaborate across schools to develop and implement high-quality curricular units to ensure student learning of meaningful disciplinary content and skills. This presentation describes how CAS curriculum units and assessments may be well-suited to provide data on student growth to contribute to evaluating teacher effectiveness. Using CAS units eliminates several threats to the use of student performance information in educator effectiveness determinations. First, CAS units provide high quality assessment information in ways that are fully integrated with instruction and the daily work of teaching and learning. In other words, “extra” standardized assessments are not needed to support effectiveness determinations. Each CAS unit contains multiple assessment tasks, so using multiple CAS units to support effectiveness decisions is a stronger model to judge a teacher’s impact on student learning than observation alone. Finally, the CAS process is an effective vehicle for supporting collaboration and serves to provide a natural bridge between the evaluation of educator practices and student performance results. In short, CAS units are designed to support high quality curriculum and instruction related to teacher and student standards and to provide meaningful classroom formative information about teaching and learning to both strengthen instruction and promote greater student growth.
DR ANNE DAVIES
Keynote Title: Practical, Research-based Ways to Use Assessment to Support Adult, School and System Learning
Leaders often ask, “How can we use assessment to support the teaching of students, the learning of adults, and their own work as educational leaders?” Many schools and school systems have been deliberately working towards full implementation of Assessment for Learning for more than a decade, yet success has seemed elusive for many. Too often leaders, with the best of intentions, tell others what they should do. At times, we, ourselves, have struggled with this stance. Yet, we realize that telling another what to do and how to do it is just not enough.
In our work with positional leaders and leadership teams over the past 15 years, we have deliberately gathered research evidence regarding the effectiveness of using Assessment for Learning as both the change and the change process. This longitudinal qualitative research study (Davies et al, 2014) drew on the experiences of eight positional leaders as they implemented both the ‘spirit and the letter’ of Assessment for Learning at all levels. These leaders came from systems in Alberta, British Columbia, Germany, Georgia, Hawai’i, Manitoba, New Zealand, and Ontario. The timeline of implementation varied from jurisdiction to jurisdiction, with the shortest implementation at the time of the interviews taking place over 3 years and the longest over 10 years. The systems ranged in size from 350 students to more than 180,000 students.
After analyzing the findings of this study, it was clear that there were three actions that the successful leaders employed:
1. Leaders must take action and move beyond words to deeds.
2. Leaders evaluate what they value and move beyond numbers to include triangulated evidence of learning.
3. Leaders find ways to collect ongoing information and use frequent feedback loops.
In this session, the research results along with accounts from schools and systems will be used to illustrate applications implications to daily leadership practice.
Spotlight Title: Using Assessment in the Service of Adult and School Learning: Taking Action
As detailed in the keynote session with Dr. Anne Davies, actions communicate more than words. In this session, Dr. Davies will involve the participants in:
· Beginning with the end in mind
· Co-constructing criteria using two different starting points
· Share ways to give specific, descriptive feedback during the teacher professional growth and supervision processes
· Collecting evidence of learning
· Communicating that learning to others
This is a practical session. Please come prepared to talk with and work alongside your colleagues.
PROFESSOR PETER TYMMS
Keynote Title: Assessment without Accountability: Is it Possible?
All parts of the education system, including teachers and principals, need to be held to account for the work that they do. But the unintended consequences of using tests and exams as part of the accountability mechanisms have gone too far. The pressure has pushed educationalists so far that we see reports of cheating in various part of the world, we see distorted curricula, inappropriate preparation for tests, distraught students and stifled creativity.
One reaction against this is to reject testing entirely. But that is to misunderstand the many purposes of assessment only one or which is accountability. Another is to ensure that the teachers understand their students’ learning and to have a clear idea where they are on their educational journey. Indeed it will be argued that they best teachers are those that have the clearest ideas of their students’ zones of proximal development through a combination of tests, discussion and observation all of which come under the heading “assessment”.
Examples will be given of successful assessments systems which are designed to be used by schools for formative purposes but not accountability. These systems have been used with millions of students over more than two decades. The systems, run from the Centre for Evaluation and Monitoring (CEM) at Durham University are designed for stages of the schooling systems through the age range from 4 to 18.
But how can accountability be maintained without the use of national exams and tests? A multiple approach is suggested. This involves some form of school inspection which ensures that schools have suitable processes in place including the tracking of student progress. It also includes national participation in international surveys such as PISA and TIMSS and the additional national collection of sample based monitoring data. The example of NEPS from Germany and the international monitoring for children starting school, iPIPS, will be discussed.
The talk will emphasise that assessments need to be designed and used for specific purposes and that their use for additional purposes can undermine their value.
Spotlight Title: Non-cognitive Assessments
Most of the testing in the world is cognitively oriented concentrating on two broad areas. One aims at curriculum-free constructs such as IQ and short term memory, the other focusses on the curriculum assessing domains such as reading, mathematics, History and Chemistry. But there is increasing recognition across the word that non-cognitive aspects of education are as important as the cognitive. The famous claim that the ability of 7-year olds to defer gratitude in the marshmallow test was a better predictor of college grades than IQ scores is one bit of evidence which is quoted in support of this assertion.
There is also the reanalysis of the ground breaking Perry pre-school project data from the USA suggesting that the value of pre-school education lies not so much in cognitive as in non-cognitive development; an idea which is being promoted by the Nobel prize winning economist James Heckman amongst others
But the phrase “non-cognitive” is used in a very broad sense and this session will break it down into various parts looking at the Big Five personality dimensions, mental health (internalising and externalising disorders), personal, social and emotional development, behaviour, concentration/inattention, grit, persistence and creativity. A structure will be provided to show how these apparently disparate measures fit together, which overlap and which address the same issue under another name.
From this generalised structure ways in which the various constructs can be measured will be discussed as well as the prospect for assessing them within classrooms. Evidence will also be set out which seeks to assess the potential impact of schooling on the measures answering questions such as: Does education change personality? Can we measure creativity well enough to use the data educationally? Are their evidence-based techniques which can reduce inattention? Can testes manipulate the results of their non-cognitive assessment results in a high stakes situation?
Finally a set of ground rules will be proposed for educationalists interested in developing and using non-cognitive measures.
ASSOC PROFESSOR KELVIN TAN
Keynote Title: Nounification and the Complex, Contested and Contextual Agendas of Assessment for Learning.
Recent focus in Assessment for Learning (AfL) has shifted from defining the scope and extent of AfL to understanding its implications and implementation. However, research reveals AfL implementation to be complex and contested, requiring much modification for various contexts. AfL implementation is especially challenging in large scale national contexts which may emphasise high stakes examination performance and grades. A coined term, ‘Nounification’, is used to describe the tendency to reduce AfL practices to nouns, and to pass off the noun as proven and useful theory. It is argued that such nounification tendencies divert teachers’ efforts from AfL practice to render AfL as forms of semantical compliance. Such oversimplification prevents, and preempts, understanding AfL in its complex, contested and contextual forms. An inquiry based approach is recommended for teachers to frame AfL implementation as a practice to be understood, rather than a package of pre-determined practices to be administered.
Spotlight Title: Asking Questions of Assessment for Learning: What does Learning require of Assessment?
The diversity of approaches to understanding and using Assessment for Learning (AfL) reflects the myriad purposes and agendas serve. Recent emphasis on the implementation of AfL signals a more pragmatic phase in its discourse. However, it is not always clear what issues and problems that AfL implementation is meant to serve. The ambiguity is exacerbated by ongoing uncertainty, and lack of consensus, over how AfL should be defined.
One way to enhance the clarity and alignment of AfL implementation and outcome is to frame its discourse as a question to be asked. “What should Assessment be doing for Learning” is suggested as a discursive inquiry alternative to the phrase ‘Assessment for Learning’. This spotlight session examines the different questions that need to be asked by policy makers, leaders and teachers, and how each different question can possibly (re)construct AfL for different agendas.
DR JAN KEIGHTLEY
Keynote Title: Assessment With Purpose – How Do We Achieve This?
Assessment is ubiquitous and is used for a diversity of purposes both within and outside of education. Often the purpose is obscure at best and sometimes with no pre-planned specific purpose.
This presentation will explore how to achieve the purpose of assessment through two lenses. Specific institutional based examples will be used to illustrate the journey from initial intention through to the final stages of assessment.
The first will be through a curriculum lens where learning outcomes, learning activities and assessment tasks interact to achieve purposes. While not a new concept (presented by Biggs (2003)), the concept of alignment will be illustrated through a current example of alignment within a higher education institution and demonstrating the strengthening of the assessment to have purpose.
The second lens will be through standards and in particular professional standards for teachers. Teacher effectiveness and impact on student learning has taxed the creativity of educators and researchers in education for an extended period of time. A number of countries have explored the nature and format of professional standards for teachers with variations in implementation. This presentation will explore the assessment of teacher effectiveness through the application of teacher professional standards as developed in Australia. Some of the challenges associated with ensuring this assessment is with purpose will be identified.
Biggs, J (2003): Aligning Teaching and Assessment to Curriculum Objectives, (Imaginative Curriculum Project, LTSN Generic Centre)
Spotlight Title: Using Professional Standards As A Basis For Assessment Of Teachers
This session will focus on teacher assessment and will provide participants with an opportunity to share their own experiences of assessment of teacher effectiveness and explore some of the implications and challenges associated with these experiences.
The Australian Professional Standards for Teachers will be elaborated in detail and the use of portfolio assessment in an electronic medium will be highlighted with specific examples of evidence presented by initial teacher education students to demonstrate the achievement of the professional standards.