Assessing technology enhanced assessment
Technology enhanced assessment should inform engagement, improvement, achievement and direction and collaboration.
To transform learning and teaching, we must start by transforming assessment, feedback and reporting.
There is considerable potential for multimedia technologies to make feedback richer and more personal and for a wider range of learner
skills and attributes to be demonstrated through assignments involving, for example, e-portfolios, blogs and wikis. In addition, online tools can support peer and self-assessment in any location and at times to suit learners – the value of peer and self-assessment in developing learners’ ability to regulate their own learning is increasingly recognised
JISC (2010: 11)
Technology enhanced assessment incorporates far more than computer-based assessment, incorporating opportunity for ‘stealth assessment’ (Oldfield, Broadfoot, Sutherland and Timmis), summative and formative assessment, valuable, continuous feedback loops and immersive experiences, all integrated to create a deep learning experience.
The nine considerations and criteria below provide an oversight of long identified requirements, aligned with current and emerging innovation and technical capabilities. They should be considered from the perspectives of students, educators, administrators and leadership, at school, jurisdictional and/or national level. All stakeholders can benefit from meeting the criteria below, but for many, these will be game changers.
These considerations should be the benchmark expectations of effective technology (platforms, systems, software, apps, devices) to support assessment, backed by pedagogical practice:
Technology enhanced assessment should promote equity and accessibility
Is equity supported through:
Consistent experience and benchmarking of results across the state, acknowledging the disparity that exists
Access to undertake assessments online and offline, with or without network access, and through network disruption
Meeting accessibility compliance and universal design across the platform and assessment activities, for all users – including capabilities for text or image-based tasks.
Offline task development and marking capability
An array of options for providing feedback (voice, commenting)
Anonymous candidature marking, ensuring transparency and reducing bias
Technology enhanced assessment should ensure safety, security, privacy and reduced plagiarism
To what degree does the technology support:
Invigilation capabilities that ensure effortless oversight of attendance, progress during assessment and completion, for all students.
Online monitoring of all candidates during assessments if required (without overloading teachers) e.g. utilising traffic light systems alerts
Improved plagiarism detection and resolution through invigilation monitoring and alerts, mechanisms for students to test their compliance
Academic integrity through locking down of restricted applications during assessment
Privacy and security for data around student and their results, through relevant profiles integrated across the system
Security around assessments and marking requirements
The capability for all users to electronically log procedures and report on their own case history for training or audit purposes.
Technology enhanced assessment should be agile and adaptable
How agile and adaptable are the assessment platform, bank and tasks:
To allow and address context, individual, localised or jurisdictional requirements?
To compliment, and integrate into existing infrastructure, through minimal change management and rapid implementation strategies.
To suitable assessment methodologies for your level, subject, compliance of policy.
Customization requirements for personalisation
In keeping with the wide range of assessment methodologies available. Simulations, MCQs, vocational education requirements.
To support the current and emerging array of assessment tools and capabilities, with consideration of age, level, stage, devices and operating systems?
To support and align summative and formative assessment?
Technology enhanced assessment should ensure validity, reliability and transparency
Is validity and reliability supported through evaluation and reporting:
Tracking all assessments and making use of the data available to determine and improve reliability, relevance and validity of assessment items at local and strategic level?
To support feedback loops encouraging teacher improvement
To inform strategic oversight, initiatives, implementation, benchmarking and scaling of exemplar practice at school, jurisdictional level
For required audit and reporting processes.
On task statistics, supporting comparators on achievement, identification of areas of weakness for a student or group of students, in syllabus items or in the assessment task.
To inform effective strategies and provide feedback on teaching approaches and initiatives
To determine exemplar marking guides
Technology enhanced assessment should be informed by and aligned to curriculum and skills
Is there capability and transparency for:
Tagging assessment tasks and questions against syllabus outcomes, learning outcomes, level of difficulty / challenge, general capabilities and across faculty, to support reporting on coverage and connection: syllabus, skills etc
Alignment to frameworks such as learning progressions
An approval process supporting the addition or amendment to the bank of assessment tasks
Technology enhanced assessment should be seamless with all systems and stakeholders
How seamless is:
Access through a single login with other elements of the eLearning ecosystem and the linked administrative systems
The experience, traversing learning activities, assessments and feedback, do they inform each other
The assessment experience in the event of network or other failure?
Teacher oversight of progress etc during assessment to inform interventions or dis-engagement
Technology enhanced assessment should be efficient and easy to implement and use
What level of training is required? For whom and at what stages?
The minimal work required by teachers, leadership and support staff through assessment design, test bank building, testing and marking?
The capabilities and support for ‘closing the loop’, utilising the data to inform changes to or scaling of practice.
The assessment banks and their scalability, sustainability and adaptability, locally and strategically?
A user-friendly experience, navigation and interface – from the perspective of all stakeholders?
Technology enhanced assessment should inform engagement, improvement, achievement and direction
Can the technology effectively support:
Alignment with other systems to ensure transparency over all aspects of learners
Adaptability during the learning experience, informed by achievement, skill, motivation and resilience
Effective and efficient capabilities, whilst considering initial and on-going cost considerations (licensing etc) and support requirements
Assessment tasks that provide immediate feedback (with or without teacher intervention)
The entire assessment journey, supported by question banks, randomisation, immediate feedback to reinforce content, inform next / future assessment tasks, ubiquitous access to assessment tasks and results?
Agile assessment journeys that react to achievement in previous assessments, current assessment, across subjects, as required.
Practice tests to motivate and build confidence
The provision of a horizontal and vertical picture of student achievement and improvement – across years and across subjects
Increased learner agency, autonomy and self-regulation
Teacher professional learning and development, to inform development required, then through the development process
To assess professional skills and attitude to provide evidence of appropriate everyday pedagogical competences.
To manage professional learning to record and reflect on activities undertaken.
Technology enhanced assessment should promote collaboration
Does the technology support:
Collaborative learning (for teachers and students)
Co-creation and sharing of modules and assessment banks, designed, maintained and evaluated at local or strategic level.
Communities of practice focusing on assessment, informed by review and data
One to many capabilities: Self-assessment, Peer assessment and group assessment, supporting demonstration of individual and group work, continuous assessment reinforced by monitoring, encouragement and review, personal development and learning throughout the course, non-bias feedback and a mix of assessment patterns.
Collaboration in assessment design between schools, across jurisdictions.
Technology enhanced assessment – moving forward
Oldfield, Broadfoot, Sutherland and Timmis recommend several areas for future research:
“Cultivate new assessment practices based on principles and theories of learning
Develop new assessment tools that reflect pedagogical principles
Construct new responses to the current emphasis on high-stakes summative assessment
Respond to ethical challenges presented by the use of digital technologies in assessment
Consider new contexts relevant to assessment using digital technologies, including learners’ lives and social, cultural, educational and technical backgrounds”
As the technical capabilities evolve and technology enhanced assessment becomes the norm, these issues will be addressed to ensure an adaptive, seamless experience for students, teachers and administrative support. This must be a co-created and co-implemented vision between pedagogical experts and technical experts, considering and harnessing all the above, ratified by agile policy reflecting current and emerging capabilities.