Platinum Sponsors

PSI will be presenting at this year's ATP Virtual Conference!

See below the sessions our PSI experts we'll be delivering and make sure you join us to learn more about innovations in testing. 

September 14th

About: The COVID-19 crisis brought about the most sudden and dramatic shift in test delivery that we have ever witnessed.  Forced with a binary choice of moving testing online or stop testing, many organisations chose the former and moved with creditable agility to adopt online proctoring in weeks rather than, more typically, months or years.

Consequently, the spotlight was shone on an aspect of high-stakes assessment that had hitherto seldom received pubic attention – and the coverage was, by no means, all positive.  There was criticism of specific failures to prevent widespread malpractice and questions whether the model provided a level playing field for candidates testing online in differing circumstances, and versus cohorts taking the test onsite.

But the strongest voices were from the candidates themselves, who cited lack of alternatives (to online proctored tests), significant privacy concerns including data usage and remote access to devices, and the charge that online proctoring was invasive and creepy.  There was undoubtedly an element of the ‘fear of the unknown’ here, but equally many valid and understandable anxieties.

In this session we will discuss the issues this episode has thrown up and explore ways in which we can ensure the candidate remains at the heart of the test experience, as a willing and informed participant.

Issues including:
• Communicating the case for proctored testing
• The role and status of professional proctors
• The use and limitations of AI
• The imperative of communication and consent
• Candidates rights, privacy and fair access
• Psychometric validity

Presenter:

Doug Winneg

About:

Advances in technology, digitalization and automation have led to changes in the workplace, with increasing importance being placed on inter- and intra-personal skills. Similarly, in education, the movement towards addressing 21st century skills has led to a broadening of focus from basic literacy and numeracy, to include noncognitive competencies such as collaboration, communication, and creativity.  Now with the pandemic, a seismic shift has taken place resulting in remote, virtual work and learning environments and new models and modes of social interaction. What competencies and characteristics will be important in the post pandemic era?  How will they be assessed and developed?  How will these assessments be used?  What are the opportunities for organizations that acquire and develop talent?

A diverse and seasoned panel of assessment experts representing European and US perspectives will examine models of personality and emotional intelligence and related assessment tools, and will explore the expected shift in importance of certain noncogntive characteristics, as well as changes in the uses and value of assessment in the post pandemic era.

Presenters: 

John Weiner, PSI

Dimmy Van Zanten 

Dan Hughes 

Maria Antoniou

Breakout Sessions

About: Classifying items to incorrect content domains impacts subsequent exam development activities and the integrity of an exam. Item classification is an integral part of multiple stages in the exam development life cycle and can be as simple as creating a crosswalk document for a programmer or as complex as facilitating a subject matter expert's (SME) review of an entire item bank. The impact of a job analysis on item classifications and the consequences of item misclassifications on exam form development can be easily overlooked when credentialing organizations are focused on expediting exam development. The presenters will discuss antecedents to item classification and reclassification; consequences of misclassifying items; methods for facilitating SME item classification activities; and contexts within which item classifications occur and their relationships with other exam development activities. Lessons learned through practical experiences from three different perspectives will be shared.

Presenters:

Daniel Breidenbach | PSI Services
Shawn Amaker | American Nurses Credentialing Center
Pamela Ing Stemmer | Professional Testing, Inc.


Breakout Sessions

About: In a candidate-centric market, organizations desire to use the recruitment and hiring process as a means to attract top talent. One way to accomplish this is to offer an efficient, accessible, and positive candidate experience through the use of new technologies, such as interactive elements and mobile-delivered assessment. Panelists representing five assessment organizations will share their perspectives and predictions on the topic of candidate experience.
Topics addressed will include:
What features have assessment publishers already established as important to the candidate/examinee experience?
What are the most recent asks from our assessment consumers? Sharing insights based on the“voice of the customer.”
Candidate feedback as a mechanism to enhance the candidate experience.
Evolving metrics: What can assessment publishers do to help their clients track and report on candidate experience to satisfy a variety of stakeholders (operations, recruiters, senior leaders,etc.)?

Presenters:
John Weiner | PSI Services
Cari Gardner | HireVue
Benjamin Granger | Qualtrics
Ken Lahti | SHL
Sara Gutierrez | SHL


About:

Advances in technology, digitalization and automation have led to changes in the workplace, with increasing importance being placed on inter- and intra-personal skills. Similarly, in education, the movement towards addressing 21st century skills has led to a broadening of focus from basic literacy and numeracy, to include noncognitive competencies such as collaboration, communication, and creativity.  Now with the pandemic, a seismic shift has taken place resulting in remote, virtual work and learning environments and new models and modes of social interaction. What competencies and characteristics will be important in the post pandemic era?  How will they be assessed and developed?  How will these assessments be used?  What are the opportunities for organizations that acquire and develop talent?

A diverse and seasoned panel of assessment experts representing European and US perspectives will examine models of personality and emotional intelligence and related assessment tools, and will explore the expected shift in importance of certain noncogntive characteristics, as well as changes in the uses and value of assessment in the post pandemic era.

Presenters: 

John Weiner, PSI

Dimmy Van Zanten 

Dan Hughes 

Maria Antoniou

About: A diverse panel representing two, large credential programs and two assessment service
providers will draw from their real-world experience and research to present and discuss
advances in technology-enabled test design, performance-based assessment, and data forensics to provide enhanced security.

Presenters:
John Weiner | PSI Services
Phillip Dickison | NCLEX
David Foster | Caveon
Liberty Munson | Microsoft

The nature of work has been rapidly evolving in recent years with advances in technology and the digitalization and automation of routine tasks and processes. Even before the pandemic, millions of jobs have been displaced by automation, particularly in manufacturing, but now also in areas ranging from finance to recruitment and customer service. The trajectory of change has been rapidly accelerated by the recent pandemic and its effects on the economy, the job market, and the now-common virtual workplace. These changes and disruptions have created a number of challenges for organizations and the people affected. One result we are seeing is a surge in outboarding and reskilling initiatives to enable people and organizations to adapt to the new world of work.


Here today to discuss these trends and strategies, we have assembled a panel of experts in economics, industrial psychology, and work placement and outplacement. The panel will discuss key questions regarding these issues, such as:


• What are the trends in the displacement of jobs near and longer term? What are some examples of jobs that have been displaced?
• What is the impact on the competencies required for individuals to succeed in future jobs?
• How are organizations using outplacement services to help employees adapt to the new economy?
• What are some examples of re-skilling initiatives?
• Can people and organizations “future proof” their skill sets? What types of jobs will be least likely to be displaced by automation?

Presented By:
John Weiner (Moderator), Chief Science Officer, PSI
Dr. Alexander Alonso, Chief Knowledge Officer, SHRM
Angie College, Senior Vice President, Adecco North America
Dr. Elena Simintzi, University of North Carolina, Kenan-Flagler Business School

About: How compatible are learning sciences and psychometrics? Both fields make inferences about candidate cognition based on manifested behaviors. However, they diverge in how they conceptualize the design and use of assessments. The aim of this presentation is to highlight fundamental differences in the two paradigms, suggest ways these differences can be addressed, as well as propose implications and opportunities in the landscape of assessment.

Presenter: Natalie Jorion, PSI

September 15th

About: For those working in the assessment industry, there are many competing demands that require constant attention. In many scenarios, these include demands on the time required to complete a project, the financial requirements of the project, and the need to develop or maintain assessments that are consistent with professional standards. During this session, a panel of seasoned measurement and educational technology professionals will discuss scenarios that require excellent judgment and experience to determine how best to meet these competing demands. Come and join us for this session and jump in and share your experiences attempting to juggle requirements to meet professional standards within the practical realities of the world.

Presenters:

Rory McCorkle

Amy Schmidt

Ye Tong

Brian Kelly

About: Automated Item Generation (AIG) is a process that uses cognitive and psychometric theories to produce hundreds of items from a single-item template utilizing computer technology. AIG may be the answer to the never-ending need for high-quality test items, but a good foundation is needed to properly design a model to generate and prepare these items for practical use. This session will explore how to effectively map and create AIG items; demonstrate the construction of AIG items through an intuitive, graphical user interface; provide examples for group discussion and activities; discuss how to implement these items in testing programs at a practical level; and present the results and findings regarding our strategies and tips for implementing AIG.

Presenters:

Brooke Dresden, PSI

Xin Li, PSI

Dennis Byrne, FAA

About: This session will focus on the rapid pace of technological innovation in assessment and the extent to which it may increase diversity and fairness, or create additional challenges.  The assessment landscape has changed significantly during the past decade in both employment and education.  Advances in technology capabilities; increased availability of technology in personal devices, in homes and in schools; the economic efficiencies introduced by expanding technology and the applications of Artificial Intelligence/Machine Learning are just some of the conditions that have stimulated different expectations by the consumers of assessments. Employers and educational policy leaders, who drive much of the market demand in assessment, increasingly call for assessments that are more engaging, more flexible, and focused on the test taker experience, as well as assessments that can be delivered on demand with one’s own device on a variety of platforms.   As technology assumes a central role in the design, development, delivery, and scoring of tests, and the reporting of test scores, new issues emerge that can threaten both the fairness and validity of test scores, as well as new opportunities that may increase access and construct-relevance to assessments. The presenters will draw from their extensive experience in high-stakes assessment to contrast the demand for innovation and flexibility in testing with the traditional requirements of standardization, replicability, and score comparability.

 

Presenters:

John Weiner

Wayne Camara

Nancy Tippins

About: The COVID-19 crisis brought about the most sudden and dramatic shift in test delivery that we have ever witnessed.  Forced with a binary choice of moving testing online or stop testing, many organisations chose the former and moved with creditable agility to adopt online proctoring in weeks rather than, more typically, months or years.

Consequently, the spotlight was shone on an aspect of high-stakes assessment that had hitherto seldom received pubic attention – and the coverage was, by no means, all positive.  There was criticism of specific failures to prevent widespread malpractice and questions whether the model provided a level playing field for candidates testing online in differing circumstances, and versus cohorts taking the test onsite.

But the strongest voices were from the candidates themselves, who cited lack of alternatives (to online proctored tests), significant privacy concerns including data usage and remote access to devices, and the charge that online proctoring was invasive and creepy.  There was undoubtedly an element of the ‘fear of the unknown’ here, but equally many valid and understandable anxieties.

In this session we will discuss the issues this episode has thrown up and explore ways in which we can ensure the candidate remains at the heart of the test experience, as a willing and informed participant.

Issues including:
• Communicating the case for proctored testing
• The role and status of professional proctors
• The use and limitations of AI
• The imperative of communication and consent
• Candidates rights, privacy and fair access
• Psychometric validity

Presenter:

Doug Winneg

About: The job analysis is the foundation of any testing program. Its ability to help define the purpose and intention of a credentialing exam is well supported, and the building blocks are well known. The benefits of carrying out a job analysis are many, but some are more obvious than others. Besides the obvious benefit of establishing content typically in the form of an exam content outline, there are benefits that a job analysis can offer that are sometimes lost, ignored, or skipped over.

One of the main outputs of the job analysis, the content outline, can be used to help inform different aspects of certification programs. Furthermore, the job analysis also provides an opportunity to help “future-proof” exam content.

This session will discuss and provide examples of these additional benefits, and how some credentialing bodies have used small changes in their process to open up the full advantages of a job analysis.

Presenter:

Cyrus Mirza, PSI

Katherine Church, Certifying Board for Dietary Managers

September 16th

TBD

About: The job analysis is the foundation of any testing program. Its ability to help define the purpose and intention of a credentialing exam is well supported, and the building blocks are well known. The benefits of carrying out a job analysis are many, but some are more obvious than others. Besides the obvious benefit of establishing content typically in the form of an exam content outline, there are benefits that a job analysis can offer that are sometimes lost, ignored, or skipped over.

One of the main outputs of the job analysis, the content outline, can be used to help inform different aspects of certification programs. Furthermore, the job analysis also provides an opportunity to help “future-proof” exam content.

This session will discuss and provide examples of these additional benefits, and how some credentialing bodies have used small changes in their process to open up the full advantages of a job analysis.

Presenter:

Cyrus Mirza, PSI

Katherine Church, Certifying Board for Dietary Managers

September 17th

About: A diverse panel representing two, large credential programs and two assessment service
providers will draw from their real-world experience and research to present and discuss
advances in technology-enabled test design, performance-based assessment, and data forensics to provide enhanced security.

Presenters:
John Weiner | PSI Services
Phillip Dickison | NCLEX
David Foster | Caveon
Liberty Munson | Microsoft

About: This session will focus on the rapid pace of technological innovation in assessment and the extent to which it may increase diversity and fairness, or create additional challenges.  The assessment landscape has changed significantly during the past decade in both employment and education.  Advances in technology capabilities; increased availability of technology in personal devices, in homes and in schools; the economic efficiencies introduced by expanding technology and the applications of Artificial Intelligence/Machine Learning are just some of the conditions that have stimulated different expectations by the consumers of assessments. Employers and educational policy leaders, who drive much of the market demand in assessment, increasingly call for assessments that are more engaging, more flexible, and focused on the test taker experience, as well as assessments that can be delivered on demand with one’s own device on a variety of platforms.   As technology assumes a central role in the design, development, delivery, and scoring of tests, and the reporting of test scores, new issues emerge that can threaten both the fairness and validity of test scores, as well as new opportunities that may increase access and construct-relevance to assessments. The presenters will draw from their extensive experience in high-stakes assessment to contrast the demand for innovation and flexibility in testing with the traditional requirements of standardization, replicability, and score comparability.

 

Presenters:

John Weiner, PSI Services

Wayne Camara, ACT

Nancy Tippins, The Nancy Tippins Group

September 18th

The unusual circumstances surrounding the closure of testing centers as well as the shortage of online proctors for high-stakes testing should cause us to wonder about proctoring and what the future might hold for humans in that role.

Is the complete replacement of proctors with enhanced-capability technology a feasible future?

Should proctors go the way of switchboard operators and computers (the women employed by NASA in the last century to calculate trajectories and landing coordinates)? Or is some small part of the human contribution a critical requirement of test monitoring?

This session will explore and debate this concept. We’ll:

• Compare and contrast the security of both online and on-site proctoring.
• Discuss possible technology to enhance the critical security roles of a proctor.
• Ask how technology might be able to take over the proctoring responsibilities completely, better serving a program’s security purposes.

Is it reasonable to assume that test security can actually be enhanced by eliminating the human element? We’ll debate this and so much more.

About: Not all testing needs are created equal - and in the case of some programs, they can be incredibly complex. In this session, PSI, TrueAbility, and one of our shared clients will discuss how we addressed their testing needs through using the strengths of both organizations. The session will describe the original goals of the program, how we went through a solutioning exercise to put together the right systems and solutions to meet the goals, and metrics of the program experience thus far. We will also discuss how the performance testing model of the program helped better ensure the skills of credential holders to hiring managers and organizations.

 
Presenters:
Rory McCorkle
Graham Livingstone

Share this page: