Psychometrics Lead - Research and Analysis

About Touchstone Institute

Touchstone Institute is a non-profit corporation governed by a Board of Directors and receives financial support from the Government of Ontario, in addition to generating revenue through direct business to business contracts seeking competency assessment and learning solution services. 

It’s the largest assessment centre of its kind in Canada, providing professional competency assessment and learning programs to support the delivery of high-quality health care. Touchstone Institute works closely with physicians, nurses and other healthcare providers who provide subject matter expertise to assess and improve the level of competency across individual practice.

Touchstone Institute has become an integral part of the commitment to internationally trained health professionals seeking entry into training and practice in Ontario and has expanded to provide assessment, certification, and quality assurance programs for practitioners across Canada.

Position Description

The Psychometrics Lead is responsible for the objective measurement of skills, knowledge, and competencies through the design, construction and validation of assessment instruments such as questionnaires, and performance exams. The Psychometrics Lead will support the design and oversight of quality assurance activities at Touchstone Institute that will be informed by evidence and research.

The Psychometrics Lead provides leadership and guidance to all program areas on design, analysis, reporting and interpretation of exams and assessments at Touchstone Institute.

Key Accountabilities

Assessment Development & Validation

Works closely with the Senior Director, Assessment and Senior Director of Program Design & Implementation and relevant SMEs to develop and operationalize clinical and written examinations and assessments that are deemed appropriate and psychometrically defensible

Create examination and assessment blueprints based on feedback and content provided by SMEs

Develops and leads workshops collaborating with SMEs in exam and assessment activities

Validates the process for developing exam questions and cases in consultation with SMEs and Lead Examiners, ensuring that all development is in line with industry standards and best practices

Develops a standardized and consistent method for assessing and scoring candidates

Ensures exams and assessments comply with industry standards (e.g. AERA, APA, NCME guidelines)

Serves as a key resource to the Assessment Oversight Committee, providing technical expertise and evidence-based recommendations.

Prepares materials, presents findings, and contributes to governance and decision-making related to assessment strategy and integrity.

Quality Assurance and Quality Improvement

Continually oversees the review of examinations, scoring systems and methodologies to ensure ongoing psychometric integrity

Leads the development of quality assurance standards and procedures that align with industry best practices

Collaborates with cross-functional teams inside the organization to turn data into ideas/insights for organizational growth

Maintains documentation/records of QA activities, audit findings, and corrective actions to support transparency and accountability

Data Analysis & Interpretation

Analyzes assessment data to generate insights that inform program design, candidate selection, and performance evaluation.

Analyzes assessment data to generate insights that inform program design, candidate selection, and performance evaluation.

Prepares technical reports and presentations for internal and external stakeholders.

Develops opportunities to integrate research and quality assurance reports to inform and support strategies and demonstrate effectiveness in measurement

Support Building networks and accessing expert advisors

Position Requirements

Knowledge

Superior understanding of exam theory, learning theory, and assessment methodologies

Strong understanding of measurement theory and methods

Knowledge of relevant health profession terminology regarding competency-based assessment

Demonstrated expertise in examination development and test analysis

Good judgment, critical thinking skills, and close attention to detail applied in this field

Skills

Strong time management skills and ability to prioritize multiple assignments and work demands

Strong analytical and report writing skills and the ability to translate statistical information for clients and regulatory bodies

Excellent verbal and written communication skills

Strong conceptual, analytical and problem-solving skills

Well-developed critical thinking skills to identify key issues and propose solutions to program setbacks

Strong interpersonal and collaboration skills to work with others to coordinate information

Strong data visualization skills, and comfort applying them in new and innovative ways

Strong leadership skills: experience managing a team of analyst staff is an asset

Proficiency in the use of word processing, and presentation software (e.g. Word, Excel, MS Project and PowerPoint)

Abilities

Ability to deal with emergent issues in a professional and timely manner

Ability to work independently or as part of a team, with a range of clients at all organizational levels

Demonstrated ability to process multiple types of information, perform multiple tasks simultaneously, and prioritize

Qualifications

Master’s degree in Statistics, Biostatistics, Quantitative Psychology, Developmental Psychology and Education, or other equivalent degrees

At least 5 years of experience in an educational or testing environment

Related educational assessment and measurement experience as well as experience in test development and analysis

What We Offer

A hybrid, collaborative, and team-oriented work environment

Comprehensive health and dental benefits and a health spending account

3 weeks’ vacation to start, plus additional personal days

Extended office closure during the holidays

Staff appreciation events

Notes for Candidates

This is an existing vacancy within the organization. 

Touchstone Institute does not use artificial intelligence (AI) tools as part of the applicant screening process. Applications will be reviewed by members of our recruitment and hiring teams to ensure a fair and thorough assessment.

The salary range considers many factors including, but not limited to experience, education, and training, etc. It is not typical for the salary to be offered near the top of the range, it is dependent on numerous factors. 

Touchstone Institute is dedicated to employment equity and fostering a diverse work environment. We encourage applications from Indigenous Peoples, racialized persons, persons with disabilities, and other equity-seeking groups. We believe that a diverse team is essential to understanding and serving the evolving needs of the healthcare sector. 

We follow a hybrid work model with two designated in-office days: Tuesdays and Wednesdays.
Successful candidates must be available to work on-site during these days at 145 Wellington St W Suite 600, Toronto, ON M5J 1H8 with or without accomodation. 

All applications should be submitted through our application tracking system via Bamboo at: https://tsin.bamboohr.com/careers/95. 

Application closing date: Februrary 27, 2026

Touchstone Institute is an equal-opportunity employer that is committed to fostering a diverse work environment and exercises accessible employment practices. In accordance with the Ontario Human Rights Code, Accessibility for Ontarians with Disabilities Act (AODA) and Touchstone Institute’s policies, a request for accommodation will be accepted in all aspects of our hiring process.  If you anticipate needing any type of accommodation during the recruitment process, please email [email protected] in advance of your appointment.