Магистратура
2026/2027
Искусственный интеллект в психометрике
Статус:
Курс по выбору (Обучение и оценивание как наука)
Кто читает:
Департамент образовательных программ
Где читается:
Институт образования
Когда читается:
2-й курс, 3 модуль
Охват аудитории:
для всех кампусов НИУ ВШЭ
Язык:
английский
Кредиты:
3
Контактные часы:
28
Course Syllabus
Abstract
Курс посвящен применению методов искусственного интеллекта в психометрике. Мы рассмотрим, как ИИ используется для автоматической генерации заданий, оценки открытых ответов и анализа текстов. Особое внимание будет уделено вопросам валидности и надежности автоматизированных оценок, а также ограничениям и перспективам использования ИИ в образовательных измерениях
Learning Objectives
- To equip students with a comprehensive understanding of the capabilities and limitations of AI in assessment, as well as the practical skills to apply these technologies across various stages of the assessment cycle: item generation → scoring → feedback provision.
Expected Learning Outcomes
- Designs agent-based architectures (agentic frameworks) for automated item generation and automated item quality evaluation
- Applies automated item generation methods, including multimodal approaches
- Employs diverse prompting techniques for automated item generation (AIG), selecting appropriate strategies aligned with specific assessment objectives
- Identifies the applications of AI in assessment and account ethical considerations and associated risks
- Formulates effective prompts for automated scoring purposes
- Compares various approaches to the automated scoring of constructed responses based on criteria such as accuracy, processing speed, interpretability, and data requirements
- Applies supervised and unsupervised methods for the automated scoring of constructed-response items
- Designs, generates, and empirically validates LLM-driven feedback integrated with psychometric data, utilizing low-code environments (both with and without API integration) without requiring machine learning engineering
Course Contents
- Use of AI in Assessment
- Technical Aspects of Utilizing Large Language Models in Assessment.
- Automated Item Generation (AIG): The evolution from template-based approaches to generative AI.
- Multimodal Items, Agentic Generation, and Automated Review
- Automated Scoring of Constructed-Response Items Using Prompting
- Automated Scoring of Short-Answer Items: Comparing ML approaches and decision-making
- LLM-Based Feedback: Design, generation, and validation
Assessment Elements
- Automated Item Generation (AIG)
- Automated Scoring
- Final Examination ProjectIn groups of 3–4, students will develop and defend a project proposal for the creation of an AI-based assessment product. The proposal must be submitted as a structured document (Word format, 3–5 pages) adhering to the specified framework below—ranging from the product title to an analysis of the team’s skill gaps. The project must demonstrate the application of all concepts covered in the course (prompt engineering, automated item generation, automated scoring, and/or AI-based feedback) and provide a robust justification for the necessity of an AI-driven approach. Project Structure (items): Short title of your AI product. Summarize the proposed development in 2–3 sentences: what specific problem is being solved and how? Attach a brief overview of the solution in the form of a text file or presentation. What specific underlying problem does your proposal address? Provide evidence-based statements confirming the existence and significance of the aforementioned problem. Do analogous solutions exist? What are their respective strengths and limitations? Why is artificial intelligence necessary to solve this problem? What constitutes the unique value proposition of your proposed AI product? Who are the target users of your AI product? Describe the potential distribution channels or go-to-market strategies for your product. Who are the members of your team (specify roles and assign them to actual team members)? What competencies are currently lacking within the team to successfully develop and distribute the AI product? Submission format: A written document accompanied by an oral defense (presentation) lasting 5–7 minutes, followed by a Q&A session with the examination committee.
Interim Assessment
- 2026/2027 3rd module0.3 * Automated Item Generation (AIG) + 0.3 * Automated Scoring + 0.4 * Final Examination Project
