Методологические эксперименты в социальных науках

2026/2027

Статус: Маго-лего

Кто читает: Департамент социологии

Где читается: Факультет социальных наук

Когда читается: 2 модуль

Охват аудитории: для всех кампусов НИУ ВШЭ

Преподаватели: Лебедев Даниил Вадимович

Язык: английский

Кредиты: 3

Контактные часы: 28

Full Syllabus Ask Question

Abstract

Social science methodology is a well-established and evolving research field concerned with evaluating and improving the tools and strategies used in data collection, analysis, and interpretation. Among its key approaches is the use of experiments to study the performance of different methodological choices in real or simulated research settings. This course focuses on the conceptual and applied aspects of methodological experimentation, with a particular emphasis on its role within the broader context of quantitative social science. Students will examine the principles of experimental design (including A/B testing), typologies of experiments, ethical considerations, practice of conducting own experiments and analytical approaches to interpreting experimental results. The course will also cover key applications of experiments in social science survey methodology: testing modes of data collection, evaluating questionnaire design, assessing response enhancement strategies, improving survey quality, and integrating auxiliary data. This course is particularly relevant for students interested in conducting their own methodological experiments or working in the field of social science survey methodology. It is also valuable for those pursuing applied research in the quantitative tradition of social sciences. Students will develop a solid understanding of experimental logic, gain practical experience in designing and analysing methodological experiments, and critically assess core research questions in contemporary quantitative methodology.

Learning Objectives

To equip students with a single, transferable command of experimental methodology - the ability to design, execute, analyse, and critically evaluate controlled experiments - using survey methodology and UX/UI A/B testing as the two domains in which that logic is tested and applied.

Expected Learning Outcomes

Define and recall the core apparatus of experimental methodology - potential outcomes, random assignment, estimand, validity typology, total survey error, OEC
Explain how experimental logic evaluates methodological choices, and translate a design between the survey and product-research domains
Analyze heterogeneous effects, multiplicity, and threats to validity in a given design or dataset
Evaluate the trustworthiness and ethical defensibility of an experiment, and justify a decision on that basis
Apply randomization checks, effect estimation, power analysis, and regression with interactions to experimental data in R
Create an original preregisterable design, or an original reproducible analysis of experimental data

Course Contents

Foundations of experimental logic
Designing the comparison: randomization and treatment construction
Outcomes, measurement, and threats to validity
Research ethics, consent, and preregistration
Beyond the average effect: binary outcomes, heterogeneity, and multiplicity
Estimation and design sizing in R
Trustworthiness diagnostics and reproducible reporting

Assessment Elements

Activity
Midterm test
Online, timed (~50 min). Mix of multiple-choice, short-answer, open-ended and one applied item. Covers experimental logic, survey experiment typologies, validity, ethics, and reading content from first part of the course.
Final assessment
Students choose either track. Both are designed for equivalent workload and graded on parallel criteria. Track A - Group Design Proposal (groups of up to 4). A ~3,000–4,000-word preregisterable proposal for an original methodological experiment (survey experiment or A/B test): motivation and hypotheses, design and randomization scheme, sample-size/power justification, planned analysis (with R code skeleton), threats to validity, and an ethics/consent plan. Includes a short synchronous group presentation (10 minutes). Track B - Individual Data-Analysis Report. A ~2,500–3,500-word reproducible Quarto/R Markdown report analyzing a real or realistic experimental dataset: randomization checks, treatment-effect estimation, effect sizes with uncertainty, at least one interaction/heterogeneous-effect analysis, multiple-comparison handling, and a trustworthiness/limitations section. Topics will be provided with data but can also be chosen by students themselves with mandatory confirmation from the instructor. Includes a short group presentation (10 minutes).

Interim Assessment

2026/2027 2nd module
0.35 * Final assessment + 0.3 * Midterm test + 0.35 * Activity

Bibliography

Recommended Core Bibliography

Experimental and quasi-experimental designs for generalized causal inference, Shadish, W. R., 2002
Field experiments : design, analysis, and interpretation, Gerber, A. S., 2012
International handbook of survey methodology, Leeuw de, E. D., 2008
Questions and answers in attitude surveys : experiments on question form, wording, and context, Schuman, H., 1996
Research design in the social sciences : declaration, diagnosis, and redesign, Blair, G., 2023
Survey methodology, Groves, R. M., 2004
The Effect: An Introduction to Research Design and Casualty, Huntington-Klein, N., 2022

Recommended Additional Bibliography

Cambridge handbook of experimental political science, , 2011
R for data science : import, tidy, transform, visualize, and model data, Wickham, H., 2017
The science of web surveys, Tourangeau, R., 2013

Authors

Lebedev Daniil Vadimovich

Course Syllabus