The “language of indirect coding” in the analysis of U.S. strategic documents: the limits of formalization, reproducibility, and validity

Abstract

The study aims to determine the methodological limits of the “language of indirect coding” (LIC), as applied by V. E. Boldyrev to the analysis of U.S. strategic documents, and the implications of these limitations for specific conclusions. The article examines the consistency of the transitions from source selection and text segmentation to codes, quantitative indicators, historical and political interpretations, and forecasts. The scientific novelty lies in a comprehensive test of the reproducibility and validity of these transitions, a mathematical reconstruction of the system stability coefficient (SSC), and a comparison of individual LIC conclusions with the underlying data. It is demonstrated that the SSC is algebraically reducible to 100/N and therefore does not measure the adaptability of subsystems; comparisons between the administrations of B. Obama and D. Trump based on this coefficient depend on the size of the coded corpus. It is shown that coincidences in codes do not, in themselves, confirm interagency interaction, and that similarities in interpretations do not establish the optimality of political development. A comparison of the forecast of J. Biden’s policy with the original U.S. document reveals insufficient justification for treating the areas of U.S. – China cooperation listed therein as an exhaustive list and does not allow the independent predictive value of the LIC to be established. Consequently, the LIC should be regarded primarily as a system for structured annotation and hypothesis generation. The measurement, explanatory, and predictive validity of the applications under consideration requires independent confirmation, while the SSC should be excluded from the category of substantively interpretable indicators.

Research materials

1. Болдырев В. Е. Обеспечение безопасности США: трансформация интерпретаций и межведомственных взаимодействий в 2009-2020 гг. Владивосток: Институт истории, археологии и этнографии народов Дальнего Востока Дальневосточного отделения Российской академии наук, 2021a.
2. Болдырев В. Е. Перспективы политики Джозефа Байдена в отношении Восточной Азии // Анализ и прогноз. Журнал Института мировой экономики и международных отношений Российской академии наук. 2021b. № 3. https://doi.org/10.20542/afij-2021-3-30-40
3. Болдырев В. Е. Эволюция политики США в отношении Северо-Восточной Азии в период президентства Барака Обамы: процесс, механизмы, результаты: дисс. … д. ист. н. Владивосток, 2025.
4. The White House.Interim National Security Strategic Guidance. Washington, 2021. https://bidenwhitehouse.archives.gov/wp-content/uploads/2021/03/NSC-1v2.pdf

References

1. Adcock R., Collier D. Measurement Validity: A Shared Standard for Qualitative and Quantitative Research // American Political Science Review. 2001. Vol. 95. Issue 3. https://doi.org/10.1017/S0003055401003100
2. Grimmer J., Stewart B. M. Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts // Political Analysis. 2013. Vol. 21. Issue 3. https://doi.org/10.1093/pan/mps028
3. Krippendorff K. Reliability in Content Analysis: Some Common Misconceptions and Recommendations // Human Communication Research. 2004. Vol. 30. Issue 3. https://doi.org/10.1111/j.1468-2958.2004.tb00738.x
4. Maier D., Niekler A., Wiedemann G., Stoltenberg D. How Document Sampling and Vocabulary Pruning Affect the Results of Topic Models // Computational Communication Research. 2020. Vol. 2. Issue 2. https://doi.org/10.5117/CCR2020.2.001.MAIE
5. Process Tracing: From Metaphor to Analytic Tool / ed. by A. Bennett, J. T. Checkel. Cambridge: Cambridge University Press, 2014. https://doi.org/10.1017/CBO9781139858472
6. Roberts D. R., Bahn V., Ciuti S., Boyce M. S., Elith J., Guillera-Arroita G., Hauenstein S., Lahoz-Monfort J. J., Schröder B., Thuiller W., Warton D. I., Wintle B. A., Hartig F., Dormann C. F. Cross-validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure // Ecography. 2017. Vol. 40. Issue 8. https://doi.org/10.1111/ecog.02881
7. Romer P. M. Mathiness in the Theory of Economic Growth // American Economic Review. 2015. Vol. 105. Issue 5. https://doi.org/10.1257/aer.p20151066
8. Ward M. D., Greenhill B. D., Bakke K. M. The Perils of Policy by P-value: Predicting Civil Conflicts // Journal of Peace Research. 2010. Vol. 47. Issue 4. https://doi.org/10.1177/0022343309356491
9. Ying L., Montgomery J. M., Stewart B. M. Topics, Concepts, and Measurement: A Crowdsourced Procedure for Validating Topics as Measures // Political Analysis. 2022. Vol. 30. Issue 4. https://doi.org/10.1017/pan.2021.33

Funding

The article was prepared within the framework of a state assignment on the topic “The Asia-Pacific Region amid the Competition between the West and the Greater East for the Formation of a New World Order. FWMU-2024-0002”.

Author information

About this article

Publication history

• Received: August 19, 2026.
• Published: November 10, 2026.

Keywords

• язык непрямого кодирования
• воспроизводимость кодирования
• измерительная валидность
• внешняя валидация
• количественный анализ текста
• language of indirect coding
• coding reproducibility
• measurement validity
• external validation
• quantitative text analysis

Copyright

© 2026 The Author(s)
© 2026 Gramota Publishing, LLC

User license

Creative Commons Attribution 4.0 International (CC BY 4.0)