Email this article A method for automated assessment of the reliability of alternative statements in a collection of scientific articles using the example of the topic “Overton windows”
- Authors: Charnine M.M.1, Somin N.V.1
-
Affiliations:
- Federal Research Center “Computer Science and Control”, RAS
- Issue: No 1 (2024)
- Pages: 118-128
- Section: Analysis of Textual and Graphical Information
- URL: https://journal-vniispk.ru/2071-8594/article/view/269807
- DOI: https://doi.org/10.14357/20718594240110
- EDN: https://elibrary.ru/YJICCC
- ID: 269807
Cite item
Full Text
Abstract
The paper proposes a method for assessing the reliability of opposing statements/facts based on trends in bibliographic data, provides an example of its use, and discusses the possibility of automating the method and replenishing the fact base. As an example, 1047 articles from the scientific eLibrary containing the words “window” and “Overton” were analyzed. Using the proposed method, it is shown that “working technology” and “pseudo-scientific concept” are alternative points of view on “Overton windows”. It is also shown that the “working technology” point of view is more reliable.
Full Text
About the authors
Michael M. Charnine
Federal Research Center “Computer Science and Control”, RAS
Author for correspondence.
Email: mc@keywen.com
Candidate of technical sciences, Senior researcher
Russian Federation, MoscowNikolay V. Somin
Federal Research Center “Computer Science and Control”, RAS
Email: chri-soc@yandex.ru
Candidate of physical and mathematical sciences, Leading researcher
Russian Federation, MoscowReferences
- Anufrieva D.Yu., Guzenko A.Yu. Soderzhaniye ponyatiya "kriticheskoye otsenivaniye informatsii" [Content of the concept of “critical assessment of information”] // Problemy sovremennogo pedagogicheskogo obrazovaniya [Problems of modern pedagogical education]. 2017. No. 56-1. P. 3-9.
- Vorontsov K.V. Tekhnologii iskusstvennogo intellekta protiv feykov, postpravdy i informatsionnykh voyn [Artificial intelligence technologies against fakes, post-truth and information wars] // Zasedaniye Ekspertnogo soveta po razvitiyu tsifrovoy ekonomiki, tekhnologiy i innovatsiy Molodozhnogo parlamenta pri Gosudarstvennoy dume Federal'nogo Sobraniya RF po teme «Protivodeystviye destruktivnoy ideologii (natsizm, terrorizm, kriminal)». [Meeting of the Expert Council for the development of the digital economy, technology and innovation of the Youth Parliament under the State Duma of the Federal Assembly of the Russian Federation on the topic “Countering destructive ideology (Nazism, terrorism, crime).”] February 17, 2023. Electronic resource. URL: https://rutube.ru/video/8628f049a17ae40e95b8aa33cbfd5b 69/ (accessed 05.10.2023).
- Efremov Yu.N. Lzhenauka i gipoteza [Pseudoscience and hypothesis] // Byulleten' «V zashchitu nauki». Komissiya RAN po bor'be s lzhenaukoy. [Bulletin “In Defense of Science”. RAS Commission on Combating Pseudoscience]. 2010. No. 7. P. 51-58.
- Melnikov O.N. Retsenzirovaniye kak faktor povysheniya kachestva nauchnoy informatsii [Peer review as a factor in improving the quality of scientific information] // Nauchnaya periodika: problemy i resheniya [Scientific periodicals: problems and solutions]. 2011. No. 2 (2). P. 20-23.
- Arazy O., Kopak R. On the measurability of information quality //Journal of the American Society for Information Science and Technology. 2011. V. 62. No. 1. P. 89-99.
- Google. Search Quality Evaluator Guidelines // Google, Mountain View, CA, USA, 2018.
- Hanselowski A., Stab C., Schulz C., Li Z., Gurevych I. A richly annotated corpus for different tasks in automated fact-checking // arXiv preprint arXiv:1911.01214. October 2019. Electronic resource. URL: https://arxiv.org/pdf/1911.01214.pdf (accessed 05.10.2023).
- Sarabadani A., Halfaker A., Taraborelli D. "Building automated vandalism detection tools for Wikidata" // Proc. 26th Int. Conf. World Wide Web Companion (WWW Companion), P. 1647-1654, 2017.
- Sugandhika C., Ahangama S. Assessing Information Quality of Wikipedia Articles Through Google’s E-A-T Model // IEEE Access, V. 10, P. 52196-52209, 2022, doi: 10.1109/ACCESS.2022.3172962.
- Thorne J., Vlachos A., Christodoulopoulos C., Mittal A. FEVER: a large-scale dataset for fact extraction and VERification // arXiv preprint arXiv:1803.05355. December 2018. Electronic resource. URL: https://arxiv.org/pdf/1803.05355.pdf (accessed 05.10.2023).
- Wadden D., Lin S., Lo K., Wang L.L., van Zuylen M., Cohan A. Hajishirzi H. Fact or fiction: Verifying scientific claims // Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics. 2020. P. 7534–7550.
- Wadden D., Lo K., Kuehl B., Cohan A., Beltagy I., Wang L.L., Hajishirzi H. SciFact-open: Towards open-domain scientific claim verification // arXiv preprint arXiv:2210.13777. October 2022. Electronic resource. URL: https://arxiv.org/pdf/2210.13777.pdf (accessed 05.10.2023).
- Zhang A. X. et al. A structured response to misinformation: Defining and annotating credibility indicators in news articles //Companion Proceedings of the The Web Conference 2018. 2018. P. 603-612. 1. Anufrieva D.Yu., Guzenko A.Yu. Soderzhaniye ponyatiya "kriticheskoye otsenivaniye informatsii" [Content of the concept of “critical assessment of information”] // Problemy sovremennogo pedagogicheskogo obrazovaniya [Problems of modern pedagogical education]. 2017. No. 56-1. P. 3-9.
Supplementary files
Supplementary Files
Action
1.
JATS XML
2.
Fig. 1. The percentage of articles from the “technology” and “pseudoscientific” groups in relation to the total number of articles in the “Overton windows” group
Download (27KB)
3.
Fig. 2. The percentage of articles from the “technology” and “pseudoscientific” groups (without review articles) in relation to the total number of articles from the “Overton windows” group
Download (24KB)
4.
Fig. 3. The 3-year moving average of the share of articles from the “patent” and “pseudoscience” groups (without review articles) as a percentage of the total number of articles from the “torsion fields” group
Download (22KB)
