Algorithmic Blind Spots in Digital Crisis Sensing
Human Validation of Social Media Sentiment on Indonesia's Free Nutritious Meals Program
Keywords:
Algorithmic Blind Spots, Digital Crisis Sensing, Human Validation, Sentiment Analysis, Free Nutritious Meals Program
Abstract
Digital social media monitoring offers governments a rapid means of detecting public concerns, but automated sentiment classification may overlook policy-crisis signals expressed in descriptive or context-dependent language. This study examines algorithmic blind spots in sentiment analysis of Indonesia’s Free Nutritious Meals (MBG) Program through a comparative content-analysis design combining automated classification with manual validation. The study used 100 high-engagement TikTok and Instagram video posts collected between January and June 2026. After duplicate removal, relevance screening, and ambiguity checks, 66 posts remained for analysis. Automated classification labelled 51 posts as neutral, 10 as positive, and five as negative, whereas manual validation identified 37 negative, 20 positive, and nine neutral posts. Exact agreement occurred in only 19 posts (28.79%). Notably, 29 of the 37 manually validated negative posts were automatically classified as neutral. The findings reveal a pattern of algorithmic neutralization, in which substantively critical events may lose their policy significance when expressed in factual or journalistic language. The study argues that effective digital crisis sensing requires a socio-technical approach that combines computational scale with structured human validation to ensure semantic relevance, contextual interpretation, risk recognition, and appropriate governmental response. This hybrid approach strengthens quality, accountability, and policy usefulness in government monitoring
References
Acikara, T., Xia, B., Yigitcanlar, T., & Hon, C. (2023). Contribution of Social Media Analytics to Disaster Response Effectiveness: A Systematic Review of Literature. Sustainability, 15(11), 8860. https://doi.org/10.3390/su15118860
Birjali, M., Kasri, M., & Beni-Hssane, A. (2021). A comprehensive survey on sentiment analysis: Approaches, challenges and trends. Knowledge-Based Systems, 226, 107134. https://doi.org/10.1016/j.knosys.2021.107134
Bulut, O., & Poth, C. N. (2022). Rapid assessment of communication consistency: sentiment analysis of public health briefings during the COVID-19 pandemic. AIMS Public Health, 9(2), 293–306. https://doi.org/10.3934/publichealth.2022020
Buzoianu, C., & Bîră, M. (2021). Using Social Media Listening in Crisis Communication and Management: New Methods and Practices for Looking into Crises. Sustainability, 13(23), 13015. https://doi.org/10.3390/su132313015
Das, R., & Singh, T. D. (2023). Multimodal Sentiment Analysis: A Survey of Methods, Trends, and Challenges. ACM Computing Surveys, 55(13s), 1–38. https://doi.org/10.1145/3586075
Duong, H.-T., & Nguyen-Thi, T.-A. (2021). A review: preprocessing techniques and data augmentation for sentiment analysis. Computational Social Networks, 8(1), 1. https://doi.org/10.1186/s40649-020-00080-x
Gandhi, A., Adhvaryu, K., Poria, S., Cambria, E., & Hussain, A. (2023). Multimodal sentiment analysis: A systematic review of history, datasets, multimodal fusion methods, applications, challenges and future directions. Information Fusion, 91, 424–444. https://doi.org/10.1016/j.inffus.2022.09.025
Gandy, L. M., Ivanitskaya, L. V, Bacon, L. L., & Bizri-Baryak, R. (2025). Public Health Discussions on Social Media: Evaluating Automated Sentiment Analysis Methods. JMIR Formative Research, 9, e57395. https://doi.org/10.2196/57395
Giest, S. N., & Klievink, B. (2024). More than a digital system: how AI is changing the role of bureaucrats in different organizational contexts. Public Management Review, 26(2), 379–398. https://doi.org/10.1080/14719037.2022.2095001
Gilardi, F., Alizadeh, M., & Kubli, M. (2023). ChatGPT outperforms crowd workers for text-annotation tasks. Proceedings of the National Academy of Sciences, 120(30). https://doi.org/10.1073/pnas.2305016120
Hartmann, J., Heitmann, M., Siebert, C., & Schamp, C. (2023). More than a Feeling: Accuracy and Application of Sentiment Analysis. International Journal of Research in Marketing, 40(1), 75–87. https://doi.org/10.1016/j.ijresmar.2022.05.005
Haug, N., Dan, S., & Mergel, I. (2024). Digitally induced change in the public sector: a systematic review and research agenda. Public Management Review, 26(7), 1963–1987. https://doi.org/10.1080/14719037.2023.2234917
He, L., Yin, T., & Zheng, K. (2022). They May Not Work! An evaluation of eleven sentiment analysis tools on seven social media datasets. Journal of Biomedical Informatics, 132, 104142. https://doi.org/10.1016/j.jbi.2022.104142
Julianto, I. T., & Nurpajar, D. S. (2025). User Sentiment Analysis X Towards Makan Bergizi Gratis Program Using Automatic Labeling Technique with Deepseek AI. Journal of Intelligent Systems Technology and Informatics, 1(2), 37–44. https://doi.org/10.64878/jistics.v1i2.43
Lohiniva, A.-L., Sibenberg, K., Austero, S., & Skogberg, N. (2022). Social Listening to Enhance Access to Appropriate Pandemic Information Among Culturally Diverse Populations: Case Study from Finland. JMIR Infodemiology, 2(2), e38343. https://doi.org/10.2196/38343
Madan, R., & Ashok, M. (2023). AI adoption and diffusion in public administration: A systematic literature review and future research agenda. Government Information Quarterly, 40(1), 101774. https://doi.org/10.1016/j.giq.2022.101774
Maragno, G., Tangi, L., Gastaldi, L., & Benedetti, M. (2023). Exploring the factors, affordances and constraints outline the implementation of Artificial Intelligence in public sector organizations. International Journal of Information Management, 73, 102686. https://doi.org/10.1016/j.ijinfomgt.2023.102686
Mergel, I., Dickinson, H., Stenvall, J., & Gasco, M. (2023). Implementing AI in the public sector. Public Management Review, 1–14. https://doi.org/10.1080/14719037.2023.2231950
Mikalef, P., Lemmer, K., Schaefer, C., Ylinen, M., Fjørtoft, S. O., Torvatn, H. Y., Gupta, M., & Niehaves, B. (2022). Enabling AI capabilities in government agencies: A study of determinants for European municipalities. Government Information Quarterly, 39(4), 101596. https://doi.org/10.1016/j.giq.2021.101596
Mosqueira-Rey, E., Hernández-Pereira, E., Alonso-Ríos, D., Bobes-Bascarán, J., & Fernández-Leal, Á. (2023). Human-in-the-loop machine learning: a state of the art. Artificial Intelligence Review, 56(4), 3005–3054. https://doi.org/10.1007/s10462-022-10246-w
Neumann, O., Guirguis, K., & Steiner, R. (2024). Exploring artificial intelligence adoption in public organizations: a comparative case study. Public Management Review, 26(1), 114–141. https://doi.org/10.1080/14719037.2022.2048685
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., … Moher, D. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ, n71. https://doi.org/10.1136/bmj.n71
Triningsih, E., Afdal, M., Permana, I., & Rozanda, N. E. (2025). Analisis Sentimen Terhadap Program Makan Bergizi Gratis Menggunakan Algoritma Machine Learning Pada Sosial Media X. Building of Informatics, Technology and Science (BITS), 6(4), 2240–2250. https://doi.org/10.47065/bits.v6i4.6534
van Atteveldt, W., van der Velden, M. A. C. G., & Boukes, M. (2021). The Validity of Sentiment Analysis: Comparing Manual Annotation, Crowd-Coding, Dictionary Approaches, and Machine Learning Algorithms. Communication Methods and Measures, 15(2), 121–140. https://doi.org/10.1080/19312458.2020.1869198
van Noordt, C., & Misuraca, G. (2022). Artificial intelligence for the public sector: results of landscaping the use of AI in government across the European Union. Government Information Quarterly, 39(3), 101714. https://doi.org/10.1016/j.giq.2022.101714
Verma, S. (2022). Sentiment analysis of public services for smart society: Literature review and future research directions. Government Information Quarterly, 39(3), 101708. https://doi.org/10.1016/j.giq.2022.101708
Villanueva-Miranda, I., Xie, Y., & Xiao, G. (2025). Sentiment analysis in public health: a systematic review of the current state, challenges, and future directions. Frontiers in Public Health, 13. https://doi.org/10.3389/fpubh.2025.1609749
Wankhade, M., Rao, A. C. S., & Kulkarni, C. (2022). A survey on sentiment analysis methods, applications, and challenges. Artificial Intelligence Review, 55(7), 5731–5780. https://doi.org/10.1007/s10462-022-10144-1
Zhu, L., Zhu, Z., Zhang, C., Xu, Y., & Kong, X. (2023). Multimodal sentiment analysis based on fusion methods: A survey. Information Fusion, 95, 306–325. https://doi.org/10.1016/j.inffus.2023.02.028
Zuiderwijk, A., Chen, Y.-C., & Salem, F. (2021). Implications of the use of artificial intelligence in public governance: A systematic literature review and a research agenda. Government Information Quarterly, 38(3), 101577. https://doi.org/10.1016/j.giq.2021.101577
Birjali, M., Kasri, M., & Beni-Hssane, A. (2021). A comprehensive survey on sentiment analysis: Approaches, challenges and trends. Knowledge-Based Systems, 226, 107134. https://doi.org/10.1016/j.knosys.2021.107134
Bulut, O., & Poth, C. N. (2022). Rapid assessment of communication consistency: sentiment analysis of public health briefings during the COVID-19 pandemic. AIMS Public Health, 9(2), 293–306. https://doi.org/10.3934/publichealth.2022020
Buzoianu, C., & Bîră, M. (2021). Using Social Media Listening in Crisis Communication and Management: New Methods and Practices for Looking into Crises. Sustainability, 13(23), 13015. https://doi.org/10.3390/su132313015
Das, R., & Singh, T. D. (2023). Multimodal Sentiment Analysis: A Survey of Methods, Trends, and Challenges. ACM Computing Surveys, 55(13s), 1–38. https://doi.org/10.1145/3586075
Duong, H.-T., & Nguyen-Thi, T.-A. (2021). A review: preprocessing techniques and data augmentation for sentiment analysis. Computational Social Networks, 8(1), 1. https://doi.org/10.1186/s40649-020-00080-x
Gandhi, A., Adhvaryu, K., Poria, S., Cambria, E., & Hussain, A. (2023). Multimodal sentiment analysis: A systematic review of history, datasets, multimodal fusion methods, applications, challenges and future directions. Information Fusion, 91, 424–444. https://doi.org/10.1016/j.inffus.2022.09.025
Gandy, L. M., Ivanitskaya, L. V, Bacon, L. L., & Bizri-Baryak, R. (2025). Public Health Discussions on Social Media: Evaluating Automated Sentiment Analysis Methods. JMIR Formative Research, 9, e57395. https://doi.org/10.2196/57395
Giest, S. N., & Klievink, B. (2024). More than a digital system: how AI is changing the role of bureaucrats in different organizational contexts. Public Management Review, 26(2), 379–398. https://doi.org/10.1080/14719037.2022.2095001
Gilardi, F., Alizadeh, M., & Kubli, M. (2023). ChatGPT outperforms crowd workers for text-annotation tasks. Proceedings of the National Academy of Sciences, 120(30). https://doi.org/10.1073/pnas.2305016120
Hartmann, J., Heitmann, M., Siebert, C., & Schamp, C. (2023). More than a Feeling: Accuracy and Application of Sentiment Analysis. International Journal of Research in Marketing, 40(1), 75–87. https://doi.org/10.1016/j.ijresmar.2022.05.005
Haug, N., Dan, S., & Mergel, I. (2024). Digitally induced change in the public sector: a systematic review and research agenda. Public Management Review, 26(7), 1963–1987. https://doi.org/10.1080/14719037.2023.2234917
He, L., Yin, T., & Zheng, K. (2022). They May Not Work! An evaluation of eleven sentiment analysis tools on seven social media datasets. Journal of Biomedical Informatics, 132, 104142. https://doi.org/10.1016/j.jbi.2022.104142
Julianto, I. T., & Nurpajar, D. S. (2025). User Sentiment Analysis X Towards Makan Bergizi Gratis Program Using Automatic Labeling Technique with Deepseek AI. Journal of Intelligent Systems Technology and Informatics, 1(2), 37–44. https://doi.org/10.64878/jistics.v1i2.43
Lohiniva, A.-L., Sibenberg, K., Austero, S., & Skogberg, N. (2022). Social Listening to Enhance Access to Appropriate Pandemic Information Among Culturally Diverse Populations: Case Study from Finland. JMIR Infodemiology, 2(2), e38343. https://doi.org/10.2196/38343
Madan, R., & Ashok, M. (2023). AI adoption and diffusion in public administration: A systematic literature review and future research agenda. Government Information Quarterly, 40(1), 101774. https://doi.org/10.1016/j.giq.2022.101774
Maragno, G., Tangi, L., Gastaldi, L., & Benedetti, M. (2023). Exploring the factors, affordances and constraints outline the implementation of Artificial Intelligence in public sector organizations. International Journal of Information Management, 73, 102686. https://doi.org/10.1016/j.ijinfomgt.2023.102686
Mergel, I., Dickinson, H., Stenvall, J., & Gasco, M. (2023). Implementing AI in the public sector. Public Management Review, 1–14. https://doi.org/10.1080/14719037.2023.2231950
Mikalef, P., Lemmer, K., Schaefer, C., Ylinen, M., Fjørtoft, S. O., Torvatn, H. Y., Gupta, M., & Niehaves, B. (2022). Enabling AI capabilities in government agencies: A study of determinants for European municipalities. Government Information Quarterly, 39(4), 101596. https://doi.org/10.1016/j.giq.2021.101596
Mosqueira-Rey, E., Hernández-Pereira, E., Alonso-Ríos, D., Bobes-Bascarán, J., & Fernández-Leal, Á. (2023). Human-in-the-loop machine learning: a state of the art. Artificial Intelligence Review, 56(4), 3005–3054. https://doi.org/10.1007/s10462-022-10246-w
Neumann, O., Guirguis, K., & Steiner, R. (2024). Exploring artificial intelligence adoption in public organizations: a comparative case study. Public Management Review, 26(1), 114–141. https://doi.org/10.1080/14719037.2022.2048685
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., … Moher, D. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ, n71. https://doi.org/10.1136/bmj.n71
Triningsih, E., Afdal, M., Permana, I., & Rozanda, N. E. (2025). Analisis Sentimen Terhadap Program Makan Bergizi Gratis Menggunakan Algoritma Machine Learning Pada Sosial Media X. Building of Informatics, Technology and Science (BITS), 6(4), 2240–2250. https://doi.org/10.47065/bits.v6i4.6534
van Atteveldt, W., van der Velden, M. A. C. G., & Boukes, M. (2021). The Validity of Sentiment Analysis: Comparing Manual Annotation, Crowd-Coding, Dictionary Approaches, and Machine Learning Algorithms. Communication Methods and Measures, 15(2), 121–140. https://doi.org/10.1080/19312458.2020.1869198
van Noordt, C., & Misuraca, G. (2022). Artificial intelligence for the public sector: results of landscaping the use of AI in government across the European Union. Government Information Quarterly, 39(3), 101714. https://doi.org/10.1016/j.giq.2022.101714
Verma, S. (2022). Sentiment analysis of public services for smart society: Literature review and future research directions. Government Information Quarterly, 39(3), 101708. https://doi.org/10.1016/j.giq.2022.101708
Villanueva-Miranda, I., Xie, Y., & Xiao, G. (2025). Sentiment analysis in public health: a systematic review of the current state, challenges, and future directions. Frontiers in Public Health, 13. https://doi.org/10.3389/fpubh.2025.1609749
Wankhade, M., Rao, A. C. S., & Kulkarni, C. (2022). A survey on sentiment analysis methods, applications, and challenges. Artificial Intelligence Review, 55(7), 5731–5780. https://doi.org/10.1007/s10462-022-10144-1
Zhu, L., Zhu, Z., Zhang, C., Xu, Y., & Kong, X. (2023). Multimodal sentiment analysis based on fusion methods: A survey. Information Fusion, 95, 306–325. https://doi.org/10.1016/j.inffus.2023.02.028
Zuiderwijk, A., Chen, Y.-C., & Salem, F. (2021). Implications of the use of artificial intelligence in public governance: A systematic literature review and a research agenda. Government Information Quarterly, 38(3), 101577. https://doi.org/10.1016/j.giq.2021.101577
Published
2026-12-01
How to Cite
Priyantomo, B. A. (2026). Algorithmic Blind Spots in Digital Crisis Sensing: Human Validation of Social Media Sentiment on Indonesia’s Free Nutritious Meals Program. Proceedings of Sunan Ampel International Conference of Political and Social Sciences, 4, 1-19. https://doi.org/10.15642/saicopss.2026.4.1.1-19
Section
Articles
Copyright (c) 2026 Bayu Aulia Priyantomo
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
