Generative Artificial Intelligence in Computer Programming Education: A Bibliometric Analysis
Abstract views: 2
/ 
PDF downloads: 3
Keywords:
Generative Artificial Intelligence, Generative Ai, Computer Programming Education, Large Language Models Bibliometric AnalysisAbstract
This study aims to investigate the emerging trends and research patterns surrounding
generative AI in computer programming education through bibliometric analysis. It seeks to identify
descriptive characteristics of existing literature, influential research contributions, core terms, and future
research directions. A bibliometric analysis was conducted using data from the Scopus database, focusing
on publications related to generative AI in programming education. The analysis employed techniques
such as citation analysis, co-author analysis, and keyword co-occurrence, utilizing R Studio and
VOSviewer for data visualization. The analysis revealed 48 publications, predominantly research articles,
with a significant increase in output during 2023-2024. Key journals identified include Applied Sciences
and Computers and Education: Artificial Intelligence. Influential authors and countries were highlighted,
with China and the USA leading in scientific production. Core terms included "programming education,"
"artificial intelligence," and "ChatGPT." The thematic map analysis identified motor themes such as
"Programming" and "AI in Education," alongside emerging themes like "computational thinking,"
indicating areas requiring further exploration. This study contributes to the understanding of generative
AI's role in programming education by mapping existing research and identifying gaps. The findings from
this bibliometric analysis provide a basis for directing future research and cultivating a thorough
understanding of how generative AI is shaping the future of computer programming education.
Downloads
References
W. M. Lim, A. Gunasekara, J. L. Pallant, J. I. Pallant, and E. Pechenkina, “Generative AI and the future of education: Ragnarök or reformation? A paradoxical perspective from management educators,” Int. J. Manag. Educ., vol. 21, no. 2, p. 100790, 2023, doi: https://doi.org/10.1016/j.ijme.2023.100790.
K.-B. Ooi et al., “The potential of generative artificial intelligence across disciplines: Perspectives and future directions,” J. Comput. Inf. Syst., pp. 1–32, 2023, doi: https://doi.org/10.1080/08874417.2023.2261010.
H. Polat, “Transforming education with artificial intelligence: Shaping the path forward,” in Empowering Education: Exploring the Potential of Artificial Intelligence, 2023, pp. 3–20. [Online]. Available: www.istes.org
B. A. Becker, P. Denny, J. Finnie-Ansley, A. Luxton-Reilly, J. Prather, and E. A. Santos, “Programming Is Hard – Or at Least It Used to Be: Educational Opportunities And Challenges of AI Code Generation,” Cornell University, vol. undefined, no. undefined. Jan. 2022. doi: 10.48550/arxiv.2212.01020.
P. Denny et al., “Computing Education in the Era of Generative AI,” Assoc. Comput. Mach., vol. 67, no. 2, pp. 56–67, Jan. 2024, doi: 10.1145/3624720.
C. Zastudil, M. Rogalska, C. Kapp, J. Vaughn, and S. MacNeil, “Generative ai in computing education: Perspectives of students and instructors,” in 2023 IEEE Frontiers in Education Conference (FIE), IEEE, 2023, pp. 1–9. doi: https://doi.org/10.1109/fie58773.2023.10343467.
N. A. Aarti, “Generative Ai in Software Development : an Overview and Evaluation of Modern Coding Tools,” Int. J. Multidiscip. Res., vol. 6, no. 3, 2024, doi: 10.36948/ijfmr.2024.v06i03.23271.
J. T. Liang, C. Yang, and B. A. Myers, “A Large-Scale Survey on the Usability of AI Programming Assistants: Successes and Challenges,” 2024, doi: 10.1145/3597503.3608128.
S. Abdulla, S. S. Ismail, Y. M. Fawzy, and A. Elhag, “Using ChatGPT in Teaching Computer Programming and Studying its Impact on Students Performance,” Electron. J. E-Learn., vol. 22, no. 6, pp. 66–81, 2024, doi: 10.34190/ejel.22.6.3380.
M. Amoozadeh et al., “Student-AI Interaction: A Case Study of CS1 students,” in Proceedings of the 24th Koli Calling International Conference on Computing Education Research, 2024, pp. 1–13.
Z. Epstein et al., “Art and the science of generative AI,” Science, vol. 380, no. 6650, pp. 1110–1111, 2023.
K. Kadaruddin, “Empowering Education through Generative AI: Innovative Instructional Strategies for Tomorrow’s Learners,” Int. J. Bus. Law Educ., 2023, doi: 10.56442/ijble.v4i2.215.
S. Altares-López, J. M. Bengochea-Guevara, C. Ranz, H. Montes, and Á. Ribeiro, “Generative AI: The power of the new education,” Cornell University, vol. undefined, no. undefined. May 2024. doi: 10.48550/arxiv.2405.13487.
S. Boguslawski, R. Deer, and M. G. Dawson, “Programming education and learner motivation in the age of generative AI: student and educator perspectives,” Information and Learning Science. Emerald Publishing, 2024. doi: 10.1108/ILS-10-2023-0163.
E. Dickey, A. Bejarano, and C. Garg, “Innovating Computer Programming Pedagogy: The AI-Lab Framework for Generative AI Adoption,” ArXiv Prepr. ArXiv230812258, 2023, doi: https://doi.org/10.48550/arxiv.2308.12258.
K. Łodzikowski, P. W. Foltz, and J. T. Behrens, “Generative AI and Its Educational Implications,” Springer International Publishing, vol. undefined, no. undefined. pp. 35–57, Jan. 2024. doi: 10.1007/978-3-031-64487-0_2.
R. F. Mello, E. Freitas, F. D. Pereira, L. Cabral, P. Tedesco, and G. Ramalho, “Education in the age of Generative AI: Context and Recent Developments,” ArXiv Prepr. ArXiv230912332, 2023.
S. Lau and P. Guo, “From" Ban it till we understand it" to" Resistance is futile": How university programming instructors plan to adapt as more students use AI code generation and explanation tools such as ChatGPT and GitHub Copilot,” in Proceedings of the 2023 ACM Conference on International Computing Education Research-Volume 1, 2023, pp. 106–121.
A. H. Mohammadkhani, C. Tantithamthavorn, and H. Hemmati, “Explainable AI for Pre-Trained Code Models: What Do They Learn? When They Do Not Work?,” arXiv.org, vol. abs/2211.12821, 2022, doi: 10.48550/arXiv.2211.12821.
M. Kazemitabaar, J. Chow, C. K. T. Ma, B. J. Ericson, D. Weintrop, and T. Grossman, “Studying the effect of AI code generators on supporting novice learners in introductory programming,” in Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, 2023, pp. 1–23.
B. R. Mohan, A. Chainani, A. K. Kartheek, C. Praneeth, and K. Shakelli, “IntelliDev: AI-Powered Command Line Tool for Developers,” Int. J. Eng. Technol. Manag. Sci., vol. 8, no. 3, pp. 161–165, 2024, doi: 10.46647/ijetms.2024.v08i03.019.
R. A. Poldrack, T. Lu, and G. Beguš, “AI-assisted coding: Experiments with GPT-4,” arXiv.org, vol. abs/2304.13187, 2023, doi: 10.48550/arXiv.2304.13187.
R. Zviel-Girshin, “The Good and Bad of AI Tools in Novice Programming Education,” Educ. Sci., vol. 14, no. 10, p. 1089, 2024, doi: https://doi.org/10.3390/educsci14101089.
C.-K. Chang and L.-C.-T. Chien, “Enhancing Academic Performance with Generative AI-Based Quiz Platform,” pp. 193–195, 2024, doi: 10.1109/icalt61570.2024.00062.
M. Lehmann, P. B. Cornelius, and F. J. Sting, “AI Meets the Classroom: When Does ChatGPT Harm Learning?,” 2024, doi: 10.48550/arxiv.2409.09047.
D. M. Johnson, W. Doss, and C. M. Estepp, “Agriculture students’ use of generative artificial intelligence for microcontroller programming,” Nat. Sci. Educ., vol. 53, no. 2, 2024, doi: 10.1002/nse2.20155.
J. Prather et al., “The Widening Gap: The Benefits and Harms of Generative AI for Novice Programmers,” in Proceedings of the 2024 ACM Conference on International Computing Education Research-Volume 1, 2024, pp. 469–486.
Y. Xue, H. Chen, G. R. Bai, R. Tairas, and Y. Huang, “Does ChatGPT Help With Introductory Programming? An Experiment of Students Using ChatGPT in CS1,” in Proceedings of the 46th International Conference on Software Engineering: Software Engineering Education and Training, 2024, pp. 331–341.
J. O. Wecks, J. Voshaar, B. J. Plate, and J. Zimmermann, “Generative AI Usage and Academic Performance,” ArXiv Prepr. ArXiv240419699, 2024, doi: https://doi.org/10.48550/arxiv.2404.19699.
J. Liu and S. Li, “Toward Artificial Intelligence-Human Paired Programming: A Review of the Educational Applications and Research on Artificial Intelligence Code-Generation Tools,” J. Educ. Comput. Res., 2024, doi: 10.1177/07356331241240460.
P. Haindl and G. Weinberger, “Students’ Experiences of Using ChatGPT in an Undergraduate Programming Course,” IEEE Access, vol. 12, pp. 43519–43529, 2024, doi: https://doi.org/10.1109/access.2024.3380909.
W. E. McGrath, “The unit of analysis (objects of study) in bibliometrics and scientometrics,” Springer Nat. Neth., vol. 35, no. 2, pp. 257–264, Feb. 1996, doi: 10.1007/bf02018483.
J. Vlachý, “Scientometrics — What to do?,” Springer Nat. Neth., vol. 30, no. 2, pp. 521–527, Jun. 1994, doi: 10.1007/bf02018134.
R. Rousseau and X. Hu, “Under-cited influential work by Eugene Garfield,” Springer Nat. Neth., vol. 114, no. 2, pp. 651–657, Sep. 2017, doi: 10.1007/s11192-017-2530-8.
W. W. Hood and C. S. Wilson, “The Literature of Bibliometrics, Scientometrics, and Informetrics,” Scientometrics, vol. 52, no. 2, pp. 291–314, Oct. 2001, doi: 10.1023/A:1017919924342.
N. Donthu, S. Kumar, D. Mukherjee, N. Pandey, and W. M. Lim, “How to conduct a bibliometric analysis: An overview and guidelines,” J. Bus. Res., vol. 133, pp. 285–296, Sep. 2021, doi: 10.1016/j.jbusres.2021.04.070.
R. Heersmink, J. van den Hoven, N. J. van Eck, and J. van den Berg, “Bibliometric mapping of computer and information ethics,” Ethics Inf. Technol., vol. 13, no. 3, pp. 241–249, Sep. 2011, doi: 10.1007/s10676-011-9273-7.
H. Polat, A. Topuz, M. Yıldız, E. Taşlıbeyaz, and E. Kurşun, “A bibliometric analysis of research on ChatGPT in education,” Int. J. Technol. Educ. IJTE, vol. 7, no. 1, 2024, doi: https://doi.org/10.46328/ijte.606.
M. Pradana, H. P. Elisa, and S. Syarifuddin, “Discussing ChatGPT in education: A literature review and bibliometric analysis,” Taylor & Francis, vol. 10, no. 2. Aug. 2023. doi: 10.1080/2331186x.2023.2243134.
P. Song and X. Wang, “A bibliometric analysis of worldwide educational artificial intelligence research development in recent twenty years,” Springer Sci. Media, vol. 21, no. 3, pp. 473–486, Aug. 2020, doi: 10.1007/s12564-020-09640-2.
M. Aria and C. Cuccurullo, “bibliometrix: An R-tool for comprehensive science mapping analysis,” J. Informetr., vol. 11, no. 4, pp. 959–975, 2017, doi: https://doi.org/10.1016/j.joi.2017.08.007.
N. J. van Eck and L. Waltman, “Software survey: VOSviewer, a computer program for bibliometric mapping,” Scientometrics, vol. 84, no. 2, pp. 523–538, Aug. 2010, doi: 10.1007/s11192-009-0146-3.
V. K. Singh, P. Singh, M. Karmakar, J. Leta, and P. Mayr, “The journal coverage of Web of Science, Scopus and Dimensions: A comparative analysis,” Springer Nat. Neth., vol. 126, no. 6, pp. 5113–5142, Mar. 2021, doi: 10.1007/s11192-021-03948-5.
J. Zhu and W. Liu, “A tale of two databases: the use of Web of Science and Scopus in academic papers,” Springer Nat. Neth., vol. 123, no. 1, pp. 321–335, Feb. 2020, doi: 10.1007/s11192-020-03387-8.
Abad-Segura, F. J. Cortés-García, and L. J. B. Ureña, “The Sustainable Approach to Corporate Social Responsibility: A Global Analysis and Future Trends,” Multidiscip. Digit. Publ. Inst., vol. 11, no. 19, pp. 5382–5382, Sep. 2019, doi: 10.3390/su11195382.
W. M. Sweileh, “Research trends on human trafficking: a bibliometric analysis using Scopus database,” BioMed Cent., vol. 14, no. 1, Nov. 2018, doi: 10.1186/s12992-018-0427-9.
Md. M. Rahman and Y. Watanobe, “ChatGPT for Education and Research: Opportunities, Threats, and Strategies,” Applied Sciences (Switzerland), vol. 13, no. 9. MDPI, 2023. doi: 10.3390/app13095783.
R. Yilmaz and F. G. Karaoglan Yilmaz, “The effect of generative artificial intelligence (AI)-based tool use on students’ computational thinking skills, programming self-efficacy and motivation,” Computers and Education: Artificial Intelligence, vol. 4. Elsevier B.V., 2023. doi: 10.1016/j.caeai.2023.100147.
C. W. Okonkwo and A. Ade-Ibijola, “Python-bot: A chatbot for teaching python programming,” Engineering Letters, vol. 29, no. 1. International Association of Engineers, pp. 25–34, 2021.
Md. M. Rahman, Y. Watanobe, and K. Nakamura, “Source Code assessment and classification based on estimated error probability using attentive lstm language model and its application in programming education,” Applied Sciences (Switzerland), vol. 10, no. 8. MDPI AG, 2020. doi: 10.3390/APP10082973.
N.-T. Le, S. Strickroth, S. Gross, and N. Pinkwart, “A review of AI-supported tutoring approaches for learning programming,” Studies in Computational Intelligence, vol. 479. Springer Verlag, pp. 267–279, 2013. doi: 10.1007/978-3-319-00293-4_20.
K.-W. Han, E. Lee, and Y. Lee, “The impact of a peer-learning agent based on pair programming in a programming course,” IEEE Transactions on Education, vol. 53, no. 2. pp. 318–327, 2010. doi: 10.1109/TE.2009.2019121.
J. McBroom, I. Koprinska, and K. Yacef, “A Survey of Automated Programming Hint Generation: The HINTS Framework,” ACM Computing Surveys, vol. 54, no. 8. Association for Computing Machinery, 2022. doi: 10.1145/3469885.
C. A. G. da Silva, F. N. Ramos, R. V. de Moraes, and E. L. dos Santos, “ChatGPT: Challenges and Benefits in Software Programming for Higher Education,” Sustainability (Switzerland), vol. 16, no. 3. Multidisciplinary Digital Publishing Institute (MDPI), 2024. doi: 10.3390/su16031245.
L. Jacques, “Teaching CS-101 at the Dawn of ChatGPT,” ACM Inroads, vol. 14, no. 2. Association for Computing Machinery, pp. 40–46, 2023. doi: 10.1145/3595634.
D. Sun, A. Boudouaia, C. Zhu, and Y. Li, “Would ChatGPT-facilitated programming mode impact college students’ programming behaviors, performances, and perceptions? An empirical study,” International Journal of Educational Technology in Higher Education, vol. 21, no. 1. Springer Science and Business Media Deutschland GmbH, 2024. doi: 10.1186/s41239-024-00446-5.
J. Zhang, Y. Qi, F. Zheng, C. Long, Z. Lu, and Z. Duan, “Comparing keywords plus of WOS and author keywords: A case study of patient adherence research,” Wiley, vol. 67, no. 4, pp. 967–972, Jan. 2015, doi: 10.1002/asi.23437.
A. Lancichinetti and S. Fortunato, “Community detection algorithms: A comparative analysis,” Am. Phys. Soc., vol. 80, no. 5, Nov. 2009, doi: 10.1103/physreve.80.056117.
M. J. Cobo, A. G. López‐Herrera, E. Herrera‐Viedma, and F. Herrera, “An approach for detecting, quantifying, and visualizing the evolution of a research field: A practical application to the Fuzzy Sets Theory field,” Elsevier BV, vol. 5, no. 1, pp. 146–166, Nov. 2010, doi: 10.1016/j.joi.2010.10.002.
