Evaluation of Automated Arabic Summaries Generated with the AI tool Chat GPT using Textual Linguistics

Document Type : Original Article

Author

Department of Arabic language, Faculty of Al Alsun, Ain Shams university, Cairo, Egypt

Abstract

With the development of artificial intelligence, summarization techniques have made remarkable progress, and that has led to the emergence of mechanisms for evaluating these summaries, but the evaluation of the quality of the summaries produced by automated systems was based entirely on mathematical algorithms and applied different evaluation criteria. Hence, this study seeks to use a linguistic mechanism to evaluate the automated summary (used by artificial intelligence) of Arabic texts, by detecting how cohesion is achieved in these summaries. Thus, the study will look at how the abstract is affected by the deletion of the cohesion from it, and how other lexical and grammatical cohesion aspects are achieved in it. The study follows the descriptive approach with textual linguistics. The study uses automated summaries of texts from various fields, generated by the artificial intelligence tool ChatGPT as its main source. The study reached a number of results, including: The tool is good at summarizing all types of texts (general and specialized), but it is less accurate in literary and economic texts. It was also found that although the tool relied on the deletion of many grammatical and lexical cohesion elements from the original text, this did not affect the coherence of the abstracts.

Keywords

Files

Share

How to cite

Statistics