###### Abstract  
Arabic raw audio datasets were initially gathered to produce a corresponding signal spectrum, which was further used to extract the Mel-Frequency Cepstral Coefficients (MFCCs). The pronunciation dictionary, language model, and acoustic model were further derived from the MFCCs' features. These output data were processed into Baidu's Deep Speech model (ASR system) to attain the text corpus. Baidu's Deep Speech model was implemented to precisely identify the global optimal value rapidly while preserving a low word and character discrepancy rate by attaining an excellent performance in isolated and end-to-end speech recognition. The desired outcome in this work is to forecast the next word and character in a sequential and systematic order that applies under natural language processing (NLP). This work combines the trained Arabic language model ARABERT with the potential of Long Short-Term Memory (LSTM) networks to predict the next word and character in an Arabic text. We used the pre-trained ARABERT embedding to improve the model's capacity and, to capture semantic relationships within the language, we educated LSTM + CNN and Markov models on Arabic text data to assess the efficacy of this model. Python libraries such as TensorFlow, Pickle, Keras, and NumPy were used to effectively design our development model. We extensively assessed the model's performance using new Arabic text, focusing on evaluation metrics like accuracy, word error rate, character error rate, BLEU score, and perplexity. The results show how well the combined LSTM + ARABERT and Markov models have outperformed the baseline models in envisaging the next word or character in the Arabic text. The accuracy rates of 64.9% for LSTM, 74.6% for ARABERT + LSTM, and 78% for Markov chain models were achieved in predicting the next word, and the accuracy rates of 72% for LSTM, 72.22% for LSTM + CNN, and 73% for ARABERT + LSTM models were achieved for the next-character prediction. This work unveils a novelty in Arabic natural language processing tasks, estimating a potential future expansion in deriving a precise next-word and next-character forecasting, which can be an efficient utility for text generation and machine translation applications. Baidu's deep speech; LSTM; CNN; ARABERT; Markov chain; sequence generation; vocab size +
Footnote †: journal: applied sciences  
using various technological advancements and linguistics. Arabic is pronounced differently depending on the phonemes and diacritical marks across the globe. Speaking patterns can also be significantly impacted by dialectal differences. With over 380 million native speakers, Arabic is one of the most widely spoken languages worldwide. There are three primary varieties of Arabic, making it a very diverse language. The written and formally spoken languages are based on classical Arabic, the language of the Quran. Derived from classical Arabic, Modern Standard Arabic (MSA) finds use in government, education, and the media. Conversely, the numerous dialects of Arabic spoken in various Arabic-speaking countries are referred to as colloquial Arabic. These dialects can differ greatly, which frequently makes it difficult for speakers of various dialects to communicate with one another. Colloquial Arabic is the language of daily living and social contact, even if MSA is the official language in many Arab nations. The foundation of comprehending and producing natural language is the capacity to anticipate the word or character that will appear in a series. It is a basic task in natural language processing (NLP), with applications that power virtual assistants and chatbots to transform text generation and machine translation. Accurately predicting the next word or character is essential for many applications. LSTM networks are well known for their capacity to represent long-term dependencies in sequences; they are perfect for simulating language's sequential structure. However, adding domain-specific knowledge can improve their performance further, and this is where ARABERT becomes useful.