Artificially Natural: Linguistics, Machine Learning, and Natural Language Processing

Anvi Padiyar Thomas Jefferson High School for Science and Technology

The human race has always found ways to communicate with one another. The earliest humans depended on hand gestures and vocal grunts to express meaning, while ancient civilizations carved symbols into stone. These forms of communication have evolved into the many languages that we know today. Linguistics, the study and scientific approach towards understanding human languages, is divided into branches such as phonetics, phonology, morphology, syntax, and semantics. Different fields of study based on linguistics include: historical linguistics, sociolinguistics, psycholinguistics, and neurolinguistics [6]. At first glance, linguistics may seem to fall within just the humanities. However, linguistics is highly dependent on neurology and has many applications in modern technology. 

Let’s look a little closer into the neurolinguistic subsection of linguistics. For language to be understood and produced, it needs to be processed in the brain. The brain stores and transmits information using networks of neurons and glial cells which transport nerve impulses. Glial cells destroy pathogens, remove dead neurons, and help neurons by providing them with nutrients and physical support. The neural networks form necessary connections between the different regions of the brain to process all kinds of information. For example, the movement regions used for speech and the internal and external sensation regions used for reading and hearing respectively form a connection which allows the brain to process language [3]. The strength of these network connections depends on how often that connection is used, and any skill, such as learning a language, is a result of strengthening previous network connections or creating new connections [3]. For example, if you practice an action, like speaking a certain word, your brain will repeatedly use that specific network connection. As a result, that connection and your skill of speaking that word will improve over time. 

You may be wondering where language resides in the brain. Unlike other functions that your brain controls, language depends on both halves of the brain’s cerebral cortex. The left half of the cortex controls the writing and speaking part of language processing, while effective communication and comprehension occur in the right half. Since the brain is mostly made up of fast, efficient neural networks, language information can be processed easily and quickly in both sides of the brain. The brain is also very flexible, allowing recovery from brain injuries or loss of knowledge and network connections to some degree. For instance, people with aphasia, a language disability caused by brain damage, can relearn a language they lost through therapy and practice. This plasticity characteristic also allows the blind to read Braille using the visual section of their brain that they can no longer use to see. The same happens for the deaf as they use the auditory section of their brain to understand sign languages [3].

As a versatile field of study, linguistics is found everywhere, but especially in computer science and technology. Artificial intelligence (AI) is a computer science field centered around creating computer programs and systems that perform tasks like a human would. Since language and communication are the basis of humanity, developers often create AI programs that tackle linguistics. These programs use Natural Language Processing (NLP), which uses human language input and observes the interaction between humans and computers. NLP also works by taking input and cross-referencing a large database of previously recorded information [7]. Used in online search engines, language translators, online customer support, autocorrect, and virtual assistants, NLP has become a significant part of our daily lives.

AIs have a few different ways to learn and adapt to their environment, but most AIs nowadays use machine learning. Similar to how we learn from our memories and our mistakes of the past, machine learning is a concept of improving an AI’s performance on a specific task by using its previous experiences [1, 5]. There are three different methods of machine learning that developers use to train their AIs: supervised, unsupervised, and reinforced. In supervised learning, the developer gives an AI a desired outcome and the AI attempts to match it as close as possible. On the other hand, in unsupervised learning, the AI is not given any desired outcomes, so it must look for important connections and patterns among the raw data on its own. Finally, reinforced learning works by putting an AI into the environment and giving it the freedom to behave however it wants. Through positive or negative reinforcement, like punishments or rewards, the AI learns and adapts from the feedback it receives [1, 7]. A more specific and advanced type of machine learning is deep learning, in which an AI can imitate the human brain with its own artificial neural networks [5]. Recall that the human brain has many of its own neural networks; therefore, AIs that use deep learning perform more alike to the human brain than AIs that rely on rudimentary machine learning. 

From calling your best friend to playing a sad song, virtual assistants like Amazon’s Alexa, Apple’s Siri, and Google Assistant can perform a variety of helpful tasks with machine learning and NLP. Of all online searches, 25% are done with voice-analyzing AIs, and over 35 million Americans use virtual assistants at least once a month [4]. Evidently, this technology has become an important part of our everyday lives. Virtual assistants work in three main steps: speech-to-text, text-to-intent, and intent-to-action. In the first step, speech-to-text, the AI receives spoken human input and converts it to text similar to what a program would normally get from typed user input. The program is able to do this by breaking down the speech into phonemes, which are small, recognizable parts of speech. The program understands the speech input from the order, combination, and context of its phonemes. With similar sounding words, the program uses context clues to differentiate between them. Then, the program matches the phonemes from the speech to established words in a database. After converting human speech to typed text, that text is then used to determine the intent of the user in the second step, text-to-intent. Intent is determined by forming multiple answers to the user’s request. The AI, then, compares the answers for its information, type of information, reliability, and relevance to create a ranking system of all the answers the AI could give. The best answer from the ranked list would show the intent of the spoken input. The last step, intent-to-action, finally outputs the highest ranked answer as a response to the user’s request. Not limited to just vocal responses or mobile phone-based actions, virtual assistants have been able to control vehicles, refrigerators, and doors during the intent-to-action step [8].

As you use Google Translate in your foreign language class, you may overlook that easily translating “Sorry, I forgot my homework” to “Désolée, j’ai oublié mes devoirs” in French is only possible due to NLP. Applications like Google Translate use NLP to easily translate and transcribe languages, such as French, Spanish, Chinese, and Hindi. However, can such applications translate ancient and lost languages? In 2010, researchers from the Massachusetts Institute of Technology and the University of Southern California developed a computer program that can do just that. The researchers’ goal was to decipher Ugaritic, an ancient Semitic language, with NLP. They started off by selecting another language that was closely related to Ugaritic. They chose Hebrew. Next, they compared the two languages’ alphabets to determine the frequency of each symbol and letter to note the common characters. Using NLP with a probabilistic model and commonly used symbols in one language mapped onto the other language, the AI is able to identify which of the mapped symbols have a consistent set of cognates, suffixes. and prefixes. After iterating through the data many times, the probabilities increase and the AI becomes more consistent due to its machine learning abilities. Since linguists had already deciphered Ugaritic, the researchers were able to judge how well their AI performed. Identifying 29 of 30 letters and 60% of cognates, the AI deciphered Ugaritic fairly well, but improvements are still needed [2]. Similar methods are also used in everyday online translations. Using probabilistic models, frequencies, and the method of mapping symbols onto one another, translation between any two languages is possible, even ancient ones.

Language has been a part of the human experience for thousands of years. With the brain’s amazing processing abilities and plasticity, we are able to speak, read, write, and bounce back from neural damage. There is nothing more human than language and the various abilities to communicate and connect with one another. Nevertheless, people have been able to train AIs that use NLP and machine learning to learn the many forms of language and create applications like chatbots, search engines, and virtual assistants. Even as I write this article, autocorrect uses NLP to fix any spelling or grammar mistakes I make. With linguistics, machine learning, and NLP, the technological possibilities are endless.

References

[1] Bringsjord, S., & Govindarajulu, N. S. (2018, July 12). Artificial intelligence. Stanford Encyclopedia of Philosophy. Retrieved June 16, 2020, from https://plato.stanford.edu/entries/artificial-intelligence/ 

[2] Hardesty, L. (2010, June 30). Computer automatically deciphers ancient language. MIT News. Retrieved June 16, 2020, from http://news.mit.edu/2010/ugaritic-barzilay-0630 

[3] Menn, L. (n.d.). Neurolinguistics. Linguistic Society of America. Retrieved June 16, 2020, from https://www.linguisticsociety.org/resource/neurolinguistics 

[4] Sharma, V. (2017, October 17). How do digital voice assistants (e.g. Alexa, Siri) work? USC Marshall. Retrieved June 16, 2020, from https://www.marshall.usc.edu/blog/How-do-digital-voice-assistants-eg-alexa-siri-work

[5] Tan, S. (2019, May 28). How to train your AI. Retrieved June 16, 2020, from https://medium.com/revain/how-to-train-your-ai-98113bdac101  

[6] What is linguistics? (n.d.). UCLA Department of Linguistics. Retrieved June 16, 2020, from https://linguistics.ucla.edu/undergraduate/what-is-linguistics/ 

[7] Zhou, M., Duan, N., Liu, S., & Shum, H.-Y. (2020). Progress in neural NLP: Modeling, learning, and reasoning. Engineering, 6(3), 275-290. https://doi.org/10.1016/j.eng.2019.12.014