AI is able to pick up additional languages, doing so without magic or pixie dust.getty
The noted author Geoffrey Willans professed to say that anyone who knows no foreign language knows nothing of their own language.
Do you agree with that bold assertion?
Lets give the matter some serious thought.
First, perhaps we can agree that anyone that knows only one language could be labeled as being monolingual. Their native language is whatever language they have come to know. All other languages are said to be foreign to them, thus, if they opt to learn an additional language we could contend that they have picked up a foreign language.
Second, I assume we can concur that anyone that knows two languages could be given the lofty title of being bilingual. For those that know three or more languages, we will reserve the impressive label of being multilingual. An aspect that we might quibble about consists of how much of a language someone must know in order to be considered fluent enough in that language to count as intrepidly knowing an additional language. Hold onto that vexing question since well come back around to it later on herein.
Got a quick question for you.
How are you when it comes to being a language-wielding wizard?
You undoubtedly have friends or colleagues that speak a handful of languages, maybe you do likewise. The odds are that you are probably stronger in just one or two. The other languages are somewhat distant and sketchy in your mind. If push comes to shove, you can at least formulate fundamental sentences and likely comprehend those other languages to some slim degree.
The apex of the language gambit seems to be those amazing polyglots that know a dozen or dozens of languages. It seems nearly impossible to pull off. They imbue languages as easily as wearing a slew of socks and shoes. One moment conveying something elegant in one language and readily jumping over into a different language, nearly at the drop of a hat.
On social media, there are those polyglots that dazzle us by quickly shifting from language to language. They make videos in which they show the surprise and awe of others that admire their ability to effortlessly use a multitude of languages. You have surely wondered whether the polyglot was born with a special knack for languages or whether they truly had to learn many languages in the same way that you learned the two or three that you know. This is the classic question of whether language learning is more so nature versus nurture. We wont be solving that one herein.
There is an important reason that I bring up this weighty discussion overall about being able to use a multitude of languages.
Get yourself ready for the twist.
Maybe sit down and prepare for it.
The latest in generative AI such as ChatGPT and other such AI apps have seemingly been able to pick up additional languages beyond the one or ones that they appeared to have been initially data trained in. AI researchers and AI developers arent exactly sure why this is attainable. We will address the matter and seek to explore various postulated ways in which this can arise.
The topic has recently become a hot one due to an episode of the famed TV show 60 Minutes that interviewed Google executives. During the interviews, a Google exec stated that their AI app was able to engage in Bengali even though it was said to not have been data trained in that language. This enlisted a burst of AI hype, suggesting in this instance that the AI somehow magically made a choice to learn the additional language and proceeded to do so on its own.
Yikes, one might assume, this is surely a sign that these AI apps are converging toward being sentient. How else could the AI make the choice to learn another language and then follow up by learning it? That seems proof positive that contemporary AI is slipping and sliding toward Artificial General Intelligence (AGI), the moniker given to AI that can perform as humans do and otherwise be construed as possessing sentience.
It might be wise to take a deep breath and not fall for these wacky notions.
The amount of fearmongering and anthropomorphizing of AI that is going on right now is beyond the pale. Sadly, it is at times simply a means of garnering views. In other cases, the person or persons involved do not know what they are talking about, or they are being loosey-goosey for a variety of reasons.
In todays column, Id like to set the record straight and examine the matter of how generative AI such as ChatGPT and other AI apps might be able to pick up additional languages. The gist is that this can be mathematically and computationally explained. We dont need to refer to voodoo dolls or create false incantations to get there.
Logic and sensibility can prevail.
Vital Background About Generative AI
Before I get further into this topic, Id like to make sure we are all on the same page overall about what generative AI is and also what ChatGPT and its successor GPT-4 are all about. For my ongoing coverage of generative AI and the latest twists and turns, see the link here.
If you are already versed in generative AI such as ChatGPT, you can skim through this foundational portion or possibly even skip ahead to the next section of this discussion. You decide what suits your background and experience.
Im sure that you already know that ChatGPT is a headline-grabbing AI app devised by AI maker OpenAI that can produce fluent essays and carry on interactive dialogues, almost as though being undertaken by human hands. A person enters a written prompt, ChatGPT responds with a few sentences or an entire essay, and the resulting encounter seems eerily as though another person is chatting with you rather than an AI application. This type of AI is classified as generative AI due to generating or producing its outputs. ChatGPT is a text-to-text generative AI app that takes text as input and produces text as output. I prefer to refer to this as text-to-essay since the outputs are usually of an essay style.
Please know though that this AI and indeed no other AI is currently sentient. Generative AI is based on a complex computational algorithm that has been data trained on text from the Internet and admittedly can do some quite impressive pattern-matching to be able to perform a mathematical mimicry of human wording and natural language. To know more about how ChatGPT works, see my explanation at the link here. If you are interested in the successor to ChatGPT, coined GPT-4, see the discussion at the link here.
There are four primary modes of being able to access or utilize ChatGPT:
The capability of being able to develop your own app and connect it to ChatGPT is quite significant. On top of that capability comes the addition of being able to craft plugins for ChatGPT. The use of plugins means that when people are using ChatGPT, they can potentially invoke your app easily and seamlessly.
I and others are saying that this will give rise to ChatGPT as a platform.
All manner of new apps and existing apps are going to hurriedly connect with ChatGPT. Doing so provides the interactive conversational functionality of ChatGPT. The users of your app will be impressed with the added facility. You will likely get a bevy of new users for your app. Furthermore, if you also provide an approved plugin, this means that anyone using ChatGPT can now make use of your app. This could demonstrably expand your audience of potential users.
The temptation to have your app connect with ChatGPT is through the roof. Even if you dont create an app, you still might be thinking of encouraging your customers or clients to use ChatGPT in conjunction with your everyday services. The problem though is that if they encroach onto banned uses, their own accounts on ChatGPT will also face scrutiny and potentially be locked out by OpenAI.
As noted, generative AI is pre-trained and makes use of a complex mathematical and computational formulation that has been set up by examining patterns in written words and stories across the web. As a result of examining thousands and millions of written passages, the AI can spew out new essays and stories that are a mishmash of what was found. By adding in various probabilistic functionality, the resulting text is pretty much unique in comparison to what has been used in the training set.
There are numerous concerns about generative AI.
One crucial downside is that the essays produced by a generative-based AI app can have various falsehoods embedded, including manifestly untrue facts, facts that are misleadingly portrayed, and apparent facts that are entirely fabricated. Those fabricated aspects are often referred to as a form of AI hallucinations, a catchphrase that I disfavor but lamentedly seems to be gaining popular traction anyway (for my detailed explanation about why this is lousy and unsuitable terminology, see my coverage at the link here).
Another concern is that humans can readily take credit for a generative AI-produced essay, despite not having composed the essay themselves. You might have heard that teachers and schools are quite concerned about the emergence of generative AI apps. Students can potentially use generative AI to write their assigned essays. If a student claims that an essay was written by their own hand, there is little chance of the teacher being able to discern whether it was instead forged by generative AI. For my analysis of this student and teacher confounding facet, see my coverage at the link here and the link here.
There have been some zany outsized claims on social media about Generative AI asserting that this latest version of AI is in fact sentient AI (nope, they are wrong!). Those in AI Ethics and AI Law are notably worried about this burgeoning trend of outstretched claims. You might politely say that some people are overstating what todays AI can do. They assume that AI has capabilities that we havent yet been able to achieve. Thats unfortunate. Worse still, they can allow themselves and others to get into dire situations because of an assumption that the AI will be sentient or human-like in being able to take action.
Do not anthropomorphize AI.
Doing so will get you caught in a sticky and dour reliance trap of expecting the AI to do things it is unable to perform. With that being said, the latest in generative AI is relatively impressive for what it can do. Be aware though that there are significant limitations that you ought to continually keep in mind when using any generative AI app.
One final forewarning for now.
Whatever you see or read in a generative AI response that seems to be conveyed as purely factual (dates, places, people, etc.), make sure to remain skeptical and be willing to double-check what you see.
Yes, dates can be concocted, places can be made up, and elements that we usually expect to be above reproach are all subject to suspicions. Do not believe what you read and keep a skeptical eye when examining any generative AI essays or outputs. If a generative AI app tells you that President Abraham Lincoln flew around the country in a private jet, you would undoubtedly know that this is malarky. Unfortunately, some people might not realize that jets werent around in his day, or they might know but fail to notice that the essay makes this brazen and outrageously false claim.
A strong dose of healthy skepticism and a persistent mindset of disbelief will be your best asset when using generative AI.
Into all of this comes a slew of AI Ethics and AI Law considerations.
There are ongoing efforts to imbue Ethical AI principles into the development and fielding of AI apps. A growing contingent of concerned and erstwhile AI ethicists are trying to ensure that efforts to devise and adopt AI takes into account a view of doing AI For Good and averting AI For Bad. Likewise, there are proposed new AI laws that are being bandied around as potential solutions to keep AI endeavors from going amok on human rights and the like. For my ongoing and extensive coverage of AI Ethics and AI Law, see the link here and the link here, just to name a few.
The development and promulgation of Ethical AI precepts are being pursued to hopefully prevent society from falling into a myriad of AI-inducing traps. For my coverage of the UN AI Ethics principles as devised and supported by nearly 200 countries via the efforts of UNESCO, see the link here. In a similar vein, new AI laws are being explored to try and keep AI on an even keel. One of the latest takes consists of a set of proposed AI Bill of Rights that the U.S. White House recently released to identify human rights in an age of AI, see the link here. It takes a village to keep AI and AI developers on a rightful path and deter the purposeful or accidental underhanded efforts that might undercut society.
Ill be interweaving AI Ethics and AI Law related considerations into this discussion.
Figuring Out The Languages Conundrum
We are ready to further unpack this thorny matter.
I would like to start by discussing how humans seem to learn languages. I do so cautiously in the sense that I am not at all going to suggest or imply that todays AI is doing anything of the same. As earlier stated, it is a misguided and misleading endeavor to associate the human mind with the mathematical and computational realm of contemporary AI.
Nonetheless, some overarching reveals might be useful to note.
We shall begin by considering the use case of humans that know only one language, ergo being monolingual. If you know only one language, there is an interesting argument to be made that you might be able to learn a second language when undergoing persistent exposure to that second language.
Consider for example this excerpt from a research study entitled English Only? Monolinguals In Linguistically Diverse Contexts Have An Edge In Language Learning by researchers Kinsey Bice and Judith Kroll:
The crux is that your awareness of a single language can be potentially leveraged toward learning a second language by mere immersion into that second language. This is described as arising when in a linguistically diverse context. You might not necessarily grasp what those words in the other language mean, but you kind of catch on by exposure to the language and presumably due to your already mindful familiarity with your primary language.
Note that you didnt particularly have to be told how the second language works.
Of course, most people take a class that entails learning a second language and are given explicit instruction. That likely is the prudent path. The other possibility is that via a semblance of mental osmosis or mental gymnastics, you can gradually glean a second language. We can make a reasonable assumption that this is due to already knowing one language. If you didnt know any language at all, presumably you wouldnt have the mental formulation that could so readily pattern onto a second language. You would be starting from a veritable blank slate (well, maybe, since there is debate over what our brains consist of about wired versus learned aspects of language as a default).
This then covers a vital aspect that when you know one language, you possibly do not need explicit teaching about another language to learn that second language. We seem to be able to use a sense of language structure and patterns to figure out a second language. Not everyone can easily do so. It might be that you would struggle mightily over a lengthy period of time to comprehend the second language. A faster path would usually consist of explicit instruction.
But anyway, we can at times make that mental leap.
Lets explore another angle to this.
There is an intriguing postulation that if you learn a second language as a child, the result is that you will be more amenable to learning additional languages as an adult. Those people that are only versed in a single language throughout childhood allegedly will have a harder time learning a second language as an adult.
Consider this excerpt from a research study entitled A Critical Period For Second Language Acquisition: Evidence From 2/3 Million English Speakers by researchers Joshua Hartshorne, Joshua Tenenbaum, and Steven Pinker:
In short, a common suspected phenomenon is that a child that learns only one language during childhood is not somehow formulating a broadened capacity for learning languages all told. If they learn at least a second language, in theory, this is enabling their mind to discern how languages contrast and compare. In turn, this is setting them up for being more versed in that second language than would an adult that learns the second language while an adult. Plus, the child is somewhat prepared to learn a third language or additional languages throughout childhood and as an adult.
The idea too is that an adult that only learned one language as a child has settled into a one-language mode. They havent had to stretch their mind to cope with a second language. Thus, even though as an adult they should be able to presumably learn a second language, they might have difficulty doing so because they had not previously formulated the mentally beneficial generic structures and patterns to tackle a second language.
Please know that there is a great deal of controversy associated with those notions. Some agree with those points, some do not. Furthermore, the explanations for why this does occur, assuming it does occur, vary quite a bit.
If you want a boatload of controversy, heres more such speculation that gets a lot of heated discourse on this topic. Hold onto your hat.
Consider this excerpt from a research study entitled The Benefits Of Multilingualism To The Personal And Professional Development Of Residents Of The US by Judith Kroll and Paola Dussias:
The contention is that individuals with exposure to multiple languages during childhood benefit in many ways including greater openness to other languages and new learning itself.
Life though is not always a bed of roses. A concern is that a child might get confused or confounded when trying to learn more than one language during their childhood. The claim is that a child might not be able to focus on their considered primary language. They could inadvertently mix the other language and end up in a nowhere zone. They arent able to pinpoint their main language, and nor are they able to pinpoint the second language.
Parents are presented with a tough choice. Do you proceed to have your child learn a second language, doing so under the hopes and belief that this is the best means of aiding your child toward language learning and perhaps other advantages of mental stimulation? Or do you focus on one language alone, believing that once they are older it might be better to have them then attempt a second language, rather than doing so as a child?
Much of our existing educational system has landed on the side that knowing a second language as a child seems to be the more prudent option. Schools typically require a minimum amount of second language learning during elementary school, and ramp this up in high school. Colleges tend to do so as well.
Returning to the cited study above, heres what the researchers further state:
The expression often used is that when you know two or more languages, you have formulated a mental juggling capacity that allows you to switch from language to language. To some extent, the two or more languages might be construed as mental competitors, fighting against each other to win in your mental contortions when interpreting language. Some people relish this. Some people have a hard time with it.
I think that covers enough of the vast topic of language learning for the purposes herein. As mentioned, the language arena is complex and a longstanding matter that continues to be bandied around. Numerous theories exist. It is a fascinating topic and one that obviously is of extreme significance to humankind due to our reliance on language.
Imagine what our lives would be like if we had no language to communicate with. Be thankful for our wonderous language capacities, no matter how they seem to arise.
Generative AI And The Languages Affair
We are now in a position to ease into the big question about generative AI such as ChatGPT and the use of languages.
AI researcher Jan Leike at OpenAI tweeted this intriguing question on February 13, 2023:
And within the InstructGPT research paper that was being referred to, this point is made about the languages in the dataset that was used:
This brings us to my list of precepts about generative AI and the pattern-matching associated with languages, specifically:
A quick unpacking might be helpful.
First, realize that words are considered to be objects by most generative AI setups.
As Ive discussed about ChatGPT and GPT-4, see the link here, text or words are divided up into tokens that are approximately 3 letters or so in length. These tokens are various assigned numbers. The numbers are used to do the pattern matching amidst the plethora of words that are for example scanned during the data training of the generative AI. All of it is tokenized and used in a numeric format.
The text you enter as a prompt is encoded into a tokenized number. The response formulated by generative AI is a series of tokenized numbers that are then mapped into the corresponding letters and word segments for presentation to you when using an AI app.
The words being scanned during data training are typically sourced on the Internet in terms of passages of text that are posted on websites. Only a tiny fraction of the text on the Internet is usually involved in this scanning for data training and pattern-matching formulation purposes. A mathematical and computational network structure is devised that attempts to statistically associate words with other words, based on how humans use words and as exhibited via the Internet sites being scanned.
You might find of interest that there are concerns that this widespread text scanning is possibly violating Intellectual Property (IP) rights and entails plagiarism, see my analysis at the link here. It is an issue being pursued in our courts and well need to wait and see how the courts rule on this.
By and large, the generative AI that you hear about is data trained on words from the Internet that are in English, including for example the data training of ChatGPT. Though the bulk of the words encountered during the Internet scanning was in English, there is nonetheless some amount of foreign or other language words that are also likely to be encountered. This could be by purposeful design as guided by the AI developers, but usually, it is more likely a happenstance as to the casting of a shall we say a rather wide net when sauntering across a swath of the Internet.
It is like aiming to catch fish in your fishnet and meanwhile, you just so happen to also get some lobsters, crabs, and other entities along the way.
What happens with those other entities that are caught in the fishnet?
One possibility is that the pattern matching of the generative AI opts to treat those encountered words as a separate category in comparison to the English words being examined. They are outliers in contrast to the preponderance of words being patterned on. In a sense, each such identification of foreign words can be classified as belonging to a different potential language. Per my analogy, if fish were being scanned, the appearance of a lobster or a crab would be quite different, and ergo could be mathematically and computationally placed into a pending separate category.
Unless the AI developers have been extraordinarily cautious, the chances are that some notable level of these non-English words will be encapsulated during the data training across the selected portions of the Internet. One devised approach would be to simply discard any words that are calculated as possibly being non-English. This is not usually the case. Most generative AI is typically programmed to take a broad-brush approach.
The point is that a generative AI is unlikely to be of a single language purity.
Ive been discussing the case of using English as the primary language for being patterned. All other languages would be considered foreign with respect to English in that instance. Of course, we could readily and AI researchers have indeed chosen other languages to be the primary language for their generative AI efforts, which in that instance would mean that English is a foreign language in that case.
For purposes of this discussion, well continue with the case of English as selected as the primary language. The same precepts apply even if some other language is the selected primary language.
We can usually assume that the data training of a generative AI is going to include encounters with a multitude of other languages. If the encounters are sufficiently numerous, the mathematical and computational pattern matching will conventionally treat those as a separate category and pattern match them as based within this newly set aside category. Furthermore, pattern matching can mathematically and computationally broaden as the encounters aid in ferreting out the patterns of one language versus the patterns of a different language.
Here are some handy rules of thumb about what the generative AI is calculating:
As the pattern matching gets enhanced via the encounters with other languages, this also has the side benefit that when encountering yet another newly encountered language. The odds are that less of the language is needed to extrapolate what the language consists of. Smaller and smaller sample sizes can be extrapolated.
There is an additional corollary associated with that hypothesis.
Suppose that an additional language that well refer to for convenience as language Z had not been encountered at all during the data training. Later on, a user decides to enter a prompt into the generative AI that consists of that particular language Z.
You might at first assume that the generative AI would summarily reject the prompt as unreadable because the user is using a language Z that has not previously been encountered. Assuming that the AI developers were mindful about devising the generative AI to fully attempt to respond to any user prompt, the generative AI might shift into a language pattern-matching mode programmatically and try to pattern match on the words that otherwise seem to be outside of the norm or primary language being used.
This could account for the possibility that such a user-entered prompt elicits a surprising response by the generative AI in that the emitted response is also shown in the language Z, or that a response in say English is emitted and has seemingly discerned part of what the prompt was asking about. You see, the newly encountered language Z is parsed based on the pattern-matching generalizations earlier formed during data training as to the structure of languages.
During the 60 Minutes interview with Google executives, the exec that brought up the instance of generative AI that suddenly and surprisingly seemed to be able to respond to a prompt that was in Bengali, further stated that after some number of prompts in Bengali, the generative AI was able to seemingly translate all of Bengali. Heres what James Manyika, Googles SVP, stated: We discovered that with very few amounts of prompting in Bengali, it can now translate all of Bengali.
See the original post here:
Recommendation and review posted by G. Smith
