A beginner’s guide to Generative AI
By Scott McLiver, Partner and Asia Pacific AI Leader
Generative AI signifies the start of a period of human progress where computers aren't just sophisticated calculators, but creators, thinkers, and even dreamers. These creations could range from an image or a video, a new piece of music, a poem, some legal advice, the computer code to create an app, or simply the answer to a question. The ability for computers to not only create and think but to interact with humans using natural language is changing the game.
How it works
Many factors and decades of research and progress have given rise to this point in the maturity curve of Artificial Intelligence. Not least of all, the massive amount of data on the internet and the huge advancements in microchip processing power. But there are some foundational elements that are helpful to understand around this technology.
Generative AI is largely made possible through a type of technology called neural networks, which are a kind of machine learning model. The idea comes from how we believe our human brains work. Just like our brain is made up of billions of neurons, all connected and communicating with each other, neural networks have virtual 'neurons' that pass information around. And just as we learn from experience, neural networks learn from data.
Pre-training and Large Language Models (LLMs)
If you were to learn a new language, you would start by studying vocabulary and grammar. That's what is called pre-training in the world of AI. We feed the AI lots of data - say, millions of books, websites, and articles - so it can learn the 'vocabulary and grammar' of the language, or in other words, patterns and structures of data.
This pre-training allows the AI to become what we call a Language Model. In our specific example, the AI model is called GPT, short for Generative Pretrained Transformer. The "language" it learns isn't just English, Spanish, or Mandarin; it learns the language of human thought and communication captured in those millions of books and websites. That's what LLMs or Large Language Models do. They're like a computer's version of an incredibly well-read scholar.
The learning doesn’t halt at the boundaries of its training data. Instead, it’s capable of extending its understanding beyond, often in ways that might surprise us. For instance, although initially trained primarily on English data, the AI’s exposure to the multilingual fabric of the internet has allowed it to understand and interact in a multitude of languages fluently. It’s like having a friend who, despite being a native English speaker, becomes multilingual by reading extensively from a global library.
This unsupervised learning aspect, where the AI discovers patterns and knowledge not explicitly part of its training, is a key element of the power of Generative AI. It’s not confined to a fixed set of capabilities; it’s a continual learner, absorbing and interpreting new data, expanding its horizons, and in turn, broadening ours. This remarkable ability to learn allows its capability to grow and grow.
How AI learns to generate new content
So after its finished pre-training, we have this 'well-read scholar' AI, trained to understand not only human language but human concepts, communication styles, and thought patterns. But how does it create new content? How does it 'imagine'? Well, just as a child initially learns to write stories by reading lots of them, the AI learns to generate new content from these patterns it's already seen in its training. This process is kind of like the AI making new but educated guesses based on what it has learned. The fact these models can imagine is equally part of the magic of them and also the risk. This ability means they are at times prone to “hallucinations”. This is the term to describe when the model provides false information or states facts which don’t exist and is one of the reasons why human oversight is very important in many use cases.
Human-level capabilities will boost efficiency
One of the key reasons that Generative AI is getting so much focus is that the capabilities of these generative models are improving really fast and they’ve now crossed human-level capability on many measures. As an example, ChatGPT-4 has passed the Bar Exam, the medical entrance exam, scored in the 99th percentile of SATs, and passed the Harvard entrance exam to name a few.
“So we’ve now got a form of technology that not only can interact with humans in a completely natural language way and be creative in its responses, but its responses have reached a human level of performance. This means we are entering a phase where human efficiency is going to get a massive boost.”
The use of AI is going to rapidly increase the productivity of knowledge workers. New technologies impact sectors differently. For example, the Industrial Revolution impacted factories and agriculture more than others. The internet impacted media, news, and entertainment more than others initially. The initial disruptive impact of Generative AI is absolutely directed at knowledge workers and professional services.
Karim R. Lakhani, a professor at Harvard Business School says that “AI won’t replace humans but humans using AI will replace humans without.” This is a really important message for businesses in the knowledge economy. In mainstream media, the narrative around Generative AI often revolves around job automation, and the replacement of human labour with machines. However, there is a far more practical and immediate opportunity that most organisations are still yet to embrace, which is to augment human knowledge work with Generative AI by automating tasks entirely or simply reducing the time needed. A working paper published by the Harvard Business School (Dell-Acqua et-al 2023, https://www.hbs.edu/) suggests that, on average, consultants using Generative AI finish 12.2% more tasks, 25.1% more efficiently at 40% higher quality.Remember that’s with current AI, imagine these numbers in two or five years.
The rate of advancement around this technology has been extraordinary and has caused many technology leaders to voice their concern.The parameters of a LLM provide a rough yet insightful gauge of a model’s complexity. While not a flawless metric, it’s revealing. For instance, GPT-2 in 2020 boasted 13 times more parameters than its predecessor. The subsequent model, GPT-3, ramped up the parameter count to over 100 times that of GPT-2. The latest iteration, ChatGPT-4, has taken a colossal leap, being over 500 times more powerful than GPT-3.
How Generative AI is helping professionals, businesses and consumers in 2023
Currently, Generative AI excels in several areas particularly around language and text. We need to remember the models that are getting all the attention are large language models. Generative AI and broader AI will make huge leaps around how to use data and big numerical based sources of information. But we need to understand that those breakthroughs are largely yet to come. It’s language and knowledge based tasks where the current Generative AI models are exceptional already and working.
In the domain of computer programming, it’s a very powerful tool, aiding in code generation and debugging. A notable example comes from GitHub, where, according to CEO Thomas Dohmke, the programming AI is credited with writing 40% of the code. In essence, for every 100 lines of code, 40 are generated by AI, leading to a 55% increase in development speed. This showcases the profound impact Generative AI is having on coding efficiency and productivity.
In the realm of written content, it’s proving to be adept not only in crafting articulate articles, blog posts, but also in composing the first draft of professional emails, legal or tax advice and comprehensive reports. For the task of summarising large documents, it’s quickly becoming a game-changer, condensing extensive text into digestible summaries focusing on key relevant points. In the visual arena, it’s facilitating the creation of engaging digital art and graphics, and at a consumer level it’s revolutionising advanced photo editing, making it significantly more intuitive and user-friendly. Each of these capabilities is a testament to Generative AI’s versatility and its role in reshaping the productivity landscape today.
This productivity element is the key area where some are overlooking the potential. Some observers are taking the view that until Generative AI can completely eliminate a human from a task it is not useful. This is the wrong lens. Automobiles were a big step forward in efficiency even though they still needed a human driver. The ability for Generative AI to augment human capability is powerful and shouldn't be overlooked.
The risks that need our attention
Undeniably, the advent of Generative AI brings with it a huge amount of possibilities but also a spectrum of risks that demand our attention. These risks pervade various dimensions, encompassing consumers, businesses, the broader society, and extend across different time horizons as we step into this new era of transformative technology.
At the forefront, the ethical use of Generative AI is a pressing concern. It's imperative that we adhere to a moral compass in deploying this technology, ensuring it serves to augment human endeavour rather than undermine it. This ethical lens is not merely a current focus but a continuous endeavour as we navigate through the unfolding landscape of Generative AI.
Further down the road, the long-term implications of Generative AI on society and employment remain veiled in uncertainty. As this technology burgeons, its impact on job markets, income distribution, and societal structures could be profound. Hence, a proactive, informed approach is essential to foster an inclusive growth trajectory and mitigate adverse repercussions.
In addition, the role of robust, forward-thinking regulation is crucial. Regulations need to serve as sturdy guardrails, guiding our journey through the uncharted road ahead of Generative AI, ensuring safety, accountability, and the common good. The objective is to foster a regulatory construct that not only keeps pace with the rapid technological advancement but also anticipates as much as possible the road ahead, helping society to harness the potential of Generative AI responsibly and beneficially.
Generative AI, thanks to the power of neural networks and pre-training, has given computers the capability to not only imagine and be creative but also interact with humans using natural language. Generative AI models are learning from massive amounts of information and gaining an incredible understanding of the world.
It's a fascinating technology, which at its core is inspired by our own brain. We are truly at the start of a new technology wave that will undoubtedly have both positive and negative impacts. Provided we are sensible and considered as a society, there is no doubt the potential for positives significantly outweigh the negatives, but this will require effort at all levels.
Generative AI was used in the creation of this article.
Contact
