The Long Road of AI
Many people first got acquainted with AI as a tool only recently, thanks to the advent of widely available LLMs. Yet, this technology has been thriving since the 1950s. In the decades between, it has seen rises and dips in popularity, including two major AI “winters.” Our article today will trace the history of AI and explore how it has led to the current AI boom.
Early Days: Perceptrons and Symbolic AI
Let’s travel back to the 1950s. The first question to address is who created AI. Realistically speaking, AI is a team effort, as multiple scientists had already spent a decade in this field. But it’s specifically Norbert Wiener, Alan Turing, and Claude Shannon whose theories led to the big discovery.
While the Turing test is probably the best-known element of AI’s infancy period, the evolution of AI also owes a lot to Frank Rosenblatt and his perceptron. First unveiled in 1958, the perceptron was a genuine neural network with a single layer, a basic AI that could only really work with binaries. Despite its simplicity from a current perspective, Rosenblatt obviously saw the perceptron as a project with massive potential and inspired optimism in him and his colleagues.
This first period of extreme optimism didn’t last, though, as Minsky and Papert, former peers of Rosenblatt, published a 1969 book of criticisms, dispelling the idea that a perceptron could change everything. Their work thoroughly explored the limitations of perceptrons, including their inability to process more complex tasks and functions. With this publication, the first AI winter started to take hold and derailed the work of the first creator of artificial intelligence.
Rise of Expert Systems and AI Winters
To clarify, an AI winter, which has happened twice so far, is an extended period where AI research stagnates, funding becomes scarce, and the direction of the field changes. Minsky and Papert’s book was a direct precursor to the first AI winter, which covered the 1970s. Despite substantial cuts, scientists kept experimenting, and the public stayed optimistic about the artificial intelligence evolution.
But as development progressed, the new AI systems made their limitations quite apparent, stifling some of the hype. For one, they were working with minimal power and had no hope of scaling effectively. Moreover, AI wasn’t as easy to train as it is now, and projects kept running into the issue of finding workable ways to “feed” data into the system.
Overcoming these issues meant that a new generation of devices had to emerge, and Japan was ready, launching its Fifth Generation projects in 1982, with the goal of creating more powerful computers. Their goal was to create a system that could engage in a conversation and provide sound, human-like reasoning. The project also aimed to provide quality machine translation. While they obviously didn’t perfect AI at the time, their drive inspired competitors to launch similar projects, with both the US and the UK pouring money into AI, marking the end of the winter period.
In the same era, Richard Greenblatt took advantage of the rise in expert systems - machines that could provide precise answers but only on a specific subject. His Lisp machines were tailor-made to run Lisp operations and had enough power to support more sophisticated AI.
The company producing these ended up splitting in two, their rivalry pushing each company to improve their offerings and change AI history. Sadly, even though this period seems fruitful and just as exciting in retrospect, it did not prevent the advent of the second AI winter. Spanning the 1990s, this change led to a minor collapse in the field, with hundreds of companies shutting down, funding dwindling once again, and surviving businesses shifting their focus.
Neural Nets Return: Backpropagation & Deep Learning
Neural networks stayed on the backburner for a while, largely thanks to Minsky and Papert, as well as their high power demands. However, the 2000s saw a new rise in this niche owing to better hardware, a new generation of GPUs, and a more successful application of backpropagation. The latter, invented in 1986, revolutionized training for AI systems and made it faster and faultproof.
After the winter of the 1990s, the 2000s saw a sharp rise in the artificial intelligence history timeline with big data availability playing a key role. Pre-cleaned and ordered data sets allowed neural networks to grow and improve at a very different pace. One big increase happened thanks to ImageNet, a massive database that started out with three million images, designed specifically for training.
Thanks to ImageNet, networks started showing better results when tasked with recognizing and identifying objects, culminating with AlexNet achieving a breakthrough record in 2012. This was a massive change as it was the first prominent deep learning model to deliver such results.
In addition to the other factors, a lot is explicitly owed to companies such as NVIDIA, who started artificial intelligence-centered development for their products. Their new generations of GPUs were made specifically to withstand the power-hungry networks and systems. As a result, training models became easier, more affordable, and less time-consuming.
Transformers: The Modern Breakthrough
A far cry from the old AI systems, modern systems can “remember” prior data inputs, hold conversations, and generally seem genuinely smart to the layperson. All of this was achieved through advancements such as Recurrent Neural Networks (RNN) and Long Short-Term Memory (LSTM) networks. These were the first major breakthroughs in allowing systems to retain data and build off of previous inputs efficiently.
Once the industry mastered RNNs and LSTMs, they were replaced by Transformers, a direct precursor to modern large language models (LLMs) and generative systems. What’s stunning, though, is that transformer architecture only really emerged in 2017 with the paper “Attention is All You Need”. Written by Google researchers, this paper laid the groundwork for the technology we use today.
All modern LLMs owe it to that paper, as even GPT-3 is a direct spin-off of that idea and, without this architecture, the AI of today would look very different, likely far less advanced. The 2020 release of GPT-3 coincided or, rather, drove a renewed interest in the tech. Investments shot up to $119 billion in 2021 and, by 2023, 30% of that was dedicated to generative AI specifically.
What Makes This History Strange
If we look at the AI history timeline as a chart, it will certainly not appear like a straight line and not even one that has a gradual climb. It’s akin to a rollercoaster - a jagged chart with cyclical turns, massive dips, and meteoric rises, all predicated on things like new research, public opinion, and investor hesitation.
As a result of this decades-long journey, we can identify patterns and make informed judgments about the sector's development. One consistent issue is that any forward movement in the AI sphere is almost immediately hailed as the brightest path to the future. Most times it’s by the public, but, as with Rosenblatt, researchers can also overestimate their capabilities.
What stems from that is a quick cooling on the technology, which comes as soon as people realize that the ideas we have for AI aren’t always supported by contemporary tech. Studying the pattern of AI history reveals that promises were often fulfilled… just a decade too late. It’s a warning sign that indicates AI ideas are outpacing the actual speed of innovation.
To prevent further loops and collapses, we can all take a few key steps that will hopefully stabilize the industry and encourage healthy, measured growth. Let’s talk about that in our parting words.
Lessons for Today’s Builders
The reason we have to go back to when AI was invented is that knowing this long, complex history helps us understand the typical faults in the way people approach AI. While S-PRO’s specialists work with AI every single day, their success with it is partly informed by avoiding our predecessors' mistakes. This means understanding how AI evolves, what makes it stagnate, and how far it can be pushed without compromising results.
We want to make sure that people who rely on AI for their work and their livelihood see the patterns that caused AI winters in the past - the overpromising and failure to live up to imagination. Avoiding this trap can help us keep the AI sector growing stably and getting new capabilities when the technology is truly ready.
The real progress for AI doesn’t happen in a step-by-step process where one release uplifts all others. Instead, it’s a cyclical journey, where a breakthrough envelops the entire industry, and if nobody successfully builds upon it, it leads to some disappointment among investors. Therefore, we find it essential to reframe that mindset and focus on the current achievements of AI.
We hope our guide to the history of AI was helpful and illuminating, showing the complex yet inspiring evolution of this indispensable technology. Our enthusiasm for AI and our desire to create high-quality projects with it are qualities that we want to share with others in this field. If you’re like-minded and have an AI project to discuss, don’t hesitate to message us.