Is reinforcement (machine) learning overhyped?

Imagine you are about to sit down to play a game with a friend. But this isn’t just any friend – it’s a computer program that doesn’t know the rules of the game. It does, however, understand that it has a goal, and that goal is to win.

Because this friend doesn’t know the rules, it starts by making random moves. Some of them make absolutely no sense, and winning for you is easy. But let’s just say you enjoy playing with this friend so much that you decide to devote the rest of your life (and future lives if you believe in that idea) to exclusively playing this game.

The digital friend will eventually win because it gradually learns the winning moves required to beat you. This scenario may seem far-fetched, but it should give you a basic idea of how reinforcement learning (RL) – an area of machine learning (ML) – roughly works.

Just How Intelligent is Reinforcement Learning?

Human intelligence encompasses many characteristics, including the attainment of knowledge, a desire to expand intellectual capacity, and intuitive thinking. Our capacity for intelligence, however, was largely questioned when Garry Kasparov, a champion chess player, lost to an IBM computer named Deep Blue. Besides capturing the attention of the public, doomsday scenarios depicting a world where robots rule humans took hold of mainstream consciousness.

Deep Blue, however, was not an average opponent. Playing with this program is analogous to a match with a thousand-year-old human that devoted their entire life to continuously playing chess. Accordingly, Deep Blue was skilled in playing a specific game – not in other intellectual pursuits like playing an instrument, writing a book, conducting a scientific experiment, raising a child, or fixing a car.

In no way am I attempting to downplay the achievement of the creation of Deep Blue. Instead, I am simply suggesting that the idea that computers can surpass us in intellectual capability requires careful examination, starting with a breakdown of RL mechanics.

How Reinforcement Learning Works

As mentioned previously, RL is a subset of ML concerned with how intelligent agents should act in
an environment to maximize the notion of cumulative reward.

In plain terms, RL robot agents are trained on a reward and punishment mechanism where they are rewarded for correct moves and punished for the wrong ones. RL Robots don’t “think” about the best actions to make – they just make all the moves possible in order to maximize chances of success.

Drawbacks of Reinforcement Learning

The main drawback of reinforcement learning is the exorbitant amount of resources it requires to achieve its goal. This is illustrated by the success of RL in another game called GO – a popular 2-player game where the goal is to use playing pieces (called stones) to maximize territory on a board while avoiding the loss of stones.

AlphaGo Master, a computer program that defeated human players in Go, required a massive investment that included many engineers, thousands of years worth of game-playing experience, and an astonishing 256 GPUs and 128,000 CPU cores. That’s a lot of energy to use in learning to win a game. This then begs the question of whether it is rational to design AI that cannot think intuitively. Shouldn’t AI research attempt to mimic human intelligence? One argument favoring RL is that we should not expect AI agents to behave like humans, and its use to solve complex problems warrants further development. On the other hand, an argument against RL is that AI research should focus on enabling machines to do things that only humans and animals are presently capable of doing. When viewed in that light, AI’s comparison to human intelligence is appropriate.

Quantum Reinforcement Learning

There’s an emerging field of reinforcement learning that purportedly solves some of
the problems outlined above. Quantum reinforcement learning (QRL) has been studied as a way to speed up calculations.

Primarily, QRL should speed up learning by optimizing the exploration (finding strategies) and exploitation (picking the best strategy) phases. Some of the current applications and proposed quantum calculations improve database search, factoring large numbers into primes, and much more. While QRL still hasn’t arrived in a groundbreaking fashion, there’s an expectation that it may resolve some of the great challenges for regular reinforcement learning.

Business Cases for RL

As I mentioned before, in no way do I want to undermine the importance of RL research and development. In fact, at Oxylabs, we have been working on RL models that will optimize web scraping resource allocation.

With that said, here is just a sample of some real-life uses for RL derived from a McKinsey report highlighting current use cases across a wide range of industries:

  1. Optimizing silicon and chip design, optimizing manufacturing processes, and improving yields for the semiconductor industry
  2. Increasing yields, optimizing logistics to reduce waste and costs, and improving margins in agriculture
  3. Reducing time to market for new systems in the aerospace and defense industries
  4. Optimizing design processes and increasing manufacturing yields for the automotive industries
  5. Increasing revenue through real-time trading and pricing strategies, improving the customer experience, and delivering advanced personalization to clients in financial services
  6. Optimizing mine design, managing power generation and applying holistic logistics scheduling to optimize operations, reduce costs and increase yields in mining
  7. Increasing yields through real- time monitoring and precision drilling, optimizing tanker routing and enabling predictive maintenance to prevent equipment failure and outages in the oil and gas industry
  8. Facilitating drug discovery, optimizing research processes, automating production and optimizing biologic methods for the pharmaceutical industry
  9. Optimizing supply chains, implementing advanced inventory modeling and delivering advanced personalizations for customers in the retail sector
  10. Optimizing and managing networks and applying customer personalization in the telecom industry
  11. Optimizing routing, network planning, warehouse operations in transport and logistics
  12. Extracting data from websites with the use of next-generation proxies

Rethinking Reinforcement Learning

Reinforcement learning may be limited, but it’s hardly overrated. Moreover, as research and development into RL increases, so do potential use cases across almost every sector of the economy. Wide-scale adoption depends on several factors, including optimizing the design of algorithms, configuring learning environments, and the availability of computing power.

Aleksandras Šulženko

Product Owner at Oxylabs.io

How E-commerce Marketers Can Win Black Friday

Sue Azari • 11th November 2024

As new global eCommerce players expand their influence across both European and US markets, traditional brands are navigating a rapidly shifting landscape. These fast-growing Asian platforms have gained traction by offering ultra-low prices, rapid product turnarounds, heavy investment in paid user acquisition, and leveraging viral social media trends to create demand almost in real-time. This...

Why microgrids are big news

Craig Tropea • 31st October 2024

As the world continues its march towards a greener future, businesses, communities, and individuals alike are all increasingly turning towards renewable energy sources to power their operations. What is most interesting, though, is how many of them are taking the pro-active position of researching, selecting, and implementing their preferred solutions without the assistance of traditional...

Is automation the silver bullet for customer retention?

Carter Busse • 22nd October 2024

CX innovation has accelerated rapidly since 2020, as business and consumer expectations evolved dramatically during the Covid-19 pandemic. Now, finding the best way to engage and respond to customers has become a top business priority and a key business challenge. Not only do customers expect the highest standard, but companies are prioritising superb CX to...

Automated Testing Tools and Their Impact on Software Quality

Natalia Yanchii • 09th October 2024

Test automation refers to using specialized software tools and frameworks to automate the execution of test cases, thereby reducing the time and effort required for manual testing. This approach ensures that automation tests run quickly and consistently, allowing development teams to identify and resolve defects more effectively. Test automation provides greater accuracy by eliminating human...

Custom Software Development

Natalia Yanchii • 04th October 2024

There is a wide performance gap between industry-leading companies and other market players. What helps these top businesses outperform their competitors? McKinsey & Company researchers are confident that these are digital technologies and custom software solutions. Nearly 70% of the top performers develop their proprietary products to differentiate themselves from competitors and drive growth. As...

The Impact of Test Automation on Software Quality

Natalia Yanchii • 04th October 2024

Software systems have become highly complex now, with multiple interconnected components, diverse user interfaces, and business logic. To ensure quality, QA engineers thoroughly test these systems through either automated or manual testing. At Testlum, we met many software development teams who were pressured to deliver new features and updates at a faster pace. The manual...