Technical Article

What is Deep Learning?

November 11, 2021 by Anish Devasia

A subset of artificial intelligence (AI) and machine learning, deep learning is a growing technique in the industrial space. What is it, how does it work, and how can it be used?

The 1990 Arnold Schwarzenegger movie Total Recall depicts a story in 2084, nearly a century in the future. The future will have self-driving cars, according to the movie. This was science fiction in 1990 but a reality today. Apple, Alphabet, Nissan, Uber, and many more companies are working on autonomous cars. Tesla sells functional self-driving cars to end-consumers, which seem to be working well. 

What made science-fiction a reality in mere three decades? The answer is deep learning.

 

From Artificial Intelligence to Deep Learning

Hephaestus, the Greek God of fire and metalworking, created golden robots and machines. As early as the mechanical Turk in the 1770s, humans have been developing contraptions that mimic human intelligence. Though mechanical Turks were deceptions, later computers capable of defeating humans were developed in the late 20th century. All of those were in efforts to create systems that can mimic the human brain. 

 

Figure 1. A cross-section of the Turk. Image used courtesy of Humboldt University Library

 

Artificial intelligence (AI) is the term given to any computing system that tries to imitate the human brain. Alan Turing’s Turing machine was a primitive AI system that used logic to reach solutions. 

Machine learning (ML) is a subset of AI that uses models to perform tasks. These models are trained with a large amount of data. Deep Blue, the computer that beat world chess champion Garry Kasparov in 1997, is an example of ML. 

Deep learning is again a subset of ML where models learn from data without any human supervision. Deep learning systems are therefore capable of unsupervised learning of unstructured data. 

 

Deep Learning: How Does It Work?

Deep learning is inspired by the structure of human brains that constitute neurons that pass along and process information. The structures used in deep learning are called artificial neural networks (ANN). ANNs can identify and classify information without human supervision and are said to be capable of unsupervised learning. This requires a much larger volume of data compared to conventional ML, which uses supervised learning.

ANNs consist of multiple layers from the input layer to the output layer, through which the data pass through. The remaining layers other than the input and output layers are called hidden layers. The first layer of ANN, or the input layer, is made of neurons. Neurons of an ANN are mathematical functions that closely represent a human neuron. 

 

Figure 2. ANN input and output layer concept.
 

Information transmission through different layers is through connecting channels. Every node in layers of ANN is connected to every node in the next layer with these channels. Each channel has a value attached to it, referred to as its weight; hence, the channels are called weighted channels. 

All neurons in the hidden layers have a unique number associated with them called bias. Information passes from one layer to the next channel, taking the weights associated with the channels. When it reaches the neuron in the next layer, the bias is added to the weighted sum of inputs. 

The result of this mathematical operation is fed to the activation function. The activation function decides whether the neuron should be active or not. This is done by applying the nonlinear activation function to the result obtained by adding bias to the weighted sums from channels. Activation functions add nonlinearity to the output of a neuron. 

Only the neurons that are active after the application of the activation function can send information to the next layer. This continues until the final layer, the output layer. The weights of the neurons’ channels and biases in hidden layers are constantly adjusted to receive a well-trained model.

Some of the most popular deep learning frameworks are:

 

Data For Deep Learning

Data is the raw material for deep learning. Theoretically, any amount of data improves the models. But considering the data collection effort, training time required, and computational power required to train models, the amount of data for deep learning cannot be infinite. Conversely, too little data will not yield a reliable deep learning model.

There is no cut-and-dry rule for the amount of data required to train a successful model. It depends primarily on the result of trained models. If the models are not adequately reliable, more data is required. There are some rules of thumb for the minimum data required to train deep learning models. 

These are two heuristics for these more common applications with deep learning. Data engineers advocate similar rules for different applications. As with all rules of thumb, these are not perfect and would need change according to the specific application.

 

Industrial Applications of Deep Learning

There are many industrial applications for deep learning. Let us go through some of them.

 

Autonomous Vehicles

Self-driving cars are sold to consumers now, but they also have many industrial applications. Autonomous driving can be incorporated into transporting devices used in factories; for instance, autonomous guided vehicles (AGVs) can go fully autonomous. This eliminates the need for manual labor in such tasks while improving safety and efficiency.

 

Figure 3. An AGV being used in a logistics warehouse.

 

Computer Vision

Computers classify and recognize objects from images. Sometimes, computer vision is part of autonomous vehicles, but it has many more uses in industrial applications. Computer vision can automate object sorting. Computer vision-aided systems can perform quality inspections. It can also automate the surveillance of plant premises and industrial processes.

 

Supply Chain Management

A company’s supply chain is a complex system that spans multiple vendors, suppliers, geographies, and regulations. Managing a huge traffic volume of goods manually is an impossible task. Deep learning can be employed to maintain a healthy supply chain by analyzing large volumes of data generated with the help of IoT (Internet of Things) devices in the supply chain.

 

Medical Applications

Deep learning also has many medical applications. It can be employed to identify anomalies from results of medical imaging like x-rays, MRIs, etc. It can also monitor patient health with attached monitoring devices around the clock. Deep learning can assist in drug discovery by giving the most probable combination of molecules. 

Deep learning has many more industrial applications in aerospace, space exploration, mining, navigation, defense systems, cybersecurity, and the list goes on. Deep learning is being adopted quickly across all industry verticals and will soon be a critical and unavoidable element of Industry 4.0.