Neural Networks: Three Key Types

Neural Networks are often seen as mysterious “black boxes”. In this article, we want to unpack them a little bit – or at least identify the three fundamental types! We’ll explore how they function, and what sort of problems they are good at solving.

General Principles:

Neural Networks are an approach to Machine Learning inspired by the way the brain works. A network is made up of many small parts called "neurons," which are connected together in layers. Each neuron takes in a number of inputs, performs some simple calculations on them, and produces a single output. The output of one neuron can be used as the input for another neuron. This process is repeated until the final layer produces an output that represents the neural network's answer to a given problem. The internal workings of the neural network are not always easy to understand, but it is possible to improve the model over time by adjusting the connections, a process called “training”.

Type 1: Dense Neural Networks

Dense Neural Networks, also known as fully connected networks, are used to make predictions or decisions based on data. They are made of many layers of interconnected, artificial neurons. All the neurons in one layer are connected to all the neurons in the next. They take an input, transform it through the layers using weights (that can be adjusted during training), and produce an output. This is a feedforward method: data moves in one direction, from input layer to output layer.

Dense Neural Networks are particularly good at processing lots of data, or data which contains a large number of features. They can predict a label or a continuous value: you might use a dense neural network to predict the weather based on past information, or to classify images.

Type 2: Recurrent Neural Networks

Recurrent neural networks (RNNs) are a type of neural network that are used to process sequential data, such as a series of words in a sentence or a series of frames in a video. They use a type of layer called a “recurrent layer”, which has connections that loop back onto itself. This enables the network to retain information from previous steps in the sequence and use it to inform the processing of the current step. This makes them great at understanding relationships within the sequence.

The most obvious application for an RNN is language translation. When translating a phrase, the model has to remember the context of the previous words in the sentence.

Type 3: Convolutional Neural Networks

Convolutional neural networks (CNNs) are for processing data that has a grid-like structure, such as an image. CNNs automatically and adaptively learn to recognize patterns and features (more technically, “spatial hierarchies”) from their input data, be they shapes, colours, or textures.

CNNs work by applying a series of "filters" to the data. Three common features in a CNN are:

• Convolutional Layers, which apply the various matrix transformations
• Nonlinear Activation Functions, which help the model learn more complex patterns
• Pooling Layers, which downsample the data to make it easier for the model to generalise

So CNNs are great at identifying objects, classifying images as a specific type of scene, or extracting certain features from images.

In our article comparing Supervised vs Unsupervised Learning, we considered the toy example of classifying cats and dogs. If you wanted to solve this problem with Neural Networks, then a CNN would be the type for you!

Using Neural Networks in the Real World

As part of our twinCity project, we are creating digital twins of our work and home environments on a city-wide basis, to help councils drive forward their sustainability agenda.

Neural networks are a key tool for this project. We are using satellite maps of cities as our input data, so that we can detect certain objects: specifically angled/oriented roof space. As a result, we will be able to compute the total available space which can be used for solar capture. And by tying this to power output data for each postcode, we can help net-zero audits identify the best areas for assistance.

Can you guess which type of neural network we are using? Would you like to learn how to build your own?

Train, Validate and Deploy Trustworthy Machine Learning Models

Our twinLab platform accelerates your time-to-value when building, validating and deploying Machine Learning models by orders of magnitude, and even makes unsolvable problems, solvable.

To find out more, get in touch with the twinLab team and book a demonstration of digiLab’s Machine Learning platform today.

digiLab Academy: Learn "AI in the Wild"

In 2023, we are launching the digiLab Academy. Our first course, “AI in the Wild”, will enable engineers in industry to apply Machine Learning methods to their real-world problems, quickly and efficiently.