Unleashing the benefits of sensor fusion

By Paul Bearpark, Head of Electronics & Software at 42 Technology

Electronic sensors abound. A typical car has in the region of 100-200 sensors. Miniaturisation through technologies such as MEMS (Micro Electro-Mechanical Systems) has enabled sensors to be employed for many more applications.

The demand for more and better sensors has arisen from a desire for improved performance, new applications, increased automation and improved reliability through condition monitoring.

Sensor fusion

An exciting application of sensors is sensor fusion which is the process of combining outputs from multiple sensors. We’re familiar with fusion in our own sensory system. We were taught from an early age to look and listen before crossing a road. By using both senses we reduce the risk that we step out in front of a vehicle especially in a noisy environment or where our view of the road might be obstructed.

Electronic sensor fusion can often be employed in a similar way to improve accuracy and reliability. However, the benefits extend well beyond this to extending and enhancing system capability, reducing power consumption in power constrained systems and even reducing system cost.

This article provides a high-level overview of the tremendous opportunities for sensor fusion and provides some specific examples.

Common sensor fusion examples

One of the most common applications of sensor fusion is GPS and INS (Inertial Navigation System) in aeronautical navigation systems. The INS can provide much faster updates to the aircraft control system than GPS but its instruments drift over time. The drift-free position provided by GPS can be used to calibrate or reset the INS. In high dynamic situations the GPS signal can be lost but the INS continues to compute position and angle.

In a novel extension to the GPS/INS navigation system, 42 Technology has been contributing to a project integrating a UWB (Ultra Wide Band) sensing system into the GPS/INS to enhance the landing capabilities of an autonomous drone. The UWB system will augment the GPS/INS system to enhance accuracy and provide the drone with commercial jet airliner level of landing reliability but at the fraction of the cost.

Another commonly deployed example of sensor fusion is multimode intrusion detection systems which are combinations of both infrared and microwave sensors. By deploying two different types of sensor, they have a high immunity to false alarms because the detectors use different sensing modalities and are therefore susceptible to different false alarm mechanisms that are unlikely to occur at the same time.

An example of a system that combines identical sensors is a microphone array for locating the direction of gunfire. Typically, the microphones are co-located. A processing unit discriminates gunshots from other noise and triangulation of the bearing to the source by measuring the time of arrival of the sound at each microphone in the array.

Distributed sensing

Often sensing systems can be distributed to provide enhanced performance or even create new applications. In the case of the gunshot detection system a distributed set of sensors enhances the system by making it much easier to distinguish gunfire from other quieter sources of noise since all the sensors must detect the sound of gunfire. It also improves the estimation of the location of source of the gunfire compared with a co-located set of microphones.

Distributed sensing is employed in many different ways. The Square Kilometre Array (SKA) radio telescope to be constructed over the next few years will employ many highly distributed radio receivers to create the world’s largest radio telescope using interferometry techniques to combine the signals. A similar distributed system combines hundreds or sometimes thousands of seismic sensors to create a large array that is capable of creating a 3-D image of geological formations to identify deposits of coal, oil, groundwater and minerals.

A simpler type of distributed sensing approach is used to regulate our driving speeds on many roads by employing ANPR (automatic number plate recognition cameras) to determine our average speed between them.

Distributed sensing requires communication between the sensors and/or communications with a central processing unit. The communications system can be employed for timing and synchronisation, data transfer, and monitoring and control. The SKA system will be transmitting data at an aggregate average rate of 8 Terabits per second!

Methods of combining sensors

The way in which the sensors are combined can occur at different levels. Raw data from the sensors can be combined. This can require high bandwidth communications. In a distributed system the data may need to be transferred over long distances. The SKA uses optical fibre communications to transmit the vast quantities of data and timing information to a central signal processor. Seismic acquisition systems also have demanding communications needs as they transmit raw data and timing.

Transmission of features rather than data generally demands much lower communications speeds. The average speed detectors transmit timing information alongside the vehicle’s registration number.

At the simplest level sensors may transmit only a decision or generate a trigger. A PIR sensor for intrusion detection might just send a message indicating that it had been activated. This might be combined with a second PIR sensor which if also triggered provides high confidence that there is an intruder rather than a false alarm.

If decision making can be made at the sensor this can considerably reduce the communications requirements resulting in lower capex and opex costs. Timing and synchronisation can still be important so communications latency must often be considered.

Managing power consumption

Fusing sensors can considerably reduce system power consumption. Camera traps are a great example of this. They are battery powered devices that can be deployed remotely for long periods on a relatively small battery. They employ low power PIR sensors that trigger the camera to wake up and start recording for a pre-defined period greatly reducing the power consumption compared with a permanently active video sensor.

Networks of portable low power seismic sensors have been deployed for long periods to wake up communications systems to alert security personnel to intrusion. Often these have been integrated with battery powered camera systems so that the alert is accompanied by real time video of the site providing security personnel with situational awareness. When they are comfortable they no longer need the video feed they can remotely return the camera to its sleep state.

Reducing cost

It’s reasonable to assume that more sensors results in higher system cost. That’s not always the case. Sometimes a combination of lower cost sensors might be more cost effective than a single expensive sensor. Networks of microphones for detecting drones might be more effective and less costly than a single radar. However, the combination of the microphones and radar might result in the highest overall performance.

A highly innovative approach to water leak detection was developed by 42 Technology. Acoustic technology is often employed for this application. However, a low cost, low power solution was required for this application. The components needed for acoustic detection of a leak and the continuous computation to distinguish a leak from background noise is a relatively costly and power hungry method. Instead, two temperature sensors were deployed.

One of the sensors measured the temperature of the pipe and the other the ambient temperature. In cold or mild climates the incoming water supply temperature into a property is generally lower than the ambient temperature. Even a small leak will result in sufficient cold water flow to prevent the pipe from reaching the ambient temperature. Therefore, the pipe temperature and ambient temperature never fully converge. This principle can be utilised to indicate a leak. This approach yielded a low cost, very low power leak detection system.

Enhancing capability

Fusing sensors can provide capabilities and enable applications that would often not be possible with a single sensor. Returning to the intrusion detection application, a 360 degree PTZ (pan-tilt-zoom) camera can be enhanced by a network of intrusion sensors. These could be PIR or seismic sensors positioned around the camera.

By setting the camera pre-sets to the positions of the sensors, the camera can automatically pan, tilt and zoom into the area around the sensor when it is triggered. This has the benefits of not requiring an ‘overwatch’ camera, alerting security personnel of an intruder and automatically setting the camera to provide the best view of the intruder.

An enhancement to this would be to add video analytics to automatically confirm the intrusion being a person rather than an animal since the original trigger could have been a false alarm.

The take-away

Reductions in the cost and size of sensors and their increasing capability mean that electronic sensors have become ubiquitous. However, combining sensors can take this to a new level. This could be by enhancing performance, reducing costs, improving accuracy and enabling completely new applications.

There are many successful examples of sensor fusion which range enormously in complexity. However, this is an area still ripe for innovation and potential. It's limited only by our imagination and capacity to create new and exciting sensor systems.

If you would like to find out more please contact Paul:
paul.bearpark@42T.com | +44 (0)1480 302700 | www.linkedin.com/in/paul-bearpark

Paul is an experienced engineer, engineering manager and product manager. He has spent over 25 years working in the field of electronics in both product development companies and technology consultancies.

He's led the development of products through their entire life-cycle from concept to volume manufacture and on-going support and upgrade. He specialises in radio communications, sensor systems and systems engineering.

He has worked in the defence and security, telecommunications, and test and measurement industries.