The vision of seeing buildings


Vision functionality from consumer electronics could revolutionise building automation and pave the way for full integration with security systems, explains Gabriel Sikorjak, European product marketing manager at Omron Electronic Components

Vision has become an accepted part of the user interface of consumer electronics. A phone can recognise its owner and respond accordingly. Games consoles and televisions respond to our gestures, dispensing with the need for a remote control and simplifying access to their increasingly advanced functions.

In this context, it seems strange vision based functionality has yet not spread into building automation systems. Premises commonly have CCTV systems, but these simply transmit an image. The interpretation of that image is left to a human operator, who is normally only responsible for security. The passive infra-red detector (PIR) in a room will switch the lights off in a room while a meeting is in progress, even though the security camera in the same space can clearly see the occupants.

Building automation potential
Building automation systems are a much more demanding application than mobile phones and games consoles, but innovative vendors are developing technology originally created for mobile phones to deliver solutions suitable for the professional environment.

Using such technology, building automation systems can interpret the image collected by a camera. They can tell the difference between a human and a cat. They can see and respond to gestures and gauge their mood, their age and their gender. They can also recognise an individual. Using these sensors, security systems and building automation systems could eventually be integrated together using one set of vision modules. The data they collect can be responded to automatically, saved or collated centrally, and passed to an operator only when necessary.

Potentially, an office can recognise an individual when he or she arrives, and set up heating and lighting just the way they like it. The next generation of lighting systems can also respond to gestures adjusting lighting, air conditioning or heating. 

The Omron HVC module is the first vision module specifically aimed at applications like building automation, available in low volumes and readily integrated by any designer without any need to understand the complex algorithms needed to recognise gestures, faces and expressions or the optical design. The module is a fully integrated, plug-in solution. The developer can just look at the outputs and configure the system to make appropriate decisions depending on their status.

Security integration
Since modules such as HVC can reliably detect the presence, location and identity of occupants in a room, they will ultimately allow full integration between access control and building services systems, enhancing the functionality of both. It can identify individuals and permit or deny access. Security can be alerted if unauthorised individuals are present in specific areas, and time-stamped photographs stored. The system can ensure that an authorised ‘host’ is present in a meeting room, and count the number of visitors to public areas. Without doubt, the introduction of vision systems will add a new dimension to commercial premises management.

Core software
Modules like the HVC rely on consumer technology. HVC builds on the Omron OKAO Vision software, a proven set of image recognition algorithms used in over 500 million digital cameras, mobile phones and surveillance robots around the world. It integrates ten image key image sensing functions, a camera and an external interface. Developers can detect a human face, hand or body, and implement face recognition, gender detection, age estimation, mood estimation, facial pose estimation, gaze estimation, and blink estimation. In each case the module returns a value together with a degree of certainty, allowing the programmer to configure the response appropriately for each individual application.

Key features of the module include speed and consistency of response, and the distance over which it can take readings. For example, HVC can capture, detect and recognise a face over a distance of 1.3m in 1.1s and will provide a confidence level with its reading. Blink and gaze estimation takes under 1s. The module can evaluate the subject’s mood based on one of five expressions. It can also detect a human body up to 2.8m away and a hand at a distance of 1.5 m. HVC implements the OKAO software on a hardware platform complete with camera, processor and data interface optimised specifically in terms of its digital and optical design for this application.

The algorithms required to implement these functions are complex and processor intensive, but they are handled entirely within the module. The memory and processor intensive computations involved make no demand at all on the host system. The module also frees the system developer from having to devote time to building and testing algorithms, a complex and time intensive task.

Specialist buildings
The potential for vision in building control and security is huge – particularly in specialist buildings.  In schools, modules can distinguish between a child or student and an adult, and a member of staff and a stranger, providing appropriate alerts as required. In healthcare, modules can recognise individual patients and even detect their mood to greatly enhance patient management. On industrial sites, the system can identify when visitors stray off the safe walkways and trigger warning messages.

Face recognition and gesture control have been features of high volume consumer electronics for some time. Their potential to revolutionise building automation systems is only just beginning to be recognised.