"Multimodal people countingvia audioand videofor needs-based air conditioningof workspaces"

ZIM funding code KK5270402PR3

Participants

• Prof. Dr Jens-Peter Akelbein
• Prof. Dr Elke Hergenröther
• Laurens Sillekens
• Oskar Rudolf
• Bettina Kurz

Project partner

• Thermokon Sensor Technology GmbH

Duration 11/2023 - 10/2025

Project description

Initial description and market demand

A poor indoor climate quickly affects our well-being. Freezing or sweating, headaches, burning eyes, dry lips or fatigue are possible consequences. In large non-residential buildings (e.g. administrative buildings, offices, schools, hotels and factories), all rooms are therefore usually heated or cooled to standardised temperatures. Modern building technology controls the air conditioning via a centralised building management system (BMS), which accesses a network of sensors for temperatures, air pressure, relative or absolute humidity,_{CO2 concentration} and consumption meter readings. The aim is to operate systems such as heating, ventilation or cooling equipment more in line with demand, thereby reducing energy consumption and operating costs.

The room climate is largely determined by the number of people in the room. For example, a fully occupied conference room requires significantly more fresh air supply, but less external heat. With information about the number of people, the BMS could calculate the radiated body heat (approx. 120 W) and the oxygen consumption of the people in the room. This could further improve the indoor climate and save energy.

Until now, automatic people counters have mainly been used at entrance areas in retail or passenger transport to compile visitor statistics. However, these high-resolution stereo vision camera systems with powerful processors are too expensive for widespread use in all rooms of a non-residential building. In addition, integrating the sensors into existing building technology would be disproportionately complex and there are also concerns about data protection.

Project objective

The intended technological development is the realisation of a multimodal sensor system for non-residential buildings for installation on the ceiling, which estimates the number of people in the room. This is a completely new class of device that is being developed specifically for simple integration into existing building technology. To minimise hardware costs, a central hub with a single low-resolution wide-angle camera is to be used in each room. People are to be recognised on image segments of only around 20 pixels. It is deliberately not possible to identify individuals at this low resolution. In addition, up to 8 microphone satellites are connected to the ceiling of the room to record sounds to recognise and localise people. The recorded audio and video data is processed by a local microprocessor in real time. The idea behind the multimodal approach is that the two channels complement and compensate for each other if one of the systems can only capture the room situation to a limited extent, e.g. if people are not in the camera's field of vision or do not emit any sounds that could be captured by the microphone. Only the number of people calculated in the microprocessor and their localisation in the room is forwarded to the BMS. Recordings of conversations or images should therefore not leave the room and are deleted after being analysed. The project includes the development of the microphone satellites, the integration of a camera into the hub and the development and implementation of algorithms for analysing the audio and video data. A combination of image and sound processing methods and artificial neural networks (ANN) will be used for this purpose.