Authors: Ove Gustafsson and Tomas Lannestedt
Published on IPQ website: 15 October 2012
The present disclosure relates to a thermography arrangement or thermal imaging device, such as a thermal camera or IR camera and further to method embodiments for providing voice feedback in such an arrangement, device or camera.
Thermographic, thermal, or infrared (IR) arrangements, devices or cameras are typically used to present thermal images of an object or an area, or in other words visual representations of incoming infrared (IR) radiation. The incoming radiation may comprise temperature information related to the images scene.
Since temperature may sometimes be indicative of undesired conditions, such as damp, over-heating, heat leakage or the malfunction of a machine part for instance, arrangements or cameras of the above described type may be used to indicate such conditions. For example, excess heat in a machine may indicate wear and possible imminent breakdown of a machine part. Excess heat in an electrical circuit may indicate malfunction of a component of the circuit. Similarly, temperatures that consistently show up as lower or higher than normal, compared for example to a reference value, can be indicative of damp, heat leakage, or of a leakage of water or other liquid.
In some areas of use the user’s main focus from the user is not on the display of the IR camera, for example because it might take a long time to reach a specific temperature, or that the user is sweeping or walking around with the arrangement or camera to cover a large area, and during the capturing of images needs to watch out for other things in the surroundings, that may be more important.
Furthermore, there may be a need to monitor areas, or objects, in places that are unsuitable or unsafe for human beings because of extreme environmental conditions due to for example excessively high or low temperatures, the presence of poisonous substances or high voltages. In such cases, it is desirable to be able to provide continuous monitoring without subjecting humans to these and other conditions, and to alert an operator or user when for example an anomaly, a risk or other temperature related information of interest is detected.
It is an object of arrangements and methods described herein to enable feedback to a user or operator of a thermography arrangement or IR camera when the user does not look directly at an IR image showing the imaged scene.
It is further an object of arrangements and methods described herein to enable feedback to a user or operator during monitoring of conditions in environments that are unsuitable and dangerous to human beings, or in situations where the user’s attention is for some reason not focused on the display of the thermography arrangement during operation.
DESCRIPTION OF THE INVENTIVE CONCEPT
By letting the thermography arrangement, or IR camera, use voice feedback it allows a user of the thermography arrangement, for example in the form of an IR camera to follow a temperature change or alert without looking at the display, the information reach the user by listening to the thermography arrangement, hereinafter sometimes referred to using the exemplifying embodiment IR camera.
By adding speech synthesis functionality, text-to-speech functionality, a dictionary of common words and/or phrases that are used in different alarm situations and/or sounds/audio files/audio clips that represent words and/or phrases that are used in different alarm situations in combination with a speaker system, the arrangement or camera is enabled “to speak” using voice feedback, produced in manners known in the art. Thereby, information may be transferred from the arrangement/camera to a user without the user having to look at a displayed image.
Fig. 1 shows a schematic view of an embodiment of a thermal image processing system, or a thermography arrangement 1, for performing a method according to the present disclosure.
A thermography arrangement 1 comprises an IR imaging device 12 having an IR sensor. The IR imaging device 12 may be any kind of thermal imaging device. According to embodiments, it is preferable that the IR imaging device 12 is able to detect thermal electromagnetic radiation at least, for example, in the interval between 0.7 and 20 µm.
According to embodiments, the thermography arrangement 1 may further comprise a visual imaging device 11 having a visual sensor, and which is mounted so that an optical axis of the visual sensor of visual imaging device 11 is at a distance d from the IR sensor of IR imaging device 12. The visual imaging device 11 may be any known type of visual imaging device, for example a CCD imaging device, an EMCCD imaging device, a CMOS imaging device or an sCMOS imaging device.
According to an embodiment, the thermography arrangement 1 comprises, is connected to or is communicatively coupled to one or more processors 2. In an embodiment, the thermography arrangement 1 is configured to capture IR and/or visible light images using the IR imaging device 12 and/or the visual imaging device 11, whereby captured images according to embodiments may be transmitted to the processor 2 for further processing.
The processor 2 may be a general or special purpose processing engine such as, for example, a central processing unit (CPU), a microprocessor, microcontroller or other control logic or an FPGA (Field-programmable gate array) that comprises sections of code, stored on a computer readable storage medium, that are fixed to perform certain tasks but also other sections of code, stored on a non-transitory computer readable storage medium, that can be altered during use.
According to embodiments, the thermography arrangement 1 may comprise, be communicatively coupled to or be configured to transfer data to and/or from a speech processor. The speech processor may be integrated in the processor 2, or be a separate processor, not shown in figure 1. According to such embodiments, alterable sections of code comprised in any of one or more processors of the thermography arrangement 1 may comprise parameters that are to be used as input for the various tasks, such as: speech synthesis, text-to-speech generation, retrieval of information from a dictionary of common words and/or phrases that are used in different alarm situations and/or sounds/audio files/audio clips that represent words and/or phrases and presentation of the retrieved information to a user of the arrangement or camera, among others. The processor or processors of the arrangement are according to embodiments configured to generate audio information in the form of a voice or speech signal based on the speech synthesis, text-to-speech generation or information retrieved from dictionaries or audio clips. According to an embodiment, the arrangement 1 further comprises sound generators 4, for example speakers 4, configured to receive or retrieve audio information from one or more processors of the arrangement and present the received or retrieved audio information to a user or operator of the arrangement in the form of general alert words or specific words that are used in different specific alarm situations.
In an embodiment, the thermography arrangement 1comprises, for example integrated in the processor 2 or any other processor of the arrangement, a signal converter for converting signals or information in different forms to voice feedback, in manners known in the art, where after the voice feedback can be presented to a user of the arrangement 1.
In an embodiment, the speakers 4 are integrated in a headset and the voice feedback is transferred, through wired or wireless communication, to the headset. Wireless communication may be performed using any suitable communications standard, including but not limited to short range radio communication such as Bluetooth, IR communication and cellular telephony such as GSM or GPRS. Thus, the arrangement 1 according to the invention enables a wireless connection between the camera itself and an earphone used by the operator. This eliminates the security risk that a cable can pose and also enables operation of the camera from greater distances compared to using a cable connection.
According to embodiments, the processor or processors may be configured to determine, based on the incoming IR radiation captured by the IR imaging device 12 and in manners known in the art, the presence of a temperature related anomaly, risk, temperature information of interest – such as for example a temperature above or below a preset threshold or within a preset temperature value interval, or other information of interest. If any such information is detected, the one or more processors may further be configured to: identify the type of temperature information detected, in manners known in the art; generate audio information in the form of speech, in any of the ways described above or any other manner known in the art; and send the generated audio information to the one or more speakers 4 of the system for presentation. Thereby, a detected anomaly, risk or other detected temperature information of interest will lead to an audible alarm/alert in the form of speech, or voice comment. According to an embodiment the voice comment may comprise general words or phrases, such as for example “warning”, or the voice comment may be specifically related to the type of anomaly, risk or other temperature related information of interest detected, for example generating voice feedback such as “hot”, “cold”, “risk for over-heating”, “moisture”, read out the currently detected temperature value: “24 degrees Celsius”, or the like. Any type of voice comment or spoken message that is suitable to the particular situation may be used. This gives the beneficial effect that the user is not only alerted when temperature related information of interest is detected, but that the user is further informed of what has been detected, without having to turn to a displayed image to interpret the IR information. Compared to for example a “beep” or other audio alarm without connection to the temperature related information, such voice feedback provides the user with much more useful information instantly.
In an embodiment, the user is enabled to input voice comments using one or more inputters 5 of the thermography arrangement 1, for example in the form of keyboard, buttons, soft buttons, touch functionality, microphone/voice input or any other suitable input functionality. Such input provided by the user may be stored in a memory of the thermography arrangement, possibly connected to a specific type of temperature related information for retrieval and output by the arrangement according to embodiments described herein.
By adding speech synthesis functionality, text-to-speech functionality, a dictionary of common words and/or phrases that are used in different alarm situations and/or sounds/audio files/audio clips that represent words and/or phrases that are used in different alarm situations in combination with a speaker system, the arrangement or camera is enabled “to speak” using voice feedback, produced in manners known in the art.
In this document, the terms “computer program product” and “computer-readable storage medium” may be used generally to refer to non-transitory media such as a memory 41 coupled to the thermography arrangement 1, the storage medium of processor 2, or the storage medium of a controller 42 coupled to the thermography arrangement 1. These and other forms of computer-readable storage media may be used to provide instructions to processor 2 for execution. Such instructions, generally referred to as “computer program code” or computer program code portions (which may be grouped in the form of computer programs or other groupings), are adapted to control a data processing system to perform the steps and functions of the inventive method. Thus when executed, the computer program code portions enable the thermography arrangement 1 or another processor to perform features or functions of embodiments of the current technology. Further the processing logic or logic describing method steps for performing any or all of the functions described herein may include hardware, software, firmware, or a combination of thereof.
According to an embodiment, the processor 2 communicates with a memory 41, wherein parameters are kept ready for use by the processor 2, and where any captured images being processed by the processor 2 may be stored. The memory 41 may be a random access memory (RAM), a register memory, a processor cache, a hard disk drive, a floppy disk drive, a magnetic tape drive, an optical disk drive, a CD or DVD drive (R or RW), or other removable or fixed media drive. In embodiments, any dictionaries, databases, audio files or logic used for speech generation may be stored in, and retrieved, from any of the memories described above.
The memory 41 may in turn communicate with a controller 42, from where such parameters may originate, for instance through input from a calibration file 43 that can be supplied from a manufacturer, by parameters being supplied by the thermography arrangement 1 itself, or by parameters being supplied by the user using one or more inputters 5 of the arrangement 1. The controller 42 can be a programmable device and determine the parameters needed for performing exemplary methods and how such parameters should interact with the processor 2, and possibly store the parameters in the memory 41 for easy retrieval by the processor 2.
According to embodiments, the thermography arrangement 1 comprises a display 3, configured to display images comprising visible light data and/or IR data to a user of the thermography arrangement 1. The images may for example be pure visible light images, pure IR images or combined visible light and IR images in the form of blended images, fused images or picture in picture images.
According to embodiments, a user is enabled to provide input to the thermography arrangement by using one or more inputters 5 integrated in or coupled to the system, for example in the form of a keyboard, buttons, soft buttons, touch functionality, microphone/voice input or any other suitable input functionality. In an embodiment, the user is enabled, by using one or more inputters 5, to store images in the memory 41 for later viewing or for transfer to another device, such as a computer, for further analysis and/or storage.
In an alternative embodiment, disclosed methods can be implemented by a computing device such as a PC that may encompass the functions of an FPGA specially adapted for performing the steps of the method for one or more embodiments described herein, or encompass a general processor 2 according to the description in connection with Fig. 1. The computing device may further comprise a memory 41, a controller 42 and/or a display 3.
The disclosed methods for producing voice feedback related to detected temperature related anomalies or risks are preferably performed live during operation of the thermography arrangement 1, but it would also be possible to add such voice feedback to still images and play the voice comment when the corresponding image is displayed, using speakers 4 of the thermography arrangement 1 or an external computing device, as described above, at a later time.