The Virginia project (https://uniter2p2.fr/en/projects/) focuses on studying the thermoregulation of newborns from thermal images.
The primary goal of this project is to detect any deterioration in the infant’s health as early as possible using their thermal profile. Over 1,000 images of newborns were captured after birth at four different time points, corresponding to Apgar assessments at 1, 3, 5, and 10 minutes after birth.
The ultimate objective is to analyze the thermal evolution of these infants at these four key moments.
Infrared images were acquired with a FLIR T650sc camera. The T650sc camera is equipped with an uncooled Vanadium Oxide (VOx) microbolometer detector that produces thermal images of 640 x 480 pixels, with an accuracy of +/- 1 °C.
The Virginia software was developed entirely within the R2P2 laboratory (by ldci) using Red programming language (https://www.red-lang.org), and the redCV library for image processing (https://github.com/ldci/redCV). The Virginia software includes add-on modules for decoding images.
THE FLIR MODULE
This module has been tested with different FLIR cameras. Its main function is to decode the metadata contained in any radiometric file and to extract the visible image (RGB), the infrared image (IR), the color palette associated with the IR image as well as the temperatures (in degrees °C) associated to each pixel.
This module uses two external programs :
ExifTool (https://exiftool.org), written and maintained by Phil Harvey, is a fabulous program written in Perl that allows you to read and write the metadata of many computer files. ExifTool supports FLIR files. It works on MacOs, Linux and Windows platforms.
ImageMagick (https://imagemagick.org/index.php) is a free software, including a library, as well as a set of command line utilities, allowing to create, convert, modify, and display images in a very large number of formats. The FLIR module mainly uses the convert utility for MacOs and Linux versions and the magick utility for Windows.
Once the metadata are extracted, we call a Python library: PixelLib
THE PIXELLIB LIBRARY
This superb library written and maintained by Ayoola Olafenwa is used for the semantic segmentation which allows to identify the newborn in the image. We use the latest version of PixelLib (https://github.com/ayoolaolafenwa/PixelLib) which supports PyTorch and is more efficient for segmentation. The PyTorch version of PixelLib uses the PointRend object segmentation architecture by Alexander Kirillov et al. 2019 to replace the Mask R-CNN. PointRend is an excellent neural network for implementing object segmentation. It generates accurate segmentation masks and runs at a high speed that meets the growing demand for real-time computer vision applications.
First, we only look for the class person without looking for other objects in the RGB image. Then, we get the detected mask as a matrix of true or false values. It is then very simple to reconstruct the binary image of the mask by replacing the true values by the white color. With a simple AND logic operator between the FLIR image and the segmentation mask image, we obtain a new image that keeps only the thermal image of the baby. Only the pixel values higher than 0.0.0 (black) will be considered. Here, for example, the values of the baby's crotch will not be included for the various calculations.
After this first operation of body segmentation, we use a double algorithm. The next step is to detect the contours of the body. This operation will detect the contours in the mask as a polygon of vertices connected by a B-Spline curve. The contour detection algorithm uses several techniques. First, two morphological operators of dilation and erosion are successively applied to smooth the contours of the mask calculated by the semantic segmentation. Then we use the Freeman coding chain technique (FCC). This technique allows the coding with a limited number of bits (8) of the local direction of a contour element defined in the image. This allows the constitution of a chain of codes from an initial pixel, considering that a contour element links two related pixels.
When the result of the edge detection is adequate we can proceed to the calculation of the body temperatures. We use a ray-tracing algorithm that makes sure that each pixel of the image belongs to the polygon representing the baby's body. This operation allows us to extract from the 2-D temperature matrix only the body temperatures in the form of a vector which is then used for the different calculations.
The code is not open-source, as we are in the process of registering patents on certain technological innovations. As soon as this is possible, I will give free access to all sources. The idea was just to show that you can do great things with Red.
Freeman H. On the encoding of arbitrary geometric configurations. IRE Transactions on Electronics Computers. 1961. 10:260–268
Panoptic Segmentation. Alexander Kirillov, Kaiming He, Ross Girshick, Carsten Rother, Piotr Dollar; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 9404-9413
Aucun commentaire:
Enregistrer un commentaire