samedi 16 décembre 2017

Red and Pandore Library

I really appreciate using Pandore library (https://clouard.users.greyc.fr/Pandore/) which provides  a lot of useful image processing operators. In many case when processing images, you have to apply successive operators: An image processing application is a chain of operators and Pandore offers very efficient operators. The library is 100% pure C++ code  and can't be easily binded to Red. But we have a solution with red call instruction which can be used to execute a shell process to run another process or program.

Installing Pandore

Go to Pandore website and download the lib which is optimized for Unix, Linux, Windows and macOS. 

The complete installation needs a C++ compiler which is required and  Qt (version >= 4.0.0)  or X11 and Motif for visualization operation (www.trolltech.com).Without these API, operators 'pvisu' and 'pdraw' are not available.  However, the rest of the operators works without Qt (or X11).

Then unpack the distribution on your computer.

Installation is straightforward:   
  1. ./configure
  2. make 
  3. make install
  4. make clean

Using Pandore with Red

The idea is now to call the operators from Red program with call. This is really simple. All operators are in pandore_6.6.7/bin directory and your red code must point to this directory
panhome: "Your access to/pandore_6.6.7"
change-dir to-file panhome
Then just use call operator 
call/output "bin/pversion" status 
The pversion operator can be used to test if the pandore lib is correctly installed. Here we use wait/output in order to redirect stdout to a string and get the result of the operator: SUCCESS or FAILURE. When using other operators which process image don't forget to use call/wait  to run the operator and wait for exit.

Pandore creates a specific image format (.pan) and thus you code must convert red image to pandore image with pany2pan operator.
In the code example we call  then the pthresholding operator which  build the output image  with the pixels of the input image  that have a value greater or equal than low or lower or equal than high provided by sliders. Other values are set to 0. Lastly we call ppan2jpeg operator which transform the filtered image to  jpg image.

Code sample

Red [
    Title:   "Pandore test"
    Author:  "Francois Jouen"
    File:    %threshold2.red
    Needs:   'View
]

status: ""
isFile: false
srcImg: none
f: ""
lowT: 0
highT: 255
prog: make string! ""

dSize: 256
gsize: as-pair dSize dSize
sldSize: as-pair ((dSize * 2) - 100) 16

; update according to you OS and pandore directory
panhome: "/Libraries/pandore_6.6.7"
change-dir to-file panhome

; is pandore installed?
call/output "bin/pversion" status

; Converts red loaded image to pandore image
red2pan: func [img [file!] return: [string!]] [
    fName: ""
    fName: form second split-path img
    fileName: copy/part fName (length? fName) - 4 ;removes .ext
    append fileName ".pan"
    prog: copy "bin/pany2pan " 
    append append append prog to-string img " /tmp/" fileName
    call/wait prog
    call/output "bin/pstatus" status
    fileName ; returns filename
]

; Pandore thresholding
{pthresholding builds the output image  with the pixels of the input image 
that have a value greater or equal than low or lower or equal than high. Other values are set to 0}

thresholdPan: func [fn [string!] t1 [integer!] t2 [integer!]] [
    prog: copy "bin/pthresholding "
    append append append prog form t1 " " form t2
    append prog " /tmp/" ; default dir for storing pandore images
    append append prog fn " /tmp/result.pan"
    call/wait prog 
    call/output "bin/pstatus" status    
]

;Converts to jpg
pan2JPG: does [
    prog: copy "bin/ppan2jpeg 1.0"
    append append prog " /tmp/result.pan" " /tmp/result.jpg"
    call/wait prog
    call/output "bin/pstatus" status 
]



; Removes all pandore images in /tmp/ directory
removePanImg: does [call "rm /tmp/*.pan" 
                    call "rm /tmp/*.jpg"
                    call "rm /tmp/pand*"
                    sb2/text: "All pandore images removed"]

; Loads red image
loadImage: does [
    isFile: false
    clear sb2/text
    canvas2/image: none
    tmpFile: request-file
    if not none? tmpFile [
        srcImg: load tmpFile
        canvas/image: srcImg
        isFile: true
        sb2/text: "Red Image loaded"
    ]
]

; Processes image
process: does [
    resImg: %/tmp/result.jpg
    sb2/text: copy "Shows filtered pan image: "
    thresholdPan f lowT highT   
    pan2JPG
    append sb2/text status
    if exists? resImg [canvas2/image: load resImg]
]

; ***************** Test Program ****************************
view win: layout [
        title "Pandore Thresholding"
        button 100 "Load Image"     [loadImage f: red2pan tmpFile process]          
        button 70 "Quit"            [removePanImg Quit]
        return
        text 50 "Low" 
        sl1: slider sldSize [lowT: to-integer face/data * 255 
                            lowsb/text: form lowT 
                            if isFile [process]] 
        lowsb: field 40 "0"
        return
        text 50 "High"
        sl2: slider sldSize [highT: to-integer face/data * 255 
                            highsb/text: form highT
                            if isFile [process]]  
        highsb: field 40 "255"
        return
        text dSize "Source"
        text dSize "Result"
        return
        canvas:  base gsize black
        canvas2: base gsize black
        return
        sb1: field dSize
        sb2: field dSize
        do [sb1/text: status sl1/data: lowT / 255.0 sl2/data: highT / 255.0]    
]

Result


Red and Pandore: a great association !



dimanche 3 décembre 2017

Face processing BRFV4

Since Red and RedCV are under development, I use other librairies for scientific research. This year we begin a series on maxillo-facial restoration in infants and children with Roselyne Lalauze-Pol from Robert Débré Hospital in Paris. I was looking for a nice library for face processing and I found BRFV4 (https://www.beyond-reality-face.com/overview) which is really useful and well-done.
Don't hesitate to test their demo code and download the trial version of the library. Really nice work !

SAMPLE

This code was written with QT 5.9.3 and BRFV4 library. Kid image was found on the Internet :)


dimanche 26 novembre 2017

RedCV: Convex Hull

The convex hull problem in geometry applied to image processing tries to find the smallest convex set containing the points. This is useful to process shapes in images for example.
There are many approaches for handling this problem, but for RedCV we focused on the Quick Hull algorithm, which is one of the easiest to implement and has a reasonable expected running time of O(n log n). A clear explanation of the algorithm can be found here: http://www.ahristov.com/tutorial/geometry-games/convex-hull.html. Thanks to Alexander Hristov for the original Java code.

Red CV rcvQuickHull Function

rcvQuickHull
Finds the convex hull of a point set
rcvQuickHull: function [points [block!] return: [block!] /cw/ccw]
points: Input 2D point set as pair!
Returns output convex hull as a block of pair
/cw/ccw : Orientation flag. If cw, the output convex hull is oriented clockwise. Otherwise, it is oriented counter-clockwise. The assumed coordinate system has its X axis pointing to the right, and its Y axis pointing upwards.


Code Sample calling rcvQuickHull

Code uses  generatePoints function to create the set of points as pair!
Code also uses rcvContourArea which calculates the area of polygon generated by rcvQuickHull function.
Data visualization is made with Red Draw DSL in showHull function.
Red [
    Title:   "Test images Red VID "
    Author:  "Francois Jouen"
    File:    %convexArea.red
    Needs:   'View
]

{Quick Hull implementation
Based on Alexander Hristov's Java code
http://www.ahristov.com/tutorial/geometry-games/convex-hull.html}

; all we need for computer vision with red
#include %../../libs/redcv.red ; for red functions

margins: 10x10      
isize: 256x256
nbpMax: 100
nbp: 3
radius: 2.5
cg: 0x0
img: rcvCreateImage isize

plot: copy []
points: copy []

random/seed now/time/precise

generatePoints: does [
    nbp: random nbpMax
    if nbp < 3 [nbp: 3]
    nbpf2/text: form nbp
    canvas/image/rgb: 0.0.0
    plot: copy [fill-pen green]
    canvas/image: draw img plot
    points: copy []
    i: 1
    while [i < (nbp + 1)] [
        p: random 128x128 
        p:  64x64 + p
        append points p
        append plot 'circle 
        append plot p
        append plot radius
        i: i + 1
    ]
    append plot [fill-pen off line-width 1 pen red]
]


showHull: does [
    clear list/data 
    either cb2/data [chull: rcvQuickHull/ccw points] [chull: rcvQuickHull/cw points]
    n: length? chull
    append plot 'polygon
    sumX: 0
    sumY: 0
    ; for centroid
    foreach p chull [append plot p sumX: sumX + p/x sumY: sumY + p/y]
    cg/x: sumX / n
    cg/y: sumY / n 
    
    if cb1/data [i: 1 foreach p chull [append plot reduce ['text p form (i)] i: i + 1]]
    i: 1
    foreach p chull [s: form i append append  s " : " to string! p
                    append list/data s i: i + 1]
    
    if cb3/data [append plot reduce ['fill-pen red 'circle (cg) radius + 1]]
    if cb4/data [
        append plot [fill-pen off line-width 1 pen green line 0x128 256x128 
        pen off pen green line 128x0 128x256]
    ]
    
    canvas/image: draw img plot
    if cb5/data [areaF/text: form rcvContourArea/signed chull]
]


view win: layout [
    title "Quick Convex Hull Area"
    origin margins space margins
    text "Max Number of points"
    nbpf: field 50 data nbpMax [if error? try [nbpMax: to integer! face/data] [nbp: nbp]]
    
    button 80x30 "Generate" [if error? try [nbpMax: to integer! nbpf/data] [nbpMax: nbpMax] 
                          generatePoints canvas/image: draw img plot showHull]
    nbpf2: field 50
    pad 180x0
    button 50 "Quit"    [Quit]
    return
    cb1: check "Show Numbers"
    cb2: check 50 "CCW"
    cb3:  check "Centroid"
    cb4: check 50 "Axes"
    cb5:  check 60 "Area"
    areaF: field 100
    return 
    canvas: base 512x512 img
    list: text-list 100x512 data []
]

 


 

 





Reading video files with Red

The support for video files is planned in the future versions of Red. But actually Red and RedCV cannot read and process video files. Sometimes I need to quickly read some video files and I appreciate using ffmpeg tools such as ffplay. This code is just a simple way to read videos with ffplay from Red. The code was developped under macOS but can be adapted to Windows OS.
Enjoy :)

A sample




The code


Red [
    Title:   "macOS FFMPEG Player"
    Author:  "Francois Jouen"
    File:    %rffplay.red
    Needs:   'View
]

fileName: ""
prog: "" 
isFile: false

command: {List of commands
q, ESC: Quit.
f: Toggle full screen.
p, SPC: Pause.
m: Toggle mute.
9, 0:  Decrease and increase volume respectively.
/, *: Decrease and increase volume respectively.
a: Cycle audio channel in the current program.
v: Cycle video channel.
t: Cycle subtitle channel in the current program.
c: Cycle program.
w: Cycle video filters or show modes.
s: Step to the next frame.
left/right: Seek backward/forward 10 seconds.
down/up: Seek backward/forward 1 minute.
page down/page up :Seek to the previous/next chapter. or if there are no chapters Seek backward/forward 10 minutes.
right mouse click: Seek to percentage in file corresponding to fraction of width.
left mouse double-click: Toggle full screen.
}


loadFile: does [
    isFile: false
    tmp: request-file
    if not none? tmp [
        fileName: to string! to-file tmp
        win/text: fileName
        isFile: true
    ]
]

playFile: does [
    if isFile [
        prog: copy "ffplay '" 
        append append prog FileName "'" 
        either cb/data [call/console prog] [call prog]
    ]
]


view win: layout [
    title "macOS ffplay"
    origin 10x10 space 10x10
    button "Load" [loadFile]
    button "Play" [playFile]
    cb: check "Show Console" 
    button "Quit" [prog: "killall ffplay" call prog Quit]
    return
    info: area 500x280
    do [info/text: command]
]

jeudi 22 juin 2017

RedCV: Normalized convolution on matrices

2-D Convolution is very useful to process image and to code various filters. A 2-D convolution can be thought of as replacing each pixel with the weighted sum of its neighbors. The kernel is another image, of smaller size, which contains the weights as illustrated below.

Depending on kernel and pixel values, weighted sum of pixel neighbors is signed (positive or negative) and can be greater than byte values [0..255] that are used to represents RGB values of the images. 
In most of cases, 2D Convolution algorithms include an implicit cut-off which is similar to a binary filter: If the weighted sum is <= to 0, then weighted sum equals to 0 and if the weighted sum is > to 255, then weighted sum equals to 255. 
In redCV, rcvConvolveMat function integrates this binary processing of weighted sum.  However, depending on the nature of the image you process, this cut-off can induce more or less correct result when applying kernel filter.

Weighted sum normalization

A way to avoid this kind of problem is to use a normalization of the weighted sums. This is done by rcvConvolveNormalizedMat function in RedCV. 
The idea is rather simple. First we look for the minimal (wsMin) and maximal (wsMax) weighted sums that are calculated when applying the kernel convolution. The difference wsMax - wsMin gives the range of the variation in image when convolution is applied. This allows to calculate a scale factor which is equal to 255 / (wsMax - wsMin). Then, it's really easy to remplace each weighted sum (wsValue) with this formula  wsValue = (wsValue - wsMin) * scale: each wsValue will be in 0..255 range.

Code sample

Red [
    Title:   "Matrix tests "
    Author:  "Francois Jouen"
    File:    %matLaplacian.red
    Needs:   'View
]

#include %../../libs/redcv.red ; for redCV functions
; laplacian convolution filter for sample
mask: [-1.0 0.0 -1.0 0.0 4.0 0.0 -1.0 0.0 -1.0]
isize: 512x512
bitSize: 32
img1: rcvCreateImage isize
img2: rcvCreateImage isize
img3: rcvCreateImage isize

loadImage: does [
    canvas1/image/rgb: black
    canvas2/image/rgb: black
    canvas3/image/rgb: black
    tmp: request-file
    if not none? tmp [
        img1: rcvLoadImage tmp
        img2: rcvCreateImage img1/size
        img3: rcvCreateImage img1/size
        mat1: rcvCreateMat 'integer! bitSize img1/size
        mat2: rcvCreateMat 'integer! bitSize img1/size
        mat3: rcvCreateMat 'integer! bitSize img1/size
        ; Converts to  grayscale image and to 1 Channel matrix [0..255]
        rcvImage2Mat img1 mat1  
        ; Standard Laplacian convolution                                    
        rcvConvolveMat mat1 mat2 img1/size mask 1.0 0.0 
        ; Normalized Laplacian convolution          
        rcvConvolveNormalizedMat mat1 mat3 img1/size mask 1.0 0.0   
        ; From matrices to Red images
        rcvMat2Image mat2 img2                                      
        rcvMat2Image mat3 img3      
        ; show results                              
        canvas1/image: img1
        canvas2/image: img2
        canvas3/image: img3
        rcvReleaseMat mat1
        rcvReleaseMat mat2
        rcvReleaseMat mat2
    ]
]


; ***************** Test Program ****************************
view win: layout [
        title "Laplacian convolution on matrix"
        button "Load" [loadImage]
        button 60 "Quit" [  rcvReleaseImage img1 
                            rcvReleaseImage img2
                            Quit]
        return
        text 100 "Source" pad 412x0 
        text 120 "Standard convolution"
        pad 402x0 
        text "Normalized convolution"
        return
        canvas1: base isize img1
        canvas2: base isize img2
        canvas3: base isize img3
]

Result



dimanche 28 mai 2017

RedCV: Motion detection with differential images

The Red code is based on the following paper: Collins, R., Lipton, A., Kanade, T., Fijiyoshi, H., Duggins, D., Tsin, Y., Tolliver, D., Enomoto, N., Hasegawa, O., Burt, P., Wixson, L.: A system for video surveillance and monitoring. Tech. rep., Carnegie Mellon University, Pittsburg, PA (2000).

The idea is to use the webcam and analyse motion using differential images.

Differential Images

Differential Images are the result of the subtraction of two images:  Idif (x, y) = I1(x, y) − I2(x, y).

A differential image shows the pixels differences between two images. With those images you can make movement visible.

Based on the quoted paper,  we use a differential image calculated from three consecutive images It−1 (previous image),  It (current image) and It+1 (next image). The advantage of this technic is that the motionless  background is removed from the result, and only motion pixels are visible.


RedCV offers the possibility to subtract two images from each other using rcvAbsDiff. Since logical operations on images are also implemented, we use rcvAnd to achieve the final differential image.

Code

Implemented code is simple to understand and just requires a time event to activate the camera. When the time event occurs, differential image is calculated and updated in the base object. To-image Red function allows to copy the webcam image to a Red image. Thanks Red!

Red [
    Title:   "Test image operators and camera Red VID "
    Author:  "Francois Jouen"
    File:    %motion.red
    Needs:   'View
]

; all we need for computer vision with red
#include %../../libs/redcv.red ; for red functions

iSize: 320x240
prevImg: rcvCreateImage iSize
currImg: rcvCreateImage iSize
nextImg: rcvCreateImage iSize
d1: rcvCreateImage iSize
d2: rcvCreateImage iSize
r1: rcvCreateImage iSize
r2: rcvCreateImage iSize

margins: 10x10
threshold: 32

to-text: function [val][form to integer! 0.5 + 128 * any [val 0]]

view win: layout [
        title "Motion Detection"
        origin margins space margins
        text "Motion " 50 
        motion: field 50 rate 0:0:1 on-time [face/text: to-text rcvCountNonZero r2]
        btnQuit: button "Quit" 60x24 on-click [
            rcvReleaseImage prevImg
            rcvReleaseImage currImg
            rcvReleaseImage nextImg
            rcvReleaseImage d1
            rcvReleaseImage d2
            rcvReleaseImage r1
            rcvReleaseImage r2
            quit]
        return
        cam: camera iSize
        canvas: base 320x240 r2 rate 0:0:1 on-time [
            rcv2gray/average nextImg currImg    ; transforms to grayscale since, we don't need color
            rcvAbsdiff  prevImg currImg d1      ; difference between previous and current image
            rcvAbsdiff  currImg nextImg d2      ; difference between current and next image
            rcvAnd d1 d2 r1                     ; AND differences
            rcv2BWFilter r1 r2 threshold        ; Applies B&W Filter to base/image
            prevImg: currImg                    ; previous image contains now the current image
            currImg: nextImg                    ; current image contains the next image             
            nextImg: to-image cam               ; updates next image
        ]
        return
        text 40 "Select" 
        cam-list: drop-list 180x32 on-create [
                face/data: cam/data
            ]
        onoff: button "Start/Stop" 80x24 on-click [
                either cam/selected [
                    cam/selected: none
                    canvas/rate: none
                    motion/rate: none
                    canvas/image: black
                ][
                    cam/selected: cam-list/selected
                    rcvZeroImage prevImg
                    rcvZeroImage currImg
                    rcvZeroImage nextImg
                    canvas/rate: 0:0:0.04;  max 1/25 fps in ms
                    motion/rate: 0:0:0.04
                    ]
            ]
        text "Filter" 40
        sl1: slider 180 [filter/text: to-text sl1/data threshold: to integer! filter/data ]
        filter: field 40 "32" 
        do [cam-list/selected: 1 motion/rate: canvas/rate: none sl1/data: 0.32 ]
]

Result


mercredi 17 mai 2017

RedCV: how to blend images

Sometimes it's interesting to superpose and mix two images to create a new image. Once again,  the variation of intensity of image can help us to solve this problem. The idea is to apply a factor (between 0. 0 and 1.0) on the image to modify intensity.  This is done for the first image and the second image. The complement of the factor (1.0 - factor) is applied to the second image.


rcvSetIntensity function


RedCV includes  a  function to do the job.

rcvSetIntensity: function [src [image!] dst [image!] alpha [float!]]


The function applies the float factor to the src image and update the result in the destination image.
Then it's very simple to combine a first call of the function on the first image with a factor and a second call on the second image with the complement of the factor:

rcvSetIntensity src1 dest1 alpha
rcvSetIntensity src2 dest2 1.0 - alpha
Then you just need to add both images to create the blended image with the rcvAdd function:

rcvAdd dest1 dest2 dst

Code sample

Red [
    Title:   "Blend Operator "
    Author:  "Francois Jouen"
    File:    %blend2.red
    Needs:   'View
]

;this version uses rcvSetIntensity and not rcvBlend

#include %../../libs/redcv.red ; for redCV functions
margins: 10x10
img1: rcvLoadImage %../../images/lena.jpg
img2: rcvLoadImage %../../images/test.jpg
dst: rcvCreateImage img1/size
tmp1: rcvCreateImage img1/size
tmp2: rcvCreateImage img2/size
alpha: 0.5

blending: function [] [
    rcvSetIntensity img1 tmp1 alpha 
    rcvSetIntensity img2 tmp2  1.0 - alpha 
    rcvAdd tmp1 tmp2 dst
]

; ***************** Test Program ****************************
view win: layout [
        title "Blend Operator Test"
        text 60 "Image 1" 
        f1: field 50 "0.5"
        sl: slider 170 [alpha: face/data * 1.0
                    f1/text: form alpha
                    f2/text: form (1 - alpha)
                    blending
                    ]
        text 60 "Image 2" 
        f2: field 50  "0.5"
        button 60 "Quit" [  rcvReleaseImage img1 
                            rcvReleaseImage img2
                            rcvReleaseImage tmp1
                            rcvReleaseImage tmp2 
                            rcvReleaseImage dst 
                            Quit]
        return
        canvas: base 512x512 dst
        do [sl/data: alpha blending]
]

And the result for a factor = 0.5



image 1


image 2


mixed image