# Grating cell operator for image processing and computer vision

M. Hol, F.E. Kalsbeek and N. Petkov, University of Groningen,
Department of Computing Science, Intelligent Systems

## Explanation of parameters

### Gabor filtering

This block implements one or multiple convolutions of an input image with a two-dimensional Gabor function: The Gabor function for the specified values of the parameters "wavelength", "orientation", "phase offset", "aspect ratio", and "bandwidth" will be calculated. If lists of values are specified under "orientation(s)" and "phase offset(s)", only the first values in these lists will be used. The output of the Gabor filter is shown and used directly as input for the grating filter.

Two-dimensional Gabor functions were proposed by Daugman  to model the spatial summation properties (of the receptive fields) of simple cells in the visual cortex. They are widely used in image processing, computer vision, neuroscience and psychophysics. The parameterization used in Eq.(1) follows references [2-5] where further details can be found.

Wavelength (λ)

This is the wavelength of the cosine factor of the Gabor filter kernel and herewith the preferred wavelength of this filter. Its value is specified in pixels. Valid values are real numbers equal to or greater than 2. To prevent the occurrence of undesired effects at the image borders the wavelength value should be smaller than one fifth of the input image size.
When the grating filter is aplied on the Gabor filter output, wavelength describes the periodicity of the bars on wich the grating filter reacts.

Orientation(s) (θ)

This parameter specifies the orientation of the normal to the parallel stripes of a Gabor function. Its value is specified in degrees. Valid values are real numbers between 0 and 360.

For one single convolution, enter one orientation value and set the value of the last parameter in the block "number of orientations" to 1.

If "number of orientations" is set to an integer value N, N >= 1, then N convolutions will be computed. The orientations of the corresponding Gabor functions are equidistantly distributed between 0 and 360 degrees in increments of 360/N, starting from the value specified in "orientation(s)".

An alternative way of computing multiple convolutions for different orientations is to specify under "orientation(s)" a list of values separated by commas (e.g. 0,45,110). In this case, the value of the parameter "number of orientations" is ignored.

The Orientation(s) are also used for the grating filter. Then the value describes the orientation of bars on wich the grating filter should react.

Phase offset(s) (φ)

The phase offset φ in the argument of the cosine factor of the Gabor function is specified in degrees. Valid values are real numbers between -180 and 180. The values 0 and 180 correspond to center-symmetric 'center-on' and 'center-off' functions, respectively, while -90 and 90 correspond to anti-symmetric functions. All other cases correspond to asymmetric functions.

If one single value is specified, one convolution per orientation will be computed. If a list of values is given (e.g. 0,90 which is default), multiple convolutions per orientation will be computed, one for each value in the phase offset list.

Aspect ratio (γ)

This parameter, called more precisely the spatial aspect ratio, specifies the ellipticity of the support of the Gabor function. For γ = 1, the support is circular. For γ < 1 the support is elongated in orientation of the parallel stripes of the function. Default value is γ = 0.5.

Bandwidth (b)

The half-response spatial frequency bandwidth b (in octaves) of a Gabor filter is related to the ratio σ / λ, where σ and λ are the standard deviation of the Gaussian factor of the Gabor function and the preferred wavelength, respectively, as follows: The value of σ cannot be specified directly. It can only be changed through the bandwidth b.

The bandwidth value must specified as a real positive number. Default is 1, in which case σ and λ are connected as follows: σ = 0.56 λ.

The smaller the bandwidth, the larger σ, the support of the Gabor function and the number of visible parallel excitatory and inhibitory stripe zones.

Number of orientations

Default value is 1. If an integer value N, N >= 1, is specified then N convolutions will computed. The orientations of the corresponding Gabor functions are equidistantly distributed between 0 and 360 degrees, with increments of 360/N, starting from the value specified in "orientation(s)". For this option to work, one single value must be specified for the parameter "orientation(s)".

### Half-wave rectification (HWR)

Enable HWR

If this option is enabled, all values in the convolution results below a certain threshold value will be set to zero (HWR is disabled by default).

HWR threshold (%)

The threshold value can be specified as a percentage of the maximum value in a given convolution result of the Gabor filter. If this percentage is set to 0, all negative values in that convolution result will be changed to 0.
Thresholding can be done Local or Global. Global thresholding will take the maximum of the Gabor filtered image and threshold accordingly. Local thresholding will use local maxima of the Gabor filtered image and threshold accordingly. The size of the window over which the local maximum is taken depends on the wavelength and the number of simplecells. The window size in X and Y direction is calculated as follows: ### Superposition of phases

If a list of multiple values is entered under parameter "Phase offset(s)" of the "Gabor filtering" block, multiple convolutions will be computed for each orientation value specified, one convolution for each phase offset value in the list. The convolution results for the different phase offset values of a given orientation can be combined in one single output image for that orientation. This combination can be done in different ways, using the L2, L1 or L-infinity norms. If the L2 norm is used, the squared values of the convolution results for the concerned orientation will be added together pixel-wise and followed by a pixel-wise square root computation to produce the combined result. The L1 and the L-infinity norms correspond to the pixel-wise sum and maximum of the absolute values, respectively. Default is the L2 norm. This choice, together with the default (0,90) of the "Phase offset(s)" of the "Gabor filtering" block, implements the Gabor energy filter that is widely uses in image processing and computer vision. One can also choose not to apply superposition of phases ("None").

Grating filtering

The grating filter responds strongly to bar gratings of a given orientation and periodicity but very weakly or not all to single bars. The grating filter uses the output of the Gabor filter as input. Parameters that the grating filter uses are "beta", "rho" and "nrsimplecells", but it also uses some of the other variables like "wavelength" and "orientation".
The model of grating cells consists of four stages, in the first stage the input image is filtered using a Gaborfilter. The second stage is calculating the simplecell response on the Gabor filtered image. The Third stage is calculating the grating subunit response on the simplecells. The fourth and final stage is calculation of the grating cell response on the grating subunits.
For a detailed explanation of the grating filter refer to  and .

Grating subunits

So called grating subunits are computed using as input the center on and center off simplecells with symmetric receptive fields. The model of the grating subunit is conceived in such a way that the unit is activated by a set of nrsimplecells/2 bars with the appropriate periodicity and are summed together within in a certain area to compute the response of the grating cell.
A quantity called the activity of the grating subunit with position , preferred orientation and preferred grating periodicity is computed as follows:  (3)
where is a threshold parameter with a value smaller than but near 1 (e.g. ) and the auxiliary quantities and  are computed as follows: (4)
and =max{ }
The quantities , are related to the activities of simple cells with symmetric receptive fields along a line segment of length passing through point in orientation . is and is . For example, if 6 simplecells are selected, = -3 and = 2. This causes to have the values: -3,-2,-1,0,1,2. When a certain position meets the condition set by formula 3 the position will get the value 1 resulting in an binary two dimensional array. For more information about these formulae, please refer to .

Beta

The parameter Beta determines the size of the area over which effective summation takes place. Shown in the formula below: (5)
Increasing beta will result in a larger Gaussian kernel, the Gaussian kernel is used in an convolution on the gratingsubunits. The size of beta determines the blur on the gratingfilter output.
A value of beta = 5 results in a good approximation of the spatial summation properties of grating cells.

Rho

Rho is the threshold parameter describing how much bars may differ in intensity from each other. In formula 3 the expression states that if a maximum found within a simplecell is lower than rho times the maximum of all simplecells s1 to s2, then those simplecells on that position don't take part in grating. In other words, rho describes how much bars may differ in intensity to still make part of a grating.
A value of rho smaller then but near one (e.g. a value of rho close to 0.85 gives a good result).

Minimum numbers of bars

Neurophysiological research has shown that the brain reacts strongly to bar grating of a given orientation and periodicity but very weakly or not at all to single bars. It is shown that the minimum number of bars to which then brain reacts strongly lies between four and eight bars of apropriate periodicity and orientation. With the parameter 'minimum number of bars' one can specify to minimum number of bars to which the filter reacts.

The top subfigure in figure 1 shows on the X-axe the spatial distribution of the bars and on the Y-axe the graylevel intensity of those bars. The intensity of the bars must be within a certain bandwith bounded by the maximum pixel intensity within a length of pixels determined by Lambda and a fraction of that intensity determined by the value Rho for grating to be detected. The bottum and middle subfigure in figure 1 show the reaction of to centre on and centre off grating subunits on the bars. As shown in the middle subfigure centre-on cells react strongly to white bars where the bottum subfigure show that centre off cells react strongly to black bars. Since one bar is detected with two simplecells a centre on and a centre off cell the number of simplecells is always two times the minimum number of bars. Figure 1

Grating consists of at least (number of simplecells)/2 consecutive bars with the right wavelength intensity and orientation. Grating subunits near the starting and ending point of a grating will show non or low activity (See formula 3). To compensate for this behavior padding to grating is introduced, padding to grading will activate gratingsubunits which take part in the grating but are not activated because of the activation constraints in formula 3.
When padding to grating is enabled (default), all bars of the grating, from the first bar to the last, will be highlighted in the output image. When padding is disabled, the first and last few bars are not highlighted in the output.
For example: when nrsimplecells=8 and padding disabled, then 4 bars to the left and 4 bars to the right will not be highlighted as grating.

## References

1. J.G. Daugman: Uncertainty relations for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters, Journal of the Optical Society of America A, 1985, vol. 2, pp. 1160-1169.
2. N. Petkov: Biologically motivated computationally intensive approaches to image pattern recognition, Future Generation Computer Systems, 11 (4-5), 1995, 451-465.
3. N. Petkov and P. Kruizinga: Computational models of visual neurons specialised in the detection of periodic and aperiodic oriented visual stimuli: bar and grating cells, Biological Cybernetics, 76 (2), 1997, 83-96.