Otoliths are calcareous structures found in the inner ear of osteichthyan fishes, and the shape and structure are species-specific (Koken 1884, Chaine and Duvergier 1931, Schmitd 1969). Evidence of this has long existed (Hecht and Appelbaum 1982, Baldás et al. 1997, Volpedo and Echeverría 2000) and is expanding as new collections are analysed (Baremore and Bethea 2005) and new studies appear (Campana 2004, Furlani et al. 2007, Tuset et al. 2008). Otoliths have been used mainly in the identification of stocks (Bird et al. 1986) and species (Schmidt 1969); phylogenetic (Gaemers 1984) and palaeontological (Nolf 1985) food webs (Fitch and Brownell 1968); eco-morphological (Lombarte and Fortuño 1992) palaeoecology (Cartes et al. 2017); and even sex identification and ageing (Cardinale et al. 2004). In trophic ecology, otoliths are used to identify species in ichthyophageal stomachs and faeces (Pierce et al. 1991, Bustos et al. 2012, 2014). This specific association between the morphology and the species to which they belong allows for the exact determination of the components of the diets of predatory fish animals. There are three pairs of otoliths in the ear, but the sagitta is the otolith with the most significant morphological variability and is therefore the one most studied. The sagitta is also generally used for this type of study because in non-otophysian fishes it is larger than the rest of the otoliths (Pierce et al. 1991), which allows for better observation of its morphology, and it does not suffer the process of calcium reabsorption (Casselman 1981).

The AFORO website (http://aforo.cmima.csic.es) is mainly an open online catalogue of images of sagitta otoliths. Throughout the text, we refer to any sagitta otolith as “otoliths” for simplicity. This website has been in development since 2003 (Lombarte et al. 2006), two decades after the first otoliths were studied using a Fourier analysis of contours (Messieh et al. 1989, Bird et al. 1986, Doering and Ludwing 1990), and offers high-resolution fish otolith images with complete morphometric information, including a shape analysis module that provides mathematical contour descriptors, such as the Fourier transform (FT), curvature scale space (CSS) and wavelet transform (WT) (Parisi-Baradad et al. 2005, 2010, Piera et al. 2005).

At present, the AFORO database contains 5793 high-resolution images that correspond to 1776 species, and its software tools have been used for otolith atlases (Tuset et al. 2008, Sadighzadeh et al. 2012), morphological indices (Tuset et al. 2006), stock identification (Capoccioni et al. 2011, Tuset et al. 2013, Sadighzadeh et al. 2014), automatic species identification (Parisi-Baradad et al. 2005, 2010), eco-morphological studies (Cruz and Lombarte 2004, Lombarte and Cruz 2007, Sadighzadeh et al. 2014), archaeological information (Rodney 2009), biodiversity (Tuset et al. 2016), and, particularly, prey identification (Veiga et al. 2011, Neves et al. 2012, Otalora-Ardila et al. 2014). Furthermore, it contains a small 3D repository of otoliths (Marti-Puig et al. 2015) of the family Sciaenidae (Perciformes), which are specialized in acoustic communication (Cruz and Lombarte 2004) and have an exceptional sulcus acusticus (SA) (Smale 1995, Volpedo and Echevarria 2000, Monteiro et al. 2005), as reported widely (Ramcharitar et al. 2006, Tuset et al. 2008, Lin and Chang 2012).

Currently, AFORO’s functionality of automatic identification of fish species and populations is based on 2D otolith closed contours. It works by comparing the contour that is extracted from an external query image against those that are in the AFORO database. All images in the database have been registered following a strict acquisition protocol, which is also required for incoming query images. However, small errors in registration of the query image are common. These introduce small rotations of the contour resulting in misalignments in the contour reference, which in turn decrease the accuracy of the classifier. Furthermore, two more critical factors cause the system to fail: otoliths in the incoming image appear rotated by approximately 180 degrees with respect to normalization because the rostrum (a part of the otolith) has been misidentified or because the incoming images have been acquired under other normalization criteria.

In addition, in some practical applications only the right sagitta is available, whereas the AFORO database only stores data on the left sagittae. To improve the automatic otolith recognition functionality that is provided by AFORO, in this paper, we study how to relax and also to completely avoid the normalization that is required for images and we explore a strategy for classifying the right sagitta by taking advantage of contour symmetries it has with the left sagitta. This is done by proposing descriptors that take advantage of the well-known discrete Fourier transform (DFT) coefficients and their properties (Proakis and Manolakis 1996). The DFT is very efficient when working with periodical functions, and the contours of the figures can be interpreted as periodical functions by continually moving through it. Features that are based on the CSS or WT are susceptible to rotations and variations in the contour sampling point, and their efficiency decreases when the standardization of input information is slightly mismatched (Marti-Puig and Reig-Bolano 2016). The potential and significant advantage of Fourier methods is that they can relate rotations in space to identify them in other changes in the transformed domain. In the same way, in a periodic time-series, DFT properties also identify positional displacements in the transformed domain. In this paper, we explore those properties to achieve robustness against positional standardization misalignments and the cost of avoiding standardization completely. Some preliminary ideas, such as the use of the major ellipse axis, can be found in Marti-Puig and Reig-Bolano (2016). In this seminal work, a seven-step algorithm was established to perform the query and to analyse the effect of rotation (concerning positional standardization) on the classification rate in the case of a DFT-based parameterization. However, as experiments take the contours from the AFORO database, all the series that defined the contours started at the same reference point. When the outline is extracted from an unknown image in which the otolith may appear rotated, it is necessary to adjust the starting point of the series because the first point of the contour that is found when scanning the image matches the reference point. The contribution of the proposed parameterization in the present work is twofold, in the sense that the same parameters include the rotational normalization and the change in the sampling origin. The use of complex coefficients allows the properties of the DFT to be fully exploited and a compact set of features for automatic identification to be established.

To highlight the contributions of this work and point out the differences from that of Marti-Puig and Reig-Bolano (2016), note that the earlier work used elliptic Fourier descriptors (EFDs), initially proposed by (Kuhl and Giardina 1982). Using EFDs, each mode, each “frequency” of contour variation, requires four real coefficients. The original EFD formulation is dependent on contour position, scale, orientation, the starting point of sampling, the sampling direction (clockwise/anti-clockwise) and the number of samples per contour. In Marti-Puig and Reig-Bolano (2016), the way to relate the EFD coefficients (a_k, b_k, c_k, d_k) of a contour with the same contour rotated, displaced or scaled (a_k’, b_k’, c_k’, d_k’) follows the matrix notation proposed in Nixon and Aguado (2008, p. 307). Note that Nixon and Aguado (2008, p. 304) also proposed invariant contour descriptors derived from EFDs, which reduce four real EFDs (a_k, b_k, c_k, d_k) into one (f_k). Those f_k are invariant to rotations, sampling point and direction and symmetries. However, their use for classification of otoliths provides very poor results compared with EFDs with handmade positional standardization (Marti-Puig and Reig-Bolano 2016). Other invariant descriptors are those proposed in (Zhang and Lu 2001), which eliminate the phase of DFT coefficients and preserve the modulus. Unfortunately, they work similarly to those pointed out in our formula (9) or those in Nixon and Aguado (2008). To avoid losing the ability to classify otoliths obtained by EFDs and positional standardization, which is manual, Marti-Puig and Reig-Bolano (2016) proposed an automatic positional contour normalization. The method was based on finding the major fundamental ellipse axis in which the contour can be decomposed, and the angle of this axis with the horizontal. This angle is used to rotate the contour in order to keep the principal ellipse axis horizontal. This strategy is independent of the human criteria and keeps the classification rate close to the results obtained by an expert doing the standardization procedure very carefully. That procedure works for rotations that do not exceed angles of ±90º from manual positional standardization.

In the present work, we employ the DFT coefficients, which unlike EFDs are complex, and we perform all the operations presented in Marti-Puig and Reig-Bolano (2016) much more efficiently because the DFT can be computed via the fast Fourier transform algorithm. We highlight some results related to the DFT formulation:

– A closed formula for the angle θ involved in the automatic positional normalization.
– A closed formula to find a starting point of sampling $\widehat{k}$ .
– The procedure to apply both sampling and positional standardizations by performing operations directly on the phase of the DFT coefficients.
– A way to deal with rotations higher than ±90° and also with symmetries.

The most relevant result is that the new strategy maintains the classification rate of otoliths of non-standardized inputs very close to that obtained when the contours are manually orientated by an expert and the samples are prepared for this purpose. The experiments were performed with the same database and the same classifier as in Marti-Puig and Reig-Bolano (2016) to facilitate comparisons.

The work is organized as follows: the Materials and Methods section covers the AFORO image acquisition protocol required to correctly position the otolith, the derivation of the proposed descriptors robust to rotations of θ (|θ|<90°), the number of points that were used to represent contours, some shape symmetries and their relation to the proposed descriptors, a baseline to achieve invariance and, finally, the method of classification and validation that was employed. The Results section presents the results under two frameworks: one in which the AFORO user applies positional normalization and in which the errors that are introduced with respect to the desired reference are small; and one that evaluates the recognition performance that could be achieved without positional standardization. We conclude the paper with some discussion.

Several experiments were conducted to evaluate the performance of the presented descriptors according to two different scenarios. The first scenario considers that the incoming image (the query image) has been taken by someone with sufficient knowledge to correctly distinguish the left sagitta, to determine the face of the otolith that shows the SA and to orient it correctly according to the indications of AFORO. In this scenario, the errors are only minor orientation errors and will always be between –90° and +90°. In the second scenario, we want to evaluate the capacity for identifying species just from the silhouette of the otolith, regardless of whether it is the right or the left sagitta and regardless of its orientation in the image. Therefore, no normalization positions are considered at all.

Common to all experiments, the effectiveness of the proposed contour descriptors will be evaluated using a K-NN classification test with (K=1) and using LOOCV. The first scenario was also used to assess the impact on the classifier accuracy depending on the number of coefficients and the number of points that are employed to represent the contour. To do this, we codified all the contours under a test, the coefficients C_l were calculated according to the expression (8) and the initial contour sampling point was forced, as has been described. The coefficient C₀ was eliminated, and the 2L coefficients with indices 1 to L and N–L to N–1 were selected. These coefficients are complex, so the descriptors that are used are the real parts followed by the imaginary parts of those C_l.

LOOCV involves selecting one contour and deleting its coefficients from the group. We apply a random rotation of between –90° and 90° to that contour. We codify it by applying (8) and selecting the same 2L coefficients as those considered, ordering real and imaginary parts as explained above, and classifying it using k-NN with Euclidian distance and k=1. This operation is repeated for all contours in several ways, and the average results are obtained as shown in Figure 6. Note that in Figure 6B we have considered the Merluccius bilinearis collected in Canada and the Merluccius bilinearis collected in the USA as belonging to two different classes, while in Figure 6A, they belong to the same class. In both cases, the abscissas represent the number # of real coefficients that were used (so a complex coefficient splits into two real coefficients). The accuracy is represented by the ordinates. In both subplots, the accuracy in the function of the number of components that are employed is computed for the same contours as those sampled at N equal to 256, 512 and 1024 points. The results suggest that contours represented by 1024 points achieve better accuracy in all cases. In addition, given a certain number of coefficients, the accuracy no longer improves, being an optimum of 28-32 real coefficients for the case of contours that are represented with 1024 points.

In the most straightforward experiment completed for the second scenario, the otolith (left or right) can appear in any position and can show any face; therefore, without observing any positional normalization, the descriptors that are presented in expression (9) as well as the expressions given in Nixon and Aguado (2008, p. 307) are used. For this purpose, all the contours of the test group are parameterized in both ways, and the 1-NN classification with the LOOCV test is carried out. The contours under the test are randomly rotated by between 0° and 360° and an inversion with the intention of simulating turning the otolith face is optionally applied. The accuracy regarding the number of real coefficients that are employed in the experiments shows that the optimum performance is observed for the range 26 to 28 real coefficients. However, the accuracy that is obtained in comparison with the first scenario has fallen considerably: from near 80% to 72.4%, as shown in Figure 7.

To improve the classification results in the second scenario (avoiding the positional normalization in the query image), we propose the use of the descriptors that were developed in (8) in the following form. The contour in which the otolith appears with any normalization is sampled counter-clockwise and is parameterized according to (8), as in the first scenario. Then, from these parameters, we reconstruct the contour again, obtaining any of the forms shown in Figure 5. Thus, the contour obtained is rotated 180°, inverted, and inverted and rotated to obtain three other silhouettes, as shown in Figure 5. We can use a fast implementation with the ifft by using the expressions that are presented in shape symmetries and their relationship with the proposed descriptors. Those four contours are parameterized according to (8) again and are classified individually in the database, saving the four results of the classification, as well as the distance to the nearest neighbour, in such a way that the winning option is the one with the lowest distance. The option that is decided by the four generated contour shapes for a single otolith image is therefore the one that obtains the lowest distance metric.

With this procedure, the classification results are close to those obtained in the first scenario. The results can be seen in Figure 8A for the second scenario. It is interesting to note that with a similar number of coefficients applying this second strategy, we achieve the same accuracy as in the first solution. The performance of the full invariant methods that are described in the paper are plotted together in Figure 8B.

The success of automatic species recognition from the images of otoliths that are sent to AFORO requires the otolith to follow positional normalization. The determination of the contour sampling starting point depends on this. Studies that are carried out in the framework of the AFORO database show that the contour descriptors that achieve the best performances, such as those based on FT, WT, and CSS, are sensitive to the determination of that point (Marti-Puig and Reig-Bolano 2016). The translation and scale invariances are addressed in a non-ambiguous and straightforward way. The shape rotation with respect to the positional normalization has a more significant impact on the starting sampling point. The reason is that it allows for a certain degree of ambiguity, because it depends on the identification of the rostrum region rather than on a single point. Actually, rotational normalization depends ultimately on the criteria of an expert. Therefore, even if specialists have taken the query images, it is inevitable that slight rotational variations will appear because of the differences in criteria.

Taking advantage of the use of the well-known DFT properties that relate rotations and sampling point changes in the spatial domain with changes in DFT coefficients, and specifically in their phase, our approach outperforms other Fourier-based methods that are dependent on a strict positional protocol. The proposed parameterization directly corrects the starting point of sampling and variations of between 90° and –90° with respect to the positional normalization that is required in AFORO.

To completely eliminate the positional standardization protocol (positional normalization), in the second framework we can directly employ the modules of the DFT coefficients f_i. To make them independent of the scale, Nixon and Aguado (2013) divide f_i by |f₁|. For automatic recognition of shape contours, we achieve the best results by normalizing by |f₁|+|f_N–1|, as given in (9), because |f₁|+|f_N–1| is a magnitude that is directly related to the length of the principal ellipse axis. However, the main contribution of this paper is that accuracy can be improved using the descriptors that are proposed in (8) by merely adding four possible variants for each contour and selecting the one that obtains the best score (the minimum Euclidean distance to the closest neighbour). From Figure 8A we observe that with only 4x(28-32) coefficients per contour (and working with 1024 points per contour) we obtain classification rates of 80%. These results are for contours that can appear randomly rotated by 360 degrees and inverted. This achievement is noteworthy because they are the same classification rates as those obtained for manual positional normalization (Marti-Puig and Reig-Bolano 2016). It is important to note that to maintain the classification rate, we need to increase the number of points used to represent the contour. In this case, decreasing the number of points means a decrease in the classification rate (see Fig. 8A). The four related shapes include robustness to the introduction of images showing the wrong otolith face (with the SA not visible) or even the introduction of the right otolith rather than the left in the recognition system. We thus reach practically the same classification accuracy performance as the one obtained using positional normalization but, in this case the user can avoid normalization completely. Therefore, the proposed descriptors facilitate the use of AFORO and other online catalogues and could open up the range of potential users and applications, thereby eliminating irregularities and errors in the image normalization process.