WOMAT Workshop On Multivariate Analysis Today Programme and Book of abstracts Scientific organisers: Frank Critchley (OU), Bing Li (Penn State), Hannu Oja (Turku) Local organisers: Sara Griffin, Tracy Johns, Radka Sabolova, Germain Van Bever Contents Programme 3 Talk abstracts 5 Yanyuan Ma: A Validated Information Criterion (VIC) to Find the Structural Dimension . . . . 5 Jo˜ao Branco: High dimensionality: the trouble with Mahalanobis distance . . . . . . . . . . . . . 5 Tim Cannings: Random projection ensemble classification . . . . . . . . . . . . . . . . . . . . . . 5 Kjersti Aas: Pair-copula constructions–even more flexible than copulas . . . . . . . . . . . . . . . 6 Sara Fontanella: A Bayesian approach to sparse latent variables modelling: Factor Analysis and Multidimensional Item Response Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Shahin Tavakoli: Dynamics of DNA Minicircles in Motion via Fourier Analysis of Functional Time Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Lutz Duembgen: New Algorithms for M -Estimation of Multivariate Scatter and Location . . . . 8 Jim Smith: Chain event graphs for discrete multivariate processes . . . . . . . . . . . . . . . . . 8 John Kent: Some new perspectives on partial least squares . . . . . . . . . . . . . . . . . . . . . 8 Poster abstracts 10 Comparison of statistical methods for multivariate outliers detection . . . . . . . . . . . . . . . . 10 On point estimation of the abnormality of a Mahalanobis distance . . . . . . . . . . . . . . . . . 11 Sparse Linear Discriminant Analysis with Common Principal Components . . . . . . . . . . . . . 12 Recovering Fisher linear discriminant subspace by Invariate Coordinate Selection . . . . . . . . . 13 Hilbertian Fourth Order Blind Identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2 Programme 9:00 Yanyuan Ma (University of South Carolina): A Validated Information Criterion to Find the Structural Dimension 9:30 Jo˜ao Branco (CEMAT, IST, Lisbon): High dimensionality: the trouble with Mahalanobis distance 10:00 Tim Cannings (Cambridge): Random projection ensemble classification 10:30 Coffee & Poster Session 11:00 Kjersti Aas (Norwegian Computing Centre): Pair-copula constructions– even more flexible than copulas 11:30 Sara Fontanella (The Open University): A Bayesian approach to sparse latent variables modelling: Factor Analysis and Multidimensional Item Response Theory 12:00 Shahin Tavakoli (Cambridge): Dynamics of DNA Minicircles in Motion via Fourier Analysis of Functional Time Series 12:30 Lutz Duembgen (Bern): New Algorithms for M -Estimation of Multivariate Scatter and Location 13:00 Lunch and Poster Session 14:30 Jim Smith (Warwick): Chain event graphs for discrete multivariate processes 15:00 John Kent (Leeds): Some new perspectives on partial least squares 15:30 Roundtable Discussion 16:00 Tea and Departures 3 Talk abstracts A Validated Information Criterion to Find the Structural Dimension Yanyuan Ma, University of South Carolina E-mail: [email protected] A crucial component in performing sufficient dimension reduction is to determine the structural dimension of the reduction model. We propose a novel information criterion-based method to achieve this purpose, whose special feature is that when examining the goodness-of-fit of the current model, we need to obtain model evaluation by using an enlarged candidate model. Although the procedure does not require estimation under the enlarged model with dimension k + 1, the decision on how well the current model with dimension k fits relies on the validation provided by the enlarged model. This leads to the name validated information criterion, calculated as VIC(k). The method is different from existing information criteria based model selection methods. It breaks free from the dependence on the connection between dimension reduction models and their corresponding matrix eigenstructures, which heavily relies on a linearity condition that we no longer assume. Its consistency is proved and its finite sample performance is demonstrated numerically. (Joint work with Xinyu Zhang.) High dimensionality: the trouble with Mahalanobis distance Jo˜ao Branco, Ana M. Pires, CEMAT, Instituto Superior Tecnico, Lisbon The recent massive production of high-dimensional data has brought great difficulties and concomitant challenges to statistics since its usual methods were not designed to cope with such kind of data. High dimensionality triggers the curse of dimensionality and unexpected behaviour of some statistical tools may surprise even those aware of the intricacies of multidimensional spaces with a large number of dimensions. We look at the Mahalanobis distance, a tool that is crucial to the functioning of the traditional multivariate statistical methods, and see how it progresses as p approaches n and when it is greater than n. Can the Mahalanobis distance keep the fundamental role in high-dimensional spaces as it does in low dimensional spaces (p n)? And if it does not what are the consequences? We will attempt to answer these questions. Random projection ensemble classification Timothy I. Cannings and Richard J. Samworth, Statistical Laboratory, University of Cambridge We introduce a very general method for high-dimensional classification, based on careful combination of the results of applying an arbitrary base classifier to random projections of the feature vectors into a lower-dimensional space. In one special case presented here, the random projections are divided into nonoverlapping blocks, and within each block we select the projection yielding the smallest estimate of the test error. Our random projection ensemble classifier then aggregates the results of applying the base classifier on the selected projections, with a data-driven voting threshold to determine the final assignment. We provide theoretical understanding to justify the methodology, and a simulation comparison with several other popular high-dimensional classifiers reveals its excellent finite-sample performance. 5 Pair-copula constructions–even more flexible than copulas Kjersti Aas, Norwegian Computing Centre A copula is a multivariate distribution with standard uniform marginal distributions. While the literature on copulas is substantial, most of the research is still limited to the bivariate case. However, some years ago hierarchical copula-based structures were proposed as an alternative to the standard copula methodology. One of the most promising of these structures is the pair-copula construction (PCC). The PCC modeling scheme is based on a decomposition of a multivariate density into a cascade of pair copulae, applied on original variables and on their conditional and unconditional distribution functions. Each pair copula can be chosen arbitrarily and the full model exhibit complex dependence patterns such as asymmetry and tail dependence. In this talk I will give an introduction to pair-copula constructions and apply the methodology to a 19-dimensional financial data set. A Bayesian approach to sparse latent variables modelling: Factor Analysis and Multidimensional Item Response Theory Sara Fontanella, N. Trendafilov, P. Valentini, L. Fontanella In the last decades, sparse modeling has inspired many studies in different research fields, such as statistics, machine learning and bioinformatics. Its importance is due to the following main advantages: first, it enhances the interpretability of the results; second, it reflects reality, as any real-world system is sparse and third, predictive performance is improved, since the sparsity helps prevent overfitting. In this work, we consider sparse modeling in the context of two multivariate statistical techniques: Factor Analysis (FA) and Multidimensional Item Response Theory (MIRT). They are strongly related to each other in terms of modeling despite the different types of data they are applied to. FA is a well-known model-based multivariate technique used to describe observed continuous variables by means of a smaller set of latent factors. Item response theory (IRT) models the probability for a correct response (to a test, questionnaire, etc) as function of disjoint sets of parameters, related respectively to the person and the item. MIRT is its multidimensional extension. Both FA and MIRT suffer from solution/factor indeterminacy. In particular, the main issue to be addressed is the rotational invariance of the final solution: for a given set of data, any orthogonal transformation of the matrix of parameters would produce the same covariance structure. In this context, we show that the sparsity plays a double role: on one side it improves the interpretability of the results, while, on the other side, it allows to overcome the rotational indeterminacy. To this end, we follow a Bayesian approach to sparse modeling. The prior belief in sparsity is modeled by a sparse-inducing prior distribution on the parameters. In this context, a popular choice is to apply spike and slab priors, which present several computational advantages. A spike and slab prior assumes that the parameters of interest are mutually independent with a two-point mixture distribution made up of a degenerate distribution at zero (the spike), to provide strong shrinkage near zero and a uniform flat distribution (the slab), to allow signals to escape strong shrinkage. The performances of the considered methods are evaluated through simulation studies. 6 Dynamics of DNA Minicircles in Motion via Fourier Analysis of Functional Time Series Shahin Tavakoli, Statslab, University of Cambridge We consider the problem of studying the dynamics of DNA minicircles that are vibrating in solution. At a large scale, DNA minicircles are modelled as elastic rods, and the problem of understanding their dynamics can be recasted into the problem of estimating the second order structure of a stationary functional time series (FTS). We tackle this problem by a frequency domain approach, where we estimate the spectral density operators (or spectra) of the DNA minicircle. We then carry out hypothesis tests to compare the spectra of two specific DNA minicircles. The comparison is broken down to a hierarchy of stages: at a global level, we compare the spectral density operators of the two DNA minicircles, across frequencies and curvelength, based on a Hilbert-Schmidt criterion; then, we localize any differences to specific frequencies; and, finally, we further localize any differences along the length of the DNA minicircles, i.e. in physical space. A hierarchical multiple testing approach guarantees control of the averaged false discovery rate over the selected frequencies. In this sense, we are able to attribute any differences to distinct dynamic (frequency) and spatial (curvelength) contributions. Keywords. Functional Data Analysis; Spectral Analysis; DNA Minicircle; Molecular Dynamics; Multiple Testing. 7 New Algorithms for M -Estimation of Multivariate Scatter and Location Lutz Duembgen, Bern We present new algorithms for M-estimators of multivariate scatter and location and for symmetrized M-estimators of multivariate scatter. The new algorithms are considerably faster than currently used fixed-point and other algorithms. The main idea is to utilize local parametrizations of scatter matrices via matrix exponentials with a corresponding second order Taylor expansion of the target functional and to devise a partial Newton-Raphson procedure. In connection with symmetrized M-estimators we work with incomplete U-statistics to accelerate our procedures initially. This talk is based on joint work with Klaus Nordhausen (Turku) and Heike Schuhmacher (Bern). Chain event graphs for discrete multivariate processes Jim Smith, Warwick Statistical models of multivariate discrete processes often need to express various hypotheses about how events might unfold and associated hypotheses about the symmetries within these unfoldings. A natural way to express such hypotheses is via a statistical model on a finite set of atoms structured around collections of different probability trees with different symmetries. One such family is the class of Chain Event Graphs. This family contains the class of discrete Bayes Nets as a very special case. It can be shown that most inferential techniques used for Bayesian Networks readily translate to this new family because of thier modular form. Furthermore because different models in the class can be associated with families of polynomials, the inferential implications of one hypothesis against another can be elegantly analysed. In this talk I will present some recent results associated with CEGs and the challenges they bring to effective model choice. This is joint work with two PhD students, Christiane Gorgen and Rodrigo Collazo. Some new perspectives on partial least squares John Kent, Department of Statistics, University of Leeds Partial least squares a regularization technique in high-dimensional multiple regression analysis. It has sometimes had a somewhat dubious reputation in mainstream statistics. Part of the reason seems to be that the methodology was originally proposed in terms of an algorithm, and only later was it noticed that it can be viewed as an attempt to fit a particular statistical model, the Krylov model. In this talk we describe how the Krylov model can be formulated most simply in the setting of inverse regression and how the PLS estimator can be viewed as an approximate MLE for this model. We then describe some comparisons with the exact MLE under this model. 8 Poster abstracts Comparison of statistical methods for multivariate outliers detection Aurore Archimbaud1 , Klaus Nordhausen2 & Anne Ruiz-Gazen1 1 2 Gremaq (TSE), Universit´e Toulouse 1 Capitole, E-mail: [email protected] [email protected] Department of Mathematics and Statistics, University of Turku, E-mail: [email protected] In this poster, we are interested in detecting outliers, like for example manufacturing defects, in multivariate numerical data sets. Several non-supervised methods that are based on robust and non-robust covariance matrix estimators exist in the statistical literature. Our first aim is to exhibit the links between three outliers detection methods: the Invariant Coordinate Selection method as proposed by Caussinus and Ruiz-Gazen (1993) and generalized by Tyler et al. (2009), the method based on the Mahalanobis distance as detailed in Rousseeuw and Van Zomeren (1990), and the robust Principal Component Analysis (PCA) method with its diagnostic plot as proposed by Hubert et al. (2005). Caussinus and Ruiz-Gazen (1993) proposed a Generalized PCA which diagonalizes a scatter matrix relative to another: V1 V2−1 where V2 is a more robust covariance estimator than V1 , the usual empirical covariance estimator. These authors compute scores by projecting V2−1 -orthogonally all the observations on some of the components and high scores are associated with potential outliers. We note that computing euclidean distances between observations using all the components is equivalent to the computation of robust Mahalanobis distances according to the matrix V2 using the initial data. Tyler et al. (2009) generalized this method and called it Invariant Coordinate Selection (ICS). Contrary to Caussinus and Ruiz-Gazen (1993), they diagonalize V1−1 V2 which leads to the same eigen elements but to different scores that are proportional to each other. As explained in Tyler et al. (2009), the method is equivalent to a robust PCA with a scatter matrix V2 after making the data spherical using V1 . However, the euclidean distances between observations based on all the components of ICS corresponds now to Mahalanobis distances according to V1 and not to V2 . Note that each of the three methods leads to a score for each observation and high scores are associated with potential outliers. We compare the three methods on some simulated and real data sets and show in particular that the ICS method is the only method that permits a selection of the relevant components for detecting outliers. Keywords. Invariant Coordinate Selection; Mahalanobis distance; robust PCA. Bibliography [1] Caussinus, H. and Ruiz-Gazen, A. (1993), Projection pursuit and generalized principal component analysis, In New Directions in Statistical Data Analysis and Robustness (eds S. Morgenthaler, E. Ronchetti and W. A. Stahel), 35–46, Basel: Birkh¨auser. [2] Hubert, M., Rousseeuw, P. J. and Vanden Branden, K. (2005), ROBPCA: a new approach to robust principal component analysis, Technometrics, 47(1), 64–79. [3] Rousseeuw, P. J. and Van Zomeren, B. C. (1990), Unmasking multivariate outliers and leverage points, Journal of the American Statistical Association, 85(411), 633–639. [4] Tyler, D. E., Critchley, F., D¨ umbgen, L. and Oja, H. (2009), Invariant coordinate selection, Journal of the Royal Statistical Society: Series B (Statistical Methodology), 71(3), 549–592. 10 On point estimation of the abnormality of a Mahalanobis distance Fadlalla G. Elfadaly1 , Paul H. Garthwaite1 & John R. Crawford2 1 The Open University University of Aberdeen Email: [email protected] 2 When a patient appears to have unusual symptoms, measurements or test scores, the degree to which this patient is unusual becomes of interest. For example, clinical neuropsychologists sometimes need to assess how a patient with some brain disorder or a head injury is different from the general population or some particular subpopulation. This is usually based on the patient’s scores in a set of tests that measure different abilities. Then, the question is “What proportion of the population would give a set of test scores as extreme as that of the patient?” The abnormality of the patient’s profile of scores is expressed in terms of the Mahalanobis distance between his profile and the average profile of the normative population. The degree to which the patient’s profile is unusual can then be equated to the proportion of the population who would have a larger Mahalanobis distance than the individual. This presentation will focus on forming an estimator of this proportion using a normative sample. The estimators that are examined include plugin maximum likelihood estimators, medians, the posterior mean from a Bayesian probability matching prior, an estimator derived from a Taylor expansion, and two forms of polynomial approximation, one based on Bernstein polynomial and one on a quadrature method. Simulations show that some estimators, including the commonly-used plug-in maximum likelihood estimators, can have substantial bias for small or moderate sample sizes. The polynomial approximations yield estimators that have low bias, with the quadrature method marginally to be preferred over Bernstein polynomials. Moreover, simulations of the median estimators have a nearly zero median error. This latter estimator has much to recommend it when unbiasedness is not of paramount importance, while the quadrature method is recommended when bias is the dominant issue. Keywords. Bernstein polynomials; Mahalanobis distance; median estimator; quadrature approximation; unbiased estimation. 11 Sparse Linear Discriminant Analysis with Common Principal Components Tsegay G. Gebru & Nickolay T. Trendafilov Department of Mathematics and Statistics, The Open University, UK Linear discriminant analysis (LDA) is a commonly used method for classifying a new observation into one of g-populations. However, in high-dimensional classification problems the classical LDA has poor performance. When the number of variables is much larger than the number of observations, the withingroup covariance matrix is singular which leads to unstable results. In addition, the large number of input variables needs considerable reduction which nowadays is addressed by producing sparse discriminant functions. Here, we propose a method to tackle the (low-sample) high-dimensional discrimination problem by using common principal components (CPC). LDA based on CPC is a general approach to the problem because it does not need the assumption of equal covariance matrix in each groups. We find sparse CPCs by modifying the stepwise estimation method proposed by Trendafilov (2010). Our aim is to find few important spare discriminant vectors which are easily interpretable. For numerical illustrations, the method is applied on some known real data sets and compared to other methods for sparse LDA. Bibliography [1] Trendafilov, N.T. Stepwise estimation of common principal components. Computational Statistics and Data Analysis 54:3446-3457, 2010. 12 Recovering Fisher linear discriminant subspace by Invariate Coordinate Selection Radka Sabolov´a1,2 , H. Oja3 , G. Van Bever1 & F. Critchley1 . 1 MCT Faculty, The Open University, Milton Keynes 2 Email: [email protected] 3 Turku University It is a remarkable fact that, using any pair of scatter matrices, invariant coordinate selection (ICS) can recover the Fisher linear discriminant subspace without knowing group membership, see [5]. The subspace is found by using two different scatter matrices S1 and S2 and joint eigendecomposition of one scatter matrix relative to another. In this poster, we focus on the two group normal subpopulation problem and discuss the optimal choice of such a pair of scatter matrices in terms of asymptotic accuracy of recovery. The first matrix is fixed as the covariance matrix while the second one is chosen within a one-parameter family based on powers of squared Mahalanobis distance, indexed by α ∈ R. Special cases of this approach include Fourth Order Blind Identification (FOBI, see [1]) and Principal Axis Analysis (PAA, see [4]). The use of two scatter matrices in discrimination was studied by [2] and later elaborated in [3], who proposed generalised PCA (GPCA) based on a family of scatter matrices with decreasing weight functions of a single real parameter β > 0. They then discussed appropriate choice of β, while concentrating on outlier detection. Their form of weight function and the consequent restriction to β > 0 implies downweighting outliers. On the other hand, in our approach, considering any α ∈ R allows us also to upweight outliers. Further, we may, in addition to the outlier case, study mixtures of subpopulations. Theoretical results are underpinned by an extensive numerical study. The UK-based authors thank the EPSRC for their support under grant EP/L010429/1. Bibliography [1] Cardoso, J.-F. Source Separation Using Higher Moments Proceedings of IEEE international conference on acoustics, speech and signal processing 2109-2112. [2] Caussinus, H. and Ruiz-Gazen, A. Projection pursuit and generalized principal component analyses New direction in Statistical Data Analysis and Robustness 35-46. [3] Caussinus, H., Fekri, M., Hakam, S. and Ruiz-Gazen, A. A monitoring display of multivariate outliers Computational Statistics & Data Analysis, 2003, 44, 237–252. [4] Critchley, F., Pires, A. and Amado, C. Principal Axis Analysis technical report, Open University, 2006. [5] Tyler, D., Critchley, F., Dumbgen, L. and Oja, H. Invariant Co-ordinate Selection J. R. Statist. Soc. B., 2009, 71, 549–592. 13 Hilbertian Fourth Order Blind Identification Germain Van Bever1,2 , B. Li3 , H. Oja4 , R. Sabolov´a1 & F. Critchley1 . 1 MCT Faculty, The Open University, Milton Keynes 2 Email: [email protected] 3 Penn State University 4 Turku University In the classical Independent Component (IC) model, the observations X1 , · · · , Xn are assumed to satisfy Xi = ΩZi , i = 1, . . . , n, where the Zi ’s are i.i.d. random vectors with independent marginals and Ω is the mixing matrix. Independent component analysis (ICA) encompasses the set of all methods aiming at unmixing X = (X1 , . . . , Xn ), that is estimating a (non unique) unmixing matrix Γ such that ΓXi , i = 1, . . . , n, has independent components. Cardoso ([1]) introduced the celebrated Fourth Order Blind Identification (FOBI) procedure, in which an estimate of Γ is provided, based on the regular covariance matrix and a scatter matrix based on fourth moments. Building on robustness considerations and generalizing FOBI, Invariant Coordinate Selection (ICS, [2]) was originally introduced as an exploratory tool generating an affine invariant coordinate system. The obtained coordinates, however, are proved to be independent in most IC models. Nowadays, functional data (FD) are occurring more and more often in practice, and relatively few statistical techniques have been developed to analyze this type of data (see, for example [3]). Functional PCA is one such technique which focuses on dimension reduction with very little theoretical considerations. We propose an extension of the FOBI methodology to the case of Hilbertian data, FD being the go-to example used throughout. When dealing with distributions on Hilbert spaces, two major problems arise: (i) the scatter operator is, in general, non-invertible and (ii) there may not exist two different affine equivariant scatter functionals. Projections on finite dimensional subspaces and Karhunen-Lo`eve expansions are used to overcome these issues and provide an alternative to FPCA. More importantly, we show that the proposed construction is Fisher consistent for the independent components of an appropriate Hilbertian IC model and enjoy the affine invariance property. This work is supported by the EPSRC grant EP/L010429/1. Keywords. Invariant Coordinate Selection; Functional Data; Symmetric Component Analysis; Independent Component Analysis. Bibliography [1] Cardoso, J.-F. (1989), Source Separation Using Higher Moments Proceedings of IEEE international conference on acoustics, speech and signal processing 2109-2112. [2] Tyler, D., Critchley, F., Dumbgen, L. and Oja, H. (2009) Invariant Co-ordinate Selection J. R. Statist. Soc. B., 71, 549–592. [3] Ramsay, J. and Silverman, B.W. (2006) Functional Data Analysis 2nd edn. Springer, New York. 14

© Copyright 2021