Variable selection and structure identification for ultrahigh-dimensional partially linear additive models with application to cardiomyopathy microarray data

Mohammad Kazemi; Davood Shahsavani; Mohammad Arashi

doi:10.19139/soic.v6i3.577

Mohammad Kazemi Department of Statistics, Faculty of Mathematical Sciences, Shahrood University of Technology, Shahrood, Iran
Davood Shahsavani Department of Statistics, Faculty of Mathematical Sciences, Shahrood University of Technology, Shahrood, Iran
Mohammad Arashi Department of Statistics, Faculty of Mathematical Sciences, Shahrood University of Technology, Shahrood, Iran

DOI: https://doi.org/10.19139/soic.v6i3.577

Keywords: Dimensionality Reduction, Partially Linear Additive Model, Structure Identification, Sure Screening, Variable Selection

Abstract

In this paper, we introduce a two-step procedure, in the context of ultrahigh-dimensional additive models, to identify nonzero and linear components. We first develop a sure independence screening procedure based on the distance correlation between predictors and marginal distribution function of the response variable to reduce the dimensionality of the feature space to a moderate scale. Then a double penalization based procedure is applied to identify nonzero and linear components, simultaneously. We conduct extensive simulation experiments to evaluate the numerical performance of the proposed method and analyze a cardiomyopathy microarray data for an illustration. Numerical studies confirm the fine performance of the proposed method for various semiparametric models.

References

J. Du, G. Li, and H. Peng, Variable selection for semiparametric partially linear Covariate-Adjusted Regression Models, Comm. Statist. Theo. Meth., vol. 44, no. 3, pp. 2809–2826, 2015.

J. Fan, and R. Li, Variable selection via nonconcave penalized likelihood and its oracle properties, J. Amer. Statist. Assoc., vol. 96, pp. 1348–1360, 2001.

J. Fan, and J. Lv, Sure independence screening for ultrahigh dimensional feature space, J. R. Statist. Soc. Ser. B Stat. Meth., vol. 70, no. 5, pp. 849-911, 2008.

J. Fan, R.J. Samworth, and Y. Wu, Ultrahigh dimensional feature selection: beyond the linear model, J. Mach. Learn. Res., vol. 10, pp. 1829–1853, 2009.

J. Fan, Y. Feng, and R. Song, Nonparametric independence screening in sparse ultrahigh-dimensional additive models, J. Amer. Statist. Assoc., vol. 106, pp. 544–557, 2011.

J. Fan, and R. Song, Sure independence screening in generalized linear models with NP-dimensionality, Ann. Statist., vol.6, pp. 3567-3604, 2010.

J. Guo, M. Tang, M. Tian, and K. Zhu, Variable selection in high-dimensional partially linear additive models for composite quantile regression, Comp. Statist. Data Anal., vol. 65, pp. 56–67, 2013.

P. Hall, H. Miller, Using generalized correlation to effect variable selection in very high dimensional problems, J. Computnl Graph. Statist., vol. 18, pp. 533-550, 2009.

J. Huang, F. Wei, and S. Ma, Semiparametric regression pursuit. Statist. Sinica , vol. 22, pp. 1403–1426, 2012.

R.Z. Li, W. Zhong, and L.P. Zhu, Feature screening via distance correlation learning, J. Amer. Statist. Assoc., vol. 107, pp.1129–1139, 2012.

G. Li, H. Peng, J. Zhang, and J. Zhu, Robust rank correlation based screening, Ann. Statist. vol. 40, pp. 1846-1877, 2012.

H. Lian, Variable selection in high-dimensional partly linear additive models, J. Nonparametric Statist., vol. 24, no. 4, pp. 825–839, 2012.

H. Lian, Shrinkage estimation for identification of linear components in additive models, Statist. Prob. Lett., vol. 82, pp. 225–231, 2012.

H. Lian, X. Chen, and JY. Yang, Identification of partially linear structure in additive models with an application to gene expression prediction from sequences, Biometrics, vol. 68, pp. 437-445, 2012.

H. Lian, H. Liang, and D. Ruppert, Separation of covariates into nonparametric and parametric parts in high-dimensional partially linear additive models, Statistica Sinica, vol. 25, pp. 591–607, 2015.

X. Liu, L. Wang, and H. Liang, Estimation and variable selection for semiparametric additive partial linear models, Statist. Sinica., vol. 21, pp. 1225–1248, 2011.

J. Lv, H. Yang, and C. Guo, Variable selection in partially linear additive models for modal regression, Comm. Statist. Sim. Comp., DOI: 10.1080/03610918.2016.1171346, 2016.

G. J. Szekely, M. L. Rizzo, and N. K. Bakirov, Measuring and testing dependence by correlation of distances, Annals of Statistics, vol. 35, pp. 2769–2794, 2007.

R. Tibshirani, Regression shrinkage and selection via the lasso, J. Royal. Statist. Soc. Ser. B, vol. 58, pp. 267–288, 1996.

C. H. Zhang, Nearly unbiased variable selection under the minimax concave penalty, Ann. Statist., vol. 83, pp. 894–942, 2010.

H. H. Zhang, G. Cheng, and Y. Liu, Linear or Nonlinear? Automatic structure discovery for partially linear models, J. Amer. Statist. Assoc., vol. 106, pp. 1099-1112, 2011.

S. D. Zhao, and Y. Li, Principled sure independence screening for Cox models with ultrahigh-dimensional covariates, J. Mult. Anal., vol. 105, pp. 397–411, 2012.

L. P. Zhu, L. Li, R. Li, and L. X. Zhu, Model-free feature screening for ultrahigh dimensional data, J. Amer. Statist. Assoc., vol. 106, pp. 1464–1475, 2011.