Article Details

Abstract

In high-dimensional classification tasks, redundant and irrelevant features increase model complexity and degrade discriminative performance. However, most existing feature selection methods operate in the original feature space and therefore have difficulty characterizing the nonlinear structure of the data. To address this issue, this paper proposes a discriminative feature selection method based on Kernel Principal Component Analysis (KPCA) and the Hilbert–Schmidt Independence Criterion (HSIC), termed KPCA-CM-HSIC. The proposed method first employs KPCA to map the data into a kernel feature space and obtain nonlinear principal component representations. It then constructs a discriminative scoring criterion that integrates class-center separability and local boundary margin to perform a rapid coarse screening of the principal components, thereby reducing the dimensionality of candidate features. Finally, HSIC is used to evaluate the nonlinear dependence between the candidate components and the class labels and to rank the candidate components accordingly, yielding a more discriminative feature subset. Experimental results demonstrate that the proposed method outperforms several classical feature selection approaches in terms of classification accuracy and computational efficiency, indicating its effectiveness and practical value.

Keywords

kernel principal component analysis; Hilbert–Schmidt independence criterion; high-dimensional data; feature selection

References

1. [1] Wei G F，Zhao J，Feng Y L，et al. A Novel Hybrid Feature Selection Method Based on Dynamic Feature Importance [J/OL].Applied Soft Computing，2020，93：106337.https://doi.org/10.1016/j.asoc.2020.106337.
2. [2] Zhang M T，Du J Q，Nie B，et al. Hybrid mRMR and Multi-Objective Particle Swarm Feature Selection Methods and Application to Metabolomics of Traditional Chinese Medicine [J/OL].PeerJ Computer Science，2024，10：e2073.https://doi.org/10.7717/peerj-cs.2073.
3. [3] Ahmad I，Yao C，Li L，et al. An Efficient Feature Selection and Explainable Classification Method for EEG-based Epileptic Seizure Detection [J/OL].Journal of Information Security and Applications，2024，80：103654.https://doi.org/10.1016/j.jisa.2023.103654.
4. [4] Omuya E O，Okeyo G O，Kimwele M W. Feature Selection for Classification Using principal Component Analysis and Information Gain [J/OL].Expert Systems with Applications，2021，174：114765.https://www.sciencedirect.com/science/article/abs/pii/S0957417421002062?via%3Dihub.
5. [5] Islam M R，Lima A A，Das S C，et al. A Comprehensive Survey on the Process, Methods, Evaluation, and Challenges of Feature Selection [J].IEEE Access，2022，10：99595-99632.
6. [6] Yang P，Wang X F. Vulnerability Extraction and Prediction Method Based on Improved Information Gain Algorithm [J/OL].PLOS ONE，2024，19（9）：e0309809.https://doi.org/10.1371/journal.pone.0309809.
7. [7] Urbanowicz R J，Meeker M，La Cava W， et al. Relief-based feature selection: Introduction and review [J].Journal of Biomedical Informatics，2018，85：189-203.
8. [8] Peng H C，Long F H，Ding C. Feature Selection Based on Mutual Information Criteria of Max-Dependency, Max-Relevance, and Min-Redundancy [J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2005，27（8）：1226-1238.
9. [9] Darst B F，Malecki K C，Engelman C D. Using Recursive Feature Elimination in Random Forest to Account for Correlated Variables in High Dimensional Data [J/OL].BMC genetics，2018，19（Suppl 1）：65.https://link.springer.com/article/10.1186/s12863-018-0633-8.
10. [10] Chotchantarakun K. Optimizing Sequential Forward Selection on Classification Using Genetic Algorithm [J/OL].Informatica，2023，47（9）.https://www.informatica.si/index.php/informatica/article/view/4964.
11. [11] Forrest S. Genetic Algorithms [J].ACM Computing Surveys (CSUR)，1996，28（1）：77-80.
12. [12] Abdel-Basset M，Ding W P，El-Shahat D. A Hybrid Harris Hawks Optimization Algorithm with Simulated Annealing for Feature Selection [J].Artificial Intelligence Review，2021，54（1）：593-637.
13. [13] Ghosh P，Azam S，Jonkman M，et al. Efficient prediction of cardiovascular disease using machine learning algorithms with relief and LASSO feature selection techniques [J].IEEE Access，2021，9：19304-19326.
14. [14] Xu Z Z，Yang F Y，Tang C S，et al. FG-HFS: A Feature Filter and Group Evolution Hybrid Feature Selection Algorithm for High-Dimensional Gene Expression Data [J/OL].Expert Systems with Applications，2024，245：123069.https://www.sciencedirect.com/science/article/pii/S0957417423035716?via%3Dihub.
15. [15] Gretton A，Fukumizu K，Teo C H，et al. A Kernel Statistical Test of Independence [C]//Advances in Neural Information Processing Systems，2007：585-592.
16. [16] Zeng Z L，Zhang H J，Zhang R，et al. A Novel Feature Selection Method Considering Feature Interaction [J].Pattern Recognition，2015，48（8）：2656-2666.
17. [17] Li A D，Xue B，Zhang M J. Improved Binary Particle Swarm Optimization for Feature Selection with new Initialization and Search Space Reduction Strategies [J/OL].Applied Soft Computing，2021，106：107302.https://doi.org/10.1016/j.asoc.2021.107302.

Funding

Development Plan for University level Science and Technology Innovation Team of Jiangxi University of Traditional Chinese Medicine (CXTD22015)

About the authors

Li Xiang (2001-), male, Han nationality, from Tongling, Anhui, master's student, research direction: pharmaceutical data mining; Corresponding author: Nie Bin (1972-), male, Han ethnicity, from Xiajiang, Jiangxi Province, professor, master's supervisor, doctoral student, research direction: pharmaceutical data mining, traditional Chinese medicine informatics, traditional Chinese medicine; Zheng Shuifei (1998-), male, Han nationality, from Shangrao, Jiangxi, master's student, research direction: pharmaceutical data mining; Zhang Zhiyao (2001-), female, Han ethnicity, born in Nanchang, Jiangxi, is a master's student with a research focus on pharmaceutical data mining.