Conferences CIMPA, 18th International Federation of Classification Societies

Font Size: 
Scalable conic optimization for feature selection in linear SVMs with cardinality control
Immanuel M. Bomze, Federico D'Onofrio, Bo Peng, Laura Palagi

Last modified: 2024-05-22

Abstract


In biclassification problems with many features, it is important to select a few of them, to ensure fairness and explainability in the original data space. We control the feature number (to be specified by the user) with a hard cardinality (zero-norm) constraint and solve the resulting difficult optimization problem by a conic decomposition approach. This exhibits promising scalability properties while maintaining both explainability and good predictive performance. From an optimization point of view, our approach is competitive with previous similar attempts employing mixed-binary linear optimization technology. These however use different techniques like bi-level formulations, surrogates for the zero-norm, or convex relaxationsIn biclassification problems with many features, it is important to select a few of them, to ensure fairness and explainability in the original data space. We control the feature number (to be specified by the user) with a hard cardinality (zero-norm) constraint and solve the resulting difficult optimization problem by a conic decomposition approach. This exhibits promising scalability properties while maintaining both explainability and good predictive performance. From an optimization point of view, our approach is competitive with previous similar attempts employing mixed-binary linear optimization technology. These however use different techniques like bi-level formulations, surrogates for the zero-norm, or convex relaxations, see for instance [1,2,3,4], and not all of them always exert strict control on the number of features., and not all of them always exert strict control on the number of features.

Keywords


Biclassification, zero norm, conic optimization

References


1. Agor, Joseph and Özaltın, Osman Y. (2019). Feature selection for classification models via bilevel optimization. Comput. OR 106, 156–168

2. Aytug, Haldun (2015). Feature selection for support vector machines using Generalized Benders Decomposition. European J. OR 244, 210–218

3. Ghaddar, Bissan and Naoum-Sawaya, Joe (2018). High dimensional data classification and feature selection using support vector machines. European J. OR 265, 993–1004

4. Labbé, Martine, Martínez-Merino, Luisa I. and Rodríguez-Chía, Antonio M. (2019). Mixed integer linear programming for feature selection in support vector machine. Discrete Appl.Math. 261, 276–304