Feature selection is to select subset of relevant features and by removing irrelevant, redundant features and improving performance of dimensionality. This paper focuses on feature selection for problems dealing with high-dimensional data. However, the large number of predictive gene sets and the disparity among them presents a challenge for identifying potential biomarkers. In this study, two novel feature ranking methods, namely, efficient feature ranking via ℓ2;1-regularization (EFRL21) and robust efficient feature ranking via ℓ2;1-regularization (REFRL21) are presented. Gene expression profiles, which represent the state of a cell at a molecular level, have great potential as a medical diagnosis tool. But compared to the number of genes involved, available training data sets generally have a fairly small sample size for classification. These training data limitations constitute a challenge to certain classification methodologies. Feature selection techniques can be used to extract the marker genes which influence the classification accuracy effectively by eliminating the unwanted noisy and redundant genes. Feature selection is an important concept in data mining. Batch learning is the mostly used learning algorithm in feature selection. Unlike Batch learning, online learning proves to be the most promising, efficient and scalable machine learning algorithm. Most existing studies of online learning require accessing all the features of training data. But, accessing all attributes becomes a problem when we deal with high dimensional data. To avoid this limitation, we investigate an online learner which will maintain a classifier having small and fixed number of attributes.