The recent and rapid increase in the size of data and the large variety of data types that are required to be processed within different scientific fields has made it extremely important to find an optimal set of features that reduces the complexity of the data and extract valuable and beneficial information from within these large sums of unorganized data. This is especially true in the cases where datasets suffer from the problem of high dimensionality. These challenges require an in depth understanding of the effect of feature selection algorithms and how to utilize them to achieve higher rates of accuracy within high dimensional data sets. In this paper we investigate the role of proper feature selection in the context of classification of anomaly network intrusion detection systems. We show that the appropriate selection of important features can have a huge effect on the accuracy level of the classifiers. We also compare the performance of different classifiers, and show that the Restricted Boltzmann Machine classifier outperforms other type of classifiers in some cases.