Application of a machine-learning pipeline to a multiclass classification problem

A problem of predicting a future event outcome from a fixed set of possible outcomes can be approached as a traditional machine-learning classification task. There are several steps that need to be done in order to build a reliable machine-learning pipeline with a solid performance on unseen future test data. First, the data should be analyzed using appropriate exploratory data analysis techniques. Various data types need to be pre-processed and transformed, utilizing different feature engineering methods before hypotheses created during the analysis can be tested. During the modeling process, some data can be omitted, while some data might need to be transformed to achieve the best predictive power. Different classification algorithms, such as k-NN, logistic regression, random forest, XGBoost, LightGBM, can be applied with respect to their assumptions and limitations. Hyper-parameter tuning process can serve to improve the predictive performance of individual models. Multiple classification models can be combined using ensemble methods to collectively outperform the best individual model. This thesis addresses all described practices and applies them to the problem of predicting the future interest level in rental apartment listings, which is a multiclass classification problem. The source of the data used for modeling is an already finished competition organized by Kaggle in cooperation with Two Sigma Connect, and Renthop. The final model is a stacked ensemble of previously mentioned models using the LightGBM classifier. The final model score would place among the top 25% of the participants’ model scores.