This paper presents a method for real-time image classification which combines Local Binary Pattern (LBP) descriptor and ensembles of Extreme Learning Machines (ELM). We start by extraction of a multi-channel LBP descriptor over separate color channels and several scales of the input image. The descriptor is characterized by compactness and robustness to illumination and resolution changes. Image category prediction is done using recently introduced specific single layer neural network called Extreme Learning Machine (ELM). To overcome main disadvantages of the ELM, instability and non-optimal output, we combined multiple ELMs into an ensemble using appropriate score aggregation strategy. Our evaluation on a standard benchmark dataset consisting of a thousand images from ten categories, has shown high accuracy of results while executing in real-time during tests.