In this dissertation, we study different machine learning algorithms including probabilistic, sparse and deep learning based models applied to multi-sensory datasets. In many machine learning problems, samples are collected from more than one source or modality. Also, various feature extraction methods can be used to provide more than one set...