Modeling and predicting human and vehicle motion is an active research domain. Owing to the difficulty in modeling the various factors that determine motion (e.g. internal state, perception) this is often tackled by applying machine learning techniques to build a statistical model, using as input a collection of trajectories gathered through a sensor (e.g. camera, laser scanner), and then using that model to predict further motion. Unfortunately, most current techniques use offline learning algorithms, meaning that they are not able to learn new motion patterns once the learning stage has finished. This books presents a lifelong learning approach where motion patterns can be learned incrementally, and in parallel with prediction. The approach is based on a novel extension to hidden Markov models, and the main contribution presented in this book, called growing hidden Markov models, which gives us the ability to learn incrementally both the parameters and the structure of the model. The proposed approach has been extensively validated with synthetic and real trajectory data. In our experiments our approach consistently learned motion models that were more compact and accurate than those produced by two other state-of-the-art techniques, confirming the viability of lifelong learning approaches to build human behavior models.