Energy balance can be maintained by adapting energy intake to changes in energy expenditure and vice versa, where short-term changes in energy expenditure are mainly caused by physical activity. Questions are whether physical activity is affected by over and under-eating, is intake affected by an increase or a decrease in physical activity, and does overweight affect physical activity? Presented evidence is largely based on studies where physical activity is quantified with doubly labeled water. Overeating does not affect physical activity while under-eating decreases habitual or voluntary physical activity. Thus, it is easier to gain weight than to lose weight. An exercise induced increase in energy requirement is compensated by intake while a change to a more sedentary routine does not induce an equivalent reduction of intake and generally results in weight gain. Overweight and obese subjects have similar activity energy expenditures than lean people despite they move less. There are two options to reverse the general population trend for an increasing body weight, reducing intake or increasing physical activity. Based on the results presented, eating less is most effective for preventing weight gain, despite a potential negative effect on physical activity when reaching a negative energy balance.