This book provides a comprehensive, authoritative and timely review of the astrophysical approach to the investigation of gravity theories. Particular attention is paid to strong-field tests of general relativity and alternative theories of gravity, performed using collapsed objects (neutron stars, black holes and white dwarfs) in relativistic binaries as laboratories. The book starts with an introduction which gives the background linking experimental gravity in cosmic laboratories to astrophysics and fundamental physics. Subsequent chapters cover observational and theoretical aspects of the following topics: from binaries as test-beds of gravity theories to binary pulsars as cosmic laboratories; from binary star evolution to the formation of relativistic binaries; from short gamma-ray bursts to low mass X-ray binaries; from stellar-mass black hole binaries to coalescing super-massive black holes in galaxy mergers. The book will be useful to researchers, PhD and graduate students in Astrophysics, Cosmology, Physics and Mathematics, who are interested in relativistic astrophysics, experimental gravity and general relativity.