In this work we present a novel second order accurate well balanced (WB) finite volume (FV) scheme for the solution of the general relativistic magnetohydrodynamics (GRMHD) equations and the first order CCZ4 formulation (FO-CCZ4) of the Einstein field equations of general relativity, as well as the fully coupled FO-CCZ4 + GRMHD system. These systems of first order hyperbolic PDEs allow to study the dynamics of the matter and the dynamics of the space-time according to the theory of general relativity. The new well balanced finite volume scheme presented here exploits the knowledge of an equilibrium solution of interest when integrating the conservative fluxes, the nonconservative products and the algebraic source terms, and also when performing the piecewise linear data reconstruction. This results in a rather simple modification of the underlying second order FV scheme, which, however, being able to cancel numerical errors committed with respect to the equilibrium component of the numerical solution, substantially improves the accuracy and long-time stability of the numerical scheme when simulating small perturbations of stationary equilibria. In particular, the need for well balanced techniques appears to be more and more crucial as the applications increase their complexity. We close the paper with a series of numerical tests of increasing difficulty, where we study the evolution of small perturbations of accretion problems and stable TOV neutron stars. Our results show that the well balancing significantly improves the long-time stability of the finite volume scheme compared to a standard one.< Réduire