The joint variability in an ensemble of neurons contains important signatures of the circuit connectivity, the ways in which synaptic plasticity is likely to affect it, and - ultimately - the computations the circuit is performing. Exploiting these links quantitatively requires a formal understanding of how correlations arise mechanistically in recurrently connected networks of neurons. Existing theories provide a somewhat blurry picture, as they often connect some high-level statistics of the circuit connectivity to other summary statistics of the pairwise correlation distribution. More elaborate theories can predict pairwise correlations for any specific pair of neurons, but assume either small firing rate variability or weak correlations. Here, we develop a novel theoretical framework to obtain the full correlational structure of a stochastic network of nonlinear neurons described by rate variables. Under the assumption that pairs of membrane potentials are jointly Gaussian - which they tend to be in large networks - we obtain deterministic equations for the temporal evolution of the mean firing rates and the noise covariance matrix that can be solved straightforwardly given the network connectivity. Importantly, our theory requires neither the fluctuations nor the pairwise noise correlations to be weak, and works for several biologically motivated, convex single-neuron gain functions. We apply our formalism to visual area MT, for which data on how particular stimuli affect neuronal variability has been published recently (Ponce-Alvarez et al., 2013). We find that a balanced ring model network with a threshold-quadratic nonlinearity captures the stimulus-dependence of both the Fano factor and the noise correlations. Interestingly, this network operates in a regime very different from previous proposals for variability quenching which all relied on multistable attractor dynamics, but closer to the behavior of the “stabilized supralinear network” of Ahmadian et al. (2013) of which we extend the analysis to the stochastic regime.