In this note, we develop semi-analytical techniques to obtain the full correlational structure of a stochastic network of nonlinear neurons described by rate variables. Under the assumption that pairs of membrane potentials are jointly Gaussian – which they tend to be in large networks – we obtain deterministic equations for the temporal evolution of the mean firing rates and the noise covariance matrix that can be solved straightforwardly given the network connectivity. We also obtain spike count statistics such as Fano factors and pairwise correlations, assuming doubly-stochastic action potential firing. Importantly, our theory does not require fluctuations to be small, and works for several biologically motivated, convex single-neuron nonlinearities.