We consider the extension of the method of Gauss-Newton from complex floating-point arithmetic to the field of truncated power series with complex floating-point coefficients. With linearization we formulate a linear system where the coefficient matrix is a series with matrix coefficients, and provide a characterization for when the matrix series is regular based on the algebraic variety of an augmented system. The structure of the linear system leads to a block triangular system. In the regular case, solving the linear system is equivalent to solving a Hermite interpolation problem. In general, we solve a Hermite-Laurent interpolation problem, via a lower triangular echelon form on the coefficient matrix. We show that this solution has cost cubic in the problem size. With a few illustrative examples, we demonstrate the application to polynomial homotopy continuation.