This article derives and implements a computational physics model for model-based image reconstruction in magnetic particle imaging (MPI) applications. To our knowledge, this is the first ever computationally tractable model-based image reconstruction in MPI, which is neither constructed from calibration or simulation experiments or limited to specific scan acquisition geometries. The derived model results in a system constructed from a series of fast linear transforms, each of which incorporate the individual components from the paramagnetic model. These include the transmit velocity, field free point location, gradient strength, receive coil sensitivity, and receive chain filtering, and each of these modeling components are amendable to any changes in the acquisition parameters. This allows us to adopt a computationally tractable system matrix modeling approach to MPI for any scan specific parameters at very high pixel resolutions. The model is derived from first principles, and it results from taking the fundamental MPI signal theory and decomposing these modeling equations into the series of linear transforms acting on a pixelated image. Each transform is formally defined in matrix form but implemented in a matrix-free fashion with fast and/or sparse operations. For these reasons, our new model should be a fundamental tool in the future of computational imaging in MPI. We demonstrate our new method on a variety of pre-clinical and simulated data sets, and these results confirm that our method is both efficient and accurate.