Discrete Element Method (DEM) simulations of granular media are very expensive, in particular during the sample initialization phase, which limits their use to short granular specimens. This paper proposes a data-driven 3D diffusion pipeline that learns packed voxel blocks from DEM databases and then stitches them into arbitrarily long assemblies. The method is demonstrated on railway ballast and lunar regolith simulants and yields speed-ups of more than two orders of magnitude with respect to DEM initialization. The generated assemblies are further exploited to run a large sleeper-track campaign, from which homogenized stress and strain fields are extracted and used to train a neural operator. The resulting surrogate provides fast predictions of continuum responses over long ballasted tracks while remaining statistically consistent with grain-scale simulations.