Railroad ballast is uniformly graded coarse aggregate placed between and immediately underneath the ties to provide drainage and structural support for the loading applied by trains. In the United States, several ballast gradation recommendations are suggested by the American Railway Engineering and Maintenance-of-Way Association (AREMA). Most of these recommended uniform gradations have somewhat wide ranges at their control sieves; this range often creates significantly different grain-size distributions within the same gradation band. In this analysis, AREMA No. 24 gradation was studied for controlled changes in grainsize distributions within its band. Gradations were created by using three control sieves within the AREMA No. 24 gradation band. In addition, to emphasize the significance of particle shape, three sets of particles having high-, medium-, and low-angularity indexes quantified by image analysis were also considered in the packing simulations carried out with an image-aided ballast aggregate assembly modeling approach with the discrete element method (DEM). The significance of passing each control sieve as well as the importance of particle angularity on number of contacts, coordination number (average number of contacts that one particle makes with its neighbors), and porosity was analyzed by using analyses of variance and regression in the DEM packing simulations. The results indicated that increasing the number of particles between ballast nominal maximum sieve size and its half size would increase the number of particle contacts created in the granular assembly. When the imaging-based angularity indexes of the aggregate particles increased, the number of contacts also increased to give a higher coordination number. The ballast DEM simulations proved to be a powerful tool to study and optimize ballast gradations.