Ballast fouling, often associated with deteriorating railroad track performance, refers to the condition in which the ballast layer changes its composition and develops a much finer grain size distribution. Fouling is commonly caused by degradation or breakage of ballast aggregates under traffic loading, although other fine materials including but not limited to coal dust, fine-grained subgrade soils, and sand can also contaminate a clean and uniformly graded ballast layer. An experimental approach is described to characterize stages of railroad ballast degradation studied through Los Angeles abrasion testing in the laboratory. An aggregate image analysis approach is used to investigate ballast particle abrasion and breakage trends at every stage through detailed quantifications of individual ballast particle size and shape properties. The experimental study indicated that the fouling index (FI) commonly used by practitioners was indeed a good indicator of fouling conditions, especially when all voids created by larger particles were filled by fine materials as FI values approached 40. Image analysis results of ballast particles larger than 9.5 mm (3∕8 in.) scanned after a number of turns of the Los Angeles abrasion drum showed good correlations between percentage changes in aggregate shape properties, that is, imaging-based flatness and elongation, angularity and surface texture indexes, and the FI. The establishment of such relationships between in-service track fouling levels and ballast size and shape properties with similar field imaging techniques would help to understand field degradation trends better and as a result improve ballast serviceability and life-cycle performance.