In recent years, the railway transportation system has become one of the main means of transportation. Therefore, driving safety is of great importance. However, because of the potential of multiple breaks of elastic rail clips in a fixed rail, accidents may occur when a train passes through the track. This paper presents the development of a computer visual recognition system which can detect the status of elastic rail clips. This visual recognition system can be used in mass rapid transit systems to reduce the substantial need of manpower for checking elastic rail clips at present. The visual recognition system under current development includes five components: preprocessing, identification of rail position, search of elastic rail clip regions, selection of the elastic rail clip, and recognition of the elastic rail clip. The preprocessing system transforms the colored images into grey-level images and eliminates noises. The identification of rail position system uses characteristics of the grey-level variation and confirms the rail position. The search system uses wavelet transformation to carry out the search of elastic rail clip regions. The selection system finds a suitable threshold, using techniques from morphological processing, object search and image processing. The recognition system processes characteristics and structures of elastic rail clips. Experimental testing shows the ability of the developed system to recognize both normal elastic rail clip images and broken elastic rail clip images. This result confirms the feasibility in developing such a visual recognition system.