The widespread availability of surveillance cameras and digital technology has improved video based security measures in public places. Surveillance systems have been assisting officials both in civil and military applications. It is helping to identify unlawful activities by means of uninterrupted transmission of surveillance videos. By this, the system is adding extraneous onus on to the already existing workload of security officers. Instead, if the surveillance system is intelligent and efficient enough to identify the events of interest and alert the officers, it alleviates the burden of continuous monitoring. In other words, our existing surveillance systems are lacking to identify the objects that are dissimilar in shape, size, and color especially in identifying human beings (nonrigid motions). Global illumination changes, frequent occurrences of shadows, insufficient lighting conditions, unique properties of slow and fast moving objects, unforeseen appearance of objects and its behavior, availability of system memory, etc., may be ascribed to the limitations of existing systems. In this paper, we present a filtering technique to extract foreground information, which uses RGB component and chrominance channels to neutralize the effects of nonuniform illumination, remove shadows, and detect both slow-moving and distant objects.