Tracking people's behaviors for Detecting and understanding their Suspicious activities.in social Environments

Abstract

This thesis represents an approach to develop a real-time autonomous invigilation and interrogation system suitable for different social environments to track people's cheating behaviors and to detect their lie. To this journey, a real-life smart class room scenario is primarily considered as social environment and we started with detecting the student's patterns of crimes by studying their psychology in smart classroom during examination period. In a smart classroom scenario, the built-in video camera of each computer will capture continuous video frames and feed it to our developed system for tracking student's suspicious behaviors. From the captured continuous video frames, we have detected target student's Visual Focus of Attention (VFOA) to track his/her suspicious gaze direction. A 3D head tracker is used to extract the facial regions from each video frame. From the extracted facial regions, face points and eye regions are detected. Using a Vector Field of Image Gradient (VFIG), the pupil of eye is pointed out from the detected face points and eye regions. In our approach, we have also used the face points to create a rectangle outside of the left eye. In our experiment, we have used the head movement along with the change of coordinate of one eye (left eye) with respect to the left eye rectangle to detect the gaze of suspect by applying different techniques under different lighting conditions and different distances from the camera and using different participants. Additionally, suspicious body movement have been tracked using Canny Edge Detector for slow, very slow and high dynamics with optimal accuracy for the same purposes. We also proposed an approach to track the fraud candidates by detecting the age and gender of the examinee in examination hall room by matching it with the pre-stored age and gender information. Texture variation of wrinkle density in the forehead, eye lids, and cheek area has been considered as key clues for age and gender classification. We have also included a real time emotion detection mechanism to our system. Besides VFOA tracking, change of shape of eyebrow has been detected using optical flow features to build a smart and robust autonomous interrogation system. Furthermore, the viability of the proposed real-time autonomous interrogation system is demonstrated by experimenting with participants in a smart classroom under controlled environment. Finally, the system is tested to validate its effectiveness. Our obtained result shows that, we can efficiently track the focus of attention of the examinee (the average accuracy is 80%) from the coordinate of the pupil of the eye combining with head position and consequently we can detect the sustained attention together with transients very effectively and control the attention by deploying an alarming system if inattention is detected. Canny edge detector shows best accuracy (>90%) when the examinee takes 3, 4 or 5 second for a proper movement. Our intelligent system works perfectly in happy, fear, surprise and neutral emotions detection and provides better accuracy to detect 2 I to 35 and 36 to 50 age groups with standard deviation 5 and 7 respectively. It also shows optimal accuracy to classify examinee of different age and genders. It is also revealed that in controlled environment our system shows fair accuracy of lie detection (upto 100% accuracy for different types of suspicious symptoms detection). So the proposed system can detect different types of crimes based on the external symptoms provided by suspect in social environments like smart class room and interrogation room . Keywords: