There is a large availability of various types of sensors at considerably cheaper prices.  These sensors generate vast amounts of data – could be scalar (temperature, luminosity, humidity, pressure, etc) or vector (image, video). Computational power is also widely accessible through mobile phones -almost all of the common computation is possible on mobile phones.

The generated heterogeneous data requires processing and extraction of intelligence to make sense from the data. This is also further accelerated by communication technology and connectivity (among humans, machines and systems), leading to Internet of Things.

Towards this development, big data analytics, machine learning, artificial intelligence and deep learning have gained popularity to provide meaningful and insightful information in an intelligent way.

My research interests intersects various aspects of Computer Vision, Machine Learning and IoT in sensing, signal processing, computation, analytics, and decision-making algorithms.


  • Computer Vision
  • Pattern Recognition
  • Machine Learning
  • Artificial Intelligence
  • Embedded Systems
  • Sensor Networks
  • Internet of Things


  • Camera based Assistive Technology for Vision-Impaired Mobility

    Portable Navigation System Using Camera and Laser for Pothole and Uneven Surface Detection

    Vision is one of the most advanced and important sensory input in humans. However, many people have vision problems due to birth defects, uncorrected errors, work nature, accidents, and aging. The white cane and guide dog are the most widely used means of navigation for the vision-impaired. With advancements in technology, electronic devices have been created using different sensors and technologies to help navigate the blind. Electronic Travel Aids (ETAs) assist in navigating a person by collecting information about the environment and relaying this information in a form that allows a blind or vision-impaired person to understand the nature of the environment.

    However, there is still a lack of devices to detect potholes and uneven pavements, which inhibits mobility after dark. This pilot study proposes a computer vision based pothole and uneven surface detection approach to assist blind people in meeting their mobility needs. The system includes projecting laser patterns, recording the patterns through a monocular video, analyzing the patterns to extract features and then providing path cues for the blind user. With over 90% accuracy in detecting potholes, the proposed system aims to assist blind people in real-time navigation.

    Our approach uses a novel feature descriptor dubbed Histogram of Intersections (HoI) to detect potholes and uneven surfaces. The system uses Two different types of laser patterns (mesh and cross-hair) are projected on to surfaces (such as roads, stairs, pedestrian pathways) and using a GoPro camera, video of the projected laser patterns are recorded. Patterns are detected based on number of laser grid (intersections) and strength of the laser pattern intensities.

  • IoT-based Real-time Urban Microclimate Monitoring

    Real-time analysis of urban environment (parks, gardens) from canopy cover

    Urban Heat Island (UHI) is an increasingly visible phenomenon. UHI is mainly attributed to urbanization with increasing buildings and reduction of urban trees. With increasing buildings, night temperatures at the center of the cities remain high compared to suburban areas. This causes the temperature at Central Business Districts (CBDs) to have higher temperatures

    Understanding the behaviour of the complex environment ecosystem requires analysis of detailed observations over a range of different conditions. One such example in urban areas includes the study of tree canopy cover over the microclimate environment using heterogeneous sensor data.

    There are several challenges that need to be addressed, such as obtaining reliable and detailed observations over monitoring area, detecting unusual events from data, and vi- sualizing events in real-time in a way that is easily understandable by the end users (e.g., City Councils).

    In this regard, we propose an integrated geovisualization framework, built for real-time wireless sensor network data on the synergy of computational intelligence and visual methods, to analyze complex patterns of urban microclimate. A Bayesian maximum entropy based method and a hyperellipsoidal model based algorithm have been build in our integrated framework to address above challenges.

    The proposed integrated framework was verified using the dataset from an indoor and two outdoor network of IoT devices deployed at two strategically selected locations in Melbourne, Australia. The data from these deployments are used for evaluation and demonstration of these components’ functionality along with the designed interactive visualization components.

  • Detecting Crowd Motion Events from Videos

    Uses optical flow and motion tracking to infer crowd motion events

    Video analytics is very helpful in learning the behavioral characteristics of humans from videos. However, detecting and predicting events in the videos is both exacting and challenging. Individual object detection and tracking is a challenging task in multi-object scenarios, and the difficulty increases further in crowded scenes. In particular, event detection in crowded scenarios becomes complex when faced with articulated human movements and occlusions

    Crowd events such as walking, running, merging, separating into groups (“splitting”), dispersing, and evacuating are critical to understanding crowd behavior. However, video data lie in a high-dimensional space, whereas events lie in a low-dimensional space. Difficulty lies in detecting people and inferring their movements in the scene.

    This work introduces a new framework based on optical flow manifolds (OFM) to detect crowd events.  Essentially, the events are recognized  from optical flow vector on a manifold. Experiment results suggest that the proposed semi-supervised approach performs best in detecting merging, separating into group (“splitting”), and dispersion events compared with existing methods. The advantages of the semi-supervised approach are the requirement of a single parameter to detect crowd events, and results that are provided on a frame-by-frame basis.

  • Detecting Loitering Behaviour in Crowded Scenes

    Method to detect loitering (suspicious) behaviour individuals from videos

    From a crowd management/surveillance viewpoint, it is important to have automated tools to detect loitering people.  This could be individuals or group of people loitering in and around an area in public spaces. Or it could be people who appear to be suspicious.

    This work provides a framework to detect and track patterns of loitering/suspicious individuals. The framework uses video as input and outputs individuals marked with possibly suspicious or exhibit loitering behaviour.

    The framework uses spatial  and spatio-temporal features. Specifically, Gray Level Co-occurrence Matrix (GLCM)-based texture features, such as contrast, correlation, energy, and homogeneity, are used. These features are encoded to represent each video frame as well as spatio-temporal relations.

    Hyperspherical clustering is used to extract trajectories of suspicious people, resulting in loitering behavior detection. Hyperspherical clustering is in general used in anomaly detection applications.

  • Design of Low-cost Water Quality Monitoring System

    Real-time water quality monitoring for catchment areas

    Water pollution remains a key factor contributing to declining ecological health in aquatic ecosystems worldwide. In Australia, the state of Victoria is facing a major challenge in maintaining water quality in the freshwater systems. Nearly 80% of waterways in Victoria are in poor to moderate condition.

    Good water quality is essential for the health of our aquatic ecosystems. Continuous water quality monitoring is an important tool for catchment management authorities, providing real-time data for environmental protection and tracking pollution sources; however, continuous water quality monitoring at high temporal and spatial resolution remains prohibitively expensive.

    An affordable wireless aquatic monitoring system will enable cost-effective water quality data collection, assisting catchment managers to maintain the health of aquatic ecosystems. In this work, a low-cost wireless water physiochemistry sensing system is presented.

    In this regard, we developed a prototype sensor as one component of the Au- tonomous Live Animal Response Monitor (ALARM) currently under development at the Victorian Center for Aquatic Pollution Identification and Management (CAPIM). As an important component of the ALARM biosensor, our system was designed to measure a suite of biologically relevant physiochemical parameters in freshwater.

    We measured temperature, light in- tensity, pH, electrical conductivity (EC), total dissolved solids (TDS), salinity (SAL), dissolved oxygen (DO) and oxidation reduction potential (ORP).  These parameters provide insights into the current status of changing water conditions and assist in identifying pollution sources. Our system was tested at CAPIM by measuring these parameters continuously.

    The system uses low-cost sensors and open-source hardware to deliver continuous measurement of water quality at substantially reduced cost (by one-third).

  • Estimating Crowd Density from Videos

    Provides automated estimates number of people in a given area from videos

    Growing population and urbanization has mobilized the day-to- day activities and consequently, people endeavor to participate more often in public events.  Often this leads to scenarios such as stampede, people crushing and unruly crowd behavior at large events. Monitoring of such activities leads to adopting mitigating strategies thereby avoiding crowd related disasters.

    Crowd density estimation is to estimate number of people per given area in a 2D scene. Accurate density estimation assists security personnel and crowd managers to have a knowledge of the ground situation in tine event of untoward incidents. Density estimation also helps to provide likelihood of stampede and adopt consequent plans for emergency plannings.

    However, automated density estimation often faces difficulties in detecting motion from the scene due to varying environmental conditions, occlusion and crowded scenes. Instead of detecting and tracking individual person, density estimation is an approximate method to count people. This work presents a semi-supervised approach using optical flow features and clustering those features to represent density in a 2D area.

  • Real-time Sensing and Asset Tracking

    Uses GPS, Short Message Service (SMS) and Google Maps

    For asset management. fleet management and logistics, it is necessary to track assets, including trucks, cars, in real time.

    The project involved developing hardware, firmware and embedded system to acquire, process and send GPS locations to a centralized server through SMS.  The platform developed is scalable to generic sensor networking platform having numerous applications ranging from medical to military and from aerospace to underwater applications.

    Figure (Courtesy: Google Maps) shows the path traced by a vehicle in real time from the developed system.