🔥 Computer Vision

Throughout my career, I have explored various challenges in the field of Computer Vision, leveraging both classical image processing techniques and state-of-the-art deep learning models.

Below is a curated list of some of the significant projects I have been involved in:

✅ Defect Detection in Semiconductor Wafers and CMP Retaining Rings

Applied image processing algorithms to detect surface defects on semiconductor wafers and CMP retaining rings used in manufacturing equipment.

✅ Semiconductor Wafer Metal Wiring Quality Detection using SEM Images

Contributed to vectorizing parts of measurement algorithms used for detecting the quality of metal wiring in semiconductor wafers using Scanning Electron Microscopy (SEM) images.

✅ Exploring Deep Learning Model Optimization Techniques

Investigated and implemented deep learning model optimization techniques, including Pruning, Quantization, Sparsity, and Knowledge Distillation.

✅ Semiconductor Defect Image Search

Implemented image similarity matching by using encodings generated from Self-Supervised Vision Transformers with DINO (by Meta).
Leveraged LaMa: Resolution-robust Large Mask Inpainting with Fourier Convolutions for removing spurious artifacts from images.

Developed custom trained SSD-MobileNetv2* models for person and intrusion detection.
Utilized DeepSort for accurate tracking of individuals.

✅ Vehicle Detection, Classification & Intrusion Detection

Experimented with YOLOv3 for vehicle detection and MobileNet, Inception, Xception, and ResNets for vehicle type classification.
Built highly optimized classification pipelines using the tf.Data API.
Custom trained SSD-MobileNetV2* models for vehicle detection.
Utilized DeepSort for precise tracking of vehicles.

*The custom models were trained using Intel OpenVINO™ Training Extensions

✅ Helmet & jacket detection and classification

Explored Focal Loss based RetinaNet (Facebook's Detectron model) for helmet and jacket detection.
Optimized the model using TensorRT and deployed it on NVIDIA T4 cards.
Custom trained MobileNetV2 models for classification.
🚀🚀 For all deep learning model inference, I harnessed the robust and high-performance Intel hardware ecosystem, which includes Intel Xeon Processors, Intel FPGAs, and Intel Server GPUs, in conjunction with the cutting-edge Intel® Distribution of OpenVINO™ Toolkit. This powerful combination allowed for accelerated and optimized model inference, enabling efficient and lightning-fast computations for a wide range of Computer Vision tasks.

*The custom models were trained using Intel OpenVINO™ Training Extensions

✅ Quality monitoring in manufacturing

Developed a custom algorithm for detecting stencil-drawn arrow direction on large aluminum blocks to optimize production processes.
- This resulted in a remarkable $10M savings per month.

✅ Person activity monitoring

Worked on camera calibration and rectification.
Monitored construction work activity at remote locations using Fisheye cameras for a wide view.
Deployed person idle time detection in user-specified areas.

✅ Vehicle Analytics

Developed a comprehensive vehicle analytics system for detecting, tracking, and monitoring vehicles in various traffic scenarios. The project had four primary use cases:
1. High-speed vehicle detection
2. Detection and tracking of vehicles moving in the wrong direction
3. Detection of vehicles violating traffic signals
4. Counting vehicles passing a particular traffic signal.
5. Traffic signal congestion monitoring using Gaussian Mixture models.
  
  Utilized YOLOv1 for vehicle detection and DeepSort for tracking.

✅ Slip and Fall Detection

Implemented a system for detecting if a person has fallen on the ground while walking.
- Utilized human pose estimation for identifying a person's posture and detecting falls.

✅ Un-attended object detection

Developed an unattended object detection system using Gaussian Mixture Model based approach.

✅ Face Detection and Recognition

Developed face detection and recognition systems using multiple techniques:
- Version 1:
  - Utilized Haar Cascades and Local Binary Patterns (LBPs) algorithms for face detection.
  - Experimented with Eigenfaces and Fisherfaces algorithms for face recognition.
  - Employed Kalman filter for face tracking, particularly useful in a pulse monitoring application based on Eulerian Motion Magnification running on real-time video streams.
- Version 2:
  - Leveraged Dlib (Histogram of Oriented Gradients (HOG)) for face detection.
  - Utilized Support Vector Machine (SVM) classifier for one-shot face matching and recognition.

✅ BlackJack Game Analytics

Worked on image segmentation techniques to differentiate and count different types of poker chips on BlackJack casino tables.
- Explored various camera systems such as Minoru 3D Stereo camera, Creative DepthSense 325 3D TOF Sensor, Microsoft Kinect, and created a stereo camera setup using two web cameras.
- Performed camera calibration and scene depth estimation using stereo vision and time-of-flight (TOF) sensing.
- Developed Python wrappers for depthsense C++ library to interface with the Creative DepthSense 325 3D TOF Sensor for camera calibration and scene depth estimation.
- Worked with various computing boards, including Raspberry Pi, Odroid, Beaglebone, Arduino, and NVIDIA Jetson to interface with the camera systems and process image data before sending it to the central server.
- Check out hardware section for more details

✅ On-shelf Products Analytics

Created a real-time system for analyzing products on retail shelves.
- Utilized Bag of Visual Words (BoVW) for classifying products based on their visual appearance.
- Implemented depth-based object counting to accurately count the number of products on shelves, even when some are occluded.

✅ Obstacle avoidance and path planning for Lego bot

Improved obstacle avoidance algorithm for a Lego bot.
Implemented the A* path planning algorithm for precise bot movement.