Using ML to Find the Semantic Region of Interest
Date & Time
Tuesday, October 25, 2022, 4:00 PM - 4:30 PM
Rob Gonsalves Zahra Montajabi

One of the most challenging problems in computer vision and image processing is the detection of the semantic regions of interest (SRoI). In this paper, we propose a method using OpenAI’s CLIP model to find SRoI by performing semantic search for objects in the image which are detected by an object detection model called Generic RoI Extractor (GRoIE). Finding the semantic regions of interest can be used in different image processing tasks such as image and video compression, enhancement, and reformatting. By knowing the semantic region of interest within images, we can improve the visual quality of images by compressing the more important parts with higher quality and the less important parts, such as the background, with a lower quality. This operation can be achieved without changing the overall compression ratio and the Peak Signal-to-noise Ratio (PSNR) quality metric. Finding the SRoI can make the processes of image enhancement and color correction more accurate by focusing only on the important parts. Moreover, for the image reformatting process, the important parts of the image may be lost. But by using the SRoI, we can reformat the image in a better way by keeping the most important regions in the frame.

Location Name
Salon 1
Take-Aways from this Presentation
• Generating importance map using CLIP • The importance map is not biased to specific objects like only people • Using importance map for image compression, image enhancement, image reformatting
featured session