AI Sitefinity Image Processor
Sitefinity has a nice feature to auto generate thumbnails. Although it is working fine most of the time, there are some cases that it is not enough. For example when there are people on the image it does not look good when their heads and faces are cropped. To illustrate this I’ll use a photo of part of our team.
You see how the heads in the top most of the photo are cropped?
Of course we can just resize it without cropping. Then we’ll have thumbnail with size 533x300 instead of 600x300 and smaller faces, which we don’t want because this photo was taken few years ago during the Christmas Holidays and we want to focus on the festive smiles. :)
Question is: How to tackle this one without doing it manually? The answer is Computer Vision.
If you are not familiar with Computer Vision, Machine Learning and Facial Recognition, you can check some resources online:
Quote from Wikipedia:
“Machine learning (ML) is the study of computer algorithms that improve automatically through experience. It is seen as a subset of artificial intelligence. Machine learning algorithms build a mathematical model based on sample data, known as "training data", in order to make predictions or decisions without being explicitly programmed to do so. Machine learning algorithms are used in a wide variety of applications, such as email filtering and computer vision, where it is difficult or infeasible to develop conventional algorithms to perform the needed tasks”
Quote from Wikipedia:
“Computer vision is an interdisciplinary scientific field that deals with how computers can gain high-level understanding from digital images or videos. From the perspective of engineering, it seeks to understand and automate tasks that the human visual system can do.
Computer vision tasks include methods for acquiring, processing, analyzing and understanding digital images, and extraction of high-dimensional data from the real world in order to produce numerical or symbolic information, e.g. in the forms of decisions. Understanding in this context means the transformation of visual images (the input of the retina) into descriptions of the world that make sense to thought processes and can elicit appropriate action. This image understanding can be seen as the disentangling of symbolic information from image data using models constructed with the aid of geometry, physics, statistics, and learning theory.”
Quote from Wikipedia:
“A facial recognition system is a technology capable of identifying or verifying a person from a digital image or a video frame from a video source. There are multiple methods in which facial recognition systems work, but in general, they work by comparing selected facial features from given image with faces within a database. It is also described as a Biometric Artificial Intelligence based application that can uniquely identify a person by analyzing patterns based on the person's facial textures and shape.”
Setting up the project
We will use the Quantum project. You can get it from GitHub and there are instructions on how to set
it up: https://github.com/Sitefinity/Telerik.Sitefinity.Samples.Quantum
For the computer vision part we will use the popular OpenCV which has a wrapper for .NET:
OpenCV - https://opencv.org/
OpenCvCsharp - https://github.com/shimat/opencvsharp
Install OpenCvCsharp nuget packages:
We will create a custom Image Processor for Sitefinty. For more details you can check the documentation:
Here is the code for our image processor:
Then we need to register it on Bootstrap (in Global.asax.cs):
protected void Application_Start(object sender, EventArgs e)
Bootstrapper.Initialized += Bootstrapper_Initialized;
void Bootstrapper_Initialized(object sender, Telerik.Sitefinity.Data.ExecutedEventArgs e)
if (e.CommandName == "Bootstrapped")
ObjectFactory.Container.RegisterType<IImageProcessor, FaceRecognitionImageProcessor>(new ContainerControlledLifetimeManager());
OpenCV in action:
- Go to Administration -> Settings -> Thumbnails
- Create new thumbnail profile
- In the “Resize image” drop down choose “Resize and center faces” (this is our custom image processor).
- Enter 600 for Width
- Enter 300 for Height
The resulting thumbnail:
Using our custom image processor
Note: Keep in mind that this
is only an example how we can utilize Computer Vision for better image manipulation. You may need to fine
tune the code or even to train your custom model to better suit your use case.