Network Video Analytics in C#: How to achieve line
detection (NVA - Object Detection)
of surveillance and security camera systems is constantly growing these days.
In the ancient times, there was a camera and a guard whose main task was to
stare at the monitor and try not to fall asleep. As technology is developing
day-by-day, new technologies arise to help the people who are in need of a
reliable security system.
One of these new
technologies is object detection.
Inside object detection, there are the so-called
feature-based methods. These methods aim to find possible matches
between the features of the object and the image of the camera. The topic of
this tutorial belongs to these feature-based methods.
The following tutorial aims to introduce line
detection. First, you can read a brief theoretical background about object
detection in general and line detection. After this part you will find a
detailed description on how to implement this exciting function in C# along
with creating a graphical user interface for the program.
I hope this article will help you in developing your
own application to perform line detection.
What will you
need to succeed?
Before we start programming,
let's see what devices are needed for this game.
Obviously, it is
necessary to have a USB camera device to work with. You can consult the webpage for USB cameras;
there are some pretty nice ones there.
software part of the business, you need to download Visual C# 2010 Express and .Net Framework 4. Those are needed for the coding procedure.
One final part
is missing now: a camera SDK that you can use for developing your line detecting
solution. I found this software, Ozeki's Camera SDK to be useful, it was easy
to work with, and so I can recommend it to you.
Now that you are
fully prepared to start working, take a look at the literature.
What is object
detection and line detection?
To get a deeper
knowledge about object detection, we can consult our old friend, the Internet.
As it is stated on Wikipedia, "object detection is a computer technology related to computer vision and image processing that
deals with detecting instances of semantic objects of a certain class (such as
humans, buildings, or cars) in digital images and videos. Well-researched
domains of object detection include face detection and pedestrian
Another source, Mathworks describes object detection as "the process of finding instances of
real-world objects such as faces, bicycles, and buildings in images or videos."
As possible fields of usage, the webpage lists image retrieval, security,
surveillance, and automated vehicle parking systems.
If we dig deeper
in this topic we can find line detection. The basic task of line detection is
to indicate if an object on the camera's image crosses any of the virtual lines
we have on our image. You can use this function for security purposes or, for
example, to observe costumer traffic in your shop.
Fine, now here
comes the exciting part of the story. Let's see how to implement this function.
Creating the C#
To detect lines
on the image of a camera, we should use the ILinedetector object.
After creating an instance of ILineDetector
with the help of the static ImageProcesserFactory class, it will
be able to detect lines on frames and on videos, too. In case of frames, we can
make the image with the help of the Process() method of the instance, while
in the case of videos, we should use the ImageProcesserHandler mediahandler.
After this tiny
piece of information, let the actual work begin!
As the first
step of coding we should apply the using lines.
Then we qualify
the namespace in our code.
basic steps, it is necessary to add the global variables to our code. Here, you
will need Webcamera_webCamera. This instance will help us to get the
image of the camera. The next variable is MediaConnector_connector. Its task
is to connect the mediahandlers. Now, here comes ImageProcesserHandler_imageProcesserHandler.
This guy is a mediahandler that runs the instances that implements the IImageProcesser interface. Now we have
the interface that is responsible for line detection and that implements the IImageProcesser interface: ILineDetector_lineDetector.
Three more variables to go. FrameCapture_frameCapture is another
mediahandler. You can configure the frequency of processing with this one. We
have the VideoViewerWF instance that is a GUI tool for Windows Forms
applications and it will help you to display the video. And last but not least,
we have the DrawingImageProvider instance that is also a mediahandler. (Mediahandlers...
mediahandlers everywhere.) Its task is to prepare the image, sent by the VideoSender
class mediahandlers, for the VideoViewerWF
instance. Now we have all the necessary global variables set in our code. Let's
see how it looks like now.
public partial class Form1 : Form
We are done with
the global variables for now, so we can move on to some methods that should be
called to work out line detecting.
The first of
these methods is Init(). This is the method that initializes most of the global
variables. This is where the FrameCapture
mediahandler instance is configured and also, where the ILineDetector instance is created with the help of ImageProcesserFactory.
We add this instance to the ImageProcesserHandler
instance. Every time the ILineDetector
instance processes an image, it will be indicated by the DetectionOccured event
which we can subscribe for here, too.
_frameCapture = new FrameCapture();
_webCamera = WebCamera.GetDefaultDevice();
_connector = new MediaConnector();
_originalImageProvider = new DrawingImageProvider();
_processedImageProvider = new DrawingImageProvider();
_lineDetector = ImageProcesserFactory.CreateLineDetector();
_lineDetector.DetectionOccurred += _lineDetector_DetectionOccurred;
_imageProcesserHandler = new ImageProcesserHandler();
method is the SetVideoViewers() method, that generates and initializes the
objects that are responsible for displaying the video. It defines the VideoViewerWF instances, sets their
properties, assigns the proper DrawingImageProvider
instances and adds them to the GUI.
_originalView = new VideoViewerWF
BackColor = Color.Black,
Location = new Point(10, 20),
Size = new Size(320, 240)
_processedView = new VideoViewerWF
BackColor = Color.Black,
Location = new Point(350, 20),
Size = new Size(320, 240)
We should not forget about the InvokeGUIThread() method. This is the method that handles the GUI
thread. It executes the specified method asynchronously on the thread that the
control's underlying handle was created on.
void InvokeGUIThread(Action action)
The next method,
fills in the Text Boxes on the GUI with the configurations of the ILineDetector with the help of the InvokeGUIThread() helper method.
chk_ShowImage.Checked = _lineDetector.ShowImage;
tb_Red.Text = _lineDetector.DrawColor.R.ToString();
tb_Green.Text = _lineDetector.DrawColor.G.ToString();
tb_Blue.Text = _lineDetector.DrawColor.B.ToString();
tb_DrawThickness.Text = _lineDetector.DrawThickness.ToString();
tb_AngleResolution.Text = _lineDetector.AngleResolution.ToString();
tb_CannyThreshold.Text = _lineDetector.CannyThreshold.ToString();
tb_CannyThresholdLinking.Text = _lineDetector.CannyThresholdLinking.ToString();
tb_DistanceResolution.Text = _lineDetector.DistanceResolution.ToString();
tb_LineGap.Text = _lineDetector.LineGap.ToString();
tb_LineWidth.Text = _lineDetector.LineWidth.ToString();
tb_Threshold.Text = _lineDetector.Threshold.ToString();
Moving on, we
have the ConnectWebcam() method. This one is responsible for connecting
the proper mediahandler instances with the help of the MediaConnector instance. One ImageProvider
object receives the original image of the camera, while the other one receives
the processed image.
through all the initialization, the mediahandlers can start operating. Now, the
method will help us.
If you press the
Set button that belongs to the Group Box on the GUI (we'll talk about it a
little later), the following event will be called whose task is to configure
void btn_Set_Click(object sender, EventArgs e)
_lineDetector.AngleResolution = Double.Parse(tb_AngleResolution.Text);
_lineDetector.CannyThreshold = Double.Parse(tb_CannyThreshold.Text);
_lineDetector.CannyThresholdLinking = Double.Parse(tb_CannyThresholdLinking.Text);
_lineDetector.DistanceResolution = Double.Parse(tb_DistanceResolution.Text);
_lineDetector.LineGap = Double.Parse(tb_LineGap.Text);
_lineDetector.LineWidth = Double.Parse(tb_LineWidth.Text);
_lineDetector.Threshold = Int32.Parse(tb_Threshold.Text);
Let's take a
closer look to see what those values in this snippet are responsible for.
you can set the resolution in the angle area.
CannyThreshold determines the value of thresholding to find initial
segments of strong edges.
value is used to determine the number of pixels in the edges of an image.
DistanceResolution defines the resolution between pixel-related units.
LineGap indicates the minimum gap between lines detectable
LineWidth indicates the minimum width of detectable lines.
For a line to be
actually considered as detected, it is necessary to reach the value of Threshold.
detection happened, you can configure how the output image should behave, what
changes it shall undergo. You can configure these features with the help of the
With the help of
checkbox you can set if you only want to see the processed image with the
detected shapes in the programme or you also want to have the original image.
You can set the
colour of the marking of the detected objects with DrawColor.
you can set the thickness of the marking of the detected objects.
are executed by the Set button on the highlight group box. If you click on this
button, the following event will be called:
void btn_HighlightSet_Click(object sender, EventArgs e)
_lineDetector.ShowImage = chk_ShowImage.Checked;
_lineDetector.DrawColor = Color.FromArgb(Int32.Parse(tb_Red.Text), Int32.Parse(tb_Green.Text), Int32.Parse(tb_Blue.Text));
_lineDetector.DrawThickness = Int32.Parse(tb_DrawThickness.Text);
each image/frame, the DetectionOccured
event will pop up.
void _lineDetector_DetectionOccurred(object sender, LineDetectedEventArgs e)
foreach (var info in e.Info)
You can find the
list of the detected lines in the arguments of this event and in that list you
can query the starting and endpoint, the direction and the length of each line.
Creating the GUI
Of course, a
nice user interface will be needed in order to manage the camera and line
detection properly. I will provide you with some code snippets I used to create
my GUI. As it can be seen on the picture, my GUI is built from 4 parts. The
first part is where the original and the processed pictures are shown. Under
this you can see the field where the list of detections is created. Two group
boxes can be seen on the right, one for highlights and one for settings.
Now take a
closer look at the different parts of the user interface.
Let's start with
the two images. On the left side you can see the original image of the camera.
this.label1.AutoSize = true;
this.label1.Font = new System.Drawing.Font("Microsoft Sans Serif", 8.25F, System.Drawing.FontStyle.Bold, System.Drawing.GraphicsUnit.Point, ((byte)(238)));
this.label1.Location = new System.Drawing.Point(30, 265);
this.label1.Name = "label1";
this.label1.Size = new System.Drawing.Size(87, 13);
this.label1.TabIndex = 0;
this.label1.Text = "Original image";
On the right
side there is the processed image.
this.label2.AutoSize = true;
this.label2.Font = new System.Drawing.Font("Microsoft Sans Serif", 8.25F, System.Drawing.FontStyle.Bold, System.Drawing.GraphicsUnit.Point, ((byte)(238)));
this.label2.Location = new System.Drawing.Point(370, 265);
this.label2.Name = "label2";
this.label2.Size = new System.Drawing.Size(103, 13);
this.label2.TabIndex = 1;
this.label2.Text = "Processed image";
The Set button under
Settings was mentioned before with which you can configure the basic settings
in your program. First, let's see how to create that almighty Set button.
this.btn_Set.Location = new System.Drawing.Point(189, 368);
this.btn_Set.Name = "btn_Set";
this.btn_Set.Size = new System.Drawing.Size(58, 23);
this.btn_Set.TabIndex = 2;
this.btn_Set.Text = "Set";
this.btn_Set.UseVisualStyleBackColor = true;
this.btn_Set.Click += new System.EventHandler(this.btn_Set_Click);
Now we have the
button. But if we want to adjust any settings with it, we need some parameters that
we can adjustJ
Let's start with
this.tb_AngleResolution.Location = new System.Drawing.Point(138, 26);
this.tb_AngleResolution.Name = "tb_AngleResolution";
this.tb_AngleResolution.Size = new System.Drawing.Size(87, 20);
this.tb_AngleResolution.TabIndex = 3;
Now the code for
the Canny Threshold comes.
this.tb_CannyThreshold.Location = new System.Drawing.Point(138, 58);
this.tb_CannyThreshold.Name = "tb_CannyThreshold";
this.tb_CannyThreshold.Size = new System.Drawing.Size(87, 20);
this.tb_CannyThreshold.TabIndex = 4;
And do not
forget about Canny Threshold Linking.
this.tb_CannyThresholdLinking.Location = new System.Drawing.Point(138, 90);
this.tb_CannyThresholdLinking.Name = "tb_CannyThresholdLinking";
this.tb_CannyThresholdLinking.Size = new System.Drawing.Size(87, 20);
this.tb_CannyThresholdLinking.TabIndex = 5;
Or about Distance
this.tb_DistanceResolution.Location = new System.Drawing.Point(138, 122);
this.tb_DistanceResolution.Name = "tb_DistanceResolution";
this.tb_DistanceResolution.Size = new System.Drawing.Size(87, 20);
this.tb_DistanceResolution.TabIndex = 6;
Here comes the Line Gap.
this.tb_LineGap.Location = new System.Drawing.Point(138, 154);
this.tb_LineGap.Name = "tb_LineGap";
this.tb_LineGap.Size = new System.Drawing.Size(87, 20);
this.tb_LineGap.TabIndex = 7;
And the Line
this.tb_LineWidth.Location = new System.Drawing.Point(138, 186);
this.tb_LineWidth.Name = "tb_LineWidth";
this.tb_LineWidth.Size = new System.Drawing.Size(87, 20);
this.tb_LineWidth.TabIndex = 8;
And finally, the
code for Threshold.
this.tb_Threshold.Location = new System.Drawing.Point(138, 218);
this.tb_Threshold.Name = "tb_Threshold";
this.tb_Threshold.Size = new System.Drawing.Size(87, 20);
this.tb_Threshold.TabIndex = 9;
These are the different settings you can adjust in
your program. After we are ready creating the boxes where you
can provide the values for these parameters, we can go on.
As it was
mentioned above, you will have a list of the detected lines. Here is how you
can create the field for the list.
this.lb_Detection.FormattingEnabled = true;
this.lb_Detection.Location = new System.Drawing.Point(10, 322);
this.lb_Detection.Name = "lb_Detection";
this.lb_Detection.Size = new System.Drawing.Size(660, 251);
this.lb_Detection.TabIndex = 14;
Let's go on with the other
group box with of Highlights.
This part also
has got its Set button (in the code it has the name of HighlightSet to
differentiate it from the other Set button).
this.btn_HighlightSet.Location = new System.Drawing.Point(189, 129);
this.btn_HighlightSet.Name = "btn_HighlightSet";
this.btn_HighlightSet.Size = new System.Drawing.Size(58, 23);
this.btn_HighlightSet.TabIndex = 19;
this.btn_HighlightSet.Text = "Set";
this.btn_HighlightSet.UseVisualStyleBackColor = true;
this.btn_HighlightSet.Click += new System.EventHandler(this.btn_HighlightSet_Click);
parameters to provide.
this.tb_DrawThickness.Location = new System.Drawing.Point(138, 95);
this.tb_DrawThickness.Name = "tb_DrawThickness";
this.tb_DrawThickness.Size = new System.Drawing.Size(87, 20);
this.tb_DrawThickness.TabIndex = 17;
this.tb_Blue.Location = new System.Drawing.Point(183, 59);
this.tb_Blue.Name = "tb_Blue";
this.tb_Blue.Size = new System.Drawing.Size(42, 20);
this.tb_Blue.TabIndex = 16;
this.tb_Green.Location = new System.Drawing.Point(138, 59);
this.tb_Green.Name = "tb_Green";
this.tb_Green.Size = new System.Drawing.Size(42, 20);
this.tb_Green.TabIndex = 15;
this.tb_Red.Location = new System.Drawing.Point(93, 59);
this.tb_Red.Name = "tb_Red";
this.tb_Red.Size = new System.Drawing.Size(42, 20);
this.tb_Red.TabIndex = 14;
There is one
checkbox here to set whether we want to see the original image or not:
this.chk_ShowImage.AutoSize = true;
this.chk_ShowImage.CheckAlign = System.Drawing.ContentAlignment.MiddleRight;
this.chk_ShowImage.Location = new System.Drawing.Point(22, 25);
this.chk_ShowImage.Name = "chk_ShowImage";
this.chk_ShowImage.Size = new System.Drawing.Size(85, 17);
this.chk_ShowImage.TabIndex = 14;
this.chk_ShowImage.Text = "ShowImage:";
this.chk_ShowImage.UseVisualStyleBackColor = true;
At the end here
you can see the Main form of the GUI.
this.AutoScaleDimensions = new System.Drawing.SizeF(6F, 13F);
this.AutoScaleMode = System.Windows.Forms.AutoScaleMode.Font;
this.ClientSize = new System.Drawing.Size(947, 581);
this.FormBorderStyle = System.Windows.Forms.FormBorderStyle.FixedSingle;
this.MaximizeBox = false;
this.Name = "MainForm";
this.Text = "Line Detection";
this.Load += new System.EventHandler(this.MainForm_Load);
And we are done!
It was fun, wasn't it?
detecting is ready to be used! In this tutorial you could read some theoretic background
about implementing line detecting in C# and a detailed practical description about
this solution, including some code examples. The article also contains a
detailed description about implementing a useful user interface for a program
I hope this
brief tutorial was useful for you all. Good luck on developing your own