Camera Based Presentation Support System forMultiscreen Venue Using Laser Pointer
Daichi Shibata

Kenji Funahashi

Department of Computer Science
Nagoya Institute of Technology
Nagoya, Japand.shibata.946@stn.nitech.ac.jp

Department of Computer Science
Nagoya Institute of Technology
Nagoya, Japankenji@nitech.ac.jp

Abstract—During a presentation, a presenter usually uses alaser pointer to point to a particular location on a screen.
However, in a multiscreen venue displaying presentation slides,
the presenter points to only one screen, and the pointed spot
is not reflected to other screens that audiences looking at. We
propose a simple method that reflects a highlighting operation
with a laser pointer to other screens without specific devices. It
is expected that all audiences can understand the presentationeven in a large venue.
Index Terms—Presentation support, Laser pointer, Multiscreenpresentation venue

I. I NTRODUCTION
In recent years, opportunities of presentations with a PC
and a video projector at academic conferences and lectures
have increased. Presenters commonly use laser pointers toemphasize designated point. An audience looks at a projection
spot of the laser pointer on the screen, and tries to understandwhat they are talking about and where they are focusing
on. In other words, the slide emphasis information by thelaser pointer is a part of the information that is necessary tounderstand the presentation content.
By the way, some screens displaying presentation slides may
be provided additionally at the middle position in a venue
other than the front position in a venue (Fig. 1). They are
installed because it is difficult to recognize the front screen
visually for the audiences at a rear area in a large venue,
and the same image is output to all screens as shown in
Fig. 2. The presenter gives a slide emphasis on the frontscreen. Only the front audiences can see the screen installed
front, and other audiences cannot see the laser pointer. An
information that is necessary to understand the presentation islacking. Therefore, we propose a presentation support systemthat detects the laser pointer position on a front screen first,
and reflects the pointing information to other sub-screens.
However, it is difficult that this system becomes popular if
this needs other special devices, that are expensive. We tried
to realize novel simple system that needs only webcam, andits image processing application works on the same PC forthe presentation. The same pointing information can be shown
audiences watching an auxiliary screen, and it is expected thataudiences will be better able to understand the presentation
content.

Fig. 1. Multiscreen presentation venue (image is quoted from [1]).

WDĂŝŶƐĐƌĞĞŶ

sŝĚĞŽŽƵƚƉƵƚ

WƌĞƐĞŶƚĞƌ

･･･^ƵďƐĐƌĞĞŶϭ^ƵďƐĐƌĞĞŶϮ

Fig. 2. Hardware environment of presentation.

II. R ELATED D EVICES
A mouse for a PC can point exactly where presenter wants
to point on a slide. However, they should operate a computer
watching it carefully in that case. They probably want toconcentrate on the presentation and audiences understanding.
It should be considered that the operation with looking at thecomputer screen is troublesome and tempo of the presentation
becomes slow. There are some presentation pointers as digital
devices that can be operated more intuitively [2] [3]. They canoperate them while looking at a projected slide on a screen
without looking at a PC screen. Although the movement issimilar to some extent, it is impossible to point to the position
pointed by the arm strictly, since a gyro sensor is used toconvert 3D hand movement to 2D movement. In our method,an usual laser pointer is used to indicate the slide, and a single
webcam is used to detect it. It is possible to point where
they want to point with maintaining the operability of thepresentation pointer using a gyro sensor.
There are some examples of a research related to detectionof a projection spot of a laser pointer. Sugawara et al. used an
infrared light laser pointer with a visible light laser pointer andreceived only infrared light with a band-pass filter [4]. Wadaet al. used a modulated light laser pointer and an image sensor
with a demodulation function [5]. They also need special
hardware devices to detect the projection spot. We implement
our new system only using a webcam because we want toimplement the system simply.
III. I MPLEMENTATION
A. Detect a Spot of a Laser PointerWe describe how to detect a projection spot of a laser
pointer on a main screen. It is assumed that there is a sufficientdistance between a presenter and a main screen when a laser
pointer is used to indicate a specific point on a slide projectedon the screen. It means that any part of the slide is not hidden,
and the presenter is not appeared in the slide image captured
by a webcam.
One of the most simple methods is to compare a preceding
frame and a following frame of a captured video for eachpixel such as the reference [4]. If only a projection spot ofa laser pointer is moving in the image, it can be detected byusing this method. However, animation effects and videos areoften used in presentations. In this case, when this method
is applied, animation effects and videos are also detected asdifferences between two frames. It is difficult to detect only aprojection spot of a laser pointer. An image of a presentation
screen projected a slide taken by a webcam and image data of apresentation slide inside a computer are compared as shown inFig. 3. Then the position of a projection spot of a laser pointeris detected. We show an example image of a presentation slide
inside a computer in Fig. 4, and a presentation screen projecteda slide in Fig. 5. The screen image taken by a webcam isdistorted and the color is different from original data. In order
to compare these images, an image distortion and an ambient
light should be considered.
It is necessary to correctly find correlate areas that show thesame part between such different images. Template matching
and feature matching are basic methods to check a matchingof two images. Template matching is a method of sequentiallycomparing images on a pixel-by-pixel basis. Although the2TGUGPVGT

2% YKVJ %COGTC2TQLGEVQT%QORCTG

5NKFGFCVCKPUKFGCEQORWVGT#UNKFGKOCIGVCMGPD[CYGDECOFig. 3. How to detect a projection spot of a laser pointer.

degree of coincidence can be recognized to some degree ofprecision, the degree of similarity is greatly reduced onlyby the difference in size and subtle angle. Further, since
the entire area is scanned at any time, the processing timeincreases as the image size increases. On the other hand, infeature matching method, feature points are detected in theimages, and comparing similarities with using them. Comparison processing is faster and that is more robustness to shapechanges than template matching. Therefore, feature quantity
matching is used in our method. There are algorithms suchas Scale Invariant Feature Transform (SIFT) [6], Speeded Up
Robust Features (SURF) [7], KAZE [8], and AcceleratedKAZE (AKAZE) [9] as types of feature matching. SIFT
feature matching smooths an input image using a Gaussian
filter, and gradually increases the smoothing level to create aplurality of smoothed images. A feature amount is generated
from the vicinity of the pixel of interest for the plurality ofcreated images. SURF feature matching is a simplification ofthe SIFT feature matching process, that processing speed isfaster than SURF feature matching algorithm. KAZE feature
matching is an improvement over the drawbacks of SIFT
feature matching and SURF feature matching. Since SIFT
feature matching and SURF feature matching do not preserve
the object boundary by isotropically blurring the image witha Gaussian filter, noise is removed while keeping importantfeatures, so the accuracy is improved. AKAZE (AcceleratedKAZE) feature matching aims to speed up KAZE featurematching. It has the same high recognition accuracy as KAZE,
but it has significantly reduced the calculation time. Therefore,
we will use this AKAZE feature matching algorithm to detectfeature points.
Now we describe a flow of comparing slides in two images.
To find a region showing a presentation slide in a captured

Fig. 4. Example image of a presentation slide inside a computer.

Fig. 5. Example image of a presentation screen projected a slide.

Fig. 6. Feature points of two images.

image is separated into two steps, the step of detecting thefeature points and the step of matching the feature points inthe two images. First, feature points are detected from each
image that resolution is decreased. Then, each feature point arematched with similar one on the other image. AKAZE feature
matching algorithm is used to detect the feature points as
described above, and simple Brute-force matching algorithm[10] is also used for matching. It takes the descriptor of one
feature in first set and is matched with all other features insecond set using some distance calculation. Then the closest
one is returned as matching feature point. An example resultof matching that calculated feature amount is shown in Fig. 6.
Feature points of two images that are simulated as same points
are connected by a straight line. Since some of them connect
the wrong points, only the higher-ranking point are used. Aprojective transformation matrix that matches two images iscreated based on the corresponding points obtained.
Here we describe a flow of detection a projection spot of alaser pointer. By performing the projective transformation of
the captured image using this transformation matrix, the slide

image in the captured image can be compared with the original
slide data inside a computer. We calculate differences between
two slide images that resolutions are decreased. Before obtaining differences, noises are removed by a median filter.
Then we create RGB and HSV channel images from eachslide images. We obtain differences from each channel images
for each pixels, and merge them by either-or operation perpixel. The threshold when obtaining the difference is decided
from pre-experience empirically. We ignore the pixels in avery small area of difference image. By these operations, theposition of the laser spot on the slide plane can be obtained as adifference area between them. We experimented the detection
flow mentioned above. Experiment environment is shown in
Fig. 7. We use liquid crystal display so far.
B. Emphasize a Projection SpotWe describe how to display a digital cursor on the screen.
A digital cursor should be displayed on each screen so thateveryone can see where the laser pointer is pointed. It takes
time and effort to create presentation software from scratch.

Fig. 9. Examples of a digital cursor.

Fig. 7. Experiment environment.

Fig. 10. Example image of a digital cursor superimposed on the slide.

Fig. 8. Example of Layered window (image is quoted from [12]).

And users would like to use a software they are used to as apresentation support system. Therefore, we implement display
digital cursor function by superimposing it over an existing
software screen, separately from the presentation software.
We use Layered window [11] to superimpose the digitalcursor that is prepared as Windows API. Layered window
is a window that colors appear to be mixed by performing
alpha blending with the window located below itself. In otherwords, we can create transparent window like Fig. 8. We use
CreateWindowEx function [13] and SetLayeredWindowAttribute function [14]. By using these functions, we can create
completely transparent window, and a digital cursor can bedisplayed in this window to realize superimposing the digitalcursor. By displaying a layered window in the foregroundof the screen and keeping it inactive, a layered window canbe used without disturbing the operations during a normalpresentation. So it is possible to operate the system without
depending on the presentation software running behind thescenes.
When the digital cursor is superimposed on a slide digitally,

it is also projected on a main screen. It may disturb to detecta projection spot. So the digital cursor as shown in Fig. 9is used. It surrounds the actual spot of a laser pointer, anddoes not disturb to detect it. Fig. 10 is example image of adigital cursor superimposed on the slide. The area except thered cursor line is transparent so the area is not displayed on
the screen. Since the digital cursor is prepared and loaded as
an image, it can be freely changed before loading that is easy
for the audience to see.
IV. C ONCLUSION AND F UTURE W ORKS
We proposed a presentation support system that detects aspot of a laser pointer on a main screen, and displays a digitalcursor on each screen at a multiscreen venue. The audience
who cannot see the main screen in front of the venue also canunderstand the presentation as well.
However, there are some tasks to improve the system. The
future improvement of the system is to reduce the time taken infeature matching process. In the Brute-force matching method,
one feature point of one image is compared with all featurepoints of the another image. So it takes time about 100 ms tocompare slides in average. Therefore, by using other matchingalgorithms with a smaller amount of calculation, it is expectedthat the calculation time can be shortened probably. There isalso a problem of detection accuracy. In our system, featurematching, especially the AKAZE algorithm, was used to detectfeature points from two images. However, since the gradient
information of the brightness in the image is used as thefeature in the feature matching, suitable parameters should beconsidered to match features when various external devices
such as a webcam and a projector are used. Therefore, weshould compare some matching methods in the future. Theexperiment system is still under construction, we would liketo complete a practical system.
ACKNOWLEDGMENT
This work was supported in part by JSPS KAKENHI Grant
Number JP20K11918.
R EFERENCES[1] sola city Conference Center. https://solacity.jp/ (accessed 26 Feb 2020).
[2] Kokuyo. AIRBEAM. https://www.kokuyo-st.co.jp/stationery/pcp/(accessed 22 Jun 2020).
[3] Logicool. Spotlight Presentation Remote. https://www.logicool.co.jp/jajp/product/spotlight-presentation-remote/ (accessed 22 Jun 2020).
[4] Toshiharu Wada, Masanobu Takahashi, Keiichiro Kagawa and Jun Ohta,“Method to Realize Mouse Functions Using Laser Pointer”, The journalof the Institute of Image Information and Television Engineers, Vol.63,No.5, pp.657-664, 2009.
[5] Shinji Sugawara and Tetsuya Miki, “A Remote Lecture System with
Laser Pointer for the Internet and Broadband Networks”, Bulletin of theUniversity of Electro-Communications, Vol.16, No.2, pp.117-123, 2004.
[6] David G. Lowe, “Distinctive image features from scale-invariant keypoints”, In Proc. IJCV2004, pp.91-110, 2004.
[7] Herbert Bay, Tinne Tuytelaars, and Luc V. Gool, “SURF: Speeded Up
Robust Features”, In Proc. ECCV2006, pp.404-417, 2006.
[8] Pablo F. Alcantarilla, Adrien Batroli and Andrew J. Davison, “KAZEfeatures”, In Proc. ECCV2012, pp.214-227, 2012.
[9] Pablo F. Alcantarilla, Jesus Nuevo and Adrien Bartoli, “Fast Explicit
Diffusion for Accelerated Features in Nonlinear Scale Spaces”, In Proc.
BMVC2013, pp.13.1-13.11, 2013.
[10] OpenCV-Python Tutorials. Feature Matching. https://opencv-pythontutroals.readthedocs.io/en/latest/py tutorials/py feature2d/py matcher/py matcher.html (accessed 4 Aug 2020).
[11] Windows technical documentation. Layered Windows. https://docs.
microsoft.com/en-us/windows/win32/winmsg/window-features#layeredwindows (accessed 4 Aug 2020).
[12] Microsoft Docs. Windows with C++ - Layered Windows withDirect2D.
https://docs.microsoft.com/en-us/archive/msdn-magazine/
2009/december/windows-with-c-layered-windows-with-direct2d(accessed 11 Aug 2020).
[13] Windows technical documentation. CreateWindowExA function.
https://docs.microsoft.com/en-us/windows/win32/api/winuser/nfwinuser-createwindowexa (accessed 4 Aug 2020).
[14] Windows technical documentation. SetLayeredWindowAttributes
function. https://docs.microsoft.com/en-us/windows/win32/api/winuser/
nf-winuser-setlayeredwindowattributes (accessed 4 Aug 2020).