I. INTRODUCTION
We conduct research mainly on assistance for safe driving as an area of Intelligent Transport Ssystem(ITS). On-vehicle cameras play an important role in capturing the information of external environment in many driver assistance systems. Some studies have been conducted for detecting traffic signs, and signal lights so on[1][2][3][4][8][9]. In this study, we propose a road-to-vehicle Visible Light Communica- tion(VLC) system using an on-vehicle high-speed camera as a receiver and LED traffic light as a transmitter. VLC is a wireless communication method using lumi- nance, transmitting data by emitting light. It is able to trans- fer data by emitting light source, and able to receive them by light sensor. There are several advantages in this commu- nication method compared to other wireless communication methods, such as radio waves and infra-red light. The visible light is not harmful to human body, and it is able to transmit with high power. There are not legal limitations for any existing light source, such as room illuminations and displays to be used. It can be used at the places where radio waves cannot be used, for example hospitals and areas around precision machines. Komine and Nakagawa[5] have achieved VLC using illumination light. It is a communication between PCs and illumination light, and considered as an alternative method for the wireless LAN. As a latest application of VLC in ITS, Suzuki et al. [6] introduced a support system for visually impaired person by utilizing VLC technology at signalized intersections. High-speed cameras are getting popular and they are also applied in ITS as well as in other scientific research, military test, and industry. High speed cameras can capture more information of fast moving objects and changing objects in high frequency, compared to the normal cameras. Specific image processing technique are also necessary when these cameras are used. In the proposed VLC system, the LEDs in the transmitter emit light in 500Hz and the images, which include those emitting LEDs are captured by the on-vehicle high-speed camera in 1000fps, while the vehicle is moving. The luminance value of LEDs in consecutive frames should be captured by processing the images from the camera for conducting communication. For this purpose, first the transmitter should be found, then it should be tracked in consecutive frames. In this paper, new effective methods are proposed for finding and tracking the transmitter concerning the above mentioned abilities of high-speed cameras. With the our previous finding and tracking methods[4][8][9], part of the LEDs were used for tracking and part of the emission used for finding, so not all LEDs could be used for continu- ous communication. With our proposed methods, continuous emission of all LEDs can be used for communication result- ing in a higher data rate. According to experiments conducted under different conditions, the proposals were effective for desired finding and tracking. This paper consists of six main sections to explain our project work. Proposed VLC system is explained in section II. Section III and IV explain the previous methods and new proposed methods for finding and tracking respectively. The experimental results are described in section V. The section VI concludes the paper.
II.  Proposed VLC system
Figure 1 shows the structure of the proposed VLC system using LED traffic light as a transmitter and high-speed camera as a receiver. If the LEDs in the transmitter could be recognized individually, it is possible to use each of them as a separate transmitter communicating in parallel at the same time. In other words, each LED transmits different data in parallel and they are received at the same time. As a result we can dramatically increase the communication speed by modulating each LED individually. Moreover, we can communicate with several transmitters and receive different information in parallel. These are the main advantage of using a camera as a receiver(Fig. 2). Another issue is the modulation method. Since this is a unique communication method using visible light and image, it requires particular modulation method which considers the characteristics of the communication. We use hierarchical coding[7] for Visible Light Communication, which modulates data on spatial fre- quency and enables long distance communication. The transmitter used for the experiments is square in shape as shown in Fig. 3 and it consists 16x16 LEDs. Figure 2 shows the high-speed camera which is used as the receiver in the experiments.
III.  PREVIOUS METHODS FOR TRANSMITTER FINDING
AND TRACKING
Figure 4 illustrates the main flow for finding and tracking the transmitter. Here, first the transmitter is found, then cut out image area( I cut ) including the transmitter from the original image as shown in Fig. 5. The cut out image area is indicated using a white square in the Fig. 5. The found transmitter is tracked detecting it only searching the corresponding image area to this cut out image area in consecutive frames. This tracking is conducted until it appear on this image area. If the transmitter stayed out from this image area, process return to the transmitter finding step. Figure 6 illustrates the emission patterns of the transmitter in our previous work. The LEDs in the transmitter are emitted sequence by sequence and in the first part of the sequence, all LEDs are blinked(ON and OFF) at the same time in 500Hz. This stage is set for finding the transmitter by processing the images from the high-speed camera. In the second part of the sequence, LEDs in the transmitter are emitted for different levels in 500Hz, except LEDs in the two exterior lines, and communication is conducted in this stage. In our previous work, these non-emitting(keep lighting without emitting) two exterior lines are kept to make it easy to track the found transmitter.
Previous transmitter finding
A.
While the vehicle is moving, the receiver (High-speed camera) installed in the vehicle takes images of the road in 1000fps. If transmitter exists on the road and it is at the first part of the sequence, it is expected to appear lighting transmitter on images once in two frames, since traffic light emits (ON and OFF) in 500Hz and high-speed camera takes images in 1000fps. Figure 7 shows the appearance of the transmitter in four consecutive frames at the first part of the sequence. Iwasaki et al. [4] proposed a method for finding the transmitter using this feature, by subtracting two consecutive frames.
B. Previous transmitter tracking
Previously, we proposed a method[8][9] for tracking the found transmitter in consecutive frames using the edge information. This method is very effective, if the non- emitting(keep lighting) two exterior lines are available. Fig- ure 8 illustrates appearance of the traffic light in four frames with non-emitting two LED lines. This method is explained in more details in section IV.B.1.
IV.  NEW METHODS FOR TRANSMITTER FINDING AND
TRACKING
In this paper, we removed the blinking part from the sequence as shown in Fig.10 and new method was pro- posed to find the transmitter only using randomly emit- ting LEDs as detailed in the section IVA. The previous tracking method[8][9] was improved to be able to track the transmitter, without keeping non-emitting(keep lighting) two exterior LED lines. The new method can track the transmitter when all LEDs in the transmitter emit light randomly. This new tracking method is explained in section IV.B. Due to above improvements, it is not necessary to have two parts in the structure of the emission sequence(Fig. 6). And the all LEDs in the transmitter cannot be used for communication and the continuous communication cannot be conducted with our previous methods[4][8][9]. All LEDs can be used for communication and continuous communication can be conducted with new proposed methods. For this reason, communication speed can be increased when new methods are applied. A. Proposed transmitter finding method The new transmitter finding method is detailed in this section. 1) Generation of differences image: In this method, dif- ferences between current frame and some just previous consecutive frames are calculated following the Equation 1.
In the above equation D t represent the current frame and m represent the number of previous frames used for the differences calculation. The total differences( Dtotal ) are projected to the new image. In this paper, this image is called as differences image. The transmitter area get highlighted there, since differences are comparatively higher than other areas due to the random emission of the transmitter. Figure 11 shows an arbitrarily example of generated differences image. 2) Binarization and morphology operation: This differ- ences image( Dtotal ) is processed for binarization using ap- propriate threshold to get the transmitter area with white pixels as shown in Fig 12. The morphological operation is conducted to the binarized difference image to connect the non-connected white pixels belong to transmitter area. In this process, first image is dilated two times and eroded it two times. There are some noise appeared in the image after these processing as shown in Fig. 13. In addition, some objects with white pixels belong to the noise have sizes as similar as transmitter in some cases. For this reason, first the transmitter candidates are selected, and then transmitter is extracted from the candidates. 3) Transmitter area candidate selection : The candidates are selected considering the geometrical information of trans- mitter. In this study, we plan to conduct communication distance between 70m and 20m, far from the transmitter. When the vehicle move this distance the side length of the transmitter has maximum length( l max ) and minimum length( l min ) on the image. l max : Vehicle is almost 20m far from the transmitter, l min : Vehicle is almost 70m far from the transmitter. The transmitter candidates are selected following below conditions calculating the circumscribing rectangle of each object.
C w and C h mean the width and height of the circumscribing rectangle belong to the searching object. Here, the object which take square circumscribing rectangle are selected as candidate by setting the aspect ratio since the desired transmitter is square in shape. In some cases, there are few candidates appeared as shown in Fig. 13 and single candidate appeared in many cases. So, it is necessary to confirm the transmitter. 4) Confirmation of the transmitter: This confirmation is conducted by searching the differences density( Dden) and the average intensity( Lavg ) in the corresponding candidates areas in the difference image. Figure 14 shows corresponding differences image for Fig. 13. D den and L avg are calculated following the Equation 2 and 3 respectively.
D pix and T pix in the above equations represent the number of differences pixels(pixel value>20) and total number of pixels inside the candidate respectively. In addition, the L piz ; represents the intensity value of each pixel inside the candidate. The candidate which are under below condition is confirmed as transmitter.
Ddn × Lavg > threshold
In the Fig. 14, the transmitter confirmation result is also in- dicated using cut out image area( I cut ) with white square. The differences density( Dden ) and the average intensity( Lavg ) values inside the transmitter area take stronger values than other objects, since differences pixels takes higher intensity value due to accumulation of m − 1 differences(differences between current frame and just previous m consecutive frames) belongs to the randomly emitted transmitter.
B. Proposed transmitter tracking method
After finding the transmitter, it is tracked only searching the corresponding cut out image area( Icut ) in consecutive frames. In this paper, at the emission sequence, all LEDs emit light randomly(Fig. 10). But, when all the LEDs emit light randomly, it is difficult to track transmitter, because, the bounding of the transmitter is not appear properly(see Fig. 9). We propose a new method for this tracking and it includes two main steps as edge-based tracking and optical flow-based tracking. 1) Edge-based tracking :  In this method[8][9], first edge detection is conducted, then the edge component whose have circumscribing rectangles as nearly similar size as just previously detected transmitter are selected as transmitter candidates. After that the transmitter is extracted from the candidates using defined transmitter likelihood conditions and transmitter likelihood probability. According to so far experiment results, it was possible to detect the transmitter in consecutive frames with at least 20% of detection rate when all LEDs emit light randomly. The reason for this is that, the above method detect transmitter considering circumscribing rectangle of the edge component. In the case of that the at least four LEDs in the each bounding LED lines of the transmitter light and those are connected each other through the other lighting LEDs, it is possible to gain the appropriate circumscribing rectangle of the edge component which belongs to the transmitter. 2) Optical flow-based tracking :  If the above edge-based method could not detect the transmitter  the movement of the transmitter in consecutive frames are measured by using the optical flow as a second step  In this paper, we use Lucas-Kanade ' s optical flow method[10]. Here, if edge- based method does not detect the transmitter, it is tracked on optical flow until is detected by edge-based method again. This optical flow-based detection is explained below. If the transmitter could not be detected in the edge-based step, preprocess the previous frame(called as base frame)of the current frame using a Gaussian low pass filter. The deforma- tion matrix( D ) is calculated using directional derivatives at each pixel in the base frame following Equation (4).
where, for example, d xx represents the 2nd order derivative in the x direction The eigenvalues of the deformation matrix( D ) are calcu- lated, and perform non-maxima suppression. In this work, we assume that local minima exist in the 5 × 5 neighborhood. Select the best N number of feature points using the eigen- values in order of priority whose do not exist in approximate transmitter area. Here, the feature points in the transmitter area is not considered. The image area( I f ea ) used for this feature point selection is indicated following Equation (5). In the above Equation (5), Itra is the just previously detected transmitter area and Ioffset is set up, since trans- mitter can have a movement in consecutive frames. After selecting the feature points in the base frame, gain the corresponding set of feature points in the current frame. Each feature point movement between current and base frame are determined. The movement of transmitter is determined using the movement of feature points. Here, the maximum frequency of the feature point movement(distance( F P dis and direction( FP dir )) between current frame and base frame is selected as the movement of transmitter. If the corresponding feature points in current frame became less than 0.7 N points, feature points are renewed. As mentioned above, the transmitter is tracked on optical flow until it is detected by edge-based method again. Figure 15 illustrates the main steps of transmitter detection by optical flow. In the above algorithm, the feature points which are in approximate transmitter area are not used. We consider the movement of the transmitter as same as the movement of the N best feature points on the background since error can be occurred in calculating feature points on the transmitter area due to its emission. While the camera is moving in a horizontal direction and if the objects are far from the camera, the affection on angular motion of the camera is more than the one on horizontal direction. As a result, the movement of the transmitter take similar movement as background. The size of the transmitter cannot be updated while it is tracked by optical flow and the size updating is conducted when the transmitter is tracked by edge-based method only. The conducted experiment to confirm the effectiveness of proposals are explained in the next section.
V.  EXPERIMENTAL RESULTS
In the experiments, we fixed the high-speed camera on a vehicle and images were captured while driving between 30km/h〜40km/h, towards the transmitter. The moved dis- tance of the vehicle is from 70m to 20m, from the transmitter. Transmitter is emitted light in 500Hz and grayscale images of emitting transmitter are captured by high-speed camera in 1000fps with size of 1024×512pixels. The cut out image area was 128×128pixels. Table 1 illustrates transmitter finding results. Each se- quence in the Table 1 includes approximately 6000frames. These sequences were captured under sunny, cloudy, and very dark(night) conditions. In the experiments, the number of previous frames used for the differences calculation( m ) was set as five. This value was decided by conducting a sub- experiment. In this method, transmitter is found in every five frames. The expected findings means number of maximum findings can be happened, when transmitter is found in five frames by five frames. Success rate shows successfully found per- centage out of expected findings. No false findings showed in any experiment. Figure 16(a)~(j) show some tracking results in consec- utive frames. In the Fig. 16 the red squares indicate the tracked transmitter by edge-based method and yellow squares indicate the tracked transmitter by optical flow. The red dashes indicate the movement of the feature points. In between Fig.16(i) and (j) it does not show any dashes, meaning that there is not a movement between these two frames. Figure 16(a), (e), (h), and(i) are some example for detecting the transmitter by edge-based method. As shown in Fig. 16 (b)~(d), the transmitter is tracked in consecutive frames on optical flow. According to the experiments so far, the transmitter could be tracked for 100 consecutive frames with high accuracy only on optical flow. But, it is not necessary to track for 100 consecutive frames, transmitter is tracked by edge-based method also within 100frames, since edge-based method can also detect it at least 20% when all LEDs emit light randomly.
VI.  CONCLUSION
The new transmitter finding and tracking methods were proposed for a road-to-vehicle Visible Light Communication system. All LEDs in the transmitter can be used for achieving continuous communication with new proposals, which was not possible before. Experimental results showed that the new proposals are very effective in desired finding and tracking.

ACKNOWLEDGMENT
This work was partially supported by the Ministry of Internal Affairs and Communications, Strategic Information and Communications RD Promotion Programme(SCOPE)
R EFERENCES
[1] K. Mori, T. Takahashi, I. Ide, H. Murase, T. Miyahara, and Y. Tamatsu, Recognition of foggy conditions by in-vehivle camera and millimeter wave radar, Proc. of IEEE Intelligent Vehicle Sym. , Turkey, 2007, pp. 87-92 [2] G. Y. Song, K. Y. Lee, and J. W. Lee, Vehicle detection by edge- based candidate generation and appearance-based classification, Proc. of IEEE Intelligent Vehicle Sym. , Netherlands, 2008, pp. 428-433. [3] Y. Shen, U. Ozguner, K. Redmill, and J. Liu, A Robust Video based Traffic Light Detection Algorithm for Intelligent Vehicles, Proc. of IEEE Intelligent Vehicle Sym. , China, 2009, pp. 521-526. [4] S. Iwasaki, H. C. N. Premachandra, T. Yendo, T. Fujii, M. Tnimoto, Y. Kimura, Visible Light Raod-to-Vehicle Communication Using High- Speed Camera, Proc. of IEEE Intelligent Vehicle Sym. , Netherlands, 2008, pp. 13-18. [5] T. Komine and M. Nakagawa, Fundement analysis for visible-light communication system using LED lights, IEEE Trans. of Consumer Electronics , Vol. 150, pp.100-107, 2004. [6] K. Suzuki, M. Fujita, Y. Hayashi, K. Fukuzono, A Study on Visually Impaired Person's Support System Utilizing Visible Light Communi- cation Technology at Signalized Intersections, Intern. Journ. of ITS Research , Vol. 5, pp.37-45, 2007 [7] S. Arai, S. Mase, T. Yamazato, T. Endo, T. Fujii, M. Tanimoto, K. Kidono, Y. Kimura and Y. Ninomiya, Experiment on Hierarchical Transmission Scheme for Visible Light Communication using LED Traffic Light and High-Speed Camera, Proc. of IEEE Intertern. Symp. on Wireless Vehicular Communica. (WiVeC' 07), 2007. [8] H. C. N. Premachandra, T. Yendo, T. Yamasato, T. Fujii, M. Tanimoto, and Y. Kimura, Detection of LED Traffic Light by Image Processing for Visible Light Communication System, Proc. of IEEE Intelligent Vehicle Sym. , China, 2009, pp. 179-184. [9] H. C. N. Premachandra, T. Yendo, M. P. Tehrani, T. Yamasato, T. Fujii, M. Tanimoto, and Y. Kimura, Edge-Based Tracking of an LED Traffic Light for a Road-to-Vehicle Visible Light Communication System, Journal of Broadcast Engineering published by KoSBE, the Korean Society for Broadcast Engineers , Vol. 14, pp.475-487, 2009. [10] D. Bruce, D. Lucas, and T. Kanade, "An Iterative Image Registration Technique with an Application to Stereo Vision", iProc. DARPA Image Understanding Workshop , pp. 121-130, 1981.