Estimation of exposure and data reading sequences on CMOS video
camera processing
Kazuhisa Miyashita(Jan. 3, 2020)

1 Introduction.
Astronomical CMOS cameras with highly sensitive CMOS image sensors, such as back-illuminated
CMOS cameras, are now available and have been used by amateur observers to observe the sueface of theMoon and planets. Since these cameras can take videos, the possibility of their application to the observationof stellar occultatioin has been discussed amongst observers. In particular, in the observation of stellar
occultations by asteroids, the target objects are often faint, and the analogue high-sensitivity cameras used in
the past required long frame integrate exposures of one to several seconds. High temporal resolution isrequired to obtain more useful data for observing the shape of asteroids and the companions of double stars,
and there are growing expectations for high-sensitivity CMOS cameras to enable observation of occultationof fainter stars. Occultation observation is a form of observation in which the position of a celestial body on
the celestial sphere is revealed by measuring its exact time. Nowadays, the accuracy of a few milliseconds orless is required as an observation value, and for this purpose, highly accurate time must be recorded in thevideo image. However, CMOS cameras have not yet developed a time inserter like analogue video. For thisreason, observers are adopting a method of obtaining precise time based on the LED light that blinks
according to the 1PPS signal generated by a GPS-based clock device, which is captured together with theoccultation event. This study attempts to clarify the temporal timing of the exposure and recording of a
CMOS camera by measuring the brightness of the 1PPS LED light recorded by the image sensor by reading
it from the video frame. The method and results are described below.
2 Recording mechanisms and issues with astronomical
CMOS cameras
Conventional analogue video cameras for astronomical applications
output composite signals, which are superimposed by a time inserter,
captured by a PC and recorded as a file. At this time, the standards for thesignals output by the camera were strictly defined as NTSC or PAL
systems, and the timing of exposure and output was assured by thecamera as a quality industrial product. Therefore, once the timing of thecamera's exposure and time recording of the time inserter (for each of themultiple shooting modes, if the camera had more than one) was
measured, all users of the camera could use the same correction values.
On the other hand, image data from astronomical CMOS video
cameras are written directly by the PC as a file via USB (2.0, 3.0), a
communication standard between PC peripherals. The time required forthe sequence of exposure, data readout, communication transmission,
processing commands by software and writing to disk is extremely likely
to be different in each environment used by the observer, and it isgenerally difficult to measure the time required for each of theseprocesses.
To solve this problem and to provide accurate time for the observed
video, the simultaneous recording of the 1 PPS LED light generated by
the GPS receiver as field illumination during the observation is being
adopted. 3 and can be used as a precise time reference. This time keeping
method was used in analogue video cameras before the advent of timeinserters, but since that time there has been a problem: if it is not known
where in the 33 msec exposure period of NTSC, for example, the lightstarted to emit, the 33 msec will be used as a time error.

Fig. 1 1PPS audio signal(red) to 'go 1 msec into'
everysecond for NTSC
frames (light blue).

3. Method for estimating the start of the LED glow in the frame exposure

Fig. 2 How 1 PPS is shifted for each frame of NTSC video
For ease of explanation, the amount of misalignment is depicted as large. The actual amount of misalignment isapproximately 1/33rd of the frame length, as shown in Figure 1.

For the above issue, the author devised a method to determine the time from simultaneously recorded time
signals, utilising the fact that the NTSC frame rate is not an integer (29.97 fps) and that the screen and 1 PPS
are shifted by approximately 1 msec when 30 frames are advanced (Miyashita, 2006). This method has also
been applied to time analysis of videos of stellar occultation by asteroids taken with an SLR camera (Fig. 1).
The mechanism is shown in Fig. 2. In this case, the light from the LEDs on the screen can be measured in thesame way as for normal celestial objects. In the NTSC case described above, the starting point of the 1PPS
signal increases by 1 msec for each frame every 30 frames, so that the 'stored amount' of LED light can beseen to increase in proportion to time. As time progresses further, the flashing light of the 1PPS signal istaken out of the frame, and at that stage the light intensity decreases in proportion to time. Therefore, when
LED light is measured using this method, the change in light intensity should be linear. Whether or not thelight changes linearly during measuring also makes it possible to detect whether the camera output isgamma-corrected, etc., and to detect fluctuations in the time it takes to record a frame into a file.
4. Investigation of camera characteristics

During observations, it is desirable toilluminate the LED light with dark-field
illumination in order to avoid affecting the partof the star to be observed. In order to find out
what position should be illuminated at that time,a basic experiment was conducted as shown in
Fig. 3.
As some conventional analogue and CMOS
camera models use a global shutter system,
there is no difference in recording time
Fig. 3 The measurement position on the screen
according to the position on the frame image.
A camera with a lens was placed on the time inserter andOn the other hand, CMOS cameras such as the unfocused so that a large LED light appeared in the image.
Photometry was carried out using the 1PPS LED light
ASI290MM, which are beginning to be used for
detector of Limovie ver. 0.9.99.4B (right). In this case, it
occultation observation in Japan, read thewas examined whether the timing of the start of the LED
optical signals received by the image sensor one light was different in the vertical and horizontal directions.
line at a time, so the start and end timing ofrecording is said to be different between the upper and lower parts of the screen. Therefore,changes in the amount of LED light with respect to time were investigated for each of the verticaland horizontal positions, as shown in Figure 3 (right).
SharpCap * developed by Robin Glover was used as the software for the capture. This software iscapable of writing the time of day of each frame. The data can then be read out by occultation

movie analysis software such as Limovie for use in time observations. However, the time written by
SharpCap is considered to be the time of the PC system clock at the stage when the software reads
the image data into the PC memory, and may indicate a later time than the time of exposure of theimage sensor. The following experiment was therefore carried out with the aim of obtaining acorrection value for the recorded time timing (delay time) by comparing the time when the 1PPS
LED light started to shine with the time when it was written to the frame.
In this experiment, the exposure time was set to 13 msec. This gives 13 msec x 77 frames = 1.001
sec, and if photometry is carried out every 77 frames, it is possible to observe how the exposure of 1
PPS and frame is shifted by 1 msec.
5. measurement results
5-1 Results at one measuring point

Fig. 4 Photometry (sum of pixel values)
within the magenta frame of the image

Fig. 5 Finding the timing of 1 PPS in a
frame

The measurement results are shown in Figures 4 and 5. The increase and decrease in light intensity
showed a linear change. This result cannot be achieved unless the recording frame rate is stable and bothlight intensity and output have a linear relationship, which confirms that the CMOS camera has suitablecharacteristics for occultation observations. In Fig. 5, the position of the rise from the noise level of thephotometric value indicates a UTC second of 1 PPS. This position can be determined as the point where thevalue at the time of light intensification is approximated to a straight line using the least-squares method andintersects the straight line averaged over the noise level. Since the interval between the measurement points
in the horizontal axis direction in Fig. 5 is 1 msec, this method can be considered to provide a time referencewith sub-millisecond accuracy for the observation.
5-2 Frame and PPS timing in the horizontal line
Figure 6 shows the results measured for thehorizontal alignment of the measurement shops in
Figure 3. The six measurement points on the left-handside are far apart, but these are areas where the LED
light is absent or extremely faint. If the brightness of thescreen is barely perceptible, this indicates that the 1 PPS
information may not be read correctly.
On the other hand, if the brightness is visible on thescreen, it does not depend on the brightness very much.
Also, regardless of whether the light is saturated or not.
No difference is found in the PPS seconds obtained
regardless of whether the light is saturated or not. The
Fig. 6 Horizontal line - LED timings
variation of the time when the 1PPS starts to shine in
The vertical axis represents the timing for 1 PPS
the brightest part of the LED light from the 10th from framing, in units of frame exposure length.
the left to the 2nd from the right is 1σ = 0.004 msec,

which is much smaller than 1 msec. From the above, it is considered that this method of finding the timing of
the 1PPS signal for a frame can provide a very high time accuracy as a time reference for time recording
during observation.
5-3 Frame and PPS timing in the vertical screen orientation
The timing results for the screen vertical direction
are shown in Figure 7. It shows that, unlike thehorizontal direction, the frames lag behind withrespect to 1 PPS towards the bottom of the screen.
Under these shooting conditions (ASI 290MM,
binning 2), there is a delay of 0.0077 msec per line.
This means that this screen, which consists of 540
pixels (lines), has an exposure timing difference of
4.16 msec at the top and bottom. Therefore, when
the object to be observed is placed in the centre of
the screen and the dark-field illumination of the
LED is shone on the bottom part of the screen, thepositive second at the position of the object isdelayed by 2 msec compared to the positive second
indicated by the LED, and the LED light as anextremely accurate time reference as described in
5-2 cannot be utilised.
Fig.6 Flash start timing of 1PPS LED in the verticalIn addition, while the timing in the horizontal
screen orientation
direction of the screen matched with high precision,
the deviation from the straight line in the verticaldirection was relatively large. This may mean that there is some degree of fluctuation (variation) in thereadout speed, but no clear conclusion can be drawn from these results alone.
6. Sequence of camera's processing
The following characteristics of the operation of ZWO's ASI 290MM astronomical CMOS camera whenconnected to a PC via USB 3.0 were identified.
1) The frame rate is stable.
2) No gamma correction or other corrections are made during output. The light level is proportional to the
output value.
3) For the horizontal direction of the screen, the timing of frames and 1 PPS is constant everywhere. This
indicates that one line is read instantaneously (0.0077 msec).
4) One line of the image sensor reads the next line at a time interval of 0.0077 msec. Therefore, if the heightof the screen is 540 pixels, there will be a time difference of 0.00416 seconds (4.16 msec) between the top
and bottom of the screen at the start and end of recording.
The sequence of operation of ZWO's ASI 290MM can be summarised from the above as shown in Fig. 8.
According to this, when recording with 4 msec set as the exposure time, image data with a time
discrepancy of exactly one frame is obtained at the top and bottom edges of the screen.
Also, when recording using a time difference between frames using 1 PPS LED light,5) A time difference of 1 msec per second can be created between a frame and a 1 PPS signal, as shown in thetable at the end of this paper. (e.g. 13msec × 77frame = 1.001sec, 11msec × 99frame = 1.001sec)
6) If photography and photometry are carried out in such a way that the LED 1PPS and frames are shifted by
1msec (e.g. photometry every 77 frames of a 13msec exposure), it is possible to capture the increase in
LED light in proportion to time. If a straight line is fitted to this using the least-squares method, theposition of the positive second can be determined at its rise.

Fig. 8 Sequence of exposure and data reading estimated from measurement results

7. Instruments and methods for obtaining accurate time
From paragraphs 3) and 4) of section 6, for precise time observations, the target object and the LED light of the 1PPPS
must have the same vertical position on the screen. Therefore, the LED light must not be point-like as used in analogue
video cameras, but must shine long enough in the vertical direction of the screen to correspond to any position of thetarget celestial body. As mentioned above, it is acceptable to have a certain degree of 'shading' of the light depending on
the location, as long as it is not extremely dark.
To satisfy this condition, Hiroyuki Watanabe developed a prototype device as shown in Fig. 9. In this, the prism of
the off-axis guide adaptor is rotated 180° to illuminate either the left or right edge of the image sensor with LED lightattached instead of a guider. As shown in the figure, the right side of the screen is illuminated from top to bottom.

Fig. 9 LED light illuminated in dark field (right), using a PPS projection unit (left, middle).
The device was conceived and built by Hiroyuki Watanabe. The off-axis guider is modified to illuminate LED lightto the right edge of the screen.
During analysis, Limovie's 1PPS signal receiving area is placed in the same vertical position as the target object andcan be used as a reference for obtaining precise time data by utilising the 0.001 second per second shift between theframe and 1PPS.

The device also allows the user to visually verify the above-estimated sequence: by following the samebrightness from frame No. 7 or No. 98 in the image to the right, one can see how the top edge is reached in
the fourth or fifth image.

Fig. 10 LED brightness change every 1 msec
Image every 91 frames at an exposure of 11 msec. Numbers represent frame numbers.

9. conclusions and future tasks
By taking video of the occultation phenomenon while irradiating 1 PPS LED light from top to bottom oneither side of the image sensor with a fixed width during the observation, and by measuring the brightness of
the irradiated light at fixed frame intervals, the time when the frame exposure was made can be determinedwith a precision higher than 1 msec. At this time, the internal clock of the PC must be set to an error of +/-0.3
s or less by an appropriate method, as it is said that the error of the PC's internal clock can sometimes reach
0..5 s. At present, it is relatively easy to obtain a precise time reference by the method described above if the
CSV file output by Limovie is processed using spreadsheet software, but it is complicated for observers to do
this for each observation, and if there are errors in the calculation method, etc., they may also become asource of error. It is necessary to have a system to support this on the analysis software side, and inparticular, it is essential to make the software easy and reliable to use. The creation of new functions for
Limovie, reflecting the opinions of observers, will be important for the effective use of this method forobservations.
This study was carried out on one of the author's own astronomical CMOS cameras; a wide variety of
CMOS camera models are available from several companies. We will continue to analyse whether thecharacteristics of the operation sequence revealed in this study are the same for other models.
*Note: The function described above has been implemented in Limovie as "SharpCap
timing analysis”. This feature eliminates the need for calculations in CSV and allows timecorrection work to be performed as part of Limovie's analysis. See appendix 3. (Jan. 23,
2021)

Acknowledgement
Hiroyuki Watanabe has been an early practitioner of the application of CMOS cameras to occultation observations andhas pointed out the problems and issues involved. He has also developed a prototype dark-field illumination device
using LED light and provided images and trial movies of the device. This study was conducted to support that practice,
and we acknowledge and pay tribute to his cooperation and advanced work in this field.

Reference
1) T. Hayamizu, Development of a New Device for Precise Timing, Report of the National Astronomical
Observatory of Japan Vol. 5 p73-79 (2001)
https://www.nao.ac.jp/contents/about/reports/report-naoj/p73.pdf

2) K. Miyashita, LIMOVIE, a New Light Measurement Tool for Occultation Observations Using Video
Recorded, Report of the National Astronomical Observatory of Japan Vol. 9 p1－26（2006)
https://www.nao.ac.jp/contents/about/reports/report-naoj/9-12.pdf
3)

Y. Fujisawa, Temporal Emission Characteristics of Light Emitting-Diodes for High-Speed Pulsed
Current, J. Illum. Engng. Jpn. Vol.81 No.8A (1997)4) Shutter methods, iDS
https://www.1stvision.com/cameras/IDS/IDS-manuals/en/basics-shutter-mode.html
Appendix 1
Settings to create a small time difference between the LED signal and the frame.

Prime factorising 1001 gives 7 x 11 x 13 = 1001. Any two of these prime factors can be chosen andmultiplied together to give the number a, and the remaining one is b. Dividing a and b so that they
are either the exposure time (msec) or the interval can make a difference of 1 msec.
143msec：
91msec：77msec：13msec：11msec：7msec；

0.143sec * 7frames = 1.001 sec
0.091sec * 11frames = 1.001 sec
0.077sec * 13frames = 1.001 sec
0.013sec * 77frames = 1.001 sec
0.011sec * 91frames = 1.001 sec
0.007sec * 143frames = 1.001 sec

In these cases, for example, for an exposure time of 11 msec, the difference is such that UTC
advances 1 msec relative to the frame recording time when 91 frames are advanced. Therefore, theframe recorded at the rise of the 1 PPS pulse (light) becomes progressively brighter as the frame
advances. From the change in brightness, determine how many milliseconds before the rise of 1 PPS(positive second) was the end of the exposure time of the first frame that became brighter.
Prime factorising 999 gives 3 x 3 x 3 x 37 = 999. As above:
333msec：
111msec：
37msec:27msec:9msec：
3msec：

0.333sec * 3frames = 0.999 sec
0.111sec * 9frames = 0.999 sec
0.037sec * 27frames = 0.999 sec
0.027sec * 37frames = 0.999 sec
0.009sec * 111frames = 0.999 sec
0.003sec * 333frames = 0.999 sec

In these cases, for example, when an exposure time of 27 msec is set, there will be a difference
where UTC is delayed by 1 msec relative to the frame recording time when 37 frames are advanced.
Therefore, the frame recorded at the rise of the 1 PPS pulse (light) becomes progressively darker as
the frame advances. From the change in brightness, the number of milliseconds after the rise of 1
PPS is determined from the frame immediately before the LED light disappears.
* An equivalent to the above is to use an exposure time that is multiplied by some number to 1002
msec. However, since the difference becomes as large as 2 msec and the accuracy of the estimation
may be halved, this method should be used for exposure times where the above is difficult to apply.
The above method should be used for exposure times to which the above is not easily applicable.
〇

Prime factorising 1002 gives 2 × 3 × 167.
501msec：

0.501sec * 2frames = 1.002 sec

334msec：

0.334sec * 3frames = 1.002 sec

167msec：

0.167sec * 6frames = 1.002 sec

6msec：

0.006sec * 167frames = 1.002 sec

Appendix 2
The exposure times available for thiscorrection method have been extended to one
decimal place, as shown on the left; byincreasing the LED light recording time,
various exposure times can be used.

Appendix 3
Limovie’s time correction system according to 1PPS LED light

Fig.11 Limovie’s time correction feature

The above features are implemented in Limovie (Jan. 23 2021)