OKU Laboratory

Random Access Vision​

​

​
In robot vision, it is often necessary to measure an area wider than the camera's field of view. Therefore, cameras are typically mounted on mechanical pan/tilt platforms, but such mechanisms have response times that cannot be ignored compared to the camera's frame rate. To address this, our lab, in collaboration with Professor Kawahito's research group at Shizuoka University, developed Random Access Vision, an imaging technique that observes multiple arbitrary gaze directions per frame.

 

The Random Access Vision system is mainly composed of a resonant mirror that vibrates at high speed to change the camera's gaze direction and a multi-tap lock-in image sensor, which has multiple image storage units (taps) and allows independent exposure control on a nanosecond level. By using this multi-tap lock-in image sensor, it was possible to capture images from multiple different directions almost simultaneously with a single camera.

Figure 1 shows the configuration of the proposed system, and Figure 2 shows the actual prototype system. In the prototype system, experiments were conducted using a resonant mirror operating at 12 kHz and a multi-tap lock-in pixel image sensor with four taps. The camera's gaze direction was constantly changing due to the resonant mirror, and the gaze direction was determined by exposing the camera when the mirror pointed in a specific direction. The resonant mirror oscillated at a high speed with a cycle of approximately 83 μs, and to avoid motion blur, the exposure time required was extremely short, around 200 ns. Due to the short exposure, the brightness obtained was low and insufficient for image measurement. Therefore, multiple short exposures were performed each time the gaze pointed in a specific direction, and these exposures were accumulated to improve image brightness. The outline of these operations is shown in Figure 3.

With this method, it was possible to select arbitrary gaze directions per frame and successfully capture images from four different directions (X-direction) simultaneously. Additionally, we confirmed that it was possible to perform operations like jumping, gathering, and spreading the gaze direction for each frame, as well as simultaneous capture from four different directions using gaze control in both X and Y axes with a two-axis mirror.