The following algorithm summarizes in more detail how the sensor tracks moving occluded objects (Figure 5h):
Step 1: The object of interest is first identified by running CNN on the ODL pixels’ electric outputs in the first image frame. For all successive image frames, the ODL pixel outputs are being continuously measured without triggering CNN. As long as these pixel values do not change, the same object is being perceived. This approach significantly reduces CNN runtime, especially at high frame rates for stationary or slow-moving objects.
Step 2: When ODL pixel values change, CNN has to determine which of the following scenarios is taking place: a) A new object has entered the scene, b) the object of the previous frame started to move, or c) the object of the previous frame is occluded. In the following, we focus on the case of occlusion as the other possibilities a) and b) can be handled at low cost by a locally trained CNN without having to invoke cloud computing.
Step 3: If occlusion occurs in the new frame, the computer probes the OHL pixels previously occupied by the travelling object (in our simple scenario, pixels 1-8 will be measured).
Step 4: These OHL pixel values together with the object’s corresponding ODL pixel outputs provide speed estimation. The positions of these active pixels reveal the spatial information required for complete path prediction.
Step 5: The probed OHL pixels are being reset by short circuiting the devices, allowing analysis of a new object. The user would need to define the condition for pixel resetting. One reasonable requirement could be resetting after the tracked object is confirmed by CNN to have reappeared after the occlusion event (Figure 5e).