Step 1: The object of interest is first identified by running CNN on the ODL pixels’ electric outputs in the first image frame. For all successive image frames, the ODL pixels are being continuously measured without triggering CNN. As long as these pixel values do not change, the same object is being perceived. This approach significantly reduces CNN runtime, especially at high frame rates for stationary or slow-moving objects. At the same time, the OHL pixels register and memorize the light intensities corresponding to the detected object (self-learning step).
Step 2: Once occlusion occurs, the ODL pixel values change. At this instance, the OHL pixel outputs are probed and should give the same values prior to the occlusion event, in this way reconstructing the visually blocked original object without having to run CNN. Hence, our sensor allows simple distinction between the occluded background (performed by the OHL) and the blocking foreground (detected by the ODL) without having to run computationally expensive cloud computing or other occlusion handling software.
To achieve such image reconstruction under occlusion, the OHL pixels have to maintain their electric outputs, and hence “recall” the image, when the light intensity or colour change due to the appearance of a different body masking the original object. We will implement such unsupervised self-learning as an invariance in the pixels’ open circuit potential VOC once light conditions are altered. This feature can be attained by dye-sensitized solar cells (DSSC) via the appropriate electrochemical modifications. Another advantage of using a DSSC is the possibility of good transparency, thus allowing the desired two functional layer tandem architecture. The DSSC based OHL pixels consist of a mesoporous transparent TiO2 layer containing chemically anchored photoactive dyes. An electrolyte infiltrates this photo sensitive film. The counter electrode allows connection of the cell to an external load for signal extraction (Figure 1c). Various charge transfer mechanisms take place during light exposure. In the following, only those processes majorly involved in the VOC build-up and decay will be summarized (Figure 1d)\cite{michael2003,nature}. Process 1: The photoactive dye molecule absorbs photons and excites an electron from the highest occupied molecular orbital (HOMO) to the lowest unoccupied molecular orbital (LUMO). Process 2: The excited electron within the LUMO is injected into the TiO2 photoanode conduction band CB (charge injection). The dye is now missing an electron; it is in its oxidized state. In order to continue absorbing photons and injecting electrons into the TiO2 photoanode, this oxidized dye molecule regains an electron back from a nearby reducing agent in the electrolyte. This process 3 is called dye regeneration. The reducing agent now becomes an oxidizing species after having transferred an electron to the oxidized dye. Some injected electrons residing within the TiO2 photoanode may transfer to these nearby oxidized shuttles to transform them back to reducing agents. This electron loss, process 4, is called charge recombination and will decrease the charge carrier density within the photoanode, lowering the TiO2 layer’s quasi Fermi level EF. As a consequence, the VOC decreases, since it is defined by the potential difference between EF and the redox potential of the redox agents\cite{michael2003,nature,letters,anders2011}. Normally, in the dark, all electrons accumulated and stored in the TiO2 film during illumination will rapidly discharge through recombination, immediately resetting the VOC close to zero, thus losing any memory effect. In the following, we use the transition from bright to dark to simulate the most extreme change in light intensity that could occur during occlusion. A more typical real life occlusion event would rather involve more moderate alterations in light intensities when two different objects intersect.