Toward Autonomous Field Inspection of CSP Collectors With a Polarimetric Imaging Drone

. We developed a polarimetric imaging drone to perform field inspections of heliostats and carried out field tests at Sandia’s National Solar Thermal Test Facility (NSTTF). The preliminary results show that Degree of Linear Polarization (DOLP) and Angle of Polarization (AOP) images greatly enhanced the edge detection results compared with the conventional visible images, supporting fast and accurate detection of heliostat mirror edges and cracks. The system holds the promise to enable future automated detection of heliostats optical errors and mirror defects.


Introduction
During the typical operation of a Concentrating Solar Power (CSP) plant, a large portion of the energy can be lost due to various imperfect conditions, such as blocking, shading, mirror soiling, tracking and canting errors, etc. Effective and efficient field inspection technology for CSP plant is necessary for various applications such as efficient control, maintenance, and repair.Currently the inspection of CSP systems relies on conventional imaging techniques [1], deflectometry [2], flux mapping [3], machine vision [4], and LIDAR scanning [5].These state-of-art heliostat inspection technologies have different limitations in terms of accuracy, speed, complexity, or cost.The limitation of accuracy results in a loss in optical efficiency due to tracking error, alignment error and defects.Despite these advances, highspeed optical inspection of thousands of heliostats in very large solar fields remains an unsolved problem.For example, systems and algorithms employing high-resolution visible drone cameras are being developed for optical error measurements of heliostats [1][6] [7].However, for visible cameras, mirror edges can be difficult to find automatically when the contrast provided by intensity and color between the target heliostat and the background is low.Additionally, small cracks on heliostat mirrors could propagate and subsequently cause final failure.Therefore, it is also beneficial to detect the small cracks prior to failure to reduce the optical loss and extend the heliostat lifetime.With the visible cameras, these cracks are difficult to detect due to the limitation of viewing angle and resolution.
Here we show that polarimetric imaging is a promising solution for significantly enhancing contrast of mirror edge and crack detection for CSP heliostat field inspection.In the following sections, we first present the configuration of a polarimetric imaging system embedded on a drone.Based upon the skylight polarization pattern simulated using the Rayleigh scattering model [8], we designed the flight path so that in the captured images, the polarization information on the heliostats can provide high contrast for mirror edge and crack detection.Then we performed theoretical study and field tests to explore the feasibility of utilizing polarization images to address the low-contrast issues in the detection of heliostat edges and corners in complex backgrounds.The superior contrast supports reliable image processing, thus facilitating fast and accurate determination of heliostat optical errors and defects using drone-based techniques for future automatic inspections.

Unmanned Aircraft System Polarimetric Imaging Setup
We built a polarization imaging drone by integrating an ultracompact polarimetric imaging system (a 5.0 MP polarization imager) onto a UAV, as shown in Figure 1.A GoPro camera (Figure 1, C) was mounted on top of the polarization camera (Figure 1, D) to record video to enable comparison with conventional images.The remote control of the camera is realized using a micro-processor, Jetson Xavier (Figure 1, A), integrated on the stabilizer (gimbal, Figure 1, E) under the UAV.We designed a program with functions of remote control, image capture, an adaptive exposure algorithm, real-time streaming, and local storage.We established a wireless connection between the imaging system and the ground control using Amimon Connex (Figure 1, B and F).During field operation, operators can control the gimbal pointing direction and trigger image capture with the Futaba controller (Figure 1, H).When we toggle the switch on Futaba Controller, it sends the signal through the Futaba Receiver (Figure 1, G) to the Connex, and wirelessly transmitted to the Jetson Xavier in the air.On the monitor (Figure 1, I), the GUI enables operator to see the livestreaming video (Figure 1, L) from the polarization camera.The status button (Figure 1, J) "Capture" turns yellow during the capturing, the information regarding frame rate, current task and image format is shown on the information window below (Figure 1, K).With this system, we were able to finish two field tests for data collection on December 16 th , 2021, and April 28 th , 2022.We implemented an adaptive exposure algorithm that utilizes the pixel intensity of the captured image to adjust the exposure automatically.Under the Mono8 mode, the pixel grayscale value ranges from 0 to 255.Each time a capture is triggered, the program takes twelve images consecutively.The first six images were captured without adjusting the exposure.The algorithm then examines the number of pixels that exceed the over-exposure threshold, which was set as 245.The total number of pixels of this imager is over 5 million, and we set 100,000 as the tolerance for the over-exposed pixel numbers.The algorithm then gradually reduces the exposure until the number of pixels that exceed 245 intensity level is lower than the tolerance, 100,000.Next, six more images are captured with the new exposure.The whole process takes approximately 15 seconds.We capture multiple images consecutively to ensure that despite a strong environmental interference, such as wind, at least one of the images taken is not significantly influenced, which usually results in motion blur.Figure 2a is an overexposed image, and Figure 2b is the image captured after the adaptive exposure algorithm adjusts the exposure automatically.This allows us to capture images with ideal intensity regardless of the environmental lighting conditions.

Results
Conventional imaging drones have been used to determine heliostat optical errors and defects.These optical error evaluation methods rely on accurate knowledge of the spatial relations between the target heliostats and the camera [7][9].It is essential to identify the edges of the target heliostat in the optical error evaluation process.Yet, with visible images, it can be quite challenging to differentiate heliostat mirrors from their background when the mirror image is similar to its background in color and intensity.We studied a few situations commonly encountered during field tests and proved the feasibility of using polarimetric imaging for improved detection of mirror edges and defects in these situations.Figure 3 shows three scenarios commonly seen in heliostat inspection.We built the UAV-integrated autonomous polarimetric imaging system and carried out two field tests at the National Solar Thermal Test Facility (NSTTF) operated by Sandia National Laboratories for the U.S. Department of Energy (DOE).The first field test was taken from 9AM to 5PM on December 16 th , 2021, with a clear sky condition.We captured 828 valid polarization images with the polarimetric imager, excluding invalid images with issues such as poor focus, motion blur, not seeing the target in camera frame, etc.Then from 10AM to 6PM on April 28 th , 2022, we carried out a second field test and captured 1356 valid polarization images.The visible images are from the GoPro Camera recording.

Sky-vs-sky scenario
When the field is in operation, all heliostats are focusing sunlight on the receiver, and thus adjacent heliostats point in a similar direction.When we take images with a drone camera, we often encounter a scenario where two adjacent heliostats overlap in the image, but only one of them is the target heliostat of interest.In the high-speed scanning method described in [7], the drone camera position is designed to look at the heliostats with their reflection towards the sky but without direct sunlight.In some situations, two overlapping heliostats both reflect blue sky, resulting in very similar intensity and color.This can result in low edge contrast between them.As shown in Figure 4a, the edge detection result of the visible image can exhibit missing edges where overlap occurs.Such situations make fast and reliable detection of mirror edges and corners more difficult.In the example of Figure 4, edge detection was performed using a structured forest algorithm [10].
On the other hand, the polarization image (DOLP) shows much better contrast (Figure 4b), because the polarization of skylight varies in different regions, which can be predicted accurately based on the Rayleigh scattering model [8].To take advantage of the polarization image, Rayleigh simulation and proper flight planning are important.When using the polarimetric imaging drone, we simulated view directions and reflected images and planned the flight accordingly to ensure the contrast was enhanced with polarization Figure 4.An example sky-vs-sky scenario where processing the conventional image did not have enough contrast to detect the edge between adjacent heliostats, but the increased contrast in the degree of linear polarization image enabled edge detection.images.For example, Figure 5 shows the simulation for the sky-vs-sky scenario.When the heliostat mirrors are reflecting the sky region with high DOLP (Figure 5b) values and high DOLP gradients (Figure 5c), the slight difference in the adjacent heliostat orientations will result in a good contrast of the DOLP reflected by the heliostat mirrors.In the reflection schematics (Figure 5a), the smiley face represents the camera position.With known coordinates of the heliostat and camera, we can calculate the corresponding sky dome region that the mirrors are reflecting.The simulated sky polarization pattern can be obtained by time and location, using the Rayleigh scattering model [8][11] [12].Here, azimuth starts at North and increases clockwise.Zenith starts from the center and goes to 90 degrees at the boundary.We designed the image capture waypoint for the drone to ensure in the camera view, the mirrors are reflecting the sky region with high DOLP values and high DOLP gradients.This way, the DOLP values on the adjacent heliostats in the captured image have a good contrast.
According to the field test results so far, if one can utilize the predicted skylight polarization pattern to position the polarimetric imaging drone to obtain mirror images of sky with a DOLP gradient of over 0.01 per degree (Figure 5c), the polarimetric images can enable heliostat edge detection with a success rate >90%.We counted the edges that were manually identified as sky-vs-sky and report the number that were found.If the number agrees with the ground truth of the heliostats, we count this as a successful detection.The results are listed below in Table 1.Each set contains six images taken consecutively.

Ground-vs-ground scenario
Another challenging scenario results from view situations with ground both in the mirror image and the adjacent background, as shown in Figure 6 (ground-vs-ground).This happens when the drone views the heliostats from a high elevation angle, and thus the reflection of the mirrors in the bottom is a reflection of the adjacent ground.Since the color and intensity of the reflection is very close to the real background, it is difficult to detect the edges using visible images (Figure 6a).As a comparison, the Angle of Polarization (AOP) image (Figure 6b) shows clear contrast, which yielded a highly improved edge detection success rate.This occurred because the polarization state of light scattered off the ground is quite different from that reflected from the mirror.For reference, the reflected AOP predicted using the Rayleigh scattering model [8] is also shown in Figure 6c.
Unlike the mirror facets, the AOP pattern of the background does not follow the simulation of the sky pattern.In our field test, some cases showed that the AOP values of the mirror facets are significantly different from the background (90 degrees or more), producing strong contrast, as shown in Figure 7a.In other cases, when the AOP values of the mirror facets are similar to the ground next to them, the contrast is low for edge detection (Figure 7b).Since the AOP values of heliostat mirror facets are predictable using the Rayleigh scattering model and reflection, we need a physical model to predict the polarization pattern of the ground.Currently, we are studying the Torrance-Sparrow Model  [13] of a rough surface, adding the scattering of light caused by soiled particles.We hope to develop a physical model that predicts the AOP values of the ground, it can guide the flight to capture images of good contrast for ground-vs-ground scenario.

Back-vs-ground scenario
Several optical error inspection methods have been developed for CSP plants.Our goal is to use the high-speed UFACET-s method [7] to analyze the backside reflection of heliostats in front to evaluate optical errors.In this method still under development, an optical model, photogrammetry, and reflection analysis are used to analyze heliostat features and compare observed feature locations to anticipated locations predicted by an ideal model.The discrepancy indicates optical error and its magnitude.We expect to see back-vsground scenarios in some of these captured images, where the backside reflection is adjacent to the background.At NSTTF, the backside reflection has similar dark color to the shadow of the heliostat, which can introduce a difficult edge detection scenario (Figure 8a).With the polarization images (DOLP), this contrast is enhanced while we look at the heliostat mirrors reflecting a high DOLP region of the sky, as shown in Figure 8b.The edge detection results showed that the DOLP images can reduce the noise introduced by background, such as the shadow, reducing clutter when seeking difficult back-vs-ground edges.

Mirror Crack Detection
After years in operation, some of heliostat mirror facets may have cracks.Mirror cracks can degrade heliostat performance or lead to failure, but cracks are hard to detect with conventional imaging technologies due to their small size and low contrast.The polarization image (DOLP) shows fine details of cracks on a heliostat mirror (Figure 9c), suggesting future use for fast screening of mirror facets for crack detection in outdoor environments.
For the visible images (Figure 9a), the detection of the cracks relies on the viewing angle and sun position.The cracks are more visible when the crack happens to be reflecting sunlight.In our observations so far, the cracks have a low DOLP, possibly because the uneven surface causes scattering.Thus, if we plan the flight to capture images where mirror facets reflecting high DOLP values, we anticipate good contrast to detect the crack regardless of viewing angle of the camera.Table 2 summarizes the results when the mirrors are reflecting the sky region with high DOLP values.Here, we count the images that show the known cracks on the facets as successful detection.Further work is required.

Conclusions
In summary, we developed a polarimetric imaging UAV (drone) to for inspection of heliostats on the CSP fields and performed field tests on the Sandia NSTTF field.The polarimetric imaging drone utilizes skylight to achieve significant enhancement of image contrast for heliostat inspection compared with conventional visible imaging.For different scenarios, different polarization information can be used to provide better contrast for edges or cracks.In sky-vs-sky scenario, we can design the drone position so that the adjacent heliostats are reflecting the sky region with high DOLP and DOLP gradient.This way the slight orientation difference of the two heliostats will result in different DOLP values, which can be captured by the polarimetric imaging system to provide good contrast on the overlayed mirror edges.In ground-vs-ground scenario, we discovered the good contrast of AOP images when the background AOP is very different from the mirror AOP.We plan to build a physics model to describe the scattering of the background to predict its AOP, so that we can plan the drone position for the camera to see good contrast of AOP from the mirrors and background.In back-vs-ground scenario, the DOLP image helps reduce the interference of irrelevant noise in the background to find the edges of mirrors more accurately.This also requires a high DOLP on the mirrors.For the mirror crack detection, we found that with the high DOLP on the mirror, we could detect the cracks regardless of the viewing angle of the camera.This provides better accuracy and stability of the autonomous crack detection.Using the skylight polarization model and locations and orientations of heliostats in the field, we can design the flight path to achieve sufficient image contrast for different detection purposes during the field inspection.These polarization images are promising visual aids for machine vision to detect the edges and cracks automatically, enabling UAV-based accurate inspection and measurement of heliostats in a CSP plant.

Figure 2 .
Figure 2. Images captured using the polarimetric camera with the adaptive exposure algorithm.2a: Before algorithm adjustment.2b: After algorithm adjustment.

Figure 3 .
Figure 3. Examples of different scenarios that are challenging for conventional image processing techniques.In each case, some mirror edges appear very similar to adjacent background.

Figure 5 .
Figure 5. Polarization analysis of the sky-vs-sky scenario.The sky polarization patterns were calculated based on the Rayleigh scattering model [8].

Figure 6 .
Figure 6.Example for ground-vs-ground scenario where the visible image (a) shows low contrast on the bottom edge to the ground, but the angle of polarization image (b) shows good contrast.The corresponding simulation of the sky [8] AOP pattern (c) is useful to predict AOP of the mirror reflection.

Figure 7 .
Figure 7. Examples for ground-vs-ground scenario.7a shows high contrast where the AOP of the mirror is significantly different from the background.7b shows low contrast where the AOP of the mirror is close to the background.

Figure 8 .
Figure 8. Example back-vs-ground scenario with the conventional image (left) and the degree of linear polarization image (right).

Figure 9 .
Figure 9. Example crack detection where the degree of linear polarization image (b) shows more of the spider web pattern indicating cracks than the visible image (a).

Table 1 .
Results summary for sky-vs-sky scenario.

Table 2 .
Results summary for cracks detection.