Hel-IoT Web Server: A Smart Heliostat Development Platform

. The smart heliostat is a concept that arises from applying artificial intelligence and computer vision techniques, mainly the machine learning technique called object detection, to the traditional heliostat to increase its efficiency and reduce costs [1,2]. Continuing with previous work at CIEMAT-PSA focused on the development of a smart heliostat and especially the solar tracking subsystem [3], a powerful tool (Hel-IoT web server) has been developed helping us with the laborious task of creating the dataset and training the model employed for object detection in solar tracking. The main goal of the Hel-IoT web server is to enhance the image data set employed to train new models and, at the same time, to improve the model that the web server is continuously running. The first conclusions that it can be extracted from the use of the Hel-IoT web Server for more than 4 working months is that it is a very powerful tool for the creation of labeled data sets with meteorological information, especially for the creation of data sets for smart heliostats. Furthermore, thanks to the joint use of CETA cluster (Centro Extremeño de Tecnologías Avanzadas), the Hel-IoT web server can improve the model automatically by periodically retraining it.


Smart heliostat
The smart heliostat is a concept based on the application of techniques derived from industry 5.0, mainly artificial intelligence and computer vision techniques.This work focuses on the development of solar trackers, which is a fundamental subsystem of heliostats.The Smart Heliostat Tracker (Hel-IoT) is based on machine learning.The main hardware elements are a low-cost low-power consumption embedded computer plus a wide-angle camera.Software includes a neural network able to detect the Sun, receiver, clouds and other heliostats.The general idea is to calculate the heliostat aiming position from detection of the Sun and receiver in the camera video frames.
It is assumed that Hel-IoT is attached to the heliostat surface mirror's center point (O') and moves with it; other positions are also possible.This example considers the target as the element where the concentrated flux is reflected; the same applies for the receiver.The camera provides a plane view (CP) of the scene.The neural network detects the Sun's (S') and target's (T') center points.The middle point (A") between S' and T' is the desired heliostat aiming point, based on the Snell's law.The current heliostat aiming point (A') is the center point in the plane view (CP).The heliostat is then moved to place the current aiming point (A') at the desired aiming point (A").Figure 1a shows a camera view for an out-of-focus heliostat, whereas Figure 1b shows the same view for a focused heliostat.Hel-IoT hardware is just composed by a low-cost wide-angle camera and a low-cost lowpower consumption computer with wireless (Wi-Fi to send and receive data and commands) and local (GPIO port to control the heliostat actuators) communication capabilities.
The tracking software has been programmed in Python [4], which is an open-source object-oriented programming language.Python is widely used for scientific computing.Tensorflow [5] can be used to add machine learning capabilities to the software; it integrates seamlessly with Python.Tensorflow is an open-source machine learning framework developed by Google.Tensorflow provides flexibility to design, implement and test different kinds of neural networks.Nevertheless, Hel-IoT is not limited to any particular hardware and software configuration.Machine learning applies statistical techniques to create algorithms and software that can learn from data (learning patterns) without being explicitly programmed to do it.Deep learning is a class of machine learning algorithms that learns data representation.Deep learning is commonly applied in computer vision, where in some cases outperforms humans on image and object recognition tasks [6].The development of the smart heliostat through the application of these tools was explained in more detail in [1,2].
Traditional solar tracking systems require strict and labour-intensive installation, expensive hardware, and periodic recalibration.Hel-IoT reduces the installation and maintenance cost.The hardware is low cost ( < 135 € ) and it is cheaper than current control system hardware in industrial heliostat trackers ( > 675 €) [7,8].Furthermore, there is room for further cost reduction, for instance bulk purchases or using less powerful custom-tailored hardware.Hel-IoT does not require an accurate installation process like traditional sun tracker systems for heliostats, i.e, positioning, vertical alignment, calibration and offset correction; among others.The installation cost could be further reduced by employing less expensive heliostat equipment.Fixed cost reduction could also lead to a reduction in the optimal heliostat area.This will also translate into a cost reduction in the heliostat structure [8] optical and economic point of view.It decreases the operation cost.If the installation of the low cost camera is done in the factory during the assembly of the heliostat, the heliostat does not require calibration in the field which reduces the spillage, thus increasing the overall plant performance.Hel-IoT hardware is equipped with Wi-Fi technology, which further reduces installation costs by not needing communication wiring.Wireless communications are only needed to receive supervision and to send local additional information.However, it should be studied and implemented wireless communication technologies applied to smart heliostat, for example Wi-Fi mesh technology appears to be a good candidate for this development.
Hel-IoT also provides additional information to detect plant design issues, i.e. blocking and shadowing due to surrounding heliostats.Moreover, cloud detection provides data to optimally control the solar plant.The detection of clouds provides useful information to make predictions about the near future collected solar radiation and the overall plant production.This information enables to optimally control the solar plant and could help to keep the energy production stable, anticipating startups and shutdowns of the system.This line of research has an enormous potential to reduce solar power tower plants costs (CAPEX and OPEX), specifically heliostat field costs, which typically account for the largest CAPEX of solar tower plants.

Hel-IoT web Server
As commented before, Hel-IoT is based on artificial intelligence, specifically a technique called object detection in where an artificial neural network is able to detect objects in images.For that, it is necessary to train a neural network with many previously labeled images.The tasks of creating the set of labeled images and then training the model with the image set can be very tedious and requires tools to reduce the great deal of labour.There are different tools developed for labeling, but all of them have a more general approach and are not adapted to Hel-IoT's requirements.For such reasons, Hel-IoT web server has been developed.It facilitates to create the labeled image set and automatically train the neural network model.Hel-IoT web server provides a friendly graphical environment that makes it possible for anyone without computer science knowledge be able to label images and retrain the model to improve its performance.

Server Architecture
In addition to the aforementioned software, Python and Tensorflow, Hel-IoT web Sever uses additional free and open source software: a) Docker [9] for the automatization of the deployment of applications within software containers, providing an additional layer of abstraction and automation of application virtualization across multiple operating systems; b) NGINX [10] web server/reverse proxy; c) Flutter [11] for the web app development; d) CloudBeaver [12] database client for data management, analysis and administration and PostgreSQL object re lational database system [13].Figure 2. shows Hel-IoT web Sever and how it accesses to the Plataforma Solar de Almeria (PSA) servers where it stores the data and creates backup copies.It also has access to the meteorological information available on the PSA network.

Server Operation
Hel-IoT server constantly stores meteorological data in the database, these data are available through API queries.Then a set of Hel-IoT systems, deployed in the CESA field in PSA, are taking images with low-cost cameras.Immediately after, the meteorological information obtained through the API is added to each image in its metadata.Then, the neural network of the Hel-IoT system processes the image to detect objects, sends the image to the store system and sends the object detection information to the server.Hel-IoT web server is responsible for managing the graphical user interface to inspect meteorological data and manage the set of images with the information on the detection of historical objects, as shown in Figure 3.

Labeling and training
Hel-IoT web server provides an image labeling tool developed for Hel-IoT.Labeling a new image or correct an image taken by one of the heliot systems deployed in the CESA field and available on the server can be done thanks to Hel-IoT web server, see Figure 4. From time to time, object detection predictions have to be corrected on certain images.Images that are then manually corrected or re-labelled, and sent to the CETA cluster.When the cluster receives a new set of relabeled images, it retrains the model in order to improve object detection accuracy.If the model retrained with the new set of corrected images shows a better performance, according to established metrics in an evaluation images set, it is deployed to the Hel-IoT systems in the CESA field.

Results
The system has been continuously running since April 2022, but we are still making some adjustments.Regarding the meteorological and image database, the system is robust and completely functional.All the data coming from different sensors or systems are available through the Hel-IoT web server.
Regarding online retraining, the full automation level of the process reduces the great deal of labour and, although it is too early to obtain conclusions about the model evolution with such periodically retraining, it has been possible to appreciate an improvement in the performance over time.For example, the first image from 23 rd of April shows that the neural network model did not detect the Sun and the target detection was imprecise.However, the second image on Figure 5 from 14 th of August, the Sun and the target detection were accurate.Also in this image, the model detected three heliostats and a low-density cloud.This shows a substantial improvement thanks to the automatic successive retraining.

Conclusions and future work
Hel-IoT web server has shown that it is a powerful tool to manage image and meteorological data for smart heliostat developing tasks.The labeling tool developed in this work adjusts perfectly to the required demands and reduces labeling time by allowing inaccurate or erroneous detections to be modified, in this way label the whole image from scratch is not needed.Periodic retraining with corrected images can improve the model and reduce retraining time by only using images where the model did not perform well.
Future work includes that Hel-IoT web server automatically evaluate the performance of different models with a set of evaluation images, store and compare additional metrics to analyze in terms of precision and accuracy different models, cameras and embedded devices to improve the development of Hel-IoT.