SUMO Roundabout Simulation with Human in the Loop

: Traffic simulators rely on calibrated driver models in order to reproduce human behavior in different traffic scenarios. Even if quite accurate results can be obtained, the actual interaction between human being and traffic cannot be completely reproduced. In particular, as automated vehicles are being developed, the human in the loop is required to understand whether drivers feel comfortable and safe in mixed traffic conditions. In recent years, dynamic driving simulators have been developed to test vehicles in complex or dangerous situations in safe and controlled environments. However, driving simulators are mostly devoted to the study of vehicle dynamics more than traffic situations. This paper presents an integration of SUMO with a high end dynamic driving simulator with the aim to study human reactions while negotiating a roundabout in mixed traffic conditions. SUMO is in charge of traffic simulation, while a full vehicle model is employed for the simulation of the dynamic of the human driven car. To allow a human to effectively drive the car, both simulation environments have to run in real time while exchanging the required information. Also, scenario graphics, sound and driving simulator feedback motion have to be accurately realized and synchronized with the simulations. A real-time server is employed for the synchronization of the different environments. As SUMO does not consider vehicle dynamics, particular attention is devoted to the a realistic reconstruction of trajectories and vehicle dynamics to be represented in the scenario. Some preliminary tests are shown where a panel of testers has been asked to negotiate the roundabout with different percentages of automated vehicles. The results of the tests show that drivers were able to perceive differences in the behavior of other vehicles and that the proposed approach is effective for understanding the feeling of comfort and safety of the human driver.


Introduction
Microscopic traffic simulators (MTS) are a powerful tool for the study of traffic and infrastructures. Each vehicle in a given road network is simulated individually allowing, among other things, for a detailed analysis of infrastructure design and modification, traffic control, behavioral studies and testing of connected and automated vehicles (CAV) [1]. CAVs are expected to provide a huge opportunity for increasing traffic flow, increase transport safety and reduce fuel consumption and emissions [2], [3]. However, the introduction of CAVs requires a huge amount of tests and data collection both by real world tests and by simulations [4]. CAVs and human driven vehicles will be required to share the same traffic environment during the transition period [5]. Reliable simulations in mixed traffic conditions require a realistic model of the human driver. To this respect, even if MTS provide several models describing driver behavior, due to human complexity and variability, such models are not able to fully catch human effects in traffic simulations [1]. Also, human acceptance and preferences when driving in a mixed traffic condition is still being researched [6].
Driving simulators, on the other hand, are developed with the aim to introduce the human driver into the simulation and study its interaction with the simulated vehicle and environment. In general, driving simulators are more focused on the simulation of the driven vehicle and the surrounding traffic simulation can be not fully accurate [5]. Those simulators employ sophisticated 3D visualization creating realistic and immersive scenarios. In cases, driving simulators can be coupled with actuated platforms and give also motion feedback to the driver. Such dynamic driving simulators often feature full scale vehicle cockpit and audio surrounding to create a completely immersive experience in order to get a more natural response of the human driver.
Several papers can be found in the literature describing the integration of MTS and driving simulators. Even if such integration is not a novelty [7], [8], only in recent years graphical and computational performances have allowed the realization of realistic scenarios [1], [4]- [6], [9]- [14]. From these papers, the principal technical challenges related to the co-simulation between MTS and driving simulators can be summarized in the following aspects.
• Network correspondence. To obtain a proper co-simulation, the same road network must be reproduced both in the MTS and in the driving simulator. This problem is discussed in the great majority of the cited papers. Depending on the chosen software for the simulations, different approaches, mostly manual, are described. • Trajectories. MTS do not consider a realistic vehicle dynamics, but the simulation is focused only on traffic flow. As a result, the trajectories of the vehicles are not realistic, but unrealistic effects such us sharp turning angles or instantaneous line changes are usually present. In [1] interpolation schemes are proposed in order to obtain smooth bending trajectories. • Synchronization and real time simulation. In order to include a human in the loop, the simulation must run in (or close to) real time. Also, the simulation time of the two software must be synchronized and frequent exchange of information has to take place. Different strategies are presented. In [1] the built-in real time function of the employed MTS has been exploited to trigger the simulations. Alternatively, when a different MTS without such function has been used, the integration parameters have been set to obtain a similar effect. In [13], a dynamic driving simulator has been employed and used to synchronize the simulations. • Delay. Delay between the two simulations is very important to provide a realistic and consistent experience to the human driver. Especially in urban scenarios, delay is very important to allow a correct perception of the positions of the other vehicles. Delay depends on the rate of data transfer between the two simulators. Usually, driver simulators run with very short simulation steps (from a maximum of 33 ms [11] up to 1 ms [15]). Smaller time steps have to be preferred to enhance the experience felt by the driver. MTS have larger integration steps, with common values between 0.1 to 1s, with the lower value typically used for these applications. This causes large delays (of the order of 0.1-0.2 s [1], [11], [13]) between the two simulations which may alter the driver perception.
In this paper, a co-simulation between SUMO [16] and a high end dynamic driving simulator is developed for the simulation of mixed traffic conditions with CAVs and human driven vehicles in a roundabout scenario. With respect to the considered papers, the described application employs a real time scheduler to have a very accurate synchronization between the two simulations and the real-time. Also, by setting up a communication frequency of 200Hz between the simulation, a very short delay of 5 ms is obtained. The employment of a high end dynamic simulator allows the driver to have a fully immersive experience, including the motion feedback. Additionally, a model of reinforcement learning artificial intelligence is run in parallel to the SUMO simulation by using the Flow library [17] to drive the CAVs. Preliminary tests with a restricted panel of drivers show the potentialities of the application.

Driving simulator and VI-Worldsim environment
The dynamic driving simulator utilized is the cable-driven DiM400 Dynamic Driving Simulator of the DRISMI laboratory [18] of Politecnico di Milano. The simulator is produced by VI-grade [19] and shown in Figure 1 The cockpit motion is obtained by a redundant system of actuators, conceived to decouple the low-frequency and high-frequency motions. A lower stage of actuation composed by a cable driven platform with in-plane degrees of freedom (longitudinal, lateral and yaw) is coupled by a higher stage realized by a Stewart platform providing all six degrees of freedom. The first stages is capable of large motion at relative low frequency (up to 3 HZ), while the second stage realized smaller motions at higher frequencies (up to 30 Hz). By combining the two stages, both low and high vehicle frequencies can be reproduced. To reproduce the higher frequencies related to NVH (noise and vibration harshness), eight shakers, able to provide vibrations up to 200 HZ, are located in engine and suspension connecting points. Table 1 reports the driving simulator performances, further details on the driving simulator can be found in [20]. Haptic seat belts, air cushions, interactive steering wheel and active brake complete the cockpit equipment. A 270°-wide 120 Hz screen surrounds the cockpit. Five speakers reproduce the sources of noise in and out of the vehicle while driving.
The motion of the dynamic driving simulator is controlled by a cueing algorithm based on a Model Predictive Control and able to provide linear and rotational acceleration consistent with the expected acceleration in the considered situation [21]. The motion of the human driven vehicle in the simulation is computed by a 14-degrees of freedom model implemented in VI-CarRealTime [19]. The graphical environment and the other vehicles are reproduced by VI-WorldSim [19]. VI-WorlSim ( Figure 2) provides a full 3D traffic visualization realized by Unreal Engine. It is a commercial software, ready to use, and it also includes a basic traffic generator. Optionally, the traffic generator can be disabled and the vehicles can be controlled by external signals. VI-WorldSim is installed on a Intel i7-9700K@3.60 GHz workstation with 32 GB and Windows 10 pro.
The driving simulator is controlled by a 2 x Intel Xeon Gold 6144@3.50 GHz with 48 GB and Linux RedHawk 7.3 real time server ( [22]). The server is in charge of synchronizing all process, run the VI-CarRealTime simulation of the human driven vehicle, run the cueing algorithm for the control of the simulator and manage all network connections, sensors and cockpit actuators. Graphic and sound are managed by six Intel i7-9700K@3.60 Ghz with 32 GB and Windows 10 pro workstations equipped with a GeForce RTX 2080 Ti. A real time database is updated at each simulation step on the real time server and is shared with all other workstations. The simulation step is set at 1 ms. The real-time server constraints each simulation step to be performed in a time interval of 1 ms, assuring a real-time simulation. All models involved must be optimized to have computational times less than the allotted time interval.

Reference scenario
The reference scenario for this application is a three-leg single-line roundabout with mixed traffic conditions. Connected and automated vehicles share the roundabout with human driven vehicles. The human driven vehicles are driven by a IDM (intelligent driver model) algorithm [23] implemented in SUMO. One of the human driven vehicles is actually driven by the human in the loop in the driving simulator. All the vehicles are considered to be connected and exchange data related to their trajectories, velocities and accelerations. CAVs are controlled by artificial intelligence defined by a reinforcement learning approach designed for realizing a policy able both to drive safely CAVs into the roundabout and to optimize the traffic flow. The communication protocol based on an innovative V2N2V (Vehicle to NEtwork to Vehicle) approach with 5G communication and edge node computing has been specifically developed in the AI@EDGE project [24]. This reference scenario is part of the AI@EDGE project and represent a use-case for the validation of the project 5G and edge computing technologies. The roundabout scenario has been chosen as roundabouts are currently one of the most critical scenarios for automated driving [25] providing a challenging real world problem to test the technologies developed in the AI@Edge project. For interested readers, more details on the project and on the AI can be found in [24]. In this paper, only the part of the project related to the integration between the two simulators is discussed.
The roundabout network has been realized in SUMO. The network has then been exported to Mathworks Roadrunner and translated by the Unreal engine to be imported in VI-WorldSim. This procedure guarantees the correspondence between the road network in SUMO and in VI-WorldSim. In Figure 3 the road network in SUMO and in Mathworks Roadrunner are depicted.

SUMO -VI-Worldsim integration
The scheme for the integration of SUMO with the driving simulator is depicted in Figure 4. The core of the connection is the real-time database located on the real-time server. The database is accessible by all the workstations of the network. The real-time server provides the base real-time clock of 1 ms to synchronize all connected workstations. In particular, by considering Figure 4, the following processes are considered.
• Human in the loop vehicle. The vehicle driven by the human in the loop is simulated by a fourteen degree of freedom model implemented in VI-CarRealTime [19] and running on a dedicated core of the real-time server. The vehicle is simulated with a simulation step of 1 ms. Each simulation step is performed within the allotted real-time clock of 1 ms. The driving commands for the vehicle are given by the human in the loop through the steering wheel and pedals of the cockpit of the simulator. The commands are stored in the real-time database and read by the simulation in VI-CarRealTime. In turns, VI-CarRealTime writes on the real-time database the state of the human driven vehicle. Such states are fed to the controller of the dynamic driving simulator for the motion feedback and to the graphic servers via VI-WorldSim for the visual and audio feedback. • SUMO connection. SUMO runs on a Intel i7-11700F@2.50 GHz with 32Gb and Linux Ubuntu 18.04.6 LTS workstation. The workstation is connected to the realtime server by a UDP connection via a python script. The python script is also in charge of communicating with SUMO by using the TraCI library. The UDP connection is used to synchronize the simulations. The real-time server sends the state of the human driven vehicle every 5 base real-time clocks of 1 ms, i.e at constant time intervals of 5 ms. The python interface waits until the vehicle state is available. When available, it reads the state, updates the vehicle position in the SUMO simulation and then launches a simulation step of 5 ms. When the simulation step is done, the python interface retrieves the states of all other vehicles and sends the information via UDP to the real-time database. It is important to notice that if the computation time required for the SUMO simulation is less than 5 ms, real-time simulation and synchronization are guaranteed. In this way, a delay of only 5 ms is present between the states of the human driven vehicle and the states of the other vehicles. The number of the vehicles that can be simulated in the network without violating the real time constraint depends on the available computational power and the network complexity. With the employed hardware configuration, the scenario considered in this paper can be simulated in real-time with up to 60 vehicles in the network. • Artificial intelligence for CAVs control. A second python instance runs on the same Linux workstation with a second instance of TraCI connected to the same SUMO simulation and to the Flow library. This interface is in charge of communicating with SUMO, retrieve the state of the simulation and provide the commands for controlling the CAVs according to the AI trained by the reinforced learning. • VI-WorldSim connection. VI-WorldSim is connected directly to the real-time database for the standard interactions with VI-CarRealTime to get the motion of the ego vehicle and set the graphical environment accordingly. A second custom connection to the real-time database is established via a Matlab/Simulink interface to provide the motion of the vehicles controlled by SUMO (either driven by a human IDM model or controlled by the IA via TraCI).
The described communication method is designed for the particular configuration of the employed driving simulator. However, the method can be applied to any generic driving simulator program. In fact, the real-time server can be configured to run with most of the most diffused driving simulator programs and graphical environment. Therefore, the general scheme of synchronization and real-time application can be adapted to any software configuration. The advantages of the proposed scheme are a rigorous real-time simulation and a delay between the different simulations of the order of the larger simulation step used (in the described case, 5 ms). Also, as the real-time scheduler can share the real-time database with any number of workstations, more driving simulators can be added to the network allowing for the inclusion of more than one human in the loop. The additional driving simulators can be of any kind (laptop or desktop workstation, static simulators, dynamic simulators) and, in principle, can run different simulation software. Some integration tests have already been run by adding a second driver by connecting a desktop driving simulator.

Trajectories
Even if SUMO integration step is very short, the trajectories in SUMO do not account for vehicle dynamics and do not appear natural when used to move vehicles in the VI-WorldSim virtual environment. However, the small integration step, corresponding to a 200 HZ sampling of the motion of the vehicles is higher than the frequency of the screen (120 HZ), thus an interpolation between steps is not necessary to obtain a fluid motion of the vehicles. Therefore, the only operation on the trajectories extracted from SUMO is a simple transformation. For each trajectory in the road network loaded in SUMO, a corresponding trajectory is modeled in an auxiliary network. The corresponding trajectory connects the same nodes of the SUMO trajectory, but with a smooth and "natural" path. At each time instant, the position of each vehicle is red in SUMO. Before sending the position to VI-WorldSim, the position is slightly modified to be consistent with the auxiliary and more natural corresponding trajectory. This operation is computationally very fast as it is just a modification of the coordinates according to a correspondence table and allows a much more realistic movements of the vehicles.
Alternatively, in some papers [1] large time increments of the order of 0.2 s are used for traffic simulation. In this case, the trajectories of the vehicles simulated by the MTS have to be interpolated to compensate for the very low update frequency of their position and orientation that prevent a fluid motion. By this approach, larger time increments of the MTS simulation allow for the simulation of larger networks, however larger position delays between the two simulation environments have to be expect. In the present paper, a smaller integration step for the MTS has been preferred to minimize the position delay.

Preliminary tests
The integrated system with SUMO and the dynamic driving simulator has been used to perform some preliminary experimental test with a relatively small panel of twelve drivers. The aim of the tests is to understand if the set up is able to let the testers appreciate different behaviors of the other vehicles. In particular, the target is if by changing the percentage of CAVs in the scenario, the participants could perceive a different traffic flow and if they feel comfortable and safe while driving.
As discussed in Sect. 3, the selected scenario is a three-legged single-lane roundabout. As the scenario is quite small and the maneuver is very quick, participants have been asked to enter the roundabout from all legs and always exit at the second exit. In this way, the driver, while on the circulatory roadway, has to cross one entry and can observe the behavior of the other vehicles when approaching the roundabout. A small queue of about three to five vehicles is present at each leg and the tester has to wait her/his turn to enter the roundabout. For each leg, the drivers repeat the maneuver two times, once with 20% and once with 80% of CAVs. Participants are not informed on the presence of automated cars. The scenario comprises forty vehicles, including the human driven one. After the test, a brief questionnaire is proposed to the participants. The aim of the questionnaire is to understand if the designed scenario permits to the participant to feel some differences in the two traffic situations. The participants are asked to indicate which of the two traffic situations, if any, has a more smooth traffic, feels more safe while driving and which they prefer. The feedback collected from the drivers involved in the test is reported in Table 2.
The results show that most of the driver were able to observe differences in the two situations. Speaking with them after the tests, they also reported that the simulation was quite realistic, the traffic was smooth and the other vehicles interacted correctly with their vehicle. Referring to the answers to the questionnaire, there is a slight trend to prefer the scenario where 80% of traffic actors were automated cars. However, the panel is too small to be able to derive conclusions and more tests will be performed. The main point, however, is that the proposed integration between SUMO and a high end dynamic driving simulator can be effectively employed for the analysis of different traffic scenarios and for the study of the interactions between human drivers and CAVs.

Conclusion
In the present paper a integration between SUMO and a high end dynamic driving simulator has been presented. The proposed scheme of integration relays on a realtime scheduler in order to guarantee a very accurate synchronization between the two simulations and the real-time. By setting an exchanging data frequency of 200 Hz between the SUMO simulation and the driver simulation, a very small delay of 5 ms is present between the two simulations. Vehicle trajectories computed by SUMO are transformed in more natural trajectories by slightly modifying the computed position of the vehicles before sending the data to the driving simulator. The high refreshment frequency of vehicle positions and the correction of the trajectories has allowed for a very smooth and realistic co-simulation.
The proposed integration scheme, even if derived for the actual software configuration used in the paper, is actually general as real-time scheduler can be easily used to interface the most diffused driving simulator software and microscopic traffic simulators. Given the utilization of the real-time scheduler and a shared real-time database, any number of driving simulators (laptop or desktop workstations, static simulators or dynamic simulators), even running different simulation software, can be added to the network.
Preliminary tests, performed on a relatively small panel of twelve drivers, have shown that the proposed approach can be effectively employed for experimentation with microscopic traffic simulators and human in the loop. The participants have been asked to navigate a roundabout in a mixed traffic condition with different percentage of connected automated vehicles. The participants were able to observe differences in the traffic flow for different percentages of connected and automated vehicles. The participants also reported that the simulation was realistic and the vehicles simulated by SUMO interacted with their vehicle as one might expect.

Data availability statement
All available data has been included in the paper.