Reinforcement Learning Technique in Multiple Motorway Access Control Strategy Design

Abstract

An appropriately designed motorway access control can decrease the total travel time spent in the system up to 30% and consequently increase the merging operations safety. To date, implemented traffic responsive motorway access control systems have been of local or regulatory type and not truly adaptive in the real sense of the meaning. Hence, traffic flow can be influenced positively by numerous intelligent transportation system (ITS) techniques. In this paper a contemporary approach is presented. It considers the design philosophy of an optimal and adaptive closed-loop multiple motorway access control strategy. The methodology proposed uses the artificial intelligence technique - known as reinforcement learning (RL) with multiple agents, and applies the Q-learning algorithm. One segment of the motorway network with three lanes in each direction and three motorway entries was designed. The detectors and traffic signals were placed at the entries (ramps). Traffic flows and traffic occupancy on the main line as well as the traffic demand on the motorway entries were taken as input model variables. The output variables referred to the travel speed on the corridor, the total travel time, and the total stop time. VISSIM micro-simulator and direct programming of the simulator functions were used in order to implement the RL technique. The peak hour was chosen for the time of simulation. The model was tested in two phases. Its effectiveness was compared to ALINEA. It was observed that the proposed strategy was capable of responding both to dynamic sensory inputs from the environment and to dynamically changing environment. The model of the environment and supervision were not required. The control policy changed as response to the inherent system characteristic changes. It was confirmed that the strategy was truly adaptive and real-time responsive to the traffic demand on the corridor. KEY WORDS: motorway access, traffic flows, control, strategy, artificial intelligence, Q-Learning, simulation