基于逆强化学习的电动汽车出行规划方法研究

Research on Electric Vehicle Travel Planning Based on Inverse Reinforcement Learning

关键词:  逆强化学习  电动汽车  出行规划  Dueling DQN  部分充电策略

Key words:  inverse reinforcement learning  electric vehicle  travel planning  Dueling DQN  partial charging strategies

摘要:

随着电动汽车的普及，对电动汽车出行规划问题的研究显得尤为重要。有别于路径规划，出行规划既需要考虑路径问题又需要考虑充电问题。本文提出了一种基于逆强化学习（Inverse Reinforcement Learning，IRL）的电动汽车出行规划（Electric Vehicle Travel Planning，EVTP）方法，有效地为电动汽车用户规划一条兼顾行驶路径短以及充电时间短的可达路径。将Dijkstra算法进行改进得到考虑充电行为的最短路径作为专家示例输入到逆强化学习算法中；利用逆强化学习算法得到兼顾行走与充电的奖励；在学习策略上，采用Dueling DQN算法高效更新Q值，提升学习性能；采用部分充电策略以及分段充电策略，提升充电效率并使研究更接近真实情况。通过对模型的工作性能和结果进行详细分析，并结合基准方法进行对比，结果表明，基于逆强化学习的电动汽车出行规划方法在行驶时间与充电时间两方面都有较好的性能，且具备很好的迁移性。

Abstract:

With the popularization of electric vehicles, the research on travel planning of electric vehicles is particularly important.Travel planning, which is different from path planning, needs to consider both path and charging problems.This article proposes a travel planning method for Electric Vehicles Travel Planning (EVTP) based on Inverse Reinforcement Learning (IRL), which can effectively plan an accessible path for electric vehicle users with a short driving path and a short charging time.The Dijkstra algorithm was improved to obtain the shortest path considering the charging behavior, which was input into the inverse reinforcement learning algorithm as an expert example.The inverse reinforcement learning algorithm was used to obtain both walking and charging rewards.In learning strategy, Dueling DQN algorithm was used to update Q-value efficiently and improve learning performance.Partial charging strategies and segmented charging strategies were adopted to improve the charging efficiency and make the research closer to the real situation.The working performance and results of the model were analyzed in detail and compared with the benchmark method.The results show that the travel planning method of electric vehicles based on inverse reinforcement learning has better performance in both driving time and charging time.Meanwhile, our method has very good performance in portability.