针对船舶网络数据量大、缓冲队列过长导致的拥塞问题，提出一种强化学习的船舶网络数据传输拥塞控制方法。针对链路拥塞节点的时延和数据流分散特点，建立拥塞问题模型，运用非线性微分法，计算拥塞前后可控和非可控数据流在预设节点处队列长度和数据传输滞留的变化，设定参考阈值，当滞留数值和队列长度超过该值时表明源端发送窗口与接收窗口间链路存在拥塞，按照数据传输平均往返时间确定具体出现拥塞的节点位置。利用强化学习算法，求得经过和未经过拥塞点的数据队列长度变化，根据数据的反馈回报，计算拥塞概率较高链路与正常链路间的窗口差值；根据数据队列长度、流量以及速率值，调节窗口大小补偿值，完成拥塞控制。实验结果表明，实施控制后船舶网络吞吐量增大，节点受限次数下降，控制效果较好。
Aiming at the problem of congestion caused by large amount of data and long buffer queue in ship network, a congestion control method for data transmission in ship network based on reinforcement learning is proposed. According to the characteristics of delay and data flow dispersion of link congestion nodes, a congestion problem model is established. The nonlinear differential method is used to calculate the change of queue length and data transmission detention of controllable and uncontrollable data flows at preset nodes before and after congestion, and set a reference threshold. When the number of detention and queue length exceed this value, it indicates that the link between the source sending window and the receiving window is congested, determine the specific node location with congestion according to the average round-trip time of data transmission. The reinforcement learning algorithm is used to obtain the change of data queue length passing through and not passing through the congestion point. According to the data feedback, the window difference between the link with high congestion probability and the normal link is calculated. According to the data queue length, flow and rate value, the window size compensation value is adjusted to complete congestion control. The experimental results show that the throughput of the ship network increases, the number of nodes restricted decreases, and the control effect is good.
2023,45(3): 165-168 收稿日期：2022-10-18
DOI：10.3404/j.issn.1672-7649.2023.03.032
分类号：TP157
基金项目：内蒙古自治区高等学校科学研究项目（NJZY22180）
作者简介：董洁(1978-),女,硕士,副教授,研究方向为计算机网络、人工智能及计算机应用
