增强许可辅助访问(eLAA)/MulteFire能够在免授权频谱上同时支持上行（Uplink, UL）下行（Downlink, DL）传输。在免授权频段上，eLAA/MulteFire采用LBT（Listen Before Talk，LBT）机制接入信道，但该机制并不能避免与隐藏节点之间的传输冲突，从而导致时延加长或WiFi丢包。合理灵活的帧配置将减少这种传输冲突，降低WiFi传输时延，提高信道的接入概率。本文提出一种基于Q学习（Q-learning，QL）的动态上下行帧配置(DFC)机制。在这个机制中，基站被视为一个智能体，并将吞吐时间和公平性的不同组合定义为智能体状态，不同帧配置定义为智能体行为。智能体基于获取到的临近基站的帧配置和WiFi的平均传输时长，学习得到最优的帧配置。仿真结果表明，提出的基于QL的帧配置方法在确定一定公平性的前提下，极大提高了免授权频段的信道接入概率和吞吐量，降低了传输时延。
Enhanced License Assisted Access (eLAA)/MulteFire can support both Uplink(UL) and Downlink(DL) transmission on the unlicensed spectrum. In the unlicensed frequency band, eLAA/MulteFire uses the Listen Before Talk (LBT) mechanism to access the channel. But this mechanism cannot avoid the transmission collision of the eLAA and hidden WiFi Access Points (WAPs), and further lead to increase the transmission delay and loss of WiFi packets. Reasonable and flexible frame configuration will minimize the transmission conflict, reduce the WiFi transmission delay, and improve the access probability of the channel. Therefore, this paper proposes a Q-learning (QL) based UL/DL Dynamic Frame Configuration (DFC) mechanism. In the mechanism, base station is regarded as an agent, and different combinations of throughput time and fairness are defined as agent state. The different frame configurations are defined as agent behavior. The agent learns the optimal frame configuration based on the acquired frame configuration of the adjacent base BS and the average transmission time of WiFi. Simulation results show that the QL based frame configuration method can greatly improve the access probability and throughput of the unlicensed spectrum and reduce the transmission delay under the premise of certain fairness.
2023,45(13): 130-135 收稿日期：2022-12-01
DOI：10.3404/j.issn.1672-7649.2023.13.026
分类号：TN919.72
作者简介：孙长龙(1979-)，男，硕士，高级工程师，研究方向为通信及指挥自动化
参考文献：
[1] MULTEFIRE A. Understanding MulteFire’s Radio Link [OL]. https://www. multefire. org/, 2017.
[2] 3GPP TR 36.889 V13.0. 0, Feasibility Study on Licensed-Assisted Access to Unlicened Spectrum[S]. 2015.
[3] 3GPP TS 36.211 V14.0. 0, TSGRAN; E-UTRA; Physical channels and modulation[S]. 2017.
[4] CHENG M, WANG Y, HWANG W, WU Y and LIN C. Adaptive adjustment of TDD uplink-downlink configuration based on cluster classification in Beyond LTE Heterogeneous networks[C]//International Conference on Applied System Innovation (ICASI), 2017: 1312–1315.
[5] LEE K, PARK Y, NA M, WANG H and HONG D. Aligned Reverse Frame Structure for Interference Mitigation in Dynamic TDD Systems[J]. IEEE Transactions on Wireless Communications, 2017, 16(10): 6967–6978
[6] LIN Yi-Ting, CHAO Cheng-Chih and WEI Hung-Yu. Dynamic TDD interference mitigation by using Soft Reconfiguration [C]//International Conference on Heterogeneous Networking for Quality, Reliability, Security and Robustness (QSHINE), Taipei, China, 2015: 352–357.
[7] MULAY A K, BASWADE A M, TAMMA B R and FRANKLIN A A. DFC: Dynamic UL–DL Frame Configuration Mechanism for Improving Channel Access in eLAA[J]. IEEE Networking Letters, 2019, 1(3): 116–119
[8] XU Y, WANG J, WU Q, ANPALAGAN A. and YAO Y. Opportunistic Spectrum Access in Cognitive Radio Networks: Global Optimization Using Local Interaction Games[J]. IEEE Journal of Selected Topics in Signal Processing, 2012, 6(2): 180–194
[9] WANG Y, TAO M. Dynamic uplink/downlink configuration using Q-learning in femtocell networks[C]//IEEE/CIC International Conference on Communications in China (ICCC), Shanghai, China, 2014: 53–58.
[10] KUSHWAHA H, KOTAGI V J and MURTHY C Siva Ram . A Novel Reinforcement Learning Based Adaptive Optimization of LTE-TDD Configurations for LTE-U/WiFi Coexistence[C]//IEEE Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), Istanbul, 2019: 1–7.
[11] ESSWIE A. A, PEDERSON K I and MOGENSEN P E. Semi-Static Radio Frame Configuration for URLLC Deployments in 5G Macro TDD Networks[C]//IEEE Wireless Communications and Networking Conference (WCNC), 2020: 1–6.
[12] DZIEDZIC A, SATHYA V, ROCHMAN M I, GHOSH M and KRISHNAN S. Machine Learning Enabled Spectrum Sharing in Dense LTE-U/Wi-Fi Coexistence Scenarios[J]. IEEE Open Journal of Vehicular Technology, 2020, 1: 173–189.
[13] JOHN I, KAMANCHI C and BHATNAGAR S. Generalized Speedy Q-Learning[J]. IEEE Control Systems Letters, 2020, 4(3): 524–529
[14] LE D Van, THAM C. A deep reinforcement learning based offloading scheme in ad-hoc mobile clouds[C]//IEEE INFOCOM 2018 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), 2018: 760–765.
