Multimodal Fusion for NextG V2X Communications

The V2X communication spans a variety of applications such as collision avoidance safety systems, traffic signal timing, safety alerts to pedestrians, and real-time traffic. In all of the above applications, the communication system must meet the requirement of either low latency or high data rate for safety or quality of service reasons. This motivates the idea of using high band mmWave frequencies for the V2X application to ensure near real-time feedback and Gbps data rates. However, the mmWave band suffers from the high overhead associated with the initial beam alignment step due to directional transmission. We propose using side information from sensor inputs such as GPS, camera, and LiDAR to assist the beam initialization. We publish different datasets to the research community to pave the way for this new interesting approach.

Fig 1:  Four categories of V2X application including, vehicle-to-Vehicle (V2V), Vehicle-to-Infrastructure (V2I), Vehicle-to-Pedestrian (V2P) and Vehicle-to-Network (V2N).

Download our datasets:
Please use below links to download the datasets. These datasets were used for the papers mentioned in the "Venue" column. Any use of these dataset, which results in an academic publication or other publication that includes a bibliography, should contain a citation to our papers.

Dataset Venue Raw Data Processed Data Portable Data[*.HDF5] Description
FLASH Infocom 2022 Link Link --
B. Salehi, J. Gu, D. Roy, and K. Chowdhury, FLASH: Federated Learning for Automated Selection of High-band mmWave Sectors, in IEEE International Conference on Computer Communications (INFOCOM), May 2022, [Accepted].
e-FLASH -- Link Link Upcoming
J. Gu, B. Salehi, D. Roy, and K. Chowdhury, Multimodality in 5G MIMO Beam Selection using Deep Learning: Datasets and Challenges. [Under Review]