Image transmission over constrained networks using segmentation and semantic communications

Abstract

This thesis addresses the problem of efficient image transmission in constrained bandwidth environments, where traditional compression methods often fail to preserve meaningful semantic information at very low bitrates. Instead of sending full images, we explore a semantic communication approach that focuses on sending only the most important information that is relevant for reconstruction. We propose a framework based on a tri-modal decomposition of the original image into three representations: a segmentation map, a structural edge map, and a textual description via a vision- language model (VLM). These components are extracted on the edge or so-called sender device (e.g., Internet of Things (IoT) camera), and then transmitted over a low-bandwidth channel. At the receiver side we do the reconstruction with a diffusion-based model with ControlNet modules. We evaluate the proposed method on the Cityscapes dataset and compare it with a recent work that also does image decomposition. The results show that our approach can reduce the transmitted data by more than 99% while still preserving key semantic properties of the original image. Both human evaluation and VLM-based evaluation show that the reconstructed images better maintain object positions and their number when compared to the baseline. We also performed an ablation study that shows that each modality contributes to the final reconstruction quality. Finally, we demonstrate that the sender-side pipeline can be deployed on resource- constrained devices, making the approach practical for real-world scenarios. Overall, this work shows that combining multiple semantic representations with generative models can be a good alternative for low-bitrate image transmission.