Hot Paper - High performance "non-local" generic face reconstruction model using the lightweight Speckle-Transformer (SpT) UNet

High performance "non-local" generic face reconstruction model using the lightweight Speckle-Transformer (SpT) UNet

使用轻量级散斑变压器（SpT）UNet的高性能“非局部”通用人脸重建模型
軽量スペックルパターン変圧器（SpT）UNetを用いた高性能「非局所」汎用顔再建モデル
경량 플레어 변압기(SpT) UNet을 사용한 고성능 "비부분" 범용 얼굴 재구성 모델
Modelo universal de reconstrucción facial "no local" de alto rendimiento con transformador de manchas ligeras (spt) unet
Modèle de reconstruction de visage universel « non localisé » haute performance utilisant unet, un transformateur léger (SPT)
Высокопроизводительная "локальная" универсальная модель восстановления лица с использованием легкого трансформатора рассеянного пятна (SpT) UNET

Yangyundou Wang 王杨云逗 ¹ ², Hao Wang 王昊 ³, Min Gu 顾敏 ¹ ²

¹ Institute of Photonic Chips, University of Shanghai for Science and Technology, Shanghai 200093, China
中国上海上海理工大学光子芯片研究院
² Centre for Artificial-Intelligence Nanophotonics, School of Optical-Electrical and Computer Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China
中国上海上海理工大学光电信息与计算机工程学院人工智能纳米光子学中心
³ School of Optical-Electrical and Computer Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China
中国上海上海理工大学光电信息与计算机工程学院

https://doi.org/10.29026/oea.2023.220049

https://www.oejournal.org/article/doi/10.29026/oea.2023.220049

Opto-Electronic Advances, 25 February 2023

Abstract

Significant progress has been made in computational imaging (CI), in which deep convolutional neural networks (CNNs) have demonstrated that sparse speckle patterns can be reconstructed. However, due to the limited “local” kernel size of the convolutional operator, for the spatially dense patterns, such as the generic face images, the performance of CNNs is limited.

Here, we propose a “non-local” model, termed the Speckle-Transformer (SpT) UNet, for speckle feature extraction of generic face images. It is worth noting that the lightweight SpT UNet reveals a high efficiency and strong comparative performance with Pearson Correlation Coefficient (PCC), and structural similarity measure (SSIM) exceeding 0.989, and 0.950, respectively.

Reviews and Discussions