From Two to One: A New Scene Text Recognizer with Visual Language Modeling Network
2 부터 1 까지:새로운 시각 언어 모델 링 네트워크 를 바탕 으로 하 는 장면 텍스트 식별 기
De dos a uno: un nuevo identificador de texto de escena basado en la red de modelado de lenguaje visual
De deux à un: un nouvel identificateur de texte de scène basé sur un réseau de modélisation de langage visuel
от 2 до 1: новая модель, основанная на визуальном языке
Yuxin Wang 王裕鑫 ¹, Hongtao Xie 谢洪涛 ¹, Shancheng Fang ¹, Jing Wang ², Shenggao Zhu ², Yongdong Zhang 张勇东 ¹
¹ University of Science and Technology of China
² Huawei Cloud & AI
arXiv, 22 August 2021

In this paper, we abandon the dominant complex language model and rethink the linguistic learning process in the scene text recognition. Different from previous methods considering the visual and linguistic information in two separate structures, we propose a Visual Language Modeling Network (VisionLAN), which views the visual and linguistic information as a union by directly enduing the vision model with language capability. Specially, we introduce the text recognition of character-wise occluded feature maps in the training stage. Such operation guides the vision model to use not only the visual texture of characters, but also the linguistic information in visual context for recognition when the visual cues are confused (e.g. occlusion, noise, etc.).

As the linguistic information is acquired along with visual features without the need of extra language model, VisionLAN significantly improves the speed by 39% and adaptively considers the linguistic information to enhance the visual features for accurate recognition. Furthermore, an Occlusion Scene Text (OST) dataset is proposed to evaluate the performance on the case of missing character-wise visual cues. The state of-the-art results on several benchmarks prove our effectiveness.
Reviews and Discussions
An overview on the principle of inkjet printing technique and its application in micro-display for augmented/virtual realities
Nonlinear optics with structured light
Femtosecond laser-induced periodic structures: mechanisms, techniques, and applications
Giant and light modifiable third-order optical nonlinearity in a free-standing h-BN film
New approach for the digital reconstruction of complex mine faults and its application in mining
p62/SQSTM1 Participates in the Innate Immune Response of Macrophages Against Candida albicans Infection
Configurable topological beam splitting via antichiral gyromagnetic photonic crystal
Single-molecule optoelectronic devices: physical mechanism and beyond
Functional nonlinear optical nanoparticles synthesized by laser ablation
The m6A methylation regulates gonadal sex differentiation in chicken embryo
A new species in the genus Synanthedon (Lepidoptera: Sesiidae) from China
A review on the forward osmosis applications and fouling control strategies for wastewater treatment

Previous Article                                Next Article
Copyright © Hot Paper