During this research period, we investigated developing a deep learning model for hyperspectral image (HSI) classification tasks based on the popular Transformer architecture. By identifying shortcomings in the existing Transformer model, we proposed a new perspective for integrating Recurrent Neural Networks (RNNs) with Transformers for complementarity. The proposal consisted of three components: 1) RNN-Transformer encoder, 2) Soft Masked Spectral-Spatial-Based Self-Attention (SMSA), and 3) Multiscanning Fusion Transformer. In this case, the transaction paper titled “Multiscanning-based RNN-Transformer for Hyperspectral Image Classification” was accepted by the IEEE Transactions on Geoscience and Remote Sensing (TGRS). Compared with baseline methods, this work achieved a 6%~11% accuracy improvement. Moreover, compared with other state-of-the-art methods, our work obtained a 1%~5% accuracy improvement and saved almost 50% of processing time with almost 40% model size reduction.

Meanwhile, the idea of the “multiscanning strategy” was extended into another field, image compression, which was studied by our coworkers. The paper, titled “Learned Image Compression with Multi-Scan Based Channel Fusion,” was accepted by the International Conference on Image Processing (ICIP) in 2023. This work verified the effectiveness of the “multiscanning strategy” and showed the general attribute of this idea. We hoped to further develop this concept into other research fields.

Furthermore, to facilitate the multiscanning strategy into a 3D version, we proposed a cubed 3D-multiscanning strategy. The manuscript, titled "Segmented Recurrent Transformer with Cubed 3D Multiscanning Strategy for Hyperspectral Image Classification", is accepted at 26 March, by IEEE Transactions on Geoscience and Remote Sensing (TGRS) .