Effective and Efficient Self-supervised Masked Model Based on Mixed Feature Training

Clicks: 5
ID: 283850
Free PDF
Article Quality & Performance Metrics
Overall Quality
Not rated
Combines reader engagement with the AI quality analysis. This article has not been analysed, so there is no overall score — reader engagement is measured and shown alongside.
AI Quality Assessment
Not analyzed
Readership in this journal
Star

Ranked #300 of 309 articles by views in Frontiers in pharmacology

Most read Least read

Bar heights use a square-root scale. Only the 120 most-read articles are drawn; the journal has 309 in total.

Mint this article as an NFT
Not yet minted

Create a permanent, verifiable on-chain record of this article on the Scimatic Network. The NFT is held in your Journament account, and you can withdraw it to your own wallet at any time.

5 SUSD one-off · no wallet required
Sign in to mint
Abstract
Under the influence of Masked Language Modeling (MLM), Masked Image Modeling (MIM) employs an attention mechanism to perform masked training on images. However, processing a single image requires numerous iterations and substantial computational resources to reconstruct the masked regions, resulting in high computational complexity and significant time costs. To address this issue, we propose an Effective and Efficient self-supervised Masked model based on Mixed feature training (EESMM). First, we stack two images for encoding and input the fused features into the network, which not only reduces computational complexity but also enables the learning of more features. Second, during decoding, we obtain the decoding features corresponding to the original images based on the decoding features of the two input original images and the mixed images, and then construct a corresponding loss function to enhance feature representation. EESMM significantly reduces pre-training time without sacrificing accuracy, achieving 83% accuracy on ImageNet in just 363 hours using four V100 GPUs—only one-tenth of the training time required by SimMIM. This validates that the method can substantially accelerate the pre-training process without noticeable performance degradation.
Reference Key
imported_1760285510_68ebd346e4236 Use this key to autocite in the manuscript while using SciMatic Manuscript Manager or Thesis Manager
Authors Cai, Chunliu
Journal Frontiers in pharmacology
Year Year not found
DOI
10.3389/fnbot.2025.1705970
URL
https://www.frontiersin.org/journals/neurorobotics/articles/10.3389/fnbot.2025.1705970/full
Keywords Keywords not found

Citations

No citations found. To add a citation, contact the admin at info@scimatic.org

Comments

No comments yet. Be the first to comment on this article.