Self-supervised learning is the Future

Self-supervised learning is the secret to ULMFiT (Universal Language Model Fine-tuning ), a natural language processing training approach that dramatically improves the state-of-the-art in this important field. In ULMFiT they start by pretraining a “language model” that is, a model that learns to predict the next word of a sentence. They are not necessarily interested in the language model itself, but it turns out that the model which can complete this task must learn about the nature of language and even a bit about the world in the process of its training.

“I think self-supervised learning is the future. This is what’s going to allow to our AI systems, deep learning system to go to the next level, perhaps learn enough background knowledge about the world by observation, so that some sort of common sense may emerge,” LeCun said in his speech at the AAAI Conference.

One of the key benefits of self-supervised learning is the immense gain in the amount of information outputted by the Artificial Intelligence (AI). In reinforcement learning, training the AI system is performed at scalar level the model receives a single numerical value as reward or punishment for its actions. In self-supervised learning, the AI system predicts a category or a numerical value for each input.

In self-supervised learning, the output improves to a whole image or set of images. “It’s a lot more information. To learn the same amount of knowledge about the world, you will require fewer samples. We must still figure out how the uncertainty problem works, but when the solution emerges, we will have unlocked a key component of the future of AI. “If artificial intelligence is a cake, self-supervised learning is the bulk of the cake,” LeCun says. Yann André LeCun is a French computer scientist working primarily in the fields of machine learning, computer vision, mobile robotics, and computational neuroscience.

E-SPIN as being a value integrator to assist enterprise customers to implement various digital transformation technology, including learning management system (LMS), e-learning system and portal, include self-supervised learning, machine learning (ML), deep learning (DL) to accelerate their speed, scale and scope objective in related to digital transformation. From e-learning systems and portal infrastructure, application and performance monitoring, continuous security protection. E-SPIN established in 2005, in the business of supply, consultancy, integration, training and maintenance of established and emerging technology and systems for enterprise customers and government agencies. Feel free to contact E-SPIN for your project and operation requirements.