Beyond AI Logo

幼児の知識獲得メカニズムを活用したAI Gaze-based minimal virtual reality paradigm for tracking developing sensitivity to dyadic interactions



研究リーダー Project Leader
辻 晶 講師 Sho Tsuji Assistant Professor
東京大学 ニューロインテリジェンス国際研究機構 International Research Center for Neurointelligence, The University of Tokyo





赤ちゃんの驚くべき言語学習能力 The Amazing Language Learning Ability of Babies


Human infants’ ability to learn language is an amazing skill that is unrivaled by existing AI systems.
In order for AI to learn and use language in the same way as humans do, it requires massive amounts of data. On the other hand, human infants learn to speak naturally as they grow up. Understanding the mechanisms behind infants’ amazing language learning ability and applying these to artificial intelligence models would enable us to take current AI approaches to the next level. Such neuro- and cognitive science inspired approaches to artificial intelligence are once again attracting interest around the world, and among those proposals to design computational models based on how infants’ language acquisition process are spotlighted as cutting-edge initiatives in this area. However, the complex mechanisms underlying infants’ amazing learning abilities have yet to be understood adequately, as they are to date very difficult to imitate artificially.


Details of Project

赤ちゃんの言語習得の謎 The Mystery of Infants’ Language Learning Ability

赤ちゃんはどのようにして、驚くべきスピードと効率で言語を習得しているのでしょうか? 赤ちゃんの素晴らしい学習能力には、母親をはじめとする赤ちゃんの周りの環境、すなわち社会的な環境要素が極めて重要な役割を担っていることが知られています。しかし、それら社会的環境要素が実際にどのように学習に影響を与えているのか、その複雑な仕組みはよく分かっていません。プロジェクトチームは、赤ちゃんの言語習得の仕組み、特に学習における社会的相互作用の役割を理解することで、乳幼児期の言語発達に関する理論構築を目指しています。

So how do babies actually learn a language with such incredible speed and efficiency? It is a known fact that infants’ social environment, for instance the mother, plays a crucial role in the amazing learning skills of babies. However, we have yet to understand the complex mechanism of how such social environmental factors actually influence learning. Our project team aims to establish a theory of early language development by understanding the mechanism of language learning in babies, in particular the role of social interaction in learning.

双方向視線追跡法(two-way gaze-contingent eye-tracking method) Two-Way Gaze-Contingent Eye-Tracking Method

本研究では、乳幼児の学習における社会的相互作用の役割を深く理解するために、双方向視線追跡法(two-way gaze-contingent eye-tracking method)に基づいた新たな実験方法を独自に確立します。この双方向視線追跡法は、被験者である赤ちゃんと大人に対し、双方の視線をそれぞれのスクリーン上で互いに追従することでインタラクションを行える実験システムです。この構成によって、二重相互作用が学習に及ぼす影響について、その複雑なダイナミクスを維持しながらも、二次元の管理可能な形態で実験・評価することが可能になりました。また、同一システムで様々な種類の学習シーケンスを評価できる点でも優れています。例えば、このシステムを使うことで、視線誘導による対話的学習と非対話的学習の比較実験を行ったり、クロスモーダルな音と物体の関連付け学習を行ったりといったように、様々な実験が実施可能です。

In this project, we established a unique new experimental method based on two-way gaze-contingent eye-tracking in order to gain an in-depth understanding of the role of social interaction in infant learning. Our experimental system simultaneously tracks the gaze of both the infant participant and an adult interaction partner on their respective screens, thereby recording their gaze interaction. This configuration enables the experimental assessment of the effects of dyadic interaction on learning while maintaining their complex dynamics, but reduced to a two-dimensional manageable format. Moreover, we will be able to evaluate various types of learning sequences with this set-up. For example, we can compare the effect of interactive and non-interactive scenarios, and the effects of such scenarios on learning of cross-modal associations between sounds and objects.


Values / Hopes

本研究プロジェクトが切り開く未来の可能性 Future Possibilities Created by This Project


The experimental data generated by this project is expected to not only provide clues for elucidating the dynamics of social interaction in infant language learning but serve as the foundation of agent-based AI systems for modeling and understanding the dynamic characteristics of interactive sequences. Moving forward, it may help establish innovative educational methods for young children and to apply them to interactive digital solutions such as video chat systems.
More speculatively, the unique contribution of this project would be to put the shortcomings of current AI systems into sharper relief: current AI methods rely on decoupled forms of learning, whereas human capacities depend on interactive forms of learning. Accordingly, the project could provide insights that pave the way toward the design of a not only developmentally-inspired, but more interaction-centered form of AI.