BabyMind
Infant-Mimic Neurocognitive Developmental Machine Learning from Interaction Experience with Real World
BabyMind is a large-scale collaborative research initiative led by Prof. Byoung-Tak Zhang at SNU Biointelligence Lab, aiming to build infant-mimic neurocognitive AI that learns from interaction with the real world the way human infants do.
Overview
Human infants acquire knowledge at a remarkable pace — learning to perceive objects, understand language, navigate space, and interact socially — all from sparse, noisy, multimodal experience and without explicit supervision. Modern machine learning systems, by contrast, require vast labeled datasets and still struggle to generalize beyond their training distribution.
The BabyMind project asks a fundamental question: can we build AI systems that learn the way babies do? The project pursues this goal by studying the computational principles underlying infant neurocognitive development and translating them into novel machine learning architectures. Core themes include curiosity-driven active learning from scarce data, complementary learning systems (fast episodic memory paired with slow semantic consolidation), developmental stage progression from sensorimotor grounding to symbolic reasoning, and multimodal grounding across vision, audio, and language.
Research activities span both physical robotic platforms and virtual-reality-based environments, enabling controlled developmental experiments that mirror the kinds of structured experiences available to human infants.
Research Team
Principal Investigator
- Prof. Byoung-Tak Zhang — Seoul National University (SNU Biointelligence Lab)
Co-Investigators and Collaborating Faculty
| Professor | Affiliation / Focus |
|---|---|
| Eon-Suk Ko | Language and speech |
| Nojun Kwak | Computer vision and semi-supervised learning |
| Hyoun Jin Kim | Robotics and mobile manipulation |
| Chong Woo Park | Human-computer interaction |
| Hyun Oh Song | Reinforcement learning and exploration |
| Woo-Young Ahn | Computational psychiatry and decision-making |
| Song Joo Oh | Cognitive vision science |
| Sung Roh Yoon | Video and sequence learning |
| Kyogu Lee | Music and audio intelligence |
| Youngki Lee | Wearable and mobile sensing |
| In Ah Lee | Language understanding |
| Jung Woo Lee | Neural imaging and neuroimaging ML |
| Hyung Bo Shim | Embedded and hardware AI |
| Choong Wan Woo | Affective and social neuroscience |
Contact: babymind-bi@bi.snu.ac.kr
Research Structure
The project was organized into four specialized sub-teams covering the full developmental stack:
| Team | Focus |
|---|---|
| Knowledge Integration | Multimodal memory, concept learning, and hypergraph-based knowledge representation |
| Vision-Audio Processing | Perceptual grounding from visual and auditory streams; face-speech synthesis |
| Language-Emotion Analysis | Pragmatic and affective language understanding; child-directed speech modeling |
| Robot-Behavior Development | Embodied learning in real and virtual-reality environments; safe policy transfer |
Key Projects and Tools
- COGNIA — cognitive architecture framework for integrating perception and memory (open source on GitHub)
- VECA — virtual environment for cognitive agent development (open source on GitHub)
- Korean Child-Language Corpus — Ko corpus collected via CHILDES TalkBank for studying Korean child language acquisition
- LENA Evaluation API — tools for evaluating child language environment metrics in Korean
NeurIPS 2020 Workshop
A major public-facing output was the NeurIPS 2020 Workshop “BabyMind: How Babies Learn and How Machines Can Imitate”, co-organized by the project. The workshop brought together researchers in cognitive developmental science, computational neuroscience, developmental robotics, and machine learning to discuss infant-inspired AI.
Publications
Selected publications produced under the BabyMind project:
2020
- Eun-Sol Kim, Woo-Young Kang, Yu-Jung Heo, Kyoung-Woon On, Byoung-Tak Zhang. “Hypergraph Attention Networks for Multimodal Learning.” CVPR 2020.
- Taehyeong Kim, Injune Hwang, Gi-Cheon Kang, Won-Seok Choi, Hyunseo Kim, Byoung-Tak Zhang. “Label Propagation Adaptive Resonance Theory for Semi-supervised Continuous Learning.” ICASSP 2020.
- Wansoo Kim, Kyogu Lee. “Digital Watermarking for Protecting Audio Classification Datasets.” ICASSP 2020.
- Kyoung-Woon On, Eun-Sol Kim, Yu-Jung Heo, Byoung-Tak Zhang. “Cut-Based Graph Learning Networks to Discover Compositional Structure of Sequential Video Data.” AAAI 2020.
- Jaemyung Kim, Chanwoo Park, Hyungjin Kim, Jungwoo Lee. “REST: Performance Improvement of a Black Box Model via RL Based Spatial Transformation.” AAAI 2020.
- Hyeong-Seok Choi, Changdae Park, Kyogu Lee. “From Inference to Generation: End-to-end Fully Self-supervised Generation of Human Face from Speech.” ICLR 2020.
- J. Hyeon Park, Jigang Kim, Younseok Jang, Inkyu Jang, H. Jin Kim. “Learning Transformable and Plannable se(3) Features for Scene Imitation of a Mobile Service Robot.” RA-L 2020.
- Lada Kohoutová, Juyeon Heo, Sungmin Cha, Sungwoo Lee, Taesup Moon, Tor D. Wager, Choong-Wan Woo. “Toward a Unified Framework for Interpreting Machine-Learning Models in Neuroimaging.” Nature Protocols 2020.
- Songjoo Oh. “The Stone-Base Illusion.” Vision Research 2020.
- Jisoo Jeong, Seungeui Lee, Nojun Kwak. “Self-Training Using Selection Network for Semi-Supervised Learning.” ICPRAM 2020.
- Hyojin Park, Lars Sjösund, Youngjoon Yoo, Nicolas Monet, Jihwan Bang, Nojun Kwak. “SINet: Extreme Lightweight Portrait Segmentation Networks with Spatial Squeeze Module and Information Blocking Decoder.” WACV 2020.
- Nathaniel Haines et al. “Anxiety Predicts Diminished Preference for Immediate Rewards in Trait-Impulsive Individuals.” Clinical Psychological Science 2020.
- Jaeyeong Yang, Mark A. Pitt, Woo-Young Ahn, Jay I Myung. “ADOpy: A Python Package for Adaptive Design Optimization.” Behavior Research Methods 2020.
2019
- Jisoo Jeong, Seungeui Lee, Jeesoo Kim, Nojun Kwak. “Consistency-based Semi-supervised Learning for Object Detection.” NeurIPS 2019.
- Gi-Cheon Kang, Jaeseo Lim, Byoung-Tak Zhang. “Dual Attention Networks for Visual Reference Resolution in Visual Dialog.” EMNLP 2019.
- Hwiyeol Jo, Ceyda Cinarel. “Delta-training: Simple Semi-Supervised Text Classification using Pretrained Word Embeddings.” EMNLP 2019.
- Jungbeom Lee, Eunji Kim, Sungmin Lee, Jangho Lee, Sungroh Yoon. “Frame-to-Frame Aggregation of Active Regions in Web Videos for Weakly Supervised Semantic Segmentation.” ICCV 2019.
- Jigang Kim, Seungwon Choi, Hyoun Jin Kim. “Fast and Safe Policy Adaptation via Alignment-based Transfer.” IROS 2019.
- Juheon Lee, Hyeong-Seok Choi, Chang-Bin Jeon, Junghyun Koo, Kyogu Lee. “Adversarially Trained End-to-end Korean Singing Voice Synthesis System.” INTERSPEECH 2019.
- Jie Hwan Lee, Hyeong-Seok Choi, Kyogu Lee. “Audio Query-Based Music Source Separation.” ISMIR 2019.
- Hyoungseok Kim, Jaekyeom Kim, Yeonwoo Jeong, Sergey Levine, Hyun Oh Song. “EMI: Exploration with Mutual Information.” ICML 2019.
- Seungyong Moon, Gaon An, Hyun Oh Song. “Parsimonious Black-Box Adversarial Attacks via Efficient Combinatorial Optimization.” ICML 2019.
- Sungjae Cho, Jaeseo Lim, Chris Hickey, Byoung-Tak Zhang. “Problem Difficulty in Arithmetic Cognition: Humans and Connectionist Models.” CogSci 2019.
- Cholmin Kang, Hyunwoo Jung, Youngki Lee. “Towards Machine Learning with Zero Real-World Data.” WearSys 2019.
- Kyoung-Woon On, Eun-Sol Kim, Yu-Jung Heo, Byoung-Tak Zhang. “Compositional Structure Learning for Sequential Video Data.” ICML 2019 Workshop on Learning and Reasoning.
- Chung-Yeon Lee, Hyundo Lee, Injune Hwang, Byoung-Tak Zhang. “Spatial Perception by Object-Aware Visual Scene Representation.” ICCV 2019 Workshop on Deep Learning for Visual SLAM.