SNU Biointelligence Lab

Research

Our research focuses on “biointelligence” i.e. the study of artificial intelligence on the basis of biological and bio-inspired information technologies, and its application to real world problems.

Preprint

Continual Vision-and-Language Navigation
Continual Vision-and-Language Navigation
Seongjun Jeong, Gi-Cheon Kang, Seongho Choi, Junghyun Kim, Byoung-Tak Zhang
preprint
Paper
Active Neural 3D Reconstruction with Colorized Surface Voxel-Based View Selection
Active Neural 3D Reconstruction with Colorized Surface Voxel-Based View Selection
Hyunseo Kim, Hyeonseo Yang, Taekyung Kim, Yoonsung Kim, Jin-Hwa Kim, Byoung-Tak Zhang
preprint
Paper
Exploring Ordinal Bias in Action Recognition for Instructional Videos
Exploring Ordinal Bias in Action Recognition for Instructional Videos
Joochan Kim, Minjoon Jung, Byoung-Tak Zhang
preprint
Paper
Variational Online Mirror Descent for Robust Learning in Schrödinger Bridge
Variational Online Mirror Descent for Robust Learning in Schrödinger Bridge
Dong-Sig Han, Jaein Kim, Hee Bin Yoo, Byoung-Tak Zhang
preprint
Paper
From Sparse to Dense: Toddler-Inspired Reward Transition in Goal-Oriented Reinforcement Learning
From Sparse to Dense: Toddler-Inspired Reward Transition in Goal-Oriented Reinforcement Learning
Junseok Park, Hyeonseo Yang, Min Whoo Lee, Won-Seok Choi, Minsu Lee, Byoung-Tak Zhang
preprint
Paper

2025

CLIP-RT: Learning Language-Conditioned Robotic Policies from Natural Language Supervision
CLIP-RT: Learning Language-Conditioned Robotic Policies from Natural Language Supervision
Gi-Cheon Kang, Junghyun Kim, Kyuhwan Shim, Jun Ki Lee, Byoung-Tak Zhang
RSS 2025
Home
Paper
Code
Mixture of Action Expert Embeddings: Multi-Task ACT
Mixture of Action Expert Embeddings: Multi-Task ACT
Suhyung Choi, Youngseok Joo, Jun Ki Lee, Byoung-Tak Zhang
AAAI 2025 GenPlan Workshop
Paper
On the Consistency of Video Large Language Models in Temporal Comprehension
On the Consistency of Video Large Language Models in Temporal Comprehension
Minjoon Jung, Junbin Xiao, Byoung-Tak Zhang, Angela Yao
CVPR 2025
Paper
Code
Socratic planner: Inquiry-based zero-shot planning for embodied instruction following
Socratic planner: Inquiry-based zero-shot planning for embodied instruction following
Suyeon Shin, Sujin Jeon, Jyunghyun Kim, Gi-Cheon Kang, Byoung-Tak Zhang
ICRA 2025
Paper
Truncated Gaussian Policy for Debiased Continuous Control
Truncated Gaussian Policy for Debiased Continuous Control
Ganghun Lee, Minji Kim, Minsu Lee, Byoung-Tak Zhang
AAAI 2025
Paper
Background-aware moment detection for video moment retrieval
Background-aware moment detection for video moment retrieval
Minjoon Jung, Youwon Jang, Seongho Choi, Joochan Kim, Jin-Hwa Kim, Byoung-Tak Zhang
WACV 2025
Paper
Code
Relaxed HLP: A Metric for Flexible High-Level Plan Evaluation in Embodied Environments
Relaxed HLP: A Metric for Flexible High-Level Plan Evaluation in Embodied Environments
Suyeon Shin, Byoung-Tak Zhang
BigComp 2025
Paper

2024

Continuous SO 3 Equivariant Convolution for 3D Point Cloud Analysis
Continuous SO(3) Equivariant Convolution for 3D Point Cloud Analysis
Jaein Kim, Hee Bin Yoo, Dong-Sig Han, Yeon-Ji Song, Byoung-Tak Zhang
ECCV 2024
Paper
Code
DUEL: Duplicate Elimination on Active Memory for Self-Supervised Class-Imbalanced Learning
DUEL: Duplicate Elimination on Active Memory for Self-Supervised Class-Imbalanced Learning
Won-Seok Choi, Hyundo Lee, Dong-Sig Han, Junseok Park, Heeyeon Koo, Byoung-Tak Zhang
AAAI 2024
Paper
Efficient Monte Carlo Tree Search via On-the-Fly State-Conditioned Action Abstraction
Efficient Monte Carlo Tree Search via On-the-Fly State-Conditioned Action Abstraction
Yunhyeok Kwak, Inwoo Hwang, Dooyoung Kim, Sanghack Lee, Byoung-Tak Zhang
UAI 2024
Paper
Code
Fine-Grained Causal Dynamics Learning with Quantization for Improving Robustness in Reinforcement Learning
Fine-Grained Causal Dynamics Learning with Quantization for Improving Robustness in Reinforcement Learning
Inwoo Hwang, Yunhyeok Kwak, Suhyung Choi, Byoung-Tak Zhang, Sanghack Lee
ICML 2024
Paper
Code
HAPFI: History-Aware Planning based on Fused Information
HAPFI: History-Aware Planning based on Fused Information
Sujin Jeon, Suyeon Shin, Byoung-Tak Zhang
ICRA 2024
Paper
Multi-Object RANSAC: Efficient Plane Clustering Method in a Clutter
Multi-Object RANSAC: Efficient Plane Clustering Method in a Clutter
Seunghyeon Lim, Youngjae Yoo, Jun Ki Lee, Byoung-Tak Zhang
ICRA 2024
Paper
OV-MAP : Open-Vocabulary Zero-Shot 3D Instance Segmentation Map for Robots
OV-MAP : Open-Vocabulary Zero-Shot 3D Instance Segmentation Map for Robots
Hye-Jung Yoon, Juno Kim, Yesol Park, Byoung-Tak Zhang
IROS 2024
Paper
PGA: Personalizing Grasping Agents with Single Human-Robot Interaction
PGA: Personalizing Grasping Agents with Single Human-Robot Interaction
Junghyun Kim, Gi-Cheon Kang, Jaein Kim, Seoyun Yang, Minjoon Jung, Byoung-Tak Zhang
IROS 2024
Paper
Code
PROGrasp: Pragmatic Human-Robot Communication for Object Grasping
PROGrasp: Pragmatic Human-Robot Communication for Object Grasping
Gi-Cheon Kang, Junghyun Kim, Jaein Kim, Byoung-Tak Zhang
ICRA 2024
Paper
Code
Seg2Grasp: A Robust Modular Suction Grasping in Bin Picking
Seg2Grasp: A Robust Modular Suction Grasping in Bin Picking
Hye-Jung Yoon, Juno Kim, Yesol Park, Jun-Ki Lee, Byoung-Tak Zhang
IROS 2024
Paper
Unveiling the Significance of Toddler-Inspired Reward Transition in Goal-Oriented Reinforcement Learning
Unveiling the Significance of Toddler-Inspired Reward Transition in Goal-Oriented Reinforcement Learning
Junseok Park, Yoonsung Kim, Hee Bin Yoo, Min Whoo Lee, Kibeom Kim, Won-Seok Choi, Minsu Lee, Byoung-Tak Zhang
AAAI 2024
Paper
Communicating Unexpectedness for Out-of-Distribution Multi-Agent Reinforcement Learning
Communicating Unexpectedness for Out-of-Distribution Multi-Agent Reinforcement Learning
Minwhoo Lee, Kibeom Kim, Soo Wung Shin, Minsu Lee, Byoung-Tak Zhang
AAAI 2024 Workshop
Paper
Decomposing the Configuration of an Articulated Object via Graph Neural Network
Decomposing the Configuration of an Articulated Object via Graph Neural Network
Seunghyeon Lim, Kisung Shin, Nakul Gopalan, Jun Ki Lee, Byoung-Tak Zhang
CoRL 2024 LFDM Workshop
Paper
Zero-Shot Vision-and-Language Navigation with Collision Mitigation in Continuous Environment
Zero-Shot Vision-and-Language Navigation with Collision Mitigation in Continuous Environment
Seongjun Jeong, Gi-Cheon Kang, Joochan Kim, Byoung-Tak Zhang
CVPR 2024 Embodied AI Workshop
Paper
Can Video Large Language Models Comprehend Language in Videos
Can Video Large Language Models Comprehend Language in Videos?
Minjoon Jung, Junbin Xiao, Byoung-Tak Zhang, Angela Yao
NeurIPS 2024 Video-Language Models Workshop
Paper
Visual Hindsight Self-Imitation Learning for Interactive Navigation
Visual Hindsight Self-Imitation Learning for Interactive Navigation
Kibeom Kim, Moonheun Lee, Min Whoo Lee, Kisung Shin, Minsu Lee, Byoung-Tak Zhang
IEEE Access
Paper

2023

Domain Adapted Broiler Density Map Estimation Using Negative-Patch Data Augmentation
Domain Adapted Broiler Density Map Estimation Using Negative-Patch Data Augmentation
Taehyung Kim, Daehyun Lee, Wansoo Kim, Byoung-Tak Zhang
Biosystems Engineering 231
Paper
Video Turing Test: A First Step Towards Human-Level AI
Video Turing Test: A First Step Towards Human-Level AI
Minsu Lee, Yu-Jung Heo, Seongho Choi, Wooseok Choi, Byoung-Tak Zhang
AI Magazine 44 (4)
Paper
EXOT: Exit-Aware Object Tracker for Safe Robotic Manipulation of Moving Object
EXOT: Exit-Aware Object Tracker for Safe Robotic Manipulation of Moving Object
Hyunseo Kim, Hye-Jung Yoon, Minji Kim, Dong-Sig Han, Byoung-Tak Zhang
ICRA 2023
Paper
Code
GVCCI: Lifelong Learning of Visual Grounding for Language-Guided Robotic Manipulation
GVCCI: Lifelong Learning of Visual Grounding for Language-Guided Robotic Manipulation
Junghyun Kim, Gi-Cheon Kang, Jaein Kim, Suyeon Shin, Byoung-Tak Zhang
IROS 2023
Paper
Code
Learning Geometry-Aware Representations by Sketching
Learning Geometry-Aware Representations by Sketching
Hyundo Lee, Inwoo Hwang, Hyunsung Go, Won-Seok Choi, Kibeom Kim, Byoung-Tak Zhang
CVPR 2023
Paper
Code
Neural Collage Transfer: Artistic Reconstruction via Material Manipulation
Neural Collage Transfer: Artistic Reconstruction via Material Manipulation
Ganghun Lee, Minji Kim, Yunsu Lee, Minsu Lee, Byoung-Tak Zhang
ICCV 2023
Paper
Code
On Discovery of Local Independence over Continuous Variables via Neural Contextual Decomposition
On Discovery of Local Independence over Continuous Variables via Neural Contextual Decomposition
Inwoo Hwang, Yunhyeok Kwak, Yeon-Ji Song, Byoung-Tak Zhang, Sanghack Lee
CLeaR 2023
Paper
Code
Robust Map Fusion with Visual Attention Utilizing Multi-Agent Rendezvous
Robust Map Fusion with Visual Attention Utilizing Multi-Agent Rendezvous
Jaein Kim, Dong-Sig Han, Byoung-Tak Zhang
ICRA 2023
Paper
Code
The Dialog Must Go On: Improving Visual Dialog via Generative Self-Training
The Dialog Must Go On: Improving Visual Dialog via Generative Self-Training
Gi-Cheon Kang, Sungdong Kim, Jin-Hwa Kim, Donghyun Kwak, Byoung-Tak Zhang
CVPR 2023
Paper
Code
Causal Dynamics Learning with Quantized Local Independence Discovery
Causal Dynamics Learning with Quantized Local Independence Discovery
Inwoo Hwang, Yunhyeok Kwak, Suhyung Choi, Byoung-Tak Zhang, Sanghack Lee
ICML 2023 Workshop on Spurious Correlations, Invariance, and Stability
Paper
H2RL: Transferring Heuristic Knowledge to a Reinforcement Learning Agent for Solving the Online 3D Bin Packing Problem
H2RL: Transferring Heuristic Knowledge to a Reinforcement Learning Agent for Solving the Online 3D Bin Packing Problem
Minji Kim, Ganghun Lee, Minsu Lee, Byoung-Tak Zhang
ICRA 2023 Workshop on Transferability in Robotics
Paper
Instruction-Tuned Self-Questioning Framework for Multimodal Reasoning
Instruction-Tuned Self-Questioning Framework for Multimodal Reasoning
Youwon Jang, Yu-Jung Heo, Jaeseok Kim, Minsu Lee, Du-Seong Chang, Byoung-Tak Zhang
ICCV 2023 CLVL Workshop
Page
Learning Object Motion and Appearance Dynamics with Object-Centric Representations
Learning Object Motion and Appearance Dynamics with Object-Centric Representations
Yeon-Ji Song, Hyunseo Kim, Suhyung Choi, Jin-Hwa Kim, Byoung-Tak Zhang
NeurIPS 2023 Workshop on Causal Representation Learning
Paper
Quantized Local Independence Discovery for Fine-Grained Causal Dynamics Learning in Reinforcement Learning
Quantized Local Independence Discovery for Fine-Grained Causal Dynamics Learning in Reinforcement Learning
Inwoo Hwang, Yunhyeok Kwak, Suhyung Choi, Byoung-Tak Zhang, Sanghack Lee
NeurIPS 2023 GenPlan Workshop
Paper
Visual Perception-Based Assistive Mobile Robot System for Manipulation Tasks
Visual Perception-Based Assistive Mobile Robot System for Manipulation Tasks
Hye-Jung Yoon, Juno Kim, Yesol Park, Byoung-Tak Zhang
ICRA 2023 Workshop on Assistive Manipulation
Page

2022

Team Tidyboy at the WRS 2020: A Modular Software Framework for Home-Service Robots
Team Tidyboy at the WRS 2020: A Modular Software Framework for Home-Service Robots
Tae-Woong Kang, Dong-Woon Song, Jae-Bong Lee, Jun-Young Kim, Choong-Yeon Lee, …, Minji Kim, Hyeon-Jun Cho, Byoung-Tak Zhang, Jae-Bok Song, Seung-Jun Lee
Advanced Robotics 36 (16)
Paper
From Scratch to Sketch: Deep Decoupled Hierarchical Reinforcement Learning for Robotic Sketching Agent
From Scratch to Sketch: Deep Decoupled Hierarchical Reinforcement Learning for Robotic Sketching Agent
Ganghun Lee, Minji Kim, Minsu Lee, Byoung-Tak Zhang
ICRA 2022
Paper
Hypergraph Transformer: Weakly-Supervised Multi-hop Reasoning for Knowledge-Based Visual Question AnSoftware Engineerring
Hypergraph Transformer: Weakly-Supervised Multi-hop Reasoning for Knowledge-Based Visual Question AnSoftware Engineerring
Yu-Jung Heo, Eunsol Kim, Wooseok Choi, Byoung-Tak Zhang
ACL 2022
Paper
Modal-Specific Pseudo Query Generation for Video Corpus Moment Retrieval
Modal-Specific Pseudo Query Generation for Video Corpus Moment Retrieval
Minjoon Jung, Seongho Choi, Joochan Kim, Jin-Hwa Kim, Byoung-Tak Zhang
EMNLP 2022
Paper
PDF
PlaceNet: Neural Spatial Representation Learning with Multimodal Attention
PlaceNet: Neural Spatial Representation Learning with Multimodal Attention
Choong-Yeon Lee, Youngjae Yoo, Byoung-Tak Zhang
IJCAI 2022
Paper
Robust Imitation via Mirror-Descent Inverse Reinforcement Learning
Robust Imitation via Mirror-Descent Inverse Reinforcement Learning
Dong-Sig Han, Hyunseo Kim, Hyundo Lee, Je-Hwan Ryu, Byoung-Tak Zhang
NeurIPS 2022
Paper
SelecMix: Debiased Learning by Contradicting-Pair Sampling
SelecMix: Debiased Learning by Contradicting-Pair Sampling
Inwoo Hwang, Sangjun Lee, Yunhyeok Kwak, Seong Joon Oh, Damien Teney, Jin-Hwa Kim, Byoung-Tak Zhang
NeurIPS 2022
Paper
Smooth-Swap: A Simple Enhancement for Face-Swapping with Smoothness
Smooth-Swap: A Simple Enhancement for Face-Swapping with Smoothness
Ji-Seob Kim, Ji-Hoon Lee, Byoung-Tak Zhang
CVPR 2022
Paper
DUEL: Adaptive Duplicate Elimination on Working Memory for Self-Supervised Learning
DUEL: Adaptive Duplicate Elimination on Working Memory for Self-Supervised Learning
Won-Seok Choi, Dong-Sig Han, Hyundo Lee, Junseok Park, Byoung-Tak Zhang
NeurIPS 2022 Workshop on Self-Supervised Learning: Theory and Practice
Paper
Generative Self-Training Improves Pre-training for Visual Dialog
Generative Self-Training Improves Pre-training for Visual Dialog
Gi-Cheon Kang, Sungdong Kim, Jin-Hwa Kim, Donghyun Kwak, Byoung-Tak Zhang
ICML 2022 Workshop on Pre-training
Paper
Improving Robustness to Texture Bias via Shape-Focused Augmentation
Improving Robustness to Texture Bias via Shape-Focused Augmentation
Sangjun Lee, Inwoo Hwang, Gi-Cheon Kang, Byoung-Tak Zhang
CVPR 2022 Human-Centered Intelligent Services Workshop
Paper
Language-Agnostic Semantic-Consistent Text-to-Image Generation
Language-Agnostic Semantic-Consistent Text-to-Image Generation
Seongjun Jung, Wooseok Choi, Seongho Choi, Byoung-Tak Zhang
ACL 2022 Workshop on Multilingual Multimodal Learning
Paper
On the Importance of Critical Period in Multi-Stage Reinforcement Learning
On the Importance of Critical Period in Multi-Stage Reinforcement Learning
Junseok Park, Inwoo Hwang, Min Whoo Lee, Hyunseok Oh, Minsu Lee, Youngki Lee, Byoung-Tak Zhang
ICML 2022 Workshop on Complex Feedback in Online Learning
Paper
Scene Graph Parsing via Abstract Meaning Representation in Pre-trained Language Models
Scene Graph Parsing via Abstract Meaning Representation in Pre-trained Language Models
Wooseok Choi, Yu-Jung Heo, Dharani Perumal, Byoung-Tak Zhang
NAACL 2022 DLG4NLP Workshop
Paper
SelecMix: Debiased Learning by Mixing-Up Contradicting Pairs
SelecMix: Debiased Learning by Mixing-Up Contradicting Pairs
Inwoo Hwang, Sangjun Lee, Yunhyeok Kwak, Seong Joon Oh, Damien Teney, Jin-Hwa Kim, Byoung-Tak Zhang
ICML 2022 Workshop on Spurious Correlations, Invariance and Stability
Paper

2021

M2FN: Multi-Step Modality Fusion for Advertisement Image Assessment
M2FN: Multi-Step Modality Fusion for Advertisement Image Assessment
Kyunghwa Park, Jeongwoo Ha, Jeonghun Lee, Sunyoung Kwon, Kyungmin Kim, Byoung-Tak Zhang
Applied Soft Computing 105
Paper
DramaQA: Character-Centered Video Story Understanding with Hierarchical QA
DramaQA: Character-Centered Video Story Understanding with Hierarchical QA
Seongho Choi, Kyungwoon On, Yu-Jung Heo, Ahjeong Seo, Yuwon Jang, Minsu Lee, Byoung-Tak Zhang
AAAI 2021
Paper
Attend What You Need: Motion-Appearance Synergistic Networks for Video Question AnSoftware Engineerring
Attend What You Need: Motion-Appearance Synergistic Networks for Video Question AnSoftware Engineerring
Ahjeong Seo, Gi-Cheon Kang, Joonhan Park, Byoung-Tak Zhang
ACL 2021
Paper
Co-Attentional Transformers for Story-Based Video Understanding
Co-Attentional Transformers for Story-Based Video Understanding
Björn Bebensee, Byoung-Tak Zhang
ICASSP 2021
Paper
Devil s Advocate: Novel Boosting Ensemble Method from Psychological Findings for Text Classification
Devil's Advocate: Novel Boosting Ensemble Method from Psychological Findings for Text Classification
Huiyeol Jo, Jaeseo Lim, Byoung-Tak Zhang
EMNLP 2021
Paper
Goal-Aware Cross-Entropy for Multi-Target Reinforcement Learning
Goal-Aware Cross-Entropy for Multi-Target Reinforcement Learning
Kibeom Kim, Min Whoo Lee, Yoonsung Kim, Je-Hwan Ryu, Minsu Lee, Byoung-Tak Zhang
NeurIPS 2021
Paper
Message Passing Adaptive Resonance Theory for Online Active Semi-Supervised Learning
Message Passing Adaptive Resonance Theory for Online Active Semi-Supervised Learning
Tae-Hyung Kim, Injun Hwang, Hyundo Lee, Hyunseo Kim, Won-Seok Choi, Joseph J. Lim, Byoung-Tak Zhang
ICML 2021
Paper
Multimodal Anomaly Detection based on Deep Auto-Encoder for Object Slip Perception of Mobile Manipulation Robots
Multimodal Anomaly Detection based on Deep Auto-Encoder for Object Slip Perception of Mobile Manipulation Robots
Youngjae Yoo, Choong-Yeon Lee, Byoung-Tak Zhang
ICRA 2021
Paper
Passive Versus Active: Frameworks of Active Learning for Linking Humans to Machines
Passive Versus Active: Frameworks of Active Learning for Linking Humans to Machines
Jaeseo Lim, Huiyeol Jo, Byoung-Tak Zhang, Jooyong Park
CogSci 2021
Paper
Reasoning Visual Dialog with Sparse Graph Learning and Knowledge Transfer
Reasoning Visual Dialog with Sparse Graph Learning and Knowledge Transfer
Gi-Cheon Kang, Junseok Park, Hwaran Lee, Byoung-Tak Zhang, Jin-Hwa Kim
EMNLP 2021
Paper
Toddler-Guidance Learning: Impacts of Critical Period on Multimodal AI Agents
Toddler-Guidance Learning: Impacts of Critical Period on Multimodal AI Agents
Junseok Park, Kwanyoung Park, Hyunseok Oh, Ganghun Lee, Minsu Lee, Youngki Lee, Byoung-Tak Zhang
ICMI 2021
Paper
Toward a Human-Level Video Understanding Intelligence
Toward a Human-Level Video Understanding Intelligence
Yu-Jung Heo, Minsu Lee, Seongho Choi, Woo Suk Choi, Minjung Shin, Minjoon Jung, Jeh-Kwang Ryu, Byoung-Tak Zhang
AAAI FSS 2021
Paper