Blending Senses and Language in Concept Learning

Published on April 27, 2022

Concept learning is a fascinating field in cognitive psychology, exploring how our brains represent knowledge. Just like mixing different ingredients to create a mouth-watering dish, researchers are investigating the combination of language-derived codes and sensory-derived codes to enhance artificial intelligence systems. Although datasets for these representations have been published individually, this study takes a groundbreaking approach by analyzing them together for the first time. Through four meticulous experiments, scientists examine whether multisensory vectors and text-derived vectors truly reflect conceptual understanding and if they complement each other in terms of cognition. The results are intriguing! Firstly, both types of representations effectively capture the essence of concepts. However, the findings reveal that these two forms of representation are quite distinct. Moreover, when it comes to highly concrete concepts, the multisensory representations outperform the text-derived ones, demonstrating a remarkable similarity to human cognition. Finally, by combining both types, researchers discovered an improvement in the overall concept representation. This research has exciting implications for developing more human-like AI systems. Ready to dive deeper? Explore the underlying study!

When learning concepts, cognitive psychology research has revealed that there are two types of concept representations in the human brain: language-derived codes and sensory-derived codes. For the objective of human-like artificial intelligence, we expect to provide multisensory and text-derived representations for concepts in AI systems. Psychologists and computer scientists have published lots of datasets for the two kinds of representations, but as far as we know, no systematic work exits to analyze them together. We do a statistical study on them in this work. We want to know if multisensory vectors and text-derived vectors reflect conceptual understanding and if they are complementary in terms of cognition. Four experiments are presented in this work, all focused on multisensory representations labeled by psychologists and text-derived representations generated by computer scientists for concept learning, and the results demonstrate that (1) for the same concept, both forms of representations can properly reflect the concept, but (2) the representational similarity analysis findings reveal that the two types of representations are significantly different, (3) as the concreteness of the concept grows larger, the multisensory representation of the concept becomes closer to human beings than the text-derived representation, and (4) we verified that combining the two improves the concept representation.

Read Full Article (External Site)

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes:

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>