Projects

Concept Alignment

Discussion of AI alignment (alignment between humans and AI systems) has focused on value alignment, broadly referring to creating AI systems that share human values. We argue that before we can even attempt to align values, it is imperative that AI systems and humans align the concepts they use to understand the world. We integrate ideas from philosophy, cognitive science, and deep learning to explain the need for concept alignment, not just value alignment, between humans and machines. We summarize existing accounts of how humans and machines currently learn concepts, and we outline opportunities and challenges in the path towards shared concepts. Finally, we explain how we can leverage the tools already being developed in cognitive science and AI research to accelerate progress towards concept alignment.

Recommended citation: Rane, S., Bruna, P. J., Sucholutsky, I., Kello, C., & Griffiths, T. (2023). Concept alignment. 1st NeurIPS Workshop on AI meets Moral Philosophy and Moral Psychology (MP2).

Psychophysical Adaptive Procedure: Developing a New, Generalizable Method

“Staircasing” is a classic experimental procedure used in psychophysics wherein when a subject successfully discriminates between two stimuli that differ on one perceptual dimension, the stimuli become more similar to each other on that dimension, and when the subject fails, the stimuli become more different. The purpose of this procedure is to find the size of the difference that produces a specific level of discrimination success. Traditional staircase procedures require subjects to be run individually for many trials. Our goal was to develop a procedure for crowd-sourced online data collection in which subjects can be tested simultaneously using only a few trials each with every new piece of data altering the stimulus pair shown on the next trial. We do this by continually fitting subject responses to a plausible function that is used to predict the difference that should produce the desired accuracy level. Only presenting differences predicted to yield the desired performance allows us to find the difference corresponding to this accuracy level faster than sampling the whole range of stimulus differences. The stimulus values we determined with this method were used for research on the phenomenon of learned categorical perception. We present this new technique as an efficient way of determining psychophysical values more generally for purposes of controlling stimulus discriminability in experimental research.

Recommended citation: Bruna, P. J., Andrews, J., & de Leeuw, J. R. (2023). Psychophysical Adaptive Procedure: Developing a New, Generalizable Method [Manuscript in preparation]. Department of Cognitive Science, Vassar College.

Probing the Methodology and Interpretation of Learned Categorical Perception Research

Learned categorical perception (CP) is a phenomenon where learning to place objects in categories influences how similar they appear, with objects in different categories becoming easier to tell apart and/or objects in the same category becoming harder to tell apart. Despite these effects being widely demonstrated, past studies exhibit low statistical power and the literature lacks a unifying theoretical framework. We seek to rectify these issues by conducting a systematic methodological investigation of learned CP, starting with replicating the effect under the conditions with which it has traditionally been reported, then exploring how successive methodological changes impact the presence of the effect. Our replication failed to show a pattern indicative of learned CP from comparing discrimination performance between a group that had learned a category distinction and a control group that had not. Through exploring our data to scrutinize possible key differences between our study and previous demonstrations of learned CP, we hypothesized that a combination of our stimuli being too easy to discriminate and the memorization of individual stimuli along each dimension obstructed the influence of category membership on discrimination behavior and was responsible for the absence of CP effects. We addressed this issue by lowering the discriminability of stimulus pairs and by increasing the number of stimuli in each category. Preliminary results suggest a possible learned CP effect and we plan to collect additional data to clarify the nature of the pattern

Recommended citation: Andrews, J. & Bruna, P. J. (2023). Probing the Methodology and Interpretation of Learned Categorical Perception Research [Manuscript in preparation]. Department of Cognitive Science, Vassar College.

What Next? : Leveraging Surprise in a Recurrent Neural Network to (de)Construct Morphological Complexity in Japanese

The question of how we as cognitive agents and biological creatures acting within a world describe, understand, and communicate about this shared world and our motivations within it has long existed at the center of cognitive science. The ability to utilize language has even been hailed as a hallmark of what it is to have a mind. Language is an action that involves transforming complex, non-linear information about the world and ourselves into linear expressions and communicating them in real time, and yet it seems every language accomplishes this in its own way. Formal language modeling exhibits a bias towards the linguistic features observed in English and related languages, namely the ability to describe language in terms of word-based units. This bias has caused language models to be ill-equipped for success in languages exhibiting a high degree of morphological complexity, such as the agglutinative morphology found in Japanese, and indicates a shortcoming in our understanding of the underlying cognitive processes that allow us to be thinkers, speakers, and listeners. This study embarks on an effort to challenge the traditional scales at which we conceptualize information processing in language. I employ a recurrent neural network (RNN) to explore character-by-character predictability in samples of contemporary Japanese text. I address what properties of agglutinative morphology are salient to neural networks and offer a comparison between meaning construction in Japanese and English. This study further investigates the unique morpho-syntactic role played by orthography in Japanese. I find success in extracting certain key features of the structure of Japanese sentences using an RNN and offer a path for deepening our understanding of the differences in information encoding and processing that we observe across languages and how these differences arise.

Recommended citation: Bruna, P. J. (2022). What Next? : Leveraging Surprise in a Recurrent Neural Network to (de)Construct Morphological Complexity in Japanese [Unpublished manuscript]. Department of Cognitive Science, Vassar College.

Words May Jump-Start Meaning More Than Vision: A Non-Replication of Early ERP Effects in Boutonnet and Lupyan (2015)

We report a replication of Boutonnet and Lupyan’s (2015) study of the effects of linguistic labelling on perceptual performance. In addition to a response time advantage of linguistic labels over non-linguistic auditory cues in judging visual objects, Boutonnet and Lupyan found that the two types of cues produced different patterns in the early perceptual ERP components P1 and P2 but not the later, semantics-relevant N4. This study thus adds an important piece of evidence supporting the claim of genuine top-down effects on perception. Given the controversy over this claim and the need for replication of key findings, we attempted to replicate Boutonnet and Lupyan (2015). We replicated their behavioral findings that response times to indicate whether an auditory cue matches a visual image of an object were faster for match than mismatch trials and faster for linguistic than non-linguistic cues. We did not replicate the main ERP effects supporting a positive effect of linguistic labels on the early perceptual ERP components P1 and P2, though we did find a congruence by cue type interaction effect on those components. Unlike Boutonnet and Lupyan, we found a main effect of cue type on the N4 in which non-linguistic cues produced more negative amplitudes. Exploratory analyses of the unpredicted N4 effect suggest that the response time advantage of linguistic labels occurred during semantic rather than early visual processing. This experiment was pre-registered at https://osf.io/cq8g4/ and conducted as part of an undergraduate cognitive science research methods class at Vassar College.

Recommended citation: de Leeuw, J. R., Andrews, J., Barney, L., Bigler, M., Bruna, P. J., Chen, Y., Cherry, R., Dowie, D. R., Forbes, E., Haffey, B., Hu, X., Jaklitsch, M., Leopold, N., Lewis, C., MacDonald, D., McShaffrey, C., Nakayama, K., Olstad, W., Peng, R., … Zhang, L. (2021). Words May Jump-Start Meaning More Than Vision: A Non-Replication of Early ERP Effects in Boutonnet and Lupyan (2015). Collabra: Psychology, 7(1). https://doi.org/10.1525/collabra.29763

Exploring Semantic Relatedness Judgments in the Structure of a Semantic Network

Drawing upon work by De Deyne et al. (2016), I explore a model of spreading activation through a semantic network in regards to how different kinds of semantic relationships are encoded in said network. In particular, I examine the contribution of indirect pathways through the network to explain differences in similarity judgments of sensorimotor and linguistic relationships between pairs of words. I propose that the structure of a semantic network encodes properties that distinguish these two types of semantic relationships that are not revealed by measures of association strength that only examine direct connections within the network. A cosine similarity measure extracted from a spreading activation model is compared to a measure of association strength in accounting for observed similarity judgments, and a model for examining the differential contributions of various random walk pathways through a semantic network in the encoding of sensorimotor and linguistic semantic relationships is presented.

Recommended citation: Bruna, P. J. (2021). Exploring Semantic Relatedness Judgments in the Structure of a Semantic Network [Unpublished manuscript]. Department of Cognitive Science, Vassar College. https://osf.io/sj7tg/

Polyphony J. Bruna