Loading…
CNS*2020 Online has ended
Welcome to the Sched instance for CNS*2020 Online! Please read the instruction document on detailed information on CNS*2020.
Back To Schedule
Sunday, July 19 • 9:00pm - 10:00pm
P153: Number-selective units can spontaneously arise in untrained deep neural networks

Log in to save this to your schedule, view media, leave feedback and see who's attending!

Feedback form is now closed.
Link : https://meet.google.com/kym-ugkp-ghx
jaesonjang@kaist.ac.kr

Jaeson Jang
, Gwangsu Kim, Seungdae Baek, Min Song, Se-Bum Paik

Number sense is an ability to estimate number of visual items (numerosity) without counting, which is observed in newborn animals of various species. In single-neuron recordings in numerically naïve monkeys, it was observed that individual neurons can respond selectively to the numerosity [1]. This suggests that number-selective neurons spontaneously arise for a foundation of innate number sense, but it remains unclear how these neurons originate in the absence of learning. Here, using a deep neural network (DNN) designed from the structure of a visual pathway (AlexNet), we show that number tuning of network units can spontaneously arise in untrained networks, even in the absence of any learning. To devise an untrained network, we randomly permuted the weights of filters in each convolutional layer of the pre-trained AlexNet and examined the response to images of dot patterns representing numbers from 1 to 30. For stimuli, we used three different sets to ensure invariance of the number tuning for certain geometric factors (stimulus size, density, and area). A network unit was considered to be number-selective if its response significantly changes across the numerosity (p < 0.01, two-way ANOVA) but there is no significant effect for the stimulus set or interaction between two factors (p > 0.01). Importantly, number-selective units were observed in the permuted AlexNet (9.58% of units in the last convolutional layer), even though the network was never trained for any task after being permuted. Observed number-selective units followed the Weber-Fechner law observed in the brain, where the width of the tuning curves increases proportionally in the numerosity. We also showed that these units enable the network to perform a number discrimination task, by training a support vector machine (SVM) to compare numerosities in two different images using the response of number- selective units. Next, to explain how number-selective units emerge in permuted networks, we hypothesized that the number tuning to various numerosities can be initiated from the monotonic unit activities in the earlier layer, the response of which monotonically decreases or increases as the given numerosity increases. To test this idea, we performed a model simulation for the randomly weighed summation of tuning curves of increasing and decreasing activities and confirmed that tuning to all the tested numerosities was successfully generated. Notably, the curve tuned to smaller numbers was generated by the summation of strongly weighted decreasing activities and weakly weighted increasing activities. As expected, in the permuted AlexNet, we observed that number-selective units tuned to smaller numbers receive strong inputs from the decreasing units and vice versa. These results suggest that number-tuned neurons may spontaneously arise from the statistical variation of feedforward projections in the visual pathway during the early development stage. This finding provides new insights into the origin of cognitive functions in biological brains, as well as in artificial neural networks.

Acknowledgments

We thank the National Research Foundation of Korea (NRF) for supporting Se-Bum Paik with 2019R1A2C4069863 and 2019M3E5D2A01058328.

References

1. Viswanathan P & Nieder A. Neuronal correlates of a visual ‘sense of number’ in primate parietal and prefrontal cortices. Proc. Natl. Acad. Sci. 2013, 110, 11187–11192.

Speakers
avatar for Jaeson Jang

Jaeson Jang

Graduate student, Department of Bio and Brain Engineering, Korea Advanced Institute of Science and Technology



Sunday July 19, 2020 9:00pm - 10:00pm CEST
Slot 13