The risks of misrecognition are, in principle, unavoidable in existing image recognition technologies using artificial intelligence (AI). For example, AI is known to falsely recognize a picture that apparently looks like a panda to humans for something completely different when a small amount of noise is added. The recent years have witnessed the remarkable development of AI using deep learning. AI has been increasingly able to recognize objects at performance levels close to those of humans and is expected to be applied to a range of areas. However, such misrecognition errors pose as a fatal drawback that can affect human lives in fields such as autonomous driving and medical image analysis.

In this research, we aim to develop robust and flexible AI preventing misrecognition errors by reproducing in AI the information processing carried out by our brain. The elucidation of how animals, including humans, process visual information may prove useful for eliminating any risks of misrecognition by AI as described above. This means that if we can elucidate the information processing actually performed in the brain and reproduce it in AI, we may be able to develop robust and flexible AI preventing misrecognition, which is unrealizable by the current principle of AI.
To this end, this research aims to comprehensively measure the neural activity of cells in many regions of the cerebral cortex and reproduce the measured brain activity in AI to build such innovative AI.

Our research team has developed a technology to comprehensively measure neuronal activity in cells in multiple areas of the cerebral cortex. It is known that visual information processing in the brains of animals including humans is performed hierarchically in many areas of the cerebral cortex. To understand the actual mechanisms of the processing in the brain, it is important to investigate such mechanisms in specific regions and between different regions of the brain.
Our team has developed a unique “two-photon calcium imaging” technique for observing the activity of neuron cells in the brain of living animals using a special two-photon microscope. Using this method, we can observe the activity of tens of thousands of cells in an area of several millimeters square, even the activity in each cell.

In this research, we aim to further develop these technologies and measure the neural activity of cells in many areas of the cerebral cortex more comprehensively.

Our research team has also developed a technique for analyzing the activity of neuron cells in detail on computer by copying the activity of neuron cells in the visual cortex from various images onto deep neural networks.
Specific steps for analyzing the activity of neuron cells include: (1) Input the image shown to an animal into the deep neural network, and have the network learn the activity of neuron cells, taking the activity responding to the image as teaching signals; and (2) on a computer, generate an image where the output of the neural network has been maximized to estimate the response characteristics of the neuron cell responses copied to the neural network. Specifically, prepare a random noise image and update the brightness value of the image so that the output of the neural network for that image becomes larger. Repeat the updating process several times to maximize the output of the neural network. The finally updated image maximizes the activity of neuron cells copied to the neural network and is presumed to indicate the response characteristics of the neuron cells.
Our team has demonstrated that the technique can be applied to the analysis of neuron cells in the mouse primary visual cortex to explain the properties of various primary visual cortex. Applying the data obtained by measuring the neural activity of cells in many regions of the cerebral cortex using the technique, we further aim to comprehensively analyze the extraction, analysis, and integration of information between layers of the brain. Furthermore, by reproducing the activity of the neurons responsible for these visual information processing on artificial intelligence, and by copying the fluctuation of neural activity, including top-down and recursive connections, which have not been used in conventional artificial intelligence, onto artificial intelligence, we aim to construct a robust, flexible, and dynamic brain-type artificial intelligence that cannot be obtained with current artificial intelligence principles.

The AI equipped with human brain functions is the target of our research. It is completely different from present AI in principle and is expected to contribute to a very wide range of industries.
Since misrecognition, which has been unavoidable from the principle of existing AI, can be avoided, our innovative AI can be applied to, for instance, diagnostic imaging such as X-rays, CT, and MRI, and to histopathological diagnosis in the medical scene, thereby preventing possible diagnostic errors. Furthermore, its application to autonomous driving will be unlikely to cause fatal accidents resulting from misrecognition.
It is hoped that the results of our research will also one day enable AI to recognize emotions from human facial expressions. If this is the case, more person-focused services will be provided in the face-to-face area such as housework and nursing care.
In addition to the aforementioned examples, we believe that our research will open up new possibilities for AI that can be applied to more complex situations in the real world.

■ We elucidated the mechanism by which the connections connecting the many areas of the cerebral cortex are precisely formed during development (Murakami et al., 2022, Nature). The human cerebral cortex contains as many as 180 areas, which are connected by thousands of precise connections, forming hierarchical and parallel circuits. In this study, we elucidated the mechanisms underlying the formation of connections among these many areas of the cortex. We found that the pathways connecting the retina to primary and many higher-order visual areas are formed first, before the connections connecting the cortical areas are formed. The retina-derived spontaneous activity propagating along these pathways conveys retinal location information to the primary and higher- visual areas, which serve as instruction signals to form precise connections between the corresponding locations in the primary and many higher-visual areas. Hierarchical and parallel information processing by precise neural circuits connecting these areas is the basis for the complex and general intelligence of our brain, and this principle may be applied to artificial intelligence to develop general artificial intelligence.

■ We developed a new way to overcome vulnerability to adversarial attacks by incorporating the properties of information processing in humans and other animals into AI (Ukita & Ohki, 2023, Neural Networks). Among the properties known in animal neurons, we focused on the variability that neurons possess. In visual information processing in animals, variability exists not only in the retina, which is located at the input of information processing, but also in the brain, which is located at the later stage of information processing. Variability is thought to be important for training circuits that can respond to variations in information from the outside world. Therefore, we first devised a new method of adversarial attack on the hidden layer. Then, by introducing variability in the hidden layer, we succeeded in reducing the vulnerability to this adversarial attack. This method is expected to increase the possibility of creating an AI that more closely resembles the behavior of humans and other animals. Furthermore, we developed an image recognition method that is robust against unknown noise during inference by performing learning-based quantization in image space and hidden (feature) space (Hataya et al., IJCNN 2022; Kishida et al., ICPR 2022).

■ We clarified how color and shape information is represented in the primate primary visual cortex (Chatterjee et al., Nature Communication, 2021). We showed that red, green, blue, and yellow information merges in the blob (a color-specific module about 300 microns in diameter) of the primary visual cortex, and that this partial blending, rather than complete blending, forms a map of hue information, including intermediate colors. Furthermore, we clarified that the blob represents color information at low spatial frequencies (such as the surface of objects) and no shape information, while just outside the blob, color information and shape information are represented in combination, and further outside the blob, only shape information at high spatial frequencies is represented and no color information is represented. In current deep learning, R, G, and B information is input as it is, but depending on the spatial frequency, shape and color information may be unnecessary in some channels, and taking this into account may contribute to the development of artificial intelligence for object recognition similar to that of primates.

■ Using the data of spontaneous activity measured from mouse brains, we developed a new learning method, Hidden-FORCE method, which is an improvement of Full-FORCE method, to reproduce the dynamics of spontaneous brain activity on a recurrent neural network. Then, by applying this method, we successfully reproduced and predicted the whole-brain spontaneous activity data with a recurrent neural network by training the recurrent network in a data-driven manner using the spontaneous activity data measured from the mouse brain.

■ As a basic model for mathematical modeling of dynamic spontaneous activity in the cerebral cortex, we developed a mean-field analysis method for nonlinear networks consisting of Stuart-Landau oscillators with noise (Li et al., 2022). Furthermore, we discovered neural activity related to the process of logical thinking using categorical information in neuroscience experiments using monkeys, and analyzed the mechanism of logical thinking in the brain by constructing a theoretical model of this activity (Hosokawa et al., 2022). These results are expected to lead to further understanding of higher cognitive functions in humans and to future applications in the development of artificial intelligence that enables logical thinking.

■ For the construction of a next-generation dynamic AI, we proposed new AI prediction theory for predicting changes in various complex large-scale data, and developed a mathematical method for utilizing spatial information of numerous big data observed from real complex systems for the prediction of specific target variables. This "auto-reserver computation" can be expected to improve the prediction accuracy of diverse complex systems for which prediction is important (P.Chen et al., Nature Communications, 2020).

■ Aiming to develop a neuro-symbolic AI, we developed “Neural Module Network” that incorporates discrete operation modules for incorporating and utilizing some kind of known structure and knowledge in neural networks in a top-down manner. This challenges the major theme of artificial intelligence; the fusion of induction and deduction. First, using an approach that relaxes a discrete problem into a continuous problem, we succeeded in making the conventional data expansion search method in image recognition several hundred times faster by introducing differentiable search technique (Hataya et al., ECCV, 2020). We also studied a more general approach that performs discrete search as is. While this approach requires searching for intermediate labels representing combinations of modules, we developed a fast search method using an efficient graph representation (Wu et al., ACCV, 2020). Furthermore, by connecting an external knowledge graph to the transformer, we realized the world's first anticipation captioning, which predicts and describes the future situation of given images (Vo et al., CVPR 2023).

■ In image generation, we obtained many results on adversarial generative networks (GANs). First, we systematically developed methods for training GANs from small amounts of data and incomplete data (Katsumata et al., ICASSP 2021; Katsumata et al., CVPR 2022; Katsumata et al., WACV 2024). We also improved the quality of image generation by improving discriminative networks and introducing intermediate distributions (Yang et al., Access 2022; Yang et al., WACV 2023), and compressing GAN models through pruning (Vo et al., WACV 2022). Furthermore, based on these basic technologies, we developed a number of applications such as multilingual handwritten text generation (Jan et al., ACMMM 2021; Jan et al., ICDAR 2023), natural text removal from images (Jan et al., WACV 2020), story image sequence generation (Chen et al., EMNLP 2022), and many other applications.

References

1. Huu Hoang, Eric J. Lang, Yoshito Hirata, Isao T. Tokuda, Kazuyuki Aihara, Keisuke Toyama, Mitsuo Kawato , Nicolas Schweighofer. Electrical coupling controls dimensionality and chaotic firing of inferior olive neurons. PLoS Computational Biology 16(7): e1008075 (2020. 7. 30)
2. Pei Chen, Rui Liu, Kazuyuki Aihara, and Luonan Chen: "Autoreservoir computing for multistep ahead prediction based on the spatiotemporal information transformation," Nature Communications, Vol.11, Article No.4568, pp.1-15 (2020) (2020.9.11)
3. Soumya Chatterjee, Kenichi Ohki, R Clay Reid. “Chromatic micromaps in primary visual cortex” Nature Communications., .Vol.12: 2315. doi: 10.1038/s41467-021-22488-3 (2021.04.19).
4. Rui Liu, Jiayuan Zhong, Renhao Hong, Ely Chen, Kazuyuki Aihara, Pei Chen, and Luonan Chen, "Predicting Local COVID-19 Outbreaks and Infectious Disease Epidemics based on Landscape Network Entropy," Science Bulletin, pp.2265-2270 (2021). (2021.3.26:online, 2021.11.30:print)
5. Murakami T, Matsui T, Uemura M, Ohki K. Modular strategy for the development of hierarchical visual network in mice. Nature 608:578-585. (2022).
6. Yang Li, Jifan Shi, and Kazuyuki Aihara: "Mean-Field Analysis of Stuart-Landau Oscillator Networks with Symmetric Coupling and Dynamical Noise," Chaos, Vol.32, No.6, Article No.063114, pp.1-23 (2022). (2022.6.3: online)
7. Takayuki Hosokawa, Muyuan Xu, Yuichi Katori, Munekazu Yamada, Kazuyuki Aihara and Ken-Ichiro Tsutsui: "Monkey Prefrontal Single-Unit Activity Reflecting Category-Based Logical Thinking Process and Its Neural Network Model," Journal of Neuroscience, Vol.42, No.33, pp.6380-6391 (2022). (2022.7.8: online)
8. Rui Yang, Raphael Shu, and Hideki Nakayama: "Improving Noised Gradient Penalty with Synchronized Activation Function for Generative Adversarial Networks," IEICE Transactions on Information and Systems, Vol.E105-D, No.9, pp.1537-1545, (2022). (2022.9.1: online)
9. Jumpei Ukita, Kenichi Ohki　Adversarial attacks and defenses using feature-space stochasticity Neural Networks 167 875-889 2023
10. Ryuichiro Hataya, Zdenek Jan, Kazuki Yoshizoe, and Hideki Nakayama. Faster AutoAugment: Learning Augmentation Strategies using Backpropagation, In Proceedings of the 16th European Conference on Computer Vision (ECCV), 2020.　(2020.8.23)
11. Kai Katsumata, Hideki Nakayama. Semantic Image Synthesis from Inaccurate and Coarse Masks. Proceedings of the 46th IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pp.2285-2289 (2021). (2021.6.11)
12. Jan Zdenek and Hideki Nakayama. JokerGAN: Memory-Efficient Model for Handwritten Text Generation with Text Line Awareness. Proceedings of the 29th ACM International Conference on Multimedia (ACMMM), pp.5655-5663 (2021). (2021.10.23)
13. Duc Minh Vo, Akihiro Sugimoto, and Hideki Nakayama: "PPCD-GAN: Progressive Pruning and Class-Aware Distillation for Large-Scale Conditional GANs Compression," In Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV), pp.2436-2444, 2022. (2022.1.4)
14. Kai Katsumata, Duc Minh Vo, and Hideki Nakayama: "OSSGAN: Open-Set Semi-Supervised Image Generation," In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), pp.11185-11193, 2022. (2022.6.19)
15. Ryuichiro Hataya and Hideki Nakayama: "DJMix: Unsupervised Task-agnostic Image Augmentation for Improving Robustness of Convolutional Neural Networks," In Proceedings of the International Joint Conference on Neural Networks (IJCNN), pp.1-8, 2022. (2022.7.18)
16. Ikki Kishida and Hideki Nakayama: "Pixel to Binary Embedding Towards Robustness for CNNs," In Proceedings of the International Conference on Pattern Recognition (ICPR), pp.2279-2285, 2022. (2022.8.21)
17. Hong Chen, Duc Minh Vo, Hiroya Takamura, Yusuke Miyao, and Hideki Nakayama: "StoryER: Automatic Story Evaluation via Ranking, Rating and Reasoning," In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp.1739-1753, 2022. (2022.12.7)
18. Jan Zdenek and Hideki Nakayama. Handwritten Text Generation with Character-specific Encoding for Style Imitation. Proceedings of the 17th International Conference on Document Analysis and Recognition (ICDAR). 2023.8.31.
19. Rui Yang, Duc Minh Vo, and Hideki Nakayama. Indirect Adversarial Losses via an Intermediate Distribution for Training GANs. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp.4641-4650, 2023.1.3.

※For more detail, please refer to the URL below.
https://beyondai.jp/contents/wp-content/uploads/2024/03/Research-report_Kenichi-Ohki_202309.pdf