Details of a Researcher - MURATA Shingo

Papers - MURATA Shingo

Division display >> ／ All the affair displays 1 - 40 of about 40

World models and predictive coding for cognitive and developmental robotics: frontiers and challenges

Tadahiro Taniguchi, Shingo Murata, Masahiro Suzuki, Dimitri Ognibene, Pablo Lanillos, Emre Ugur, Lorenzo Jamone, Tomoaki Nakamura, Alejandra Ciria, Bruno Lara, Giovanni Pezzulo

Advanced Robotics (Informa UK Limited) 37 ( 13 ) 780 - 806 2023.06

Accepted, ISSN 0169-1864

Creating autonomous robots that can actively explore the environment, acquire knowledge and learn skills continuously is the ultimate achievement envisioned in cognitive and developmental robotics. Importantly, if the aim is to create robots that can continuously develop through interactions with their environment, their learning processes should be based on interactions with their physical and social world in the manner of human learning and cognitive development. Based on this context, in this paper, we focus on the two concepts of world models and predictive coding. Recently, world models have attracted renewed attention as a topic of considerable interest in artificial intelligence. Cognitive systems learn world models to better predict future sensory observations and optimize their policies, i.e. controllers. Alternatively, in neuroscience, predictive coding proposes that the brain continuously predicts its inputs and adapts to model its own dynamics and control behavior in its environment. Both ideas may be considered as underpinning the cognitive development of robots and humans capable of continual or lifelong learning. Although many studies have been conducted on predictive coding in cognitive robotics and neurorobotics, the relationship between world model-based approaches in AI and predictive coding in robotics has rarely been discussed. Therefore, in this paper, we clarify the definitions, relationships, and status of current research on these topics, as well as missing pieces of world models and predictive coding in conjunction with crucially related concepts such as the free-energy principle and active inference in the context of cognitive and developmental robotics. Furthermore, we outline the frontiers and challenges involved in world models and predictive coding toward the further integration of AI and robotics, as well as the creation of robots with real cognitive and developmental capabilities in the future.

Access to Document (DOI)

Interaction between Functional Connectivity and Neural Excitability in Autism: A Novel Framework for Computational Modeling and Application to Biological Data

Yuta Takahashi, Shingo Murata, Masao Ueki, Hiroaki Tomita, Yuichi Yamashita

Computational Psychiatry (Ubiquity Press, Ltd.) 7 ( 1 ) 14 - 29 2023.01

Accepted

Functional connectivity (FC) and neural excitability may interact to affect symptoms of autism spectrum disorder (ASD). We tested this hypothesis with neural network simulations, and applied it with functional magnetic resonance imaging (fMRI). A hierarchical recurrent neural network embodying predictive processing theory was subjected to a facial emotion recognition task. Neural network simulations examined the effects of FC and neural excitability on changes in neural representations by developmental learning, and eventually on ASD-like performance. Next, by mapping each neural network condition to subject subgroups on the basis of fMRI parameters, the association between ASD-like performance in the simulation and ASD diagnosis in the corresponding subject subgroup was examined. In the neural network simulation, the more homogeneous the neural excitability of the lower-level network, the more ASD-like the performance (reduced generalization and emotion recognition capability). In addition, in homogeneous networks, the higher the FC, the more ASD-like performance, while in heterogeneous networks, the higher the FC, the less ASD-like performance, demonstrating that FC and neural excitability interact. As an underlying mechanism, neural excitability determines the generalization capability of top-down prediction, and FC determines whether the model’s information processing will be top-down prediction-dependent or bottom-up sensory-input dependent. In fMRI datasets, ASD was actually more prevalent in subject subgroups corresponding to the network condition showing ASD-like performance. The current study suggests an interaction between FC and neural excitability, and presents a novel framework for computational modeling and biological application of a developmental learning process underlying cognitive alterations in ASD.

Access to Document (DOI)

Application of Robotic Predicitve Learning to Computational Psychiatry

Shingo Murata

Journal of the Robotics Society of Japan (The Robotics Society of Japan) 40 ( 9 ) 796 - 801 2022.11

Lead author, Corresponding author, ISSN 0289-1824

Access to Document (DOI)

Latent Representation in Human-Robot Interaction With Explicit Consideration of Periodic Dynamics

Taisuke Kobayashi, Shingo Murata, Tetsunari Inamura

IEEE Transactions on Human-Machine Systems (IEEE Transactions on Human-Machine Systems) 52 ( 5 ) 928 - 940 2022.10

Accepted, ISSN 2168-2291

This article presents a new data-driven framework for analyzing periodic physical human-robot interaction (pHRI) in latent state space. The model representing pHRI is critical for elaborating human understanding and/or robot control during pHRI. Recent advancements in deep learning technology would allow us to train such a model on a dataset collected from the actual pHRI. Our framework is based on a variational recurrent neural network (VRNN), which can process time-series data generated by a pHRI. This study modifies VRNN to explicitly integrate the latent dynamics from robot to human and to distinguish it from a human state estimate module. Furthermore, to analyze periodic motions, such as walking, we integrate VRNN with a new recurrent network based on reservoir computing (RC), which has random and fixed connections between numerous neurons. By boosting RC into a complex domain, periodic behavior can be represented as phase rotation in the complex domain without decaying the amplitude. A rope rotation/swinging experiment was used to validate the proposed framework. The proposed framework, trained on the collected experiment dataset, achieved the latent state space in which variation in periodic motions can be distinguished. The best prediction accuracy of the human observations and robot actions was obtained in such a well-distinguished space.

Access to Document (DOI)

Neural network modeling of altered facial expression recognition in autism spectrum disorders based on predictive processing framework

Yuta Takahashi, Shingo Murata, Hayato Idei, Hiroaki Tomita, Yuichi Yamashita

Scientific Reports (Springer Science and Business Media LLC) 11 ( 1 ) 2021.12

Accepted

<title>Abstract</title>The mechanism underlying the emergence of emotional categories from visual facial expression information during the developmental process is largely unknown. Therefore, this study proposes a system-level explanation for understanding the facial emotion recognition process and its alteration in autism spectrum disorder (ASD) from the perspective of predictive processing theory. Predictive processing for facial emotion recognition was implemented as a hierarchical recurrent neural network (RNN). The RNNs were trained to predict the dynamic changes of facial expression movies for six basic emotions without explicit emotion labels as a developmental learning process, and were evaluated by the performance of recognizing unseen facial expressions for the test phase. In addition, the causal relationship between the network characteristics assumed in ASD and ASD-like cognition was investigated. After the developmental learning process, emotional clusters emerged in the natural course of self-organization in higher-level neurons, even though emotional labels were not explicitly instructed. In addition, the network successfully recognized unseen test facial sequences by adjusting higher-level activity through the process of minimizing precision-weighted prediction error. In contrast, the network simulating altered intrinsic neural excitability demonstrated reduced generalization capability and impaired emotional clustering in higher-level neurons. Consistent with previous findings from human behavioral studies, an excessive precision estimation of noisy details underlies this ASD-like cognition. These results support the idea that impaired facial emotion recognition in ASD can be explained by altered predictive processing, and provide possible insight for investigating the neurophysiological basis of affective contact.

Access to Document (DOI)

Tool-Use Model to Reproduce the Goal Situations Considering Relationship Among Tools, Objects, Actions and Effects Using Multimodal Deep Neural Networks

Namiko Saito, Tetsuya Ogata, Hiroki Mori, Shingo Murata, Shigeki Sugano

Frontiers in Robotics and AI (Frontiers Media SA) 8 2021.09

Accepted

We propose a tool-use model that enables a robot to act toward a provided goal. It is important to consider features of the four factors; tools, objects actions, and effects at the same time because they are related to each other and one factor can influence the others. The tool-use model is constructed with deep neural networks (DNNs) using multimodal sensorimotor data; image, force, and joint angle information. To allow the robot to learn tool-use, we collect training data by controlling the robot to perform various object operations using several tools with multiple actions that leads different effects. Then the tool-use model is thereby trained and learns sensorimotor coordination and acquires relationships among tools, objects, actions and effects in its latent space. We can give the robot a task goal by providing an image showing the target placement and orientation of the object. Using the goal image with the tool-use model, the robot detects the features of tools and objects, and determines how to act to reproduce the target effects automatically. Then the robot generates actions adjusting to the real time situations even though the tools and objects are unknown and more complicated than trained ones.

Access to Document (DOI)

Homogeneous Intrinsic Neuronal Excitability Induces Overfitting to Sensory Noise: A Robot Model of Neurodevelopmental Disorder

Hayato Idei, Shingo Murata, Yuichi Yamashita, Tetsuya Ogata

Frontiers in Psychiatry (Frontiers Media SA) 11 ( 762 ) 1 - 15 2020.08

Research paper (scientific journal), Joint Work, Accepted

Neurodevelopmental disorders, including autism spectrum disorder, have been intensively investigated at the neural, cognitive, and behavioral levels, but the accumulated knowledge remains fragmented. In particular, developmental learning aspects of symptoms and interactions with the physical environment remain largely unexplored in computational modeling studies, although a leading computational theory has posited associations between psychiatric symptoms and an unusual estimation of information uncertainty (precision), which is an essential aspect of the real world and is estimated through learning processes. Here, we propose a mechanistic explanation that unifies the disparate observations via a hierarchical predictive coding and developmental learning framework, which is demonstrated in experiments using a neural network-controlled robot. The results show that, through the developmental learning process, homogeneous intrinsic neuronal excitability at the neural level induced via self-organization changes at the information processing level, such as hyper sensory precision and overfitting to sensory noise. These changes led to multifaceted alterations at the behavioral level, such as inflexibility, reduced generalization, and motor clumsiness. In addition, these behavioral alterations were accompanied by fluctuating neural activity and excessive development of synaptic connections. These findings might bridge various levels of understandings in autism spectrum and other neurodevelopmental disorders and provide insights into the disease processes underlying observed behaviors and brain activities in individual patients. This study shows the potential of neurorobotics frameworks for modeling how psychiatric disorders arise from dynamic interactions among the brain, body, and uncertain environments.

Access to Document (DOI)

Paradoxical sensory reactivity induced by functional disconnection in a robot model of neurodevelopmental disorder

Hayato Idei, Shingo Murata, Yuichi Yamashita, Tetsuya Ogata

Neural Networks (Elsevier BV) 138 150 - 163 2020.06

Joint Work, Accepted, ISSN 08936080

Neurodevelopmental disorders are characterized by heterogeneous and non-specific nature of their clinical symptoms. In particular, hyper- and hypo-reactivity to sensory stimuli are diagnostic features of autism spectrum disorder and are reported across many neurodevelopmental disorders. However, computational mechanisms underlying the unusual paradoxical behaviors remain unclear. In this study, using a robot controlled by a hierarchical recurrent neural network model with predictive processing and learning mechanism, we simulated how functional disconnection altered the learning process and subsequent behavioral reactivity to environmental change. The results show that, through the learning process, long-range functional disconnection between distinct network levels could simultaneously lower the precision of sensory information and higher-level prediction. The alteration caused a robot to exhibit sensory-dominated and sensory-ignoring behaviors ascribed to sensory hyper- and hypo-reactivity, respectively. As long-range functional disconnection became more severe, a frequency shift from hyporeactivity to hyperreactivity was observed, paralleling an early sign of autism spectrum disorder. Furthermore, local functional disconnection at the level of sensory processing similarly induced hyporeactivity due to low sensory precision. These findings suggest a computational explanation for paradoxical sensory behaviors in neurodevelopmental disorders, such as coexisting hyper- and hypo-reactivity to sensory stimulus. A neurorobotics approach may be useful for bridging various levels of understanding in neurodevelopmental disorders and providing insights into mechanisms underlying complex clinical symptoms.

Access to Document (DOI)

Sensory Uncertainty and Autism Spectrum Disorder: A Neurorobotics Simulation of Symptoms

Hayato Idei, Shingo Murata, Tetsuya Ogata, Yuichi Yamashita

Seishin Igaku ((株)医学書院) 62 ( 2 ) 219 - 229 2020.02

Research paper (scientific journal), Joint Work, Accepted, ISSN 0488-1281

＜文献概要＞脳の計算理論の発展に伴い,それを精神疾患の病態理解に応用する試みへの期待が,近年,高まっている。特に,脳の予測符号化モデルに基づいた研究では,感覚情報の不確実性を推定する神経機能の変調と精神疾患との関連を示唆する概念的な仮説が提示されてきたが,実際の臨床で観察される行動レベルでの病像との間にはギャップが存在している。本論文では,神経ロボティクスの手法を用いることで,実験的にこの理論と臨床像との橋渡しを行った研究を紹介する。実験では,自閉スペクトラム症に特徴的な限定された反復的な行動様式や行為の停止といった多様な症状が,不確実性の過小評価と過大評価による異なるプロセスから生じ得ることが示され,精神症状のmultifinal性とequifinal性の一側面が捉えられた。また,病態理解にとどまらず,症状改善のための環境調整的な介入への理論的示唆といった,計算論的精神医学の臨床研究への貢献可能性が示された。

Achieving Human–Robot Collaboration with Dynamic Goal Inference by Gradient Descent

Shingo Murata, Wataru Masuda, Jiayi Chen, Hiroaki Arie, Tetsuya Ogata, Shigeki Sugano

In Proceedings of the 26th International Conference on Neural Information Processing (ICONIP 2019) (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)) 11954 LNCS 579 - 590 2019.12

Research paper (international conference proceedings), Joint Work, Accepted, ISSN 9783030367107

Collaboration with a human partner is a challenging task expected of intelligent robots. To realize this, robots need the ability to share a particular goal with a human and dynamically infer whether the goal state is changed by the human. In this paper, we propose a neural network-based computational framework with a gradient-based optimization of the goal state that enables robots to achieve this ability. The proposed framework consists of convolutional variational autoencoders (ConvVAEs) and a recurrent neural network (RNN) with a long short-term memory (LSTM) architecture that learns to map a given goal image for collaboration to visuomotor predictions. More specifically, visual and goal feature states are first extracted by the encoder of the respective ConvVAEs. Visual feature and motor predictions are then generated by the LSTM based on their current state and are conditioned according to the extracted goal feature state. During collaboration after the learning process, the goal feature state is optimized by gradient descent to minimize errors between the predicted and actual visual feature states. This enables the robot to dynamically infer situational (goal) changes of the human partner from visual observations alone. The proposed framework is evaluated by conducting experiments on a human–robot collaboration task involving object assembly. Experimental results demonstrate that a robot equipped with the proposed framework can collaborate with a human partner through dynamic goal inference even when the situation is ambiguous.

Access to Document (DOI)

Large-scale Data Collection for Goal-directed Drawing Task with Self-report Psychiatric Symptom Questionnaires via Crowdsourcing

Shingo Murata, Hikaru Yanagida, Kentaro Katahira, Shinsuke Suzuki, Tetsuya Ogata, Yuichi Yamashita

In Proceedings of the 2019 IEEE International Conference on Systems, Man, and Cybernetics (SMC 2019) (Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics) 2019-October 3839 - 3845 2019.10

Research paper (international conference proceedings), Joint Work, Accepted, ISSN 9781728145693

Drawing is a representative human cognitive ability and may mirror cognitive characteristics including those associated with psychiatric symptoms. Therefore, analysis of drawing data collected from various populations such as healthy people and psychiatric patients may be beneficial for better understanding human cognition. However, collecting such large-scale data about the relationship between drawing and cognitive/personality traits offline-in a laboratory-is a difficult issue. To overcome this issue, we devised a novel experimental paradigm involving a goal-directed drawing task conducted online-on the eb-with participants recruited via a crowdsourcing platform. With the assistance of 1155 participants with differing levels of psychiatric symptoms, we collected a total of 194, 040 trajectory data and answers to seven different self-report psychiatric symptom questionnaires comprising 181 items. We visualized the collected trajectory data and performed an exploratory factor analysis on the correlation matrix of the psychiatric symptom questionnaire items. Our results suggest that there were associations between psychiatric symptoms represented by specific psychiatric factors and atypical behavior observed while performing the goal-directed drawing task. This indicates the efficacy of a dimensional approach to large-scale online experiments with respect to clinical psychiatry.

Access to Document (DOI)

Looking Back and Ahead: Adaptation and Planning by Gradient Descent

Shingo Murata, Hiroki Sawa, Shigeki Sugano, Tetsuya Ogata

In Proceedings of the Ninth Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics (ICDL-EpiRob 2019) (2019 Joint IEEE 9th International Conference on Development and Learning and Epigenetic Robotics, ICDL-EpiRob 2019) 151 - 156 2019.08

Research paper (international conference proceedings), Joint Work, Accepted, ISSN 9781538681282

Adaptation and planning are crucial for both biological and artificial agents. In this study, we treat these as an inference problem that we solve using a gradient-based optimization approach. We propose adaptation and planning by gradient descent (APGraDe), a gradient-based computational framework with a hierarchical recurrent neural network (RNN) for adaptation and planning. This framework computes (counterfactual) prediction errors by looking back on past situations based on actual observations and by looking ahead to future situations based on preferred observations (or goal). The internal state of the higher level of the RNN is optimized in the direction of minimizing these errors. The errors for the past contribute to the adaptation while errors for the future contribute to the planning. The proposed APGraDe framework is implemented in a humanoid robot and the robot performs a ball manipulation task with a human experimenter. Experimental results show that given a particular preference, the robot can adapt to unexpected situations while pursuing its own preference through the planning of future actions.

Access to Document (DOI)

A Neurorobotics Simulation of Autistic Behavior Induced by Unusual Sensory Precision.

Hayato Idei, Shingo Murata, Yiwen Chen, Yuichi Yamashita, Jun Tani, Tetsuya Ogata

Computational psychiatry (Cambridge, Mass.) 2 164 - 182 2018.12

Research paper (scientific journal), Joint Work, Accepted

Recently, applying computational models developed in cognitive science to psychiatric disorders has been recognized as an essential approach for understanding cognitive mechanisms underlying psychiatric symptoms. Autism spectrum disorder is a neurodevelopmental disorder that is hypothesized to affect information processes in the brain involving the estimation of sensory precision (uncertainty), but the mechanism by which observed symptoms are generated from such abnormalities has not been thoroughly investigated. Using a humanoid robot controlled by a neural network using a precision-weighted prediction error minimization mechanism, it is suggested that both increased and decreased sensory precision could induce the behavioral rigidity characterized by resistance to change that is characteristic of autistic behavior. Specifically, decreased sensory precision caused any error signals to be disregarded, leading to invariability of the robot's intention, while increased sensory precision caused an excessive response to error signals, leading to fluctuations and subsequent fixation of intention. The results may provide a system-level explanation of mechanisms underlying different types of behavioral rigidity in autism spectrum and other psychiatric disorders. In addition, our findings suggest that symptoms caused by decreased and increased sensory precision could be distinguishable by examining the internal experience of patients and neural activity coding prediction error signals in the biological brain.

Access to Document (DOI)

Tool-use Model Considering Tool Selection by a Robot using Deep Learning

Namiko Saito, Kitae Kim, Shingo Murata, Tetsuya Ogata, Shigeki Sugano

In Proceedings of the 2018 IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids 2018) (IEEE-RAS International Conference on Humanoid Robots) 2018-November 814 - 819 2018.11

Research paper (international conference proceedings), Joint Work, Accepted, ISSN 9781538672839

We propose a tool-use model that can select tools that require neither labeling nor modeling of the environment and actions. With this model, a robot can choose a tool by itself and perform the operation that matches a human command and the environmental situation. To realize this, we use deep learning to train sensory motor data recorded during tool selection and tool use as experienced by a robot. The experience includes two types of selection, namely according to function and according to size, thereby allowing the robot to handle both situations. For evaluation, the robot is required to generate motion either in an untrained situation or using an untrained tool. We confirm that the robot can choose and use a tool that is suitable for achieving the target task.

Access to Document (DOI)

Detecting Features of Tools, Objects, and Actions from Effects in a Robot using Deep Learning

Namiko Saito, Kitae Kim, Shingo Murata, Tetsuya Ogata, Shigeki Sugano

In Proceedings of the Eighth Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics (ICDL-EpiRob 2018) (2018 Joint IEEE 8th International Conference on Development and Learning and Epigenetic Robotics, ICDL-EpiRob 2018) 91 - 96 2018.09

Research paper (international conference proceedings), Joint Work, Accepted, ISSN 9781538661109

We propose a tool-use model that can detect the features of tools, target objects, and actions from the provided effects of object manipulation. We construct a model that enables robots to manipulate objects with tools, using infant learning as a concept. To realize this, we train sensory-motor data recorded during a tool-use task performed by a robot with deep learning. Experiments include four factors: (1) tools, (2) objects, (3) actions, and (4) effects, which the model considers simultaneously. For evaluation, the robot generates predicted images and motions given information of the effects of using unknown tools and objects. We confirm that the robot is capable of detecting features of tools, objects, and actions by learning the effects and executing the task.

Access to Document (DOI)

Dynamic Motion Generation by Flexible-Joint Robot based on Deep Learning using Images

Yuheng Wu, Kuniyuki Takahashi, Hiroki Yamada, Kitae Kim, Shingo Murata, Shigeki Sugano, Tetsuya Ogata

Research paper (international conference proceedings), Joint Work, Accepted, ISSN 9781538661109

Robots with flexible joints have recently been attracting attention from researchers because such robots can passively adapt to environmental changes and realize dynamic motion that uses inertia. In previous research, body-model acquisition using deep learning was proposed and dynamic motion learning was achieved. However, using the end-effector position as a visual feedback signal to train a robot limits what the robot can know to only the relation between the task and itself, instead of the relation between the environment and itself. In this research, we propose to use images as a feedback signal so that the robot can have a sense of the overall situation within the task environment. This motion learning is performed via deep learning using raw image data. In an experiment, we let a robot perform task motions once to acquire motor and image data. Then, we used a convolutional auto-encoder to extract image features from raw image data. The extracted image features were used in combination with motor data to train a recurrent neural network. As a result, motion learning through deep learning from image data allowed the robot to acquire environmental information and conduct tasks that require consideration of environmental changes, making use of its advantage of passive adaptation.

Access to Document (DOI)

Multiclass Classification of Driver Perceived Workload Using Long Short-Term Memory based Recurrent Neural Network

Udara Manawadu, Takahiro Kawano, Shingo Murata, Mitsuhiro Kamezaki, Junya Muramatsu, Shigeki Sugano

In Proceedings of the 2018 IEEE Intelligent Vehicles Symposium (IV'18) (IEEE Intelligent Vehicles Symposium, Proceedings) 2018-June 2009 - 2014 2018.06

Research paper (international conference proceedings), Joint Work, Accepted, ISSN 9781538644522

Human sensing enables intelligent vehicles to provide driver-adaptive support by classifying perceived workload into multiple levels. Objective of this study is to classify driver workload associated with traffic complexity into five levels. We conducted driving experiments in systematically varied traffic complexity levels in a simulator. We recorded driver physiological signals including electrocardiography, electrodermal activity, and electroencephalography. In addition, we integrated driver performance and subjective workload measures. Deep learning based models outperform statistical machine learning methods when dealing with dynamic time-series data with variable sequence lengths. We show that our long short-term memory based recurrent neural network model can classify driver perceived-workload into five classes with an accuracy of 74.5%. Since perceived workload differ between individual drivers for the same traffic situation, our results further highlight the significance of including driver characteristics such as driving style and workload sensitivity to achieve higher classification accuracy.

Access to Document (DOI)

Acquisition of Viewpoint Transformation and Action Mappings via Sequence to Sequence Imitative Learning by Deep Neural Networks

Ryoichi Nakajo, Shingo Murata, Hiroaki Arie, Tetsuya Ogata

Frontiers in Neurorobotics (Frontiers in Neurorobotics) 12 ( 46 ) 1 - 12 2018

Research paper (scientific journal), Joint Work, Accepted

We propose an imitative learning model that allows a robot to acquire positional relations between the demonstrator and the robot, and to transform observed actions into robotic actions. Providing robots with imitative capabilities allows us to teach novel actions to them without resorting to trial-and-error approaches. Existing methods for imitative robotic learning require mathematical formulations or conversion modules to translate positional relations between demonstrators and robots. The proposed model uses two neural networks, a convolutional autoencoder (CAE) and a multiple timescale recurrent neural network (MTRNN). The CAE is trained to extract visual features from raw images captured by a camera. The MTRNN is trained to integrate sensory-motor information and to predict next states. We implement this model on a robot and conducted sequence to sequence learning that allows the robot to transform demonstrator actions into robot actions. Through training of the proposedmodel, representations of actions, manipulated objects, and positional relations are formed in the hierarchical structure of the MTRNN. After training, we confirm capability for generating unlearned imitative patterns.

Access to Document (DOI)

Learning to Achieve Different Levels of Adaptability for Human–Robot Collaboration Utilizing a Neuro-dynamical System

Shingo Murata, Yuxi Li, Hiroaki Arie, Tetsuya Ogata, Shigeki Sugano

IEEE Transactions on Cognitive and Developmental Systems (IEEE Transactions on Cognitive and Developmental Systems) 10 ( 3 ) 712 - 725 2018

Research paper (scientific journal), Joint Work, Accepted, ISSN 23798920

Intelligent robots are expected to collaboratively work with humans in dynamically changing daily life environments. To realize successful human-robot collaboration, robots need to deal with latent spatiotemporal complexity in the workspace and the task. To overcome this crucial issue, three levels of adaptability-motion modification, action selection, and role switching-should be considered. This paper demonstrates that a single hierarchically organized neuro-dynamical system called a multiple timescale recurrent neural network can achieve these levels of adaptability by utilizing hierarchical and bidirectional information processing. The system is implemented in a humanoid robot and the robot is required to learn to perform collaborative tasks in which some parts must be performed by a human partner and others by the robot. Experimental results show that the robot can perform collaborative tasks under dynamically changing environments, including both learned and unlearned situations, thanks to different levels of adaptability acquired in the system.

Access to Document (DOI)

Estimating driver workload with systematically varying traffic complexity using machine learning: Experimental design

Udara E. Manawadu, Takahiro Kawano, Shingo Murata, Mitsuhiro Kamezaki, Shigeki Sugano

Advances in Intelligent Systems and Computing (Springer Verlag) 722 106 - 111 2018

Research paper (international conference proceedings), Joint Work, Accepted, ISSN 2194-5357

Traffic complexity is one of the factors affecting driver workload. In order to study the relationship between traffic complexity levels and workload, a designed experiment is required, especially to vary traffic flow parameters systematically in a simulated environment. This paper describes the experimental design of a simulator study for developing a computational model to estimate the behavior of driver workload based on traffic complexity. Driving simulators allow creating and testing different traffic scenarios and manipulating independent variables to improve the quality of data, as compared to real world experiments. Physiological responses such as heart rate, skin conductance, and pupil size have been found to be related to workload. By adapting a data-driven method, we integrated electrocardiography sensors, electro-dermal activity sensors, and eye-tracker to acquire driver physiological signals and gaze information. Preliminary results show a positive correlation between traffic complexity levels and corresponding physiological responses, performance, and subjective measures.

Access to Document (DOI)

Representation Learning of Logic Words by an RNN: From Word Sequences to Robot Actions

Tatsuro Yamada, Shingo Murata, Hiroaki Arie, Tetsuya Ogata

FRONTIERS IN NEUROROBOTICS (FRONTIERS MEDIA SA) 11 ( 70 ) 1 - 18 2017.12

Research paper (scientific journal), Joint Work, Accepted, ISSN 1662-5218

An important characteristic of human language is compositionality. We can efficiently express a wide variety of real-world situations, events, and behaviors by compositionally constructing the meaning of a complex expression from a finite number of elements. Previous studies have analyzed how machine-learning models, particularly neural networks, can learn from experience to represent compositional relationships between language and robot actions with the aim of understanding the symbol grounding structure and achieving intelligent communicative agents. Such studies have mainly dealt with the words (nouns, adjectives, and verbs) that directly refer to real-world matters. In addition to these words, the current study deals with logic words, such as "not," "and," and "or" simultaneously. These words are not directly referring to the real world, but are logical operators that contribute to the construction of meaning in sentences. In human-robot communication, these words may be used often. The current study builds a recurrent neural network model with long short-term memory units and trains it to learn to translate sentences including logic words into robot actions. We investigate what kind of compositional representations, which mediate sentences and robot actions, emerge as the network's internal states via the learning process. Analysis after learning shows that referential words are merged with visual information and the robot's own current state, and the logical words are represented by the model in accordance with their functions as logical operators. Words such as "true," "false," and "not" work as non-linear transformations to encode orthogonal phrases into the same area in a memory cell state space. The word "and," which required a robot to lift up both its hands, worked as if it was a universal quantifier. The word "or," which required action generation that looked apparently random, was represented as an unstable space of the network's dynamical system.

Access to Document (DOI)

Reduced Behavioral Flexibility by Aberrant Sensory Precision in Autism Spectrum Disorder: A Neurorobotics Experiment

Hayato Idei, Shingo Murata, Yiwen Chen, Yuichi Yamashita, Jun Tani, Tetsuya Ogata

In Proceedings of the Seventh Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics (ICDL-EpiRob 2017) (7th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics, ICDL-EpiRob 2017) 2018-January 271 - 276 2017.09

Research paper (international conference proceedings), Joint Work, Accepted, ISSN 9781538637159

Recently, the importance of the application of computational models utilized in cognitive neuroscience to psychiatric disorders has been recognized. This study utilizes a recurrent neural network model to test aberrant sensory precision, a normative theory of autism spectrum disorder. We particularly focus on the effects of increased and decreased sensory precision on adaptive behavior based on a prediction error minimization mechanism. To distinguish dysfunction at the behavioral and network levels, we employ a humanoid robot driven by a neural network and observe ball-playing interactions with a human experimenter. Experimental results show that behavioral rigidity characteristic of autism spectrum disorder - including stopping movement and repetitive behavior - was generated from both increased and decreased sensory precision, but through different processes at the network level. These results may provide a system-level explanation of different types of behavioral rigidity in psychiatric diseases such as compulsions and stereotypies. The results also support a system-level model for autism spectrum disorder that suggests core deficits in estimating the uncertainty of sensory evidence.

Access to Document (DOI)

Learning to Perceive the World as Probabilistic or Deterministic via Interaction With Others: A Neuro-Robotics Experiment

Shingo Murata, Yuichi Yamashita, Hiroaki Arie, Tetsuya Ogata, Shigeki Sugano, Jun Tani

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS (IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC) 28 ( 4 ) 830 - 848 2017.04

Research paper (scientific journal), Joint Work, Accepted, ISSN 2162-237X

We suggest that different behavior generation schemes, such as sensory reflex behavior and intentional proactive behavior, can be developed by a newly proposed dynamic neural network model, named stochastic multiple timescale recurrent neural network (S-MTRNN). The model learns to predict subsequent sensory inputs, generating both their means and their uncertainty levels in terms of variance (or inverse precision) by utilizing its multiple timescale property. This model was employed in robotics learning experiments in which one robot controlled by the S-MTRNN was required to interact with another robot under the condition of uncertainty about the other's behavior. The experimental results show that self-organized and sensory reflex behavior-based on probabilistic prediction-emerges when learning proceeds without a precise specification of initial conditions. In contrast, intentional proactive behavior with deterministic predictions emerges when precise initial conditions are available. The results also showed that, in situations where unanticipated behavior of the other robot was perceived, the behavioral context was revised adequately by adaptation of the internal neural dynamics to respond to sensory inputs during sensory reflex behavior generation. On the other hand, during intentional proactive behavior generation, an error regression scheme by which the internal neural activity was modified in the direction of minimizing prediction errors was needed for adequately revising the behavioral context. These results indicate that two different ways of treating uncertainty about perceptual events in learning, namely, probabilistic modeling and deterministic modeling, contribute to the development of different dynamic neuronal structures governing the two types of behavior generation schemes.

Access to Document (DOI)

Mixing actual and predicted sensory states based on uncertainty estimation for flexible and robust robot behavior

Shingo Murata, Wataru Masuda, Saki Tomioka, Tetsuya Ogata, Shigeki Sugano

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Springer Verlag) 10613 11 - 18 2017

Research paper (international conference proceedings), Joint Work, Accepted, ISSN 1611-3349

In this paper, we propose a method to dynamically modulate the input state of recurrent neural networks (RNNs) so as to realize flexible and robust robot behavior. We employ the so-called stochastic continuous-time RNN (S-CTRNN), which can learn to predict the mean and variance (or uncertainty) of subsequent sensorimotor information. Our proposed method uses this estimated uncertainty to determine a mixture ratio for combining actual and predicted sensory states of network input. The method is evaluated by conducting a robot learning experiment in which a robot is required to perform a sensory-dependent task and a sensory-independent task. The sensory-dependent task requires the robot to incorporate meaningful sensory information, and the sensory-independent task requires the robot to ignore irrelevant sensory information. Experimental results demonstrate that a robot controlled by our proposed method exhibits flexible and robust behavior, which results from dynamic modulation of the network input on the basis of the estimated uncertainty of actual sensory states.

Access to Document (DOI)

Achieving Different Levels of Adaptability for Human–Robot Collaboration Utilizing a Neuro-Dynamical System

Yuxi Li, Shingo Murata, Hiroaki Arie, Tetsuya Ogata, Shigeki Sugano

Workshop on Bio-inspired Social Robot Learning in Home Scenarios, The 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2016) 1 - 6 2016.10

Research paper (international conference proceedings), Joint Work, Accepted

Dynamical Integration of Language and Behavior in a Recurrent Neural Network for Human-Robot Interaction

Tatsuro Yamada, Shingo Murata, Hiroaki Arie, Tetsuya Ogata

FRONTIERS IN NEUROROBOTICS (FRONTIERS MEDIA SA) 10 ( 5 ) 1 - 17 2016.07

Research paper (scientific journal), Joint Work, Accepted, ISSN 1662-5218

To work cooperatively with humans by using language, robots must not only acquire a mapping between language and their behavior but also autonomously utilize the mapping in appropriate contexts of interactive tasks online. To this end, we propose a novel learning method linking language to robot behavior by means of a recurrent neural network. In this method, the network learns from correct examples of the imposed task that are given not as explicitly separated sets of language and behavior but as sequential data constructed from the actual temporal flow of the task. By doing this, the internal dynamics of the network models both language behavior relationships and the temporal patterns of interaction. Here, "internal dynamics" refers to the time development of the system defined on the fixed-dimensional space of the internal states of the context layer. Thus, in the execution phase, by constantly representing where in the interaction context it is as its current state, the network autonomously switches between recognition and generation phases without any explicit signs and utilizes the acquired mapping in appropriate contexts. To evaluate our method, we conducted an experiment in which a robot generates appropriate behavior responding to a human's linguistic instruction. After learning, the network actually formed the attractor structure representing both language behavior relationships and the task's temporal pattern in its internal dynamics. In the dynamics, language behavior mapping was achieved by the branching structure. Repetition of human's instruction and robot's behavioral response was represented as the cyclic structure, and besides, waiting to a subsequent instruction was represented as the fixed-point attractor. Thanks to this structure, the robot was able to interact online with a human concerning the given task by autonomously switching phases.

Access to Document (DOI)

Self and Non-self Discrimination Mechanism Based on Predictive Learning with Estimation of Uncertainty

Ryoichi Nakajo, Maasa Takahashi, Shingo Murata, Hiroaki Arie, Tetsuya Ogata

NEURAL INFORMATION PROCESSING, ICONIP 2016, PT IV (SPRINGER INT PUBLISHING AG) 9950 228 - 235 2016

Research paper (international conference proceedings), Joint Work, Accepted, ISSN 0302-9743

In this paper, we propose a model that can explain the mechanism of self and non-self discrimination. Infants gradually develop their abilities for self-other cognition through interaction with the environment. Predictive learning has been widely used to explain the mechanism of infants' development. We hypothesized that infants' cognitive abilities are developed through predictive learning and the uncertainty estimation of their sensory-motor inputs. We chose a stochastic continuous time recurrent neural network, which is a dynamical neural network model, to predict uncertainties as variances. From the perspective of cognitive developmental robotics, a predictive learning experiment with a robot was performed. The results indicate that training made the robot predict the regions related to its body more easily. We confirmed that self and non-self cognitive abilities might be acquired through predictive learning with uncertainty estimation.

Access to Document (DOI)

Emergence of Interactive Behaviors between Two Robots by Prediction Error Minimization Mechanism

Yiwen Chen, Shingo Murata, Hiroaki Arie, Tetsuya Ogata, Jun Tani, Shigeki Sugano

2016 JOINT IEEE INTERNATIONAL CONFERENCE ON DEVELOPMENT AND LEARNING AND EPIGENETIC ROBOTICS (ICDL-EPIROB) (IEEE) 302 - 307 2016

Research paper (international conference proceedings), Joint Work, Accepted

This study demonstrates that the prediction error minimization (PEM) mechanism can account for the emergence of reciprocal interaction between two cognitive agents. During interactive processes, alternation of forming and deforming interactions may be triggered by various internal and external causes. We focus in particular on external causes derived from a dynamic and uncertain environment. Two small humanoid robots controlled by an identical dynamic neural network model using the PEM mechanism were trained to achieve a set of coherent ball-playing interactions between them. The two robots predict each other in a top-down way while they try to minimize the prediction errors derived from the unstable ball dynamics or the external cause in a bottom-up way by using the PEM mechanism. The experimental results showed that switching among the set of trained interactive ball plays between the two robots appears spontaneously. The analysis clarified how each complementary behavior can be generated via mutual adaptation between the two robots by undertaking top-down and bottom-up interaction in each individual dynamic neural network model by using the PEM mechanism.

Access to Document (DOI)

Dynamical Linking of Positive and Negative Sentences to Goal-Oriented Robot Behavior by Hierarchical RNN

Tatsuro Yamada, Shingo Murata, Hiroaki Arie, Tetsuya Ogata

ARTIFICIAL NEURAL NETWORKS AND MACHINE LEARNING - ICANN 2016, PT I (SPRINGER INT PUBLISHING AG) 9886 339 - 346 2016

Research paper (international conference proceedings), Joint Work, Accepted, ISSN 0302-9743

Meanings of language expressions are constructed not only from words grounded in real-world matters, but also from words such as "not" that participate in the construction by working as logical operators. This study proposes a connectionist method for learning and internally representing functions that deal with both of these word groups, and grounding sentences constructed from them in corresponding behaviors just by experiencing raw sequential data of an imposed task. In the experiment, a robot implemented with a recurrent neural network is required to ground imperative positive and negative sentences given as a sequence of words in corresponding goal-oriented behavior. Analysis of the internal representations reveals that the network fulfilled the requirement by extracting XOR problems implicitly included in the target sequences and solving them by learning to represent the logical operations in its nonlinear dynamics in a self-organizing manner.

Access to Document (DOI)

Analysis of Imitative Interactions between Humans and a Robot with a Neuro-dynamical System

Shingo Murata, Kai Hirano, Hiroaki Arie, Shigeki Sugano, Tetsuya Ogata

2016 IEEE/SICE INTERNATIONAL SYMPOSIUM ON SYSTEM INTEGRATION (SII) (IEEE) 343 - 348 2016

Research paper (international conference proceedings), Joint Work, Accepted, ISSN 2474-2317

Human communicative behavior is both dynamic and bidirectional. This study aims to analyze such behavior by conducting imitative interactions between human subjects and a humanoid robot that has a neuro-dynamical system. For this purpose, we take a robot-centered approach in which the change in robot performance according to difference in human partner is analyzed, rather than adopting the typical human-centered approach. A small humanoid robot equipped with a neuro-dynamical system learns imitative arm movement patterns and interacts with humans after the learning process. We analyze the interactive phenomena by different methods, including principal component analysis and use of a recurrence plot. Through this analysis, we demonstrate that different classes of interactions can be observed in the contextual dynamics of the neuro-dynamical system.

Access to Document (DOI)

Predictive Learning with Uncertainty Estimation for Modeling Infants' Cognitive Development with Caregivers: A Neurorobotics Experiment

Shingo Murata, Saki Tomioka, Ryoichi Nakajo, Tatsuro Yamada, Hiroaki Arie, Tetsuya Ogata, Shigeki Sugano

5TH INTERNATIONAL CONFERENCE ON DEVELOPMENT AND LEARNING AND ON EPIGENETIC ROBOTICS (ICDL-EPIROB) (IEEE) 302 - 307 2015

Research paper (international conference proceedings), Joint Work, Accepted

Dynamic interactions with caregivers are essential for infants to develop cognitive abilities, including aspects of action, perception, and attention. We hypothesized that these abilities can be acquired through the predictive learning of sensory inputs including their uncertainty (inverse precision) in terms of variance. To examine our hypothesis from the perspective of cognitive developmental robotics, we conducted a neurorobotics experiment involving a ball-playing interaction task between a human experimenter representing a caregiver and a small humanoid robot representing an infant. The robot was equipped with a dynamic generative model called a stochastic continuous-time recurrent neural network (S-CTRNN). The S-CTRNN learned to generate predictions about both the visuo-proprioceptive states of the robot and the uncertainty of these states by minimizing a negative log-likelihood consisting of log-uncertainty and precision-weighted prediction error. The experimental results showed that predictive learning with uncertainty estimation enabled the robot to acquire infant-like cognitive abilities through dynamic interactions with the experimenter. We also discuss the effects of infant-directed modifications observed in caregiver-infant interactions on the development of these abilities.

Access to Document (DOI)

Attractor Representations of Language-behavior Structure in a Recurrent Neural Network for Human-robot Interaction

Tatsuro Yamada, Shingo Murata, Hiroaki Arie, Tetsuya Ogata

2015 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS) (IEEE) 4179 - 4184 2015

Research paper (international conference proceedings), Joint Work, Accepted, ISSN 2153-0858

In recent years there has been increased interest in studies that explore integrative learning of language and other modalities by using neural network models. However, for practical application to human-robot interaction, the acquired semantic structure between language and meaning has to be available immediately and repeatably whenever necessary, just as in everyday communication. As a solution to this problem, this study proposes a method in which a recurrent neural network self-organizes cyclic attractors that reflect semantic structure and represent interaction flows in its internal dynamics. To evaluate this method we design a simple task in which a human verbally directs a robot, which responds appropriately. Training the network with training data that represent the interaction series, the cyclic attractors that reflect the semantic structure is self-organized. The network first receives a verbal direction, and its internal state moves according to the first half of the cyclic attractors with branch structures corresponding to semantics. After that, the internal state reaches a potential to generate appropriate behavior. Finally, the internal state moves to the second half and converges on the initial point of the cycle while generating the appropriate behavior. By self-organizing such an internal structure in its forward dynamics, the model achieves immediate and repeatable response to linguistic directions. Furthermore, the network self-organizes a fixed-point attractor, and so able to wait for directions. It can thus repeat the interaction flexibly without explicit turn-taking signs.

Access to Document (DOI)

Acquisition of Viewpoint Representation in Imitative Learning from Own Sensory-Motor Experiences

Ryoichi Nakajo, Shingo Murata, Hiroaki Arie, Tetsuya Ogata

5TH INTERNATIONAL CONFERENCE ON DEVELOPMENT AND LEARNING AND ON EPIGENETIC ROBOTICS (ICDL-EPIROB) (IEEE) 326 - 331 2015

Research paper (international conference proceedings), Joint Work, Accepted

This paper introduces an imitative model that enables a robot to acquire viewpoints of the self and others from its own sensory-motor experiences. This is important for recognizing and imitating actions generated from various directions. Existing methods require coordinate transformations input by human designers or complex learning modules to acquire a viewpoint. In the proposed model, several neurons dedicated to generated actions and viewpoints of the self and others are added to a dynamic nueral network model reffered as continuous time recurrent neural network (CTRNN). The training data are labeled with types of actions and viewpoints, and are linked to each internal state. We implemented this model in a robot and trained the model to perform actions of object manipulation. Representations of behavior and viewpoint were formed in the internal states of the CTRNN. In addition, we analyzed the initial values of the internal states that represent the viewpoint information. We confirmed the distinction of the observational perspective of other's actions self-organized in the space of the initial values. Combining the initial values of the internal states that describe the behavior and the viewpoint, the system can generate unlearned data.

Access to Document (DOI)

Generation of Sensory Reflex Behavior versus Intentional Proactive Behavior in Robot Learning of Cooperative Interactions with Others

Shingo Murata, Yuichi Yamashita, Hiroaki Arie, Tetsuya Ogata, Jun Tani, Shigeki Sugano

FOUTH JOINT IEEE INTERNATIONAL CONFERENCES ON DEVELOPMENT AND LEARNING AND EPIGENETIC ROBOTICS (IEEE ICDL-EPIROB 2014) (IEEE) 242 - 248 2014

Research paper (international conference proceedings), Joint Work, Accepted

This paper investigates the essential difference between two types of behavior generation schemes, namely, sensory reflex behavior generation and intentional proactive behavior generation, by proposing a dynamic neural network model referred to as stochastic multiple-timescale recurrent neural network (S-MTRNN). The proposed model was employed in an experiment involving robots learning to cooperate with others under the condition of potential unpredictability of the others' behaviors. The results of the learning experiment showed that sensory reflex behavior was generated by a self-organizing probabilistic prediction mechanism when the initial sensitivity characteristics in the network dynamics were not utilized in the learning process. In contrast, proactive behavior with a deterministic prediction mechanism was developed when the initial sensitivity was utilized. It was further shown that in situations where unexpected behaviors of others were observed, the behavioral context was re-situated by adaptation of the internal neural dynamics by means of simple sensory reflexes in the former case. In the latter case, the behavioral context was re-situated by error regression of the internal neural activity rather than by sensory reflex. The role of the top-down and bottom-up interactions in dealing with unexpected situations is discussed.

Access to Document (DOI)

Tool-body assimilation model based on body babbling and a neuro-dynamical system for motion generation

Kuniyuki Takahashi, Tetsuya Ogata, Hadi Tjandra, Shingo Murata, Hiroaki Arie, Shigeki Sugano

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Springer Verlag) 8681 363 - 370 2014

Research paper (international conference proceedings), Joint Work, Accepted, ISSN 1611-3349

We propose a model for robots to use tools without predetermined parameters based on a human cognitive model. Almost all existing studies of robot using tool require predetermined motions and tool features, so the motion patterns are limited and the robots cannot use new tools. Other studies use a full search for new tools
however, this entails an enormous number of calculations. We built a model for tool use based on the phenomenon of tool-body assimilation using the following approach: We used a humanoid robot model to generate random motion, based on human body babbling. These rich motion experiences were then used to train a recurrent neural network for modeling a body image. Tool features were self-organized in the parametric bias modulating the body image according to the used tool. Finally, we designed the neural network for the robot to generate motion only from the target image. © 2014 Springer International Publishing Switzerland.

Access to Document (DOI)

Learning to generate proactive and reactive behavior using a dynamic neural network model with time-varying variance prediction mechanism

Shingo Murata, Hiroaki Arie, Tetsuya Ogata, Shigeki Sugano, Jun Tani

ADVANCED ROBOTICS (TAYLOR & FRANCIS LTD) 28 ( 17 ) 1189 - 1203 2014

Research paper (scientific journal), Joint Work, Accepted, ISSN 0169-1864

This paper discusses a possible neurodynamic mechanism that enables self-organization of two basic behavioral modes, namely a 'proactive mode' and a 'reactive mode,' and of autonomous switching between these modes depending on the situation. In the proactive mode, actions are generated based on an internal prediction, whereas in the reactive mode actions are generated in response to sensory inputs in unpredictable situations. In order to investigate how these two behavioral modes can be self-organized and how autonomous switching between the two modes can be achieved, we conducted neurorobotics experiments by using our recently developed dynamic neural network model that has a capability to learn to predict time-varying variance of the observable variables. In a set of robot experiments under various conditions, the robot was required to imitate other's movements consisting of alternating predictable and unpredictable patterns. The experimental results showed that the robot controlled by the neural network model was able to proactively imitate predictable patterns and reactively follow unpredictable patterns by autonomously switching its behavioral modes. Our analysis revealed that variance prediction mechanism can lead to self-organization of these abilities with sufficient robustness and generalization capabilities.

Access to Document (DOI)

Learning and recognition of multiple fluctuating temporal patterns using S-CTRNN

Shingo Murata, Hiroaki Arie, Tetsuya Ogata, Jun Tani, Shigeki Sugano

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Springer Verlag) 8681 9 - 16 2014

Research paper (international conference proceedings), Joint Work, Accepted, ISSN 1611-3349

In the present study, we demonstrate the learning and recognition capabilities of our recently proposed recurrent neural network (RNN) model called stochastic continuous-time RNN (S-CTRNN). S-CTRNN can learn to predict not only the mean but also the variance of the next state of the learning targets. The network parameters consisting of weights, biases, and initial states of context neurons are optimized through maximum likelihood estimation (MLE) using the gradient descent method. First, we clarify the essential difference between the learning capabilities of conventional CTRNN and S-CTRNN by analyzing the results of a numerical experiment in which multiple fluctuating temporal patterns were used as training data, where the variance of the Gaussian noise varied among the patterns. Furthermore, we also show that the trained S-CTRNN can recognize given fluctuating patterns by inferring the initial states that can reproduce the patterns through the same MLE scheme as that used for network training. © 2014 Springer International Publishing Switzerland.

Access to Document (DOI)

Learning to reproduce fluctuating time series by inferring their time-dependent stochastic properties: Application in Robot learning via tutoring

Shingo Murata, Jun Namikawa, Hiroaki Arie, Shigeki Sugano, Jun Tani

IEEE Transactions on Autonomous Mental Development 5 ( 4 ) 298 - 310 2013.12

Research paper (scientific journal), Joint Work, Accepted, ISSN 1943-0604

This study proposes a novel type of dynamic neural network model that can learn to extract stochastic or fluctuating structures hidden in time series data. The network learns to predict not only the mean of the next input state, but also its time-dependent variance. The training method is based on maximum likelihood estimation by using the gradient descent method and the likelihood function is expressed as a function of the estimated variance. Regarding the model evaluation, we present numerical experiments in which training data were generated in different ways utilizing Gaussian noise. Our analysis showed that the network can predict the time-dependent variance and the mean and it can also reproduce the target stochastic sequence data by utilizing the estimated variance. Furthermore, it was shown that a humanoid robot using the proposed network can learn to reproduce latent stochastic structures hidden in fluctuating tutoring trajectories. This learning scheme is essential for the acquisition of sensory-guided skilled behavior. © 2013 IEEE.

Access to Document (DOI)

Learning to Reproduce Fluctuating Behavioral Sequences Using a Dynamic Neural Network Model with Time-Varying Variance Estimation Mechanism

Shingo Murata, Jun Namikawa, Hiroaki Arie, Jun Tani, Shigeki Sugano

In Proceedings of the Third Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics (ICDL-EpiRob 2013) (IEEE) 1 - 6 2013.08

Research paper (international conference proceedings), Joint Work, Accepted

This study shows that a novel type of recurrent neural network model can learn to reproduce fluctuating training sequences by inferring their stochastic structures. The network learns to predict not only the mean of the next input state, but also its time-varying variance. The network is trained through maximum likelihood estimation by utilizing the gradient descent method, and the likelihood function is expressed as a function of both the predicted mean and variance. In a numerical experiment, in order to evaluate the performance of the model, we first tested its ability to reproduce fluctuating training sequences generated by a known dynamical system that were perturbed by Gaussian noise with state-dependent variance. Our analysis showed that the network can reproduce the sequences by predicting the variance correctly. Furthermore, the other experiment showed that a humanoid robot equipped with the network can learn to reproduce fluctuating tutoring sequences by inferring latent stochastic structures hidden in the sequences.

Access to Document (DOI)

Development of proactive and reactive behavior via meta-learning of prediction error variance

Shingo Murata, Jun Namikawa, Hiroaki Arie, Jun Tani, Shigeki Sugano

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 8226 ( 1 ) 537 - 544 2013

Research paper (international conference proceedings), Joint Work, Accepted, ISSN 0302-9743

This paper investigates a possible neurodynamic mechanism that enables autonomous switching between two basic behavioral modes, namely a "proactive mode" and a "reactive mode." In the proactive mode, actions are generated as intended, whereas in the reactive mode actions are generated in response to the sensory state.We conducted neurorobotics experiments to investigate how these two modes can develop and how a robot can learn to switch autonomously between the two modes as necessary by utilizing our recently developed dynamic neural network model. Tasks designed for the robot included switching between proactive imitation of other's predictable movements using acquired memories and reactive following of other's unpredictable movements through iterative learning of alternating predictable and unpredictable movement patterns. The experimental results revealed that this "meta-learning" capability can lead to self-organization of adequate contextual dynamical structures that can perform autonomous switching between the different behavioral modes. © Springer-Verlag 2013.

Access to Document (DOI)

　 Previous page - Next page 　