Details of a Researcher - SUGIURA, Komei

Papers - SUGIURA, Komei

Division display 21 - 40 of about 211 ／ All the affair displays >>

Layer-Wise Relevance Propagation for ResNet: 保全公理を満たす視覚的説明生成

小槻誠太郎，飯田紡，平川翼，山下隆義，藤吉弘亘，杉浦孔明

人工知能学会全国大会論文集第 38 回 (2024), 4I1GS703-4I1GS703 (The Japanese Society for Artificial Intelligence) JSAI2024 ( 0 ) 4I1GS703 - 4I1GS703 2024

In the modern era, where deep learning is applied across a wide range of fields, the explainability of deep learning models is crucial, and the generation of explanations with high transparency is desirable. Layer-wise Relevance Propagation (LRP) is mentioned as a method for generating explanations by backpropagating relevance scores, offering high transparency. However, this method has been applied only to models without skip connections, as its application to models with skip connections does not satisfy the conservation property, resulting in poor quality explanations. Therefore, this paper proposes a method for calculating the backpropagation of relevance scores that satisfies the conservation property in models with skip connections. The proposed method outperformed existing methods in terms of Insertion-Deletion Score while satisfying the conservation property.

C3-LRP: Visual Explanation Generation based on Layer-Wise Relevance Propagation for ResNet

F Doublet, S OTSUKI, I TSUMUGI, K SUGIURA

人工知能学会全国大会論文集第 38 回 (2024), 4Q1IS2c03-4Q1IS2c03 (The Japanese Society for Artificial Intelligence) JSAI2024 ( 0 ) 4Q1IS2c03 - 4Q1IS2c03 2024

In this paper, we focus on the task of visualizing important regions in an image as high-quality visual explanations of the model’s decisions with a clear theoretical background. We introduce a novel calculation method for Layer-wise Relevance Propagation (LRP) specifically tailored to models featuring skip connections such as ResNet. This method’s strength lies in its adaptability, as the backpropagation technique is distinctly defined for each layer, enhancing its extensibility. To validate our method, we conduct an experiment on the CUB-200-2011 dataset. The proposed method successfully generates appropriate explanations and, based on the Insertion-Deletion score, outperforms the baseline methods.

Alternative Adapter Model: 視覚言語基盤モデルのための視覚的説明生成

平野愼之助，飯田紡，杉浦孔明

人工知能学会全国大会論文集第 38 回 (2024), 1D3GS705-1D3GS705 (The Japanese Society for Artificial Intelligence) JSAI2024 ( 0 ) 1D3GS705 - 1D3GS705 2024

In the modern era where deep learning is applied across a wide range of fields, the explainability of models is of paramount importance. However, existing methods are not optimized for vision-language foundation models, leading to lower explanation quality for such models. Therefore, this study proposes the Alternative Adapter Model, an explanation generation model tailored to vision-language foundation models. By introducing a Side Branch Network connected to the vision-language foundation model, the proposed method extracts features suitable for explanation generation. Furthermore, by implementing the Alternative Epoch Architecture, which dynamically changes the outputs of modules and the layers to be frozen, we address the issue of overly narrow focus areas. To evaluate the proposed method, experiments were conducted using the CUB-200-2011 dataset. The results demonstrate that the proposed method surpasses existing methods in mean IoU, Insertion Score, Deletion Score, and Insertion-Deletion Score, which are standard metrics for visual explanation generation tasks.

Polos: Multimodal Metric Learning from Human Feedback for Image Captioning.

Yuiga Wada, Kanta Kaneda, Daichi Saito, Komei Sugiura

CVPR 13559 - 13568 2024

Access to Document (DOI)

Retention 機構に基づく頭蓋内脳波の分類と BMI の構築

長嶋隼矢，兼田寛大，飯田紡，田口美紗，平田雅之，杉浦孔明

人工知能学会全国大会論文集第 38 回 (2024), 4N1GS101-4N1GS101 (The Japanese Society for Artificial Intelligence) JSAI2024 ( 0 ) 4N1GS101 - 4N1GS101 2024

Speech impairments from conditions like Amyotrophic Lateral Sclerosis and muscular dystrophy severely restrict patient communication, affecting daily and social life. Decoding technology based on Electrocorticography (ECoG) is essential for supporting these patients' communication. In this study, we propose a novel architecture combining a specialized convolutional layer for electrode feature extraction and a retentive network for ECoG signal classification of motor imagery, outperforming all baselines in accuracy.

Switching Text-based Image Encoders for Captioning Images with Text

A Ueda, W Yang, K Sugiura

IEEE Access 11 55706 - 55715 2023

Access to Document (DOI)

Affective Image Captioning for Visual Artworks Using Emotion-Based Cross-Attention Mechanisms

S Ishikawa, K Sugiura

IEEE Access 11, 24527-24534 11 24527 - 24534 2023

Access to Document (DOI)

Switching Head-Tail Funnel UNITER for Dual Referring Expression Comprehension with Fetch-and-Carry Tasks

R Korekata, M Kambara, Y Yoshida, S Ishikawa, Y Kawasaki, ...

2023 IEEE/RSJ International Conference on Intelligent Robots and Systems … 2023

Prototypical Contrastive Transfer Learning for Multimodal Language Understanding

S Otsuki, S Ishikawa, K Sugiura

2023 IEEE/RSJ International Conference on Intelligent Robots and Systems … 2023

Multimodal Diffusion Segmentation Model for Object Segmentation from Manipulation Instructions

Y Iioka, Y Yoshida, Y Wada, S Hatanaka, K Sugiura

2023 IEEE/RSJ International Conference on Intelligent Robots and Systems … 2023

Nearest Neighbor Future Captioning: 物体配置タスクにおける衝突リスクに関する説明文生成

小松拓実，神原元就，畑中駿平，松尾榛夏，平川翼，山下隆義，藤吉弘亘， ...

人工知能学会全国大会論文集第 37 回 (2023), 3G1OS24a05-3G1OS24a05 (The Japanese Society for Artificial Intelligence) JSAI2023 ( 0 ) 3G1OS24a05 - 3G1OS24a05 2023

The practical implementation of domestic support robots that can communicate using natural language is a promising solution for those in need of assistance. In particular, the ability to predict potential hazards associated with task execution and to prompt the user for judgment can enhance safety and convenience. However, accurate prediction is difficult because information about future events cannot be utilized. In existing methods, representation of the grasped object is insufficient because the image of the grasped object is not used as input. Additionally, there is a drawback that it is impossible to avoid collision during collision prediction as it requires input of the previous image. In this study, we propose the addition of an attention map visualization module for collision prediction and the enhancement of model representation through the use of k-nearest neighbor method. We conduct comparative experiments using standard evaluation metrics for generated text such as BLEU4, METEOR, ROUGE-L, and CIDEr-D. Experimental results show that the proposed method outperforms the baseline method in all evaluation metrics.

Multimodal encoder with gated cross-attention for text-vqa tasks

W Yang, A Ueda, K Sugiura

29th Annual Conference of the Language Processing Society, 1580-1585 2023

Learning to Rank Physical Objects: ランキング学習による物理世界検索エンジン

兼田寛大，神原元就，杉浦孔明

人工知能学会全国大会論文集第 37 回 (2023), 3G1OS24a01-3G1OS24a01 (The Japanese Society for Artificial Intelligence) JSAI2023 ( 0 ) 3G1OS24a01 - 3G1OS24a01 2023

In this study, we focus on the learning-to-rank physical objects task, which involves retrieving target objects from open-vocabulary user instructions in a human-in-the-loop setting. We propose MultiRankIt, which introduces the Crossmodal Noun Phrase Encoder to model the relationship between referring expressions and target bounding box, and the Crossmodal Region Feature Encoder to model the relationship between the target object and its surrounding contextual environment. Our model outperforms the baseline method in terms of mean reciprocal rank and recall@K.

Shared transformer encoder with mask-based 3D model estimation for container mass estimation

T Matsubara, S Otsuki, Y Wada, H Matsuo, T Komatsu, Y Iioka, K Sugiura, ...

ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and … 9142 - 9146 2022

Access to Document (DOI)

生活支援ロボットによる物体操作タスクにおける Funnel UNITER に基づく指示文理解

吉田悠，石川慎太郎，杉浦孔明

人工知能学会全国大会論文集第 36 回 (2022), 2O4GS701-2O4GS701 (The Japanese Society for Artificial Intelligence) JSAI2022 ( 0 ) 2O4GS701 - 2O4GS701 2022

In this study, we develop a multimodal language comprehension model that allows domestic service robots to understand object fetching instructions. We propose a multimodal language understanding model, Funnel UNITER, which gradually reduces the dimensions of the query, key, and value in each transformer layer to reduce the computational cost of self-attention. We also built a new dataset for the multimodal language understanding for fetching instruction (MLU-FI) task called the ALFRED-fetch dataset. Our model outperformed the baseline method in both classification accuracy and training time.

日常タスクにおける将来イベントのクロスモーダル説明文生成

神原元就，杉浦孔明

人工知能学会全国大会論文集第 36 回 (2022), 2O1GS702-2O1GS702 (The Japanese Society for Artificial Intelligence) JSAI2022 ( 0 ) 2O1GS702 - 2O1GS702 2022

In this paper, our aim is to generate a caption about a future event. We propose the Relational Future Captioning Model (RFCM), a crossmodal language generation model for the future captioning task. The RFCM has the Relational Self-Attention Encoder to extract the relationships between events more effectively than the conventional self-attention in transformers. We conducted comparison experiments, and the results show the RFCM outperforms a baseline method on two datasets.

Target Position Prediction Using UNITER Regressor for Understanding Navigation Instructions in Urban Areas

HATANAKA Shumpei, WEI Yang, HOSOMI Naoki, MISU Teruhisa, YAMADA Kentarou, SUGIURA Komei

JSAI Technical Report, SIG-KBS (The Japanese Society for Artificial Intelligence) 126 ( 0 ) 34 - 39 2022

Vision-and-Language Navigation タスクにおける敵対的サブゴール生成

石川慎太朗，杉浦孔明

人工知能学会全国大会論文集第 36 回 (2022), 2O1GS701-2O1GS701 (The Japanese Society for Artificial Intelligence) JSAI2022 ( 0 ) 2O1GS701 - 2O1GS701 2022

In this paper, we focus on a vision-and-language task in which a robot is instructed to execute household tasks. We propose Moment-based Adversarial Training (MAT), which uses two types of moments for perturbation updates in adversarial training. We introduce MAT to the embedding spaces of the instruction, subgoals, and state representations to handle their varieties. We validated our method on the ALFRED benchmark, and the results demonstrated that our method outperformed the baseline method for all the metrics on the benchmark.

Language identifying device and computer program for same, and speech processing device

P Shen, K Sugiura, H Kawai

US Patent App. 17/593,822 2022

Lambda Attention Branch Networks による視覚的説明生成

飯田紡，兼田寛大，平川翼，山下隆義，藤吉弘亘，杉浦孔明

人工知能学会全国大会論文集第 36 回 (2022), 2O1GS703-2O1GS703 (The Japanese Society for Artificial Intelligence) JSAI2022 ( 0 ) 2O1GS703 - 2O1GS703 2022

Explanation generation for transformers enhances accountability for their predictions. However, there have been few studies on generating visual explanations for the transformers that use multidimensional context, such as LambdaNetworks. In this paper, we propose the Lambda Attention Branch Networks, which attend to important regions in detail and generate easily interpretable visual explanations. We also propose the Patch Insertion-Deletion score, an extension of the Insertion-Deletion score, as an effective evaluation metric for images with sparse important regions. Experimental results on two public datasets indicate that the proposed method successfully generates visual explanations.

　 Previous page - Next page 　