Details of a Researcher - Aoki, Yoshimitsu

Papers - Aoki, Yoshimitsu

Division display 61 - 80 of about 192 ／ All the affair displays >>

Akimoto N., Zhu H., Jin Y., Aoki Y.

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition) 8274 - 8283 2020

ISSN 10636919

We address the problem of soft color segmentation, defined as decomposing a given image into several RGBA layers, each containing only homogeneous color regions. The resulting layers from decomposition pave the way for applications that benefit from layer-based editing, such as recoloring and compositing of images and videos. The current state-of-the-art approach for this problem is hindered by slow processing time due to its iterative nature, and consequently does not scale to certain real-world scenarios. To address this issue, we propose a neural network based method for this task that decomposes a given image into multiple layers in a single forward pass. Furthermore, our method separately decomposes the color layers and the alpha channel layers. By leveraging a novel training objective, our method achieves proper assignment of colors amongst layers. As a consequence, our method achieve promising quality without existing issue of inference speed for iterative approaches. Our thorough experimental analysis shows that our method produces qualitative and quantitative results comparable to previous methods while achieving a 300,000x speed improvement. Finally, we utilize our proposed method on several applications, and demonstrate its speed advantage, especially in video editing.

Access to Document (DOI)

Weakly Supervised Domain Adaptation with Point Supervision in Histopathological Image Segmentation

Obikane S., Aoki Y.

Communications in Computer and Information Science (Communications in Computer and Information Science) 1180 CCIS 127 - 140 2020

ISSN 9789811536502

When a model learned in a domain is applied to a different domain, even if in the same task, there is no guarantee of accuracy. This is a very important issue when deep learning and machine learning are applied in the field. In medical applications, there is a wide variety of domain bias, making it very difficult to create a model appropriate for all domains. Furthermore, semantic segmentation needs fine annotation and its high labor cost makes its application difficult. Histopathological image segmentation enables drug discovery and medical image analysis, but it is expensive due to its annotation cost and the need for the skills of histopathological experts. In this paper, we focus on a weakly supervised method using point annotation unique to histopathological image segmentation, and tackled on weakly supervised domain adaptation to suppress domain gaps. Providing point level annotation instead of fine annotation decreases the high cost of labor normally required.

Access to Document (DOI)

Flood Susceptibility Prediction via Data-Mining Based Bell-Curve Analogical-Hydrographs Analysis: A Case Study of Langat River Basin, Selangor, Malaysia

Khuzaimah S.N., Kiyoki Y., Aoki Y.

Frontiers in Artificial Intelligence and Applications (Frontiers in Artificial Intelligence and Applications) 321 442 - 457 2019.12

ISSN 9781643680446

In this study, we proposed data-mining based bell-curve analogical hydrographs analysis with lag time vertical axes and bankfull discharge horizontal axes to make flood susceptibility prediction. We utilized flood data reports, hourly/daily rainfall data and daily water discharge of Hulu Langat district, Selangor Malaysia from the year 2013-2016 to do flood susceptibility. We implement data mining concept by sorting the database, followed by plotting hydrograph to identify flood patterns and establish relationships to predict flood trends. This method is an intersection between the knowledge field of hydrology and mathematical modeling. When an outlier from the graph is detected, the knowledge from hydrology can be applied to understand the reason behind the appearance of outliers. Besides, the knowledge of mathematical modeling is necessary to assist us in predicting flood susceptibility. The purpose of this study is to predict the flood susceptibility which is vital to prepare the users/public well prepared for smooth and efficient evacuation. In 4 years context, our flood depth predictions are nearly 100% accurate. Factor influencing the lag time and steepness of rising limb are related to land use and topographical features. Implications of the results and future research directions are also presented.

Access to Document (DOI)

An application of artificial intelligence to diagnostic imaging of spine disease: Estimating spinal alignment from moiré images

Watanabe K., Aoki Y., Matsumoto M.

Neurospine (Neurospine) 16 ( 4 ) 697 - 702 2019.12

ISSN 25866583

The use of artificial intelligence (AI) as a tool supporting the diagnosis and treatment of spinal diseases is eagerly anticipated. In the field of diagnostic imaging, the possible application of AI includes diagnostic support for diseases requiring highly specialized expertise, such as trauma in children, scoliosis, symptomatic diseases, and spinal cord tumors. Moiré topography, which describes the 3-dimensional surface of the trunk with band patterns, has been used to screen students for scoliosis, but the interpretation of the band patterns can be ambiguous. Thus, we created a scoliosis screening system that estimates spinal alignment, the Cobb angle, and vertebral rotation from moiré images. In our system, a convolutional neural network (CNN) estimates the positions of 12 thoracic and 5 lumbar vertebrae, 17 spinous processes, and the vertebral rotation angle of each vertebra. We used this information to estimate the Cobb angle. The mean absolute error (MAE) of the estimated vertebral positions was 3.6 pixels (~5.4 mm) per person. T1 and L5 had smaller MAEs than the other levels. The MAE per person between the Cobb angle measured by doctors and the estimated Cobb angle was 3.42°. The MAE was 4.38° in normal spines, 3.13° in spines with a slight deformity, and 2.74° in spines with a mild to severe deformity. The MAE of the angle of vertebral rotation was 2.9° ± 1.4°, and was smaller when the deformity was milder. The proposed method of estimating the Cobb angle and AVR from moiré images using a CNN is expected to enhance the accuracy of scoliosis screening.

Access to Document (DOI)

Task-oriented function detection based on operational tasks

Ishikawa Y., Ishikawa H., Akizuki S., Yamazaki M., Taniguchi Y., Aoki Y.

2019 19th International Conference on Advanced Robotics, ICAR 2019 (2019 19th International Conference on Advanced Robotics, ICAR 2019) 635 - 640 2019.12

ISSN 9781728124674

We propose novel representations for functions of an object, namely Task-oriented Function, which is improved upon the idea of Afforadance in the field of Robotics Vision. We also propose a convolutional neural network to detect task-oriented functions. This network takes as input an operational task as well as an RGB image and assign each pixel an appropriate label for every task. Task-oriented funciton makes it possible to descibe various ways to use an object because the outputs from the network differ depending on operational tasks. We introduce a new dataset for task-oriented function detection, which contains about 1200 RGB images and 6000 pixel-level annotations assuming five tasks. Our proposed method reached 0.80 mean IOU in our dataset.

Access to Document (DOI)

Fine-grained action recognition in assembly work scenes by drawing attention to the hands

Kobayashi T., Aoki Y., Shimizu S., Kusano K., Okumura S.

Proceedings - 15th International Conference on Signal Image Technology and Internet Based Systems, SISITS 2019 (Proceedings - 15th International Conference on Signal Image Technology and Internet Based Systems, SISITS 2019) 440 - 446 2019.11

ISSN 9781728156866

The analysis of assembly working scenes is an important tool for improving work efficiency and detecting worker error. Many factories choose to analyze videos of employees working by human observation. It is more efficient, however, to approach this as an action segmentation task, but this is difficult due to the fine-grained actions. In this paper, we focus on featuring the workers hands in action segmentation to describe the detailed moves in assembly work scenes and utilize the tool or product information that the worker is using. We propose two methods to draw attention to the hand from each image in the video: the first is by cutting out the workers hand image and extracting features, and the second is by using an attention module specialized to extract hand features by the training of a regression network. For both methods we combine different types of features - image features and pose features - which are both important information in fine-grained actions. Due to the lack of action datasets that focus on assembly work scenes, we create a new assembly work dataset for our experiments, and the results of those experiments show that our method is effective for analyzing fine-grained action segmentation tasks.

Access to Document (DOI)

Shot detection in racket sport video at the frame level using a recurrent neural network

Horie S., Sato Y., Furuyama J., Tanabiki M., Aoki Y.

ISSN 9781728156866

In recent years, there has been a demand in the sports industry to reduce the burden of data collection and video editing for tactical analysis. To achieve these, a system that can recognize the game context is needed. In this study, we proposed a method to identify the player's shot timing at the frame level during a ball-striking sport. In this study, players' shots were detected in video of a tennis match. It was shown that shots could be detected with an F-score value of 87% or more within an error range of 1 frame (0.033 sec) by considering time-series information using a recurrent neural network. This technology is expected to be applied not only to tennis, but also to other sports that involve ball shots, such as table tennis, baseball, and volleyball. At the same time, it can be used to detect moments of a specific action (for example, touching or hitting an object).

Access to Document (DOI)

360-Degree Image Completion by Two-Stage Conditional Gans

Akimoto N., Kasai S., Hayashi M., Aoki Y.

Proceedings - International Conference on Image Processing, ICIP (Proceedings - International Conference on Image Processing, ICIP) 2019-September 4704 - 4708 2019.09

ISSN 9781538662496

The latest generative adversarial networks (GANs) can generate realistic high resolution images. However, to the best of our knowledge, there are no GANs for generating 360-degree images. Therefore, this paper proposes the novel problem setting that by using a known area from the 360-degree image as an input, the remainder of the image can be completed with the GANs. To do so, we propose the approach of two-stage generation using network architecture with series-parallel dilated convolution layers. Moreover, we present how to rearrange images for data augmentation, simplify the problem, and make inputs for training the 2 stage generator. Our experiments show that these methods generate the distortion seen in 360-degree images in the outlines of buildings and roads, and their boundaries are clearer than those of baseline methods. Furthermore, we discuss and clarify the difficulty of our proposed problem. Our work is the first step towards GANs predicting an unseen area within a 360-degree space. nd

Access to Document (DOI)

Human-object maps for daily activity recognition

Ishikawa H., Ishikawa Y., Akizuki S., Aoki Y.

Proceedings of the 16th International Conference on Machine Vision Applications, MVA 2019 (Proceedings of the 16th International Conference on Machine Vision Applications, MVA 2019) 2019.05

ISSN 9784901122184

In the field of action recognition, when and where an interaction between a human and an object happens has the potential to be valid information in enhancing action recognition accuracy. Especially, in daily life where each activities are performed in longer time frame, conventional short term action recognition may fail to generalize do to the variety of shorter actions that could take place during the activity. In this paper, we propose a novel representation of human object interaction called Human-Object Maps (HOMs) for recognition of long term daily activities. HOMs are 2D probability maps that represents spatio-temporal information of human object interaction in a given scene. We analyzed the effectiveness of HOMs as well as features relating to the time of the day in daily activity recognition. Since there are no publicly available daily activity dataset that depicts daily routines needed for our task, we have created a new dataset that contains long term activities. Using this dataset, we confirm that our method enhances the prediction accuracy of the conventional 3D ResNeXt action recognition method from 86.31% to 97.89%.

Access to Document (DOI)

Detect functions of objects based on operational task input

Ishikawa Y., Ishikawa H., Akizuki S., Aoki Y.

Seimitsu Kogaku Kaishi/Journal of the Japan Society for Precision Engineering (Seimitsu Kogaku Kaishi/Journal of the Japan Society for Precision Engineering) 85 ( 12 ) 1136 - 1142 2019

ISSN 09120289

Recently, the development of deep learning has enabled robots to grasp objects more reliably than ever. Given this fact, there is an increasing demand for helper robots or home robots. To make these robots real, robots need to understand not only how to grasp objects but also their functions. We propose a new representation for the functions of objects, task-oriented function, which is based on operational task input. This representation makes it possible to describe a variety of ways to use an object. We also propose a new dataset for task-oriented function and a network to detect it. This model reached 79.7% mean IOU in our dataset.

Access to Document (DOI)

6D0F pose estimation for similar objects using spatial relationship of part-affordance

Akizuki S., Aoki Y.

Seimitsu Kogaku Kaishi/Journal of the Japan Society for Precision Engineering (Seimitsu Kogaku Kaishi/Journal of the Japan Society for Precision Engineering) 85 ( 1 ) 107 - 112 2019

ISSN 09120289

In this paper, we propose a method to estimate 6D pose (3D position and pose) of the everyday objects, especially Kitchen or DIY tools, even if there are not the same 3D model of the target object. We assume that the tools that belong to same category have common spatial relation in part-affordances, the role of each part. The proposed method uses this assumption for pose estimation. By minimizing two types of consistencies, the spatial relationship of part-affordances and the shapes, we have confirmed that the proposed method achieved higher performance compared with traditional pose estimation method.

Access to Document (DOI)

FOE-based regularization for optical flow estimation from an in-vehicle event camera

Nagata J., Sekikawa Y., Hara K., Aoki Y.

Proceedings of SPIE - The International Society for Optical Engineering (Proceedings of SPIE - The International Society for Optical Engineering) 11049 2019

ISSN 9781510627734

Optical flow estimation in onboard cameras is an important task in automatic driving and advanced driver- assistance systems. However, there is a problem that calculation is mistakable with high contrast and high speed. Event cameras have great features such as high dynamic range and low latency that can overcome these problems. Event cameras report only the change in the logarithmic intensity per pixel rather than the absolute brightness. There is a method of estimating the optical ow simultaneously with the luminance restoration from the event data. The regularization using the L1 norm of differentiation is insufficient for spatially sparse event data. Therefore, we propose to use the focus of expansion (FOE) for regularization of optical ow estimation in event camera. The FOE is defined as the intersection of the translation vector of the camera and the image plane. The optical ow becomes radial from the FOE excluding the rotational component. Using the property, the optical ow can be regularized in the correct direction in the optimization process. We demonstrated that the optical ow was improved by introducing our regularization using the public dataset.

Access to Document (DOI)

Visual attribute manipulation using natural language commands

Kasai S., Akimoto N., Hayashi M., Aoki Y.

Seimitsu Kogaku Kaishi/Journal of the Japan Society for Precision Engineering (Seimitsu Kogaku Kaishi/Journal of the Japan Society for Precision Engineering) 85 ( 12 ) 1102 - 1109 2019

ISSN 09120289

In this paper, a novel setting is tackled in which a neural network generates object images with transferred attributes, by conditioning on natural language commands. Conventional methods for object image transformation have used visual attributes, which are components that describe the object's color, posture, etc. This paper builds on this approach and finds an algorithm to precisely extract information from natural language commands, which transfers the attributes of an image and completes this image translation model. The effectiveness of our information extraction model is experimented, with additional tests to see if the change in visual attributes is correctly seen in the image.

Access to Document (DOI)

Tactile logging for understanding plausible tool use based on human demonstration

Akizuki S., Aoki Y.

British Machine Vision Conference 2018, BMVC 2018 (British Machine Vision Conference 2018, BMVC 2018) 2019

In this paper, we propose a novel human-object interaction (HOI) representation for understanding plausible tool use; this is done by using an object-centric method. Everyday tools (such as kitchen or DIY utensils) have the ideal region and direction for being touched by hands and surrounding objects. By analyzing the human demonstrations, tactile histories that occurred on the tools are accumulated to the tool's surface as the "Tactile Log", which can help to achieve the plausible tool use by robot arms. We also proposed a dataset for the evaluation of generated tactile logs, which is called the Object-Centric Interaction (OCI) dataset consists of RGB-D videos of plausible tool use by human and 3D models of the tools used in the videos. We confirm that the precision of our tactile logging is 0.77.

FOE-based regularization for optical flow estimation from an in-vehicle event camera

Nagata J., Sekikawa Y., Hara K., Aoki Y.

IEEJ Transactions on Electronics, Information and Systems (IEEJ Transactions on Electronics, Information and Systems) 139 ( 10 ) 1113 - 1118 2019

ISSN 03854221

Optical flow estimation from an in-vehicle camera is an important task in automatic driving and advanced driver- A ssistance systems. However, there is a problem that optical flow estimation is mistakable with high contrast and high speed. Event camera can overcome these situations because it reports only the per-pixel intensity change with high dynamic range and low latency. However, the LI smoothness regularization in the conventional optical flow estimation method is not suitable for radial optical flow in the driving scene. Therefore, we propose to use the focus of expansion (FOE) for regularization of optical flow estimation in event camera. The FOE is defined as the intersection of the translation vector of the camera and the image plane. The optical flow becomes radial from the FOE excluding the rotational component. Using the property, the optical flow can be regularized in the correct direction in the optimization process. We demonstrated that the optical flow was improved by introducing our regularization using the public dataset.

Access to Document (DOI)

Measurement of Vertebral Rotation from Moire Image for Screening of Adolescent Idiopathic Scoliosis

Ran CHOI, Kota WATANABE, Yoshimitsu AOKI

IIEEJ Transactions on Image Electronics and Visual Computing (IIEEJ) 6 ( 2 ) 56 - 64 2018.12

Research paper (scientific journal), Joint Work, Accepted

Anticipating Traffic Accidents with Adaptive Loss and Large-Scale Incident DB

Suzuki T., Kataoka H., Aoki Y., Satoh Y.

ISSN 9781538664209

© 2018 IEEE. In this paper, we propose a novel approach for traffic accident anticipation through (i) Adaptive Loss for Early Anticipation (AdaLEA) and (ii) a large-scale self-annotated incident database for anticipation. The proposed AdaLEA allows a model to gradually learn an earlier anticipation as training progresses. The loss function adaptively assigns penalty weights depending on how early the model can anticipate a traffic accident at each epoch. Additionally, we construct a Near-miss Incident DataBase for anticipation. This database contains an enormous number of traffic near-miss incident videos and annotations for detail evaluation of two tasks, risk anticipation and risk-factor anticipation. In our experimental results, we found our proposal achieved the highest scores for risk anticipation (+6.6% better on mean average precision (mAP) and 2.36 sec earlier than previous work on the average time-to-collision (ATTC)) and risk-factor anticipation (+4.3% better on mAP and 0.70 sec earlier than previous work on ATTC).

Access to Document (DOI)

Generating effect animation with conditional GANs

Akimoto N., Kataoka H., Aoki Y.

SIGGRAPH Asia 2018 Posters, SA 2018 (SIGGRAPH Asia 2018 Posters, SA 2018) 2018.12

ISSN 9781450360630

Access to Document (DOI)

機能属性の空間配置に着目した類似形状物体の6自由度姿勢推定

秋月秀一, 青木義満

精密工学会誌 (精密工学会) 2018.12

Research paper (scientific journal), Joint Work, Accepted

GAN によるカラー調整と画像補完の同時実行

秋本直郁，秋月秀一，林昌希，青木義満

精密工学会誌 (精密工学会) 84 ( 12 ) 1033 - 1040 2018.12

Research paper (scientific journal), Joint Work, Accepted

　 Previous page - Next page 　