論文 - 青木　義満

FOE-based regularization for optical flow estimation from an in-vehicle event camera

Nagata J., Sekikawa Y., Hara K., Aoki Y.

Proceedings of SPIE - The International Society for Optical Engineering （Proceedings of SPIE - The International Society for Optical Engineering）  11049 2019年

ISSN  9781510627734

　概要を見る

Optical flow estimation in onboard cameras is an important task in automatic driving and advanced driver- assistance systems. However, there is a problem that calculation is mistakable with high contrast and high speed. Event cameras have great features such as high dynamic range and low latency that can overcome these problems. Event cameras report only the change in the logarithmic intensity per pixel rather than the absolute brightness. There is a method of estimating the optical ow simultaneously with the luminance restoration from the event data. The regularization using the L1 norm of differentiation is insufficient for spatially sparse event data. Therefore, we propose to use the focus of expansion (FOE) for regularization of optical ow estimation in event camera. The FOE is defined as the intersection of the translation vector of the camera and the image plane. The optical ow becomes radial from the FOE excluding the rotational component. Using the property, the optical ow can be regularized in the correct direction in the optimization process. We demonstrated that the optical ow was improved by introducing our regularization using the public dataset.

• Access to Document (DOI)

Detect functions of objects based on operational task input

Ishikawa Y., Ishikawa H., Akizuki S., Aoki Y.

Seimitsu Kogaku Kaishi/Journal of the Japan Society for Precision Engineering （Seimitsu Kogaku Kaishi/Journal of the Japan Society for Precision Engineering）  85 （ 12 ） 1136 - 1142 2019年

ISSN  09120289

　概要を見る

Recently, the development of deep learning has enabled robots to grasp objects more reliably than ever. Given this fact, there is an increasing demand for helper robots or home robots. To make these robots real, robots need to understand not only how to grasp objects but also their functions. We propose a new representation for the functions of objects, task-oriented function, which is based on operational task input. This representation makes it possible to describe a variety of ways to use an object. We also propose a new dataset for task-oriented function and a network to detect it. This model reached 79.7% mean IOU in our dataset.

• Access to Document (DOI)

6D0F pose estimation for similar objects using spatial relationship of part-affordance

Akizuki S., Aoki Y.

Seimitsu Kogaku Kaishi/Journal of the Japan Society for Precision Engineering （Seimitsu Kogaku Kaishi/Journal of the Japan Society for Precision Engineering）  85 （ 1 ） 107 - 112 2019年

ISSN  09120289

　概要を見る

In this paper, we propose a method to estimate 6D pose (3D position and pose) of the everyday objects, especially Kitchen or DIY tools, even if there are not the same 3D model of the target object. We assume that the tools that belong to same category have common spatial relation in part-affordances, the role of each part. The proposed method uses this assumption for pose estimation. By minimizing two types of consistencies, the spatial relationship of part-affordances and the shapes, we have confirmed that the proposed method achieved higher performance compared with traditional pose estimation method.

• Access to Document (DOI)

Visual attribute manipulation using natural language commands

Kasai S., Akimoto N., Hayashi M., Aoki Y.

Seimitsu Kogaku Kaishi/Journal of the Japan Society for Precision Engineering （Seimitsu Kogaku Kaishi/Journal of the Japan Society for Precision Engineering）  85 （ 12 ） 1102 - 1109 2019年

ISSN  09120289

　概要を見る

In this paper, a novel setting is tackled in which a neural network generates object images with transferred attributes, by conditioning on natural language commands. Conventional methods for object image transformation have used visual attributes, which are components that describe the object's color, posture, etc. This paper builds on this approach and finds an algorithm to precisely extract information from natural language commands, which transfers the attributes of an image and completes this image translation model. The effectiveness of our information extraction model is experimented, with additional tests to see if the change in visual attributes is correctly seen in the image.

• Access to Document (DOI)

Measurement of Vertebral Rotation from Moire Image for Screening of Adolescent Idiopathic Scoliosis

Ran CHOI, Kota WATANABE, Yoshimitsu AOKI

IIEEJ Transactions on Image Electronics and Visual Computing （IIEEJ）  6 （ 2 ） 56 - 64 2018年12月

研究論文（学術雑誌）, 共著, 査読有り

Anticipating Traffic Accidents with Adaptive Loss and Large-Scale Incident DB

Suzuki T., Kataoka H., Aoki Y., Satoh Y.

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition （Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition）     3521 - 3529 2018年12月

ISSN  9781538664209

　概要を見る

© 2018 IEEE. In this paper, we propose a novel approach for traffic accident anticipation through (i) Adaptive Loss for Early Anticipation (AdaLEA) and (ii) a large-scale self-annotated incident database for anticipation. The proposed AdaLEA allows a model to gradually learn an earlier anticipation as training progresses. The loss function adaptively assigns penalty weights depending on how early the model can anticipate a traffic accident at each epoch. Additionally, we construct a Near-miss Incident DataBase for anticipation. This database contains an enormous number of traffic near-miss incident videos and annotations for detail evaluation of two tasks, risk anticipation and risk-factor anticipation. In our experimental results, we found our proposal achieved the highest scores for risk anticipation (+6.6% better on mean average precision (mAP) and 2.36 sec earlier than previous work on the average time-to-collision (ATTC)) and risk-factor anticipation (+4.3% better on mAP and 0.70 sec earlier than previous work on ATTC).

• Access to Document (DOI)

Generating effect animation with conditional GANs

Akimoto N., Kataoka H., Aoki Y.

SIGGRAPH Asia 2018 Posters, SA 2018 （SIGGRAPH Asia 2018 Posters, SA 2018）  2018年12月

ISSN  9781450360630

• Access to Document (DOI)

機能属性の空間配置に着目した類似形状物体の6自由度姿勢推定

秋月秀一, 青木義満

精密工学会誌 （精密工学会）  2018年12月

研究論文（学術雑誌）, 共著, 査読有り

GAN によるカラー調整と画像補完の同時実行

秋本直郁，秋月秀一，林昌希，青木義満

精密工学会誌 （精密工学会）  84 （ 12 ） 1033 - 1040 2018年12月

研究論文（学術雑誌）, 共著, 査読有り

追跡軌跡の再同定を用いたオンライン複数物体追跡

加藤直樹，田靡雅基，古山純子，里 雄二，青木義満

画像電子学会論文誌 （画像電子学会）  47 （ 4 ） 391 - 400 2018年11月

研究論文（学術雑誌）, 共著, 査読有り

Superpixel convolution for segmentation

Suzuki T., Akizuki S., Kato N., Aoki Y.

Proceedings - International Conference on Image Processing, ICIP （Proceedings - International Conference on Image Processing, ICIP）     3249 - 3253 2018年08月

ISSN  9781479970612

　概要を見る

© 2018 IEEE. In this paper, we propose a novel segmentation algorithm based on convolutional neural networks (CNNs) on superpix-else CNNs are powerful methods for several computer vision tasks, but spatial information disappears through the pooling process. Moreover, since pooling compresses different types of pixels into a single value, pooling sometimes negatively affects the results of inference in segmentation task. We use superpixel pooling instead of general pooling to resolve this problem. However, general CNNs can't use superpixel images in which the adjacency relationships between pixels are broken. Therefore, we define CNNs and Dilated Convolution on superpixels. Finally, we show the effectiveness of proposed method on an HKU-IS dataset.

• Access to Document (DOI)

Segmentationのためのスーパーピクセル上でのGraph Convolutional Neural Networksの提案

鈴木哲平，青木義満

電子情報通信学会和文論文誌D （電子情報通信学会）  J101-D （ 8 ） 1120 - 1129 2018年08月

研究論文（学術雑誌）, 共著, 査読有り

Estimation of Scenes Contributing to Score in Tennis Video Using Attention

Kurose R., Hayashi M., Aoki Y.

Proceedings - 14th International Conference on Signal Image Technology and Internet Based Systems, SITIS 2018 （Proceedings - 14th International Conference on Signal Image Technology and Internet Based Systems, SITIS 2018）     672 - 675 2018年07月

ISSN  9781538693858

　概要を見る

The use of image processing technology for sports is increasing. By analyzing athletes and teams based on video analysis, scientific and objective analysis can be conducted separately from subjective analysis of experts, and individuals or teams can be evaluated with the same index. As can be seen from such a trend, recently, technological progress has made it possible to detect and analyze what can be visually confirmed in the sports video. It is certain that these technologies are helping coaches in the analysis of movement. However, these cannot play the role of coaches. The role is to judge what is important from what can be visually confirmed. There are also empirical and qualitative parts in coach's judgment, and no reproducibility of understanding the important parts unless it is a specialist. Based on this background, we thought that it would be useful to extract more important scenes during the game using quantitative information. Such technology can be applied to various sports, but this paper, we focused on tennis. In this thesis, the aim is to estimate which play was largely contributed to the tennis game result (score, failure). In addition, we don't use supervised information that can be obtained from an empirical point of view. This is to eliminate dataset dependency caused by using data created from qualitative information. Specifically, based on quantitative information such as athletes' movement and score result, we attempted to estimate the attention from unsupervised method.

• Access to Document (DOI)

Single Image, Context Aware Action Estimation in Sports

Lanius C., Kobayashi D., Ouchi K., Aoki Y.

Proceedings - 14th International Conference on Signal Image Technology and Internet Based Systems, SITIS 2018 （Proceedings - 14th International Conference on Signal Image Technology and Internet Based Systems, SITIS 2018）     664 - 671 2018年07月

ISSN  9781538693858

　概要を見る

Sports video analysis enables professional teams to prepare and practice more efficiently than ever before. It allows the team to work on their weak points and analyze upcoming opponents' strategies. As segmentation is a well researched topic, we assume a sports video where each frame is segmented into actors. We show that the location of the actors is correlated to their action. We develop a model to predict each actor's action in a context consistent and location aware manner. The set of context actors is an unordered set, thus we propose a permutation invariant structure for our classifier. We evaluate a location and a context aware network for single image sports action recognition on two datasets. One is an existing dataset of volleyball games. We present a new dataset for actor recognition in rugby. We compare the results to using only the image of the actor.

• Access to Document (DOI)

Swimmer's stroke estimation using CNN and MultiLSTM

Kohei Hakozaki, Noki Kato, Masamoto Tanabiki, Junko Furuyama, Yuji Sato, Yoshimitsu Aoki

Journal of Signal Processing （RISP）  22 （ 4 ） 219 - 222 2018年07月

研究論文（学術雑誌）, 共著, 査読有り

Player pose analysis in tennis video based on pose estimation

Kurose R., Hayashi M., Ishii T., Aoki Y.

2018 International Workshop on Advanced Image Technology, IWAIT 2018 （2018 International Workshop on Advanced Image Technology, IWAIT 2018）     1 - 4 2018年05月

ISSN  9781538626153

　概要を見る

© 2018 IEEE. The demand for sports video analysis is expanding. Sports video analysis is used to analyze their own play and play of opponent players, and visualize movement and ability of players. Scientific and objective analysis become possible by incorporating video analysis. It is desired to improve the competition level by feeding back the analysis results to the players. Therefore, in this research, in the case of feeding back the analysis result to the athlete, research purpose is to realize a method which can understand and evaluate the details of the form hitting the ball in detail. First, joint position coordinates are estimated from input RGB tennis images by a posture estimation method. The joint position coordinates in each frame at the time of shot are classified using unsupervised method and represented by BoW. The feature vector is designed by combining this with the shot position. The probability of shot success is predicted using this feature vector. By visualizing BoW of the shot with the high probability of success and the high failure probability, it is possible to extract and compare poses that are likely to appear in each case without giving correct labels.

• Access to Document (DOI)

Pose alignment for different objects using affordance cues

Akizuki S., Aoki Y.

2018 International Workshop on Advanced Image Technology, IWAIT 2018 （2018 International Workshop on Advanced Image Technology, IWAIT 2018）     1 - 3 2018年05月

ISSN  9781538626153

　概要を見る

© 2018 IEEE. In this paper, we propose a method to estimate 6D pose (3D position and pose) of the everyday objects, especially tools, even if there are not the same 3D model of the target object. We assume that the tools that belong to same category have common spatial relation in part-affordances, the role of each part. The proposed method uses this assumption for pose estimation. By minimizing two types of consistencies, the spatial relationship of part-affordances and the shapes, we have confirmed that the proposed method achieved higher performance compared with traditional pose estimation method.

• Access to Document (DOI)

Temporal and fine-grained pedestrian action recognition on driving recorder database

Kataoka H., Satoh Y., Aoki Y., Oikawa S., Matsui Y.

Sensors (Switzerland) （Sensors (Switzerland)）  18 （ 2 ）  2018年02月

研究論文（学術雑誌）, 共著, 査読有り,  ISSN  14248220

　概要を見る

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. The paper presents an emerging issue of fine-grained pedestrian action recognition that induces an advanced pre-crush safety to estimate a pedestrian intention in advance. The fine-grained pedestrian actions include visually slight differences (e.g., walking straight and crossing), which are difficult to distinguish from each other. It is believed that the fine-grained action recognition induces a pedestrian intention estimation for a helpful advanced driver-assistance systems (ADAS). The following difficulties have been studied to achieve a fine-grained and accurate pedestrian action recognition: (i) In order to analyze the fine-grained motion of a pedestrian appearance in the vehicle-mounted drive recorder, a method to describe subtle change of motion characteristics occurring in a short time is necessary; (ii) even when the background moves greatly due to the driving of the vehicle, it is necessary to detect changes in subtle motion of the pedestrian; (iii) the collection of large-scale fine-grained actions is very difficult, and therefore a relatively small database should be focused. We find out how to learn an effective recognition model with only a small-scale database. Here, we have thoroughly evaluated several types of configurations to explore an effective approach in fine-grained pedestrian action recognition without a large-scale database. Moreover, two different datasets have been collected in order to raise the issue. Finally, our proposal attained 91.01% on National Traffic Science and Environment Laboratory database (NTSEL) and 53.23% on the near-miss driving recorder database (NDRDB). The paper has improved +8.28% and +6.53% from baseline two-stream fusion convnets.

• Access to Document (DOI)

Video-Based Person Re-identification by 3D Convolutional Neural Networks and Improved Parameter Learning

Kato N., Hakozaki K., Tanabiki M., Furuyama J., Sato Y., Aoki Y.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) （Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)）  10882 LNCS   153 - 163 2018年

ISSN  9783319929996

　概要を見る

© 2018, Springer International Publishing AG, part of Springer Nature. In this paper we propose a novel approach for video-based person re-identification that exploits convolutional neural networks to learn the similarity of persons observed from video camera. We take 3-dimensional convolutional neural networks (3D CNN) to extract fine-grained spatiotemporal features from the video sequence of a person. Unlike recurrent neural networks, 3D CNN preserves the spatial patterns of the input, which works well on re-identification problem. The network maps each video sequence of a person to a Euclidean space where distances between feature embeddings directly correspond to measures of person similarity. By our improved parameter learning method called entire triplet loss, all possible triplets in the mini-batch are taken into account to update network parameters. This parameter updating method significantly improves training, enabling the embeddings to be more discriminative. Experimental results show that our model achieves new state of the art identification rate on iLIDS-VID dataset and PRID-2011 dataset with 82.0%, 83.3% at rank 1, respectively.

• Access to Document (DOI)

Simultaneous execution of color adjustment and image completion by generative adversarial network

Akimoto N., Hayashi M., Akizuki S., Aoki Y.

Seimitsu Kogaku Kaishi/Journal of the Japan Society for Precision Engineering （Seimitsu Kogaku Kaishi/Journal of the Japan Society for Precision Engineering）  84 （ 12 ） 1033 - 1040 2018年

ISSN  09120289

　概要を見る

In this paper, we address the problem of performing natural paste synthesis by color adjustment and image completion, in order to solve the completion problem that can specify an object appearing in a completion area. We propose a synthesis network that can extract the context features of the input image and reconstruct an image with the feature, making the inserted object appear in the completion region. In addition, we propose a ingenious method to make input images and learning method using Generative Adversarial Network (GAN) that do not require collection of high cost learning data. We show that color adjustment and image completion based on context features are executed at the same time, and natural pasting synthesis can be performed by using these proposal methods.

• Access to Document (DOI)