研究者詳細 - 青木　義満

論文 - 青木　義満

分割表示 181 件中 41 - 60 件目／全件表示 >>

Scenery Image Extension via Inpainting with a Mirrored Input

Akimoto N., Ito D., Aoki Y.

IEEE Access （IEEE Access） 9 59286 - 59300 2021年

Generative image extension has the advantage of extending the overall image size while preserving the target image because, unlike other image extensions using interpolation, it completes the surroundings of the target image. However, existing generative image extension methods tend to have poor quality in the generation of outer pixels. One method deals only with a limited number of scene classes because the extension repeats the same semantics. We propose a mirrored input, which sandwiches the extended region by mirroring a part of the target image. This replaces generative image extension with an image inpainting problem and thus helps to achieve higher quality pixel generation and can extend semantics with more complex shapes than horizontal repetition. Experimental results show that our proposed method achieves a scenery image extension that exceeds the state-of-the-art generative image extension methods in both visual quality and FID score for datasets containing diverse scenes.

Access to Document (DOI)

Analysis of recent re-identification architectures for tracking-by-detection paradigm in multi-object tracking

Ishikawa H., Hayashi M., Phan T.H., Yamamoto K., Masuda M., Aoki Y.

VISIGRAPP 2021 - Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications （VISIGRAPP 2021 - Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications） 5 234 - 244 2021年

ISSN 9789897584886

Person re-identification is a vital module of the tracking-by-detection framework for online multi-object tracking. Despite recent advances in multi-object tracking and person re-identification, inadequate attention was given to integrating these technologies to provide a robust multi-object tracker. In this work, we combine modern state-of-the-art re-identification models and modeling techniques on the basic tracking-by-detection framework and benchmark them on heavily occluded scenes to understand their effect. We hypothesize that temporal modeling for re-identification is crucial for training robust re-identification models for they are conditioned on sequences containing occlusions. Along with traditional image-based re-identification methods, we analyze temporal modeling methods used in video-based re-identification tasks. We also train re-identification models with different embedding methods, including triplet loss, and analyze their effect. We benchmark the re-identification models on the challenging MOT20 dataset containing crowded scenes with various occlusions. We provide a thorough assessment and investigation of the usage of modern re-identification modeling methods and prove that these methods are, in fact, effective for multi-object tracking. Compared to baseline methods, results show that these models can provide robust re-identification proved by improvements in the number of identity switching, MOTA, IDF1, and other metrics.

Multi-modal action segmentation in the kitchen with a feature fusion approach

Kogure S., Aoki Y.

Proceedings of SPIE - The International Society for Optical Engineering （Proceedings of SPIE - The International Society for Optical Engineering） 11794 2021年

ISSN 0277786X

In this paper, we propose a "Multi-modal Action Segmentation approach"that uses three modalities: (i) video, (ii) audio, (iii) thermal to classify cooking behavior in the kitchen. These 3 modalities are assumed to be features related to cooking. However, there is no public dataset containing these three modalities. Therefore, we built the original dataset and frame-level annotation. We then examined the usefulness of Action Segmentation using multi-modal features. We analyzed the effects of each modality using three evaluation metrics. As a result, the accuracy, edit distance, and F1 value were improved by up to about 1%, 2%, and 8%, respectively, compared to the case when only images were used.

Access to Document (DOI)

Out-of-distribution detection for fungi images with similar features

Kawashima Y., Higo M., Tokiwa T., Asami Y., Nonaka K., Aoki Y.

Proceedings of SPIE - The International Society for Optical Engineering （Proceedings of SPIE - The International Society for Optical Engineering） 11794 2021年

ISSN 0277786X

In order to create a classification model for fungi, it is necessary to have robustness against out-of-distribution data from the viewpoint of practicality. Therefore, in this paper, we perform out-of-distribution detection on a fungi. Unlike the case of conventional out-of-distribution detection, the characteristics of in-distribution data and out-of-distribution data in this paper are very similar. Therefore, the problem in which conventional methods using out-of-distribution data for validation are not effective is mentioned. We also verify whether the accuracy of out-of-distribution detection can be improved using the attention branch network.

Access to Document (DOI)

Retrieving and Highlighting Action with Spatiotemporal Reference

Kasai S., Ishikawa Y., Hayashi M., Aoki Y., Hara K., Kataoka H.

Proceedings - International Conference on Image Processing, ICIP （Proceedings - International Conference on Image Processing, ICIP） 2020-October 1401 - 1405 2020年10月

ISSN 9781728163956

In this paper, we present a framework thatjointly retrieves and spatiotemporally highlights actions in videos by enhancing current deep cross-modal retrieval methods. Our work takes on the novel task of action highlighting, which visualizes where and when actions occur in an untrimmed video setting. Action highlighting is a fine-grained task, compared to conventional action recognition tasks which focus on classification or window-based localization. Leveraging weak supervision from annotated captions, our framework acquires spatiotemporal relevance maps and generates local embeddings which relate to the nouns and verbs in captions. Through experiments, we show that our model generates various maps conditioned on different actions, in which conventional visual reasoning methods only go as far as to show a single deterministic saliency map. Also, our model improves retrieval recall over our baseline without alignment by 2-3% on the MSR-VTT dataset.

Access to Document (DOI)

Joint Pedestrian Detection and Risk-level Prediction with Motion-Representation-by-Detection

Kataoka H., Suzuki T., Nakashima K., Satoh Y., Aoki Y.

Proceedings - IEEE International Conference on Robotics and Automation （Proceedings - IEEE International Conference on Robotics and Automation） 1021 - 1027 2020年05月

ISSN 9781728173955

The paper presents a pedestrian near-miss detector with temporal analysis that provides both pedestrian detection and risk-level predictions which are demonstrated on a self-collected database. Our work makes three primary contributions: (i) The framework of pedestrian near-miss detection is proposed by providing both a pedestrian detection and risk-level assignment. Specifically, we have created a Pedestrian Near-Miss (PNM) dataset that categorizes traffic near-miss incidents based on their risk levels (high-, low-, and no-risk). Unlike existing databases, our dataset also includes manually localized pedestrian labels as well as a large number of incident-related videos. (ii) Single-Shot MultiBox Detector with Motion Representation (SSD-MR) is implemented to effectively extract motion-based features in a detected pedestrian. (iii) Using the self-collected PNM dataset and SSD-MR, our proposed method achieved +19.38% (on risk-level prediction) and +13.00% (on joint pedestrian detection and risk-level prediction) higher scores than that of the baseline SSD and LSTM. Additionally, the running time of our system is over 50 fps on a graphics processing unit (GPU).

Access to Document (DOI)

FOE-based regularization for optical flow estimation from an in-vehicle event camera

Nagata J., Sekikawa Y., Hara K., Aoki Y.

Electronics and Communications in Japan （Electronics and Communications in Japan） 103 （ 1-4 ） 19 - 25 2020年04月

ISSN 19429533

Optical flow estimation from an in-vehicle camera is an important task in automatic driving and advanced driver-assistance systems. However, there is a problem that optical flow estimation is mistakable with high contrast and high speed. Event camera can overcome these situations because it reports only the per-pixel intensity change with high dynamic range and low latency. However, the L1 smoothness regularization in the conventional optical flow estimation method is not suitable for radial optical flow in the driving scene. Therefore, we propose to use the focus of expansion (FOE) for regularization of optical flow estimation in event camera. The FOE is defined as the intersection of the translation vector of the camera and the image plane. The optical flow becomes radial from the FOE excluding the rotational component. Using the property, the optical flow can be regularized in the correct direction in the optimization process. We demonstrated that the optical flow was improved by introducing our regularization using the public dataset.

Access to Document (DOI)

QR-code reconstruction from event data via optimization in code subspace

Nagata J., Sekikawa Y., Hara K., Suzuki T., Yoshimitsu A.

Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020 （Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020） 2113 - 2121 2020年03月

ISSN 9781728165530

We propose an image reconstruction method from event data, assuming the target images belong to a prespecified class like QR codes. Instead of solving the reconstruction problem in the image space, we introduce a code space that covers all the noiseless target class images and solves the reconstruction problem on it. This restriction enormously reduces the number of optimizing parameters and makes the reconstruction problem well posed and robust to noise. We demonstrate fast and robust QR-code scanning in difficult, high-speed scenes with industrial high-speed cameras and other reconstruction methods.

Access to Document (DOI)

Fast soft color segmentation

Akimoto N., Zhu H., Jin Y., Aoki Y.

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition （Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition） 8274 - 8283 2020年

ISSN 10636919

We address the problem of soft color segmentation, defined as decomposing a given image into several RGBA layers, each containing only homogeneous color regions. The resulting layers from decomposition pave the way for applications that benefit from layer-based editing, such as recoloring and compositing of images and videos. The current state-of-the-art approach for this problem is hindered by slow processing time due to its iterative nature, and consequently does not scale to certain real-world scenarios. To address this issue, we propose a neural network based method for this task that decomposes a given image into multiple layers in a single forward pass. Furthermore, our method separately decomposes the color layers and the alpha channel layers. By leveraging a novel training objective, our method achieves proper assignment of colors amongst layers. As a consequence, our method achieve promising quality without existing issue of inference speed for iterative approaches. Our thorough experimental analysis shows that our method produces qualitative and quantitative results comparable to previous methods while achieving a 300,000x speed improvement. Finally, we utilize our proposed method on several applications, and demonstrate its speed advantage, especially in video editing.

Access to Document (DOI)

Action Recognition in Sports Video Considering Location Information

Ichige R., Aoki Y.

Communications in Computer and Information Science （Communications in Computer and Information Science） 1212 CCIS 150 - 164 2020年

ISSN 9789811548178

The purpose of this study is to develop a tactics analysis system using image recognition for rugby. With the Rugby World Cup in 2019 and the Tokyo Olympics in 2020, demand for sports video analysis is increasing. Rugby has more complicated play such as dense play than other sports, and the ball is hidden between players, making it difficult to track. By developing a high-precision analysis technology for rugby with few research cases, we thought that it could be used for other sports and industrial fields other than sports. In this research, we propose a method that adds spatial information to time-series information as a new feature. Using the coordinates obtained by projectively transforming the match video onto the bird’s-eye view image, play classification was performed using the player position, the ball position, and the dense area position as feature amounts. Also, in order to further improve the detection accuracy of the boundaries between plays, attention was paid to the positional relationship of each player on the field.

Access to Document (DOI)

Weakly Supervised Domain Adaptation with Point Supervision in Histopathological Image Segmentation

Obikane S., Aoki Y.

Communications in Computer and Information Science （Communications in Computer and Information Science） 1180 CCIS 127 - 140 2020年

ISSN 9789811536502

When a model learned in a domain is applied to a different domain, even if in the same task, there is no guarantee of accuracy. This is a very important issue when deep learning and machine learning are applied in the field. In medical applications, there is a wide variety of domain bias, making it very difficult to create a model appropriate for all domains. Furthermore, semantic segmentation needs fine annotation and its high labor cost makes its application difficult. Histopathological image segmentation enables drug discovery and medical image analysis, but it is expensive due to its annotation cost and the need for the skills of histopathological experts. In this paper, we focus on a weakly supervised method using point annotation unique to histopathological image segmentation, and tackled on weakly supervised domain adaptation to suppress domain gaps. Providing point level annotation instead of fine annotation decreases the high cost of labor normally required.

Access to Document (DOI)

Flood Susceptibility Prediction via Data-Mining Based Bell-Curve Analogical-Hydrographs Analysis: A Case Study of Langat River Basin, Selangor, Malaysia

Khuzaimah S.N., Kiyoki Y., Aoki Y.

Frontiers in Artificial Intelligence and Applications （Frontiers in Artificial Intelligence and Applications） 321 442 - 457 2019年12月

ISSN 9781643680446

In this study, we proposed data-mining based bell-curve analogical hydrographs analysis with lag time vertical axes and bankfull discharge horizontal axes to make flood susceptibility prediction. We utilized flood data reports, hourly/daily rainfall data and daily water discharge of Hulu Langat district, Selangor Malaysia from the year 2013-2016 to do flood susceptibility. We implement data mining concept by sorting the database, followed by plotting hydrograph to identify flood patterns and establish relationships to predict flood trends. This method is an intersection between the knowledge field of hydrology and mathematical modeling. When an outlier from the graph is detected, the knowledge from hydrology can be applied to understand the reason behind the appearance of outliers. Besides, the knowledge of mathematical modeling is necessary to assist us in predicting flood susceptibility. The purpose of this study is to predict the flood susceptibility which is vital to prepare the users/public well prepared for smooth and efficient evacuation. In 4 years context, our flood depth predictions are nearly 100% accurate. Factor influencing the lag time and steepness of rising limb are related to land use and topographical features. Implications of the results and future research directions are also presented.

Access to Document (DOI)

An application of artificial intelligence to diagnostic imaging of spine disease: Estimating spinal alignment from moiré images

Watanabe K., Aoki Y., Matsumoto M.

Neurospine （Neurospine） 16 （ 4 ） 697 - 702 2019年12月

ISSN 25866583

The use of artificial intelligence (AI) as a tool supporting the diagnosis and treatment of spinal diseases is eagerly anticipated. In the field of diagnostic imaging, the possible application of AI includes diagnostic support for diseases requiring highly specialized expertise, such as trauma in children, scoliosis, symptomatic diseases, and spinal cord tumors. Moiré topography, which describes the 3-dimensional surface of the trunk with band patterns, has been used to screen students for scoliosis, but the interpretation of the band patterns can be ambiguous. Thus, we created a scoliosis screening system that estimates spinal alignment, the Cobb angle, and vertebral rotation from moiré images. In our system, a convolutional neural network (CNN) estimates the positions of 12 thoracic and 5 lumbar vertebrae, 17 spinous processes, and the vertebral rotation angle of each vertebra. We used this information to estimate the Cobb angle. The mean absolute error (MAE) of the estimated vertebral positions was 3.6 pixels (~5.4 mm) per person. T1 and L5 had smaller MAEs than the other levels. The MAE per person between the Cobb angle measured by doctors and the estimated Cobb angle was 3.42°. The MAE was 4.38° in normal spines, 3.13° in spines with a slight deformity, and 2.74° in spines with a mild to severe deformity. The MAE of the angle of vertebral rotation was 2.9° ± 1.4°, and was smaller when the deformity was milder. The proposed method of estimating the Cobb angle and AVR from moiré images using a CNN is expected to enhance the accuracy of scoliosis screening.

Access to Document (DOI)

Task-oriented function detection based on operational tasks

Ishikawa Y., Ishikawa H., Akizuki S., Yamazaki M., Taniguchi Y., Aoki Y.

2019 19th International Conference on Advanced Robotics, ICAR 2019 （2019 19th International Conference on Advanced Robotics, ICAR 2019） 635 - 640 2019年12月

ISSN 9781728124674

We propose novel representations for functions of an object, namely Task-oriented Function, which is improved upon the idea of Afforadance in the field of Robotics Vision. We also propose a convolutional neural network to detect task-oriented functions. This network takes as input an operational task as well as an RGB image and assign each pixel an appropriate label for every task. Task-oriented funciton makes it possible to descibe various ways to use an object because the outputs from the network differ depending on operational tasks. We introduce a new dataset for task-oriented function detection, which contains about 1200 RGB images and 6000 pixel-level annotations assuming five tasks. Our proposed method reached 0.80 mean IOU in our dataset.

Access to Document (DOI)

Shot detection in racket sport video at the frame level using a recurrent neural network

Horie S., Sato Y., Furuyama J., Tanabiki M., Aoki Y.

Proceedings - 15th International Conference on Signal Image Technology and Internet Based Systems, SISITS 2019 （Proceedings - 15th International Conference on Signal Image Technology and Internet Based Systems, SISITS 2019） 447 - 453 2019年11月

ISSN 9781728156866

In recent years, there has been a demand in the sports industry to reduce the burden of data collection and video editing for tactical analysis. To achieve these, a system that can recognize the game context is needed. In this study, we proposed a method to identify the player's shot timing at the frame level during a ball-striking sport. In this study, players' shots were detected in video of a tennis match. It was shown that shots could be detected with an F-score value of 87% or more within an error range of 1 frame (0.033 sec) by considering time-series information using a recurrent neural network. This technology is expected to be applied not only to tennis, but also to other sports that involve ball shots, such as table tennis, baseball, and volleyball. At the same time, it can be used to detect moments of a specific action (for example, touching or hitting an object).

Access to Document (DOI)

Fine-grained action recognition in assembly work scenes by drawing attention to the hands

Kobayashi T., Aoki Y., Shimizu S., Kusano K., Okumura S.

ISSN 9781728156866

The analysis of assembly working scenes is an important tool for improving work efficiency and detecting worker error. Many factories choose to analyze videos of employees working by human observation. It is more efficient, however, to approach this as an action segmentation task, but this is difficult due to the fine-grained actions. In this paper, we focus on featuring the workers hands in action segmentation to describe the detailed moves in assembly work scenes and utilize the tool or product information that the worker is using. We propose two methods to draw attention to the hand from each image in the video: the first is by cutting out the workers hand image and extracting features, and the second is by using an attention module specialized to extract hand features by the training of a regression network. For both methods we combine different types of features - image features and pose features - which are both important information in fine-grained actions. Due to the lack of action datasets that focus on assembly work scenes, we create a new assembly work dataset for our experiments, and the results of those experiments show that our method is effective for analyzing fine-grained action segmentation tasks.

Access to Document (DOI)

360-Degree Image Completion by Two-Stage Conditional Gans

Akimoto N., Kasai S., Hayashi M., Aoki Y.

Proceedings - International Conference on Image Processing, ICIP （Proceedings - International Conference on Image Processing, ICIP） 2019-September 4704 - 4708 2019年09月

ISSN 9781538662496

The latest generative adversarial networks (GANs) can generate realistic high resolution images. However, to the best of our knowledge, there are no GANs for generating 360-degree images. Therefore, this paper proposes the novel problem setting that by using a known area from the 360-degree image as an input, the remainder of the image can be completed with the GANs. To do so, we propose the approach of two-stage generation using network architecture with series-parallel dilated convolution layers. Moreover, we present how to rearrange images for data augmentation, simplify the problem, and make inputs for training the 2 stage generator. Our experiments show that these methods generate the distortion seen in 360-degree images in the outlines of buildings and roads, and their boundaries are clearer than those of baseline methods. Furthermore, we discuss and clarify the difficulty of our proposed problem. Our work is the first step towards GANs predicting an unseen area within a 360-degree space. nd

Access to Document (DOI)

Human-object maps for daily activity recognition

Ishikawa H., Ishikawa Y., Akizuki S., Aoki Y.

Proceedings of the 16th International Conference on Machine Vision Applications, MVA 2019 （Proceedings of the 16th International Conference on Machine Vision Applications, MVA 2019） 2019年05月

ISSN 9784901122184

In the field of action recognition, when and where an interaction between a human and an object happens has the potential to be valid information in enhancing action recognition accuracy. Especially, in daily life where each activities are performed in longer time frame, conventional short term action recognition may fail to generalize do to the variety of shorter actions that could take place during the activity. In this paper, we propose a novel representation of human object interaction called Human-Object Maps (HOMs) for recognition of long term daily activities. HOMs are 2D probability maps that represents spatio-temporal information of human object interaction in a given scene. We analyzed the effectiveness of HOMs as well as features relating to the time of the day in daily activity recognition. Since there are no publicly available daily activity dataset that depicts daily routines needed for our task, we have created a new dataset that contains long term activities. Using this dataset, we confirm that our method enhances the prediction accuracy of the conventional 3D ResNeXt action recognition method from 86.31% to 97.89%.

Access to Document (DOI)

Tactile logging for understanding plausible tool use based on human demonstration

Akizuki S., Aoki Y.

British Machine Vision Conference 2018, BMVC 2018 （British Machine Vision Conference 2018, BMVC 2018） 2019年

In this paper, we propose a novel human-object interaction (HOI) representation for understanding plausible tool use; this is done by using an object-centric method. Everyday tools (such as kitchen or DIY utensils) have the ideal region and direction for being touched by hands and surrounding objects. By analyzing the human demonstrations, tactile histories that occurred on the tools are accumulated to the tool's surface as the "Tactile Log", which can help to achieve the plausible tool use by robot arms. We also proposed a dataset for the evaluation of generated tactile logs, which is called the Object-Centric Interaction (OCI) dataset consists of RGB-D videos of plausible tool use by human and 3D models of the tools used in the videos. We confirm that the precision of our tactile logging is 0.77.

FOE-based regularization for optical flow estimation from an in-vehicle event camera

Nagata J., Sekikawa Y., Hara K., Aoki Y.

IEEJ Transactions on Electronics, Information and Systems （IEEJ Transactions on Electronics, Information and Systems） 139 （ 10 ） 1113 - 1118 2019年

ISSN 03854221

Optical flow estimation from an in-vehicle camera is an important task in automatic driving and advanced driver- A ssistance systems. However, there is a problem that optical flow estimation is mistakable with high contrast and high speed. Event camera can overcome these situations because it reports only the per-pixel intensity change with high dynamic range and low latency. However, the LI smoothness regularization in the conventional optical flow estimation method is not suitable for radial optical flow in the driving scene. Therefore, we propose to use the focus of expansion (FOE) for regularization of optical flow estimation in event camera. The FOE is defined as the intersection of the translation vector of the camera and the image plane. The optical flow becomes radial from the FOE excluding the rotational component. Using the property, the optical flow can be regularized in the correct direction in the optimization process. We demonstrated that the optical flow was improved by introducing our regularization using the public dataset.

Access to Document (DOI)

　前のページ - 次のページ　

青木 義満

論文 - 青木 義満

青木　義満

論文 - 青木　義満