Details of a Researcher

Papers - Saito, Hideo

Division display >> ／ All the affair displays 1 - 356 of about 356

InpaintFusion: Incremental RGB-D Inpainting for 3D Scenes

Mori S., Erat O., Broll W., Saito H., Schmalstieg D., Kalkofen D.

IEEE Transactions on Visualization and Computer Graphics (IEEE Transactions on Visualization and Computer Graphics) 26 ( 10 ) 2994 - 3007 2020.10

ISSN 10772626

© 1995-2012 IEEE. State-of-the-art methods for diminished reality propagate pixel information from a keyframe to subsequent frames for real-time inpainting. However, these approaches produce artifacts, if the scene geometry is not sufficiently planar. In this article, we present InpaintFusion, a new real-time method that extends inpainting to non-planar scenes by considering both color and depth information in the inpainting process. We use an RGB-D sensor for simultaneous localization and mapping, in order to both track the camera and obtain a surfel map in addition to RGB images. We use the RGB-D information in a cost function for both the color and the geometric appearance to derive a global optimization for simultaneous inpainting of color and depth. The inpainted depth is merged in a global map by depth fusion. For the final rendering, we project the map model into image space, where we can use it for effects such as relighting and stereo rendering of otherwise hidden structures. We demonstrate the capabilities of our method by comparing it to inpainting results with methods using planar geometric proxies.

Access to Document (DOI)

Resolving position ambiguity of imu-based human pose with a single RGB camera

Kaichi T., Maruyama T., Tada M., Saito H.

Sensors (Switzerland) (Sensors (Switzerland)) 20 ( 19 ) 1 - 12 2020.10

ISSN 14248220

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. Human motion capture (MoCap) plays a key role in healthcare and human–robot collaboration. Some researchers have combined orientation measurements from inertial measurement units (IMUs) and positional inference from cameras to reconstruct the 3D human motion. Their works utilize multiple cameras or depth sensors to localize the human in three dimensions. Such multiple cameras are not always available in our daily life, but just a single camera attached in a smart IP devices has recently been popular. Therefore, we present a 3D pose estimation approach from IMUs and a single camera. In order to resolve the depth ambiguity of the single camera configuration and localize the global position of the subject, we present a constraint which optimizes the foot-ground contact points. The timing and 3D positions of the ground contact are calculated from the acceleration of IMUs on foot and geometric transformation of foot position detected on image, respectively. Since the results of pose estimation is greatly affected by the failure of the detection, we design the image-based constraints to handle the outliers of positional estimates. We evaluated the performance of our approach on public 3D human pose dataset. The experiments demonstrated that the proposed constraints contributed to improve the accuracy of pose estimation in single and multiple camera setting.

Access to Document (DOI)

Pose estimation of primitive-shaped objects from a depth image using superquadric representation

Hachiuma R., Saito H.

Applied Sciences (Switzerland) (Applied Sciences (Switzerland)) 10 ( 16 ) 2020.08

© 2020 by the authors. This paper presents a method for estimating the six Degrees of Freedom (6DoF) pose of texture-less primitive-shaped objects from depth images. As the conventional methods for object pose estimation require rich texture or geometric features to the target objects, these methods are not suitable for texture-less and geometrically simple shaped objects. In order to estimate the pose of the primitive-shaped object, the parameters that represent primitive shapes are estimated. However, these methods explicitly limit the number of types of primitive shapes that can be estimated. We employ superquadrics as a primitive shape representation that can represent various types of primitive shapes with only a few parameters. In order to estimate the superquadric parameters of primitive-shaped objects, the point cloud of the object must be segmented from a depth image. It is known that the parameter estimation is sensitive to outliers, which are caused by the miss-segmentation of the depth image. Therefore, we propose a novel estimation method for superquadric parameters that are robust to outliers. In the experiment, we constructed a dataset in which the person grasps and moves the primitive-shaped objects. The experimental results show that our estimation method outperformed three conventional methods and the baseline method.

Access to Document (DOI)

CorsNet: 3D Point Cloud Registration by Deep Neural Network

Kurobe A., Sekikawa Y., Ishikawa K., Saito H.

IEEE Robotics and Automation Letters (IEEE Robotics and Automation Letters) 5 ( 3 ) 3960 - 3966 2020.07

© 2016 IEEE. Point cloud registration is a key problem for robotics and computer vision communities. This represents estimating a rigid transform which aligns one point cloud to another. Iterative closest point (ICP) is a well-known classical method for this problem, yet it generally achieves high alignment only when the source and template point cloud are mostly pre-aligned. If each point cloud is far away or contains a repeating structure, the registration often fails because of being fallen into a local minimum. Recently, inspired by PointNet, several deep learning-based methods have been developed. PointNetLK is a representative approach, which directly optimizes the distance of aggregated features using gradient method by Jacobian. In this paper, we propose a point cloud registration system based on deep learning: CorsNet. Since CorsNet concatenates the local features with the global features and regresses correspondences between point clouds, not directly pose or aggregated features, more useful information is integrated than the conventional approaches. For comparison, we also developed a novel deep learning approach (DirectNet) that directly regresses the pose between point clouds. Through our experiments, we show that CorsNet achieves higher accuracy than not only the classic ICP method, but also the recently proposed learning-based proposal PointNetLK and DirectNet, including on seen and unseen categories.

Access to Document (DOI)

Pose Estimation of Stacked Rectangular Objects from Depth Images

Matsuno D., Hachiuma R., Saito H., Sugano J., Adachi H.

IEEE International Symposium on Industrial Electronics (IEEE International Symposium on Industrial Electronics) 2020-June 1409 - 1414 2020.06

ISSN 9781728156354

© 2020 IEEE. This paper addresses the task of six degrees of freedom (6-DoF) pose estimation of stacked rectangular objects from depth images. Object pose estimation is one of the key challenges for visual processing systems since it plays a vital role in many situations such as warehouse/factory automation, robotic manipulation, and augmented reality. Many recent approaches to object pose estimation use RGB information for detecting and estimating the pose of objects. However, in warehouse/factory automation, objects are often small, occluded, cluttered, and texture-less which makes it difficult to utilize RGB features for detection and pose estimation. In order to overcome this restriction, we use only the depth information (without RGB information) and its geometric features to segment each object and to estimate the 6-DoF (position and orientation) in a stacked scene. We segment the rectangular objects in each scene from the depth and surface normal discontinuities (geometric segmentation). From the geometrically segmented image, four object corner points can be estimated using the convex hull detection and the eight corner points, which are required for the 6-DoF pose estimation, can be calculated. To improve the accuracy of orientation estimation, we estimate four orientation candidates and select the best among them. Experimental results using two evaluation methods show that our method outperformed the baseline method.

Access to Document (DOI)

Detection of cervical myelopathy with Leap Motion Sensor by random forests

Watanabe M., Sugiura Y., Saito H., Koyama T., Fujita K.

LifeTech 2020 - 2020 IEEE 2nd Global Conference on Life Sciences and Technologies (LifeTech 2020 - 2020 IEEE 2nd Global Conference on Life Sciences and Technologies) 214 - 216 2020.03

ISSN 9781728170633

© 2020 IEEE. This paper presents a method for automatic detection of cervical myelopathy by using Leap Motion Sensor for sensing hand motion to apply a method called the 10-s grip and release test. Leap Motion data is analyzed for computing feature of hand motion, which is used for automatic detection of patient or healthy people by random forests algorithm for machine learning. For evaluating automatic detection performance, we collect real data of hand motion 54 of patient and healthy people, which is used for validating accuracy of detection.

Access to Document (DOI)

Gait Measurement at Home Using A Single RGB Camera

Yagi K., Sugiura Y., Hasegawa K., Saito H.

Gait and Posture (Gait and Posture) 76 136 - 140 2020.02

ISSN 09666362

© 2019 Elsevier B.V. Background: An individual's gait is a key factor for consideration in evaluating their overall health. Several medical studies have demonstrated the correlation between gait and incidence rate of diseases, mortality, and risk of fall. However, gait is only occasionally evaluated during medical visits, which may delay the detection of health problems. Methods: In this paper, we propose a gait measurement system that is suitable for use at home. Our method requires only a single RGB camera, whereas other visionary sensor-based methods require depth sensors or multiple RGB cameras. In addition, the setup for the measurement is easy. What the user has to do is only putting a single camera in a room and choosing four location known points on the floor. Our method can measure step positions and step timings, and therefore, other important parameters such as stride length, step width, walking speed, and cadence may also be captured. The individual's gait is captured by the camera, and therefore the user is not required to wear any devices. Results: In the experiment described herein, we demonstrate our method's accuracy by comparing it with the motion capture system. The results indicate that our method can measure walking speed with an error of 3.62 cm/s from the side view, and which is too small a change to be clinically meaningful.

Access to Document (DOI)

Rgb-d image inpainting using generative adversarial network with a late fusion approach

Fujii R., Hachiuma R., Saito H.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)) 12242 LNCS 440 - 451 2020

ISSN 9783030584641

© Springer Nature Switzerland AG 2020. Diminished reality is a technology that aims to remove objects from video images and fills in the missing region with plausible pixels. Most conventional methods utilize the different cameras that capture the same scene from different viewpoints to allow regions to be removed and restored. In this paper, we propose an RGB-D image inpainting method using generative adversarial network, which does not require multiple cameras. Recently, an RGB image inpainting method has achieved outstanding results by employing a generative adversarial network. However, RGB inpainting methods aim to restore only the texture of the missing region and, therefore, does not recover geometric information (i.e, 3D structure of the scene). We expand conventional image inpainting method to RGB-D image inpainting to jointly restore the texture and geometry of missing regions from a pair of RGB and depth images. Inspired by other tasks that use RGB and depth images (e.g., semantic segmentation and object detection), we propose late fusion approach that exploits the advantage of RGB and depth information each other. The experimental results verify the effectiveness of our proposed method.

Access to Document (DOI)

In-Plane Rotation-Aware Monocular Depth Estimation Using SLAM

Saito Y., Hachiuma R., Yamaguchi M., Saito H.

Communications in Computer and Information Science (Communications in Computer and Information Science) 1212 CCIS 305 - 317 2020

ISSN 9789811548178

© 2020, Springer Nature Singapore Pte Ltd. Estimating accurate depth from an RGB image in any environment is challenging task in computer vision. Recent learning based method using deep Convolutional Neural Networks (CNNs) have driven plausible appearance, but these conventional methods are not good at estimating scenes that have a pure rotation of camera, such as in-plane rolling. This movement imposes perturbations on learning-based methods because gravity direction is considered to be strong prior to CNN depth estimation (i.e., the top region of an image has a relatively large depth, whereas bottom region tends to have a small depth). To overcome this crucial weakness in depth estimation with CNN, we propose a simple but effective refining method that incorporates in-plane roll alignment using camera poses of monocular Simultaneous Localization and Mapping (SLAM). For the experiment, we used public datasets and also created our own dataset composed of mostly in-plane roll camera movements. Evaluation results on these datasets show the effectiveness of our approach.

Access to Document (DOI)

Global-map-registered local visual odometry using on-the-fly pose graph updates

Yamaguchi M., Mori S., Saito H., Yachida S., Shibata T.

ISSN 9783030584641

© Springer Nature Switzerland AG 2020. Real-time camera pose estimation is one of the indispensable technologies for Augmented Reality (AR). While a large body of work in Visual Odometry (VO) has been proposed for AR, practical challenges such as scale ambiguities and accumulative errors still remain especially when we apply VO to large-scale scenes due to limited hardware and resources. We propose a camera pose registration method, where a local VO is consecutively optimized with respect to a large-scale scene map on the fly. This framework enables the scale estimation between a VO map and a scene map and reduces accumulative errors by finding corresponding locations in the map to the current frame and by on-the-fly pose graph optimization. The results using public datasets demonstrated that our approach reduces the accumulative errors of naïve VO.

Access to Document (DOI)

DetectFusion: Detecting and segmenting both known and unknown dynamic objects in real-time slam

Hachiuma R., Pirchheim C., Schmalstieg D., Saito H.

30th British Machine Vision Conference 2019, BMVC 2019 (30th British Machine Vision Conference 2019, BMVC 2019) 2020

© 2019. The copyright of this document resides with its authors. We present DetectFusion, an RGB-D SLAM system that runs in real time and can robustly handle semantically known and unknown objects that can move dynamically in the scene. Our system detects, segments and assigns semantic class labels to known objects in the scene, while tracking and reconstructing them even when they move independently in front of the monocular camera. In contrast to related work, we achieve real-time computational performance on semantic instance segmentation with a novel method combining 2D object detection and 3D geometric segmentation. In addition, we propose a method for detecting and segmenting the motion of semantically unknown objects, thus further improving the accuracy of camera tracking and map reconstruction. We show that our method performs on par or better than previous work in terms of localization and object reconstruction accuracy, while achieving about 20 fps even if the objects are segmented in each frame.

Deep Selection: A Fully Supervised Camera Selection Network for Surgery Recordings

Hachiuma R., Shimizu T., Saito H., Kajita H., Takatsume Y.

ISSN 9783030597153

© 2020, Springer Nature Switzerland AG. Recording surgery in operating rooms is an essential task for education and evaluation of medical treatment. However, recording the desired targets, such as the surgery field, surgical tools, or doctor’s hands, is difficult because the targets are heavily occluded during surgery. We use a recording system in which multiple cameras are embedded in the surgical lamp, and we assume that at least one camera is recording the target without occlusion at any given time. As the embedded cameras obtain multiple video sequences, we address the task of selecting the camera with the best view of the surgery. Unlike the conventional method, which selects the camera based on the area size of the surgery field, we propose a deep neural network that predicts the camera selection probability from multiple video sequences by learning the supervision of the expert annotation. We created a dataset in which six different types of plastic surgery are recorded, and we provided the annotation of camera switching. Our experiments show that our approach successfully switched between cameras and outperformed three baseline methods.

Access to Document (DOI)

Surgery recording without occlusions by multi-view surgical videos

Shimizu T., Oishi K., Hachiuma R., Kajita H., Takatsume Y., Saito H.

VISIGRAPP 2020 - Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2020 - Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications) 5 837 - 844 2020

ISSN 9789897584022

Copyright © 2020 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved. Recording surgery is important for sharing various operating techniques. In most surgical rooms, fixed surgical cameras are already installed, but it is almost impossible to capture the surgical field because of occlusion by the surgeon’s head and body. In order to capture the surgical field, we propose the installation of multiple cameras in a surgical lamp system, so that at least one camera can capture the surgical field even when the surgeon’s head and body occlude other cameras. In this paper, we present a method for automatic viewpoint switching from multi-view surgical videos, so that the surgical field can always be recorded. We employ a method for learning-based object detection from videos for automatic evaluation of the surgical field from multi-view surgical videos. In general, frequent camera switching degrades the video quality of view (QoV). Therefore, we apply Dijkstra’s algorithm, widely used in the shortest path problem, as an optimization method for this problem. Our camera scheduling method works so that camera switching is not performed for the minimum frame we specified, and therefore the surgical field observed in the entire video is maximized.

MMSports'19: 2nd ACM international workshop on multimedia content analysis in sports

Lienhart R., Moeslund T.B., Saito H.

MM 2019 - Proceedings of the 27th ACM International Conference on Multimedia (MM 2019 - Proceedings of the 27th ACM International Conference on Multimedia) 2732 - 2733 2019.10

ISSN 9781450368896

© 2019 Copyright is held by the owner/author(s). ACM ISBN 978-1-4503-6889-6/19/10. The second ACM International Workshop on Multimedia Content Analysis in Sports (ACM MMSports'19) is held in Nice, France on October 25th, 2019 co-located with the ACM International Conference on Multimedia 2019 (ACM Multimedia 2019). The goal of this workshop is to bring together researchers and practitioners from academia and industry to address challenges and report progress in mining, analyzing, understanding and visualizing the multimedia/multimodal data in sports. The combination of sports and modern technology offers a novel and intriguing field of research with promising approaches for visual broadcast augmentation, understanding, statistical analysis and evaluation, and sensor fusion. There is a lack of research communities focusing on the fusion of multiple modalities. We are helping to close this research gap with this workshop series on multimedia content analysis in sports.

Access to Document (DOI)

Prediction of future shot direction using pose and position of tennis player

Shimizu T., Hachiuma R., Saito H., Yoshikawa T., Lee C.

MMSports 2019 - Proceedings of the 2nd International Workshop on Multimedia Content Analysis in Sports, co-located with MM 2019 (MMSports 2019 - Proceedings of the 2nd International Workshop on Multimedia Content Analysis in Sports, co-located with MM 2019) 59 - 66 2019.10

ISSN 9781450369114

© 2019 Copyright held by the owner/author(s). In this paper, we propose a method to predict the future shot direction in a tennis match using pose information and player position. As far as we know, there is no work that deals with such a predictive task, so there is no shot direction dataset as yet. Therefore, using a YouTube tennis match video, we construct an time of impact and shot direction dataset. To reduce annotation costs, we propose a method to automatically label the shot direction. Moreover, we propose a method to predict the future shot direction using the constructed dataset. The shot direction is predicted using LSTM(long short-time memory), from sequential pose information up to the time of impact and the player position. We employ OpenPose to extract the position of skeleton joints. In the experiment, we evaluate the accuracy of shot direction prediction and verify the effectiveness of the proposed method. Since there are no studies that predict future shot direction, we set four baseline methods to evaluate the effectiveness of our proposed method.

Access to Document (DOI)

Volumetric representation of human body parts using superquadrics

Hachiuma R., Saito H.

Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019 (Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019) 187 - 188 2019.10

ISSN 9781728147659

© 2019 IEEE. Superquadrics are one of the ideal shape representations for adapting various kinds of primitive shapes with a single equation. This paper revisits the task of representing a 3D human body with multiple superquadrics. As a single superquadric surface can only represent symmetric primitive shapes, we present a method that segments the human body into body parts to estimate their superquadric parameters. Moreover, we propose a novel initial parameter estimation method by using 3D skeleton joints. The results show that superquadric parameters are estimated, which represent human body parts volumetrically.

Access to Document (DOI)

Automatic viewpoint switching for multi-view surgical videos

Shimizu T., Oishi K., Saito H., Kajita H., Takatsume Y.

ISSN 9781728147659

© 2019 IEEE. Recording medical surgery operations is important for sharing the various operating techniques. In most operating rooms, fixed surgery cameras are already installed, but it is almost impossible to capture the surgical field because of occlusion by the surgeon's head and body. In order to capture the surgical field in real surgery operations, we proposed the installation of multiple cameras in a surgical lighting system, so that at least one camera can capture the target surgical field even when the surgeon's head and body occlude other cameras. In this paper, we present a method for automatic viewpoint selection from multi-view surgical videos, so that the surgical field can always be recorded in the output video. We employ a method for learning-based object detection from videos for automatic evaluation of the surgical field area from multiple input videos. By selecting the viewpoint with the largest area of the surgical field, we can virtually reduce the area of the surgeon's head and hands. In general, frequent camera switching degrades the video quality of view (QoV). Therefore, we apply the Dijkstra method widely used in the shortest path problem as a combinatorial optimization method for this problem. Our camera scheduling method is that the camera switching is not performed for a certain period of time, and the surgical field observed in the entire video is maximized.

Access to Document (DOI)

Joint inpainting of RGB and depth images by generative adversarial network with a late fusion approach

Fujii R., Hachiuma R., Saito H.

ISSN 9781728147659

© 2019 IEEE. Image inpainting aims to restore texture of missing regions in scene from an RGB image. In this paper, we aim to restore not only the texture but also the geometry of the missing regions in scene from a pair of RGB and depth images. Inspired by the recent development of generative adversarial network, we employ an encoder-decoderbased generative adversarial network with the input of RGB and depth image. The experimental results show that our method restores the missing region of both RGB and depth image.

Access to Document (DOI)

Incremental class discovery for semantic segmentation with rgbd sensing

Nakajima Y., Kang B., Saito H., Kitani K.

Proceedings of the IEEE International Conference on Computer Vision (Proceedings of the IEEE International Conference on Computer Vision) 2019-October 972 - 981 2019.10

ISSN 9781728148038

© 2019 IEEE. This work addresses the task of open world semantic segmentation using RGBD sensing to discover new semantic classes over time. Although there are many types of objects in the real-word, current semantic segmentation methods make a closed world assumption and are trained only to segment a limited number of object classes. Towards a more open world approach, we propose a novel method that incrementally learns new classes for image segmentation. The proposed system first segments each RGBD frame using both color and geometric information, and then aggregates that information to build a single segmented dense 3D map of the environment. The segmented 3D map representation is a key component of our approach as it is used to discover new object classes by identifying coherent regions in the 3D map that have no semantic label. The use of coherent region in the 3D map as a primitive element, rather than traditional elements such as surfels or voxels, also significantly reduces the computational complexity and memory use of our method. It thus leads to semi-real-time performance at 10.7 Hz when incrementally updating the dense 3D map at every frame. Through experiments on the NYUDv2 dataset, we demonstrate that the proposed method is able to correctly cluster objects of both known and unseen classes. We also show the quantitative comparison with the state-of-the-art supervised methods, the processing time of each step, and the influences of each component.

Access to Document (DOI)

Mobile Photometric Stereo with Keypoint-Based SLAM for Dense 3D Reconstruction

Maxence R., Uchiyama H., Kawasaki H., Thomas D., Nozick V., Saito H.

Proceedings - 2019 International Conference on 3D Vision, 3DV 2019 (Proceedings - 2019 International Conference on 3D Vision, 3DV 2019) 574 - 582 2019.09

ISSN 9781728131313

© 2019 IEEE. The standard photometric stereo is a technique to densely reconstruct objects' surfaces using light variation under the assumption of a static camera with a moving light source. In this work, we use photometric stereo to reconstruct dense 3D scenes while moving the camera and the light altogether. In such non-static case, camera poses as well as correspondences between pixels of each frame to apply photometric stereo are required. ORB-SLAM is a technique that can be used to estimate camera poses. To retrieve correspondences, our idea is to start from a sparse 3D mesh obtained with ORB SLAM and then densify the mesh by a plane sweep method using a multi-view photometric consistency. By combining ORB-SLAM and photometric stereo, it is possible to reconstruct dense 3D scenes with a off-the-shelf smartphone and its embedded torchlight. Note that SLAM systems usually struggle with textureless object, which is effectively compensated by the photometric stereo in our method. Experiments are conducted to show that our proposed method gives better results than SLAM alone or COLMAP, especially for partially textureless surfaces.

Access to Document (DOI)

Eventnet: Asynchronous recursive event processing

Sekikawa Y., Hara K., Saito H.

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition) 2019-June 3882 - 3891 2019.06

ISSN 9781728132938

© 2019 IEEE. Event cameras are bio-inspired vision sensors that mimic retinas to asynchronously report per-pixel intensity changes rather than outputting an actual intensity image at regular intervals. This new paradigm of image sensor offers significant potential advantages; namely, sparse and non-redundant data representation. Unfortunately, however, most of the existing artificial neural network architectures, such as a CNN, require dense synchronous input data, and therefore, cannot make use of the sparseness of the data. We propose EventNet, a neural network designed for real-time processing of asynchronous event streams in a recursive and event-wise manner. EventNet models dependence of the output on tens of thousands of causal events recursively using a novel temporal coding scheme. As a result, at inference time, our network operates in an event-wise manner that is realized with very few sum-of-the-product operations--look-up table and temporal feature aggregation--which enables processing of 1 mega or more events per second on standard CPU. In experiments using real data, we demonstrated the real-time performance and robustness of our framework.

Access to Document (DOI)

Simultaneous 3D Tracking and Reconstruction of Multiple Moving Rigid Objects

Ozawa T., Nakajima Y., Saito H.

Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019 (Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019) 2019.05

ISSN 9781728115719

© 2019 IEEE. Most SLAM works based on the assumption of static scene, so localization of the camera and mapping of the scene can fail and lose accuracy of the scene, including moving objects. This paper presents a method for simultaneous mapping of moving objects in the target scene and localization of the moving camera, based on geometrical segmentation of each temporal frame. Taking advantage of segmentation of the target scene, using only the geometric structure of the scene, our method can estimate relative pose for camera and every geometrically segmented areas even without recognizing each object. For confirming the effectiveness of the proposed method, we experimentally show that our method can estimate relative poses for all segmented areas in the scene, so that we can achieve SLAM for the scene, including multiple moving objects.

Access to Document (DOI)

Semantic Segmentation of 3D Point Cloud to Virtually Manipulate Real Living Space

Ishikawa Y., Hachiuma R., Ienaga N., Kuno W., Sugiura Y., Saito H.

Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019 (Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019) 2019.05

ISSN 9781728115719

© 2019 IEEE. This paper presents a method for the virtual manipulation of real living space using semantic segmentation of a 3D point cloud captured in the real world. We applied PointNet to segment each piece of furniture from the point cloud of a real indoor environment captured by moving a RGB-D camera. For semantic segmentation, we focused on local geometric information not used in PointNet, and we proposed a method to refine the class probability of labels attached to each point in PointNet's output. The effectiveness of our method was experimentally confirmed. We then created 3D models of real-world furniture using a point cloud with corrected labels, and we virtually manipulated real living space using Dollhouse VR, a layout system.

Access to Document (DOI)

Automatic eyeglasses replacement for a 3d virtual try-on system

Kobayashi T., Sugiura Y., Saito H., Uema Y.

ACM International Conference Proceeding Series (ACM International Conference Proceeding Series) 2019.03

ISSN 9781450365475

©2019 Association for Computing Machinery. This paper presents a 3D virtual eyeglasses try-on system for practical use. For fitting eyeglasses in a shop, consumers wish to look at themselves in a mirror while trying on various eyeglass styles. However, for people who need to wear eyeglasses for correcting problems with eyesight, it is impossible for them to clearly observe their face in the mirror without wearing eyeglasses. This makes fitting them for new eyeglasses difficult. This research proposes a virtual try-on system that can be used while wearing eyeglasses.We replace the user's eyeglasses in the input video with new eyeglasses virtually. Moreover, a fast and accurate face tracking tool enables our system to automatically display 3D virtual glasses following a user's head motion. Experimental results demonstrate that the proposed method can render virtual glasses naturally while the user is wearing real eyeglasses.

Access to Document (DOI)

Removing fences from sweep motion videos using global 3D reconstruction and fence-aware light field rendering

Lueangwattana C., Mori S., Saito H.

Computational Visual Media (Computational Visual Media) 5 ( 1 ) 21 - 32 2019.03

ISSN 20960433

© 2018, The Author(s). Diminishing the appearance of a fence in an image is a challenging research area due to the characteristics of fences (thinness, lack of texture, etc.) and the need for occluded background restoration. In this paper, we describe a fence removal method for an image sequence captured by a user making a sweep motion, in which occluded background is potentially observed. To make use of geometric and appearance information such as consecutive images, we use two well-known approaches: structure from motion and light field rendering. Results using real image sequences show that our method can stably segment fences and preserve background details for various fence and background combinations. A new video without the fence, with frame coherence, can be successfully provided.

Access to Document (DOI)

Self-assessment application of flexion and extension

Ienaga N., Sugiura Y., Saito H., Fujita K.

2019 IEEE 1st Global Conference on Life Sciences and Technologies, LifeTech 2019 (2019 IEEE 1st Global Conference on Life Sciences and Technologies, LifeTech 2019) 150 - 152 2019.03

ISSN 9781728105437

© 2019 IEEE. The measurement of range of motion during medical examination is a basic procedure in diagnosis and treatment evaluation. However, accurate measurement requires experience, and it is difficult for patients to measure range of motion by themselves. If patients can easily and accurately measure the range by their own, it will encourage patients to continue rehabilitation and will be useful for helping patients to determine when they should see a doctor. We developed a wrist flexion and extension measurement system using images captured by a smart phone so that patients can measure range of motion without expert knowledge or experience. First, the system calculates a middle line for the forearm and an outside line for the dorsal hand based on user inputs. Afterward, the angle formed by these lines is calculated. The experimental results show that the average absolute errors are 4.82° for flexion and 8.74° for extension.

Access to Document (DOI)

Volumetric Representation of Semantically Segmented Human Body Parts Using Superquadrics

Hachiuma R., Saito H.

ISSN 9783030319076

© Springer Nature Switzerland AG 2019. Superquadrics are one of the ideal shape representations for adapting various kinds of primitive shapes with a single equation. This paper revisits the task of representing a 3D human body with multiple superquadrics. As a single superquadric surface can only represent symmetric primitive shapes, we present a method that segments the human body into body parts to estimate their superquadric parameters. Moreover, we propose a novel initial parameter estimation method by using 3D skeleton joints. The results show that superquadric parameters are estimated, which represent human body parts volumetrically.

Access to Document (DOI)

Vehicle trajectory estimation method by drive recorder images and point cloud of surrounding environment

Kurobe A., Kinoshita H., Saito H.

Seimitsu Kogaku Kaishi/Journal of the Japan Society for Precision Engineering (Seimitsu Kogaku Kaishi/Journal of the Japan Society for Precision Engineering) 85 ( 3 ) 274 - 281 2019

ISSN 09120289

© 2019 Japan Society for Precision Engineering. All rights reserved. The on-the-spot inspections after accident are based on information obtained from the brake marks left on the road and the damage situation of the surroundings. However, this method is affected by the situation before the accident and cannot estimate quantitative guessing. Moreover, technologies such as automatic braking system and lane keeping system, have been developed actively. In developing these technologies, vehicle trajectory is one of the most important data. In this paper, we propose a novel method of estimating vehicle trajectory by using drive recorder images and point cloud which is measured by Mobile Mapping System and LiDAR. In the evaluation experiments, we confirmed the effectiveness of our method by comparing the vehicle poses estimated with RTK-GPS and showing the average errors are within one meter.

Access to Document (DOI)

A Thumb Tip Wearable Device Consisting of Multiple Cameras to Measure Thumb Posture

Ienaga N., Kawai W., Fujita K., Miyata N., Sugiura Y., Saito H.

ISSN 9783030210731

© 2019, Springer Nature Switzerland AG. Today, cameras have become smaller and cheaper and can be utilized in various scenes. We took advantage of that to develop a thumb tip wearable device to estimate joint angles of a thumb as measuring human finger postures is important in terms of human-computer interface and to analyze human behavior. The device we developed consists of three small cameras attached at different angles so the cameras can capture the four fingers. We assumed that the appearance of the four fingers would change depending on the joint angles of the thumb. We made a convolutional neural network learn a regression relationship between the joint angles of the thumb and the images taken by the cameras. In this paper, we captured the keypoint positions of the thumb with a USB sensor device and calculated the joint angles to construct a dataset. The root mean squared error of the test data was 6.23° and 4.75° .

Access to Document (DOI)

Semi-independent stereo visual odometry for different field of view cameras

Truong T.P., Nozick V., Saito H.

ISSN 9783030110086

© Springer Nature Switzerland AG 2019. This paper presents a pipeline for stereo visual odometry using cameras with different fields of view. It gives a proof of concept about how a constraint on the respective field of view of each camera can lead to both an accurate 3D reconstruction and a robust pose estimation. Indeed, when considering a fixed resolution, a narrow field of view has a higher angular resolution and can preserve image texture details. On the other hand, a wide field of view allows to track features over longer periods since the overlap between two successive frames is more substantial. We propose a semi-independent stereo system where each camera performs individually temporal multi-view optimization but their initial parameters are still jointly optimized in an iterative framework. Furthermore, the concept of lead and follow camera is introduced to adaptively propagate information between the cameras. We evaluate the method qualitatively on two indoor datasets, and quantitatively on a synthetic dataset to allow the comparison across different fields of view.

Access to Document (DOI)

Line-based SLAM considering prior distribution of distance and angle of line features in an urban environment

Uehara K., Saito H., Hara K.

Communications in Computer and Information Science (Communications in Computer and Information Science) 983 105 - 127 2019

ISSN 9783030122089

© Springer Nature Switzerland AG 2019. In this paper, we propose a line-based SLAM from an image sequence captured by a camera mounted on a vehicle in consideration with the prior distribution of line features that detected in an urban environments. Since such scenes captured by the vehicle in urban envirounments can be expected to include a lot of line segments detected from road markings and buildings, we employ line segments as features for our SLAM. We use additional prior regarding the line segments so that we can improve the accuracy of the SLAM. We assume that the angle of the vector of the line segments to the vehicle’s direction of travel conform to four-component Gaussian mixture distribution. We define a new cost function considering the prior distribution and optimize the relative camera pose, position, and the 3D line segments by bundle adjustment. The prior distribution is also extended into 2D, the distance and angle of the line segments. In addition, we make digital maps from the detected line segments. Our method increases the accuracy of localization and corrects tilted lines in the digital maps. We apply our method to both the single-camera system and the multi-camera system for demonstrate the accuracy improvement by the prior distribution of distance and angle of line features.

Access to Document (DOI)

Learning an optimisable semantic segmentation map with image conditioned variational autoencoder

Zhuang P., Sekikawa Y., Hara K., Saito H.

ISSN 9783030306441

© Springer Nature Switzerland AG 2019. Recent semantic segmentation systems have achieved significant improvement by performing pixel-wise training with hierarchical features using deep convolutional neural network models. While the learning process usually requires pixel-level annotated images, it is difficult to get desirable amounts of fine-labeled data and thus the training set size is more likely to be limited, often in thousands. This means that top methods for a dataset can be fine-tuned for a specific situation, making the generalization ability unclear. In real-world applications like self-driving systems, ambiguous region or lack of context information can cause errors in the predicted results. Resolving such ambiguities is crucial for subsequent operations to be performed safely. We are inspired by work from CodeSLAM where optimizable pixel-wise depth representation is learned. We modify the regression method to work on the pixel-wise classification problem. By training a variational auto-encoder network conditioned with a color image, the computed latent space works as a low-dimensional representation of semantic segmentation, which can be efficiently optimized. As a consequence, our model can correct the error or ambiguity of the prediction during the inference phase given useful scene information. We show how this approach works by giving partial scene truth and perform optimization on the latent variable.

Access to Document (DOI)

Image-based center of mass estimation of the human body via 3D shape and kinematic structure

Kaichi T., Mori S., Saito H., Takahashi K., Mikami D., Isogawa M., Kusachi Y.

Sports Engineering (Sports Engineering) 22 ( 3-4 ) 2019

ISSN 13697072

© 2019, International Sports Engineering Association. This paper presents a method to estimate a time-sequential trajectory of the center of mass (CoM) of an athlete from a multi-view set of cameras. Collecting the CoM typically requires large-scale measuring systems or attaching sensors to the athletes. To mitigate such hardware limitations, the present study takes a multi-view video-based approach. The proposed method reconstructs subjects’ voxels from a set of multi-view frames and weights each voxel with body part-dependent weights to calculate a CoM. Our results, using real data measured in a studio, showed that the proposed method can estimate CoM within 20 mm concerning center of pressure measures.

Access to Document (DOI)

Efficient object-oriented semantic mapping with object detector

Nakajima Y., Saito H.

IEEE Access (IEEE Access) 7 3206 - 3213 2019

© 2018 IEEE. Incrementally, building a 3D map in which object instances are semantically annotated has a wide range of applications, including scene understanding, human-robot interactions, and simultaneous localization and mapping extensions. Although researchers are developing efficient and accurate systems, these methods still face a critical issue: Real-time processing, because the task requires a series of heavy processing components, e.g., camera pose estimation, 3D map reconstruction, and especially recognition. In this paper, we propose a novel object-oriented semantic mapping approach aiming at overcoming such issues by introducing highly accurate object-oriented semantic scene reconstruction in real time. For high efficiency, the proposed method employs a fast and scalable object detection algorithm for exploiting semantic information from the incoming frames. These outputs are integrated into geometric regions of the 3D map, which are carried by the geometric-based incremental segmentation method. The strategy of assigning class probabilities to each segmented region, not each element (e.g., surfels and voxels), notably reduces the computational cost, as well as the memory footprint. In addition to efficiency, by geometrically segmenting the 3D map first, clear boundaries between objects appear. We complementarily improve the geometric-based segmentation results beyond the geometric only to the semantic-aware representation. We validate the proposed method's accuracy and computational efficiency through experiments in a common office scene.

Access to Document (DOI)

Fast and Accurate Semantic Mapping through Geometric-based Incremental Segmentation

Nakajima Y., Tateno K., Tombari F., Saito H.

IEEE International Conference on Intelligent Robots and Systems (IEEE International Conference on Intelligent Robots and Systems) 385 - 392 2018.12

ISSN 9781538680940

© 2018 IEEE. We propose an efficient and scalable method for incrementally building a dense, semantically annotated 3D map in real-time. The proposed method assigns class probabilities to each region, not each element (e.g., surfel and voxel), of the 3D map which is built up through a robust SLAM framework and incrementally segmented with a geometric-based segmentation method. Differently from all other approaches, our method has a capability of running at over 30Hz while performing all processing components, including SLAM, segmentation, 2D recognition, and updating class probabilities of each segmentation label at every incoming frame, thanks to the high efficiency that characterizes the computationally intensive stages of our framework. By utilizing a specifically designed CNN to improve the frame-wise segmentation result, we can also achieve high accuracy. We validate our method on the NYUv2 dataset by comparing with the state of the art in terms of accuracy and computational efficiency, and by means of an analysis in terms of time and space complexity.

Access to Document (DOI)

Estimation of center of mass for sports scene using weighted visual hull

Kaichi T., Mori S., Saito H., Takahashi K., Mikami D., Isogawa M., Kimata H.

IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops) 2018-June 1890 - 1896 2018.12

ISSN 9781538661000

© 2018 IEEE. This paper presents a method to estimate the 3D position of a center of mass (CoM) of a human body from a set of multi-view images. As a well-known fact, in sports, collections of CoM are important for analyzing the athletes' performance. Most conventional studies in CoM estimation require installing a measuring system (e.g., a force plate or optical motion capture system) or attaching sensors to the athlete. While such systems reliably estimate CoM, casual settings are preferable for simplifying preparations. To address this issue, the proposed method takes a vision-based approach that does not require specialized hardware and wearable devices. Our method calculates subject's CoM using voxels with body parts dependent weighting. This individual voxel reconstruction and voxel-wise weighting reflects the differences in each body shape, and are expected to contribute to higher performance in analysis. The results using real data demonstrated the performance of the proposed method were compared to force plate data, and provided a 3D CoM visualization in a dynamic scene.

Access to Document (DOI)

FusionMLS: Highly dynamic 3D reconstruction with consumer-grade RGB-D cameras

Meerits S., Thomas D., Nozick V., Saito H.

Computational Visual Media (Computational Visual Media) 4 ( 4 ) 287 - 303 2018.12

ISSN 20960433

© 2018, The Author(s). Multi-view dynamic three-dimensional reconstruction has typically required the use of custom shutter-synchronized camera rigs in order to capture scenes containing rapid movements or complex topology changes. In this paper, we demonstrate that multiple unsynchronized low-cost RGB-D cameras can be used for the same purpose. To alleviate issues caused by unsynchronized shutters, we propose a novel depth frame interpolation technique that allows synchronized data capture from highly dynamic 3D scenes. To manage the resulting huge number of input depth images, we also introduce an efficient moving least squares-based volumetric reconstruction method that generates triangle meshes of the scene. Our approach does not store the reconstruction volume in memory, making it memory-efficient and scalable to large scenes. Our implementation is completely GPU based and works in real time. The results shown herein, obtained with real data, demonstrate the effectiveness of our proposed method and its advantages compared to state-of-the-art approaches.

Access to Document (DOI)

Estimation of runners’ number of steps, stride length and speed transition from video of a 100-meter race

Yagi K., Hasegawa K., Sugiura Y., Saito H.

MMSports 2018 - Proceedings of the 1st International Workshop on Multimedia Content Analysis in Sports, Co-located with MM 2018 (MMSports 2018 - Proceedings of the 1st International Workshop on Multimedia Content Analysis in Sports, Co-located with MM 2018) 87 - 95 2018.10

ISSN 9781450359818

© 2018 Copyright held by the owner/author(s). Publication rights licensed to ACM. The purpose of this study is sensing movements of 100-m runners from video that is publicly available, for example, Internet broadcasts. Normally, information that can be obtained from a video is limited to the number of steps and average stride length. However, our proposed method makes it possible to measure not only this information, but also time-scale information like every stride length and speed transition from the same input. Our proposed method can be divided into three steps. First, we generate a panoramic image of the 100-m track. By doing this, we can estimate where the runners are running in a frame at the 100-meter scale. Second, we detect whether the runner steps in the frame. For this process, we utilize the detected track lines and leg joint positions of runners. Finally, we project every steps to the overview image of the 100-m track to estimate the stride length at the 100-m scale. In the experiment part, we apply our method to various race videos. We evaluate the accuracy of our method via comparison with the data measured using typical methods. In addition, we evaluate the accuracy of estimation of the number of steps and show visualized runners’ steps and speed transitions.

Access to Document (DOI)

1 st ACM international workshop on multimedia content analysis in sports

Lienhart R., Moeslund T., Saito H.

MM 2018 - Proceedings of the 2018 ACM Multimedia Conference (MM 2018 - Proceedings of the 2018 ACM Multimedia Conference) 2125 - 2126 2018.10

ISSN 9781450356657

© 2018 Association for Computing Machinery. The first ACM International Workshop on Multimedia Content Analysis in Sports (ACM MMSports'18) is held in Seoul, South Korea on October 26th, 2018 and is co-located with the ACM International Conference on Multimedia 2018 (ACM Multimedia 2018). The goal of this workshop is to bring together researchers and practitioners from academia and industry to address challenges and report progress in mining and content analysis of multimedia/multimodal data in sports. The combination of sports and modern technology offers a novel and intriguing field of research with promising approaches for visual broadcast augmentation, understanding, statistical analysis and evaluation, and sensor fusion. There is a lack of research communities focusing on the fusion of multiple modalities. We are helping to close this research gap with this first workshop of a serious workshops on multimedia content analysis in sports.

Access to Document (DOI)

Constant velocity 3D convolution

Sekikawa Y., Ishikawa K., Hara K., Yoshida Y., Suzuki K., Sato I., Saito H.

Proceedings - 2018 International Conference on 3D Vision, 3DV 2018 (Proceedings - 2018 International Conference on 3D Vision, 3DV 2018) 343 - 351 2018.10

ISSN 9781538684252

© 2018 IEEE. We propose a novel three-dimensional (3D)-convolution method, cv3dconv, for detecting spatiotemporal features from videos. It reduces the number of sum-of-products of 3D convolution by thousands of times by assuming the constant moving velocity of the camera. We observed that a specific class of video sequences, such as those captured by an in-vehicle camera, can be well approximated with piece-wise linear movements of 2D features in the temporal dimension. Our principal finding is that the 3D kernel, represented by the constant-velocity, can be decomposed into a convolution of a 2D kernel representing the shapes and a 3D kernel representing the velocity. We derived the efficient recursive algorithm for this class of 3D convolution which is exceptionally suited for sparse data, and this parameterized decomposed representation imposes a structured regularization along the temporal direction. We experimentally verified the validity of our approximation using a controlled dataset, and we also showed the effectiveness of cv3dconv for the visual odometry estimation task using real event camera data captured in urban road scene.

Access to Document (DOI)

3D PixMix: Image Inpainting in 3D Environments

Mori S., Herling J., Broll W., Kawai N., Saito H., Schmalstieg D., Kalkofen D.

Adjunct Proceedings - 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018 (Adjunct Proceedings - 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018) 1 - 2 2018.07

ISSN 9781538675922

© 2018 IEEE. State of the art methods for diminished reality (DR) propagate pixel information from keyframes to later frames to achieve real-time image inpainting in 3D spaces. However, this approach assumes a planar scene and produces artifacts, if the scene geometry is not sufficiently planar. In this paper, we present 3D PixMix, a new real-time inpainting method that addresses non-planar scenes by considering both color and depth information in the inpainting process. We define cost functions for both the color and the geometric appearance in the inpainting scheme and use an RGB-D sensor for depth fusion using a SLAM. Comparison results against the conventional PixMix show that 3D PixMix obtains the equivalent or even better quality in 3D scenes with additional depth information.

Access to Document (DOI)

Superimposing thermal-infrared data on 3D structure reconstructed by RGB visual odometry

Yamaguchi M., Truong T.P., Mori S., Nozick V., Saito H., Yachida S., Sato H.

IEICE Transactions on Information and Systems (IEICE Transactions on Information and Systems) E101D ( 5 ) 1296 - 1307 2018.05

ISSN 09168532

© 2018 The Institute of Electronics, Information and Communication Engineers. In this paper, we propose a method to generate a threedimensional (3D) thermal map and RGB + thermal (RGB-T) images of a scene from thermal-infrared and RGB images. The scene images are acquired by moving both a RGB camera and an thermal-infrared camera mounted on a stereo rig. Before capturing the scene with those cameras, we estimate their respective intrinsic parameters and their relative pose. Then, we reconstruct the 3D structures of the scene by using Direct Sparse Odometry (DSO) using the RGB images. In order to superimpose thermal information onto each point generated from DSO, we propose a method for estimating the scale of the point cloud corresponding to the extrinsic parameters between both cameras by matching depth images recovered from the RGB camera and the thermal-infrared camera based on mutual information. We also generate RGB-T images using the 3D structure of the scene and Delaunay triangulation. We do not rely on depth cameras and, therefore, our technique is not limited to scenes within the measurement range of the depth cameras. To demonstrate this technique, we generate 3D thermal maps and RGB-T images for both indoor and outdoor scenes.

Access to Document (DOI)

Line-based SLAM using non-overlapping cameras in an urban environment

Kawasaki A., Hara K., Saito H.

IEICE Transactions on Information and Systems (IEICE Transactions on Information and Systems) E101D ( 5 ) 1232 - 1242 2018.05

ISSN 09168532

© 2018 The Institute of Electronics, Information and Communication Engineers. We propose a method of line-based Simultaneous Localization and Mapping (SLAM) using non-overlapping multiple cameras for vehicles running in an urban environment. It uses corresponding line segments between images taken by different frames and different cameras. The contribution is a novel line segment matching algorithm by warping processing based on urban structures. This idea significantly improves the accuracy of line segment matching when viewing direction are very different, so that a number of correspondences between front-view and rear-view cameras can be found and the accuracy of SLAM can be improved. Additionally, to enhance the accuracy of SLAM we apply a geometrical constraint of urban area for initial estimation of 3D mapping of line segments and optimization by bundle adjustment. We can further improve the accuracy of SLAM by combining points and lines. The position error is stable within 1.5m for the entire image dataset evaluated in this paper. The estimation accuracy of our method is as high as that of ground truth captured by RTK-GPS. Our high accuracy SLAM algorithm can be apply for generating a road map represented by line segments. According to an evaluation of our generating map, true positive rate around the vehicle exceeding 70% is achieved.

Access to Document (DOI)

Simultaneous object segmentation and recognition by merging CNN outputs from uniformly distributed multiple viewpoints

Nakajima Y., Saito H.

IEICE Transactions on Information and Systems (IEICE Transactions on Information and Systems) E101D ( 5 ) 1308 - 1316 2018.05

ISSN 09168532

© 2018 The Institute of Electronics, Information and Communication Engineers. We propose a novel object recognition system that is able to (i) work in real-time while reconstructing segmented 3D maps and simultaneously recognize objects in a scene, (ii) manage various kinds of objects, including those with smooth surfaces and those with a large number of categories, utilizing a CNN for feature extraction, and (iii) maintain high accuracy no matter how the camera moves by distributing the viewpoints for each object uniformly and aggregating recognition results from each distributed viewpoint as the same weight. Through experiments, the advantages of our system with respect to current state-of-the-art object recognition approaches are demonstrated on the UW RGB-D Dataset and Scenes and on our own scenes prepared to verify the effectiveness of the Viewpoint-Class-based approach.

Access to Document (DOI)

Synchronizing 3D point cloud from 3D scene flow estimation with 3D Lidar and RGB camera

Hiroki Usami, Hideo Saito, Jun Kawai and Noriko Itani

IS&T International Symposium on Electronic Imaging 2018: 3D Image Processing, Measurement (3DIPM), and Applications 2018 proceedings. (IS and T International Symposium on Electronic Imaging Science and Technology) 1 - 6 2018.01

Research paper (international conference proceedings), Joint Work, Accepted

We present a method for synchronizing three-dimensional (3D)
point cloud from 3D scene with estimation using a 3D Lidar and an
RGB camera. These 3D points sensed by the 3D Lidar are not
captured at the same time, which makes it difficult to measure the
correct shape of the object in a dynamic scene. In our method, we
generate synchronized 3D points at arbitrary times using linear
interpolation in four-dimensional space, time and space. For
interpolating the 3D point, we obtain corresponding 3D point
matching with the pixel value captured by the RGB camera in a
continuous frame. The experimental results demonstrate the
effectiveness of the presented method by depicting a synchronized
3D point cloud that is correctly shaped.

Access to Document (DOI)

Vehicle pose estimation from drive recorder images by monocular SLAM and matching with rendered 3D point cloud of surrounding environment

Kurobe A., Kinoshita H., Saito H.

IS and T International Symposium on Electronic Imaging Science and Technology (IS and T International Symposium on Electronic Imaging Science and Technology) 2371 - 2376 2018

© 2018, Society for Imaging Science and Technology. Vehicle pose estimation is a vital technology for reconstructing the circumstaces of traffic accidents. We propose a novel method for reconstructing the trajectory of vehicles from drive recorder images and a point cloud around the road. First, we apply ORB-SLAM to image sequence of the drive recorder for obtaining the vehicle pose trajectory; however this is based on relative coordinates and a relative scale. For estimating the absolute coordinates and scale of the trajectory, which cannot be obtained from a monocular SLAM like ORB-SLAM, we match the feature points detected in the image sequence with the three-dimensional (3D) point cloud of surrounding environment. For finding 3D points matching the feature points, we generate candidate images by the rendering 3D point cloud of the surrounding environment using the position initially estimated by the Global Positioning System (GPS). Next, we match to obtain the 3D two-dimensional (2D) generated images and drive recorder image to get 3D-2D point correspondences between the 3D point cloud and the drive recorder images; thus, we can convert the relative estimation of the camera pose by ORB-SLAM to the coordinates of the 3D point cloud of the surrounding environment. In the evaluation experiments, we confirmed the effectiveness of our method by comparing the vehicle poses estimated by our method, with those of RTKGPS, which exhibits high measurement precision.

Access to Document (DOI)

Synthesis of wide-FOV RGB-D images by registration and upsampling of 3D Lidar with omnidirectional RGB camera

Usami H., Miyata S., Saito H.

23rd International Display Workshops in conjunction with Asia Display, IDW/AD 2016 (23rd International Display Workshops in conjunction with Asia Display, IDW/AD 2016) 4 2340 - 2342 2018

ISSN 9781510845510

Multi-view surface inspection using a rotating table

Kaichi T., Mori S., Saito H., Sugano J., Adachi H.

IS and T International Symposium on Electronic Imaging Science and Technology (IS and T International Symposium on Electronic Imaging Science and Technology) 2021 - 2026 2018

© 2018, Society for Imaging Science and Technology. In this paper, we introduce a method to visually inspect in- dustrial parts by using multi-view images of an industrial part on a rotating table. During the visual inspection of the parts' sur- faces, the relationship between the camera pose, the light vectors, and the normal vectors of a part's surface is the key factor in de- tecting abnormalities. We can change the relationship between the three factors by rotating the part; then, the abnormalities are visible in certain positional relationships. We, therefore, track the points on a part's surfaces in an image sequence and discrimi- nate abnormal points (such as points on scratches or dents) from normal points based on pixel value transitions while rotating the part. The experimental results, which were based on real data of industrial parts, showed that the proposed method could detect the abnormalities on the surfaces.

Access to Document (DOI)

Constant velocity 3d convolution

Sekikawa Y., Ishikawa K., Saito H.

IEEE Access (IEEE Access) 6 76490 - 76501 2018

© 2018 IEEE. We propose a novel 3-D convolution method, cv3dconv, for extracting spatiotemporal features from videos. It reduces the number of sum-of-products operations in 3-D convolution by thousands of times by assuming the constant moving velocity of the features. We observed that a specific class of video sequences, such as video captured by an in-vehicle camera, can be well approximated with piece-wise linear movements of 2-D features in a temporal dimension. Our principal finding is that a 3-D kernel, represented by constant velocity, can be decomposed into a convolution of a 2-D-shaped kernel and a 3-D-velocity kernel, which is parameterized using only two parameters. We derived an efficient recursive algorithm for this class of 3-D convolution, which is exceptionally suited for sparse spatiotemporal data, and this parameterized decomposed representation imposes a structured regularization along a temporal direction. We experimentally verified the validity of our approximation using a controlled dataset, and we also showed the effectiveness of the cv3dconv by adopting it for deep neural networks (DNNs) in visual odometry estimation task using publicly available event-based camera dataset captured in urban road scene. Our DNN architecture improves the estimation accuracy for about 30% compared with the existing states-of-the-arts architecture designed for event data.

Access to Document (DOI)

An overview of augmented visualization: Observing the real world as desired

Mori S., Saito H.

APSIPA Transactions on Signal and Information Processing (APSIPA Transactions on Signal and Information Processing) 7 2018

Copyright © The Authors, 2018 This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.. Over 20 years have passed since a free-viewpoint video technology has been proposed with which a user's viewpoint can be freely set up in a reconstructed three-dimensional space of a target scene photographed by multi-view cameras. This technology allows us to capture and reproduce the real world as recorded. Once we capture the world in a digital form, we can modify it as augmented reality (i.e., placing virtual objects in the digitized real world). Unlike this concept, the augmented world allows us to see through real objects by synthesizing the backgrounds that cannot be observed in our raw perspective directly. The key idea is to generate the background image using multi-view cameras, observing the backgrounds at different positions and seamlessly overlaying the recovered image in our digitized perspective. In this paper, we review such desired view-generation techniques from the perspective of free-view point image generation and discuss challenges and open problems through a case study of our implementations.

Access to Document (DOI)

A simple 3D scanning system of human foot using a smartphone with a depth camera

Kobayashi T., Ienaga N., Sugiura Y., Saito H., Miyata N., Tada M.

Seimitsu Kogaku Kaishi/Journal of the Japan Society for Precision Engineering (Seimitsu Kogaku Kaishi/Journal of the Japan Society for Precision Engineering) 84 ( 12 ) 996 - 1002 2018

ISSN 09120289

© 2018 Japan Society for Precision Engineering. All rights reserved. This paper presents a method to measure foot shape from a 3D point cloud as input captured from multiple directions using a smartphone depth camera. Such a 3D point cloud could potentially include noise or omit parts of the foot due to occlusion. To deal with this occlusion problem, we propose to use a dataset of 3D foot shapes collected by a precise 3D shape scanner of foot shapes. According to the dataset of 3D foot shapes, we can generate a deformable model by performing a principal component analysis (PCA) on the dataset. Then we minimize the error of the shape represented by the deformable model and the 3D point cloud acquired by the smartphone camera, to recover a complete 3D shape of the entire foot with high accuracy. We test this method by comparing the 3D shape produced by our proposed method to the 3D shape precisely measured by the 3D scanner. Our proposed method can scan the foot shape with an error of about 1,13mm.

Access to Document (DOI)

A refocus-interface for diminished reality work area visualization

Maezawa M., Mori S., Saito H.

IS and T International Symposium on Electronic Imaging Science and Technology (IS and T International Symposium on Electronic Imaging Science and Technology) 1121 - 1124 2018

© 2018, Society for Imaging Science and Technology. In this paper, we present a refocus interface to set the parameters used for diminished reality (DR)-based work area visualization and a multiview camera-based rendering scheme. The refocus interface allows the user to determine two planes - one for setting a virtual window, through which the user can observe the background occluded by an object, and the other for a background plane, which is used for the subsequent background rendering. The background is rendered considering the geometric and appearance relationships of the multiview cameras observing the scene. Our preliminary results demonstrate that our DR system can visualize the hidden background.

Access to Document (DOI)

Synthesis of top view image and the detection of obstacles using multiple cameras for monitoring rea around a truck

Uehara K., Saito H., Yamamoto K., Sato H.

23rd International Display Workshops in conjunction with Asia Display, IDW/AD 2016 (23rd International Display Workshops in conjunction with Asia Display, IDW/AD 2016) 4 2304 - 2307 2018

ISSN 9781510845510

Augmented Visualization: Observing as Desired

Shohei Mori and Hideo Saito

Proceedings of 2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) (Proceedings - 9th Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2017) 2018-February 351 - 356 2017.12

Research paper (international conference proceedings), Accepted, ISSN 9781538615423

Over 20 years have passed since a free-viewpoint video technology has been proposed in which a user's viewpoint can be freely set up reconstructed 3D space of a target scene photographed by multi-view cameras. This technology allows us to capture and reproduce the real world as it recorded. Once we capture the world in a digital form, we can modify the world as augmented reality does (i.e., placing virtual objects in the digitized real world). As oppose to this concept, the augmented world also allows us to see through real objects by synthesizing the backgrounds that cannot be observed in our raw perspective directly. The key idea is to generate the background image using multi-view cameras observing the backgrounds at different positions, and seamlessly overlaying the recovered image in our digitized perspective. In this article, our studies regarding such desired view generation techniques are reviewed, and its future directions and open problems are discussed.

Access to Document (DOI)

Visual Inspection by Capturing a Rotating Industrial Part

Tomoya KAICHI, Shohei MORI, Hideo SAITO, Junichi SUGANO, Hideyuki ADACHI

Journal of the Japan Society for Precision Engineering 83 ( 12 ) 1184 - 1191 2017.12

Research paper (scientific journal), Accepted

Visual inspection is a vital step for maintaining the quality of industrial parts. In visual inspection, the relation between the three factors, camera pose, lights direction and normal vector of a part is very important to detect abnormalities on industrial part. We propose a visual inspection method for industrial parts using an image sequence captured while rotating an industrial part using a rotating table. By rotating a part, we can easily change the relation between three factors. We track the points on part's surfaces in an image sequence. We discriminate abnormal surface points such as scratches from the normal ones based on pixel value transitions in an image sequence. For accurately tracking the points on the part surface, we propose a novel camera calibration method of the rotating table and the camera with a telecentric lens and a pose estimation method for the rotating part. We use angular invariant feature vectors for improving robustness against types and shapes of industrial parts to be inspected. We presented experimental results using real data of industrial parts and verified that the proposed method could detect scratches on surfaces.

Access to Document (DOI)

Line-Based SLAM Considering Directional Distribution of Line Features in an Urban Environment

Kei Uehara, Hideo Saito, Kosuke Hara

The journal of the Institute of Image Electronics Engineers of Japan : visual computing, devices & communications 46 ( 4 ) 522 - 532 2017.12

Research paper (scientific journal), Joint Work, Accepted

本論文では，線分特徴の事前分布を考慮した自動運転車のためのSLAM (Simultaneous Localization and Mapping) を提案する．提案手法は，道路標示や建物など，線分を検出しやすい物体が存在する都市環境を走行する車両に搭載されたカメラにより撮影される動画像を入力とすることを前提とし，この動画像から検出される線分特徴の方向に関する制約を加えることによりSLAM の精度を向上させるものである．線分特徴の方向に関する制約を加えるため，本手法では道路標示中の線分が車両進行方向に対して平行，もしくは垂直であるものが多いことに着目し，それら2 つの線分間の角度分布が混合ガウス分布に則ると仮定し，バンドル調整による最適化に導入する．これにより，カメラの位置姿勢と線分特徴の位置推定の精度を高めることが期待できる．本手法の有効性を確かめるため，カメラを1 つ用いたシングルカメラシステムと，2 つ用いたマルチカメラシステムの両方に提案手法を適用し精度検証を行った．その結果マルチカメラによる従来手法の精度と，シングルカメラによる本手法の精度が同等となった．また提案手法により生成した路面地図は，線分の方向に関する制約を導入したことにより補正され，精度向上を確認した．

Real-time scene reconstruction and triangle mesh generation using multiple RGB-D cameras

Meerits Siim, Nozick Vincent, Saito Hideo

Journal of Real-Time Image Processing (Springer) 16 ( 6 ) 1 - 13 2017.11

Research paper (scientific journal), ISSN 1861-8200

We present a novel 3D reconstruction system that can generate a stable triangle mesh using data from multiple RGB-D sensors in real time for dynamic scenes. The first part of the system uses moving least squares (MLS) point set surfaces to smooth and filter point clouds acquired from RGB-D sensors. The second part of the system generates triangle meshes from point clouds. The whole pipeline is executed on the GPU and is tailored to scale linearly with the size of the input data. Our contributions include changes to the MLS method for improving meshing, a fast triangle mesh generation method and GPU implementations of all parts of the pipeline.

Access to Document (DOI)

Semantic Object Selection and Detection for Diminished Reality Based on SLAM with Viewpoint Class

Nakajima Yoshikatsu, Mori Shohei, Saito Hideo

2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct) 338 - 343 2017.10

Research paper (international conference proceedings), Accepted

We propose a novel diminished reality method which is able to (i) automatically recognize the region to be diminished, (ii) work with a single RGB-D sensor, and (iii) work without pre-processing to generate a 3D model of the target scene by utilizing SLAM, segmentation, and recognition framework. Especially, regarding the recognition of the area to be diminished, our method is able to maintain high accuracy no matter how the camera moves by distributing the viewpoints for each object uniformly and aggregating recognition results from each distributed viewpoint as the same weight. These advantages are demonstrated on the UW RGB-D Dataset and Scenes.

Access to Document (DOI)

An Instant See-Through Vision System Using a Wide Field-of-View Camera and a 3D-Lidar

Oishi Kei, Mori Shohei, Saito Hideo

2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct) 344 - 347 2017.10

Research paper (international conference proceedings)

Diminished reality (DR) enables us to see through real objects occluding some areas in our field of view. This interactive display has various applications, such as see-through vision to visualize invisible areas, work area visualization in surgery and landscape simulation. In this paper, we propose two underlying problems in see-through vision, in which hidden areas are observed in real time. First, see-through vision methods require a common area to calibrate every camera in the environment. However, the field of view is limited and many approaches rely on a time-consuming calibration, sensors, or fiducial markers. Second, see-through vision applications assume that the background is planar to ease image alignment. We therefore present a place-and-play see-through vision system using a wide field-of-view RGB-D camera. We validated the accuracy and the robustness of our system and showed results in various environments to show the applicability.

Access to Document (DOI)

Diminished Reality for Privacy Protection by Hiding Pedestrians in Motion Image Sequences Using Structure from Motion

Yagi Kentaro, Hasegawa Kunihiro, Saito Hideo

2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct) 334 - 337 2017.10

Research paper (international conference proceedings), Accepted

We present a method for generating images in which people are hidden from image sequences taken with a hand-held camera. Our method is basically used for privacy protection of people whose images are unintentionally captured in image sequences. We hide people from images by reconstructing a 3D model of background and projecting it to 2D images. By detecting the area in which people are present beforehand, we can reconstruct a 3D model of the background without people. In the experiment, We compare our method with some conventional approaches for diminished reality.

Access to Document (DOI)

See-Through Vision Using a Fish-Eye Camera and a 3D-LIDAR

Kei Oishi, Shohei Mori, Hideo Saito

Journal of the Institute of Image Information and Television Engineers 71 ( 11 ) J276 - J279 2017.10

Research paper (scientific journal), Joint Work, Accepted, ISSN 1342-6907

現実世界から実物体を消し去る隠消現実感 (Diminished Reality; DR) 技術は，見通しの悪い交差点におけるSee-Through Visionなど，さまざま応用例が考えられている．しかし，そこにはいくつかの課題が存在した．第一に隠背景を観測するカメラと，体験者のカメラの直接的なカメラの位置関係を取得することが困難であること．第二に，屋外において隠背景が平面で近似できる場面でしか応用できなかったことである．本稿では，魚眼カメラと3D-LIDARを組み合わせた，高画角のRGB-Dカメラを用いて，両者のカメラの共通の視野を獲得することにより上記の問題解決を図る．また，屋外実験を通して，提案手法によるSee-Through Visionの実現可能性について議論する．

Access to Document (DOI)

3D reconstruction of grasp posture using an embedded camera

Kashiwagi Naoaki, Sugiura Yuta, Miyata Natsuki, Tada Mitsunori, Sugimoto Maki, Saito Hideo

Journal of the Institute of Image Information and Television Engineers 71 ( 11 ) J283 - J286 2017.10

Research paper (scientific journal), Joint Work, Accepted, ISSN 1342-6907

本研究では，物体に対する人間の把持姿勢を3D復元するシステムを提案する．ペットボトルやビンといった筒状の物体の蓋の上面に魚眼レンズを装着したカメラを付与し，肌色抽出やセグメンテーションといった画像処理を加えて指を認識する．取得した指位置をディジタルハンドモデルと組合せることで手全体の位置姿勢を復元する．本システムは，日用品の利用状況の計測に基づく製品改良や，スポーツのトレーニングに活用できる．

Access to Document (DOI)

Registration of RGB and thermal point clouds generated by structure from motion

Trong Phuc Truong, Masahiro Yamaguchi, Shohei Mori, Vincent Nozick, and Hideo Saito

2017 IEEE International Conference on Computer Vision Workshops (ICCVW) 419 - 427 2017.10

Research paper (international conference proceedings), Joint Work, Accepted

Thermal imaging has become a valuable tool in various
fields for remote sensing and can provide relevant information
to perform object recognition or classification.
In this paper, we present an automated method to obtain a
3D model fusing data from a visible and a thermal camera.
The RGB and thermal point clouds are generated independently
by structure from motion. The registration process
includes a normalization of the point cloud scale, a global
registration based on calibration data and the output of the
structure from motion, and a fine registration employing a
variant of the Iterative Closest Point optimization. Experimental
results demonstrate the accuracy and robustness of
the overall process.

Access to Document (DOI)

A Work Area Visualization by Multi-View Camera-Based Diminished Reality

Shohei Mori, Momoko Maezawa and Hideo Saito

Multimodal Technologies Interaction 1 ( 3 ) 2017.09

Research paper (scientific journal), Joint Work

Hand-held tools are indispensable for efficient manual working in fields ranging from
woodworking to surgery. In this paper, we present a diminished reality (DR) method to visualize areas
occluded by hands and tools in various hand-working scenarios. We propose a redesigned existing
arbitrary viewpoint image generation method for DR applications as a core DR background rendering
to recover views without undesirable objects. We conducted quantitative and qualitative experiments
using real data to validate the performance of the proposed method. The experimental results showed
that our method runs in real-time (40.1 fps) and surpasses conventional methods (including image
inpainting-based and geometry-based approaches) in terms of image similarity measures.

Access to Document (DOI)

Structure Based Matching between Aerial and Map Images using Brightness- and Rotation-Invariant Curve Features

Yoshikatsu Nakajima, Hideo Saito

Proceedings of the Irish Machine Vision and Image Processing Conference (IMVIP2017) 233 - 236 2017.08

Research paper (international conference proceedings), Accepted

Since conventional feature descriptor algorithms depend on brightness values, those algorithms cannot
obtain point correspondences between aerial and map images where brightness values do not correspond but
geometric structures do correspond. The research in this paper focused on obtaining point correspondences
between those images. This study proposes a novel method by which curve features, edge shape, and
road extraction by a CNN, independent on brightness value, produce robustness in image matching. Its
effectiveness was confirmed through experiments on Massachusetts Roads Dataset.

輝度値変化及び回転に頑健な曲線特徴量

SAITO HIDEO

電子情報通信学会論文誌D J100-D ( 12 ) 978 - ９８１ 2017.08

Research paper (scientific journal), Accepted

Access to Document (DOI)

Ball 3D Trajectory Reconstruction without Preliminary Temporal and Geometrical Camera Calibration

Miyata Shogo, Saito Hideo, Takahashi Kosuke, Mikami Dan, Isogawa Mariko, Kimata Hideaki

2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) 2017-July 164 - 169 2017.08

Research paper (international conference proceedings), Joint Work, Accepted

This paper proposes a method for reconstructing 3D ball trajectories by using multiple temporally and geometrically uncalibrated cameras. To use cameras to measure the trajectory of a fast-moving object, such as a ball thrown by a pitcher, the cameras must be temporally synchronized and their position and orientation should be calibrated. In some cases, these conditions cannot be met, e.g., one cannot geometrically calibrate cameras when one cannot step into a baseball stadium. The basic idea of the proposed method is to use a ball captured by multiple cameras as a corresponding point. The method first detects a ball. Then, it estimates temporal difference between cameras. After that, the ball positions are used as corresponding points for geometrically calibrating the cameras. Experiments using actual pitching videos verify the effectiveness of our method.

Access to Document (DOI)

Extrinsic Camera Calibration Without Visible Corresponding Points Using Omnidirectional Cameras

Miyata Shogo, Saito Hideo, Takahashi Kosuke, Mikami Dan, Isogawa Mariko, Kojima Akira

IEEE Transactions on Circuits and Systems for Video Technology (IEEE Transactions on Circuits and Systems for Video Technology) 28 ( 9 ) 2210 - 2219 2017.07

Research paper (scientific journal), Joint Work, Accepted, ISSN 1051-8215

This paper proposes a novel algorithm that calibrates multiple cameras scattered across a broad area. The key idea of the proposed method is &#x201C;using the position of an omnidirectional camera as a reference point.&#x201D; The common approach to calibrating multiple cameras assumes that the cameras capture at least some common points. This means calibration becomes quite difficult if there are no shared points in each camera&#x2019;s field of view (FOV). The proposed method uses the position of an omnidirectional camera to determine point correspondence. The position of an omnidirectional camera relative to the calibrated camera is estimated by the theory of epipolar geometry, even if the omnidirectional camera is placed outside the camera&#x2019;s FOV. This property makes our method applicable to multiple cameras scattered across a broad area. Qualitative and quantitative evaluations using synthesized and real data, e.g., a sports field, demonstrate the advantages of the proposed method.

Access to Document (DOI)

Removing Redundant Line Segments of 3D Model for Camera Pose Estimation

Naoto Ienaga, Siim Meerits, Shohei Mori, Hideo Saito

映像情報メディア学会誌 71 ( 6 ) J189 - J191 2017.06

Research paper (scientific journal), Joint Work, Accepted

Access to Document (DOI)

A survey of diminished reality: Techniques for visually concealing, eliminating, and seeing through real objects

Shohei Mori, Sei Ikeda and Hideo Saito

IPSJ Transactions on Computer Vision and Applications (Springer) 9 ( 17 ) 1 - 14 2017.06

Research paper (scientific journal)

In this paper, we review diminished reality (DR) studies that visually remove, hide, and see through real objects from
the real world. We systematically analyze and classify publications and present a technology map as a reference for
future research. We also discuss future directions, including multimodal diminished reality. We believe that this paper
will be useful mainly for students who are interested in DR, beginning DR researchers, and teachers who introduce DR
in their classes.

Access to Document (DOI)

Diminished hand: A Diminished Reality-Based Work Area Visualization

Mori Shohei, Maezawa Momoko, Ienaga Naoto, Saito Hideo

2017 IEEE Virtual Reality (VR) 443 - 444 2017.04

Research paper (international conference proceedings), Joint Work, Accepted

Live instructors perspective videos are useful to present intuitive visual instructions for trainees in medical and industrial settings. In such videos, the instructors hands often hide the work area. In this demo, we present a diminished hand for visualizing the work area hidden by hands by capturing the work area with multiple cameras. To achieve the diminished reality, we use a light field rendering technique, in which light rays avoid passing through penalty points set in the unstructured light fields reconstructed from the multiple viewpoint images.

Access to Document (DOI)

Speaker Identification Based on Integrated Face Direction in a Group Conversation

Naoto Ienaga, Yuko Ozasa, and Hideo Saito

2017 IEEE Winter Applications of Computer Vision Workshops (WACVW) (IEEE) 53 - 57 2017.03

Research paper (international conference proceedings), Accepted

Access to Document (DOI)

Measuring grasp posture using an embedded camera

Naoaki Kashiwagi, Yuta Sugiura, Natsuki Miyata, Mitsunori Tada, Maki Sugimoto, and Hideo Saito

2017 IEEE Winter Applications of Computer Vision Workshops (WACVW) 42 - 47 2017.03

Research paper (international conference proceedings), Joint Work

Access to Document (DOI)

Line-based SLAM Considering Directional Distribution of Line Features in an Urban Environment

Kei Uehara, Hideo Saito and Kosuke Hara

Proceedings of the 12th International Conference on Computer Vision Theory and Applications (VISAPP2017) 6 255 - 264 2017.02

Research paper (international conference proceedings), Joint Work, Accepted

In this paper, we propose a line-based SLAM from an image sequence captured by a vehicle in consideration
with the directional distribution of line features that detected in an urban environments. The proposed SLAM
is based on line segments detected from objects in an urban environment, for example, road markings and
buildings, that are too conspicuous to be detected. We use additional constraints regarding the line segments
so that we can improve the accuracy of the SLAM. We assume that the angle of the vector of the line segments
to the vehicle’s direction of travel conform to four-component Gaussian mixture distribution. We define a
new cost function considering the distribution and optimize the relative camera pose, position, and the 3D
line segments by bundle adjustment. In addition, we make digital maps from the detected line segments. Our
method increases the accuracy of localization and revises tilted lines in the digital maps. We implement our
method to both the single-camera system and the multi-camera system. The accuracy of SLAM, which uses
a single-camera system with our constraint, works just as well as a method that uses a multi-camera system
without our constraint.

Primitive Shape Recognition via Superquadric Representation using Large Margin Nearest Neighbor Classifier

Ryo Hachiuma, Yuko Ozasa and Hideo Saito

Proceedings of the 12th International Conference on Computer Vision Theory and Applications (VISAPP2017) 5 325 - 332 2017.02

Research paper (international conference proceedings), Joint Work, Accepted

It is known that humans recognize objects using combinations and positional relations of primitive shapes.
The first step of such recognition is to recognize 3D primitive shapes. In this paper, we propose a method for
primitive shape recognition using superquadric parameters with a metric learning method, large margin nearest
neighbor (LMNN). Superquadrics can represent various types of primitive shapes using a single equation with
few parameters. These parameters are used as the feature vector of classification. The real objects of primitive
shapes are used in our experiment, and the results show the effectiveness of using LMNN for recognition based
on superquadrics. Compared to the previous methods, which used k-nearest neighbors (76.5%) and Support
Vector Machines (73.5%), our LMNN method has the best performance (79.5%).

Application of LSD-SLAM for Visualization Temperature in Wide-area Environment

Masahiro Yamaguchi, Hideo Saito and Shoji Yachida

Proceedings of the 12th International Conference on Computer Vision Theory and Applications (VISAPP2017) 4 216 - 223 2017.02

Research paper (international conference proceedings), Joint Work, Accepted

In this paper, we propose a method to generate a three-dimensional (3D) thermal map by overlaying thermal
images onto a 3D surface reconstructed by a monocular RGB camera. In this method, we capture the target
scene moving both an RGB camera and a thermal camera, which are mounted on the same zig. From the
RGB image sequence, we reconstruct 3D structures of the scene by using Large-Scale Direct Monocular
Simultaneous Localization and Mapping (LSD-SLAM), on which temperature distribution captured by the
thermal camera is overlaid, thus generate a 3D thermal map. The geometrical relationship between those
cameras is calibrated beforehand by using a calibration board that can be detected by both cameras. Since we
do not use depth cameras such as Kinect, the depth of the target scene is not limited by the measurement range
of the depth camera; any depth range can be captured. To demonstrating this technique, we show synthesized
3D thermal maps for both indoor and outdoor scenes.

Eating and Drinking Recognition via Integrated Information of Head Directions and Joint Positions in a Group

Naoto Ienaga, Yuko Ozasa and Hideo Saito

Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods (ICPRAM 2017) (Science and Technology Publications) 527 - 533 2017.02

Research paper (international conference proceedings), Joint Work, Accepted

Recent years have seen the introduction of service robots as waiters or waitresses in restaurants and cafes.
In such venues, it is common for customers to visit in groups as well as for them to engage in conversation
while eating and drinking. It is important for cyber serving staff to understand whether they are eating and
drinking, or not, in order to wait on tables at appropriate times. In this paper, we present a method by which
the robots can recognize eating and drinking actions performed by individuals in a group. Our approach uses
the positions of joints in the human body as a feature and long short-term memory to achieve a recognition
task on time-series data. We also used head directions in our method, as we assumed that it is effective for
recognition in a group. The information garnered from head directions and joint positions is integrated via
logistic regression and employed in recognition. The results show that this yielded the highest accuracy and
effectiveness of the robots’ tasks.

First Deployment of Diminished Reality for Anatomy Education

Ienaga Naoto, Bork Felix, Meerits Siim, Mori Shohei, Fallavollita Pascal, Navab Nassir, Saito Hideo

The second International Workshop on Diminished Reality as Challenging Issue in Mixed and Augmented Reality (IWDR2016), 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct) (IEEE) 294 - 296 2017.01

Research paper (international conference proceedings), Accepted

Understanding the anatomy of the human body is vital for everyone working in the medical domain. Augmented reality (AR) systems for anatomy teaching, which display virtual information directly on top of a users' body, have proven to facilitate mental mapping compared to traditional teaching paradigms. In this paper, we explore the potential of diminished reality (DR) in the context of anatomy education. As a first necessary step to achieving a DR anatomy education system, parts of the human body have to be extracted and diminished from the video stream. Our system diminishes either the arm or head of the user by projecting a background image recovered using RGB-D cameras. Such a system, if combined with an accurate overlay of virtual counterparts, could potentially improve the learning effect by attracting the users' attention to the virtual information and improve visual perception by avoiding the well-known floating effect of AR.

Access to Document (DOI)

Detour Light Field Rendering for Diminished Reality Using Unstructured Multiple Views

Mori Shohei, Maezawa Momoko, Ienaga Naoto, Saito Hideo

Research paper (international conference proceedings), Joint Work, Accepted

Instructor's perspective videos are useful for presenting intuitive visual instructions for trainees in medical and industrial settings. In such videos, the instructor's arms often obstruct the trainee's view of the work area. In this article, we present a diminished reality method for visualizing the work area hidden by an instructor's arms by capturing the work area with multiple cameras. To achieve such diminished reality, we propose detour light field rendering (DLFR), in which light rays avoid passing through penalty points set in the unstructured light fields reconstructed from multiple viewpoint images. In DLFR, the camera blending field used in an existing freeviewpoint image generation method known as unstructured lumigraph is re-designed based on our use cases. In this re-design, lesser weights are given to light rays as they pass close to given penalty points. Experimental results demonstrate that using DLFR, the appearance of an undesirable object can be removed from an image in real time.

Access to Document (DOI)

Robust camera pose estimation by viewpoint classification using deep learning

Yoshikatsu Nakajima and Hideo Saito

Computational Visual Media (Springer) 3 ( 2 ) 189 - 198 2016.12

Research paper (scientific journal), Joint Work, Accepted

Camera pose estimation with respect to target scenes is an important technology for superimposing virtual information in augmented reality (AR). However, it is difficult to estimate the camera pose for all possible view angles because feature descriptors such as SIFT are not completely invariant from every perspective. We propose a novel method of robust camera pose estimation using multiple feature descriptor databases generated for each partitioned viewpoint, in which the feature descriptor of each keypoint is almost invariant. Our method estimates the viewpoint class for each input image using deep learning based on a set of training images prepared for each viewpoint class. We give two ways to prepare these images for deep learning and generating databases. In the first method, images are generated using a projection matrix to ensure robust learning in a range of environments with changing backgrounds. The second method uses real images to learn a given environment around a planar pattern. Our evaluation results confirm that our approach increases the number of correct matches and the accuracy of camera pose estimation compared to the conventional method.

Access to Document (DOI)

Reconstruction of 3D models consisting of line segments

Ienaga Naoto, Saito Hideo

ACCV 2016 International Workshops, Revised Selected Papers, Part II, Lecture Notes in Computer Science book series (LNCS, volume 10117) (Springer International Publishing) 100 - 113 2016.11

Research paper (international conference proceedings), Accepted

Reconstruction of three-dimensional (3D) models from images is currently one of the most important areas of research in computer vision. In this paper, we propose a method to recover 3D models using the minimum number of line segments. By using structure-from-motion, the proposed method first recovers a 3D model of line segments detected from an input image sequence. We then detect overlapping 3D line segments that redundantly represent a single line structure so that the number of 3D line segments representing the target scene can be reduced without losing the detailed geometry of the structure. We apply matching and depth information to remove redundant line segments from the model while keeping the necessary segments. In experiments, we confirm that the proposed method can greatly reduce the number of line segments. We also demonstrate that the accuracy and computational time for camera pose estimation can be significantly improved with the 3D line segment model recovered by the proposed method. Moreover, we have applied the proposed method to see through occluded areas.

Access to Document (DOI)

Synthesis of a stroboscopic image from a hand-held camera sequence for a sports analysis

Kunihiro Hasegawa and Hideo Saito

Computational Visual Media (Springer) 2 ( 3 ) 277 - 289 2016.09

Research paper (scientific journal), Joint Work, Accepted

This paper presents a method for synthesizing a stroboscopic image of a moving sports player from a hand-held camera sequence. This method has three steps: synthesis of background image, synthesis of stroboscopic image, and removal of player’s shadow. In synthesis of background image step, all input frames masked a bounding box of the player are stitched together to generate a background image. The player is extracted by an HOG-based people detector. In synthesis of stroboscopic image step, the background image, the input frame, and a mask of the player synthesize a stroboscopic image. In removal of shadow step, we remove the player’s shadow which negatively affects an analysis by using mean-shift. In our previous work, synthesis of background image has been timeconsuming. In this paper, by using the bounding box of the player detected by HOG and by subtracting the images for synthesizing a mask, computational speed and accuracy can be improved. These have contributed greatly to the improvement from the previous method. These are main improvements and novelty points from our previous method. In experiments, we confirmed the effectiveness of the proposed method, measured the player’s speed and stride length, and made a footprint image. The image sequence was captured under a simple condition that no other people were in the background and the person controlling the video camera was standing still, such like a motion parallax was not occurred. In addition, we applied the synthesis method to various scenes to confirm its versatility.

Access to Document (DOI)

Recognition and pose estimation of primitive shapes from depth images for spatial augmented reality

Ryo Hachiuma and Hideo Saito

Proceedings of IEEE 2nd Workshop on Everyday Virtual Reality (WEVR2016) 2016.03

Research paper (international conference proceedings), Joint Work, Accepted

Access to Document (DOI)

Marker-less augmented reality framework using on-site 3D line-segment-basedmodel generation

Nakayama Yusuke, Saito Hideo, Shimizu Masayoshi, Yamaguchi Nobuyasu

Journal of Imaging Science and Technology 60 ( 2 ) 20401-1 - 20401-24 2016.03

Research paper (scientific journal), Joint Work, Accepted, ISSN 1062-3701

The authors propose a line-segment-based marker-less augmented reality (AR) framework that involves an on-site model-generation method and on-line camera tracking. In most conventional model-based marker-less AR frameworks, correspondences between the 3D model and the 2D frame for camera-pose estimation are obtained by feature-point matching. However, 3D models of the target scene are not always available, and feature points are not detected from texture-less objects. The authors' framework is based on a model-generation method with an RGB-D camera and model-based tracking using line segments, which can be detected even with only a few feature points. The camera pose of the input images can be estimated from the 2D-3D line-segment correspondences given by a line-segment feature descriptor. The experimental results show that the proposed framework can achieve AR when other point-based frameworks cannot. The authors also argue that their framework can generate a model and estimate camera pose more accurately than their previous study.

Access to Document (DOI)

Chromaticity-matched Superimposition of Foreground Objects in Different Environments

Yohei Ogura, Yoichi Nakasone, Shao Ming and Hideo Saito

Proceedings of the 22nd Korea-Japan Joint Workshop on Frontiers of Computer Vision (FCV2016) 141 - 145 2016.02

Research paper (international conference proceedings), Joint Work, Accepted

The illumination chromaticity is one of the important
factor for appearance of objects in images. When we
superimpose some objects on another scene to generate composite
images, the resulting image may give us strangeness if the
illumination chromaticity of foreground scene and background
scene are different. In this paper, we propose a composite
image synthesis method which can achieve illumination chromaticity
consistency. Our goal is to generate a illumination
chromaticity-matched composite image from two images which
are taken under different illumination environment. We estimate
chromaticity of two scenes based on segmented color regions.
Subsequently illumination chromaticity matching is performed
based on estimated chromaticity of two scenes. After the chromaticity
matching process, we extract the target person by using
the grab cut segmentation. The result image is obtained by
superimposing the target person or object onto background
scene. We demonstrate that our method can perform some
different illumination chromaticity environment.

Camera Pose Estimation Based on Keypoints Matching with Pre-Captured Set of Real Images

Kei Obata and Hideo Saito

Proceedings of the 22nd Korea-Japan Joint Workshop on Frontiers of Computer Vision (FCV2016) 76 - 80 2016.02

Research paper (international conference proceedings), Joint Work, Accepted

In this paper, we propose a method for camera pose
estimation based on keypoints matching with images capturing
the target scene from multiple viewpoints. For estimating camera
pose from an image, 3D structure of the target scene is needed.
Then a set of correspondences between 2D position in the image
and 3D positions in the target scene provides the camera pose by
solving Perspective-n-Point problem. Image keypoints are used
as these positions, but finding corresponding 3D positions of the
3D structure is not easy problem even if a 3D textured model
of the target scene is given. This is because the appearance of
each 3D position is not always the same. In order to solve this
problem, we adopt real images as a source of keypoints contained
in the database. Real image represents accurate appearance for
each 3D position, so keypoint matching with the input images
can be more accurate. We constructed database based on the
keypoint using the local image feature derived from real images
and evaluated the accuracy of camera pose estimation. The result
showed that the database constructed by our method is effective
to accurate camera tracking.

Stroboscopic Image Synthesis of Sports Player from Hand-Held Camera Sequence

Hasegawa Kunihiro, Saito Hideo

Proceedings of 2015 IEEE International Conference on Computer Vision Workshop (ICCVW) 726 - 733 2016.02

Research paper (international conference proceedings), Joint Work, Accepted

A method for synthesizing the stroboscopic image of a moving sports player from a hand-held camera sequence is proposed. In this method, a player is extracted from each frame of an input video sequence using a HOG-based people detector. After removing the bounding box of the extracted player from each frame, all frames are stitched together to generate a background image without the player. The player's area is then overlaid onto the stitched background image and a stroboscopic image of the player is synthesized. By using the bounding box of the player detected by HOG and by subtracting the images, computational speed and accuracy can be improved compared with the conventional method that segments the player's region. In addition to the stroboscopic image synthesis, we also remove the player's shadow to improve the appearance of the resultant stroboscopic image. Experimental results demonstrate the effectiveness of the proposed method.

Access to Document (DOI)

勾配画像処理に基づく三次元点群からの高精度地図の自動生成

Hara Kosuke, Saito Hideo

自動車技術会論文集 47 ( 1 ) 183 - 188 2016.01

Research paper (scientific journal), Joint Work, Accepted

Relighting for an arbitrary shape object under unknown illumination environment

Ogura Yohei, Saito Hideo

Proceedings of 11th International Symposium, ISVC 2015,Lecture Notes in Computer Science Volume 9475 (Springer) 433 - 442 2015.12

Research paper (international conference proceedings), Joint Work

Relighting techniques can achieve the photometric consistency in synthesizing a composite images. Relighting generally need the object’s shape and illumination environment. Recent research demonstrates possibilities of relighting for an unknown shape object or the relighting the object under unknown illumination environment. However, achieving both tasks are still challenging issue. In this paper, we propose a relighting method for an unknown shape object captured under unknown illumination environment by using an RGB-D camera. The relighted object can be rendered from pixel intensity, surface albedo and shape of the object and illumination environment. The pixel intensity and the shape of the object can simultaneously be obtained from an RGB-D camera. Then surface albedo and illumination environment are iteratively estimated from the pixel intensity and shape of the object.We demonstrate that our method can perform relighting for a dynamic shape object captured under unknown illumination using an RGB-D camera.

Access to Document (DOI)

アルベドと法線ベクトルの緊密度に着目した影と鏡面反射除去のための組み合わせ照度差ステレオ法

家永　直人, 斎藤英雄, 清水雅芳，手塚耕一，岩村康正

精密工学会誌 (公益社団法人精密工学会) 81 ( 12 ) 1154 - 1161 2015.12

Research paper (scientific journal), Joint Work, Accepted

We propose a method of ego-motion estimation for a self-driving car on which we installed cameras with non-overlapping views. By finding corresponding points between the multi-camera images, we aim to enhance the accuracy of the ego-motion estimation. However since the viewing directions are very different from one camera to the other, a conventional algorithm such as SURF cannot detect a sufficient number of correspondences. Additionally in the case where cameras have low frame rate and the vehicle has high speed, the scene changes might be too big to find correspondences between the same camera images. We propose a novel matching algorithm by warping feature patches detected in different cameras based on urban 3D structure. We assume that detected features exist on the surface of buildings or roads and the patch around the feature is planar. Based on this assumption, we can warp the patches so that the feature descriptors are similar for the corresponding feature points. We apply Bundle Adjustment to the found correspondences to optimize the odometry. The result shows higher estimation accuracy when compared to other matching methods.

Access to Document (DOI)

都市の3次元構造を利用した非重複カメラ間の特徴点マッチングによる車両運動の推定

Kazasaki Atsushi, Saito Hideo, Hara Kosuke

精密工学会誌 (公益社団法人精密工学会) 81 ( 12 ) 1146 - 1153 2015.12

Research paper (scientific journal), Joint Work, Accepted

Access to Document (DOI)

Vehicle localization based on the detection of line segments from multi-camera images

Hara Kosuke, Saito Hideo

Journal of Robotics and Mechatronics 27 ( 6 ) 617 - 626 2015.12

Research paper (scientific journal), Joint Work, Accepted, ISSN 0915-3942

For realizing autonomous vehicle driving and advanced safety systems, it is necessary to achieve accurate vehicle localization in cities. This paper proposes a method of accurately estimating vehicle position by matching a map and line segment features detected from images captured by a camera. Features such as white road lines, yellow road lines, road signs, and curb stones, which could be used as clues for vehicle localization, were expressed as line segment features on a two-dimensional road plane in an integrated manner. The detected line segments were subjected to bird’s-eye view transformation to transform them to the vehicle coordinate system so that they could be used for vehicle localization regardless of the camera configuration. Moreover, an extended Kalman filter was applied after a detailed study of the line observation errors for realizing real-time estimation. Vehicle localization was tested under city driving conditions, and the vehicle position was identified with sub-meter accuracy.

Access to Document (DOI)

Geolocation for printed maps using line segment-based SIFT-like feature matching

Minster Gautier, Moreau Guillaume, Saito Hideo

Proceedings of 2015 IEEE International Symposium on Mixed and Augmented Reality Workshops, Challenges and Applications of Urban Augmented Reality 88 - 93 2015.10

Research paper (international conference proceedings), Joint Work, Accepted

This paper presents a method for the geolocation of printed maps. It enables the registration of unprepared maps with a Geographical Information System (GIS) database, and can for example be used as a first step to augment an unknown map. We define and match local road pattern descriptors, which are similar to SIFT descriptors [6], but adapted to the case of simple textureless line segments. Using a processing pipeline commonly encountered in the feature-point based matching of texture images - composed of offline description and indexing, followed by an online description, matching and robust transformation estimation - we show that local descriptors can successfully register unprepared maps using only geographic features and no texture information. Our method is scale and rotation invariant, and circumvents the two hurdles that are the level-of-detail, and the changing colormaps and textures, allowing the processing of large classes of printed maps.

Access to Document (DOI)

Diminished reality for hiding a pedestrian using hand-held camera

Hasegawa Kunihiro, Saito Hideo

Proceedings of 2015 IEEE International Symposium on Mixed and Augmented Reality Workshops, International Workshop on Diminished Reality as Challenging Issue in Mixed and Augmented Reality (IWDR2015) 47 - 52 2015.09

Research paper (international conference proceedings), Joint Work, Accepted

This paper proposes a diminished reality method for hiding a pedestrian from a hand-held camera sequence. In the proposed method, the pedestrian is extracted from each frame of input video sequence using HOG based people detector. The detected area is excluded from each frame by masking, and then two frames are stitched for generating a stitched background image. By using the stitched background image, the pedestrian's mask for hiding is made. It is used for overlaying the background pixel to the pedestrian in the original frame. The presented experimental result is demonstrate the effectiveness of the proposed method.

Access to Document (DOI)

Motion estimation for non-overlapping cameras by improvement of feature points matching based on urban 3D structure

Kawasaki Atsushi, Saito Hideo, Hara Kosuke

Proceedings of 2015 IEEE International Conference on Image Processing (ICIP) 1230 - 1234 2015.09

Research paper (international conference proceedings), Joint Work, Accepted

We propose a method of ego-motion estimation for a self-driving vehicle using multiple cameras. By finding corresponding points between the multi-camera images, we aim to enhance the accuracy of the ego-motion estimation. However since the viewing directions are very different from one camera to the other, a conventional algorithm such as SURF cannot detect a sufficient number of correspondences. We propose a novel matching algorithm by warping feature patches detected in different cameras based on urban 3D structure. We assume that detected features exist on the surface of buildings or roads and the patch around the feature is planar. Based on this assumption, we can warp the patches so that the feature descriptors are similar for the corresponding feature points. We apply Bundle Adjustment to the found correspondences to optimizes the odometry. The result shows higher estimation accuracy when compared to other matching method.

Access to Document (DOI)

Real-time diminished reality for dynamic scenes

Meerits Siim, Saito Hideo

Research paper (international conference proceedings), Accepted

Diminished reality (DR) research aims to develop methods to visually hide objects from real scenes. One or more cameras capture a scene where some objects, called the occluding objects, are 'diminished' by an DR system. To do this in real time we need to use multiple cameras. We introduce a novel multi-camera DR system utilizing an RGB-D camera to hide arbitrary trackable objects from a scene. In our case, the scene background does not have to be planar for the system to work and we can handle scene changes in real time. Additonally, we introduce color correction technique to merge image content originating from different cameras and also demonstrate method to repair images with partially missing content.

Access to Document (DOI)

Combination photometric stereo using compactness of albedo and surface normal in the presence of shadows and specular reflection

Ienaga Naoto, Saito Hideo, Tezuka Kouichi, Iwamura Yasumasa, Shimizu Masayoshi

Proceedings of the 16th International Conference on Computer Analysis of Images and Patterns, CAIP 2015, LNCS 9256 (Springer) 445 - 455 2015.09

Research paper (international conference proceedings), Joint Work, Accepted

We present a novel combination photometric stereo which can estimate surface normals precisely even for images including shadows and specular reflection. We can use photometric stereo if there are more than three input images. Therefore we can employ photometric stereo with nC3 combinations for n input images. We make 3D distribution of albedos and surface normals estimated from pixel intensities of nC3 pixel combinations. In the distribution, we define a novel value “compactness” to distinguish pixels which are included in neither shadows nor specular reflection from pixels which are included in shadows or specular reflection. Through experimental results, we demonstrate that the proposed method can estimate surface normals in the presence of shadows and specular reflection. Moreover the proposed method is superior to previous works in better accuracy.

Access to Document (DOI)

RGB-Dスーパーピクセル領域分割された局所領域内の平面性を利用した長距離デプス画像の高解像度化

松本一紀，斎藤英雄

電子情報通信学会論文誌 D J98-D ( 9 ) 1226 - 1229 2015.09

Research paper (scientific journal), Joint Work, Accepted, ISSN 1881-0225

Reconstructing the 3D Trajectory of a Ball with Unsynchronized Cameras

Tamaki Sho, Saito Hideo

International journal of computer science and sport 14 ( 1 ) 51 - 68 2015.09

Research paper (scientific journal), Accepted, ISSN 1684-4769

The 3D trajectory of the ball is one of the most important performance indicators
in ball games. However, it often cannot be measured because of the requirements:
synchronized cameras and control points within the playing field. Thus, this paper
proposes a method that reconstructs the 3D trajectory of a ball with
unsynchronized cameras. The proposed method consists of ball detection, camera
calibration, and trajectory reconstruction. At first, ball candidates are detected on
the basis of their appearance. Balls are then extracted from the candidates on the
basis of their motion. After the extraction, ball trajectories are reconstructed in
image spaces. The corresponding points in two views are estimated on the basis
of the ball trajectory in image spaces and the temporal offset between cameras,
which is supposed to be recorded with a few erroneous frames. The matrix
including the geometrical relationship between cameras, namely the fundamental
matrix, is then computed from the corresponding points. The estimated one is
inaccurate due to the error of the temporal offset. The key feature of this method
is to optimize the temporal offset and fundamental matrix simultaneously. After
the optimization, the geometrical relationship between cameras is computed from
the matrix. Balls are extracted again by using the temporal offsets and the
geometrical relationship between cameras. A ball trajectory is finally
reconstructed as connected trajectories that are separated at collisions. It is
experimentally demonstrated that the proposed method accurately calibrates
cameras and successfully reconstructs the 3D trajectory of the ball.

Visualization of Dynamic Hidden Areas by Real-Time 3D Structure Acquisition Using RGB-D Camera

Siim Meerits and Hideo Saito

Proceedings of International Display Manufacturing Conference (IDMC) & 3D Systems and Applications (3DSA) 2015 AP4-008 2015.08

Research paper (international conference proceedings), Joint Work, Accepted

An omnidirectional vision system for bus safety surveillance

Hung Dao Huu, Saito Hideo, Yamamoto Keiichi, Sato Hiromitsu

Proceedings of 3rd Workshop on Activity Monitoring by Multiple Distributed Sensing (AMMDS), 12th edition of the IEEE AVSS conference 2015.08

Research paper (international conference proceedings), Joint Work, Accepted

Sudden brake seems to be unavoidable in driving practice to prevent from accidence. The sudden brake may often cause falling risks to on-board passengers in public transport and even threaten the lives of the senior. Thus, this paper presents an omnidirectional vision system to assist bus drivers in ensuring the safety for passengers. To this end, we configure an auxiliary camera in the boarding gate to recognize the age of passengers and an omnidirectional camera on the ceiling of the bus to detect and track the passengers. Their trajectories and positions are mapped into a bus layout, together with their age information transferred from the auxiliary camera. This bus layout is shown on a small display in front of bus drivers. Instead of looking to the mirror to guess and gather information of passengers, the driver can collect such necessary information for safe driving. We set up experiments on a real bus and demonstrate the system by preliminary results.

Access to Document (DOI)

Mutliple Object Tracking for End-to-End Transmission System of Free-Viewpoint Video

Shogo Miyata, Hideo Saito

Proceedings of International Display Manufacturing Conference (IDMC) & 3D Systems and Applications (3DSA) 2015 3DSA S5-5 2015.08

Research paper (international conference proceedings), Joint Work, Accepted

Polymorphic cataloguing and viewing system for using digital archives

Ishikawa Hiroyo, Saito Hideo, Miyashita Yamato, Kaneko Kunitake

Proceedings of 2014 International Conference on Virtual Systems & Multimedia (VSMM) (IEEE) 176 - 179 2015.06

Research paper (international conference proceedings), Joint Work

We propose a digital content viewing system which visualizes the polymorphic relationships among digital content interactively and uses it as a visual interface. The relationships are expressed by associating of two content and grouping and shown by directed graphs. On the graphs, nodes mean the content files or the groups, and edges mean correspondences or group members. The graphs are archived as relation catalogues separately from content files. We visualize the relations of content on a directed graph. To visualize the graphs, we use a method of locating nodes by horizontal directions and height. Nodes linked with a target node will be located on a horizontal circle and their height is decided by their relation catalogue. After deciding the node location, nodes and links are drawn using 3D graphics and animation. The visualization result called 'polymorphic view' is used as a visual interface for selecting content. Selecting content on the visualized graph will expand new links and new content nodes additionally. We can show another content by tracing graph links. It becomes easy to understand the relations between digital content, because we can change the direction and size of the directed graph. We developed a content viewer system (MoSaIC-II) using the polymorphic view as a visual interface and we can see the usability of the polymorphic view.

Access to Document (DOI)

3D Line Segment Based Model Generation by RGB-D Camera for Camera Pose Estimation

Yusuke Nakayama, Hideo Saito, Masayoshi Shimizu, Nobuyasu Yamaguchi

Computer Vision - ACCV 2014 Workshops Lecture Notes in Computer Science 9010 459 - 472 2015.04

Research paper (international conference proceedings), Joint Work, Accepted

In this paper, we propose a novel method for generating 3D line segment based model from an image sequence taken with a RGB-D camera. Constructing 3D geometrical representation by 3D model is essential for model based camera pose estimation that can be performed by corresponding 2D features in images with 3D features of the captured scene. While point features are mostly used as such features for conventional camera pose estimation, we aim to use line segment features for improving the performance of the camera pose estimation. In this method, using RGB images and depth images of two continuous frames, 2D line segments from the current frame and 3D line segments from the previous frame are corresponded. The 2D-3D line segment correspondences provide camera pose of the current frame. All of 2D line segments are finally back-projected to the world coordinate based on the estimated camera pose for generating 3D line segment based model of the target scene. In experiments, we confirmed that the proposed method can successfully generate line segment based models, while 3D models based on the point features often fail to successfully represent the target scene.

Access to Document (DOI)

Measuring the Rotation of the Ball Using the Ball Mark and the Trajecto

Sho TAMAKI, Hideo SAITO, and Kazuto YOSHIDA

電子情報通信学会論文誌 D J98-D ( 4 ) 630 - 650 2015.04

Research paper (scientific journal), Joint Work, Accepted, ISSN 1881-0225

卓球においてボールの回転は戦術的に重要な要素と考えられている．しかし，従来の回転計測法は，表面全体にマークを追加したボールを高速度カメラを用いて撮影するものであり，撮影されるデータ量が膨大になる，あるいは試合と同じ条件下の回転による攻防を分析できない等の問題がある．ボールの運動モデルに基づき軌跡から回転を推定する方法も提案されているが，正確な回転計測は難しい．そこで本研究では，高速度カメラを用いることを前提とせず，且つ試合で用いられる卓球ボールを対象とできる回転計測法を提案する．提案手法ではまずボールのマークを検出し，検出されたマーク領域の重心から回転軸を算出する．高速度カメラを用いないことから，回転軸周りの回転方向及び隣接フレーム間の回転角速度は一意には求まらないが，回転角速度の全ての候補値に対応する軌跡を運動モデルに基づきシミュレートし，実測された軌跡に最も近い軌跡を探索することで最も真値に近い回転角速度を選択する．回転する卓球ボールを対象とした実画像実験により，本手法が実用的な精度で回転軸及び回転角速度を計測できることを示す．

Access to Document (DOI)

3D line segment based model generation by RGB-D camera for camera pose estimation

Nakayama Yusuke, Saito Hideo, Shimizu Masayoshi, Yamaguchi Nobuyasu

Kyokai Joho Imeji Zasshi/Journal of the Institute of Image Information and Television Engineers 69 ( 4 ) J148 - J159 2015.03

Research paper (scientific journal), Accepted, ISSN 1342-6907

本論文では，カメラ位置姿勢推定のための3次元線分モデルを，対象シーンをRGB-Dカメラで撮影するだけで簡易に生成する手法を提案する．従来の特徴点マッチングによるモデル生成は，特徴点が検出されない場合は失敗するという問題点があった．そのような場面でも線分は安定して検出されるため，3次元線分モデルの生成法が求められている．本提案手法では線分特徴記述子を用いた線分マッチングにより，簡易に，そして3次元情報が必要となるその場で3次元線分モデルを生成する．まず，入力されたRGB画像と距離画像から前フレームの3次元線分と現フレームの2次元線分の対応付けを行う．この2次元3次元線分対応から現フレームのカメラ位置姿勢を推定し，フレーム中の2次元線分を3次元空間上へ逆投影して3次元線分モデルとする．実験により，提案手法が特徴点マッチングではモデルが生成できないシーンでも正確なモデルを生成できることを確認した．

Access to Document (DOI)

AR display system of temperature by camera tracking based on 3D environmental model

Nakagawa Wataru, Matsumoto Kazuki, De Sorbier Francois, Sugimoto Maki, Saito Hideo, Senda Shuji, Shibata Takashi, Iketani Akihiko

Kyokai Joho Imeji Zasshi/Journal of the Institute of Image Information and Television Engineers 69 ( 4 ) J160 - J168 2015.03

Research paper (scientific journal), Joint Work, ISSN 1342-6907

近年、温度カメラは、機械設備・建築物の検査や，人間や動物の体温の検査等，多くの目的に利用されるようになってきた．本論文では，RGB-Dカメラで取得した3次元環境モデルを利用して、このモデル上に温度分布をレンダリング表示することにより，対象環境の温度分布を自由な視点から観察可能なシステムを提案する．本システムでは，３Ｄ環境モデルを利用したカメラ位置姿勢のトラッキング手法に基づき，事前に取得しておいた温度分布に対して，現在の温度変化を表示することも可能である．提案システムの有効性を検証するために，幾つかのシーンに対して本システムを利用して表示した温度分布や温度変化分布を示す．

Access to Document (DOI)

Visualization of Temperature Change Using RGB-D Camera and Thermal Camera

Wataru Nakagawa, Kazuki Matsumoto, Francois de Sorbier, Maki Sugimoto, Hideo Saito, Shuji Senda, Takashi Shibata, Akihiko Iketani

Computer Vision - ECCV 2014 Workshops Lecture Notes in Computer Science Volume (Springer International Publishing) 8925 386 - 400 2015.03

Research paper (international conference proceedings), Joint Work, Accepted, ISSN 0302-9743

In this paper, we present a system for visualizing temperature changes in a scene using an RGB-D camera coupled with a thermal camera. This system has applications in the context of maintenance of power equipments. We propose a two-stage approach made of with an offline and an online phases. During the first stage, after the calibration, we generate a 3D reconstruction of the scene with the color and the thermal data. We then apply the Viewpoint Generative Learning (VGL) method on the colored 3D model for creating a database of descriptors obtained from features robust to strong viewpoint changes. During the second online phase we compare the descriptors extracted from the current view against the ones in the database for estimating the pose of the camera. In this situation, we can display the current thermal data and compare it with the data saved during the offline phase.

Access to Document (DOI)

Marker-less AR system based on line segment feature, The Engineering Reality of Virtual Reality 2015

Yusuke Nakayama, Hideo Saito, Masayoshi Shimizu, and Nobuyasu Yamaguchi

Proc. SPIE 9392, The Engineering Reality of Virtual Reality 2015 93920I 2015.03

Research paper (international conference proceedings), Joint Work, Accepted

We propose a method of marker-less AR which uses line segment feature. Estimating camera poses is important
part of AR system. In most of conventional marker-less AR system, feature point matching between a model and
its image is required for the camera pose estimation. However, a sufficient number of correspondence points is not
always detected to estimate accurate camera poses. To solve this problem, we propose the use of line segments
feature that can possibly be detected even when only a few feature points can be detected. In this paper, we
propose a marker-less AR system that uses line segment features for camera pose estimation. In this system,
we propose a novel descriptor of the line segment feature for achieving fast computation of the camera pose
estimation. We also construct a database which contains a k-d tree of line feature and 3D line segment position
for nding 2D-3D line segment correspondences from input image and the database, so that we can estimate the
camera pose and perform AR. We demonstrated that our proposed method can estimate the camera pose and
provide robust marker-less AR in the situation where point matching method fails.

Access to Document (DOI)

AR visualization of thermal 3D model by hand-held cameras

Kazuki Matsumoto, Wataru Nakagawa, Hideo Saito, Maki Sugimoto, Takashi Shibata, and Shoji Yachida

Proceedings of the Int. Conf. on Computer Vision Theory and Applications (VISAPP2015) 480 - 487 2015.03

Research paper (international conference proceedings), Joint Work, Accepted

Illumination estimation and relighting using an RGB-D camera

Yohei Ogura, Takuya Ikeda, Francois de Sorbier, and Hideo Saito

Proceedings of the Int. Conf. on Computer Vision Theory and Applications (VISAPP2015) 305 - 312 2015.03

Research paper (international conference proceedings), Joint Work, Accepted

n this paper, we propose a relighting system combined with a
n illumination estimation method using RGB-
D camera. Relighting techniques can achieve the photometri
c registration of composite images. They often
need illumination environments of the scene which include a
target object and the background scene. Some
relighting methods obtain the illumination environments b
eforehand. In this case, they cannot be used under
the unknown dynamic illumination environment. Some on-lin
e illumination estimation methods need light
probes which can be invade the scene geometry. In our method,
the illumination environment is estimated
from pixel intensity, normal map and surface reflectance bas
ed on inverse rendering in on-line processing.
The normal map of the arbitrary object which is used in the ill
umination estimation part and the relighting part
is calculated from the denoised depth image on each frame. Re
lighting is achieved by calculating the ratio for
the estimated Illumination environment of the each scene. T
hus our implementation can be used for dynamic
illumination or a dynamic object.

Extraction of blood vessels in retinal images using resampling high-order background estimation

Paripurana Sukritta, Chiracharit Werapon, Chamnongthai Kosin, Saito Hideo

IEICE Transactions on Information and Systems E98D ( 3 ) 692 - 703 2015.03

Research paper (scientific journal), Joint Work, ISSN 0916-8532

In retinal blood vessel extraction through background removal, the vessels in a fundus image which appear in a higher illumination variance area are often missing after the background is removed. This is because the intensity values of the vessel and the background are nearly the same. Thus, the estimated background should be robust to changes of the illumination intensity. This paper proposes retinal blood vessel extraction using background estimation. The estimated background is calculated by using a weight surface fitting method with a high degree polynomial. Bright pixels are defined as unwanted data and are set as zero in a weight matrix. To fit a retinal surface with a higher degree polynomial, fundus images are reduced in size by different scaling parameters in order to reduce the processing time and complexity in calculation. The estimated background is then removed from the original image. The candidate vessel pixels are extracted from the image by using the local threshold values. To identify the true vessel region, the candidate vessel pixels are dilated from the candidate. After that, the active contour without edge method is applied. The experimental results show that the efficiency of the proposed method is higher than the conventional low-pass filter and the conventional surface fitting method. Moreover, rescaling an image down using the scaling parameter at 0.25 before background estimation provides as good a result as a non-rescaled image does. The correlation value between the non-rescaled image and the rescaled image is 0.99. The results of the proposed method in the sensitivity, the specificity, the accuracy, the area under the receiver operating characteristic (ROC) curve (AUC) and the processing time per image are 0.7994, 0.9717, 0.9543, 0.9676 and 1.8320 seconds for the DRIVE database respectively.

Access to Document (DOI)

Realtime depth map noise reduction for RGB-D range data using plane fitting

Matsumoto Kazuki, De Sorbier Francois, Saito Hideo

Kyokai Joho Imeji Zasshi/Journal of the Institute of Image Information and Television Engineers 69 ( 3 ) J113 - J120 2015.02

Research paper (scientific journal), Accepted, ISSN 1342-6907

近年登場した距離センサーを搭載したデバイスの中でもその価格と汎用性ゆえに最も多く利用されているのがMicrosoftのKinectに代表されるRGB-Dカメラである．しかし，RGB-Dカメラから得られる３次元点群は多くのノイズを含んでおり，精密な距離情報の取得が求められるアプリケーションでの利用は依然として困難である．そこで本論文ではRGB-Dカメラから得られた３次元点群の観測ノイズの低減と補間を実時間で行う手法を提案する．本手法はRGB-Dカメラから得られた３次元点群からSuperpixel Segmentaiotnとラベリングを併用して平面で構成される小領域群を検出し，領域毎に平面方程式を推定し３次元点群をフィッティングさせることにより，もとの距離画像に含まれるノイズの低減と補間を行うものである．本手法では検出された領域内の点群の法線分布を算出し，その領域が平面の場合平面フィッティングを適用し曲面の場合Joint Bilateral Filterを適用することで，曲面分布した点群を保持しながら平面分布した点群のみを平面フィッティングすることができる．また，領域分割と平面フィッティングはすべて局所領域ごとに並列化できる手法を用いているため，本手法の実装はすべてGPUで行い実時間での処理が可能である．従来の手法と比べて，本手法はRGB-Dカメラから取得した点群に対して，曲面領域を保持したまま平面領域を平滑化することでノイズ低減と補間を行うことができる．

Access to Document (DOI)

手持ちカメラを用いた走者ストロボ画像の自動生成

SAITO HIDEO

精密工学会誌 81 ( 2 ) 156 - 163 2015.02

Research paper (scientific journal), Joint Work, Accepted

This paper discusses the method for the automatically generating stroboscopic image from a video image using a handheld camera comprehensively. The stroboscopic image can be used in order to measure stride length and speed of the runner based on the authors' previously proposed method, in which it is shown that measuring stride length and speed of the runner from the stroboscopic image is theoretically possible. The proposed automatic stroboscopic image generating method has two steps. The first step is generating a stitched background image from input video image. In the second step, runner images extracted from arbitrary frames are overlapped onto the stitched background image. Thus, this method can generate a stroboscopic image including the runners in arbitrary frames without overwriting them in some frames. These processes can be executed automatically, and these are noteworthy features. In the experiments under various environment to show the effectiveness of this method, it was confirmed that could generate a stroboscopic images well in respectively environment. It is also demonstrated that stride length and speed of the runner captured into the generated stroboscopic image can be measured with satisfactory accuracy by manually selected landing points in the stroboscopic image.

Access to Document (DOI)

Plane fitting and depth variance based upsampling for noisy depth map from 3D-ToF cameras in real-time

Matsumoto Kazuki, De Sorbier Francois, Saito Hideo

Proceedings of the International Conference on Pattern Recognition Applications and Methods (ICPRAM-2015) (SCITEPRESS (Science and Technology Publications, Lda.)) 150 - 157 2015.01

Research paper (international conference proceedings), Joint Work, Accepted

Recent advances of ToF depth sensor devices enables us to easily retrieve scene depth data with high frame rates. However, the resolution of the depth map captured from these devices is much lower than that of color images and the depth data suffers from the optical noise effects. In this paper, we propose an efficient algorithm that upsamples depth map captured by ToF depth cameras and reduces noise. The upsampling is carried out by applying plane based interpolation to the groups of points similar to planar structures and depth variance based joint bilateral upsampling to curved or bumpy surface points. For dividing the depth map into piecewise planar areas, we apply superpixel segmentation and graph component labeling. In order to distinguish planar areas and curved areas, we evaluate the reliability of detected plane structures. Compared with other state-of-the-art algorithms, our method is observed to produce an upsampled depth map that is smoothed and closer to the ground truth depth map both visually and numerically. Since the algorithm is parallelizable, it can work in real-time by utilizing highly parallel processing capabilities of modern commodity GPUs.

Access to Document (DOI)

Real Time Detection and Tracking of Printed Maps Based on Road Structure

Sebastien Callier, SAITO HIDEO, Guillaume Moreau

ITE Transactions on Media Technology and Applications 3 ( 1 ) 85 - 94 2015.01

Research paper (scientific journal), Joint Work, Accepted

This paper presents a method to detect and track a paper map and overlay virtual data over it. First, we take a picture of the map and identify the road colors using our road area extraction method. Using the color information we can then extract in real time the roads from a video and use the position of the intersections to identify the map from a database without any further user input. The main issue is to get reliable intersection positions in real time even though the map can contain a lot of overlapping data. Finally, using the location of the intersections we can track and augment the map with virtual information. Our method enables the real time matching, 3D tracking and augmentation of a paper map based on a database that can contain several thousands of intersections coordinates.

Access to Document (DOI)

REAL-TIME ENHANCEMENT OF RGB-D POINT CLOUDS USING PIECEWISE PLANE FITTING

Kazuki Matsumoto, Francois de Sorbier and Hideo Saito

Proceedings of the 2014 5th European Workshop on Visual Information Processing (EUVIP2014) (IEEE) 2014.12

Research paper (international conference proceedings), Joint Work, Accepted

Access to Document (DOI)

A Method for Reconstructing 3D Trajectory of Ball with Unsynchronized Cameras

SAITO HIDEO

The Japan Society for Precision Engineering (The Japan Society for Precision Engineering) 80 ( 12 ) 1157 - 1165 2014.12

Research paper (scientific journal), Joint Work, Accepted, ISSN 0912-0289

Ball trajectory is an important performance indicator in ball game. However, Measuring 3D trajectories is often unusable because synchronized cameras and control points inside the playing field are generally required. A method that reconstructs 3D trajectory of ball with unsynchronized cameras is proposed in this paper. Proposed method consists of ball detection, camera calibration, and trajectory reconstruction. At first, ball candidates are detected based on the appearance. For the camera calibration, highly probable balls are extracted from the candidates based on the motion. Imaged ball trajectories are then reconstructed. Next, fundamental matrix is computed from the corresponding points, which are estimated with the ball trajectories, and roughly estimated temporal offset, which is estimated manually. The estimated fundamental matrix is assumed to be inaccurate due to the inaccuracy of the temporal offset. The key feature of this method is to accurately estimate temporal offset and fundamental matrix by optimizing them iteratively based on the error of fundamental matrix. After the calibration, balls are extracted robustly with geometrical and temporal relationship between cameras. A ball trajectory is finally reconstructed as connected trajectories which separated at collisions. It is experimentally demonstrated that the proposed method can calibrate cameras precisely and reconstruct accurate ball trajectories.

Access to Document (DOI)

RGB-Dカメラを用いた実時間リライティング映像生成システムの開発

SAITO HIDEO

映像情報メディア学会誌 (映像情報メディア学会) 68 ( 12 ) J558 - J568 2014.11

Research paper (scientific journal), Accepted, ISSN 1342-6907

リライティングとは，実画像から，それを撮影した照明環境を別の正面環境で撮影したかのような画像を合成する処理のことであり，その実現には，対象物体の3次元形状（法線分布），反射特性，光源環境が必要である．本論文では，RGB-Dカメラを用いて実時間リライティングを実現可能なシステムを提案する．本システムでは，正確な物体領域抽出のためにノイズを含む距離画像の修正を行う．領域抽出を行った後，対象物体の法線分布を推定する．光源環境は，観測された鏡面球から取得し，任意の環境に対してリライティングを行う．また，実時間処理を実現するために，GPUを用いた並列処理を行う．我々の実装では，約10fpsの速度でリライティングが可能である．提案システムの有効性を確認するために行った実験では，対象物体に対して事前に撮影した動的光源環境へリライティングを行い，提案システムの性能を評価した．

Access to Document (DOI)

Auto-Generation of Runner's Stroboscopic Image and Measuring Landing Points Using a Handheld Camera

K. Hasegawa and H. Saito

Proc. 2014 Irish Machine Vision and Image Processing (IMVIP2014) 169 - 174 2014.08

Research paper (international conference proceedings), Joint Work, Accepted

Specular 3D Object Tracking by View Generative Learning

Yukiko Shinozuka, Francois de Sorbier and Hideo Saito

Proc. 2014 Irish Machine Vision and Image Processing (IMVIP2014) 9 - 14 2014.08

Research paper (international conference proceedings), Joint Work, Accepted

Combination Method: Photometric Stereo with Shadows

A. Dulac, S. Martedi, H. Saito, K. Tezuka and M. Shimizu

Proc. 2014 Irish Machine Vision and Image Processing (IMVIP2014) 137 - 142 2014.08

Research paper (international conference proceedings), Joint Work, Accepted

Human Trajectory Tracking Using a Single Omnidirectional Camera

A. Kawasaki, D. Hung and H. Saito

Proc. 2014 Irish Machine Vision and Image Processing (IMVIP2014) 157 - 162 2014.08

Research paper (international conference proceedings), Joint Work, Accepted

Accurate Camera Pose Estimation for KinectFusion Based on Line Segment Matching by LEHF

Yusuke Nakayama, Toshihiro Honda, Hideo Saito, Masayoshi Shimizu,and Nobuyasu Yamaguchi Accurate Camera Pose Estimation for KinectFusion

Proceedings of the 2014 22nd International Conference on Pattern Recognition (IEEE Computer Society Washington, DC, USA) 2149 - 2154 2014.08

Research paper (international conference proceedings), Joint Work, Accepted, ISSN 978-1-4799-5209-0

Access to Document (DOI)

Camera Pose Estimation for Mixed and Diminished Reality in FTV

Hideo Saito, Toshihiro Honda, Yusuke Nakayama, Francois de Sorbier

3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON), 2014 (IEEE) 2014.07

Research paper (international conference proceedings), Joint Work, Accepted

Access to Document (DOI)

RGB-Dカメラによる環境の3次元計測に基づく実時間隠消現実感

SAITO HIDEO

日本バーチャルリアリティ学会論文誌 (日本バーチャルリアリティ学会) 19 ( 2 ) 105 - 116 2014.06

Research paper (scientific journal), Joint Work, Accepted, ISSN 1344011X

Dynamic learning, retrieval, and tracking to augment hundreds of photographs

Julien Pilet, Hideo Saito

Virtual Reality (Springer London) 18 ( 2 ) 89 - 100 2014.06

Research paper (scientific journal), Joint Work, Accepted, ISSN 1434-9957

In this paper, we present a method that combines the speed and accuracy of tracking with the scalability of image retrieval. At the heart of our approach is a bi-layer clustering process that allows our system to index and retrieve objects based on tracks of features, thereby effectively summarizing the information available on multiple video frames. Dynamic learning of new viewpoints as the camera moves naturally yields the kind of robustness and reliability expected from an augmented reality engine. As a result, our system is able to track in real-time multiple objects, recognized with low delay from a database of more than 300 entries. We released the source code of our system in a package called Polyora.

Access to Document (DOI)

Rapid Feedback of Table Tennis Ball Trajectory with Unsynchronized Cameras

Sho Tamaki, Hideo Saito

Proc. International Association of Computer Science in Sports 2014 Conference 39 - 45 2014.06

Research paper (international conference proceedings), Joint Work, Accepted

Virtual rope slider

Tatsuya Kodera, Naoto Tani, Jun Morita, Naoya Maeda, Kazuna Tsuboi, Motoko Kanegae, Yukiko Shinozuka, Sho Shimamura, Kadoki Kubo, Yusuke Nakayama, Jaejun Lee, Maxime Pruneau, Hideo Saito, Maki Sugimoto

Proceedings of the 2014 Virtual Reality International Conference ( Article No. 36 ) 2014.04

Research paper (international conference proceedings), Joint Work, Accepted

Access to Document (DOI)

Virtual Slicer: Interactive visualizer for tomographic medical images based on position and orientation of handheld device

Sho Shimamura, Motoko Kanegae, Yuji Uema, Masahiko Inami, Tetsu Hayashida, Hideo Saito, and Maki Sugimoto

Proceedings of the 2014 Virtual Reality International Conference ( Article No. 12 ) 2014.04

Research paper (international conference proceedings), Joint Work, Accepted

This paper introduces an interface that helps understand the
correspondence between the patient and medical images.
Surgeons determine the extent of resection by using tomographic
images such as MRI (Magnetic Resonance Imaging) data
.
However, understanding the relationship between the patient and
tomographic images is di
ffi
cult. This study aims to visualize the
correspond
en
ce more intuitive
ly
.
In this paper, we propose an
interactive visualizer for medical images based on the relative
position and orientation of the handheld device and the patient.
We conducted an experiment to verify the performances of the
proposed method and several other methods. In the experiment,
t
he
proposed method showed the minimum error.

Access to Document (DOI)

Sharing 3D object with multiple clients via networks using vision-based 3D object tracking

Yukiko Shinozuka and Hideo Saito

Proceedings of the 2014 Virtual Reality International Conference ( Article No. 34 ) 2014.04

Research paper (international conference proceedings), Joint Work, Accepted

This paper proposes a new system for 3D object sharing with
multiple clients using networks. Our system aims to support group
communication across devices over the same 3D object and puts
the annotations with augmented reality automatically. In order to
keep putting the annotations during the interaction, we apply the
vision-based 3D object tracking algorithm which estimates 3D
position and pose of the object just by capturing the object with a
single RGB camera. By employing the tracking algorithm robust
to viewpoint changes and occlusion, we develop the 3D object
sharing system with a desktop as a server and a laptop and a smart
phone as clients. We conducted the experiments to show the
tracking algorithm is tolerant of interaction and can be applied to
various objects with texture.

Access to Document (DOI)

An AR edutainment system supporting bone anatomy learning

Wucherer, P. ; Oyamada, Y. ; Ma, M. ; Schoch, A. ; Kanegae, M. ; Shimizu, N. ; Kodera, T. ; Cahier, S. ; Weigl, M. ; Sugimoto, M. ; Fallavollita, P. ; SAITO HIDEO ; Navab, N.

IEEE Virtual Reality (VR2014) 113 - 114 2014.03

Research paper (international conference proceedings), Joint Work, Accepted

We present a medical Augmented Reality (AR) edutainment system for bone anatomy learning. This learning environment, called AR bone puzzle, is a metaphor for bone anatomy learning with AR visualization and intuitive interaction. AR bone puzzle uses its user's body as a puzzle frame and computer generated virtual bones as puzzle pieces. Users learn bone anatomy by assembling the virtual bone pieces on their body. Key features of this system are 3D AR visualization and intuitive gesture based user interaction.

Access to Document (DOI)

Stereoscopic augmented reality with pseudo-realistic global illumination effects

Francois de Sorbier; Hideo Saito

Proc. SPIE. 9011, Stereoscopic Displays and Applications XXV 90111W 2014.03

Research paper (international conference proceedings), Accepted

Access to Document (DOI)

Joint upsampling and noise reduction for real-time depth map enhancement

Kazuki Matsumoto, Chiyoung Song, Francois de Sorbier, Hideo Saito

Proc. SPIE 9011, Stereoscopic Displays and Applications XXV 901120 2014.03

Research paper (international conference proceedings), Joint Work, Accepted

An efficient system that upsamples depth map captured by Microsoft Kinect while jointly reducing the effect of noise is presented. The upsampling is carried by detecting and exploiting the piecewise locally planar structures of the downsampled depth map, based on corresponding high-resolution RGB image. The amount of noise is reduced by accumulating the downsampled data simultaneously. By benefiting from massively parallel computing capability of modern commodity GPUs, the system is able to maintain high frame rate. Our system is observed to produce the upsampled depth map that is very close to the original depth map both visually and mathematically.

Access to Document (DOI)

A calibration method for performance analysis in table tennis

Sho Tamaki, Hideo Saito

Proc. 20th Korea-Japan Joint Workshop on Frontiers of Computer Vision 330 - 334 2014.02

Research paper (international conference proceedings), Joint Work, Accepted

3D Surface Reconstruction of a BGA Connector with Large Specular Highlightsby Photometric Stereo

Maxime Vallee, Hideo Saito, Junichi Sugano, Yoshinobu Takizawa

Proc. 20th Korea-Japan Joint Workshop on Frontiers of Computer Vision 2014.02

Research paper (international conference proceedings), Joint Work, Accepted

View Independent Vehicle Make and Model Recognition with Virtual Frontal View

Yukiko Shinozuka, Takuya Minagawa, Hideo Saito

IEEJ Transactions on Electronics, Information and Systems 134 ( 2 ) 233 - 241 2014.02

Research paper (scientific journal), Accepted

This paper proposes a novel view independent vehicle make and model recognition method (VMMR). Our system identifies the make and model from the variety of viewpoints while the conventional methods for VMMR work only for the fixed frontal or rear images. In addition, it needs only the 2D images not CAD data for database. To solve the alignment issue, our method uses SIFT, that has scale and rotation invariance. For the view independent recognition, it creates less distorted frontal view images by view morphing or homography matrix calculated by the position of the license plate and extracts the keypoints from them. Our method enables to recognize up to 60-degree angle with high accuracy due to the less distorted virtual frontal images.

Access to Document (DOI)

View Independent Recognition of Vehicle Make and Model from View Morphed Frontal Image

Yukiko Shinozuka, Takuya Minagawa, Hideo Saito

2014 Int. Workshop on Advanced Image Technology (IWAIT 2014) 451 - 455 2014.01

Research paper (international conference proceedings), Joint Work, Accepted

This paper proposes a novel view independent
vehicle make and model recognition method(VMMR). Our system
identifies the make and model from the variety of viewpoints while
the conventional methods for VMMR work only for the fixed
frontal or rear images. In addition, it needs only the 2D images
not CAD data for database. To solve the alignment issue, our
method uses SIFT, that has scale and rotation invariance. For the
view independent recognition, it creates the more realistic and less
distorted frontal view images by view morphing and extracts the
keypoints from them. Our method enables to recognize up to 40-
degree angle with high accuracy due to the less distorted morphed
images. Our method can be extended to the other product model
recognition.

Augmented reality system using a smartphone based on getting a 3D environment model in real-time with an RGB-D camera

Toshihiro Honda, Francois de Sorbier, and Hideo Saito

2014 Int. Workshop on Advanced Image Technology (IWAIT 2014) 355 - 359 2014.01

Research paper (international conference proceedings), Joint Work, Accepted

n this paper, we propose a system to achieve
Augmented Reality (AR) on a smartphone. In this system, we
assume a fixed RGB-D camera connected to a server is installed in
the environment, and perform AR based on the 3D environment
shape got by the RGB-D camera in real-time. On the smartphone
side, it sends the image captured by its camera, and receives
the output image processed to perform AR by the server. On
the server side, it sends back the image that virtual objects
are superimposed on. At this time, by constructing the 3D
environment model in real-time using the RGB-D camera, it is
possible to calculate the smartphone pose and the interaction
between the virtual objects and the environment properly. It is
also possible to change the lighting virtually. Our experiment
revealed it is possible to superimpose virtual objects on the
smartphone in approximately 8fps.

Real time relighting with dynamic light environment using an RGB-D camera

Takuya Ikeda, Francois de Sorbier, and Hideo Saito

2014 Int. Workshop on Advanced Image Technology (IWAIT 2014) 420 - 425 2014.01

Research paper (international conference proceedings), Joint Work, Accepted

elighting techniques often require knowledge
about the shape and the reflectance property of a target object
and the light environment. The quality of the relighting is highly
dependent on the normals accuracy of the object since they
are used for the computation of the illumination. We propose
a new relighting approach for arbitrary shaped objects with
dynamic light environment using an RGB-D camera such as
the Microsoft’s Kinect. The generated depth map is useful to
estimate the normals of the object, but it’s inaccurate because of
the noise such as discrete depth values or missing data. We focus
on the depth map modification to segment the accurate object
region, and normal estimation for relighting. Our implementation
achieves a relighting at 10fps with GPU. Therefore, our relighting
system can apply for a moving object and a dynamic light
environment in real time.

Speed up in Computation of HMAX Features for Object Localization

Takuya Minagawa, Hideo Saito

ITE Transactions on Media Technology and Applications 2 ( 1 ) 60 - 73 2014.01

Research paper (scientific journal), Accepted

While HMAX features have been proved to have excellent performance in image categorization tasks, the computational cost of recognition is expensive. If we aim to apply the HMAX features to object localization tasks, in which the categorization tasks are repeatedly performed by sliding windows, their processing time increases enormously. In this paper, we propose a method for speed up in computation of object localization based on HMAX features. We found that the HMAX features cause specific redundancies in the sliding window approach. The speed up is achieved by eliminating the redundancies in our method. The results from experiments using the University of Illinois-Urbana-Champaign (UIUC) car dataset and the face detection dataset benchmark (FDDB) indicate that this modification improved processing speeds significantly with insignificant reductions in precision.

Access to Document (DOI)

Plane approximation-based approach for 3d reconstruction of ball trajectory for performance analysis in table tennis

Tamaki Sho, Saito Hideo

Advances in Computer Vision and Pattern Recognition 71 47 - 66 2014

Research paper (scientific journal), Joint Work, ISSN 2191-6586

A plane approximation-based approach for 3D reconstruction of ball trajectory is introduced, which was developed as a solution to the problem with conventional analysis in table tennis. There are methods of reconstructing 2D ball trajectories or 3D trajectories of balls heavier than those in table tennis. However, these methods cannot be adopted to reconstruct the 3D trajectories of table tennis balls, because there are problems that are attributed to the dimensions of the trajectories and weight of the balls. The method proposed in this chapter could reconstruct the 3D trajectories of a table tennis ball. The key feature of the method is that it approximates that a ball is traveling on tilted planes. This approximation makes reconstruction robust against failure to measure 3D ball positions. Two systems were developed based on the new method. A system using two RGB cameras experimentally demonstrated that it could provide accurate information for match analysis. A system using an RGB-D camera was experimentally demonstrated that the system could provide accurate information for service analysis.

Access to Document (DOI)

A Tracking Method for 2D Canvas in MR-Based Interactive Painting System

Sandy Martedi, Mai Otsuki, Hideo Saito, Maki Sugimoto, Asako Kimura, Fumihisa Shibata

International Conference on Signal-Image Technology & Internet-Based Systems (SITIS2013) (IEEE) 790 - 794 2013.12

Research paper (international conference proceedings), Accepted

We have proposed a mixed reality based painting system. In this paper, we tackle a problem in our previous painting system that fully relies on magnetic sensor that is attached on the canvas, the users needed to detach and attach a sensor on the canvas during painting when they want to switch the canvas. Instead of that, in this paper, we aim to automatically detect the shape of the canvas for registration purpose. Using the shape or region detection method such as MSER (maximally stable extremal regions), we detect and track the shape on the canvas on the captured camera image. We then compute the camera pose for virtually overlay the painting result. Using the brush device, we can draw and paint freely on the tracked canvases. We show that using visual based tracking method, we can generate the equivalent result compared to the result of using the sensor.

Access to Document (DOI)

Arbitrarily Shaped Objects Relighting Using an RGB-D Camera

Takuya Ikeda, Francois de Sorbier, Hideo Saito

2nd IAPR Asian Conference on Pattern Recognition (ACPR), 2013 631 - 636 2013.11

Research paper (international conference proceedings), Accepted

Access to Document (DOI)

Detecting Fall Incidents of the Elderly Based on Human-Ground Contact Areas

Dao Huu Hung, Hideo Saito, Gee-Sern Hsu

2nd IAPR Asian Conference on Pattern Recognition (ACPR), 2013 516 - 521 2013.11

Research paper (international conference proceedings), Accepted

Access to Document (DOI)

パフォーマンス分析への応用を目的とした卓球ボールの3次元軌跡復元システム

SAITO HIDEO

精密工学会誌 79 ( 11 ) 1102 - 1109 2013.11

Research paper (scientific journal), Joint Work, Accepted

Virtual Slicer: Development of Interactive Visualizer for Tomographic Medical Images Based on Position and Orientation of Handheld Device

Sho Shimamura, Motoko Kanegae, Yuji Uema, Masahiko Inami, Tetsu Hayashida, Hideo Saito, Maki Sugimoto

International Conference on Cyberworlds (CW), 2013 (IEEE) 383 - 383 2013.10

Research paper (international conference proceedings), Accepted

Access to Document (DOI)

Region-based tracking using sequences of relevance measures

SAITO HIDEO

IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2013 (IEEE) 1 - 6 2013.10

Research paper (international conference proceedings), Accepted

We present the preliminary results of our proposal: a region-based detection and tracking method of arbitrary shapes. The method is designed to be robust against orientation and scale changes and also occlusions. In this work, we study the effectiveness of sequence of shape descriptors for matching purpose. We detect and track surfaces by matching the sequences of descriptor so called relevance measures with their correspondences in the database. First, we extract stable shapes as the detection target using Maximally Stable Extreme Region (MSER) method. The keypoints on the stable shapes are then extracted by simplifying the outline of the stable regions. The relevance measures that are composed by three keypoints are then computed and the sequences of them are composed as descriptors. During runtime, the sequences of relevance measures are extracted from the captured image and are matched with those in the database. When a particular region is matched with one in the database, the orientation of the region is then estimated and virtual annotations can be superimposed. We apply this approach in an interactive task support system that helps users for creating paper craft objects.

Access to Document (DOI)

An Automatic Broadcast System for a Weather Report Radio Program

Hiroyuki Segi, Reiko Takou, Nobumasa Seiyama, Tohru Takagi, Yuko Uematsu, Hideo Saito, Shinji Ozawa

IEEE Transactions on Broadcasting (IEEE) 59 ( 3 ) 548 - 555 2013.09

Research paper (scientific journal), Accepted, ISSN 0018-9316

Access to Document (DOI)

From smartphone to virtual window

Zhang, E.; Saito, H. ; De Sorbier, F.

IEEE International Conference on Multimedia and Expo Workshops (ICMEW), 2013 (IEEE) 1 - 6 2013.07

Research paper (international conference proceedings), Accepted

We propose a vehicle make and model recognition method for the smart phone, and implemented it onto them. Our method identifies the make and model from the variety of viewpoints while the conventional methods for VMMR work only for the frontal or rear view images. This method enables the users to take the pictures from a certain range of angles. Our method uses SIFT, that has scale and rotation invariance to solve the alignment issue. It creates the pseudo frontal view images by homography matrix and extracts the keypoints. Homography matrix is calculated with the position of the license plate. Our experimental result shows our method enables to recognize up to 60-degree angle.

Access to Document (DOI)

Vehicle make and model recognition by keypoint matching of pseudo frontal view

Yukiko Shinozuka, Ruiko Miyano, Takuya Minagawa, Hideo Saito

IEEE International Conference on Multimedia and Expo Workshops (ICMEW), 2013 (IEEE) 1 - 6 2013.07

Research paper (international conference proceedings), Accepted

Access to Document (DOI)

Reconstruction of 3D Trajectories for Performance Analysis in Table Tennis

Sho Tamaki, Hideo Saito

IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2013 (IEEE) 1019 - 1026 2013.06

Research paper (international conference proceedings), Accepted

A method of reconstructing the 3D trajectories of a table tennis ball is introduced, which was developed to solve the problem with conventional analysis in table tennis. There are several methods of reconstructing 2D ball trajectories or 3D trajectories of balls heavier than those in table tennis. However, these methods cannot be adopted to reconstruct the 3D trajectories of table tennis balls, because there are problems that are attributed to the dimensions of the trajectories and weight of the balls. The method proposed in this paper could reconstruct the 3D trajectories of a table tennis ball. The key feature of the method is that it approximates that a ball is traveling on tilted planes. This approximation makes reconstruction robust against failure to measure 3D ball positions. A system using two RGB cameras was developed based on the new method. The system experimentally demonstrated that it could provide accurate information for match analysis. A system using an RGB-D camera was then developed to optimize usability for practitioners. We experimentally demonstrated that the system could provide accurate information for service analysis.

Access to Document (DOI)

Displayed Object Recognition for Smartphone Interaction

Karim Kadar, Francois de Sorbier, Hideo Saito

13th IAPR International Conference on Machine Vision Applications (MVA 2013) 105 - 108 2013.05

Research paper (international conference proceedings), Joint Work, Accepted

A Training Support System for Table Tennis Service using Kinect

Tamaki Sho, Nakamura Jun, Saito Hideo

The 13th ITTF Sports Science Congress 2013.05

Research paper (international conference proceedings), Joint Work, Accepted

Illumination Estimation from Object Shadow and Incomplete Object Shape Information Captured by an RGB-D Camera

Ikeda Takuya, Oyamada Yuji, Sugimoto Maki, Saito Hideo

The journal of the Institute of Image Information and Television Engineers (社団法人映像情報メディア学会) 67 ( 4 ) J124-J133 2013.03

Research paper (scientific journal), Joint Work, Accepted, ISSN 13426907

Feature-based Alignment Method for Projecting Virtual Content on a Movable Paper Map

Sandy Martedi, Maki Sugimoto, Hideo Saito, Bruce Thomas

IEEJ Trans. on Electronics, Information, and Systems (電気学会) 133 ( 3 ) 672-679 2013.03

Research paper (scientific journal), Joint Work, Accepted

Stable Keypoint Recognition using Viewpoint Generative Learning

Takumi Yoshida, Hideo Saito, Masayoshi Shimizu and Akinori Taguchi

the International Conference on Computer Vision Theory and Applications(VISAPP2013) (The Institute for Systems and Technologies of Information, Control and Communication (INSTICC)) 2 310 - 315 2013.02

Research paper (international conference proceedings), Joint Work, Accepted

We propose a stable keypoint recognition method that is robust to viewpoint changes. Conventional local
features such as SIFT, SURF, etc., have scale and rotation invariance but often fail in matching points when
the camera pose significantly changes. In order to solve this problem, we adopt viewpoint generative learning.
By generating various patterns as seen from different viewpoints and collecting local invariant features, our
system can learn feature descriptors under various camera poses for each keypoint before actual matching. Ex-
perimental results comparing usual local feature matching or patch classification method show both robustness
and fastness of learning.

A Mobile AR System for Sports Spectators using Multiple Viewpoint Cameras

Ruiko Miyano, Takuya Inoue, Takuya Minagawa, Yuko Uematsu, Hideo Saito

the International Conference on Computer Vision Theory and Applications(VISAPP2013) (The Institute for Systems and Technologies of Information, Control and Communication (INSTICC)) 1 23 - 32 2013.02

Joint Work, Accepted

In this paper, we aimto develop an AR system which supports spectators who are watching a sports game using
smartphones in a spectators’ stand. The final goal of this system is that a spectator can watch information of
players through a smartphone and share experiences with other spectators. For this goal, we propose a system
which consists of smartphones and fixed cameras. Fixed cameras are set to cover the whole sports field and
used to analyze players. Smartphones held by spectators are used to estimate positions where they are looking
on the sports field. We built an AR system which makes annotation of players’ information onto a smartphone
image. And we evaluated the accuracy and the processing time of our system and revealed its practicality.

Non-rigid Surface Tracking for Virtual Fitting System

Naoki Shimizu, Takumi Yoshida, Tomoki Hayashi, Francois de Sorbier, Hideo Saito

the International Conference on Computer Vision Theory and Applications(VISAPP2013) (The Institute for Systems and Technologies of Information, Control and Communication (INSTICC)) 2 12 - 18 2013.02

Research paper (international conference proceedings), Joint Work, Accepted

In this paper, we describe a method for overlaying a texture onto a T-shirt, for improving current virtual fitting
system. In such systems, users can try on clothes virtually. In order to realize such a system, a depth camera
has been used. These depth cameras can capture 3D data in real time and have been used by some industrial
virtual cloth fitting systems. However, these systems roughly, or just do not, consider the shape of the clothes
that user is wearing. So the appearance of these virtual fitting systems looks unnaturally. For a better fitting,
we need to estimate 3D shape of cloth surface, and overlay a texture of the cloth that the user wants to see
onto the surface. There are some methods that register a 3D deformable mesh onto captured depth data of a
target surface. Although those registration methods are very accurate, most of them require large amount of
processing time or either manually-set markers or special rectangles. The main contribution of our method is
to overlay a texture onto a texture of T-shirt in real-time without modifying the surface.

The Estimation of Heights and Occupied Areas of Humans from Two Orthogonal Views for Fall Detection

Dao Huu Hung and Hideo Saito

IEEJ Trans. on Electronics, Information, and Systems (電気学会) 133 ( 1 ) 117-127 2013.01

Research paper (scientific journal), Joint Work, Accepted

Violin pedagogy for finger and bow placement using augmented reality

F. de Sorbier, H. Shiino, H. Saito

Asia-Pacific Signal & Information Processing Association Annual Summit and Conference (APSIPA ASC 2012) (IEEE) 1 - 5 2012.12

Research paper (international conference proceedings), Joint Work, Accepted

Beginners need a long time before being able to
play correctly violin. Learning the bowing technique appears to
be a difficult task and retains most of the attention of beginners.
Besides this point, the finger placement is also an important part
of the learning but often under estimated. One difficulty is that
the fingerboard of the violin does not have frets. In this on-going
work, we present a marker-less augmented reality system that
advises the novice players about their fingering and bowing. We
display in real-time the virtual frets by tracking the violin with a
depth camera. We also capture and recognize the note currently
played to direct the placement of the bow on the strings.

Resource-Aware Rendering for Laser Plasma Scanning 3D Display Devices Using Surface Descriptors

Nakatani Ayaka, Issei Fujishiro, Ishikawa Hiroyo, Saito Hideo

Transactions on Virtual Reality Society of Japan (日本バーチャルリアリティ学会) 17 ( 4 ) 419-428 2012.12

Research paper (scientific journal), Joint Work, Accepted

Diminished Reality using Real-Time Region Extraction with Depth Cameras

Naoki Shimizu, Takanori Hashimoto, Yuko Uematsu, Hideo Saito

The journal of the Institute of Image Information and Television Engineers (社団法人映像情報メディア学会) 66 ( 12 ) J549-J552 2012.11

Research paper (scientific journal), Joint Work, Accepted, ISSN 13426907

Toward an augmented reality system for violin learning support

Hiroyuki Shiino, François de Sorbier, Hideo Saito

Advances in Depth Image Analysis and Applications Lecture Notes in Computer Science (Springer Berlin Heidelberg) 7854 136 - 145 2012.11

Research paper (international conference proceedings), Joint Work, Accepted, ISSN 0302-9743

Violin is one of the most beautiful but also one of the most difficult musical instruments for a beginner. This paper presents an on-going work about a new augmented reality system for training how to play violin. We propose to help the players by virtually guiding the movement of the bow and the correct position of their fingers for pressing the strings. Our system also recognizes the musical note played and the correctness of its pitch. The main benefit of our system is that it does not require any specific marker since our real-time solu-tion is based on a depth camera.

Access to Document (DOI)

Camera Pose Estimation of a Smartphone at a Field without Interest Points

Ruiko Miyano, Takuya Inoue, Takuya Minagawa, Yuko Uematsu, Hideo Saito

Computer Vision - ACCV 2012 Workshops Lecture Notes in Computer Science (Springer Berlin Heidelberg) 7729 545 - 555 2012.11

Research paper (international conference proceedings), Joint Work, Accepted, ISSN 0302-9743

Access to Document (DOI)

Colour Descriptors for Tracking in Spatial Augmented Reality

Thijs Kooi, Francois de Sorbier, Hideo Saito

Computer Vision - ACCV 2012 Workshops Lecture Notes in Computer Science (Springer Berlin Heidelberg) 7729 387 - 399 2012.11

Research paper (international conference proceedings), Joint Work, Accepted, ISSN 0302-9743

Augmented Reality is an emerging research field, that aims
for the composition of real and virtual imagery, by means of a camera
and display device. Spatial augmented reality employs data projectors
to augment the real world. In this setting, traditional tracking methods
fall short due to the interference caused by the projector. Recent works
assume a calibration process to model the projector and assume continuity
in movement of the object being tracked. In this paper we present
a tracking-by-detection system that does not require such a procedure
and makes use of natural features represented by SIFT descriptors. We
evaluate a set of photometric invariants that have previously been shown
to improve the performance of object recognition, added to the descriptor
to reduce the influence of the projector. We evaluate the descriptors
based on precision-recall under projector distortion and the total system
based on its tracking performance. Results show tracking is significantly
more precise using one of the invariants.

Access to Document (DOI)

Illumination Estimation from Shadow and Incomplete Object Shape Using an RGB-D Camera

Takuya Ikeda, Yuji Oyamada, Maki Sugimoto, and Hideo Saito

International Conference on Pattern Recognition (ICPR2012) (IEEE) 165 - 169 2012.11

Research paper (international conference proceedings), Joint Work, Accepted, ISSN 1051-4651

Optical consistency between the real world and the
virtual objects is one of the important issues in Augmented
Reality (AR). This paper proposes a method
to estimate illuminations from an object shadow and
incomplete object shape information captured by an
RGB-D camera. The environmental illumination can
be estimated without any prior knowledge of the object
shape. The radiance of each light source is computed by
solving linear equations derived from color and depth
images. Since a priori knowledge of object shape is
not necessary, we can flexibly use/combine any objects.
Thus, we can use/combine multiple objects in the target
scene to increase the accuracy and flexibility. Experimental
results show the characteristics of our proposed
method.

Calibration-Free Projector-Camera System for Spatial Augmented Reality on Planar Surfaces

Takayuki Nakamura, Francois de Sorbier, Sandy Martedi, Hideo Saito

International Conference on Pattern Recognition (ICPR2012) (IEEE) 85 - 88 2012.11

Research paper (international conference proceedings), Joint Work, Accepted, ISSN 1051-4651

Spatial augmented reality extends augmented real-
ity by projecting virtual data directly on a target sur-
face, but requires to calibrate a projector-camera sys-
tem. This paper introduces a free-calibration projector-
camera system for spatial augmented reality with a pla-
nar surface. A pattern is projected on the target surface
that can be freely moved. The main difficulty is to make
the projected image fitting the target surface. This is of-
ten achieved by calibrating the projector according to
the camera’s pose, but the calibration process is offline
and restricts the configuration of the setup. Our pro-
posed method allows to independently move the projec-
tor, the camera or the surface while the projected image
continuously fit the target surface. The experimental re-
sults demonstrate the efficiency of our calibration-free
spatial augmented reality approach when applied on a
moving planar surface.

Visualization and Image Analysis: (2) 3D Shape Recovery Technique in Computer Vision and its Applications

SAITO HIDEO

Ｍｅｄｉｃａｌ　Ｉｍａｇｉｎｇ　Ｔｅｃｈｎｏｌｏｇｙ (日本医用画像工学会) 30 ( 4 ) 209-214, 2012.10

Research paper (scientific journal), Single Work

Real-Time Diminished Reality using Multiple Smartphone Cameras

Honda Toshihiro, Saito Hideo

Transactions on Virtual Reality Society of Japan (日本バーチャルリアリティ学会) 17 ( 3 ) 181-190 2012.09

Research paper (scientific journal), Joint Work, Accepted

Real-time SLAM Using New Line Descriptor (LEHF)

Hirose Keisuke, Saito Hideo

Transactions on Virtual Reality Society of Japan (日本バーチャルリアリティ学会) 17 ( 3 ) 201-208 2012.09

Research paper (scientific journal), Joint Work, Accepted

Robust Planar Pose Estimation by Viewpoint Generative Learning

Hirose Keisuke, Saito Hideo

Transactions on Virtual Reality Society of Japan (日本バーチャルリアリティ学会) 17 ( 3 ) 191-200 2012.09

Research paper (scientific journal), Joint Work, Accepted

Fast Line Description for Line-based SLAM

Keisuke Hirose, Hideo Saito

Proceedings of the British Machine Vision Conference (BMVA Press) 83.1 - 83.11 2012.09

Research paper (international conference proceedings), Joint Work, Accepted

Simultaneous localization and mapping (SLAM) is a technique to simultaneously
perform mapping of environments and localization of a camera in real-time. Most existing
monocular vision based SLAM techniques use point features as landmarks. However,
images of artificial environments with little texture often contain many line segments,
whereas few point features can be localized in such a scene. We propose here a realtime
line-based SLAM system, and a novel method for describing the features of line
segments (LEHF:Line-based Eight-directional Histogram Feature) in order to establish
correct 2D and 3D line correspondences (2D-3D correspondences). LEHF is a fast and
efficient way of describing features of line segments, which are detected by the line segment
detector (LSD) method. The line-based orthogonal iteration (LBOI) method and
the RANSAC algorithm are applied for the camera pose estimation. We conducted an
experiment in order to test our SLAM system in a desktop environment and to perform
augmented reality (AR). Moreover our SLAM system was evaluated by synthetic data.

Access to Document (DOI)

Automatic Road Area Extraction from Printed Maps Based on Linear Feature Detection

Sebastien CALLIER Hideo SAITO

IEICE TRANSACTIONS on Information and Systems (電子情報通信学会) E95-D ( 7 ) 1758-1765 2012.07

Research paper (scientific journal), Joint Work, Accepted

Real-Time People Counting in Crowded Areas by using Local Empirical Templates and Density Ratios

Dao Huu Hung, Sheng-Luen Chung, Gee-Sern Hsu, and Hideo Saito

IEICE TRANSACTIONS on Information and Systems (電子情報通信学会) E95-D ( 7 ) 1791 - 1803 2012.07

Research paper (scientific journal), Joint Work, Accepted

全方位カメラを用いたテーブルトップ型ARシステム

SAITO HIDEO

画像ラボ 7-15 2012.05

Research paper (scientific journal), Joint Work

FOOD REGION DETECTION USING BAG-OF-FEATURES REPRESENTATION AND COLOR FEATURE

Ruiko Miyano, Yuko Uematsu, SAITO HIDEO

International Conference on Computer Vision Theory and Applications 2012 (VISAPP2012) (SciTePress) 1 709 - 713 2012.02

Research paper (international conference proceedings), Joint Work, Accepted

TEXTURE OVERLAY ONTO NON-RIGID SURFACE USING COMMODITY DEPTH CAMERA

Tomoki Hayashi, Francois de Sorbier, SAITO HIDEO

International Conference on Computer Vision Theory and Applications 2012 (VISAPP2012) (SciTePress) 2 66 - 71 2012.02

Research paper (international conference proceedings), Joint Work, Accepted

We present a method for overlaying a texture onto a non-rigid surface using a commodity depth camera. The
depth cameras are able to capture 3-D data of a surface in real-time, and have several advantages compared
with methods using only standard color cameras. However, it is not easy to register a 3-D deformable mesh to a
point cloud of the non-rigid surface while keeping its geometrical topology. In order to solve this problem, our
method starts by learning many representative meshes to generate surface deformation models. Then, while
capturing 3-D data, we register a feasible 3-D mesh to the target surface and overlay a template texture onto
the registered mesh. Even if the depth data are noisy or sparse, the learning-based method provides us with a
smooth surface mesh. In addition, our method can be applied to real-time applications. In our experiments,
we show some augmented reality results of texture overlay onto a non-textured T-shirt.

Foldable Augmented Maps

MARTEDI Sandy, UCHIYAMA Hideaki, ENRIQUEZ Guillermo, SAITO HIDEO, MIYASHITA Tsutomu, HARA Takenori

IEICE TRANSACTIONS on Information and Systems (電子情報通信学会) E95-D ( 1 ) 256-266 2012.01

Research paper (scientific journal), Joint Work, Accepted, ISSN 1745-1361

Camera tracking by online learning of keypoint arrangements using LLAH in augmented reality applications

Uchiyama Hideaki, SAITO HIDEO, Servières Myriam, Moreau Guillaume

Virtual Reality (Springer) 15 ( 2-3 ) 109-117 2011.10

Research paper (scientific journal), Joint Work, Accepted

A novel Inpainting-based Layered Depth Video for 3DTV

I. Daribo, SAITO HIDEO

IEEE TRANSACTIONS ON BROADCASTING (IEEE) 57 ( 2 ) 533-541 2011.06

Research paper (scientific journal), Joint Work, Accepted

Estimation of Rotational Velocity of Baseball Using High-Speed Camera Movies

INOUE TAKUYA, UEMATSU YUKO, SAITO HIDEO

The transactions of the Institute of Electrical Engineers of Japan. D, A publication of Industry Applications Society (社団法人電気学会) 131 ( 4 ) 608-615 2011.04

Research paper (scientific journal), Joint Work, Accepted, ISSN 09136339

See-Through View Generation from Multi-Camera Baseball Movies

HASHIMOTO TAKANORI, UEMATSU YUKO, SAITO HIDEO

The journal of the Institute of Image Information and Television Engineers (社団法人映像情報メディア学会) 65 ( 4 ) 505-513 2011.04

Research paper (scientific journal), Joint Work, Accepted, ISSN 13426907

Dynamics Analysis of Facial Expression Changes for Person Identification

TANAKA HIDENORI, SAITO HIDEO

The transactions of the Institute of Electrical Engineers of Japan. C, A publication of Electronics, Information and System Society (社団法人電気学会) 130 ( 11 ) 2047-2057 2010.11

Research paper (scientific journal), Joint Work, Accepted, ISSN 03854221

Real-time video-based rendering from uncalibrated cameras using plane-sweep algorithm

Songkran Jarusirisawad, Vincent Nozick, Hideo Saito

Journal of Visual Communication and Image Representation (Elsevier) 21 577 - 585 2010.01

Research paper (scientific journal), Accepted

In this paper, we present a new online video-based rendering (VBR) method that creates new views of a
scene from uncalibrated cameras. Our method does not require information about the cameras intrinsic
parameters. For obtaining a geometrical relation among the cameras, we use projective grid space (PGS)
which is 3D space defined by epipolar geometry between two basis cameras. The other cameras are registered
to the same 3D space by trifocal tensors between these basis cameras. We simultaneously reconstruct
and render novel view using our proposed plane-sweep algorithm in PGS. To achieve real-time
performance, we implemented the proposed algorithm in graphics processing unit (GPU). We succeed
to create novel view images in real-time from uncalibrated cameras and the results show the efficiency
of our proposed method.

Access to Document (DOI)

光源マーカを用いた可視光通信に基づく写真測量システム

内山英昭、吉野昌樹、春山真一郎、斎藤英雄、中川正雄、掛橋孝夫、永元直樹

画像電子学会誌、Vol. 38、No.5、pp. 703-711、2009 2009

Accepted

3DTV View Generation Using Uncalibrated Pure Rotating and Zooming Cameras

Songkran Jarusirisawad and Hideo Saito

the Special Issue on Advances in Three Dimensional Television and Video of the Journal Signal Processing: Image Communication 24 17-30 2009

Research paper (scientific journal), Joint Work, Accepted

Multiple Planes Based Registration Using 3D Projective Space for Augmented Reality

''Yuko Uematsu, Hideo Saito''

''Journal of Image and Vision Computing, Vol.27, pp.1484-1496, 2009.'' 2009

Joint Work, Accepted

Model-Based Hand Tracking by Chamfer Distance and Adaptive Color Learning Using Particle Filter

Chutisant Kerdvibulvech and Hideo Saito

''EURASIP Journal on Image and Video Processing, Volume 2009 (2009), Article ID 724947, 10 pages'' 2009

Joint Work, Accepted

Calibration of a Structured Light System Using Planar Objects

''Koichiro Yamauchi, Hideo Saito, and Yukio Sato''

''In Peng-Yeng Yin, editor. Pattern Recognition pp. 1-14, IN-TECH, Vienna, Austria, October 2009'' 2009

Joint Work, Accepted

フリーハンド撮影された多視点スナップショットからの顔形状復元

鈴木誠司，斎藤英雄，持丸正明

''映像情報メディア学会誌，Vol.63,No.4, pp.506-515,2009'' 2009

Accepted

レーザプラズマ式３Dディスプレイにおける点列を用いた物体表現

石川尋代，斎藤英雄

''映像情報メディア学会誌，Vol.63,No.5, pp.665-672,2009'' 2009

Accepted

画像クエリによる情報検索システム構築のための階層型オブジェクトカテゴリ認識手法

皆川卓也，斎藤英雄

''電気学会論文誌C，Volume 129-C, Number 5, pp.947-955, 2009'' 2009

Accepted

基準平面を用いたスリット光投影式レンジファインダの校正

山内航一郎，斎藤英雄，佐藤幸男

''電気学会論文誌C，Volume 129-C, Number 6, pp.1162-1168, 2009'' 2009

Accepted

単眼カメラを用いた車両の実時間蛇行検出手法

手島知昭，斎藤英雄，小澤慎治(愛知工科大学)，山本恵一 (三菱ふそう)，伊原　徹(三菱ふそう)

''電気学会論文誌C，Volume 129-C, Number 9, pp.1714-1723, 2009'' 2009

Accepted

時間・空間的にまばらな多視点画像を用いた個人識別のための動的な見え方学習法

田中秀典、北原格、斎藤英雄、村瀬洋、小暮潔、萩田紀博

画像電子学会誌、Vol. 38、No.5、pp. 623-633、2009 2009

Accepted

レスキュー用アレイアンテナ搭載レーダシステム

''竹内敏夫，斎藤英雄，大矢晃久,青木義満，松野文俊，秋山いわき''

''計測自動制御学会論文集, Vol.44, No.12, pp.962-969, 2008'' 2008

Accepted

Calibration of a Structured Light System by Observing Planar Object

''Koichiro Yamauchi, Hideo Saito and Yukio Sato''

''International Conference on Pattern Recognition, ICPR08, pp.1-4, December, 2008'' 2008

Joint Work, Accepted

PCA-based 3D Shape Reconstruction of Human Foot Using Multiple Viewpoint Cameras

''Edmee Amstutz, Tomoaki Teshima, Makoto Kimura, Masaaki Mochimaru, Hideo Saito''

''International Journal of Automation and Computing, Volume 5, Issue 3, pp. 217-225, July 2008.'' 2008

Joint Work, Accepted

On-line Free-Viewpoint Video: From Single to Multiple View Rendering

''Vincent Nozick, Hideo Saito''

''International Journal of Automation and Computing, Volume 5, Issue 3, pp. 257-265, July 2008.'' 2008

Research paper (conference, symposium, etc.), Joint Work, Accepted

Visual Enhancement for Sports Entertainment by Vision-Based Augmented Reality

Yuko Uematsu and Hideo Saito

''Advances in Human-Computer Interaction, Volume 2008 (2008), Article ID 145363, 14 pages.'' 2008

Joint Work, Accepted

AR Supporting System for Pool Games Using a Camera Mounted Handheld Display

Hideaki Uchiyama and Hideo Saito

''Advances in Human-Computer Interaction, Volume 2008 (2008), Article ID 357270, 13 pages'' 2008

Joint Work, Accepted

Guitarist Fingertip Tracking by Integrating a Bayesian Classifier into Particle Filters

Chutisant Kerdvibulvech and Hideo Saito

''Advances in Human-Computer Interaction, Volume 2008 (2008) , Article ID 384749, 10 pages.'' 2008

Joint Work, Accepted

ギター演奏支援のための構造特徴追跡を利用したマーカレスAR表示

元川洋一，斎藤英雄

''日本バーチャルリアリティ学会論文，Vol. 13, No. 2 pp.267-278, 2008'' 2008

Accepted

複数のハンディカメラの協調利用による遮蔽物体除去映像のオンライン生成

榎本暁人，斎藤英雄

''映像情報メディア学会誌，Vol.62,No.6, pp.901-908,2008'' 2008

Accepted

鼻孔のテンプレート探索とパーティクルフィルタによるドライバの頭部姿勢追跡

久保田整(ズズキ），武士正美(ズズキ），斎藤英雄

''電気学会論文誌C, Volume 128-C Number 9, pp.1447-1454, 2008'' 2008

Accepted

プロジェクタスクリーン上に生じる焦点ボケ解消のための投影画像の事前補正法

小山田雄仁，斎藤英雄

''日本バーチャルリアリティ学会論文誌, vol.12, No.4, pp.479-486, 2007'' 2007

Accepted

Online Multiple View Computation for Autostereoscopic Display

''Vincent Nozick, Hideo Saito''

''Proc. 2007 Pacific Rim Symposium on Image Video and Technology (PSIVT2007), Lecture Note in Computer Science 4872, pp. 399-412, 2007.'' 2007

Joint Work, Accepted

Free viewpoint video synthesis and presentation from multiple sporting videos

''Naho Inamoto, Hideo Saito''

''Electronics and Communications in Japan (Part III: Fundamental Electronic Science), Volume 90, Issue 2, pp. 40-49, February 2007.'' 2007

Research paper (conference, symposium, etc.), Joint Work, Accepted

Real-Time Guitar Chord Recognition System Using Stereo Cameras for Supporting Guitarists

''Chutisant Kerdvibulvech, Hideo Saito''

''ECTI TRANSACTIONS ON ELECTRICAL ENG., ELECTRONICS, AND COMMUNICATIONS VOL.5, NO.2, pp.147-157, August 2007.'' 2007

Joint Work, Accepted

Free Viewpoint Video Synthesis based on Natural Features Using Uncalibrated Moving Cameras

''Songkran Jarusirisawad, Hideo Saito''

''ECTI TRANSACTIONS ON ELECTRICAL ENG., ELECTRONICS, AND COMMUNICATIONS VOL.5, NO.2, pp.181-190, August 2007.'' 2007

Joint Work, Accepted

Virtual Viewpoint Replay for a Soccer Match by View Interpolation From Multiple Cameras

''Naho Inamoto, Hideo Saito''

''IEEE Trans. Multimedia, Vol.9, Issue 6, pp. 1155-1166, October 2007.'' 2007

Joint Work, Accepted

拡張現実表示技術を用いたギターの演奏支援システム

元川洋一，斎藤英雄

''映像情報メディア学会誌，Vol.61,No.6, pp.789 −796, 2007'' 2007

Accepted

ハンディカメラ入力からのシーン解析に基づくビリヤード戦略発想支援用AR表示システム

内山英昭，斎藤英雄

''日本バーチャルリアリティ学会論文誌，VOL.12，NO.2，pp.159−170, 2007'' 2007

Accepted

任意位置姿勢に配置された複数マーカの統合によるオンラインARシステム

植松裕子，斎藤英雄

''映像情報メディア学会誌，Vol.60, No.6, 2006.'' 2006

Accepted

Calibration Free Virtual Display System Using Video Projector onto Real Object Surface

''Shinichiro HIROOKA, Hideo SAITO''

''IEICE TRANSACTIONS on Information and Systems, Volume E89-D, No.1,pp.88-97, 2006'' 2006

Joint Work, Accepted

Improving Quality of Free-Viewpoint Image by Mesh Based 3D Shape Deformation

S.Yaguchi，H.Saito

''Journal of WSCG, Vol. 14, No. 1-3, pp. 57-64, 2006'' 2006

Joint Work, Accepted

Human Foot Reconstruction from Multiple Camera Images with Foot Shape Database

''Jiahui WANG, Hideo SAITO, Makoto KIMURA, Masaaki MOCHIMARU, Takeo KANADE''

''IEICE TRANSACTIONS on Information and Systems, Volume E89-D, No.5, pp.1732-1742, 2006'' 2006

Joint Work, Accepted

平面領域抽出に基づく距離画像の位置合わせ法

渡辺大樹，斎藤英雄

''映像情報メディア学会誌，Vol.60, No.4, pp.576-579, 2006.'' 2006

Accepted

未校正多視点カメラからの自由視点画像生成のための擬似正射影グリッド空間

磯大輔，斎藤英雄

''画像電子学会誌，Vol.34, No.5, pp.512-521, 2005.'' 2005

Accepted

未校正カメラ及びプロジェクタによるディジタルコンテンツの実物展示システム

広岡慎一郎，斎藤英雄

''画像電子学会誌，Vol.34, No.1, pp.45-52, 2005.'' 2005

Accepted

視点間の射影幾何的関係を利用した多視点動画像のデータ圧縮

秋山修，斎藤英雄

''映像情報メディア学会誌，Vol.59, No.1, pp.136-145. 2005.'' 2005

Accepted

胴体に装着したステレオカメラを用いたゲーム操作用3次元ハンドコントローラ

保利栄作・斎藤英雄

''知能と情報（日本知能情報ファジィ学会誌） Vol.17 , No.2, pp.58-66, 2005年4月'' 2005

Accepted

3次元動的輪郭モデルによる皮膚内部組織3次元画像からの境界面検出

王　加輝，斎藤英雄，小沢慎治，桑原智裕，山下豊信，高橋元次

''電気学会論文誌C(電子・情報・システム部門誌) 2005年 Vol. 125-C No.5, pp.756-764,2005.'' 2005

Accepted

プロジェクタと2台のカメラを用いた任意位置姿勢平面への手書きパターンの入力・表示システム

伴　仁志，斎藤英雄

''日本バーチャルリアリティ学会論文誌，投影型インタフェース特集, Vol.10 No.2, pp.173-182, 2005.'' 2005

Accepted

多視点スポーツ映像からの自由視点映像合成と提示

稲本奈穂，斎藤英雄

''電子情報通信学会論文誌D-II, Vol.J88-D-II, No.8, pp.1693-1701, 2005.'' 2005

Accepted

全方位カメラ画像からの光線空間構築による任意視点画像生成

等々力寛，斎藤英雄

''映像情報メディア学会誌，Vol.58, No.10, pp.489-1496, 2004.'' 2004

Accepted

Arbitrary Viewpoint Video Synthesis from Multiple Uncalibrated Cameras

S.Yaguchi，H.Saito

''IEEE Trans. on Systems, Man and Cybernetics, PartB, vol. 34, no.1, pp.430-439, Feb. 2004.'' 2004

Joint Work, Accepted

Arbitrary viewpoint image synthesis based on light field construction from uncalibrated image sequence

''Kenkichi Kobayashi, Hideo Saito''

''Electronics and Communications in Japan (Part III: Fundamental Electronic Science), Volume 87, Issue 9, pp.11-21, September 2004.'' 2004

Joint Work, Accepted

視点位置の内挿に基づく３次元サッカー映像の自由視点観賞システム

稲本奈穂，斎藤英雄

''映像情報メディア学会誌，Vol.58, No.4, pp.529-539, 2004.'' 2004

Accepted

多視点画像からの薄板状柔軟物体の形状と姿勢の推定

久保田　整，小野勝一，武士正美，斎藤英雄

''電気学会論文誌C(電子・情報・システム部門誌), Vol. 124-C, No.3, pp.659-665, 2004.'' 2004

Accepted

単眼カメラを用いたインストゥルメンタルパネル操作用ハンドシェープスイッチの構築

五十嵐弘也，斎藤英雄，小沢慎治，山本恵一，佐藤広充

''電気学会論文誌C(電子・情報・システム部門誌), Vol. 124-C, pp.1820-1826, 2004.'' 2004

Accepted

因子分解法のための射影復元を利用した計測行列の補間法

上島重治，斎藤英雄

''画像電子学会誌，Vol.33, No.4-B,pp.576-585，2004.'' 2004

Accepted

HMDを用いた形状可変物体表面へのテクスチャ重畳表示システム

江森基倫，斎藤英雄

''日本バーチャルリアリティ学会論文誌，Vol.9 No.3, pp.309-317, 2004.'' 2004

Accepted

多視点テニス画像からのプレイヤー視点映像提示システム

木村健二，斎藤英雄

''映像情報メディア学会誌，Vol.58, No.10, pp.1503-1510, 2004.'' 2004

Accepted

Appearance-Based Virtual View Generation From Multicamera Videos Captured in the 3-D Room

''H.Saito, S.Baba, T.Kanade''

IEEE Trans. on Multimedia vol.5 ( no.3 ) 303-316 2003.09

Joint Work, Accepted

Stereo Matching between Three Images by Iterative Refinement in PVS

''M.Kimura, H.Saito, T.Kanade''

IEICE Transactions on Information and Systems Vol.E86-D ( No.1 ) 89-100 2003.01

Joint Work, Accepted

未校正の移動カメラによる複数平面シーンの超解像度画像生成

清真一朗，斎藤英雄

''電子情報通信学会論文誌D-II，VoL.J86-D-II, No.5, pp.688-696, May 2003'' 2003

Joint Work, Accepted

High-Resolution Image Synthesis from Video Sequence by Light Field

''Kenkichi Kobayashi, Hideo Saito''

Journal of Robotics and Mechatronics Vol.15 ( No.3 ) 254-262 2003

Joint Work, Accepted

Virtual Object Overlay onto Uncalibrated Camera Image Sequence Enabling Tracking of Natural Features

''Shunsuke Harasaki, Hideo Saito''

Journal of Robotics and Mechatronics Vol.15 ( No.3 ) 263-270 2003

Joint Work, Accepted

Surface Extraction of Dermo-epidermal Surface from 3-D Volumetric Images of Human Skin

''J.Wang, H.Saito, S.Ozawa, T.Yamashita, T.Kuwahara, M.Takahashi''

International Journal of Image and Graphics，Special Issue on　3D Vision and Medical Image Processing Vol. 3 ( No. 4 ) 589-608 2003

Joint Work, Accepted

画面一体型のマルチカメラシステムによる手指の三次元位置の推定

高橋伸昌，斎藤英雄

''計測自動制御学会論文集, Vol.39, No.1, Jan.2003'' 2003

Joint Work, Accepted

未校正画像列からの光線空間生成に基づく任意視点画像合成

小林賢吉，斎藤英雄

''電子情報通信学会論文誌D-II，VoL.J86-D-II, No.2, pp.272-281, Feb.2003'' 2003

Joint Work, Accepted

拡張現実感のための未校正カメラ画像への仮想物体の重ね合わせ

原崎俊介、斎藤英雄

'電子情報通信学会論文誌DII,Jan.2002' VOL.J85-D-II ( NO.1 ) 46-55 2002.01

Joint Work, Accepted

サッカーシーンにおける自由視点映像生成のための視点内挿法

稲本奈穂，斎藤英雄

''日本バーチャルリアリティー学会論文誌TVRSJ, Vol.7, No.4, pp.513-520, Dec.2002'' 2002

Joint Work, Accepted

Virtual 3-D Interface System via Hand Motion Recognition from Two Cameras

''K.Abe, H.Saito, S.Ozawa''

''IEEE Transactions on Systems, Man, and Cybernetics, Part A'' Vol.32 ( No.4 ) 536-540 2002

Joint Work, Accepted

拡張現実感のための未校正カメラ画像への仮想物体の重ね合わせ

原崎俊介，斎藤英雄

''電子情報通信学会論文誌D-II, VOL.J85-D-II, NO.1, pp.46-55, Jan.2002'' 2002

Joint Work, Accepted

距離情報とカラー情報の融合によるシルエット検出に基づく任意視点画像描画システム

''磯大輔, 斎藤英雄, 小沢慎治''

''画像電子学会誌，ビジュアルコンピューティング論文特集号，Vol.31, No.4, pp.525-533, Aug.2002.'' 2002

Joint Work, Accepted

平面射影変換を利用した多視点動画像の圧縮

''秋山　修，小沢慎治, 斎藤英雄''

''情報技術レターズ, Sep.2002.'' 2002

Joint Work, Accepted

射影幾何を利用した多視点画像からの中間視点映像生成

斎藤英雄，木村誠，矢口悟志，稲本奈穂

''情報処理学会コンピュータビジョンとイメージメディア研究会論文誌，Vol.43, No.SIG11, pp.21-32, Dec.2002'' 2002

Joint Work, Accepted

3D Reconstruction Based on Epipolar Geometry

'Makoto Kimura, Hideo Saito'

IEICE Transactions on Information and Systems Vol.E84-D ( No.12 ) 1690-1697 2001.12

Joint Work, Accepted

皮膚表面画像を用いた肌診断システム

竹前嘉修、斎藤英雄、小沢慎治

'計測自動制御学会論文集, 2001' Vol.37 ( No.11 ) 1097-1103 2001.11

Joint Work, Accepted

Recording Multiple Videos in a Large-scale Space for Large-scale Virtualized Reality

'I. Kitahara, Y.Ohta, H.Saito, S.Akimichi, T.Ono, T.Kanade'

'The 8th International Display Workshops, Nagoya, Oct.2001' 2001.10

Research paper (conference, symposium, etc.), Joint Work

サッカー放送における視点選択のための多視点画像の統合によるボール追跡

松本圭介、須藤智、斎藤英雄、小沢慎治

電気学会論文誌Ｃ Vol.121-C ( No.10 ) 1530-1539 2001.10

Joint Work, Accepted

Three-dimensional shape reconstructing system from multiple view images using octree and silhouette

'D. Iso, H. Saito, S. Ozawa'

'Intelligent Robots and Computer Vision XX, SPIE Conference 4572, Boston' 2001.10

Research paper (conference, symposium, etc.), Joint Work, Accepted

Three-dimensional shape model reconstruction from multiple view range images and color images

'K. Yoshida, H. Saito'

'Intelligent Robots and Computer Vision XX, SPIE Conference 4572, Boston' 2001.10

Research paper (conference, symposium, etc.), Joint Work, Accepted

Three-dimensional reconstruction of environment using factorization method and EPI analysis

'S. Kamijima, H. Saito'

'Intelligent Robots and Computer Vision XX, SPIE Conference 4572, Boston' 2001.10

Research paper (conference, symposium, etc.), Joint Work, Accepted

Robot motion tracking system with multiple views

'H. Yamano, H. Saito'

'Intelligent Robots and Computer Vision XX, SPIE Conference 4572, Boston' 2001.10

Research paper (conference, symposium, etc.), Joint Work, Accepted

Vision-based overlay of a virtual object into real scene for designing room interior

'S. Harasaki, H. Saito'

'Intelligent Robots and Computer Vision XX, SPIE Conference 4572, Boston' 2001.10

Research paper (conference, symposium, etc.), Joint Work, Accepted

Arbitrary View Video Generation from Multiple Camera System

Hideo Saito

'2001 International Technical Conference on Circuits/Systems, Computers and Communications,Vol.2' 672-675 2001.07

Research paper (conference, symposium, etc.), Single Work, Accepted

未校正多視点カメラシステムを用いた任意視点画像生成

矢口悟志、木村誠、斎藤英雄、金出武雄

情報処理学会コンピュータビジョンとイメージメディア研究会論文誌 Vol.42 ( No.SIG6(CVIM2) ) 9月21日 2001.06

Joint Work, Accepted

High-resolution image generation from video image sequence by light field

'K.Kobayashi, H.Saito'

Proceedings of SPIE Vol.4309 (Videometrics and Optical Methods for 3D Shape Measurement) 2001.01

Research paper (conference, symposium, etc.), Joint Work, Accepted

Interpolation of Three Views Based on Epipolar Geometry

'Makoto Kimura, Hideo Saito'

Proceedings of SPIE Vol.4310 (Visual Communication and Image Processing 2001) 218-227 2001.01

Research paper (conference, symposium, etc.), Joint Work, Accepted

Arbitrary View Image Generation from Multiple Silhouette Images in Projective Grid Space

'S. Yaguchi, H.Saito'

Proceedings of SPIE Vol.4309　(Videometrics and Optical Methods for 3D Shape Measurement) 294-304 2001.01

Research paper (conference, symposium, etc.), Joint Work, Accepted

Three Bodies Trading Model in Financial Markets and Its Numerical Simulation Methodology with Genetic Algorithms

'Hiwon Yoon, Hideo Saito, Takahiro Tanahashi'

'Empirical Science of Financial Fluctuations, Springer, 2001' 159-170 2001

Joint Work, Accepted

金融市場における日中変動シミュレーション

尹　煕元，斎藤英雄，棚橋隆彦

'日本計算工学会論文集, 2001年号，論文番号20010036' 1月8日 2001

Joint Work, Accepted

カラー陰影解析とシルエット解析の融合による回転非凸物体の３次元形状復元

佐藤太一、斎藤英雄、小沢慎治

情報処理学会コンピュータビジョンとイメージメディア研究会論文誌 Vol.42 ( No.SIG10(CVIM1) ) 41-51 2000.12

Joint Work, Accepted

3D Reconstruction Based on Epipolar Geometry

'Makoto Kimura, Hideo Saito'

IAPR Workshop on Machine Vision Applications (MVA00) 578-581 2000.11

Research paper (conference, symposium, etc.), Joint Work, Accepted

Optimized Camera Viewpoint Determination System for Soccer Game Broadcasting

'Keisuke Matsumoto, Hideo Saito, Shinji Ozawa'

IAPR Workshop on Machine Vision Applications (MVA00) 115-118 2000.11

Research paper (conference, symposium, etc.), Joint Work, Accepted

Generation of 3D model with super resolved texture from image sequence

'Kenji Nakamura, Hideo Saito, Shinji Ozawa'

'Proc. IEEE International Conference on Systems, Man, and Cybernetics' 1406-1411 2000.10

Research paper (conference, symposium, etc.), Joint Work, Accepted

Super resolving texture mapping from multiple view images for 3D model construction

Hideo Saito

'Proc. IEEE International Conference on Systems, Man, and Cybernetics' 2000.10

Research paper (conference, symposium, etc.), Joint Work, Accepted

3-D drawing system via hand motion recognition from two cameras

'Kiyofumi Abe, Hideo Saito, Shinji Ozawa'

'Proc. IEEE International Conference on Systems, Man, and Cybernetics' 840-845 2000.10

Research paper (conference, symposium, etc.), Joint Work, Accepted

The evaluating system of human skin surface condition by image processing

'Yoshinao Takemae, Hideo Saito, Shinji Ozawa'

'Proc. IEEE International Conference on Systems, Man, and Cybernetics' 218-223 2000.10

Research paper (conference, symposium, etc.), Joint Work, Accepted

3D Reconstruction of Book Surface Taken from Image Sequence

'Kenji Nakamura, Hideo Saito, Shinji Ozawa'

15th International Conference on Pattern Recognition (ICPR00) vol.4 575-578 2000.09

Research paper (conference, symposium, etc.), Joint Work, Accepted

3D Shape Recovery of Non-Convex Object from Rotation

'Taichi Sato, Hideo Saito, and Shinji Ozawa'

15th International Conference on Pattern Recognition (ICPR00) vol.1 742-745 2000.09

Research paper (conference, symposium, etc.), Joint Work, Accepted

3D Reconstruction of Skin Surface from Image Sequence

'T.Yamada, H.Saito, S.Ozawa'

IEICE Transactions on Information and Systems Vol.E83-D ( No.7 ) 1415-1421. 2000.07

Joint Work, Accepted

Appearance-Based Virtual View Generation for Fly Through in a Real Dynamic Scene

'Shigeyuki Baba, Hideo Saito, Sundar Vedula, Kong Man Cheng, Takeo Kanade'

Proceedings of the Joint EUROGRAPHICS and IEEE TVCG Symposium on Visualization 179-188 2000.05

Research paper (conference, symposium, etc.), Joint Work, Accepted

光源の相対的回転による物体形状と表面反射特性の推定

小俣和子，斎藤英雄，小沢慎治

電子情報通信学会論文誌DII VOL.J83-D-II ( No.3 ) 927-937 2000.03

Joint Work, Accepted

ハンディカメラを用いて撮影された画像列からのEPIの補正に基づく室内環境三次元再構築

中川雅朗、斎藤英雄、小沢慎治

電子情報通信学会論文誌DII Vol.J84-DII ( No.2 ) 266-275 2000.02

Joint Work, Accepted

3-D Drawing System via Hand Motion Recognition from Two Cameras

'Kiyofumi Abe, Hideo Saito, Shinji Ozawa'

Proceedings of the 6th Korea-Japan Joint Workshop on Computer Vision 138-143 2000.01

Research paper (conference, symposium, etc.), Joint Work, Accepted

Face Pose Estimating System Based on Eigen Space Analysis

'Hideo Saito, Akihiro Watanabe, Shinji Ozawa'

1999 IEEE International Conference on Image Processing (ICIP99) Vol.I 638-642 1999.10

Research paper (conference, symposium, etc.), Joint Work, Accepted

Appearance-Based Virtual View Generation of Temporally-Varying Events from Multi-Camera Images in the 3D Room

'Hideo Saito, Shigeyuki Baba, Makoto Kimura, Sundar Vedula, Takeo Kanade'

Second International Conference on 3-D Digital Imaging and Modeling (3DIM99) 516-525 1999.10

Research paper (conference, symposium, etc.), Joint Work, Accepted

Recovery of Shape and Surface Reflectance of Specular Object from Rotation of Light Source

'H.Saito, K.Omata, S.Ozawa'

Second International Conference on 3-D Digital Imaging and Modeling (3DIM99) 526-535 1999.10

Research paper (conference, symposium, etc.), Joint Work, Accepted

3D Voxel Construction Based on Epipolar Geometry

'Makoto Kimura, Hideo Saito, Takeo Kanade'

1999 IEEE International Conference on Image Processing (ICIP99) Vol. III 135-139. 1999.10

Research paper (conference, symposium, etc.), Joint Work, Accepted

Shape Reconstruction in Projective Grid Space from Large Number of Images

'Hideo Saito, Takeo Kanade'

IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'99) Vol.2 49-54 1999.06

Research paper (conference, symposium, etc.), Joint Work, Accepted

多視点カラー画像からの3次元形状および物体色の推定

西山芳生，斎藤英雄，小沢慎治

電子情報通信学会論文誌DII VOL.J82-D-II ( No.6 ) 990-1000 1999.06

Joint Work, Accepted

3D Shape Inspection of Skin Surface from Rotation of Light Source

'T.Yamada, H. Saito, S.Ozawa, M.Yanai, M.Takahashi'

5th International Conference on Quality Control by Artificial Vision (QCAV99) 247-254 1999.05

Research paper (conference, symposium, etc.), Joint Work, Accepted

Virtualized Reality: Digitizing a 3D Time-Varying Event As Is and in Real Time

'T.Kanade, P.W.Rander, S.Vedula, H.Saito'

International Symposium on Mixed Reality(ISMR99) 41-57 1999.05

Research paper (conference, symposium, etc.), Joint Work, Accepted

音声における感情表現語と感情表現パラメータの対応づけ

森山剛，斎藤英雄，小沢慎治

電子情報通信学会論文誌DII VOL.J82-D-II ( No.4 ) 703-711 1999.04

Joint Work, Accepted

High Description Image Acquisition for Digital Archiving of Rare Books

'M. Kashimura, T.Nakajima, T.Maeda, N.Onda, H.Saito, S.Ozawa'

SPIE Visual Communications and Image Processing '99 Vol.3653 1430-1440 1999.01

Research paper (conference, symposium, etc.), Joint Work, Accepted

Shspe Recovery of Book Surface Using Two Shade Images Under Perspective Condition

'S.I. Cho, H. Saito and S. Ozawa'

Int'L Conf. on Quality Control by Artificial Vision (QCAV'98) 116-122 1998.11

Research paper (conference, symposium, etc.), Joint Work, Accepted

'Modeling, Combining, and Rendering Dynamic Real-World Events from Image Sequences'

'S.Vedula, P.W.Rander, H.Saito, T.Kanade'

4th Conference on Virtual Systems and Multimedia (VSMM98) Vol.1 326-332 1998.11

Research paper (conference, symposium, etc.), Joint Work, Accepted

3D Reconstruction of Skin Surface from Image Sequence

'T.Yamada, H.Saito, S.Ozawa'

IAPR Workshop on Machine Vision Applications (MVA98) 384-387 1998.11

Research paper (conference, symposium, etc.), Joint Work, Accepted

Estimation of Face Pose from an Image using Eigen Space Method

'A.Watanabe, H.Saito, S.Ozawa'

'International Conference on Computer Vision, Pattern Recognition, and Image Processing' 287-290 1998.10

Research paper (conference, symposium, etc.), Joint Work, Accepted

Practical Introduction of Image Processing Technology to Digital Archiving of Rare Books

'M. Kashimura, T.Nakajima, N.Onda, H.Saito, S.Ozawa'

Internationa Conference on Signal Processing Application Technology (ISPAT98) 1025-1029 1998.10

Research paper (conference, symposium, etc.), Joint Work, Accepted

Recovery of Shape and Body Reflectance of Specular Object by Rotating Light Source

'K.Omata, H.Saito, S.Ozawa'

'International Conference on Computer Vision, Pattern Recognition, and Image Processing' 421-424 1998.10

Research paper (conference, symposium, etc.), Joint Work, Accepted

遺伝的アルゴリズムによる多視点画像からの多面体モデリング

桐原俊、森政之、斎藤英雄

電子情報通信学会論文誌DII Vol.J81-D-II ( No. 7 ) 1545-1555 1998.07

Joint Work, Accepted

Geometric Solution of Shape from Shading Problem

'S.I.Cho, H. Saito, and S. Ozawa'

Frontier of Computer Vision (FCV'98): 4th Japan-Korea Joint Workshop on Computer Vision 19-24 1998.01

Research paper (conference, symposium, etc.), Joint Work, Accepted

Shape Modeling from Multiple View Images Using Gas

S. Kirihara and H. Saito

'Third Asian Conference on Computer Vision (ACCV'98), Lecture Notes in Computer Science 1352 Vol.II' 448-454 1998.01

Research paper (conference, symposium, etc.), Joint Work, Accepted

Obtaining Polyhedral Model by Integration of Multiview Images via Genetic Algorithms

'H. Saito, S. Kirihara'

'Three-Dimensional Imaging and Laser-based Systems for Metrology and Inspection III, Proc.of SPIE Vol.3204' 174-184 1997.11

Research paper (conference, symposium, etc.), Joint Work, Accepted

Face-Direction Estimating System Using Stereo Vision

'T. Minagawa, H. Saito and S. Ozawa'

'23th International Conference on Industrial Electronics, Control, and Instrumentation' 1454-1459 1997.11

Research paper (conference, symposium, etc.), Joint Work, Accepted

Shape Recovery of Book Surface from Shade Images Under Perspective Condition

'Ik Song Cho, H. Saito, and S. Ozawa'

電気学会論文誌C Vol.117-C ( No.10 ) 1384-1390 1997.10

Joint Work, Accepted

鏡面反射・相互反射を考慮した皮膚表面微細形状の三次元再構成

松本敦之，斎藤英雄，小沢慎治

'電気学会論文誌C, 1997.10,' Vol.117-C ( No.10 ) 1391-1397 1997.10

Joint Work, Accepted

濃淡情報を用いた顔の向きの推定

鈴木香織，斎藤英雄，小沢慎治

'電気学会論文誌C, 1997.10,' Vol.117-C ( No.10 ) 1377-1383 1997.10

Joint Work, Accepted

Analytical Solution of Shape from Shading Problem

'S.I. Cho, H. Saito and S. Ozawa.'

British Machine Vision Conference (BMVC'97) Vol. 1 51-59 1997.09

Research paper (conference, symposium, etc.), Joint Work, Accepted

A Divide and Conquer Strategy in Recovering Shape of Book Surface from Shading

'S.I. Cho, H. Saito, and S. Ozawa'

'Image Analysis and Processing (9th Int. Conf.ICIAP'97) Lecture Notes in Computer Science 1311, Springer Vol. 2' 262-269 1997.09

Research paper (conference, symposium, etc.), Joint Work, Accepted

GAによる楕円当てはめとSNAKESを用いた任意背景からの顔輪郭抽出法

鈴木雄仁，斎藤英雄，小沢慎治

電子情報通信学会論文誌D-II Vol.J80-D-II ( No.8 ) 2186-2193 1997.08

Joint Work, Accepted

走行車両ナンバープレート画像のブレ補正

佐藤徹也，斎藤英雄，小沢慎治，小松雅俊，小林彰，佐藤元久

電気学会論文誌C Vol.117-C ( No.6 ) 777-784 1997.06

Joint Work, Accepted

Divide-and-conquer Strategy in Shape from Shading Problems

'S.I.Cho, H. Saito, and S. Ozawa'

IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR97) 413-419 1997.06

Joint Work, Accepted

Image Retrieval System Capable of Learning the User's Sensibility Using Neural Networks

'Yoshiteru Kageyama, Hideo Saito'

IEEE International Conference on Neural Networks (ICNN97) 1563-1567 1997.06

Joint Work, Accepted

遺伝的アルゴリズムを用いた混合音推定

小野徹太郎，斎藤英雄，小沢慎治

計測自動制御学会論文集 vol.33 ( no.5 ) 417-23 1997.05

Joint Work, Accepted

Evaluation of the relationship between emotional concepts and emotional parameters on speech

'Moriyama, T.; Saito, H.; Ozawa, S.,'

'1997 IEEE International Conference on Acoustics, Speech, and Signal Processing vol.2' 1431-1434 1997.04

Research paper (conference, symposium, etc.), Joint Work, Accepted

高速道路走行車両のナンバープレート画像の劣化補正

斎藤英雄，小沢慎治

画像ラボ Vol.8 ( No.1 ) 5月8日 1997.01

Research paper (scientific journal), Joint Work

Divide-and-conquer Strategy in Recovering Surface Shape of Book from Shading

'S.I.Cho, H. Saito, and S. Ozawa'

3rd Korea-Japan Joint Workshop on Computer Vision 47-52 1997.01

Research paper (conference, symposium, etc.), Joint Work, Accepted

3-D Reconstruction of Skin Surface from Photometric Stereo Images with Specular and Inter Reflections

'A. Matsumoto, H. Saito, and S. Ozawa'

'The 4th International Conference on Control, Automation, Robotics, and Vision (ICARCV'96) Vol.2' 778-782 1996.12

Research paper (conference, symposium, etc.), Joint Work, Accepted

Extraction of the Human Face from Natural Background Using Gas

'Y. Suzuki, H. Saito, and S. Ozawa'

1996 IEEE TENCON -Digital Signal Processing Applications- Vol.1 221-226 1996.11

Research paper (conference, symposium, etc.), Joint Work, Accepted

Shape Modeling of Multiple Objects from Shading Images Using Genetic Algorithms

H. Saito and M. Kimura

'IEEE International Conference on Systems, Man and Cybernetics (SMC'96)' 2463-2468 1996.10

Research paper (conference, symposium, etc.), Joint Work, Accepted

Shape Modeling from Multiple Images Using Genetic Algorithms

H. Saito and M. Mori

International Conference on Virtual Systems and Multimedia '96 (VSMM '96) 251-256 1996.09

Research paper (conference, symposium, etc.), Joint Work, Accepted

An Interactive Image Retrieval System Using Genetic Algorithms

H. Shiraki and H. Saito

International Conference on Virtual Systems and Multimedia '96 (VSMM '96) 257-261 1996.09

Research paper (conference, symposium, etc.), Joint Work, Accepted

影を含んだ複数の陰影画像からの皮膚表面形状の3次元再構成

斎藤英雄，宗宮由紀子，小沢慎治

計測自動制御学会論文集 Vol.32 ( No.8 ) 1160-1167 1996.08

Joint Work, Accepted

Object Modeling from Multiple Images Using Genetic Algorithms

H. Saito and M. Mori

13th International Conference on Pattern Recognition Vol. IV 669-673 1996.08

Research paper (conference, symposium, etc.), Joint Work, Accepted

Superquadrics Modeling of Multiple Objects from Shading Images Using Genetic Algorithms

Hideo Saito and M. Kimura

'22th International Conference on Industrial Electronics, Control, and Instrumentation' 1589-1593 1996.08

Research paper (conference, symposium, etc.), Joint Work, Accepted

Human Hand Tracking from Binocular Image Sequences

'Kenichi Nirei, Hideo Saito, Masaaki Mochimaru, and S. Ozawa'

'22th International Conference on Industrial Electronics, Control, and Instrumentation' 297-302 1996.08

Research paper (conference, symposium, etc.), Joint Work, Accepted

遺伝的アルゴリズムを用いた濃淡画像からの超２次関数のパラメータ推定

綱島宣浩，斎藤英雄

電子情報通信学会論文誌ＤII Vol.J79-D-II ( No.5 ) 795-802 1996.05

Joint Work, Accepted

時空間画像の濃度勾配による移動カメラの位置と姿勢の推定

宇佐美潔忠，斎藤英雄

電子情報通信学会論文誌ＤII Vol.J79-D-II ( No.5 ) 737-746 1996.05

Joint Work, Accepted

時空間画像解析による水中物体の３次元形状計測

河村拓史，斎藤英雄，中島真人

計測自動制御学会論文集 Vol.32 ( No.4 ) 607-609 1996.04

Joint Work, Accepted

知能情報メディアシンポジウム傍聴記

斎藤英雄、

画像ラボ Vol.7 ( No.1 ) 65-67 1996.01

Research paper (scientific journal), Single Work

Restoration of Space-Variant Blurred Image Using a Wavelet Transform

S. Hashimoto and H. Saito

Systems and Computers In Japan，1996年 Vol.27 ( No.14 ) 76-84 1996

Joint Work, Accepted

Halftone Algorithm Using Genetic Algorithm

N. Kobayashi and H. Saito

Systems and Computers In Japan，1996年 Vol.27 ( No.10 ) 89-97 1996

Joint Work, Accepted

Road Correspondence from Image Sequence Based on Road Matching Technique

'J. Yang, H. Horie, S.Ozawa, H. Saito'

1995 International Symposium on Nonlinear Theory and Its Applications (NOLTA'95) 8G-5 1017-1020 1995.12

Research paper (conference, symposium, etc.), Joint Work, Accepted

ウエーブレット変換を用いたスペースバリアントな劣化画像の復元

橋本正一，斎藤英雄

電子情報通信学会論文誌ＤII Vol.J78-D-II ( No.12 ) 1821-1830 1995.12

Joint Work, Accepted

遺伝的アルゴリズムを用いた擬似濃淡表示法

小林直樹，斎藤英雄、

電子情報通信学会論文誌ＤII Vol.J78-D-II ( No.10 ) 1450-1459 1995.10

Joint Work, Accepted

High-Speed Imaging System of Luminous Intensity Distribution of Plasma Using Light Emission CT

'H. Kawamura, H. Saito, K. Yuasa, Y. Yuge, and M. Nakajima　　　'

Optics Communication Vol.119 197-206 1995.08

Joint Work, Accepted

Application of genetic algorithms to stereo matching of images

Hideo Saito and Masayuki Mori

Pattern Recognition Letters Vol.16 815-821 1995

Joint Work, Accepted

遺伝的アルゴリズムを用いた２次元濃淡画像からの３次元形状推定

宇佐美潔忠，斎藤英雄

計測自動制御学会論文集 Vol.30 ( No.11 ) 1378-1384 1994.11

Joint Work, Accepted

Application of Genetic Algorithm to Stereo Matching

'H. Saito, M. Mori'

Japan-Israel Workshop on Computer Vision and Visual Communication 64-67 1994.10

Research paper (conference, symposium, etc.), Joint Work, Accepted

Superquadrics Parameter Estimation from Shading Images Using Genetic Algorithm'

'H. Saito, N. Tsunashima'

'20th International Conference on Industrial Electronics, Control, and Instrumentation Vol. 2' 978-983 (3rd Best aer Award) 1994.09

Research paper (conference, symposium, etc.), Joint Work, Accepted

ＣＴ手法による磁界断面測定法

'斎藤英雄, 中島真人'

固体物理 Vol.29 ( No.6 ) 541-547 1994.06

Research paper (scientific journal), Joint Work

Evolutionary Computation Approaches to Halftoning Algorithm

H. Saito and N. Kobayashi

The First IEEE Conference on Evolutionary Computation Vol.2 787-791 1994.06

Research paper (conference, symposium, etc.), Joint Work, Accepted

ＰＳＦのパラメータ分布を推定するシフト・バリアントなボケ画像の復元法

橋本正一，斎藤英雄

電子情報通信学会論文誌ＤII Vol.J77-D-II ( No.4 ) 719-728 1994.04

Joint Work, Accepted

自然画像のための局所的コントラスト強調法

小林直樹，斎藤英雄，中島真人

電子情報通信学会論文誌ＤII Vol.J77-D-II ( No.3 ) 502-509 1994.03

Joint Work, Accepted

偏平断面蛍光ランプ中の励起種分布

湯浅邦夫，弓削洋二，斎藤英雄，中島真人

照明学会誌 Vol.78 ( No.10 ) 497-503 1994

Joint Work, Accepted

光ECTによる放電管内プラズマ発光強度分布の動画像化９

斎藤英雄，畠中秀晃，湯浅邦夫，弓削洋二，中島真人

計測自動制御学会論文集 Vol.29 ( No.9 ) 1043-1049 1993.09

Joint Work, Accepted

ＣＴ手法による磁界の映像化−ＲＯＩ再構成システムの開発−

'斎藤英雄, 藤田健二, 中島真人'

電子情報通信学会論文誌ＤII Vol.J75-DII ( No.7 ) 1222-1231 1992.07

Joint Work, Accepted

ＣＴ手法を用いた交流磁界分布の映像化

'斎藤英雄, 谷口幸治, 中島真人'

電子情報通信学会論文誌ＤII Vol.J75-DII ( No.3 ) 530-537 1992.03

Joint Work, Accepted

ＣＴ手法による磁界の映像化−回転走査方式による投影データ収集法−

'斎藤英雄, 油田信一, 中島真人'

電子情報通信学会論文誌ＤII Vol.J74-D-II ( No.2 ) 282-284 1991.02

Joint Work, Accepted

High Speed Data Collection System for Magnetic Field Imaging by CT Technique

'H. Saito, F. Saito, S. Yuta, and M. Nakajima'

IEEE Transactions on Instrumentation and Measurement Vol.IM-39 ( N0.5 ) 739-744 1990.10

Joint Work, Accepted

ＣＴ手法による磁界の高速映像化システム

'斎藤英雄, 中島真人'

センサ技術 Vol.10 ( No.2 ) 17-21 1990.02

Research paper (scientific journal), Joint Work

ＣＴ手法による磁界の映像化−ベクトル量としての再構成−

'斎藤英雄, 宮本晃宏, 諸角建, 油田信一, 中島真人'

電子情報通信学会論文誌Ｄ-II Vol.J72-D-II ( No.11 ) 1839-1847 1989.11

Joint Work, Accepted

Reconstruction of Magnetic Flux Density as Vector Quantity by CT Technique

'H. Saito, A. Miyamoto, T. Morozumi, S. Yuta, and M. Nakajima'

IEEE Transactions on Instrumentation and Measurement Vol.IM-38 ( No.2 ) 415-420 1989.04

Joint Work, Accepted

Reconstruction of magnetic flux density as vector quantity by CT technique

Saito Hideo, Miyamoto Akihiro, Morozumi Tatsuru, Yuta Shin'ichi, Nakajima Masato

353 - 354 1988.06

ＣＴ手法による不可視物理量の映像化

'斎藤英雄, 中島真人'

流れの可視化 Vol.8 ( No.29 ) 104-109 1988.04

Research paper (scientific journal), Joint Work

ＣＴ手法による磁界の映像化

'草場律, 斎藤英雄, 中島真人, 油田信一'

電子情報通信学会論文誌Ｄ Vol.J71-D ( No.2 ) 424-429 1988.02

Joint Work, Accepted

Magnetic Field Imaging by CT Technique

'H. Saito, R. Kusaba, M. Nakajima, and S. Yuta'

IEEE Transactions on Magnetics Vol. MAG-23 ( No.5 ) 2587-2589 1987.09

Research paper (conference, symposium, etc.), Joint Work, Accepted

MAGNETIC FIELD IMAGING BY CT TECHNIQUE.

Saito Hideo, Kusaba Ritsu, Nakajima Masato, Yuta Shin'ichi

IEEE Transactions on Magnetics MAG-23 ( 5 ) 2587 - 2589 1987.09

ISSN 0018-9464

In this paper, we propose a method for measuring the magnetic field as a 2-D tomographic image, applyinig an algorithm of CT to the electromagnetic induction phenomenon, and also show some experimental results to demonstrate the efficacy. When a line conductor moves relative to a filed of magnetic flux density, the voltage is induced between both ends of the line conductor. The measurement of the voltage during the field movement provides a projection data of the flux density distribution in the plane where the line conductor passed. Collecting this projection data at every angle, and applying the CT algorithm to them, the flux density distribution can be reconstructed in a 2-D tomographic image.

MAGNETIC FIELD RECONSTRUCTION BY CT TECHNIQUE.

Kusaba R., Saito H., Nakajima M., Yuta S.

IEEE translation journal on magnetics in Japan TJMJ-2 ( 12 ) 1122 - 1123 1985.12

ISSN 0882-4959

Computer tomography (CT) techniques were used to obtain cross-sectional images of a magnetic field. A conducting wire was passed through the field at constant velocity and at a series of orientations (in 3 degree increments) with respect to a horseshoe magnet. The resulting induced voltages were used as projection data to extract two-dimensional tomographic images of the vertical component of the magnetic field of the magnet. The results confirm the validity of the method.

　 Previous page - Next page 　