研究者詳細 - 松谷　宏紀

論文 - 松谷　宏紀

分割表示 90 件中 21 - 40 件目／全件表示 >>

Accelerating Deep Learning using Multiple GPUs and FPGA-Based 10GbE Switch

Itsubo T., Koibuchi M., Amano H., Matsutani H.

Proceedings - 2020 28th Euromicro International Conference on Parallel, Distributed and Network-Based Processing, PDP 2020 （Proceedings - 2020 28th Euromicro International Conference on Parallel, Distributed and Network-Based Processing, PDP 2020） 102 - 109 2020年03月

ISSN 9781728165820

A back-propagation algorithm following a gradient descent approach is used for training deep neural networks. Since it iteratively performs a large number of matrix operations to compute the gradients, GPUs (Graphics Processing Units) are efficient especially for the training phase. Thus, a cluster of computers each of which equips multiple GPUs can significantly accelerate the training phase. Although the gradient computation is still a major bottleneck of the training, gradient aggregation and parameter optimization impose both communication and computation overheads, which should also be reduced for further shortening the training time. To address this issue, in this paper, multiple GPUs are interconnected with a PCI Express (PCIe) over 10 Gbit Ethernet (10GbE) technology. Since these remote GPUs are interconnected via network switches, gradient aggregation and optimizers (e.g., SGD, Adagrad, Adam, and SMORMS3) are offloaded to an FPGA-based network switch between a host machine and remote GPUs; thus, the gradient aggregation and optimization are completed in the network. Evaluation results using four remote GPUs connected via the FPGA-based 10GbE switch that implements the four optimizers demonstrate that these optimization algorithms are accelerated by up to 3. 0x and 1. 25x compared to CPU and GPU implementations, respectively. Also, the gradient aggregation throughput by the FPGA-based switch achieves 98.3% of the 10GbE line rate.

Access to Document (DOI)

A generalized theory based on the turn model for deadlock-free irregular networks

Kawano R., Yasudo R., Matsutani H., Koibuchi M., Amano H.

IEICE Transactions on Information and Systems （IEICE Transactions on Information and Systems） E103D （ 1 ） 101 - 110 2020年

ISSN 09168532

Recently proposed irregular networks can reduce the latency for both on-chip and off-chip systems with a large number of computing nodes and thus can improve the performance of parallel applications. However, these networks usually suffer from deadlocks in routing packets when using a naive minimal path routing algorithm. To solve this problem, we focus attention on a lately proposed theory that generalizes the turn model to maintain the network performance with deadlock-freedom. The theorems remain a challenge of applying themselves to arbitrary topologies including fully irregular networks. In this paper, we advance the theorems to completely general ones. Moreover, we provide a feasible implementation of a deadlock-free routing method based on our advanced theorem. Experimental results show that the routing method based on our proposed theorem can improve the network throughput by up to 138 % compared to a conventional deterministic minimal routing method. Moreover, when utilized as the escape path in Duato’s protocol, it can improve the throughput by up to 26.3 % compared with the conventional up*/down* routing.

Access to Document (DOI)

Designing low-diameter interconnection networks with multi-ported host-switch graphs

Yasudo R., Nakano K., Koibuchi M., Matsutani H., Amano H.

Concurrency Computation （Concurrency Computation） 2020年

ISSN 15320626

A host-switch graph was originally proposed as a graph that represents a network topology of a computer systems with 1-port host computers and (Formula presented.) -port switches. It has been studied from both theoretical and practical aspects in terms of the diameter, the average shortest path length, and the performance of real applications. In recent high-performance computing systems, however, a host computer is connected to multiple switches by using InfiniBand, NVSwitch, or Omni-Path, and consequently they provide high bandwidths. Since a host-switch graph cannot represent such systems, this article extends a host-switch graph so that it can represent such systems. As a result, a host-switch graph can include multi-ported hosts. Furthermore, we propose to use multi-port hosts for reducing the diameter. We show that the diameter minimization is equivalent to solving the degree diameter problem for bipartite graphs of diameter three. Our experimental results show that we can drastically reduce the diameter as well as increasing the bandwidth and improves performance of MPI applications by up to 162% as compared with networks with single-ported hosts.

Access to Document (DOI)

Deadlock-free layered routing for infiniband networks

Kawano R., Matsutani H., Amano H.

Proceedings - 2019 7th International Symposium on Computing and Networking Workshops, CANDARW 2019 （Proceedings - 2019 7th International Symposium on Computing and Networking Workshops, CANDARW 2019） 84 - 90 2019年11月

ISSN 9781728152684

End-to-end network latency has become an important issue for big data processing on large-scale and heterogeneous data centers. To realize them, a low-latency inter-switch network is required. In recent years, a novel network based on a random regular graph such as Jellyfish has been proposed as a solution to these problems. To realize this network on Infiniband, we explore layered shortest path routing that guarantees deadlock freedom. Infiniband can avoid deadlocks by using Virtual Lane (VL), which is a QoS guarantee mechanism. However, the output VL is indirectly determined by using Service Level (SL) held by each packet. In this research, to minimize the number of consumed SLs, we propose an algorithm based on a graph coloring problem to assign SLs to all source-destination pairs. The proposed method can reduce the number of consumed SLs by up to 38%compared to the conventional SL mapping method.

Access to Document (DOI)

An adaptive abnormal behavior detection using online sequential learning

Ito R., Tsukada M., Kondo M., Matsutani H.

Proceedings - 22nd IEEE International Conference on Computational Science and Engineering and 17th IEEE International Conference on Embedded and Ubiquitous Computing, CSE/EUC 2019 （Proceedings - 22nd IEEE International Conference on Computational Science and Engineering and 17th IEEE International Conference on Embedded and Ubiquitous Computing, CSE/EUC 2019） 436 - 440 2019年08月

ISSN 9781728116631

In the real world, normal and abnormal behavior patterns vary depending on a given environment, which means that the abnormal behavior detection model should be customized. To address this issue, in this paper, we employ OS-ELM (Online Sequential Extreme Learning Machine) and Autoencoder for adaptive abnormal behavior detection. First, state-transition probability tables of a target during an initial learning period are learned as normal behaviors. Then, Autoencoder-based anomaly detection is performed for the state-transition probability tables of subsequent time frames. The abnormal behavior detection model is updated by using OS-ELM algorithm every time a new probability table or behavior comes. The number of abnormal behavior detection instances is dynamically tuned to reflect the recent normal patterns or modes. Also, the table is compressed to reduce the computation cost. Evaluation results using a driving dataset of cars show that the proposed abnormal behavior detection accurately identifies normal and aggressive driving patterns with the optimal number of the abnormal behavior detection instances.

Access to Document (DOI)

Sparse 3-D NoCs with inductive coupling

Koibuchi M., Leong L., Totoki T., Niwa N., Matsutani H., Amano H., Casanova H.

Proceedings - Design Automation Conference （Proceedings - Design Automation Conference） 2019年06月

研究論文（国際会議プロシーディングス）, 共著, 査読有り, ISSN 9781450367257

Wireless interconnects based on inductive coupling technology are compelling propositions for designing 3-D integrated chips. This work addresses the heat dissipation problem on such systems. Although effective cooling technologies have been proposed for systems designed based on Through Silicon Via (TSV), their application to systems that use inductive coupling is problematic because of increased wireless-communication distance. For this reason, we propose two methods for designing sparse 3-D chips layouts and Networks on Chip (NoCs) based on inductive coupling. The first method computes an optimized 3-D chip layout and then generates a randomized network topology for this layout. The second method uses a standard stack chip layout with a standard network topology as a starting point, and then deterministically transforms it into either a "staircase" or a "checkerboard" layout. We quantitatively compare the designs produced by these two methods in terms of network and application performance. Our main finding is that the first method produces designs that ultimately lead to higher parallel application performance, as demonstrated for nine OpenMP applications in the NAS Parallel Benchmarks.

Access to Document (DOI)

Performance and Cost Evaluations of Online Sequential Learning and Unsupervised Anomaly Detection Core

Itsubo T., Tsukada M., Matsutani H.

IEEE Symposium on Low-Power and High-Speed Chips and Systems, COOL CHIPS 2019 - Proceedings （IEEE Symposium on Low-Power and High-Speed Chips and Systems, COOL CHIPS 2019 - Proceedings） 2019年05月

ISSN 9781728117485

Access to Document (DOI)

Key-value Store Chip Design for Low Power Consumption

Tokusashi Y., Matsutani H., Amano H.

ISSN 9781728117485

Low-power embedded systems often require simple but flexible functionality for data management. Key-value store is one of data stores, which provides simple API and is easy to scale-out. We developed a dedicated key-value store core as a low-power embedded storage. In this paper, key-value store chip is fabricated with Renesas SOTB 65-nm process technology. We evalute the real chip in terms of power consumption and operating frequency by tuning VDD and body bias voltage. The chip achieved 11.2mW as power consumption with 0.7V VDD if the target frequency is 40MHz. When data rate is low, the system can reduce power consumption by tuning VDD and body bias voltage.

Access to Document (DOI)

Network optimizations on prediction server with multiple predictors

Okuyama K., Tokusashi Y., Iwata T., Tsukada M., Kishiki K., Matsutani H.

Proceedings - 16th IEEE International Symposium on Parallel and Distributed Processing with Applications, 17th IEEE International Conference on Ubiquitous Computing and Communications, 8th IEEE International Conference on Big Data and Cloud Computing, 11th IEEE International Conference on Social Computing and Networking and 8th IEEE International Conference on Sustainable Computing and Communications, ISPA/IUCC/BDCloud/SocialCom/SustainCom 2018 （Proceedings - 16th IEEE International Symposium on Parallel and Distributed Processing with Applications, 17th IEEE International Conference on Ubiquitous Computing and Communications, 8th IEEE International Conference on Big Data and Cloud Computing, 11th IEEE International Conference on Social Computing and Networking and 8th IEEE International Conference on Sustainable Computing and Communications, ISPA/IUCC/BDCloud/SocialCom/SustainCom 2018） 1044 - 1045 2019年03月

ISSN 9781728111414

Toward machine learning based prediction services, the prediction server has multiple predictors and selects an appropriate one based on past feedbacks from the clients. In this case, three messages including request, reply, and feedback, are required for each prediction request. Packets are typically transmitted and received via a network protocol stack in OS kernel, and performance improvement can be expected by avoiding the protocol stack since it degrades the communication performance especially for small packets. We implement the prediction server using network optimization approaches including kernel-bypassing and in-NIC processing approaches. Evaluation results show that these network optimizations are beneficial to improve the prediction server performance compared to a baseline prediction server using a standard network protocol stack.

Access to Document (DOI)

Accelerating blockchain transfer system using FPGA-Based NIC

Sakakibara Y., Tokusashi Y., Morishima S., Matsutani H.

ISSN 9781728111414

Blockchain is core technology for cryptocurrency and it is possible to become fundamental platform for industry and business. Especially, a blockchain-based digital asset transfer system using Internet of Things (IoT) products has recently been considered as a new practical platform, but the protocol limits performance. Previous research has improved performance by proposing new protocols, but further improvement is necessary for dealing with increasing transactions via IoT products. Therefore, we propose an in-Network Interface Card (in-NIC) processing approach using a Field Programmable Gate Array (FPGA) to improve performance of a blockchain-based transfer system. To be more concrete, we design and implement a prototype NIC with a key-value data store written in a P4 language on the FPGA that has four 10Gigabit Ethernet (10GbE) network interfaces. The prototype system supports frequently-used commands (CREATE, ISSUE, TRANSFER and REFER) for transferring digital asset. It reduces time for processing a kernel network protocol stack and accessing the data store. In fact, we measured throughput and latency of our prototype system compared to those of a blockchain software application. As a result, we found that our solution is able to obtain throughput 6.04 times higher on average and latency 15.4 times lower on average for all typical blockchain operations.

Access to Document (DOI)

Acceleration of anomaly detection in blockchain using in-GPU Cache

Morishima S., Matsutani H.

ISSN 9781728111414

Blockchain is a distributed ledger system composed of a P2P network and is used for a wide range of applications, such as international remittance, inter-individual transactions, and asset conservation. In Blockchain systems, tamper resistance is enhanced by the property of transaction that cannot be changed or deleted by everyone including the creator of the transaction. However, this property also becomes a problem that unintended transaction created by miss operation or secret key theft cannot be corrected later. Due to this problem, once an illegal transaction such as theft occurs, the damage will expand. To suppress the damage, we need countermeasures, such as detecting illegal transaction at high speed and correcting the transaction before approval. However, abnormality detection in the Blockchain at high speed is computationally heavy, because we need to repeat the detection process using various feature quantities and the feature extractions become overhead. In this paper, to accelerate abnormality detection, we propose to cache transaction information necessary for extracting feature in GPU device memory and perform both feature extraction and abnormality detection in the GPU. We employ abnormality detection using K-means algorithm based on the conditional features. When the number of users is one million and the number of transactions is 100 millions, our proposed method achieves 37.1 times faster than CPU processing method and 16.1 times faster than GPU processing method that does not perform feature extraction on the GPU.

Access to Document (DOI)

LaKe: The power of in-network computing

Tokusashi Y., Matsutani H., Zilberman N.

2018 International Conference on Reconfigurable Computing and FPGAs, ReConFig 2018 （2018 International Conference on Reconfigurable Computing and FPGAs, ReConFig 2018） 2019年02月

ISSN 9781728119687

In-network computing accelerates applications natively running on the host by executing them within network devices. While in-network computing offers significant performance improvements, its limitations and design trade-offs have not been explored. To usefully and efficiently run applications within the network, we first need to understand the implications of their design. In this work we introduce LaKe, a Layered Key-Value Store design, running as an in-network application. LaKe is a scalable design, enabling the exploration of design decisions and their effect on throughput, latency and power efficiency. LaKe achieves full line rate throughput, while maintaining a latency of 1.1 μs and better power efficiency than existing hardware based memcached designs.

Access to Document (DOI)

Designing High-Performance Interconnection Networks with Host-Switch Graphs

Yasudo R., Koibuchi M., Nakano K., Matsutani H., Amano H.

IEEE Transactions on Parallel and Distributed Systems （IEEE Transactions on Parallel and Distributed Systems） 30 （ 2 ） 315 - 330 2019年02月

研究論文（学術雑誌）, 共著, 査読有り, ISSN 10459219

This paper aims at establishing a method for designing high-performance network topologies to bridge a gap between theoretical and practical studies. To this end, we present a novel graph called a host-switch graph, which consists of host vertices and switch vertices with maximum degree 1 and r, respectively. This graph represents a network topology of a practical parallel/distributed computer system with host computers connected by r-port switches. We discuss important metrics for designing high-performance interconnection networks: the host-to-host average shortest path length (h-ASPL) and the bisection width (BiW). In particular, we explore a method for constructing host-switch graphs with low h-ASPL and high BiW that connect the fixed number of hosts via any number of r-port switches. We demonstrate that the number of switches that provides the minimum h-ASPL can mathematically be approximated, and the minimum number of switches that provides a certain BiW can experimentally be approximated. On the basis of the approximations, we propose a randomized algorithm for searching host-switch graphs. We then apply the graphs to interconnection networks and compare them with typical network topologies. As compared with the torus, the dragonfly, and the fat-tree, our networks attain higher performance and smaller power and costs.

Access to Document (DOI)

OS-ELM-FPGA: An FPGA-based online sequential unsupervised anomaly detector

Tsukada M., Kondo M., Matsutani H.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) （Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)） 11339 LNCS 518 - 529 2019年

ISSN 9783030105488

Autoencoder, a neural-network based dimensionality reduction algorithm has demonstrated its effectiveness in anomaly detection. It can detect whether an input sample is normal or abnormal by just training only with normal data. In general, Autoencoder is built on backpropagation-based neural networks (BP-NNs). When BP-NNs are implemented in edge devices, they are typically specialized only for prediction with weight matrices precomputed offline due to the high computational cost. However, such devices cannot be immediately adapted to time-series trend changes of input data. In this paper, we propose an FPGA-based unsupervised anomaly detector, called OS-ELM-FPGA, that combines Autoencoder and an online sequential learning algorithm OS-ELM. Based on our theoretical analysis of the algorithm, the proposed OS-ELM-FPGA completely eliminates matrix pseudoinversions while improving the learning throughput. Simulation results using open-source datasets show that OS-ELM-FPGA achieves favorable anomaly detection accuracy compared to CPU and GPU implementations of BP-NNs. Learning throughput of OS-ELM-FPGA is 3.47x to 27.99x and 5.22x to 78.06x higher than those of CPU and GPU implementations of OS-ELM. It is also 3.62x to 36.15x and 1.53x to 43.44x higher than those of CPU and GPU implementations of BP-NNs.

Access to Document (DOI)

Accelerating online change-point detection algorithm using 10 GbE FPGA NIC

Iwata T., Nakamura K., Tokusashi Y., Matsutani H.

ISSN 9783030105488

In statistical analysis and data mining, change-point detection that identifies the change-points which are times when the probability distribution of time series changes has been used for various purposes, such as anomaly detections on network traffic and transaction data. However, computation cost of a conventional AR (Auto-Regression) model based approach is too high and infeasible for online. In this paper, an AR model based online change-point detection algorithm, called ChangeFinder, is implemented on an FPGA (Field Programmable Gate Array) based NIC (Network Interface Card). The proposed system computes the change-point score from time series data received from 10 GbE (10 Gbit Ethernet). More specifically, it computes the change-point score at the 10 GbE NIC in advance of host applications. This paper aims to reduce the host workload and improve change-point detection performance by offloading ChangeFinder algorithm from host to the NIC. As evaluations, change-point detection in the FPGA NIC is compared with a baseline software implementation and those enhanced by two network optimization techniques using DPDK and Netfilter in terms of throughput. The result demonstrates 16.8x improvement in change-point detection throughput compared to the baseline software implementation. The throughput achieves 83.4% of the 10 GbE line rate.

Access to Document (DOI)

An FPGA-based change-point detection for 10Gbps packet stream

Iwata T., Nakamura K., Tokusashi Y., Matsutani H.

IEICE Transactions on Information and Systems （IEICE Transactions on Information and Systems） E102D （ 12 ） 2366 - 2376 2019年

ISSN 09168532

In statistical analysis and data mining, change-point detection that identifies the change-points which are times when the probability distribution of time series changes has been used for various purposes, such as anomaly detections on network traffic and transaction data. However, computation cost of a conventional AR (Auto-Regression) model based approach is too high and infeasible for online. In this paper, an AR model based online change-point detection algorithm, called ChangeFinder, is implemented on an FPGA (Field Programmable Gate Array) based NIC (Network Interface Card). The proposed system computes the change-point score from time series data received from 10GbE (10Gbit Ethernet). More specifically, it computes the change-point score at the 10GbE NIC in advance of host applications. It can find change-points on single or multiple streams using a context memory. This paper aims to reduce the host workload and improve change-point detection performance by offloading ChangeFinder algorithm from host to the NIC. As evaluations, change-point detection in the FPGA NIC is compared with a baseline software implementation and those enhanced by two network optimization techniques using DPDK and Netfilter in terms of throughput. The result demonstrates 16.8x improvement in change-point detection throughput compared to the baseline software implementation. It is corresponding to the 10GbE line rate. Performance and area overheads when supporting multiple streams are also evaluated.

Access to Document (DOI)

K-Optimized Path Routing for High-Throughput Data Center Networks

Kawano R., Yasudo R., Matsutani H., Amano H.

Proceedings - 2018 6th International Symposium on Computing and Networking, CANDAR 2018 （Proceedings - 2018 6th International Symposium on Computing and Networking, CANDAR 2018） 99 - 105 2018年12月

ISSN 9781538691823

Network throughput has become an important issue for big-data analysis on Warehouse-Scale Computing (WSC) systems. It has been reported that randomly-connected inter-switch networks can enlarge the network throughput. For irregular networks, a multi-path routing method called k-shortest path routing is conventionally utilized. However, it cannot efficiently exploit longer-than-shortest paths that would be detour paths to avoid bottlenecks. In this work, a novel routing method called k-optimized path routing to achieve high throughput is proposed for irregular networks. We introduce a heuristic to select detour paths that can avoid bottlenecks in the network to improve the network throughput. Experimental results show that the proposed k-optimized path routing can improve the throughput by up to 133 % compared to the conventional k-shortest path routing. Moreover, it can improve the network bandwidth while maintaining the low network latency.

Access to Document (DOI)

An trace-driven performance prediction method for exploring noc design optimization

Niwa N., Totoki T., Matsutani H., Koibuchi M., Amano H.

Proceedings - 2018 6th International Symposium on Computing and Networking Workshops, CANDARW 2018 （Proceedings - 2018 6th International Symposium on Computing and Networking Workshops, CANDARW 2018） 182 - 185 2018年12月

ISSN 9781538691847

The performance prediction for a NoC-based Chip Multi-Processor (CMP) is one of the main design concerns. Generally, there is a trade-off between accuracy and time overhead on the performance prediction of computer systems. In particular, the time overhead is proportional or exponential to the number of cores when using a cycle-accurate full-system simulation, such as gem5. In this study, we propose an accurate and scalable method to predict the influence of design NoC parameters on its performance. Our method counts the number of execution cycles when employing the target NoC based on the statistics of one-time execution of a full-system simulation using a fully-connected NoC. To evaluate the accuracy and execution time overhead, we use the case that randomly generates allocations of processors with 3D mesh topology NoC. Its Mean Absolute Percentage Error of the estimated cycles is about 4.7%, and the Maximum Absolute Percentage Error is about 8.5%.

Access to Document (DOI)

MultiMQC: A Multilevel Message Queuing Cache Combining In-NIC and In-Kernel Memories

Mitsuzuka K., Tokusashi Y., Matsutani H.

Proceedings - 2018 International Conference on Field-Programmable Technology, FPT 2018 （Proceedings - 2018 International Conference on Field-Programmable Technology, FPT 2018） 137 - 144 2018年12月

研究論文（国際会議プロシーディングス）, 共著, 査読有り, ISSN 9781728102139

Message queuing systems that deliver messages from publishers to subscribers play an important role to collect data from IoT devices. Traditional message queuing systems have improved their performance in the context of transferring log data from publishers such as Web servers to subscribers that analyze the log data. In this case, both publishers and subscribers have been assumed to have enough buffer capacity and can transfer data as jumbo frame packets for high efficiency. In recent IoT applications, however, publishers are small sensors or edge devices with low-power processors and limited memory capacity. Vast numbers of such publishers produce relatively small packets. Such a lot of small messages significantly decrease the efficiency of conventional message queuing systems. To address this issue, a dedicated message queuing logic can be implemented on FPGA-based network interface card (FPGA NIC). However, a serious issue of such in-NIC approach is a limited memory capacity on the FPGA NIC. To handle message overflow of the in-NIC cache, in this paper, it is combined with a large in-kernel software cache. More specifically, we propose a multilevel message queuing cache combining in-NIC and in-kernel memories, called MultiMQC. The multilevel cache improves the read performance. Regarding the write performance, MultiMQC introduces a batch transfer that packs small incoming messages into a single batch. We implemented MultiMQC using NetFPGA-SUME board as in-NIC cache and Linux Netfilter framework as in-kernel cache. The experimental results demonstrate that the write throughput is increased in proportion to the batch size. When pull requests hit in the in-NIC cache, the read throughput reaches 95.8% of 10GbE line rate in four 10GbE interfaces.

Access to Document (DOI)

Low-reliable low-latency networks optimized for HPC parallel applications

Nguyen T., Matsutani H., Koibuchi M.

NCA 2018 - 2018 IEEE 17th International Symposium on Network Computing and Applications （NCA 2018 - 2018 IEEE 17th International Symposium on Network Computing and Applications） 2018年11月

ISSN 9781538676592

High-end network standards, such as 400GbE, have been introduced Forwarding Error Correction (FEC) for maintaining the same bit error rate (BER) as that in traditional low-bandwidth interconnection networks. However, FEC operation latency overhead surprisingly becomes higher than the sum of all the other switch operation overheads, e.g., routing computation and switch allocation. FEC operation latency overhead significantly degrades the performance of parallel applications in HPC systems. Instead, in this study, we exploit the low-latency network design using a Hamming code that does not provide rigid error-free communication. Since it is consistent with existing frame format based on standard Reed-Solomon RS(544,514) with DC(64b/66b) direct linecode and TC(256b/257b) transcode, respectively, the influences upon the other network layer design are limited. Interestingly, a large number of parallel applications can accept the BER in such a Hamming code. Since lowering such a BER improves switch operation latency, the proposed network using the Hamming code improves the execution time of NAS Parallel Benchmarks by 56% on average when compared to the counterpart RS-FEC networks.

Access to Document (DOI)

　前のページ - 次のページ　

松谷 宏紀

論文 - 松谷 宏紀

松谷　宏紀

論文 - 松谷　宏紀