Change point detection of time series - technical survey

Surveyed theories and technologies of detecting change points on a given time series in some meaning and summarized as a report document.

These technologies are generally called "change point detection", "anomaly detection", or "motif discovery", and many methods have been developed. The fields in need of these technologies are wide, not only signal processing but also security system, bioinformatics, multimedia processing and so on.

Mainly surveyed the following methods.

• Symbolic Aggregation Approximation Method
• Random Projection Pursuit Method
• Markov-Suffix Tree Method
• Normalized Edit Distance Method
• Isomap Nonlinear Dimension Reduction Method
• Kolmogorov Complexity Anomaly Detection Method
• Motif Discovery by PCA and MDL Principal

Independent component analysis - technical survey and tutorial

We made a survey of various methods of Independent Component Analysis, summarized mathematical theories to understand these theories, submitted as a report document, and tutorial based on the report.

Mainly surveyed the following methods:

• Negentropy Minimizing Method (FastICA)
• Cumulant Tensor Eigen Decomposition Method (JADE)
• Non-Normal Component Extraction by Projection Pursuit
• Beta-Divergence Minimizing Method
• Space Mixture Separation utilizing Time Structures.
• Mutual Covariance Elimination (Simultaneous Diagonalization of Self-Covariance Matrix by Eigenvalue Decomposition)
• Mutual Decorrelation Method (Simultaneous Diagonalization of Nonstationary Covariance Matrix by Gradient Method)
• Variance-Nonstationarity Maximizing Method.

Summarized the following mathematical theories:

Bayesian networks - technical survey

Bayesian Network is a technology of modeling causal relations between stochastic variables. In this work, we surveyed

• Inference Algorithms
• Parameter Learning Algorithms
• Structure Learning Algorithms

We made a report document which can be used until implementing as software program. We also surveyed concrete applied examples of Bayesian Networks to real world.

Mainly surveyed the following methods:

• Message Passing Algorithm (Probability Propagation)
• SPI Algorithm (Symbolic Probability Inference)
• Likelihood Weighted Sampling Method
• Parameter Learning by EM Algorithm based on Dirichlet Distribution
• K2 Structure Learning Algorithm

Mainly surveyed the following application examples:

• Abnormal Identification of Aircraft Gas Turbine Engine
• Abnormal Diagnosis of Ship Engine Room
• Abnormal Diagnosis of Turbine Power Generator
• Anomaly Detection of Machines by Vibrations and Sounds
• Anomaly Detection of Auto Door
• Anomaly Detection of Space Appliances

Periodic impulse waveform extraction - technical survey and implementation

Surveyed technologies which detect and extract periodic impulse waveform from time series data, and summarized as a report document. Also implemented five algorithm chosen from these many methods and compared accuracy and performance.

Mainly surveyed the following methods:

• SVR Spectrum Method
• Matrix Algebra Separation Method (MAS)
• Cumulant Harmonic Decomposition Method
• Orthogonal Partial Space Decomposition Method
• Periodic Transform Method (PT)
• Filter-Selecting Adaptive Comb Filter (ACF)
• Uniform Q-Comb Filter
• Fast Whitening Comb Filter
• Harmonic Wave Inference Whitening Comb Filter
• Lattice-type IIR Adaptive Notch Filter
• Correlation Impulse Adaptive Filter (CIAF)
• Rotating Adaptive Filter (RAF)
• Cyclic-Shift Intermissive Adaptive Filter (CSSGAF)
• Canonical Self Regression Decomposition (CARD)
• Global Amplitude Phase Modulation Method
• Fourier Linear Combinator (FLC)
• Recursive Discrete Fourier Transform Method
• Adaptive Fourier Analysis by Fast Filter-bank

Process mining - technical survey

We investigated various methods of process mining, a technique for inferring the underlying process structure from time-series event logs, and compiled them into a report. We also implemented one of the algorithms in Java.

Mainly the following methods are surveyed.

• Heuristic Algorithm
• Genetic Algorithm
• Duplicate Task GA Algorithm
• alpha Algorithm
• alpha++ Algorithm
• Parikh Language-based Region Algorithm
• Region Algorithm
• Tsinghua-alpha Algorithm
• Fuzzy Algorithm
• DWS mining Algorithm

Order tracking of mechanical vibration by Bayesian estimation - development

Developed a MATLAB program of Bayesian estimation for order tracking which estimates frequency of rotating machines or reciprocating machines from vibration and sound coming up from them.

Technologies

• MATLAB
• Signal Processing
• Numerical Computation
• Bayesian Estimation
• Maximum Likelihood Method

Implementation of a program to detect natural vibrations of machine

We implemented a program in MATLAB to estimate a machine's natural vibration from its vibration spectrum.

Technologies

• MATLAB
• Signal Processing
• Spectrum Analysis
• Nonparametric Smoothing
• Nadaraya-Watson Estimator
• Gaussian Mixture Models
• EM Algorithm

Tone representation and resynthesis system for speech and sound - R&D.

This system extracts essential data structures which represents tone of sound (tone representation), and inversely reconstruct and resynthesis original tone from the data structures.

Developed the system which can experiment several algorithms, based on band-width extended sinusoidal model which improves McAulay-Quatieri Sinusoidal Analysis utilizing auditory psychology model, making high accuracy of tone analysis using spectrogram reassignment method based on Cohen's kernel function classes, and adding auditory energy model of our own.

Technologies

• MATLAB
• Signal Processing
• Sound Processing
• McAulay-Quatieri Sinusoidal Analysis
• Auditory Psychology Model
• Spectrogram Reassignment Method
• Cohen's Kernel Functions Classes

Text-to-speech synthesis program - porting and development

We ported text-to-speech synthesis program which is originally developed in CMU and Edinburg University implemented in C++ and Scheme language to lightweight C language.

The original program is highly customizable software because it implements speech signal processing and various algorithms in C++ and call them through Scheme interpreter. But the original program is so large to use in real environments, and it includes redundant processing all over the place, we simplified these architectures as well as porting to lightweight C language.

In C language, we not embedded Scheme interpreter, but used simple Scheme object with reference counting instead. Various inference processing used in speech synthesis are realized by Viterbi algorithm with n-gram dictionaries. We wrote Scheme program which converts pre-learned decision trees written in Scheme to decision trees written in C.

Technologies

• Text-To-Speech Synthesis
• C++
• Scheme
• Viterbi Algorithm
• Decision Tree

Noise reduction algorithm by wavelet analysis - R&D

A system was developed which experiments and analyze reduction algorithms of noise included in speech or vibration by using wavelet analysis related techniques.

Global GCV Method, Level-Dependent GCV Method

Calculating threshold of minimizing GCV (Global Cross-Validation) against discrete wavelet coefficients, this method reduces noise by eliminating components less than the threshold.

Tree-Structured Method

Calculating sub-tree having strength more than some threshold from wavelet coefficients tree, this method reduces noise which have no relations with the essential waveform. Implemented Dyadic CART (Classification And Regression Trees) algorithm for searching optimal sub-tree. This algorithm does model regression against (sparse) binary-tree whose node has numerical value by posing penalty for complexity of the binary-tree.

To develop this system, we have needed processing of complex data structure, but MATLAB is not so good at such a work, so we implemented core algorithm in C++ and let MATLAB call it as a plug-in.

Technologies

• MATLAB
• C++
• MATLAB Plugin
• Wavelet Analysis
• GCV
• Tree Model Regression

Abnormal detection algorithm by mechanical vibrations and sounds - R&D

Researched and developed algorithms which detects abnormality of machines by vibrations and sounds originated from machines.

Features are extracted which are considered to be efficient for detecting abnormality using Spectrum Analysis and Statistic Analysis. These features are input to decision algorithms of various pattern recognition. We verified effectiveness of several combinations of these features and recognition algorithms.

Technologies

• Spectrum Analysis
• Statistical Analysis
• Time Series Analysis
• Pattern Recognition
• MATLAB
• C++

Pre-processing and unsupervised machine learning methods for anomaly detection from spatio-temporal data - technical survey

Technologies

• Proper Orthogonal Decomposition (POD)
• Dynamic Mode Decomposition (DMD)
• Robust Principal Component Analysis (Robust PCA)
• Local Outlier Factor (LOF)
• Isolation Forest (iForest)
• One-Class Support Vector Machine (One-Class SVM)

Optimized numerical control for semiconductor production device - software R&D

Existing program which solves AR model estimation problem by numerical iteration method was very slow and cannot endure for practical use.

We explored the existing program and suggested several alternative numerical calculation algorithm and implementation methods. Finally compared performance of these alternatives, we achieved 1000 times faster program than the existing program.

Technologies

• Time Series Analysis
• AR model Estimation
• Numerical Computation
• C

An automated fragmentation library of low molecular weight compounds for application in the FMO method - development

Technologies

• Quantum Chemical Calculation
• Fragment Molecular Orbital Method (FMO Method)
• Combinatorial Optimization
• C++

Advancement of biomolecular analysis workflows running on supercomputer "Fugaku" - R&D

Technologies

• Prefect
• Workflow
• Supercomputer "Fugaku"
• HPC
• PJM
• AbinitMP
• Fragment Molecular Orbital Method (FMO Method)
• IFIE
• PIEDA
• Extensive exploration of biomolecular structures
• Gromacs
• CafeMOL
• Incremental PCA
• Dask-ML
• KMR
• Python

Workflow program for binding free energy calculations of multiple ligands onto supercomputer "Fugaku" - deployment

Technologies

• Free Energy Perturbation
• Supercomputer "Fugaku"
• HPC
• PJM
• RDKit
• Gromacs
• Python

2-D region optimal partitioning - software development

We have developed software that numerically finds an approximate solution to the optimization problem of partitioning a finite and connected two-dimensional region in a two-dimensional plane into a specified number of subregions and finding a region partitioning that minimizes the sum of the first eigenvalues (minimum eigenvalues) of the Laplace-Dirichlet eigenvalue problem in each of these subregions.

Since it is difficult to solve such an optimization problem directly, a relaxed formulation was developed by introducing a density function that approximates the characteristic function of the partitioned region and imposing a penalty outside the region in order to compute approximate eigenvalues of the partitioned region. In such a relaxed formulation, it is known that the density function converges to the characteristic function of the region as the penalty approaches infinity.

Therefore, we implemented an algorithm in which each density function is optimized by the gradient descent method so that the sum of eigenvalues becomes small while solving the approximate eigenvalue problem for each partitioned regions with appropriate initial settings. Parallel computation was also used to speed up the process.

We have also implemented an algorithm to optimize the same problem from a thermodynamic point of view. Since the property that a multi-component system of particles in Brownian motion is regionally distributed in such a way as to minimize Renyi entropy production in a stationary state is an equivalent formulation to this optimization problem, we implemented an algorithm to approximate this optimization problem by simulating this thermodynamic system.

Technologies

• Laplace-Dirichlet Eigenvalue Problem
• Eigenvalue Optimization
• Region Partitioning
• Parallel Computing
• C++

Optimal fitting applications for nonlinear functions - development

We developed an application to find a nonlinear function to fit to data from a measurement instrument.

This application allows you to find the coefficients of a desired nonlinear function.

The derivative free least squares method is used to estimate nonlinear functions.
Wavelet is used for data preprocessing, and cubic spline interpolation is used to interpolate data between measurement points.

Technologies

• Nonlinear Least Squares Method
• Wavelet
• Spline Interpolation
• Jacobian
• QR Decomposition
• C#

Data assimilation system for seismic waves - development

We have developed a data assimilation system that optimizes the mass density and elastic constants of seismic wave simulation model based on ground-based pseudo-seismic observation data.

Although it is difficult to directly observe the subsurface mass density and elastic constants, they can be estimated by data assimilation using seismic data from the ground.

The data assimilation method used was the four-dimensional variational method (4DVar or adjoint method). This method uses an adjoint model based on a simulation model. This allows us to optimize the parameters over a period of time to satisfy the governing equations of the simulation model.

Technologies

• Data Assimilation
• 4DVar
• Fortran

Mathematical Optimization of Redundant Manipulator Inverse Kinematics

This software development is for solving the inverse kinematics of redundant manipulators. We have derived analytical solutions using a mathematical processing system and calculated practical solutions by means of mathematical optimization.

In the inverse kinematics of redundant systems, there are countless analytical solutions, of which only a few are valid for practical use. Therefore, mathematical optimization is performed to select a solution that fits practical use from among analytical solutions with degrees of freedom. Also, although inverse kinematics theoretically yields an analytical solution, it involves the computation of an inverse matrix, and as the number of joints increases, the solution equation expands to a huge number of terms. If it is implemented as it is, it is not expected that a solution can be obtained within a practical time frame. By combining analytical solution and mathematical optimization, we developed an algorithm to obtain a solution at high speed and implemented it in software.

For the implementation of mathematical optimization, we implemented and compared several algorithms, including the conjugate gradient method, and selected the best algorithm with a good balance between accuracy and speed.

Technologies

• Redundant Inverse Kinematics
• Mathematical Processing System
• Mathematical Optimization
• C++

Experimental condition optimization program using Bayesian optimization - development

Technologies

• Bayesian Optimization
• RPA (Robotic Process Automation)
• Sequential Experimental Design
• Python

Vehicle routing problem solver - research and development

Technologies

• Operations research
• Combinatorial optimization
• Meta-heuristics

Machine design parameters analysis - tool development

We have developed a set of tools to analyze and visualize machine design parameters.

Self-Organizing Map

The information necessary for 2D drawing by SOM is output and visualized by ESOM.

Clustering

The parameters are clustered by K-means, and the data for graphical display is output by automatically extracting the two axes for which the representative points of the clusters are easily visible.

Least-Squares

Given 𝒚 and 𝑿, we obtain 𝒘 which satisfies 𝒚≈𝑿𝒘 by least-squares method. QR decomposition by Gram-Schmidt orthogonalization was used to compute the inverse matrix.

Technologies

• Self-Organizing Map
• ESOM
• K-means
• Gram-Schmidt
• QR Decomposition
• Java

Circuit layout diagram visualization software for reconfigurable devices - development

We have developed the software to create and visualize the layout diagrams of large-scale circuits for reconfigurable devices.

Basic circuit layouts (basic tiles), which serve as the unit circuit, can be multiplied in a variety of placement patterns to create large-scale circuit layouts. In addition, it is possible to search for circuit elements and find the corresponding wiring based on the wiring length and display it in the center of the view. Also, the software provides other functions to meet your research objectives, such as color highlighting of circuit elements to visualize the crowdedness of placement and routing in terms of the entire layout.

In order to draw large-scale circuit layout diagrams at a high speed, we implemented it using the Java OpenGL (JOGL) library.

Technologies

• Reconfigurable device
• FPGA
• Visualization
• Java
• C++
• OpenGL

Sound propagation calculation - software development

For a single sound source, we have developed a program that calculates the difference between the A-weighted sound pressure level of the reference point and the sound receiving point for each frequency. By visualizing the calculation results, it is possible to evaluate diffraction effects and road surface effects in the noise propagation from the road to the areas along the road.

In graphing the difference in sound pressure levels at different frequencies from the calculation results, we discovered an error in the calculation process of relative complex sound pressure that had been used.

Technologies

• Sound Propagation
• Complex Claculation
• Numerical Integration
• C#

Surface variation in three-dimensional euclidean space - numerical experiment application development

This system performs numerical experiments on boundary problems.

Functions for mapping polar coordinates to 3-dimensional space can be entered in text format, and initial conditions for surfaces can be freely set. In addition, the conditions for numerical experiments can be freely changed, allowing a variety of experimental cases to be performed.

The surfaces on the screen can be viewed from any viewpoint by mouse operation. In addition, the progress of surface changes in numerical experiments can be replayed and animated.

Numerical computation and real-time 3D surface rendering are performed in native language for high speed. By adopting C++ and Java as implementation languages which have low environmental dependency, we developed both Mac OS X and Windows versions at low cost.

Technologies

• Boundary Problem
• Numerical Experiment
• C++
• Java
• JNI
• OpenGL
• OpenCV

Parallelization of algorithms with parallel programming models

We developed software that runs efficiently in a multiprocessor environment by parallelizing programs that previously ran on a single CPU.

OpenMP is used for shared memory type parallelization and MPI for distributed memory type parallelization to achieve parallelization suitable for the execution environment.

Technologies

• C++
• OpenMP
• MPI
• Parallel Programming Model

Research software enhancement and multiplatformization

We have enhanced the functionality of the research software for the Windows environment and made it multi-platform with the Java framework.

Three-dimensional functions that rely on OpenGL were ported using JOGL. At the same time, the software's numerical simulation capabilities were extended.

Technologies

• Java
• VB6.0
• OpenGL
• JOGL
• C
• C++
• Windows
• Mac OS
• Linux Numerical Simulation

Phase recovery algorithms for coherent x-ray analysis - implementation and acceleration

We implemented and accelerated the phase recovery algorithm for diffraction analysis using coherent X-rays produced at SPring-8 and other beamlines.

Several algorithms, based on holography and other technologies specified by the customer, were implemented.

In terms of algorithm acceleration, the phase recovery algorithm used by the customer was parallelized on CPUs and GPUs. As a result, the analysis process, which used to take several hours, can now be processed in a few minutes.

Technologies

• C/C++
• OpenMP
• CUDA
• Python
• X-Ray Diffraction
• Beam line
• Phase Recovery

Computational software for tomographic image reconstruction using GPGPU - development

Tomography, derived from the Greek word "tomos" meaning "to cut," is a technique for observing cross sections of a sample to evaluate its interior. CT, in CT scans performed in hospitals, is an abbreviation for Computerized Tomography, an examination technique that uses X-rays to capture cross-sectional images of the body. The cross-sectional image obtained in this way is called a tomographic image.

To reconstruct a tomographic image, it is necessary to calculate the scattering of a quantum beam penetrating a sample. In this case, we have streamlined and implemented the algorithm used for this calculation, and developed calculation software to generate the parameter space required for reconstruction.

The implemented algorithm uses a computer GPU to perform parallel computation of sparse matrices at high speed. As a result, high-resolution processing that was previously impossible due to memory limitations can now be computed in about one minute.

Technologies

• Tomography (ART)
• GPGPU
• Sparse Matrix
• OpenGL

Numerical analysis software development and theoretical analysis of superconducting circuits

We have developed numerical analysis software for basic research on circuit quantum electrodynamics. We also performed theoretical analysis to clarify the characteristics of superconducting circuits.

This numerical analysis software calculates the distribution function and calculates and outputs physical quantities such as specific heat when the Hamiltonian parameters of a superconducting circuit are entered.

In the theoretical analysis, parameters were added to the equations used in the numerical analysis to increase the degrees of freedom and provide a more general derivation.

Technologies

• Numerical Analysis
• Quantum Mechanics
• Statistical Mechanics
• Circuit Quantum Electrodynamics
• MATLAB

Machine translation algorithms - R&D

We developed an algorithm to improve the accuracy of Neural Machine Translation (NMT) by combining the results of statistical machine translation for input sentences that are difficult to be translated with high accuracy.

We also implemented training and evaluation pipelines for neural machine translation and statistical machine translation. In addition, we developed and performance-tuned an algorithm for aligning clauses to check for missing translations.

Technologies

• GIZA++
• NMT
• Statistical Machine Translation
• SentencePiece
• Attention

Automatic summarization algorithm - R&D

We developed an algorithm for automatic summarization which is effective for documents in domains where a characteristic style of writing is used.

Specifically, multiple algorithms were selected for each of the two main approaches to automatic summarization - extractive and generative summarization - and a training and evaluation pipeline was implemented.

Technologies

• Extractive Summarization
• Abstractive Summarization
• BertSum
• LexRank
• Deep Language Models

Document classification algorithms - R&D

We classified documents in a corpus with a small sample size compared to a large number of categories.

Due to the low accuracy of the usual supervised learning approach, a similarity-based approach was adopted. In selecting the best approach, several methods for generating document vectors were implemented and tested.

Technologies

• BERT
• Word2Vec
• fastText
• GloVe
• SIF
• TF-IDF
• SCDV
• Sentence2Vec
• Doc2Vec
• Gensim
• Universal Sentence Encoder
• Sentence BERT

Question answering from large-scale unstructured data using LLM - R&D

We built a system that enables question answering on large unstructured data without fine-tuning, using a large-scale language model, and evaluated its accuracy.

Technologies

• GPT-4
• LangChain
• Llama-Index
• in-context learning
• embedding
• Semantic Search

Web-embedded 3DCG editor - development

We developed a system using Java3D that allows users to easily edit 3D animations from a web browser using a Java applet.

Animation data can be saved as its own file format, which is compatible with VRML and can import and export VRML files. This allows you to animate 3D characters exported from common 3D modelers. However, VRML animations exported from 3D modelers are sometimes described as vertex morphing rather than affine transformations of parts. Morphing cannot be supported by this web-embedded sytem due to the increased data size.

To solve this problem, we developed an algorithm that approximates morphing animation to an affine transformation by least-squares estimation.

Technologies

• Java Applet
• Java Swing
• Java3D
• VRML
• Numerical Computation

Web information guidance system using ontology - development

This system uses an ontology (knowledge data structure) obtained through web mining to provide web information guidance. You can efficiently browse information, terms, links, reputations, metaphors, and episodes related to a specific topic and answer questions in natural language. It is equipped with a 3D character agent, which behaves in a specific way and speaks in response to the user's actions. Speech is automatically synthesized by commercially available text-to-speech software, with a highly interactive and friendly interface using Ajax.

Technologies

• Ontology
• Knowledge Processing
• Natural Language Processing
• Speech Synthesis
• C++
• PHP
• Ajax

3DCG display system using natural language processing - development

We developed a Java applet in Java3D for editing 3D animations in the browser.

The server stores the modeling files of the 3D characters. The Java applet connects to the server and fetches the 3D characters, allowing the user to easily create 3D animations with the mouse. This system performs 3D rendering on the web server and displays it as an animated GIF image in the browser.

Technologies

• Java Applet
• Java Servlet
• Java3D
• Natural Language Processing

Educational systems using natural language processing - development

This system provides users with educational content in natural language. A natural language database and animation data are located on the server side, and asynchronous communication from the browser enables real-time communication with the server to provide interactive content to users.

Technologies

• Natural Language Processing
• Servlet

Natural language processing for Twitter timelines - technical survey

Surveyed application cases of natural language processing for Twitter timelines and the use of the processed results.

Published papers were surveyed and a list of implementers, purpose of implementation, analysis methods, and tools/libraries used was summarized in the report.

Technologies

• Twitter
• Natural Language Processing
• Python

Remote control system for simultaneous measurement by multiple PCs - development

Measurement equipment generally comes with dedicated measurement software, and a PC is generally prepared for each device for measurement. In order to perform multimodal measurement using multiple measurement devices, it is necessary to prepare multiple PCs and operate the measurement software installed on each PC.

Therefore, we developed a remote control system that remotely sends commands to start and end measurements from a management PC and operates the measurement software according to the commands received by each measurement PC.

AutoIt was used to operate the measurement software on the PCs, and PsExec was used to send commands remotely, enabling remote control of four PCs over a wireless LAN. We have also developed an interface for sending commands that can be flexibly adapted to increases or decreases in the number of measurement devices and PCs used for measurement.

Technologies

• Remote Control
• Multimodal Measurement
• PsExec
• AutoIt
• RPA
• C#

Zigbee wireless sensor network monitoring system - development

This system receives measurement data from various sensors connected to a Zigbee specification wireless chip and monitors the data on a PC screen while operating various hardware via a control board.

For example, a cleanroom monitoring system uses sensors to measure temperature, humidity, static electricity, and particle volume in a cleanroom and transmits the data to a PC via a Zigbee wireless chip. The PC can display and monitor that sensor data. If the data meets certain conditions, the system determines that it is not clean and turns on a warning light or adjusts the fan air volume via the control board.

Other developed systems include one that detects earthquakes by receiving data from acceleration sensors installed in buildings and another that regulates CO2 concentration by controlling fans by receiving CO2 concentration in plant cultivation greenhouses.

Technologies

• Zigbee Wireless Chip
• Serial Communication
• DIO Board Control
• C#

Validation of the accuracy of a real-time situation assessment model using time-series sensor data - application development

We have developed an application that analyzes time-series data measured by physical sensors, estimates a prediction model to determine the situation in real time, and verifies the accuracy of the prediction.

Situation judgments were made using binary values. To learn to judge the situation, we set a threshold value for the frequency distribution of each sensor data and developed a annotation support application to classify positive and negative cases.

We estimated two types of situation judgment prediction models: a logistic regression model and an SVM learning model. In order to verify the prediction accuracy of the situation judgment prediction model, we developed an application that synchronizes sensor data with corresponding video data, and visually verified the prediction accuracy in real time.

Technologies

• R
• C++/CLI
• Feature Selection
• Logistic Regression
• SVM
• Cross Validation

Multi-sensor simultaneous measurement management system for human body - development

This system attaches various types of sensors to the human body for simultaneous measurement and examines the time-series relationship of the data.

To prevent inaccurate measurements due to CPU load, a PC is installed for each sensor to perform measurements, and a server program for remote control is run on that PC. This server program receives requests from the measurement management PC and controls the sensor via the sensor's device API. The measured data is temporarily recorded on each PC, and the measurement management PC can collect the data, display graphs, and analyze the data.

Technologies

• Biomedical Measurement
• Motion Capture
• Remoce Control
• C++

Medical support image processing system - development

We have developed a system to automatically detect abnormal tissue areas from HE-stained tissue images.

In this field, pattern matching methods have been mainly used, but instead of using this, we adopt a method that uses topological geometry to extract image features.
Compared to similar systems, the processing speed is significantly improved.

Technologies

• C#
• Histogram Processing
• Peak Extraction Algorithm
• Labeling Algorithm
• Shape Extraction Process
• Convolution Integral
• Histogram Correction
• Automatic White Balance Correction Algorithm
• Color Deviation Automatic Correction Algorithm

Layer discrimination method for tomographic image - R&D

In a 3D tomographic image consisting of multiple layers, we developed a method to discriminate the layer to which each image belongs.

This method is divided into a learning step and an estimation step.

In the learning step, the relationship between image features and layers is learned using support vector regression, a type of machine learning technique, to create a model.

In the estimation step, features are extracted from the image whose layer is to be discriminated and fed to a trained estimation model to estimate the layer to which the image belongs.

The above series of methods were implemented in Python.

Technologies

• Image Analysis
• Python
• OpenCV
• Machine Learning
• Support Vector Regression

Estimating environmental conditions using metagenomic Data - R&D

We developed a method for estimating environmental conditions using information on microbial communities obtained from metagenomic data acquired using next-generation sequencers.

In this work, we create regression models (Lasso regression and support vector regression) to estimate environmental conditions based on the abundance of each microorganism.

In addition, to forecast future microbiota, we created regression models that estimates species abundance of the following month based on data from a given month.

We used Python for the above work.

Technologies

• Metagenomics
• Machine Learning
• Lasso regression
• Support Vector regression
• Python

Medical data analysis and modeling - research support

Technologies

• Python
• Stan
• TensorFlow Probability
• Probabilistic Programming
• Variational Inference
• MCMC

Stochastic process estimation algorithm - porting development

Technologies

• Stochastic Process
• Pseudo-Likelihood
• R
• Python

Statistical causal inference algorithms - development

Technologies

• Causal Inference
• Hierarchical Bayes
• Confounding
• MCMC
• Variational Bayes
• Hamiltonian Monte Carlo

Price prediction model using quantile regression - development

A model has been developed to predict the resale price of products with large value fluctuations over time.

Data on product attributes and data on the auction history of products were used to develop the model.

By using quantile-point regression models, the output from the model is not a single predicted value of resale price, but a predictive distribution.

The predictive distribution of resale prices can be used to optimize purchase prices. It can also identify high-risk, high-return transactions, as the predictive distribution contains certainty information. This can also help determine whether resale price forecasts should be transferred from the AI to an expert (called the reject option).

Technologies

• Quantile Regression
• Decision Making
• Demand Function
• Optimal Bidding

Observer evaluation support software using item response theory - development

Technologies

• Python
• Item Response Theory
• Rasch Model
• Social Psychology
• Psychometric Scales
• Bayesian Models
• Multiple OS
• JAX
• AMPS

Economic simulation result visualization program - development support

Technologies

• GitHub
• R
• Shiny
• ggplot2
• Package Release
• Open Source
• Travis CI

Factor analysis for data with missing values - R&D

Development Environment and Technical Field

• Multiple Imputation Method
• Structural Equation Modeling
• Factor Analysis
• Principal Component Analysis

State estimation program using unscented Kalman filter and particle filter - development

Development Environment and Technical Field

• C++
• Unscented Kalman Filter
• Particle Filter
• State Space Model

Sensory evaluation program for gemstones - research and development

Technologies

• Machine Learning
• Image Processing

Real-time object recognition agent system - development

Technologies

• Local feature
• Octree
• Complex background
• C++

High-speed detection of obstacles when traveling with moving objects - R&D

As an assistive technology to help moving objects avoid hazards during automatic driving, we have developed an algorithm to detect obstacles at high speed from images acquired by a camera in the direction of travel.

The algorithm had two requirements

(1) High speed processing
(2) Ability to adapt to changing daylight and lighting conditions

The algorithm used was the grid partitioning pattern matching method.
The pattern matching algorithm implements logic to find robust correlation coefficients for illumination change, shape deformation, and magnification change.

This has confirmed that obstacles can be detected at high speed even in situations where daylight and proving conditions change.

Technologies

• Zero Mean Normalized Cross-correlation
• Incremental Code Correlation
• C/C++
• C#

Consulting for the introduction of deep learning techniques

Technologies

• Deep Learning
• Technical Consulting
• Python

Face image recognition system - algorithm R&D

We developed Face Image Recognition System with cameras placed in real environments.

Roughly speaking, face recognition systems are classified in two application form: "Validation" and "Identification". Validation is used for users to identify themselves in front of door access control system. Meanwhile, identification is used for estimating identity of unspecified person from candidate list in such a system as urban monitoring.

Identification results are computed based on face image of unspecified person and validation results of the face image in candidate list It is said that identification is more difficult in general than validation. For example, because of dispersion of validation scores of each person and face feature changes by disturbances, identification results sometimes contain rejection of identical person or misidentification of person.

In this work, to improve identification accuracy, we implemented validation score normalization algorithm in C++. Also we implemented tuning tools in MATLAB which choose learning data tolerant of disturbances and image processing parameters.

Technologies

• Biometrics
• Image Processing
• Pattern Recognition
• Statistical Analysis
• MATLAB
• C++

Face identification method based on a small number of data - algorithm development

Face recognition by deep learning often requires large amounts of labeled training data, which can be very costly. To overcome this problem, we have developed a method for face identification with a small number of samples in this work.

This method pre-trains a deep network based on existing labeled data, and then transfers the feature extraction part obtained from the pre-training to the deep network for the data given the label of the person to be recognized. We have confirmed that this method can achieve a high discrimination rate even with a small amount of data.

Because training deep networks is very time consuming, multiple GPU instances were launched on AWS to evaluate discrimination accuracy for multiple network structures in parallel.

Technologies

• Python
• Deep Learning
• fine-tuning
• CNN
• AWS
• GPGPU

Anomaly detection methods based on image reconstruction using deep Learning - survey and implementation

Technologies

• Automated Visual Inspection
• Anomaly Detection
• Autoencoder
• Variational Autoencoder (VAE)
• Python
• TensorFlow/Keras

High-resolution film scanner control system - development

This system controls a high-resolution film scanner capable of scanning large format film of aerial photographs with a resolution of up to 6.2 μm and converting it into digital data.

The high-resolution film scanner consists of a table driven by the XY axis and a line sensor mounted on the table. Since large format film is approximately four times wider than the line sensor can capture, the entire film cannot be converted to digital data in a single scan. Therefore, this system scans the entire film in four areas, aligns each area, and combines them into a single image. If alignment lines are noticeable due to the limitations of the drive system's accuracy, they are corrected by image processing.

To increase the scanning speed per film, this system utilizes multi-core CPUs for parallel processing.

Technologies

• Line Sensor
• Serial Communication
• Motor Control
• Image Processing
• Parallel Processing
• CameraLink
• C++/CLI
• C#

Digital camera shooting system for aerial photography - development

This system is designed to take high speed interval shots with a high-resolution digital camera for aerial photography and store the images on an external storage device.

In aerial photography for surveying, the speed of the aircraft and the field of view of the camera require that hundreds of images be taken at high speed intervals, but the camera itself cannot hold that much memory. Therefore, the high-resolution camera used by this system distributes the Bayer array image data output by the CCD to multiple CameraLink channels and transfers it to the PC via grabber board at high speed. This system saves Bayer array image data transferred from the camera to the storage device at high speed. It also provides a preview function for test photography.

Technologies

• CCD Bayer array
• CameraLink
• Image Processing
• C++/CLI
• C#

Trademark similarity determining algorithms - algorithm development

Technologies

• Python
• Deep Learning
• TensorFlow

Machine learning on sets of samples where individual labels are unknown - R&D

We developed a machine learning algorithm to estimate whether a certain number of samples of a given type are present in a large number of samples.

The underlying technique is Multiple Instance Learning (MIL), which is a learning method used when labels for a set of samples are given as supervised data. In general supervised learning, each sample needs to be labelled in advance, but it can be applied when individual samples do not have labels.

Standard MIL assumes that the presence or absence of a certain type of sample in the set is used as a label. Even if the available label is whether or not a certain number of samples of a certain species in a set are present, it can be treated as standard MIL with special pre-processing.

Technologies

• Machine Learning
• Multiple Instance Learning
• Python

Complex problem guide system applying data mining - development

This system extracts tacit knowledge in a specific field by data mining and automatically converts it into formal knowledge. The extracted formal knowledge is used to guide beginners in the field through various problems.

Process mining, Bayesian networks, self-organizing maps, and ontology were used as data mining techniques.

In this R&D, several algorithms were surveyed in papers and selected and implemented to be effective for the purposes of this system.

Technologies

• Java Servlet
• MySQL
• Cloud
• Process Mining
• Bayesian Networks
• Self-Organizing Map
• Ontology

Biometric coupling hardware control system - development

We developed a research system that uses measurement data from a living body as input to operate externally connected equipment.

The system takes ever-changing biological information as input and generates signals to control external devices accordingly. In this example, a control time resolution of 1 millisecond was achieved.

Technologies

• C#
• Laser Control
• Real-time Processing

EEG measurements - experimental support

In a project to build a large-scale brain activity database, we provided experimental support for measuring the EEG of a large number of subjects by installing and adjusting electrodes, recording and monitoring EEG data, and detecting and recording the causes of noise generation.

Electrode placement is the key to EEG measurement. If the electrodes are not placed properly, the measurement data will contain large noise, and accurate analysis will not be possible. Therefore, it is necessary to adjust the placement while checking the measurement data from time to time.

In addition, because EEG is a very small electrical signal, slight movements of the subject or changes in the environment can result in significant noise. In this case, too, later analysis cannot be performed correctly, so it is necessary to constantly check the measurement data, estimate what kind of noise has been mixed in, and record it.

In this way, we provided consistent experimental support from measurement preparation to recording in experiments that required experience and expertise in EEG measurement.

Technologies

• Human electroencephalography

BMI framework - development

We have developed a brain-machine interface (BMI) framework that detects targeted brain activity from brain signals such as electroencephalogram (EEG) in real time and outputs feedback signals to operate external devices.

Since real-time performance is important for BMI, a C++ multi-threading mechanism was used to minimize the delay time to feedback. The framework is designed to be flexible enough to incorporate MATLAB or Python scripts which implemented the algorithms for brain signal preprocessing and brain activity detection, depending on the experiment. The BMI also uses the lab streaming layer communication technology to provide multimodal input from multiple devices (e.g., eye-tracking devices) in addition to brain signals.

Technologies

• Brain-machine interface (BMI)
• Electroencephalogram (EEG)
• C++
• MATLAB
• Python
• Lab streaming layer (LSL)

Decoding analysis of brain activity data - R&D

Decoding analysis was performed to estimate the task being performed by the subject from electroencephalogram (EEG) data and functional magnetic resonance imaging (fMRI) data, which measure human brain activity.

A person's actions, thoughts, and stimuli to the five senses are always affecting brain activities. The decoding technique which uses machine learning technologies, has been attracting attention as a method to decipher a person's state from the data recorded in this brain activity.

In this case, we used machine learning to learn the relationship between EEG and fMRI data, which record brain activity during task execution, and the corresponding task. Using the learning results, we implemented a program to estimate the type of task from brain activity and performed decoding analysis.

EEGLAB, VBMEG, and SPM, which are standard tools, were used to analyze brain activity. The Python library scikit-learn was used for machine learning.

Technologies

• Human brain activity analysis
• Decoding
• Machine Learning
• MATLAB
• Python
• EEGLAB
• VBMEG
• SPM

Visual stimulus presentation program - development

Technologies

• Visual presentation
• Neuroscience
• Presentation
• Psychtoolbox
• MATLAB
• PsychoPy
• Python

Heart rate variability parameter calculation Android App - development

Developed an Android application that acquires data from a heart rate (pulse wave) detection device via Bluetooth communication and calculates heart rate variability parameters.

Both Bluetooth Classic and BLE standards were supported, and an autoregressive model (AR model) power spectrum density calculation method was used to calculate heart rate variability parameters.

Heart rate variability parameters calculated in real time are displayed graphically and simultaneously output to a CSV file for separate analysis and confirmation after measurement is finished. In the development process, we designed the algorithm to be easily extended and designed to be easily ported to a separate Java platform.

Technologies

• Android
• Java
• Bluetooth Classic
• BLE
• Heart Rate Variability Analysis
• AR Model
• Time Series Analysis

BLE sensor data analysis windows application - development

We developed an application that can acquire sensor data in real time via BLE communication on Windows and analyze the data in the MATLAB environment.

Unlike iOS and Android, Windows does not support the BLE standard at the OS level, so a dedicated dongle and SDK were used to realize BLE communication.

Furthermore, to achieve real-time analysis of sensor data on the MATLAB environment, inter-process communication was used for real-time sensor data acquisition.

Technologies

• Windows
• BLE
• MATLAB
• Inter-Process Communication
• C/C++

Visualizing brain oscillatory activity using maximum entropy method - software development

We developed software that performs frequency analysis using the maximum entropy method on data obtained by optical imaging technique.

In order to capture the oscillatory phenomenon, a form of neural activity, in the brain in a spatiotemporal manner, we performed frequency analysis on the imaging data. The duration of oscillatory activity in the brain is short and the number of time-series data obtained during oscillatory activity is small, we applied the maximum entropy method.

The following processing functions have been implemented for over 10,000 pieces of pixel data output by imaging devices.

• Photobleaching correction of voltage-sensitive dyes
• FIR Filtering
• Calculation of power spectrum by maximum entropy method
• Image smoothing using a Gaussian filter
• Calculation of oscillatory power distribution at each frequency and derivation of contour plot in brain
• Deriving spectrograms of specific brain regions

Development environment/Technical field

• FIR Filter
• Detrending
• Maximum Entropy Method
• C

Neural spike sorting - algorithm development

A widely used method for recording neural signals is to simultaneously record signals from multiple nerves (recording of population potentials). In this case, one measurement record is recorded as a composite signal of multiple neural spikes. The process of the features of each spike from the recording of the population potential and separating the spikes specific to each nerve is called spike sorting.

In spike sorting, spike amplitude and spike duration are usually used as features. To improve the accuracy of spike sorting, we developed an algorithm that compares spike amplitude and spike shape itself.

We succeeded in isolating about 10 spike types from a single nerve bundle measurement signal, and then, for confirmation, we histologically confirmed the number of nerves that the nerve bundle, and the number of isolated nerve spike types and the histologically confirmed nerve number matched well.

Finite impulse response filter (FIR)

Spike sorting results (separation of 11 types of spikes)

Technologies

• FIR Filter
• Detrending
• Histogram
• Spline Interpolation
• Peak Detection Logic
• C
• Visual Basic

Evaluating the stimulus response of spike information - software development

We developed software that recorded population potentials and then used a sorting technique to separate each spike signal from the population potential, then clustered them by using their responsiveness to stimuli.

In this example, we gave 11 types of oscillatory stimuli. The evaluation items of the neural response to each stimulus were (1) the presence or absence of a response, (2) the dominant frequency when a constant frequency response was observed, and (3) the time variation of the dominant frequency.

The main frequency components were evaluated using the autocorrelation function. The response oscillatory characteristics were classified based on the time change of the autocorrelation function. These parameters were automatically extracted and an interface was created so that they could be passed to statistical processing software.

Technologies

• autocorrelation function
• C
• Visual Basic

Image processing program for high-speed camera - development

We performed information extraction processing on high-speed video images that measured the behavior of insects when they were given an odor stimulus.

In high-speed photography, high-intensity lights are generally used to overcome the lack of light caused by the short exposure time. However, when photographing living organisms, high-intensity lights can not be used to eliminate the effects of light, and the images may be insufficient in light.
Edges were extracted from this image data using differential filtering, and optical flow was obtained using the frame difference method. This revealed the locations where insect behavior in response to odor stimuli occurs and the sequence in which it occurs.

Technologies

• Differential Filter
• Optical Flow
• Frame Difference Method
• High-speed Camera
• C

Ca2+ imaging data processing program - development

We developed a program that directly inputs the binary output data from the Ca2+ imaging device into the program and performs processes such as calcium indicator bleaching correction, noise filtering using a digital filter, calculation of relative light intensity change, and spatial noise filtering using a Gaussian filter and a median filter.

The processed image data was output in the same format as the Ca2+ imaging device output, allowing the use of the functions of the software that comes with the Ca2+ imaging device.

Technologies

• Bleaching Correction
• FIR Filter
• DF/F Calculation
• Gaussian Filter
• Median Filter
• C

Digitizing and mapping system for animal behavior experiments - development

In animal behavior experiments such as the Morris water maze, this system acquires location information from video footage of the target animal (e.g., mouse) moving (digitizing), and creates a heat-map that indicates the frequency of the target animal's stay using colors based on the location information (mapping).

In digitizing, each frame of the video image is processed to detect the position of the target animal. Simple GUI operation can be used to set image processing settings for digitizing, as well as markers, color bars, and other settings related to mapping.

Technologies

• Image Processing
• C#
• OpenCV
• Animal Behavior Experiments

Small animal behavior recording system - development

In neuroethology experiments, we have developed a system to simultaneously monitor neural activity and behavioral changes in small animals such as mice and insects.

Imaging techniques that use cameras to record the movement of a trackball are often used to measure the behavior of small animals. However, the use of inexpensive cameras with a time resolution of about 30 milliseconds makes it impossible to compare brain activity (on the order of a few milliseconds) and behavioral records, and therefore requires expensive high-speed camera systems.

This system is an inexpensive configuration that measures trackball movements with two optical mice. This enables behavioral recording with a high time resolution of up to 2 milliseconds.

Technologies

• C#
• RawInput API
• Behavioral Measurement
• Animal Behavior Experiments