Nature Language Processing

Knowledge Graph

In computer science and information science, an ontology encompasses a representation, formal naming and definition of the categories, properties and relations between the concepts, data and entities that substantiate one, many or all domains of discourse. More simply, an ontology is a way of showing the properties of a subject area and how they are related, by defining a set of concepts and categories that represent the subject. (from wiki)

Building and Iteratively Updating Medical Knowledge Graph

When numerous literature emerges, how can we quickly obtain the information we concerned about?
For clinicians, it is necessary to pay attention to the latest medical progress while treating patients, and sometimes new decisions can be made based on new knowledge.

In this project, we are researching on an automatic information extraction method to obtain medical knowledge from the up-to-date medical literature. Then, based on this knowledge graph, a system can be developed to assist clinicians to make decisions.

How to make a delicious home-cooked dish?

In this project, a small recipe domain knowledge graph was automatically constructed using the crawled data from Xiachufang, and a recipe-related question answering system based on rule matching was implemented based on this knowledge graph (Neo4j+python).

Here, I completed the whole process of knowledge graph construction and question answering system implementation dependently, including the web data crawling, data cleaning, knowledge graph construction, question analysis, graph-based query, etc.

Polyphone Disambiguation in Mandarin Chinese

The text-to-speech (TTS) system is an essential component in the human-computer voice interaction framework, which aims to “translate” natural language texts into speech. Although most of Chinese characters have a fixed pronunciation, there are some special cases, i.e. the polyphonic characters.

We propose a framework that can predict the pronunciations of Chinese characters, and the core model is trained in a distantly supervised way.
Experimental results demonstrate that even without additional syntactic features and pre-trained embeddings, our approach achieves competitive prediction results, and especially improves the predictive accuracy for unbalanced polyphonic characters. In addition, compared with the manually annotated training datasets, the auto-generated one is more diversified and makes the results more consistent with the pronunciation habits of most people.

Text Sentiment Classiﬁcation

How to determine the emotional polarity of a text?

In this project, we implemented a sentimental polarity classification of commodity reviews based on sentiment dictionary and some traditional machine learning methods (KNNentropy). The chi-square test was employed to select the features, and the weighted voting mechanism was utilized to improve the performance of the classifier (F1=0.896).

Human-Computer Interaction

Eye-tracking Study

Based on the eye-mind assumption that eye fixation locations reflect attention distributions (Just, M. A., 1980), we using eye-tracking techniques to understand how teacher’s behaviour affects students’ distribution of visual attention.

Eye tracking is the process of measuring either the point of gaze or the motion of an eye relative to the head. A simple calibration procedure is usually needed. In this project we used Tobii 4C eye tracker, which can provide simultaneous eye and head tracking.

Areas of Interest and transitions are used as the measurements, and we employed the attention maps and temporal evolution of scanpaths to visualize the experimental results.

Augmented Reality

This project aims at applying Augmented Reality (AR) technology to distance education by creating an AR system with a three-dimensional virtual teaching agent (here is a pepper robot) to enhance students' experience of distance learning.

AR is a technique that combines virtual objects with real environments to provide information expansion and enhancement for real scene environments, which means it can "augmented" real-world environment by computer-generated perceptual information.

Based on the results of eye-tracking study, the movement of a pedagogical agents were redesigned, which only retain the body language that has been previously found to be effective in guiding students' attention.Blender was used to model and animate the pedagogical agent, and the overall AR application was implemented in Unity 3D.

Epson smart glasses were employed in this project, which can be classified as an optical see-through head-mounted display (S-HMD). S-HMD shows the virtual environment directly over the real world by placing optical combiners in front of the user's eyes.

We also recorded the teaching video of the real pepper robot and conducted a comparative experiment with the AR system, and obtained the feedback from the audiences through questionnaire.