Uma abordagem incremental para resolução de entidades descritas por dados textuais curtos

Abstract

Several Web applications maintain data repositories containing references to thousands of realworld entities originating from multiple sources, and they continually receive new data. Identifying the distinct entities and associating the correct references to each one is a problem known as entity resolution. The challenge is to solve the problem incrementally, as the data arrive, especially when those data are described by a single textual attribute. In this work, we propose a approach for incremental entity resolution. Unlike traditional approaches, the method we implemented, called AssocIER, uses an ensemble of multiclass classifiers with self-training and detection of novel classes to incrementally group entity references. Self-training allows the learning model to be automatically updated during the prediction phase, and the novel class detection mechanism allows the identification of records of unknown classes in the training time. Our main classifier is based on a restricted case of association rules, which can be implemented efficiently. We evaluated our method in various real-world datasets and scenarios, comparing it with a traditional entity resolution approach. The results show that AssocIER is effective and efficient to solve unstructured data in collections with a very large number of entities and features, and is able to detect hundreds of novel classes. We found that AssocIER can greatly improve the performance of resolving product data, which is a weakness of the baseline, achieving gains of 149% in effectiveness and being up to 385 times faster in the prediction phase. The results also show that it is important to incorporate new data into the learning model, especially for datasets with fewer records per class. Furthermore, our method behaves well in scenarios of scarce availability of examples for training, being able to run even with no training data.